Hi Team,
We’ve created an Azure Data Lake Storage Gen1 storage connector and would like to use it as offline feature group. However when we try to access a datalake path it is giving an error like below. Is this supported in Community edition?
Code:
sc=fs.get_storage_connector(“adls”)
sc.read(query=None, data_format=None, options={}, path=“adl://dlsazewsmlitdatalake001.azuredatalakestore.net/adls/<########/<##########>”)
Error :
NotImplementedError Traceback (most recent call last)
in
----> 1 sc.read(query=None, data_format=None, options={}, path="/adls/<########/<##########>")
/srv/hops/anaconda/envs/theenv/lib/python3.7/site-packages/hsfs/storage_connector.py in read(self, query, data_format, options, path)
104 queries are meant for JDBC or databases like Redshift and Snowflake.
105 “”"
→ 106 return engine.get_instance().read(self, data_format, options, path)
107
108 def _get_path(self, sub_path: str):
/srv/hops/anaconda/envs/theenv/lib/python3.7/site-packages/hsfs/engine/hive.py in read(self, storage_connector, data_format, read_options, location)
77 raise NotImplementedError(
78 “{} Storage Connectors for training datasets are not supported yet for external environments.”.format(
—> 79 storage_connector.type
80 )
81 )
NotImplementedError: ADLS Storage Connectors for training datasets are not supported yet for external environments.