we are testing the integration of Hopsworks FS into Databricks and therefore installed Hopsworks using the cloud installer on a single VM in Azure. We also setup a databricks instance (both same RG and region). Now we are trying to connect from Databricks to Hopsworks FS using the python hops module. The call of “fs.setup_databricks” in a python notebook fails with an error:
SSLError: HTTPSConnectionPool(host=‘10.0.0.4’, port=443): Max retries exceeded with url: /hopsworks-api/api/project/getProjectInfo/Test (Caused by SSLError(SSLError(“bad handshake: Error([(‘SSL routines’, ‘tls_process_server_certificate’, ‘certificate verify failed’)])”)))
the code is
fs.setup_databricks( '10.0.0.4', # Hopsworks.ai address of your Feature Store instance 'Test', # Name of your Hopsworks Feature Store project port=443, secrets_store='local', api_key_file='/dbfs/fs_apikey.txt', # This should point to a path in you Databricks cluster hostname_verification=False)
10.0.0.4 is the internal IP of the network adapter associated to the Hopsworks VM.
Until this we went along the documentation under https://hopsworks.readthedocs.io/en/latest/getting_started/hopsworksai/guides/databricks_quick_start_azure.html# successfully.
VNet peering was successfull showing the peering status as “Connected”. A API key was also created and added to ‘/dbfs/fs_apikey.txt’.
The above statment fails with either “hostname_verification” set to True or False.
Thanks and best, Roberto