SSL Error from SageMaker Jupyter Notebook whilst retrieving features

Feature Store
#1

Hi

I’m having an SSL error when trying to retrieve features from the feature store. I’ve created an AWS instance using https://www.hopsworks.ai/ and I am trying to connect to it from SageMaker Jupyter Notebook.

I assume my connection is successful since the keystore and truststore have been transferred to my notebook’s directory without errors .

However, I get the following error when trying to get a feature :

featurestore.get_featuregroup(“teams_features”).head(5)

failed to initialize SSL
Traceback (most recent call last):
File “/usr/local/lib/python3.6/site-packages/thrift/transport/TSSLSocket.py”, line 293, in _do_open
return self._wrap_socket(plain_sock)
File “/usr/local/lib/python3.6/site-packages/thrift/transport/TSSLSocket.py”, line 177, in _wrap_socket
self.keyfile)
ssl.SSLError: [SSL: EE_KEY_TOO_SMALL] ee key too small (_ssl.c:3503)
SQL string for the query created successfully
Running sql: SELECT * FROM teams_features_1 against the offline feature store

#2

Hi and welcome!

I think the issue happens when we convert the certificates from jks format to pem so that they can be used by the Python libraries.

Could you please make sure that the keyStore.jks and trustStore.jks are not empty (if they are, could you please remove them and try to run the connect again) ?

When you run the featurestore.get_featuregroup("teams_features").head(5), do you see three files being created (ca_chain, certfile and keyfile).

Are you using plain SageMaker instances or do you apply any configuration on top of it? What’s the output of running the following cell:

!openssl version

Also, which kernel are you using?

–
Fabio

#3

Hi Fabio, thank you for your response.

The keyStore.jks and trustStore.jks are not empty since I can read them using the java keytool.

After making the API call, the 3 files ( ca_chain, certfile and keyfile) were transferred into my notebook.

I am using Sagemaker as per the integration documentation , I don’t have any additional configurations.

!openssl version => OpenSSL 1.1.1d 10 Sep 2019

kernel => Python 3 (Base Python)

Sipho

#4

Ok, I think we found the issue.

The new version of Openssl requries the key length for the certificate to be at least 2048 bytes, but Hopsworks generates keys of 1024 bytes.

(Unfortunately all our testing SageMaker instances have an old version of OpenSSL which means that we never encountered the issue before)

We’ll push a fix for hopsworks.ai during the day. I’ll ping you again when it’s done.

–
Fabio

#5

Could it not also be the kernel type?

I took your suggestion of investigating different kernels.

It worked with python 3 (MXNet CPU Optimized) .

1 Like
#6

Ok, that’s good to hear. That could also be the case, as SageMaker uses Conda to manage the python environments, it could be that in some environments there is an older version of OpenSSL.

In any case, we also fixed Hopsworks to generate longer keys to be compatible with the newer versions of OpenSSL. If you re-create your https://hopsworks.ai cluster, it should work with all the environments.

–
Fabio