I have mighty issues installing hopsworks on my local machine with poetry or pip. One package seem to be required with specific version numbers that cannot be installed at the moment:
The following error code is returned.
1 error generated. error: command '/usr/bin/clang' failed with exit code 1
Note: This error originates from the build backend, and is likely not a problem with poetry but with fastavro (1.7.3) not supporting PEP 517 builds. You can verify this by running 'pip wheel --use-pep517 "fastavro (==1.7.3)"'.
I have so far tried to install the packages separately but this runs into version issues with hopsworks 3.2.0.
Hardware: MacBook M2 (2023), Apple M2 Max, 64 GB Memory, Ventura 13.4
If more logs are required I will added them as soon as possible.
Thanks for flagging this. What Python version are you using?
I am using python version 3.10.1 within a poetry virtual environment. Fwiw, I had the same error with confluent-kafka, and there is a solution in your documentation. So maybe that is related but I couldn’t get it to work with fastarvo.
This is the error code I get when installing the packages seperately and the most recent version. This was for confulent-kafka, but I got the same error for fastavro.
Because hsfs (3.2.0) depends on confluent-kafka (<=1.9.0) and no versions of hsfs match >3.2.0,<3.3.0, hsfs (>=3.2.0,<3.3.0) requires confluent-kafka (<=1.9.0). And because hopsworks (3.2.0) depends on hsfs (>=3.2.0,<3.3.0), hopsworks (3.2.0) requires confluent-kafka (<=1.9.0). So, because <PROJECT> depends on both confluent-kafka (2.1.0) and hopsworks (3.2.0), version solving failed.
i was also able to install hopsworks 3.3.0rc (not 100% sure on the letters in the version number) but that gave me HttpsConnectionError on port 433 issues and I cannot seem to replicate the installation again at the moment for some reason.
Hope this helps.
Hi @llctrautmann ,
The issue is related to fastavro doesn't compile with Cython 3 · Issue #701 · fastavro/fastavro · GitHub
The Hopsworks library has a dependency on:
"fastavro>=1.4.11,<=1.7.3". The fix for the issue above is available in fastavro 1.8.1. We are making a new release of the Hopsworks library to relax the fastavro version constraints.
I’ll let you know when the release is going to be available on PyPi.
In the meantime you can try this workaround to get fastavro installed: fastavro doesn't compile with Cython 3 · Issue #701 · fastavro/fastavro · GitHub
This should be solved in the latest release of hsfs - https://pypi.org/project/hsfs/3.2.1/
Let me know if that works for you
I can confirm that the install now works. Thank you for the great support and quick fix.
EDIT: The package install now works but I cannot import the package. I get the following error
AttributeError Traceback (most recent call last)
Cell In, line 7
5 import pandas as pd
6 from functions import *
----> 7 import hsfs
17 import warnings
19 from hsfs import util, version
---> 20 from hsfs.connection import Connection
22 __version__ = version.__version__
24 connection = Connection.connection
20 from requests.exceptions import ConnectionError
22 from hsfs.decorators import connected, not_connected
---> 23 from hsfs import engine, client, util
24 from hsfs.core import feature_store_api, project_api, hosts_api, services_api
26 AWS_DEFAULT_REGION = "default"
2 # Copyright 2020 Logical Clocks AB
113 def id(self) -> Optional[int]:
AttributeError: module 'great_expectations' has no attribute 'core'
Restarting the kernel solved the issue. So I think this works now.