Error executing action `run` on resource 'bash[install_airflow_mysql]'

Hopsworks
#1

When I tried to install hops_airflow__default (hops_airflow__default.sh), I encountered an error, some of the error messages are as follows :

Building wheels for collected packages: mysqlclient
Building wheel for mysqlclient (setup.py): started
Building wheel for mysqlclient (setup.py): finished with status ‘error’
Running setup.py clean for mysqlclient
Failed to build mysqlclient
Installing collected packages: mysqlclient
Running setup.py install for mysqlclient: started
Running setup.py install for mysqlclient: finished with status ‘error’
STDERR: ERROR: Command errored out with exit status 1:
command: /srv/hops/anaconda/anaconda/envs/airflow/bin/python -u -c ‘import io, os, sys, setuptools, tokenize; sys.argv[0] = ‘"’"’/tmp/pip-install-1llp1aov/mysqlclient_b8c24fd2d834460b9fa2cb8e08ca2498/setup.py’"’"’; file=’"’"’/tmp/pip-install-1llp1aov/mysqlclient_b8c24fd2d834460b9fa2cb8e08ca2498/setup.py’"’"’;f = getattr(tokenize, ‘"’"‘open’"’"’, open)(file) if os.path.exists(file) else io.StringIO(’"’"‘from setuptools import setup; setup()’"’"’);code = f.read().replace(’"’"’\r\n’"’"’, ‘"’"’\n’"’"’);f.close();exec(compile(code, file, ‘"’"‘exec’"’"’))’ bdist_wheel -d /tmp/pip-wheel-t49e9x8w
cwd: /tmp/pip-install-1llp1aov/mysqlclient_b8c24fd2d834460b9fa2cb8e08ca2498/
Complete output (72 lines):

gcc -pthread -shared -B /srv/hops/anaconda/anaconda/envs/airflow/compiler_compat -L/srv/hops/anaconda/anaconda/envs/airflow/lib -Wl,-rpath=/srv/hops/anaconda/anaconda/envs/airflow/lib -Wl,–no-as-needed -Wl,–sysroot=/ build/temp.linux-x86_64-3.7/_mysql.o -L/usr/lib64/ -lmariadb -o build/lib.linux-x86_64-3.7/_mysql.cpython-37m-x86_64-linux-gnu.so
/srv/hops/anaconda/anaconda/envs/airflow/compiler_compat/ld: cannot find -lmariadb
collect2: error: ld returned 1 exit status
error: command ‘gcc’ failed with exit status 1

Has anyone ever had this problem? Any comments will be much appreciated. The attachment is the full log.hops_airflow__default.zip (6.7 KB)

#2

@Freeman - I’m not sure why it cannot find the mariadb library, from the logs you sent it seems that mariadb was installed correctly.

Can you check that you have the .so files in /usr/lib64/?
Alternatively, if for any reasons, the OS placed the libraries in some other places, you can try to modify the recipe and export LD_LIBRARY_PATH in this part of the packages.rb recipe:

    # In /tmp/chef-solo/cookbooks/hops_airflow/recipes/packages.rb
117:   bash 'install_airflow_' + operator do
118:     umask "022"
119:     user node['conda']['user']
120:     group node['conda']['group']
121:     environment ({'SLUGIFY_USES_TEXT_UNIDECODE' => 'yes',
122:                   'AIRFLOW_HOME' => node['airflow']['base_dir']})
123:     code <<-EOF
124:       set -e
125:       #{node['conda']['base_dir']}/envs/airflow/bin/pip install apache-airflow["#{operator}"]==#{node['airflow']['version']} --constraint #{node['airflow']['url']}
126:     EOF
127:   end
128: end

so that LD_LIBRARY_PATH contains the path to the .so files for mariadb

#3

Hi @Fabio ,
Thank you for your reply. By executing the following command, I have solved this issue.

yum install MariaDB-shared

1 Like