I installed hopsworks 3.0 on Ubuntu 18.04 but when i try to open airflow gui from hopsworks interface, it dosen’t work with this error:
Error
AirflowManager is not initialized
On server.log i found this error:
[#|2022-08-19T13:35:19.895+0200|WARNING|Payara 4.1|io.hops.hopsworks.api.exception.mapper.RESTApiThrowableMapper|_ThreadID=670;_ThreadName=http-thread-pool::http-listener-2(7);_TimeMillis=1660908919895;_LevelValue=900;|errorCode=290005, usrMsg=AirflowManager is not initialized, devMsg=AirflowManager failed to initialize or Airflow is not deployed
io.hops.hopsworks.exceptions.AirflowException
The airflow services are correctly started:
root@xxxxx:/srv/hops/airflow# systemctl status airflow-webserver airflow-scheduler
● airflow-webserver.service - Airflow webserver daemon
Loaded: loaded (/lib/systemd/system/airflow-webserver.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2022-08-19 13:34:28 CEST; 4min 47s ago
Process: 10991 ExecStop=/usr/bin/docker stop airflow-webserver (code=exited, status=0/SUCCESS)
Main PID: 11088 (docker)
Tasks: 12 (limit: 4915)
CGroup: /system.slice/airflow-webserver.service
└─11088 /usr/bin/docker run --rm --name airflow-webserver -v /srv/hops/airflow/airflow.cfg:/airflow/airflow.cfg
Aug 19 13:38:55 xxxxx docker[11088]: 10.206.195.58 - - [19/Aug/2022:11:38:55 +0000] “GET /metrics HTTP/1.1” 404 38
Aug 19 13:39:01 xxxxx docker[11088]: 127.0.0.1 - - [19/Aug/2022:11:39:01 +0000] "GET /hopsworks-api/airflow/api/ex
Aug 19 13:39:02 xxxxx docker[11088]: 10.206.195.58 - - [19/Aug/2022:11:39:02 +0000] "GET /hopsworks-api/airflow/ad
Aug 19 13:39:02 xxxxx docker[11088]: 10.206.195.58 - - [19/Aug/2022:11:39:02 +0000] "GET /hopsworks-api/airflow/ad
Aug 19 13:39:05 xxxxx docker[11088]: 10.206.195.58 - - [19/Aug/2022:11:39:05 +0000] “GET /metrics HTTP/1.1” 404 38
Aug 19 13:39:12 xxxxx docker[11088]: 10.206.195.58 - - [19/Aug/2022:11:39:12 +0000] "GET /hopsworks-api/airflow/ad
Aug 19 13:39:12 xxxxx docker[11088]: 10.206.195.58 - - [19/Aug/2022:11:39:12 +0000] "GET /hopsworks-api/airflow/ad
Aug 19 13:39:15 xxxxx docker[11088]: [2022-08-19 11:39:15,194] {init.py:51} INFO - Using executor LocalExecuto
Aug 19 13:39:15 xxxxx docker[11088]: [2022-08-19 11:39:15,195] {dagbag.py:396} INFO - Filling up the DagBag from /
Aug 19 13:39:15 xxxxx docker[11088]: 10.206.195.58 - - [19/Aug/2022:11:39:15 +0000] “GET /metrics HTTP/1.1” 404 38
● airflow-scheduler.service - Airflow scheduler daemon
Loaded: loaded (/lib/systemd/system/airflow-scheduler.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2022-08-19 13:35:01 CEST; 4min 14s ago
Process: 12044 ExecStop=/usr/bin/docker stop airflow-scheduler (code=exited, status=1/FAILURE)
Main PID: 12120 (docker)
Tasks: 14 (limit: 4915)
CGroup: /system.slice/airflow-scheduler.service
└─12120 /usr/bin/docker run --rm --name airfow-scheduler -v /srv/hops/airflow/airflow.cfg:/airflow/airflow.cfg -
Aug 19 13:35:04 xxxxx docker[12120]: // |// // // // _/__/|__/
Aug 19 13:35:04 xxxxx docker[12120]: [2022-08-19 11:35:04,201] {init.py:51} INFO - Using executor LocalExecuto
Aug 19 13:35:04 xxxxx docker[12120]: [2022-08-19 11:35:04,205] {scheduler_job.py:1346} INFO - Starting the schedul
Aug 19 13:35:04 xxxxx docker[12120]: [2022-08-19 11:35:04,205] {scheduler_job.py:1354} INFO - Running execute loop
Aug 19 13:35:04 xxxxx docker[12120]: [2022-08-19 11:35:04,206] {scheduler_job.py:1355} INFO - Processing each file
Aug 19 13:35:04 xxxxx docker[12120]: [2022-08-19 11:35:04,206] {scheduler_job.py:1358} INFO - Searching for files
Aug 19 13:35:04 xxxxx docker[12120]: [2022-08-19 11:35:04,206] {scheduler_job.py:1360} INFO - There are 0 files in
Aug 19 13:35:04 xxxxx docker[12120]: [2022-08-19 11:35:04,353] {scheduler_job.py:1411} INFO - Resetting orphaned t
Aug 19 13:35:04 xxxxx docker[12120]: [2022-08-19 11:35:04,367] {dag_processing.py:556} INFO - Launched DagFileProc
Aug 19 13:35:04 xxxxx docker[12120]: [2022-08-19 11:35:04,372] {settings.py:54} INFO - Configured default timezone
And airflow containers on docker are running:
root@xxxxx:/var/log# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
58e7fabfbe78 registry.service.consul:4443/airflow:1.10.10 “airflow scheduler” 11 minutes ago Up 11 minutes airfow-scheduler
261ddf675c2f registry.service.consul:4443/airflow:1.10.10 “airflow webserver” 11 minutes ago Up 11 minutes airflow-webserver
55fb2d489126 docker.hops.works/onlinefs:1.3 “/bin/sh -c 'java -c…” 27 minutes ago Up 27 minutes onlinefs.service
2d6062f28590 registry.service.consul:4443/airflow:1.10.10 “airflow upgradedb” 37 minutes ago Exited (0) 36 minutes ago mystifying_burnell
86a406013f1a registry “/entrypoint.sh /etc…” 55 minutes ago Up 55 minutes 5000/tcp, 0.0.0.0:4443->443/tcp registry
Any suggestions ?
Thank’s