I have a single machine instance created from an ova image. There is no api key generator module. I wish to connect to this instance from python environment from my local machine. Can you help me with the fs.connect()
Hi amritsh,
What version of Hopsworks did you install?
the image was- hopsworks-0.10.0.ova
0.10 is an old release and does not support api keys. Please follow our instructions to install the latest version: https://hopsworks.readthedocs.io/en/latest/getting_started/installation_guide/installation.html
Is there an image available for newer version? The normal single machine setup is getting stuck
We stopped supporting ova. In what way is the setup getting stuck?
is it possible to change the port for hopsworks UI?
usually it’s deployed at 9090, but that is already being used
we are following this -> https://hopsworks.readthedocs.io/en/latest/getting_started/installation_guide/platforms/baremetal.html#quickstart-single-host-installation
There’s a karamel UI at 9090 port and I am getting these logs
(base) [subexgpu5@master subex_dss]$ tail -f installation.log
ERROR [2020-09-10 16:22:37,816] se.kth.karamel.webservice.KaramelServiceApplication: An instance of Karamel is already running. Exiting…
ERROR [2020-09-10 16:22:37,820] se.kth.karamel.webservice.KaramelServiceApplication: java.net.BindException: Address already in use (Bind failed)
! java.net.BindException: Address already in use (Bind failed)
! at java.net.PlainSocketImpl.socketBind(Native Method) ~[na:1.8.0_161]
! at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) ~[na:1.8.0_161]
! at java.net.ServerSocket.bind(ServerSocket.java:375) ~[na:1.8.0_161]
! at java.net.ServerSocket.(ServerSocket.java:237) ~[na:1.8.0_161]
! at se.kth.karamel.webservice.KaramelServiceApplication.(KaramelServiceApplication.java:111) ~[karamel-ui-0.6.jar:na]
Karamel can unfortunately not use any other port than 9090.
If there is already an instance of karamel running then you should be able to identify the process with netstat -tulpen | grep 9090 and then kill it or use jps -l to find the relevant process.
I stopped the process using 9090 port and reran the setup. There’s just a karamel ui at port 9090, nothing like the hopsworks page
Karamel is the installer. Wait for it to finish and Hopsworks will appear on port 443. You can check the state of the installation in the karamel ui under Menu, Terminal, status.
the karamel status is stuck at-
Hops is RUNNING_DAG but it is on pause.
Passed Phases:
| Phase | Status | Duration|
- | PRECLEANING | SUCCEED| 2 |
- | FORKING_GROUPS | SUCCEED| 1 |
- | FORKING_MACHINES| SUCCEED| 131 |
Tasks’ Status:
| OS Family| Public IP | Private IP | SSH Port| SSH User | Life Status| Task Status|
| ? | 10.113.41.11| 10.113.41.11| 22 | subexgpu5| FORKED | ONGOING |
__________________________________________________________________
| Task | Status | Actions| Duration(ms)|
==================================================================
- | find os-type | READY | | |
- | apt-get essentials | WAITING| | |
- | install chefdk | WAITING| | |
- | make solo.rb | WAITING| | |
- | clone and vendor hopsworks-chef| WAITING| | |
- | epipe::install | WAITING| | |
- | epipe::default | WAITING| | |
- | hops_airflow::install | WAITING| | |
- | hops_airflow::default | WAITING| | |
- | hops_airflow::sqoop | WAITING| | |
- | hopslog::install | WAITING| | |
- | hopslog::_filebeat-spark | WAITING| | |
- | hopslog::_filebeat-beam | WAITING| | |
- | hopslog::default | WAITING| | |
- | hopslog::_filebeat-serving | WAITING| | |
- | hopslog::_filebeat-kagent | WAITING| | |
- | hops::install | WAITING| | |
- | hops::dn | WAITING| | |
- | hops::rm | WAITING| | |
- | livy::default | WAITING| | |
- | hops::nm | WAITING| | |
- | hops::jhs | WAITING| | |
- | flink::yarn | WAITING| | |
- | flink::historyserver | WAITING| | |
- | tensorflow::default | WAITING| | |
- | hadoop_spark::yarn | WAITING| | |
- | hadoop_spark::historyserver | WAITING| | |
- | hadoop_spark::certs | WAITING| | |
- | hive2::default | WAITING| | |
- | hops::nn | WAITING| | |
- | hopsworks::install | WAITING| | |
- | hopsworks::default | WAITING| | |
- | kagent::default | WAITING| | |
- | consul::master | WAITING| | |
- | kzookeeper::default | WAITING| | |
- | kkafka::default | WAITING| | |
- | elastic::default | WAITING| | |
- | hopsmonitor::default | WAITING| | |
- | hops::ndb | WAITING| | |
- | kkafka::install | WAITING| | |
- | kzookeeper::install | WAITING| | |
- | conda::install | WAITING| | |
- | conda::default | WAITING| | |
- | hopsmonitor::install | WAITING| | |
- | hopsmonitor::prometheus | WAITING| | |
- | hopsmonitor::node_exporter | WAITING| | |
- | hopsmonitor::alertmanager | WAITING| | |
- | hopsmonitor::purge_telegraf | WAITING| | |
- | ndb::install | WAITING| | |
- | ndb::ndbd | WAITING| | |
- | ndb::mysqld | WAITING| | |
- | ndb::mgmd | WAITING| | |
- | elastic::install | WAITING| | |
- | livy::install | WAITING| | |
- | hadoop_spark::install | WAITING| | |
- | tensorflow::install | WAITING| | |
- | flink::install | WAITING| | |
- | kagent::install | WAITING| | |
- | consul::install | WAITING| | |
- | hive2::install | WAITING| | |
also the installation logs-
ERROR [2020-09-11 06:09:05,449] se.kth.karamel.backend.machines.MachinesMonitor:
! net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
! at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225) ~[sshj-0.20.0.jar:na]
! at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340) ~[sshj-0.20.0.jar:na]
! at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359) ~[sshj-0.20.0.jar:na]
! at se.kth.karamel.backend.machines.SshMachine.connect(SshMachine.java:491) ~[karamel-core-0.6.jar:na]
! … 7 common frames omitted
! Causing: se.kth.karamel.common.exception.KaramelException: 10.113.41.11: Authentication problem using ssh keys.
! at se.kth.karamel.backend.machines.SshMachine.connect(SshMachine.java:523) ~[karamel-core-0.6.jar:na]
! at se.kth.karamel.backend.machines.SshMachine.ping(SshMachine.java:549) ~[karamel-core-0.6.jar:na]
! at se.kth.karamel.backend.machines.MachinesMonitor.run(MachinesMonitor.java:101) ~[karamel-core-0.6.jar:na]
! at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_161]
! at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_161]
! at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_161]
! at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_161]
! at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
Does the user you use for installation have sudo rights? Also can you check if it is able to ssh into localhost?
Yes, the user has sudo privelage and can ssh onto localhost
The error is indicating that karamel is failing to SSH into localhost. It does that for running the setup. Can you double-check that ssh for that user is set up passwordless and localhost was added to trusted host by sshing into localhost with that user?
That helped, Thanks!
the “conda::install” task keeps failing, even after “retry”
REDHAT task has been in “paused” for some time
Tasks’ Status:
| OS Family| Public IP | Private IP | SSH Port| SSH User | Life Status| Task Status|
| REDHAT | 10.113.41.11| 10.113.41.11| 22 | subexgpu5| CONNECTED | PAUSED |
Can you send the logs/error from the conda install task?