Authentication problem

get the following error. The passwordless ssh is setup for workernode. The given ip (10.11.40.5) is for the head node.
INFO [2020-05-14 23:15:01,652] se.kth.karamel.backend.ClusterManager: Going to serve ‘SUBMIT_INSTALL_DAG’
INFO [2020-05-14 23:15:01,652] se.kth.karamel.backend.ClusterManager: Running the DAG for ‘Hops’ …
INFO [2020-05-14 23:15:01,758] se.kth.karamel.client.api.CookbookCacheIml: 0-level cookbooks for Hops is 21
INFO [2020-05-14 23:15:01,758] se.kth.karamel.client.api.CookbookCacheIml: 1-level cookbooks for Hops is 20
INFO [2020-05-14 23:15:11,549] net.schmizz.sshj.transport.random.BouncyCastleRandom: Generating random seed from SecureRandom.
INFO [2020-05-14 23:15:11,623] se.kth.karamel.backend.machines.SshMachine: 10.11.40.4: connecting …
INFO [2020-05-14 23:15:11,624] net.schmizz.sshj.transport.TransportImpl: Client identity string: SSH-2.0-SSHJ_0.20.0
INFO [2020-05-14 23:15:11,635] net.schmizz.sshj.transport.TransportImpl: Server identity string: SSH-2.0-OpenSSH_7.4
INFO [2020-05-14 23:15:11,761] se.kth.karamel.backend.machines.SshMachine: 10.11.40.4: Yey!! connected ^-^
ERROR [2020-05-14 23:15:11,885] se.kth.karamel.backend.machines.MachinesMonitor:
! net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
! at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225) ~[sshj-0.20.0.jar:na]
! at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340) ~[sshj-0.20.0.jar:na]
! at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359) ~[sshj-0.20.0.jar:na]
! at se.kth.karamel.backend.machines.SshMachine.connect(SshMachine.java:479) ~[karamel-core-0.6.jar:na]
! … 7 common frames omitted
! Causing: se.kth.karamel.common.exception.KaramelException: 10.11.40.4: Authentication problem using ssh keys.
! at se.kth.karamel.backend.machines.SshMachine.connect(SshMachine.java:511) ~[karamel-core-0.6.jar:na]
! at se.kth.karamel.backend.machines.SshMachine.ping(SshMachine.java:537) ~[karamel-core-0.6.jar:na]
! at se.kth.karamel.backend.machines.MachinesMonitor.run(MachinesMonitor.java:101) ~[karamel-core-0.6.jar:na]
! at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_252]
! at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_252]
! at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_252]
! at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_252]
! at java.lang.Thread.run(Thread.java:748) [na:1.8.0_252]
INFO [2020-05-14 23:15:16,632] se.kth.karamel.backend.machines.MachinesMonitor: Sending pause signal to all machines

You need to be able to ssh into the head node from itself. Karamel will ssh into all machines to run the chef recipes, including the head machine itself (if it is installing software also on the head machine).
Do:
cd ~/.ssh
cat id_rsa.pub >> authorized_keys
ssh 10.11.40.5

Then try again.

Thanks.
The key was in authorized_keys. The problem was permission. The following command fix the problem.
chmod og-wx ~/.ssh/authorized_keys

However, the rest of installation gets stucks or fail.

  1. At second step, installation of essentials fails. I installed it manually.
  2. At chefdk step, the correct rpm was not picked/downloaed. This this step also manually.
    2.a. Needed libX11, so installed it manually.

error in make solo.rb
sudo: no tty present and no askpass program specified

./hops_airflow__install.sh failed with the error "sudo: no tty present and no askpass program specified
"
Then, after running manually, got this error
“Chef Infra Client cannot execute without accepting the license”
tried adding chef_license ‘accept’
to /etc/chef/client.rb … but it is not solving problem.
I guess i am gonna give up for now. Not interested in learning chef.
extra info which might help to fix these: trying to install on cenos 7

It looks like your sudo account needs a sudo password.
If you are using hopsworks-installer.sh, there is a switch:
-pwd PASSWORD
that you can pass to set the sudo password.

-pwd might be helpful for running some commands which need sudo.
But, my installation still get stuck in step 6 | hops_airflow::install
with this error
[sudo] password for user1: Chef Infra Client cannot execute without accepting the license

(BTW: in hopsworks-installer.sh, shift command is missing before setting SUDO_PWD)

The license error comes from this step. Newer releases of Chef have a different license ( not Apache V2 ) which you need to accept before using it.
Moreover, our recipes work with Chef 14 so, even if you accept the license, you’ll have problems installing Hopsworks.

You can download the correct version of the RPM from here: https://repo.hops.works/master/chef/chefdk-3.7.23-1.el7.x86_64.rpm

Or from the official website, if it’s still available.


Fabio

Thanks. It solved the problem. But, now have errors similar to this on error handling.

Can you post the logs of the failed recipe?
You can find the logs in /home/[user]/.karamel/install - for instance, if the recipe hops_airflow::install failed, you’ll find a hops_airflow__install.log file there.

It fails at hops. log

Hi. We just fixed the installer to install the recently released 1.3 version. It should be stable now. Could you re-download the installer script (the password switch is fixed) and try it out?

I think the problem is related to this line:
not_if "yum list installed docker-ce-#{node['hops']['docker_version']['centos']}"
all the packages are installed. However, the checking is not working because of the package naming style, as shown below.

`$ yum list installed docker-ce-19.03.8-3
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile

  • epel: mirror.umd.edu
    Error: No matching Packages to list`

'$ yum list installed docker-ce
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile

  • epel: mirror.umd.edu
    Installed Packages
    docker-ce.x86_64 3:19.03.8-3.el7 installed’

Yep, that guard is not correct we need to fix it - thanks :slight_smile:

However Docker support was just recently merged and we are in the process of stabilizing it more, but currently there are still a few bugs left.

If you want to use Hopsworks and the feature store, I suggest you try the 1.3 version which is the latest stable version.


Fabio

with version 1.3, receiving an error in hops__ndb

I suspect you are re-running the script on a “dirty” machine (on which you have already run it) to the system is half configured.

If that’s the case, the installer has a purge command so that you can start form a clean state.


Fabio