ERROR: Validate failed: Detected failed migration to version 2.8.2.8 (hops)

Hopsworks
#21

I solved it !
My problem for ndb::mysqld was the access of mysql and mysqld for the installation.
But now there is a problem on kzookeeper::default . This are the log of th error:

[sudo] password for nikokks: Starting Chef Infra Client, version 15.7.32e[0m
resolving cookbooks for run list: ["kzookeeper::default"]e[0m
Synchronizing Cookbooks:e[0m
  - kzookeeper (1.3.0)e[0m
  - java (7.0.0)e[0m
  - kagent (1.3.0)e[0m
  - consul (1.3.0)e[0m
  - homebrew (5.0.8)e[0m
  - windows (7.0.0)e[0m
  - openssl (4.4.0)e[0m
  - hostsfile (2.4.6)e[0m
  - sudo (4.0.1)e[0m
  - ntp (2.0.3)e[0m
  - magic_shell (1.0.0)e[0m
  - conda (1.3.0)e[0m
  - chef-sugar (5.1.8)e[0m
  - ulimit (1.3.0)e[0m
Installing Cookbook Gems:e[0m
Compiling Cookbooks...e[0m
[2020-04-24T14:46:33+02:00] WARN: Resource openssl_dhparam from the client is overriding the resource from a cookbook. Please upgrade your cookbook or remove the cookbook from your run_list.
[2020-04-24T14:46:33+02:00] WARN: Resource openssl_rsa_key from the client is overriding the resource from a cookbook. Please upgrade your cookbook or remove the cookbook from your run_list.
[2020-04-24T14:46:33+02:00] WARN: Resource openssl_x509 from the client is overriding the resource from a cookbook. Please upgrade your cookbook or remove the cookbook from your run_list.
[2020-04-24T14:46:33+02:00] WARN: Resource sudo from the client is overriding the resource from a cookbook. Please upgrade your cookbook or remove the cookbook from your run_list.
Converging 28 resourcese[0m
Recipe: kzookeeper::defaulte[0m
  * group[zookeeper] action create
    e[32m- create group zookeepere[0m
e[0m  * linux_user[zookeeper] action create
    e[32m- create user zookeepere[0m
e[0m  * group[zookeeper] action modify
    e[32m- modify group zookeepere[0m
    e[32m- add missing member(s): zookeepere[0m
e[0m  * directory[/srv/hops] action create (skipped due to not_if)
  * directory[/srv/hops/zookeeper] action create
    e[32m- create new directory /srv/hops/zookeepere[0m
    e[32m- change mode from '' to '0750'e[0m
    e[32m- change owner from '' to 'zookeeper'e[0m
    e[32m- change group from '' to 'zookeeper'e[0m
e[0mRecipe: java::notifye[0m
  * log[jdk-version-changed] action nothing (skipped due to action :nothing)
Recipe: java::openjdke[0m
  * apt_package[openjdk-8-jdk, openjdk-8-jre-headless] action install (up to date)
  * java_alternatives[set-java-alternatives] action set (up to date)
Recipe: java::default_java_symlinke[0m
  * link[/usr/lib/jvm/default-java] action create (up to date)
Recipe: java::set_java_homee[0m
  * directory[/etc/profile.d] action create (up to date)
  * template[/etc/profile.d/jdk.sh] action create (up to date)
  * ruby_block[Set JAVA_HOME in /etc/environment] action run
    e[32m- execute the ruby block Set JAVA_HOME in /etc/environmente[0m
e[0mRecipe: kzookeeper::defaulte[0m
  * kzookeeper[3.4.7] action install
  Recipe: <Dynamically Defined Resource>e[0m
    * chef_gem[zookeeper] action install
      e[32m- install version 1.4.11 of package zookeepere[0m
e[0m    * chef_gem[json] action install (up to date)
    * group[zookeeper] action create
      e[32m- alter group zookeepere[0m
      e[32m- replace group members with new list of memberse[0m
e[0m    * dscl_user[zookeeper] action create
      e[0m
      ================================================================================e[0m
      e[31mError executing action `create` on resource 'dscl_user[zookeeper]'e[0m
      ================================================================================e[0m
      
e[0m      Errno::ENOENTe[0m
      -------------e[0m
      No such file or directory - dscacheutile[0m
      
e[0m      Cookbook Trace:e[0m
      ---------------e[0m
      /tmp/chef-solo/cookbooks/kzookeeper/providers/default.rb:44:in `block in class_from_file'e[0m
      
e[0m      Compiled Resource:e[0m
      ------------------e[0m
      # Declared in 
e[0m      
e[0m      dscl_user("zookeeper") do
e[0m        action [:create]
e[0m        default_guard_interpreter :default
e[0m        gid "zookeeper"
e[0m      end
e[0m      
e[0m      System Info:e[0m
      ------------e[0m
      chef_version=15.7.32
e[0m      platform=ubuntu
e[0m      platform_version=18.04
e[0m      ruby=ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]
e[0m      program_name=/usr/bin/chef-solo
e[0m      executable=/opt/chefdk/bin/chef-soloe[0m
      
e[0m    e[0m
    ================================================================================e[0m
    e[31mError executing action `install` on resource 'kzookeeper[3.4.7]'e[0m
    ================================================================================e[0m
    
e[0m    Errno::ENOENTe[0m
    -------------e[0m
    dscl_user[zookeeper] (dynamically defined) had an error: Errno::ENOENT: No such file or directory - dscacheutile[0m
    
e[0m    Cookbook Trace:e[0m
    ---------------e[0m
    /tmp/chef-solo/cookbooks/kzookeeper/providers/default.rb:44:in `block in class_from_file'e[0m
    
e[0m    Resource Declaration:e[0m
    ---------------------e[0m
    # In /tmp/chef-solo/cookbooks/kzookeeper/recipes/default.rb
e[0m    
e[0m     72: kzookeeper "#{node['kzookeeper']['version']}" do
e[0m     73:   user        node['kzookeeper']['user']
e[0m     74:   mirror      node['kzookeeper']['mirror']
e[0m     75:   checksum    node['kzookeeper']['checksum']
e[0m     76:   install_dir node['kzookeeper']['install_dir']
e[0m     77:   data_dir    node['kzookeeper']['config']['dataDir']
e[0m     78:   action      :install
e[0m     79: end
e[0m     80: 
e[0m    
e[0m    Compiled Resource:e[0m
    ------------------e[0m
    # Declared in /tmp/chef-solo/cookbooks/kzookeeper/recipes/default.rb:72:in `from_file'
e[0m    
e[0m    kzookeeper("3.4.7") do
e[0m      action [:install]
e[0m      default_guard_interpreter :default
e[0m      declared_type :kzookeeper
e[0m      cookbook_name "kzookeeper"
e[0m      recipe_name "default"
e[0m      user "zookeeper"
e[0m      mirror "http://193.10.67.171/hops"
e[0m      checksum "2e043e04c4da82fbdb38a68e585f3317535b3842c726e0993312948afcc83870"
e[0m      install_dir "/srv/hops/zookeeper"
e[0m      data_dir "/srv/hops/zookeeper/zookeeper-3.4.7/data"
e[0m      version "3.4.7"
e[0m    end
e[0m    
e[0m    System Info:e[0m
    ------------e[0m
    chef_version=15.7.32
e[0m    platform=ubuntu
e[0m    platform_version=18.04
e[0m    ruby=ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]
e[0m    program_name=/usr/bin/chef-solo
e[0m    executable=/opt/chefdk/bin/chef-soloe[0m
    
e[0me[0m
Running handlers:e[0m
[2020-04-24T14:47:40+02:00] ERROR: Running exception handlers
Running handlers complete
e[0m[2020-04-24T14:47:40+02:00] ERROR: Exception handlers complete
Chef Infra Client failed. 7 resources updated in 01 minutes 28 secondse[0m
[2020-04-24T14:47:40+02:00] FATAL: Stacktrace dumped to /tmp/chef-solo/chef-stacktrace.out
[2020-04-24T14:47:40+02:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2020-04-24T14:47:40+02:00] FATAL: Errno::ENOENT: kzookeeper[3.4.7] (kzookeeper::default line 72) had an error: Errno::ENOENT: dscl_user[zookeeper] (dynamically defined) had an error: Errno::ENOENT: No such file or directory - dscacheutil
[sudo] password for nikokks: Starting Chef Infra Client, version 15.7.32e[0m
resolving cookbooks for run list: ["kzookeeper::default"]e[0m
Synchronizing Cookbooks:e[0m
  - kzookeeper (1.3.0)e[0m
  - java (7.0.0)e[0m
  - kagent (1.3.0)e[0m
  - consul (1.3.0)e[0m
  - homebrew (5.0.8)e[0m
  - windows (7.0.0)e[0m
  - openssl (4.4.0)e[0m
  - hostsfile (2.4.6)e[0m
  - ntp (2.0.3)e[0m
  - magic_shell (1.0.0)e[0m
  - sudo (4.0.1)e[0m
  - conda (1.3.0)e[0m
  - chef-sugar (5.1.8)e[0m
  - ulimit (1.3.0)e[0m
Installing Cookbook Gems:e[0m
Compiling Cookbooks...e[0m
[2020-04-24T14:48:15+02:00] WARN: Resource openssl_dhparam from the client is overriding the resource from a cookbook. Please upgrade your cookbook or remove the cookbook from your run_list.
[2020-04-24T14:48:15+02:00] WARN: Resource openssl_rsa_key from the client is overriding the resource from a cookbook. Please upgrade your cookbook or remove the cookbook from your run_list.
[2020-04-24T14:48:15+02:00] WARN: Resource openssl_x509 from the client is overriding the resource from a cookbook. Please upgrade your cookbook or remove the cookbook from your run_list.
[2020-04-24T14:48:15+02:00] WARN: Resource sudo from the client is overriding the resource from a cookbook. Please upgrade your cookbook or remove the cookbook from your run_list.
Converging 28 resourcese[0m
Recipe: kzookeeper::defaulte[0m
  * group[zookeeper] action create (skipped due to not_if)
  * linux_user[zookeeper] action create (skipped due to not_if)
  * group[zookeeper] action modify
    e[32m- modify group zookeepere[0m
    e[32m- add missing member(s): zookeepere[0m
e[0m  * directory[/srv/hops] action create (skipped due to not_if)
  * directory[/srv/hops/zookeeper] action create (up to date)
Recipe: java::notifye[0m
  * log[jdk-version-changed] action nothing (skipped due to action :nothing)
Recipe: java::openjdke[0m
  * apt_package[openjdk-8-jdk, openjdk-8-jre-headless] action install (up to date)
  * java_alternatives[set-java-alternatives] action set (up to date)
Recipe: java::default_java_symlinke[0m
  * link[/usr/lib/jvm/default-java] action create (up to date)
Recipe: java::set_java_homee[0m
  * directory[/etc/profile.d] action create (up to date)
  * template[/etc/profile.d/jdk.sh] action create (up to date)
  * ruby_block[Set JAVA_HOME in /etc/environment] action run
    e[32m- execute the ruby block Set JAVA_HOME in /etc/environmente[0m
e[0mRecipe: kzookeeper::defaulte[0m
  * kzookeeper[3.4.7] action install
  Recipe: <Dynamically Defined Resource>e[0m
    * chef_gem[zookeeper] action install (up to date)
    * chef_gem[json] action install (up to date)
    * group[zookeeper] action create
      e[32m- alter group zookeepere[0m
      e[32m- replace group members with new list of memberse[0m
e[0m    * dscl_user[zookeeper] action create
      e[0m
      ================================================================================e[0m
      e[31mError executing action `create` on resource 'dscl_user[zookeeper]'e[0m
      ================================================================================e[0m
      
e[0m      Errno::ENOENTe[0m
      -------------e[0m
      No such file or directory - dscacheutile[0m
      
e[0m      Cookbook Trace:e[0m
      ---------------e[0m
      /tmp/chef-solo/cookbooks/kzookeeper/providers/default.rb:44:in `block in class_from_file'e[0m
      
e[0m      Compiled Resource:e[0m
      ------------------e[0m
      # Declared in 
e[0m      
e[0m      dscl_user("zookeeper") do
e[0m        action [:create]
e[0m        default_guard_interpreter :default
e[0m        gid "zookeeper"
e[0m      end
e[0m      
e[0m      System Info:e[0m
      ------------e[0m
      chef_version=15.7.32
e[0m      platform=ubuntu
e[0m      platform_version=18.04
e[0m      ruby=ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]
e[0m      program_name=/usr/bin/chef-solo
e[0m      executable=/opt/chefdk/bin/chef-soloe[0m
      
e[0m    e[0m
    ================================================================================e[0m
    e[31mError executing action `install` on resource 'kzookeeper[3.4.7]'e[0m
    ================================================================================e[0m
    
e[0m    Errno::ENOENTe[0m
    -------------e[0m
    dscl_user[zookeeper] (dynamically defined) had an error: Errno::ENOENT: No such file or directory - dscacheutile[0m
    
e[0m    Cookbook Trace:e[0m
    ---------------e[0m
    /tmp/chef-solo/cookbooks/kzookeeper/providers/default.rb:44:in `block in class_from_file'e[0m
    
e[0m    Resource Declaration:e[0m
    ---------------------e[0m
    # In /tmp/chef-solo/cookbooks/kzookeeper/recipes/default.rb
e[0m    
e[0m     72: kzookeeper "#{node['kzookeeper']['version']}" do
e[0m     73:   user        node['kzookeeper']['user']
e[0m     74:   mirror      node['kzookeeper']['mirror']
e[0m     75:   checksum    node['kzookeeper']['checksum']
e[0m     76:   install_dir node['kzookeeper']['install_dir']
e[0m     77:   data_dir    node['kzookeeper']['config']['dataDir']
e[0m     78:   action      :install
e[0m     79: end
e[0m     80: 
e[0m    
e[0m    Compiled Resource:e[0m
    ------------------e[0m
    # Declared in /tmp/chef-solo/cookbooks/kzookeeper/recipes/default.rb:72:in `from_file'
e[0m    
e[0m    kzookeeper("3.4.7") do
e[0m      action [:install]
e[0m      default_guard_interpreter :default
e[0m      declared_type :kzookeeper
e[0m      cookbook_name "kzookeeper"
e[0m      recipe_name "default"
e[0m      user "zookeeper"
e[0m      mirror "http://193.10.67.171/hops"
e[0m      checksum "2e043e04c4da82fbdb38a68e585f3317535b3842c726e0993312948afcc83870"
e[0m      install_dir "/srv/hops/zookeeper"
e[0m      data_dir "/srv/hops/zookeeper/zookeeper-3.4.7/data"
e[0m      version "3.4.7"
e[0m    end
e[0m    
e[0m    System Info:e[0m
    ------------e[0m
    chef_version=15.7.32
e[0m    platform=ubuntu
e[0m    platform_version=18.04
e[0m    ruby=ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]
e[0m    program_name=/usr/bin/chef-solo
e[0m    executable=/opt/chefdk/bin/chef-soloe[0m
    
e[0me[0m
Running handlers:e[0m
[2020-04-24T14:48:25+02:00] ERROR: Running exception handlers
Running handlers complete
e[0m[2020-04-24T14:48:25+02:00] ERROR: Exception handlers complete
Chef Infra Client failed. 3 resources updated in 32 secondse[0m
[2020-04-24T14:48:25+02:00] FATAL: Stacktrace dumped to /tmp/chef-solo/chef-stacktrace.out
[2020-04-24T14:48:25+02:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2020-04-24T14:48:25+02:00] FATAL: Errno::ENOENT: kzookeeper[3.4.7] (kzookeeper::default line 72) had an error: Errno::ENOENT: dscl_user[zookeeper] (dynamically defined) had an error: Errno::ENOENT: No such file or directory - dscacheutil
#22

Hi,
I couldn’t solve the problem over the weekend.
Do you have an idea ?

#23

Hi.

That’s an error I haven’t seen before. Do you have free disk space? Also, what OS you’re installing Hopsworks?

#24

Hi,
I have 300 Go of free space and I am installing Hopswork on Ubuntu 18.04.

#25

Honestly it the first time I encounter this error :slight_smile: Is MacOS anyhow involved in the installation? dscl_user is a resource used to manage users in MacOS.

#26

Have you manually installed chefdk ? The version is not the one we install. Try removing what it’s already there and install for here

#27

Hello Antonios,
Can we re-run the recipe from terminal? I mean, how can we resume process through terminal if I don’t have Karamel UI.

Thanks,
Krishna

#28

Hi Krishna,

From the terminal you can re-run an individual recipe but you can’t resume the installation. All run files are in $HOME/.karamel/install, each recipe should have a Bash script. Karamel is a web application which listens on port 9090, so assuming that Karamel is still running netstat -lpn | grep 9090 - you can tunnel this port to your computer with ssh. From your computer you can do ssh -L9090:localhost:9090 REMOTE_USER@REMOTE_IP

Then open your browser and go to http://localhost:9090/index.html then Menu > Terminal, click on Status and then re-run the failed recipe. The installation will resume afterwards.