Hi @Alex
once solved the chef license acceptance, and 2 hrs later conda fails
[fmarines@PER320-2 cluster]$ tail installation.log
“run_list”: [
“conda::install”
]
}
END_OF_FILE
sudo chef-solo -c /home/fmarines/.karamel/install/solo.rb -j /home/fmarines/.karamel/install/conda__install.json 2>&1 | tee conda__install.log
echo ‘https://github.com/logicalclocks/conda-chef/tree/1.3/conda::install’ >> succeed_list
’ > conda__install.sh ; chmod +x conda__install.sh ; ./conda__install.sh
', DAG is stuck here
INFO [2020-07-23 20:45:52,763] se.kth.karamel.backend.machines.MachinesMonitor: Sending pause signal to all machines
this is from install__log:
Recipe: conda::install^[[0m
- ulimit_domain[anaconda] action create
Recipe: ^[[0m
- ulimit_rule[ulimit_rule[anaconda:nice-hard–10]] action createCreate: {“anaconda”=>{“nice”=>{“hard”=>-10}}}
(up to date)
- ulimit_rule[ulimit_rule[anaconda:nice-soft–10]] action createCreate: {“anaconda”=>{“nice”=>{“hard”=>-10, “soft”=>-10}}}
(up to date)
- template[/etc/security/limits.d/anaconda.conf] action create (up to date)
(up to date)
Recipe: conda::install^[[0m
- template[/home/anaconda/hops-system-environment.yml] action create (up to date)
- directory[/home/anaconda/.conda] action create (up to date)
- directory[/home/anaconda/.conda/pkgs] action create (up to date)
- file[/home/anaconda/.conda/environments.txt] action create (up to date)
- bash[update_conda] action run
^[[0m
================================================================================^[[0m
^[[31mError executing action run
on resource ‘bash[update_conda]’^[[0m
================================================================================^[[0m
^[[0m Mixlib::ShellOut::ShellCommandFailed^[[0m
------------------------------------^[[0m
Expected process to exit with [0], but received ‘1’
^[[0m ---- Begin output of “bash” “/tmp/chef-script20200723-37215-euo5m1” ----
^[[0m STDOUT: Collecting package metadata (current_repodata.json): …working… done
^[[0m Solving environment: …working… done
^[[0m
^[[0m # All requested packages already installed.
^[[0m
^[[0m Collecting package metadata (current_repodata.json): …working… done
^[[0m Solving environment: …working… failed with repodata from current_repodata.json, will retry with next repodata source.
^[[0m STDERR: ==> WARNING: A newer version of conda exists. <==
^[[0m current version: 4.8.2
^[[0m latest version: 4.8.3
^[[0m
^[[0m Please update conda by running
^[[0m
^[[0m $ conda update -n base -c defaults conda
^[[0m
^[[0m
^[[0m
^[[0m # >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<
^[[0m
^[[0m Traceback (most recent call last):
^[[0m File “/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/exceptions.py”, line 1079, in call
^[[0m return func(*args, **kwargs)
^[[0m File “/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/cli/main.py”, line 84, in _main
^[[0m exit_code = do_call(args, p)
^[[0m File “/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/cli/conda_argparse.py”, line 82, in do_call
^[[0m return getattr(module, func_name)(args, parser)
^[[0m File “/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/cli/main_update.py”, line 20, in execute
^[[0m install(args, parser, ‘update’)
^[[0m File “/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/cli/install.py”, line 265, in install
^[[0m should_retry_solve=(_should_retry_unfrozen or repodata_fn != repodata_fns[-1]),
^[[0m File “/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/core/solve.py”, line 117, in solve_for_transaction
^[[0m should_retry_solve)
^[[0m File “/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/core/solve.py”, line 158, in solve_for_diff
^[[0m force_remove, should_retry_solve)
^[[0m File “/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/core/solve.py”, line 281, in solve_final_state
^[[0m ssc = self._run_sat(ssc)
^[[0m File “/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/common/io.py”, line 88, in decorated
^[[0m return f(*args, **kwds)
^[[0m File “/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/core/solve.py”, line 808, in _run_sat
^[[0m should_retry_solve=ssc.should_retry_solve
^[[0m File “/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/common/io.py”, line 88, in decorated
^[[0m return f(*args, **kwds)
^[[0m File “/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/resolve.py”, line 1412, in solve
^[[0m if not is_converged(solution):
^[[0m File “/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/resolve.py”, line 1304, in is_converged
^[[0m psolution = clean(solution)
^[[0m File “/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/resolve.py”, line 1293, in clean
^[[0m return [q for q in (C.from_index(s) for s in sol)
^[[0m TypeError: ‘NoneType’ object is not iterable
^[[0m
^[[0m $ /srv/hops/anaconda/anaconda/bin/conda update anaconda -y -q
^[[0m
^[[0m environment variables:
^[[0m CIO_TEST=
^[[0m CONDA_ROOT=/srv/hops/anaconda/anaconda-3-2020.02
^[[0m PATH=/sbin:/bin:/usr/sbin:/usr/bin
^[[0m PYTHONUNBUFFERED=1
^[[0m REQUESTS_CA_BUNDLE=
^[[0m SSL_CERT_FILE=
^[[0m SUDO_COMMAND=/bin/chef-solo -c /home/fmarines/.karamel/install/solo.rb -j
^[[0m /home/fmarines/.karamel/install/conda__install.json
^[[0m SUDO_GID=1000
^[[0m SUDO_UID=1000
^[[0m SUDO_USER=fmarines
^[[0m
^[[0m active environment : None
^[[0m user config file : /home/anaconda/.condarc
^[[0m populated config files : /home/anaconda/.condarc
^[[0m conda version : 4.8.2
^[[0m conda-build version : 3.18.11
^[[0m python version : 3.7.6.final.0
^[[0m virtual packages : __cuda=10.2
^[[0m __glibc=2.17
^[[0m base environment : /srv/hops/anaconda/anaconda-3-2020.02 (writable)
^[[0m channel URLs : https://conda.anaconda.org/pytorch/linux-64
^[[0m https://conda.anaconda.org/pytorch/noarch
^[[0m main/linux-64
^[[0m main/noarch
^[[0m r/linux-64
^[[0m r/noarch
^[[0m package cache : /srv/hops/anaconda/anaconda/pkgs
^[[0m envs directories : /srv/hops/anaconda/anaconda/envs
^[[0m /srv/hops/anaconda/anaconda-3-2020.02/envs
^[[0m /home/anaconda/.conda/envs
^[[0m platform : linux-64
^[[0m user-agent : conda/4.8.2 requests/2.22.0 CPython/3.7.6 Linux/3.10.0-1127.13.1.el7.x86_64 centos/7.8.2003 glibc/2.17
^[[0m UID:GID : 968:1011
^[[0m netrc file : None
^[[0m offline mode : False
^[[0m
^[[0m
^[[0m An unexpected error has occurred. Conda has prepared the above report.
^[[0m
^[[0m Upload successful.
^[[0m ---- End output of “bash” “/tmp/chef-script20200723-37215-euo5m1” ----
^[[0m Ran “bash” “/tmp/chef-script20200723-37215-euo5m1” returned 1^[[0m
^[[0m Resource Declaration:^[[0m
---------------------^[[0m
# In /tmp/chef-solo/cookbooks/conda/recipes/install.rb
^[[0m
^[[0m 191: bash “update_conda” do
^[[0m 192: user node[‘conda’][‘user’]
^[[0m 193: group node[‘conda’][‘group’]
^[[0m 194: environment ({‘HOME’ => “/home/#{node[‘conda’][‘user’]}”})
^[[0m 195: cwd “/home/#{node[‘conda’][‘user’]}”
^[[0m 196: retries 1
^[[0m 197: retry_delay 10
^[[0m 198: code <<-EOF
^[[0m 199: #{node[‘conda’][‘base_dir’]}/bin/conda install --no-deps pycryptosat libcryptominisat
^[[0m 200: #{node[‘conda’][‘base_dir’]}/bin/conda config --set sat_solver pycryptosat
^[[0m 201: #{node[‘conda’][‘base_dir’]}/bin/conda update anaconda -y -q
^[[0m 202: EOF
^[[0m 203: end
^[[0m
^[[0m Compiled Resource:^[[0m
------------------^[[0m
# Declared in /tmp/chef-solo/cookbooks/conda/recipes/install.rb:191:in `from_file’
^[[0m
^[[0m bash(“update_conda”) do
^[[0m action [:run]
^[[0m default_guard_interpreter :default
^[[0m command nil
^[[0m backup 5
^[[0m interpreter “bash”
^[[0m declared_type :bash
^[[0m cookbook_name “conda”
^[[0m recipe_name “install”
^[[0m user “anaconda”
^[[0m group “anaconda”
^[[0m code " /srv/hops/anaconda/anaconda/bin/conda install --no-deps pycryptosat libcryptominisat\n /srv/hops/anaconda/anaconda/bin/conda config --set sat_solver pycryptosat\n /srv/hops/anaconda/anaconda/bin/conda update anaconda -y -q\n"
^[[0m domain nil
^[[0m environment {“HOME”=>"/home/anaconda"}
^[[0m cwd “/home/anaconda”
^[[0m retries 1
^[[0m retry_delay 10
^[[0m end
^[[0m
^[[0m System Info:^[[0m
------------^[[0m
chef_version=15.12.22
^[[0m platform=centos
^[[0m platform_version=7.8.2003
^[[0m ruby=ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux]
^[[0m program_name=/bin/chef-solo
^[[0m executable=/opt/chefdk/bin/chef-solo^[[0m
^[[0m^[[0m
Running handlers:^[[0m
[2020-07-23T16:45:44-04:00] ERROR: Running exception handlers
Running handlers complete
^[[0m[2020-07-23T16:45:44-04:00] ERROR: Exception handlers complete
Chef Infra Client failed. 5 resources updated in 01 minutes 06 seconds^[[0m
[2020-07-23T16:45:44-04:00] FATAL: Stacktrace dumped to /tmp/chef-solo/chef-stacktrace.out
[2020-07-23T16:45:44-04:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2020-07-23T16:45:44-04:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: bash[update_conda] (conda::install line 191) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received ‘1’
---- Begin output of “bash” “/tmp/chef-script20200723-37215-euo5m1” ----
STDOUT: Collecting package metadata (current_repodata.json): …working… done
Solving environment: …working… done
All requested packages already installed.
Collecting package metadata (current_repodata.json): …working… done
Solving environment: …working… failed with repodata from current_repodata.json, will retry with next repodata source.
STDERR: ==> WARNING: A newer version of conda exists. <==
current version: 4.8.2
latest version: 4.8.3
Please update conda by running
$ conda update -n base -c defaults conda
>>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<
Traceback (most recent call last):
File "/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/exceptions.py", line 1079, in __call__
return func(*args, **kwargs)
File "/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/cli/main.py", line 84, in _main
exit_code = do_call(args, p)
File "/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/cli/conda_argparse.py", line 82, in do_call
return getattr(module, func_name)(args, parser)
File "/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/cli/main_update.py", line 20, in execute
install(args, parser, 'update')
File "/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/cli/install.py", line 265, in install
should_retry_solve=(_should_retry_unfrozen or repodata_fn != repodata_fns[-1]),
File "/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/core/solve.py", line 117, in solve_for_transaction
should_retry_solve)
File "/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/core/solve.py", line 158, in solve_for_diff
force_remove, should_retry_solve)
File "/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/core/solve.py", line 281, in solve_final_state
ssc = self._run_sat(ssc)
File "/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/core/solve.py", line 808, in _run_sat
should_retry_solve=ssc.should_retry_solve
File "/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/resolve.py", line 1412, in solve
if not is_converged(solution):
File "/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/resolve.py", line 1304, in is_converged
psolution = clean(solution)
File "/srv/hops/anaconda/anaconda-3-2020.02/lib/python3.7/site-packages/conda/resolve.py", line 1293, in clean
return [q for q in (C.from_index(s) for s in sol)
TypeError: 'NoneType' object is not iterable
$ /srv/hops/anaconda/anaconda/bin/conda update anaconda -y -q
environment variables:
CIO_TEST=
CONDA_ROOT=/srv/hops/anaconda/anaconda-3-2020.02
PATH=/sbin:/bin:/usr/sbin:/usr/bin
PYTHONUNBUFFERED=1
REQUESTS_CA_BUNDLE=
SSL_CERT_FILE=
SUDO_COMMAND=/bin/chef-solo -c /home/fmarines/.karamel/install/solo.rb -j
/home/fmarines/.karamel/install/conda__install.json
SUDO_GID=1000
SUDO_UID=1000
SUDO_USER=fmarines
active environment : None
user config file : /home/anaconda/.condarc
populated config files : /home/anaconda/.condarc
conda version : 4.8.2
conda-build version : 3.18.11
python version : 3.7.6.final.0
virtual packages : __cuda=10.2
__glibc=2.17
base environment : /srv/hops/anaconda/anaconda-3-2020.02 (writable)
channel URLs : https://conda.anaconda.org/pytorch/linux-64
https://conda.anaconda.org/pytorch/noarch
main/linux-64
main/noarch
r/linux-64
r/noarch
package cache : /srv/hops/anaconda/anaconda/pkgs
envs directories : /srv/hops/anaconda/anaconda/envs
/srv/hops/anaconda/anaconda-3-2020.02/envs
/home/anaconda/.conda/envs
platform : linux-64
user-agent : conda/4.8.2 requests/2.22.0 CPython/3.7.6 Linux/3.10.0-1127.13.1.el7.x86_64 centos/7.8.2003 glibc/2.17
UID:GID : 968:1011
netrc file : None
offline mode : False
An unexpected error has occurred. Conda has prepared the above report.
Upload successful.
---- End output of “bash” “/tmp/chef-script20200723-37215-euo5m1” ----
Ran “bash” “/tmp/chef-script20200723-37215-euo5m1” returned 1