Cookbook exceding the timeout for chef-server deb download
dfduarte opened this issue · comments
Good afternoon,
Since yesterday, I'm facing a problem with the chef-server cookbook.
The cookbook runs well and when it reaches the download chef-server from repository stage, the node starts the download, and fails with a timeout error, as:
efault: * apt_package[chef-server] action install
==> default:
==> default:
==> default:
==> default: ================================================================================
==> default: Error executing action install on resource 'apt_package[chef-server]'
==> default:
==> default: ================================================================================
==> default:
==> default:
==> default:
==> default:
==> default: Mixlib::ShellOut::CommandTimeout
==> default:
==> default: --------------------------------
==> default: Command timed out after 900s:
==> default: Command exceeded allowed execution time, process terminated
==> default: ---- Begin output of apt-get -q -y install chef-server-core=12.2.0-1 ----
==> default: STDOUT: Reading package lists...
==> default: Building dependency tree...
==> default: Reading state information...
==> default: The following packages were automatically installed and are no longer required:
==> default: chef-zero erubis ohai ruby-diff-lcs ruby-erubis ruby-hashie ruby-highline
==> default: ruby-ipaddress ruby-mime-types ruby-mixlib-authentication ruby-mixlib-cli
==> default: ruby-mixlib-config ruby-mixlib-log ruby-mixlib-shellout ruby-net-ssh
==> default: ruby-net-ssh-gateway ruby-net-ssh-multi ruby-rack ruby-rest-client
==> default: ruby-sigar ruby-systemu ruby-yajl
==> default: Use 'apt-get autoremove' to remove them.
==> default: The following NEW packages will be installed:
==> default: chef-server-core
==> default: STDERR:
==> default: ---- End output of apt-get -q -y install chef-server-core=12.2.0-1 ----
==> default: Ran apt-get -q -y install chef-server-core=12.2.0-1 returned
==> default:
==> default: Resource Declaration:
==> default: ---------------------
==> default: # In /var/chef/cache/cookbooks/chef-ingredient/libraries/helpers.rb
==> default:
==> default: 46: declare_resource presource, new_resource.product_name do
==> default: 47: package_name ingredient_package_name
==> default: 48: options new_resource.options
==> default: 49: version install_version if Mixlib::Versioning.parse(version_string(new_resource.version)) > '0.0.0'
==> default: 50: source new_resource.package_source
==> default: 51: timeout new_resource.timeout
==> default: 52: action ingredient_action
==> default: 53: end
==> default: 54: end
==> default:
==> default: Compiled Resource:
==> default: ------------------
==> default: # Declared in /var/chef/cache/cookbooks/chef-ingredient/libraries/helpers.rb:46:in package_resource' ==> default: ==> default: apt_package("chef-server") do ==> default: action [:install] ==> default: retries 0 ==> default: retry_delay 2 ==> default: default_guard_interpreter :default ==> default: package_name "chef-server-core" ==> default: version "12.2.0-1" ==> default: declared_type :package ==> default: cookbook_name "chef-server" ==> default: end ==> default: ==> default: ==> default: [2015-09-10T16:16:00+00:00] INFO: Running queued delayed notifications before re-raising exception ==> default: ==> default: ==> default: ================================================================================ ==> default: Error executing actioninstallon resource 'chef_ingredient[chef-server]' ==> default: ==> default: ================================================================================ ==> default: ==> default: ==> default: ==> default: ==> default: Mixlib::ShellOut::CommandTimeout ==> default: -------------------------------- ==> default: ==> default: apt_package[chef-server] (/var/chef/cache/cookbooks/chef-ingredient/libraries/helpers.rb line 46) had an error: Mixlib::ShellOut::CommandTimeout: Command timed out after 900s: ==> default: ==> default: Command exceeded allowed execution time, process terminated ==> default: ---- Begin output of apt-get -q -y install chef-server-core=12.2.0-1 ---- ==> default: STDOUT: Reading package lists... ==> default: ==> default: ==> default: Building dependency tree... ==> default: ==> default: Reading state information... ==> default: ==> default: The following packages were automatically installed and are no longer required: ==> default: ==> default: chef-zero erubis ohai ruby-diff-lcs ruby-erubis ruby-hashie ruby-highline ==> default: ruby-ipaddress ruby-mime-types ruby-mixlib-authentication ruby-mixlib-cli ==> default: ruby-mixlib-config ruby-mixlib-log ruby-mixlib-shellout ruby-net-ssh ==> default: ==> default: ruby-net-ssh-gateway ruby-net-ssh-multi ruby-rack ruby-rest-client ==> default: ruby-sigar ruby-systemu ruby-yajl ==> default: Use 'apt-get autoremove' to remove them. ==> default: The following NEW packages will be installed: ==> default: chef-server-core ==> default: STDERR: ==> default: ==> default: ---- End output of apt-get -q -y install chef-server-core=12.2.0-1 ---- ==> default: ==> default: Ran apt-get -q -y install chef-server-core=12.2.0-1 returned ==> default: ==> default: ==> default: ==> default: Resource Declaration: ==> default: ==> default: --------------------- ==> default: ==> default: # In /var/chef/cache/cookbooks/chef-server/recipes/default.rb ==> default: ==> default: 27: chef_ingredient 'chef-server' do ==> default: ==> default: ==> default: 28: extend ChefServerCoobook::Helpers ==> default: 29: version node['chef-server']['version'] ==> default: ==> default: 30: package_source node['chef-server']['package_source'] ==> default: 31: config <<-EOS ==> default: ==> default: 32: topology "#{node['chef-server']['topology']}" ==> default: 33: #{"api_fqdn \"#{node['chef-server']['api_fqdn']}\"" if api_fqdn_available?} ==> default: 34: #{node['chef-server']['configuration']} ==> default: ==> default: 35: EOS ==> default: 36: action :install ==> default: ==> default: 37: end ==> default: ==> default: 38: ==> default: ==> default: ==> default: ==> default: ==> default: Compiled Resource: ==> default: ==> default: ------------------ ==> default: # Declared in /var/chef/cache/cookbooks/chef-server/recipes/default.rb:27:infrom_file'
==> default:
==> default:
==> default:
==> default: chef_ingredient("chef-server") do
==> default:
==> default: action [:install]
==> default:
==> default: updated true
==> default:
==> default: updated_by_last_action true
==> default:
==> default: retries 0
==> default:
==> default: retry_delay 2
==> default:
==> default: default_guard_interpreter :default
==> default:
==> default: declared_type :chef_ingredient
==> default:
==> default: cookbook_name "chef-server"
==> default: recipe_name "default"
==> default:
==> default: version :latest
==> default:
==> default: config "topology "standalone"\napi_fqdn "vagrant-ubuntu-trusty-64"\n\n"
==> default:
==> default: product_name "chef-server"
==> default: end
==> default:
==> default:
==> default: [2015-09-10T16:16:00+00:00] INFO: Running queued delayed notifications before re-raising exception
==> default:
==> default: Running handlers:
==> default: [2015-09-10T16:16:00+00:00] ERROR: Running exception handlers
==> default: Running handlers complete
==> default:
==> default: [2015-09-10T16:16:00+00:00] ERROR: Exception handlers complete
==> default: Chef Client failed. 9 resources updated in 988.655551312 seconds
==> default: [2015-09-10T16:16:00+00:00] INFO: Sending resource update report (run-id: 5d590a0a-7958-4824-9171-7774db100bb7)
==> default: [2015-09-10T16:16:01+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
==> default: [2015-09-10T16:16:01+00:00] ERROR: chef_ingredient[chef-server](chef-server::default line 27) had an error: Mixlib::ShellOut::CommandTimeout: apt_package[chef-server](/var/chef/cache/cookbooks/chef-ingredient/libraries/helpers.rb line 46) had an error: Mixlib::ShellOut::CommandTimeout: Command timed out after 900s:
==> default: Command exceeded allowed execution time, process terminated
==> default: ---- Begin output of apt-get -q -y install chef-server-core=12.2.0-1 ----
==> default: STDOUT: Reading package lists...
==> default: Building dependency tree...
==> default: Reading state information...
==> default: The following packages were automatically installed and are no longer required:
==> default: chef-zero erubis ohai ruby-diff-lcs ruby-erubis ruby-hashie ruby-highline
==> default: ruby-ipaddress ruby-mime-types ruby-mixlib-authentication ruby-mixlib-cli
==> default: ruby-mixlib-config ruby-mixlib-log ruby-mixlib-shellout ruby-net-ssh
==> default: ruby-net-ssh-gateway ruby-net-ssh-multi ruby-rack ruby-rest-client
==> default: ruby-sigar ruby-systemu ruby-yajl
==> default: Use 'apt-get autoremove' to remove them.
==> default: The following NEW packages will be installed:
==> default: chef-server-core
==> default: STDERR:
==> default: ---- End output of apt-get -q -y install chef-server-core=12.2.0-1 ----
==> default: Ran apt-get -q -y install chef-server-core=12.2.0-1 returned
==> default: [2015-09-10T16:16:01+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
Chef never successfully completed! Any errors should be visible in the
output above. Please fix your recipes so that they properly complete.
Note: This errors appears either the AWS, or Vagrant machines.
Why is this issue closed? What is the solution to fix this?
I just encountered the same issue...
I closed since nobody has reported and from my side is ok. There is a way to re-open this ticket, then?
Reopened as requested by @turkenh
Hi, I have got the same error! The real problem seems to be that the gpg key for the repo is not imported.
==============================================================================
Error executing action `install` on resource 'chef_ingredient[chef-server]'
==============================================================================
Mixlib::ShellOut::CommandTimeout
--------------------------------
apt_package[chef-server] (/var/chef/cookbooks/chef-ingredient/libraries/debian_handler.rb line 71) had an error: Mixlib::ShellOut::CommandTimeout: Command timed out after 900s:
Command exceeded allowed execution time, process terminated
---- Begin output of apt-get -q -y install chef-server-core=12.4.1-1 ----
STDOUT: Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
chef-zero erubis ohai ruby-diff-lcs ruby-erubis ruby-hashie ruby-highline
ruby-ipaddress ruby-mime-types ruby-mixlib-authentication ruby-mixlib-cli
ruby-mixlib-config ruby-mixlib-log ruby-mixlib-shellout ruby-net-ssh
ruby-net-ssh-gateway ruby-net-ssh-multi ruby-rack ruby-rest-client
ruby-sigar ruby-systemu ruby-yajl
Use 'apt-get autoremove' to remove them.
The following NEW packages will be installed:
chef-server-core
STDERR:
---- End output of apt-get -q -y install chef-server-core=12.4.1-1 ----
Ran apt-get -q -y install chef-server-core=12.4.1-1 returned
When we execute the failing apt command we get a little hint:
root@chef:/vagrant# apt-get -q -y install chef-server-core=12.4.1-1
Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
chef-zero erubis ohai ruby-diff-lcs ruby-erubis ruby-hashie ruby-highline
ruby-ipaddress ruby-mime-types ruby-mixlib-authentication ruby-mixlib-cli
ruby-mixlib-config ruby-mixlib-log ruby-mixlib-shellout ruby-net-ssh
ruby-net-ssh-gateway ruby-net-ssh-multi ruby-rack ruby-rest-client
ruby-sigar ruby-systemu ruby-yajl
Use 'apt-get autoremove' to remove them.
The following NEW packages will be installed:
chef-server-core
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 463 MB of archives.
After this operation, 1,225 MB of additional disk space will be used.
Fetched 463 MB in 7min 32s (1,025 kB/s)
E: Failed to fetch https://packagecloud.io/chef/stable/ubuntu/pool/trusty/main/c/chef-server-core/chef-server-core_12.4.1-1_amd64.deb Hash Sum mismatch
The last two words! hashsum mismatch! OK, the real error can be seen after an apt-get update:
W: GPG error: https://packagecloud.io trusty InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 37BBEE3F7AD95B3F
So there is no valid gpg key for the package cloud repo. This key should be imported, then it must work.
I am not sure atm how this counts in, but tis recipe also installed a wrong package repository - in /etc/apt/sources.list.d/ I see:
deb [trusted=yes] https://packagecloud.io/chef/stable/ubuntu/ lucid main
but this is a trusty server.
I've added a key curl https://packagecloud.io/gpg.key | sudo /usr/bin/apt-key add - now apt-get update isn't complaining about the lack of a key, but I still get Failed to fetch https://packagecloud.io/chef/stable/ubuntu/pool/trusty/main/c/chef-server-core/chef-server-core_12.4.1-1_amd64.deb Hash Sum mismatch
and just to re the above comment, this is trusty and i see: cat /etc/apt/sources.list.d/chef-stable.list deb [trusted=yes] "https://packagecloud.io/chef/stable/ubuntu/" trusty main
@tomharvey I am not able to reproduce this in kitchen just yet -- is aside from setting node['chef-server']['version'] = 12.4.1 is there anything else you are pinning by any chance? Any additional information would be appreciated.
I'm getting the same results as @devopstiger when I run the install command manually.
See https://github.com/cat5inthecradle/chef-server-vagrant/tree/4e4df3251108aadf76731f01eebb67860db9c2d9 for my full dev environment.
sudo apt-get -q -y install chef-server-core=12.4.1-1 returns
Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
chef-zero erubis ohai ruby-diff-lcs ruby-erubis ruby-hashie ruby-highline
ruby-ipaddress ruby-mime-types ruby-mixlib-authentication ruby-mixlib-cli
ruby-mixlib-config ruby-mixlib-log ruby-mixlib-shellout ruby-net-ssh
ruby-net-ssh-gateway ruby-net-ssh-multi ruby-rack ruby-rest-client
ruby-sigar ruby-systemu ruby-yajl
Use 'apt-get autoremove' to remove them.
The following NEW packages will be installed:
chef-server-core
0 upgraded, 1 newly installed, 0 to remove and 8 not upgraded.
Need to get 463 MB of archives.
After this operation, 1,225 MB of additional disk space will be used.
Fetched 463 MB in 3min 5s (2,493 kB/s)
E: Failed to fetch https://packagecloud.io/chef/stable/ubuntu/pool/trusty/main/c/chef-server-core/chef-server-core_12.4.1-1_amd64.deb Hash Sum mismatch
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
When I run sudo apt-get update --fix-missing I get the following as the last line of output:
W: GPG error: https://packagecloud.io trusty InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 37BBEE3F7AD95B3F
This was fixed for ubuntu 10.04 and debian 6 based things in chef-ingredient 0.17.0, I'm guessing this will be resolved by doing a berks update chef-ingredient
The chef-ingredient dependency for this chef-server cookbook is 0.11.0, berks update doesn't change it - so I added 0.17.0 as a dependency for my project. Currently running now but looks like it's going to timeout again.
The depends is a >= depends so a berks update will bring in the latest cookbook unless you're constrained elsewhere in the node runlist. I'm going to close this out though since we fixed the issue in chef-ingredient.