test-kitchen / busser

Originally reported at test-kitchen/test-kitchen#616 but re-reporting here as the symptoms of that bug are now fixed, but the cause remains.

On Windows, the batchfile binstubs generated by busser in %TEMP%\verifier\gems\bin\ do not point at a valid ruby. The ruby.exe that they try to call does not exist on the machine.

In my case, bundle.bat looks like this:

C:\Users\vagrant>more C:\Users\vagrant\AppData\Local\Temp\verifier\gems\bin\bundle.bat                                                                                                                                               
@ECHO OFF                                                                                                                                                                                                                            
IF NOT "%~f0" == "~f0" GOTO :WinNT                                                                                                                                                                                                   
@"C:\Users\vagrant\AppData\Local\Temp\verifier\gems\bin\ruby.exe" "C:/Users/vagrant/AppData/Local/Temp/verifier/gems/bin/bundle" %1 %2 %3 %4 %5 %6 %7 %8 %9                                                                          
GOTO :EOF                                                                                                                                                                                                                            
:WinNT                                                                                                                                                                                                                               
@"%~dp0ruby.exe" "%~dpn0" %*

Where bundle in the same directory, looks like this:

C:\Users\vagrant>more C:\Users\vagrant\AppData\Local\Temp\verifier\gems\bin\bundle                                                                                                                                                   
#!C:/opscode/chef/embedded/bin/ruby.exe                                                                                                                                                                                              
#                                                                                                                                                                                                                                    
# This file was generated by RubyGems.                                                                                                                                                                                               
#                                                                    
....

The latter is correct.

This leads to errors when trying to run binaries, even with the correct environment. An example would be running serverspec tests with a Gemfile present (which runs 'bundle'):

-> bundle exec kitchen verify default-opentable-win-2012r2-standard-amd64-nocm
-----> Starting Kitchen (v1.4.0.rc.1)
-----> Verifying <default-opentable-win-2012r2-standard-amd64-nocm>...
       Preparing files for transfer
-----> Busser installation detected (busser)
       Installing Busser plugins: busser-serverspec
       Plugin serverspec already installed
       Removing C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec
       Transferring files to <default-opentable-win-2012r2-standard-amd64-nocm>
-----> Running serverspec test suite
-----> Bundle Installing..
         run  bundle install --gemfile C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec/Gemfile --local || bundle install --gemfile C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec/Gemfile from "."
       C:/opscode/chef/embedded/bin/ruby.exe -IC:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec -I'C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-support-3.2.2/lib';'C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib' 'C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/exe/rspec' --pattern 'C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec/**/*_spec.rb' --color --format documentation --default-path C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec
$$$$$$ C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/busser-serverspec-0.5.5/lib/busser/runner_plugin/serverspec.rb:49:in `run_bundle_install': Use RbConfig instead of obsolete and deprecated Config.
$$$$$$ C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/busser-serverspec-0.5.5/lib/busser/runner_plugin/serverspec.rb:49:in `run_bundle_install': Use RbConfig instead of obsolete and deprecated Config.
$$$$$$ '"C:\Users\vagrant\AppData\Local\Temp\verifier\gems\bin\ruby.exe"' is not recognized as an internal or external command,
$$$$$$ operable program or batch file.
$$$$$$ '"C:\Users\vagrant\AppData\Local\Temp\verifier\gems\bin\ruby.exe"' is not recognized as an internal or external command,
$$$$$$ operable program or batch file.
$$$$$$ C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec/spec_helper.rb:17:in `<top (required)>': uninitialized constant Rspec (NameError)
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec/default_spec.rb:1:in `require_relative'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec/default_spec.rb:1:in `<top (required)>'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib/rspec/core/configuration.rb:1226:in `load'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib/rspec/core/configuration.rb:1226:in `block in load_spec_files'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib/rspec/core/configuration.rb:1224:in `each'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib/rspec/core/configuration.rb:1224:in `load_spec_files'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:97:in `setup'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:85:in `run'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:70:in `run'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:38:in `invoke'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/exe/rspec:4:in `<main>'
$$$$$$ C:/opscode/chef/embedded/bin/ruby.exe -IC:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec -I'C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-support-3.2.2/lib';'C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib' 'C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/exe/rspec' --pattern 'C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec/**/*_spec.rb' --color --format documentation --default-path C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec failed
$$$$$$ !!!!!! Ruby Script [C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/busser-serverspec-0.5.5/lib/busser/runner_plugin/../serverspec/runner.rb C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec] exit code was 1
>>>>>> Verify failed on instance <default-opentable-win-2012r2-standard-amd64-nocm>.
>>>>>> Please see .kitchen/logs/default-opentable-win-2012r2-standard-amd64-nocm.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: WinRM exited (1) for command: [
$env:BUSSER_ROOT = "$env:TEMP\verifier"
$env:GEM_HOME = "$env:TEMP\verifier\gems"
$env:GEM_PATH = "$env:TEMP\verifier\gems"
$env:GEM_CACHE = "$env:TEMP\verifier\gems\cache"

& $env:TEMP\verifier\bin\busser.bat test
]
>>>>>> ----------------------

Note, this error no longer occurs as it was worked around by test-kitchen/busser-serverspec#27 which calls bundler explicitly via /path/to/ruby.exe /path/to/bundle/script instead.

Agree, that is odd. However, the fix in busser-serverspec is ultimately the right way to resolve this--I've had issues getting the RubyGems-build .bat wrapper coming out consistently correctly and the test-kitchen code falls back to using the /path/to/ruby /path/to/ruby/binstub pattern.

I'll investigate this a little to see if Busser is setting up an incorrect path to Ruby, but otherwise I'm hoping we're okay here.

@cheeseplus It seems

busser/lib/busser/command/setup.rb

Lines 59 to 97 in 13115a2

    
                 def generate_busser_binstub_for_bat 
        
                   binstub = root_path + "bin/busser.bat" 
        
                   busser_root = root_path.to_s.gsub("/", "\\") 
        
                   File.unlink(binstub) if File.exists?(binstub) 
        
                   create_file(binstub, :verbose => false) do 
        
                     <<-BUSSER_BINSTUB.gsub(/^ {12}/, '') 
        
                       @ECHO OFF 
        
                       REM This file was generated by Busser. 
        
                       REM 
        
                       REM The application 'busser' is installed as part of a gem, and 
        
                       REM this file is here to facilitate running it. 
        
                       REM Make sure any variables we set exist only for this batch file 
        
                       SETLOCAL 
        
                       REM Set Busser Root Path 
        
                       SET "BUSSER_ROOT=#{busser_root}" 
        
                       REM Export gem paths so that we use the isolated gems. 
        
                       SET "GEM_HOME=#{gem_home}" 
        
                       SET "GEM_PATH=#{gem_path}" 
        
                       SET "GEM_CACHE=#{gem_home}\\cache" 
        
                       REM Unset RUBYOPT, we don't want this bleeding into our runtime. 
        
                       SET RUBYOPT= 
        
                       SET GEMRC= 
        
                       REM Call the actual Busser bin with our arguments 
        
                       "#{ruby_bin}" "#{gem_bindir}\\busser" %* 
        
                       REM Store the exit status so we can re-use it later 
        
                       SET "BUSSER_EXIT_STATUS=%ERRORLEVEL%" 
        
                       REM Exit with the proper exit status from Busser 
        
                       exit /b %BUSSER_EXIT_STATUS% 
        
                     BUSSER_BINSTUB 
        
                   end 
        
                 end

is not being written in Window 2012r2 for Chef 11. Instead, the default bat file created by rubygems (https://github.com/rubygems/rubygems/search?utf8=%E2%9C%93&q=goto) is being used. This fails because ruby is not on the path when test-kitchen calls it.

busser.bat file created by rubygems installer:

@ECHO OFF
IF NOT "%~f0" == "~f0" GOTO :WinNT
@"ruby.exe" "C:/Users/vagrant/AppData/Local/Temp/verifier/bin/busser" %1 %2 %3 %4 %5 %6 %7 %8 %9
GOTO :EOF
:WinNT
@"ruby.exe" "%~dpn0" %*

@dhoer / @cheeseplus We're seeing the same thing in EC2. Currently, our CI uses chef-client 11.18.6. Although, we use Chef-Client 12 for local development. We cannot yet upgrade to Chef-Client 12 because our Production estate is still using Chef-Client 11.

Is there a way to fix this ? I'm trying to investigate what exactly has changed. Because this worked on the 24th of March.

This seems to reproducible with
ChefDK 0.12.0,
TK 1.6.0,
Chef-Client 11.18.6
Only in AWS EC2.

Here is a snippet of a kitchen test.

-----> Verifying <default-Windows2012>...
       Preparing files for transfer
D      Creating local sandbox in C:/Users/CHRIST~1.CLA/AppData/Local/Temp/default-Windows2012-sandbox-20160404-33148-vl3r8k
D      [WinRM] reusing existing connection negotiate::http://10.103.2.130:5985/wsman<{:user=>"vagrant", :pass=>"vagrant", :no_ssl_peer_verification=>true, :disable_sspi=>false, :basic_auth_only=>false}>
D      [WinRM] negotiate::http://10.103.2.130:5985/wsman<{:user=>"vagrant", :pass=>"vagrant", :no_ssl_peer_verification=>true, :disable_sspi=>false, :basic_auth_only=>false}> (
$env:BUSSER_ROOT = "$env:TEMP\verifier"
$env:GEM_HOME = "$env:TEMP\verifier\gems"
$env:GEM_PATH = "$env:TEMP\verifier\gems"
$env:GEM_CACHE = "$env:TEMP\verifier\gems\cache"
$ruby = "$env:systemdrive\opscode\chef\embedded\bin\ruby.exe"
$gem = "$env:systemdrive\opscode\chef\embedded\bin\gem"
$version = "busser"
$gem_install_args = "busser --no-rdoc --no-ri --no-format-executable -n $env:TEMP\verifier\bin --no-user-install"
$busser = "$env:TEMP\verifier\bin\busser.bat"
$plugins = "busser-serverspec"

if ((& "$ruby" "$gem" list busser -i) -ne "true") {
  Write-Host "-----> Installing Busser ($version)`n"
  & "$ruby" "$gem" install $gem_install_args.Split() 2>&1
} else {
  Write-Host "-----> Busser installation detected ($version)`n"
}

if (-Not (Test-Path "$busser")) {
  $gem_bindir = & "$ruby" -rrubygems -e "puts Gem.bindir.dup.gsub('/', '\\')"
  & "$ruby" "$gem_bindir\busser" setup --type bat 2>&1
}

Write-Host "       Installing Busser plugins: $plugins`n"
& "$busser" plugin install $plugins.Split() 2>&1
)
-----> Installing Busser (busser)

       Successfully installed thor-0.19.0
       Successfully installed busser-0.7.1
       2 gems installed
       Installing Busser plugins: busser-serverspec

$$$$$$ busser.bat : '"ruby.exe"' is not recognized as an internal or external command,
$$$$$$ At line:26 char:1
$$$$$$ + &amp; "$busser" plugin install $plugins.Split() 2&gt;&amp;1
$$$$$$ + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
$$$$$$     + CategoryInfo          : NotSpecified: ('"ruby.exe"' is...ternal command,
$$$$$$    :String) [], RemoteException
$$$$$$     + FullyQualifiedErrorId : NativeCommandError
$$$$$$
D      Cleaning up local sandbox in C:/Users/CHRIST~1.CLA/AppData/Local/Temp/default-Windows2012-sandbox-20160404-33148-vl3r8k
>>>>>> Verify failed on instance <default-Windows2012>.
>>>>>> Please see .kitchen/logs/default-Windows2012.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: WinRM exited (1) for command: [
$env:BUSSER_ROOT = "$env:TEMP\verifier"
$env:GEM_HOME = "$env:TEMP\verifier\gems"
$env:GEM_PATH = "$env:TEMP\verifier\gems"
$env:GEM_CACHE = "$env:TEMP\verifier\gems\cache"
$ruby = "$env:systemdrive\opscode\chef\embedded\bin\ruby.exe"
$gem = "$env:systemdrive\opscode\chef\embedded\bin\gem"
$version = "busser"
$gem_install_args = "busser --no-rdoc --no-ri --no-format-executable -n $env:TEMP\verifier\bin --no-user-install"
$busser = "$env:TEMP\verifier\bin\busser.bat"
$plugins = "busser-serverspec"

if ((& "$ruby" "$gem" list busser -i) -ne "true") {
  Write-Host "-----> Installing Busser ($version)`n"
  & "$ruby" "$gem" install $gem_install_args.Split() 2>&1
} else {
  Write-Host "-----> Busser installation detected ($version)`n"
}

if (-Not (Test-Path "$busser")) {
  $gem_bindir = & "$ruby" -rrubygems -e "puts Gem.bindir.dup.gsub('/', '\\')"
  & "$ruby" "$gem_bindir\busser" setup --type bat 2>&1
}

Write-Host "       Installing Busser plugins: $plugins`n"
& "$busser" plugin install $plugins.Split() 2>&1
]
>>>>>> ----------------------
D      Verify failed on instance <default-Windows2012>.
D      ------Exception-------
D      Class: Kitchen::InstanceFailure
D      Message: Verify failed on instance <default-Windows2012>.  Please see .kitchen/logs/default-Windows2012.log for more details
D      ---Nested Exception---
D      Class: Kitchen::ActionFailed
D      Message: WinRM exited (1) for command: [
$env:BUSSER_ROOT = "$env:TEMP\verifier"
$env:GEM_HOME = "$env:TEMP\verifier\gems"
$env:GEM_PATH = "$env:TEMP\verifier\gems"
$env:GEM_CACHE = "$env:TEMP\verifier\gems\cache"
$ruby = "$env:systemdrive\opscode\chef\embedded\bin\ruby.exe"
$gem = "$env:systemdrive\opscode\chef\embedded\bin\gem"
$version = "busser"
$gem_install_args = "busser --no-rdoc --no-ri --no-format-executable -n $env:TEMP\verifier\bin --no-user-install"
$busser = "$env:TEMP\verifier\bin\busser.bat"
$plugins = "busser-serverspec"

if ((& "$ruby" "$gem" list busser -i) -ne "true") {
  Write-Host "-----> Installing Busser ($version)`n"
  & "$ruby" "$gem" install $gem_install_args.Split() 2>&1
} else {
  Write-Host "-----> Busser installation detected ($version)`n"
}

if (-Not (Test-Path "$busser")) {
  $gem_bindir = & "$ruby" -rrubygems -e "puts Gem.bindir.dup.gsub('/', '\\')"
  & "$ruby" "$gem_bindir\busser" setup --type bat 2>&1
}

Write-Host "       Installing Busser plugins: $plugins`n"
& "$busser" plugin install $plugins.Split() 2>&1
]
D      ------Backtrace-------
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.6.0/lib/kitchen/verifier/base.rb:83:in `rescue in call'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.6.0/lib/kitchen/verifier/base.rb:86:in `call' D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.6.0/lib/kitchen/instance.rb:423:in `block in verify_action'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.6.0/lib/kitchen/instance.rb:513:in `call'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.6.0/lib/kitchen/instance.rb:513:in `synchronize_or_call'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.6.0/lib/kitchen/instance.rb:478:in `block in action'
D      C:/opscode/chefdk/embedded/lib/ruby/2.1.0/benchmark.rb:279:in `measure'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.6.0/lib/kitchen/instance.rb:477:in `action'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.6.0/lib/kitchen/instance.rb:415:in `verify_action'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.6.0/lib/kitchen/instance.rb:348:in `block in transition_to'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.6.0/lib/kitchen/instance.rb:347:in `each'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.6.0/lib/kitchen/instance.rb:347:in `transition_to'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.6.0/lib/kitchen/instance.rb:160:in `verify'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.6.0/lib/kitchen/instance.rb:189:in `block in test'
D      C:/opscode/chefdk/embedded/lib/ruby/2.1.0/benchmark.rb:279:in `measure'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.6.0/lib/kitchen/instance.rb:185:in `test'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.6.0/lib/kitchen/command.rb:176:in `public_send'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.6.0/lib/kitchen/command.rb:176:in `block (2 levels) in run_action'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/logging-2.1.0/lib/logging/diagnostic_context.rb:450:in `call'

D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/logging-2.1.0/lib/logging/diagnostic_context.rb:450:in `block in create_with_logging_context'
D      ----------------------
D      [WinRM] closing remote shell F024F848-44CD-4CD6-AF29-0271037E7A95 on http://10.103.2.130:5985/wsman
D      [WinRM] remote shell F024F848-44CD-4CD6-AF29-0271037E7A95 closed

I have submitted a fix for this here: test-kitchen/test-kitchen#1003

Thanks @mwrock! 👍

I see test-kitchen released v1.7.2 yesterday. When can we expect this patch to be released?

Likely sometime before the next dk this month.

	def generate_busser_binstub_for_bat
	binstub = root_path + "bin/busser.bat"
	busser_root = root_path.to_s.gsub("/", "\\")

	File.unlink(binstub) if File.exists?(binstub)
	create_file(binstub, :verbose => false) do
	<<-BUSSER_BINSTUB.gsub(/^ {12}/, '')
	@ECHO OFF
	REM This file was generated by Busser.
	REM
	REM The application 'busser' is installed as part of a gem, and
	REM this file is here to facilitate running it.

	REM Make sure any variables we set exist only for this batch file
	SETLOCAL

	REM Set Busser Root Path
	SET "BUSSER_ROOT=#{busser_root}"

	REM Export gem paths so that we use the isolated gems.
	SET "GEM_HOME=#{gem_home}"
	SET "GEM_PATH=#{gem_path}"
	SET "GEM_CACHE=#{gem_home}\\cache"

	REM Unset RUBYOPT, we don't want this bleeding into our runtime.
	SET RUBYOPT=
	SET GEMRC=

	REM Call the actual Busser bin with our arguments
	"#{ruby_bin}" "#{gem_bindir}\\busser" %*

	REM Store the exit status so we can re-use it later
	SET "BUSSER_EXIT_STATUS=%ERRORLEVEL%"

	REM Exit with the proper exit status from Busser
	exit /b %BUSSER_EXIT_STATUS%
	BUSSER_BINSTUB
	end
	end

Binstubs in 'verifier' do not point at a valid ruby interpreter on Windows