Windows: puma_http11.so breaking when ruby folder is relocated
nathan-nhm opened this issue · comments
I'm trying to repackage rubyinstaller-3.2.2-1-x64.7z with binary gems compiled, including Puma, so we don't have to install msys64 on customer's machines.
We don't always dictate where we install. It may go on C: , P:, N:
When unzipped on a different drive - paths properly set up - I get the dump
<internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:38:in `require': 126: The specified module could not be found. - p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/puma-6.4.0/lib/puma/puma_http11.so (LoadError)
from <internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:38:in `require'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/bootsnap-1.17.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/zeitwerk-2.6.12/lib/zeitwerk/kernel.rb:38:in `require'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/puma-6.4.0/lib/puma.rb:12:in `<main>'
from <internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:38:in `require'
from <internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:38:in `require'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/bootsnap-1.17.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/zeitwerk-2.6.12/lib/zeitwerk/kernel.rb:38:in `require'
from p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/bundler/runtime.rb:60:in `block (2 levels) in require'
from p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/bundler/runtime.rb:55:in `each'
from p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/bundler/runtime.rb:55:in `block in require'
from p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/bundler/runtime.rb:44:in `each'
from p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/bundler/runtime.rb:44:in `require'
from p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/bundler.rb:187:in `require'
from P:/NHM/mdw63/plugins/mdw_core/config/application.rb:19:in `<main>'
from <internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:38:in `require'
from <internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:38:in `require'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/bootsnap-1.17.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
from P:/NHM/mdw63/solutions/automator/config/application.rb:1:in `<main>'
from <internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:38:in `require'
from <internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:38:in `require'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/bootsnap-1.17.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/railties-7.1.1/lib/rails/commands/server/server_command.rb:139:in `block in perform'
from <internal:kernel>:90:in `tap'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/railties-7.1.1/lib/rails/commands/server/server_command.rb:136:in `perform'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/thor-1.3.0/lib/thor/command.rb:28:in `run'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/thor-1.3.0/lib/thor/invocation.rb:127:in `invoke_command'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/railties-7.1.1/lib/rails/command/base.rb:178:in `invoke_command'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/thor-1.3.0/lib/thor.rb:527:in `dispatch'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/railties-7.1.1/lib/rails/command/base.rb:73:in `perform'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/railties-7.1.1/lib/rails/command.rb:71:in `block in invoke'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/railties-7.1.1/lib/rails/command.rb:149:in `with_argv'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/railties-7.1.1/lib/rails/command.rb:69:in `invoke'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/railties-7.1.1/lib/rails/commands.rb:18:in `<main>'
from <internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:38:in `require'
from <internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:38:in `require'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/bootsnap-1.17.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
from bin/rails:4:in `<main>'
p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/puma-6.4.0/lib/puma/puma_http11.so Exists.
P:\NHM\Ruby3221a\lib\ruby\gems\3.2.0\extensions\x64-mingw-ucrt\3.2.0\puma-6.4.0\puma/puma_http11.so also exists.
If I force re-installation of the gem - so it builds with the paths correct for the system - I have no dump, the server starts. But again, would like to avoid compiling on customer systems if possible.
Normally The specified module could not be found indicates that a dll cannot be loaded. Does require 'openssl' work on the p drive?
Did you change anything in the bin/ruby_builtin_dlls folder? Unlike *unix based systems, Windows does not store 'paths' for library/dll files, they're found in other ways.
Irb - require works, class responds after.
rails console - After puma is commented out / bundle updated, OpenSSL is pre-loaded.
(Incedentally, with puma not in the Gemfile, webrick does run.)
Nothing intentionally in bin/ruby_builtin_dlls - I did update versions of the default gems but it doesn't look like it touched that folder. (all date modifieds are 1 April 2023)
So libcrypto-3-x64.dll & libssl-3-x64.dll are both in bin/ruby_builtin_dlls and ruby_builtin_dlls.manifest also exists (XML doc)?
They do.
What is the output from:
ruby -rpuma -e "puts Puma::MiniSSL::OPENSSL_LIBRARY_VERSION"
>ruby -rpuma -e "puts Puma::MiniSSL::OPENSSL_LIBRARY_VERSION"
<internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require': 126: The specified module could not be found. - p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/puma-6.4.0/lib/puma/puma_http11.so (LoadError)
from <internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
from p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/puma-6.4.0/lib/puma.rb:12:in `<top (required)>'
from <internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:160:in `require'
from <internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:160:in `rescue in require'
from <internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:40:in `require'
<internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require': cannot load such file -- puma (LoadError)
from <internal:p:/NHM/Ruby3221a/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
At least it's a more compact error.
And yes, p:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/puma-6.4.0/lib/puma/puma_http11.so does exist.
Ok. This shouldn't be happening. I'm going to copy my Ruby 3.2.2 folder to my D drive, and see what happens. Problem is, I'm about 100% sure it will work. Give me a minute.
I should mention - Ruby is not in PATH- In my case, it's in %RUBY3_PATH% if I need a script to resolve it. If that has anything to do with puma's resolution.
(So If I need the temporary enhancement, it's )
@echo off
setlocal
SET PATH=%RUBY3_PATH%;%PATH%
(because I need to run multiple rubies. And no, no change if the older ruby is running or not)
because I need to run multiple rubies
Years ago I wrote a PowerShell path switcher.
Anyway, I see you've updated rubygems/bundler. That may also cause an issue.
But, the command I listed should work. I copied the Ruby from C: to D:, and the code loads Puma. Below is console output, note that I didn't switch Path:
PS C:\> ruby -v
ruby 3.3.0dev (2023-11-14T14:34:48Z master 336d81a671) [x64-mingw-ucrt]
PS C:\> gem list puma
*** LOCAL GEMS ***
PS C:\> D:/Ruby32-x64/bin/ruby -vrpuma -e "puts Puma::MiniSSL::OPENSSL_LIBRARY_VERSION, Puma::MiniSSL::OPENSSL_VERSION"
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x64-mingw-ucrt]
OpenSSL 3.1.0 14 Mar 2023
OpenSSL 3.1.4 24 Oct 2023
PS C:\>
If I rename libcrypto-3-x64.dll or libssl-3-x64.dll in bin/ruby_builtin_dlls, I get exactly the same error you've shown.
Just to clarify, Puma was compiled/installed with Ruby 3.2.x?
Last item, lets check the puma file for the dll files it's loading. Using a PowerShell terminal, run the following (change paths to match your system):
C:/msys64/ucrt64/bin/objdump.exe -p D:/Ruby32-x64/lib/ruby/gems/3.2.0/gems/puma-6.4.0/lib/puma/puma_http11.so | Select-String -Pattern 'DLL Name:' | Sort-Object
Output should be identical to:
DLL Name: api-ms-win-crt-environment-l1-1-0.dll
DLL Name: api-ms-win-crt-heap-l1-1-0.dll
DLL Name: api-ms-win-crt-private-l1-1-0.dll
DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
DLL Name: api-ms-win-crt-string-l1-1-0.dll
DLL Name: api-ms-win-crt-time-l1-1-0.dll
DLL Name: api-ms-win-crt-utility-l1-1-0.dll
DLL Name: KERNEL32.dll
DLL Name: libcrypto-3-x64.dll
DLL Name: libssl-3-x64.dll
DLL Name: x64-ucrt-ruby320.dll
- Yes, Ruby 3.2.2
p:/NHM/msys64/ucrt64/bin/objdump.exe -p P:/NHM/Ruby3221a/lib/ruby/gems/3.2.0/gems/puma-6.4.0/lib/puma/puma_http1
1.so | Select-String -Pattern 'DLL Name:' | Sort-Object
DLL Name: api-ms-win-crt-environment-l1-1-0.dll
DLL Name: api-ms-win-crt-heap-l1-1-0.dll
DLL Name: api-ms-win-crt-private-l1-1-0.dll
DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
DLL Name: api-ms-win-crt-string-l1-1-0.dll
DLL Name: api-ms-win-crt-time-l1-1-0.dll
DLL Name: api-ms-win-crt-utility-l1-1-0.dll
DLL Name: KERNEL32.dll
DLL Name: libcrypto-1_1-x64.dll
DLL Name: libssl-1_1-x64.dll
DLL Name: x64-ucrt-ruby320.dll
So if there's a difference, I'm not seeing it.
I guess then I report back when I figure out how I screwed myself up, right?
(maybe I messed somthing up at the original compile time when old ruby was still in system path)
(Nope, still getting it)
So if there's a difference, I'm not seeing it.
I guess then I report back when I figure out how I screwed myself up, right?
No, actually the problem has a pretty simple fix. You need to update the openssl package in your MSYS2 install. Your Ruby (3.2.2) uses (and has bundled) OpenSSL 3. Your MSYS2 has OpenSSL 1.1. But, Windows Rubies 2.5 thru 3.1 are built with OpenSSL 1.1, so be aware of that issue.
Notice the difference:
Your output
DLL Name: libcrypto-1_1-x64.dll
DLL Name: libssl-1_1-x64.dll
My output
DLL Name: libcrypto-3-x64.dll
DLL Name: libssl-3-x64.dll
I think if you do the following in a windows shell, you should be good to go.
You may need to do it twice (the console may give you a message like To complete this update all MSYS2 processes including this terminal will be closed, let it close, and run the command again. If you have more than one MSYS2 install, ping me back.
ridk exec pacman -Syuu
JFYI, I've been involved with 'Windows' Ruby since Ruby 2.4, I also work with OpenSSL & Puma.
ridk exec pacman -Syuu
Ran twice ... how did I not recognize pacman the first time
ruby -vrpuma -e "puts Puma::MiniSSL::OPENSSL_LIBRARY_VERSION, Puma::MiniSSL::OPENSSL_VERSION"
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x64-mingw-ucrt]
OpenSSL 3.1.0 14 Mar 2023
OpenSSL 3.1.4 24 Oct 2023
And works after being relocated now.
Lovely. Thank you so much @MSP-Greg
Checking Ruby's OpenSSL version against Puma's OpenSSL version is a good check, especially when working with MSYS2 on Windows or Homebrew on macOS. Anytime the OpenSSL version is independent of the OS...
We have the below logging that info in our test suite.
puma/test/test_puma_server_ssl.rb
Lines 19 to 22 in cdb2341