ruby 3.2.2. build with OpenSSL 3.0 fails on FreeBSD 13.
opened this issue · comments
I'm trying to build ruby 3.2.2 on a FreeBSD 13.2-p3 system:
stevewi@dave:~ $
uname -a
FreeBSD dave 13.2-RELEASE-p3 FreeBSD 13.2-RELEASE-p3 hp800g3 amd64
from the source package I acquired at ruby-lang.org:
This system has openssl 3.0 installed:
stevewi@dave:~ $
pkg info | grep openssl
openssl-3.0.11,1 TLSv1.3 capable SSL and crypto library
-- Everything builds... I get lots of warnings when ruby's openssl support is built. See the output of the make in the make.out.txt attachment. The output of make install is in the install.out.txt attachment. So far, so good...
-- I update all the default gems. I have a script that does this. The output is in update.out.txt. So far, so good...
-- I rebundle all my projects using bundler (default: 2.4.21) . I have a script that does this. This fails with the following errors:
ld-elf.so.1: /usr/local/lib/ruby/gems/3.2.0/gems/openssl-3.2.0/lib/openssl.so: Undefined symbol "EVP_PKEY_base_id"
ld-elf.so.1: /usr/local/lib/ruby/3.2.0/x86_64-freebsd13.2/openssl.so: Undefined symbol "EVP_MD_CTX_get0_md"
The output of the re-bundling process is in rebundle.out.txt.
The only combination that seems to work is ruby 3.2.2 and openssl 1.1.1w. However, openssl 1.1.1w is deprecated:
===> NOTICE:
This port is deprecated; you may wish to reconsider installing it:
End-of-life since 2023-09-11, see https://www.openssl.org/blog/blog/2023/09/11/eol-111/ port will be removed when FreeBSD 13 is EoL.
It is scheduled to be removed on or after 2026-01-31.
I'd really like to know the magic required to get this combination (ruby 3.2.2 and openssl 3.0) to build and work properly. FWIW, I get exactly the same results with ruby 3.3.0 preview2 and openssl 3.0...
This is a test system I have at home so I can blow away ruby at will and build things differently. Thanks in advance...
make.out.txt
install.out.txt
update.out.txt
rebundle.out.txt
You don't have to describe your negative emotions here with a "sigh". Please be patient and kind. Did you install the OpenSSL 3.0.11 headers (devel) package such as /usr/include/openssl/*.h files?
Note that we are testing the openssl gem with OpenSSL 3.0.x on Ubuntu
openssl/.github/workflows/test.yml
Line 81 in f948e6b
We are also testing the openssl gem bundled in Ruby (ruby/ruby) on our Ruby project's CI on FreeBSD
FreeBSD 13.1 x64
FreeBSD 12.4 x64
In make.out.txt:
[...]
ossl.c:322:21: warning: implicit declaration of function 'ERR_get_error_all' is invalid in C99 [-Wimplicit-function-declaration]
while ((e = ERR_get_error_all(&file, &line, &func, &data, &flags))) {
^
1 warning generated.
[...]
This isn't quite right. ERR_get_error_all() is declared in OpenSSL 3.0's openssl/err.h. ossl.c does include that file, so this warning should never happen.
ERR_get_error_all() is a new function in OpenSSL 3.0. My best guess is that header files from OpenSSL <= 1.1 sneaked into the search paths somehow.
What options did you pass to the configure script? Also please provide ext/openssl/mkmf.log and ext/openssl/Makefile which are generated while running make (after it prints configuring openssl).
You don't have to describe your negative emotions here with a "sigh".
I've been at this for two days... I've done hundreds of builds with differing variations. I've even transplanted the ext/openssl files from your master branch into the directory created when I extracted the tar.gz file I got from ruby-lang.org. I'm incredibly frustrated...
Did you install the OpenSSL 3.0.11 headers (devel) package such as /usr/include/openssl/*.h files?
On FreeBSD there is no openssl-devel...like on Debian/Ubuntu. There is only the openssl port (which installs libraries and headers)...this goes for all the versions available: 1.1.1, 3.0, 3.1, 3.2alpha, etc. I've tried building/installing the openssl (3.0) port too (instead of relying on pkg...which works fine -- no warnings). I verified the headers are installed using either pkg or building openssl from the port. Here's the openssl version in the headers:
opensslv.h:
# define OPENSSL_VERSION_MAJOR 3
opensslv.h:# define OPENSSL_VERSION_MINOR 0
opensslv.h:# define OPENSSL_VERSION_PATCH 11
...
opensslv.h:# define OPENSSL_VERSION_STR "3.0.11"
opensslv.h:# define OPENSSL_FULL_VERSION_STR "3.0.11"
opensslv.h:# define OPENSSL_VERSION_TEXT "OpenSSL 3.0.11 19 Sep 2023"
What options did you pass to the configure script?
No options...just cd ~/ruby-3.2.2; ./configure. The output is attached as configure.out.txt.
Thanks very much for your response. Attached are the two files you requested (I added the .txt extension so github wouldn't complain).
I found it! There's a set of openssl 1.1.1 headers installed in /usr/include and a set of openssl 1.1.1 libraries installed in /lib and /usr/lib. These directories are searched before /usr/local/lib by the C/C++ compiler (clang) and the linker (ld).
These openssl 1.1.1 headers/libraries were put there when the system was installed...so I can't get rid of them (they're used by things like sudo and pkg...and probably the FreeBSD crypto drivers). FreeBSD 14.0 (real soon now) is supposed to upgrade all this to openssl 3.X.
I solved my immediate problem by running configure --with-openssl --with-openssl-dir=/usr/local. This causes the location of openssl 3.0 to be set explicitly. This is confirmed by
stevewi@dave:~/ruby-3.2.2 $
ldd .ext/x86_64-freebsd13.2/openssl.so
.ext/x86_64-freebsd13.2/openssl.so:
libssl.so.12 => /usr/local/lib/libssl.so.12 (0x13cdcc26000)<-- openssl 3.0 shared library
libcrypto.so.12 => /usr/local/lib/libcrypto.so.12 (0x13cddfad000)<-- openssl 3.0 shared library
libm.so.5 => /lib/libm.so.5 (0x13cdd5f9000)
libthr.so.3 => /lib/libthr.so.3 (0x13cdf901000)
libc.so.7 => /lib/libc.so.7 (0x13cda2ba000)
after the build is complete.
stevewi@dave:~/ruby-3.2.2 $
ldd /usr/local/lib/ruby/gems/3.2.0/gems/openssl-3.2.0/lib/openssl.so
/usr/local/lib/ruby/gems/3.2.0/gems/openssl-3.2.0/lib/openssl.so:
libssl.so.12 => /usr/local/lib/libssl.so.12 (0x13cdcc26000)<-- openssl 3.0 shared library
libcrypto.so.12 => /usr/local/lib/libcrypto.so.12 (0x13cddfad000)<-- openssl 3.0 shared library
libm.so.5 => /lib/libm.so.5 (0x13cdd5f9000)
libthr.so.3 => /lib/libthr.so.3 (0x13cdf901000)
libc.so.7 => /lib/libc.so.7 (0x13cda2ba000)
after the new build is installed confirms this as well.
This was the clue:
ERR_get_error_all()is a new function in OpenSSL 3.0. My best guess is that header files from OpenSSL <= 1.1 sneaked into the search paths somehow.
Thanks for your help.
This issue is now resolved.