Giters

pyenv / pyenv

Simple Python version management

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

persistent SSL_get_peer_certificate when building Python 2.7

davidsiaw opened this issue · comments

commented

Too many issues will kill our team's development velocity, drastically.
Make sure you have checked all steps below.

Prerequisite

  • [ x ] Make sure your problem is not listed in the common build problems.
  • [ x ] Make sure no duplicated issue has already been reported in the pyenv issues. You should look for closed issues, too.
  • [ x ] Make sure you are not asking us to help solving your specific issue.
    • GitHub issues is opened mainly for development purposes. If you want to ask someone to help solving your problem, go to some community site like Gitter, StackOverflow, etc.
  • [ x ] Make sure your problem is not derived from packaging (e.g. Homebrew).
    • Please refer to the package documentation for the installation issues, etc.
  • [ x ] Make sure your problem is not derived from plugins.
    • This repository is maintaining pyenv and the default python-build plugin only. Please refrain from reporting issues of other plugins here.

Description

  • [ x ] Platform information (e.g. Ubuntu Linux 16.04): 22.04
  • [ x ] OS architecture (e.g. amd64): amd64
  • [ x ] pyenv version: pyenv 2.4.0-2-ge1f07da7
  • [ x ] Python version: 3.10.13
  • [ x ] C Compiler information (e.g. gcc 7.3): gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04)
  • [ x ] Please attach the debug trace of the failing command as a gist:
    • Run env PYENV_DEBUG=1 <faulty command> 2>&1 | tee trace.log and attach trace.log. E.g. if you have a problem with installing Python, run env PYENV_DEBUG=1 pyenv install -v <version> 2>&1 | tee trace.log (note the -v option to pyenv install).
  • [ x ] If you have a problem with installing Python, please also attach config.log from the build directory
    • The build directory is reported after the "BUILD FAILED" message and is usually under /tmp.
  • If the build succeeds but the problem is still with the build process (e.g. the resulting Python is missing a feature), please attach
    • the debug trace from reinstalling the faulty version with env PYENV_DEBUG=1 pyenv install -f -k -v <version> 2>&1 | tee trace.log
    • config.log from the build directory. When using pyenv install with -k as per above, the build directory will be under $PYENV_ROOT/sources.

Here is the gist with the trace.log and the build log. https://gist.github.com/davidsiaw/e545c540170096d3f3d7309caac51959

I am attempting to pyenv install 2.7

The issue is the _ssl module could not be imported and therefore it failed. Looking up further there is an error saying this:

*** WARNING: renaming "_ssl" since importing it failed: build/lib.linux-x86_64-2.7/_ssl.so: undefined symbol: SSL_get_peer_certificate

All the guidance including that of the wiki points to an older version of SSL required. Hence I checked out openssl branches 1.1.1 and 1.0.2, and set the CFLAGS and LDFLAGS appropriately, but the same issue occurs. The local version of openssl is OpenSSL 1.1.1q 5 Jul 2022

Homebrew is not used, and no openssl-related closed issues show relevance.

I got this with compiling Ruby as well and managed to solve it by using a lower OpenSSL lib, so I am guessing this is an issue in pyenv or python-build, or it is not entirely clear which version of openssl I have to refer to.

commented

I checked out openssl branches 1.1.1 and 1.0.2

But did you actually build and install them from source?

P.S. 1.1.1 should be enough as per https://github.com/pyenv/pyenv/wiki/Common-build-problems#2-your-openssl-version-is-incompatible-with-the-python-version-youre-trying-to-install

commented

Yes. config --prefix=$HOME/ssl-1.... && make -j6 && make install yes built them from source.

P.S. 1.1.1 should be enough as per https://github.com/pyenv/pyenv/wiki/Common-build-problems#2-your-openssl-version-is-incompatible-with-the-python-version-youre-trying-to-install

this is puzzling because the thing you linked to me literally says Old Python versions (for CPython, <3.5.3 and <2.7.13) require OpenSSL 1.0 . Hence I used a 1.0.2 instead of 1.1.1. Am I reading that wrong? (1.0.2 and 1.1.1 both didn't work though, I haven't bothered trying older versions)

commented

Okay. Please add config.log and trace.log as per the issue template. Just the build log is not enough.

commented

Okay. Please add config.log and trace.log as per the issue template. Just the build log is not enough.

I apologize. It is in the link, but hard to find. Here is a shortcut to it.

https://gist.github.com/davidsiaw/e545c540170096d3f3d7309caac51959#file-trace-log

commented

Old Python versions (for CPython, <3.5.3 and <2.7.13) require OpenSSL 1.0

$ pyenv latest -k 2.7
2.7.18
commented

Ah. Thanks. My mistake. I do have 1.1.1 installed from my Ruby build, but it seems to be missing that one function thats preventing it from building. So I am at my wits end at the moment. I apologize if this seems like I am asking for support. I understand python 2 is very old and everyone is discouraging its use, but this should really just work. Maybe pyenv should pull the appropriate old openssl? If you could point me to the right place I can try and add this capability.

commented

It's EOL, you can't expect it to "just work". We only promise that we won't be actively breaking it and will accept extended support patches from interested parties.

Now, from the trace log, I don't see you passing any FLAGS envvars. You didn't provide config.log which shows more envvars than trace.log, so I can't check there.

running build_ext
warning: openssl 0x00000000 is too old for _hashlib

clearly shows that OpenSSL has NOT been recognized.
Looking at setup.py, this value is extracted from the line #define OPENSSL_VERSION_NUMBER <...> in openssl/opensslv.h, and it expects additional include directories to reside in CPPFLAGS

commented

This worked for me (edited, initial version succeeded but linked against a wrong OpenSSL):

curl https://www.openssl.org/source/old/1.1.1/openssl-1.1.1w.tar.gz | tar -zx
cd openssl-1.1.1w/
./config --prefix=$HOME/.local/opt/openssl-1.1.1
make -j$(nproc) && make install
cd
CPPFLAGS=-I$HOME/.local/opt/openssl-1.1.1/include LDFLAGS="-L$HOME/.local/opt/openssl-1.1.1/lib -Wl,-rpath,$HOME/.local/opt/openssl-1.1.1/lib" pyenv install 2.7

To check which OpenSSL you have linked against:

$ ldd ~/.pyenv/versions/2.7.18/lib/python2.7/lib-dynload/_ssl.so 
	linux-vdso.so.1 (0x00007ffd181fd000)
	libssl.so.1.1 => /home/vmuser/.local/opt/openssl-1.1.1/lib/libssl.so.1.1 (0x00007efc4c652000)
	libcrypto.so.1.1 => /home/vmuser/.local/opt/openssl-1.1.1/lib/libcrypto.so.1.1 (0x00007efc4c200000)
	libpython2.7.so.1.0 => /home/vmuser/.pyenv/versions/2.7.18/lib/libpython2.7.so.1.0 (0x00007efc4be00000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007efc4ba00000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007efc4c55c000)
	/lib64/ld-linux-x86-64.so.2 (0x00007efc4c708000)
commented

... and to test if it works (because the build can successfully link it to e.g. OpenSSL 3 -- but the tests fail due to incompatibilities):

pyenv shell 2.7
python $(pyenv prefix)/lib/python2.7/test/test_ssl.py