Republished/re-pushed python:3 image 10/11 breaks builds. What was changed

Question

Republished/re-pushed python:3 image 10/11 breaks builds. What was changed

vladimirpodr opened this issue 10 months ago · comments

Volodymyr Podrushniak commented 10 months ago

With the previous image python:3 (before 10/11) everything was ok.
But now:

#10 7.719 [pipenv.exceptions.InstallError]: Building wheels for collected packages: python-rapidjson
#10 7.719 [pipenv.exceptions.InstallError]:   Building wheel for python-rapidjson (setup.py): started
#10 7.719 [pipenv.exceptions.InstallError]:   Building wheel for python-rapidjson (setup.py): finished with status 'error'
#10 7.719 [pipenv.exceptions.InstallError]:   Running setup.py clean for python-rapidjson
#10 7.719 [pipenv.exceptions.InstallError]: Failed to build python-rapidjson
#10 7.719 [pipenv.exceptions.InstallError]: error: subprocess-exited-with-error
#10 7.719 [pipenv.exceptions.InstallError]:   
#10 7.719 [pipenv.exceptions.InstallError]:   × python setup.py bdist_wheel did not run successfully.
#10 7.719 [pipenv.exceptions.InstallError]:   │ exit code: 1
#10 7.719 [pipenv.exceptions.InstallError]:   ╰─> [12 lines of output]
#10 7.719 [pipenv.exceptions.InstallError]:       running bdist_wheel
#10 7.719 [pipenv.exceptions.InstallError]:       running build
#10 7.719 [pipenv.exceptions.InstallError]:       running build_ext
#10 7.719 [pipenv.exceptions.InstallError]:       building 'rapidjson' extension
#10 7.719 [pipenv.exceptions.InstallError]:       creating build
#10 7.719 [pipenv.exceptions.InstallError]:       creating build/temp.linux-x86_64-cpython-311
#10 7.719 [pipenv.exceptions.InstallError]:       x86_64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DPYTHON_RAPIDJSON_VERSION=1.6 -DRAPIDJSON_EXACT_VERSION=v1.1.0-677-g4d6cb081 -I./rapidjson/include -I/root/.local/share/virtualenvs/app-4PlAip0Q/include -I/usr/include/python3.11 -c ./rapidjson.cpp -o build/temp.linux-x86_64-cpython-311/./rapidjson.o -pedantic -Wno-long-long
#10 7.719 [pipenv.exceptions.InstallError]:       ./rapidjson.cpp:11:10: fatal error: Python.h: No such file or directory
#10 7.719 [pipenv.exceptions.InstallError]:          11 | #include <Python.h>
#10 7.719 [pipenv.exceptions.InstallError]:             |          ^~~~~~~~~~
#10 7.719 [pipenv.exceptions.InstallError]:       compilation terminated.
#10 7.720 [pipenv.exceptions.InstallError]:       error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
#10 7.720 [pipenv.exceptions.InstallError]:       [end of output]
#10 7.720 [pipenv.exceptions.InstallError]:   
#10 7.720 [pipenv.exceptions.InstallError]:   note: This error originates from a subprocess, and is likely not a problem with pip.
#10 7.720 [pipenv.exceptions.InstallError]:   ERROR: Failed building wheel for python-rapidjson
#10 7.720 [pipenv.exceptions.InstallError]: ERROR: Could not build wheels for python-rapidjson, which is required to install pyproject.toml-based projects
#10 7.720 ERROR: Couldn't install package: ***
#10 7.720  Package installation failed...
#10 7.726 /usr/local/lib/python3.9/subprocess.py:1052: ResourceWarning: subprocess 32 is still running
#10 7.726   _warn("subprocess %s is still running" % self.pid,
#10 7.726 ResourceWarning: Enable tracemalloc to get the object allocation traceback
#10 7.726 sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=4 encoding='utf-8'>
#10 7.726 ResourceWarning: Enable tracemalloc to get the object allocation traceback
#10 7.726 sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=7 encoding='utf-8'>
#10 7.726 ResourceWarning: Enable tracemalloc to get the object allocation traceback
#10 ERROR: executor failed running [pipenv install]: exit code: 1
------
 > [6/6] RUN ["pipenv", "install"]:
7.720 [pipenv.exceptions.InstallError]: ERROR: Could not build wheels for python-rapidjson, which is required to install pyproject.toml-based projects

Dockerfile:

FROM python:3

# Install necessary packages
RUN apt-get update && \
    apt-get install -y postgresql-client && \
    apt-get clean


RUN pip install pipenv
WORKDIR /app
COPY . .
RUN ["pipenv", "install"]
ENTRYPOINT ["pipenv", "run", "python", "main.py"]

Pipfile:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
pyyaml = "*"
psycopg = "*"
azure-storage-blob = "*"
azure-identity = "*"
docker = "*"
pygithub = "*"

yosifkit · Answer 1 · Thu Oct 12 2023 05:16:56 GMT+0800 (China Standard Time)

You mention the python:3 image, but your Dockerfile is using the python:3.11.5 image. So which version tags don't work, and which do?

The python:3 image recently moved from Python 3.11 to 3.12; this was built/pushed on Oct 2 once docker-library/official-images#15464 merged. The only other change in python images from Oct 2 to today was an update of the Windows-based images, but that has no effect on the Debian based images. (The repo-info repo is useful for exploring what changed, esp. the git history of the python directory or specific files therein).

I am unable to reproduce a failure on WSL2 with Docker Desktop (amd64 CPU). I used the provided Dockerfile and Pipfile. I tried building with all of the following Python images: 3.11.4, 3.11.5, 3.11.6, and 3.12.0. I even tried just installing python-rapidjson directly:

$ docker run -it --rm python:3.11.5 pip install python-rapidjson
Collecting python-rapidjson
  Obtaining dependency information for python-rapidjson from https://files.pythonhosted.org/packages/ce/0b/1a15f73ffe1b3eed2dd017e1733db9f9fe52ac871836bfb35a8bc9a8be64/python_rapidjson-1.12-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading python_rapidjson-1.12-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (20 kB)
Downloading python_rapidjson-1.12-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 22.6 MB/s eta 0:00:00
Installing collected packages: python-rapidjson
Successfully installed python-rapidjson-1.12
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

$ docker run -it --rm python:3.11.6 pip install python-rapidjson
Collecting python-rapidjson
  Obtaining dependency information for python-rapidjson from https://files.pythonhosted.org/packages/ce/0b/1a15f73ffe1b3eed2dd017e1733db9f9fe52ac871836bfb35a8bc9a8be64/python_rapidjson-1.12-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading python_rapidjson-1.12-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (20 kB)
Downloading python_rapidjson-1.12-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 22.4 MB/s eta 0:00:00
Installing collected packages: python-rapidjson
Successfully installed python-rapidjson-1.12
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

$ docker run -it --rm python:3.12.0 pip install python-rapidjson
Collecting python-rapidjson
  Obtaining dependency information for python-rapidjson from https://files.pythonhosted.org/packages/cc/df/055c133881ccf119f3fb9839612c5728726e30cb4c9f2ed69204a6a8ff2f/python_rapidjson-1.12-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading python_rapidjson-1.12-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (20 kB)
Downloading python_rapidjson-1.12-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 22.5 MB/s eta 0:00:00
Installing collected packages: python-rapidjson
Successfully installed python-rapidjson-1.12
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Volodymyr Podrushniak · Answer 2 · Thu Oct 12 2023 21:17:42 GMT+0800 (China Standard Time)

@yosifkit Sorry for the confusion. python:3.11.5 from my testing different versions.
python:3 image version does not work. And it has happened from the time when python:3 image was republished.

Laurent Goderre · Answer 3 · Thu Oct 12 2023 21:34:09 GMT+0800 (China Standard Time)

python3 now points to 3.12 though. Could that be the issue? There seems to be a few issues coming lately with installing packages in 3.12