[BUG] pyproject.toml: name, version not recognized (UNKNOWN 0.0.0)
nschloe opened this issue · comments
setuptools version
62.1.0
Python version
3.10
OS
Ubuntu 22.04
Additional environment information
No response
Description
I recently started using Python 3.10 and found the pyproject.toml-based installation lacking. It now doesn't reads name and version incorrectly:
pip install .
Defaulting to user installation because normal site-packages is not writeable
Processing /path/to/foobar
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: UNKNOWN
Building wheel for UNKNOWN (pyproject.toml) ... done
Created wheel for UNKNOWN: filename=UNKNOWN-0.0.0-py3-none-any.whl size=1227 sha256=sgrhihehihg
Stored in directory: /path/to/.cache/pip/wheels/dsliughredshg
Successfully built UNKNOWN
Installing collected packages: UNKNOWN
Attempting uninstall: UNKNOWN
Found existing installation: UNKNOWN 0.0.0
Uninstalling UNKNOWN-0.0.0:
Successfully uninstalled UNKNOWN-0.0.0
Successfully installed UNKNOWN-0.0.0
Any idea why this might be? Perhaps package that's missing in my 3.10 installation? I was surprised not to get an error message here either.
Hi @nschloe , is it the case you have apyproject.toml file without the [project] section? Or that the directory doesn't have a pyproject.toml at all?
No, it's all there. Let me try and construct an MWE.
Thank you very much Nico.
I don't know exactly what is the problem, it seems to be working fine when installed in a virtual environment:
cd /tmp
git clone https://github.com/nschloe/setuptools-UNKNOWN-bug.git
cd setuptools-UNKNOWN-bug
virtualenv -p python3.10 .venv
.venv/bin/python -m pip install .Cloning into 'setuptools-UNKNOWN-bug'...
remote: Enumerating objects: 12, done.
remote: Counting objects: 100% (12/12), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 12 (delta 0), reused 12 (delta 0), pack-reused 0
created virtual environment CPython3.10.4.final.0-64 in 6163ms
creator CPython3Posix(dest=/tmp/setuptools-UNKNOWN-bug/.venv, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/abravalheri/.local/share/virtualenv)
added seed packages: pip==22.0.4, setuptools==60.9.3, wheel==0.37.1
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
Processing /tmp/setuptools-UNKNOWN-bug
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: foobar
Building wheel for foobar (pyproject.toml) ... done
Created wheel for foobar: filename=foobar-1.2.3-py3-none-any.whl size=1637 sha256=f907c18db190782ff7e3066eecd072fd76ec5a4777809cf7f8325309cb24937a
Stored in directory: /home/abravalheri/.cache/pip/wheels/d5/45/27/903aaeecfb5f483be8a106ef7e408201b853f57c4b50642397
Successfully built foobar
Installing collected packages: foobar
Successfully installed foobar-1.2.3It also seem to be OK when running from a container:
> docker run --rm -it python:3.10-bullseye /bin/bash
cd /tmp
git clone https://github.com/nschloe/setuptools-UNKNOWN-bug.git
cd setuptools-UNKNOWN-bug
pip install .Cloning into 'setuptools-UNKNOWN-bug'...
remote: Enumerating objects: 12, done.
remote: Counting objects: 100% (12/12), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 12 (delta 0), reused 12 (delta 0), pack-reused 0
Receiving objects: 100% (12/12), done.
Processing /tmp/setuptools-UNKNOWN-bug
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: foobar
Building wheel for foobar (pyproject.toml) ... done
Created wheel for foobar: filename=foobar-1.2.3-py3-none-any.whl size=1637 sha256=652ef47f8ff8cd03e54c981d121e61618e95a60d1ca5f7592588f0baecc75def
Stored in directory: /root/.cache/pip/wheels/d5/45/27/903aaeecfb5f483be8a106ef7e408201b853f57c4b50642397
Successfully built foobar
Installing collected packages: foobar
Successfully installed foobar-1.2.3
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venvHave you tried updating pip? Maybe that would help?
It also works in a venv here.
pip is at the latest version. There's got to be something wrong with my installation. Geez, I wish I had any clue at all.
Are you using the system-provided Python and pip (i.e. the ones in /usr/bin), or are you building your own?
It's all in ~/.local, from pypi. The issue remain on main/master (both setuptools and pip) which I'm using for debugging.
Any chance this originates from caches of previous builds?
Does it persist even after a rm -rf build dist *.egg-info src/*.egg-info?
Yup, even then. I noticed, however, that pip creates a UNKNOWN.egg-info dir in the root directory of the project, not in src/ as I would have expected. Perhaps that's a clue?
Can you run pip debug and share the output from that? And, run pip install with -vv?
pip debug
WARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice.
pip version: pip 22.1.dev0 from /home/nschloe/.local/lib/python3.10/site-packages/pip (python 3.10)
sys.version: 3.10.4 (main, Apr 2 2022, 09:04:19) [GCC 11.2.0]
sys.executable: /usr/bin/python3
sys.getdefaultencoding: utf-8
sys.getfilesystemencoding: utf-8
locale.getpreferredencoding: UTF-8
sys.platform: linux
sys.implementation:
name: cpython
'cert' config value: Not specified
REQUESTS_CA_BUNDLE: None
CURL_CA_BUNDLE: None
pip._vendor.certifi.where(): /home/nschloe/.local/lib/python3.10/site-packages/pip/_vendor/certifi/cacert.pem
pip._vendor.DEBUNDLED: False
vendored library versions:
CacheControl==0.12.10
colorama==0.4.4
distlib==0.3.3
distro==1.6.0
html5lib==1.1
msgpack==1.0.3 (Unable to locate actual module version, using vendor.txt specified version)
packaging==21.3
pep517==0.12.0
platformdirs==2.4.1
pyparsing==3.0.8
requests==2.27.1
certifi==2021.10.08
chardet==4.0.0
idna==3.3
urllib3==1.26.8
rich==11.0.0 (Unable to locate actual module version, using vendor.txt specified version)
pygments==2.11.2
typing_extensions==4.0.1 (Unable to locate actual module version, using vendor.txt specified version)
resolvelib==0.8.1
setuptools==44.0.0 (Unable to locate actual module version, using vendor.txt specified version)
six==1.16.0
tenacity==8.0.1 (Unable to locate actual module version, using vendor.txt specified version)
tomli==1.0.3
webencodings==0.5.1 (Unable to locate actual module version, using vendor.txt specified version)
Compatible tags: 818
cp310-cp310-manylinux_2_35_x86_64
cp310-cp310-manylinux_2_34_x86_64
cp310-cp310-manylinux_2_33_x86_64
cp310-cp310-manylinux_2_32_x86_64
cp310-cp310-manylinux_2_31_x86_64
cp310-cp310-manylinux_2_30_x86_64
cp310-cp310-manylinux_2_29_x86_64
cp310-cp310-manylinux_2_28_x86_64
cp310-cp310-manylinux_2_27_x86_64
cp310-cp310-manylinux_2_26_x86_64
...
[First 10 tags shown. Pass --verbose to show all.]
pip install . -vv
Using pip 22.1.dev0 from /home/nschloe/.local/lib/python3.10/site-packages/pip (python 3.10)
Defaulting to user installation because normal site-packages is not writeable
Created temporary directory: /tmp/pip-ephem-wheel-cache-5upjyai5
Created temporary directory: /tmp/pip-build-tracker-fj5v1v9p
Initialized build tracking at /tmp/pip-build-tracker-fj5v1v9p
Created build tracker: /tmp/pip-build-tracker-fj5v1v9p
Entered build tracker: /tmp/pip-build-tracker-fj5v1v9p
Created temporary directory: /tmp/pip-install-kn0svgpe
Processing /home/nschloe/software/nschloe/setuptools-UNKNOWN-bug
Added file:///home/nschloe/software/nschloe/setuptools-UNKNOWN-bug to build tracker '/tmp/pip-build-tracker-fj5v1v9p'
Created temporary directory: /tmp/pip-build-env-lvx4ym5h
Created temporary directory: /tmp/pip-standalone-pip-jtuo8gm7
Running command pip subprocess to install build dependencies
Using pip 22.1.dev0 from /tmp/pip-standalone-pip-jtuo8gm7/__env_pip__.zip/pip (python 3.10)
Collecting setuptools>=61
Using cached setuptools-62.1.0-py3-none-any.whl (1.1 MB)
Installing collected packages: setuptools
Successfully installed setuptools-62.1.0
Installing build dependencies ... done
Running command Getting requirements to build wheel
running egg_info
writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
Getting requirements to build wheel ... done
Created temporary directory: /tmp/pip-standalone-pip-jed8u07y
Running command pip subprocess to install backend dependencies
Using pip 22.1.dev0 from /tmp/pip-standalone-pip-jed8u07y/__env_pip__.zip/pip (python 3.10)
Collecting wheel
Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel
Creating /tmp/pip-build-env-lvx4ym5h/normal/local/bin
changing mode of /tmp/pip-build-env-lvx4ym5h/normal/local/bin/wheel to 775
Successfully installed wheel-0.37.1
Installing backend dependencies ... done
Created temporary directory: /tmp/pip-modern-metadata-hhyyiox_
Running command Preparing metadata (pyproject.toml)
running dist_info
creating /tmp/pip-modern-metadata-hhyyiox_/UNKNOWN.egg-info
writing manifest file '/tmp/pip-modern-metadata-hhyyiox_/UNKNOWN.egg-info/SOURCES.txt'
writing manifest file '/tmp/pip-modern-metadata-hhyyiox_/UNKNOWN.egg-info/SOURCES.txt'
Preparing metadata (pyproject.toml) ... done
Source in /home/nschloe/software/nschloe/setuptools-UNKNOWN-bug has version 0.0.0, which satisfies requirement UNKNOWN==0.0.0 from file:///home/nschloe/software/nschloe/setuptools-UNKNOWN-bug
Removed UNKNOWN==0.0.0 from file:///home/nschloe/software/nschloe/setuptools-UNKNOWN-bug from build tracker '/tmp/pip-build-tracker-fj5v1v9p'
Created temporary directory: /tmp/pip-unpack-w1yz2tuj
Building wheels for collected packages: UNKNOWN
Created temporary directory: /tmp/pip-wheel-t0daq_rt
Destination directory: /tmp/pip-wheel-t0daq_rt
Running command Building wheel for UNKNOWN (pyproject.toml)
running bdist_wheel
running build
running install
running install_egg_info
running egg_info
writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
Copying UNKNOWN.egg-info to build/bdist.linux-x86_64/wheel/UNKNOWN-0.0.0.egg-info
running install_scripts
Building wheel for UNKNOWN (pyproject.toml) ... done
Created wheel for UNKNOWN: filename=UNKNOWN-0.0.0-py3-none-any.whl size=961 sha256=fbe801efc0ed589ed6e268eabeef9a3c21e21aebb5708474998de84ecab9c4ee
Stored in directory: /home/nschloe/.cache/pip/wheels/8c/53/6f/b8f9907b9ed054106c802dcf084e59af6c78e96adc189caa9c
Successfully built UNKNOWN
Installing collected packages: UNKNOWN
Attempting uninstall: UNKNOWN
Found existing installation: UNKNOWN 0.0.0
Uninstalling UNKNOWN-0.0.0:
Created temporary directory: /home/nschloe/.local/lib/python3.10/site-packages/~NKNOWN-0.0.0.dist-info
Removing file or directory /home/nschloe/.local/lib/python3.10/site-packages/UNKNOWN-0.0.0.dist-info/
Successfully uninstalled UNKNOWN-0.0.0
Successfully installed UNKNOWN-0.0.0
Removed build tracker: '/tmp/pip-build-tracker-fj5v1v9p'
I could also add some print()s to setuptools or pip.
I wonder if this is related to pypa/pip#6264 (comment)
It would seem that an older version of setuptools is being loaded from outside the environment prepared by pip.
Yup, that’s what this looks like. You’re almost certainly using a Debian-patched Python which behaves incorrectly with pip and how it handles build isolation.
I'm indeed on Ubuntu 22.04. I've now tried the fixed (?) Python from the deadsnakes PPA (https://launchpad.net/~deadsnakes/+archive/ubuntu/python3.10-jammy), still to no avail.
pip install . -vv
Using pip 22.0.4 from /home/nschloe/.local/lib/python3.10/site-packages/pip (python 3.10)
Defaulting to user installation because normal site-packages is not writeable
Created temporary directory: /tmp/pip-ephem-wheel-cache-cp8n1mrt
Created temporary directory: /tmp/pip-req-tracker-2wziyn5f
Initialized build tracking at /tmp/pip-req-tracker-2wziyn5f
Created build tracker: /tmp/pip-req-tracker-2wziyn5f
Entered build tracker: /tmp/pip-req-tracker-2wziyn5f
Created temporary directory: /tmp/pip-install-cugd9fnz
Processing /home/nschloe/software/nschloe/setuptools-UNKNOWN-bug
Added file:///home/nschloe/software/nschloe/setuptools-UNKNOWN-bug to build tracker '/tmp/pip-req-tracker-2wziyn5f'
Created temporary directory: /tmp/pip-build-env-grzhsya1
Created temporary directory: /tmp/pip-standalone-pip-7g1ikiwq
Running command pip subprocess to install build dependencies
Using pip 22.0.4 from /tmp/pip-standalone-pip-7g1ikiwq/__env_pip__.zip/pip (python 3.10)
Collecting setuptools>=61
Using cached setuptools-62.1.0-py3-none-any.whl (1.1 MB)
Installing collected packages: setuptools
Successfully installed setuptools-62.1.0
Installing build dependencies ... done
Running command Getting requirements to build wheel
running egg_info
writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
Getting requirements to build wheel ... done
Created temporary directory: /tmp/pip-standalone-pip-x8kpa2d7
Running command pip subprocess to install backend dependencies
Using pip 22.0.4 from /tmp/pip-standalone-pip-x8kpa2d7/__env_pip__.zip/pip (python 3.10)
Collecting wheel
Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel
Creating /tmp/pip-build-env-grzhsya1/normal/local/bin
changing mode of /tmp/pip-build-env-grzhsya1/normal/local/bin/wheel to 775
Successfully installed wheel-0.37.1
Installing backend dependencies ... done
Created temporary directory: /tmp/pip-modern-metadata-cx_uo7qm
Running command Preparing metadata (pyproject.toml)
running dist_info
creating /tmp/pip-modern-metadata-cx_uo7qm/UNKNOWN.egg-info
writing manifest file '/tmp/pip-modern-metadata-cx_uo7qm/UNKNOWN.egg-info/SOURCES.txt'
writing manifest file '/tmp/pip-modern-metadata-cx_uo7qm/UNKNOWN.egg-info/SOURCES.txt'
Preparing metadata (pyproject.toml) ... done
Source in /home/nschloe/software/nschloe/setuptools-UNKNOWN-bug has version 0.0.0, which satisfies requirement UNKNOWN==0.0.0 from file:///home/nschloe/software/nschloe/setuptools-UNKNOWN-bug
Removed UNKNOWN==0.0.0 from file:///home/nschloe/software/nschloe/setuptools-UNKNOWN-bug from build tracker '/tmp/pip-req-tracker-2wziyn5f'
Created temporary directory: /tmp/pip-unpack-hsq4yaq7
Building wheels for collected packages: UNKNOWN
Created temporary directory: /tmp/pip-wheel-ngwdy_74
Destination directory: /tmp/pip-wheel-ngwdy_74
Running command Building wheel for UNKNOWN (pyproject.toml)
running bdist_wheel
running build
running install
running install_egg_info
running egg_info
writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
Copying UNKNOWN.egg-info to build/bdist.linux-x86_64/wheel/UNKNOWN-0.0.0.egg-info
running install_scripts
Building wheel for UNKNOWN (pyproject.toml) ... done
Created wheel for UNKNOWN: filename=UNKNOWN-0.0.0-py3-none-any.whl size=961 sha256=558fec805c83652321354cbcc23881544f1a163e9bfcb556db1ce4539a779e36
Stored in directory: /home/nschloe/.cache/pip/wheels/8c/53/6f/b8f9907b9ed054106c802dcf084e59af6c78e96adc189caa9c
Successfully built UNKNOWN
Installing collected packages: UNKNOWN
Successfully installed UNKNOWN-0.0.0
Removed build tracker: '/tmp/pip-req-tracker-2wziyn5f'
I don't know what this patch does, but just to make sure the problem we discuss in pypa/pip#6264 (comment) is the case, would it be possible for you to try updating the version of setuptools in /usr/local/lib/python3.10/dist-packages/setuptools to 62.1.0 and running again? That could clarify some things...
deadsnake (intentionally) distributes exactly the same Python patches as Debian, simply backporting (and forward-poarting, is that a thing?) them to more system versions. So if something does not work right with the system Python, it’s likely to also not work with a deadsnake Python—that’s an explicit design goal.
But whatever the problem is, it very likely roots in the a patch that breaks build isolation.
This is a reproducer based on the issue previously reported in the pip tracker:
> docker run --rm -it debian:11 /bin/bash
apt update && apt install -y python3 python3-dev curl git
cd /tmp
curl -sSL https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python3 get-pip.py --no-setuptools --no-wheel
python3 -m pip install -Uv setuptools==44.0.0
python3 -m pip list
# Package Version
# ---------- -------
# pip 22.0.4
# setuptools 44.0.0
python3 -c 'import setuptools; print(setuptools.__path__, setuptools.__version__)'
# ['/usr/local/lib/python3.9/dist-packages/setuptools'] 44.0.0
git clone https://github.com/nschloe/setuptools-UNKNOWN-bug
cd setuptools-UNKNOWN-bug
pip install . -vv
# ...
# Successfully installed UNKNOWN-0.0.0
# ...Note that when we update the root installation of setuptools, the project builds correctly, which indicates that the dist-packages folder is somehow leaking into the build environment:
python3 -m pip install -Uv setuptools==62.1.0
pip install . -vv
# ...
# Successfully installed foobar-1.2.3
# ...My conclusion is that it is the same problem, and based on what @uranusjr just discussed, deadsnakes patches are either the same as upstream debian/ubuntu or not enough to workaround this issue.
@nschloe, if you can confirm that is the case, I think we can close this issue as a duplicate and follow the original discussion in pypa/pip#6264 for updates.
Indeed, this is the problem. I've "fixed" it by removing the system setuptools,
sudo apt purge python3-setuptools
Now it all builds correctly.
Thank you very much for confirming @nschloe
I think this person had the same problem :-)
https://pypi.org/project/UNKNOWN/
Judging from the 2004 upload, I’m pretty sure that’s a differnt problem. pyproject.toml wasn’t invented until 2016.
TLDR; On Ubuntu 22.04 (Jammy) ensure the environment variable DEB_PYTHON_INSTALL_LAYOUT=deb_system before installing a PEP621 compliant project with pip and setuptools.
This issue is occurring on Jammy because pip is unable to get an accurate list of system package directories to remove from its temporary build environment's sys.path. As a result, when the temporary build env's site/sitecustomize.py script runs /usr/lib/python3/dist-packages is left at the end of sys.path, before the overlay and normal paths are appended, thus placing it before those dirs in the search path. This means Jammy's debian packaged setuptools (at version 59) is found and loaded ahead of whatever version the pyproject.toml requested. That version does not support PEP621 and is unable to extract name and version metadata, resulting in UNKNOWN-0.0.0.
This was not an issue on 20.04 (Focal) because the get_purelib and get_platlib logic inside pip/_internal/locations/__init__.py switched at runtime between using the older distutils and newer sysconfig modules for determining purelib and platlib paths. That choice is based on the value of sysconfig._PIP_USE_SYSCONFIG, which only defaults to False if the python version is less than 3.10. The older distuils knew how to deal with Debian's install scheme layout, where the newer sysconfig doesn't unless the env var DEB_PYTHON_INSTALL_LAYOUT is set to either deb or deb_system.
This means this is a distro configuration/packaging issue, but I'm posting this comment on a closed bug because this bug report comes up as the top result when googling for this issue.
This was purportedly fixed toward the end of September 2022 in pypa/pip#11466
For mac users -
Turns out, there was an outdated setuptools installed in my mac too. It took me some but I finally found it and removed it -
$ sudo rm ./Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/site-packages/setuptools*
For me it helped to update pip
python3 -m pip install pip -U
I've read through all of this but still have no idea what the solution is. I've updated to the latest pip. Can anyone succinctly describe a solution on Ubuntu 22.04? I'm in a dev container already, so I'm not using venv.
I recall I might have worked around this by specifying these things in any other way, such as the traditional setup.py .
Which is aggravating since then you get deprecation warnings non-stop...
I've read through all of this but still have no idea what the solution is. I've updated to the latest pip. Can anyone succinctly describe a solution on Ubuntu 22.04? I'm in a dev container already, so I'm not using venv.
Have you tried setting DEB_PYTHON_INSTALL_LAYOUT=deb_system, in you dev container, or exporting it before running pip? There is a longer explanation here #3269 (comment), but I found that help me. Though I thought it had been fixed by now... as I don't seem to have that set /etc/environment any more.
I've read through all of this but still have no idea what the solution is. I've updated to the latest pip. Can anyone succinctly describe a solution on Ubuntu 22.04? I'm in a dev container already, so I'm not using venv.
Have you tried setting DEB_PYTHON_INSTALL_LAYOUT=deb_system, in you dev container, or exporting it before running pip? There is a longer explanation here #3269 (comment), but I found that help me. Though I thought it had been fixed by now... as I don't seem to have that set /etc/environment any more.
The issue is occurring at build time, not at install time. I'm trying to build a wheel using this command line:
python3 -m build --no-isolation --wheel .
I just changed back to a setup.py.
I ran into this issue using pipx run build -n -s in GitHub CI (ubuntu-22.04).
pip install build && python -m build -n -s works.
Ran into this issue installing my package in a Docker. Fixed it by just upgrading pip before installing my package:
pip install --upgrade pip
pip install .