pip 23.0.1 and anyio==4.0.0 break installation with --upgrade --no-binary :all: options
i-chvets opened this issue · comments
Description
When trying to install dependency anyio, setuptools_scm installation is broken when using options --upgrade --no-binary :all:
pip version is 23.0.1
Error observed:
LookupError: https://files.pythonhosted.org/packages/a7/16/8aef42ae27036ceb201f1774940357da18ae7daac45bdd721da554892012/setuptools-scm-8.0.1.tar.gz (from https://pypi.org/simple/setuptools-scm/) (requires-python:>=3.8) is already being built: setuptools_scm>=6.4 from https://files.pythonhosted.org/packages/a7/16/8aef42ae27036ceb201f1774940357da18ae7daac45bdd721da554892012/setuptools-scm-8.0.1.tar.gz
Steps to reproduce
pip3 install --upgrade --no-binary :all: anyio==4.0.0
Collecting anyio==4.0.0
Using cached anyio-4.0.0.tar.gz (153 kB)
Installing build dependencies ... error
error: subprocess-exited-with-error
Full log
$ pip3 install --upgrade --no-binary :all: anyio==4.0.0
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
Collecting anyio==4.0.0
Using cached anyio-4.0.0.tar.gz (153 kB)
Installing build dependencies ... error
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> [116 lines of output]
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
Collecting setuptools>=64
Using cached setuptools-68.2.2.tar.gz (2.2 MB)
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Collecting setuptools_scm>=6.4
Using cached setuptools-scm-8.0.1.tar.gz (71 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'error'
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> [85 lines of output]
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
Ignoring typing_extensions: markers 'python_version < "3.8"' don't match your environment
Collecting importlib-metadata>=4.6
Using cached importlib_metadata-6.8.0.tar.gz (53 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'error'
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 2
╰─> [61 lines of output]
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
Collecting setuptools>=56
Using cached setuptools-68.2.2.tar.gz (2.2 MB)
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Collecting setuptools_scm[toml]>=3.4.1
Using cached setuptools-scm-8.0.1.tar.gz (71 kB)
ERROR: Exception:
Traceback (most recent call last):
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
status = run_func(*args)
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
return func(self, options, args)
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 419, in run
requirement_set = resolver.resolve(
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
result = self._result = resolver.resolve(
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 348, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
if not criterion.candidates:
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
return bool(self._sequence)
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
return any(self)
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
return (c for c in iterator if id(c) not in self._incompatible_ids)
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
candidate = func()
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
self._link_candidate_cache[link] = LinkCandidate(
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 297, in __init__
super().__init__(
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 162, in __init__
self.dist = self._prepare()
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 231, in _prepare
dist = self._prepare_distribution()
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 491, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 577, in _prepare_linked_requirement
dist = _get_prepared_distribution(
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 68, in _get_prepared_distribution
with build_tracker.track(req):
File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
return next(self.gen)
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/operations/build/build_tracker.py", line 122, in track
self.add(req)
File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/operations/build/build_tracker.py", line 92, in add
raise LookupError(message)
LookupError: https://files.pythonhosted.org/packages/a7/16/8aef42ae27036ceb201f1774940357da18ae7daac45bdd721da554892012/setuptools-scm-8.0.1.tar.gz (from https://pypi.org/simple/setuptools-scm/) (requires-python:>=3.8) is already being built: setuptools_scm>=6.4 from https://files.pythonhosted.org/packages/a7/16/8aef42ae27036ceb201f1774940357da18ae7daac45bdd721da554892012/setuptools-scm-8.0.1.tar.gz
[notice] A new release of pip is available: 23.0.1 -> 23.2.1
[notice] To update, run: pip install --upgrade pip
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 2
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
[notice] A new release of pip is available: 23.0.1 -> 23.2.1
[notice] To update, run: pip install --upgrade pip
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
[notice] A new release of pip is available: 23.0.1 -> 23.2.1
[notice] To update, run: pip install --upgrade pip
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Expected behaviour
setuptools_scm is installed with no error messages.
Workaround
Not available.
this cycle was previously missed, im not quite sure how to unravel
packaging will be removed as build dependency and i'll use the setuptools vendored one in that case
Just verified and it still fails with 8.0.2 release.Can we re-open issue to track this?
Full log of test:
$ pip3 install --upgrade --no-binary :all: anyio==4.0.0
Collecting anyio==4.0.0
Downloading anyio-4.0.0.tar.gz (153 kB)
|████████████████████████████████| 153 kB 4.0 MB/s
Installing build dependencies ... error
ERROR: Command errored out with exit status 1:
command: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-hyjnx6vc/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools >= 64' 'setuptools_scm >= 6.4'
cwd: None
Complete output (62 lines):
Collecting setuptools>=64
Downloading setuptools-68.2.2.tar.gz (2.2 MB)
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Collecting setuptools_scm>=6.4
Downloading setuptools-scm-8.0.2.tar.gz (72 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-hov0ps1j/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'importlib-metadata>=4.6; python_version < "3.10"' rich 'setuptools>=61' 'tomli; python_version < "3.11"' 'typing_extensions; python_version < "3.8"'
cwd: None
Complete output (44 lines):
Ignoring typing-extensions: markers 'python_version < "3.8"' don't match your environment
Collecting importlib-metadata>=4.6
Downloading importlib_metadata-6.8.0.tar.gz (53 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'error'
ERROR: Command errored out with exit status 2:
command: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-m7mg97oz/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=56' 'setuptools_scm[toml]>=3.4.1'
cwd: None
Complete output (33 lines):
Collecting setuptools>=56
Using cached setuptools-68.2.2.tar.gz (2.2 MB)
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Collecting setuptools_scm[toml]>=3.4.1
Using cached setuptools-scm-8.0.2.tar.gz (72 kB)
ERROR: Exception:
Traceback (most recent call last):
File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
status = self.run(options, args)
File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 357, in run
resolver.resolve(requirement_set)
File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve
discovered_reqs.extend(self._resolve_one(requirement_set, req))
File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_for
abstract_dist = self.preparer.prepare_linked_requirement(req)
File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 515, in prepare_linked_requirement
abstract_dist = _get_prepared_distribution(
File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 94, in _get_prepared_distribution
with req_tracker.track(req):
File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
return next(self.gen)
File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/req/req_tracker.py", line 148, in track
self.add(req)
File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/req/req_tracker.py", line 115, in add
raise LookupError(message)
LookupError: https://files.pythonhosted.org/packages/51/11/58d3bdef61f2090330292b52d390d57b76dfc08f003174952eff0fafa10e/setuptools-scm-8.0.2.tar.gz#sha256=e45c8c87719b753b6d47cf09907d1239540c7e150cd44f06f658b602f402b005 (from https://pypi.org/simple/setuptools-scm/) (requires-python:>=3.8) is already being built: setuptools_scm>=6.4 from https://files.pythonhosted.org/packages/51/11/58d3bdef61f2090330292b52d390d57b76dfc08f003174952eff0fafa10e/setuptools-scm-8.0.2.tar.gz#sha256=e45c8c87719b753b6d47cf09907d1239540c7e150cd44f06f658b602f402b005
----------------------------------------
ERROR: Command errored out with exit status 2: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-m7mg97oz/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=56' 'setuptools_scm[toml]>=3.4.1' Check the logs for full command output.
----------------------------------------
ERROR: Command errored out with exit status 1: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-hov0ps1j/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'importlib-metadata>=4.6; python_version < "3.10"' rich 'setuptools>=61' 'tomli; python_version < "3.11"' 'typing_extensions; python_version < "3.8"' Check the logs for full command output.
----------------------------------------
ERROR: Command errored out with exit status 1: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-hyjnx6vc/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools >= 64' 'setuptools_scm >= 6.4' Check the logs for full command output.
podman run -it --entrypoint /bin/bash python "-c" "pip3 install --upgrade --no-binary :all: anyio==4.0.0"
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
Collecting anyio==4.0.0
Downloading anyio-4.0.0.tar.gz (153 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 153.3/153.3 kB 3.9 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Collecting idna>=2.8
Downloading idna-3.4.tar.gz (183 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 183.1/183.1 kB 9.0 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting sniffio>=1.1
Downloading sniffio-1.3.0.tar.gz (17 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: anyio, idna, sniffio
Building wheel for anyio (pyproject.toml) ... done
Created wheel for anyio: filename=anyio-4.0.0-py3-none-any.whl size=83122 sha256=14d6a7627545464b0f19e8b7ff6a1a87fbba475785eee35e7d134bcab8858cf6
Stored in directory: /root/.cache/pip/wheels/f5/23/5a/b1dfb39b8a1fef8c850b021366051c9b8017b320a185faa7a1
Building wheel for idna (pyproject.toml) ... done
Created wheel for idna: filename=idna-3.4-py3-none-any.whl size=61538 sha256=eaf916f74ea1a0d0a5a2de80ad62503d1a42d2e7a413e512808add7765e645e6
Stored in directory: /root/.cache/pip/wheels/73/14/5f/87e450624ca0598cd1347aaac250ba069435fdc821c99b0e3b
Building wheel for sniffio (pyproject.toml) ... done
Created wheel for sniffio: filename=sniffio-1.3.0-py3-none-any.whl size=10166 sha256=12bf3c7f732ddbcb94fed28d717a5c1920fd7050a3bdda1aae2153a406830325
Stored in directory: /root/.cache/pip/wheels/d5/1c/0a/a5c736dd17400583869e23f40145928205a7a57d7304c491df
Successfully built anyio idna sniffio
Installing collected packages: sniffio, idna, anyio
Successfully installed anyio-4.0.0 idna-3.4 sniffio-1.3.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
seems to hit pypa/pip#11453
@i-chvets so the current recommendation is a pip upgrade it seems
Hey @RonnyPfannschmidt, I tried upgrading pip, but I still cannot manage to solve the issue:
(venv) ubuntu@charm-dev-focal:~$ pip --version
pip 23.2.1 from /home/ubuntu/venv/lib/python3.8/site-packages/pip (python 3.8)
(venv) ubuntu@charm-dev-focal:~$ pip3 install --upgrade --no-binary :all: setuptools-scm
...
LookupError: https://files.pythonhosted.org/packages/51/11/58d3bdef61f2090330292b52d390d57b76dfc08f003174952eff0fafa10e/setuptools-scm-8.0.2.tar.gz (from https://pypi.org/simple/setuptools-scm/) (requires-python:>=3.8) is already being built: setuptools_scm>=6.4 from https://files.pythonhosted.org/packages/51/11/58d3bdef61f2090330292b52d390d57b76dfc08f003174952eff0fafa10e/setuptools-scm-8.0.2.tar.gz
[end of output]
@DnPlas whats the output of pip3 -- version?
do you have a docker container that can replicate
I've got one that demonstrates the issue with 8.0.2:
# syntax=docker/dockerfile:1
FROM debian:11
RUN apt-get update && apt-get install -y python3-pip
RUN pip3 install --upgrade pip==23.2.1
RUN pip3 --version
RUN pip3 install "anyio==4.0.0" --no-binary :all:
When running the docker build, if you add in the env var BUILDKIT_PROGRESS=plain, you can see the full output, including the pip3 --version command, which reports: pip 23.2.1 from /usr/local/lib/python3.9/dist-packages/pip (python 3.9)
EDIT: Probably better to replace that last line with RUN pip3 install "setuptools_scm==8.0.2" --no-binary :all: to get to the heart of the issue.
@DnPlas whats the output of
pip3 -- version?do you have a docker container that can replicate
pip is the same as pip3:
(venv) ubuntu@charm-dev-focal:~$ pip3 --version
pip 23.2.1 from /home/ubuntu/venv/lib/python3.8/site-packages/pip (python 3.8)
Is there any workaround in the meantime? Without dropping the --no-binary :all: option.
I haven't yet figured out how to override the version of the build dependency of my dependencies.
Most of them only have a minimum version for setuptools_scm so for now I depend on the latest version on pypi.
I have tried in the following environment and it seems like working:
- Ubuntu 22.04
- python3.10
- pip (and pip3) 22.0.2
pip3 install --upgrade --no-binary :all: setuptools-scm
...
Successfully installed packaging-23.1 setuptools-scm-8.0.2 tomli-2.0.1 typing-extensions-4.8.0
It's important because in the original description and in my case as well, the environment was Ubuntu 20.04 with python3.8 and pip 23.x.x.
I identified that a pip downgrade IS necessary atm
I identified that a pip downgrade IS necessary atm
could you share which environments have you tried for this?
I've tried with this combination and the command failed
- Ubuntu 20.04
- python 3.8.10
- pip (and pip3) 20.0.2
the key issue is python 3.9 vs 3.10 - 3.10+ works out of the box, 3.9 needs some extra hacks
i found a hack that resolves it, mr incoming
@RonnyPfannschmidt how about python 3.8?
@addyess same hack works
merge and release tommorow in the morning
Everything seems to be working now, thanks!