Bug: cyclic dependency with `tomli==1.2.1` and `flit_core==3.2.0`
lummax opened this issue · comments
I can not download tomli
because there is a cyclic dependency with flit_core
.
Reproducer:
python3.9 -m venv test
./test/bin/pip download --no-binary=":all:" tomli # fails
./test/bin/pip download --no-binary=":all:" flit_core # fails
Example output:
Collecting tomli
File was already downloaded XXX/tomli-1.2.1.tar.gz
Installing build dependencies ... error
ERROR: Command errored out with exit status 2:
command: XXX/test/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-91g152mw/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'flit_core>=3.2.0,<4'
cwd: None
Complete output (31 lines):
Collecting flit_core<4,>=3.2.0
Downloading flit_core-3.4.0.tar.gz (27 kB)
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Collecting tomli
Downloading tomli-1.2.1.tar.gz (14 kB)
ERROR: Exception:
Traceback (most recent call last):
File "XXX/test/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
status = self.run(options, args)
File "XXX/test/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 357, in run
resolver.resolve(requirement_set)
File "XXX/test/lib/python3.9/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve
discovered_reqs.extend(self._resolve_one(requirement_set, req))
File "XXX/test/lib/python3.9/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File "XXX/test/lib/python3.9/site-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_for
abstract_dist = self.preparer.prepare_linked_requirement(req)
File "XXX/test/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 515, in prepare_linked_requirement
abstract_dist = _get_prepared_distribution(
File "XXX/test/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 94, in _get_prepared_distribution
with req_tracker.track(req):
File "/usr/lib/python3.9/contextlib.py", line 117, in __enter__
return next(self.gen)
File "XXX/test/lib/python3.9/site-packages/pip/_internal/req/req_tracker.py", line 148, in track
self.add(req)
File "XXX/test/lib/python3.9/site-packages/pip/_internal/req/req_tracker.py", line 115, in add
raise LookupError(message)
LookupError: https://files.pythonhosted.org/packages/75/50/973397c5ba854445bcc396b593b5db1958da6ab8d665b27397daa1497018/tomli-1.2.1.tar.gz#sha256=a5b75cb6f3968abb47af1b40c1819dc519ea82bcc065776a866e8d74c5ca9442 (from https://pypi.org/simple/tomli/) (requires-python:>=3.6) is already being built: tomli from https://files.pythonhosted.org/packages/75/50/973397c5ba854445bcc396b593b5db1958da6ab8d665b27397daa1497018/tomli-1.2.1.tar.gz#sha256=a5b75cb6f3968abb47af1b40c1819dc519ea82bcc065776a866e8d74c5ca9442
----------------------------------------
ERROR: Command errored out with exit status 2: XXX/test/bin/python3.9 XXX/test/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-91g152mw/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'flit_core>=3.2.0,<4' Check the logs for full command output.
Companion ticket: pypa/flit#451
I don't think this is a Tomli issue. You may want to look into bootstraping the foundational build/installer packages https://flit.readthedocs.io/en/latest/bootstrap.html
I agree, to some extend this is a problem with pip download
installing build-dependencies even though I only requested to download a source-package (without building it).
But even installing flit_core
does not help. The following still reproduces the problem. Only --no-build-isolation
helps.
~ ➤ python3.9 -m venv test
~ ➤ ./test/bin/pip install flit_core
~ ➤ ./test/bin/pip download --no-binary=":all:" tomli # fails
~ ➤ ./test/bin/pip download --no-binary=":all:" tomli --no-build-isolation # works
But I am fine with having this closed. Although some part of me wishes for stricter dependency requirements so that problems like this don't happen again.
Hmm, yeah maybe pip's issue tracker is the right place for this?
The thing with --no-build-isolation
makes sense - by default, pip creates a new environment to build packages, and installs build dependencies into that. --no-build-isolation
tells it to use the current environment instead. Why pip download
tries to do any kind of building, I'm not sure, but given that it does, that's what I'd expect.
My best guess is that it wants to get the list of dependencies even if you tell it not to fetch them (--no-deps
). Getting package metadata is similar to building.
# This works - flit_core has no build dependencies
pip download --no-binary :all: flit_core --no-deps
# This doesn't
pip download --no-binary :all: tomli --no-deps
Why
pip download
tries to do any kind of building, I'm not sure, but given that it does, that's what I'd expect.