🐛 Pinning importlib-metadata from above creates incompatibility with twine
FrancescoCasalegno opened this issue · comments
Dear explosion,
In your release v2.0.2 you have introduced a max version constraint for importlib-metadata which needs to be <3.3.0.
Line 1 in 7a39fc7
This makes catalogue incompatible with the twine (which is a quite useful package to test the packaging + upload to PyPI), since twine has been requiring importlib-metadata >= 3.6 since their release v3.4.0.
Was there any specific reason for pinning importlib-metadata to <3.3.0?
Can you please suggest some solution to allow compatibility with twine?
Thank you in advance!
Ugh, this is all very frustrating. We don't really want to have an upper pin here, but we've had multiple reports of performance regressions in spacy with importlib-metadata>=3.9 plus some internal test failures with >=3.3. The internal test failures are secondary (it's using an internal part of API that works with the built-in importlib.metadata and with importlib-metadata<3.3.0), but the performance regressions with spacy are a serious concern.
We could modify the pin to <3.9.0, but this is more of a band-aid than a longer-term solution, I suspect. In trying to track down the test failures and performance regressions, our impression is that the importlib-metadata package is just not stable enough for our usage in catalogue. We are discussing alternatives and will hopefully have a better solution soon in the next release of catalogue.
There are no other differences between v2.0.1 and v2.0.2, so you can downgrade to v2.0.1 in the meanwhile.
Dear @adrianeboyd,
Thanks for your reply!
so you can downgrade to v2.0.1 in the meanwhile
Sure, that's indeed what we are currently doing, i.e. pinning v2.0.1 (notice indeed that spacy requires catalogue>=2.0.1 so this is the only version that works with twine).
At the same time we would like to remove pins from above in our setup.py so if you can manage to solve the issue more radically (i.e. remove importlib-metadata altogether, or similar) it would be great!
Looking forward to see the next release of catalogue 😃
Okay, we decided to vendor importlib-metadata v3.2.0 with catalogue, so catalogue v2.0.3 should resolve this issue. We'll remove the importlib-metadata requirement from all our packages in the next patch releases.
Since catalogue==2.0.3 we get this error if we do this 2 imports in this order:
import thinc
import hypothesis
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/conda/lib/python3.7/site-packages/hypothesis/__init__.py", line 61, in <module>
run()
File "/opt/conda/lib/python3.7/site-packages/hypothesis/entry_points.py", line 61, in run
for entry in get_entry_points(): # pragma: no cover
File "/opt/conda/lib/python3.7/site-packages/hypothesis/entry_points.py", line 32, in get_entry_points
yield from importlib_metadata.entry_points().get("hypothesis", [])
File "/opt/conda/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 884, in entry_points
return SelectableGroups.load(eps).select(**params)
File "/opt/conda/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 340, in load
ordered = sorted(eps, key=by_group)
File "/opt/conda/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 882, in <genexpr>
dist.entry_points for dist in unique(distributions())
File "/opt/conda/lib/python3.7/site-packages/importlib_metadata/_itertools.py", line 16, in unique_everseen
k = key(element)
AttributeError: 'PathDistribution' object has no attribute 'name'
Thanks for the report! We vendored an older version of importlib-metadata with catalogue with this version and thought it was separate enough, but it still ended up interacting with importlib-metadata if it's installed, effectively leading to a mixture of two incompatible versions.
This should be fixed in the upcoming v2.0.4.
As a workaround in the meanwhile, I think you should be able to use importlib-metadata<3.4.0 instead with hypothesis.