Jinja2 v 2.11.3 has bad dependency on markupsafe

[Claudiu]

If we install Jinja2<3.0,>=2.10.1 (like it is from flask<1.2,>=1.1.1) -- we get a non-working install.

$ python3 -mvenv env/tmp
$ source env/tmp/bin/activate
(tmp) $ pip install "Jinja2<3.0,>=2.10.1"
Collecting Jinja2<3.0,>=2.10.1
  Using cached Jinja2-2.11.3-py2.py3-none-any.whl (125 kB)
Collecting MarkupSafe>=0.23
  Using cached MarkupSafe-2.1.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Installing collected packages: MarkupSafe, Jinja2
Successfully installed Jinja2-2.11.3 MarkupSafe-2.1.1
(tmp) $ python
Python 3.10.5 (main, Jun  6 2022, 18:49:26) [GCC 12.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import jinja2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/csaftoiu/work/git/testtest/env/tmp/lib/python3.10/site-packages/jinja2/__init__.py", line 12, in <module>
    from .environment import Environment
  File "/home/csaftoiu/work/git/testtest/env/tmp/lib/python3.10/site-packages/jinja2/environment.py", line 25, in <module>
    from .defaults import BLOCK_END_STRING
  File "/home/csaftoiu/work/git/testtest/env/tmp/lib/python3.10/site-packages/jinja2/defaults.py", line 3, in <module>
    from .filters import FILTERS as DEFAULT_FILTERS  # noqa: F401
  File "/home/csaftoiu/work/git/testtest/env/tmp/lib/python3.10/site-packages/jinja2/filters.py", line 13, in <module>
    from markupsafe import soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/home/csaftoiu/work/git/testtest/env/tmp/lib/python3.10/site-packages/markupsafe/__init__.py)
>>> 
(tmp) $ 

It should import and work correctly, but it fails to import.

The correct dependency should be MarkupSafe<2.1.0,>=0.23:

(tmp) $ pip install "MarkupSafe<2.1.0,>=0.23"
Collecting MarkupSafe<2.1.0,>=0.23
  Using cached MarkupSafe-2.0.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (30 kB)
Installing collected packages: MarkupSafe
  Attempting uninstall: MarkupSafe
    Found existing installation: MarkupSafe 2.1.1
    Uninstalling MarkupSafe-2.1.1:
      Successfully uninstalled MarkupSafe-2.1.1
Successfully installed MarkupSafe-2.0.1
(tmp) $ python
Python 3.10.5 (main, Jun  6 2022, 18:49:26) [GCC 12.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import jinja2
>>> 

Environment:

• Python version: 3.10.5
• Jinja version: 2.11.3

[Adrian]

But what's the problem there? You are explicitly asking for some strange version range that's not compatible. You need to pin ALL your (transitive) dependencies if you pin your top-level ones.

Also, there are tons of existing issues related to this, including detailed links on why this is important and Jinja isn't broken.

[Claudiu]

Hmm, the only pip install I did initially was for Jinja2. There was no other dependency. To put it differently, if I just pip install Jinja2==2.11.3, then it results in a non-working Jinja2 install. Shouldn't every version of Jinja2 work if there's no other dependencies interfering?

[Adrian]

No, pip will pull in the latest version of Jinja's dependencies which are no longer compatible.

For a new install, you should always use the latest version -> no conflicts.
For an old application, you should have everything pinned -> no conflicts either.

See e.g. this comment for details: #1592 (comment)

[Claudiu]

oh! ok, I didn't realize pip worked that way. thanks for explaining!