AssertionError in import_proxifier.create_module while importing from django.utils.six.moves.urllib.parse

[John Vandenberg]

Running Django 3.0.8 and django-utils-six-2.0 and django-compat-patcher master , when using https://github.com/andersinno/django-form-designer-ai I encounter an assertion error in compat_patcher_core/import_proxifier.py create_module
when it tries to load https://github.com/andersinno/django-form-designer-ai/blob/master/form_designer/views.py#L9

from django.utils.six.moves.urllib.parse import urlencode

The first assertion is at

assert module.__name__ == self.real_name, module.__name__

module.__name__ reported by the assertion is six.moves.urllib_parse

But if I stupidly comment out that assertion, there are several assertions which occur during exec_module phase of module loading.

In case it helps andersinno/django-form-designer-ai#38 gets that app roughly working.

[John Vandenberg]

Occurring again at https://travis-ci.org/github/jayvdb/bluebottle/jobs/732751957#L6827

    module = import_module(module_path)
  File "/opt/python/3.8.3/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/travis/virtualenv/python3.8.3/lib/python3.8/site-packages/corsheaders/middleware.py", line 6, in <module>
    from django.utils.six.moves.urllib.parse import urlparse
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 657, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 556, in module_from_spec
  File "/home/travis/virtualenv/python3.8.3/lib/python3.8/site-packages/compat_patcher_core/import_proxifier.py", line 124, in create_module
    assert module.__name__ == self.real_name, module.__name__
AssertionError: six.moves.urllib_parse

[John Vandenberg]

Bypassing that one

  File "/home/travis/virtualenv/python3.8.3/lib/python3.8/site-packages/django/utils/module_loading.py", line 17, in import_string
    module = import_module(module_path)
  File "/opt/python/3.8.3/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/travis/virtualenv/python3.8.3/lib/python3.8/site-packages/corsheaders/middleware.py", line 6, in <module>
    from django.utils.six.moves.urllib.parse import urlparse
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/home/travis/virtualenv/python3.8.3/lib/python3.8/site-packages/compat_patcher_core/import_proxifier.py", line 129, in exec_module
    assert module.__name__ in (self.alias_name, self.real_name), module.__name__
AssertionError: six.moves.urllib_parse

[John Vandenberg]

Removing that assert solves the problem.
Obviously just removing them isnt the best solution, and the problem wont be limited to Django and its inclusion of six as a subpackage. Many other packages vendor six, and other packages.

One way to keep the assert is to add config to identify imports to allow bypassing these asserts.

It might be possible to detect that this import doesnt need to be proxied. Possibly django-compat-patcher can declare that there should be no patching of django.utils.six.

[Pascal Chambon]

Sorry, I completely missed enabling notifications on this repository ^^'

Double-patching django.utils.six seems is unnecessary, the project maintainer should just remove django-utils-six-2.0, or disable the "fix_deletion_utils_six" fixer of DCP.

I couldn't reproduce the problem since when i try it, the patched django.utils.six has a proper "name" on Python3.8. Weird.

However, this error is very unclear, and indeed I should auto-test what happens when the compat-patcher-core tries to inject an already-injected module. I don't know if the patcher should fail, or warn and proceed, or warn and abort... or maybe it can be left as an option for the specific fixer.

[Pascal Chambon]

Ok I managed to reproduce it, I removed assertions that were wrong due to the interfering custom module importer of six.

Latest django-compat-patcher 0.10, with compat-patcher-core 1.2, should solve this.