JelleZijlstra / typeshed_client

Retrieve information from typeshed and other typing stubs

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

`typeshed_client` crashes on typeshed's stubs for `six.moves.collections_abc`

AlexWaygood opened this issue · comments

Reproducer from inside a typeshed clone:

>>> import typeshed_client
>>> from pathlib import Path
>>> context = typeshed_client.finder.get_search_context(typeshed=Path("stdlib"), search_path=[Path("stubs/six")], version=(3, 11))
>>> typeshed_client.get_stub_names("six.moves.collections_abc", search_context=context)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\alexw\coding\typeshed\stubdefaulter-venv\Lib\site-packages\typeshed_client\parser.py", line 61, in get_stub_names
    return parse_ast(
           ^^^^^^^^^^
  File "C:\Users\alexw\coding\typeshed\stubdefaulter-venv\Lib\site-packages\typeshed_client\parser.py", line 76, in parse_ast
    names = visitor.visit(ast)
            ^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\AppData\Local\Programs\Python\Python311\Lib\ast.py", line 418, in visit
    return visitor(node)
           ^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\typeshed\stubdefaulter-venv\Lib\site-packages\typeshed_client\parser.py", line 188, in visit_Module
    return [info for child in node.body for info in self.visit(child)]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\typeshed\stubdefaulter-venv\Lib\site-packages\typeshed_client\parser.py", line 188, in <listcomp>
    return [info for child in node.body for info in self.visit(child)]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\typeshed\stubdefaulter-venv\Lib\site-packages\typeshed_client\parser.py", line 298, in visit_ImportFrom
    names = get_import_star_names(
            ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\typeshed\stubdefaulter-venv\Lib\site-packages\typeshed_client\parser.py", line 127, in get_import_star_names
    return get_dunder_all_from_info(info)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\typeshed\stubdefaulter-venv\Lib\site-packages\typeshed_client\parser.py", line 141, in get_dunder_all_from_info
    raise InvalidStub(f"Invalid __all__: {info}")
typeshed_client.parser.InvalidStub: Invalid __all__: NameInfo(name='__all__', is_exported=False, ast=ImportedName(module_name=('_collections_abc',), name='__all__'), child_nodes=None)

It looks like an issue to do with __all__. Possibly because collections.abc does the unusual from _collections_abc import __all__ as __all__ thing.

On the other hand, maybe typeshed_client is right: maybe this is an invalid stub.

In stdlib/collections/abc.pyi, we do from _collections_abc import __all__ as __all__. That implies that _collections.abc.__all__ is being re-exported from collections.abc in stdlib/collections/abc.pyi. But it's not, as "__all__" is not included in _collections.abc.__all__, and the contents of __all__ is what decides whether something is publicly re-exported or not.

Maybe in the stub, we should actually be doing:

from _collections_abc import __all__ as _collections_abc___all__
__all__ = _collections_abc___all__