damnever / pigar

:coffee: A tool to generate requirements.txt for Python project, and more than that. (IT IS NOT A PACKAGE MANAGEMENT TOOL)

Home Page:https://damnever.github.io/pigar/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Bug with apache-airflow package recognition.

Al-Clutch opened this issue · comments

commented

Describe the bug
pigar throws an error when creating requirements.txt that includes the apache-airflow(airflow) packages.

To Reproduce
Steps to reproduce the behavior:

  1. Setup directories and files..
  2. Install: pip install pigar
  3. Run: pigar generate
  4. See error..

Expected behavior
airflow or apache-airflow added to requirements.txt

Context (Environment)

  • OS: MacOS Monterey 12.6.1
  • IDE: PyCharm 2022.2.3 (Professional Edition)
  • Version of Python: 3.9.x
  • Version of pigar: 2.0.3

Additional context
Issue package: https://pypi.org/project/apache-airflow/

Tested the standard import practice for these packages (e.g. from airflow import DAG) as well as from apache-airflow import DAG. The method using apache-airflow isn't recognized by my IDE. The method using airflow recognizes the package, but breaks pigar generate.

Please select one or more packages from the below list for the module "airflow" (input * to select all, multiple values can be sperated by ",")
[apache-airflow, apache-airflow-providers-common-sql, apache-airflow-providers-ftp, apache-airflow-providers-http, apache-airflow-providers-imap, apache-airflow-providers-sqlite]
(the best match may be "apache-airflow"): apache-airflow
[2022-12-21 08:32:56,850] {configuration.py:1450} INFO - Reading the config from /Users/bronze.toad/airflow/airflow.cfg
[2022-12-21 08:32:56,863] {settings.py:58} INFO - Configured default timezone Timezone('UTC')
Traceback (most recent call last):
  File "/Users/bronze.toad/Documents/Projects/ProjectWithApacheAirflowDAGs/.venv/bin/pigar", line 8, in <module>
    sys.exit(main())
  File "/Users/bronze.toad/Documents/Projects/ProjectWithApacheAirflowDAGs/.venv/lib/python3.9/site-packages/pigar/__main__.py", line 475, in main
    cli()
  File "/Users/bronze.toad/Documents/Projects/ProjectWithApacheAirflowDAGs/.venv/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/bronze.toad/Documents/Projects/ProjectWithApacheAirflowDAGs/.venv/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/bronze.toad/Documents/Projects/ProjectWithApacheAirflowDAGs/.venv/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/bronze.toad/Documents/Projects/ProjectWithApacheAirflowDAGs/.venv/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/bronze.toad/Documents/Projects/ProjectWithApacheAirflowDAGs/.venv/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/bronze.toad/Documents/Projects/ProjectWithApacheAirflowDAGs/.venv/lib/python3.9/site-packages/pigar/__main__.py", line 236, in generate
    analyzer.analyze_requirements(
  File "/Users/bronze.toad/Documents/Projects/ProjectWithApacheAirflowDAGs/.venv/lib/python3.9/site-packages/pigar/core.py", line 70, in analyze_requirements
    is_stdlib, code_path = check_stdlib(name)
  File "/Users/bronze.toad/Documents/Projects/ProjectWithApacheAirflowDAGs/.venv/lib/python3.9/site-packages/pigar/core.py", line 491, in _wrapper
    checked[name] = func(name)
  File "/Users/bronze.toad/Documents/Projects/ProjectWithApacheAirflowDAGs/.venv/lib/python3.9/site-packages/pigar/core.py", line 509, in check_stdlib
    spec = importlib.util.find_spec(name)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/util.py", line 114, in find_spec
    raise ValueError('{}.__spec__ is None'.format(name))
ValueError: airflow.operators.dummy.__spec__ is None

pip install pigar==2.0.4 should have fixed this.