robinhood / airflow-prometheus-exporter

Prometheus Exporter for Airflow

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Breaks airflow initdb when no config.yml is found

Limess opened this issue · comments

The recent change #11 added a requirement to define a config.yml file. This does not gracefully handle the case where no config.yml file is found which breaks existing setups.

The README also contains no information about this required file.

The error is as follows:

touch airflow.db && airflow initdb

Results in:

Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 25, in <module>
    from airflow.configuration import conf
  File "/usr/local/lib/python3.7/site-packages/airflow/__init__.py", line 97, in <module>
    operators._integrate_plugins()
  File "/usr/local/lib/python3.7/site-packages/airflow/operators/__init__.py", line 104, in _integrate_plugins
    from airflow.plugins_manager import operators_modules, register_inbuilt_operator_links
  File "/usr/local/lib/python3.7/site-packages/airflow/plugins_manager.py", line 194, in <module>
    plugins
  File "/usr/local/lib/python3.7/site-packages/airflow/plugins_manager.py", line 104, in load_entrypoint_plugins
    plugin_obj = entry_point.load()
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2443, in load
    return self.resolve()
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2449, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python3.7/site-packages/airflow_prometheus_exporter/prometheus_exporter.py", line 11, in <module>
    from airflow_prometheus_exporter.xcom_config import xcom_config
  File "/usr/local/lib/python3.7/site-packages/airflow_prometheus_exporter/xcom_config.py", line 9, in <module>
    with open(filename) as file:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.7/site-packages/airflow_prometheus_exporter/../config.yaml'
The command '/bin/sh -c touch airflow.db && airflow initdb' returned a non-zero code: 1

yes...i'm noticing this too now that i am trying to install from pypi rather than locally (sorry!) would need to add something like:

package_data={'airflow_prometheus_exporter': ['config.yaml']}, to the setup.py file....but this would still require digging into site-packages to actually configure the xcom exporter..which is probably not a great solution. maybe the configuration could be put into airflow.cfg?

Yeah, I should have been more careful during the review/test for that diff. Just pushed and released a fix so things should hopefully be working again. You might have to bump your version and re-install the package.

Sorry for the trouble!

38559b6 seems to have fixed this, I'll close the issue.