qubole / afctl

afctl helps to manage and deploy Apache Airflow projects faster and smoother.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

commons - the main idea and usage for import

TheML9I opened this issue · comments

Hi.

Could please someone explain what was the idea of commons folder usage? Properly, I mean how to import?.

Airflow has the init func prepare_syspath() that ensures that certain subfolders of AIRFLOW_HOME are on the classpath.
That allows to import modules from folders like DAGS_FOLDER/third_party, PLUGINS_FOLDER/third_party.

For instance i have the next structure:
── module /
│ ├── commons
│ └── dags

What have to be specified for DAGS_FOLDER to be able to import modules from commons into DAG-files.
Do I have to extend sys.path manually? If I keep this by default I will get a module import error.

I'd like to make it clear how to import modules from commons folders. Why commons dirs are on that project's structure level?