elyra-ai / elyra

Elyra extends JupyterLab with an AI centric approach.

Home Page:https://elyra.readthedocs.io/en/stable/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Missing Airflow Images

dvaldivia opened this issue · comments

Hi, I'm trying to get Elyra to work with the latest Airflow chart, the instructions on the website are not up to date but that's not a problem I can send a PR with the updated chart changes, however the documentation suggest we use elyra/airflow image, but that image on Docker hub is about 2 years old and the Makefile on the repo doesn't build that container, only elyra and kf-notebooks.

What is the proper way to build the airflow container?

Right now attempting to submit an elyra pipeline to airflow causes the following exception:

Broken DAG: [/opt/airflow/dags/repo/sentiment_analysis-0509204952.py] Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/airflow/dags/repo/sentiment_analysis-0509204952.py", line 2, in <module>
    from airflow.contrib.kubernetes.volume_mount import VolumeMount
ModuleNotFoundError: No module named 'airflow.contrib.kubernetes'

I tried using the elyra/airflow:3.0.0rc1 and 2.3.0rc1 images, but they fail to start for migrations as the airflow binary is not up to date with airflow

Could you please clarify? We are currently supporting Airflow 1.x at the moment with Elyra.