restackio / starter-airflow

Starter repository for Airflow CI/CD with Restack

Airflow Restack repository

This is the default Airflow repository to get you started for generating preview environments from a custom Airflow image with Restack github application.

Dags

1. If you want to extend the image with custom dags, just add them to the dags directory.

Plugins

1. If you want to extend the image with custom plugins, just add them to the plugins directory.

Config

1. If you want to extend the image with custom configuration, just add them to the config directory.

XComs

1. It is possible to pass data between tasks natively just using the Astronomer provider package. To try XCOM feature just run simple_xcom_dag or fetch_pokemon_data DAGs.

Restack Product Version

Restack will expose a build arg for the Dockerfile called: RESTACK_PRODUCT_VERSION. This will contain the image tag of the product. As seen on this starter repo's Dockerfile you can use it as such:

ARG RESTACK_PRODUCT_VERSION=2.8.0
FROM apache/airflow:${RESTACK_PRODUCT_VERSION}

This way once a new version of airflow is released and you upgrade your app via Restack console, your ci/cd builds will use the latest version to build your airflow custom image.

Generating a preview environment

1. Make sure to fork this repository.
2. Follow steps in the official Restack documentation
3. Once you open a pull request a preview environment will be generated.
4. Once your pull request is merged your initial Airflow application will be provisioned with latest code from the "main" branch.