BasPH / airflow-rocket

Airflow code accompanying blog post.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Airflow Rocket

This repository contains code accompanying this Airflow blog post seriesWIP. To get started, run docker run -d -p 8080:8080 basph/airflow-rocket.

Throughout the blog post series, I explain various Airflow concepts and give examples using the Launch Library API, hence the repository name "Airflow Rocket".

IMPORTANT: The Dockerfile in this repository creates a single image containing all Airflow components, and examples and demos shown in the blog post. It is NOT intended for production usage! For more information on running Airflow in production, read the blog post part 4.

If you mount your own DAGs volume, the container takes 10-15 seconds to start up. This is because airflow initdb is executed at startup (if no .airflow-rocket file was found in the DAGs folder). The idea is to clear all built-in DAGs and only display your own, mounted, DAGs.

Versions used:

  • Airflow 1.10.2
  • Python 3.6.6
  • Launch Library API 1.4
Rocket icon made by Freepik from www.flaticon.com, licensed by CC 3.0 BY.

About

Airflow code accompanying blog post.


Languages

Language:Python 92.6%Language:Makefile 3.1%Language:Dockerfile 3.0%Language:Shell 1.4%