onesky / docker-airflow

Simple Airbnb Airflow container

Airflow Dockerfile

This repository contains Dockerfile of airflow for Docker's automated build published to the public Docker Hub Registry.

Base Docker Image

• debian:wheezy

Installation

1. Install Docker.

2. Install Docker-compose.

3. Download automated build from public Docker Hub Registry: docker pull puckel/docker-airflow

Alternatively, you can build an image from Dockerfile

Usage

Start the stack (mysql, rabbitmq, airflow-webserver, airflow-flower & airflow-worker) :

  docker-compose up

UI Interface :

Airflow: http://container-ip:8080/ Flower (Celery): http://container-ip:5555/ RabbitMQ: http://container-ip:15672/

To scale the number of workers :

  docker-compose scale worker=5

Then you can run the "tutorial" :

  docker exec dockerairflow_webserver_1 airflow backfill tutorial -s 2015-05-01 -e 2015-06-01