Airflow docker container based on Miniconda
By default airflow --help is run:
docker run godatadriven/airflow
To run the webserver, try our upgradedb_webserver command which first runs upgradedb (inits and upgrades the airflow database) before it starts the webserver
docker run godatadriven/airflow upgradedb_webserver
When starting the container, the entrypoint script will look for additional dependencies to install in /dependencies.
It will install:
- Debian dependencies specified in
/dependencies/packages.txt - Pip dependencies specified in
/dependencies/requirements.txt - Conda dependencies specified in
/dependencies/environment.yml
In the dependencies_example folder we've included some examples.
Additionally, you can customize the container at runtime with a pre and post hook. These will be called before, and after calling upgrade db.
We've build three compose files which use postgresql as a database.
- The first compose file starts the webserver and localexecutor as separate containers.
- The second compose file uses celery to communicate with the executors.
- Finally the last compose file shows how to add additional dependencies at runtime.