hubmapconsortium / airflow-dev

Configuration, workflows, and DAGs for the development instance of Airflow on HIVE infrastructure

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

airflow-dev

This repo, and the purposes outlined below, have been mostly replaced by ingest-pipeline. Currently moving relevant issues over to ingest-pipeline or other active repos.

Previous Purpose: This was used for configuration, workflows, and DAGs for the development instance of Airflow on HIVE infrastructure.

If you had a workflow you'd like to run as data is submitted to HuBMAP, make a PR here and then ...?

The details of most workflows will be in docker containers. For workflows which take intermediate formats and translate them into formats for visualization on the web, see the portal-containers repo.

About

Configuration, workflows, and DAGs for the development instance of Airflow on HIVE infrastructure

License:MIT License


Languages

Language:Python 62.9%Language:Common Workflow Language 18.0%Language:Shell 14.0%Language:Dockerfile 5.1%