There are 1 repository under airflow-docker topic.
Source code of the Apache Airflow Tutorial for Beginners on YouTube Channel Coder2j (https://www.youtube.com/c/coder2j)
The goal of this project is to track the expenses of Uber Rides and Uber Eats through data Engineering processes using technologies such as Apache Airflow, AWS Redshift and Power BI.
Repo for building docker based airflow image. Containers support multiple features like writing logs to local or S3 folder and Initializing GCP while container booting. https://abhioncbr.github.io/docker-airflow/
My self-learning about Apache Airflow
An open-source project dedicated to constructing robust data pipelines and scalable software infrastructure. We leverage industry-standard tools favored by developers to enhance efficiency and reliability. Uniquely, these pipelines are field-tested on farms across Sumatra, Indonesia, ensuring real-world applicability and resilience.
Sync DAG changes from Git to Airflow
This is my Apache Airflow Local development setup on Windows 10 WSL2/Mac using docker-compose. It will also include some sample DAGs and workflows.
Airflow Pipeline for Machine Learning
Apache Airflow Guide
An End-to-End ETL data pipeline that leverages pyspark parallel processing to process about 25 million rows of data coming from a SaaS application using Apache Airflow as an orchestration tool and various data warehouse technologies and finally using Apache Superset to connect to DWH for generating BI dashboards for weekly reports
Docker for airflow with mysql as backend
From data gathering to model deployment. Complete ML pipeline using Docker, Airflow and Python.
An airflow deployment configuration with sane defaults
Building Data Warehouse on BigQuery which takes flat file as the data sources with Airflow as the Orchestrator
A starting point for a data stack using Python, Apache Airflow and Metabase.
Quick Docker Compose deployment for Apache Airflow version 2.x. Ideal for local development
In this project, you will be building a Twitter Scheduler using Apache Airflow on Docker.
Run Apache Airflow using Docker containers
🎵 LyricWave - Your AI Music Composer 🎶 Compose Unique MP4 Songs Effortlessly! LyricWave uses AI to create personalized music by harmonizing lyrics with captivating melodies and synthetic vocals. Unleash your musical creativity today! 🚀🎶
Tools to streamline Jupyter Notebook Prototypes into robust Data Products
Shares ETL to scrape historical data of one shares and import in metabase
Airflow, Spark and Kafka example
This repository includes data engineering projects using Apache Airflow. I hope to add more projects using different technologies soon!
ETL que extraí diariamente os trending topics do Twitter em diversos países, realiza transformações com PySpark e envia os dados para o Amazon S3 utilizando Apache Airflow como orquestrador.
Airflow docker boiler plate with LocalExecutor, CeleryExecutor. #TODO Kubernetes
to start airflow in local with basic setup
Airflow DAG tutorial with docker compose local setup
Automated Indeed Job Offer Scraper: Airflow Orchestrated and Scheduled, Data Loaded into PostgreSQL Database
Airflow helm chart (KubernetesExecutor)
This set of code and instructions has the porpouse to instanciate a compiled environment with set of docker images like airflow webserver, airflow scheduler, postgresql, pyspark, Data Pipeline consuming data from weather api , processing with pyspark and storing in postgresql
Apache Airflow quick setup orchestrated with Docker Compose
Simple python script for easy local airflow deployment with docker. Packed with additional components. Will be adding more going forward.
Modelt(mow·delt) is a modern data integration solution that connects data to data for advanced analytics.
A project of Airflow DAGs created using Airflow operators
Deploying Classification Model on GCP for detecting anomalous behavior in credit-card transactions. A crucial aspect for credit-card companies to recognize fraudulent activity and not charge customers for items they didn't purchase. A subtle line needs to be drawn and my main focus here is to reduce blocked-transactions of legitimate customers.