stemitom / postgres-pipeline

A simple pipeline infrastructure with ETL pipeline contained in a Docker environment on Apache Airflow for orchestration and Postgres for data warehousing

Postgres Pipeline

About   |   Technologies   |   Requirements   |   Starting   |   License   |   Author

🎯 About

A simple pipeline infrastructure with ETL pipeline contained in a docker environment using apache airflow for orchestration and postgres for data warehousing

🚀 Technologies

The following tools were used in this project:

• Apache Airflow
• Postgres
• Docker

✅ Requirements

Before starting 🏁, you need to have Git, Python3 and Docker-Compose installed.

🏁 Starting

# Clone this project
$ git clone https://github.com/stemitom/postgres-pipeline

# Access
$ cd postgres-pipeline

# Install dependencies in your environment
$ pip install -r requirements.txt

# Run the project
$ docker-compose up -d

# The airflow server will initialize at <http://localhost:8080>

✍️ Todo

• Add visualization with Metabase

📝 License

This project is under license from MIT. For more details, see the LICENSE file.

Made with ❤️ by Temiloluwa Samuel

 

Back to top