tmat84 / ETL_Pipeline

This is a simple data pipeline that downloads holiday data from https://calendarific.com website

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Build Status codecov

The small ETL project inspired by Karolina Sowinska

This is a simple data pipeline that downloads holiday data from https://calendarific.com website on a defined month and country and saves that data in a postgress database.

The pipeline will be scheduled to run monthly using Airflow.

The postgreDB and Airflow are installed locally using docker.

Generate your access token here: https://calendarific.com
Set up postgres database using docker here: https://www.youtube.com/watch?v=aHbE3pTyG-Q&t=177s
Set network for postgres inside docker here: https://www.youtube.com/watch?v=lNsJRM9ga-s&t=750s
Set airflow using docker here: https://www.youtube.com/watch?v=aTaytcxy2Ck

About

This is a simple data pipeline that downloads holiday data from https://calendarific.com website

License:MIT License


Languages

Language:Python 100.0%