The objective is to build an ETL pipeline orchestrated using Apache Airflow. Extract data from Spotify using Spotify's Web API, transform extracted data using Python and finally load data into Postgres database.
![spotify_etl_diagram](https://private-user-images.githubusercontent.com/64268620/308783186-c6163f70-4a81-4bd6-91bb-517db7febc40.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTI5NTY4OTQsIm5iZiI6MTcxMjk1NjU5NCwicGF0aCI6Ii82NDI2ODYyMC8zMDg3ODMxODYtYzYxNjNmNzAtNGE4MS00YmQ2LTkxYmItNTE3ZGI3ZmViYzQwLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA0MTIlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNDEyVDIxMTYzNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWU0ZDI4OWE1ZTgyOTJlNWU4MzM1MTE1NWU3YjdlYWZkMmMxMzQ2YjI4MmMwMTNkZTM2ZjZjMTQ2NDcwODQ5MzQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.AudD-n28mRtlVeydL1KdieK-wp55BxA6MvGzhp7InCI)
![AirflowDAG](https://private-user-images.githubusercontent.com/64268620/308783238-5d3c3da1-d02a-4d99-ad5c-f2e304ff12c7.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTI5NTY4OTQsIm5iZiI6MTcxMjk1NjU5NCwicGF0aCI6Ii82NDI2ODYyMC8zMDg3ODMyMzgtNWQzYzNkYTEtZDAyYS00ZDk5LWFkNWMtZjJlMzA0ZmYxMmM3LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA0MTIlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNDEyVDIxMTYzNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTdlMjhhYjM3MGY0Y2NkNTEwMmZmNGZkY2U4NWMwZjE4MjdhMjI2YzIyOTZjMjgyNmY3ZDI4MzU3ZTNiZjlhNjQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.U8DRcRQ_54fi8XtZ6dlTio7qB3IPHrScv6w5AA7HWh0)
![PSQL](https://private-user-images.githubusercontent.com/64268620/308783310-6876d77b-1343-4c86-9688-58c457fc412e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTI5NTY4OTQsIm5iZiI6MTcxMjk1NjU5NCwicGF0aCI6Ii82NDI2ODYyMC8zMDg3ODMzMTAtNjg3NmQ3N2ItMTM0My00Yzg2LTk2ODgtNThjNDU3ZmM0MTJlLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA0MTIlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNDEyVDIxMTYzNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWU0NjQ4MThkMzhiZGNlNjhjMGZlNmI4NDY5MDM5YzA5ZjQzNTdiNWEyZGQ5NjI0YWFlNjAwNDMwZDkzMGNjMjUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.US-dQzITT-jL-YxtHh3pDvuZRX4A7fuUHI69O2tde6I)
ETL Pipeline for Spotify API on Airflow