Main goals of this project:
- To Define fact and dimension tables for a star schema for a particular analytic focus
- Build an ETL pipeline that transfers data local files into these tables in Postgres using Python and SQL.
- Navigate to the project folder
- Create the postgres container:
docker-compose up
- Create the sparkifydb database and tables:
python3 create_tables.py
- Navigate to the project folder
- Create the environment:
conda env create -f environment.yml
- Activate the new environment:
conda activate rubrics
- Run the ETL pipeline:
python3 etl.py