Mcamilo / Bari-DataEngineer-Test

Code exercise for a data engineer position at Bari Bank

Bari-DataEngineer-Test

Code exercise for a data engineer position at Bari Bank

Requirements

• Jupyter Notebook (https://jupyter.org/install)
• Python 3.8 (https://www.python.org/downloads/release/python-380/)
• pip (https://pypi.org/project/pip/)

Setup

Install required packages for the mock module

pip install requirements.txt

Run

Run Jupyter and open the coding_test.ipynb notebook

Mock

The mock module creates a mock s3/raw dir with data distributed by date as such:

├── s3
│   ├── raw 
│   │   ├── year-month-day
│   │   │   ├── album_randomId.json