Code example to go along with my talk Dynamic Data Pipelining with Luigi at PyOhio 2019.
To get started:
# To run a postgres db, if you don't want to run one yourself
docker-compose up -d
# Setup python environment
python3 -m venv env
source env/bin/activate
pip install -r requirements.txt
# Run luigi!
PYTHONPATH='.' luigi --module tasks UploadTask --fname "./data/sample.csv" --local-scheduler