khuyentran1401 / prefect-dvc

Prefect + DVC Demo

A demo on how to use Prefect with DVC

Tools Used in This Project

• DVC: Data version control
• Prefect: Orchestrate and observe dataflow

Project Structure

• src: consists of Python scripts
• config: consists of configuration files
• data: consists of data
• notebook: consists of Jupyter Notebooks
• tests: consists of test files

Set Up the Project

• Install requirements:

pip install -r requirements.txt

Run the Project

To run all flows, type:

python src/main.py