eyadsibai / ds-template

Template for a data science project

Data Science Cookie Cutter

Quick Start

Set up the environment

1. Install Poetry
2. Set up the environment:

make setup
make activate

Install new packages

To install new PyPI packages, run:

poetry add <package-name>

Run Python scripts

To run the Python scripts to process data, train model, and run a notebook, type the following:

make pipeline

View all flow runs

A flow is the basis of all Prefect workflows.

To view your flow runs from a UI, sign in to your Prefect Cloud account or spin up a Prefect Orion server on your local machine:

prefect orion start

Open the URL http://127.0.0.1:4200/, and you should see the Prefect UI:

Run flows from the UI

After creating a deployment, you can run a flow from the UI with default parameters:

or custom parameters:

Auto-generate API documentation

To auto-generate API document for your project, run:

make docs_save

Run tests when creating a PR

When creating a PR, the tests in your tests folder will automatically run.