Software engineering best practices for Databricks notebooks
This repository is a companion for the example article "Software engineering best practices for Databricks notebooks" (AWS | Azure | GCP).
Going through the example, you will:
- Add notebooks to Databricks Repos for version control.
- Extracts portions of code from one of the notebooks into a shareable component.
- Test the shared code.
- Automatically run notebooks in git on a schedule using a Databricks job.
- Optionally, apply CI/CD to the notebooks and the shared code.
The example is hands-on. We recommend working it step-by-step to learn how to apply these techniques to your own Databricks notebooks.