Welcome to the GIT repository for the code samples included in the book. The code included in this repository are subject to change without prior notice.
This book is an essential guide to leveraging the Databricks Lakehouse for a development and deployment platform of data products. You will build your expertise and resume by doing hands-on projects that cover the entire data lifecycle.
This book is for machine learning engineers, data scientists and analysts that want to learn hands-on about implementing and leveraging the Databricks Lakehouse Platform to create data products.
Each chapter folder contains code examples shared in the book using one of our data sources, and the links shared in the README file:
- Chapter 1 - Getting Started
- Chapter 2 - Designing Databricks Day One
- Chapter 3 - Building Out Our Bronze Layer
- Chapter 4 - Exploring and Cleaning Toward a Silver Layer
- Chapter 5 - Feature Engineering on the Lakehouse
- Chapter 6 - Searching for Signal
- Chapter 7 - Productionizing Machine Learning
- Chapter 8 - Sharing Insights
You will need a Databricks environment and permissions to run a cluster in order to follow along. There is a Databricks community Edition that you can use to run the provided notebooks and code.
The authors will do its best to keep the code and examples provided as up-to-date as possible, but we understand that you may encounter outdated snippets or other issues. Please post your enquiries in the issues page) should you require further assistance.
Stephanie Rivera has been working in big data and machine learning for 12 years. She collaborates with teams and companies as they design their Lakehouse as a Sr. Solutions Architect for Databricks.
Previously Stephanie was the VP, Data Intelligence for a global company, taking in 20+ terabytes of data daily. She led the data science, data engineering, and business intelligence teams.
Her data career has also included contributing to and leading a team in creating software that teaches people to explore fictional planets using data science algorithms. Stephanie authored numerous sections of Booz Allen Hamilton’s publication, The Field Guide to Data Science.
I want to thank my loving partner, Rami Alba, Databricks coworkers, and friends who have supported me.
Mandy Baker began her career in data 8 years ago. She loves leveraging her skills as a data scientist to orchestrate transformative journeys for companies across diverse industries as a Solutions Architect for Databricks. Her experiences have brought her from large corporations to small startups and everything in between. Mandy is a graduate of Carnegie Mellon University and the University of Washington.
Thank you to my partner Emmanuel, my parents, sisters, and friends for their enduring love and support.
Hayley Horn started her data career 15 years ago as a data quality consultant on enterprise data integration projects. As a data scientist, she specialized in customer insights and strategy, and presented at Data Science and AI conferences in the US and Europe. She is currently a Sr. Solutions Architect for Databricks, with expertise in data science and technology modernization.
A graduate of the MS Data Science program at Southern Methodist University in Dallas, Texas, USA, she is now a capstone advisor to students in their final semesters of the program.
I’d like to thank my husband, Kevin, and my sons Dyson and Dalton for their encouragement and enthusiastic support.
Anastasia Prokaieva began her career 9 years ago, as a research scientist at CEA (France), focusing on large data analysis and satellite data assimilation, treating terabytes of data. She has been working within the big data analysis and machine learning domain since then. In 2021, she joined Databricks and became the regional AI subject matter expert.
On a daily basis, Anastasia consults Databricks users on best practices implementation of AI projects end-to-end, she delivers trainings, workshops to democratize AI. Anastasia holds two MSc degrees in theoretical physics and energy science.
I would like to thank my partner, Julien, and my family for their tremendous support. My gratitude to my talented teammates all around the globe, as you inspire me every day!
Hope you will enjoy it as much as we enjoyed writing it for you.