By Thomas J. Fan
Scikit-learn is a machine learning library in Python that is used by many data science practitioners. In this training, we will learn about model evaluation, model calibration, and model inspection. For model evaluation, we will compare various metrics such as ROC AUC and mean average precision and see how they behave on datasets with different characteristics. We will use scikit-learn's plotting API to easily visualize the performance of a model and to compare multiple models. A well-calibrated model will predict probabilities that reflect the true likelihood of an event. Next, we will learn about techniques used for inspecting open-box machine learning models after they are trained. Afterwards, we will learn about inspection techniques used for more opaque models such as random forests or gradient boosted trees. These techniques are flexible because they can be applied to any machine learning model and gives a glimpse into how the model is generating its predictions.
The most convenient way to download the material is with git:
git clone https://github.com/thomasjpfan/ml-workshop-intermediate-2-of-2
Please note that I may add and improve the material until shortly before the session. You can update your copy by running:
git pull origin master
If you are not familiar with git, you can download this repository as a zip file at: github.com/thomasjpfan/ml-workshop-intermediate-2-of-2/archive/master.zip. Please note that I may add and improve the material until shortly before the session. To update your copy please re-download the material a day before the session.
Local installation requires conda
to be installed on your machine. The simplest way to install conda
is to install miniconda
by using an installer for your operating system provided at docs.conda.io/en/latest/miniconda.html. After conda
is installed, navigate to this repository on your local machine:
cd ml-workshop-intermediate-2-of-2
Then download and install the dependencies:
conda env create -f environment.yml
This will create a virtual environment named ml-workshop-intermediate-2-of-2
. To activate this environment:
conda activate ml-workshop-intermediate-2-of-2
Finally, to start jupyterlab
run:
jupyter lab
This should open a browser window with the jupterlab
interface.
If you have any issues with installing conda
or running jupyter
on your local computer, then you can run the notebooks on Google's Colab:
This repo is under the MIT License.