KAUST-Academy / practical-tools-for-machine-learning

Course materials for a practical, tools-focused course on machine learning

Practical Tools for Machine Learning

There is strong demand for machine learning (ML) skills and expertise to solve challenging business problems both globally and locally in KSA. This course will help learners build capacity in core ML tools and methods and enable them to develop their own machine learning applications. This course covers the basic theory behind ML algorithms but the majority of the focus is on hands-on examples using Scikit Learn and PyTorch.

Learning Objectives

The primary learning objective of this course is to provide students with practical, hands-on experience with state-of-the-art machine learning and deep learning tools that are widely used in industry.

The first part of this course covers chapters 1-10 of Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow. The following topics will be discussed.

• The Machine Learning Landscape 
• End-to-End Machine Learning Projects 
• Classification 
• Training Models 
• Support Vector Machines (SVMs) 
• Decision Trees 
• Ensemble Methods and Random Forests 
• Dimensionality Reduction 
• Unsupervised Learning Techniques 
• Introduction to Artificial Neural Networks 

Lessons

Day 1: Introduction to Machine Learning, Part I

• The morning session will focus on the theory behind linear models for solving basic classification and regression problems by covering chapters 1-4 of Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow.
• The afternoon session will focus on applying the techniques learned in the morning session using Scikit Learn, followed by a short assessment on the Kaggle data science competition platform.

Day 2: Introduction to Machine Learning, Part II

• Consolidation of previous days content via Q/A and live coding demonstrations.
• The morning session will focus on the theory behind non-linear models for solving basic classification and regression problems by covering chapters 5-7 of Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow.
• The afternoon session will focus on applying the techniques learned in the morning session using Scikit Learn, followed by a short assessment on the Kaggle data science competition platform.

Day 3: Introduction to Machine Learning, Part III

• Consolidation of previous days content via Q/A and live coding demonstrations.
• The morning session will focus on the theory behind dimensionality reduction and unsupervised learning techniques as well as introducing the theory behind neural networks by covering chapters 8-10 of Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow.
• The afternoon session will focus on applying the techniques learned in the morning session using Scikit Learn, followed by a short assessment using the Kaggle data science competition platform.

Day 4: Introduction to Machine Learning, Part IV

• Consolidation of previous days content via Q/A and live coding demonstrations.
• The morning session will focus on applying the techniques learned over the previous days using Scikit-Learn.
• The afternoon session will allow time for a final assessment as well as additional time for learners to complete any of the previous assessments.

Organization

Project organization is based on ideas from Good Enough Practices for Scientific Computing.

1. Put each project in its own directory, which is named after the project.
2. Put external scripts or compiled programs in the bin directory.
3. Put raw data and metadata in a data directory.
4. Put text documents associated with the project in the doc directory.
5. Put all Docker related files in the docker directory.
6. Install the Conda environment into an env directory.
7. Put all notebooks in the notebooks directory.
8. Put files generated during cleanup and analysis in a results directory.
9. Put project source code in the src directory.
10. Name all files to reflect their content or function.

Building the Conda environment

After adding any necessary dependencies that should be downloaded via conda to the environment.yml file and any dependencies that should be downloaded via pip to the requirements.txt file you create the Conda environment in a sub-directory ./envof your project directory by running the following commands.

export ENV_PREFIX=$PWD/env
mamba env create --prefix $ENV_PREFIX --file environment.yml --force

Once the new environment has been created you can activate the environment with the following command.

conda activate $ENV_PREFIX

Note that the ENV_PREFIX directory is not under version control as it can always be re-created as necessary.

For your convenience these commands have been combined in a shell script ./bin/create-conda-env.sh. Running the shell script will create the Conda environment, activate the Conda environment, and build JupyterLab with any additional extensions. The script should be run from the project root directory as follows.

./bin/create-conda-env.sh

Ibex

The most efficient way to build Conda environments on Ibex is to launch the environment creation script as a job on the debug partition via Slurm. For your convenience a Slurm job script ./bin/create-conda-env.sbatch is included. The script should be run from the project root directory as follows.

sbatch ./bin/create-conda-env.sbatch

Listing the full contents of the Conda environment

The list of explicit dependencies for the project are listed in the environment.yml file. To see the full lost of packages installed into the environment run the following command.

conda list --prefix $ENV_PREFIX

Updating the Conda environment

If you add (remove) dependencies to (from) the environment.yml file or the requirements.txt file after the environment has already been created, then you can re-create the environment with the following command.

$ mamba env create --prefix $ENV_PREFIX --file environment.yml --force

Installing the NVIDIA CUDA Compiler (NVCC) (Optional)

Installing the NVIDIA CUDA Toolkit manually is only required if your project needs to use the nvcc compiler. Note that even if you have not written any custom CUDA code that needs to be compiled with nvcc, if your project uses packages that include custom CUDA extensions for PyTorch then you will need nvcc installed in order to build these packages.

If you don't need nvcc, then you can skip this section as conda will install a cudatoolkit package which includes all the necessary runtime CUDA dependencies (but not the nvcc compiler).

Workstation

You will need to have the appropriate version of the NVIDIA CUDA Toolkit installed on your workstation. If using the most recent versionf of PyTorch, then you should install NVIDIA CUDA Toolkit 11.1 (documentation).

After installing the appropriate version of the NVIDIA CUDA Toolkit you will need to set the following environment variables.

$ export CUDA_HOME=/usr/local/cuda-11.1
$ export PATH=$CUDA_HOME/bin:$PATH
$ export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH

Ibex

Ibex users do not neet to install NVIDIA CUDA Toolkit as the relevant versions have already been made available on Ibex by the Ibex Systems team. Users simply need to load the appropriate version using the module tool.

$ module load cuda/11.1.1

Using Docker

In order to build Docker images for your project and run containers with GPU acceleration you will need to install Docker, Docker Compose and the NVIDIA Docker runtime.

Detailed instructions for using Docker to build and image and launch containers can be found in the docker/README.md.