IanWangg / MetaDrive-Tutorials

Reinforcement Learning tutorials with Metadrive: A collection of hands-on notebooks and resources to get started with reinforcement learning using the powerful MetaDrive simulation environment.

MetaDrive Tutorials

Are you interested in self-driving cars? Do you want to learn RL? If so, look no further! In this series of tutorials, we'll teach you how to train an RL model to drive a simulated car in Metadrive, an environment for testing self-driving cars. We assume only knowledge of basic ML, and will teach you any RL concepts. To ensure your understanding, we provide coding exercises for you to fill out, as well as solutions to check your work against.

Screenshots

Prerequisite Knowledge

Here's a list of the topics you should be familiar with, alongside some sources that you can review them with:

• Partial Derivatives and Gradients
  • https://www.khanacademy.org/math/multivariable-calculus/multivariable-derivatives/partial-derivative-and-gradient-articles/a/introduction-to-partial-derivatives
  • https://www.khanacademy.org/math/multivariable-calculus/multivariable-derivatives/partial-derivative-and-gradient-articles/a/the-gradient
• Probability Distributions
  • https://www.khanacademy.org/math/statistics-probability/random-variables-stats-library/random-variables-discrete/v/discrete-and-continuous-random-variables
  • https://www.khanacademy.org/math/statistics-probability/random-variables-stats-library/random-variables-continuous/v/random-variables
• Backprop
  • 3Blue1Brown's Backprop Video: https://www.youtube.com/watch?v=tIeHLnjs5U8
  • Andrei Karpathy's Micrograd Video: https://www.youtube.com/watch?v=VMj-3S1tku0
• PyTorch
  • https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html
  • https://pytorch.org/tutorials/beginner/pytorch_with_examples.html

Installing Metadrive

Note: Metadrive only officially supports Windows and Linux systems

The most up-to-date instructions for installing Metadrive can be found on the Metadrive repository

We reccomend directly git cloning the latest version of the repository, as it contains recent updates that allow you to use it with newer libraries:

git clone https://github.com/metadriverse/metadrive.git
cd metadrive
pip install -e .

Table of Contents

The tutorials build off of each other, so we reccomend reading them in the order listed:

1. Policy Gradients Explanation
   • Discrete Policy Gradient Exercise
     • Discrete Policy Gradient Solution
   • Continuous Policy Gradient Exercise
     • Continuous Policy Gradient Solution
2. Deep Q Network Explanation
   • Deep Q Network Exercise
     • Deep Q Network Solution
3. Double and Dueling DQN Explanation
   • Double and Dueling DQN Exercise
     • Double and Dueling DQN Solution
4. PPO Explanation
   • PPO Exercise
     • PPO Solution
5. Behavior Cloning Explanation
   • BC Exercise
     • BC Solution