PyTorch official tutorial to build an AI-powered Mario.
Install dependencies with
conda env create -f environment.yml
Check the new environment mario is created successfully.
Activate mario enviroment
conda activate myenv
To start the learning process for Mario,
This starts the double Q-learning and logs key training metrics to
checkpoints. In addition, a copy of
MarioNet and current exploration rate will be saved.
GPU will automatically be used if available. Training time is around 80 hours on CPU and 20 hours on GPU.
To evaluate a trained Mario,
This visualizes Mario playing the game in a window. Performance metrics will be logged to a new folder under
checkpoints. Change the
Mario.load() to check a specific timestamp.
main.py Main loop between Environment and Mario
agent.py Define how the agent collects experiences, makes actions given observations and updates the action policy.
wrappers.py Environment pre-processing logics, including observation resizing, rgb to grayscale, etc.
neural.py Define Q-value estimators backed by a convolution neural network.
MetricLogger that helps track training/evaluation performance.
tutorial.ipynb Interactive tutorial with extensive explanation and feedback. Run it on Google Colab.
- Episode: current episode
- Step: total number of steps Mario played
- Epsilon: current exploration rate
- MeanReward: moving average of episode reward in past 100 episodes
- MeanLength: moving average of episode length in past 100 episodes
- MeanLoss: moving average of step loss in past 100 episodes
- MeanQValue: moving average of step Q value (predicted) in past 100 episodes
Checkpoint for a trained Mario: https://drive.google.com/file/d/1RRwhSMUrpBBRyAsfHLPGt1rlYFoiuus2/view?usp=sharing
Deep Reinforcement Learning with Double Q-learning, Hado V. Hasselt et al, NIPS 2015: https://arxiv.org/abs/1509.06461
OpenAI Spinning Up tutorial: https://spinningup.openai.com/en/latest/
Reinforcement Learning: An Introduction, Richard S. Sutton et al. https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf
super-mario-reinforcement-learning, GitHub: https://github.com/sebastianheinz/super-mario-reinforcement-learning
Deep Reinforcement Learning Doesn't Work Yet: https://www.alexirpan.com/2018/02/14/rl-hard.html