abhimanyudubey / Rainbow

Rainbow: Combining Improvements in Deep Reinforcement Learning

Rainbow

Rainbow: Combining Improvements in Deep Reinforcement Learning [1].

Results and pretrained models can be found in the releases.

• DQN [2]
• Double DQN [3]
• Prioritised Experience Replay [4]
• Dueling Network Architecture [5]
• Multi-step Returns [6]
• Distributional RL [7]
• Noisy Nets [8]

Data-efficient Rainbow [9] can be run using the following options:

python main.py --target-update 2000 \
               --T-max 100000 \
               --learn-start 1600 \
               --memory-capacity 100000 \
               --replay-frequency 1 \
               --multi-step 20 \
               --architecture data-efficient \
               --hidden-size 256 \
               --learning-rate 0.0001 \
               --evaluation-interval 10000

Requirements

• atari-py
• OpenCV Python
• Plotly
• PyTorch

To install all dependencies with Anaconda run conda env create -f environment.yml and use source activate rainbow to activate the environment.

Available Atari games can be found in the atari-py ROMs folder.

Acknowledgements

• @floringogianu for categorical-dqn
• @jvmancuso for Noisy layer
• @jaara for AI-blog
• @openai for Baselines
• @mtthss for implementation details

References

[1] Rainbow: Combining Improvements in Deep Reinforcement Learning
[2] Playing Atari with Deep Reinforcement Learning
[3] Deep Reinforcement Learning with Double Q-learning
[4] Prioritized Experience Replay
[5] Dueling Network Architectures for Deep Reinforcement Learning
[6] Reinforcement Learning: An Introduction
[7] A Distributional Perspective on Reinforcement Learning
[8] Noisy Networks for Exploration
[8] When to Use Parametric Models in Reinforcement Learning?