hitersyw / Deep-Reinforcement-Learning-Algorithms-with-Pytorch

Clean and robust implementations of Reinforcement Learning algorithms by Pytorch

Clean, Robust, and Unified implementation of classical Deep Reinforcement Learning Algorithms

Link of my code:

• Q-learning:
• DQN/DDQN on Classic Control:
• DQN/DDQN on Atari Game:
• Prioritized DQN/DDQN on Classic Control:
• Proximal Policy Optimization(PPO) for Discrete Action Space:
• Proximal Policy Optimization(PPO) for Continuous Action Space:
• Deep Deternimistic Policy Gradient(DDPG):
• Twin Delayed Deep Deterministic Policy Gradient(TD3):
• Soft Actor Critic(SAC) for Discrete Action Space:
• Soft Actor Critic(SAC) for Continuous Action Space:
• Actor-Sharer-Learner(ASL):

Recommended Resources for DRL

Books：

• 《Reinforcement learning: An introduction》--Richard S. Sutton
• 《深度学习入门：基于Python的理论与实现》--斋藤康毅

Online Courses:

• RL Courses(bilibili)--李宏毅(Hongyi Li)
• RL Courses(Youtube)--李宏毅(Hongyi Li)
• UCL Course on RL--David Silver
• 动手强化学习--上海交通大学

Blogs:

• OpenAI Spinning Up
• Policy Gradient Theorem --Cangxi
• Policy Gradient Algorithms --Lilian
• Theorem of PPO
• The 37 Implementation Details of Proximal Policy Optimization
• Prioritized Experience Replay
• Soft Actor Critic
• A (Long) Peek into Reinforcement Learning --Lilian
• Introduction to TD3

Simulation Environments:

• gym
• gymnasium
• Sparrow
• Envpool
• ROS
• Webots

Important Papers

DQN: Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning[J]. nature, 2015, 518(7540): 529-533.

Double DQN: Van Hasselt H, Guez A, Silver D. Deep reinforcement learning with double q-learning[C]//Proceedings of the AAAI conference on artificial intelligence. 2016, 30(1).

PER: Schaul T, Quan J, Antonoglou I, et al. Prioritized experience replay[J]. arXiv preprint arXiv:1511.05952, 2015.

PPO: Schulman J, Wolski F, Dhariwal P, et al. Proximal policy optimization algorithms[J]. arXiv preprint arXiv:1707.06347, 2017.

DDPG: Lillicrap T P, Hunt J J, Pritzel A, et al. Continuous control with deep reinforcement learning[J]. arXiv preprint arXiv:1509.02971, 2015.

TD3: Fujimoto S, Hoof H, Meger D. Addressing function approximation error in actor-critic methods[C]//International conference on machine learning. PMLR, 2018: 1587-1596.

SAC: Haarnoja T, Zhou A, Abbeel P, et al. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor[C]//International conference on machine learning. PMLR, 2018: 1861-1870.

ASL: Train a Real-world Local Path Planner in One Hour via Partially Decoupled Reinforcement Learning and Vectorized Diversity

Training Curves of my Code:

Q-learning:

DQN/DDQN on Classic Control:

DQN/DDQN on Atari Game:

Pong	Enduro

Prioritized DQN/DDQN on Classic Control:

CartPole	LunarLander

PPO Discrete:

PPO Continuous:

DDPG:

Pendulum	LunarLanderContinuous

TD3:

SAC Continuous:

SAC Discrete:

Actor-Sharer-Learner (ASL):