yangyi0318 / Tabular-RL-with-Python

Tabular Reinforcement Learning Algorithms with NumPy & Visualizations with Seaborn

Tabular Reinforcement Learning with Algorithms Python

Python implementation of Tabular RL Algorithms in Sutton & Barto 2017 (Reinforcement Learning: An Introduction) Using only NumPy & basic Python data structures (list, tuple, set, and dictionary) to create environment & create algorithms

Algorithms learning from 4X4 Grid World Environment (From Sutton & Barto 2017, pp. 61)

Tabular Reinforcement Learning Algorithms with NumPy

Visualizations with Seaborn (Policy & Value function)

Contents

0. MDP Environment (Chapter 3, Sutton & Barto 2017)

1. Introduction to gridworld environment

1. Dynamic Programming (Chapter 4, Sutton & Barto 2017)

1. Policy Evaluation and improvement
2. Policy Iteration
3. Value Iteration

2. Monte Carlo Methods (Chapter 5, Sutton & Barto 2017)

1. Monte Carlo Prediction
2. Monte Carlo Exploring Starts
3. On Policy Monte Carlo
4. Off Policy Monte Carlo

3. Temporal Difference Learning (Chapter 6, Sutton & Barto 2017)

1. TD Prediction
2. SARSA - On-policy Control
3. Q-learning - Off-policy Control
4. Double Q-learning - Off-policy Control

4. n-step Bootstrapping (Chapter 7, Sutton & Barto 2017)

1. n-step TD Prediction
2. n-step SARSA - On-policy Control
3. n-step Off-policy learning by Importance Sampling
4. n-step Off-policy learning without Importance Sampling