fyenne / reinforcement-learning-an-introduction

Solutions to exercises in Reinforcement Learning: An Introduction (2nd Edition).

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Reinforcement Learning: An Introduction

by Richard Sutton & Andrew Barto (2nd edition)

Solutions to Exercises and Programming Problems

This repository contains my answers to exercises and programming problems from the Reinforcement Learning: An Introduction. I'm not sure if it's a good idea to make the solutions public because authors' intention is clearly the opposite. Despite initially not liking it, I have to admit it's a brilliant decision. Not having the answers easily available makes one think of creative ways to check whether the idea he just came up with is correct. The sanity checking process frequently reveals much more about the topic than working out the answer.

On the other hand, Googling for the answer helped me to get unstuck in many cases. So it would be selfish of me not to return the favor to others. The answers tend to be are scattered all over the internet and it seems like there's no single place to look them up. Besides, formulation of many of the programming problems is very deceiving. Initially it looks like an innocent, hour long, fun, coding challenge. But more often than not it turns into a weekend long wrestling with the pig in the mud :)

If you see a wrong answer, please, let me know by opening an issue. All the programming problems use OpenAI gym as the API between the agent and the environment. So checking the documentation may help to get a faster start. The code is written in Python 3 and uses numpy library for vectorized linear algebra operations.


Chapter 1 - Introduction

Part I - Tabular Solution Methods

Chapter 2 - Multi-armed Bandits

Chapter 3 - Finite Markov Decision Processes

Chapter 4 - Dynamic Programming

Chapter 5 - Monte Carlo Methods

Chapter 6 - Temporal-Difference Learning

Chapter 7 - n-step Bootstrapping

Chapter 8 - Planning and Learning with Tabular Methods

Part II - Approximate Methods

Chapter 9 - On-policy Prediction with Approximation

Chapter 10 - On-policy Control with Approximation

Chapter 11 - Off-policy Methods with Approximation

About

Solutions to exercises in Reinforcement Learning: An Introduction (2nd Edition).


Languages

Language:Jupyter Notebook 96.3%Language:Python 3.7%