Simple Q-Learning Implementation
q_learning.py is the script accompanying the talk "Reinforcement Learning, An Introduction", Dr. Sven Mika Duesseldorf Germany Aug 20th 2017
1) Run the code:
python q_learning.py (python3 only)
2) Alter the parameters of the algo:
Play around with the epsilon and alpha parameters to lower the number of necessary iterations to a minimum. Also, if you start with large values (close to 1.0) for both and slowly reduce both values of time (by multiplying with a factor < 1.0 each iteration), you will get better results (faster convergence). In the end, your table should give you the expected accumulated future rewards for each state/action pair.
3) Expected Output:
For epsilon=alpha=0.1 (no reduction of these over time) and 5000 iterations, you should get something like:
[s] /[a] | [q-value]
-----------------------
A/N | 3.0
B/W | 5.0
D/None | 0.0
A/W | 3.0
A/SWIM | 3.4
LAKE/SWIM | 5.0
C/N | 5.0