LyWangPX / Reinforcement-Learning-2nd-Edition-by-Sutton-Exercise-Solutions

Solutions of Reinforcement Learning, An Introduction

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

ex 3.18

burmecia opened this issue · comments

commented

Should we add an condition to the expectation?
image

What about a~\pi under E? Since s is given in the left, and small s is always a particular state.
But q itself is dependen on \pi, that's why I think expectation over pi is enough.

commented

I was following the 2nd equation notation from formula 3.14, although I think your thinking is probably right.

OK.
I will leave the annation unchanged for now.