`LogWrapper` should indicate missing data until first episode terminates
EmptyJackson opened this issue · comments
Matthew Jackson commented
In the LogWrapper
initialization, both returned_episode_returns
and returned_episode_lengths
are set to 0.
Since 0 is a valid return for many environments, is it impossible (in isolation) to tell apart an initial return of 0 and a non-terminated first episode.
In order to indicate missing data before the first episode terminates, these should be set to NaN
(or None
).