Problem with AverageReturnMetric
billh0420 opened this issue · comments
It seems to be necessary to increase the buffer_size for AverageReturnMetric to get better precision.
One would think that increasing buffer_size would only speed up the computation (at a cost of increase memory usage). But this is not the case. Using the default buffer size of 10 yields only an accuracy to one decimal place. I notices this problem when trying to compute the AverageReturnMetric for 100 episodes: the results were always like .50... or .70... etc and never .87000....
If I provide a buffer size of 200, then I get satisfactory results.