Add a score method to an agent
DJCordhose opened this issue · comments
like 'play' should play a number of episodes and output the total rewards for them. Maybe also give min, max, mean, and std.
We already support:
Would that be similar, but having an additional min/max area ?
I was rather thinking of something like the score method in sklearn or evaluate in Keras, which do not plot, but give a list of values based on what metrics you want
could you make an example of what you would pass into agent.score and how the output would look like ?
min, max, mean, all = agent.score(num_episodes=1000, metrics=['min', 'max', 'mean', 'list'])
min => 0.2
max => 0.8
mean => 0.7
all => [0.2, ....., 0.7]
ah I see. You could try this one (already supported):
import statistics
pc = agent.play(num_episodes=1000)
r = list(pc.sum_of_rewards.values())
min(r), max(r), statistics.mean(r), statistics.stdev(r), r
what do you think ?
what about a score() function on play_context yielding all available scores as a dictioinary?
agent.play(num_episodes=1000).score()
=> {'min':0.2, 'max': 0.8, 'mean':0.7,'std':0.1,'all'=[...]}
good point. let's do that.
I would be happy to take care of this, but it will take some time until I get to it.
thats great. no problem.
implemented directly into main branch, as minimal implementation in abstract base class 'EasyAgent'