How many environment steps per update?
mctigger opened this issue · comments
Hi danijar,
how many environment steps are you running per update?
In the paper it is 4 (so after every step the agent makes it is updated because of action repeat?), but here in the config it says train_every: 16. What is the correct number?
Best,
Tim
Hi Tim, those are both correct. It's 16 env steps or 4 agent steps. The action repeat introduces the factor of 4 between the two.
Thank you for answering so quickly!
In the paper in the hyperparameters section it says Environment steps per update: 4. So in the paper it should actually be Agent Steps per update: 4 or Environment steps per update: 16? Just want to make sure I understand you correctly.
Ah, yes. It's every 16 frames or 4 actions. I'll update the paper to make it clearer.