How many step does it take to converge?

[Yu Zhang]

Hi, I have trained the policy for over 50M steps, but it seems not converge yet.

I wonder how many steps in total it need to finish trianing?

(BTW, I use 16 way parallel training)

Thanks.

[erwincoumans]

Somewhere around 100 million samples and for some of the more complex motions even more (150-200 million). Note that quality of the policies is not as good as our internal version we used for the paper, it may require a bit more hyper-parameter tuning.

[Yu Zhang]

Thanks for your reply.

I've noticed that the trained lakaigo can already walk smoothly while the vf loss constantly increases. Does it make any sense? or it means I did something wrong?

[Akhtyamov Timur]

Dear @erwincoumans,

Can you please clarify the parameters of your computational setup that you used to training in original paper? And how much time did training take for one motion skill?

[erwincoumans]

Computation was in the order of day(s) typically. The computational setup are a cluster of cloud machines running Linux, similar to Google Cloud.