Jax vs normal agent

Question

Jax vs normal agent

satst27 opened this issue 4 years ago · comments

Hi,

I think it will helpful to clarify that the checkpoints that you have provided are for the Jax verion and not the normal version:

"The TensorFlow checkpoint files for 5 independent runs of the 4 agents on all 60 games are available. The format for each of the files is: https://storage.cloud.google.com/download-dopamine-rl/lucid/${AGENT}/${GAME}/${RUN}/tf_ckpt-199.${SUFFIX}, where:

AGENT can be "dqn", "c51", "rainbow", or "iqn"."

Are there checkpoints available for all games for non-Jax version as well?

thanks,
satst27

Pablo Samuel Castro · Answer 1 · Wed Oct 07 2020 21:36:41 GMT+0800 (China Standard Time)

hi, the checkpoints are for the tensorflow version, not for the jax version. we don't have checkpoints for all games and all agents on JAX, but i'm hoping to be able to release the quantile checkpoints on JAX sometime in the near future (since quantile is only available on JAX).

…

On Wed, Oct 7, 2020 at 5:20 AM satst27 ***@***.***> wrote: Hi, I think it will helpful to clarify that the checkpoints that you have provided are for the Jax verion and not the normal version: "The TensorFlow checkpoint files for 5 independent runs of the 4 agents on all 60 games are available. The format for each of the files is: https://storage.cloud.google.com/download-dopamine-rl/lucid/${AGENT}/${GAME}/${RUN}/tf_ckpt-199.${SUFFIX}, where: AGENT can be "dqn", "c51", "rainbow", or "iqn"." Are there checkpoints available for all games for non-Jax version as well? thanks, satst27 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#153>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE3CCMM2473JHHVYL6LTOZDSJQXHDANCNFSM4SHED4DQ> .

Tor Gunnar Høst Houeland · Answer 2 · Wed Sep 21 2022 03:12:37 GMT+0800 (China Standard Time)

Bump, is this actually completed now? I see that e.g. https://console.cloud.google.com/storage/browser/_details/download-dopamine-rl/jax/quantile/Pong/1/ckpt.199 exists now.

(I currently only see 6 games there, but at least it's not 0. download-dopamine-rl/jax/dqn_adam_mse looks like it actually has checkpoints for all the trained games(?), while download-dopamine-rl/jax/iqn has 0.)

Pablo Samuel Castro · Answer 3 · Fri Sep 23 2022 22:24:55 GMT+0800 (China Standard Time)

sorry for delay, will get to this after the iclr deadline! the dqn_adam_mse folder does have for all games, but as the directory suggests it is with Adam optimizer and MSE loss (as we explored in Revisiting Rainbow: http://proceedings.mlr.press/v139/ceron21a.html) we haven't released checkpoints externally for all games for the jax agents. after the iclr deadline i'll see about releasing all of them.

…

On Tue, Sep 20, 2022 at 3:12 PM Tor Gunnar Houeland < ***@***.***> wrote: Bump, is this actually completed? I see that e.g. https://console.cloud.google.com/storage/browser/_details/download-dopamine-rl/jax/quantile/Pong/1/ckpt.199 exists now. (I currently only see 6 games there, but at least it's not 0. download-dopamine-rl/jax/dqn_adam_mse looks like it actually has checkpoints for all the trained games(?), while download-dopamine-rl/jax/iqn has 0.) — Reply to this email directly, view it on GitHub <#153 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE3CCMIUNQ5JNYQ7ZHKQCPDV7IEDBANCNFSM4SHED4DQ> . You are receiving this because you commented.Message ID: ***@***.***>