[QUESTION] Logging metrics at every step of training

[NicholasCL1994]

Hello, @takuseno! First I would like to congratulate you on this package!

By default, it saves the last step of every epoch in the csv file of a given metric, but I would like to keep record of the metrics every step of training while using .fit() (I am doing offline RL). I have being through the docs extensively but could not find a way to achieve what I am looking for.

Best,

[Takuma Seno]

@NicholasCL1994 Thanks for the issue and sorry for the late response. Currently, it's not directly supported. However, you can create a custom logger to achieve your goal.

References are here:

• https://d3rlpy.readthedocs.io/en/v2.3.0/references/logging.html
• https://github.com/takuseno/d3rlpy/blob/master/d3rlpy/logging/file_adapter.py

[NicholasCL1994]

Thanks for the reply! I have tried to use the custom logger but in fitter(), the function commit(), which as I understood correctly saves the metrics into the csv file, is only called in the outside loop (for the epochs) and not inside the training loop. So I don't see how a custom logger would allow me to save the metrics at every training step. Sorry if I missed something.

[Takuma Seno]

Sorry for being late to respond. Recently, logging_strategy option has been added to fit method. If you do as follows, you can save metrics every steps:

import d3rlpy
from d3rlpy.constants import LoggingStrategy

dqn = d3rlpy.algos.DQNConfig.create()
dqn.fit(..., logging_strategy=LoggingStrategy.STEPS, logging_steps=1)

Please make sure that you install v2.4.0 package or latest source code.

[NicholasCL1994]

Thanks! I really appreciate all the help, @takuseno !

[Takuma Seno]

Feel free to reopen this issue if there is any further discussion.