Performance of DQN agent on "small-linear" scenario

[chgeneve]

Hi,

I have run multiple training with the DQN on the "small-linear" scenario with training_steps=5000000 and fully_obs=False. The network doesn't seem to compromise the two targets on this scenario (and doesn't seem to converge in terms of episodic return). Have you ever managed to get results that show that the DQN converges and can get positive episodic return on this scenario (with the above mentioned parameters) ?

Best regards,

[Jonathon Schwartz]

Hey,

I haven't tried it with fully_obs=False, but it's not surprising that the DQN implementation that comes with the library isn't able to solve it, since it's very basic. To solve the non fully observable setting you need an agent that can handle partially observable environments, e.g. agents that use recurrent networks.

There are a lot of resources and existing implementations of such algorithms, the following are some examples I found after a brief search, I'm sure there are many more:

• https://sb3-contrib.readthedocs.io/en/master/modules/ppo_recurrent.html
• https://github.com/mynkpl1998/Recurrent-Deep-Q-Learning

Best of luck

[chgeneve]

Thanks for you answer! I will try with recurrent networks.