What should I do if I want to train a DDPG model?

[Yurong You]

Hi, I am currently doing a project about trying to use DDPG to train the same agent in this paper. It seems that I need to change the network architecture and the way to update the network a little bit. I have tried diving into the source code for two days, but there are just too many classes and I still have not found what I should modify. Could you give me some advice or explain a little bit which class is doing what in training? (e.g. cScenarioTrain, cScenarioExp, cNeuralNetLearner, etc.) Thanks a lot!

[xbpeng]

I think implementing DDPG will be pretty difficult. But a good start might be to look into cNeuralNetTrainer. Trainers are the set of classes responsible for updating the networks. ScenarioExp are the classes responsible for generating the training data, and ScenarioTrain manages a bunch of ScenarioExps to generate data in parallel to feed into the trainer. A good example to follow might be to look at ScenarioTrainCacla, which uses an actor-critic framework. But again, it will be pretty tricky to get DDPG setup.

[Yurong You]

Thank you very much! This would help a lot~