questions about the Dueling logic in network.py
liurida opened this issue · comments
liurida commented
Hi
It's really a good code for learning Reinforcement Learning.
In the network.py, I have 2 questions.
- I think you want to assert len(value_hidden_sizes) != 0 and len(advantage_hidden_sizes) != 0.
- About the Dueling part, the logic in code is layer contains value_hidden_sizes linear, then the layer is delivered to the next advantage logic. But I read the related paper, if I understand correct, it describes that the state-value and advantage are generated from the same source observation, then they're added together, and minus the mean advantage value.
Looking forward to your further response.
Taehoon Kim commented
@liurida Thanks! You're totally right. That's a huge bug and I just fixed and commit the new code.