NAF generative models - mode collapse
pfrendl opened this issue · comments
Hi!
I have tried to create a NAF-based generative model outputting samples from the gaussian grid in your paper. As far as I understand, you use reverse-direction KL divergence as the loss in this case. This results in my generator failing to capure all the modes of the 5x5 grid, though. Did I misunderstand something? Do you have a solution for this problem?
Density estimation works great on this same distribution.
Edit:
In the sinewave experiment, you scale the energy function up in the loss gradually. Also, the unit-weight initialization in the forst weight vector of the DSFs could help, I guess. Is there something else that could be done?