Improved WGAN memory leak and training issue

Question

Improved WGAN memory leak and training issue

vishwakftw opened this issue 7 years ago · comments

Vishwak Srinivasan commented 7 years ago

There seems to be a memory leak in the improved WGAN code. This is due to the BatchNorm2d layers in the Discriminator and Generator of the DCGAN, which is used for the improved WGAN and the torch.autograd.grad function. This is also referenced in this issue: PyTorch Double Backward on BatchNorm2d.

The patch is ready, but we might have to wait until the next release.

There seems to be problem with training without the BatchNorm2d layers. This needs to be tracked and fixed.

Vishwak Srinivasan · Answer 1 · Sun Aug 27 2017 23:34:53 GMT+0800 (China Standard Time)

The problem now is that the training is extremely unstable. The D-loss and G-loss are decreasing constantly and don't seem to flatten out/saturate.