lilianweng / unified-gan-tensorflow

A Tensorflow implementation of GAN, WGAN and WGAN with gradient penalty.

Original, Wasserstein, and Wasserstein-Gradient-Penalty DCGAN

(*) This repo is a modification of carpedm20/DCGAN-tensorflow.

(*) The full credit of the model structure design goes to carpedm20/DCGAN-tensorflow.

I started with carpedm20/DCGAN-tensorflow because its DCGAN implementation is not fixed for one dataset, which is not a common setting. Most WGAN and WGAN-GP implementations only work on 'mnist' or one given dataset.

Modifications

A couple of modifications I've made that could be helpful to people who try to implement GAN on their own for the first time.

1. Added model_type which could be one of 'GAN' (original), 'WGAN' (Wasserstein distance as loss), and 'WGAN_GP' (Wasserstein distance as loss function with gradient penalty), each corresponding to one variation of GAN model.
2. UnifiedDCGAN can build and train the graph differently according to model_type.
3. Some model methods were reconstructed so that the code is easier to read through.
4. Many comments were added for important, or potential confusing functions, like conv and deconv operations in ops.py.

The download.py file stays same as in carpedm20/DCGAN-tensorflow. I keep this file in the repo for the sake of easily fetching dataset for testing.

Reading

If you are interested in the math behind the loss functions of GAN and WGAN, read here.

Related Papers

• Goodfellow, Ian, et al. "Generative adversarial nets." NIPS, 2014.
• Martin Arjovsky, Soumith Chintala, and Léon Bottou. "Wasserstein GAN." arXiv preprint arXiv:1701.07875 (2017).
• Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, Aaron Courville. Improved training of wasserstein gans. arXiv preprint arXiv:1704.00028 (2017).

Test Runs:

(left) python main.py --dataset=mnist --model_type=GAN --batch_size=64 --input_height=28 --output_height=28 --max_iter=10000 --learning_rate=0.0002 --train
(middle) python main.py --dataset=mnist --model_type=WGAN --batch_size=64 --input_height=28 --output_height=28 --d_iter=5 --max_iter=10000 --learning_rate=0.00005 --train
(right) python main.py --dataset=mnist --model_type=WGAN_GP --batch_size=64 --input_height=28 --output_height=28 --d_iter=5 --max_iter=10000 --learning_rate=0.0001 --train