chrismattmann / DCGAN-Dog-Generator

DCGAN-Dog-Generator

DCGAN model (and tutorial) for generating fake images of dogs based on the Generative Dog Images Kaggle Competition (2019).

Google Colab version

To run this example locally or using Colab, you will need a Kaggle account, in order to retrieve its API key and use the provided datasets. Full tutorial is available here.

Nbviewer link of the Kaggle notebook (better readability)

Visualizing the generation over epochs (~9 hrs of training)

                                   

Model architecture

My DCGAN model uses the standard DCGAN generator and discriminator structure from the official paper(with some modifications) shown below:

Generator

Discriminator

Model improvements for this task include:

• Weight initialization (Truncated normal distribution)
• Spectral Normalization for each Conv layer in the discriminator
• 2 Dropout layers after the first 2 TP_Conv layers in the generator
• Label smoothing
• Instance noise
• Cosine learning rate decay for Adam
• Data Augmentation (horizontal flips)

Final scores:

Evaluation was based on the MIFID(Memorization Informed Frechet Inception Distance) metric, where the lower the distance score is, the better the quality of the images will be. During the competition, submissions were evaluated on a public test set of dog images, while the final scoring was based on a private dataset.

• First submission - Public - 59.05, Private - 122.55
• Second submission - Public - 55.87, Private - 125.34

My References

• The official tutorial kernel
• My best submission for this competition)
• Blog post (https://medium.com/@dragonflareful/dcgans-generating-dog-images-with-tensorflow-and-keras-fb51a1071432)

Resources

[1]. Generative Adversarial Networks official paper

[2]. Understanding Generative Adversarial Networks (GANs)

[3]. A Gentle Introduction to Generative Adversarial Networks (GANs)

[4]. DCGAN official paper

[5]. GAN — DCGAN (Deep convolutional generative adversarial networks)

[6]. Weight Initialization Techniques in Neural Networks

[7]. Spectral Normalization for Generative Adversarial Networks paper

[8]. Spectral Normalization implemented in Keras

[9]. Spectral Normalization Explained

[10]. GAN — Ways to improve GAN performance

[11]. How to Implement GAN Hacks in Keras to Train Stable Models

[12]. Tricks of GANS

[13]. Instance Noise: A trick for stabilising GAN training

[14]. GAN — RSGAN & RaGAN (A new generation of cost function.)

[15]. A simple explanation of the Inception Score

[16]. GAN — How to measure GAN performance?

[17]. All you need is GAN Hacks

[18]. How to train your touchy GANs - Things that seem to work.

[19]. Explaining the metric FID