crossphoton / DF-GAN

Deep Fusion Generative Adversarial Networks for Text-to-Image Synthesis

DF-GAN: Deep Fusion Generative Adversarial Networks for Text-to-Image Synthesis

(A novel and effective one-stage Text-to-Image Backbone)

Official Pytorch implementation for our paper DF-GAN: Deep Fusion Generative Adversarial Networks for Text-to-Image Synthesis by Ming Tao, Hao Tang, Songsong Wu, Nicu Sebe, Xiaoyuan Jing, Fei Wu, Bingkun Bao.

---

Requirements

• python 3.6+
• Pytorch 1.0+
• easydict
• nltk
• scikit-image
• A titan xp (set nf=32 in *.yaml) or a V100 32GB (set nf=64 in *.yaml)

Installation

Clone this repo.

git clone https://github.com/tobran/DF-GAN
cd DF-GAN/code/

Datasets Preparation

1. Download the preprocessed metadata for birds coco and save them to data/
2. Download the birds image data. Extract them to data/birds/
3. Download coco dataset and extract the images to data/coco/

Pre-trained text encoder

1. Download the pre-trained text encoder for CUB and save it to DAMSMencoders/bird/inception/
2. Download the pre-trained text encoder for coco and save it to DAMSMencoders/coco/inception/

---

Training

Train DF-GAN models:

• For bird dataset: python main.py --cfg cfg/bird.yml

• For coco dataset: python main.py --cfg cfg/coco.yml

• *.yml files are example configuration files for training/evaluation our models.

Evaluating

Dwonload Pretrained Model

• DF-GAN for bird. Download and save it to models/bird/
• DF-GAN for coco. Download and save it to models/coco/

Evaluate DF-GAN models:

• To evaluate our DF-GAN on CUB, change B_VALIDATION to True in the bird.yml. and then run python main.py --cfg cfg/bird.yml
• To evaluate our DF-GAN on coco, change B_VALIDATION to True in the coco.yml. and then run python main.py --cfg cfg/coco.yml
• We compute inception score for models trained on birds using StackGAN-inception-model.
• We compute FID for CUB and coco using DM-GAN/eval/FID.

---

Citing DF-GAN

If you find DF-GAN useful in your research, please consider citing our paper:

@article{ming2020DFGAN,
  title={DF-GAN: Deep Fusion Generative Adversarial Networks for Text-to-Image Synthesis},
  author={Ming Tao, Hao Tang, Songsong Wu, Nicu Sebe, Fei Wu, Xiao-Yuan Jing, Bingkun Bao},
  journal={arXiv preprint arXiv:2008.05865},
  year={2020}
}

The code is released for academic research use only. For commercial use, please contact Ming Tao.

Reference

• StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks [code]
• AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks [code]
• DM-GAN: Realistic Image Synthesis with Stacked Generative Adversarial Networks [code]