Perceptual features (PFs) have been used with great success in tasks such as transfer learning, style transfer, and super-resolution. However, the efficacy of PFs as key source of information for learning generative models is not well studied. We investigate here the use of PFs in the context of learning implicit generative models through moment matching (MM). More specifically, we propose a new effective MM approach that learns implicit generative models by performing mean and covariance matching of features extracted from pretrained ConvNets. Our proposed approach improves upon existing MM methods by: (1) breaking away from the problematic min/max game of adversarial learning; (2) avoiding online learning of kernel functions; and (3) being efficient with respect to both number of used moments and required minibatch size. Our experimental results demonstrate that, due to the expressiveness of PFs from pretrained deep ConvNets, our method achieves state-of-the-art results for challenging benchmarks.
- python 2.7
- pytorch 0.3.0
Please install requirements by pip install -r requirements.txt
- Feature Extractor :
VGG19andResnet18 - Generator Architecture :
Resnet
python gfmn.py --netGType resnet --netEncType vgg19 resnet18 --dataset cifar10 \
--netEnc [path-to-pretrained-vgg19-model] [path-to-pretrained-resnet18-model][path-to-pretrained-X-model] : Path to pretrained VGG19/Resnet18 classifiers. Refer downloads section.
- Feature Extractor :
VGG19andResnet18 - Generator Architecture :
Resnet
python gfmn.py --netGType resnet --netEncType vgg19 resnet18 --dataset stl10 \
--netEnc [path-to-pretrained-vgg19-model] [path-to-pretrained-resnet18-model][path-to-pretrained-X-model] : Path to pretrained VGG19/Resnet18 classifiers. Refer downloads section.
You can download pre-trained VGG19/Resnet18 classifiers from this link. These are the feature extractors we used in the above scripts to replicate the results, with --netEnc option.
@InProceedings{Santos_2019_ICCV,
author = {dos Santos, Cicero Nogueira and Mroueh, Youssef and Padhi, Inkit and Dognin, Pierre},
title = {Learning Implicit Generative Models by Matching Perceptual Features},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month = {October},
year = {2019}}