SFTGAN [project page] [paper]

😜 PyTorch - BasicSR

The pytorch version of SFTGAN, including training and testing codes, is in PyTorch - BasicSR.

BasicSR contains basic codes for Super-Resolution. It has also provided basic SR models like SRResNet, SRGAN and etc.

This repo is the original torch implementation (test code) for Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform.

Feature Modulation

Spatial feature modulation is motivated by Conditional Batch Normalization (e.g., image style transfer [1, 2, 3] and visual reasoning [1, 2) ] and also feature modulation [FiLM].

Introduction

We have explored the use of semantic segmentation maps as categorical prior for SR.

A Spatial Feature Transform (SFT) layer has been proposed to efficiently incorporate the categorical conditions into a CNN network.

Requirements and Dependencies

Torch
cuda & cudnn
other torch dependencies, e.g. nngraph / paths / image (install them by luarocks install xxx)

Test

We test our model with Titan X/XP GPU.

Note that our model is limited to some outdoor scenes. It is an unsatisfying limitation that we need to relax in future.

Download segmentation model (OutdoorSceneSeg_bic_iter_30000.t7) and SFT-GAN model (SFT-GAN.t7) from google drive. Put them in the model folder.
There are 2 sample images in data/samples folder. You can put your images inside this folder.
Run th test_seg.lua The segmentation results are then in data/ with _segprob/_colorimg/_byteimg suffix.
Run th test_SFT-GAN.lua The results are then in data/ with prefix rlt_.

Citation

If you find the code and datasets useful in your research, please cite:

@inproceedings{wang2018sftgan,
    author = {Wang, Xintao and Yu, Ke and Dong, Chao and Loy, Chen Change},
    title = {Recovering realistic texture in image super-resolution by deep spatial feature transform},
    booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year = {2018}

😆 Image Viewer

If you have trouble in comparing image details, may have a try for HandyViewer - an image viewer that you can switch image with a fixed zoom ratio.

zijundeng / CVPR18-SFTGAN