windcr / CVPR18-SFTGAN

Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform (CVPR 2018) http://mmlab.ie.cuhk.edu.hk/projects/SFTGAN/

CVPR18-SFTGAN

[PyTorch(Under Construction)] [project page] [paper]

Torch implementation for Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform

Training code is coming soon...

Table of Contents

1. Introduction
2. Requirements and Dependencies
3. Test
4. Citation

Introduction

We have explored the use of semantic segmentation maps as categorical prior for SR.

A Spatial Feature Transform (SFT) layer has been proposed to efficiently incorporate the categorical conditions into a CNN network.

For more details, please check out our project webpage and paper.

Requirements and Dependencies

• Torch
• cuda & cudnn
• other torch dependencies, e.g. nngraph / paths / image (install them by luarocks install xxx)

Test

We test our model with Titan X/XP GPU.

1. Download segmentation model (OutdoorSceneSeg_bic_iter_30000.t7) and SFT-GAN model (SFT-GAN.t7) from google drive. Put them in the model folder.
2. There are 2 sample images in data/samples folder. You can put your images inside this folder.
3. Run th test_seg.t7 The segmentation results are then in data/ with _segprob/_colorimg/_byteimg suffix.
4. Run th test_SFT-GAN.lua The results are then in data/ with prefix rlt_.

Citation

If you find the code and datasets useful in your research, please cite:

@inproceedings{wang2018sftgan,
    author = {Xintao Wang, Ke Yu, Chao Dong and Chen Change Loy},
    title = {Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform},
    booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year = {2018}