ShirleyYim / caffe-vdsr

A Caffe-based implementation of very deep convolution network for image super-resolution

Caffe_VDSR

This is an implementation of "Accurate Image Super-Resolution Using Very Deep Convolutional Networks" (CVPR 2016 Oral Paper) in caffe.

Update

Training Samples

I trained my "VDSR_170000" model by about 300000 training samples (1638 images augumented from 91 images). So, please do data augumentation if you want to get good performances.

Multi Scale Training

I trained a new model by multi scale data augumentation. But there are not obvious improvemets between multi scale model and my single scale model. Replies welcome, if anyone has experience on multi scale training.

Instruction

VDSR (Very Deep network for Super-Resolution) is an end-to-end network with 20 convolutional layers for single image super-resolution. The performance of VDSR is better than other state-of-the-art SISR methods, such as SRCNN, A+ and CSCN (My implementation of CSCN).

Dependencies

Train

• Caffe

Test

• MatConvNet

Usage

Train

1. Place the "Train" folder into "($Caffe_Dir)/examples/", and rename "Train" to "VDSR"

2. Open MATLAB and direct to ($Caffe_Dir)/example/VDSR, run "generate_train.m" and "generate_test.m" to generate training and test data (Code from SRCNN)

3. To train VDSR, run ./build/tools/caffe train --solver examples/VDSR/VDSR_solver.prototxt

4. Set clip_gradients in VDSR_solver.prototxt to solve gradient explosion problem, 0.1 or 1 is a good choice

5. Change the learning rate when the error plateaus

6. After training, run "caffemodel2mat.m" to convert caffemodel to mat for testing (matcaffe is required)

Test

1. "Demo_SR_Conv.m" is a simple test code. Just run it and you will get the result

2. "VDSR_170000.mat" is a model trained by myself

3. "VDSR_Official.mat" is an official model converted from Official Test Code

Different from original paper

Because of the limitation of hardware conditions, I didn't do complete training. So there are some differences between this implementation ("VDSR_170000.mat") and original paper ("VDSR_Official.mat").

Training Dataset

This implementation: 91 images (with data augumentation and only factor 2)

Original paper: 291 images (with data augumentation and factor 2, 3 and 4)

Multi Scale

This implementation: Casade of 2x to generate 3x and 4x result

Original paper: Multi scale in one model

Training Time of Final Model

This implementation: about 30 epoch

Original paper: about 80 epoch Performance in PSNR

Factor 2

DataSet	VDSR_Official	VDSR_170000
Set5	37.53	37.46
Set14	33.03	32.83
BSD100	31.90	31.65

Factor 3

DataSet	VDSR_Official	VDSR_170000
Set5	33.66	33.52
Set14	29.77	29.55
BSD100	28.82	28.62

Factor 4

DataSet	VDSR_Official	VDSR_170000
Set5	31.35	31.14
Set14	28.01	27.81
BSD100	27.29	27.13

References

Please cite [1] if you use this code in your work, thank you!

[1] Jiwon Kim, Jung Kwon Lee and Kyoung Mu Lee, "Accurate Image Super-Resolution Using Very Deep Convolutional Networks", Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016