MSG-Net (Multi-scale guidance network)

Multi-scale guidance network

This repository (https://github.com/twhui/MSG-Net) is the offical release of MSG-Net for our paper Depth Map Super-Resolution by Deep Multi-Scale Guidance in ECCV16. It comes with four trained networks (x2, x4, x8, and x16), one hole-filled RGBD training set, and three hole-filled RGBD testing sets (A, B, and C).

To the best of our knowledge, MSG-Net is the FIRST convolution neural networkwhich attempts to upsample depth images under multi-scale guidance from the corresponding high-resolution RGB images.

Another repository for MS-Net (without multi-scale guidance) is also available.

For more details, please visit my project page.

License and Citation

All code and other materials (including but not limited to the paper, figures, and tables) are provided for research purposes only and without any warranty. Any commercial use requires our consent. When using any parts of the code package or the paper (Depth Map Super-Resolution by Deep Multi-Scale Guidance) in your work, please cite the following paper:

@InProceedings{hui16msgnet,    
 author = {Tak-Wai Hui and Chen Change Loy and and Xiaoou Tang},    
 title  = {Depth Map Super-Resolution by Deep Multi-Scale Guidance},    
 booktitle  = {Proceedings of European Conference on Computer Vision (ECCV)},    
 pages = {353--369},  
 year = {2016},    
 url = {http://mmlab.ie.cuhk.edu.hk/projects/guidance_SR_depth.html}}

Dependency

We train our models using caffe and evaluate the results on Matlab.

Installation and Running

You need to install caffe and remeber to complie matcaffe. You can put the folder MSGNet-release in caffe/examples. Finally, you need to get into the the directory of examples/MSGNet-release/util, and run MSGNet.m.

Training data

Our RGBD training set consists of 58 RGBD images from MPI Sintel depth dataset, and 34 RGBD images from Middlebury dataset. 82 images are used for training and 10 images (frames 1, 20, 28, 58, 64, 66, 69, 73, 75 and 79) are used for validation.

Training sub-images/patches (hdf5 format)

An example for generating training patches of the scale factor 2 can be found here util/gen_train_data_x2.m.

Testing data

Testig set is available at the folder MSGNet-release/testing sets.

kushalvyas / MSG-Net