neycyanshi / DDRNet

Depth Map Denoising and Refinement for Consumer Depth Cameras Using Cascaded CNNs (ECCV 2018)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

DDRNet: Depth Map Denoising and Refinement for Consumer Depth Cameras Using Cascaded CNNs

The major contributers of this repository include Shi Yan and Lizhen Wang.


DDRNet is a cascaded Depth Denoising and Refinement Network, which achieves superior performance over the state-of-the-art techniques.

DDRNet is described in ECCV 2018 paper. It is worth noticing that:

  • DDRNet leverages the multi-frame fused geometry and the accompanying high quality color image through a joint training strategy.
  • Each sub-network focuses on lifting the quality in different frequency domains.
  • It combines supervised and unsupervised learning to solve the issue of lacking ground truth training data.

Requirements: Software

  • Python > 2.7
  • TensorFlow >= 1.3.0
  • numpy, scipy, scikit-image

Requirements: Hardware

Any NVIDIA GPUs with 5GB memory suffices. For training, we recommend 4xGPU with 12G memory.

Before Start

Clone the repository.

git clone


  1. Download dataset or prepare your own data. In the dataset folder:

    ├── 20170907
    │   ├── group1
    │   │   ├── depth_map
    │   │   ├── high_quality_depth (depth_ref)
    │   │   ├── depth_filled
    │   │   ├── color_map
    │   │   ├── mask
    │   │   └── albedo
    │   ├── group2
    │   └── ... 
    ├── 20170910
    └── ...
    • color_map and depth_map are captured and aligned raw data.
    • high_quality_depth (depth_ref, D_ref) is generated using fusion method, paired with depth_map.
    • depth_filled, mask and albedo are generated by preprocess(optional).
  2. Please prepare index files for training or testing. train.csv has at least 3 columns, pre-computed mask and albedo is optional, and can be added to the 4,5th column. test.csv is similar, and depth_ref path is not necessary.

  3. (Optional) Preproccess

    • depth_filled and mask: fill small holes in depth map and get salient mask using the following command.
    python data_utils/ dataset/20170907


  1. Please download our pre-trained model from Google Drive or Baidu Pan, and put them under folder log/cscd/noBN_L1_sd100_B16/, and we call this checkpoint directory as $CKPT_DIR. Make sure it looks like this:
  2. Edit by modifing checkpoint_dir and csv_path to yours.
  3. Run test script and save results to $RESULT_DIR directory. Denoised depth map (D_dn) dn_frame_%6d.png and refined depth map (D_dt) dt_frame_%6d.png will be saved.
    sh $RESULT_DIR/  # RESULT_DIR=sample


  1. is an example of how to train this network, please edit index_file and dataset_dir to yours.
  2. Run train script and save checkpoints to specified directory. Here $LOG_DIR is the parent directory of $EXPM_DIR. Log, network graph and model parameters (checkpoints) are saved in $EXPM_DIR. $GPU_ID is the selected gpu id. If GPU_ID=0, training is done on the first GPU.
    sh $LOG_DIR/$EXPM_DIR/ $GPU_ID  # LOG_DIR=cscd EXPM_DIR=noBN_L1_sd100_B16
    tensorboard --logdir=log/$LOG_DIR/$EXPM_DIR --port=6006  # visualizing training.

Citing DDRNet

If you find DDRNet useful in your research, please consider citing:

  author = {Shi Yan, Chenglei Wu, Lizhen Wang, Feng Xu, Liang An, Kaiwen Guo, and Yebin Liu},  
  title = {DDRNet: Depth Map Denoising and Refinement for Consumer Depth Cameras Using Cascaded CNNs},  
  booktitle = {ECCV},  
  year = {2018}  


Code has been tested under:

  • Ubuntu 16.04 with 3 Maxwell Titan X GPU and Intel i7-6900K CPU @ 3.20GHz
  • Ubuntu 14.04 with a GTX 1070 GPU and Intel Xeon CPU E3-1231 v3 @ 3.40GHz


Depth Map Denoising and Refinement for Consumer Depth Cameras Using Cascaded CNNs (ECCV 2018)

License:MIT License


Language:Python 98.8%Language:Shell 1.2%