YvanYin/VNL_Monocular_Depth_Prediction

Enforcing geometric constraints of virtual normal for depth prediction.

[NEW] Training codes have been uploaded!

This repository contains the source code of our paper: Yin Wei, Yifan Liu, Chunhua Shen, Youliang Yan, Enforcing geometric constraints of virtual normal for depth prediction (accepted for publication in ICCV' 2019).

Some Results

Framework

Hightlights

State-of-the-art performance: The comparisons with other SOTA methods on NYU and KITTI are illustrated as follow. The published model trained on NYU can achieve 10.5% on absrel.

Installation

Please refer to Installation.

Datasets

NYUDV2 The details of datasets can be found here. The Eigen split of labeled images can be downloaded here. Please extract it to ./datasets. Our SOTA model is trained on the around 20K unlabled images.
KITTI The details of KITTI benchmark for monocular depth prediction is here. We use both the official split and Eigen split. You can find the filenames here.

Model Zoo

ResNext101_32x4d backbone, trained on NYU dataset, download here
ResNext101_32x4d backbone, trained on KITTI dataset (Eigen split), download here
ResNext101_32x4d backbone, trained on KITTI dataset (Official split), download here
ImageNet pretrained weight, download here.

Inference

# Run the inferece on NYUDV2 dataset
 python  ./tools/test_nyu_metric.py \
		--dataroot    ./datasets/NYUDV2 \
		--dataset     nyudv2 \
		--cfg_file     lib/configs/resnext101_32x4d_nyudv2_class \
		--load_ckpt   ./nyu_rawdata.pth 
		
# Test depth predictions on any images, please replace the data dir in test_any_images.py
 python  ./tools/test_any_images.py \
		--dataroot    ./ \
		--dataset     any \
		--cfg_file     lib/configs/resnext101_32x4d_nyudv2_class \
		--load_ckpt   ./nyu_rawdata.pth

If you want to test the kitti dataset, please see here

Training

# Run the inferece on NYUDV2 dataset
 python  ./tools/train_nyu_metric.py \
		--dataroot    ./datasets/NYUDV2 \
		--dataset     nyudv2 \
		--cfg_file     lib/configs/resnext101_32x4d_nyudv2_class 
                #--load_ckpt *.pth
                #--resume

Citation

@inproceedings{Yin2019enforcing,
  title={Enforcing geometric constraints of virtual normal for depth prediction},
  author={Yin, Wei and Liu, Yifan and Shen, Chunhua and Yan, Youliang},
  booktitle= {The IEEE International Conference on Computer Vision (ICCV)},
  year={2019}
}

@article{yin2021virtual,
  title={Virtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Prediction},
  author={Yin, Wei and Liu, Yifan and Shen, Chunhua},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
  year={2021}
}

Other works :)

We have done a new work to aim to solve the generalization problem of monocular depth prediction. Please refer to following link. :)

Paper: DiverseDepth: Affine-invariant Depth Prediction Using Diverse Data
Code: https://github.com/YvanYin/DiverseDepth

Contact

Wei Yin: wei.yin@adelaide.edu.au

YvanYin / VNL_Monocular_Depth_Prediction