nv-tlabs / DIB-R

Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer (NeurIPS 2019)

DIB-Render

This is the official inference code for:

Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer (NeurIPS 2019)

Wenzheng Chen, Jun Gao*, Huan Ling*, Edward J. Smith*, Jaakko Lehtinen, Alec Jacobson, Sanja Fidler

[Paper] [Project Page]

Note: key functions from this work have also been ported to Kaolin Library, where they continue to be maintained. See this example repo, as well as this tutorial. Please cite our paper regardless of the implementation you end up using for your research.

Usage

Install dependencies

This code requires PyTorch 1.1.0 and python 3+. Please install dependencies by

pip install -r requirments.txt

Compile the DIB-Render

cd dib-render/cuda_dib_render
python build.py install

Inference

python test-all.py \
 --g_model_dir ./checkpoints/g_model.pth \
 --svfolder ./prediction \
 --data_folder ./dataset \
 --filelist ./test_list.txt

To get the evaluation IOU, please first download the tool Binvox and install it's dependencies,

Voxelize the prediction using Binvox

python voxelization.py  --folder ./prediction

To evaluate the IOU, please first install binvox-rw-py following this Link, then run the script

python check_iou.py --folder ./prediction  --gt_folder ./dataset 

To get the boundary F-score, please run the following script

python check_chamfer.py --folder ./prediction  --gt_folder ./dataset 

Ciatation

If you use the code, please cite our paper:

@inproceedings{chen2019dibrender,
title={Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer},
author={Wenzheng Chen and Jun Gao and Huan Ling and Edward Smith and Jaakko Lehtinen and Alec Jacobson and Sanja Fidler},
booktitle={Advances In Neural Information Processing Systems},
year={2019}
}