Learning Descriptor Networks for 3D Shape Synthesis and Analysis

This repository contains a tensorflow implementation for the paper "Learning Descriptor Networks for 3D Shape Synthesis and Analysis ". (http://www.stat.ucla.edu/~jxie/3DDescriptorNet/3DDescriptorNet.html)

Requirements

Python 2.7 or Python 3.3+
Tensorflow r1.3+
Install required Python libraries
```
pip install numpy scipy
```

Getting Started

Clone this repo:

git clone https://github.com/jianwen-xie/3DDescriptorNet.git
cd 3DDescriptorNet

Download volumetric data and save it to ./data directory. The dataset contains 10 categories of voxelizations of ModelNet10.
Download pretrained models and save it to the cloned directory.

Exp1: 3D object synthesis

Train the synthesis model on night stand category:

python train.py --category night_stand --data_dir ./data/volumetric_data/ModelNet10 --output_dir ./output

Visualize the generated results using the MATLAB code in visualization/visualize.m, e.g.

addpath('visualization')
visualize('./output/night_stand/synthesis', 'sample2990.mat')

Evaluate synthesized results using the evaluation code in ./evaluation
You can download our synthesized results and test on it.

Exp2: 3D object recovery

Train the recovery model on sofa category:

python rec_exp.py --category sofa \
                  --num_epochs 1000 \
                  --batch_size 50 \
                  --step_size 0.07 \
                  --sample_steps 90

Test the recovery model:
1. Download the incomplete data and save it to ./data directory. For each category in volumetric_data, the incomplete data contains: 1) incomplete_test.mat: 70% randomly corrupted testing data 2) masks.mat: The mask to corrupt the testing data. 3. original_test.mat: original testing data for comparison.
2. You can download our pretrained model to test recovery.
3. Run recovery on the corrupted data
```
python rec_exp.py --test --category sofa \
                  --ckpt pretrained_model/recovery/sofa/sofa.ckpt \
                  --incomp_data_path ./data/incomplete_data \
                  --batch_size 50 \
                  --step_size 0.07 \
                  --sample_steps 90 
```

Exp3: 3D object super resolution

Train the super resolution model on toilet category:

python sr_exp.py --category toilet \
                  --cube_len 64 \
                  --scale 4 \
                  --num_epochs 500 \
                  --batch_size 50 \
                  --step_size 0.01 \
                  --sample_steps 10

Test the super resolution model:

python rec_exp.py --test --category toilet \
                  --ckpt ./output/toilet/checkpoints/model.ckpt-490 \
                  --cube_len 64 \
                  --scale 4 \
                  --batch_size 50 \
                  --step_size 0.01 \
                  --sample_steps 10

Exp4: 3D object classification

Method	Classification
Geometry Image	88.4%
PANORAMA-NN	91.1%
ECC	90.0%
3D ShapeNets	83.5%
DeepPana	85.5%
SPH	79.8%
VConv-DAE	80.5%
3D-GAN	91.0%
3D DescriptorNet (ours)	92.4%

Train Classification using Logistic Regression (pretrained model):

python train_classification.py --classifier_type logistic --ckpt pretrained_models/classification/model.ckpt

Train Classification using SVM:

python train_classification.py --classifier_type svm --ckpt pretrained_models/classification/model.ckpt

References

@inproceedings{3DDesNet,
    title={Learning Descriptor Networks for 3D Shape Synthesis and Analysis},
    author={Xie, Jianwen and Zheng, Zilong and Gao, Ruiqi and Wang, Wenguan and Zhu Song-Chun and Wu, Ying Nian},
    booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year={2018}
}

For any questions, please contact Jianwen Xie (jianwen@ucla.edu) and Zilong Zheng (zilongzheng0318@ucla.edu).

jianwen-xie / 3DDescriptorNet