lkeab / EvoSkeleton

Official project website for the CVPR 2020 paper (Oral Presentation) "Cascaded Deep Monocular 3D Human Pose Estimation With Evolutionary Training Data"

EvoSkeleton

This is the project website containing relevant files for the CVPR 2020 paper "Cascaded Deep Monocular 3D Human Pose Estimation with Evolutionary Training Data". The usage and instructions are organized into several parts serving distinct purposes. Please visit the corresponding sub-page for details.

Temporary Note: This repo is currently a Beta version and certain modules are still under cleaning. More comments and documentation will be added in a future update.

Hierarchical Human Representation and Data Synthesis

This sub-page gives instructions on how to use the 3D skeleton model and how the evolution algorithm can be used to discover novel data.

Cascaded 2D-to-3D Lifting

This sub-page details how to train a cascaded model to lift 2D key-points to 3D skeletons.

If you do not want to prepare synthetic data and train the model by yourself, you can access an examplar pre-trained model here and follow the inference section in the document.

Dataset: Unconstrained 3D Pose in the Wild

This sub-page describs the newly collected dataset Unconstrained 3D Human Pose in the Wild (U3DPW) and gives instructions on how to download it.

Interactive Annotation Tool

This sub-page provides usage of an annotator that can be used to label 2D and 3D skeleton for an input image. U3DPW was obtained created with this tool and this tool may help increasing the scale of 3D annotation for in-the-wild images.

Environment

• Python 3.6
• Numpy 1.16
• PyTorch 1.0.1
• CUDA 9

For a complete list of other python packages, please refer to spec-list.txt. The recommended environment manager is Anaconda, which can create an environment using the provided spec-list. Certain tool in this project may need other specified environment, which is detailed in its corresponding page.

License

A MIT license is used for this repository. However, certain third-party dataset (Human 3.6M) and tool (SMPLify) are subject to their respective licenses and may not grant commercial use.

Citation

Please star this repository and cite the following paper in your publications if it helps your research:

@InProceedings{Li_2020_CVPR,
author = {Li, Shichao and Ke, Lei and Pratama, Kevin and Tai, Yu-Wing and Tang, Chi-Keung and Cheng, Kwang-Ting},
title = {Cascaded Deep Monocular 3D Human Pose Estimation With Evolutionary Training Data},
booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}

Link to the paper: Cascaded Deep Monocular 3D Human Pose Estimation With Evolutionary Training Data