State-of-the-art methods on monocular 3D pose estimation / 3D mesh recovery
Benchmark Leaderboard and state-of-the-art method list on monocular 3D pose estimation / 3D mesh recovery
Feel free to contribute!
Contents
Benchmark Leaderboard
Evaluation Matrix
See evaluation_matrix.py in Numpy / Pytorch to get more details on evaluation matrix for 3D pose estimation / mesh recovery. In general, MPJPE and PA-MPJPE are mean 3D joint errors. The lower, the better.
Comparisons on Human3.6M.
See datasets/Human3.6M to get more details on Human3.6M
State-of-the-art 3D pose estimation methods
| Methods | Publication | MPJPE | PA-MPJPE | link |
|---|---|---|---|---|
| OANet | ICCV19 | 42.9 | 32.8 | [conf] |
| 3DMPPE | ICCV19 | 54.4 | - | [conf] [code] |
| SemGCN | CVPR19 | 57.6 | - | [conf] [code] |
| VideoPose3D | CVPR19 | 46.8 | 36.5 | [conf] [code] |
| EpipolarPose | CVPR19 | 51.8 | 45.0 | [conf] [code] |
| FBN | TPAMI | 61.1 | - | [arxiv] |
| TP-Net | ECCV18 | 58.2 | 44.1 | [conf] [code] |
| IHPR | ECCV18 | 64.1 | 49.6 | [conf] [code] |
| MVC | CVPR18 | 66.8 | 51.6 | [conf] |
| Deephar | CVPR18 | 53.2 | - | [conf] [code] |
| ODS | CVPR18 | 56.2 | 41.8 | [conf] [code] |
| LPG | AAAI18 | 60.4 | 45.7 | [conf] |
| CHPR | ICCV17 | 92.4 | 59.1 | [conf] |
| 3DPB | ICCV17 | 62.9 | 47.7 | [conf] [code] |
| RPSM | CVPR17 | 73.1 | - | [conf] [code] |
| C2F | CVPR17 | 71.9 | 51.9 | [conf] [code] |
| LFD | CVPR17 | 113.0 | - | [conf] [code] |
| SMD | CVPR16 | 88.4 | - | [conf] [code] |
State-of-the-art 3D mesh recovery methods
See datasets/3DPW to get more details on 3DPW
Evaluation Datasets of MPJPE: Human3.6M | Human3.6M | HumanEva-I | 3DPW
| Methods | Publication | MPJPE | PA-MPJPE | MPJPE | PA-MPJPE | link |
|---|---|---|---|---|---|---|
| ROMP | ICCV21 | - | - | - | 47.3 | [arxiv] [code] |
| VIBE | CVPR20 | 65.6 | 41.4 | - | 51.9 | [arxiv] [code] |
| DSD-SATN | ICCV19 | 59.1 | 42.4 | - | 69.5 | [conf] [code] |
| SPIN | ICCV19 | - | 41.1 | - | 59.2 | [conf] [code] |
| Texturepose | ICCV19 | - | 49.7 | - | - | [conf] [code] |
| DenseRaC | ICCV19 | 76.8 | 48.0 | - | - | [conf] |
| HoloPose | CVPR19 | 60.2 | 46.5 | - | - | [conf] [code] |
| GCMR | CVPR19 | 71.9 | 50.1 | - | - | [conf] [code] |
| HMR-video | CVPR19 | - | 56.9 | - | 72.6 | [conf] [code] |
| HMR | CVPR18 | 87.9 | 58.1 | - | - | [conf] [code] [Pytorch] |
| UP | CVPR17 | 80.7 | - | 74.5 | - | [conf] [code] |
| SMPLify | ECCV16 | 82.3 | - | 79.9 | - | [Paper] [code] |
Valuable Code
Differential Renderer
NMR CVPR18
Neural 3D Mesh Renderer
Hiroharu Kato, Yoshitaka Ushiku, Tatsuya Harada [conf] [Project] [Tensorflow - official] [Pytorch]
Method Zoo
arXiv Papers
[2008.12272] CenterHMR: a Bottom-up Single-shot Method for Multi-person 3D Mesh Recovery from a Single Image
Yu Sun, Qian Bao, Wu Liu, Yili Fu, Tao Mei
[2004.13985] Motion Guided 3D Pose Estimation from Videos
Jingbo Wang, Sijie Yan, Yuanjun Xiong, Dahua Lin
[2003.10350] Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flows
Andrei Zanfir, Eduard Gabriel Bazavan, Hongyi Xu, Bill Freeman, Rahul Sukthankar, Cristian Sminchisescu
[2003.10873] EllipBody: A Light-weight and Part-based Representation for Human Pose and Shape Recovery
Min Wang, Feng Qiu, Wentao Liu, Chen Qian, Xiaowei Zhou, Lizhuang Ma
[2004.01166] Bodies at Rest: 3D Human Pose and Shape Estimation from a Pressure Image using Synthetic Data
Henry M. Clever, Zackory Erickson, Ariel Kapusta, Greg Turk, C. Karen Liu, Charles C. Kemp
[2004.02186] Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representation
Edoardo Remelli, Shangchen Han, Sina Honari, Pascal Fua, Robert Wang
[2004.03143] Predicting Camera Viewpoint Improves Cross-dataset Generalization for 3D Human Pose Estimation
Zhe Wang, Daeyun Shin, Charless C. Fowlkes
[2004.03989] Multi-Person Absolute 3D Human Pose Estimation with Weak Depth Supervision
Marton Veges, Andras Lorincz
[Code]
[2004.05275] Multi-View Matching (MVM): Facilitating Multi-Person 3D Pose Estimation Learning with Action-Frozen People Video
Yeji Shen, C.-C. Jay Kuo
[2004.05815] MulayCap: Multi-layer Human Performance Capture Using A Monocular Video Camera
Zhaoqi Su, Weilin Wan, Tao Yu, Lingjie Liu, Lu Fang, Wenping Wang, Yebin Liu
[2004.03686] Exemplar Fine-Tuning for 3D Human Pose Fitting Towards In-the-Wild 3D Human Pose Estimation
Hanbyul Joo, Natalia Neverova, Andrea Vedaldi
Conference Papers
2021 ICCV
ROMP
Monocular, One-stage, Regression of Multiple 3D People
Yu Sun, Qian Bao, Wu Liu, Yili Fu, Michael J. Black, Tao Mei
[arxiv] [Pytorch code]
2020 ECCV
Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose
Hongsuk Choi, Gyeongsik Moon, and Kyoung Mu Lee
[arxiv]
Full-Body Awareness from Partial Observations
Chris Rockwell and David F. Fouhey
[paper]
Unsupervised Human 3D Pose Representation with Viewpoint and Pose Disentanglement
Qiang Nie, Ziwei Liu, Yunhui Liu
[arxiv]
2020 CVPR
ETD
Cascaded Deep Monocular 3D Human Pose Estimation With Evolutionary Training Data
Shichao Li, Lei Ke, Kevin Pratama, Yu-Wing Tai, Chi-Keung Tang, Kwang-Ting Cheng
[conf] [Pytorch code]
DKA
Deep Kinematics Analysis for Monocular 3D Human Pose Estimation
Jingwei Xu, Zhenbo Yu, Bingbing Ni, Jiancheng Yang, Xiaokang Yang, Wenjun Zhang
[conf]
3D Human Mesh Regression With Dense Correspondence
Wang Zeng, Wanli Ouyang, Ping Luo, Wentao Liu, Xiaogang Wang
[conf]
Object-Occluded Human Shape and Pose Estimation from a Single Color Image
Tianshu Zhang, Buzhen Huang and Yangang Wang
[pdf]
Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wild
Umar Iqbal, Pavlo Molchanov, Jan Kautz
[arxiv]
Attention Mechanism Exploits Temporal Contexts: Real-time 3D Human Pose Reconstruction
Ruixu Liu1, Ju Shen1, He Wang1, Chen Chen2, Sen-ching Cheung3, Vijayan Asari1
[pdf]
Multiview-Consistent Semi-Supervised Learning for 3D Human Pose Estimation
Rahul Mitra, Nitesh B. Gundavarapu, Abhishek Sharma, Arjun Jain
[conf]
VIBE
VIBE: Video Inference for Human Body Pose and Shape Estimation
Muhammed Kocabas, Nikos Athanasiou, Michael J. Black
[arxiv] [Pytorch code]
Multiperson
Coherent Reconstruction of Multiple Humans From a Single Image*
Wen Jiang, Nikos Kolotouros, Georgios Pavlakos, Xiaowei Zhou, Kostas Daniilidis
[conf]
ActiveMoCap
ActiveMoCap: Optimized Viewpoint Selection for Active Human Motion Capture
Sena Kiciroglu, Helge Rhodin, Sudipta N. Sinha, Mathieu Salzmann, Pascal Fua
[conf]
PandaNet
PandaNet: Anchor-Based Single-Shot Multi-Person 3D Pose Estimation
Abdallah Benzine, Florian Chabot, Bertrand Luvison, Quoc Cuong Pham, Catherine Achard
[conf]
LoCO
Compressed Volumetric Heatmaps for Multi-Person 3D Pose Estimation
Matteo Fabbri, Fabio Lanzi, Simone Calderara, Stefano Alletto, Rita Cucchiara
[arxiv] [Code]
Pgp
Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis
Jogendra Nath Kundu, Siddharth Seth, Varun Jampani, Mugalodi Rakesh, R. Venkatesh Babu, Anirban Chakraborty
[arxiv] [Project]
2020-Others
AAAI2020 - 3D Human Pose Estimation using Spatio-Temporal Networks with Explicit Occlusion Training
Yu Cheng, Bo Yang, Bo Wang, Robby T. Tan [arxiv]
2019 ICCV
DSD-SATN
Human Mesh Recovery From Monocular Images via a Skeleton-Disentangled Representation
Yu Sun, Yun Ye, Wu Liu, Wenpeng Gao, Yili Fu, Tao Mei
SPIN
Learning to Reconstruct 3D Human Pose and Shape via Model-Fitting in the Loop
Nikos Kolotouros, Georgios Pavlakos, Michael J. Black, Kostas Daniilidis
TexturePose
TexturePose: Supervising Human Mesh Estimation With Texture Consistency
Georgios Pavlakos, Nikos Kolotouros, Kostas Daniilidis
DenseRaC
DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Compare
Yuanlu Xu, Song-Chun Zhu, Tony Tung
BSFMV
On Boosting Single-Frame 3D Human Pose Estimation via Monocular Videos
Zhi Li, Xuan Wang, Fei Wang, Peilin Jiang [conf]
STR-GCN
Exploiting Spatial-Temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks
Yujun Cai, Liuhao Ge, Jun Liu, Jianfei Cai, Tat-Jen Cham, Junsong Yuan, Nadia Magnenat Thalmann [conf]
GOR
Monocular 3D Human Pose Estimation by Generation and Ordinal Ranking
Saurabh Sharma, Pavan Teja Varigonda, Prashast Bindal, Abhishek Sharma, Arjun Jain [conf]
C3DPO
C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion
David Novotny, Nikhila Ravi, Benjamin Graham, Natalia Neverova, Andrea Vedaldi [conf]
HEMlets
HEMlets Pose: Learning Part-Centric Heatmap Triplets for Accurate 3D Human Pose Estimation
Kun Zhou, Xiaoguang Han, Nianjuan Jiang, Kui Jia, Jiangbo Lu [conf]
ONS
Optimizing Network Structure for 3D Human Pose Estimation
Hai Ci, Chunyu Wang, Xiaoxuan Ma, Yizhou Wang [conf]
DK
Distill Knowledge From NRSfM for Weakly Supervised 3D Pose Learning
Chaoyang Wang, Chen Kong, Simon Lucey [conf]
OANet
Occlusion-Aware Networks for 3D Human Pose Estimation in Video
Yu Cheng, Bo Yang, Bo Wang, Wending Yan, Robby T. Tan
3DMPPE
Camera Distance-Aware Top-Down Approach for 3D Multi-Person Pose Estimation From a Single RGB Image
Gyeongsik Moon, Ju Yong Chang, Kyoung Mu Lee
2019 CVPR
GCMR
Convolutional Mesh Regression for Single-Image Human Shape Reconstruction
Nikos Kolotouros, Georgios Pavlakos, Kostas Daniilidis
HoloPose
HoloPose: Holistic 3D Human Reconstruction In-The-Wild
Riza Alp Guler, Iasonas Kokkinos
HMR-video
Learning 3D Human Dynamics From Video
Angjoo Kanazawa, Jason Y. Zhang, Panna Felsen, Jitendra Malik
SemGCN
Semantic Graph Convolutional Networks for 3D Human Pose Regression
Long Zhao, Xi Peng, Yu Tian, Mubbasir Kapadia, Dimitris N. Metaxas
VideoPose3D
3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training
Dario Pavllo, Christoph Feichtenhofer, David Grangier, Michael Auli
EpipolarPose
Self-Supervised Learning of 3D Human Pose Using Multi-View Geometry
Muhammed Kocabas, Salih Karagoz, Emre Akbas
2018 ECCV
TP-Net
Exploiting temporal information for 3D human pose estimation
Mir Rayat Imtiaz Hossain, James J. Little
IHPR
Integral Human Pose Regression
Xiao Sun, Bin Xiao, Fangyin Wei, Shuang Liang, Yichen Wei
2018 CVPR
HMR
End-to-End Recovery of Human Shape and Pose
Angjoo Kanazawa, Michael J. Black, David W. Jacobs, Jitendra Malik
[conf] [Tensorflow - official] [Pytorch]
MVC
Learning Monocular 3D Human Pose Estimation From Multi-View Images
Helge Rhodin, Jörg Spörri, Isinsu Katircioglu, Victor Constantin, Frédéric Meyer, Erich Müller, Mathieu Salzmann, Pascal Fua
Deephar
2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning
Diogo C. Luvizon, David Picard, Hedi Tabia
ODS
Ordinal Depth Supervision for 3D Human Pose Estimation
Georgios Pavlakos, Xiaowei Zhou, Kostas Daniilidis
Deephar
2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning
Diogo C. Luvizon, David Picard, Hedi Tabia
2018 Others
LPG-AAAI18
Learning Pose Grammar to Encode Human Body Configuration for 3D Pose Estimation
Hao-Shu Fang, Yuanlu Xu, Wenguan Wang, Xiaobai Liu, Song-Chun Zhu
OriNet-BMVC18
OriNet: A Fully Convolutional Network for 3D Human Pose Estimation
Chenxu Luo, Xiao Chu, Alan Yuille
[arxiv] [code]
2017 ICCV
comming soon...
2017 CVPR
UP
Unite the People: Closing the Loop Between 3D and 2D Human Representations
Christoph Lassner, Javier Romero, Martin Kiefel, Federica Bogo, Michael J. Black, Peter V. Gehler
RPSM
Recurrent 3D Pose Sequence Machines
Mude Lin, Liang Lin, Xiaodan Liang, Keze Wang, Hui Cheng
C2F
Coarse-To-Fine Volumetric Prediction for Single-Image 3D Human Pose
Georgios Pavlakos, Xiaowei Zhou, Konstantinos G. Derpanis, Kostas Daniilidis
LFD
Lifting From the Deep: Convolutional 3D Pose Estimation From a Single Image
Denis Tome, Chris Russell, Lourdes Agapito
2016 ECCV
SMPLify
Keep it SMPL: Automatic Estimation of {3D} Human Pose and Shape from a Single Image
Bogo Federica, Kanazawa Angjoo, Lassner Christoph, Gehler Peter, Romero Javier, Black Michael J.
2016 CVPR
SMD
Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video
Xiaowei Zhou, Menglong Zhu, Spyridon Leonardos, Kosta Derpanis, Kostas Daniilidis
Journal-Papers
FBN
Feature Boosting Network For 3D Pose Estimation
Jun Liu, Henghui Ding, Amir Shahroudy, Ling-Yu Duan, Xudong Jiang, Gang Wang, Alex C. Kot
Datasets
Deep Fashion3D
Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images
Heming Zhu and Yu Cao and Hang Jin and Weikai Chen and Dong Du and Zhangye Wang and Shuguang Cui and Xiaoguang Han
[arxiv] [project]
3DBodyTex.Pose
Towards Generalization of 3D Human Pose Estimation In The Wild
Renato Baptista, Alexandre Saint, Kassem Al Ismaeil, Djamila Aouada
[arxiv]
GPA
Geometric Pose Affordance: 3D Human Pose with Scene Constraints
Zhe Wang, Liyan Chen, Shaurya Rathore, Daeyun Shin, Charless Fowlkes
[arxiv] [project]
JTA
Learning to Detect and Track Visible and Occluded Body Joints in a Virtual World
Matteo Fabbri, Fabio Lanzi, Simone Calderara, Andrea Palazzi, Roberto Vezzani, Rita Cucchiara
[arxiv] [project]
Contribute
Contributions are welcome! Please feel free to pull requests, open an issue or send me email (yusun@stu.hit.edu.cn) to add/correct state-of-the-art methods.