This repository is an official Pytorch implementation of the paper "Sampling is Matter: Point-guided 3D Human Mesh Reconstruction"
Jeonghwan Kim*, Mi-Gyeong Gwon*, Hyunwoo Park, Hyukmin Kwon, Gi-Mun Um, and Wonjun Kim (Corresponding Author)
* equally contributed
🍁 IEEE/CVF International Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2023. 🍁
- We propose to utilize the correspondence between encoded features and vertex positions, which are projected into the 2D space, via our point-guided feature sampling scheme. By explicitly indicating such vertex-relevant features to the transformer encoder, coordinates of the 3D human mesh are accurately estimated.
- Our progressive attention masking scheme helps the model efficiently deal with local vertex-to-vertex relations even under complicated poses and occlusions.
Please refer to Installation.md for installation.
We provide guidelines to download pre-trained models and datasets.
Please check Download.md for more information.
We provide demo codes to run end-to-end inference on the test images.
Please check Demo.md for more information.
We provide guidelines to train and evaluate our model on Human3.6M and 3DPW.
Please check Experiments.md for more information.
| Model | Dataset | MPJPE | PA-MPJPE | Checkpoint |
|---|---|---|---|---|
| PointHMR-HR32 | Human3.6M | 48.3 | 32.9 | Download |
| PointHMR-HR32 | 3DPW | 73.9 | 44.9 | Download |
Results on 3DPW dataset:
Results on COCO dataset:
This research code is released under the MIT license. Please see LICENSE for more information.
SMPL and MANO models are subject to Software Copyright License for non-commercial scientific research purposes. Please see SMPL-Model License and MANO License for more information.
We use submodules from third party (hassony2/manopth). Please see NOTICE for more information.
This work was supported by Institute of Information & communications Technology Planning & Evaluation(IITP) grant funded by the Korea government(MSIT) (2021-0-02084, eXtended Reality and Volumetric media generation and transmission technology for immersive experience sharing in noncontact environment with a Korea-EU international cooperative research).
Our implementation and experiments are built on top of open-source GitHub repositories. We thank all the authors who made their code public, which tremendously accelerates our project progress. If you find these works helpful, please consider citing them as well.
microsoft/MeshTransformer
microsoft/MeshGraphormer
postech-ami/FastMETRO
Arthur151/ROMP
@InProceedings{PointHMR,
author = {Kim, Jeonghwan and Gwon, Mi-Gyeong and Park, Hyunwoo and Kwon, Hyukmin and Um, Gi-Mun and Kim, Wonjun},
title = {{Sampling is Matter}: Point-guided 3D Human Mesh Reconstruction},
booktitle = {CVPR},
month = {June},
year = {2023}
}