yfeng95 / PoseGPT

ChatPose: Chatting about 3D Human Pose

This is the implementation of ChatPose (formerly known as PoseGPT). More details please check our Project page.

ChatPose is a Multi-modal LLM to understand and reason about 3D Human poses (as SMPL pose format). ChatPose enables users to ask questions regarding human poses and infer these poses from both images and text descriptions.

Getting Started

Clone the repo:

git clone https://github.com/yfeng95/PoseGPT
cd PoseGPT

Requirements

bash install_conda.sh

Download data

bash fetch_data.sh

This step will download SMPL-X model for visualization.

Inference

• chatting:

python main_chat.py 

• chatting with image input:

python main_chat.py --image_file dataset/baber.png

Citation

@InProceedings{feng2024chatpose,
    author = {Feng, Yao and Lin, Jing and Dwivedi, Sai Kumar and Sun, Yu and Patel, Priyanka and Black, Michael J.},
    title = {{ChatPose}: Chatting about 3D Human Pose},
    booktitle = {CVPR},
    year = {2024}
}  

Acknowledgments

This repository is built extensively on top of LLaVA and LISA. Some other great resources we benefit from:
TokenHMR, PoseScript and 4D-Humans for 3D human pose.