HandTailor

This repository is the implementation code and model of the paper "HandTailor: Towards High-Precision Monocular 3D Hand Recovery" (arXiv)

Get the Code

git clone https://github.com/LyuJ1998/HandTailor.git
cd HandTailor

Install Requirements

Please install the dependencies listed in requirements.txt.

pip install -r requirements.txt

Download Model Files

Pretrain Model

Download the pretrain model from Google Drive, and put the model.pt in ./checkpoints

MANO Model

Go to MANO website
Create an account by clicking Sign Up and provide your information
Download Models and Code (the downloaded file should have the format mano_v*_*.zip). Note that all code and data from this download falls under the MANO license.
unzip and copy the MANO_RIGHT.pkl file into the folder

Demo

To process the image provided in ./demo, run

python demo.py

You can also put your data in the fold ./demo, but remember to use the proper camera intrinsic like

python demo.py --fx=612.0206 --fy=612.2821 --cx=321.2842 --cy=235.8609

If camera information is unavailable, run

python demo_in_the_wild.py

We recommand you to utilize the camera intrinsic, which will improve the performance a lot.

Realtime Demo

To reconstruct the hand from image captured with a webcam,run the following command. Also remember to use the proper camera intrinsic, the following command is for RealSense D435

python app.py --fx=612.0206 --fy=612.2821 --cx=321.2842 --cy=235.8609

When camera information is absence

python app_in_the_wild.py

Citation

If you find this work helpful, please consider citing us

@article{lv2021handtailor,
  title={HandTailor: Towards High-Precision Monocular 3D Hand Recovery},
  author={Lv, Jun and Xu, Wenqiang and Yang, Lixin and Qian, Sucheng and Mao, Chongzhao and Lu, Cewu},
  journal={arXiv preprint arXiv:2102.09244},
  year={2021}
}

maitetsu / HandTailor