HiFiHR: High-Fidelity Hand Reconstruction

Enhancing 3D Hand Reconstruction from a Single Image via High-Fidelity Texture

Development Time: +

📒 Table of Contents

📍 Overview
🚀 Getting Started
🛠 Training and Evaluation
👏 Acknowledgments
📄 Citation

📍 Overview

FreiHAND	HO-3Dv2

🎯 Features

Objective: Generate realistic 3D hand meshes with accurate textures from a single image.
Supervision Levels: Utilize self-supervision, weak supervision, and full supervision.
Contributions of High-Fidelity Textures: Enhance hand pose and shape estimation with learned high-fidelity textures.
Benchmark Performance: Experimental evaluations on public benchmarks (FreiHAND and HO-3D). Outperform state-of-the-art methods in texture quality, while maintaining accurate pose and shape estimation.

🚀 Getting Started

📦 Environment

This code is developed under Python 3.9, Pytorch 1.13, and cuda 11.7.

(Optional) You may need to wake up your conda:

conda update -n base -c default conda
conda config --append channels conda-forge
conda update --all

Create the environment and install the requirements:

conda env remove -n hifihr
conda create -n hifihr python=3.9
conda activate hifihr
conda install pytorch=1.13.0 torchvision pytorch-cuda=11.7 -c pytorch -c nvidia

conda install -c fvcore -c iopath -c conda-forge fvcore iopath
# conda install pytorch3d -c pytorch3d
pip install "git+https://github.com/facebookresearch/pytorch3d.git"

conda install tqdm tensorboard transforms3d scikit-image timm trimesh rtree opencv matplotlib rich lpips
pip install chumpy

📂 Datasets

For 3D hand reconstruction task on the FreiHAND dataset:

Download the FreiHAND dataset from the website.

For HO3D dataset:

Download the HO-3Dv2 dataset from the website.

🛠 Training and Evaluation

Pre-trained models can be downloaded from the Google Drive link.

🧪 FreiHAND

Evaluation:

python train_hrnet.py --config_json config/FreiHAND/evaluation.json

Training:

python train_hrnet.py --config_json config/FreiHAND/full_rhd_freihand.json

Note: remember to check and inplace the dirs and files in the *.json files.

🧪 HO3D

Evaluation:

python3 train_hrnet.py --config_json config/HO3D/evaluation.json

Training: Please refer to FreiHAND training scripts.

👏 Acknowledgements

We would like to thank to the great project in S²HAND.

📄 Citation

If you find this code useful for your research, please consider citing:

@inproceedings{zhu2023hifihr,
    title={HiFiHR: Enhancing 3D Hand Reconstruction from a Single Image via High-Fidelity Texture},
    author={Zhu, Jiayin and Zhao, Zhuoran and Yang, Linlin and Yao, Angela},
    booktitle={German Conference on Pattern Recognition},
    year={2023},
    organization={Springer}
}

delaprada / HiFiHR