reyuwei / NIMBLE_model

repo for NIMBLE: A Non-rigid Hand Model with Bones and Muscles

NIMBLE: A Non-rigid Hand Model with Bones and Muscles

We present NIMBLE, a non-rigid parametric hand model that includes bones and muscles, bringing 3D hand model to a new level of realism. By enforcing the inner bones and muscles to match anatomic and kinematic rules, NIMBLE can animate 3D hands to new poses at unprecedented realism.

To learn about NIMBLE, please visit our website: https://liyuwei.cc/proj/nimble

You can find the NIMBLE paper at: https://arxiv.org/abs/2202.04533

---

Requirements:

1. numpy
2. Pytorch
3. Pytorch3d
4. opencv-python
5. trimesh (with pyembree for faster collision detection)

Getting Started:

1. Download model files from here.
2. python demo.py
   • *_bone.obj: bone mesh
   • *_joint.xyz: joints (order see here)
   • *_muscle.obj: muscle mesh
   • *_skin.obj/mtl: skin mesh with diffuse texture
   • *_diffuse/normal/spec.png: texture images
   • *_manov.xyz: corresponding skin vertices in MANO topology
3. Render in assets/render.blend
   • Scale down the meshes and manually assign texture images.
   • Select non-color for color space of normal and specular textures.
   • Change mesh to shade smooth.
   • Adjust normal map strength.

Notes:

1. Use handle_collision=True to make sure skin is always outside muscle. Otherwise, skin and muscle might collide.
2. Joint order see PIANO Dataset.

Acknowledgements

This model and code was developped and used for the paper NIMBLE: A Non-rigid Hand Model with Bones and Muscles for SIGGRAPH22. See project page.

It reuses part of the great code from manopth by Yana Hasson, pytorch_HMR by Zhang Xiong and SMPLX by Vassilis Choutas!

If you find this code useful for your research, consider citing:

@article{li2022nimble,
        author = {Li, Yuwei and Zhang, Longwen and Qiu, Zesong and Jiang, 
            Yingwenqi and Li, Nianyi and Ma, Yuexin and Zhang, Yuyao and
            Xu, Lan and Yu, Jingyi},
        title = {NIMBLE: A Non-Rigid Hand Model with Bones and Muscles},
        year = {2022},
        issue_date = {July 2022},
        publisher = {Association for Computing Machinery},
        address = {New York, NY, USA},
        volume = {41},
        number = {4},
        issn = {0730-0301},
        url = {https://doi.org/10.1145/3528223.3530079},
        doi = {10.1145/3528223.3530079},
        journal = {ACM Trans. Graph.},
        month = {jul},
        articleno = {120},
        numpages = {16}
        }