massimilianoviola / arctic-hands-object-viewer

Simple 3D rendering of hand-object interactions for the ARCTIC dataset

ARCTIC 🥶 Hands & Object Viewer

This repository allows you to easily render hand-object interactions in 3D from the ARCTIC dataset. Pick a raw data sequence and a frame number and interactively explore the 3D scene in detail!

Requirements

The ARCTIC dataset and codebase are what this repository is based on. In particular, it requires having a copy of the dataset and models in the /data folder, which can be obtained following the instructions here. Credits to the original work.

To run the viewer, first install the environment and packages:

conda create -n arctic_viz python=3.10
conda activate arctic_viz
pip install -r requirements.txt

Then download some scripts from the arctic repository:

chmod +x ./bash/*.sh
./bash/download_scripts.sh

After, modify smplx package to return 21 joints instead of 16:

vim /home/<user_name>/anaconda3/envs/arctic_viz/lib/python3.10/site-packages/smplx/body_models.py

# uncomment L1681
joints = self.vertex_joint_selector(vertices, joints)

Finally, copy the previously prepared /data folder into the current directory. If the repository structure looks like this, you are ready to go!

.
├── bash/
├── common/
├── data/
├── README.md
├── requirements.txt
├── res/
├── src/
└── utils/

Usage

To render a scene, provide the path to a raw .mano.npy sequence and a valid frame number to the hands_object_viewer.py script.

usage: hands_object_viewer.py [-h] [-m MANO_P] [-f FRAME]

options:
  -h, --help            show this help message and exit
  -m MANO_P, --mano_p MANO_P
                        Path to raw .mano.npy sequence to process
  -f FRAME, --frame FRAME
                        Frame number to visualize

For example, try:

python src/hands_object_viewer.py --mano_p ./data/arctic_data/data/raw_seqs/s01/laptop_use_01.mano.npy --frame 250

References

[1] Fan et al., "ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation", http://download.is.tue.mpg.de/arctic/arctic_april_24.pdf
[2] ARCTIC 🥶: A Dataset for Dexterous Bimanual Hand-Object Manipulation, https://github.com/zc-alexfan/arctic