runningpp / SIF

SIF: Skeleton-aware Implicit Function for Single-view Human Reconstruction

SIF: Skeleton-aware Implicit Function for Single-view Human Reconstruction

• PARE (SMPL), PyMAF (SMPL) are all supported as optional HPS.

Table of Contents

1. Who needs SIF
2. TODO
3. Installation
4. RGB based Experiments
5. Train Eval

Who needs SIF?

• Given an RGB image, you could get:
  • image (png): segmentation, normal images (body + cloth), overlap result (rgb + normal)
  • mesh (obj): SMPL body, reconstructed clothed human

TODO

• pretrained models (*self-implemented version)
• PIFu* (RGB image + predicted normal map as input)
• PaMIR* (RGB image + predicted normal map as input, w/ PyMAF/PARE as HPS)
• dataset processing pipeline
• training and evaluation codes

Installation

Please follow the requirenments and environment to setup all the required packages
Please follow the Data Preprocess to generate the train/val/test dataset from raw scans (THuman2.0).

RGB based Experiments

We have show more RGB base experiments, i.e. RGB based ablation study. RGB base experiments

Train & Eval

You should get the dataset (inclued SMPL normal, SMPL detpth, joints,Normal,RGB), and set your own path.
For the training details, you can refer config.py,train_eval_sif and tainer_sif
For the training loss, you can refer tainer_sif

python train_eval_sif.py

More Qualitative Results

Acknowledgments

Here are some great resources we benefit from:

• ICON for normal estimation and SMPL refinement, PaMIR, PIFu, PIFuHD, and MonoPort for Benchmark
• rembg for Human Segmentation
• PARE, PyMAF, and PIXIE for Human Pose & Shape Estimation
• TwinDom and THuman for Dataset
• PyTorch3D for Differential Rendering