syunar / GPAvatar

[ICLR 2024] Generalizable and Precise Head Avatar from Image(s)

     

Xuangeng Chu^1,2  Yu Li²  Ailing Zeng²  Tianyu Yang²  Lijian Lin²  Yunfei Liu²  Tatsuya Harada^1,3
1The University of Tokyo, ²International Digital Economy Academy (IDEA), ³RIKEN AIP

🤩ICLR 2024🤩

GPAvatar reconstructs controllable 3D head avatars from one or several images in a single forward pass.
More results can be seen from our Project Page.

Installation

Install step by step

``` conda create -n track python=3.9 conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.8 -c pytorch -c nvidia conda install -c fvcore -c iopath -c conda-forge fvcore iopath conda install pytorch3d -c pytorch3d pip3 install mediapipe tqdm rich lmdb einops colored ninja av opencv-python scikit-image onnxruntime-gpu onnx transformers pykalman ```

Preparation

Build resources with bash ./build_resources.sh.

Download the model checkpoint and put it at checkpoints/one_model.ckpt.

Quick Start

Driven by images:

python inference.py -r ./checkpoints/one_model.ckpt --driver ./demos/drivers/pdriver --input ./demos/examples/real1

or driven by video:

python inference.py -r ./checkpoints/one_model.ckpt --driver ./demos/drivers/vdriver1 --input ./demos/examples/art1 -v

Citation

If you find our work useful in your research, please consider citing:

@inproceedings{
    chu2024gpavatar,
    title   = {GPAvatar: Generalizable and Precise Head Avatar from Image(s)}, 
    author  = {Xuangeng Chu and Yu Li and Ailing Zeng and Tianyu Yang and Lijian Lin and Yunfei Liu and Tatsuya Harada},
    journal = {ICLR},
    year    = {2024},
}

Acknowledgements

Some part of our work is built based on FLAME, StyleMatte, EMOCA and MICA. The GPAvatar Logo is designed by Caihong Ning. We thank you for sharing their wonderful code and their wonderful work.

• FLAME: https://flame.is.tue.mpg.de
• StyleMatte: https://github.com/chroneus/stylematte
• EMOCA: https://github.com/radekd91/emoca
• MICA: https://github.com/Zielon/MICA