zeng-yifei / AvatarBooth

Official implementation of “AvatarBooth: High-Quality and Customizable 3D Human Avatar Generation”

AvatarBooth: High-Quality and Customizable 3D Human Avatar Generation

Yifei Zeng¹  Yuanxun Lu¹  Xinya Ji¹  Yao Yao¹  Hao Zhu¹⁺  Xun Cao¹

¹Nanjing University

⁺corresponding author

[Project Page] •

Install

For package installation, ensure that you have installed pytorch (tested on pytorch 2.1 cuda121 and pytorch 1.13 cuda 117):

pip install -r requirements.txt

For data preparation, register and download SMPL models here. Put the downloaded models in the folder smpl_models. The folder structure should look like

./
├── ...
└── smpl_models/
    ├── smpl/
        ├── SMPL_FEMALE.pkl
        ├── SMPL_MALE.pkl
        └── SMPL_NEUTRAL.pkl

Usage

To generate avatars. you can use:

python main.py --mode train --conf confs/examples/obama.conf

To use personalized model like LoRA or DreamBooth model, you can assign the corresponding file path in config file like:

general {
    sd_path = ... # assign DreamBooth path for whole body in huggingface format, e.g. stabilityai/stable-diffusion-2-1-base or stablediffusionapi/realistic-vision(recommanded and by default)
    sd_face_path = ... # assign DreamBooth path for face in huggingface format

    lora_path = ... # assign lora path with safetensors, e.g. ./pretrained_models/A.safetensors
}

To animate the avatar, you can refer to the AvatarCLIP. With the same procedure, you can obtain a animatable fbx after processing the A-pose ply model.

Teaser

Animation

Citation

If you find our work useful for your research, please consider citing the paper:

@inproceedings{Zeng2023AvatarBoothHA,
  title={AvatarBooth: High-Quality and Customizable 3D Human Avatar Generation},
  author={Yifei Zeng and Yuanxun Lu and Xinya Ji and Yao Yao and Hao Zhu and Xun Cao},
  year={2023}
}

Acknowledgement

The code is built upon AvatarCLIP and Stable DreamFusion, we express great appreciation to the authors for their great work.