HairCLIPv2: Unifying Hair Editing via Proxy Feature Blending (ICCV2023)

This repository hosts the official PyTorch implementation of the paper: "HairCLIPv2: Unifying Hair Editing via Proxy Feature Blending".

HairCLIPv2 supports hairstyle and color editing individually or jointly with unprecedented user interaction mode support, including text, mask, sketch, reference image, etc.

Tianyi Wei¹, Dongdong Chen², Wenbo Zhou¹, Jing Liao³, Weiming Zhang¹, Gang Hua⁴, Nenghai Yu¹
¹University of Science and Technology of China, ²Microsoft Cloud AI, ³City University of Hong Kong, ⁴Xi'an Jiaotong University

News

2023.12.08: The initial version of the code is released.
2023.10.12: HairCLIPv2 is accepted by ICCV2023! 🎉

Getting Started

Prerequisites

$ pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 -f https://download.pytorch.org/whl/torch_stable.html
$ pip install ftfy regex tqdm matplotlib jupyter ipykernel opencv-python scikit-image kornia==0.6.7 face-alignment==1.3.5 dlib==19.22.1
$ pip install git+https://github.com/openai/CLIP.git

Pretrained Models

Download and put all the downloaded pretrained weights into the pretrained_models directory.

Path	Description
FFHQ StyleGAN	StyleGAN model pretrained on FFHQ with 1024x1024 output resolution.
Face Parse Model	Pretrained face parse model taken from Barbershop.
Face Landmark Model	Used to align unprocessed images.
Bald Proxy	Bald proxy weights from HairMapper.
Sketch Proxy	Sketch proxy weights trained on hair-sketch dataset using E2style.

Input Instructions

All inputs are in the test_images folder and all images should be made sure to be aligned to 1024x1024 according to FFHQ standards.

Path	Description
`unaligned_img`	Natural images that should be aligned using `python scripts/align_face.py` Results are stored in `aligned_img`.
`aligned_img`	Depending on the usage, they should be moved to `src_img` or `ref_img`.
`src_img`	Aligned image to be edited.
`src_F`	The latent code of `src_img` being inverted is not required to be provided by the user.
`ref_img`	Aligned hairstyle reference image, hair color reference image.
`ref_latent`	The latent code of `ref_img` being inverted is not required to be provided by the user.

Enjoy

Open hairclip_v2_demo.ipynb to unlock your versatile hairstyles!

If you run it on a remote server, you need to run jupyter notebook --port=20367 --allow-root --ip=0.0.0.0 first. then use e.g. VS Code to select that Jupyter Server as your kernel.

To Do

Develop an interactive interface

Acknowledgements

This code is based on Barbershop.

Citation

If you find our work useful for your research, please consider citing the following papers :)

@article{wei2023hairclipv2,
  title={HairCLIPv2: Unifying Hair Editing via Proxy Feature Blending},
  author={Wei, Tianyi and Chen, Dongdong and Zhou, Wenbo and Liao, Jing and Zhang, Weiming and Hua, Gang and Yu, Nenghai},
  journal={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  year={2023}
}

@article{wei2022hairclip,
  title={Hairclip: Design your hair by text and reference image},
  author={Wei, Tianyi and Chen, Dongdong and Zhou, Wenbo and Liao, Jing and Tan, Zhentao and Yuan, Lu and Zhang, Weiming and Yu, Nenghai},
  journal={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2022}
}

XiaoqingWang / HairCLIPv2