wl-zhao / DiffSwap

[CVPR 2023] DiffSwap is a diffusion-based face-swapping framework.

DiffSwap

Created by Wenliang Zhao, Yongming Rao, Weikang Shi, Zuyan Liu, Jie Zhou, Jiwen Lu†

This repository contains PyTorch implementation for paper "DiffSwap: High-Fidelity and Controllable Face Swapping via 3D-Aware Masked Diffusion"

[paper]

Installation

Please first install the environment following stable-diffusion, and then run pip install -r requirements.txt.

Please download the checkpoints from [here], and put them under the checkpoints/ folder. The resulting file structure should be:

├── checkpoints
│   ├── diffswap.pth
│   ├── glint360k_r100.pth
│   └── shape_predictor_68_face_landmarks.dat

Inference

We provide a sample code to perform face swapping given the portrait source and target images. Please put the source images and target images in data/portrait_jpg and run

python pipeline.py

the swapped results are saved in data/portrait/swap_res_ori.

Citation

If you find our work useful in your research, please consider citing:

@article{zhao2023diffswap,
  title={DiffSwap: High-Fidelity and Controllable Face Swapping via 3D-Aware Masked Diffusion},
  author={Zhao, Wenliang and Rao, Yongming and Shi, Weikang and Liu, Zuyan and Zhou, Jie and Lu, Jiwen},
  journal={CVPR},
  year={2023}
}