Deep Fakes Lightning
This project is a study project about deep fakes technology. In this project, it used the original deep fakes architecture, and trained the model with Pytorch lightning. The aim is to make an open-source project which is similar to DeepFaceLab.
Deep Fakes
Deep Fakes is kind of a technology using for face-swapping. These years, with the emerging of AI and GAN, the Deep Fakes technology become maturer and maturer. We can easily find fantastic face-swapping videos on Youtube. And most of them are made from DeepFaceLab
Why DeepFaceLab
DeepFaceLab is the most well-known face-swapping open-source library. It's easy to use and can fully use the GPU power on Windows system. With its power, many ameture can make a master-piece deepfakes video. However, its code is highily abstract and encapsulated, make you uneasy to make this codebase scalable. So this project is aim to build pipeline of face-swapping, and make it scalable to add any other open-source model.
Why Pytorch Lightning
Since Pytorch is more and more popular for AI developers, Pytorch Lightning is kind of library totally based on Pytorch. It can help you decouple your Pytorch research code and engineering code, makes your code clean and speeds up your development.
Model Architecture
This project follows the deep fakes architecture, uses a shared weight encoder, and trains seperated decoder for both src and dst faces. Finally, it uses the dst face as input to the src decoder to do the face-swapping.
Training Result
In each batch is: src/dst face, reconstructed face, swapped face.
MSELoss with src/dst faces and reconstructed faces
To Do
data
- data preprocessing
- face alignment
- data augmentation
model
- add landmark loss for more accurate expression transformation.
- add discriminator
production
- add merging pipeline
- add colab notebooks