sarumaj / persona

Talking head video AI generator

Persona

Tool for creating talking head videos using generative AI. Uses the following projects/libraries:

1. Tortoise-TTS - generating speech
2. SDXL-Turbo - generating the digital avatar image
3. One-Shot Free-View Neural Talking Head Synthesis - animating the face
4. Wav2Lip - using speech to generate lip movement and superimposing it on the animated face
5. Real ESRGAN - improving the image and scaling it

Setup

$ git clone https://github.com/sarumaj/persona.git
$ cd persona
$ pip install -r requirements.txt

1. Download Way2Lib weights and put them in the wav2lip directory

2. Download Face detection weights and place them in the directory wave2lip/face_detection/detection/sfd/s3fd.pth

3. Create a folder a folder weights and place the Real-ESRGAN weight files in there

Run

$ python persona.py