hajungong007 / wunjo.wladradchenko.ru

Wunjo AI: Synthesize & clone voices in English, Russian & Chinese, real-time speech recognition, deepfake face & lips animation, face swap with one photo, change video by text prompts, segmentation, and retouching. Open-source, local & free.

(RU)

Wunjo AI: Advanced Speech & Deepfake Neural Network Tool

Documentation
Issue · Discussions · Tutorial

About

Unlock the unparalleled capabilities of neural networks with Wunjo AI. Whether you're delving into speech synthesis, crafting deepfake animations, drawing Stable Diffusion video by text prompt or video making, Wunjo AI has got you covered.

Key Features:

• Speech Synthesis: Effortlessly convert text into human-like speech.
• Voice Cloning: Clone voices from provided audio files or directly record your voice within the app for real-time cloning.
• Multilingual Support: Currently supports English, Russian, Chinese for voice cloning (from any language audio) and English, Russian synthesis, with plans to extend voice cloning synthesis model for Spanish.
• Real-time Speech Recognition: Dictate text and get instant transcriptions. An efficient tool for hands-free content creation.
• Multidialogue Creation: Craft multi-dialogues using unlimited characters with distinct voice profiles.
• Video-to-Video by Text Prompt:
  • Reshape videos with by text prompt with difference models of Stable Diffusion. Let generative neural networks craft a new visual narrative.
  • Change individual objects in a video by text prompt with one click, changing them throughout the video with unique text queries.
  • Preserve specific objects without change by using the «pass» keyword.
• Deepfake Animation:
  • Animate faces using just one photo combined with audio.
  • Achieve precise lip syncing with your audio using our deepfake lips feature.
  • Effortlessly swap faces in videos, GIFs, and photos using just a single photograph with our "Face Swap" feature.
  • Experimental feature. Change the emotions of a person in the video, with the help of a text description.
• AI Retouch Tool: Elevate your videos by removing unwanted objects or refining the quality of your deepfakes.
• Automatic Segmentation Mask: Select any object at any time period and get a storyboard of the selected object with a transparent or colored background.

Applications: From voiceovers in commercials to character voicing in games, from audiobook narrations to fun deepfake projects, Wunjo AI offers endless possibilities and all is free and local on your device.

Why Choose Wunjo AI?:

• All-in-One: A comprehensive tool catering to both your voice and visual AI needs.
• User-friendly: Designed for all, from beginners to professionals.
• Privacy First: Functions locally on your desktop, ensuring your data remains private.
• Open-source & Free: Benefit from community-driven enhancements and enjoy the app without any cost.

Step into the future of AI-powered creativity with Wunjo AI.

Setup

Requirements Python version 3.10 and ffmpeg.

For detailed instructions about setup Wunjo AI from GitHub, refer to the Launch Project from GitHub section in our wiki.

Install packets

Ubuntu / Debian v1.6 (GPU version)

For detailed instructions about install Wunjo AI on Ubuntu / Debian OS from installer

MacOS v1.6 (CPU version)

Due to the fact that the author of the project does not have an Apple license, there is currently no way to create an official installer.

Windows v1.6 (CPU version)

For detailed instructions about install Wunjo AI on Windows from installer

Read in Wunjo AI documentation how use GPU on Windows.

Example

Speech synthesis and voice cloning

• Russian synthesized voice from text
• English voice cloned from previously synthesized Russian voice
• Chinese voice cloned from a previously synthesized Russian voice

Face animation from image src

Original	Move face + Enhancer	Fix face + Enhancer

Mouth animation from video src

Original	Mouth animation	Mouth animation + Enhancer

Face swap by one photo

Original photo	Original video	Face swap + Background enhancer

Remove object by Retouch AI

Original video	Remove object

Retouch AI to improve quality of deepfake

Defective lines on the chins after animation lip	Retouch lines on the chins + Face swap

Get segmentation mask by one click

Original	Mask

Video-to-Video by Text Prompt (Only for GPU)

The higher the video resolution, the better the quality of the drawn frames.

Video resolution 512x512 default model for deepfake

Original	Blonde hair + Brown jacket

Video resolution 512x512 custom model for anime

Additionally, you can use your custom stable diffusion model to redraw video or objects in video with difference timeline.

Original	Pass object + Background change	Full change

Limit resolution video by GPU VRAM

24 GB	18 GB	14 GB	10 GB	8 GB	7 GB
1280x1280	1024x1024	768x768	640x640	576x576	512x512

Emotion deepfake [Experimental]

This is an experimental feature that is under development, but you can take a look at some of the work right now in Wunjo AI.

Original	Happy	Angry
Fear	Sad	Disgust

Language

The application comes with built-in support for the following languages: English, Russian, Chinese, Portuguese, and Korean.

If you wish to add a new language:

Navigate to .wunjo/settings/settings.json. Add your desired language in the format: "default_language": {"name": "code"}. To find the appropriate code for your language, please refer to the Google Cloud Translate Language Codes.

Update 1.6.0

• Improved and automated remove object from image or video
• Improved edit video element
• Added auto segmentation mask with save
• Added Video2Video with ControlNet by text prompt tool
• Added InpaintVideoMask2Video with ControlNet by text prompt tool
• Optimized using memory for face swapping for long video
• Optimized using memory for retouch and remove object for long video

Update 1.6.1

• Fix bug with enhancer. Improve enhancer for video and face. Added enhancer for drawing video
• Imitate emotions in voice and improve voice cloning
• Music generation
• Adding a new tool for creating a user-drawable mask that is attached to the segmentation object and moves with it

Video

Review	How install on Windows?

Support the Project

You can support the author of the project in the development of his creative ideas, or just treat him to a cup of coffee in USD or a slice of pizza in RUB. There are other ways to support the development of the project, more details on page.

Buy a cup of coffee in USD	Buy a slice of pizza in RUB

Contact

Owner: Wladislav Radchenko

Email: [e-mail]

Project: https://github.com/wladradchenko/wunjo.wladradchenko.ru

Web site: wladradchenko.ru/wunjo

Premise

Wunjo comes from the ancient runic alphabet and represents joy and contentment, which could tie into the idea of using the application to create engaging and expressive speech. Vunyo (ᚹ) is the eighth rune of the Elder and Anglo-Saxon Futhark. Prior to the introduction of the letter W into the Latin alphabet, the letter Ƿynn (Ƿƿ) was used instead in English, derived from this rune.

Credits

• Tacatron 2 - https://github.com/NVIDIA/tacotron2
• Waveglow - https://github.com/NVIDIA/waveglow
• Flask UI - https://github.com/ClimenteA/flaskwebgui
• BeeWare - https://beeware.org/project/projects/tools/briefcase/
• Sad Talker - https://github.com/OpenTalker/SadTalker
• Wav2lip - https://github.com/Rudrabha/Wav2Lip
• Face Utils - https://github.com/xinntao/facexlib
• Face Enhancement - https://github.com/TencentARC/GFPGAN
• Image/Video Enhancement - https://github.com/xinntao/Real-ESRGAN
• Real-Time Voice Cloning - https://github.com/CorentinJ/Real-Time-Voice-Cloning
• Segment Anything - https://github.com/facebookresearch/segment-anything
• Rerender a Video - https://github.com/williamyang1991/Rerender_A_Video
• GMFlow - https://github.com/haofeixu/gmflow
• ControlNet - https://github.com/lllyasviel/ControlNet
• Stable Diffusion - https://github.com/Stability-AI/stablediffusion
• Ebsynth - https://github.com/jamriska/ebsynth

(to top)