Maoshuiyang

VITA

✨✨VITA: Towards Open-Source Interactive Omni Multimodal LLM

audiocraft

Audiocraft is a library for audio processing and generation with deep learning. It features the state-of-the-art EnCodec audio compressor / tokenizer, along with MusicGen, a simple and controllable music generation LM with textual and melodic conditioning.

Amphion

Amphion (/æmˈfaɪən/) is a toolkit for Audio, Music, and Speech Generation. Its purpose is to support reproducible research and help junior researchers and engineers get started in the field of audio, music, and speech generation research and development.

naturalspeech3_facodec

FACodec: Speech Codec with Attribute Factorization used for NaturalSpeech 3

seed-tts-eval

WenetSpeechSpeakerCluster

RepCodec

Models and code for RepCodec: A Speech Representation Codec for Speech Tokenization

Speech-Editing-Toolkit

It's a repository for implementations of neural speech editing algorithms.

open-speech-corpora

💎 A list of accessible speech corpora for ASR, TTS, and other Speech Technologies

trl

Train transformer language models with reinforcement learning.

litgpt

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

Make-A-Scene

Pytorch implementation of Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors

SpeechT5

Unified-Modal Speech-Text Pre-Training for Spoken Language Processing

pyllama

LLaMA: Open and Efficient Foundation Language Models

SpeechGen

《SpeechGen: Unlocking the Generative Power of Speech Language Models with Prompts》

TTS-TextAnalyzer

TTS Text Analyzer

Text-to-sound-Synthesis

The source code of our paper "Diffsound: discrete diffusion model for text-to-sound generation"

lyra

A Very Low-Bitrate Codec for Speech Compression

chinese_speech_pretrain

chinese speech pretrained models

vocos

Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis

VQ-Diffusion

g2p-kd

Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversion

phonemizer

Simple text to phones converter for multiple languages

SoundStorm

The reproduced code for Google's SoundStorm

AcademiCodec

AcademiCodec: An Open Source Audio Codec Model for Academic Research

NaturalSpeech2

Meta-voicebox

Implementation of Meta-Voicebox : The first generative AI model for speech to generalize across tasks with state-of-the-art performance.

naturalspeech2-pytorch

Implementation of Natural Speech 2, Zero-shot Speech and Singing Synthesizer, in Pytorch

vall-e

PyTorch implementation of VALL-E(Zero-Shot Text-To-Speech), Reproduced Demo https://lifeiteng.github.io/valle/index.html

bark

🔊 Text-Prompted Generative Audio Model