i-MaTh

GPT-SoVITS

1 min voice data can also be used to train a good TTS model! (few shot voice cloning)

ControlNet

Let us control diffusion models!

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Open-Sora-Plan

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.

whisperX

WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization)

StreamDiffusion

StreamDiffusion: A Pipeline-Level Solution for Real-Time Interactive Generation

minbpe

Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.

Depth-Anything

[CVPR 2024] Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data. Foundation Model for Monocular Depth Estimation

streaming-llm

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

instruct-pix2pix

gpt-fast

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

cleanrl

High-quality single file implementation of Deep Reinforcement Learning algorithms with research-friendly features (PPO, DQN, C51, DDPG, TD3, SAC, PPG)

Amphion

Amphion (/æmˈfaɪən/) is a toolkit for Audio, Music, and Speech Generation. Its purpose is to support reproducible research and help junior researchers and engineers get started in the field of audio, music, and speech generation research and development.

latent-consistency-model

Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference

metavoice-src

Foundational model for human-like, expressive TTS

stable-audio-tools

Generative models for conditional audio generation

AudioLDM2

Text-to-Audio/Music Generation

OpenDiT

OpenDiT: An Easy, Fast and Memory-Efficient System for DiT Training and Inference

voicebox-pytorch

Implementation of Voicebox, new SOTA Text-to-speech network from MetaAI, in Pytorch

SiT

Official PyTorch Implementation of "SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers"

UniSpeech

UniSpeech - Large Scale Self-Supervised Learning for Speech

SpeechTokenizer

This is the code for the SpeechTokenizer presented in the SpeechTokenizer: Unified Speech Tokenizer for Speech Language Models. Samples are presented on

MiniGPT4Qwen

Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train your own 8B/14B LLaVA-training-like MLLM on RTX3090/4090 24GB.

spear-tts-pytorch

Implementation of Spear-TTS - multi-speaker text-to-speech attention network, in Pytorch

tts-scores

Scripts for computing the Intelligibility and CLVP scores for evaluating TTS models

stable-audio-metrics

Metrics for evaluating music and audio generative models – with a focus on long-form, full-band, and stereo generations.

UniCATS-CTX-vec2wav

[AAAI 2024] Code for CTX-vec2wav in UniCATS

agc

Audiogen Codec

whisper-punctuator

Zero-shot multimodal punctuation insertion and truecasing using Whisper

OpenSora

Exquisite video generation