LuckerYi

Interpret_Instruction_Tuning_LLMs

Understanding Why and How Instruction Tuning Changes Pre-trained Models

videollm-online

VideoLLM-online: Online Video Large Language Model for Streaming Video (CVPR 2024)

LibriTTS-P

LibriTTS-P: A Corpus with Speaking Style and Speaker Identity Prompts for Text-to-Speech and Style Captioning

emphassess

This repository presents an evaluation framework for speech-to-speech (S2S) models, following the methodology described in the EmphAsses paper (de Seyssel et al., 2023).

Token-level-Direct-Preference-Optimization

Reference implementation for Token-level Direct Preference Optimization(TDPO)

Awesome-Diffusion-Models

A collection of resources and papers on Diffusion Models

diffusers

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.

DiT

Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

GPT-SoVITS

1 min voice data can also be used to train a good TTS model! (few shot voice cloning)

best-rq-pytorch

Implementation of BEST-RQ - a model for self-supervised learning of speech signals using a random projection quantizer, in Pytorch.

SECap

CLAP

Learning audio concepts from natural language supervision

emotion2vec

[ACL 2024] Official PyTorch code for extracting features and training downstream models with emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation

PL-BERT

Phoneme-Level BERT for Enhanced Prosody of Text-to-Speech with Grapheme Predictions

tortoise-tts

A multi-voice TTS system trained with an emphasis on quality

torchscale

Foundation Architecture for (M)LLMs

normalizing-flows

PyTorch implementation of normalizing flow models

MM-Diffusion

[CVPR'23] MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation

normalizing_flows

Pytorch implementations of density estimation algorithms: BNAF, Glow, MAF, RealNVP, planar flows

glow-tts

A Generative Flow for Text-to-Speech via Monotonic Alignment Search

glow

Code for reproducing results in "Glow: Generative Flow with Invertible 1x1 Convolutions"

Diffusion-SVC

speechbrain

A PyTorch-based Speech Toolkit

so-vits-svc

SoftVC VITS Singing Voice Conversion

torchdiffeq

Differentiable ODE solvers with full GPU support and O(1)-memory backpropagation.

Meta-voicebox

Implementation of Meta-Voicebox : The first generative AI model for speech to generalize across tasks with state-of-the-art performance.

VISinger2

VISinger 2: High-Fidelity End-to-End Singing Voice Synthesis Enhanced by Digital Signal Processing Synthesizer

ControlNet

Let us control diffusion models!

stable-diffusion-webui

Stable Diffusion web UI

HTS-Audio-Transformer

The official code repo of "HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection"