whatissimondoing

TaskMatrix

ChatTTS

A generative speech model for daily dialogue.

LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

ragflow

RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.

fish-speech

Brand new TTS solution

gorilla

Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)

livekit

End-to-end stack for WebRTC. SFU media server and SDKs.

BBDown

Bilibili Downloader. 一款命令行式哔哩哔哩下载器.

sglang

SGLang is a fast serving framework for large language models and vision language models.

AniPortrait

AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation

Amphion

Amphion (/æmˈfaɪən/) is a toolkit for Audio, Music, and Speech Generation. Its purpose is to support reproducible research and help junior researchers and engineers get started in the field of audio, music, and speech generation research and development.

xtuner

An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)

V-Express

V-Express aims to generate a talking head video under the control of a reference image, an audio, and a sequence of V-Kps images.

OS-Copilot

An self-improving embodied conversational agent seamlessly integrated into the operating system to automate our daily tasks.

distilabel

Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verified research papers.

Bunny

A family of lightweight multimodal models.

bm25s

Fast lexical search library implementing BM25 in Python using Numpy and Scipy

EmoLLM

心理健康大模型、LLM、The Big Model of Mental Health、Finetune、InternLM2、InternLM2.5、Qwen、ChatGLM、Baichuan、DeepSeek、Mixtral、LLama3、GLM4、Qwen2、LLama3.1

UMOE-Scaling-Unified-Multimodal-LLMs

The codes about "Uni-MoE: Scaling Unified Multimodal Models with Mixture of Experts"

MInference

To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

emotion2vec

[ACL 2024] Official PyTorch code for extracting features and training downstream models with emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation

LLaSM

第一个支持中英文双语语音-文本多模态对话的开源可商用对话模型。便捷的语音输入将大幅改善以文本为输入的大模型的使用体验，同时避免了基于 ASR 解决方案的繁琐流程以及可能引入的错误。

SLAM-LLM

Speech, Language, Audio, Music Processing with Large Language Model

Vach

Real time streaming talking head

QiaoBan

SECap

EmoBox

[INTERSPEECH 2024] EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmark

xRAG

Source code for xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token

Flash-VStream

Please refer to our official repo at https://github.com/IVGSZ/Flash-VStream.

EMO-SUPERB-submission

EMO-SUPERB submission