mikeluck

pipecat

Open Source framework for voice and multimodal conversational AI

sglang

SGLang is yet another fast serving framework for large language models and vision language models.

SpeechGPT

SpeechGPT Series: Speech Large Language Models

descript-audio-codec

State-of-the-art audio codec with 90x compression factor. Supports 44.1kHz, 24kHz, and 16kHz mono/stereo audio.

audiocraft

Audiocraft is a library for audio processing and generation with deep learning. It features the state-of-the-art EnCodec audio compressor / tokenizer, along with MusicGen, a simple and controllable music generation LM with textual and melodic conditioning.

build-nanogpt

Video+code lecture on building nanoGPT from scratch

LLM101n

LLM101n: Let's build a Storyteller

MobileLLM

MobileLLM Optimizing Sub-billion Parameter Language Models for On-Device Use Cases. In ICML 2024.

AudioNotes

快速提取音视频内容，整理成一份结构化的markdown笔记

mistral-finetune

open_lm

A repository for research on medium sized language models.

UltraPixel

Implementation of UltraPixel: Advancing Ultra-High-Resolution Image Synthesis to New Peaks

florence2-finetuning

Quick exploration into fine tuning florence 2

hammal

docker-registry proxy run in cloudflare workers

hammal

docker-registry proxy run in cloudflare workers

vision-llms-are-blind

CLIP

CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image

OneLLM

[CVPR 2024] OneLLM: One Framework to Align All Modalities with Language

LLaMA-Adapter

[ICLR 2024] Fine-tuning LLaMA to follow Instructions within 1 Hour and 1.2M Parameters

Awesome-Multimodal-Large-Language-Models

:sparkles::sparkles:Latest Advances on Multimodal Large Language Models

CosyVoice

Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.

SenseVoice

Multilingual Voice Understanding Model

graphrag

A modular graph-based Retrieval-Augmented Generation (RAG) system

KG-LLM-MDQA

cloudflare-docker-proxy

A docker registry proxy run on cloudflare worker.

LAS

LAS Specification

LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

python-markdownify

Convert HTML to Markdown

RAGatouille

Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline. Designed for modularity and ease-of-use, backed by research.

piccolo-embedding

code for piccolo embedding model from SenseTime