linyueqian

VideoLISA

[NeurlPS 2024] One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos

stanford_alpaca

Code and documentation to train Stanford's Alpaca models, and generate the data.

LLaVA-PruMerge

LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models

CosyVoice

Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.

moshi

Bay-CAT

[ECCV’24] Official Implementation for CAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual Scenarios

unified-io-2

NExT-GPT

Code and models for NExT-GPT: Any-to-Any Multimodal Large Language Model

LLaMA-Omni

LLaMA-Omni is a low-latency and high-quality end-to-end speech interaction model built upon Llama-3.1-8B-Instruct, aiming to achieve speech capabilities at the GPT-4o level.

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Call-for-Reviewers

This project aims to collect the latest "call for reviewers" links from various top CS/ML/AI conferences/journals

LinFusion

Official PyTorch and Diffusers Implementation of "LinFusion: 1 GPU, 1 Minute, 16K Image"

VCD

[CVPR 2024 Highlight] Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding

SpeculativeDecodingPapers

📰 Must-read papers and blogs on Speculative Decoding ⚡️

UTMOS22

UT-Sarulab MOS prediction system using SSL models

WavTokenizer

SOTA discrete acoustic codec models with 40 tokens per second for audio language modeling

Show-o

Repository for Show-o, One Single Transformer to Unify Multimodal Understanding and Generation.

Awesome-Unified-Multimodal-Models

📖 This is a repository for organizing papers, codes and other resources related to unified multimodal models.

RoFormer_pytorch

RoFormer V1 & V2 pytorch

llama-recipes

Scripts for fine-tuning Meta Llama with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama for WhatsApp & Messenger.

llama-stack

Model components of the Llama Stack APIs

llama-models

Utilities intended for use with Llama models.

modin

Modin: Scale your Pandas workflows by changing a single line of code

kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.

AI-System-School

🚀 AI System Papers and Industry Practice. ⚡️ System for Machine Learning, LLM (Large Language Model), GenAI (Generative AI). 🍻 OSDI, NSDI, SIGCOMM, SoCC, MLSys, etc. 🗃️ Llama3, Mistral, etc. 🧑‍💻 Video Tutorials.

LLaMA-Factory

Efficiently Fine-Tune 100+ LLMs in WebUI (ACL 2024)

homeassistant-smartrent

Home Assistant Custom Component for SmartRent Locks 🔐, Thermostats 🌡, Sensors 💧 and Switches💡

icloud_photos_downloader

A command-line tool to download photos from iCloud

lmms-finetune

A minimal codebase for finetuning large multimodal models, supporting llava-1.5/1.6, llava-interleave, llava-next-video, qwen-vl, phi3-v etc.

Awesome-OOD-VLM

Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey [Miyai+, arXiv2024]