DachengLi1

ring-attention-pytorch

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

rectified-flow-pytorch

Implementation of rectified flow and some of its followup research / improvements in Pytorch

SEED-Story

SEED-Story: Multimodal Long Story Generation with Large Language Model

flute

Fast Matrix Multiplications for Lookup Table-Quantized LLMs

Video-MME

✨✨Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis

big_vision

Official codebase used to develop Vision Transformer, SigLIP, MLP-Mixer, LiT and more.

LivePortrait

Bring portraits to life!

VAR

[GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ultra-simple, user-friendly yet state-of-the-art* codebase for autoregressive image generation!

retrieval-scaling

Official repository for "Scaling Retrieval-Based Langauge Models with a Trillion-Token Datastore".

GRUtopia

GRUtopia: Dream General Robots in a City at Scale

Video-ChatGPT

[ACL 2024 🔥] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted for spatiotemporal video representation. We also introduce a rigorous 'Quantitative Evaluation Benchmarking' for video-based conversational models.

WildAvatar_Toolbox

[ArXiv 2024] WildAvatar: Web-scale In-the-wild Video Dataset for 3D Avatar Creation

VEnhancer

qserve

QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving

VADER

Video Diffusion Alignment via Reward Gradients. We improve a variety of video diffusion models such as VideoCrafter, OpenSora, ModelScope and StableVideoDiffusion by finetuning them using various reward models such as HPS, PickScore, VideoMAE, VJEPA, YOLO, Aesthetics etc.

MPS

prm800k

800,000 step-level correctness labels on LLM solutions to MATH problems

LongLoRA

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

FreeInit

[ECCV 2024] FreeInit: Bridging Initialization Gap in Video Diffusion Models

PixArt-sigma

PixArt-Σ: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation

HPSv2

Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis

unsloth

Finetune Llama 3, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory

RouteLLM

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality!

diffusion-forcing

code for "Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion"

OpenVid-1M

IRASim

q-diffusion

[ICCV 2023] Q-Diffusion: Quantizing Diffusion Models.

delta-iris

Efficient World Models with Context-Aware Tokenization. ICML 2024

VBench

[CVPR2024 Highlight] VBench - We Evaluate Video Generation

PipeFusion

A Suite for Parallel Inference of Diffusion Transformers (DiTs) on multi-GPU Clusters