canqin001

UniLMMV

switch-cuda

A simple bash script for switching between installed versions of CUDA.

Video-Dataset-Loading-Pytorch

Generic PyTorch dataset implementation to load and augment VIDEOS for deep learning training loops.

llama.cpp

LLM inference in C/C++

Online-RLHF

A recipe for online RLHF.

scaling_on_scales

When do we not need larger vision models?

MiniGPT4-video

Official code for Goldfish model for long video understanding and MiniGPT4-video for short video understanding

DoRA

[ICML2024 (Oral)] Official PyTorch implementation of DoRA: Weight-Decomposed Low-Rank Adaptation

SoM-LLaVA

[COLM-2024] List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMs

emoji-cheat-sheet

A markdown version emoji cheat sheet

llama3

The official Meta Llama 3 GitHub site

T2I-CompBench

[Neurips 2023] T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional Text-to-image Generation

LLaMA2-Accessory

An Open-source Toolkit for LLM Development

PixArt-alpha

PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis

SQ-LLaVA

Visual self-questioning for large vision-language assistant.

Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

Maskgit-pytorch

vector-quantize-pytorch

Vector (and Scalar) Quantization, in Pytorch

Latte

Latte: Latent Diffusion Transformer for Video Generation.

DiT

Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

Video-ChatGPT

[ACL 2024 🔥] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted for spatiotemporal video representation. We also introduce a rigorous 'Quantitative Evaluation Benchmarking' for video-based conversational models.

DiffSynth-Studio

Enjoy the magic of Diffusion models!

VBench

[CVPR2024 Highlight] VBench - We Evaluate Video Generation

AIGCBench

Official repo for AIGCBench: Comprehensive Evaluation of Image-to-Video Content Generated by AI

decord

An efficient video loader for deep learning with smart shuffling that's super easy to digest

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Awesome-Multimodal-Large-Language-Models

:sparkles::sparkles:Latest Advances on Multimodal Large Language Models

MMVP

DragNUWA