raoyongming

cambrian

Cambrian-1 is a family of multimodal LLMs with a vision-centric design.

MM-NIAH

This is the official implementation of the paper "Needle In A Multimodal Haystack"

PLLaVA

Official repository for the paper PLLaVA

Lumina-T2X

Lumina-T2X is a unified framework for Text to Any Modality Generation

ShareGPT4Video

An official implementation of ShareGPT4Video: Improving Video Understanding and Generation with Better Captions

Omost

Your image is almost there!

flash-linear-attention

Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton

LaViLa

Code release for "Learning Video Representations from Large Language Models"

MiniCPM-V

MiniCPM-Llama3-V 2.5: A GPT-4V Level Multimodal LLM on Your Phone

HallusionBench

[CVPR'24] HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models

HunyuanDiT

Hunyuan-DiT : A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding

LLaVA-NeXT

DeepSeek-V2

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

RADIO

Official repository for "AM-RADIO: Reduce All Domains Into One"

HPT

HPT - Open Multimodal LLMs from HyperGAI

G-LLaVA

Official github repo of G-LLaVA

MGM

Official repo for "Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models"

VAR

[GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ultra-simple, user-friendly yet state-of-the-art* codebase for autoregressive image generation!

FeatUp

Official code for "FeatUp: A Model-Agnostic Frameworkfor Features at Any Resolution" ICLR 2024

SpeeD

SpeeD: A Closer Look at Time Steps is Worthy of Triple Speed-Up for Diffusion Model Training

LLaVA-RLHF

Aligning LMMs with Factually Augmented RLHF

dust3r

DUSt3R: Geometric 3D Vision Made Easy

VILA

VILA - a multi-image visual language model with training, inference and evaluation recipe, deployable from cloud to edge (Jetson Orin and laptops)

Chain-of-Spot

Chain-of-Spot: Interactive Reasoning Improves Large Vision-language Models

Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

CapsFusion

[CVPR 2024] CapsFusion: Rethinking Image-Text Data at Scale

grok-1

Grok open release

VQASynth

Compose multimodal datasets 🎹

DeepSeek-VL

DeepSeek-VL: Towards Real-World Vision-Language Understanding

LLaMA2-Accessory

An Open-source Toolkit for LLM Development