Yuezeyi

InternVL

[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. 接近GPT-4o表现的开源多模态对话模型

SenseVoice

Multilingual Voice Understanding Model

LLaVA-NeXT

Qwen-Audio

The official repo of Qwen-Audio (通义千问-Audio) chat & pretrained large audio language model proposed by Alibaba Cloud.

lmms-eval

Accelerating the development of large multimodal models (LMMs) with lmms-eval

Qwen2-Audio

The official repo of Qwen2-Audio chat & pretrained large audio language model proposed by Alibaba Cloud.

LLaVA-pp

🔥🔥 LLaVA++: Extending LLaVA with Phi-3 and LLaMA-3 (LLaVA LLaMA-3, LLaVA Phi-3)

AnyGPT

Code for "AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling"

anole

Anole: An Open, Autoregressive and Native Multimodal Models for Interleaved Image-Text Generation

Groma

[ECCV2024] Grounded Multimodal Large Language Model with Localized Visual Tokenization

tevatron

Tevatron - A flexible toolkit for neural retrieval research and development.

honeybee

Official implementation of project Honeybee (CVPR 2024)

SpeechTokenizer

This is the code for the SpeechTokenizer presented in the SpeechTokenizer: Unified Speech Tokenizer for Speech Language Models. Samples are presented on

SEED-X

Multimodal Models in Real World

scaling_on_scales

When do we not need larger vision models?

Vitron

A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing

animate-your-word

Official implementations for paper: Dynamic Typography: Bringing Text to Life via Video Diffusion Prior

MotionLLM

[Arxiv-2024] MotionLLM: Understanding Human Behaviors from Human Motions and Videos

RLAIF-V

RLAIF-V: Aligning MLLMs through Open-Source AI Feedback for Super GPT-4V Trustworthiness

imp

a family of highly capabale yet efficient large multimodal models

TokenPacker

The code for "TokenPacker: Efficient Visual Projector for Multimodal LLM".

MuLan

MuLan: Adapting Multilingual Diffusion Models for 110+ Languages (无需额外训练为任意扩散模型支持多语言能力)

MultiBooth

[arXiv 2024] MultiBooth: This repo is the official implementation of "MultiBooth: Towards Generating All Your Concepts in an Image from Text"

VL-InterpreT

Visual Language Transformer Interpreter - An interactive visualization tool for interpreting vision-language transformers

XVERSE-V-13B

MiCo

Explore the Limits of Omni-modal Pretraining at Scale

unified-io-2.pytorch

Koala-video-llm

lvlm-interpret

diffusion-model-hallucination