WasedaMagina

VILA

VILA - A multi-image visual language model with training, inference and evaluation recipe, deployable from cloud to edge (Jetson Orin and laptops)

ml-4m

4M: Massively Multimodal Masked Modeling

LlamaGen

Autoregressive Model Beats Diffusion: 🦙 Llama for Scalable Image Generation

LWM

taming-transformers

Taming Transformers for High-Resolution Image Synthesis

MotionLLM

[Arxiv-2024] MotionLLM: Understanding Human Behaviors from Human Motions and Videos

enhancing-transformers

An unofficial implementation of both ViT-VQGAN and RQ-VAE in Pytorch

Emu

Emu Series: Generative Multimodal Models from BAAI

llama3-from-scratch

llama3 implementation one matrix multiplication at a time

MiraData

SuperCLUE-Role

SuperCLUE-Role中文原生角色扮演测评基准

ffmpeg-build-script

The FFmpeg build script provides an easy way to build a static FFmpeg on OSX and Linux with non-free codecs included.

TransNetV2

TransNet V2: Shot Boundary Detection Neural Network

MathVerse

Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

xtuner

An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)

MultiInstruct

MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuning

Awesome-Multimodal-Large-Language-Models

:sparkles::sparkles:Latest Advances on Multimodal Large Language Models

self-instruct

Aligning pretrained language models with instruction data generated by themselves.

LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

StableLLAVA

Official repo for StableLLAVA

LaVIN

[NeurIPS 2023] Official implementations of "Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Models"

DECOLA

Code release for "Language-conditioned Detection Transformer"

Awesome-Parameter-Efficient-Transfer-Learning

A collection of parameter-efficient transfer learning papers focusing on computer vision and multimodal domains.

peft

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

VLM_survey

Collection of AWESOME vision-language models for vision tasks

decision-transformer

Official codebase for Decision Transformer: Reinforcement Learning via Sequence Modeling.

FlagEmbedding

Retrieval and Retrieval-augmented LLMs

LURE

[ICLR 2024] Analyzing and Mitigating Object Hallucination in Large Vision-Language Models

FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.