jihanyang

Qwen2

Qwen2 is the large language model series developed by Qwen team, Alibaba Cloud.

EgoSchema

IG-VLM

ALLaVA

Harnessing 1.4M GPT4V-synthesized Data for A Lite Vision-Language Model

clip-beyond-tail

Generalization Beyond Data Imbalance: A Controlled Study on CLIP for Transferable Insights

VideoTree

Code for paper "VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos"

LLoVi

Official implementation for "A Simple LLM Framework for Long-Range Video Question-Answering"

DeepSeek-VL

DeepSeek-VL: Towards Real-World Vision-Language Understanding

LanguageBind

【ICLR 2024🔥】 Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment

OpenGlass

Turn any glasses into AI-powered smart glasses

VQASynth

Compose multimodal datasets 🎹

Video-ChatGPT

[ACL 2024 🔥] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted for spatiotemporal video representation. We also introduce a rigorous 'Quantitative Evaluation Benchmarking' for video-based conversational models.

visualwebarena

VisualWebArena is a benchmark for multimodal agents.

mmc4

MultimodalC4 is a multimodal extension of c4 that interleaves millions of images with text.

Kosmos-G

Official Implementation of ICLR'24: Kosmos-G: Generating Images in Context with Multimodal Large Language Models

Groma

Grounded Multimodal Large Language Model with Localized Visual Tokenization

llama3

The official Meta Llama 3 GitHub site

nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.

open-eqa

OpenEQA Embodied Question Answering in the Era of Foundation Models

eai-vc

The repository for the largest and most comprehensive empirical study of visual foundation models for Embodied AI (EAI).

datacomp

DataComp: In search of the next generation of multimodal datasets

paperbot

PaperBot: Learning to Design Real-World Tools Using Paper

VAR

[GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ultra-simple, user-friendly yet state-of-the-art* codebase for autoregressive image generation!

ffcv

FFCV: Fast Forward Computer Vision (and other ML workloads!)

VLN-CE

Vision-and-Language Navigation in Continuous Environments using Habitat

Total-Decom

GiT

Official Implementation of "GiT: Towards Generalist Vision Transformer through Universal Language Interface"

3D-LR

Can 3D Vision-Language Models Truly Understand Natural Language?

grok-1

Grok open release

gemma_pytorch

The official PyTorch implementation of Google's Gemma models