JosephPai

VidProM

[NeurIPS 2024] VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Models

VideoLISA

[NeurlPS 2024] One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos

Awesome-Unified-Multimodal-Models

📖 This is a repository for organizing papers, codes and other resources related to unified multimodal models.

AdaSlot

Official implementation of the CVPR'24 paper [Adaptive Slot Attention: Object Discovery with Dynamic Slot Number]

Show-o

Repository for Show-o, One Single Transformer to Unify Multimodal Understanding and Generation.

AI-Scientist

The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery 🧑‍🔬

MINT-1T

MINT-1T: A one trillion token multimodal interleaved dataset.

sam2

The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

SOLO

Public code repo for paper "A Single Transformer for Scalable Vision-Language Modeling"

fucking-algorithm

刷算法全靠套路，认准 labuladong 就够了！English version supported! Crack LeetCode, not only how, but also why.

Maskgit-pytorch

ShiArthur03

SliME

✨✨Beyond LLaVA-HD: Diving into High-Resolution Large Multimodal Models

enhancing-transformers

An unofficial implementation of both ViT-VQGAN and RQ-VAE in Pytorch

cambrian

Cambrian-1 is a family of multimodal LLMs with a vision-centric design.

chameleon

Repository for Meta Chameleon, a mixed-modal early-fusion foundation model from FAIR.

Open-MAGVIT2

Open-MAGVIT2: Democratizing Autoregressive Visual Generation

lmms-eval

Accelerating the development of large multimodal models (LMMs) with lmms-eval

Awesome-World-Model

Collect some World Models for Autonomous Driving papers.

gpt-computer-assistant

Intelligence development framework in python for your product like Apple Intelligence

schedule_free

Schedule-Free Optimization in PyTorch

World-Models-Autonomous-Driving-Latest-Survey

A curated list of world models for autonomous driving. Keep updated.

supervision

We write your reusable computer vision tools. 💜

Lumina-T2X

Lumina-T2X is a unified framework for Text to Any Modality Generation

HALC

[ICML 2024] Official implementation for "HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding"

Awesome-MLLM-Hallucination

📖 A curated list of resources dedicated to hallucination of multimodal large language models (MLLM).

LLaVA-pp

🔥🔥 LLaVA++: Extending LLaVA with Phi-3 and LLaMA-3 (LLaVA LLaMA-3, LLaVA Phi-3)

XMem2

A tool for efficient semi-supervised video object segmentation (great results with minimal manual labor) and a dataset for benchmarking

Tracking-Anything-with-DEVA

[ICCV 2023] Tracking Anything with Decoupled Video Segmentation

mergekit

Tools for merging pretrained large language models.