MinghanLi

LAVIS

LAVIS - A One-stop Library for Language-Vision Intelligence

moment_detr

[NeurIPS 2021] Moment-DETR code and QVHighlights dataset

direct-preference-optimization

Reference implementation for DPO (Direct Preference Optimization)

deep-residual-networks

Deep Residual Learning for Image Recognition

OmniDrive

open-images-dataset

Open Images is a dataset of ~9 million images that have been annotated with image-level labels and bounding boxes spanning thousands of classes.

conceptual-12m

Conceptual 12M is a dataset containing (image-URL, caption) pairs collected for vision-and-language pre-training.

cld3

localized-narratives

Localized Narratives

VideoCrafter

VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models

LongVA

Long Context Transfer from Language to Vision

LLaVA-NeXT

TokenPacker

The code for "TokenPacker: Efficient Visual Projector for Multimodal LLM".

Video-MME

✨✨Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis

Video-Bench

A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models!

MMDU

Official repository of MMDU dataset

ShareGPT4Video

An official implementation of ShareGPT4Video: Improving Video Understanding and Generation with Better Captions

Valley

The official repository of "Video assistant towards large language model makes everything easy"

PLLaVA

Official repository for the paper PLLaVA

matmulfreellm

Implementation for MatMul-free LM.

Chinese-LLaMA-Alpaca

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

OmniVid

video-question-answering

Video Question Answering via Gradually Refined Attention over Appearance and Motion

Video-guided-Machine-Translation

Starter code for the VMT task and challenge

MiniCPM-V

MiniCPM-V 2.6: A GPT-4V Level MLLM for Single Image, Multi Image and Video on Your Phone

pycocoevalcap

Python 3 support for the MS COCO caption evaluation tools

coco-caption

ClipBERT

[CVPR 2021 Best Student Paper Honorable Mention, Oral] Official PyTorch code for ClipBERT, an efficient framework for end-to-end learning on image-text and video-text tasks.

MambaOut

MambaOut: Do We Really Need Mamba for Vision?

youtube-dl

Command-line program to download videos from YouTube.com and other video sites