lntzm

Awesome-Multimodal-Large-Language-Models

:sparkles::sparkles:Latest Advances on Multimodal Large Language Models

Otter

🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.

Vim

[ICML 2024] Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

Transformer-Explainability

[CVPR 2021] Official PyTorch implementation for Transformer Interpretability Beyond Attention Visualization, a novel method to visualize classifications by Transformer based networks.

VideoX

VideoX: a collection of video cross-modal models

VideoLLaMA2

VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs

PLLaVA

Official repository for the paper PLLaVA

MovieChat

[CVPR 2024] 🎬💭 chat with over 10K frames of video!

SAM-6D

[CVPR2024] Code for "SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation".

UniVTG

[ICCV2023] UniVTG: Towards Unified Video-Language Temporal Grounding

GroundingGPT

[ACL 2024] GroundingGPT: Language-Enhanced Multi-modal Grounding Model

DEADiff

[CVPR 2024] Official implementation of "DEADiff: An Efficient Stylization Diffusion Model with Disentangled Representations"

VTimeLLM

[CVPR'2024 Highlight] Official PyTorch implementation of the paper "VTimeLLM: Empower LLM to Grasp Video Moments".

QD-DETR

Official pytorch repository for "QD-DETR : Query-Dependent Video Representation for Moment Retrieval and Highlight Detection" (CVPR 2023 Paper)

Awesome_Long_Form_Video_Understanding

Awesome papers & datasets specifically focused on long-term videos.

TriDet

[CVPR2023] Code for the paper, TriDet: Temporal Action Detection with Relative Boundary Modeling

HBI

[CVPR 2023 Highlight] Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning

BACL

Balanced Classification: A Unified Framework for Long-Tailed Object Detection (TMM 2023)

ProST

Progressive Spatio-Temporal Prototype Matching for Text-Video Retrieval --ICCV2023 Oral

HiREST

Hierarchical Video-Moment Retrieval and Step-Captioning (CVPR 2023)

MomentDiff

MomentDiff: Generative Video Moment Retrieval from Random to Real--NeurIPS 2023

VideoTree

Code for paper "VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos"

ViGA

"Video Moment Retrieval from Text Queries via Single Frame Annotation" in SIGIR 2022.

DHVT

This is an official implementation of our NeurIPS 2022 paper "Bridging the Gap Between Vision Transformers and Convolutional Neural Networks on Small Datasets".

LocVTP

[ECCV 22] LocVTP: Video-Text Pre-training for Temporal Localization

LPV

The official code of Linguistic More: Taking a Further Step toward Efficient and Accurate Scene Text Recognition (IJCAI2023)

DenseEventsGrounding

MS-DETR

An official implementation for MS-DETR in ACL'23

SSM

[IJCAI-2024] The official code of Self-Supervised Pre-training with Symmetric Superimposition Modeling for Scene Text Recognition

Language-Enhanced-CLIP-For-Multi-label-Image-Recognition

3rd Place, Visual Prompt Tuning Challenge @ CVPR 2023 HIT Workshop (2023)