wh0330

MA-LMM

(2024CVPR) MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding

VTimeLLM

[CVPR'2024 Highlight] Official PyTorch implementation of the paper "VTimeLLM: Empower LLM to Grasp Video Moments".

Awesome-Multimodal-Large-Language-Models

:sparkles::sparkles:Latest Papers and Datasets on Multimodal Large Language Models, and Their Evaluation.

AppAgent

AppAgent: Multimodal Agents as Smartphone Users, an LLM-based multimodal agent framework designed to operate smartphone apps.

VGT

Video Graph Transformer for Video Question Answering (ECCV'22)

PDVC

End-to-End Dense Video Captioning with Parallel Decoding (ICCV 2021)

hgr_v2t

Code accompanying the paper "Fine-grained Video-Text Retrieval with Hierarchical Graph Reasoning".

ClipBERT

[CVPR 2021 Best Student Paper Honorable Mention, Oral] Official PyTorch code for ClipBERT, an efficient framework for end-to-end learning on image-text and video-text tasks.

bottom-up-attention.pytorch

A PyTorch reimplementation of bottom-up-attention models

SceneGraphParser

A python toolkit for parsing captions (in natural language) into scene graphs (as symbolic representations).

HRNAT

Hierarchical Representation Network with AuxiliaryTasks for Video Captioning and Video QuestionAnswering

Graph-Optimal-Transport

Code for ICML 2020 "Graph Optimal Transport for Cross-Domain Alignment"

SUTD-TrafficQA

[CVPR2021] SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Events

vRGV

Visual Relation Grounding in Videos (ECCV'20, Spotlight)

VidHOI

Official implementation of "ST-HOI: A Spatial-Temporal Baseline for Human-Object Interaction Detection in Videos" (ACM ICMRW 2021)

NExT-QA

NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions (CVPR'21)

cheatsheets

Official Matplotlib cheat sheets

FINCH-Clustering

Source Code for FINCH Clustering Algorithm

mmt

Multi-Modal Transformer for Video Retrieval

Mixture-of-Embedding-Experts

Mixture-of-Embeddings-Experts

NeXtVLAD.pytorch

Pytorch implementation of NetVlad for classification on UCF101

DRN

Dense Regression Network for Video Grounding (CVPR2020)

CCL

PyTorch Implementation on Paper [CVPR2021]Distilling Audio-Visual Knowledge by Compositional Contrastive Learning

CrossViT-pytorch

Implementation of CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification

CVPR-2021-Papers

SimCSE

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821

Multi-HT100M

PoseFormer

The project is an official implementation of our paper "3D Human Pose Estimation with Spatial and Temporal Transformers".

BiST

Code for the paper BiST: Bi-directional Spatio-Temporal Reasoning for Video-Grounded Dialogues (EMNLP20)

visdial_conv

This repository contains code used in our ACL'20 paper History for Visual Dialog: Do we really need it?