TGLTommy

StructEval

This is the office repository for ACL 2024 paper "StructEval: Deepen and Broaden Large Language Assessment via Structured Evaluation"

MedTrinity-25M

omages

We present Object Images (Omages): An homage to the classic Geometry Images.

MMIU

MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models

ExoViP

[COLM 2024] ExoViP: Step-by-step Verification and Exploration with Exoskeleton Modules for Compositional Visual Reasoning

fantastic-data-engineering

Fantastic Data Engineering for Large Language Models

Lumina-mGPT

Hallu-PI

The code and datasets of our ACM MM 2024 paper "Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed Inputs".

stable-fast-3d

flux

Official inference repo for FLUX.1 models

nano-llama31

nanoGPT style version of Llama 3.1

redel

ReDel is a toolkit for researchers and developers to build, iterate on, and analyze recursive multi-agent systems.

UnifiedMLLM

UnifiedMLLM: Enabling Unified Representation for Multi-modal Multi-tasks With Large Language Model

RAGFoundry

Framework for specializing LLMs for retrieval-augmented-generation tasks using fine-tuning.

POA

Official implementation of ECCV24 paper: POA

RagLLaVA

Official PyTorch Implementation of MLLM Is a Strong Reranker: Advancing Multimodal Retrieval-augmented Generation via Knowledge-enhanced Reranking and Noise-injected Training.

TEVAD

Official implementation for paper TEVAD: Improved video anomaly detection with captions

R2D2

SwinBERT

Research code for CVPR 2022 paper "SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning"

CoCap

[ICCV 2023] Accurate and Fast Compressed Video Captioning

segment-anything-2

The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

GalleryGPT

MHR

MovieSeq

[ECCV2024] Learning Video Context as Interleaved Multimodal Sequences

DQU-CIR

[SIGIR 2024] - Simple but Effective Raw-Data Level Multimodal Fusion for Composed Image Retrieval

SynthVLM

theia

KIVI

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

XHand

Official pytorch implementation of "XHand: Real-time Expressive Hand Avatar"

AdaCLIP

This repository contains the code for AdaCLIP, a computation and latency-aware system for pragmatic multimodal video retrieval.