ChocoWu

scene_graph_commonsense

This is the official implementation of the paper "Enhancing Scene Graph Generation with Hierarchical Relationships and Commonsense Knowledge" in PyTorch.

Awesome-Scene-Graph-for-CrossModal-Learning

This is a repository for listing papers on scene graph generation and application.

VSD

Code for "Visual Spatial Description: Controlled Spatial-Oriented Image-to-Text Generation"

SymbCoT

Codes and Data for ACL 2024 Paper "Faithful Logical Reasoning via Symbolic Chain-of-Thought".

path2generalist.github.io

Path to Multimodal Generalist: Level, Benchmark and Model

EmpathyEar

Multimodal Empathetic Chatbot

SEED-X

Multimodal Models in Real World

HiKER-SGG

Code for HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generation (CVPR 2024)

IntrinsicHallu

NExT-GPT

Code and models for NExT-GPT: Any-to-Any Multimodal Large Language Model

coconut_cvpr2024

torch_kmeans

PyTorch implementations of KMeans, Soft-KMeans and Constrained-KMeans which can be run on GPU and work on (mini-)batches of data.

Vitron

A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing

NExT-Chat

The code of the paper "NExT-Chat: An LMM for Chat, Detection and Segmentation".

DiaASQ

ACL 2023 (Findings) : DiaASQ: A Benchmark of Conversational Aspect-based Sentiment Quadruple Analysis

instruct-pix2pix

NExT-QA

NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions (CVPR'21)

GLIGEN

Open-Set Grounded Text-to-Image Generation

FactualSceneGraph

FACTUAL benchmark dataset, the pre-trained textual scene graph parser trained on FACTUAL.

LayoutLLM-T2I

DisNER-PtrNet

Codes for the AAAI 2021 paper Rethinking Boundaries: End-To-End Recognition of Discontinuous Mentions with Pointer Networks.

THOR-ISA

Codes for ACL 2023 paper: Reasoning Implicit Sentiment with Chain-of-Thought Prompting

StruMatchDL

Codes for ICML 2022 paper: Matching Structure for Dual Learning

mmre-page

DiaRE-D2G

Codes of of the IJCAI 2022 Paper Global Inference with Explicit Syntactic and Discourse Structures for Dialogue-Level Relation Extraction

mmc4

MultimodalC4 is a multimodal extension of c4 that interleaves millions of images with text.

APTM

The official code of "Towards Unified Text-based Person Retrieval: A Large-scale Multi-Attribute and Language Search Benchmark"

NExT-GPT.github.io

NExT-GPT: Any-to-Any Multimodal Large Language Model

PENET

[CVPR 2023]Official Pytorch code for paper "Prototype-based Embedding Network for Scene Graph Generation"

MRE-ISE

About Codes for ACL 2023 paper: Exploiting! Multimodal Relation Extraction with Feature Denoising and Multimodal Topic Modeling.