Multimodal Machine Learning Group (MMLG)

multimodal-ml-reading-list

Reading list for research topics in multimodal machine learning

Multimodal-Emotion-Recognition

A real time Multimodal Emotion Recognition web app for text, sound and video inputs

awesome-multimodal-knowledge-graph

A curated list of awesome papers, datasets and tutorials within Multimodal Knowledge Graph.

iPerceive

Applying Common-Sense Reasoning to Multi-Modal Dense Video Captioning and Video Question Answering | Python3 | PyTorch | CNNs | Causality | Reasoning | LSTMs | Transformers | Multi-Head Self Attention

MultimodalNMT

contextual-multimodal-fusion

Contextual inter modal attention for multimodal sentiment analysis

MTN

Code for the paper Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems (ACL19)

multimodal-sentiment-analysis

Attention-based multimodal fusion for sentiment analysis

nmtpytorch

Sequence-to-Sequence Framework in PyTorch

pythia

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

Video-guided-Machine-Translation

Starter code for the VMT task and challenge

vilbert_beta

visualbert

Code for the paper "VisualBERT: A Simple and Performant Baseline for Vision and Language"

awesome-multimodal-machine-translation

A curated list of awesome papers, datasets and tutorials within Multimodal Machine Learning.

CVSE

The official source code for the paper Consensus-Aware Visual-Semantic Embedding for Image-Text Matching (ECCV 2020)

Grounding_in_Dialogue

ACL 2020 Tutorial by Malihe Alikhani and Matthew Stone

how2-dataset

This repository contains code and metadata of How2 dataset

lxmert

PyTorch code for EMNLP 2019 paper "LXMERT: Learning Cross-Modality Encoder Representations from Transformers".

Multimodal-Transformer

[ACL'19] [PyTorch] Multimodal Transformer