multimodal-fusion

There are 10 repositories under multimodal-fusion topic.

BMT
v-iashin / BMT
Source code for "Bi-modal Transformer for Dense Video Captioning" (BMVC 2020)
pytorch bi-modal-transformer dense-video-captioning audio video i3d activitynet-captions multimodal-fusion temporal-action-proposals bmvc bmvc20 bmt proposal-generator video-features bi-modal-encoder transformer
Language:Jupyter Notebook 220
declare-lab / Multimodal-Infomax
This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis, accepted at EMNLP 2021.
multimodal-deep-learning multimodal-fusion multimodal-sentiment-analysis
Language:Python 140
mahmoodlab / MCAT
Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images - ICCV 2021
early-fusion genomics mahmoodlab mcat multimodal multimodal-deep-learning multimodal-fusion pathology
Language:Jupyter Notebook 135
akashe / Multimodal-action-recognition
Code on selecting an action based on multimodal inputs. Here in this case inputs are voice and text.
multimodal-deep-learning multimodality multimodal-fusion multimodal-learning multimodal-data multimodal-action-recognition cross-attention
Language:Python 70
thuiar / MIntRec
MIntRec: A New Dataset for Multimodal Intent Recognition (ACM MM 2022)
acm-mm acm-mm-22 artificial-intelligence multimodal-deep-learning multimodal-fusion multimodal-intent-analysis speaker-recognition
Language:Python 59
ai-forever / fusion_brain_aij2021
Creating multimodal multitask models
multitask multimodal-fusion zero-shot-object-detection handwritten-text-recognition visual-question-answering java-to-python bilingual
Language:Jupyter Notebook 47
declare-lab / hfusion
Multimodal sentiment analysis using hierarchical fusion with context modeling
sentiment-analysis multimodal-sentiment-analysis multimodal-interactions multimodal-fusion fusion emotion-recognition emotion-detection emotion-analysis data-fusion
Language:Python 44
gholste / breast_mri_fusion
[CVAMD 2021] "End-to-End Learning of Fused Image and Non-Image Feature for Improved Breast Cancer Classification from MRI"
multimodal-fusion breast-cancer deep-learning
Language:Python 27
Asichurter / MalFusionFSL
Few-Shot malware classification using fused features of static analysis and dynamic analysis （基于静态+动态分析的混合特征的小样本恶意代码分类框架）
malware-classification multimodal-fusion simple malware fused-features few-shot-learning few-shot-malware-classification
Language:Python 18
ai-forever / fbc2_aij2022
FusionBrain Challenge 2.0: creating multimodal multitask model
multimodal-fusion multitask-learning
Language:Python 16
declare-lab / M2H2-dataset
This repository contains the dataset and baselines explained in the paper: M2H2: A Multimodal Multiparty Hindi Dataset For HumorRecognition in Conversations
humor-detection emotion-recognition-in-conversation multimodal-deep-learning multimodal-fusion
Language:Python 16
sverma88 / Deep-HOSeq--ICDM-2020
Deep-HOSeq: Deep Higher-Order Sequence Fusion for Multimodal Sentiment Analysis.
tensor multimodal-fusion tensor-factorization convolutional-neural-networks
Language:Python 11
imadhou / multimodal-sentiment-analysis
Multimodal sentiment analysis
multimodal-classification multimodal-fusion multimodal-learning multimodal-sentiment-analysis sentiment-analysis sentiment-classification twitter-sentiment-analysis
Language:Jupyter Notebook 8
marcomoldovan / multimodal-self-distillation
A generalized self-supervised training paradigm for unimodal and multimodal alignment and fusion.
multimodal-deep-learning pytorch self-distillation self-supervised-learning multimodal-alignment multimodal-fusion multimodal-retrieval
Language:Python 5
Clealiya / Multimodal-model
[FR|EN - Trio] 2023 - 2024 Centrale Méditerranée AI Master | Multimodal retranscription with text, audio and video
ai deep-learning machine-learning multimodal multimodal-fusion multimodality
Language:Python 3
sustainable-computing / Centaur
Repo for "Centaur: Robust Multimodal Fusion for Human Activity Recognition"
human-activity-recognition multimodal-fusion sensor-faults
Language:Jupyter Notebook 3
brian-zZZ / Guided-PLI
A Transferability-guided Protein-Ligand Interaction Prediction Method
multimodal-fusion protein-ligand-interactions representation-learning transferability
Language:Python 1
kassy11 / Awesome_Visually-Augmented_NLP
🖼️Latest Papers on Visually(Imagination)-Augmented NLP
awesome multimodal multimodal-deep-learning nlp dialogue-generation multimodal-fusion multimodality vision-and-language dialogue-system llm mllm multimodal-dialogue
1
usc-sail / mica-context-emotion-recognition
Repository for context based emotion recognition
computer-vision context-understanding emotion-recognition multimodal-deep-learning multimodal-fusion
Language:Python 1
kaykobad / MMSFormer
We propose Multi-Modal Segmentation TransFormer (MMSFormer) that incorporates a novel fusion strategy to perform multimodal material segmentation.
computer-vision multimodal-fusion multimodal-learning pytorch segmentation
Language:Python 0