MANN - Multimodal Attention-based Neural Network
This is a part of Bittersweet's Graduation project. This repo implements the MANN framework (brought up by this paper) using TensorFlow.
PIP dependencies
- tensorflow (or tensorflow-gpu)
- sklearn
- numpy
- matplotlib
- lxml
Project structure
-
MANN_naive: MANN implemented without any attention
- using dummy data to test convergence:
python train_test.py - training:
python train.py - inference:
python inference.py - evaluate:
python evaluate.py - hyper parameters are read from hyper_params.py
- constants are read from constants.py
- using dummy data to test convergence:
-
MANN_SA: MANN implemented with SA(Similarity Attention)
- using dummy data to test convergence:
python train_test.py - training:
python train.py - inference:
python inference.py - evaluate:
python evaluate.py - hyper parameters are read from hyper_params.py
- constants are read from constants.py
- using dummy data to test convergence:
-
MANN_TCA: MANN implemented with TCA(Text-Concept Attention)
- training:
python train.py - inference:
python inference.py - evaluate:
python evaluate.py - hyper parameters are read from hyper_params.py
- constants are read from constants.py
- training:
-
MANN_TCA_cudnn: MANN implemented with TCA(Text-Concept Attention) written with CuDNNLSTM
- training:
python train.py - hyper parameters are read from hyper_params.py
- constants are read from constants.py
- training:
-
MANN_TCA_SA: MANN implemented with TCA(Text-Concept Attention) and SA(Similarity Attention)
- training:
python train.py - inference:
python inference.py - evaluate:
python evaluate.py - hyper parameters are read from hyper_params.py
- constants are read from constants.py
- training:
-
LCS: Least Common Sub-sequence
- inference:
python inference.py - evaluate:
python evaluate.py
- inference:
-
TF-IDF: Term Frequency - Inverse Document Frequency
- inference:
python inference.py - evaluate:
python evaluate.py
- inference: