This repository contains implementation of paper "Message Passing Attention Networks for Document Understanding" as part of my coursewrok BITS F312 - Neural Networks and Fuzzy Logic at BITS Pilani.
- Reproduce results of MPAD and any one of its variants on one topic modelling dataset and one binary sentiment analysis dataset and one multi-class sentiment analysis dataset. (total 2 models * 3 datasets) = 6 results to be reproduced. [You are free to pick these 3 datasets from the given 10 and also one variant of MPAD from the given 3]
- Pick any 2 of the ablation studies and reproduce results for each of the ablation studies on any 1 of the datasets.
- In all above reports, in addition to accuracy, report F1 scores, precision and recall.
- Plot loss, accuracy and f1 during training
The three datasets selected are:
- Binary Sentiment Analysis: Polarity rt-polarity.txt
- Multi-class Sentiment Analysis: SST1 tab_sst1_train_n_test.txt: Contains both test and train data (Both training and testing are kept here as per our requirement for graph representation to be done at same time) tab_sst1_test.txt: Contains test data
- Topic Modelling Dataset: TREC tab_trec_train_n_test: Contains both test and train data (Both training and testing are kept here as per our requirement for graph representation to be done at same time) tab_trec_test: Contains test data
Drive link to datasets is here
-
MPAD(mpad folder contains the details regarding this). Markup:
- main_cross_val.py => Main function using k-fold cross validation for final evaluation. This is used for Polarity dataset only.
- main_test.py => Main function using testing datasets for final evaluation.
- utils.py => Contains all utility functions. Graphical representation functions are also present here.
- mlp.py => Pytorch model for Multi-layer perceptron used in AGGREGATE phase.
- layers.py => Pytorch model for attention mechanism and Message passing mechanism (to be used in models.py)
- models.py => Main Pytorch model used for training & testing.
-
Hierarchical MPAD(hierarchial_mpad folder contains the details regarding this). Markup:
- hmain_cross_val.py => Main function using k-fold cross validation for final evaluation. This is used for Polarity dataset only.
- hmain_test.py => Main function using testing datasets for final evaluation.
- hutils.py => Contains all utility functions. Graphical representation functions are also present here. This is to be used for hmain_cross_val.py. Utility functions for main cross validation and test differ slightly.
- hutils_test.py => Contains all utility functions. Graphical representation functions are also present here. This is to be used for hmain_test.py.
- mlp.py => Pytorch model for Multi-layer perceptron used in AGGREGATE phase.
- layers.py => Pytorch model for attention mechanism and Message passing mechanism (to be used in models.py)
- models.py => Main Pytorch model used for training & testing.
Execution Commands can be found here (also in colab file provided, Execution.ipynb )
- Message Passing Attention Networks for Document Understanding (Paper)
- Giannis Nikolentzos, MPAD GitHub Repository
- Lakshya Agarwal, 2017B5A70904P
- Naga Siva Sai Reddy, 2017B3A70779P
- Rahul Poddar, 2017B3A70746P