HLTCHKUST/dialogue-emotion

Hierarchical Attention for Dialogue Emotion Classification

CAiRE_HKUST submission for SemEval-2019 Task 3 Emo-Context

This is the implementation of our submission to Emo-Context. You can find our paper here. Shared task website: https://www.humanizing-ai.com/emocontext.html

This code has been written using PyTorch >= 1.0. If you use any source codes or datasets included in this toolkit in your work, please cite the following paper. The bibtex is listed below:

@inproceedings{winata-etal-2019-caire,
    title = "{CA}i{RE}{\_}{HKUST} at {S}em{E}val-2019 Task 3: Hierarchical Attention for Dialogue Emotion Classification",
    author = "Winata, Genta Indra  and
      Madotto, Andrea  and
      Lin, Zhaojiang  and
      Shin, Jamin  and
      Xu, Yan  and
      Xu, Peng  and
      Fung, Pascale",
    booktitle = "Proceedings of the 13th International Workshop on Semantic Evaluation",
    month = jun,
    year = "2019",
    address = "Minneapolis, Minnesota, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/S19-2021",
    pages = "142--147",
}

Abstract

Detecting emotion from dialogue is a challenge that has not yet been extensively surveyed. One could consider the emotion of each dialogue turn to be independent, but in this paper, we introduce a hierarchical approach to classify emotion, hypothesizing that the current emotional state depends on previous latent emotions. We benchmark several feature-based classifiers using pre-trained word and emotion embeddings, state-of-the-art end-to-end neural network models, and Gaussian processes for automatic hyper-parameter search. In our experiments, hierarchical architectures consistently give significant improvements, and our best model achieves a 76.77% F1-score on the test set.

Model Architecture

Data

You can find the data in Linkedin page.

Setup

To begin, you need to install libraries and their dependencies

bash setup.sh

Train

model: LSTM or UTRS (Universal Transformer)
attn: True (to add word-level attention)
fix_pretrain: True (To avoid any update on pretrained embedding)
--dev_with_label: to evaluate with a development set
--include_test: to merge train and development set, split the merged dataset, and construct a new set of train and development set. The model will be evaluated with a test set

Flat model

❱❱❱ python3 main.py --num_split 10 --max_epochs=100 --pretrain_list glove840B~300 --emb_dim 300 --cuda --hidden_dim 1000 --model LSTM --patient 5 --drop 0.1 --model LSTM --noam --save_path save_final/LSTM_1000_DROP0.1_ATTN_GLOVE/ --pretrain_emb --attn

Hierarchical model

❱❱❱ python3 main_hier.py --num_split 10 --max_epochs=100 --pretrain_list glove840B~300 --emb_dim 300 --cuda --hidden_dim 1000 --model LSTM --patient 5 --drop 0.1 --model LSTM --noam --save_path save_final/HLSTM_1000_DROP0.1_GLOVE/ --pretrain_emb

Evaluation

--pred_file_path: prediction file path, --ground_file_path: ground truth file path

❱❱❱ python3 eval.py --pred_file_path save/HLSTM_1000_DROP0.4_ALL_ATTN/test_voting.txt --ground_file_path data/dev.txt

Predict

--load_model_path: trained model path, --save_path: prediction file path

❱❱❱ python3 predict.py --load_model_path save/TEST2/model_0 --cuda --save_path save/TEST2/
❱❱❱ python3 predict_hier.py --load_model_path save/HLSTM_1000_DROP0.4_ATTN_DOUBLE_0.1_GLOVE/model_5 --cuda --double_supervision --save_prediction_path save_final/HLSTM_1000_DROP0.4_ATTN_DOUBLE_0.1_GLOVE/model_5.txt --save_confidence_path save_final/HLSTM_1000_DROP0.4_ATTN_DOUBLE_0.1_GLOVE/model_5_confidence.txt

Voting

❱❱❱ python3 voting.py --voting-dir-list save_final/HLSTM_1000_DROP0.4_ATTN_DOUBLE_0.1_GLOVE save_final/HLSTM_1000_DROP0.1_ATTN_DOUBLE_0.1_GLOVE --save_path voting_prediction.txt

Bug Report

Feel free to create an issue or send email to giwinata@connect.ust.hk

HLTCHKUST / dialogue-emotion