DC-NeuralConversation
OpenNMT based Neural Conversation model which implements Topic and Semantic Distributional Constraints to improve quality of generated responses. paper
Installing dependencies
- Install the following pip packages -
six,tqdm,torchtext=0.2.1,future - Install pytorch 0.3.0; Choose the appropriate link from the website based on your cuda version (our model was trained on CUDA 9)
- Create an empty directory inside the main directory for caching -
syntax_topic_cache - Copy this embeddings file to
/data/sif_data/
How to evaluate models
If you want to re-train the model yourself, or train it on a different dataset then look at the next section "Training the model yourself". Evalutating with the pre-trained model doesn't require preprocessing or traning.
Loading pretrained model
- Download and extract the pretrained models and save them in the main directory. Skip the training and jump to evaluation.
Evaluating on the sample of Cornell Movie Dialogue Dataset
- To run evaluate DC-10 run -
python translate.py -model saved_adadelta_opensubtitles_models/opensubtitles_2_6_t_given_s_<10th epoch model file> -src data/cornell_test/s_cornell_dev_test.txt -tgt data/cornell_test/t_cornell_dev_test.txt -output DC_10_decoding_cornell_dev_test_predictions.txt -attn_debug -beam_size 10 -n_best 10 -batch_size 1 -verbose - To evaluate DC-MMI10 run -
python translate.py -model saved_adadelta_opensubtitles_models/opensubtitles_2_6_t_given_s_<10th epoch model file> -mmi_model saved_adadelta_opensubtitles_s_given_t_models/opensubtitles_2_6_s_given_t_<10th epoch model file> -src data/cornell_test/s_cornell_dev_test.txt -tgt data/cornell_test/t_cornell_dev_test.txt -output DC_MMI10_decoding_cornell_dev_test_predictions.txt -attn_debug -beam_size 10 -n_best 10 -batch_size 1 -verbose
Changing the \alpha and \beta (Topic and Semantic constraint weights)
Go the file /onmt/translate/Beam.py and update adjust self.alpha at line 93(for \alpha) and self.gamma at line 99(for \beta)
Traning the model yourself
Preprocessing the data
- Download the train data (sources and targets) and save it in the
data/opensubtitles_data/directory along with the validation and test files. - Run the following preprocessing command for T|S model -
python preprocess.py -train_src data/opensubtitles_data/s_train_dialogue_length2_6.txt -train_tgt data/opensubtitles_data/t_train_dialogue_length2_6.txt -valid_src data/opensubtitles_data/s_val_dialogue_length2_6.txt -valid_tgt data/opensubtitles_data/t_val_dialogue_length2_6.txt -save_data data/opensubtitles_2_6 -dynamic_dict -share_vocab - Run the following preprocessing command for S|T model -
python preprocess.py -train_src data/opensubtitles_data/t_train_dialogue_length2_6.txt -train_tgt data/opensubtitles_data/s_train_dialogue_length2_6.txt -valid_src data/opensubtitles_data/t_val_dialogue_length2_6.txt -valid_tgt data/opensubtitles_data/s_val_dialogue_length2_6.txt -save_data data/opensubtitles_2_6_s_given_t -dynamic_dict -share_vocab
Training the model
- Run the training command for T|S model -
python train.py -data data/opensubtitles_2_6 -save_model saved_adadelta_opensubtitles_models/opensubtitles_2_6_t_given_s -gpuid 2 -encoder_type brnn -param_init 0.08 -batch_size 256 -learning_rate 0.1 -optim adadelta -max_grad_norm 1 -word_vec_size 1000 -layers 4 -rnn_size 1000 -epochs 10 - Run the training command for S|T model -
python train.py -data data/opensubtitles_2_6_s_given_t -save_model saved_adadelta_opensubtitles_s_given_t_models/opensubtitles_2_6_s_given_t -gpuid 1 -encoder_type brnn -param_init 0.08 -batch_size 256 -learning_rate 0.1 -optim adadelta -max_grad_norm 1 -word_vec_size 1000 -layers 4 -rnn_size 1000 -epochs 10
Reference
@article{baheti2018generating,
title={Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints},
author={Baheti, Ashutosh and Ritter, Alan and Li, Jiwei and Dolan, Bill},
booktitle={Conference on Empirical Methods in Natural Language Processing (EMNLP 2018)},
year={2018}
}