microsoft / glge

Code for ACL2021 paper: "GLGE: A New General Language Generation Evaluation Benchmark"

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Reproduce the result of LSTM on PersonaChat

StevenTang1998 opened this issue · comments

Hi, I followed the instruction from this page.
I tested the result of the LSTM model on easy PersonaChat dataset.
I used the original script to preprocess the dataset combined with the vocab.txt in this page (because I can't find it in GLGE page).

./preprocessed.sh personachat easy

And then I used the original scipt to train and evaluate the lstm model without modifying anything.

./run.sh personachat easy lstm test

However, there is a big gap between the results of mine and the ones in the paper.
The results of paper are 42.2/35.9/0.2/0.7.
The results of mine are 31.75/19.63/0.3/1.2.

So, I want to know how to get the results in the paper or where I am wrong.

Hi, are you aware how to get the test.tgt file?
I downloaded the GLGE dataset on this page, but the test.tgt file is not included.