Giters

martiansideofthemoon / style-transfer-paraphrase

Official code and data repository for our EMNLP 2020 long paper "Reformulating Unsupervised Style Transfer as Paraphrase Generation" (https://arxiv.org/abs/2010.05700).

Home Page:http://style.cs.umass.edu

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Perplexity value during training remains constant

Yaffa16 opened this issue · comments

commented

When i train on the Shakespeare data, for both shakespeare_0 or shakespeare_1, the perplexity value in eval_results remins constant
perplexity= tensor(135.8836)

Although the model trains till checkpoint=13797 , the quality of trained output is consistently low .

I also tried to run the demoparaphraser script while pointing to the Shakespeare model directory downloaded from the GoogleDrive here, however the quality was still not the same as on the web demo API .

Can you please help with why is this happening

commented

hi @Yaffa16,
Thanks for reporting the issue! Is your training loss going down? This is logged in Tensorboard (https://github.com/martiansideofthemoon/style-transfer-paraphrase/blob/master/style_paraphrase/run_lm_finetuning.py#L245) but you could also print it directly.

Also, what's the logging of the evaluation script? (this loop - https://github.com/martiansideofthemoon/style-transfer-paraphrase/blob/master/style_paraphrase/run_lm_finetuning.py#L444)

It should output the perplexity of every checkpoint.