lipiji / neural-summ-reproduce-lcsts-pytorch

Abstractive summarization on LCSTS dataset

neural-summ-reproduce-lcsts-pytorch

no copy, no coverage:

---------------------------------------------
C ROUGE-1 Average_R: 0.35439 (95%-conf.int. 0.33951 - 0.36968)
C ROUGE-1 Average_P: 0.37997 (95%-conf.int. 0.36300 - 0.39634)
C ROUGE-1 Average_F: 0.35953 (95%-conf.int. 0.34439 - 0.37447)
---------------------------------------------
C ROUGE-2 Average_R: 0.22892 (95%-conf.int. 0.21546 - 0.24284)
C ROUGE-2 Average_P: 0.24884 (95%-conf.int. 0.23375 - 0.26358)
C ROUGE-2 Average_F: 0.23322 (95%-conf.int. 0.21973 - 0.24709)
---------------------------------------------
C ROUGE-3 Average_R: 0.14141 (95%-conf.int. 0.12925 - 0.15499)
C ROUGE-3 Average_P: 0.15530 (95%-conf.int. 0.14111 - 0.16899)
C ROUGE-3 Average_F: 0.14424 (95%-conf.int. 0.13191 - 0.15719)
---------------------------------------------
C ROUGE-4 Average_R: 0.09420 (95%-conf.int. 0.08294 - 0.10710)
C ROUGE-4 Average_P: 0.10428 (95%-conf.int. 0.09186 - 0.11677)
C ROUGE-4 Average_F: 0.09609 (95%-conf.int. 0.08503 - 0.10808)
---------------------------------------------
C ROUGE-L Average_R: 0.32715 (95%-conf.int. 0.31242 - 0.34159)
C ROUGE-L Average_P: 0.35167 (95%-conf.int. 0.33494 - 0.36755)
C ROUGE-L Average_F: 0.33223 (95%-conf.int. 0.31733 - 0.34644)
---------------------------------------------
C ROUGE-W-1.2 Average_R: 0.17257 (95%-conf.int. 0.16448 - 0.18078)
C ROUGE-W-1.2 Average_P: 0.32348 (95%-conf.int. 0.30847 - 0.33776)
C ROUGE-W-1.2 Average_F: 0.21979 (95%-conf.int. 0.21020 - 0.22939)
---------------------------------------------
C ROUGE-SU4 Average_R: 0.19150 (95%-conf.int. 0.17923 - 0.20433)
C ROUGE-SU4 Average_P: 0.21034 (95%-conf.int. 0.19606 - 0.22356)
C ROUGE-SU4 Average_F: 0.19474 (95%-conf.int. 0.18208 - 0.20699)
BLEU =  0.18136493752655353
BLEU1 =  0.3730240549828179
BLEU2 =  0.24246587890446092
BLEU3 =  0.15048067490680792
BLEU4 =  0.10104529616724739
ratio =  0.9434268114767386

with copy and coverage:

---------------------------------------------
C ROUGE-1 Average_R: 0.33102 (95%-conf.int. 0.31656 - 0.34519)
C ROUGE-1 Average_P: 0.34201 (95%-conf.int. 0.32745 - 0.35644)
C ROUGE-1 Average_F: 0.33011 (95%-conf.int. 0.31617 - 0.34376)
---------------------------------------------
C ROUGE-2 Average_R: 0.20637 (95%-conf.int. 0.19407 - 0.21946)
C ROUGE-2 Average_P: 0.21596 (95%-conf.int. 0.20305 - 0.22923)
C ROUGE-2 Average_F: 0.20642 (95%-conf.int. 0.19455 - 0.21910)
---------------------------------------------
C ROUGE-3 Average_R: 0.12132 (95%-conf.int. 0.11058 - 0.13260)
C ROUGE-3 Average_P: 0.12907 (95%-conf.int. 0.11743 - 0.14130)
C ROUGE-3 Average_F: 0.12180 (95%-conf.int. 0.11103 - 0.13311)
---------------------------------------------
C ROUGE-4 Average_R: 0.07703 (95%-conf.int. 0.06724 - 0.08697)
C ROUGE-4 Average_P: 0.08351 (95%-conf.int. 0.07304 - 0.09441)
C ROUGE-4 Average_F: 0.07760 (95%-conf.int. 0.06797 - 0.08765)
---------------------------------------------
C ROUGE-L Average_R: 0.30072 (95%-conf.int. 0.28691 - 0.31424)
C ROUGE-L Average_P: 0.31092 (95%-conf.int. 0.29655 - 0.32563)
C ROUGE-L Average_F: 0.29991 (95%-conf.int. 0.28663 - 0.31275)
---------------------------------------------
C ROUGE-W-1.2 Average_R: 0.15968 (95%-conf.int. 0.15227 - 0.16706)
C ROUGE-W-1.2 Average_P: 0.28814 (95%-conf.int. 0.27478 - 0.30157)
C ROUGE-W-1.2 Average_F: 0.20073 (95%-conf.int. 0.19172 - 0.20975)
---------------------------------------------
C ROUGE-SU4 Average_R: 0.16902 (95%-conf.int. 0.15814 - 0.18020)
C ROUGE-SU4 Average_P: 0.17881 (95%-conf.int. 0.16648 - 0.19117)
C ROUGE-SU4 Average_F: 0.16894 (95%-conf.int. 0.15815 - 0.18017)
BLEU =  0.15822399118081293
BLEU1 =  0.3330862977602108
BLEU2 =  0.20803782505910165
BLEU3 =  0.12282669657879977
BLEU4 =  0.07849624060150376
ratio =  0.984276219808721