pltrdy / rouge

A full Python Implementation of the ROUGE Metric (not a wrapper)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Typo in documentation

jindrahelcl opened this issue · comments

Hi, I noticed an error in the documentation you have on PyPI (maybe it's used elsewhere as well):

from rouge import Rouge 

hypothesis = "the #### transcript is a written version of each day 's cnn student news program use this transcript to he lp students with reading comprehension and vocabulary use the weekly newsquiz to test your knowledge of storie s you saw on cnn student news"

reference = "this page includes the show transcript use the transcript to help students with reading comprehension and vocabulary at the bottom of the page , comment for a chance to be mentioned on cnn student news . you must be a teac her or a student age # # or older to request a mention on the cnn student news roll call . the weekly newsquiz tests students ' knowledge of even ts in the news"

rouge = Rouge()
scores = rouge.get_scores(reference, hypothesis)

according to the code (and the examples below this excerpt), hypothesis should go first.

@jindrahelcl thanks for pointing this out, that's true.

I'll fix it, just note that it is not really important as the ROUGE scoring is commutative

commented

Empirically score varied for me, I think it maybe because the reference should be a triply nested list and hypothesis doubly nested. So the get_scores is not commutative eventhough rouge recall is.