grammarly / gector

Official implementation of the papers "GECToR – Grammatical Error Correction: Tag, Not Rewrite" (BEA-20) and "Text Simplification by Tagging" (BEA-21)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Achievement of metric values on conll-2014

kristinaivanova97 opened this issue · comments

Hello! I am trying to obtain the same metrics on conll-14 test dataset as mentioned in the GECToR paper and with the instructions on the GitHub page I was able to obtain only these:

Precision : 0.4167
Recall : 0.3538
F_0.5 : 0.4024

Can you help me with obtaining better values (from article)?
I've used xlnet_0_gector.th weights and vocabulary from the GitHub.
Other params:

max_len=128,
min_len=0,
min_error_probability=0,
special_tokens_fix=0

Hi! Here's the table with min_error_probability & additional_confidence you need to use:
https://github.com/grammarly/gector#pretrained-models