Giters

marcotcr / checklist

Beyond Accuracy: Behavioral Testing of NLP models with CheckList

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

NER example

ninamalina opened this issue · comments

commented

Hi!
In the paper you describe some invariant tests for NER models, however, in the library I can't find any code to this. Could you share this as well? Thanks :)

commented

Hi! I don't think our papers involved tests for [NER models] though;
Did you mean the invariance tests w.r.t [capability: named entity] for sentiment/QQP/squad models? It was a capability we tested, e.g., changing the named entity from Dallas -> Seattle should not change the sentiment in "I liked my flight to Dallas."
If so, those tests are in the paper testing notebooks

Please feel free to reopen if this doesn't answer your question. I can try to further clarify if you can point me to which tests you are referring to in the paper :)