marcotcr / checklist

Beyond Accuracy: Behavioral Testing of NLP models with CheckList

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Any suggestion on how to block masked words?

imranraad07 opened this issue · comments

commented

I was wondering whether the checklist provides any way when masked suggestions would not include certain words? For example, for the given query below, I don't want to get suggestion words such as - a or the.

ret = editor.template('this is {mask} movie', remove_duplicates=True)

We don't have that functionality in editor.template, but here is a quick and easy way to get at the same thing:

forbidden = set(['a', 'the'])
suggestions = [x for x in editor.suggest('this is {mask} movie') if x not in forbidden]
ret = editor.template('this is {sug} movie', sug=suggestions, remove_duplicates=True)