This is the official repo for the Findings of EMNLP 2020 paper: The Box is in the Pen: Evaluating Commonsense Reasoning in Neural Machine Translation .
This repository contains three types of test suite:
-
lexical ambiguity
-
contextless syntactic ambiguity
-
contextual syntactic ambiguity