This is an extended version of the QALD dataset, a well-known benchmark resource used for the task of Question Answering over knowledge graphs (KGQA). The purpose of this work is to make available a) high-quality data even for languages other than English, and b) multiple reformulations of the same question, to test systems' robustness. The QALD version we used is the one released for the 9th edition of the challenge of Question Answering over Linked Data QALD9, and the languages involved are English and Italian. Besides a revised and improved quality of Italian translations of questions, the resource presented here features a number of alternative rewordings of both English and Italian questions. The dataset has been encoded in the QALD-JSON format.
The dataset includes, in addition to the approximately 550 original questions, a total of 995 paraphrases for the English section and 1,156 for the Italian section, and an overall amount of 1546 questions for English and 1707 for Italian. Few instances of duplicate questions from the original QALD9 were also removed.
| Lang. | Split | Orig. questions | Paraphrases | Tot. Questions |
|---|---|---|---|---|
| En | Train | 405 | 729 | 1134 |
| Test | 146 | 266 | 412 | |
| It | Train | 405 | 843 | 1248 |
| Test | 146 | 313 | 459 |
A preliminary baseline evaluation was conducted using QAnswer via GERBIL.
Here are the main results on the rewordQALD9 test set and for the F1-QALD measure only. To see the complete results for Macro/Micro-Precision/Recall/F1, please follow through GERBIL's permanent URIs of the experiments:
| EN | IT | |
|---|---|---|
| rewordQALD9 | 0.3186 | 0.2603 |
| rewordQALD9 (paraph.only) | 0.3113 | 0.244 |