| # Questions | # Non-CG Questions | # CG Questions | |
|---|---|---|---|
| sparc | 1203 | 355 | 98 |
| sparc-cg | 1669 | 355 | 372 |
| cosql | 1007 | 117 | 46 |
| cosql-cg | 814 | 117 | 167 |
-
This is the combined generalization test set generated by sparc and cosql based on the validation set, and its detailed parameters are the numbers of SPARC-CG and COSQL-CG in the table below.
-
Question is the total number of questions, # CG Questions is the number of questions that meet the definition of compositional generalization and # Non-CG Questions is the number of in-domain questions (the templates and combination of templates are both seen in training).
-
In a turn with a key called "cg," if the value of the key is true, it's a CG question, while if it is false, it is a non-cg question. If there is no key named "cg" in the turn of a conversation, it is an irrelevant question used to form a context.