[SST-2] Different test.tsv for original and k-shot data

[skull8888888]

Hello, I noticed that test.tsv for k-shot data contains 872 sentences, whereas original folder contains 1820 for SST-2 task. Is it correct behavior? Perhaps to save on computational time?

[Tianyu Gao]

Hi,

Thanks for pointing this out. For SST-2, we use the original development set as our test set, so the numbers of instances are different. You can also check out Appendix A in the paper for details.

[skull8888888]

So when I train the model, which test dataset does the model use, the one in k-shot/SST-2/ or in original?

[Tianyu Gao]

In our framework, the one in k-shot will be used.