A little question about the evaluation

[Lingfeng Shen]

Hi! Thanks for your wonderful work!
I got a little question about the zero-shot evaluation on text classification. When doing argmax to get the predicted label, are we going to argmax on the whole vocab or on the specific label set (e.g., pos/neg in sentiment analysis)

Thanks a LOT

[Tianyu Gao]

Hi,

We did it only on the specific label set for both zero-shot and few-shot settings.