Why is imagenet-1k considered zero shot?

[Zhang Peiyuan]

I thought Imagenet 21k is a superset of imagenet-1k, as written in the ViT paper.

If Imagenet-21K is allowed for pre-training, I assume the evaluation on imagenet1k cannot be considered as zero shot?

[ChunyuanLI]

This confusion comes from the inconsistency of terms in the literature.

• In our setup and paper, we call the full ImageNet dataset as ImageNet-22K, the standard subset with 1K classes as ImageNet-1K. Therefor, ImageNet-21K in our case is defined as the subset of full ImageNet that excluded all the images in ImageNet-1K (ie, ImageNet-21K = ImageNet-22K - ImageNet-1K).

• You screenshot has a different definition of ImageNet-21K, which is equivalent to ImageNet-22K in our case.

If people follow our setting by training on ImageNet-21K, and evaluating on ImageNet-1K, it is zero-shot task transfer on ImageNet-1K.

In case you have further concerns on the definition of ``zero-shot'', I think the definition of zero-shot is also evolving. Here is an summary:

Hope this resolve your concerns. Thanks.