Can you give the accurate number of few-shot accuracy linear prob?

[Keumgang Cha]

Can you give the accurate number of few-shot accuracy linear prob?

[Delong Chen (陈德龙)]

Hi, the followings are the few-shot evaluation result of RemoteCLIP in the Fig. 8 of our paper:

---

ResNet-50

n-shot	RSI-CB128	RSI-CB256	WHU-earth	EuroSAT	MLRSNet	PatternNet	RESISC45	AID	RS2800	OPTIMAL-31	RSC11	WHU-RS19
1-shot	35.59	42.52	44.25	43.20	31.75	46.10	39.33	36.95	41.93	42.80	48.13	45.15
4-shot	60.04	65.44	62.96	55.53	46.90	66.99	52.11	63.13	56.21	62.63	61.67	73.59
8-shot	69.55	75.89	75.38	61.75	55.02	77.07	61.75	70.50	63.29	73.01	72.51	85.44
16-shot	77.58	83.72	77.67	70.36	59.74	82.93	69.51	75.12	73.86	75.38	76.57	89.32
32-shot	82.02	87.04	85.08	77.44	64.99	88.32	75.71	82.46	77.71	81.61	83.82	93.79

ViT-B-32

n-shot	RSI-CB128	RSI-CB256	WHU-earth	EuroSAT	MLRSNet	PatternNet	RESISC45	AID	RS2800	OPTIMAL-31	RSC11	WHU-RS19
1-shot	34.31	44.28	41.25	44.89	34.14	45.98	42.10	37.04	39.29	41.13	44.14	40.78
4-shot	64.49	70.33	60.67	55.99	54.52	70.98	60.91	65.59	55.96	69.30	61.99	68.16
8-shot	76.13	83.73	75.71	65.76	64.24	82.53	70.92	75.72	63.75	77.85	72.43	80.68
16-shot	82.63	89.12	77.50	75.73	67.45	88.13	75.83	81.05	71.61	82.20	77.13	89.51
32-shot	88.11	91.83	87.08	83.30	71.58	91.87	81.77	86.67	81.61	89.14	85.50	93.40

[Keumgang Cha]

So, in the case of 32-shot, only 32 shots per class were used for learning and the rest were used for testing?

[Delong Chen (陈德龙)]

Actually no, we randomly sample different amount (1-shot to 32-shot) of few-shot examples from the 80% training split, and keep the testing set the same - all settings use the remaining 20% testing split.

[Keumgang Cha]

Thank you!