Question about the number of testing data

[LebronXu]

Hi, It's a very nice work!
I don't understand the division of data volume when testing,
for example, in the mini-imagenet dataset, the images num is 12000, but when 'TEST WITH test', you load 15000 images
the same question with 'val dataset'
why set like this num,
Looking forward to your reply ！

[vgsatorras]

Hi,

Few-shot learning builds a "episode" at every training/test iteration. In other words, it samples a support set of samples at every iteration that contains N classes and K samples per class, for example N=5 classes, and K=1 sample per class. Therefore, given for example, a full dataset of 12.000 samples (e.g. 100 classes and 120 samples per class) you could sample a huge amount of episodes by first sampling N classes and then K samples per class.

Therefore 15.000 value you see is actually and arbitrary parameter, you could create more or less episodes to evaluate your model. But 15.000 seems to be enough for the metric to converge.

Best,

[LebronXu]

Thank you for your reply.
I may have misunderstood these theories, this is means that some test-set images are reused when testing.
Can I understand it in this way ?

[LebronXu]

And I am also puzzled about the "Best test accuracy",
in the main.py line 196-198, when calculating the ''test_acc '':

if val_acc_aux is not None and val_acc_aux >= val_acc:
    test_acc = test_acc_aux
    val_acc = val_acc_aux

Why not use

if val_acc_aux is not None and val_acc_aux >= val_acc:
    val_acc = val_acc_aux
if test_acc_aux is not None and test_acc_aux >= test_acc:
    test_acc = test_acc_aux

I'm sorry to ask so many questions.
Looking forward to your reply ！

[Daniel123jia]

Hi, It's a very nice work!
I don't understand the division of data volume when testing,
for example, in the mini-imagenet dataset, the images num is 12000, but when 'TEST WITH test', you load 15000 images
the same question with 'val dataset'
why set like this num,
Looking forward to your reply ！

I am now study this code. we can communicate each other if you would like to and contact with me with qq 904948231