kakaobrain / miro

Official PyTorch implementation of MIRO (ECCV 2022)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Standard error of CLIP-ViT experiments

korts-lew opened this issue · comments

Hello, I make this issue to ask standard errors of experiments using CLIP-ViT.
Unlike the results reported using ResNet-50 and RegNet w/ SWAG, there is no standard errors for the results of CLIP-ViT 😢
Can I get standard errors for it? (stds of 3 runs in PACS, VLCS, OfficeHome, TerraInc, DomainNet)
If it is impossible, can you give me hyperparameters used for the experiments instead?

Thank you for your great work again

Here is the results with standard errors of CLIP-ViT:

Algorithm PACS VLCS OfficeHome TerraInc DomainNet
ERM 83.4 ± 0.5 75.9 ± 1.3 66.4 ± 0.5 35.3 ± 0.8 44.4 ± 0.6
MIRO 95.6 ± 0.8 82.2 ± 0.3 82.5 ± 0.1 54.3 ± 0.4 54.0 ± 0.3

We omitted this due to lack of the space. Sorry for the inconvenience!

Thank you for your kind reply !! 😄