Question about applying MIRO to ViT

[Suho Lee]

Hello, Thanks for sharing your great works.

Currently, I'm trying to apply MIRO to ViT architecture and I have some questions about it. According to the paper, you've applied MIRO to CLIP ViT and I found that you tuned lambda only for such non-main experiments from the appendix. Does that mean you used learning rate of 5e-05, no dropout and no weight decay with Adam optimizer? If not, can you please share the algorithm-agnostic hyperparameters you used for CLIP ViT experiments?

Also, I was wondering that if you ever tried MIRO to ImageNet pretrained ViT(for example, torchvision.models.vit_b_16) instead of CLIP.

[Junbum Cha]

Yes, and no.
For CLIP ViT experiments, we used learning rate of 5e-5, no dropout, and no weight decay.
Also, we have not applied MIRO to ImageNet-pretrained ViT.

[Suho Lee]

Thanks for kind reply!