Training time cost

[Ferenas]

Hi, thanks for sharing this impressive work! I would like to know what is the specific GPU you used in experiments, like 3090 or v100? And how long it takes to finish the pre train process on X and OS+X?

[Sungnyun Kim]

Hello @Ferenas,

I'm glad to help you. For pretraining, we used 4 GPUS (RTX 3090) for 2 days. However, since OS+X is much larger than any other combinations of dataset, we restricted to 200 epochs training, which might take more than 2 days (I remember ~4 days?).

For relevant details, please refer to Appendix B.2: Basic settings.

Thanks.