Roughly how much time does the zero-shot retrieval evaluation take?

[Yash Patel]

Hi,

Many thanks for the great work and releasing the code. How much time does the inference of zero-shot retrieval take? On 2 V100s, it takes around 2 hours for me on MS-COCO. Is that normal or something is potentially wrong with my setup?

Many thanks in advance.

[viyjy]

Thanks for your interest in our work. I remember that it takes about 30mins on MS-COCO. Our machine is A100 with GPU memory size=40GB. Are you using the default batch size?

[Yash Patel]

Yes, default batch size from the retrieval_coco config.

[Yash Patel]

How many A100s do you use for the evaluation?

[viyjy]

As you can see here https://github.com/uta-smile/TCL#image-text-retrieval, we are using 8 GPUs.

[Yash Patel]

Thanks.