Giters

salesforce / ALPRO

Align and Prompt: Video-and-Language Pre-training with Entity Prompts

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Using multiple GPUs vs single GPU

avinashsai opened this issue · comments

commented

Hi,

Congratulations on the amazing work. Will there be any difference in performance if I use just a single GPU and what are the changes to be made in eg: msvd_qa.json?

Thank you.

commented

Using single gpu ends up with smaller number of samples in a batch. So you might need to increase the "gradient_accumulation_steps" to simulate multiple GPUs case. See how it works here. Conceptually this should give you same results after a longer training time, though non-determinism might cause minor performance difference, too.