What batch size should be used on an A100?
ALGW71 opened this issue · comments
Cannot see any advice on batch size. If we want to improve the speed of obtaining embeddings, what batch sizes would you recommend on an A100?
Also is a dataloader used?
I found a batch size of 4096 can be used and this did improve speed.