GPU requirement for sv-eres2netv2
Tortoise17 opened this issue · comments
How much GPU memory is requirement for training sv-eres2netv2 model? I tried with 48 GB but failed. Is there any way to utilize the multi GPUs to avoid memory allocation failure?
I'm using 4 32G V100 GPUs, and 26G of memory per card in use, which is sufficient. Is your batch size too large? You could try running with the standard configuration.
@yfchenlucky I have tried run.sh file. so far I have not changed batch size. But I have used 48 G 2X cards, didn't try more than this.
You can change the batch_size to 128 or lower in https://github.com/modelscope/3D-Speaker/blob/main/egs/voxceleb/sv-eres2netv2/conf/eres2netv2.yaml#L24 and try again.