关于 Supervised SimCSE 的 GPU Memory Usage

[AnonymXXXXX]

Hi~

根据论文附录，有监督场景下（使用 NLI 数据集）SimCSE-BERT-base 和 SimCSE-RoBERTa-base 的 batch size 为 512，seq len 为 32。请问这样的训练配置需要多少显存（GPU Memory Usage）？以及 SimCSE-BERT-base 和 SimCSE-RoBERTa-base 的显存占用是否一致？（需要据此确定自己的硬件环境，谢谢！）

[Tianyu Gao]

Hi,

The unsupervised model was trained with 1 3090 and the supervised model was trained with 4x3090.

[AnonymXXXXX]

Ok, that should be 24G*4, thanks for your reply.