vicuna 7b oom

[yjdy]

I'm trying to retrain the autoregression head with your train code on vicuna 7b
I have 8 v100 with 32G. But even I set bs=1, it still out of memory.

My environment is as follow:
CUDA 11.7
python 3.10
pytorch 2.1.2
transformers 4.37.2
accelerate 0.27.2

By the way, I used code of bsne1 branch

Best regards