--mixed_precision=fp16
WGS-note opened this issue · comments
多卡训练需要去掉混合精度，否则会：RuntimeError: expected scalar type Half but found Float，因为模型是Half加载的
用p-tuning还是lora做的，lora多卡我都跑不起来，量化直接报错
我没有用--mixed_precision=fp16没有报错
HarderThenHarder / transformers_tasks
⭐️ NLP Algorithms with transformers lib. Supporting Text-Classification, Text-Generation, Information-Extraction, Text-Matching, RLHF, SFT etc.