不用deepspeed init ？
XiaoqingNLP opened this issue · comments
按deepspeed 的教程，用deepspeed 先替换原torch 初始化，使用deepspeed init
THUDM / SwissArmyTransformer
SwissArmyTransformer is a flexible and powerful library to develop your own Transformer variants.
SwissArmyTransformer is a flexible and powerful library to develop your own Transformer variants.
XiaoqingNLP opened this issue · comments
按deepspeed 的教程，用deepspeed 先替换原torch 初始化，使用deepspeed init