THUDM / GLM

GLM (General Language Model)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

请问单机8卡v100 32G能跑seq2seq的fine tune吗?我跑着会work = _default_pg.barrier()

webYFDT opened this issue · comments

commented

启动命令:bash scripts/ds_finetune_seq2seq.sh config_tasks/model_blocklm_10B.sh config_tasks/seq_cnndm_org.sh
1、scripts/ds_finetune_seq2seq.sh jia脚本配置
image
2、config_tasks/model_blocklm_10B.sh config_tasks/seq_cnndm_org.sh保持模型
3、运行10s内报错,但是监控显存并没有涨得很高,最高到400MB
image
请这种爆显存不够是哪个地方配置错误吗?还是单机8卡v100 32G不能跑seq2seq的fine tune?那么至少需要什么卡、多少显存才能跑上。