Giters

bigscience-workshop / Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Can I use python only apex for gpt_pretrain?

Luoyang144 opened this issue · comments

commented

I'm tring to run pretrain_gpt_single_node.sh in this link, and find not amp_C installed, it needs gcc>=6.0 but I can't change gcc version for now. So will it possible to run this script without amp_C?