Ongoing research training transformer language models at scale, including: BERT & GPT-2
Repository from Github https://github.comzdevito/Megatron-LMRepository from Github https://github.comzdevito/Megatron-LM