Giters

EleutherAI / gpt-neox

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

Home Page:https://www.eleuther.ai/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Convert HF format or raw weights of Llama2 to NEOX format

fmh1art opened this issue · comments

commented

Thanks for your great contribution! Recently, I want to continuously pretrain llama2 model but I have problems to load a llama2 checkpoints. Is there any instruction for loading Llama2 checkpoint? I notice that you have already offerred scripts for converting llama to neox format. While it fails on llama2.