Giters

EleutherAI / gpt-neox

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

Home Page:https://www.eleuther.ai/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How to convert gpt-neox to llama architecture..?

yuri-son opened this issue · comments

commented

Hello developer,

I want to replacing the polyglot based on gpt-neox model with the llama tokenizer and model architecture.
Could you please advise if there's a way to make this conversion?

commented

GPT-NeoX is a library for training models, not a specific architecture. You can find the configs for training a model using the LLaMA architecture in our example configs.