EleutherAI / gpt-neox

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

Home Page:https://www.eleuther.ai/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add Basic RWKV Block to GPT-NeoX

Quentin-Anthony opened this issue · comments

We want to add RWKV to gpt-neox: