RWKV

RWKV is an RNN with Transformer-level LLM performance. It is hosted as an incubation project in LF AI & Data Foundation.

United States of America

https://github.com/BlinkDL/RWKV-LM

RWKV's repositories

rwkv.cpp

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

Language:C++MIT1418 22 81

RWKV-infctx-trainer

RWKV infctx trainer, for training arbitary context sizes, to 10k and beyond!

Language:Jupyter NotebookApache-2.0133 5 23

rwkv-onnx

A converter and basic tester for rwkv onnx

Language:Python41 2 8

RWKV-LM

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

Language:PythonApache-2.02100

RWKV-cpp-node

Node.js implementation binding for the RWKV.cpp module

Language:JavaScriptMIT20 2 3

RWKV-wiki

RWKV centralised docs for the community

rwkv-tensorflow

RWKV ported to tensorflow

Language:Python11 2 1

ChatRWKV

ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.

Language:PythonApache-2.08 10

RWKV-tokenizer-node

RWKV tokenizer for node.js

Language:JavaScript4 2 1

ggml

Tensor library for machine learning

Language:CMIT300

llama.cpp

LLM inference in C/C++

Language:C++MIT2 10

World-Tokenizer-Typescript

Language:TypeScript2 10

artwork

Logos of RWKV projects

1 20

lm-evaluation-harness

A framework for few-shot evaluation of language models.

Language:PythonMIT101