josStorer / web-rwkv-py

Web-RWKV-Py

Python binding for web-rwkv.

Todos

• Basic V5 inference support
• Support V4, V5 and V6
• Batched inference

Usage

1. Install python and rust.

2. Install maturin by

   $ pip install maturin

3. Build and install:

   $ maturin develop

4. Try using web-rwkv in python:

   import web_rwkv_py as wrp
   
   model = wrp.v5.Model("/path/to/model.st", turbo=False)
   logits, state = wrp.v5.run_one(model, [114, 514], state=None)