J.L's repositories
opencompass-rwkv
OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.
ChatRWKV
ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.
flash-linear-attention
Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton
peft
🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
PiSSA
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models
RWKV-LM-statetuning
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
RWKV_LM_EXT
This project is to extend RWKV LM's capabilities including sequence classification/embedding/peft/cross encoder/bi encoder/multi modalities, etc.