ring-attention

There are 0 repository under ring-attention topic.

feifeibear / long-context-attention
Sequence Parallel Attention for Long Context LLM Model Training and Inference
attention-is-all-you-need deepspeed-ulysses llm-inference llm-training pytorch ring-attention
Language:Python 173
dtunai / Ring-Attention-Jax
Packaged Ring Attention with Blockwise Transformers for Near-Infinite Context implemented in Jax + Flax.
jax ring-attention
Language:Python 1

feifeibear / long-context-attention