radarFudan

Awesome-state-space-models

Collection of papers on state-space models

mamba-minimal-jax

mamba

Curse-of-memory

Curse-of-memory phenomenon of RNNs in sequence modelling

Mamba_State_Space_Model_Paper_List

Paper list for State-Space-Model and it's Applications

profiling-cuda-in-torch

radarFudan.github.io

StableSSM

annotated-mamba

Annotated version of the Mamba paper

flash-fft-conv

FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores

S5

attention_with_linear_biases

Code for the ALiBi method for transformer language models (ICLR 2022)

causal-conv1d

Causal depthwise conv1d in CUDA, with a PyTorch interface

EffHDC

flash-attention

Fast and memory-efficient exact attention

google-research

Google Research

lightning-hydra-template

PyTorch Lightning + Hydra. A very user-friendly template for ML experimentation. ⚡🔥⚡

radarFudan

RWKV-CUDA

The CUDA version of the RWKV language model ( https://github.com/BlinkDL/RWKV-LM )

SSM_examples

t5-pegasus-pytorch

TinyLlama

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

triton

Development repository for the Triton language and compiler

gateloop-transformer

Implementation of GateLoop Transformer in Pytorch and Jax

hasee

in-context-operator-networks

ICON for in-context operator learning

LongMamba

Some preliminary explorations of Mamba's context scaling.

mamba-minimal

Simple, minimal implementation of the Mamba SSM in one file of PyTorch.

s4

Structured state space sequence models

S5_StableSSM