rmsnorm

There are 0 repository under rmsnorm topic.

DefTruth / CUDA-Learn-Notes
📚Modern CUDA Learn Notes with PyTorch: Tensor/CUDA Cores, 📖150+ CUDA Kernels with PyTorch bindings, 📖HGEMM/SGEMM (95%~99% cuBLAS performance), 📖100+ LLM/CUDA Blogs.
block-reduce cuda cuda-programming elementwise flash-attention flash-attention-2 flash-attention-3 gemm gemv layernorm pytorch rmsnorm softmax triton warp-reduce
Language:Cuda 1444
bzhangGo / rmsnorm
Root Mean Square Layer Normalization
layernorm rmsnorm
Language:Python 213
knotgrass / Griffin
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models
attention conv1d gelu griffin h3 language-model linear mamba rg-lru rmsnorm shift-ssm
Language:Python 8
dtunai / Tri-RMSNorm
Efficient kernel for RMS normalization with fused operations, includes both forward and backward passes, compatibility with PyTorch.
ai machine-learning rmsnorm triton
Language:Python 7
rmgogogo / nano-aigc
Generative models nano version for fun. No STOA here, nano first.
bert clip conditional-diffusion conditional-vae ddim diffusion gan gemma gpt2 huggingface latent-diffusion llama llama2 multiqueryattention rmsnorm rope vae variational-autoencoder vit vq-vae
Language:Jupyter Notebook 0
sushantkumar23 / nano-gpt
Simple character level Transformer
attention attention-mechanism llm multi-head-attention pytorch rmsnorm rope self-attention transformer-architecture transformers shakespeare-dataset
Language:Jupyter Notebook 0

DefTruth / CUDA-Learn-Notes