shreyansh26

neovim

Vim-fork focused on extensibility and usability

llama-stack-apps

Agentic components of the Llama Stack APIs

Liger-Kernel

Efficient Triton Kernels for LLM Training

flash-linear-attention

Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton

nano-llama31

nanoGPT style version of Llama 3.1

FBGEMM

FB (Facebook) + GEMM (General Matrix-Matrix Multiplication) - https://code.fb.com/ml-applications/fbgemm/

Efficient-LLMs-Survey

[TMLR 2024] Efficient Large Language Models: A Survey

nanoT5

Fast & Simple repository for pre-training and fine-tuning T5-style models

rerankers

A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models.

prompt-poet

Streamlines and simplifies prompt design for both developers and non-technical users with a low code approach.

bm25s

Fast lexical search library implementing BM25 in Python using Numpy and Scipy

cookbook

Deep learning for dummies. All the practical details and useful utilities that go into working with real models.

Nanoflow

A throughput-oriented high-performance serving framework for LLMs

llm-compressor

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Sophia

Effortless plugin and play Optimizer to cut model training costs by 50%. New optimizer that is 2x faster than Adam on LLMs.

cuda_hgemm

Several optimization methods of half-precision general matrix multiplication (HGEMM) using tensor core with WMMA API and MMA PTX instruction.

hash-hop

Long context evaluation for large language models

Awesome_LLM_System-PaperList

Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of papers on accelerating LLMs, currently focusing mainly on inference acceleration, and related works will be gradually added in the future. Welcome contributions!

AutoFP8

awesome-llm-planning-reasoning

A curated collection of LLM reasoning and planning resources, including key papers, limitations, benchmarks, and additional learning materials.

applied-ai

Applied AI experiments and examples for PyTorch

flashattention2-custom-mask

Triton implementation of FlashAttention2 that adds Custom Masks.

ml-recurrent-drafter

Guide-NVIDIA-Tools

NVIDIA tools guide

benchmark

Benchmark suite for LLMs from Fireworks.ai

cuda_hgemv

Several optimization methods of half-precision general matrix vector multiplication (HGEMV) using CUDA core.

Palu

Code for Palu: Compressing KV-Cache with Low-Rank Projection

lovely-llama

An implementation of the Llama architecture, to instruct and delight

hydragen

Hydragen: High-Throughput LLM Inference with Shared Prefixes

hip-attention

Training-free Post-training Efficient Sub-quadratic Complexity Attention. Implemented with OpenAI Triton.