leeyeehoo

text-generation-inference

Large Language Model Text Generation Inference

SnapKV

InfLLM

The code of our paper "InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory"

JetMoE

Reaching LLaMA2 Performance with 0.1M Dollars

yarn

YaRN: Efficient Context Window Extension of Large Language Models

BitDelta

LLMTest_NeedleInAHaystack

Doing simple retrieval from LLM models at various context lengths to measure accuracy

Long-Context-Data-Engineering

Implementation of paper Data Engineering for Scaling Language Models to 128K Context

LEval

[ACL'24] Data and code for L-Eval, a comprehensive long context language models evaluation benchmark

fstattention

Memory bandwidth efficient sparse tree attention

KVQuant

KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

axlearn

An Extensible Deep Learning Library

EasyKV

Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)

LongLM

LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning

flashinfer

FlashInfer: Kernel Library for LLM Serving

jax

Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more

LLMFarm

llama and other large language models on iOS and MacOS offline using GGML library.

search_with_lepton

Building a quick conversation-based search demo with Lepton AI.

prompt-lookup-decoding

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

alphageometry

ELF

ELF: a platform for game research with AlphaGoZero/AlphaZero reimplementation

magicoder

Magicoder: Source Code Is All You Need

chex

mlx-examples

Examples in the MLX framework

cleanrl

High-quality single file implementation of Deep Reinforcement Learning algorithms with research-friendly features (PPO, DQN, C51, DDPG, TD3, SAC, PPG)

chatgpt-retrieval

PPO-PyTorch

Minimal implementation of clipped objective Proximal Policy Optimization (PPO) in PyTorch

mamba-minimal

Simple, minimal implementation of the Mamba SSM in one file of PyTorch.

lm-evaluation-harness

A framework for few-shot evaluation of language models.