zhouyuan

minbpe

Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.

vanna

🤖 Chat with your SQL database 📊. Accurate Text-to-SQL Generation via LLMs using RAG 🔄.

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

DeepSeek-Coder

DeepSeek Coder: Let the Code Write Itself

LLMLingua

To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

Anima

33B Chinese LLM, DPO QLORA, 100K context, AirLLM 70B inference with single 4GB GPU

sglang

SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with models faster and more controllable.

blitzar

Zero-knowledge proof acceleration with GPUs for C++ and Rust

DeepSpeed-MII

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.

executorch

On-device AI across mobile, embedded and edge for PyTorch

Lumos

A RAG LLM co-pilot for browsing the web, powered by local LLMs

splink

Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends

sneller

World's fastest log analysis: λ + SQL + JSON + S3

gluten

Gluten: Plugin to Double SparkSQL's Performance

llm-app-stack

yet-another-applied-llm-benchmark

A benchmark to evaluate language models on questions I've previously asked them to solve.

incubator-xtable

Apache XTable (incubating) is a cross-table converter for lakehouse table formats that facilitates interoperability across data processing systems and query engines.

IvorySQL

Open Source Oracle Compatible PostgreSQL.

flashinfer

FlashInfer: Kernel Library for LLM Serving

RingAttention

Transformers with Arbitrarily Large Context

dbchaos

Stress-test your database with pre-defined queries. Generate synthetic data and events statically or with GPT.

sql-eval

Evaluate the accuracy of LLM generated outputs

rtp-llm

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

sort-research-rs

Test and benchmark suite for sort implementations.

mamba.c

Inference of Mamba models in pure C

libCacheSim

a high performance cache simulator and library

avx_qsort

Quick sort code using AVX2 instructions

Gluten-Trino

Gluten: Plugin to Boost Trino's Performance

crystal

GPU library for writing SQL queries

storage-testbench

A testbench for Google Cloud Storage client libraries.