qtwang

cutlass

CUDA Templates for Linear Algebra Subroutines

blis

BLAS-like Library Instantiation Software Framework

rocBLAS

Next generation BLAS implementation for ROCm platform

RouteLLM

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality!

SGEMM_CUDA

Fast CUDA matrix multiplication from scratch

NVIDIA_SGEMM_PRACTICE

Step-by-step optimization of CUDA SGEMM

faiss-learned-termination

Source code for SIGMOD 2020 paper "Improving Approximate Nearest Neighbor Search through Learned Adaptive Early Termination"

AutoCompressors

[EMNLP 2023] Adapting Language Models to Compress Long Contexts

TimeMixer

[ICLR 2024] Official implementation of "TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting"

dfn

packaging.python.org

Python Packaging User Guide

cmake_template

CMake for C++ Best Practices

cppbestpractices

Collaborative Collection of C++ Best Practices. This online resource is part of Jason Turner's collection of C++ Best Practices resources. See README.md for more information.

resume

An elegant \LaTeX\ résumé template. 大陆镜像 https://gods.coding.net/p/resume/git

PowerInfer

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

Dumpy

Dumpy: A Compact and Adaptive Index for Large Data Series Collections (SIGMOD'23)

pgvector

Open-source vector similarity search for Postgres

llama.cpp

LLM inference in C/C++

NeuTraj

Seed guided neural metric learning approach for calculating trajectory similarities

xformers

Hackable and optimized Transformers building blocks, supporting a composable construction.

DejaVu

FlexGen

Running large language models on a single GPU for throughput-oriented scenarios.

gisting

Learning to Compress Prompts with Gist Tokens - https://arxiv.org/abs/2304.08467

H2O

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

CUDA-Programs

Examples from Programming in Parallel with CUDA

FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

zero_scrolls

Running inference on the ZeroSCROLLS benchmark

LongChat

Official repository for LongChat and LongEval

sentencepiece

Unsupervised text tokenizer for Neural Network-based text generation.

StableLM

StableLM: Stability AI Language Models