justheuristic

llama

Inference code for LLaMA models

text-generation-webui

A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

numpy

The fundamental package for scientific computing with Python.

llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.

peft

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

qlora

QLoRA: Efficient Finetuning of Quantized LLMs

al-folio

A beautiful, simple, clean, and responsive Jekyll theme for academics

llama-recipes

Scripts for fine-tuning Llama2 with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization & question answering. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment.Demo apps to showcase Llama2 for WhatsApp & Messenger

StableCascade

Official Code for Stable Cascade

GitTorrent

A decentralization of GitHub using BitTorrent and Bitcoin

AutoGPTQ

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

alpa

Training and serving large-scale neural networks with auto parallelization.

GPTQ-for-LLaMa

4 bits quantization of LLaMA using GPTQ

mixtral-offloading

Run Mixtral-8x7B models in Colab or consumer desktops

llm-awq

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

ytsaurus

YTsaurus is a scalable and fault-tolerant open-source big data platform.

aesara

Aesara is a Python library for defining, optimizing, and efficiently evaluating mathematical expressions involving multi-dimensional arrays.

FBGEMM

FB (Facebook) + GEMM (General Matrix-Matrix Multiplication) - https://code.fb.com/ml-applications/fbgemm/

AQLM

Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.pdf

SpQR

hqq

Official implementation of Half-Quadratic Quantization (HQQ)

quip-sharp

H2O

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

knn-transformers

PyTorch + HuggingFace code for RetoMaton: "Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval" (ICML 2022), including an implementation of kNN-LM and kNN-MT

torch_cg

Preconditioned Conjugate Gradient in Pytorch

FastBinarySearch

Fast and vectorizable algorithms for searching in a vector of sorted floating point numbers

local-search-quantization

State-of-the-art method for large-scale ANN search as of Oct 2016. Presented at ECCV 16.

fast-hadamard-transform

Fast Hadamard transform in CUDA, with a PyTorch interface

huffman

Generate Huffman codes with Python

weighted-low-rank-bert-compression

Using weighted low-rank approximation to compress BERT.