oobabooga

text-generation-webui

The definitive Web UI for local AI, with powerful features and easy setup.

text-generation-webui-extensions

one-click-installers

Simplified installers for oobabooga/text-generation-webui.

flash-attention

Fast and memory-efficient exact attention - Windows wheels

oobabooga.github.io

llama-cpp-python-cuBLAS-wheels

Wheels for llama-cpp-python compiled with cuBLAS support

llm-tools

Various scripts for working with local LLMs

AI-Notebooks

SillyTavern

LLM Frontend for Power Users.

transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs

llama-cpp-python-basic

Python bindings for llama.cpp

torch-grammar

chatbot-ui

An open source ChatGPT UI.

quip-sharp

bitsandbytes-windows-webui

Windows compile of bitsandbytes for use in text-generation-webui.

gradio

Create UIs for your machine learning model in Python in 3 minutes

whisper

Robust Speech Recognition via Large-Scale Weak Supervision

AutoGPTQ

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

llama-cpp-binaries

llama.cpp server in a Python wheel.

screenshots

AutoAWQ

AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:

GPTQ-for-LLaMa-CUDA

A combination of Oobabooga's fork and the main cuda branch of GPTQ-for-LLaMa in a package format.

exllamav3

An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs

llama.cpp

Port of Facebook's LLaMA model in C/C++

test-repo

AutoAWQ_kernels

bitsandbytes

8-bit CUDA functions for PyTorch

BlockMerge_Gradient

Merge Transformers language models by use of gradient parameters.

llama-cpp-wrapper