shamio

llama.cpp

LLM inference in C/C++

text-generation-webui

A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

unsloth

Finetune Llama 3, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory

MemGPT

Create LLM agents with long-term memory and custom tools 📚🦙

PowerInfer

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

streaming-llm

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

koboldcpp

A simple one-file way to run various GGML and GGUF models with KoboldAI's UI

OLMo

Modeling, training, eval, and inference code for OLMo

lollms-webui

Lord of Large Language Models Web User Interface

mergekit

Tools for merging pretrained large language models.

intel-extension-for-transformers

⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡

languagemodels

Explore large language models in 512MB of RAM

Long-Context

This repository contains code and tooling for the Abacus.AI LLM Context Expansion project. Also included are evaluation scripts and benchmark tasks that evaluate a model’s information retrieval capabilities with context expansion. We also include key experimental results and instructions for reproducing and building on them.

exui

Web UI for ExLlamaV2

neural-speed

An innovative library for efficient LLM inference via low-bit quantization

qmoe

Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".

LongAlign

LongAlign: A Recipe for Long Context Alignment Encompassing Data, Training, and Evaluation

Entropy-ABF

Official implementation for 'Extending LLMs’ Context Window with 100 Samples'

read-agent.github.io