andysalerno

hoppscotch

Open source API development ecosystem - https://hoppscotch.io (open-source alternative to Postman, Insomnia)

llm-course

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

autogen

A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap

chatbot-ui

AI chat for every model.

jan

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)

mlc-llm

Universal LLM Deployment Engine with ML Compilation

llamafile

Distribute and run LLMs with a single file.

peft

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

promptflow

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

litgpt

Pretrain, finetune, deploy 20+ LLMs on your own data. Uses state-of-the-art techniques: flash attention, FSDP, 4-bit, LoRA, and more.

PowerInfer

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

outlines

Structured Text Generation

gpt-fast

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

Autofac

An addictive .NET IoC container

mergekit

Tools for merging pretrained large language models.

sglang

SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with models faster and more controllable.

mixtral-offloading

Run Mixtral-8x7B models in Colab or consumer desktops

rustc_codegen_clr

This rust compiler backend(module) emmits valid CIL (.NET IR), enabling you to use Rust in .NET projects.

open-instruct

AQLM

Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.pdf

aphrodite-engine

PygmalionAI's large-scale inference engine

llm-autoeval

Automatically evaluate your LLMs in Google Colab

tabbyAPI

An OAI compatible exllamav2 API that's both lightweight and fast

weather.gov

weather.gov 2.0

openchat-ui

An open source UI for OpenChat models

cookbook

Deep learning for dummies. All the practical details and useful utilities that go into working with real models.

lm-inference-engines

Comparison of Language Model Inference Engines

NPHardEval

Repository for NPHardEval, a quantified-dynamic benchmark of LLMs

aphrodite-guidance

Guidence support for aphrodite-engine

clembench

A Framework for the Systematic Evaluation of Chat-Optimized Language Models as Conversational Agents and an Extensible Benchmark