Xu-Chen

ollama

Get up and running with Llama 3, Mistral, Gemma 2, and other large language models.

LLaMA-Factory

Unify Efficient Fine-Tuning of 100+ LLMs

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Perplexica

Perplexica is an AI-powered search engine. It is an Open source alternative to Perplexity AI

mistral-inference

Official inference library for Mistral models

Qwen2

Qwen2 is the large language model series developed by Qwen team, Alibaba Cloud.

gpt-fast

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

AutoGPTQ

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs

xtuner

An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)

DeepSeek-V2

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

aspoem

Learn Chinese Poetry With AsPoem.com

farfalle

🔍 AI search engine - self-host with local or cloud LLMs

swift

ms-swift: Use PEFT or Full-parameter to finetune 250+ LLMs or 40+ MLLMs. (Qwen2, GLM4, Internlm2, Yi, Llama3, Llava, MiniCPM-V, Deepseek, Baichuan2, Gemma2, Phi3-Vision, ...)

AutoAWQ

AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:

clarity-ai

A simple Perplexity AI clone.

aphrodite-engine

PygmalionAI's large-scale inference engine

EAGLE

Official Implementation of EAGLE

BMTrain

Efficient Training (including pre-training and fine-tuning) for Big Models

marlin

FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

ocular

AI Powered Search and Chat for Orgs - Think ChatGpt meets Google Search but powered by your data.

llamafia.github

Eurus

nm-vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

KwaiYii

openai-scala-client

Scala client for OpenAI API

transformer_vq

QQQ

QQQ is an innovative and hardware-optimized W4A8 quantization solution.

GPTQModel

Sparse-Marlin