firejq

Mooncake

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

resolvelib

Resolve abstract dependencies into concrete ones

pip

The Python package installer

DeepSeek-MoE

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

libarchive

Multi-format archive and compression library

DiT

Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

Chinese-Mixtral

中文Mixtral混合专家大模型（Chinese Mixtral MoE LLMs）

WeChatMsg

提取微信聊天记录，将其导出成HTML、Word、Excel文档永久保存，对聊天记录进行分析生成年度聊天报告，用聊天数据训练专属于个人的AI聊天助手

PaddleNLP

👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.

Chinese-LLaMA-Alpaca

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

Chinese-LLaMA-Alpaca-2

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)

llama

Inference code for Llama models

Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

lightllm

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

aliendao

huggingface mirror download

ChatGLM3

ChatGLM3 series: Open Bilingual Chat LLMs | 开源双语对话语言模型

smoothquant

[ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

flash-attention

Fast and memory-efficient exact attention

tensorrtllm_backend

The Triton TensorRT-LLM Backend

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

AutoGPTQ

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

text-generation-inference

Large Language Model Text Generation Inference

peft

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

fastllm

纯c++的全平台llm加速库，支持python调用，chatglm-6B级模型单卡可达10000+token / s，支持glm, llama, moss基座，手机端流畅运行

text-generation-webui

A Gradio web UI for Large Language Models.

ppl.nn.llm

ControlNet

Let us control diffusion models!

generative-models

Generative Models by Stability AI

AITemplate

AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.

stablediffusion

High-Resolution Image Synthesis with Latent Diffusion Models