brucechin

FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

LLaMA-Factory

A WebUI for Efficient Fine-Tuning of 100+ LLMs (ACL 2024)

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

chatbox

User-friendly Desktop Client App for AI Models/LLMs (GPT, Claude, Gemini, Ollama...)

awesome-quant

A curated list of insanely awesome libraries, packages and resources for Quants (Quantitative Finance)

flash-attention

Fast and memory-efficient exact attention

triton

Development repository for the Triton language and compiler

NeMo

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

FlexGen

Running large language models on a single GPU for throughput-oriented scenarios.

text-generation-inference

Large Language Model Text Generation Inference

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

lora

Using Low-rank adaptation to quickly fine-tune diffusion models.

DiT

Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

FasterTransformer

Transformer related optimization, including BERT, GPT

FlagAI

FlagAI (Fast LArge-scale General AI models) is a fast, easy-to-use and extensible toolkit for large-scale model.

beringei

Beringei is a high performance, in-memory storage engine for time series data.

deepsparse

Sparsity-aware deep learning inference runtime for CPUs

risc0

RISC Zero is a zero-knowledge verifiable general computing platform based on zk-STARKs and the RISC-V microarchitecture.

OpenDiT

OpenDiT: An Easy, Fast and Memory-Efficient System for DiT Training and Inference

DeepSeek-LLM

DeepSeek LLM: Let there be answers

LLMs_interview_notes

该仓库主要记录 大模型（LLMs） 算法工程师相关的面试题

MetaCLIP

ICLR2024 Spotlight: curation/training code, metadata, distribution and pre-trained models for MetaCLIP; CVPR 2024: MoDE: CLIP Data Experts via Clustering

llm-hallucination-survey

Reading list of hallucination in LLMs. Check out our new survey paper: "Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models"

llm-engine

Scale LLM Engine public repository

system-design

veScale

A PyTorch Native LLM Training Framework

Awesome-LLM-System-Papers

cuda-handbook

pdf

VALI

Video processing in Python