wangdach

gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

llama.cpp

LLM inference in C/C++

alpaca.cpp

Locally run an Instruction-Tuned Chat-Style LLM

GPT-4-LLM

Instruction Tuning with GPT-4

Mooncake

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

ktransformers

A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations

DiT

Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

DeepLearing-Interview-Awesome-2024

AIGC-interview/CV-interview/LLMs-interview面试问题与答案集合仓，同时包含工作和科研过程中的新想法、新问题、新资源与新项目

llm_interview_note

主要记录大语言大模型（LLMs） 算法（应用）工程师相关的知识及面试题

torchtune

A Native-PyTorch Library for LLM Fine-tuning

ArticleCatalog

typora_plugin

Typora plugin. Feature enhancement tool | Typora 插件，功能增强工具

bitsandbytes

Accessible large language models via k-bit quantization for PyTorch.

xla

Enabling PyTorch on XLA Devices (e.g. Google TPU)

pytorch-model-train-template

pytorch单精度、半精度、混合精度、单卡、多卡（DP / DDP）、FSDP、DeepSpeed模型训练代码，并对比不同方法的训练速度以及GPU内存的使用

TransformerEngine

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.

transformers_zh_docs

Huggingface transformers的中文文档

Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

tutorials

PyTorch tutorials.

x-transformers

A simple but complete full-attention transformer with a set of promising experimental features from various papers

accelerate

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

peft

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

zig

General-purpose programming language and toolchain for maintaining robust, optimal, and reusable software.

python_backend

Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.

common

Common source, scripts and utilities shared across all Triton repositories.

triton

Development repository for the Triton language and compiler

tutorials

This repository contains tutorials and examples for Triton Inference Server

client

Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.

DeepSpeedExamples

Example models using DeepSpeed