Yi Liu's repositories
DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
neural-compressor
Intel® Neural Compressor (formerly known as Intel® Low Precision Optimization Tool), targeting to provide unified APIs for network compression technologies, such as low precision quantization, sparsity, pruning, knowledge distillation, across different deep learning frameworks to pursue optimal inference performance.
oneDNN
oneAPI Deep Neural Network Library (oneDNN)
pytorch
Tensors and Dynamic neural networks in Python with strong GPU acceleration
torchao-fork
The torchao repository contains api's and workflows for quantization and pruning gpu models.
accelerate
🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision
ai-pr-reviewer
AI-based Pull Request Summarizer and Reviewer with Chat Capabilities.
auto-awq-fork
AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:
auto-round
SOTA Weight-only Quantization Algorithm for LLMs
CodeXGLUE
CodeXGLUE
gemma.cpp
lightweight, standalone C++ inference engine for Google's Gemma models.
gpt-fast
Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.
hqq
Official implementation of Half-Quadratic Quantization (HQQ)
intel-extension-for-transformers
⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡
ipex-llm
Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, Phi, etc.) on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, DeepSpeed, vLLM, FastChat, Axolotl, etc.
marlin
FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
mpi-operator
Kubernetes Operator for MPI-based applications (distributed training, HPC, etc.)
nn-zero-to-hero
Neural Networks: Zero to Hero
optimum-habana
Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)
optimum-intel
🤗 Optimum Intel: Accelerate inference with Intel optimization tools
tgi
Large Language Model Text Generation Inference
Torch-Fx-Graph-Visualizer
Visualizer for neural network, deep learning and machine learning models
training-operator
Training operators on Kubernetes.
transformers
🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
tutorials
PyTorch tutorials.
xTuring
Easily build, customize and control your own LLMs