TaoLbr1993

ShiArthur03

KIVI

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

Pytorch-XNOR-Net

XNOR-Net, with binary gemm and binary conv2d kernels, support both CPU and GPU.

binary-networks-pytorch

Binarize convolutional neural networks using pytorch :fire:

kernel_tuner

Kernel Tuner

NVIDIA_SGEMM_PRACTICE

Step-by-step optimization of CUDA SGEMM

XNOR-Net

ImageNet classification using binary Convolutional Neural Networks

BiLLM

(ICML 2024) BiLLM: Pushing the Limit of Post-Training Quantization for LLMs

marlin

FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

graphlearn-for-pytorch

A GPU-accelerated graph learning library for PyTorch, facilitating the scaling of GNN training and inference.

plot_demo

论文里可以用到的实验图示例

neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

graphcast

ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, Phi, MiniCPM, etc.) on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, GraphRAG, DeepSpeed, vLLM, FastChat, Axolotl, etc.

AutoGPTQ

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Edge-MoE

Edge-MoE: Memory-Efficient Multi-Task Vision Transformer Architecture with Task-level Sparsity via Mixture-of-Experts

LLMSpeculativeSampling

Fast inference from large lauguage models via speculative decoding

PygHO

A library for subgraph GNN based on pyg

DejaVu

FlexGen

Running large language models on a single GPU for throughput-oriented scenarios.

sparsegpt

Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".

Megatron-LM

Ongoing research training transformer models at scale

metaseq

Repo for external large-scale work

tuning_playbook

A playbook for systematically maximizing the performance of deep learning models.

vsc-cec-ide

一个插件，国产化你的VSCode，来源于CEC-IDE，有敏感词检测、防沉迷等功能。

Book

llama

Inference code for Llama models

pretrain-gnns

Strategies for Pre-training Graph Neural Networks

SUN

Understanding and Extending Subgraph GNNs by Rethinking their Symmetries (NeurIPS 2022 Oral)