OswaldHe

qserve

QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving

SET-ISCA2023

The framework for the paper "Inter-layer Scheduling Space Definition and Exploration for Tiled Accelerators" in ISCA 2023.

TAPA-CS

HMT-pytorch

Official Implementation of "HMT: Hierarchical Memory Transformer for Long Context Language Processing"

mlirPyoclExec

Enabling OpenCL in MLIR via Python

pykan

Kolmogorov Arnold Networks

brevitas

Brevitas: neural network quantization in PyTorch

recut

Large-scale medical image processing and reconstruction toolbox

allo

Allo: A Programming Model for Composable Accelerator Design

JetMoE

Reaching LLaMA2 Performance with 0.1M Dollars

unlimiformer

Public repo for the NeurIPS 2023 paper "Unlimiformer: Long-Range Transformers with Unlimited Length Input"

LevelST

[FPGA 2024] Source code and bitstream for LevelST: Stream-based Accelerator for Sparse Triangular Solver

SSR

SSR: Spatial Sequential Hybrid Architecture for Latency Throughput Tradeoff in Transformer Acceleration (Full Paper Accepted in FPGA'24)

CHARM

CHARM: Composing Heterogeneous Accelerators on Versal ACAP Architecture

LM-RMT

Recurrent Memory Transformer

SqueezeLLM

[ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization

sparsegpt

Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".

llama2.cpp

Inference Llama 2 in one file of pure C++

FlexCNN

llama

Inference code for Llama models

llm-awq

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

flash-attention

Fast and memory-efficient exact attention

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

mosaic

pasta

[FCCM 2023] PASTA: Programming and Automation Support for Scalable Task-Parallel HLS Programs on Modern Multi-Die FPGAs

LightningSim

A fast, accurate trace-based simulator for High-Level Synthesis.

YuenyeungSpTRSV

A Thread-Level and Warp-Level Fusion Synchronization-Free Sparse Triangular Solve on GPUs

Callipepla

Large-scale sparse Conjugate Gradient (CG) solvers on High Bandwidth Memory (HBM) FPGAs

Serpens

Serpens is an HBM FPGA accelerator for SpMV

tapa

TAPA is a dataflow HLS framework that features fast compilation, expressive programming model and generates high-frequency FPGA accelerators.