licj15

Fov-3DGS

Official Implementation of RTGS: Enabling Real-Time Gaussian Splatting on Mobile Devices Using Efficiency-Guided Pruning and Foveated Rendering.

ServerlessLLM

Cost-efficient and fast multi-LLM serving.

ThunderKittens

Tile primitives for speedy kernels

grayskull-attention

Attention in SRAM on Tenstorrent Grayskull :metal:

QuaRot

Code for QuaRot, an end-to-end 4-bit inference of large language models.

unsloth

Finetune Llama 3.1, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory

ac_math

Algorithmic C Math Library

LLM4HWDesign_Starting_Toolkit

LLM4HWDesign Starting Toolkit

ACT

[ICML 2024] Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration

Grendel-GS

Ongoing research training gaussian splatting at scale by distributed system

cs249r_book

Collaborative book Machine Learning Systems

3D-Carbon

3D-Carbon: An Analytical Carbon Modeling Tool for 3D and 2.5D Integrated Circuits

LogarithmicPosit

[DAC'24] Official Implementation of the Logarithmic Posit (LP) Number System

Edge-LLM

[DAC 2024] EDGE-LLM: Enabling Efficient Large Language Model Adaptation on Edge Devices via Layerwise Unified Compression and Adaptive Layer Tuning and Voting

DeepSeek-V2

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

ShiftAddLLM

ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization

arxiv-latex-cleaner

arXiv LaTeX Cleaner: Easily clean the LaTeX code of your paper to submit to arXiv

TensorRT-Model-Optimizer

TensorRT Model Optimizer is a unified library of state-of-the-art model optimization techniques such as quantization and sparsity. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM or TensorRT to optimize inference speed on NVIDIA GPUs.

mg-verilog

Awesome-LLM-Inference

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

tiny-gpu

A minimal GPU design in Verilog to learn how GPUs work from the ground up

llama3

The official Meta Llama 3 GitHub site

gpt-fast

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

owl

SA-GS

research-career-tools

ray-tracing-in-cuda

cutlass

CUDA Templates for Linear Algebra Subroutines

warp

A Python framework for high performance GPU simulation and graphics

KIVI

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache