SubjectNoi

stable-diffusion-webui

Stable Diffusion web UI

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

mlx

MLX: An array framework for Apple silicon

xformers

Hackable and optimized Transformers building blocks, supporting a composable construction.

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

tiny-gpu

A minimal GPU design in Verilog to learn how GPUs work from the ground up

FasterTransformer

Transformer related optimization, including BERT, GPT

hnswlib

Header-only C++/python library for fast approximate nearest neighbors

chain-of-thought-hub

Benchmarking large language models' complex reasoning ability with chain-of-thought prompting

ispc

Intel® Implicit SPMD Program Compiler

llm-awq

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

flashinfer

FlashInfer: Kernel Library for LLM Serving

openqasm

Quantum assembly language for extended quantum circuits

unlimiformer

Public repo for the NeurIPS 2023 paper "Unlimiformer: Long-Range Transformers with Unlimited Length Input"

CompilerGym

Reinforcement learning environments for compiler and program optimization tasks

How_to_optimize_in_GPU

This is a series of GPU optimization topics. Here we will introduce how to optimize the CUDA kernel in detail. I will introduce several basic kernel optimizations, including: elementwise, reduce, sgemv, sgemm, etc. The performance of these kernels is basically at or near the theoretical limit.

how-to-optimize-gemm

row-major matmul optimization

glake

GLake: optimizing GPU memory management and IO transmission.

kokkos-tutorials

Tutorials for the Kokkos C++ Performance Portability Programming Ecosystem

cuda_hgemm

Several optimization methods of half-precision general matrix multiplication (HGEMM) using tensor core with WMMA API and MMA PTX instruction.

dnnweaver2

Open Source Specialized Computing Stack for Accelerating Deep Neural Networks.

TiledCUDA

TiledCUDA is a highly efficient kernel template library designed to elevate CUDA C’s level of abstraction for processing tiles.

dsa-framework

Release of stream-specialization software/hardware stack.

bitfusion

Simulator for BitFusion

RayTracingToInfinity

A feature packed raytracer built with C++

tvm_gpu_gemm

play gemm with tvm

brainstorm

Compiler for Dynamic Neural Networks

mononn

amanda