zobinHuang

Self-Hosting-Guide

Self-Hosting Guide. Learn all about locally hosting (on premises & private web servers) and managing software applications by yourself or your organization. Including Cloud, LLMs, WireGuard, Automation, Home Assistant, and Networking.

generative-recommenders

Repository hosting code used to reproduce results in "Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations" (https://arxiv.org/abs/2402.17152).

pyo3

Rust bindings for the Python interpreter

cudaparsers

Parsers for CUDA binary files

ecapture

Capturing SSL/TLS plaintext without a CA certificate using eBPF. Supported on Linux/Android kernels for amd64/arm64.

Awesome-LLM-Strawberry

A collection of LLM papers, blogs, and projects, with a focus on OpenAI o1 and reasoning techniques.

quiet-star

Code for Quiet-STaR

g1

g1: Using Llama-3.1 70b on Groq to create o1-like reasoning chains

rStar

xla

A machine learning compiler for GPUs, CPUs, and ML accelerators

k8s-dra-driver

Dynamic Resource Allocation (DRA) for NVIDIA GPUs in Kubernetes

sgl-learning-materials

Materials for learning SGLang

terway

CNI plugin for Alibaba Cloud VPC/ENI

CLFuzz

FluxMusic

Text-to-Music Generation with Rectified Flow Transformers

TiledCUDA

TiledCUDA is a highly efficient kernel template library designed to elevate CUDA C’s level of abstraction for processing tiles.

nnsmith

Automatic DNN generation for fuzzing and more

CasaOS

CasaOS - A simple, easy-to-use, elegant open-source Personal Cloud system.

gpumembench

A GPU benchmark suite for assessing on-chip GPU memory bandwidth

HAMi-core

HAMi-core compiles libvgpu.so, which ensures hard limit on GPU in container

astra-sim

ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale

Liger-Kernel

Efficient Triton Kernels for LLM Training

VideoSys

VideoSys: An easy and efficient system for video generation

rocm_bandwidth_test

Bandwidth test for ROCm

Awesome_LLM_System-PaperList

Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of papers on accelerating LLMs, currently focusing mainly on inference acceleration, and related works will be gradually added in the future. Welcome contributions!

LoongServe

torchdynamo

A Python-level JIT compiler designed to make unmodified PyTorch programs faster.

MIT-6.5940

All Homeworks for TinyML and Efficient Deep Learning Computing 6.5940 • Fall • 2023 • https://efficientml.ai

citus

Distributed PostgreSQL as an extension

MInference

To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.