DicardoX

Oobleck

A resilient distributed training framework

fastapi

FastAPI framework, high performance, easy to learn, fast to code, ready for production

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

triton

Development repository for the Triton language and compiler

carrot

Free ChatGPT Site List 这儿为你准备了众多免费好用的ChatGPT镜像站点

pybind11

Seamless operability between C++11 and Python

LLM-Pruner

[NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support Llama-3/3.1, Llama-2, LLaMA, BLOOM, Vicuna, Baichuan, TinyLlama, etc.

habitat

🔮 Execution time predictions for deep neural network training iterations across different GPUs.

xla

A machine learning compiler for GPUs, CPUs, and ML accelerators

web-llm

High-performance In-browser LLM Inference Engine

ComScribe

ComScribe is a tool to identify communication among all GPU-GPU and CPU-GPU pairs in a single-node multi-GPU system.

nccl-tests

NCCL Tests

kernel_tuner

Kernel Tuner

mlc-llm

Universal LLM Deployment Engine with ML Compilation

chatgpt-web

用 Express 和 Vue3 搭建的 ChatGPT 演示网页

awesome-free-chatgpt

🆓免费的 ChatGPT 镜像网站列表，持续更新。List of free ChatGPT mirror sites, continuously updated.

grpc

The C based gRPC (C++, Python, Ruby, Objective-C, PHP, C#)

elasticflow-traces

Integrated Training Platform (ITP) traces used in ElasticFlow paper.

philly-traces

Information-Retrieval

Programming Assignments done using Python

nvidia-docker

Build and run Docker containers leveraging NVIDIA GPUs

ElasticFlow

Artifacts for our ASPLOS'23 paper ElasticFlow

tutorials

PyTorch tutorials.

HeliosArtifact

HeliosArtifact

HeliosData

Helios Traces from SenseTime

ray

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

clusterdata

cluster data collected from production clusters in Alibaba for cluster management research

hivedscheduler

Kubernetes Scheduler for Deep Learning

tutel

Tutel MoE: An Optimized Mixture-of-Experts Implementation