UCCL

uccl

UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

uccl-project.github.io

nccl

Optimized primitives for collective multi-GPU communication

nixl

NVIDIA Inference Xfer Library (NIXL)

ray-uccl

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs