openppl-public / ppl.llm.kernel.cuda

PPL LLM Kernel CUDA

Overview

ppl.llm.kernel.cuda is a part of PPL.LLM system.

We recommend users who are new to this project to read the Overview of system.

---

Primitive cuda kernel library for ppl.nn.llm

Currently, only Ampere and Hopper have been tested.

Prerequisites

• Linux running on x86_64 or arm64 CPUs
• GCC >= 9.4.0
• CMake >= 3.18
• Git >= 2.7.0
• CUDA Toolkit >= 11.4. 11.6 recommended. (for CUDA)

Quick Start

• Installing Prerequisites(on Debian or Ubuntu for example)

  apt-get install build-essential cmake git

• Cloning Source Code

  git clone https://github.com/openppl-public/ppl.llm.kernel.cuda.git

• Building from Source

  ./build.sh -DPPLNN_CUDA_ENABLE_NCCL=ON -DPPLNN_ENABLE_CUDA_JIT=OFF -DPPLNN_CUDA_ARCHITECTURES="'80;86;87'" -DPPLCOMMON_CUDA_ARCHITECTURES="'80;86;87'"

License

This project is distributed under the Apache License, Version 2.0.