rudeigerc / kuberay

A toolkit to run Ray applications on Kubernetes

KubeRay

KubeRay is a powerful, open-source Kubernetes operator that simplifies the deployment and management of Ray applications on Kubernetes. It offers several key components:

KubeRay core: This is the official, fully-maintained component of KubeRay that provides three custom resource definitions, RayCluster, RayJob, and RayService. These resources are designed to help you run a wide range of workloads with ease.

• RayCluster: KubeRay fully manages the lifecycle of RayCluster, including cluster creation/deletion, autoscaling, and ensuring fault tolerance.

• RayJob: With RayJob, KubeRay automatically creates a RayCluster and submits a job when the cluster is ready. You can also configure RayJob to automatically delete the RayCluster once the job finishes.

• RayService: RayService is made up of two parts: a RayCluster and a Ray Serve deployment graph. RayService offers zero-downtime upgrades for RayCluster and high availability.

Community-managed components (optional): Some components are maintained by the KubeRay community.

• KubeRay APIServer: It provides a layer of simplified configuration for KubeRay resources. The KubeRay API server is used internally by some organizations to back user interfaces for KubeRay resource management.

• KubeRay Python client: This Python client library provides APIs to handle RayCluster from your Python application.

• KubeRay CLI: KubeRay CLI provides the ability to manage KubeRay resources through command-line interface.

Documentation

From September 2023, all user-facing KubeRay documentation will be hosted on the Ray documentation. The KubeRay repository only contains documentation related to the development and maintenance of KubeRay.

Quick Start

• RayCluster Quickstart
• RayJob Quickstart
• RayService Quickstart

Examples

• Ray Train XGBoostTrainer on Kubernetes (CPU-only)
• Train PyTorch ResNet model with GPUs on Kubernetes
• Serve a MobileNet image classifier on Kubernetes (CPU-only)
• Serve a StableDiffusion text-to-image model on Kubernetes
• Serve a text summarizer on Kubernetes
• RayJob Batch Inference Example

Kubernetes Ecosystem

• Ingress: AWS Application Load Balancer, GKE Ingress, Nginx
• Using Prometheus and Grafana
• Profiling with py-spy
• KubeRay integration with Volcano
• Kubeflow: an interactive development solution
• MCAD: A Kubernetes Solution for Queuing and Gang Dispatching jobs on Single or Multi-Cluster environment

Blogs

• A cloud-native, open-source stack for accelerating foundation model innovation IBM (May 9, 2023).
• AI/ML Models Batch Training at Scale with Open Data Hub Red Hat (May 15, 2023).

Helm Charts

KubeRay Helm charts are hosted on the ray-project/kuberay-helm repository. Please read kuberay-operator to deploy the operator and ray-cluster to deploy a configurable Ray cluster. To deploy the optional KubeRay API Server, see kuberay-apiserver.

# Add the Helm repo
helm repo add kuberay https://ray-project.github.io/kuberay-helm/
helm repo update

# Confirm the repo exists
helm search repo kuberay --devel

# Install both CRDs and KubeRay operator v1.0.0.
helm install kuberay-operator kuberay/kuberay-operator --version 1.0.0

# Check the KubeRay operator Pod in `default` namespace
kubectl get pods
# NAME                                READY   STATUS    RESTARTS   AGE
# kuberay-operator-6fcbb94f64-mbfnr   1/1     Running   0          17s

Development

Please read our CONTRIBUTING guide before making a pull request. Refer to our DEVELOPMENT to build and run tests locally.

Getting Involved

Join Ray's Slack workspace, and search the following public channels:

• #kuberay-questions (KubeRay users): This channel aims to help KubeRay users with their questions. The messages will be closely monitored by the Ray and KubeRay maintainers.

• #kuberay-discuss (KubeRay contributors): This channel is for contributors to discuss what to do next with KubeRay (e.g. issues, pull requests, feature requests, design docs, KubeRay ecosystem integrations). All KubeRay maintainers and core contributors are in the channel.

Security

If you discover a potential security issue in this project, or think you may have discovered a security issue, we ask that you notify KubeRay Security via our Slack Channel. Please do not create a public GitHub issue.

License

This project is licensed under the Apache-2.0 License.