KubeRay
KubeRay is a powerful, open-source Kubernetes operator that simplifies the deployment and management of Ray applications on Kubernetes. It offers several key components:
KubeRay core: This is the official, fully-maintained component of KubeRay that provides three custom resource definitions, RayCluster, RayJob, and RayService. These resources are designed to help you run a wide range of workloads with ease.
-
RayCluster: KubeRay fully manages the lifecycle of RayCluster, including cluster creation/deletion, autoscaling, and ensuring fault tolerance.
-
RayJob: With RayJob, KubeRay automatically creates a RayCluster and submits a job when the cluster is ready. You can also configure RayJob to automatically delete the RayCluster once the job finishes.
-
RayService: RayService is made up of two parts: a RayCluster and a Ray Serve deployment graph. RayService offers zero-downtime upgrades for RayCluster and high availability.
Community-managed components (optional): Some components are maintained by the KubeRay community.
-
KubeRay APIServer: It provides a layer of simplified configuration for KubeRay resources. The KubeRay API server is used internally by some organizations to back user interfaces for KubeRay resource management.
-
KubeRay Python client: This Python client library provides APIs to handle RayCluster from your Python application.
-
KubeRay CLI: KubeRay CLI provides the ability to manage KubeRay resources through command-line interface.
Documentation
From September 2023, all user-facing KubeRay documentation will be hosted on the Ray documentation. The KubeRay repository only contains documentation related to the development and maintenance of KubeRay.
Quick Start
Examples
- Ray Train XGBoostTrainer on Kubernetes (CPU-only)
- Train PyTorch ResNet model with GPUs on Kubernetes
- Serve a MobileNet image classifier on Kubernetes (CPU-only)
- Serve a StableDiffusion text-to-image model on Kubernetes
- Serve a text summarizer on Kubernetes
- RayJob Batch Inference Example
Kubernetes Ecosystem
- Ingress: AWS Application Load Balancer, GKE Ingress, Nginx
- Using Prometheus and Grafana
- Profiling with py-spy
- KubeRay integration with Volcano
- Kubeflow: an interactive development solution
Blogs
- A cloud-native, open-source stack for accelerating foundation model innovation IBM (May 9, 2023).
- AI/ML Models Batch Training at Scale with Open Data Hub Red Hat (May 15, 2023).
Helm Charts
KubeRay Helm charts are hosted on the ray-project/kuberay-helm repository. Please read kuberay-operator to deploy the operator and ray-cluster to deploy a configurable Ray cluster. To deploy the optional KubeRay API Server, see kuberay-apiserver.
# Add the Helm repo
helm repo add kuberay https://ray-project.github.io/kuberay-helm/
helm repo update
# Confirm the repo exists
helm search repo kuberay --devel
# Install both CRDs and KubeRay operator v0.6.0.
helm install kuberay-operator kuberay/kuberay-operator --version 1.0.0-rc.0
# Check the KubeRay operator Pod in `default` namespace
kubectl get pods
# NAME READY STATUS RESTARTS AGE
# kuberay-operator-6fcbb94f64-mbfnr 1/1 Running 0 17s
Development
Please read our CONTRIBUTING guide before making a pull request. Refer to our DEVELOPMENT to build and run tests locally.
Getting Involved
Join Ray's Slack workspace, and search the following public channels:
-
#kuberay-questions
(KubeRay users): This channel aims to help KubeRay users with their questions. The messages will be closely monitored by the Ray and KubeRay maintainers. -
#kuberay-discuss
(KubeRay contributors): This channel is for contributors to discuss what to do next with KubeRay (e.g. issues, pull requests, feature requests, design docs, KubeRay ecosystem integrations). All KubeRay maintainers and core contributors are in the channel.
Security
If you discover a potential security issue in this project, or think you may have discovered a security issue, we ask that you notify KubeRay Security via our Slack Channel. Please do not create a public GitHub issue.
License
This project is licensed under the Apache-2.0 License.