flyer103 / kcp

kcp is a prototype of a Kubernetes API server that is not a Kubernetes cluster - a place to create, update, and maintain Kube-like APIs with controllers above or without clusters.

kcp is a minimal Kubernetes API server

How minimal exactly? kcp doesn't know about Pods or Nodes, let alone Deployments, Services, LoadBalancers, etc.

By default, kcp only knows about:

• Namespaces
• ServiceAccounts and role-based access control types like Role and RoleBinding
• Secrets and ConfigMaps, to store configuration data
• CustomResourceDefinitions, to define new types
• a handful of other low-level resources like Leases, Events, etc.

Like vanilla Kubernetes, kcp persists these resources in etcd for durable storage.

Any other resources, including Kubernetes-standard resources like Pods, Nodes and the rest, can be added as CRDs and reconciled using the standard controllers.

Why would I want that?

Kubernetes is mainly known as a container orchestration platform today, but we believe it can be even more.

With the power of CustomResourceDefinitions, Kubernetes provides a flexible platform for declarative APIs of all types, and the reconciliation pattern common to Kubernetes controllers is a powerful tool in building robust, expressive systems.

At the same time, a diverse and creative community of tools and services has sprung up around Kubernetes APIs.

Imagine a declarative Kubernetes-style API for anything, supported by an ecosystem of Kubernetes-aware tooling, separate from Kubernetes-the-container-orchestrator.

That's kcp.

Is kcp a "fork" of Kubernetes? 🍴

No.

kcp as a prototype currently depends on some unmerged changes to Kubernetes, but we intend to pursue these changes through the usual KEP process, until (hopefully!) Kubernetes can be configured to run as kcp runs today.

Our intention is that our experiments improve Kubernetes for everyone, by improving CRDs and scaling resource watching, and enabling more, better controllers for everyone, whether you're using Kubernetes as a container orchestrator or not.

Our kcp specific patches are in the feature-logical-clusters feature branch in the kcp-dev/kubernetes repo. See DEVELOPMENT.md for how the patches are structured and how they must be formatted during our experimentation phase. See GOALS.md for more info on how we intend to use kcp as a test-bed for exploring ideas that improve the entire ecosystem.

What's in this repo?

First off, this is a prototype, not a project. We're exploring these ideas here to try them out, experiment, and bounce them off each other. Our basic demo leverages the following components to show off these ideas:

• kcp, which serves a Kubernetes-style API with a minimum of built-in types.
• cluster-controller, which along with the Cluster CRD allows kcp to connect to other full-featured Kubernetes clusters, and includes these components:
  • syncer, which runs on Kubernetes clusters registered with the cluster-controller, and watches kcp for resources assigned to the cluster
  • deployment-splitter, which demonstrates a controller that can split a Deployment object into multiple "virtual Deployment" objects across multiple clusters.
  • crd-puller which demonstrates mirroring CRDs from a cluster back to kcp

So what's this for?

Multi-Cluster Kubernetes?

kcp could be useful for multi-cluster scenarios, by running kcp as a control plane outside of any of your workload clusters.

Multi-Tenant Kubernetes?

kcp could be useful for multi-tenancy scenarios, by allowing multiple tenant clusters inside a cluster to be managed by a single kcp control plane.

Local Kubernetes Development?

kcp could be useful for local development scenarios, where you don't necessarily care about all of Kubernetes' many built-in resources and their reconciling controllers.

Embedded/low-resource scenarios?

kcp could be useful for environments where resources are scarce, by limiting the number of controllers that need to run. Kubernetes' asynchronous reconciliation pattern can also be very powerful in disconnected or intermittently connected environments, regardless of how workloads actually run.

Is that all?

No! See our GOALS.md doc for more on what we are trying to accomplish with this prototype and our docs/ directory.

What does kcp stand for?

kcp as a project stands for equality and justice for all people.

However, kcp is not an acronym.

How do I get started?

1. Clone the repository.
2. Install Go (1.16+).
3. Download the latest kubectl binary for your OS.
4. Build and start kcp in the background: go run ./cmd/kcp start.
5. Tell kubectl where to find the kubeconfig: export KUBECONFIG=.kcp/admin.kubeconfig (this assumes your working directory is the root directory of the repository).
6. Confirm you can connect to kcp: kubectl api-resources.

For more scenarios, see DEVELOPMENT.md.

This sounds cool and I want to help!

Thanks! And great!

This work is still in early development, which means it's not ready for production, but also that your feedback can have a big impact.

You can reach us here, in this repository via issues and discussions, or:

• Join the #kcp-prototype channel in the Kubernetes Slack workspace
• Join the mailing lists
  • kcp-dev for development discussions
  • kcp-users for discussions among users and potential users
• Subscribe to the community calendar for community meetings and events
  • The kcp-dev mailing list is subscribed to this calendar
• See recordings of past community meetings on YouTube
• See upcoming and past community meeting agendas and notes
• Browse the shared Google Drive to share design docs, notes, etc.
  • Members of the kcp-dev mailing list can view this drive

References

• KubeCon EU 2021: Kubernetes as the Hybrid Cloud Control Plane Keynote - Clayton Coleman (video)
• OpenShift Commons: Kubernetes as the Control Plane for the Hybrid Cloud - Clayton Coleman (video)
• TGI Kubernetes 157: Exploring kcp: apiserver without Kubernetes
• K8s SIG Architecture meeting discussing kcp, June 29 2021
• Let's Learn kcp - A minimal Kubernetes API server with Saiyam Pathak, July 7 2021