Extension registration and automation for `gardener-operator`
timuthy opened this issue · comments
How to categorize this issue?
/area usability
/area ops-productivity
/area open-source
/area ipcei
/kind enhancement
What would you like to be added:
The following concept aims at improving the overall experience and productivity for operators. A Gardener installation usually needs additional and tedious preparation tasks to be done, e.g. creating storage buckets for backups or managing DNS entries. All of those can be automated via gardener-operator
.
They even overlap with requirements that were already implemented for shoot clusters, but never made it to the Garden due to conceptional reasons.
Therefore, we plan to add a new custom resource Extension
:
🎯 Goals
- Registration of a provider extension for the Garden-Runtime and Shoot clusters at the same time.
- Serves common tasks in Runtime cluster, e.g. reconciling
DNSRecord
orBackupBucket
resources (gardener-operator
will need to create those). - Will be translated into and deployed as
ControllerRegistration
/ControllerDeployment
(ref) in Garden cluster.
- Serves common tasks in Runtime cluster, e.g. reconciling
- Registration of an optional admission controller (example).
- Stick to Helm deployment logic (ref) but only rely on OCI repositories to fetch charts/assets (similar to Flux's support for OCI Repositories).
- Add great defaulting for known extensions, hosted in
github.com/gardener
. This allows smaller and trial landscapes to be setup with a minimalExtension
resources (see Example 2 below).
🙅♂️ Non Goals
- It's not planned to eliminate
ControllerRegistration
s. For complex or advanced configurations, operators should still consider their additional usage.
Extension
API:
Fields of the Extension
API are very similar to ControllerRegistration
/ControllerDeployment
as requirements and goals overlap for the main part.
Example 1:
apiVersion: operator.gardener.cloud/v1alpha1
kind: Extension
metadata:
name: aws
spec:
resources: # optional - defaulted by Operator for well-known extensions
# - kind: BackupBucket
# type: aws
# primary: true|false
# globallyEnabled: true|false # only valid if kind=Extension
# reconcileTimeout: 30s # only valid if kind=Extension
# lifecycle: {}
# workerlessSupported: true|false # only valid if kind=Extension
version: # optional - defaulted by Operator for well-known extensions
deployment: # optional - defaulted by Operator for well-known extensions
admission:
ociRepositoryURL: # optional - defaulted by Operator for well-known extensions
type: helm
providerConfig:
values: {}
extension:
ociRepositoryURL: # optional - defaulted by Operator for well-known extensions
type: helm
providerConfig:
values: {}
# policy: OnDemand|Always
The resources
configuration merges the extension handling for Garden and Shoot clusters, whereas the most part is irrelevant for the Garden and only needed to craft the ControllerRegistration
.
With decent defaulting, we want the extension registration for operators to be as simple as the following:
Example 2:
apiVersion: operator.gardener.cloud
kind: Extension
metadata:
name: aws
Tasks:
- Prerequisites
- #9924
- Introduce
class
inextensions.gardener.cloud/v1alpha1.DefaultSpec
- Default values for well known extensions, e.g. AWS, GCP, etc. Those should be maintained by the Renovate bot (see hackathon code)
- Registration and Deployment
- Extension registration in Virtual Garden Cluster for Seeds
- Extension Deployment in Runtime Garden Cluster
- Admission controller deployment (example)
Garden
controller adaptations- #9940
- Do we need
extensions.gardener.cloud/v1alpha1.BackupEntry
deployment?
- Adapt
Backup{Bucket,Entry}
andDNSRecord
extensions (for now, others maybe later) to be able to run in a cluster twice (one resp. for garden, one resp. for seed) - Push Helm charts to OCI registries (for both extension and admission components)
-
gardener/gardener
: -
gardener/gardener-extension-provider-alicloud
: -
gardener/gardener-extension-provider-aws
: -
gardener/gardener-extension-provider-azure
: -
gardener/gardener-extension-provider-gcp
: -
gardener/gardener-extension-provider-openstack
: -
gardener/gardener-extension-provider-equinix-metal
: -
gardener/gardener-extension-networking-calico
: -
gardener/gardener-extension-networking-cilium
: -
gardener/gardener-extension-os-gardenlinux
: -
gardener/gardener-extension-os-suse-chost
: -
gardener/gardener-extension-os-coreos
: -
gardener/gardener-extension-os-ubuntu
: -
gardener/gardener-extension-runtime-gvisor
: -
gardener/gardener-extension-registry-cache
: -
gardener/gardener-extension-shoot-dns-service
: -
gardener/gardener-extension-shoot-cert-service
: -
gardener/gardener-extension-shoot-oidc-service
: -
gardener/gardener-extension-shoot-lakom-service
: -
gardener/gardener-extension-shoot-falco-service
: -
gardener/gardener-extension-shoot-networking-problemdetector
: -
gardener/gardener-extension-shoot-networking-filter
: -
gardener/gardener-extension-shoot-rsyslog-relp
: -
stackitcloud/gardener-extension-acl
:
-