Auto Rollback Controller
A Kubernetes controller example for learning purposes.
Usage
In v1.5 Kubernetes added a new field to the Deployments
spec, progressDeadlineSeconds
. As of v1.5
a Deployment
will be marked as failed if it fails to make progress within the allotted time. However, it won't automatically roll back.
This means the following deployment:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: hello
spec:
progressDeadlineSeconds: 5
replicas: 3
template:
metadata:
labels:
app: hello
spec:
containers:
- name: hello
image: alpine:3.5
command:
- /bin/sh
- -c
- "while :; do echo 'Goodbye'; exit 1; sleep 1; done"
doesn't roll back in v1.5.
The rollback-controller
loops, looking for failed deployments, then automatically does this rollback.
Example
In one terminal, start the rollback controller:
$ go get github.com/coreos/rollback-controller/cmd/rollback-controller
$ rollback-controller --kubeconfig=$PATH_TO_KUBECONFIG --namespace=default
In another create a deployment, then roll to a bad version of the deployment:
$ kubectl create -f examples/good.yaml
$ # Wait a bit for the deployment to succeed...
$ kubectl replace -f examples/bad.yaml
The second deployment should roll back to the first.
Exercises
Once you get the rollback controller working, fork the project and try adding the following features:
Opt-in annotation
Rather then rolling back all deployments in the cluster, modify the controller to only rollback deployments with a specific annotation. This is a common pattern for new controller features, letting users slowly opt into new functionality before making it the default.
An example annotation:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: hello
annotations:
coreos.com/auto-rollback: true
The rollback controller should ignore all other deployments.
Run the controller in the cluster
Containerize the rollback controller using a Dockerfile, push it to an image registry (e.g. Quay), and write a deployment manifest for this app. Deploy the controller as a pod in the cluster instead of running it from your laptop.
Optimizing performance
Instead of looping and listing, use client-go's imformers framework to optimize the controllers performance using the watch API.