Blue Green with Helm
This spike is about implementing a Blue/Green deployment strategy using helm package manager. To be honest Helm wasn't born thinking about B/G but with some trick we are able to do it.
This is an implementation related to the article here https://medium.com/@saraswatpuneet/blue-green-deployments-using-helm-charts-93ec479c0282
What we need...
- docker desktop, minikube or a real K8s cluster
- Helm version 3 or above
- Nginx as ingress
- A simple service (build using dot net core)
Installing NGINX
In the nginx folder you can find two yaml file coming directly from nginx we site. The only difference is about the namespace. We are going to use default instead of ingress-nginx.
> kubectl apply -f nginx/mandatory.yaml
> kubectl apply -f nginx/cloud-generic.yaml
my-service
In order to test the blue/green deployment we are going to use a little service developed using dotnet core. This service expose a resource /weatherforecast/slot returning a simple string like this
[HttpGet("slot")]
public string Slot()
=> $"{_serviceDescription.Slot} - {_serviceDescription.Version}";Topology
Looking at the yaml files in the template folder
.
├── Chart.yaml
├── deploy.yaml
├── templates
│ ├── blue
│ │ ├── config.yaml
│ │ └── deployment.yaml
│ ├── green
│ │ ├── config.yaml
│ │ └── deployment.yaml
│ ├── ingress.yaml
│ └── service.yaml
└── values.yaml
we get the following topology of the system
where the service use the selector slot:blue|green to point the the right deployment.
Release 0.1.0
deploy.yaml
blue:
version: "0.1.0"
green:
version: "0.1.0"
productionSlot: greencommand
helm upgrade my-service helm/ --debug --install -f helm/deploy.yamlresult
> kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx LoadBalancer 10.97.201.105 localhost 80:31940/TCP,443:30232/TCP 42h
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 2d3h
my-service ClusterIP 10.102.1.193 <none> 80/TCP 38m
> kubectl get configmap
NAME DATA AGE
ingress-controller-leader-nginx 0 42h
my-service-blue 1 39m
my-service-green 1 39m
nginx-configuration 0 42h
tcp-services 0 42h
udp-services 0 42h
> kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
my-service-blue 1/1 1 1 40m
my-service-green 1/1 1 1 40m
nginx-ingress-controller 1/1 1 1 42h
> kubectl get pods
NAME READY STATUS RESTARTS AGE
my-service-blue-598ffcc8d5-qwbbn 1/1 Running 0 2m37s
my-service-green-7fb46c446d-mx8gs 1/1 Running 0 40m
nginx-ingress-controller-7f74f657bd-gfxcv 1/1 Running 0 42h
❯ curl http://localhost/weatherforecast/slot -v
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost.prd (127.0.0.1) port 80 (#0)
> GET /weatherforecast/slot HTTP/1.1
> Host: localhost.prd
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.17.8
< Date: Thu, 05 Mar 2020 12:04:25 GMT
< Content-Type: text/plain; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Vary: Accept-Encoding
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
< Access-Control-Allow-Methods: GET, PUT, POST, DELETE, PATCH, OPTIONS
< Access-Control-Allow-Headers: DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization
<
* Connection #0 to host localhost.prd left intact
green - 0.1.0#
Deploy 0.2.0 on stage
deploy.yaml
blue:
version: "0.2.0"
green:
version: "0.1.0"
productionSlot: greencommand
> helm upgrade my-service helm/ --debug --install -f helm/deploy.yamlresult
❯ curl http://localhost/weatherforecast/slot -v
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost.stg (127.0.0.1) port 80 (#0)
> GET /weatherforecast/slot HTTP/1.1
> Host: localhost.stg
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.17.8
< Date: Thu, 05 Mar 2020 12:08:58 GMT
< Content-Type: text/plain; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Vary: Accept-Encoding
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
< Access-Control-Allow-Methods: GET, PUT, POST, DELETE, PATCH, OPTIONS
< Access-Control-Allow-Headers: DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization
<
* Connection #0 to host localhost.stg left intact
green - 0.1.0#
Switching prod to blue
deploy.yaml
blue:
version: "0.2.0"
green:
version: "0.1.0"
productionSlot: bluecommand
> helm upgrade my-service helm/ --debug --install -f helm/deploy.yamlresult
❯ curl http://localhost/weatherforecast/slot -v
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost.stg (127.0.0.1) port 80 (#0)
> GET /weatherforecast/slot HTTP/1.1
> Host: localhost.stg
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.17.8
< Date: Thu, 05 Mar 2020 12:08:58 GMT
< Content-Type: text/plain; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Vary: Accept-Encoding
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
< Access-Control-Allow-Methods: GET, PUT, POST, DELETE, PATCH, OPTIONS
< Access-Control-Allow-Headers: DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization
<
* Connection #0 to host localhost.stg left intact
blue - 0.2.0#
Now the command to switch immediately from green to blue is:
> kubectl patch svc my-service -p '{"spec":{"selector":{"slot": "blue"}}}'That allow you to change the selector of the service in order to route the traffic to the blue deployment set instead of the green one.