echel0n / k8s-gitops

GitOps principles to define kubernetes cluster state via code

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

GitOps workflow for kubernetes cluster

Leverage WeaveWorks Flux to automate cluster state using code residing in this repo

Setup

See cluster bootstrap instructions for bootstrapping a kubernetes cluster for using this repo

Deep-Dive

System-level configuration

See kube-system for details on system-level configurations (cert-manager, traefik, decsheduler, fluxcloud, forwardauth OAuth, heapster, intel gpu plugin, dashboard, kured, metallb, sealed-secrets)

Storage

See storage for details on storage type services (local storage provider, minio, nfs-client, external NFS mounts, external ceph, stash)

Deployments

See deployments for details on regular workloads (home-assistant, hubot, minecraft, node-red, nzbget, plex, rabbitmq, radarr, rtorrent-flood, sonarr, unifi)

Monitoring

See monitoring for details on regular workloads (chronograf, comcast usage, grafana, influxdb, cable modem stats, prometheus-operator, speedtest results, uptimerobot agent)

Logging

See logging for details on logging solutions (loki, EFK Stack (elasticSearch, fluentd, kibana), elasticsearch-curator)

Caveats

Manual actions

See manual-steps for instructions things that cannot be handled by flux

New namespaces

If deploying a helm chart that needs to live in a new namespace, Flux seems to expect that the namespace is already created, or else the helm deployment will fail. When deploying a helm chart in the traditional approach via the helm CLI, it would handle the namespace creation for you. In Flx, you must explicitly create a helm chart (see storage/rook/namespace.yaml for an example of this)

Deletions

Flux doesn't handle deletions. What this means is that if you remove something from the repo (or even change something to run in a different namespace), it will not clean-up the removed item. This is a task that you must manually do.

To remove HelmRelease type entities from flux, you must manually delete the helmrelease object, e.g. to clean-up a helm release named forwardauth. This should properly remove the helm chart and associated objects

kubectl -n kube-system delete helmrelease/forwardauth

Secrets & Sensitive information

  • sealed-secrets works really well for encrypting secret and sensitive information for certain situations:
    • Kubernetes Secret primitives
    • The usage of those primitives in Deployments ENV variables and volume mounts
    • Helm chart values.yaml merging: You can leverage flux & sealed-secrets to automatically merge-in a secured set of values into the helm deployment
  • Securing other sensitive things that don't fall into the above categories must be handled manually outside of Flux

About

GitOps principles to define kubernetes cluster state via code

License:Apache License 2.0


Languages

Language:Shell 100.0%