k8-containers / local-gitops

An automated local cluster setup w/ tls, monitoring, ingress and DNS configuration.

Local Gitops

Project Objective

The goal of this project is to provide a simplified approach to creating your own local cluster, as well all the utilities you'll need to run a functional cluster, such as a image registry, garbage collectors, ingress configuration with tls and monitoring tools.

It is also meant to provide a way for newcomers on the area both access to a functional GitOps configuration, and the capability to run it easily, to understand how it works in practice.

Resources

Kubernetes Version: v1.24.4-k3s1

Avaliable Kubernetes services:

• ArgoCD as the main GitOps tool | Available at argocd.k8s.localhost
• Access to the cluster using Nginx Ingress.
• Secret Management with Hashicorp's Vault | Available at vault.k8s.localhost
  • Secret access integration between Vault and ArgoCD using argocd-vault-plugin
• On-demand databases clusters with Zalando Operator for PostgreSQL | UI available at dbs.k8s.localhost
• Hot-Reload secrets and configmaps to pods using Reloader.
• Metrics monitoring with Prometheus's Stack (Also includes Grafana)
• Automated resource management with kube-cleanup-operator and Descheduler.
• Mirror resources between namespaces using Reflector.
• Image caching for hot-swapping using Kube-fledged. (Only useful for multi-node k3d setups)
• Cluster monitoring for problematic resources using kubenurse.

Tools required locally

• k3d running atop of either (pick one):
  • Rancher Desktop (Recommended)
  • Docker for Desktop
  • Podman (works but requires extra steps)
• Task as a more modern iteration of the Makefile utility
• mkcert for creating locally based TLS certificates for your ingress proxy
• kubectl | kustomize | helm to apply local commands to the cluster
• jq to manipulate the resulting JSON files and extract the required strings
• hostctl to create the local domain on your hosts file (optional, but recommended)
• vault-cli to create the local domain on your hosts file (required if Vault is chosen as the secret manager)

---

To list available commands, install Task and run:

task help

Installing requirements (optional)

To install the required tools automatically _{(Requires brew for Linux/MacOS and Chocolatey for Windows.)}

task tools:install

Initializing the repository

First: You have to fork this repository.

The task commands will ensure the repository is correctly configured once you run the bootstrapping command.

cd into your forked repository, then run:

task

---

Plan to use but are not k3d-compatible for now:

• Storage: Longhorn, require iscsi-compatible drivers. See Using Longhorn in k3d #478.
  • Alternative: Local Volumes. Uncomment config/cluster.yaml lines 31-36.
• Cluster Backup: Velero, requires compatible storage provider. See the official compatible list.
  • Alternative: kube-dump backup a k8s cluster as yaml manifests
  • Alternative: Stash, specifically for stateful applications.