kube-vip
and MetalLB
Automated build of HA k3s Cluster with
This playbook will build an HA Kubernetes cluster with k3s
, kube-vip
and MetalLB via ansible
.
This is based on the work from this fork which is based on the work from k3s-io/k3s-ansible. It uses kube-vip to create a load balancer for control plane, and metal-lb for its service LoadBalancer
.
If you want more context on how this works, see:
๐ k3s Ansible Playbook
Build a Kubernetes cluster using Ansible with k3s. The goal is easily install a HA Kubernetes cluster on machines running:
- Debian (tested on version 11)
- Ubuntu (tested on version 22.04)
- Rocky (tested on version 9)
on processor architecture:
- x64
- arm64
- armhf
โ
System requirements
-
Deployment environment must have Ansible 2.4.0+. If you need a quick primer on Ansible you can check out my docs and setting up Ansible.
-
You will also need to install collections that this playbook uses by running
ansible-galaxy collection install -r ./collections/requirements.yml
(importantโ ) -
netaddr
package must be available to Ansible. If you have installed Ansible via apt, this is already taken care of. If you have installed Ansible viapip
, make sure to installnetaddr
into the respective virtual environment. -
server
andagent
nodes should have passwordless SSH access, if not you can supply arguments to provide credentials--ask-pass --ask-become-pass
to each command.
๐ Getting Started
๐ด Preparation
First create a new directory based on the sample
directory within the inventory
directory:
cp -R inventory/sample inventory/my-cluster
Second, edit inventory/my-cluster/hosts.ini
to match the system information gathered above
For example:
[master]
192.168.30.38
192.168.30.39
192.168.30.40
[node]
192.168.30.41
192.168.30.42
[k3s_cluster:children]
master
node
If multiple hosts are in the master group, the playbook will automatically set up k3s in HA mode with etcd.
This requires at least k3s version 1.19.1
however the version is configurable by using the k3s_version
variable.
If needed, you can also edit inventory/my-cluster/group_vars/all.yml
to match your environment.
โธ๏ธ Create Cluster
Start provisioning of the cluster using the following command:
ansible-playbook site.yml -i inventory/my-cluster/hosts.ini
After deployment control plane will be accessible via virtual ip-address which is defined in inventory/group_vars/all.yml as apiserver_endpoint
๐ฅ Remove k3s cluster
ansible-playbook reset.yml -i inventory/my-cluster/hosts.ini
You should also reboot these nodes due to the VIP not being destroyed
โ๏ธ Kube Config
To copy your kube config
locally so that you can access your Kubernetes cluster run:
scp debian@master_ip:~/.kube/config ~/.kube/config
๐จ Testing your cluster
See the commands here.
Troubleshooting
Be sure to see this post on how to troubleshoot common problems
Testing the playbook using molecule
This playbook includes a molecule-based test setup. It is run automatically in CI, but you can also run the tests locally. This might be helpful for quick feedback in a few cases. You can find more information about it here.
Pre-commit Hooks
This repo uses pre-commit
and pre-commit-hooks
to lint and fix common style and syntax errors. Be sure to install python packages and then run pre-commit install
. For more information, see pre-commit
๐ค
Thanks This repo is really standing on the shoulders of giants. Thank you to all those who have contributed and thanks to these repos for code and ideas: