dsmith73 / pi-cluster

raspberry pi cluster to use with docker swarm and k3s

Building a Raspberry PI cluster

I wanted to have a small development environment that I could use to test out docker swarms and kubernetes in the k3s configuration. This would allow me to stop using Windows Subsystem for Linux (WSL) or VMs to run them, and let me use them for the management of the cluster.

I looked at several different raspberry pi images, weighted their pros and cons, and then finally settled on using DietPi, over a standard minimalist install, or the hypriot image. I felt that DietPi would give me most of the configurability of hypriot, while staying closer to the OOB configuration offered by Raspberry.

---

Architecture:

Hardware

Amount	Description
5	Raspberry pi 3B+
1	switch
1	VIXMINI wireless router
1	Ankr power supply

Software

• dietpi os
• Etcher
• Ansible

---

Issues & Fixes

Issues

• Python apparently wasn't installed on the images that I used - so I made a script to handle that part before running the plays (as part of the playbook)

#!/bin/bash
# start_here.sh
# don't forget to "chmod +x start_here.sh"  

USERNAME=root
PASSWORD="XXXXXXXXXXXXXXXXXXX"
HOSTS="n99.ds"
SCRIPT="apt install python -y"

sshpass -p ${PASSWORD} ssh -o StrictHostKeyChecking=no -l ${USERNAME} ${HOSTNAME} "${SCRIPT}"

---

Templates

To make things consistent, I templated the config

• Update the following files -
  • Automation_Custom_PreScript.sh with pre-network scripts and place in /boot
  • Automation_Custom_Script.sh with post-install scripts and place in /boot
    • any additional packages that I want installed or configurations that I want performed
  • dietpi.txt with desired settings and place in /boot - I changed the following -
    • AUTO_SETUP_LOCALE=en_US.UTF-8
    • AUTO_SETUP_KEYBOARD_LAYOUT=us
    • AUTO_SETUP_TIMEZONE=America/New_York
    • AUTO_SETUP_NET_ETHERNET_ENABLED=1
    • AUTO_SETUP_NET_WIFI_ENABLED=1
    • AUTO_SETUP_NET_USESTATIC=1
    • AUTO_SETUP_NET_STATIC_IP=10.11.0.99
    • AUTO_SETUP_NET_STATIC_MASK=255.255.255.0
    • AUTO_SETUP_NET_STATIC_GATEWAY=10.11.0.1
    • AUTO_SETUP_NET_STATIC_DNS=10.11.0.1 8.8.8.8
    • AUTO_SETUP_NET_HOSTNAME=n99
    • AUTO_SETUP_HEADLESS=1
    • AUTO_SETUP_AUTOMATED=1
    • AUTO_SETUP_GLOBAL_PASSWORD=XXXXXXX
    • CONFIG_NTP_MODE=3
    • CONFIG_WIFI_COUNTRY_CODE=US
  • dietpi-wifi.txt with desired settings and place in /boot
    • aWIFI_SSID[0]='WIFI SSID'
    • aWIFI_KEY[0]='WIFI PASSWORD'
• Enable SSH
  • put an empty ssh file (no extension) in the /boot directory

---

Playbook Outline

---

keys.yml

JOIN_K3S_CLUSTER     : "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
JOIN_DOCKER_SWARM    : "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"

---

Knowledge:

Some of the resources that I used for this

• Setup DietPi on Raspberry
• Cluster computing on the Raspberry Pi with Kubernetes
• Kubernetes Raspberry Pi 3 cluster
• SSL certs easy with k3s
• Grafana Docker image
• BrewPi

---

#dsmith73