Piraeus is a high performance (i.e., in Linux kernel), highly-available, simple, secure, and cloud agnostic data store for Kubernetes.
Using a Kubernetes cluster with at least 4 nodes is recommended. The hosts should use Docker as their container runtime and be running one of the following distributions to enable automatic DRBD kernel module injection:
- CentOS/RHEL 7
- CentOS/RHEL 8
- Ubuntu 18.04
The nodes on which piraeus should provide or consume storage should be labelled as follows:
kubectl label nodes $NODE_NAME piraeus/node=true
These nodes must also have the appropriate kernel development package installed unless DRBD is already present.
This is kernel-devel for CentOS based distributions and linux-headers-`uname -r` for Ubuntu.
Install as follows:
kubectl apply -f https://raw.githubusercontent.com/piraeusdatastore/piraeus/master/deploy/all.yaml
This may take several minutes. Once the pods have started, the status of Piraeus can be checked with:
kubectl -n kube-system exec piraeus-controller-0 -- linstor node list
This should show that the selected nodes are Online at the LINSTOR level.
Piraeus can use storage that is local to the application as well as storage on other nodes. On the nodes that should provide storage, backing devices must be available. Assuming the hosts have empty storage devices of at least 1GB capacity, they can be listed as follows:
kubectl -n kube-system exec piraeus-controller-0 -- linstor physical-storage list
Piraeus can then configure LVM on these devices and create a storage pool. Use the following steps for each node:
kubectl -n kube-system exec piraeus-controller-0 -- linstor physical-storage create-device-pool --pool-name pool0 LVM $NODE_NAME /dev/$DEVICE
kubectl -n kube-system exec piraeus-controller-0 -- linstor storage-pool create lvm $NODE_NAME DfltStorPool pool0
The demo directory contains examples of how to use the newly configured storage. For instance:
kubectl apply -f https://raw.githubusercontent.com/piraeusdatastore/piraeus/master/demo/piraeus-demo-pvc.yaml
kubectl label nodes $NODE_NAME piraeus/demo=true
kubectl apply -f https://raw.githubusercontent.com/piraeusdatastore/piraeus/master/demo/piraeus-demo-app.yaml
This demo app exposes some stats about the new volume as an HTTP page available on port 31279 of its host.
Piraeus consists of a number of open source components.
DRBD is used as the underlying storage replication mechanism. Documentation is provided by LINBIT.
LINSTOR is used for storage management. Documentation is provided by LINBIT.
The LINSTOR CSI plugin integrates LINSTOR with the Container Storage Interface. Documentation is included with the LINSTOR documentation.
This directory contains Dockerfiles and shell scripts for Piraeus components. Each of the subdirectories has
a README.md with more information.
- drbd-driver-loader contains is a collection of Piraeus container images containing the DRBD kernel source code that can be used to compile DRBD kernel modules from source and load them into the host kernel.
- piraeus-server combines LINBIT's linstor-controller and linstor-satellite components, as well as the linstor-client which can be used for cluster setup. It also contains some useful debugging tools.
- piraeus-client is a stand alone version of the linstor-client.