cblack34 / k8s-plugin-flexvolume-cifs

CIFS Flexvolume Plugin for Kubernetes

CIFS Flexvolume Plugin for Kubernetes

Driver for CIFS (SMB, Samba, Windows Share) network filesystems as Kubernetes volumes.

Background

Docker containers running in Kubernetes have an ephemeral file system: Once a container is terminated, all files are gone. In order to store persistent data in Kubernetes, you need to mount a Persistent Volume into your container. Kubernetes has built-in support for network filesystems found in the most common cloud providers, like Amazon's EBS, Microsoft's Azure disk, etc. However, some cloud hosting services, like the Hetzner cloud, provide network storage using the CIFS (SMB, Samba, Windows Share) protocol, which is not natively supported in Kubernetes.

Fortunately, Kubernetes provides Flexvolume, which is a plugin mechanism enabling users to write their own drivers. There are a few flexvolume drivers for CIFS out there, but for different reasons none of them seemed to work for me. So I wrote my own, which can be found on github.com/fstab/cifs.

Installing

The flexvolume plugin is a single shell script named cifs. This shell script must be available on the Kubernetes master and on each of the Kubernetes nodes. By default, Kubernetes searches for third party volume plugins in /usr/libexec/kubernetes/kubelet-plugins/volume/exec/. The plugin directory can be configured with the kubelet's --volume-plugin-dir parameter, run ps aux | grep kubelet to learn the location of the plugin directory on your system (see #1). The cifs script must be located in a subdirectory named fstab~cifs/. The directory name fstab~cifs/ will be mapped to the Flexvolume driver name fstab/cifs.

On the Kubernetes master and on each Kubernetes node run the following commands:

VOLUME_PLUGIN_DIR="/usr/libexec/kubernetes/kubelet-plugins/volume/exec"
mkdir -p "$VOLUME_PLUGIN_DIR/fstab~cifs"
cd "$VOLUME_PLUGIN_DIR/fstab~cifs"
curl -L -O https://raw.githubusercontent.com/fstab/cifs/master/cifs
chmod 755 cifs

The cifs script requires a few executables to be available on each host system:

• mount.cifs, on Ubuntu this is in the cifs-utils package.
• jq, on Ubuntu this is in the jq package.
• mountpoint, on Ubuntu this is in the util-linux package.
• base64, on Ubuntu this is in the coreutils package.

To check if the installation was successful, run the following command:

VOLUME_PLUGIN_DIR="/usr/libexec/kubernetes/kubelet-plugins/volume/exec"
$VOLUME_PLUGIN_DIR/fstab~cifs/cifs init

It should output a JSON string containing "status": "Success". This command is also run by Kubernetes itself when the cifs plugin is detected on the file system.

Running

The plugin takes the CIFS username, password, and optionally domain from a Kubernetes Secret. To create the secret, you first have to convert your username, domain, and password to base64 encoding:

echo -n username | base64
echo -n password | base64
echo -n domain | base64 # optional

Then, create a file secret.yml and use the ouput of the above commands as username, domain, and password:

apiVersion: v1
kind: Secret
metadata:
  name: cifs-secret
  namespace: default
type: fstab/cifs
data:
  username: 'ZXhhbXBsZQ=='
  password: 'bXktc2VjcmV0LXBhc3N3b3Jk'
  domain: 'ZG9tYWluLmNvbQ==' # optional

Apply the secret:

kubectl apply -f secret.yml

You can check if the secret was installed successfully using kubectl describe secret cifs-secret.

Next, create a file pod.yml with a test pod (replace //server/share with the network path of your CIFS share):

apiVersion: v1
kind: Pod
metadata:
  name: busybox
  namespace: default
spec:
  containers:
  - name: busybox
    image: busybox
    command:
      - sleep
      - "3600"
    imagePullPolicy: IfNotPresent
    volumeMounts:
    - name: test
      mountPath: /data
  volumes:
  - name: test
    flexVolume:
      driver: "fstab/cifs"
      fsType: "cifs"
      secretRef:
        name: "cifs-secret"
      options:
        networkPath: "//server/share"
        mountOptions: "dir_mode=0755,file_mode=0644,noperm"

Start the pod:

kubectl apply -f pod.yml

You can verify that the volume was mounted successfully using kubectl describe pod busybox.

Testing

If everything is fine, start a shell inside the container to see if it worked:

kubectl exec -ti busybox /bin/sh

Inside the container, you should see the CIFS share mounted to /data.