Drop capabilities CAP_SYS_ADMIN or all by default
NissesSenap opened this issue · comments
More or less nothing good can come from letting pods have the capabilities CAP_SYS_ADMIN.
CAP_SYS_ADMIN
Perform a range of system administration operations including: [quotactl](https://linux.die.net/man/2/quotactl)(2), [mount](https://linux.die.net/man/2/mount)(2), [umount](https://linux.die.net/man/2/umount)(2), [swapon](https://linux.die.net/man/2/swapon)(2), [swapoff](https://linux.die.net/man/2/swapoff)(2), [sethostname](https://linux.die.net/man/2/sethostname)(2), and [setdomainname](https://linux.die.net/man/2/setdomainname)(2);
Preferably we would drop all by default.
This might create a issues on the platform level but at the same time we ignore many of the platform namespaces.
If we are using mutating webhooks that applies drop "all" is it still possible to let the users to apply add
?
If so that could be an option if our tenants have very special needs.
containers:
- name: payment
image: nginx
securityContext:
capabilities:
drop:
- all
add:
- NET_BIND_SERVICE
DoD:
- Can we use add and a specific capability at the same time as we are using a mutating webhook?
- If so drop
all
- If not only drop
CAP_SYS_ADMIN
to all our containers - release a new helm chart version
The current constraints don't enforce this. My suggestion is that we disable the default enforce and we instead write our own that includes the config to also drop CAP_SYS_ADMIN. Else we entrust everything to the mutating webhook.
What do you think @phillebaba
My initial idea was to DROP all. But If i define a ADD capability I get:
apiVersion: v1
kind: Pod
metadata:
name: debu22
namespace: lab
spec:
containers:
- command:
- /bin/sh
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- NET_RAW
add:
- NET_BIND_SERVICE
readOnlyRootFilesystem: true
resources:
requests:
memory: "16Mi"
cpu: "10m"
limits:
memory: "64Mi"
cpu: "100m"
image: alpine:latest
name: container-00
tty: true
Error from server ([psp-capabilities] container <container-00> has a disallowed capability. Allowed capabilities are [""]): error when creating "debug.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [psp-capabilities] container <container-00> has a disallowed capability. Allowed capabilities are [""]
This is due to k get k8spspcapabilities.constraints.gatekeeper.sh psp-capabilities -o yaml
allowedCapabilities=""
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sPSPCapabilities
metadata:
annotations:
meta.helm.sh/release-name: gatekeeper-library-constraints
meta.helm.sh/release-namespace: gatekeeper-system
creationTimestamp: "2022-01-31T09:22:28Z"
generation: 1
labels:
app.kubernetes.io/managed-by: Helm
name: psp-capabilities
resourceVersion: "273515965"
uid: fe3d5f29-7f21-4489-aeb6-ae72aacd1514
spec:
enforcementAction: deny
match:
kinds:
- apiGroups:
- ""
kinds:
- Pod
parameters:
allowedCapabilities:
- ""
requiredDropCapabilities:
- NET_RAW
We can of course change this. But the question is what should we set it to? Should we DROP all by default but allow users to set whatever they want?
I think if we set drop all we need to capability to ether ignore the rule or allow our users to add capabilities.
So I have read up more on how this helm chart actually works...
As it actually says there are very few default values and allot can be overwritten, in our case we do it in: https://github.com/XenitAB/terraform-modules/blob/main/modules/kubernetes/opa-gatekeeper/locals.tf
So If I want to enforce the drop of CAP_SYS_ADMIN or even ALL then I can do that there.
If we ever get a need to allow a specific capability we can control this through the same config.
Currently we have setup ignore lists for namespace but if we ever need we can always add the possibility of exemptImages
to our helmchart.
I'm leaning more towards using the default value of drop all after actually understanding all this.
Verify the Drop the of CAP_SYS_ADMIN XenitAB/terraform-modules#590
This is solved