Updating vulnerable images at runtime

[Mateus Caruccio]

Feature Description

Hability to update packages at pod runtime.

Is your feature request related to a problem?

Usually reports are too long and prone to be ignored by developers. This proposal is to be used as a last resource in order to minimize vulnerable code to run.

Thes idea is as follow:

• Marvin/Zora/whatever scans the image and generates a vulnerability report.
• A workload ValidatingWebhook inside the cluster checks for matches with vulnerable imagens from the reports
• For a given match, webhook changes each containers.command with a pre-exec script/binary which updates vulnerable dependencies (add, remove, update packages/files/configs) and exec() the original command (either explicitly set in the original containers.command or from image COMMAND.
• Upon failure, it could fallback to dont-update and leave the next pod untuched by labeling the owner workload object.

What alternatives have you considered?

Slim detects and minifies the image by using probe heuristics. The drawback is you need to interact with the application in order to generate data for slim to detect (access files, network activity, exec commands...)

https://github.com/slimtoolkit/slim

It's also an standalone tool, running outside of the cluster, thus it have no knowledge of the environment the app runs.

Additional Context

For example the webhook could change a deployment to this:

kind: Deployment:
...
spec:
  template:
    spec:
      container:
      - name: web
        commands:
        - sh
        - -c
        - 'curl https://undistro.io/update/alpine/3.2 | sh /app/server --listen 8080'

The update script will "fix" the image and exec() the original command /app/server from the image's COMMAND or from the original Deployment object.

To ignore certain deployments, a label marvin.io/dont-auto-update may be set to it. This could be used to ignore subsequent auto-updates to be applied by the webhook.

Caveats

This is certainly an antipattern since on-the-fly updated images will not be immutable anymore.
Also, this only work when pod.spec.containers.securityContext.readOnlyRootFilesystem=false.

An alternative solution is to have a daemonset agent to rebuild the images locally in all nodes with a new name.
All the webhook now has to do is mutate the workload's image name to the secured one.