aquasecurity / starboard

What steps did you take and what happened:

I installed the starboard operator with Helm to use conftest (instead of polaris):

helm -n starboard-operator install starboard-operator aqua/starboard-operator --set targetNamespaces=default,starboard.configAuditReportsPlugin=Conftest

The chart then installs the config map starboard-conftest-config which contains keys conftest.imageRef and conftest.resources.*.*.

starboard/deploy/helm/templates/config.yaml

Lines 95 to 100 in 467c2eb

    
           data: 
        
             conftest.imageRef: {{ required ".Values.conftest.imageRef is required" .Values.conftest.imageRef | quote }} 
        
             conftest.resources.requests.cpu: {{ .Values.conftest.resources.requests.cpu | quote }} 
        
             conftest.resources.requests.memory: {{ .Values.conftest.resources.requests.memory | quote }} 
        
             conftest.resources.limits.cpu: {{ .Values.conftest.resources.limits.cpu | quote }} 
        
             conftest.resources.limits.memory: {{ .Values.conftest.resources.limits.memory | quote }}

Every time a new scan needs to be done, the Conftest plugin creates a job having a volume mounting in the directory /project/policy/ all the keys conftest.policy.* present in the config map starboard-conftest-config. The problem is that this config map does not contain such keys. The chart does not allow to complete this config map, and the operator does not mount files from others sources.

I wouldn't want to patch the config map starboard-conftest-config because it is a bad practice to manually modify a resource created by a Helm chart. Moreover, the modification would be overwritten if a GitOps CD pipeline regularly reconciles the resources installed by the Helm release.

What did you expect to happen:

I would like a chart option allows to configure the policies used by conftest.

Anything else you would like to add:

I see several ways to configure the policies that the conftest job pod should use :

Add to the chart a variable conftest.policies whose values would be added to the config map starboard-conftest-config
Allow the operator to load policies from another optional config map (like starboard-conftest-policies), not installed by this chart.

The second way requires to modify the plugin code but it is much more practical.
Indeed, with the first way, the policies are passed as parameters (values) of the Helm release. Pass rego policies as Helm values of a release is not very appropriate. First because policies are complex and not a scalar value; then because to change some policies the values of all the releases (for example for several clusters) need to be updated. With the second way, if the additional config map is packaged by a custom chart, then a new version of this chart could be automatically upgraded for example by a CD (GitOps) pipeline.

Environment:

Versions :

starboard operator: 0.12.0
chart: 0.7.0

Thanks for the feedback @fredgate .

Regarding the second option you mentioned, allow loading policies from the starboard-contest-policies ConfigMap, how is that different from creating the starboard-conftest-config ConfigMap with all policies before you run the helm install command? The operator will not override existing configuration object if it's present.

I was also wondering if we could use the --set-file flag to load Rego policies as Helm values? WDYT?

helm -n starboard-operator install starboard-operator aqua/starboard-operator \
  --set=targetNamespaces=default \
  --set=starboard.configAuditReportsPlugin=Conftest \
  --set-file=conftest.policy.kubernetes.rego=kubernetes/lib/kubernetes.rego \
  --set-file=conftest.policy.utils.rego=kubernetes/lib/utils.rego \
  --set-file=conftest.policy.file_system_not_read_only.rego=kubernetes/policies/general/file_system_not_read_only.rego \
  --set-file=conftest.policy.uses_image_tag_latest.rego=kubernetes/policies/general/uses_image_tag_latest.rego

We can't create the config map starboard-conftest-config before to install the Helm release of the starboard operator chart, else the release will fail because (by design) Helm refuses to modify a resource that has not been installed by itself.
A second config map (like starboard-conftest-policies) which would be optional and not installed by the starboard chart, could solve this issue. This config map could be installed manually or by a custom (corporate) chart whose update would be automatic with a GitOps CD pipeline.
I don't like the idea to pass policies as chart values, because when we need to change the policies we need to manually change the definition of the Helm release. In addition, the usual artifact registries (like Harbor...) can easily store a custom Helm chart but not rego policies.

I manage several Kubernetes clusters; also for that I use a GitOps CD pipeline (Flux Toolkit) which ensures the continuous reconciliation and the automatic update of the releases when new versions of the charts are available.
I currently cannot use conftest check using conftest, yet I am very interested in starboard. If a solution that works well with a CD pipeline can be found, I can participate in its development.

👋 @fredgate We have other integrations that rely on the current implementation of the Conftest config objects. Therefore, for v0.13.0 we came up with the following solution that might be suitable for what you are trying to achieve.

We'll be more than happy to discuss further and improve if needed it in the upcoming releases.

Conftest policies created separately

kubectl create ns $STARBOARD_NAMESPACE

kubectl create -n $STARBOARD_NAMESPACE -f your-conftest-policies.yaml

helm install starboard-operator aqua/starboard-operator \
  --namespace $STARBOARD_NAMESPACE \
  --set="targetNamespaces=default" \
  --set="starboard.configAuditReportsPlugin=Conftest" \
  --set="conftest.createConfig=false"

Conftest policies created with Helm

helm install starboard-operator aqua/starboard-operator \
  --namespace $STARBOARD_NAMESPACE --create-namespace \
  --set="targetNamespaces=default" \
  --set="starboard.configAuditReportsPlugin=Conftest" \
  --set-file="conftest.library.kubernetes\.rego=kubernetes/lib/kubernetes.rego" \
  --set-file="conftest.library.utils\.rego=kubernetes/lib/utils.rego" \
  --set-file="conftest.policy.file_system_not_read_only.rego=kubernetes/policies/general/file_system_not_read_only.rego" \
  --set-file="conftest.policy.uses_image_tag_latest.rego=kubernetes/policies/general/uses_image_tag_latest.rego" \
  --set-string="conftest.policy.file_system_not_read_only.kinds=Workload" \
  --set-string="conftest.policy.uses_image_tag_latest.kinds=Workload"

The new flag conftest.createConfig is a good alternative. I tested it; it works fine. It meets my need.

	data:
	conftest.imageRef: {{ required ".Values.conftest.imageRef is required" .Values.conftest.imageRef \| quote }}
	conftest.resources.requests.cpu: {{ .Values.conftest.resources.requests.cpu \| quote }}
	conftest.resources.requests.memory: {{ .Values.conftest.resources.requests.memory \| quote }}
	conftest.resources.limits.cpu: {{ .Values.conftest.resources.limits.cpu \| quote }}
	conftest.resources.limits.memory: {{ .Values.conftest.resources.limits.memory \| quote }}

Configure Conftest policies with the Helm chart

Conftest policies created separately

Conftest policies created with Helm