Giters

alekc / terraform-provider-kubectl

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

v2.0.4 processing of Secrets not compliant with API and breaks expected behaviour

joaocc opened this issue · comments

commented

As per Secret API spec (https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#secret-v1-core):
stringData (...)It is provided as a write-only input field for convenience. All keys and values are merged into the data field on write, overwriting any existing values. The stringData field is never output when reading from the API.

However, it seems the implementation of ca6ec65 (to avoid plan issues) seems to overwrite data with stringData if stringData exists, instead of merging them as per the spec.

This can cause subtle issues as the provider is breaking the expected behaviour from the user point of view.
A suggestion would be to convert this to a merge as per the spec.

commented

This is related to #120 but they are not the same

commented

Testing it now but it looks like I can't reproduce it...

Given initial manifest:

resource "kubectl_manifest" "example" {
  yaml_body = <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: secret-basic-auth
  namespace: default
type: Opaque
stringData:
  username: admin # required field for kubernetes.io/basic-auth
  password: t0p-Secret # required field for kubernetes.io/basic-auth
EOF
}

We get a new secret created

apiVersion: v1
kind: Secret
metadata:
  name: secret-basic-auth
  namespace: default
  uid: 0449fb6c-6bbe-4a2f-be2d-642dfaa9f163
  resourceVersion: '904359108'
  creationTimestamp: '2024-04-06T18:42:51Z'
data:
  password: dDBwLVNlY3JldA==
  username: YWRtaW4=
type: Opaque

Adding a new field manually via kubectl edit

apiVersion: v1
kind: Secret
metadata:
  name: secret-basic-auth
  namespace: default
  uid: 0449fb6c-6bbe-4a2f-be2d-642dfaa9f163
  resourceVersion: '904359522'
  creationTimestamp: '2024-04-06T18:42:51Z'
  labels:
    k8slens-edit-resource-version: v1
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: >
      {"apiVersion":"v1","kind":"Secret","metadata":{"annotations":{},"name":"secret-basic-auth","namespace":"default"},"stringData":{"password":"t0p-Secret","username":"admin"},"type":"Opaque"}
  managedFields:
    - manager: HashiCorp
      operation: Update
      apiVersion: v1
      time: '2024-04-06T18:42:51Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:data:
          .: {}
          f:password: {}
          f:username: {}
        f:metadata:
          f:annotations:
            .: {}
            f:kubectl.kubernetes.io/last-applied-configuration: {}
        f:type: {}
    - manager: node-fetch
      operation: Update
      apiVersion: v1
      time: '2024-04-06T18:43:53Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:data:
          f:foo: {}
        f:metadata:
          f:labels:
            .: {}
            f:k8slens-edit-resource-version: {}
  selfLink: /api/v1/namespaces/default/secrets/secret-basic-auth
data:
  foo: YWRtaW4=
  password: dDBwLVNlY3JldA==
  username: YWRtaW4=
type: Opaque

Running terraform plan doesn't show any changes.

Changing one of the values

resource "kubectl_manifest" "example" {
  yaml_body = <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: secret-basic-auth
  namespace: default
type: Opaque
stringData:
  username: admin1 # required field for kubernetes.io/basic-auth
  password: t0p-Secret # required field for kubernetes.io/basic-auth
EOF
}

manual field is still maintained

apiVersion: v1
kind: Secret
metadata:
  name: secret-basic-auth
  namespace: default
  uid: 0449fb6c-6bbe-4a2f-be2d-642dfaa9f163
  resourceVersion: '904360377'
  creationTimestamp: '2024-04-06T18:42:51Z'
  labels:
    k8slens-edit-resource-version: v1
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: >
      {"apiVersion":"v1","kind":"Secret","metadata":{"annotations":{},"name":"secret-basic-auth","namespace":"default"},"stringData":{"password":"t0p-Secret","username":"admin1"},"type":"Opaque"}
  managedFields:
    - manager: node-fetch
      operation: Update
      apiVersion: v1
      time: '2024-04-06T18:43:53Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:data:
          f:foo: {}
        f:metadata:
          f:labels:
            .: {}
            f:k8slens-edit-resource-version: {}
    - manager: HashiCorp
      operation: Update
      apiVersion: v1
      time: '2024-04-06T18:45:26Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:data:
          .: {}
          f:password: {}
          f:username: {}
        f:metadata:
          f:annotations:
            .: {}
            f:kubectl.kubernetes.io/last-applied-configuration: {}
        f:type: {}
  selfLink: /api/v1/namespaces/default/secrets/secret-basic-auth
data:
  foo: YWRtaW4=
  password: dDBwLVNlY3JldA==
  username: YWRtaW4x
type: Opaque

adding a new field in data on manifest

resource "kubectl_manifest" "example" {
  yaml_body = <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: secret-basic-auth
  namespace: default
type: Opaque
data:
  bar: YmFyZS1hdXRoCg==
stringData:
  username: admin1 # required field for kubernetes.io/basic-auth
  password: t0p-Secret # required field for kubernetes.io/basic-auth
EOF
}

All 4 fields are maintained

apiVersion: v1
kind: Secret
metadata:
  name: secret-basic-auth
  namespace: default
  uid: 0449fb6c-6bbe-4a2f-be2d-642dfaa9f163
  resourceVersion: '904360882'
  creationTimestamp: '2024-04-06T18:42:51Z'
  labels:
    k8slens-edit-resource-version: v1
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: >
      {"apiVersion":"v1","data":{"bar":"YmFyZS1hdXRoCg=="},"kind":"Secret","metadata":{"annotations":{},"name":"secret-basic-auth","namespace":"default"},"stringData":{"password":"t0p-Secret","username":"admin1"},"type":"Opaque"}
  managedFields:
    - manager: node-fetch
      operation: Update
      apiVersion: v1
      time: '2024-04-06T18:43:53Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:data:
          f:foo: {}
        f:metadata:
          f:labels:
            .: {}
            f:k8slens-edit-resource-version: {}
    - manager: HashiCorp
      operation: Update
      apiVersion: v1
      time: '2024-04-06T18:46:42Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:data:
          .: {}
          f:bar: {}
          f:password: {}
          f:username: {}
        f:metadata:
          f:annotations:
            .: {}
            f:kubectl.kubernetes.io/last-applied-configuration: {}
        f:type: {}
  selfLink: /api/v1/namespaces/default/secrets/secret-basic-auth
data:
  bar: YmFyZS1hdXRoCg==
  foo: YWRtaW4=
  password: dDBwLVNlY3JldA==
  username: YWRtaW4x
type: Opaque

Did I miss some scenario?

commented

I believe the issue is with something like this (sorry for the succinctness, but don't have time for much more today) in yaml pseudo-code

data:
  field1: in-data-1    # assume base 64 encoded
  field2 in-data-2    # assume base 64 encoded
stringData:
  field1: in-stringdata-1
  field3: in-stringdata-2

As per https://kubernetes.io/docs/concepts/configuration/secret/#restriction-names-data (the second source, in addition to the API link on original post),
The keys of data and stringData must consist of alphanumeric characters, -, _ or .. All key-value pairs in the stringData field are internally merged into the data field. If a key appears in both the data and the stringData field, the value specified in the stringData field takes precedence.
So the resulting equivalent if using only data should be:

data:
  field1: in-stringdata-1  # assume base 64 encoded
  field2 in-data-2    # assume base 64 encoded
  field3: in-stringdata-2

When I looked at the code it seemed this would not be the case, but I may have seen it incorrectly.

On this topic of consistency with the API, while looking for the extra quote above, found another line in kubernetes docs on (https://kubernetes.io/docs/concepts/configuration/secret/#bootstrap-token-secrets)
Note: The stringData field for a Secret does not work well with server-side apply.
To avoid some errors, this should prob be enforced at the level of kube_manifest (either by overriding or aborting if set).

Thanks

commented

It's how it's working now

resource "kubectl_manifest" "example" {
  yaml_body = <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: secret-basic-auth
  namespace: default
type: Opaque
data:
  username: bm90YWRtaW4K # base64 encoded string for "notadmin"
stringData:
  username: admin # required field for kubernetes.io/basic-auth
  password: t0p-Secret # required field for kubernetes.io/basic-auth
EOF
}

resulting manifest

apiVersion: v1
kind: Secret
metadata:
  name: secret-basic-auth
  namespace: default
  uid: 938ccfee-a03d-46b3-930a-73e78f23be06
  resourceVersion: '904817565'
  creationTimestamp: '2024-04-07T10:48:10Z'
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: >
      {"apiVersion":"v1","data":{"username":"bm90YWRtaW4K"},"kind":"Secret","metadata":{"annotations":{},"name":"secret-basic-auth","namespace":"default"},"stringData":{"password":"t0p-Secret","username":"admin"},"type":"Opaque"}
  managedFields:
    - manager: HashiCorp
      operation: Update
      apiVersion: v1
      time: '2024-04-07T10:48:10Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:data:
          .: {}
          f:password: {}
          f:username: {}
        f:metadata:
          f:annotations:
            .: {}
            f:kubectl.kubernetes.io/last-applied-configuration: {}
        f:type: {}
  selfLink: /api/v1/namespaces/default/secrets/secret-basic-auth
data:
  password: dDBwLVNlY3JldA==
  username: YWRtaW4=
type: Opaque

As for the serverside apply, it doesn't matter. As you can see

if stringData, found := userProvided.Raw.Object["stringData"]; found {
			// there is an edge case where stringData might be nil and not a map[string]interface{}
			// in this case we will just ignore it
			if stringData, ok := stringData.(map[string]interface{}); ok {
				// move all stringdata values to the data
				for k, v := range stringData {
					encodedString := base64.StdEncoding.EncodeToString([]byte(fmt.Sprintf("%v", v)))
					meta_v1_unstruct.SetNestedField(userProvided.Raw.Object, encodedString, "data", k)
				}
				// and unset the stringData entirely
				meta_v1_unstruct.RemoveNestedField(userProvided.Raw.Object, "stringData")
			}
		}

Once I've moved stringdata into data, I remove stringData from the manifest via meta_v1_unstruct.RemoveNestedField(userProvided.Raw.Object, "stringData")

Since it looks like the provider is working correctly, I am going to close this one. Feel free to comment for reopening in case I've missed something

commented

Hi. You are correct. My go understanding was not as good as to let me see the setting of data. Sorry for that. Thanks