Ansible variable evaluation type mismatch

Question

Ansible variable evaluation type mismatch

Nik-Novak opened this issue 4 years ago · comments

Nik-Novak commented 4 years ago

Bug Report

What did you do?
A clear and concise description of the steps you took (or insert a code snippet).

CR spec:
spec: workshopID: 30001

tasks/main.yaml:
`- debug:
msg: "{{ workshop_id | type_debug }}"

debug:
msg: "{{ workshop_id | int | type_debug }}"`

These statements produced:
`TASK [cleanup : debug] *********************************************************
task path: /opt/ansible/roles/cleanup/tasks/main.yml:17
ok: [localhost] => {
"msg": "int"
}

TASK [cleanup : debug] *********************************************************
task path: /opt/ansible/roles/cleanup/tasks/main.yml:19
ok: [localhost] => {
"msg": "int"
}`

So it's evaluating as int, but then causes an error upon use in port field:
ports: - protocol: TCP nodePort: "{{workshop_id}}" port: 80 # pod/svc internal port maps to... targetPort: 80 # container internal port

The error:
TASK [resources : Create a nodeport service - Workshop's Resources] ********** task path: /opt/ansible/roles/resources/tasks/main.yml:11 fatal: [localhost]: FAILED! => {"changed": false, "error": 400, "msg": "Failed to create object: b'{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"Service in version \\\\\"v1\\\\\" cannot be handled as a Service: v1.Service.Spec: v1.ServiceSpec.Ports: []v1.ServicePort: v1.ServicePort.NodePort: readUint32: unexpected character: \\\\ufffd, error found in #10 byte of ...|odePort\\\\\":\\\\\"30001\\\\\",\\\\\"po|..., bigger context ...|3b-5bc6dba6cc92\\\\\"}]},\\\\\"spec\\\\\":{\\\\\"ports\\\\\":[{\\\\\"nodePort\\\\\":\\\\\"30001\\\\\",\\\\\"port\\\\\":80,\\\\\"protocol\\\\\":\\\\\"TCP\\\\\",\\\\\"targetPort\\\\\":80}|...\",\"reason\":\"BadRequest\",\"code\":400}\\n'", "reason": "Bad Request", "status": 400}

I also tried casting to int with "{{workshop_id | int}}", but received the same error:
TASK [resources : Create a nodeport service - Workshop's Resources] ********** task path: /opt/ansible/roles/resources/tasks/main.yml:11 fatal: [localhost]: FAILED! => {"changed": false, "error": 400, "msg": "Failed to create object: b'{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"Service in version \\\\\"v1\\\\\" cannot be handled as a Service: v1.Service.Spec: v1.ServiceSpec.Ports: []v1.ServicePort: v1.ServicePort.NodePort: readUint32: unexpected character: \\\\ufffd, error found in #10 byte of ...|odePort\\\\\":\\\\\"30001\\\\\",\\\\\"po|..., bigger context ...|26-a1189a80de41\\\\\"}]},\\\\\"spec\\\\\":{\\\\\"ports\\\\\":[{\\\\\"nodePort\\\\\":\\\\\"30001\\\\\",\\\\\"port\\\\\":80,\\\\\"protocol\\\\\":\\\\\"TCP\\\\\",\\\\\"targetPort\\\\\":80}|...\",\"reason\":\"BadRequest\",\"code\":400}\\n'", "reason": "Bad Request", "status": 400}

What did you expect to see?
A clear and concise description of what you expected to happen (or insert a code snippet).
Expected to see a service created, and the integer value specified passing Resource Creation Validation. Jinja and ansible variables re supposed to preserve their native definition type now, as of this: pallets/jinja#708

What did you see instead? Under which circumstances?
A clear and concise description of what you expected to happen (or insert a code snippet).
Failed builds due to this error:
TASK [resources : Create a nodeport service - Workshop's Resources] ********** task path: /opt/ansible/roles/resources/tasks/main.yml:11 fatal: [localhost]: FAILED! => {"changed": false, "error": 400, "msg": "Failed to create object: b'{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"Service in version \\\\\"v1\\\\\" cannot be handled as a Service: v1.Service.Spec: v1.ServiceSpec.Ports: []v1.ServicePort: v1.ServicePort.NodePort: readUint32: unexpected character: \\\\ufffd, error found in #10 byte of ...|odePort\\\\\":\\\\\"30001\\\\\",\\\\\"po|..., bigger context ...|26-a1189a80de41\\\\\"}]},\\\\\"spec\\\\\":{\\\\\"ports\\\\\":[{\\\\\"nodePort\\\\\":\\\\\"30001\\\\\",\\\\\"port\\\\\":80,\\\\\"protocol\\\\\":\\\\\"TCP\\\\\",\\\\\"targetPort\\\\\":80}|...\",\"reason\":\"BadRequest\",\"code\":400}\\n'", "reason": "Bad Request", "status": 400}

Environment

operator-sdk version:
operator-sdk version: "v0.15.2", commit: "ffaf278993c8fcb00c6f527c9f20091eb8dd3352", go version: "go1.13.3 linux/amd64"

go version: n/a
go version go1.13.8 linux/amd64

Kubernetes version information:
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"06ad960bfd03b39c8310aaf92d1e7c12ce618213", GitTreeState:"clean", BuildDate:"2020-02-11T18:14:22Z", GoVersion:"go1.13.6", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"06ad960bfd03b39c8310aaf92d1e7c12ce618213", GitTreeState:"clean", BuildDate:"2020-02-11T18:07:13Z", GoVersion:"go1.13.6", Compiler:"gc", Platform:"linux/amd64"}

Kubernetes cluster kind:
default/regular
Are you writing your operator in ansible, helm, or go?
ansible

Possible Solution

Tried casting to int with "int" filter.
Tried looking for configuration files to enable jinja NativeTypes.
Hard-coding any value works fine.

Additional context
These sources show that significant effort was put into attempting to solve this exact issue, yet the solutions dont seem to work for me.
pallets/jinja#708
https://jinja.palletsprojects.com/en/2.10.x/nativetypes/

Also, here's my CRD validation spec for Workshop:
validation: openAPIV3Schema: type: object properties: spec: type: object required: - workshopID properties: workshopID: # type: integer format: int32 minimum: 30000 maximum: 32767 description: Unique integer identifier for this particular virtual workshop. 30000 <= workshopID <= 32767 example: 30001

Nik-Novak · Answer 1 · Fri Feb 28 2020 21:48:21 GMT+0800 (China Standard Time)

Other people are also able to reproduce this

Camila Macedo · Answer 2 · Fri Feb 28 2020 21:56:45 GMT+0800 (China Standard Time)

I think it is related to how are you creating the resource. See an example which works in GO;

                    Ports: []corev1.ServicePort{
				{
					Name: db.Name,
					TargetPort: intstr.IntOrString{
						Type:   intstr.Int,
						IntVal: db.Spec.DatabasePort,
					},
					Port:     db.Spec.DatabasePort,
					Protocol: "TCP",
				},
			},

You need to define the TargetPort.Type.

If it does not help you solve the issue. Could you please share the code which will create your Create a nodeport service?

Nik-Novak · Answer 3 · Fri Feb 28 2020 22:52:23 GMT+0800 (China Standard Time)

I have a hunch the issue is with Jinja returning a Python-native integer type which is not translating somehow at the kubernetes API level.

Regardless here's an operator with all of the relevant code:
https://drive.google.com/file/d/1cYlu1ty-zn7SNv6-QOqTNiJN05W8l5Mn/view?usp=sharing

See roles/resources/templates/svc-gmeclient.yaml.j2 for relevant problem

I've made an example using another project, since I could not share the Workshop code. Confirmed that this also has the same issue.

Will update when I try to define port types

Camila Macedo · Answer 4 · Fri Feb 28 2020 23:02:40 GMT+0800 (China Standard Time)

HI @Nik-Novak,

Thank you for the data provided. But it is a big project which we have not knowledge about.
Note that it shows that you are not creating the resource correctly. IHMO shows that you are missing to define the TargetPort.Type.

So, could you please add here the snippet code that you are using to create the resource and/or share a POC based in a new project done from the scratch or the steps required for we reproduce it?

Nik-Novak · Answer 5 · Fri Feb 28 2020 23:11:03 GMT+0800 (China Standard Time)

I'm putting together a bare-bones operator with a script to recreate the issue. Will post in about 15 minutes

Nik-Novak · Answer 6 · Sat Feb 29 2020 02:05:15 GMT+0800 (China Standard Time)

We figured out the issue here: https://kubernetes.slack.com/archives/CAW0GV7A5/p1582893529107700

it has to do with ansible vs jinja being used to source templated variables. Thanks to @fabianvf

It's not necessarily a bug, but a very misleading implementation of Jinja and Ansible variable sourcing

Ryan Conway · Answer 7 · Sat Jun 12 2021 01:41:17 GMT+0800 (China Standard Time)

@Nik-Novak any chance you could share the discussion from that Slack link please? I don't have access to it, nor does it seem that I can sign up for access.

I've got this same problem, and not sure if it's possible to work around it.

Jeff Gibson · Answer 8 · Thu Mar 10 2022 05:50:29 GMT+0800 (China Standard Time)

@Nik-Novak seconding @Rylon - can you share? Unable to view the kube slack archive.

Jeff Gibson · Answer 9 · Thu Mar 10 2022 06:47:51 GMT+0800 (China Standard Time)

To save future people time-

Setting jinja2_native = True in ansbile.cfg or running ansible like ANSIBLE_JINJA2_NATIVE=True ansible-playbook... works.

Trying to set:

  - mytask:
    environment:
      ANSIBLE_JINJA2_NATIVE: True

did not.

Reference - #1701

This issue is extremely opaque and I spent a lot of time finding the solution. Maybe the docs could be better?