"kubecfg update" fails with SIGSEGV when updating object from invalid manifest
seh opened this issue · comments
Observed Behavior
Using kubecfg update against a server running Kubernetes version 1.14.3, attempting to update an existing object, kubecfg crashes with the following stack trace:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1d20a61]
goroutine 1 [running]:
k8s.io/apimachinery/pkg/apis/meta/v1/unstructured.(*Unstructured).GetUID(...)
/private/tmp/kubecfg-20190528-66151-n1u7p4/kubecfg-0.12.0/src/github.com/bitnami/kubecfg/vendor/k8s.io/apimachinery/pkg/apis/meta/v1/unstructured/unstructured.go:252
github.com/bitnami/kubecfg/pkg/kubecfg.UpdateCmd.Run(0x23a60e0, 0xc0000c2378, 0x23edee0, 0xc00036e930, 0x23f2800, 0xc00004dfc0, 0xc000584a70, 0x8, 0x1, 0x7ffeefbff5a0, ...)
/private/tmp/kubecfg-20190528-66151-n1u7p4/kubecfg-0.12.0/src/github.com/bitnami/kubecfg/pkg/kubecfg/update.go:315 +0x281
github.com/bitnami/kubecfg/cmd.glob..func6(0x322eda0, 0xc000430750, 0x1, 0x3, 0x0, 0x0)
/private/tmp/kubecfg-20190528-66151-n1u7p4/kubecfg-0.12.0/src/github.com/bitnami/kubecfg/cmd/update.go:107 +0x328
github.com/spf13/cobra.(*Command).execute(0x322eda0, 0xc0004305d0, 0x3, 0x3, 0x322eda0, 0xc0004305d0)
/private/tmp/kubecfg-20190528-66151-n1u7p4/kubecfg-0.12.0/src/github.com/bitnami/kubecfg/vendor/github.com/spf13/cobra/command.go:756 +0x465
github.com/spf13/cobra.(*Command).ExecuteC(0x322e1c0, 0x100570f, 0xc0000a2058, 0x0)
/private/tmp/kubecfg-20190528-66151-n1u7p4/kubecfg-0.12.0/src/github.com/bitnami/kubecfg/vendor/github.com/spf13/cobra/command.go:846 +0x2ec
github.com/spf13/cobra.(*Command).Execute(...)
/private/tmp/kubecfg-20190528-66151-n1u7p4/kubecfg-0.12.0/src/github.com/bitnami/kubecfg/vendor/github.com/spf13/cobra/command.go:794
main.main()
/private/tmp/kubecfg-20190528-66151-n1u7p4/kubecfg-0.12.0/src/github.com/bitnami/kubecfg/main.go:33 +0x5b
At line 315 in file pkg/kubecfg/update.go, it appears that the newobj pointer is nil.
Running kubecfg update --dry-run against the same configuration succeeds with error, but running kubecfg update again against the same configuration fails the same way.
Diagnosis
Running kubectl update --v=8 reveals more detail: in particular, one of the objects is deemed invalid (a field for a Deployment's "spec.template.spec.containers.resources.requests" field has the wrong name, "mem" instead of "memory"). kubecfg attempts to compute the difference between the proposed and existing objects. The existing object retrieved from the server is null. Hence, it appears that kubecfg is not detecting the server-side schema validation correctly.
I0610 15:34:48.269317 78935 request.go:942] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Deployment.apps \"m3-collector\" is invalid: [spec.template.spec.containers[0].resources.requests[mem]: Invalid value: \"mem\": must be a standard resource type or fully qualified, spec.template.spec.containers[0].resources.requests[mem]: Invalid value: \"mem\": must be a standard resource for containers]","reason":"Invalid","details":{"name":"m3-collector","group":"apps","kind":"Deployment","causes":[{"reason":"FieldValueInvalid","message":"Invalid value: \"mem\": must be a standard resource type or fully qualified","field":"spec.template.spec.containers[0].resources.requests[mem]"},{"reason":"FieldValueInvalid","message":"Invalid value: \"mem\": must be a standard resource for containers","field":"spec.template.spec.containers[0].resources.requests[mem]"}]},"code":422}
DEBUG Update(m3-collector) returned (<nil>, Deployment.apps "m3-collector" is invalid: [spec.template.spec.containers[0].resources.requests[mem]: Invalid value: "mem": must be a standard resource type or fully qualified, spec.template.spec.containers[0].resources.requests[mem]: Invalid value: "mem": must be a standard resource for containers])
DEBUG Updated object:
A: {"apiVersion":"apps/v1","kind":"Deployment"...
B: null
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1d20a61]
At line 199 in file pkg/kubecfg/update.go in the createOrUpdate function, we receive a nil *Unstructured back from ResourceInterface.Update, together with a non-nil error. Five lines later, though, we unconditionally return a nil error value. At line 204, it looks like it should read as follows:
return newobj, errAlternately, return early after line 202.
Software
kubecfg version
kubecfg version: v0.12.0
jsonnet version: v0.12.0
client-go version: v0.0.0-master+6d1a9b9
Kubernetes version
version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.2", GitCommit:"66049e3b21efe110454d67df4fa62b08ea79a19b", GitTreeState:"clean", BuildDate:"2019-05-16T16:14:56Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
It would be nice to publish a release that includes the fix for this problem. It bites us frequently, as we're still using the most recently released version 0.12.0.
@seh; on it (fighting some release automation issue)
Thank you. I see evidence of the struggle. We will prevail.
Do I dare ask: Does any of this trigger new Homebrew releases too?
Do I dare ask: Does any of this trigger new Homebrew releases too?
I think somebody have to trigger them manually, see Homebrew/homebrew-core#40385
@seh; turns out I have to re-create the github secrets so that travis can upload the binaries.
I uploaded the binaries manually, PTAL
I've done it before, though I wasn't able to get that bump-formula-pr tool to work on my computer.
somebody is already doing it Homebrew/homebrew-core#42677 :-)