vmware-tanzu / sonobuoy

Sonobuoy is a diagnostic tool that makes it easier to understand the state of a Kubernetes cluster by running a set of Kubernetes conformance tests and other plugins in an accessible and non-destructive manner.

Home Page:https://sonobuoy.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

gojson processing passes failing tests

Smeb opened this issue · comments

commented

What steps did you take and what happened:
We had a test fail in the Setup phase, meaning that the logs show a failure, but the plugin detects a passed test.

What did you expect to happen:
We expect that the test is reported as failing

Anything else you would like to add:
Here are (sanitised) test logs. The only change made was renaming the package.

{"Time":"2023-10-17T11:38:30.536126509Z","Action":"start","Package":"/test-suite/k8s-cluster-tests/clustertests/preflight/kube-system"}
{"Time":"2023-10-17T11:38:30.556740676Z","Action":"output","Package":"/test-suite/k8s-cluster-tests/clustertests/preflight/kube-system","Output":"F1017 11:38:30.556634      77 env.go:375] Setup failure: loading aws config\n"}
{"Time":"2023-10-17T11:38:30.558280565Z","Action":"output","Package":"/test-suite/k8s-cluster-tests/clustertests/preflight/kube-system","Output":"FAIL\t/test-suite/k8s-cluster-tests/clustertests/preflight/kube-system\t0.022s\n"}
{"Time":"2023-10-17T11:38:30.558315464Z","Action":"fail","Package":"/test-suite/k8s-cluster-tests/clustertests/preflight/kube-system","Elapsed":0.022}

This happens on the current main version.