Giters

vmware-tanzu / sonobuoy

Sonobuoy is a diagnostic tool that makes it easier to understand the state of a Kubernetes cluster by running a set of Kubernetes conformance tests and other plugins in an accessible and non-destructive manner.

Home Page:https://sonobuoy.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

error retrieving results: non-zero data ! (byte 33) read after tar EOF

katexochen opened this issue · comments

commented

What steps did you take and what happened:

Running sonobuoy retrieve --kubeconfig constellation-admin.conf in a CI run.

What did you expect to happen:

Getting the results. What causes this type of error?

Environment:

  • Sonobuoy version: v0.56.16
  • Kubernetes version: (use kubectl version): v1.27.0
  • Kubernetes installer & version: constellation (v2.8.0-pre, dev-build)
  • Cloud provider or hardware configuration: Azure
  • OS (e.g. from /etc/os-release): Fedora (custom)
  • Sonobuoy tarball (which contains * below)
commented

Hi @katexochen

I believe this may be related to #1633. There is a WIP pull request to add retries though I believe this to be a race condition between the initial calling function spawning a go routine to fetch the asset and returning pre-emptively a reader that may or may not be populated. To help in attempting reproduction. I have a few more questions.

Are the log files/test outputs being retrieved from a large count of nodes, or potentially large file sizes? or is there increased latency in the network between sonobuoy CLI and the cluster the results bundle is being retrieved from.

commented

Are the log files/test outputs being retrieved from a large count of nodes

This was a cluster with 3 control planes and 2 worker nodes

or potentially large file sizes?

Our logs are usually ~3 MB.
We run the following plugins (in case that influences the file size):

--plugin e2e
--plugin-env e2e.E2E_FOCUS="\[Conformance\]" --plugin-env e2e.E2E_SKIP="for service with type clusterIP|HostPort validates that there is no conflict between pods with same hostPort but different hostIP and protocol"
--plugin https://raw.githubusercontent.com/vmware-tanzu/sonobuoy-plugins/master/cis-benchmarks/kube-bench-plugin.yaml
--plugin https://raw.githubusercontent.com/vmware-tanzu/sonobuoy-plugins/master/cis-benchmarks/kube-bench-master-plugin.yaml'

or is there increased latency in the network between sonobuoy CLI and the cluster the results bundle is being retrieved from

Might be the case. The network between GitHub runner and major cloud providers is quite slow from time to time. But no other indication for slow network in this specific run.

commented

There has not been much activity here. We'll be closing this issue if there are no follow-ups within 15 days.