Long execution time

[Prince Rachit Sinha]

I tried running kubectl-tree (after modifying the code so that 403 errors are ignored). It took long time for the plugin to print the result.

I0106 13:19:41.260134   50748 query.go:45] all goroutines have returned in 7m53.641176705s

Continued discussion from #14

[Prince Rachit Sinha]

@ahmetb FYI

[Ahmet Alp Balkan]

thanks, no need to @ since it's my repo and I get notifications about everything

[Prince Rachit Sinha]

Found out the issue was the bug I introduced in the code. Still it takes time but significantly less:

I0106 15:02:16.109438   53594 query.go:45] all goroutines have returned in 19.125264491s
I0106 15:02:16.109445   53594 query.go:46] query result: error=<nil>, objects=8991

[Prince Rachit Sinha]

Increased the burst and qps to 2000 each and increased pagination value to 1000. Still no significant improvement. It went down to 17s.

What I can understand is that the large the cluster the more objects are listed. If I have 10k pods running then it will cause listing of all 10k pods which is definitely going to be slow.

The ideal way would require a change in design st. we start using labels of deployment and other objects to list down the children. Or maybe better if we could directly use field selector with ownerID (Not sure if there is a support for this in client-go).

func (od objectDirectory) ownedBy(id types.UID) []unstructured.Unstructured {
	var out sortedObjects
	for k := range od.ownership[id] { // instead of finding owner here make modification in snippet pasted below
		out = append(out, od.getObject(k))
	}
	sort.Sort(out)
	return out
}

Query api:

func queryAPI(client dynamic.Interface, api apiResource) ([]unstructured.Unstructured, error) {
	var out []unstructured.Unstructured

	var next string
	for {
		resp, err := client.Resource(api.GroupVersionResource()).List(metav1.ListOptions{
			Limit:    1000,
			Continue: next,
// The logic to filter should go here
		})

[Ahmet Alp Balkan]

Sadly, server side filtering is not possible.

https://stackoverflow.com/questions/59442065/how-to-use-kubernetes-fieldselector-to-query-ownerreferences

The only improvement I can think of is to query resources only in specified namespace (Except nonnamespaced apis) and separately do a --all-namespaces flag for the current behavior.

I am not sure it’s worth fixing just yet. If it effects 1% users and takes only 20s waiting, I don’t want to fix it and change experience for everyone.