kapp should support `CiliumIdentity` resources out of the box
praveenrewar opened this issue · comments
What steps did you take:
I am using [Cilium](https://cilium.io] and I deployed a simple app with a Deployment and a Service.
$ kapp deploy -a simple-app -f https://github.com/carvel-dev/kapp/blob/develop/examples/simple-app-example/config-1.yml
Target cluster 'https://xx.xxx.xx.xxx' (nodes: gk3-xxx-xxx-default-pool, 2+)
Changes
Namespace Name Kind Age Op Op st. Wait to Rs Ri
default simple-app Deployment - create - reconcile - -
^ simple-app Service - create - reconcile - -
Op: 2 create, 0 delete, 0 update, 0 noop, 0 exists
Wait to: 2 reconcile, 0 delete, 0 noop
Continue? [yN]: y
11:07:03AM: ---- applying 2 changes [0/2 done] ----
Warning: Autopilot set default resource requests for Deployment default/simple-app, as resource requests were not specified. See http://g.co/gke/autopilot-defaults
11:07:04AM: create service/simple-app (v1) namespace: default
11:07:06AM: create deployment/simple-app (apps/v1) namespace: default
...snip...
11:09:13AM: ---- applying complete [2/2 done] ----
11:09:13AM: ---- waiting complete [2/2 done] ----
SucceededWhat happened:
When I delete the app, kapp gets stuck on waiting for the CiliumIdentity resource to get deleted by the cluster.
$ kapp delete -a simple-app
Target cluster 'https://xx.xxx.xx.xxx' (nodes: gk3-xxx-xxx-default-pool, 2+)
Changes
Namespace Name Kind Age Op Op st. Wait to Rs Ri
(cluster) 22690 CiliumIdentity 3m - - delete ok -
default simple-app Deployment 5m delete - delete ok -
^ simple-app Endpoints 5m - - delete ok -
^ simple-app Service 5m delete - delete ok -
^ simple-app-64dccdbdf5 ReplicaSet 5m - - delete ok -
^ simple-app-64dccdbdf5-smkjb CiliumEndpoint 3m - - delete ok -
^ simple-app-64dccdbdf5-smkjb Pod 5m - - delete ok -
^ simple-app-64dccdbdf5-smkjb PodMetrics 2s - - delete ok -
^ simple-app-7mdbq EndpointSlice 5m - - delete ok -
Op: 0 create, 2 delete, 0 update, 7 noop, 0 exists
Wait to: 0 reconcile, 9 delete, 0 noop
Continue? [yN]: y
11:12:13AM: ---- applying 9 changes [0/9 done] ----
11:12:13AM: noop ciliumendpoint/simple-app-64dccdbdf5-smkjb (cilium.io/v2) namespace: default
11:12:13AM: noop pod/simple-app-64dccdbdf5-smkjb (v1) namespace: default
11:12:13AM: noop replicaset/simple-app-64dccdbdf5 (apps/v1) namespace: default
11:12:13AM: noop endpoints/simple-app (v1) namespace: default
11:12:13AM: noop endpointslice/simple-app-7mdbq (discovery.k8s.io/v1) namespace: default
11:12:13AM: noop podmetrics/simple-app-64dccdbdf5-smkjb (metrics.k8s.io/v1beta1) namespace: default
11:12:13AM: noop ciliumidentity/22690 (cilium.io/v2) cluster
11:12:13AM: delete deployment/simple-app (apps/v1) namespace: default
11:12:13AM: delete service/simple-app (v1) namespace: default
...snip...
11:12:17AM: ---- waiting on 1 changes [8/9 done] ----
11:13:15AM: ongoing: delete ciliumidentity/22690 (cilium.io/v2) cluster
11:13:18AM: ---- waiting on 1 changes [8/9 done] ----
11:14:16AM: ongoing: delete ciliumidentity/22690 (cilium.io/v2) cluster
11:14:19AM: ---- waiting on 1 changes [8/9 done] ----
^CWhat did you expect:
I expected kapp delete to succeed.
Anything else you would like to add:
When we create a an app using kapp which has a pod resource (like Deployment), then the ownership labels are injected into the Pods and the ReplicaSets as well, and into the CiliumIdentity resource in case of GKE autopilot cluster. While deleting the app, since CiliumIdentity is a cluster owned resource, kapp waits for the cluster to delete it, but these resources are not deleted immediately so kapp gets stuck waiting for the cluster to delete them.
Slack thread: https://kubernetes.slack.com/archives/CH8KCCKA5/p1683315623187379
Environment:
- kapp version (use
kapp --version): - OS (e.g. from
/etc/os-release): - Kubernetes version (use
kubectl version)
Vote on this request
This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.
👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"
We are also happy to receive and review Pull Requests if you want to help working on this issue.
kapp v0.56.0 contains a fix for this issue which excludes the CiliumIdentity resources during listing. Next steps would be to provide this functionally via the kapp configuration and also provide a a way to disable it. Keeping this issue open to track that work.