Handle eraser pod and PodTemplate deletion on controller manager restart
ashnamehrotra opened this issue · comments
What steps did you take and what happened:
[A clear and concise description of what the bug is.]
AKS ImageCleaner user raised issue:
If eraser-controller-manager pod is restarted, the ImageJob is cleaned up but the PodTemplate and eraser pods are left behind (since controller-manager-pod is not deleted in this case). In this example, manager pod restarted twice and two PodTemplates are left behind:
...
eraser-aks-upools1z3-40582422-vmss00003t-7wsdb 0/3 Completed 0 43h
eraser-aks-upools1z3-40582422-vmss00003t-d5hbv 0/3 Completed 0 2d20h
eraser-aks-upools1z3-40582422-vmss00003u-ckw2s 0/3 Completed 0 2d20h
eraser-aks-upools1z3-40582422-vmss00003u-x8sg7 0/3 Completed 0 43h
eraser-aks-upools1z3-40582422-vmss00003v-2hgsf 0/3 Completed 0 43h
eraser-aks-upools1z3-40582422-vmss00003v-5wbp6 0/3 Completed 0 2d20h
eraser-controller-manager-7bcc57b846-n52f7 1/1 Running 2 (43h ago) 3d20h
...
kubectl get imagejob -A
No resources found
kubectl get podtemplate -A
NAMESPACE NAME CONTAINERS IMAGES POD LABELS
kube-system imagejob-dhtbn collector,remover,trivy-scanner mcr.microsoft.com/oss/eraser/collector:v1.2.2,mcr.microsoft.com/oss/eraser/remover:v1.2.2,mcr.microsoft.com/oss/eraser/eraser-trivy-scanner:v1.2.2
kube-system imagejob-r5vlc collector,remover,trivy-scanner mcr.microsoft.com/oss/eraser/collector:v1.2.2,mcr.microsoft.com/oss/eraser/remover:v1.2.2,mcr.microsoft.com/oss/eraser/eraser-trivy-scanner:v1.2.2
What did you expect to happen:
PodTemplates and Pods should be cleaned up if manager restarts, not just when it is deleted.
Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]
Environment:
- Eraser version: v1.2.2
- Kubernetes version: (use
kubectl version):