Proxy returns 500 on image HEAD requests when CachedImage does not exist
dennislapchenko opened this issue · comments
There are cases (ie. gitlab-runner jobs) where returning a 500 for uncached image is not an option, because the caller does not have retry capability.
This is supposedly acceptable behaviour because k8s self-heals well. But some actors (gitlab jobs) do not.
Currently have built own version which checks if image is cached during pod mutation, if not - no mutation happens. This solves gitlab job issues, but not sure if this is a solution that should be merged into master with a feature flag. LMK
Perhaps there is a more elegant solution to NOT return 500 to HEAD requests when image is going to be proxied from remote origin.
Your issue here is that there is no CachedImage for this repository, as the first line states. This is most probably because of your patch, indeed, if you don't mutate the pod, the CachedImage is not created, and with your patch, if it is not created, the pod is not mutated. So basically you just prevented kuik to ever work.
I'm closing the issue since the bug most probably comes from this patch. If you're sure it doesn't, feel free to re-open it with a minimal reproductible example.
@paullaffitte the log screenshot is before the patch. It does seem this 500 always happens on first image appearance.
But what you are saying is that without mutation, which includes new labels - the image never turns to CachedImage?
Is there a reason the (unpatched) controller is returning 500 on initial CachedImage nonexistence?
I will try to come up with an easily reproducible behaviour but it does seem any fresh image in the cluster produces this