How to watch resources in the same group with different served versions?
ethnB opened this issue · comments
Keywords
watch-stream multiple resources
Problem
When using a very simple operator to watch two different kinds of resource in the same group, it seems to only be able to watch one 'served' version at once. Would you please be able to tell me if I'm doing something wrong, or if this is expected behaviour?
My use-case is that a separate service creates CRDs (in the group being watched, usually with different versions) before my kopf operator starts. My understanding was that the decorator I've used should allow watching of all resources in the group, but again, it seems to only be able to handle one 'served' version at once.
Deployed the following (with all the required RBAC pieces):
My operator
CRD 1
CRD 2
First deployment log
Bumped the version of CRD 2 and redeployed:
CRD 2 (version incremented)
Second deployment log
As you can see in the log, the second deployment only starts a single watch stream, whereas I thought I'd get a watch-stream per resource per served version in that group.
kubectl api-resources | grep github
firstresources com.github/v1-alpha1 true FirstResource
secondresources com.github/v1-alpha2 true SecondResource
I did find a bug logged in March 23, but was unsure if related (as this refers to admission webhooks)? #1013
Seems if I mark both versions of CRD 2 as served then I get two watch-streams (one per resource), but only for the v1-alpha1 version. In that case, I'd have thought there would have been 3 streams (two for v1-alpha1, one for v1-alpha2)?
#1013 is related to admission webhooks. When trying to run admission webhooks on API groups with multiple versions the webhook configuration only includes the "preferred" version, so resources in other versions are not subjected to the admission policy.
It does seem like kopf is only able to start a watch-stream for the highest priority version returned from kubectl api-versions when using a group wide handler like mine - is this intended behaviour?
Hello. Yes, this is the intended explicitly coded behavior — serving only the default/preferred version out of the group.
The logic behind this is, first, that different resource versions have different structure and therefore must have different code to serve them. And second, that other versions are “converted” to the served ones by k8s itself, but I might have misunderstood how it works.
If you know that your code can safely handle both versions, you list the version selectors explicitly (as 2+ decorators on the same handler function).
There is an idea to implement customizable resource selectors in addition to those string specifiers, so you can choose which versions to choose yourself. This is not in the making yet.
Besides, dynamic resource & namespace control was discussed in #1040 & co — which would be another approach to this problem.
PS: If explicit decorators serve the wrong versions, this is a bug. Explicit specifiers must work as declared with no hidden logic behind.
@nolar thank you for answering, understandable logic.
P.S. great work with kopf - really useful framework.