HNC: Should not allow creating subns with excluded namespace names, e.g. `kube-system`
yiqigao217 opened this issue · comments
While testing configurable excluded namespaces, I found that we may want to add these two new rules in our valiators:
##### Should deny creating subnamespaces with excluded namespace names, but got: #####
$ kubectl hns create kube-public -n a
Successfully created "kube-public" subnamespace anchor in "a" namespace
##### Should output a better message, but got: #####
$ kubectl hns set kube-node-lease --parent a
Setting the parent of kube-node-lease to a
Could not update the hierarchical configuration of kube-node-lease.
Reason: admission webhook "hierarchyconfigurations.hnc.x-k8s.io" denied the request: HNC has not reconciled namespace "kube-node-lease" yet - please try again in a few moments.
Will also look into conditions if the webhook is bypassed. Currently when
- Setting a non-excluded namespace as child of excluded namespace, we get:
$ kubectl hns set a --parent kube-system
Setting the parent of a to kube-system
Succesfully updated 1 property of the hierarchical configuration of a
$ kubectl hns tree kube-system
kube-system
$ kubectl hns tree a
a (1)
Conditions:
1) ActivitiesHalted (ParentMissing): Parent "kube-system" does not exist
- Setting an excluded namespace to a non-excluded namespace, we get nothing but the HC object is created.
- Creating subns in excluded namespace with an existing name, we get
Conflictinsubns.status. - Creating subns in exlcuded namespace with a new name, we get
Forbiddeninsubns.status. - Creating subns using excluded namespace name in a non-excluded namespace, we get
Forbiddeninsubns.status.
We need to make the first two conditions more explicit and specific for excluded namespaces.
My proposal for the above webhook-bypassed cases are
- Setting a non-excluded namespace as child of excluded namespace - The HC singleton will get
ActivitiesHalted (ParentMissing): Parent "kube-system" is an excluded namespacecondition; - Setting an excluded namespace as a child of a non-excluded namespace - Our reconciler will remove the HC singleton directly instead of adding a new condition for it (say
BadConfiguration (NamespaceExcluded): "kube-system" is an excluded namespace and should not have a hierarchyconfiguration object); - All 3 subns status mentioned above remain the same.
Since webhook being bypassed is rare and so are the above cases, I'm trying to make the minimum changes to our current conditions and the HC reconciler. Does this sgty? @adrianludwin @rjbez17
For the first one, I think I'd prefer something like IllegalParent instead of ParentMissing - the parent isn't actually missing, it's just not allowed.
For the second one - hmm. I'm not sure. @rjbez17 wdyt? I like that it makes it very clear that a bug will never allow us to accidentally put an HC object into kube-system, but it seems a little weird to delete it. I really appreciate that it's a minimal change to the current code though! I think I'm ok with it if Ryan is.
Otherwise lgtm, thanks!
The first one sgtm.
As for the 2nd one, OR we can have a condition called Illegal with message telling users to delete the HC singleton (in the excluded namespace). However, it will be much more complicated than the HNC direct deleting approach mentioned above. Currently, HNC doesn't reconcile resources in excluded namespaces at all (with an early exist in all Reconcile() functions). Adding conditions to resources, HC singleton specifially, in excluded namespace would require reconciling these resources.
Wdyt @rjbez17 ?
Sorry, I was OOO the last few days. As an SRE, I have a slight preference to not magically delete things. However, as a dev, I agree that the added complexity is simply not worth it. I'd prefer to keep it simple now, we can always add complexity later.