Method to determine whether kotsadm-operator was able to get elevated permissions to deploy CRDs

[Dex]

As an application packager, I'd like to configure my application to deploy CRDs along with the application itself. This only works, however, if kotsadm-operator was able to get elevated privileges at the cluster scope. I'd like to be able to query whether this happened before deploying my app, because if we don't have permission to add CRDs, then I will need kotsadm to run a preflight check to check those CRDs exist.

Not sure if this is a template function or annotation or what. The acceptance criteria for this story is the ability to dynamically decide whether to either

1. include a preflight check for CRDs and skip CRD deploy
2. deploy CRDs at runtime and skip preflight check

and to execute that. I'm not sure if preflight supports conditional evaluation yet but if not then that should part of the scope. We also need to make sure CRDs can be dynamically excluded.

Example (aspirational)

In preflight.yaml

analyzers:
  - customResourceDefinition:
       apiVersion: some.group.com
       kind: myobject
       when: repl{{ not (KotsadmOperatorClusterAdmin) }}

In crds.yaml

kind: CustomResourceDefinition
metadata:
  name: myobject
  annotations:
    kots.io/when: repl{{ KotsadmOperatorClusterAdmin }}

[Marc Campbell]

Beginning in KOTS 1.14.0, we've made it possible for an application packager to create more predictable RBAC setups. The pre 1.14.0 behavior of requesting cluster-scoped RBAC but falling back to namespace-scoped RBAC is really fragile and bound to break and produce unpredictable installations.

Starting in 1.14, we've added a few new fields to the apiVersion: kots.io/v1beta1, Kind=Application spec to allow a vendor to define their RBAC expectations for existing cluster installations.

Instead of recommending that a vendor push CRDs when permissions are elevated enough, I think we'd be better served by creating options for vendor to include the option on the config screen and expose the option to the user to include the CRD to existing clusters.

Currently, this issue could be confusing because a CRD could be added at an update in the future, and it's ability to be deployed would be dependent on the user's kubeconfig permissions at installation time in the past, while not being obvious or easy to debug. Using the new functionality, a vendor can define the permissions that they'd like here.

There's a lot more to do on permissions and RBAC, including giving the user a preflight output that allows a vendor to change RBAC requirements, including giving a vendor the ability to have multiple RBAC levels in an application.