Using CRD resources for get commands

[Robert Bost]

Hello,

I'm curious if you can use the CRD inside of must-gathers as a source of information to complement the resource map that is manually maintained right now. The CRD contains names of resources you can omg get as well as the column names you would print in output.

I was going to start a PR on this, but curious if you've gone through this already @kxr. A small bump in the road I have hit is that loading all the CRD is quite slow (~0.6s); that would introduce quite a bit of latency if start parsing it each omg get command.

[Khizer Naeem]

If I got your idea right, I actually thought about it when I initially wrote the tool. I didn't go as far as you have, to actually test it and check the performance. However my thought was that we anyway have to have a database of where to load the yaml files from, so why not let it (e.g, the map dict) be the source of info for the sake of simplicity and completion. Hence I scratched the idea.

Having said that, we can revisit the idea. So suppose if you load the CRDs from inside the must-gather, how will you tell omg where to look for yaml(s) for a specific CRD or standard resource (pod, secrets etc.) ? May be search for filenames inside the must-gather? If there is someway to address the file location problem and performance is the only issue may be we can generate the map dict at time when user runs omg use and then save it on disk, which can then be used for subsequent calls?

Hmm, may be this can solve the current feature that I am contemplating on. i.e, I want to be able to provide multiple directories to omg. This would enable us to troubleshoot a custom namespace. So for example, if we are intested in a (one or more) custom projects, we can ask the user to provide must-gather + inspect of that particular namespace(s) and point omg to those (one or more) directories. And omg will combine the yamls from all directories when showing us the omg outputs. This will also address must-gathers ran with custom (e.g, service mesh etc.) images. I have done some work on this, but I stopped and was waiting to move to click before spending more time on this.

If I didn't get your idea right, I apologize :)

[Robert Bost]

So suppose if you load the CRDs from inside the must-gather, how will you tell omg where to look for yaml(s) for a specific CRD or standard resource (pod, secrets etc.) ?

The CRD live in a standard location inside must-gathers, so that location could be hard-coded in omg. The location is cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/. I believe these files contain all of the information we would need to locate the resources. Of course, we'd still have to maintain a mapping for non-CRD resources (Pods, Nodes, etc).

I'll use InstallPlans as an example, matching the CRD contents with fields in the resource map dict we use now.

• The following satisfies the type and aliases fields in resource map dict. This could also be used to build the yaml_loc path (<scope>/<%s namespace>/<group>/<plural>.yaml or just <plural>/)

  group: operators.coreos.com
  names:
    categories:
    - olm
    kind: InstallPlan
    listKind: InstallPlanList
    plural: installplans
    shortNames:
    - ip
    singular: installplan
  scope: Namespaced

• The need_ns map entry would be satisfied by the scope field above.
• The additionalPrinterColumns could be used to replace the type specific *_out.py files with some more generic code printing the "simple_out" columns and then whatever is listed in additional printer columns:

  versions:
  - additionalPrinterColumns:
    - description: The first CSV in the list of clusterServiceVersionNames
      jsonPath: .spec.clusterServiceVersionNames[0]
      name: CSV
      type: string
    - description: The approval mode
      jsonPath: .spec.approval
      name: Approval
      type: string
    - jsonPath: .spec.approved
      name: Approved
      type: boolean

May be search for filenames inside the must-gather? If there is someway to address the file location problem and performance is the only issue may be we can generate the map dict at time when user runs omg use and then save it on disk, which can then be used for subsequent calls?

Yeah, that's a good idea. Caching a quick-access reference for the CRD would probably work and speed things up. The CRD shouldn't be changing on disk for a must-gather, but that's probably easy to address if it becomes a use-case (using file timestamps or something).

[Khizer Naeem]

@bostrt I implemented a fallback. What do you think?

[Robert Bost]

Yes, I just tested that and it works great :) There's no autocomplete but I think its a worthwhile start. Having that logic run each time you hit tab might be too slow right now anyways.