Resources plugin

[Alex Collins]

Summary

Today - the resources template has two downsides:

• It is expensive as it requires one pod per resource.
• It only supports a single resource.
• Resource manifests must come from the workflow.
• Resources are strings, not structured YAML

It would also be an opportunity to complete this:

• #7725
• #7179
• #7143
• #6862

I propose a new plugin:

plugins:
  resources:
    -  manifest:
          value: 
          valueFrom: 
            url: ...
       wait:
          for: condition=Ready
          status:
            phase: Running

---

Message from the maintainers:

Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.

[ybyang]

very useful feature. Does this plugin means a implment of excutor plugin? @alexec

[Alex Collins]

Correct. Plug-ins are coming in v3.3 and it would be easy for anyone to write a turbo-charged version of the resources template themselves.

[ybyang]

Correct. Plug-ins are coming in v3.3 and it would be easy for anyone to write a turbo-charged version of the resources template themselves.

Good. I have read the doc , and i'll do a job for adapting the Volcano Job, when the v3.3 will be released ? @alexec

[ybyang]

@alexec hi , i just implement a demo plugin , and i found that , every workflow will start an agent with a plugin sidecar container, i have some question here, do we really need sidecar? why not we make the plugin run forever and all the same extended type workflow share the plugin? i think per workflow with an sidecar is expensive too.

[Niklas Hansson]

i think per workflow with an sidecar is expensive too.

I think it would great if it is possible to have plugins that lived outside the scope of a workflow in ordet to allow plugins to persist over multiple workflows.

[Alex Collins]

@berlinsaint that is currently the architecture. There is an open enhancement to run one agent per namespace. However, there are benefits of running one per workflow, e.g. it can use a different service account, it is not running when not needed.

If your plugin is heavy weight, then you could split it into two parts, one part is a normal Kubernetes, and another part is normal plugin which proxies requests.

Alternativey, maybe we need a new type of plugin other than sidecar, maybe endpoint - i.e. it does not start a sidecar, it just expect an endpoint to be running.

[Alex Collins]

#8142

[ybyang]

@berlinsaint that is currently the architecture. There is an open enhancement to run one agent per namespace. However, there are benefits of running one per workflow, e.g. it can use a different service account, it is not running when not needed.

If your plugin is heavy weight, then you could split it into two parts, one part is a normal Kubernetes, and another part is normal plugin which proxies requests.

Alternativey, maybe we need a new type of plugin other than sidecar, maybe endpoint - i.e. it does not start a sidecar, it just expect an endpoint to be running.

kind of understanding that. thanks

[Tan-JiaLiang]

Hi @alexec , I tried to implement v3.4.4's Executor Plugin, but i found i could not catch the action like suspend/terminate/stop/resume etc. Did executor plugin has something API we can use? For example, I want to implement a spark/flink plugin, which will submit a job into spark/flink cluster. When i stop the workflow, the plugin should stop the spark/flink job too.

[Alex Collins]

I don’t think those actions are implemented I’m afraid. Maybe create a new issue?

[Tan-JiaLiang]

See #10476

[Stephen]

is there example for this proposal?