StatCan / aaw-kubeflow-mlops

Kubeflow MLOps pipeline using GitHub Actions

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Reusable Yaml components

blairdrummond opened this issue · comments

Might be good to have pluggable yaml components for basic tasks like data loading. Example

If we make them documented and modular, we might also want to clone them into the notebook-servers by default.

CC @ca-scribner

hah you beat me to the card writing. 100% agree!

I actually started putting some together for minio earlier this week. Haven't found a home in /Statcan yet (was going to but then got side tracked with other work), but what do you think of these? So far just minio in there, but they're generic components that can optionally have creds (or find them for you), they wait for creds to be mounted, etc.

I've got a few more ideas for others but if you've got any in mind that would be great