StatCan / aaw-kubeflow-mlops

Kubeflow MLOps pipeline using GitHub Actions

Kubeflow MLOps

This repository contains examples of integrated Kubeflow End-to-End Pipelines (KFP) using GitHub actions. Focus is on the creation of pluggable components that can make it easier to build your own pipelines enabling more advanced machine learning projects.

The architecture is shown in the following diagram:

The current pipelines that have been created provide the following pluggable components:

• Default
• Convolutional Neural Network (CNN)
• Convolutional Neural Network (CNN) with DataBricks

KubeFlow

• List Pipelines
• Publish Pipeline
• Run Pipeline

Integrations

• DataBricks

Registration

• Azure Machine Learning
• Kubeflow Artifacts
• MLFlow

Scoring

• Seldon
• KFServing

TensorFlow

• Preprocess
• Training

Acknowledgements

Extended with code and lessons learnt from the amazing work done by the Kaizen Team over at KaizenTM