alexcombessie / example-seldon

Example for end-to-end machine learning on Kubernetes using Kubeflow and Seldon Core

End-To-End Train and Deploy Machine Learning Model on Kubernetes (on GKE)

Using:

• kubeflow
• seldon-core

The example will be the MNIST handwriiten digit classification task.

In the follow we will:

1. Do the data science
2. Train the model
3. Serve the model
4. Get predictions

Prerequisites

Either :

1. Follow the kubeflow docs to
   1. Create a persistent disk for NFS. Call it nfs-1.
   2. Install kubeflow with an NFS volume, Argo and seldon-core onto your cluster.
2. Follow a consolidated guide to do the steps in 1.

Data Science

In this demo we provide some example models for MNIST classification. Firstly, a Tensorflow model

• A simple TensorFlow model for MNIST classification
• A script to build and optionaly push to DockerHub a Docker image for the training code
• A runtime TensorFlow inference module that provides a predict method that can be wrapped by seldon-core for deployment.
• A script to build and optionally push to DockerHub a Docker image for the run time predictions

We also provide a simple scikit-learn random forest model which is used in the notebooks to illustrate A-B tests and multi-armed bandits.

• A simple scikit-learn model for MNIST classification
• A runtime scikit-learn inference module that provides a predict method that can be wrapped by seldon-core for deployment.

Train Model

The training and serving steps are written as Argo jobs.

argo submit workflows/training-tf-mnist-workflow.yaml -p tfjob-version-hack=$RANDOM

To understand the workflow in detail and run it with optional parameters to build and push to your own repo see here.

To check on your Argo jobs use argo list and argo get or the Argo UI discussed above.

Serve Model

To wrap our model as a Docker container and launch we create:

argo submit workflows/serving-tf-mnist-workflow.yaml

• See here for detailed comments on workflow

Get Predictions

The cluster is using Ambassador so your model will be exposed by REST and gRPC on the Ambassador reverse proxy.

To expose the ambassador reverse proxy to a local port do

kubectl port-forward $(kubectl get pods -n kubeflow-seldon -l service=ambassador -o jsonpath='{.items[0].metadata.name}') -n kubeflow-seldon 8002:80

You can test the service by following the example jupyter notebook

cd notebooks
jupyter notebook

Next Steps

There is a second model for mnist using an sklearn random forest which can be found in models/sk_mnist.

• You start an A-B test using the two models by using the deployment in k8s_serving/ab_test_sklearn_tensorflow.json

See Next Steps in jupyter notebook