Where do you add --model_config_file argument to a yaml configuration?

[rolandwang19]

I've been having a hell of a time trying to figure-out how to serve multiple models using a yaml configuration file for K8s.

I can run directly in Bash using the following, but having trouble converting it to yaml.

docker run -p 8500:8500 -p 8501:8501 \
    [container id] \
    --model_config_file=/models/model_config.config \
    --model_config_file_poll_wait_seconds=60

Using the configuration below, where would you add model_config_file = /models/model_config.config?

apiVersion: apps/v1
kind: Deployment
metadata:
  name: resnet-deployment
spec:
  selector:
    matchLabels:
      app: resnet-server
  replicas: 3
  template:
    metadata:
      labels:
        app: resnet-server
    spec:
      containers:
      - name: resnet-container
        image: gcr.io/tensorflow-serving/resnet
        ports:
        - containerPort: 8500
---
apiVersion: v1
kind: Service
metadata:
  labels:
    run: resnet-service
  name: resnet-service
spec:
  ports:
  - port: 8500
    targetPort: 8500
  selector:
    app: resnet-server
  type: LoadBalancer

[Mehran Salmani]

@rolandwang19
One approach I have tested is to use Kubernetes' ConfigMap for your models_config.config; Then you need to define a Kubernetes volume of type configmap, and mount it inside your container in the specified path. Here I give you a sample:

apiVersion: v1
kind: ConfigMap
metadata:
  name: tfserving-resnet-cm
data:
  models.config: |-
    model_config_list {
      config {
        name: 'resnet'
        base_path: '/models/resnet/'
        model_platform: 'tensorflow'
        model_version_policy {
          specific {
            versions: 50
          }
        }
      }
    }

apiVersion: apps/v1
kind: Deployment
metadata:
  name: resnet-deployment
spec:
  selector:
    matchLabels:
      app: resnet-server
  replicas: 3
  template:
    metadata:
      labels:
        app: resnet-server
    spec:
      containers:
      - name: resnet-container
        image: gcr.io/tensorflow-serving/resnet
        ports:
        - containerPort: 8500
        args:
          - --model_config_file=/etc/tfserving/models.config
          - --model_config_file_poll_wait_seconds=60
        volumeMounts:
          - name: tfserving-resnet-vol
            mountPath: /etc/tfserving
      volumes:
      - name: tfserving-resnet-vol
        configMap:
          name: tfserving-resnet-cm