Discussion about multiple runtimes on a Kubernetes pod?

[Thang Chung]

Just have a question on multiple runtimes on POD:

Currently, we have used some kind of sidecar (dapr or istio) tool on the Kubernetes cluster.

The sample file can be found below

apiVersion: apps/v1
kind: Deployment
metadata:
  name: product-api
spec:
  replicas: 1
  selector:
    matchLabels:
      app: product-api
  template:
    metadata:
      labels:
        app: product-api
      annotations:
        dapr.io/enabled: "true"
        dapr.io/app-id: "product-api"
        dapr.io/app-port: "3000"
        dapr.io/enable-api-logging: "true"
    spec:
      runtimeClassName: wasmtime-spin-v1
      containers:
        - name: product-api
          image: ghcr.io/thangchung/dapr-labs/product-api-spin:1.0.0
          command: ["/"]
          ports:
          - containerPort: 3000
          env:
          - name: RUST_BACKTRACE
            value: "1"
          resources: # limit the resources to 128Mi of memory and 100m of CPU
            limits:
              cpu: 100m
              memory: 128Mi
            requests:
              cpu: 100m
              memory: 128Mi
---
apiVersion: v1
kind: Service
metadata:
  name: product-api
spec:
  type: LoadBalancer
  ports:
    - protocol: TCP
      port: 5001
      targetPort: 3000
  selector:
    app: product-api

When we run it, it always throws the exception that the daprd didn't have spin.toml file, and we found out that because runtimeClassName: wasmtime-spin will scan all containers (include any sidecar in this pod). I know this is by the design of Kubernetes because I searched around and maybe we cannot run multiple runtimes on 1 pod.

Could anyone shed light for me on it? Thank you very much.

[Jiaxiao Zhou]

Hey there, that's an excellent question!

In fact, I am working on a new feature in runwasi, which provides the shim library for the spin shim, that allows the sidecar behavior you are proposing here: containerd/runwasi#64

It's still under development and will keep you updated on it.

[Thang Chung]

Thanks for the information. That's interesting to see that it is working, just have a look at containerd/runwasi#142, waiting for the merging.

[Jiaxiao Zhou]

#121 should bring the ability to run Dapr as a sidecar to the spin shim.

[Thang Chung]

@Mossaka, /cc @0xE282B0 because of #130, I just tried to daprized the app, but when I ran kubectl apply -f iac/kind-spin/product-api-deploy.yaml, and it threw the exception as below

"Error: failed to start containerd task "daprd": Cannot read spin.toml manifest from "/run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/daprd/rootfs/spin.toml": unknown"

I run it on k3d, and get the latest guidance at https://github.com/deislabs/containerd-wasm-shims/blob/main/deployments/k3d/README.md

My product-api-deploy.yaml as below

apiVersion: apps/v1
kind: Deployment
metadata:
  name: product-api
spec:
  replicas: 1
  selector:
    matchLabels:
      app: product-api
  template:
    metadata:
      labels:
        app: product-api
      annotations:
        dapr.io/enabled: "true"
        dapr.io/app-id: "productapi"
        dapr.io/app-port: "3000"
        dapr.io/enable-api-logging: "true"
    spec:
      runtimeClassName: wasmtime-spin
      containers:
        - name: product-api
          image: ghcr.io/thangchung/dapr-labs/product-api-spin:1.0.0
          command: ["/"]
          ports:
          - containerPort: 3000
          env:
          - name: RUST_BACKTRACE
            value: "1"
          resources: # limit the resources to 128Mi of memory and 100m of CPU
            limits:
              cpu: 100m
              memory: 128Mi
            requests:
              cpu: 100m
              memory: 128Mi
---
apiVersion: v1
kind: Service
metadata:
  name: product-api
spec:
  type: LoadBalancer
  ports:
    - protocol: TCP
      port: 5001
      targetPort: 3000
  selector:
    app: product-api

My source code is at https://github.com/thangchung/dapr-labs/blob/feat/spin-refactor/polyglot/iac/kind-spin/product-api-deploy.yaml

[Jiaxiao Zhou]

Did the new release fix this issue? Note that the Spin shim is expect spin.toml at rootfs in the spin image.

[Thang Chung]

Nevermind. Yes, it worked with 0.9.0. The error above was because I ran with 0.8.0.