Masahigo / video-indexer

Building an event driven .NET Core app with Dapr

Video indexer application using Dapr

Running locally in Standalone mode (Windows file system)

Refer to Dapr's documentation for more details.

• Open PowerShell as Administrator and run:

    powershell -Command "iwr -useb https://raw.githubusercontent.com/dapr/cli/master/install/install.ps1 | iex"

• Open Command Prompt as Administrator and run:

    dapr init

• Check Dapr runtime containers:

    docker ps

    CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                     NAMES
    523eed39c32d        daprio/dapr         "./placement"            8 hours ago         Up 29 minutes       0.0.0.0:6050->50005/tcp   dapr_placement
    7f9c30b58968        redis               "docker-entrypoint.s…"   8 hours ago         Up 29 minutes       0.0.0.0:6379->6379/tcp    dapr_redis

• Get Ngrok

• Run Ngrok with following command

    ngrok http -host-header=localhost 9000

• Configure your event grid binding (eg. /components/eventgrid.yaml) with required metadata, using the HTTPS endpoint from ngrok for subscriberEndpoint

..
spec
  - name: subscriberEndpoint
    value: "https://xxx.ngrok.io/api/events"    
  - name: handshakePort
    value: 9000
  ..

• Configure your cosmosdb binding (eg. /components/cosmosdb.yaml) with required metadata

• Start application with Dapr

    dapr run --app-id video-indexer --app-port 5000 --port 3500 dotnet run

Steps to deploy to Kubernetes

1. Create Kubernetes cluster and get admin credentials

2. Install Helm 3 from script: https://helm.sh/docs/intro/install/

curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | sudo bash

3. Install Dapr using Helm3: https://github.com/dapr/docs/blob/master/getting-started/environment-setup.md#using-helm-advanced

helm repo add dapr https://daprio.azurecr.io/helm/v1/repo
helm repo update

kubectl create namespace dapr-system
helm install dapr dapr/dapr --namespace dapr-system
kubectl get pods -n dapr-system -w

4. Install Redis

helm repo add bitnami https://charts.bitnami.com/bitnami
helm install redis bitnami/redis --set "usePassword=false"

5. Install Nginx with Dapr annotations to default namespace

helm repo add stable https://kubernetes-charts.storage.googleapis.com/
helm install nginx stable/nginx-ingress -f ./deploy/dapr-annotations.yaml -n default

# Get public IP from ingress controller
kubectl get svc -l component=controller -o jsonpath='Public IP is: {.items[0].status.loadBalancer.ingress[0].ip}{"\n"}'

6. Configure DNS A record pointing to the public IP

7. Install Cert manager to default namespace

# Install the CustomResourceDefinition resources separately
kubectl apply --validate=false -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.13/deploy/manifests/00-crds.yaml

# Label the ingress-basic namespace to disable resource validation
kubectl label namespace default cert-manager.io/disable-validation=true

# Add the Jetstack Helm repository
helm repo add jetstack https://charts.jetstack.io

# Update your local Helm chart repository cache
helm repo update

# Install the cert-manager Helm chart
helm install \
  cert-manager \
  --namespace default \
  --version v0.13.0 \
  jetstack/cert-manager

8. Create a CA cluster issuer: https://docs.microsoft.com/en-us/azure/aks/ingress-tls#create-a-ca-cluster-issuer

9. Add Azure Key cert file to k8s secret for enabling secrets in Dapr bindings: https://github.com/dapr/docs/blob/master/howto/setup-secret-store/azure-keyvault.md#use-azure-key-vault-secret-store-in-kubernetes-mode

kubectl create secret generic azure-keyvault-cert --from-file=VideoIndexerCert.pfx

10. Deploy akv2k8s for k8s app's secrets: https://akv2k8s.io/installation/installing-with-helm/#installing-with-helm-on-azure-aks

kubectl create ns akv2k8s
helm repo add spv-charts http://charts.spvapi.no
helm repo update

helm install azure-key-vault-controller \
  spv-charts/azure-key-vault-controller \
  --namespace akv2k8s

helm install azure-key-vault-env-injector \
  spv-charts/azure-key-vault-env-injector \
  --set installCrd=false \
  --namespace akv2k8s

kubectl apply -f ./deploy/azurekeyvaultsecrets.yaml

$ kubectl get akvs

NAME                           VAULT                 VAULT OBJECT              SECRET NAME   SYNCHED
secret-client-id               video-indexer-we-kv   clientId
secret-client-secret           video-indexer-we-kv   clientSecret
secret-storage-account-name    video-indexer-we-kv   azureStorageAccountName
secret-tenant-id               video-indexer-we-kv   tenantId
secret-video-indexer-api-key   video-indexer-we-kv   videoIndexerApiKey

# Remember to authorize get permission for k8s cluster's spn: https://akv2k8s.io/tutorials/prerequisites/#add-secret---required-for-secret-tutorials
# Otherwise akv2k8s fails to initialize along with your app's pod

10. Deploy Dapr components

$ kubectl apply -f ./deploy/azurekeyvault.yaml
$ kubectl apply -f ./deploy/redis.yaml
$ kubectl apply -f ./deploy/eventgrid_binding.yaml
$ kubectl apply -f ./deploy/cosmosdb_binding.yaml

11. Deploy app and ingress + verify that the app works

$ kubectl apply -f ./deploy/dotnetwebapi.yaml
$ kubectl get pods -l app=dotnetwebapi
NAME                                                   READY   STATUS            RESTARTS   AGE
video-indexer-6cb78f555c-j8vpv                         0/2     PodInitializing   0          4s

$ kubectl logs video-indexer-6cb78f555c-j8vpv webapi
time="2020-06-10T06:32:34Z" level=info msg="found original container command to be /usr/bin/dotnet [dotnet VideoIndexerApi.dll]" application=env-injector component=akv2k8s custom_auth=false namespace=default
time="2020-06-10T06:32:34Z" level=info msg="secret clientId injected into evn var AZURE_CLIENT_ID for executable /usr/bin/dotnet" application=env-injector component=akv2k8s custom_auth=false namespace=default
time="2020-06-10T06:32:35Z" level=info msg="secret clientSecret injected into evn var AZURE_CLIENT_SECRET for executable /usr/bin/dotnet" application=env-injector component=akv2k8s custom_auth=false namespace=default
time="2020-06-10T06:32:35Z" level=info msg="secret videoIndexerApiKey injected into evn var VIDEO_INDEXER_API_KEY for executable /usr/bin/dotnet" application=env-injector component=akv2k8s custom_auth=false namespace=default
time="2020-06-10T06:32:35Z" level=info msg="secret azureStorageAccountName injected into evn var AZURE_STORAGE_ACCOUNT_NAME for executable /usr/bin/dotnet" application=env-injector component=akv2k8s custom_auth=false namespace=default
time="2020-06-10T06:32:35Z" level=info msg="secret tenantId injected into evn var AZURE_TENANT_ID for executable /usr/bin/dotnet" application=env-injector component=akv2k8s custom_auth=false namespace=default
time="2020-06-10T06:32:35Z" level=info msg="starting process /usr/bin/dotnet [dotnet VideoIndexerApi.dll] with secrets in env vars" application=env-injector component=akv2k8s custom_auth=false namespace=default
info: VideoIndexerApi.Services.QueuedHostedService[0]
      Queued Hosted Service is running.

      Tap W to add a work item to the background queue.

info: Microsoft.Hosting.Lifetime[0]
      Now listening on: http://0.0.0.0:5000
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Production
info: Microsoft.Hosting.Lifetime[0]
      Content root path: /app
info: VideoIndexerApi.HealthChecks.LivenessHealthCheck[0]
      LivenessHealthCheck executed.
info: VideoIndexerApi.HealthChecks.ReadinessHealthCheck[0]
      Readiness health check executed.
info: VideoIndexerApi.HealthChecks.ReadinessHealthCheck[0]
      Connection to video indexer API is working.

12. Check Nginx controller logs and restart pod if needed (https://kubernetes.github.io/ingress-nginx/troubleshooting/)

$ kubectl get pods -l app=nginx-ingress

NAME                                                   READY   STATUS    RESTARTS   AGE
nginx-nginx-ingress-controller-649df94867-fp6mg        2/2     Running   0          51m
nginx-nginx-ingress-default-backend-6d96c457f6-4nbj5   1/1     Running   0          55m

$ kubectl logs nginx-nginx-ingress-controller-649df94867-fp6mg nginx-ingress-controller

# If you see 503s logged from calls to webhook endpoint '/api/events' restart the pod
# .."OPTIONS /api/events HTTP/1.1" 503..

kubectl delete pod nginx-nginx-ingress-controller-649df94867-fp6mg

12. Observe your Event Subscription being created to Storage account