Azure / application-gateway-kubernetes-ingress

This is an ingress controller that can be run on Azure Kubernetes Service (AKS) to allow an Azure Application Gateway to act as the ingress for an AKS cluster.

Home Page:https://azure.github.io/application-gateway-kubernetes-ingress

Repository from Github https://github.comAzure/application-gateway-kubernetes-ingressRepository from Github https://github.comAzure/application-gateway-kubernetes-ingress

AGIC resets verifyClientRevocation from "OCSP" back to "None" for pre-existing SSL Profiles every time it creates or updates a listener

juanbora opened this issue · comments

Describe the bug
I think the title is pretty much self explanatory, but I'll elaborate a bit better here.
So basically, if you set AppGateway with an SSL Profile which includes verifyClientRevocation set to OCSP

        "sslProfiles": [
            {
                "name": "somesslprofile",
                "properties": {
                    "clientAuthConfiguration": {
                        "verifyClientCertIssuerDN": false,
                        "verifyClientRevocation": "OCSP"   <-- THIS!
                    },

when AGIC runs (let's say, to create or update an existing listener), it'll set verifyClientRevocation back to None.

I've been researching into this a little bit, and I noted that AGIC logs, don't show the field verifyClientRevocation when it reads the existing AppGateway configuration from Azure (-- Existing App Gwy Config --), so when it creates or updates a listener, it posts to Azure API an AppGateway config without the verifyClientRevocation parameter, thus resulting in Azure setting it to None as it's probably the default.

I doubt this is a permission issue, because we are at a development stage, and we are being pretty generous with permissions.

To Reproduce

  • Create a network with an AppGateway.
  • Create an AKS cluster in Azure, and associate it with AppGateway (in my case, done via Terraform with ingress_application_gateway block).
  • Create an SSL Profile.
  • Enable OCSP for the SSL Profile using az cli or Azure API (no possible to enable through the UI at the moment I guess?).
az network application-gateway update -n ${app_gateway_name} -g ${resource_group} --ssl-profiles [0].client-auth-configuration.verify-client-revocation=OCSP
  • Notice sslProfiles[0].properties.clientAuthConfiguration.verifyClientRevocation is set to OCSP now.
  • Create an ingress in AKS cluster.
  • Verify that sslProfiles[0].properties.clientAuthConfiguration.verifyClientRevocation is set to None.

Ingress Controller details

  • Output of kubectl describe pod <ingress controller> . The pod name can be obtained by running helm list.
Name:             ingress-appgw-deployment-d76b949b4-q7zv7
Namespace:        kube-system
Priority:         0
Service Account:  ingress-appgw-sa
Node:             aks-default-39169157-vmss000003/10.66.63.7
Start Time:       Wed, 04 Dec 2024 00:07:02 +0000
Labels:           app=ingress-appgw
                  kubernetes.azure.com/managedby=aks
                  pod-template-hash=d76b949b4
Annotations:      checksum/config: dcfd3e5e7b7211f8e24bb6598651c107e41127e4b7ae4325a45aa06d2a748f8a
                  cluster-autoscaler.kubernetes.io/safe-to-evict: true
                  kubernetes.azure.com/metrics-scrape: true
                  prometheus.io/path: /metrics
                  prometheus.io/port: 8123
                  prometheus.io/scrape: true
                  resource-id:
                    <AKS-CLUSTER-ID-HERE>
Status:           Running
IP:               10.66.56.93
IPs:
  IP:           10.66.56.93
Controlled By:  ReplicaSet/ingress-appgw-deployment-d76b949b4
Containers:
  ingress-appgw-container:
    Container ID:   containerd://02a5951d419c0444d9765575c352e9faf6191a4402884bbc1acff93a5a101203
    Image:          mcr.microsoft.com/azure-application-gateway/kubernetes-ingress:1.7.4
    Image ID:       mcr.microsoft.com/azure-application-gateway/kubernetes-ingress@sha256:92cb29fe2583e4e332f4a6610dd3cd812c2c975d388817934c5c33ff50452a75
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Wed, 04 Dec 2024 00:07:10 +0000
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     700m
      memory:  600Mi
    Requests:
      cpu:      100m
      memory:   20Mi
    Liveness:   http-get http://:8123/health/alive delay=15s timeout=1s period=20s #success=1 #failure=3
    Readiness:  http-get http://:8123/health/ready delay=5s timeout=1s period=10s #success=1 #failure=3
    Environment Variables from:
      ingress-appgw-cm  ConfigMap  Optional: false
    Environment:
      KUBERNETES_SERVICE_HOST:        10.66.63.196
      KUBERNETES_PORT:                tcp://10.66.63.196:443
      KUBERNETES_PORT_443_TCP:        tcp://10.66.63.196:443
      KUBERNETES_PORT_443_TCP_ADDR:   10.66.63.196
      AZURE_CLOUD_PROVIDER_LOCATION:  /etc/kubernetes/azure.json
      AGIC_POD_NAME:                  ingress-appgw-deployment-d76b949b4-q7zv7 (v1:metadata.name)
      AGIC_POD_NAMESPACE:             kube-system (v1:metadata.namespace)
      AZURE_ENVIRONMENT:              AZUREPUBLICCLOUD
    Mounts:
      /etc/kubernetes/azure.json from cloud-provider-config (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-w7c4l (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True 
  Initialized                 True 
  Ready                       True 
  ContainersReady             True 
  PodScheduled                True 
Volumes:
  cloud-provider-config:
    Type:          HostPath (bare host directory volume)
    Path:          /etc/kubernetes/azure.json
    HostPathType:  File
  kube-api-access-w7c4l:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 CriticalAddonsOnly op=Exists
                             node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:                      <none>
  • Output of `kubectl logs .
-- Existing App Gwy Config --        "sslProfiles": [
-- Existing App Gwy Config --            {
-- Existing App Gwy Config --                "id": "<SSL-PROFILE-ID-HERE>",
-- Existing App Gwy Config --                "name": "<SSL-PROFILE-NAME-HER>",
-- Existing App Gwy Config --                "properties": {
-- Existing App Gwy Config --                    "clientAuthConfiguration": {
-- Existing App Gwy Config --                        "verifyClientCertIssuerDN": false
-- Existing App Gwy Config --                    },
  • Any Azure support tickets associated with this issue.