"rate-limit" deployment fails if redis.disabled is set to "true"
htech7x opened this issue · comments
Gloo Edge Product
Enterprise
Gloo Edge Version
1.16.9
Kubernetes Version
1.28.5
Describe the bug
In case we need to disable "redis" but need to have "rate-limit" enabled we will have issue.
According to "2-rate-limit-deployment.yaml" template, "rate-limit-sidecar-config" configmap is used during deployment "rate-limit":
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: gloo
gloo: {{ $rateLimitName }}
name: {{ $rateLimitName }}
...
volumeMounts:
- mountPath: /etc/envoy
name: {{ $rateLimitName }}-sidecar-config
...
But according to another template "25-rate-limit-sidecar-config-and-redis-secret.yaml" configmap "rate-limit-sidecar-config" will not be created if "redis" is disabled:
...
{{- if or .Values.global.glooMtls.enabled (and .Values.redis.clientSideShardingEnabled (not .Values.redis.disabled) ) }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ $rateLimitName }}-sidecar-config
...
Thus, without "redis", "rate-limit-sidecar-config" will not be created and "rate-limit" deployment will fail
Expected Behavior
"rate-limit" is deployed successfully with redis.disabled is set to "true"
Steps to reproduce the bug
Deploy Gloo EE 1.16.9 with the following settings:
helm install gloo glooe/gloo-ee --version 1.16.9 --namespace gloo-system --create-namespace --set-string license_key=$GLOO_LICENSE_KEY --set redis.disabled=true --set global.glooMtls.enabled=true --set global.extensions.rateLimit.enabled=true
Check status of "rate-limit" pod:
kubectl get pod -n gloo-system |grep rate
rate-limit-6997c4cb45-h6x9s 0/3 ContainerCreating 0 8m12s
kubectl describe pod rate-limit-6997c4cb45-h6x9s -n gloo-system
...
Volumes:
gloo-mtls-certs:
Type: Secret (a volume populated by a Secret)
SecretName: gloo-mtls-certs
Optional: false
rate-limit-sidecar-config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: rate-limit-sidecar-config
Optional: false
kube-api-access-86t8b:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 8m51s default-scheduler Successfully assigned gloo-system/rate-limit-6997c4cb45-h6x9s to gloo-mesh-cluster1-worker2
Warning FailedMount 37s (x12 over 8m51s) kubelet MountVolume.SetUp failed for volume "rate-limit-sidecar-config" : configmap "rate-limit-sidecar-config" not found
Check "rate-limit-sidecar-config" configmap was not created during Gloo EE deployment:
kubectl get cm -n gloo-system
NAME DATA AGE
envoy-sidecar-config 1 10m
extauth-sidecar-config 1 10m
gateway-proxy-envoy-config 1 10m
gloo-custom-resource-config 4 10m
gloo-fed-default-apiserver-envoy-config 1 10m
gloo-observability-config 2 10m
glooe-grafana 3 10m
glooe-grafana-custom-dashboards 5 10m
glooe-grafana-custom-dashboards-v2 5 10m
glooe-prometheus-server 6 10m
kube-root-ca.crt 1 10m
Additional Environment Detail
No response
Additional Context
No response
┆Issue is synchronized with this Asana task by Unito
Zendesk ticket #3821 has been linked to this issue.
It's actually this line in the Helm chart that is the problem: https://github.com/solo-io/solo-projects/blob/54e505cc5af0c16ae905f54e14845bf9d5f42fa4/install/helm/gloo-ee/templates/25-rate-limit-sidecar-config-and-redis-secret.yaml#L191
The section mentioned in the issue description actually states that the ConfigMap will be created, even when Redis is disabled:
...
{{- if or .Values.global.glooMtls.enabled (and .Values.redis.clientSideShardingEnabled (not .Values.redis.disabled) ) }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ $rateLimitName }}-sidecar-config
...
This is an "or" statement, so as long as global.glooMtls.enabled=true, the ConfigMap should be created.
However, the if statement at line 191 only evaluates to true when redis.disabled=false
{{- if and .Values.global.extensions.rateLimit.enabled (not .Values.redis.disabled) }}
..... and that prevents the execution of this statement at line 196:
{{- include "gloo.util.merge" (list $ctx $override "ratelimit.sidecarConfigAndRedisSecret") -}}
... which causes the generated ConfigMap not to be included in the output.
Lets take the simplest scope at first
Custom redis + no clientsidesharding should work out of the box
@nfuden Will we be able to backport this to 1.16?
A fix for this has been merged in 1.18, 1.17, and 1.16 and will be released in 1.18.0-beta1, 1.17.0-beta6, and 1.16.12