Giters

hashicorp / vault-secrets-operator

The Vault Secrets Operator (VSO) allows Pods to consume Vault secrets natively from Kubernetes Secrets.

Home Page:https://hashicorp.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

VSO failed to connect through CloudFlare Zero Trust

kalote opened this issue · comments

commented

Describe the bug
We have our vault protected by CloudFlare Zero Trust. I have a client_id and client_secret tokens that I need to pass to go through the ZT. When using the CLI locally, the tokens works fine (using vault login -header=cf-client=xxx ...).
When adding those information as headers in the default vaultConnection, I get a 403:

Failed to check Vault seal status: Error making API request. URL: GET https://example.cloudflareaccess.com/cdn-cgi/access/login/vault.example.com Code: 403. Raw Message: <html> <head><title>403 Forbidden</title></head> <body> <center><h1>403 Forbidden</h1></center> <hr><center>cloudflare</center> </body> </html>

We're using helm charts with following info:

  • vault-secrets-operator version: 0.4.3
  • vault version: 0.27.0

The values.yaml (for the operator):

operator:
  enabled: true
  controller:
    replicas: 1
  defaultVaultConnection:
    enabled: true
    address: "https://vault.example.com"
    skipTLSVerify: false
    headers:
      CF-Access-Client-Id: "1f..."
      CF-Access-Client-Secret: "58..."
  defaultAuthMethod:
    enabled: false

When checking in my K8s cluster the vaultConnection resource, I can see the headers:

apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultConnection
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: > {longobject}
  finalizers:
    - vaultconnection.secrets.hashicorp.com/finalizer
  generation: 4
  labels:
    app.kubernetes.io/component: controller-manager
    app.kubernetes.io/instance: vault-dev
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: operator
    app.kubernetes.io/version: 0.4.3
    argocd.argoproj.io/instance: vault-dev
    component: controller-manager
    control-plane: controller-manager
    helm.sh/chart: operator-0.4.3
  name: default
  namespace: vault
spec:
  address: 'https://vault.example.com'
  headers:
    CF-Access-Client-Id: 1f...
    CF-Access-Client-Secret: 58...
  skipTLSVerify: false
status:
  valid: true

Expected behaviour
The vault-secrets-operator will use the headers to bypass CloudFlare Zero Trust

Environment

  • Kubernetes version:
    • Distribution or cloud vendor (OpenShift, EKS, GKE, AKS, etc.): EKS, version 1.28

Is there something that I miss here? I haven't been able to find a good documentation on this specific setup =/

Also, To make it work, do I need to update additional resources (VaultAuth, VaultStaticSecret, ...)?
Thanks 🙏

commented

I've update the vault-secrets-operator version to 0.5.1, but still the same issue.

commented

Hi @kalote, thanks for pointing this out! Indeed it looks like spec.headers from VaultConnection is not being set correctly on the vault client. We should be able to get a fix for this into the next release.

commented

Thanks a lot @tvoran !

QQ:

  • Do you know when will be the next release?
  • Is there a flag that you could add to the helm chart to help debug a bit more ... e.g., being able to see the requests that are sent to the vault for instance? something like:
operator:
  enabled: true
  debug: true # or log-level: debug

Thanks 🙏

commented

Setting -zap-log-level=5 in controller.manager.extraArgs may give some of what you're looking for in terms of debugging.

No ETA yet on the next release.