Allow default VaultAuth to suffix kubernetes auth role with provider/target namespace
mkilchhofer opened this issue · comments
Is your feature request related to a problem? Please describe.
In our current setup, our K8s platform customers use https://github.com/postfinance/vault-kubernetes to sync Secrets from Vault OSS into their desired Kubernetes namespace.
Our Vault is fully configured via Terraform. The k8s integration is done via Kubernetes Auth method on namespace basis (a Vault role per K8s namespace). In detail, we have this structure:
module "my_k8s_cluster" {
source = "./k8s-clusters"
cluster_name = "my-k8s-cluster"
kubernetes_host = "https://<api-server-url>"
kubernetes_ca_cert = <<EOF
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
EOF
disable_iss_validation = true
reviewer_jwt = var.my_k8s_cluster-jwt
namespaces = {
# Customer namespaces
"app-a-namespace" = {
secret_engine = "solution/my-solution-a/blabla"
secret_path = "general/my-app-a"
}
"my-app-b-namespace" = {
secret_engine = "solution/my-solution-b/blabla"
secret_path = "github-actions/*"
}
"my-app-c-namespace" = {
secret_engine = "solution/my-solution-c/blabla"
secret_path = "test/*"
}
# And also some platform namespaces
"test-vault-sync" = {
secret_engine = "realm/k8s/kv"
secret_path = "testing/test-vault-sync/*"
}
"monitoring" = {
secret_engine = "realm/k8s/kv"
secret_path = "alertmanager/*"
}
}
}
This module then installs multiple config resources inside Vault:
########################
# Configure Vault to talk to Kubernetes
########################
resource "vault_auth_backend" "kubernetes" {
type = "kubernetes"
path = "k8s-${var.cluster_name}"
description = "kubernetes auth for cluster ${var.cluster_name}"
}
resource "vault_kubernetes_auth_backend_config" "kubernetes" {
backend = vault_auth_backend.kubernetes.path
kubernetes_host = var.kubernetes_host
kubernetes_ca_cert = var.kubernetes_ca_cert
token_reviewer_jwt = var.reviewer_jwt
disable_iss_validation = var.disable_iss_validation
}
########################
# K8s Namespace to Policy mapping
########################
resource "vault_kubernetes_auth_backend_role" "vault_sync" {
for_each = var.namespaces
backend = vault_auth_backend.kubernetes.path
role_name = "vault-sync-${each.key}" # <-- see here
bound_service_account_names = ["vault-sync*"]
bound_service_account_namespaces = [each.key] # <-- here
token_policies = ["k8s-${var.cluster_name}-${each.key}-read"] # <-- and here
}
########################
# Read policies per Namespace
########################
data "vault_policy_document" "k8s-policies" {
for_each = var.namespaces
rule {
path = "${each.value.secret_engine}/data/${each.value.secret_path}"
capabilities = ["read"]
}
rule {
path = "${each.value.secret_engine}/metadata/${each.value.secret_path}"
capabilities = ["list"]
}
rule {
path = "sys/mounts"
capabilities = ["read"]
}
}
resource "vault_policy" "k8s-policies" {
for_each = var.namespaces
name = "k8s-${var.cluster_name}-${each.key}-read"
policy = data.vault_policy_document.k8s-policies[each.key].hcl
}
So in practice we have:
- 1 vault_auth_backend per cluster
- 1 vault_kubernetes_auth_backend_config per cluster
- N vault_policy per cluster (1 per namespace)
- N vault_kubernetes_auth_backend_role per cluster (1 per namespace)
Describe the solution you'd like
It would be nice to share both
- the vault default Connection
- the vault default Auth (Kubernetes Auth)
with all customer namespaces, so they only need to define the VaultStaticSecret
etc. :
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: default
namespace: vault-secrets-operator # <-- sits in the VSO namespace
spec:
method: kubernetes # same as tool from PF
mount: {{ printf "k8s-%s" .Values.clusterName }}
kubernetes:
role: vault-sync-vso
serviceAccount: vault-sync-vso
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultConnection
metadata:
name: default
namespace: vault-secrets-operator # <-- sits in the VSO namespace
spec:
address: https://vault.example.com
caCertSecretRef: vault-ca
Describe alternatives you've considered
Only sharing the kind: VaultConnection
with all namespaces and let our platform customers define the kind: VaultAuth
in each namespace.
Additional context
I did a short PoC without spending too much time on the golang code which worked:
diff --git a/internal/credentials/vault/kubernetes.go b/internal/credentials/vault/kubernetes.go
index 830c1f4..0ed9af2 100644
--- a/internal/credentials/vault/kubernetes.go
+++ b/internal/credentials/vault/kubernetes.go
@@ -5,6 +5,7 @@ package vault
import (
"context"
+ "fmt"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/types"
@@ -82,9 +83,11 @@ func (l *KubernetesCredentialProvider) GetCreds(ctx context.Context, client ctrl
return nil, err
}
+ role := fmt.Sprintf("%s-%s", l.authObj.Spec.Kubernetes.Role, l.providerNamespace)
+
// credentials needed for Kubernetes auth
return map[string]interface{}{
- "role": l.authObj.Spec.Kubernetes.Role,
+ "role": role,
"jwt": tr.Status.Token,
}, nil
}
Is this a valable approach? If so, I could invest some more time to make it configurable.
Maybe somehow related:
ping @benashz