rimusz / kubenab

Kubernetes Admission Webhook to enforce pulling of Docker images from the private registry.

Kubenab

What does Kubenab do?

Kubenab is Kubernetes Admission webhook to enforce pulling of docker images from private registry.

Prerequisites

Kubernetes 1.12.0 or above with the admissionregistration.k8s.io/v1beta1 API enabled. Verify that by the following command:

kubectl api-versions | grep admissionregistration.k8s.io/v1beta1

The result should be:

admissionregistration.k8s.io/v1beta1

In addition, the MutatingAdmissionWebhook and ValidatingAdmissionWebhook admission controllers should be added and listed in the correct order in the admission-control flag of kube-apiserver.

Build and Push Kubenab Docker Image

# Build docker image
docker build -t my-registry/kubenab:0.3.3 .

# Push it to Docker Registry
docker push my-registry/kubenab:0.3.3

Create Kubernetes Docker registry secret

# Create a Docker registry secret called 'regsecret'
kubectl create secret docker-registry regsecret --docker-server=${DOCKER_REGISTRY} --docker-username=${DOCKER_USER} --docker-password=${DOCKER_PASS} --docker-email=${DOCKER_EMAIL}

Note: Create Docker registry secret in each non-whitelisted namespaces.

Generate TLS Certs for Kubenab

./tls/gen-cert.sh

Get CA Bundle

./webhook/webhook-patch-ca-bundle.sh

Note: You can skip this step and use Helm chart install with TLS certs generated by the cert-manager

Deploy Kubenab to Kubernetes with self generated TLS Certs

• Deploy using kubectl

# Run deployment
kubectl create -f deployment/kubenab-deployment.yaml

# Create service
kubectl create -f  deployment/kubenab-svc.yaml

• Deploy using Helm Chart

helm install --name kubenab --set docker.registrySecret=regsecret,docker.registryUrl=jfrog,whitelistNamespaces="kube-system,default",whitelistRegistries="jfrog",tls.secretName=kubenab-certs chart/kubenab/

Configure MutatingAdmissionWebhook and ValidatingAdmissionWebhook

Note: Replace ${CA_BUNDLE} with value generated by running ./webhook/webhook-patch-ca-bundle.sh

# Configure MutatingAdmissionWebhook
kubectl create -f webhook/kubenab-mutating-webhook-configuration.yaml

Note: Use MutatingAdmissionWebhook only if you want to enforce pulling of docker image from Private Docker Registry e.g JFrog Artifactory. If your container image is nginx then Kubenab will append REGISTRY_URL to it. e.g nginx will become jfrog/nginx

# Configure ValidatingAdmissionWebhook
kubectl create -f webhook/kubenab-validating-webhook-configuration.yaml

Note: Use ValidatingAdmissionWebhook only if you want to check pulling of docker image from Private Docker Registry e.g JFrog Artifactory. If your container image does not contain REGISTRY_URL then Kubenab will deny request to run that pod.

Deploy Kubenab to Kubernetes with cert-manager generated TLS Certs

• Deploy using Helm Chart

helm install --name kubenab --set docker.registrySecret=regsecret,docker.registryUrl=jfrog,whitelistNamespaces="kube-system,default",whitelistRegistries="jfrog" chart/kubenab/

ValidatingAdmissionWebhook is enabled by default, use mutatingWebhook.enabled="true" to enable MutatingAdmissionWebhook.

Test Kubenab

# Deploy nginx
kubectl apply -f test/nginx.yaml

Benchmark

Run the benchmark.sh Script in benchmark/, you only need go, docker and openssl.

Benchmark Results

==> Mutate Webhook


 1000000 / 1000000 [=====================================================================================================================] 100.00% 6850/s 2m25s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec      6906,14     612,30    7957,42
  Latency      285,93us   171,41us    27,91ms
  Latency Distribution
     50%   257,00us
     75%   309,00us
     90%   376,00us
     95%   440,00us
     99%   764,00us
  HTTP codes:
    1xx - 0, 2xx - 100000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:    15.05MB/s


==> Validate Webhook


 1000000 / 1000000 [=====================================================================================================================] 100.00% 6669/s 2m29s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec      6675.83     515.18    8017.83
  Latency      295.81us    99.07us    22.86ms
  Latency Distribution
     50%   267.00us
     75%   319.00us
     90%   388.00us
     95%   453.00us
     99%   759.00us
  HTTP codes:
    1xx - 0, 2xx - 1000000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:    16.20MB/s

ATTENTION: This Benchmark was done on an non optimized Laptop! (Intel(R) Core(TM) i5-3230M CPU @ 2.60GHz ; 8GB RAM)