rosenhouse/cf-k8s-networking

cf-k8s-networking

Routing and networking for Cloud Foundry running on Kubernetes.

Installation

Prerequisites

A Cloud Foundry deployment using Eirini for app workloads
kubectl installed and access to the Kubernetes cluster backing Eirini
kapp installed
ytt installed

Metacontroller

Install Metacontroller to the Kubernetes cluster

Istio

Install Istio to the Kubernetes cluster.
Include the istio-values.yaml in your Istio installation.

Note: As an example, in our CI we are installing Istio via the deploy-istio.sh task.
Enable automatic sidecar injection by labeling the following namespaces with istio-injection=enabled: cf-system, cf-workloads, metacontroller:
```
kubectl label namespaces cf-system cf-workloads metacontroller istio-injection=enabled --overwrite
```

Sidecars are required for automatic mTLS between workloads so it is important that this is enabled. Confirm that the namespaces are labeled correctly:

$ kubectl get namespaces -l istio-injection=enabled

NAME             STATUS   AGE
cf-system        Active   50d
cf-workloads     Active   50d
metacontroller   Active   50d

CF-K8s-Networking

cfroutesync needs to be able to authenticate with UAA and fetch routes from Cloud Controller. To do this you must override the following properties from install/ytt/networking/values.yaml. You can do this by creating a new file /tmp/values.yaml that contains the following information:
```
#@data/values
---
cfroutesync:
  ccCA: 'pem_encoded_cloud_controller_ca'
  ccBaseURL: 'https://api.example.com'
  uaaCA: 'pem_encoded_uaa_ca'
  uaaBaseURL: 'https://uaa.example.com'
  clientName: 'uaaClientName'
  clientSecret: 'uaaClientSecret'
```
The UAA client specified by clientName is used for fetching routing data from Cloud Controller. It must have permission to access all routes and domains in the deployment. We recommend using a client with at least the cloud_controller.admin_read_only authority. For example, see the network-policy client in cf-deployment.

As an example, for our dev environments we are using the generate_values.rb script to populate these values from the bbl-state.json and secrets in CredHub.

Deploy the cf-k8s-networking CRDs and components using ytt and kapp:

system_namespace="cf-system"

ytt -f config/cfroutesync/ -f /tmp/values.yaml \
    -f cfroutesync/crds/routebulksync.yaml | \
    kapp deploy -n "${system_namespace}" -a cfroutesync \
    -f - \
    -y

Update the Prometheus configuration so metrics from cf-k8s-networking can be queried.

prometheus_file="$(mktemp -u).yml"
kubectl get -n istio-system configmap prometheus -o yaml > ${prometheus_file}

ytt \
 -f "config/values.yaml" \
 -f "${prometheus_file}" \
 -f "config/deps/prometheus-config.yaml" | \
 kubectl apply -f -

Note: you might need to restart Prometheus pod(s) in the istio-system namespace after updating the ConfigMap 🧐🥺

kubectl -n istio-system delete pod -l app=prometheus

rosenhouse / cf-k8s-networking