CF App Service Discovery Release (EXPERIMENTAL)

High Level Overview

Problem we are trying to solve

Application Developers who want to use container to container networking today are required to bring their own service discovery. While we have provided examples with Eureka and Amalgam8, we have received user feedback that usage of c2c is very difficult, with some common themes emerging: Polyglot microservices written in languages/frameworks other than Java/Spring cannot easily use Eureka Clustering applications have a requirement to address individual instances Additional VMs need to be deployed and managed to provide external service discovery

In order to support all types of apps, languages and frameworks, we plan to build service discovery for c2c into the platform. With this feature, users will no longer have to bring their own service discovery.

App Developer Experience

When a user pushes an app, their app is automatically given an internal DNS entry, e.g.

app-guid.apps.internal

This DNS entry will automatically load balance between all instances of this app. For the time being, your tld will always be apps.internal.

Each instance of the application also receives a specific DNS entry, e.g.

instance-index.app-guid.apps.internal

that you can use to access one instance of the app, so it will not load balance between all instances.

Interaction with Policy

By default, apps cannot talk to each other over cf networking. In order for an app to talk to another app, you must still set a policy allowing access.

Example usage

cf push consumer-app --no-start
cf push server-app

cf add-network-policy consumer-app --destination-app server-app --port 8080 --protocol tcp

cf set-env consumer SERVER_HOSTNAME "$(cf app --guid server-app).apps.internal"
cf start consumer-app

You can run cf add-network-policy even after both apps are started, and you don't need to restart the apps for the policy to start working.

From consumer-app, the following will work:

curl "$SERVER_HOSTNAME.apps.internal:8080"

Architecture

Architecture Diagram

How To Deploy

To add service discovery to cf-deployment, include the following experimental ops-files:

Experimental Ops File for cf-deployment

Pull down your current manifest with

bosh manifest > /tmp/{env}-manifest.yml

Update your deployment with the ops files

bosh deploy /tmp/{env}-manifest.yml \
  -o ~/workspace/cf-deployment/operations/experimental/use-bosh-dns-for-containers.yml \
  -o ~/workspace/cf-deployment/operations/experimental/use-bosh-dns.yml \
  -o ~/workspace/cf-deployment/operations/experimental/enable-service-discovery.yml \
  --vars-store ~/workspace/cf-networking-deployments/environments/{env}/vars-store.yml

See opsfile and our pipeline that uses this opsfile.

Debugging problems

To change logging for service-discovery-controller, ssh onto the VM holding the service-discovery-controller and make a request to the log-level server:

curl -X POST -d 'debug' localhost:8055/log-level

where 8055 is the default value of service-discovery-controller.log_level_port.

To switch back to info logging:

curl -X POST -d 'info' localhost:8055/log-level

To change logging for bosh-dns-adapter, ssh onto the VM holding the bosh-dns-adapter and make a request to the log-level server:

curl -X POST -d 'debug' localhost:8066/log-level

To switch back to info logging:

curl -X POST -d 'info' localhost:8066/log-level

Deploying to BOSH-lite

Run the scripts/deploy-to-bosh-lite script.

To deploy you will need cf-networking-release, bosh-deployment, and cf-deployment.

camelpunch / cf-app-sd-release