This project explores Knative Eventing. The aim of the project is to show a how a stream of events can be consumed and displayed in real-time using Knative Eventing.
In this tutorial, you will deploy a go application that streams messages to a Knative broker. A service (event display) can subscribe to the events and they can be displayed in real time via a UI.
The basic architecture that will be deployed by the end of this tutorial will look as follows:
In the case where you only have one consumer/subscriber, you can also sink straight to there (in this case that would be the event-display service).
There is also the option to have the service reply back to the broker with another event, which can then be consumed by another service from the Knative Eventing ecosystem. The architecture then looks as follows:
The message stream source is the blockchain.info WebSocket API and sends information about new bitcoin transactions in real-time. You can find more information about this service here.
In order to run this demo, you will need:
- A Kubernetes cluster (a single node cluster running on Docker desktop is fine and is used to build this example)
- kubectl
- Istio installed (or another Gateway provider such as Gloo). You will need cluster local gateway set up (instructions can be found at this link).
- Knative installed as per this documentation.
To confirm your install is complete, you can run the following command:
kubectl get pods --all-namespacesYou should have namespaces for istio-system, knative-eventing and knative-serving
You can also see the components deployed by running:
kubectl get pods -n <namespace>The Docker images are already available for each service in this tutorial, however if you want to make your own then you can build the image yourself. Each application/service folder has it's own Dockerfile.
To build a container image, from the application folder you can run the following commands (ensure you are also logged into your image repo account e.g Dockerhub):
example to build a Docker image:
docker build -t <username>/bitcoinfrontend .example to push:
docker push <username>/bitcoinfrontendThe 001-namespace.yaml creates a new namespace and also adds a knative-eventing injection label, which in turn creates a default broker.
First, navigate to the yaml folder:
cd yamlNow:
kubectl apply -f 001-namespace.yamlThis will deploy the knative-eventing-websocket-source namespace and enable the knative-eventing injection.
Verify that the default broker is ready by running kubectl get broker -n <namespace> and you should see that the default broker has a status of READY=True.
You now have a broker within a knative-eventing-websocket-source namespace.
The 010-deployment.yaml deploys a Kubernetes application (written in Go), which connects to the blockchain websocket stream and sends the messages to the broker.
Navigate to the blockchain-app folder:
cd blockchain-appDeploy the blockchain stream service:
kubectl apply -f 010-deployment.yamlYou can verify messages by using the following kubectl command to check the logs of the app:
kubectl --namespace knative-eventing-websocket-source logs -l app=wseventsource --tail=100You can also verify the broker logs:
kubectl --namespace knative-eventing-websocket-source logs -l eventing.knative.dev/broker=default --tail=100You now have a service that listens for events on a websocket and then send these on as CloudEvents into the Knative Eventing broker, which is our sink:
040-trigger.yaml - This specifies that all messages that come to the default broker from our blockchain source should be sent to the Event Display service (which is not deployed yet).
Create a trigger using the following command:
kubectl apply -f 040-trigger.yamlYou now have a trigger:
050-kubernetesdeploy.yaml - This is a Kubernetes app deployment for consuming the CloudEvents and serving them at a front-end UI.
060-kubernetesservice.yaml - This file defines a Kubernetes service in order to specify the requirements of our Event Display application. It includes the ports needed to run the application and also sets it to be accessible on NodePort 31234 (you can change this if you want).
Apply both of these files using the following commands:
kubectl apply -f 050-kubernetesdeploy.yaml
kubectl apply -f 060-kubernetesservice.yamlNote on UI Display:
Knative serving does not currently allow multiple ports so the Event Display part is a Kubernetes deployment (consisting of two files; 050 & 060). This is because we need to expose both 8080 (to receive the events) and 9080 (to serve the events).
Now you have messages from the blockchain application being sent as real time events to the UI front-end service:
Head to localhost:31234 and you should see the events in the UI
Alternatively you can just curl the service from the terminal window and you should see the stream of events.
curl http://localhost:31234/eventsThe files to build your own image for the classifier service are in the classifier-transaction-size folder.
cd into the yaml/classification folder and apply the 031-classifier-service.yaml and the two triggers that are in there.
Return to the main yaml folder. This time, our subscriber is a Knative service called test-display, which we still need to deploy.
030-test-display-service.yaml - This deploys a Knative service that will be subscribed to the reply events. It uses the Knative event-display image.
Deploy the simple display service, which is subscribed only to the reply messages.
kubectl apply -f 030-test-display-service.yamlWe can now get the logs of the test-display service and you should only see the reply messages:
kubectl logs -l serving.knative.dev/service=test-display -c user-container --tail=100 -n knative-eventing-websocket-sourceYou now have the following deployed:
