██████╗ ██╗ ██╗██████╗ ███████╗██╗ ██╗██████╗
██╔══██╗██║ ██║██╔══██╗ ██╔════╝██║ ██║██╔══██╗
██████╔╝██║ ██║██████╔╝ ███████╗██║ ██║██████╔╝
██╔═══╝ ██║ ██║██╔══██╗ ╚════██║██║ ██║██╔══██╗
██║ ╚██████╔╝██████╔╝ ███████║╚██████╔╝██████╔╝
╚═╝ ╚═════╝ ╚═════╝ ╚══════╝ ╚═════╝ ╚═════╝
-
Repo:
-
Full Redpanda Cluster with 3 Brokers running on Kubernetes
-
Applications to consume and produce events from the Central Cluster
-
-
Prerequisites
-
k3
Running locally -
kubectl
+helm
cli -
Docker
# Install k3d
wget -q -O - https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash
# Create cluster
k3d cluster create event-streaming
# One time setup for red panda repo
helm repo add redpanda https://charts.redpanda.com/
helm repo update
# Install red-panda
helm upgrade --install redpanda redpanda/redpanda --namespace redpanda --create-namespace --wait --values=./redpanda/k3d-values.yaml
helm repo add kubernetes-dashboard https://kubernetes.github.io/dashboard/
helm upgrade --install kubernetes-dashboard kubernetes-dashboard/kubernetes-dashboard --create-namespace --namespace kubernetes-dashboard
This command deploys kubernetes-dashboard on the Kubernetes cluster in the kubernetes-dashboard
namespace with default configuration.
kubectl apply -f ./redpanda/dashboard-adminuser.yaml
kubectl apply -f ./redpanda/cluster-role-binding.yaml
export POD_NAME=$(kubectl get pods -n kubernetes-dashboard -l "app.kubernetes.io/name=kubernetes-dashboard,app.kubernetes.io/instance=kubernetes-dashboard" -o jsonpath="{.items[0].metadata.name}")
echo https://127.0.0.1:8443/
kubectl -n kubernetes-dashboard port-forward $POD_NAME 8443:8443
NOTE: You need a user token to log into the dashboard.
kubectl -n kubernetes-dashboard create token admin-user
NOTE: You need to periodically log into dashboard again with new token.
# BUILD image
docker build -t redpanda-node-app .
# Import image
k3d image import redpanda-node-app -c event-streaming
NOTE: You need to repeate these steps if you change source code.
NOTE: This image also includes the "rpk" redpanda CLI.
NOTE: cluster info
is the rpk command. You can use any other rpk commands here as well.
kubectl create namespace redpanda
kubectl apply -f ./redpanda/node-kafka-app.yaml
Make sure everything is working...
kubectl get pod -n redpanda
This should show result like this:
NAME READY STATUS RESTARTS AGE
redpanda-0 2/2 Running 0 3h43m
redpanda-configuration-7586j 0/1 Completed 0 3h43m
kefka-app-57776b785d-58pkg 1/1 Running 0 33m
kubectl get replicaset -n redpanda
This should show result like this:
NAME DESIRED CURRENT READY AGE
kefka-app-57776b785d 1 1 1 35m
kubectl get statefulset -n redpanda
This should show result like this:
NAME READY AGE
redpanda 1/1 3h45m
-
Get the name of "kafla app" pod (using
kubectl get pod -n redpanda
) -
Log into the "kafka app" pod
kubectl exec -n redpanda -it redpanda-node-app-6d46cb8bb6-5nwl4 -- /bin/sh
- Check Redpanda Cluster (using rpk)
./rpk cluster info --brokers redpanda-0.redpanda.redpanda.svc.cluster.local.:9093
- Run the chat room sample app (You need to log into two separate terminals and run the same app)
node ./src/index.mjs
- Building Java App and importing to Cluster (can use an Image from your Image Repo as well)
./gradlew fatJar
docker build -t redpanda-java-app:latest -f Dockerfile ..
k3d image import redpanda-java-app:latest -c event-streaming
- Run the JVM App that sends and produces messages
kubectl apply -f java-kafka-app.yaml
# Get correct deployment value by checking kubectl pods -n redpanda
kubectl exec -n redpanda -it redpanda-java-app-c67c4ddd9-62gdq -- /bin/sh
# Run java app
java -jar app.jar
# Now you can send messages -
# Or type poll to use the Consumer Group and offsets to retrieve latest messages