(with Bitnami Helm Chart)
- Docker
- Kubernetes
- Helm
- Kafka Client
$ git clone git@github.com:LOG-INFO/kafka-clsuter-with-bitnami.git
2. Bitnami Helm Repository를 추가한다 (Helm 설치)
$ helm repo add bitnami https://charts.bitnami.com/bitnami
# 기본 values.yaml이 있긴 하지만, 덮어쓰고 싶은 내용들은 kafka/values.yaml을 참조한다
# 설정값을 변경하고 싶다면 https://github.com/bitnami/charts/tree/master/bitnami/kafka/#installing-the-chart 를 참고한다
$ helm install kafka -f kafka/values.yaml bitnami/kafka
$ kubectl get pods INT rancher-desktop kube
NAME READY STATUS RESTARTS AGE
kafka-0 2/2 Running 6 1m
kafka-1 2/2 Running 6 1m
kafka-2 2/2 Running 6 1m
kafka-zookeeper-0 1/1 Running 1 1m
kafka-zookeeper-1 1/1 Running 1 1m
kafka-zookeeper-2 1/1 Running 1 1m
kafka-exporter-97b464596-c46zm 1/1 Running 6 1m
$ kubectl get services ok rancher-desktop kube
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kafka ClusterIP 10.43.175.234 <none> 9092/TCP 1m
kafka-headless ClusterIP None <none> 9092/TCP,9093/TCP 1m
kafka-0-external NodePort 10.43.5.144 <none> 9094:30001/TCP 1m
kafka-1-external NodePort 10.43.83.102 <none> 9094:30002/TCP 1m
kafka-2-external NodePort 10.43.73.183 <none> 9094:30003/TCP 1m
kafka-zookeeper ClusterIP 10.43.201.237 <none> 2181/TCP,2888/TCP,3888/TCP 1m
kafka-zookeeper-headless ClusterIP None <none> 2181/TCP,2888/TCP,3888/TCP 1m
kafka-metrics ClusterIP 10.43.242.108 <none> 9308/TCP 1m
kafka-jmx-metrics ClusterIP 10.43.201.19 <none> 5556/TCP 1m
동작수행 확인을 위한 임시 컨테이너를 띄운다.
- 임시 컨테이너에서 콘솔 프로듀서를 띄워서 카프카 메시지 전송을 테스트
$ docker run --network host --rm -it -u 0 bitnami/kafka /bin/bash
root@docker-desktop:/# /opt/bitnami/kafka/bin/kafka-console-producer.sh --bootstrap-server localhost:30001,localhost:30002,localhost:30003 --topic test
>{"message":"test_message"}
>^C # Ctrl + C
- 테스트 메시지 컨슈밍 테스트
root@docker-desktop:/# /opt/bitnami/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:30001,localhost:30002,localhost:30003 --topic test --from-beginning --group test
{"message":"test_message"} # 메시지 입수 확인
$ kubectl apply -f prometheus,grafana
kubectl get pods INT rancher-desktop kube
NAME READY STATUS RESTARTS AGE
prometheus-776fd895bc-vg87g 1/1 Running 1 1m
grafana-9bd5bbd6b-x5khn 1/1 Running 1 1m
$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
grafana NodePort 10.43.194.39 <none> 3000:31000/TCP 1m
prometheus NodePort 10.43.224.108 <none> 9090:31001/TCP 1m
- Prometheus: http://localhost:31001/
- Grafana: http://localhost:31000/
- Kafka JMX Grafana Dashboard (Grafana Dashboard ID:
12483
) - Kafka Grafana Dashboard (Grafana Dashboard ID:
7589
)
$ helm delete kafka
$ kubectl delete -f prometheus,grafana