A simple environment to test out system design concept
A simple system to allow user to login and save some random message. A summary dashboard to show the total word count of all message.
- Web API
- Web
- Zookeeper
- Redis
- Kafka
- Minio
- Spark
- Notification and STOMP
Deployed following cluster on Kubernetes for fun.
| Tech | Spec | Usage |
|---|---|---|
| Redis | cluster of 3 by bitnami/redis-cluster | As a cache cluster |
| Kafka | cluster of 3 by bitnami/kafka | As message queue |
| Minio | cluster of 4 by minio/minio | Store data for Spark to process. |
| Spark | Use GoogleCloudPlatform/spark-on-k8s-operator | Ask backend wordcount processing |
| Zookeeper | cluster of 3 by bitnami/zookeeper | Keep track of Kafka and spring boot discovery |
| MySQL | A single host database | As database |
| docker hub | function |
|---|---|
| sd-read-api | Read API |
| sd-write-api | Write API |
| sd-main-api | Main API |
| sd-wordcount-spark | Spark job |
| sd-kafka-consumer | Kafka consumer |
| sd-web | Web |