Name: Gustian
Kelompok: A
Group: 3
This project demonstrates the use of Apache Kafka to stream Bitcoin price data from a CSV file into a Kafka topic and consume it. The setup involves Docker for running Kafka, Zookeeper, Schema Registry, and Control Center, along with Python scripts for producing and consuming messages.
- Docker and Docker Compose
- Python 3.x
- Pip
-
Clone the repository:
https://github.com/gustianmajs/Session-82-Homework-Kafka-III.git
-
Install Python dependencies:
pip install -r requirements.txt
-
Start the Kafka services using Docker Compose:
docker-compose up -d
This will start the following services:
- Zookeeper
- Kafka Broker
- Schema Registry
- Control Center
The producer reads data from data/bitcoin_price_Training.csv
and sends it to the Kafka topic bitcoin_prices
.
python producer.py
The consumer reads data from the Kafka topic bitcoin_prices
.
python consumer.py
The consumer will continuously poll the Kafka topic and print the received messages.
- Zookeeper: Manages and coordinates the Kafka brokers.
- Kafka Broker: The Kafka server that handles message production and consumption.
- Schema Registry: Manages Avro schemas for Kafka topics, ensuring data compatibility.
- Control Center: A web interface to monitor and manage the Kafka cluster. Accessible at http://localhost:9021.