Apache Kafka
Start:
nohup ./bin/zookeeper-server-start.sh config/zookeeper.properties > /dev/null 2>&1 &
nohup ./bin/kafka-server-start.sh config/server.properties > /dev/null 2>&1 & # Kafka Broker 1
nohup ./bin/kafka-server-start.sh config/server2.properties > /dev/null 2>&1 & # Kafka Broker 2
nohup ./bin/kafka-server-start.sh config/server3.properties > /dev/null 2>&1 & # Kafka Broker 3
Create Topic:
./bin/kafka-topics.sh --create --zookeeper zkhost:2181 --replication-factor 1 --partitions 1 --topic dztopic1
Delete Topic:
./bin/kafka-topics.sh --delete --zookeeper localhost:2181 --topic dztopic1
List Topics:
./bin/kafka-topics.sh --zookeeper zkhost:2181 --list
Describe Topic:
./bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic dztopic1
CMD Line Producer: (Default Broker Port: 9092, Hortonworks uses broker port: 6667)
echo "DZ Kafka Event at $(date)" | ./bin/kafka-console-producer.sh --broker-list kafkahost:9092 --topic dztopic1 > /dev/null
CMD Line Consumer: (Default Broker Port: 9092, Hortonworks uses broker port: 6667)
./bin/kafka-console-consumer.sh --bootstrap-server kafka_broker:6667 --topic dztopic1 --from-beginning
Purge Kafka Topic
/opt/kafka/bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic eye_tracking --config retention.ms=1000
References:
Kafka Documentation
Python PyKafka
Python Kafka