The goal is be able to use the KafkaPlugin created by @metadady to Visualize Apache log data in Minecraft with StreamSets Data Collector using docker. This branch takes care of setting up Minecraft Spigot server along with Kafka and Zookeeper. It also includes a SCD container with the necessary libraries.
- Docker
- Docker compose
- Minecraft client
- Minecraft account
- Add your Minecraft user's name to .env. See example.env
Once this repo has been cloned, open your command line and initialize the docker containers using: $ docker-compose up.
This can take a while, be aware Minecraft Spigot server can take a while.
Once the containers are up and running follow these steps:
- Import the Kakfa producer pipeline into SDC by going to http://localhost:18630
You should see the following pipeline
You can Preview or Start the pipeline right away.
- Load the KafkkaCraft plugin
A slightly modified version of the original plugin has been created for this repo, here is the source.
Place it into ./minecraft/data/plugins and restart the Spigot server $docker restart mc.
- Connect your Minecraft (client) to the local Spigot server
- Run the kafka command and you'll see a map rendering in front of you
To render the map from scratch:
/kafka world log true kafka
To reuse a map already rendered:
/kafka world log false kafka
For more information regarding the parameters see the yml file or /kafka help.
- Play some data into the Minecraft world
Once the map has been rendered, the plugin will subscribe a Kafka consumer to the topic log.
Play some data from your command line using the script readfile.sh
$ cd ./streamsets/apache-logs
$ ./readfile.sh apache_to_read.log >> apache_access.log
A small Apache file have been included but you can use any.