iwpnd / geo-stream-kafka

producer and consumer FastAPI endpoints to handle incoming geo data

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

geo-stream-kafka

setup geostream fastapi aiokafka

geostream gif

see blogpost for a more detailed description

docker-compose up -d

If you're on MacOS set an environment variable like:

export DOCKER_KAFKA_HOST=$(ipconfig getifaddr en0)

that is afterwards used in docker-compose.yml to identify the KAFKA_ADVERTISED_HOST_NAME. Some similar workaround has to exist for Windows users. For linux I assume you can just set it to localhost if you're only running on Kafka node. See github.com/wurstmeister/kafka-docker/wiki/Connectivity.

Kafka

Kafka will be served on localhost:9092

Frontend

open /geostream/frontend/app/index.html and associated map.js will consume messages when they come in

Producer

can produce messages to a topic at localhost:8002/producer/<topicname>

Consumer

can consume kafka messages through at localhost:8003/consumer/<topicname>

About

producer and consumer FastAPI endpoints to handle incoming geo data

License:MIT License


Languages

Language:Python 59.8%Language:Jupyter Notebook 28.7%Language:JavaScript 5.7%Language:HTML 3.1%Language:Dockerfile 1.8%Language:CSS 0.9%