perdy / poc-stream-processor-faust-avro

A Proof of Concept of a streaming application based on Faust that reads messages from a Kafka topic using Avro as deserializer.

PoC Stream Processor

A Proof of Concept of a stream processing application to read and process messages from a Kafka topic.

This PoC is implemented based on a Faust application using Avro as serializing mechanism. To integrate both it's using a couple of open source libraries:

• python-schema-registry-client
• dataclasses-avroschema

Quick Start

1. Build the docker image:

python make build

It could ask for installing some dependencies, such as Clinner and Jinja2, if you aceept it will install them and once all requirements are installed you can run the script.

2. Run the application:

python make run

Requirements

• Python 3.6+

Environment variables

It is required defining some environment variables to have the application working:

• KAFKA_URL
• KAFKA_TOPIC