altworx / kafka_gen_stage

"Klarna's Brod Kafka client wrapped into GenStages."

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

KafkaGenStage

Fast GenStages for reading and writting into Apache Kafka. GenStages are built using excelent feature-full Klarna's Brod Kafka client.

Unfortunately both Kafka & GenStage are using consumer/producer naming. Note that what is consumer from Kafka's perspective, is producer from GenStage's.

KafkaGenStage.Consumer (so:producer GenStage) is buffering events as well as demand.

KafkaGenStage.Producer (coming soon) is controlling demand manually, synchronizing GenStage's ask with brod's ack.

Example

{:ok, consumer} = KafkaGenStage.Consumer.start_link(:started_brod_client, "topic_id", read_until: :latest)

About

"Klarna's Brod Kafka client wrapped into GenStages."

License:Apache License 2.0


Languages

Language:Elixir 100.0%