Giters

quixio / quix-streams

Quix Streams - A library for data streaming and Python Stream Processing

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Allow using consumer group prefix when not using consumer group

peter-quix opened this issue · comments

commented

Is your feature request related to a problem? Please describe.
While the title sounds counter intuitive, the problem is real. When there is no consumer group in use, librdkafka still requires a consumer group to connect to Kafka with, but it will not commit anything. Because of this quirk of the underlying library we need to be able to specify a prefix to use when a random consumer group is generated for this purpose.

Describe the solution you'd like
Add consumer_prefix="" in python and relevant in C# when opening a topic for consumption.

Describe alternatives you've considered
Not an alternative solution, but a bad workaround: specify consumer group with your prefix and a random id. Won't behave exactly the same, as by default commit is enabled, but future runs won't use it if you always use a new one.