confluentinc / parallel-consumer

Parallel Apache Kafka client wrapper with per message ACK, client side queueing, a simpler consumer/producer API with key concurrency and extendable non-blocking IO processing.

Home Page:https://confluent.io/confluent-accelerators/#parallel-consumer

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Dynamic concurrency control with flow control or tcp congestion control theory

astubbs opened this issue · comments

Prevent user from having to choose a level of concurrency with either the core engine or the vertx engine.

Uses concepts from tcp congestion control to set the thread pool size by measuring performance of execution.
Originally planned to do this by hand using concepts from Engineering Control Theory, but stumbled upon Netflix's concurrency-limits library, which uses very similar theory from TCP congestion control theory.

Closing Issue