A flexible framework to process Kafka messages with multithreading, middlewares, serialization, compression, typed handlers and message order support.
- Fluent configuration
- In memory message buffering
- Improve client code testability
- Multiple cluster support
- Multiple workers consuming the same topic (limited by the machine resources)
- Message order guarantee for the same partition key
- Multiple consumer groups in the same topic
- Multiple topics in the same consumer
- Different work distribution strategies
- Middleware support for consumers and producers implementing
IMessageMiddlewareinterface - Serialization middleware (ProtoBuf and Json are shipped with the framework but they have different nuget packages, you can support custom serialization using
IMessageSerializerinterface) - Compression middleware (Gzip is shipped with the framework but it has a different nuget package, you can support custom compressions using
IMessageCompressorinterface) - Graceful shutdown (waits for the message processor ends to shutdown)
- Store message offset only when message processing ends, avoiding message loss (or you can store it manually)
- Read or write message headers
- Ignore messages
- Manipulate the message
- Custom error handling and retry policies
- Monitoring and performance measurement
- Tracing
- Maintain compatibility with other frameworks
- And so on...
You should install KafkaFlow with NuGet:
Install-Package KafkaFlowOr via the .NET Core command line interface:
dotnet add package KafkaFlowEither commands, from Package Manager Console or .NET Core CLI, will download and install KafkaFlow and all required dependencies.
See the samples folder to see Producer and Consumer samples
- Fork this repository
- Follow project guidelines
- Do your stuff
- Open a merge request using the template
By sending us your contributions, you are agreeing that your contribution is made subject to the terms of our Contributor Ownership Statement