The purpose of this project is to reproduce the issue related to kafka consumer count number reducing from a defined value to default value inline with the ThrottlingExceptionRoutePolicy when defined. This project describes the behaviour of how a ThrottlingExceptionRoutePolicy pitches in when a server is unavailable while processing the messages using Camel Kafka. The project runs initially with 15 consumer threads, but drops to 1 after the consumers are restarted after the server is available.
- JDK 1.8
- Maven 3.6
- Apache Kafka 1.0.1 or higher
- Configure below properties in application.properties
kafka.topic
kafka.camelKafkaOptions.brokers
kafka.camelKafkaOptions.groupId
kafka.camelKafkaOptions.sslTruststorePassword
kafka.camelKafkaOptions.sslTruststoreLocation
kafka.camelKafkaOptions.sslKeystorePassword
kafka.camelKafkaOptions.sslKeystoreLocation
kafka.camelKafkaOptions.sslKeyPassword
kafka.camelKafkaOptions.sslEndpointAlgorithm
kafka.camelKafkaOptions.securityProtocol
Ensure that the below 2 applications are running before starting SimpleCamelKafkaConsumer application.
SimpleRestInterface
SimpleCamelKafkaProducer
Run the application using the below command
mvn package
mvn spring-boot:run
The below steps are the observation after application is started
- KafkaConsumer calls the /process endpoint to process the message
- Server times out and throws exception
- Circuit is opened
- Stopping consumers on the topic
- Health check is performed by calling /healthCheck endpoint
- HealthCheck succeeds and Circuit is Closed
- All the kafka consumers start subscribing and un-subscribing one by one, the last thread remains active
The consumer application should run with defined number of threads after server is available.