j-tim / kafka-admin-client-topic-partition-offsets-playground

Playground for Kafka Admin Client API - Topic Partition Offsets. Using Spring Boot

Kafka Admin Client - Topic Partition offset playground

Playground to play around with the Kafka (Admin) client version 3.0.0.

Next to:

• OffsetSpec.EarliestSpec
• OffsetSpec.LatestSpec
• OffsetSpec.TimestampSpec

Since version 3.0.0 you can get :

• OffsetSpec.MaxTimestampSpec: this returns the offset and timestamp for the record with the highest timestamp.

In combination with listOffsets method on the org.apache.kafka.clients.admin.AdminClient.

See also:

• Extend ListOffset to fetch offset with max timestamp (KIP-734)
• KIP-734

Notes

• This project is using Spring Boot 2.6.0 release candidate 1.
• Spring Boot 2.6.0 RC1 bring the latest version 2.8.0-RC1 of Spring Kafka
• Spring Kafka brings version 3.0.0 of the Kafka (Admin) client

How to run

Run Kafka, Zookeeper and Kafka manager using Docker:

docker-compose up -d

Build the project:

./mvnw clean package

Start the Spring Boot application using Maven. Or import the project in your favorite IDE.

./mvnw spring-boot:run

Every second a Chuck Norris 'fact' will be produced to topic with name chuck-norris. Every 10 seconds the latest offsets and timestamps for each partition in the topic will be printed.

To change the defaults check the: application.yml:

• print.offsets.rate.ms: To change the rate in milliseconds to print the partition offsets.
• chuck.norris.fact.producer.cron: Change the cron of the producer

Errors

In case you are running into this issue your broker don't support MAX_TIMESTAMP offset feature. Use Kafka version 3.0.0 or Confluent Kafka version 7.0.0 or later!

java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.UnsupportedVersionException: Broker 1 does not support MAX_TIMESTAMP offset spec
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395) ~[na:na]
at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999) ~[na:na]
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:165) ~[kafka-clients-3.0.0.jar:na]
at ScheduledTopicOffsetPrinter.doSomething(AdminService.java:38) ~[classes/:na]
at jdk.internal.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) ~[spring-context-5.3.12.jar:5.3.12]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.3.12.jar:5.3.12]
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
at java.base/java.util.concurrent.FutureTask.runAndReset$$$capture(FutureTask.java:305) ~[na:na]
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java) ~[na:na]
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: org.apache.kafka.common.errors.UnsupportedVersionException: Broker 1 does not support MAX_TIMESTAMP offset spec