Timeout overflows when over 2047.48
nhaq-confluent opened this issue · comments
Read the FAQ first: https://github.com/confluentinc/librdkafka/wiki/FAQ
Do NOT create issues for questions, use the discussion forum: https://github.com/confluentinc/librdkafka/discussions
Description
In the confluent-kafka-python library, this issue was raised: confluentinc/confluent-kafka-python#1080
It was deemed the root would likely be in librdkafka.
I found that whenever I set the timeout for the Consumer.poll method > 2047.4... it will start behaving incorrectly.
It looks like there is a overflow in the timeout where the timeout is going to be modulo 2**31 // 10^6 ~= 2147.483648
If I set the timeout to 4305 I will get a ~ 10 second timeout.
How to reproduce
prepare a consumer to connect to an empty topic
consumer = Consumer(**props)
start = time.time()
msg = consumer.poll(2048.0)
print(f"This should take many seconds {time.time - start}")
Versions:
confluent-kafka==1.5.0
confluent_kafka.libversion(): ('1.4.4', 17040639)
Consumer config
- properties:
- auto.offset.reset='earliest'
- bootstrap.servers='kafka:9092'
- enable.auto.commit=False
- group.id='relay.testing'
OS: Ubuntu 16.04.5 LTS
Kafka version: tested against docker image bitnami/kafka:2.6.0
Checklist
IMPORTANT: We will close issues where the checklist has not been completed.
Please provide the following information:
- librdkafka version (release number or git tag):
<REPLACE with e.g., v0.10.5 or a git sha. NOT "latest" or "current">
- Apache Kafka version:
<REPLACE with e.g., 0.10.2.3>
- librdkafka client configuration:
<REPLACE with e.g., message.timeout.ms=123, auto.reset.offset=earliest, ..>
- Operating system:
<REPLACE with e.g., Centos 5 (x64)>
- Provide logs (with
debug=..
as necessary) from librdkafka - Provide broker log excerpts
- Critical issue