rd_kafka_produceva: double free headers on message sending error
blindspotbounty opened this issue · comments
Description
Accidentally, we found a problem related to headers double free in rd_kafka_produceva
method:
=================================================================
==49557==ERROR: AddressSanitizer: heap-use-after-free on address 0x0001178e9dd8 at pc 0x00010f6813b4 bp 0x00016b719000 sp 0x00016b718ff8
READ of size 8 at 0x0001178e9dd8 thread T18
#0 0x10f6813b0 in rd_list_destroy_elems rdlist.c:276
#1 0x10f681828 in rd_list_destroy rdlist.c:300
#2 0x10f2fb2d0 in rd_kafka_headers_destroy rdkafka_header.c:37
#3 0x10f465414 in rd_kafka_produceva rdkafka_msg.c:521
#4 0x10fbcc5dc in RDKafkaClient._produceVariadic(topicHandle:partition:messageFlags:key:value:opaque:cHeaders:) RDKafkaClient.swift:218
#5 0x10fbc8910 in closure #1 in closure #1 in closure #1 in RDKafkaClient.produce<A, B>(message:newMessageID:topicConfiguration:topicHandles:) RDKafkaClient.swift:137
#6 0x10fbe5188 in partial apply for closure #1 in closure #1 in closure #1 in RDKafkaClient.produce<A, B>(message:newMessageID:topicConfiguration:topicHandles:) <compiler-generated>
...
0x0001178e9dd8 is located 8 bytes inside of 48-byte region [0x0001178e9dd0,0x0001178e9e00)
freed by thread T18 here:
#0 0x105133380 in wrap_free+0x98 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x53380)
#1 0x10f2fb2fc in rd_free rd.h:151
#2 0x10f2fb2d8 in rd_kafka_headers_destroy rdkafka_header.c:38
#3 0x10f460c04 in rd_kafka_msg_destroy rdkafka_msg.c:118
#4 0x10f4652ec in rd_kafka_produceva rdkafka_msg.c:505
...
previously allocated by thread T18 here:
#0 0x105133244 in wrap_malloc+0x94 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x53244)
#1 0x10f2fb3b4 in rd_malloc rd.h:139
#2 0x10f2fb330 in rd_kafka_headers_new rdkafka_header.c:44
#3 0x10f4646f4 in rd_kafka_produceva rdkafka_msg.c:431
How to reproduce
I was reproducing it with swift-kafka-client wrapping client within issue swift-server/swift-kafka-client#150
git clone --recursive https://github.com/swift-server/swift-kafka-client
git apply diff // (from issue)
swift test --sanitize=address --filter testProduceAndConsumeWithMessageHeaders
Checklist
IMPORTANT: We will close issues where the checklist has not been completed.
Please provide the following information:
- librdkafka version (release number or git tag):
v2.3.0
- Apache Kafka version: N/A
- librdkafka client configuration: any invalid broker address
- Operating system:
ubuntu 22.04 lts
- Provide logs (with
debug=..
as necessary) from librdkafka: ASAN logs instead - Provide broker log excerpts: N/A
- Not a critical issue
Yeah, the hdrs
variable should be set to NULL before rd_kafka_msg_destroy
if it corresponds to the message headers, otherwise it's freed a second time here
Yes, exactly.
Btw: fixed comment as requested in PR.