confluentinc / librdkafka

The Apache Kafka C/C++ library

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

rd_kafka_produceva: double free headers on message sending error

blindspotbounty opened this issue · comments

Description

Accidentally, we found a problem related to headers double free in rd_kafka_produceva method:

=================================================================
==49557==ERROR: AddressSanitizer: heap-use-after-free on address 0x0001178e9dd8 at pc 0x00010f6813b4 bp 0x00016b719000 sp 0x00016b718ff8
READ of size 8 at 0x0001178e9dd8 thread T18
#0 0x10f6813b0 in rd_list_destroy_elems rdlist.c:276
    #1 0x10f681828 in rd_list_destroy rdlist.c:300
    #2 0x10f2fb2d0 in rd_kafka_headers_destroy rdkafka_header.c:37
    #3 0x10f465414 in rd_kafka_produceva rdkafka_msg.c:521
    #4 0x10fbcc5dc in RDKafkaClient._produceVariadic(topicHandle:partition:messageFlags:key:value:opaque:cHeaders:) RDKafkaClient.swift:218
    #5 0x10fbc8910 in closure #1 in closure #1 in closure #1 in RDKafkaClient.produce<A, B>(message:newMessageID:topicConfiguration:topicHandles:) RDKafkaClient.swift:137
    #6 0x10fbe5188 in partial apply for closure #1 in closure #1 in closure #1 in RDKafkaClient.produce<A, B>(message:newMessageID:topicConfiguration:topicHandles:) <compiler-generated>
    ...
    
0x0001178e9dd8 is located 8 bytes inside of 48-byte region [0x0001178e9dd0,0x0001178e9e00)
freed by thread T18 here:
#0 0x105133380 in wrap_free+0x98 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x53380)
    #1 0x10f2fb2fc in rd_free rd.h:151
    #2 0x10f2fb2d8 in rd_kafka_headers_destroy rdkafka_header.c:38
    #3 0x10f460c04 in rd_kafka_msg_destroy rdkafka_msg.c:118
    #4 0x10f4652ec in rd_kafka_produceva rdkafka_msg.c:505
...

previously allocated by thread T18 here:
#0 0x105133244 in wrap_malloc+0x94 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x53244)
    #1 0x10f2fb3b4 in rd_malloc rd.h:139
    #2 0x10f2fb330 in rd_kafka_headers_new rdkafka_header.c:44
    #3 0x10f4646f4 in rd_kafka_produceva rdkafka_msg.c:431

How to reproduce

I was reproducing it with swift-kafka-client wrapping client within issue swift-server/swift-kafka-client#150

git clone --recursive https://github.com/swift-server/swift-kafka-client
git apply diff // (from issue)
swift test --sanitize=address --filter testProduceAndConsumeWithMessageHeaders

Checklist

IMPORTANT: We will close issues where the checklist has not been completed.

Please provide the following information:

  • librdkafka version (release number or git tag): v2.3.0
  • Apache Kafka version: N/A
  • librdkafka client configuration: any invalid broker address
  • Operating system: ubuntu 22.04 lts
  • Provide logs (with debug=.. as necessary) from librdkafka: ASAN logs instead
  • Provide broker log excerpts: N/A
  • Not a critical issue

Yeah, the hdrs variable should be set to NULL before rd_kafka_msg_destroy
if it corresponds to the message headers, otherwise it's freed a second time here

Yes, exactly.
Btw: fixed comment as requested in PR.