ASAN bug missed by GCC_PLUGIN mode
bendrissou opened this issue · comments
When ffmpeg project is compiled with gcc-asan toolchain, a certain heap-buffer-overflow error can be reproduced as shown here. However, when compiling the same project with afl-cc/afl-gcc-fast, the bug goes undetected.
Note that when compiling the projcet with afl-gcc, the bug is detected successfully. Which means the issue is caused by the GCC_PLUGIN mode.
Thanks @vanhauser-thc - we'll take a look.
@bendrissou can you confirm that you are enabling ASAN for the fuzzing session. It's obviously not something that happens by default; not everyone would want this as there is a performance overhead. ASAN is also not as useful for fuzzing applications developed in memory safe languages languages with built in runtime checks like Ada.
Have a look at the -fsanitize=address switch.
Or are you implying that afl-gcc-fast is cancelling out the effects of ASAN?
I can confirm that ASAN is enabled, as I can trigger other memory errors, just not this one.
Ok, but AFL can also trigger memory errors without ASAN enabled. For example, ASAN is very good at triggering buffer overflows that don't cause seg faults, but if an overflow does trigger the OS to raise a fault, AFL can already detect that through an analysis of the associated core dump.
Before we start investigating, please can you confirm that you have used the GCC -fsanitize=address switch to turn on ASAN within the fuzzing session.
@thebutcherboy that is what he means. afl-gcc-fast -fsanitize=address does not trigger a specific crash (but others are detected), whereas without afl-gcc-fast ASAN triggers.
Understood, thanks both. I'll ask our compiler team to investigate.
ASAN is enabled by setting AFL_USE_ASAN variable before compiling the target. And this is confirmed by running some inputs and getting ASAN reports.
Many thanks @bendrissou - that's good info.
@vanhauser-thc I debug on this to check if it is caused by afl-cc, and fortunately not :)
However it can be observered that, with afl-gcc-fast, afl-compiler-rt.o is linked in. So probably duplicate of #1716 .
My env:
root@localhost:~# gcc --version
gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Confirmed:
(gdb) b __asan_region_is_poisoned
Breakpoint 11 at 0x55555ab21280: __asan_region_is_poisoned. (3 locations)
(gdb) c
Continuing.
Thread 19 "vf#0:0" hit Breakpoint 11, __asan_region_is_poisoned (beg=beg@entry=0x603000000bb1, size=size@entry=1) at instrumentation/afl-compiler-rt.o.c:2255
2255 instrumentation/afl-compiler-rt.o.c: No such file or directory.
(gdb) where
#0 __asan_region_is_poisoned (beg=beg@entry=0x603000000bb1, size=size@entry=1) at instrumentation/afl-compiler-rt.o.c:2255
#1 0x00007ffff7605c9b in __interceptor_strchr (s=0x603000000bb1 "", c=<optimized out>) at ../../../../src/libsanitizer/asan/asan_mapping.h:401
#2 0x0000555555c26b0f in ff_rate_control_init (s=s@entry=0x625000002900) at src/libavcodec/ratecontrol.c:560
#3 0x0000555555bfdcdd in ff_mpv_encode_init (avctx=avctx@entry=0x619000001480) at src/libavcodec/mpegvideo_enc.c:956
#4 0x0000555555be8596 in encode_init (avctx=0x619000001480) at src/libavcodec/mpeg4videoenc.c:1291
#5 0x000055555779e3b9 in avcodec_open2 (avctx=0x619000001480, codec=codec@entry=0x55555b3961e0 <ff_mpeg4_encoder>, options=options@entry=0x617000000890) at src/libavcodec/avcodec.c:337
#6 0x0000555555dbf319 in enc_open (opaque=0x617000000780, frame=<optimized out>) at src/fftools/ffmpeg_enc.c:319
#7 0x0000555555e903b2 in enc_open (sch=0x613000000040, frame=0x61500000cb00, enc=0x60c000000d00) at src/fftools/ffmpeg_sched.c:1571
#8 send_to_enc (frame=0x61500000cb00, enc=0x60c000000d00, sch=0x613000000040) at src/fftools/ffmpeg_sched.c:1571
#9 sch_filter_send (sch=0x613000000040, fg_idx=<optimized out>, out_idx=<optimized out>, frame=frame@entry=0x61500000cb00) at src/fftools/ffmpeg_sched.c:2154
#10 0x0000555555ddcbab in fg_output_frame (ofp=ofp@entry=0x612000000f40, fgt=fgt@entry=0x7fffea89e5f0, frame=frame@entry=0x6150000dff00) at src/fftools/ffmpeg_filter.c:2303
#11 0x0000555555de35a8 in fg_output_step (frame=<optimized out>, fgt=<optimized out>, ofp=<optimized out>) at src/fftools/ffmpeg_filter.c:2406
#12 read_frames (fg=fg@entry=0x60e0000002e0, fgt=fgt@entry=0x7fffea89e5f0, frame=0x6150000dff00) at src/fftools/ffmpeg_filter.c:2467
#13 0x0000555555df0ead in filter_thread (arg=0x60e0000002e0) at src/fftools/ffmpeg_filter.c:2879
#14 0x0000555555e7c460 in task_wrapper (arg=0x611000000f68) at src/fftools/ffmpeg_sched.c:2200
#15 0x00007ffff7427609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#16 0x00007ffff734c353 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
The const NULL from __asan_region_is_poisoned in afl-compiler-rt.o will break the interceptor of strchr. This happens at libsanitizer/asan/asan_interceptors_memintrinsics.h:60 (gcc 9.4.0):
#define ACCESS_MEMORY_RANGE(ctx, offset, size, isWrite) do { \
uptr __offset = (uptr)(offset); \
uptr __size = (uptr)(size); \
uptr __bad = 0; \
if (__offset > __offset + __size) { \
GET_STACK_TRACE_FATAL_HERE; \
ReportStringFunctionSizeOverflow(__offset, __size, &stack); \
} \
if (!QuickCheckForUnpoisonedRegion(__offset, __size) && \
(__bad = __asan_region_is_poisoned(__offset, __size))) { \
AsanInterceptorContext *_ctx = (AsanInterceptorContext *)ctx; \
bool suppressed = false; \
if (_ctx) { \
suppressed = IsInterceptorSuppressed(_ctx->interceptor_name); \
if (!suppressed && HaveStackTraceBasedSuppressions()) { \
GET_STACK_TRACE_FATAL_HERE; \
suppressed = IsStackTraceSuppressed(&stack); \
} \
} \
if (!suppressed) { \
GET_CURRENT_PC_BP_SP; \
ReportGenericError(pc, bp, sp, __bad, isWrite, __size, 0, false);\
} \
} \
} while (0)The expression
!QuickCheckForUnpoisonedRegion(__offset, __size)is evaluated as True. But
(__bad = __asan_region_is_poisoned(__offset, __size))is evalutated as False when afl-compiler-rt.o is here, which is expected to be True, for the bug @bendrissou mentioned.
@SonicStark oh right, yeah that is likely the case.
maybe setting -static-libasan in afl-cc if ASAN and afl-gcc-fast is used is the right move here?
@SonicStark @bendrissou
can you please test the following branch: https://github.com/AFLplusplus/AFLplusplus/tree/gccasan
it enforces -staticlibasan if GCC_PLUGIN mode is used.
I don't have all the details of the bug on my mind anymore, but I suspect enforcing static asan is more a workaround than an actual solution.
It is totally fine and a good solution though for AFL++ :)
Note that the same issue exists in LibAFL: AFLplusplus/LibAFL#1231
I don't have all the details of the bug on my mind anymore, but I suspect enforcing static asan is more a workaround than an actual solution.
It is totally fine and a good solution though for AFL++ :)
yes it is just a workaround - if it works (still needs someone to test), but that is all we can do on our end :)
Just tested it, and the workaround works.
If you two have any opinions on who is at fault or if this can be fixed somehow feel free to leave a comment at llvm/llvm-project#62431
I think it is wild that shared ASan is only working on Ubuntu.
Maybe the fix is to force the compilation of ASAN to use -Wl,-Bsymbolic-functions. That way ASab will at least behave the same on Ubuntu and Debian.
Works for me, too :)
@vanhauser-thc However it fails on some regression tests I did before for #1912 . See the comments I left for 7ec3388 .
I can also confirm that this fixes ASan with afl-g++-fast
thanks guys! merging this then.