Decorrelation unpacking assembly crash
mark4o opened this issue · comments
Invalid wavpack files can trigger an invalid memory access and crash in the decorrelation unpacking assembly optimizations. The crash does not occur if configured with --disable-asm.
An example wavpack file (40 bytes, base64-encoded):
d3ZwayAAAAAQBDAwMDAwMDAwMDAwMAAAMBgwfzAwMDACAQPoCgEwMA==
Attempting to decode on x86_64, the crash occurs at src/unpack_x64.s line 775:
default_mono_loop:
mov eax, [rdi+rbx]
rax = 0x00000000fffffffe (i.e. (uint32_t)-2) and imul rbx, rax, -4 sets rbx = 0xfffffffc00000008. Adding this to rdi produces an out of range memory address.
Crash found with afl-fuzz on OS X 10.10.5 clang 3.7 x86_64.
Fixed and closed. Thanks for reporting!