assertion failed
dovahcrow opened this issue · comments
When using libflate to ungzip gz files from s3, I met this failure.
Not sure how to fix it.
thread 'main' panicked at 'assertion failed: `(left == right)` (left: `1541`, right: `16`)', /home/wooya/.cargo/registry/src/github.com-1ecc6299db9ec823/libflate-0.1.3/src/huffman.rs:97
Is it possible to tell me how to reproduce the error?
If the problematic file is publicly accessible, it will be helpful if you let me know its URL.
I tried to decode some gzip files downloaded from s3 (e.g., emsdk-portable.tar.gz ), but no problem was found.
Sorry there're some sensitive data inside the file so I cannot give it to you directly.
I added these to huffman.rs: 97 and saw some interesting results.
for padding in 0..(1 << (self.max_bitwidth - code.width)) {
let i = ((padding << code.width) | code_be.bits) as usize;
if i == 0 {
println!("
symbol: {}
code: Code {{ width: {}, bits: {:b} }}
code_be.bits: {:b}
i: {}
self.max_bitwidth: {}
padding: {}
value: {}",
symbol,
code.width,
code.bits,
code_be.bits,
i,
self.max_bitwidth,
padding,
value)
}
debug_assert_eq!(self.table[i], MAX_BITWIDTH as u16 + 1);
unsafe {
*self.table.get_unchecked_mut(i) = value;
}
Prints:
symbol: 0
code: Code { width: 3, bits: 0 }
code_be.bits: 0
i: 0
self.max_bitwidth: 7
padding: 0
value: 3
symbol: 48
code: Code { width: 5, bits: 0 }
code_be.bits: 0
i: 0
self.max_bitwidth: 15
padding: 0
value: 1541
symbol: 230
code: Code { width: 13, bits: 10000000000000 }
code_be.bits: 0
i: 0
self.max_bitwidth: 15
padding: 0
value: 7373
It seems like the index 0 has been repeatedly used.
Thank you for your information.
It seems like the index 0 has been repeatedly used.
Typically, it indicates that "the input data is broken".
DecoderBuilder::set_mapping method (and its callers) tries to reconstruct huffman codes from the canonical huffman codes stored in the input data.
But for some reason, the canonical huffman codes have used overlapped (conflicted) bits spaces for different symbols (so the assertion raised).
Of course, the cause may be simply a bug in the libflate.
But, unfortunately, there is no sufficient information to determine the exact reason of the error ... .
Hmm... .
I can not do it for a while because I do not have time, but if I can afford, I will try to reproduce it in my environment.
Thanks!
FYI, I fixed a bug (at the commit a03c2f3 ) which seems the cause of the problem reported by this issue.
Thanks!
I hope this will solve your problem.
Great, I can confirm the bug have been fixed! Thanks for your help!
Good!
Thank you for confirmation.
(And sorry for the inconvenience for a long time)