benchmark mode claims a decompression error on this data w/huf
adamdmoss opened this issue · comments
% ./fse -h -b ./xxx
FSE : Finite State Entropy, 64-bits demo by Yann Collet (Jul 17 2020)
!! Error decompressing block 4 of cSize 18041 !! => (Corrupted block detected)
gunzip the below file and run the above.
xxx.gz
The 'xxx' file appears to survive a huf compress and then a huf decompress intact when doing them individually, so perhaps this is an issue specific to the benchmark mode.
Tested with 3865a70
% cc --version
cc (Ubuntu 8.4.0-1ubuntu1~18.04) 8.4.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
also occurs with:
% clang --version
clang version 9.0.0-2~ubuntu18.04.2 (tags/RELEASE_900/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
... so I suppose, not a compiler issue.
Oh, yikes, might be a dupe of #95 ... except this repro data is tiny. :)
I think the issue is that the xxx file is > 128 KB.
The huffman format requires input data to be provided in blocks <= 128 KB.
When it's longer than that, it must be split accordingly.
Splitting is performed by the I/O layer, but is not present in the benchmark module.
Nevertheless, while there is a explanation, it underlines that the information message is not clear enough about the root cause.
Ah! That was really unobvious. Thanks for the explanation. :)