ignore compress level options
calestyo opened this issue · comments
Hey.
Many other programs, where bzip3 might be used as drop-in replacement for other well-known compression tools use -0 to -9 as well as --fast and --best.
Even if there is no such difference in bzip3 it would make life easier if these could be just ignored, or there was a switch that could make them silently ignored.
Thanks,
Chris.
My issue with compression level options is that we would have to map them to block sizes, and there is some data for which a bigger block size would negatively impact compression ratio and performance (in a parallel setting).
The biggest block size is 511M and uses approximately 3GB of memory. If anything, I think I am just scared of people running out of memory in an episode of brain fog or instantly assuming that bigger block = better compression.
I think we could maybe clarify these in the manpage and add -1..-9 flags either way...
Well if there was serious concern, that it could generally lead to worse compression... one could in principle still go the way of having a switch like --no-ignore-compression-levels,... but arguably that could be considered quite some strange behaviour (i.e. needing an option, to enable an actually given one).
If memory isn't enough, one could perhaps warn an just use lower settings?
I don't like this solution. I don't want to add "decoy" compression levels, because I can see someone misunderstanding that.
I'm already settled on the fact that bzip3 will not support setting the compression level (whatever it is defined as), unless someone comes up with a solution that is not misleading.
Ok,... then I guess we can close this issue and any other programs that want to use bzip3 as drop-in replacement would need to work around the inability to set the levels.
My suggestion is to map -9 to -b511, -8 to -b256, -7 to -b128, -6 to -b64, -5 to -b32, -4 to -b16, -3 to -b8, -2 to -b4 and -1 to -b2, should you want to take the "liability" of this going wrong yourself.
Well that wouldn't really help for my use case... if you look at the reference to the btrbk issue above...
It uses allows using various compression tools… and setting a level (since all support them). I've made a PR that adds bzip3 support, but that doesn't have levels.
So when I do any such mapping as you suggest, one can also just extend btrbk to allow not setting a level in the first place.
The idea of this issue really was, to make it easier to use bzip3 as a drop-in for other such tools, of which most follow similar syntax for some common options. (-c, etc.).
bzip3 strives to be as compatible with other compressors as possible not counting the levels. The levels mechanic doesn't work for bzip3, it was deliberately not implemented - you have to (sadly) specially accommodate for bzip3 in one way or another.
Well that's why I've originally proposed to have them simply ignored.
Anyway ... closing :-)