Please support generating a single gzip-compatible or deflate-compatible stream
joshtriplett opened this issue · comments
Currently, pargz uses many GzEncoder instances, which each emit a gzip header, so decoding it requires MultiGzDecoder. If pargz instead used DeflateEncoder to emit raw deflate streams, with an optional gzip or zlib header in front, the resulting stream would be a single gzip/zlib/deflate file, and wouldn't require MultiGzDecoder.
I'd love to be able to use pargz in file formats that expect raw deflate, or that expect zlib, or that expect gzip but don't handle multiple gzip headers.
I'll look into this. 👍
Thanks for opening the issue!
@joshtriplett See linked PR on this ticket for details and the v0.5.0 release which addresses the following items:
- Single stream deflate for Zlib, Gzip, and Raw Deflate
- Uses last 32K of previous block as dictionary for current block
Not yet addressed:
- A pure rust version of adler32 that has a combine method
- passing lots of details to the gzip / zlib headers
Please feel to reopen this issue or make new ones if you run into problems or have ideas for improving things!
Thank you!