Implement Write
Pr0methean opened this issue · comments
I'm trying to integrate Zopfli into https://crates.io/crates/zip_next, but an obstacle to this is that it's designed to let users write the file to be compressed incrementally. Thus, I can't provide a Read implementation to https://docs.rs/zopfli/latest/zopfli/fn.compress.html (even in a separate thread) except by buffering the entire file in memory; otherwise, the reader might catch up to the writer, read 0 bytes, and conclude EOF prematurely. Since the crate targets stable Rust 1.66, coroutines (where the reader would block instead of reading 0 bytes, until the actual EOF) aren't an option either. A compressing implementation of Write would solve this problem.
This is a very good idea, thank you for sharing it!
The current compress function could be trivially implemented by copying from the reader to the compressing Zopfli writer, and in fact it could provide for a simpler end of Deflate stream detection technique: just wait until the writer is dropped.
Edit: by the way, if getting the functionality working for you is urgent and the input data always is a file in an OS-mounted filesystem, you can map it to memory to work with it as if it was a big slice. Note that the soundness and safety of byte slices backed by memory maps in Rust is a topic of debate.
The requested feature has been implemented and lightly tested in the latest commits. Would you like to get a new release on crates.io with it? 😄
Yes please; otherwise I won't be able to add it to zip_next as a dependency.
I've released the changes made so far, including this feature, as v0.7.3. Everything should work fine, but feel free to open a new issue if you experience any problems with the new update 👍
Could you please make DeflateEncoder own the Options rather than borrow them? I'm having trouble creating storage for and passing them with the correct lifetime. All the structs I'm currently using own their members.
Also, it would probably be useful if Options derived Copy.