Evaluate pure rust compression crates
gwihlidal opened this issue · comments
An evaluation should be done for using pure rust compression crates, such as miniz_oxide.
The current implementation uses miniz_oxide_c_api, which aside from being an unsafe cc crate, it has some very weird nuances like the compression bounds having different sizes between Windows and Linux/OSX.
Current workaround hack in demo.rs:
// GW-TODO: Wow, on Windows the bound type is u32, and on OSX the bound type is u64 (fix me)
#[cfg(windows)]
type BoundsType = u32;
#[cfg(not(windows))]
type BoundsType = u64;
let compress_bound = miniz_oxide_c_api::mz_compressBound(input_size as BoundsType);
FWIW the reason why miniz is used in the demo program in C++ is:
- vertex/index compressors are designed to compress data quickly and meaningfully but still leave the output compressible by a slower general purpose compressor such as zlib or zstd
- miniz was the smallest minimal dependency library that I could include on the C side
If there's some idiomatic Rust compression crate that uses deflate or zstd or equivalent, it would definitely make sense to switch to that here. The library itself doesn't use miniz anyway, this is just an example showing the extra gains you can get by applying a general purpose compressor to the encoded output.
Yup! I want to get the rust version as close as possible to the c/c++ version initially, but then want to improve (in the rust sense) what makes sense.