BitShuffle#shuffle does not allow to pass arbitrary type sizes (even if the underlying library does support that)
foosmate opened this issue · comments
foosmate commented
For data encoding, sometimes integers with intermediate sizes (e.g. of 3 or 5 bytes) are used (e.g. to reduce bandwith). These arrays currently cannot be passed to the external BitShuffle library with snappy-java, because in snappy-java the typesize is predefined as constant. Since the external library actually supports arbitrary type sizes this is an unnecessary restriction.
How to fix:
Would be great to have the following methods:
public static int shuffle(ByteBuffer input, int typeSize, ByteBuffer shuffled) throws IOException {
...
}
public static int unshuffle(ByteBuffer shuffled, int typeSize, ByteBuffer output) throws IOException {
...
}
Taro L. Saito commented
Related #443