xerial / snappy-java

Snappy compressor/decompressor for Java

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

BitShuffle#shuffle does not allow to pass arbitrary type sizes (even if the underlying library does support that)

foosmate opened this issue · comments

For data encoding, sometimes integers with intermediate sizes (e.g. of 3 or 5 bytes) are used (e.g. to reduce bandwith). These arrays currently cannot be passed to the external BitShuffle library with snappy-java, because in snappy-java the typesize is predefined as constant. Since the external library actually supports arbitrary type sizes this is an unnecessary restriction.

How to fix:

Would be great to have the following methods:

public static int shuffle(ByteBuffer input, int typeSize, ByteBuffer shuffled) throws IOException {
    ...
}

public static int unshuffle(ByteBuffer shuffled, int typeSize, ByteBuffer output) throws IOException {
    ...
}

Related #443