ArrayIndexOutOfBoundsException when using large set of keys with CHDMinimalPerfectHashFunction

Question

ArrayIndexOutOfBoundsException when using large set of keys with CHDMinimalPerfectHashFunction

binarybit10 opened this issue 4 years ago · comments

Hi,

I'm using CHDMinimalPerfectHashFunction<byte[]> for about 1_00_000 keys. I see ArrayIndexOutOfBoundsException , while the same works if I use 5_00_00 keys.

version: sux4j:5.1.0
jdk 8

Stacktrace
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at it.unimi.dsi.bits.LongArrayBitVector.getLong(LongArrayBitVector.java:488)
at it.unimi.dsi.sux4j.util.EliasFanoLongBigList.getLong(EliasFanoLongBigList.java:242)
at it.unimi.dsi.sux4j.mph.CHDMinimalPerfectHashFunction.getLong(CHDMinimalPerfectHashFunction.java:589)

I tried running GOVMinimalPerfectHashFunction, it takes several minutes to complete for the 5_00_00 keys, so I switched to CHDMinimalPerfectHashFunction which completed in under 5 seconds, but this fails for large keys.

Sebastiano Vigna · Answer 1 · Thu Aug 27 2020 14:43:27 GMT+0800 (China Standard Time)

Please provide a replicable unit test.

binarybit10 · Answer 2 · Sun Aug 30 2020 04:28:57 GMT+0800 (China Standard Time)

Apologies, I think the issue was due to something else. I tested with 10 million keys and works brilliantly!