Giters

metamx / bytebuffer-collections

ByteBuffer collection classes for java and jvm-based languages.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Wrong result with RoaringBitmapFactory

binlijin opened this issue · comments

commented

In a case i find Roaring and Concise return different results.

import org.junit.Test;
import org.roaringbitmap.IntIterator;
public class BitMapTest
{
  @Test
  public void test() throws Exception
  {
    System.out.println("Test ConciseBitmapFactory");
    printBitMap(new ConciseBitmapFactory());
    System.out.println("Test RoaringBitmapFactory");
    printBitMap(new RoaringBitmapFactory());
  }

  private void printBitMap(BitmapFactory bitmapFactory) throws Exception
  {
    int numRow = 5104234;
    ImmutableBitmap bitmap = bitmapFactory.complement(bitmapFactory.makeEmptyImmutableBitmap(), numRow);
    ImmutableBitmap notBitmap = bitmapFactory.complement(bitmap,numRow);
    System.out.println("size " + notBitmap.size());
    IntIterator intIter = notBitmap.iterator();
    while(intIter.hasNext()) {
      System.out.println(intIter.next());
    }
  }
}

Output is：

Test ConciseBitmapFactory
size 0
Test RoaringBitmapFactory
size 78
65535
131071
196607
262143
327679
393215
458751
524287
589823
655359
720895
786431
851967
917503
983039
1048575
1114111
1179647
1245183
1310719
1376255
1441791
1507327
1572863
1638399
1703935
1769471
1835007
1900543
1966079
2031615
2097151
2162687
2228223
2293759
2359295
2424831
2490367
2555903
2621439
2686975
2752511
2818047
2883583
2949119
3014655
3080191
3145727
3211263
3276799
3342335
3407871
3473407
3538943
3604479
3670015
3735551
3801087
3866623
3932159
3997695
4063231
4128767
4194303
4259839
4325375
4390911
4456447
4521983
4587519
4653055
4718591
4784127
4849663
4915199
4980735
5046271
5104233
commented

Thanks. I'll investigate in the near future (though not right now).

commented

@binlijin
I copied the test you added and run it. But I did not reproduce the issue: both bitmaps are empty.
I'm on the master branch with Roaring 0.5.16. I've also tested it with 0.5.17 and 0.5.18.
Let me know if I missed something.

commented

Thanks @gssiyankai.

To clarify the issue, I have proposed a PR that should reproduce the problem. It works for me, but being paranoid, I prefer to propose a new unit test rather than argue that "it works for me".

commented

Thank you @lemire! @binlijin can you confirm that this test would have caught the bug you see?

commented

@binlijin there have been some updates to Roaring which druid has been trying to keep up with, so can you please confirm which roaring version you are seeing this fail against?

commented

Note that the pom.xml file contains the following...

        <dependency>
            <groupId>org.roaringbitmap</groupId>
            <artifactId>RoaringBitmap</artifactId>
            <version>0.5.16</version>
        </dependency>

Presumably if the bug was reported against the current version of bytebuffer-collections, one should assume that the Roaring version that is concerned is 0.5.16 as the pom.xml specifies this version.

@gssiyankai tested with more recent versions of Roaring and could not verify the bug.

Could it be that the bug was reported against an outdated version of bytebuffer-collections?

In any case, the wisest thing is probably just to add in a unit test.

commented

Test fails on 0.5.15 and earlier, passes on 0.5.16 and later. Closing as resolved.

commented

Thanks all for this, i will update the Roaring version.