Tricky NullPointerException in IntObjectMap.forEachKeyValue

[Andrej Fink]

If you remove key 0 (int; aka EMPTY_KEY) in forEachKeyValue - you will enjoy NullPointerException

@Test public void testBugRemoveNPE (){
  try {
    var m = IntObjectMaps.mutable.of(0, 'a');
    m.forEachKeyValue((k,v)->m.removeKey(k));
    fail();
  } catch (NullPointerException e){
    //assertEquals("Cannot read field \"containsOneKey\" because \"this.sentinelValues\" is null", e.getMessage());
  }
}

Problem is here:
org.eclipse.collections.impl.map.mutable.primitive.IntObjectHashMap#forEachKeyValue

if (this.sentinelValues.containsOneKey)

after removing key 0 aka EMPTY_KEY - sentinelValues can be null

The problem is easy to fix:

if (this.sentinelValues != null && this.sentinelValues.containsZeroKey)
{
  procedure.value(EMPTY_KEY, this.sentinelValues.zeroValue);
}
if (this.sentinelValues != null && this.sentinelValues.containsOneKey)
{
  procedure.value(REMOVED_KEY, this.sentinelValues.oneValue);
}

[Mohammad Rezaei]

While the above is true, I don't believe we support map modifications within forEachKeyValue. remove is "safe" by accident (and not safe in the case of the zero key). put would cause all manner of problems.

Perhaps that needs to be better documented.

Additionally, we could add removeIf to primitive maps (I'm assuming this is how you ran into this, as the code sample to reproduce this is not likely your real code).

[Andrej Fink]

It is absolutely safe (even if by accident). Only this case with key==0 is a problem.

I didn't want to call removeKey in forEachKeyValue in the first place, BUT

keyValuesView iterator doesn't support remove (throws exception) and it was harder to tell how safe to call removeKey from this iterator...

Real code (migrated from JDK Map) looks like this:

  void verify (@Nonnull MutableIntObjectMap<byte[]> m){
    m.forEachKeyValue((ie,b)->{
      int iel = b.length;//actual ie len

      Integer ielMin = IE_LEN_MIN.get(ie);
      if (ielMin != null && iel < ielMin) {
        m.removeKey(ie);
      }

      Integer ielMax = IE_LEN_MAX.get(ie);
      if (ielMax != null && iel > ielMax) {
        m.removeKey(ie);
      }
    });
  }

I use EC only few hours. Probably there is a more idiomatic way to do this.

PS: BUT bug with NPE is easy to fix. So, why keep it lying there ;-) Other primitive key Map implementations have it probably too.

[Mohammad Rezaei]

The above code fits well within a removeIf implementation.

We have two choices here:

1. Document forEachKeyValue as not allowing modifications. This allows us to change the map implementation internals much easier. For example, if we implement auto-shrinking, remove will no longer be safe, even by accident. Additionally, add removeIf. Maybe implement keyValuesView.iterator().remove().
2. Document forEachKeyValue as allowing remove (but not put!) and test all scenarios with this and fix various issues.

Relying on an accident, instead of an intent, is neither good for your code nor for the library.

[Andrej Fink]

I would choose both :-)

1. implement keyValuesView.iterator().remove() - is a very good thing!
2. Document forEachKeyValue as allowing remove (but not put!) and test all scenarios with this and fix various issues. And fix it (no more NPE), obviously!

[Andrej Fink]

I've just checked: this pattern is everywhere

        if (this.sentinelValues != null)
        {
            if (this.sentinelValues.containsZeroKey)
            {
                procedure.value(EMPTY_KEY);
            }
            if (this.sentinelValues.containsOneKey)// a place to hit your head
            {
                procedure.value(REMOVED_KEY);
            }
        }

in forEachKey, forEachValue, etc...
So they all have this exactly same problem.

And it means: it is really hard to remove from primitive Maps...