Race detector warnings

[Lukáš Němec]

Hi, I'm getting race detector warnings when accessing a hashmap from multiple goroutines:

WARNING: DATA RACE
Read at 0x00c001b7dbd0 by goroutine 64:
  sync/atomic.LoadInt64()
      /usr/local/Cellar/go/1.11.1/libexec/src/runtime/race_amd64.s:211 +0xb
  .../vendor/github.com/cornelk/hashmap.(*ListElement).Next()
      /.../vendor/github.com/cornelk/hashmap/listelement.go:25 +0x3e
  .../vendor/github.com/cornelk/hashmap.(*HashMap).fillIndexItems()
      /.../vendor/github.com/cornelk/hashmap/hashmap.go:373 +0x77
  .../vendor/github.com/cornelk/hashmap.(*HashMap).grow()
      /.../vendor/github.com/cornelk/hashmap/hashmap.go:339 +0x292

Previous write at 0x00c001b7dbd0 by main goroutine:
  .../vendor/github.com/cornelk/hashmap.(*List).insertAt()
      /.../vendor/github.com/cornelk/hashmap/list.go:128 +0x234
  .../vendor/github.com/cornelk/hashmap.(*List).AddOrUpdate()
      /.../vendor/github.com/cornelk/hashmap/list.go:65 +0xcf
  .../vendor/github.com/cornelk/hashmap.(*HashMap).insertListElement()
      /.../vendor/github.com/cornelk/hashmap/hashmap.go:237 +0xc8
  core-next/vendor/github.com/cornelk/hashmap.(*HashMap).Set()
      /.../vendor/github.com/cornelk/hashmap/hashmap.go:211 +0x172
      /.../main.go:45 +0xd3

But the strange part is, this is code running in a single goroutine - just iterating over set of data and calling .Set() on the hashmap.Hashmap.

Example:

var dataMap hashmap.HashMap
for _, data := range listOfData {
	var innerMap hashmap.HashMap
	for _, innerData := range data.InnerList {
		innerMap.Set(innerData, struct{}{})
	}
	dataMap.Set(data.ID, innerMap)
}

Any ideas what to do here? Am I using the hashmap.Hashmap wrong?

[Cornel]

@lunemec are you using the current master branch that included a race fix? if so, can you create a unit test to replicate that behavior?

[Lukáš Němec]

Well, I only noticed when creating kinda huge hashmaps. I tested this with my tests, but it did not trigger. Only when I load real data 70MB in heap, then it triggers this. My current revision is this: b893f49

[Lukáš Němec]

I'll try to force the newest commit with the fix - dep automatically takes released version.

[Lukáš Němec]

Nope, the current master does not work either. I'll write a test case for this.

[Lukáš Němec]

Well, I kinda created this problem:

func TestHashMap_SetWithContainer(t *testing.T) {
	type Container struct {
		Map HashMap
	}
	type mapItem struct {
		InnerData HashMap
	}
	var container Container
	for i := 0; i < 10; i++ {
		var innerMap HashMap
		for z := 0; z < 10; z++ {
			innerMap.Set(z, struct{}{})
		}
		container.Map.Set(i, mapItem{InnerData: innerMap})
	}
}

[Lukáš Němec]

Well at least I can now fix it by simply moving the HashMap outside of the struct, but I have some metadata packed with it, so that would not be ideal.

[Lukáš Němec]

@cornelk do you think a fix is possible? Or should I work around this?

[Cornel]

this is the fixed unit test without a race condition:

func TestHashMap_SetWithContainer(t *testing.T) {
	type Container struct {
		Map *HashMap
	}
	type mapItem struct {
		InnerData *HashMap
	}
	container := Container{Map: &HashMap{}}
	for i := 0; i < 10; i++ {
		innerMap := &HashMap{}
		for z := 0; z < 10; z++ {
			innerMap.Set(z, struct{}{})
		}
		container.Map.Set(i, mapItem{InnerData: innerMap})
	}
}

[Lukáš Němec]

Okay, so it was misunderstanding of usage on my part. Could you maybe update readme/doc with a example of this?

[Cornel]

The readme and tests already contain code that shows how to initiate the map: m := &HashMap{}