SIGSEGV in badactor.Director.maintenance()

[Max Hilbrunner]

I'm seeing the following crash after my application is running for some time.

>panic: runtime error: invalid memory address or nil pointer dereference
>[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x8c3fc2]
>
>goroutine 9 [running]:
[PATH]/vendor/github.com/jaredfolkins/badactor.(*Director).maintenance(0xc420283710, 0xc436b3f2b0, 0xc)
       [PATH]/vendor/github.com/jaredfolkins/badactor/director.go:181 +0x72
[PATH]/vendor/github.com/jaredfolkins/badactor.(*Director).lMaintenance(0xc420283710)
        [PATH]/vendor/github.com/jaredfolkins/badactor/director.go:38 +0xad
[PATH]/vendor/github.com/jaredfolkins/badactor.(*Studio).StartReaper.func1(0xc4201f26c0, 0xc4202448a0)
        [PATH]/vendor/github.com/jaredfolkins/badactor/studio.go:118 +0x132
created by [PATH]/vendor/github.com/jaredfolkins/badactor.(*Studio).StartReaper
        [PATH]/vendor/github.com/jaredfolkins/badactor/studio.go:127 +0x57

[Jared Folkins]

What version of Go are you using?

Have you ran your application with the -race flag?

Is this the entirety of the trace?

The trace seems light. And by reading that section with BadActor, BadActor simply appears to be running in its own goroutine like it is supposed to. There is nothing pointing to a panic from BadActor that I can read.

Thoughts?

[Jared Folkins]

goroutine 9 [running]:

That states that BadActor is running. No IO Wait, no lock. So I am confused is all. But if you can get me more data I'll try to help.

Laterz

[Max Hilbrunner]

go version go1.9 windows/amd64

Cross-compiled the app to run on a Linux server.
Thats the whole crash log, unfortunately :(

I'll see if I can debug this some more tomorrow!

[Max Hilbrunner]

Can't reproduce this on any other machine, will keep debugging & reopen this if I get anything more concrete.

[Jared Folkins]

Bummer. I'd love to know if you find something interesting. Take care.

[Max Hilbrunner]

Just to give you peace of mind, it indeed seems like it was caused by a race condition unrelated to badactor. Found and fixed it and now the crash no longer happens. Thanks for pointing out -race :)

[Jared Folkins]

THANK YOU! Just coming back and stating this means a lot. And yes -race is required in Go 💯

[Andrew LeFevre]

I actually just came across this issue, and think I found the cause: when actors are deleted by the reaper in Director.maintenance(), the Director's index field is not updated accordingly. So the next call to Director.lMaintenance() will then call Director.maintenance() on the now deleted actor, causing a panic.

I think I fixed the issue by deleting the actor from the index in Director.lMaintenance() if the actor was deleted, keeping the actor map and actor indexes in sync:

func (d *Director) lMaintenance() {
	d.Lock()
	defer d.Unlock()
	for e := d.index.Front(); e != nil; e = e.Next() {
		an := e.Value.(*Actor).name
		d.maintenance(an)
		if !d.actorExists(an) {
			d.index.Remove(e)
		}
	}
}

Though I'm not sure if there's a reason you don't keep the index in sync with the actor map, as I can see that Director.deleteOldest() removes old actors, so I will wait to submit a PR until you can verify that this would be a sufficient solution @jaredfolkins.

Edit: I also ran my server with -race, and confirmed that this bug is not born of a race condition of my code.

[Andrew LeFevre]

Sorry about that, I just realized you fixed this in c3185d0. I am using go modules, and because you tagged a 1.0 release before that commit, go geting badactor with recent versions of Go pulls a broken version of badactor by default. I think making a new release should fix this.

Also, I can make a PR for adding modules support if you want.

[Jared Folkins]

Nice! Make the PR and I'll pull it in and cut a release. Thanks!