panic on Writer.Close, runtime error: index out of range

Question

panic on Writer.Close, runtime error: index out of range

toffaletti opened this issue 8 years ago · comments

Unfortunately I don't have a test to reproduce this, but over hundreds of thousands of calls we started seeing a few of these panics:

runtime error: index out of range
goroutine 148112966 [running]:
net/http.func·011()
        /usr/lib/go/src/net/http/server.go:1130 +0xbb
github.com/klauspost/compress/flate.(*compressor).deflateNoSkip(0xc20e431600)
        _vendor/src/github.com/klauspost/compress/flate/deflate.go:594 +0xc68
github.com/klauspost/compress/flate.(*compressor).close(0xc20e431600, 0x0, 0x0)
        _vendor/src/github.com/klauspost/compress/flate/deflate.go:773 +0x49
github.com/klauspost/compress/flate.(*Writer).Close(0xc20e431600, 0x0, 0x0)
        _vendor/src/github.com/klauspost/compress/flate/deflate.go:854

We never saw this before I updated our snapshot on Nov 2nd. The previous update was Sep 8th.

Klaus Post · Answer 1 · Tue Nov 17 2015 16:00:14 GMT+0800 (China Standard Time)

Thanks for the report - I will investigate at once!

Klaus Post · Answer 2 · Tue Nov 17 2015 16:33:32 GMT+0800 (China Standard Time)

Can you confirm that deflate.go:594 is also https://github.com/klauspost/compress/blob/master/flate/deflate.go#L594 ?

Jason Toffaletti · Answer 3 · Tue Nov 17 2015 17:17:31 GMT+0800 (China Standard Time)

Yes, and unfortunately there are two indexes used on that line. I'm not sure which is out of range.

tAndrii · Answer 4 · Tue Nov 17 2015 18:04:25 GMT+0800 (China Standard Time)

@maruel has found this bug too https://code.google.com/p/chromium/issues/detail?id=552697#c4 .
The unittest in our code is a bit large but can reproduce this.

Klaus Post · Answer 5 · Tue Nov 17 2015 22:08:14 GMT+0800 (China Standard Time)

From the other report it is clear that it is d.tokens.n that somehow gets too high. I am trying to recreate a scenario where that happens. Unfortunately my 500k files "webserver" test does not reproduce it.

I am of course continuing work on this.

Klaus Post · Answer 6 · Tue Nov 17 2015 22:16:12 GMT+0800 (China Standard Time)

OK. Are you checking returned error values?

If you call Write after a write has already failed (in case of the chrominum bug) I can see this error occurring.

Close after a failed write will do the same.

Both are of course bugs, and will be fixed - I just want to be sure I haven't overlooked anything.

M-A · Answer 7 · Tue Nov 17 2015 22:40:56 GMT+0800 (China Standard Time)

We reproduce in this case:

Compress, writing to an io.Pipe().
Using io.Copy() to write the input data to the compressor.
Length of data is >= 64kb + 1.
The read pipe from io.Pipe() is closed in the middle of the stream.

I was planning try to write a regression test before even bothering you about this but too late, eh. :)

M-A · Answer 8 · Tue Nov 17 2015 22:54:15 GMT+0800 (China Standard Time)

@toffaletti (shameless) plug for panicparse to make your stack traces shorter.

Klaus Post · Answer 9 · Wed Nov 18 2015 02:14:52 GMT+0800 (China Standard Time)

Thanks for helping identify this issue. I have merged the fix after testing.

Jason Toffaletti · Answer 10 · Wed Nov 18 2015 05:39:10 GMT+0800 (China Standard Time)

In our code I don't control whether Write is called again after an error because our write is essentially json.NewEncoder(gzipWriter).Encode(). I will open a new issue if this fix didn't clear up the panic. Thanks so much for the prompt response despite so few details.