image/jpeg: Decode hangs

[Dmitry Vyukov]

Run the following program on the following input:

package main

import (
    "bytes"
    "image/jpeg"
    "io/ioutil"
    "os"
)

func main() {
    data, _ := ioutil.ReadFile(os.Args[1])
    img, err := jpeg.Decode(bytes.NewReader(data))
    if err != nil {
        return
    }
    var w bytes.Buffer
    err = jpeg.Encode(&w, img, nil)
    if err != nil {
        panic(err)
    }
}

https://drive.google.com/file/d/0B20Uwp8Hs1oCTzlVWFBmZ2lHS0k/view?usp=sharing

The program hangs (waited for several minutes, while file size is 502 bytes).

Abort stack:

SIGABRT: abort
PC=0x450c5d m=0

goroutine 1 [running]:
image/jpeg.(*decoder).readByte(0xc20806c000, 0xc208047ad9, 0x0, 0x0)
    /ssd/src/go10/src/image/jpeg/reader.go:188 +0x5d fp=0xc2080479b8 sp=0xc208047998
image/jpeg.(*decoder).readByteStuffedByte(0xc20806c000, 0x4a94d9, 0x0, 0x0)
    /ssd/src/go10/src/image/jpeg/reader.go:220 +0x167 fp=0xc2080479f8 sp=0xc2080479b8
image/jpeg.(*decoder).ensureNBits(0xc20806c000, 0x8, 0x0, 0x0)
    /ssd/src/go10/src/image/jpeg/huffman.go:50 +0x34 fp=0xc208047a48 sp=0xc2080479f8
image/jpeg.(*decoder).decodeHuffman(0xc20806c000, 0xc20806e060, 0x2, 0x0, 0x0)
    /ssd/src/go10/src/image/jpeg/huffman.go:183 +0xfb fp=0xc208047ae8 sp=0xc208047a48
image/jpeg.(*decoder).processSOS(0xc20806c000, 0x6, 0x0, 0x0)
    /ssd/src/go10/src/image/jpeg/scan.go:253 +0x1a33 fp=0xc208047de8 sp=0xc208047ae8
image/jpeg.(*decoder).decode(0xc20806c000, 0x7fe7cbfa51c0, 0xc208016450, 0x573300, 0x0, 0x0, 0x0, 0x0)
    /ssd/src/go10/src/image/jpeg/reader.go:622 +0xa4e fp=0xc208047e88 sp=0xc208047de8
image/jpeg.Decode(0x7fe7cbfa51c0, 0xc208016450, 0x0, 0x0, 0x0, 0x0)
    /ssd/src/go10/src/image/jpeg/reader.go:765 +0x69 fp=0xc208047ed0 sp=0xc208047e88
main.main()
    /tmp/jpeg.go:12 +0x132 fp=0xc208047f90 sp=0xc208047ed0

Perf profile:

 13.59%  jpeg  jpeg               [.] image/jpeg.(*decoder).decodeHuffman                                                                                 10.97%  jpeg  jpeg               [.] image/jpeg.(*decoder).processSOS                                                            
 10.60%  jpeg  jpeg               [.] runtime.mallocgc                                                                                                     5.12%  jpeg  jpeg               [.] image/jpeg.(*decoder).receiveExtend                                                           
  4.73%  jpeg  jpeg               [.] image/jpeg.(*decoder).ensureNBits                                                                                    4.44%  jpeg  jpeg               [.] runtime.memmove                                                                                                      4.31%  jpeg  jpeg               [.] image/jpeg.idct 

The code seems to be looping in processSOS.

My repository is on commit 8ac129e.

[Nigel Tao]

Technically, it's not an infinite loop. The actual image is 38655 wide and 16384 high. Your program will actually spit out an error if you wait long enough. It only took my computer a minute or so.

There's still a bug here, that we don't seem to notice that we're not making progress during the Huffman decode, but I don't think it's a hang. More investigation coming...

As for whether we should not try to decode 'unreasonably large images', that's issue #5050 to discuss what 'unreasonable' means and how it's expressed in API.

[Dmitry Vyukov]

Input is just few bytes. Is it OK for such a small input to take so much time and memory? Looks like a zip bomb. I.e. I can force your server to spend arbitrary amount of CPU by sending just few bytes. Or it is OK for this format because of compression (e.g. you encode a huge black image)?

[Nigel Tao]

As I said, not making progress during the Huffman decode is still a bug, and I'm still investigating. All I'm saying is that it's not an infinite loop bug.