tomas / needle

I use node v12.16.1 and the code below throws me an error: unexpected end of file at BrotliDecoder.zlibOnError [as onerror] (zlib.js:180:17)

var needle = require("needle")

needle("get", 'https://www.cloudflare.com/', {compressed: true}).then((r) => {
    console.log(r.headers)
}).catch((e) => {
    throw e
})

It works if i comment out this line:

needle/lib/needle.js

Line 62 in 0ef49f6

flush: zlib.Z_SYNC_FLUSH,

from:

needle/lib/needle.js

Lines 60 to 64 in 0ef49f6

    
           // Enable Z_SYNC_FLUSH to avoid Z_BUF_ERROR errors (Node PR #2595) 
        
           var zlib_options = { 
        
             flush: zlib.Z_SYNC_FLUSH, 
        
             finishFlush: zlib.Z_SYNC_FLUSH 
        
           }

I couldn't reproduce issues from #202 but the code from above still works on all version mentioned

v5.12.0 (which is the latest v5), v6.9.5 (latest v6), and v7.5.0 (latest v7)

I ran into a similar error as the OP on the current nodejs 12.x:

Error: unexpected end of file
    at BrotliDecoder.zlibOnError [as onerror] (zlib.js:180:17)

The error does not happen on every request.

The zlib_options mentioned above appear to be wrong in case we decompress a brotli stream.

 // Enable Z_SYNC_FLUSH to avoid Z_BUF_ERROR errors (Node PR #2595) 
 var zlib_options = { 
   flush: zlib.Z_SYNC_FLUSH, 
   finishFlush: zlib.Z_SYNC_FLUSH 
 }

According to the nodejs docs,

  // For Brotli, the equivalent is zlib.constants.BROTLI_OPERATION_FLUSH.
  { finishFlush: zlib.constants.Z_SYNC_FLUSH },

the values have to be BROTLI_OPERATION_FLUSH instead of Z_SYNC_FLUSH if decompressing a brotli stream.
I tried with both flush and finishFlush set to BROTLI_OPERATION_FLUSH, and I could decompress br encoded responses without an error.

Fixed by #326, available in npm as needle@2.5.2

	// Enable Z_SYNC_FLUSH to avoid Z_BUF_ERROR errors (Node PR #2595)
	var zlib_options = {
	flush: zlib.Z_SYNC_FLUSH,
	finishFlush: zlib.Z_SYNC_FLUSH
	}

zlib error with brotli