nodeca / pako

high speed zlib port to javascript, works in browser & node.js

Home Page:http://nodeca.github.io/pako/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Pako returns undefined

kalvenschraut opened this issue · comments

I made a reproduction repo here https://github.com/RtVision/pako-undefined-return

System information
image

From what debugging have done it seems the status in in Inflate.push is never being set to Z_STEAM_END so it ends up hitting this line and ending the execution. Since the status in never set to Z_STEAM_END and there exists room in the output buffer it never gets into here to be able to add the data to the chunks array. Also since onEnd called but the function returns true the code tries to access this.result which doesn't exist.

I did try changing the check to be the below and then call this.onEnd here in the if before it breaks and that fixed this issue specifically. I did try running your tests and I broke most of them with that change so I am not sure what the actual solution to this issue should be. As far as I can tell it stops trying to inflate the input if it has no more left in the in buffer, but never adds that to the chunks array and call end to join that into the result variable. This is only a problem if the status ends up being Z_OK and not Z_STREAM_END so some reason the zlib library isn't saying the stream ended when there is nothing more to process?

if (strm.avail_in === 0 || status === Z_STREAM_END) {
   // rest of the code
}
``

zlib itself is located in /lib/zlib. Users are expected to create their own wrappers, for concrete cases. Our wrapper is universal, but can not cover 100% of anything people can imagine. So, probably you need your own.

The data itself was actually a mix of two encrypted texts that got mixed up in a beta environment by the server code, I guess only real change I would like to see is instead of returning undefined can pako throw an error in this situation?

Known bug, see #231 .

You can reproduce it like this:

let sample = zlib.deflateSync(Buffer.from('foobar'));
sample = sample.slice(0, -1);
console.log(pako.inflate(sample));

Here's a fix:

diff --git a/lib/inflate.js b/lib/inflate.js
index d95bf14..9047973 100644
--- a/lib/inflate.js
+++ b/lib/inflate.js
@@ -383,6 +383,10 @@ function inflate(input, options) {
   // That will never happens, if you don't cheat with options :)
   if (inflator.err) throw inflator.msg || msg[inflator.err];

+  if (!inflator.ended) {
+    throw new Error('unexpected end of file');
+  }
+
   return inflator.result;
 }

And here's a custom wrapper you can use instead:

const pako = require('pako');

function pako_inflate(input, options) {
  const inflator = new pako.Inflate(options);

  inflator.push(input);

  if (inflator.err) throw inflator.msg || msg[inflator.err];

  if (!inflator.ended) {
    throw new Error('unexpected end of file');
  }

  return inflator.result;
}

console.log(pako_inflate(Buffer.from([ 0x78, 0x9c, 0x73, 0x74, 0x02, 0x00, 0x00, 0xc6, 0x00, 0x84 ])));
console.log(pako_inflate(Buffer.from([ 0x78, 0x9c, 0x73, 0x74, 0x02, 0x00, 0x00, 0xc6 ])));

my case is golang server, like this, change defer, because of close will append gzip end flag, like:

le.PutUint32(z.buf[:4], z.digest)
le.PutUint32(z.buf[4:8], z.size)

func (c GzipCompressor) Compress(i []byte) (o []byte, err error) {
var buf bytes.Buffer
w := gzip.NewWriter(&buf)
defer w.Close()

_, err = w.Write(i)
if err != nil {
	return
}

err = w.Flush()
if err != nil {
	return
}
    w.Close()
return

}

fix this "unexpected end of file", must delete "defer w.Close()"
like this:

func (c GzipCompressor) Compress(i []byte) (o []byte, err error) {
var buf bytes.Buffer
w := gzip.NewWriter(&buf)
// defer w.Close()

_, err = w.Write(i)
if err != nil {
             w.Close()
	return
}

err = w.Flush()
if err != nil {
             w.Close()
	return
}
    w.Close()
o = buf.Bytes()
return

}