Faleij / json-stream-stringify

JSON.Stringify as a readable stream

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Throws: Uncaught RangeError: Invalid string length

framp opened this issue · comments

Thanks for the library!

I tried using this library to bypass this JSON.stringify bug: nodejs/help#1061
But the library is throwing the same error.

I don't have time to debug it but I thought I'd flag it. I wonder if maybe this library is using JSON.stringify at some point and it's not navigating recursively into the object?

How to reproduce (it takes a while, maybe you'll be able to get it with a smaller array):

> JSON.stringify( Array.from({length: 10000000}).fill({a: 'streaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringify'}))
Uncaught RangeError: Invalid string length
    at JSON.stringify (<anonymous>)
> new JSONStreamStringify(Array.from({length: 10000000}).fill({a: 'streaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringify'})).then(console.log)
Promise { <pending> }
Uncaught RangeError: Invalid string length

then is not a function of JSONStreamStringify, you have some library that makes readable streams have a then method and that method that just concatenates all strings that the stream outputs and that will eventually exceed buffer.constants.MAX_STRING_LENGTH. Use the streaming nature of this library to bypass the MAX_STRING_LENGTH and pipe the data to its destination or write it to a buffer.

const buffers = [];
let len = 0;
new JsonStreamStringify(Array.from({length: 10000000}).fill({a: 'streaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringify'}))
    .on('data', data => {
        buffers.push(data);
        len += data.length;
    })
    .on('end', () => {
        const wholeBuffer = Buffer.concat(buffers, len);
        console.log({
            numberOfBuffers: buffers.length,
            bytes: len,
            wholeBufferBytes: wholeBuffer.length,
            startOfWholeBuffer: wholeBuffer.slice(0, 60).toString('UTF8') + '...',
            endOfWholeBuffer: '...' + wholeBuffer.slice(-60).toString('UTF8'),
        });
    });