Throws: Uncaught RangeError: Invalid string length
framp opened this issue · comments
Thanks for the library!
I tried using this library to bypass this JSON.stringify bug: nodejs/help#1061
But the library is throwing the same error.
I don't have time to debug it but I thought I'd flag it. I wonder if maybe this library is using JSON.stringify at some point and it's not navigating recursively into the object?
How to reproduce (it takes a while, maybe you'll be able to get it with a smaller array):
> JSON.stringify( Array.from({length: 10000000}).fill({a: 'streaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringify'}))
Uncaught RangeError: Invalid string length
at JSON.stringify (<anonymous>)
> new JSONStreamStringify(Array.from({length: 10000000}).fill({a: 'streaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringify'})).then(console.log)
Promise { <pending> }
Uncaught RangeError: Invalid string length
then
is not a function of JSONStreamStringify, you have some library that makes readable streams have a then
method and that method that just concatenates all strings that the stream outputs and that will eventually exceed buffer.constants.MAX_STRING_LENGTH. Use the streaming nature of this library to bypass the MAX_STRING_LENGTH and pipe the data to its destination or write it to a buffer.
const buffers = [];
let len = 0;
new JsonStreamStringify(Array.from({length: 10000000}).fill({a: 'streaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringifystreaming-json-stringify'}))
.on('data', data => {
buffers.push(data);
len += data.length;
})
.on('end', () => {
const wholeBuffer = Buffer.concat(buffers, len);
console.log({
numberOfBuffers: buffers.length,
bytes: len,
wholeBufferBytes: wholeBuffer.length,
startOfWholeBuffer: wholeBuffer.slice(0, 60).toString('UTF8') + '...',
endOfWholeBuffer: '...' + wholeBuffer.slice(-60).toString('UTF8'),
});
});