Getting "Error: Data read, but end of buffer not reached"

[Ahmad Salama]

Hey, I am getting "Error: Data read, but end of buffer not reached" and I don't know why.
Can you help, please?

Code:
const msgpack = require('msgpackr')
let unpacked = msgpack.unpack(Buffer.from("4234235rfasdfasdf", 'base64'));

Thanks

[Kris Zyp]

This doesn't appear to be valid MessagePack document. The first three bytes are each individual integers (starting with -29) followed by a string that declares a length over 2GB, which doesn't make sense. Where did this data come from?

[Ahmad Salama]

I am migrating from msgpack-node (https://github.com/msgpack/msgpack-node) to msgpackr.
We had this string as test to msgpack unpack function and it returned -29 with the old msgpack.
Can you explain what changed, please?

Thanks for the fast response.

[Ahmad Salama]

And do you have any tips on migrating between the two repos?

[Ahmad Salama]

I see in our code that this string ("4234235rfasdfasdf") is random (not packed), so it's not relevant.
How to determine if a string is legal or not?

[Gabriel]

You can unpack with msgpack-node and pack the result with msgpackr to get the new test string.

[Kris Zyp]

The first byte, 0xe3 is indeed the MessagePack encoding of -29. However it is followed by 11 more random bytes and for decoding that expects a single decoded value, this trailing data is considered an error; you should haven't trailing data, just the MessagePack data itself. So this should work:

msgpackr.unpack(Buffer.from("42", 'base64'));
// or for a complete byte
msgpackr.unpack(Buffer.from([0xe3]));

If you want to process sequential MessagePack values (where one encoded value may immediately be followed by an other encoded value in the same buffer/stream), you can use unpackMultiple:
msgpackr.unpackMultiple(data);
This could unpack the first few bytes (that are integers), but you would eventually run into the invalid string declaration. You could observe the first few values before it throws an error though:

msgpackr.unpackMultiple(Buffer.from("4234235rfasdfasdf", 'base64'), (item) => console.log(item));

How to determine if a string is legal or not?

If it is not legal, it will throw an error, like "Error: Data read, but end of buffer not reached"! :)

And do you have any tips on migrating between the two repos?

You should just be able to directly replace it (that's kind of the point of using a specification like MessagePack is that libraries should be interchangeable), but of course where there's... let's say "bugs", in msgpack-node, than yeah, that might not exactly replicate in msgpackr :).

[Ahmad Salama]

Awesome, thank you very much for the declarations :D