Error: Data read, but end of buffer not reached 64

[Sayan J. Das]

Hi, I'm facing this error while trying to unpack serialized data received from a server. The server packs the data with record structures enabled, precisely through this Packr:

const clientPackr = new Packr({
    structures: [['m', 'e'], ['action', 'share']]
});

On the client side, the Packr is also initialized with the exact same structures. I would expect it to unpack the received data successfully with pre-knowing the structures, but it doesn't. It seems to be only able to do it on the server side through the same clientPackr.unpack (I think it has something to do with the offset property not automatically set in client side).
I would appreciate your help, thanks.

[Kris Zyp]

Yes, this should definitely work. Here is what I have tried to reproduce the issue:

		const packr = new Packr({
			structures: [['m', 'e'], ['action', 'share']]
		});
		const packr2 = new Packr({
			structures: [['m', 'e'], ['action', 'share']]
		});
		let packed = packr.pack([{m: 1, e: 2}, {action: 3, share: 4}]);
		// also tried directly decoding this without the first Packr instance: packed = new Uint8Array([0x92, 0x40, 0x01, 0x02, 0x41, 0x03, 0x04]);
		console.log(packr2.unpack(packed));

And this seems to work for me on both node.js and browser. Do you have any other suggestions on how to reproduce this, or am I misunderstanding what you are trying to do?

[Sayan J. Das]

Ah, I was wrong about the issue being only unpack-able from the same Packr instance; Your code ran fine on the server.
The problem seems to be related to structures:
All the objects that I pack are of this format:

{
    m: <Nested Object>
    e: <String>
}

Here, the hierarchy of <Nested Object> is unknown to me and will change frequently, but the structure m & e will always be constant. But this code fails:

const packr = new Packr({
    structures: [['m', 'e']]
});
const packr2 = new Packr({
    structures: [['m', 'e']]
});
let packed = packr.pack({m: {a: 'b', c: 'd'}, e: 2});
// Error: Data read, but end of buffer not reached {"m":65,"e":"b"}
console.log(packr2.unpack(packed));

This succeeds if I pack and unpack from the same Packr instance, but in my case they'll be different instances and I don't know the entire nested structure of the object.
I understand it's because there are fields a and c nested within m, but how can I configure msgpackr to only use the first level properties as the struct only, and the value can be nested objects can be anything, without failing?

[Kris Zyp]

Ah, I see, this means you are using a combination of "shared" structures (known ahead of time on both sides), and "local" structures (communicated in the data). You can configure this by doing:

let structures = [['m', 'e']];
const packr = new Packr({
    structures,
    maxSharedStructures: structures.length, // indicate no more shared structures
});
let packed = packr.pack({m: {a: 'b', c: 'd'}, e: 2}); // now packed should be readable by the client code that has base shared structures

[Sayan J. Das]

That did it, thank you so much!