[question] problem encoding types

[Endel Dreyer]

Hey there,

I'm afraid there are some differences between the serialization of this library and Node's msgpack-lite

Still trying to debug to find where the difference is, I'd appreciate if you can give me any hints!

Here's the differences when trying to encode this object:

{ players: {}, messages: [] }

JavaScript:

[130, 167, 112, 108, 97, 121, 101, 114, 115, 128, 168, 109, 101, 115, 115, 97, 103, 101, 115, 144]

C#

// second and third bytes differs from JavaScript
[130, 217, 7, 112, 108, 97, 121, 101, 114, 115, 128, 217, 8, 109, 101, 115, 115, 97, 103, 101, 115, 144]

Still, in JavaScript, I have the same output when trying to decode both byte arrays:

> msgpack.decode([130, 167, 112, 108, 97, 121, 101, 114, 115, 128, 168, 109, 101, 115, 115, 97, 103, 101, 115, 144])
{ players: {}, messages: [] }

> msgpack.decode([130, 217, 7, 112, 108, 97, 121, 101, 114, 115, 128, 217, 8, 109, 101, 115, 115, 97, 103, 101, 115, 144])
{ players: {}, messages: [] }

My guess is that C# is serializing empty objects as a different type.

This issue looks harmless on the surface but unfortunately, it's preventing me from applying byte patches using FossilDelta due to this byte difference.

Looking forward to hearing your thoughts on this!
Cheers

[Denis Zykov]

C# tokens
[FixMapStart, Str8 ...

JavaScript:
[FixMapStart, FixStrStart...

Looks line msgpack-lite choose better type for encoding this string. I will address this problem soon.
But I think this is not the last place where a binary representation of same data could be different. This format was not designed for data storage and patch applying, it was created for messages passing over network. BSON is better for storage, traversal and patching.

[Endel Dreyer]

Thanks for your swift reply @deniszykov!

I've checked the byte size of that same object ({ players: {}, messages: [] }) and it is quite larger using BSON. 34 bytes using BSON and 20 using msgpack. I'd like to keep it as small as possible. :/

If you can give me some direction where to look at in your C# implementation I'd be glad to help.

Cheers! Have a nice weekend :)

[Denis Zykov]

Hi, I have fixed string serialization issue in commit cfd4017. And submitted Unity asset in store (it would take up to one week to update).

Unfortunately I do not have a serializer in the BSON format for Unity.

[Endel Dreyer]

Thanks a lot @deniszykov! Everything looks fine now!

[Denis Zykov]

I have fixed bug in d72d1cb. It is related to small length strings serialization in MessagePack (rare case with non-latin letters).