Standardize message packing
doodles526 opened this issue · comments
Currently wire messages are JSON and release messages are MessagePack. I think it would be nice to standardize our messaging. Either switching to MessagePack for everything, or ProtoBuf would be my personal suggestion.
Biggest pros I see to MessagePack is it's simple, and available in many languages if WH gets a community of folks who want to build out clients in languages other than protobuf supported languages.
With ProtoBuf, the static typing and ability to have back-compatible messages seem to be the biggest draw.
From what I gather perf differences should have relatively little impact since we see a relatively little amount of these messages in comparison to through-traffic.
Personally, I don't see much use in having clients in other languages(maybe c for better perf, but it supports proto bufs anyways). And I think having static typing/back compat could be really nice for expanding features in the future without breaking outdated clients. So my vote here is for full protobuf.
Thoughts @mrkurt @mbyczkowski
Another item of interest in my exploration is that protobuf doesn't support very many types. your standard variety of int/uint/float etc. Strings, and that's about it. Given you could have application specific twiddling to hack a solution if we need it. But MessagePack has facilities out of the box for everything protobuf does, plus arrays and maps.
I don't know if we need those extra features or not. Right now the only non-natural representation in WH is the VCSTimestamp which we represent as a time.Time. But again, we could just as easily rep it as a uint64 as seconds post epoch
All in all, the 2 arguments I see for protobufs are:
- Use a template file, so always ensured to generate correctly(don't change the template, code doesn't break) - however, extensions for messagepack exist for templates
- Designed to allow back-compat for messages for clients running old versions. Careful use of messagepack can achieve the same result though
Personally I'm leaning to messagepack
And with the custom extensions as a first-class feature in this implemention I really think running with this repo would be awesome. https://github.com/tinylib/msgp. Fixes our Template issue above - and will also cleanup the use of the reflection package we're currently using in message/wire.go
Closed by #42