Protocol structure changes – avoid layering

[Alexander Emelin]

I am thinking about changing a structure of client protocol schema in a way so that we won't have several Raw data layers in messages. Specifically I mean sth like this:

diff --git a/client.proto b/client.proto
index abe5268..53bae89 100644
--- a/client.proto
+++ b/client.proto
@@ -27,12 +27,26 @@ message Command {
   }
   MethodType method = 2;
   bytes params = 3;
+
+  ConnectRequest connect = 4;
+  SubscribeRequest subscribe = 5;
+  UnsubscribeRequest unsubscribe = 6;
+  PublishRequest publish = 7;
+  PresenceRequest presence = 8;
+  PresenceStatsRequest presence_stats = 9;
+  HistoryRequest history = 10;
+  PingRequest ping = 11;
+  SendRequest send = 12;
+  RPCRequest rpc = 13;
+  RefreshRequest refresh = 14;
+  SubRefreshRequest sub_refresh = 15;
 }
 
 message Reply {
   uint32 id = 1;
   Error error = 2;
   bytes result = 3;
+  Push push = 4;
 }
 
 message Push {
@@ -50,6 +64,16 @@ message Push {
   PushType type = 1;
   string channel = 2;
   bytes data = 3;
+
+  Publication pub = 4;
+  Join join = 5;
+  Leave leave = 6;
+  Unsubscribe unsubscribe = 7;
+  Message message = 8;
+  Subscribe subscribe = 9;
+  Connect connect = 10;
+  Disconnect disconnect = 11;
+  Refresh refresh = 12;
 }

So publication in channel will look:

{"push":{"channel":"test","pub":{"data":{"input":"i"}}}}

instead of current:

{"result":{"channel":"test","data":{"data":{"input":"i"}}}}

And similar changes for requests.

Why?

• Better introspection of messages written into connection in OnTransportWrite callback - no need to decode to get contents.
• Removing many complicated pieces currently used to make serialization fast (buffer pools, byte slice pools)
• Constructing messages will be more straighforward
• More readable JSON representation

Performance

Overall the performance should be a bit faster for proposed schema as shown below. But while new structure is faster in terms of serialization speed it involves more allocs on heap since more pointer types escape from function inside Reply object on marshaling. For unmarshaling we benefit in both allocs and speed.

BenchmarkReplyJSONMarshal-12                      	 2218264	       540.1 ns/op	     816 B/op	       3 allocs/op
BenchmarkReplyJSONMarshalNew-12                   	 2512666	       470.6 ns/op	     752 B/op	       4 allocs/op

BenchmarkReplyJSONMarshalParallel-12              	 4736026	       254.9 ns/op	     817 B/op	       3 allocs/op
BenchmarkReplyJSONMarshalParallelNew-12           	 6682945	       176.6 ns/op	     752 B/op	       4 allocs/op

BenchmarkReplyJSONUnmarshal-12                    	 3032642	       395.2 ns/op	     328 B/op	       3 allocs/op
BenchmarkReplyJSONUnmarshalNew-12                 	 4051048	       285.7 ns/op	     304 B/op	       2 allocs/op

BenchmarkReplyJSONUnmarshalParallel-12            	10702466	       113.4 ns/op	     328 B/op	       3 allocs/op
BenchmarkReplyJSONUnmarshalParallelNew-12         	12536810	        86.55 ns/op	     304 B/op	       2 allocs/op

BenchmarkReplyProtobufMarshal-12                  	 4115287	       288.6 ns/op	     752 B/op	       3 allocs/op
BenchmarkReplyProtobufMarshalNew-12               	 4451221	       265.2 ns/op	     720 B/op	       4 allocs/op

BenchmarkReplyProtobufMarshalParallel-12          	 7056967	       160.2 ns/op	     752 B/op	       3 allocs/op
BenchmarkReplyProtobufMarshalParallelNew-12       	10134794	       119.0 ns/op	     720 B/op	       4 allocs/op

BenchmarkReplyProtobufUnmarshal-12               	 5471642	       207.9 ns/op	     320 B/op	       4 allocs/op
BenchmarkReplyProtobufUnmarshalNew-12            	 6323160	       182.9 ns/op	     309 B/op	       3 allocs/op

BenchmarkReplyProtobufUnmarshalParallel-12       	16424490	        69.96 ns/op	     320 B/op	       4 allocs/op
BenchmarkReplyProtobufUnmarshalParallelNew-12    	18111056	        63.67 ns/op	     309 B/op	       3 allocs/op

Also benchmark in centrifuge-go repo shows CPU reduction, the bigger payload size the bigger CPU drop.

Not a one of?

There is a problem with easyjson message generation – so we should most probably go without oneof.

Backwards compatibility

I don't want to break existing clients. Client connectors most probably need to support both protocol formats for some time. Old approach will be deprecated - but it should be possible to support old format using an option on the server.