Where a JSON message exceeds the TCP MSS, the Listener should wait until it has a complete message before parsing it
Ozzard opened this issue · comments
As modern receivers get more sensitive, it's getting more common to see very large hunks of JSON coming out of gpsd. SKY messages from my device often exceed 2.5k.
When gpsd sends these large messages, they're split into multiple frames (TCP segments) across the network, each no larger than TCP's max segment size (MSS). The "data" event of the socket triggers for each segment, meaning you get two "bad message format" events - one for the front-end of the message, and then one for the back-end.
The Listener should combine segments that don't end with a newline until it finds one that does, then parse the whole thing.
I've got working code to fix this and will submit a PR.
Thank you for your contribution. I will take a look at it before the weekend.
Thanks! I hit this with SKY messages coming from gpsd fed from a u-blox M8 board on a Raspberry Pi, which I was then accessing across the network from node-red-contrib-gpsd, which has node-gpsd as a dependency. Took a little working out as to what was going on. I suspect most people use gpsd locally, and the loopback network has a massive MSS so this would never arise.
I ran into the same issue with node-red-contrib-gpsd which uses node-gpsd
the objects emitted by GPSD are technically NDJSON
I had good results with using carrier.js for handling TCP-chunked NDJSON in a different setting
here's a PR which works for me: #21
Thank you for reminding me @mhaberler. I just merged @Ozzard merge request and published version 0.3.3 that fixes this issue to npm.
appreciated!