Websocket API: count is parsed as 63 bit unsigned integer

[Simeon Miteff]

The zeek count type is defined as a 64 bit unsigned integer but the websocket API does not allow values > 2^63 - 1 (0x7FFFFFFFFFFFFFFF / 9223372036854775807):

listen.zeek:

redef exit_only_after_terminate = T;
redef check_for_unused_event_handlers = F;
redef Broker::disable_ssl = T;

global test_topic = "/topic/test";

export {
    global test: event(c: count);
}

event test(c: count)
    {
    print fmt("got test: %d", c);
    }

event zeek_init()
    {
    Broker::listen_websocket();
    Broker::subscribe(test_topic);
    }

Testing with websocat:

$ websocat ws://[::]:9997/v1/messages/json
["/topic/test"]
{"type": "ack", "endpoint": "626ff631-492e-5ade-a321-94ba5d073c8c", "version": "2.5.0-dev"}
{"@data-type":"vector","data":[{"@data-type":"count","data":1},{"@data-type":"count","data":1},{"@data-type":"vector","data":[{"@data-type":"string","data":"test"},{"@data-type":"vector","data":[{"@data-type":"count","data":9223372036854775808}]}]}],"topic":"/topic/test","type":"data-message"}
{"type": "error", "code": "deserialization_failed", "context": "input #1 contained malformed JSON -> caf::pec::integer_overflow(1, 243)"}
{"@data-type":"vector","data":[{"@data-type":"count","data":1},{"@data-type":"count","data":1},{"@data-type":"vector","data":[{"@data-type":"string","data":"test"},{"@data-type":"vector","data":[{"@data-type":"count","data":9223372036854775807}]}]}],"topic":"/topic/test","type":"data-message"}

The first event message above generates the "input #1 contained malformed JSON -> caf::pec::integer_overflow(1, 243)" error while the second is accepted:

$ zeek listen.zeek 
got test: 9223372036854775807

[Simeon Miteff]

Ooooh is this because the JSON parser in broker strictly treats JSON "numbers" as doubles?

[Dominik Charousset]

Ooooh is this because the JSON parser in broker strictly treats JSON "numbers" as doubles?

The parser internally treats numbers as either double (if the string has a decimal point or uses scientific notation) or int64_t. That's why it's overflowing here. The parser does this conversion eagerly. I'll have to think a bit about the best way forward.