Size Prefixed Verifier checks for length equality

[Derek Bailey]

For sized prefixed buffers, the verifier explicitly checks for length equality of the prefixed size. While correct, it makes the caller pass in the exact size of the buffer, even if they a larger buffer with multiple flatbuffers in it. Changing it to <= might be better.

This was brought up in https://stackoverflow.com/questions/76185538/flatbuffers-verifysizeprefixedbuffer-from-stream.

[tchatow]

While we're at it, I think an interface like this would be useful:

flatbuffers::uoffset_t message_size;
// message_size out-parameter returns sizeof(prefix) + prefix
while (flatbuffers::Verifier(buf.data(), buf.size())
           .VerifySizePrefixedBuffer<Message>(nullptr, &message_size)) {
  const auto msg = flatbuffers::GetSizePrefixedRoot<Message>(buf.data());

  // Use message

  buf = buf.subspan(message_size);
}

The idea would be to encapsulate the responsibility managing the size prefix. It doesn't seem obvious why FinishSizePrefixed does this for the user, while on the read side it must be read out separately with GetPrefixedSize.

Equivalently could be having message_size returned from GetSizePrefixedRoot instead of VerifySizePrefixedBuffer - this might be more consistent.

[Derek Bailey]

Not sure I understand the FinishedSizedPrefixed part, as that is in the serialization path, not the deserialization path where the verifier is used. And when we are building the buffer, we know the exact size, so it is easy for us to place it in the buffer without the caller having to do it.

I think just calling GetPrefixedSize() is the correct approach instead of an out value. Yes its convenient for your case, but there are users who might not want the size and now they have to provide an out-value. Why isn't something like:

 buf = buf.subspan(flatbuffers::GetPrefixedSize(buf.data() + sizeof(uoffset_t));

OK?

[tchatow]

Yes, that will work fine with the change to <=. The only part that felt weird is sizeof(uoffset_t) - it's not immediately obvious that sizeof(uoffset_t) == size of serialized prefix.

[Derek Bailey]

That's fair. In the PR i added a new method GetSizePrefixedBufferLength(buf) that returns the length + the sizeof(prefix).

I didn't want to change the semantics of the existing GetPrefixedSize(buf) function, so that's why i added a new one.