google / flatbuffers

FlatBuffers: Memory Efficient Serialization Library

Home Page:http://google.github.io/flatbuffers/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[swift] Generated code does not allow for differentiating between null vectors and vectors with 0 elements

hassila opened this issue · comments

Just found the support for optional scalars in flat buffers (#6014) and the implementation for scalars (#6038) and played a bit with it now to see how the generated Swift code would look and found an issue:

For a struct inside a table, an optional accessor is correctly generated by default:

schema extract:
struct UUID {
    var f: UInt64 = 0
    var s: UInt64 = 0
}
...
generated accessor example with optional:
public var id: XXX_UUID? { let o = _accessor.offset(VTOFFSET.id.v); return o == 0 ? nil : _accessor.readBuffer(of: XXX_UUID.self, at: o) }

likewise for scalars:

schema extract:
table RandomData {
  intVector:[ulong];
  doubleVector:[double];
  uuidVector:[UUID];
  sideVector:[Side];
  x:long = null;
  y:ulong = null;
  a:uint = null;
  b:int = null;
  c:ushort = null;
  d:short = null;
  e:ubyte = null;
  f:byte = null;
}
...
generated accessor example with optional:
  public var x: Int64? { let o = _accessor.offset(VTOFFSET.x.v); return o == 0 ? nil : _accessor.readBuffer(of: Int64.self, at: o) }

The issue now, is with vectors:

schema extract:
table RandomData {
  intVector:[ulong];
  doubleVector:[double];
  uuidVector:[UUID];
  sideVector:[Side];
  x:long = null;
  y:ulong = null;
  a:uint = null;
  b:int = null;
  c:ushort = null;
  d:short = null;
  e:ubyte = null;
  f:byte = null;
}
...
generated accessor example for vectors:

  public var intVectorCount: Int32 { let o = _accessor.offset(VTOFFSET.intVector.v); return o == 0 ? 0 : _accessor.vector(count: o) }
  public func intVector(at index: Int32) -> UInt64 { let o = _accessor.offset(VTOFFSET.intVector.v); return o == 0 ? 0 : _accessor.directRead(of: UInt64.self, offset: _accessor.vector(at: o) + index * 8) }

The problem here is that both the lack of the vector field in the message (the check with _accessor.offset) AND the _accessor.vector count, will both return zero for intVectorCount.

This is problematic if using flat buffers for delta updates of entities and wanting to zero out the vector (sending the actual vector with 0 elements on the wire to clear it, which is different from it not being on the wire at all).

A few different thoughts:

A. One small change would be to change intVectorCount to be optional, but that would break current code.
B. Another option would be to generate an additional optional (pun intended) optional check for count, I would suggest similar to how swift protobuf does it with e.g. "hasIntVector" that would just return true if _accessor.offset is non-zero, otherwise false.

I think B would be nice as it is non-intrusive.

Also, on a related issue in terms of API:

I wonder if it would be reasonable to consider adding iterator support wrappers for vectors in flatbuffers? it would be nice to access them with usual for x in y syntax.

Thoughts?

Forget about the question about iterator support, I just saw that I missed the third aspects of the accessor and that it already exists that way;

That gives option C, which is also source breaking:

public var intVector: [UInt64] { return _accessor.getVector(at: VTOFFSET.intVector.v) ?? [] }
should be:
public var intVector: [UInt64]? { return _accessor.getVector(at: VTOFFSET.intVector.v) }
(B would still be pragmatic short-term, but for a future api-breaking tweak maybe C can be considered)

I think just adding a var hasIntVector: bool { let o = _accessor.offset(VTOFFSET.doubleVector.v); return o == 0 ? false : true } or something like that.

Yes, that's basically option B as I see it, would be good enough I think

ezoic increase your site revenue