Get rid of FlatBufferReader and reimplement LazyAcces
mzaks opened this issue · comments
#46 shows that by using structs and direct reading from buffer we can achieve fast performance with nice API. Getting rid of FlatBuffersReader should also benefit the eager implementation without the need of instance pooling. The FlatBuffersReader.swift file should only contain the pure functions and no class definition.
Also, as we can operate directly on data read from external sources (e.g. Network or mapped files) true zero copy is possible for streaming applications which is really nice.
Should also change getters to assume 0 offset and instead adjust the pointer for structs, then we get rid of the myOffset property completely...
So your were thinking of having the Eager implementation being a ref counter object and let the lazy implementation be struct based?
I think that would be quite nice, one only need to be clear in terms of documentation that the structs only are valid as long as the underlying buffer pointer is valid.
Btw, it seems that all table properties must use optionals to be able to fully reflect the flat buffers protocol (in the current codegen it does not seems to be the case for some scalars).
Scalars are not optional because of default values. Even if you don't define the default value in schema when vTable for a property is set to 0 and the property is a scalar you have to return a default default :) in case of numbers it is a 0, in case of boolean it is false, in case of an enum it is the 0 case. Structs hm there I am not sure, it is probably the struct with scalars set to default values. This is how Googles implementation works. As in most languages there is no concept for optional scalars. Nil/Null will be returned only for strings, tables and unions.
I am already breaking the compatibility as I transform enum values to real enums and therefor this scalar can be nil. As in case where I have a new value and an old client which has no case for such value.
Here is a discussion I had on the google group channel
https://groups.google.com/forum/#!searchin/flatbuffers/enum/flatbuffers/kpea0C6hXgk/FWZQYz4VCAAJ
Wow, that was a bit unexpected design choice (by Google), thanks for clarifying.
It kind of defeats a great feature (being able to represent optionals on the wire without ugly/fragile workarounds, like stuffing each value in a struct with a bool, or having 'magic values') - it would make much more sense to separate out presence checking and querying for the default value so it is not done behind-the-scenes IMO.
Btw, having null-representation for scalars have been very common in e.g. SQL databases for ~40 years, so there are clear use cases (and I have encountered plenty of them in real-world scenarios when it comes to serialization of data as well...).
Ok, it is what it is now from a spec perspective, probably not much to do about it, I just read the doc and saw that they supported optional values and thought it was great... If one must return a default value in the implementation, there are truly no optionals from an application developers POV, even if the wire format has the information, bit of a pity really.
Btw, I do agree with your arguments for having proper Enums in that thread, it seems dicey to let an old client manipulate data it can't understand properly as the types used actually have changed. I would argue that extending the possible values of the enum is a semantical version break. Perhaps we could provide an additional accessor the returns back the int that could be used if the Enum is nil (together with the ability to set such a value), to allow the possibility to write code that handles such cases.
Ok, I will stop here, this is on a tangent from what this issue truly was about.
Ok, there is a workaround that actually solves it when needed:
"Another option that works in all languages is to wrap a scalar field in a struct. This way it will return null if it is not present. The cool thing is that structs don't take up any more space than the scalar they represent."
struct Integer { i:int }
https://groups.google.com/forum/#!topic/flatbuffers/OdI1xmEU0rA
So for those cases one would need this functionality, one can always work around it in this way.
It means though that struct references (e,g, 'sibling' in the test) must be optionals.
Then of course, there is really a difference between a field not being present, or it being set to nil - that can't be expressed (it is useful for nil:ing out existing values).
So you kind of build your own optional struct :)
struct OptionalInt {
value : int;
}
struct OptionalByte {
value : byte;
}
and so on ...
Exactly!
The reader is already designed in a way that it returns an optional if default value is not provided
https://github.com/mzaks/FlatBuffersSwift/blob/master/FlatBuffersSwift/FlatBufferReader.swift#L65
And it is also supported by the code generator:
https://github.com/mzaks/FlatBuffersSwift/blob/master/FlatBuffersPerformanceTestDesktop/bench.swift#L30
Btw about the FlatBufferReader - perhaps it could be implemented as a struct to namespace the 'raw functions'.
Will close this one because of #64