Add support for pickled fields
densh opened this issue · comments
Introduce a new special kind of fields that would let one transparently serialize and deserialize on-heap objects:
@data class Person(@pickle name: String)
Whenever a pickled field is written it would use an implicit serializer to convert objects to bytes and move those bytes to an off-heap array. Reading would materialize on-heap version on every access.
Serialisation mechanism is going to be based on implicit class that would make it easy to bridge to any serialisation framework user might want to use (e.g. scala.pickling, java serialization etc)
This partially addresses issue #15 as one would be able to define a string serializer that encodes string into UTF-8 sequence of bytes. Operations on those strings would materialize an on-heap version though.
Another alternative for the UI is to put pickles into some generic container type:
@data class Person(name: Pickle[String])
It would let one add extra functionality on Pickle[T] type but would make it a bit harder to use.
An interesting development in more or less this area: https://google.github.io/flatbuffers/
FYI I have a project to support fast Scala collections over FlatBuffers: github.com/velvia/filo
-Evan
"Never doubt that a small group of thoughtful, committed citizens can change the world" - M. Mead
On Aug 10, 2015, at 5:39 AM, Denys Shabalin notifications@github.com wrote:
An interesting development in more or less this area: https://google.github.io/flatbuffers/
—
Reply to this email directly or view it on GitHub.
People should use normal serialisation frameworks instead.