[JavaScript] Type Forward/Backward compatible object serialization for Fury JavaScript
chaokunyang opened this issue · comments
Is your feature request related to a problem? Please describe.
Type forward and backward compatible serialization is a common requirement for RPC between microservices. As the calling and receiving ends of the service are deployed and upgraded independently, the serialized objects may exist in different versions across services.
The object structure may have a different number of fields between services. Serialization frameworks need to be able to efficiently support object serialization under situations of type inconsistency between the two ends, ensuring accuracy while providing more compact data and extreme performance.
The Flatbuffer serialization framework suffers from significant data inflation and lacks flexibility, whereas Protobuf achieves type forward and backward compatibility through a KV structure. For each field of the object, during serialization, the object's TAG and type are written.
During deserialization, this information can be used to skip fields not present in the current version. This approach incurs additional performance overhead and space overhead for Message List serialization.
Fury has designed a type compatibility protocol based on metadata compression shared, which at runtime, performs extreme compression of the object's field names and types and other metadata, and shares it across multiple object serializations to minimize metadata overhead. Furthermore, based on this metadata, a deserializer fully consistent with serialization can be dynamically generated during deserialization, completely solving the issues of performance, usability, and space redundancy.
The detailed spec can be fount at https://fury.apache.org/docs/specification/fury_xlang_serialization_spec/#schema-evolution