Stream writer with custom mapper

[Artur Owczarek]

Background: After migrating one of the services from Insert API to Write API, we've observed a significant performance decline. The JSON to Protobuf conversion step is using 2/3 of CPU time. Our application internally uses Avro records, transforms them to JSON and sends with JsonStreamWriter. The transformation to and from JSON is unnecessary and we should skip it.

There should be a generic class similar to JsonStreamWriter where the developer could provide their own mapper. The mapper should have access to Protobuf (table) schema. It should be able to make not only direct transformations but also some adjustments (for example transforming maps to lists). The generic writer class should take care of schema refreshes. The mapper should inform in a uniform manner that retriable error occurred (and schema should be retried).