working with avro decimal type
sfsf9797 opened this issue · comments
Since this crate use apache avro [to_value] (https://github.com/gklijs/schema_registry_converter/blob/main/src/avro_common.rs#L144) function to convert serializable item to Value
type, it might not work with avro schema with decimal field as the avro serializer might not convert it properly.
use apache_avro::Decimal;
#[derive(Serialize, Clone, Debug)]
pub struct item {
// indexes
pub total_successful_transaction_blocks: Decimal,
The total_successful_transaction_blocks will be serialized to
"total_successful_transaction_blocks", Record([("value", Array([Int(1), Array([Int(7)])])), ("len", Long(1))]))
The issue is not very clear to me...
Is there something that should be fixed/improved in the apache_avro crate or it is an issue only in schema_registry_converter ?
The issue is not very clear to me... Is there something that should be fixed/improved in the apache_avro crate or it is an issue only in schema_registry_converter ?
Hi @martin-g I realize that my message wasn’t very clear—apologies. I see it as issues with apache_avro crate but there is probably something we can do on schema_registry_converter side.
For example, if we have an Avro Union
schema for the following struct:
use apache_avro::Decimal;
pub struct tmp {
// indexes
pub id: String,
pub data: Decimal
}
Either EasyAvroEncoder::encode
or EasyAvroEncoder::encode_struct
will not be able to encode the struct tmp
.
To unblock myself, I implemented a function that take apache_avro::types::Value
and does the encoding. A sample PR
We can pass this
Value::Union(1, Box::new(Value::Record(values)))
into the encode_value function.