Serialization of type with multiple dictionaries
thomas-rabiller opened this issue · comments
What is the bug?
I am trying to use the serializer for a round trip Serialize/Deserialize of the same C# class type. For most types I've used this has worked fine but with a specific model containing two Dictionary properties, the deserialization fails.
How to reproduce?
This sample code fails using AvroConvert 3.4.6
using SolTechnology.Avro;
public class SampleType
{
public Dictionary<int, byte[]> ByteArrayData { get; set; } = new();
public Dictionary<int, float> FloatData { get; set; } = new();
}
var data = new[]
{
new SampleType {
FloatData = new Dictionary<int, float>{ [1] = 23.0f } },
new SampleType {
ByteArrayData = new Dictionary<int, byte[]>{ [1] = [0,1,2,3,4] } }
};
var file = AvroConvert.Serialize(data);
System.IO.File.WriteAllBytes("test.avro", file);
var read = AvroConvert.Deserialize<SampleType[]>(file);
Exception during Deserialize():
Error: SolTechnology.Avro.Infrastructure.Exceptions.AvroTypeMismatchException: Unable to deserialize [array] of schema [Array] to the target type [Submission#4+SampleType[]]. Inner exception:
---> SolTechnology.Avro.Infrastructure.Exceptions.AvroTypeMismatchException: Unable to deserialize [SampleType] of schema [Record] to the target type [Submission#4+SampleType]. Inner exception:
---> SolTechnology.Avro.Infrastructure.Exceptions.AvroTypeMismatchException: Unable to deserialize [array] of schema [Array] to the target type [System.Collections.Generic.Dictionary`2[System.Int32,System.Single]]. Inner exception:
---> Microsoft.CSharp.RuntimeBinder.RuntimeBinderException: The best overloaded method match for 'System.Collections.Generic.Dictionary<int,float>.Add(int, float)' has some invalid arguments
at CallSite.Target(Closure, CallSite, Object, Object, Object)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.ResolveDictionary(RecordSchema writerSchema, RecordSchema readerSchema, IReader d, Type type)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.ResolveArray(TypeSchema writerSchema, TypeSchema readerSchema, IReader reader, Type type, Int64 itemsCount)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.Resolve(TypeSchema writerSchema, TypeSchema readerSchema, IReader reader, Type type)
--- End of inner exception stack trace ---
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.Resolve(TypeSchema writerSchema, TypeSchema readerSchema, IReader reader, Type type)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.GetValue(RecordFieldSchema wf, RecordFieldSchema rf, Member memberInfo, IReader dec)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.ResolveRecord(RecordSchema writerSchema, RecordSchema readerSchema, IReader reader, Type type)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.Resolve(TypeSchema writerSchema, TypeSchema readerSchema, IReader reader, Type type)
--- End of inner exception stack trace ---
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.Resolve(TypeSchema writerSchema, TypeSchema readerSchema, IReader reader, Type type)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.ResolveArray(TypeSchema writerSchema, TypeSchema readerSchema, IReader reader, Type type, Int64 itemsCount)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.Resolve(TypeSchema writerSchema, TypeSchema readerSchema, IReader reader, Type type)
--- End of inner exception stack trace ---
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.Resolve(TypeSchema writerSchema, TypeSchema readerSchema, IReader reader, Type type)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.Resolve[T](IReader reader, Int64 itemsCount)
at SolTechnology.Avro.Features.Deserialize.Decoder.Read[T](Reader reader, Header header, AbstractCodec codec, Resolver resolver)
at SolTechnology.Avro.Features.Deserialize.Decoder.Decode[T](Stream stream, TypeSchema readSchema)
at SolTechnology.Avro.AvroConvert.Deserialize[T](Byte[] avroBytes)
at Submission#4.<<Initialize>>d__0.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.CodeAnalysis.Scripting.ScriptExecutionState.RunSubmissionsAsync[TResult](ImmutableArray`1 precedingExecutors, Func`2 currentExecutor, StrongBox`1 exceptionHolderOpt, Func`2 catchExceptionOpt, CancellationToken cancellationToken)
Please note that the same code works fine for a class with only one of the two dictionary properties. I suspect there is a mixup between the two during type resolution.
What is the Avro schema?
The schema is auto-generated from this class:
public class SampleType
{
public Dictionary<int, byte[]> ByteArrayData { get; set; } = new();
public Dictionary<int, float> FloatData { get; set; } = new();
}
and produces the following json schema (please note the two dictionary types with same name:
{"type":"array","items":{"name":"SampleType","type":"record","fields":[{"name":"ByteArrayData","type":{"type":"array","items":{"name":"KeyValuePair2","namespace":"System.Collections.Generic","type":"record","fields":[{"name":"Key","type":"int"},{"name":"Value","type":"bytes"}]}}},{"name":"FloatData","type":{"type":"array","items":{"name":"KeyValuePair2","namespace":"System.Collections.Generic","type":"record","fields":[{"name":"Key","type":"int"},{"name":"Value","type":"float"}]}}}]}}
What is the Avro data? Fill up the section or provide a sample file
Test avro file produced by the repro code here:
test.zip
What is the expected behavior?
Both properties should be deserialized correctly based on the same reader and writer schemas.
Hey,
Thank you for reporting the bug. I will work on a fix.
Best,
Adrian
Hello @thomas-rabiller
Thank you for your contribution. The fix is available from v.3.4.7 of AvroConvert.
Regards,
Adrian
Many thanks! I confirm this fixes the problem on my end.
Thanks again for this great library
Kind regards
Thomas
Hey @thomas-rabiller,
I had to change the solution, as the previous introduced breaking changes to other scenarios. Please consider moving to v.3.4.8
Adrian