Serialization error when properties / attributes present
rootix opened this issue · comments
Hi,
I'm banging my head around deserializing a Feature from a simple JSON structure with the System.Text.Json version of the library.
I have the following code which throws an Exception:
var options = new JsonSerializerOptions();
options.Converters.Add(new NetTopologySuite.IO.Converters.GeoJsonConverterFactory());
var feature = JsonSerializer.Deserialize<Feature>("{\"type\":\"Feature\",\"geometry\":{\"type\":\"Point\",\"coordinates\":[7.485730780104628,46.95471171052063]},\"properties\":{\"id\":\"new\"}}", options);
Console.WriteLine("Re-serialized Feature: " + JsonSerializer.Serialize(feature, options));The beautified JSON looks like:
{
"type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [
7.485730780104628,
46.95471171052063
]
},
"properties": {
"id": "new"
}
}I get the following exception on deserialization time (i use this in a Blazor app, the stacktrace is from the Browser console):
blazor.webassembly.js:1
Uncaught (in promise) Error: System.TypeInitializationException: The type initializer for 'NetTopologySuite.IO.Converters.StjAttributesTableConverter' threw an exception.
---> System.Text.Json.JsonReaderException: The input does not contain any JSON tokens. Expected the input to start with a valid JSON token, when isFinalBlock is true. LineNumber: 0 | BytePositionInLine: 0.
at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan`1 bytes)
at System.Text.Json.Utf8JsonReader.Read()
at System.Text.Json.JsonDocument.Parse(ReadOnlySpan`1 utf8JsonSpan, JsonReaderOptions readerOptions, MetadataDb& database, StackRowStack& stack)
at System.Text.Json.JsonDocument.Parse(ReadOnlyMemory`1 utf8Json, JsonReaderOptions readerOptions, Byte[] extraRentedArrayPoolBytes, PooledByteBufferWriter extraPooledByteBufferWriter)
at System.Text.Json.JsonDocument.Parse(ReadOnlyMemory`1 json, JsonDocumentOptions options)
at System.Text.Json.JsonDocument.Parse(String json, JsonDocumentOptions options)
at NetTopologySuite.IO.Converters.StjAttributesTable..ctor()
at NetTopologySuite.IO.Converters.StjAttributesTableConverter..cctor()
--- End of inner exception stack trace ---
at NetTopologySuite.IO.Converters.GeoJsonConverterFactory.CreateConverter(Type typeToConvert, JsonSerializerOptions options)
at System.Text.Json.Serialization.JsonConverterFactory.GetConverterInternal(Type typeToConvert, JsonSerializerOptions options)
at System.Text.Json.JsonSerializerOptions.GetConverterInternal(Type typeToConvert)
at System.Text.Json.JsonSerializerOptions.DetermineConverter(Type parentClassType, Type runtimePropertyType, MemberInfo memberInfo)
at System.Text.Json.Serialization.Metadata.JsonTypeInfo.GetConverter(Type type, Type parentClassType, MemberInfo memberInfo, Type& runtimeType, JsonSerializerOptions options)
at System.Text.Json.Serialization.Metadata.JsonTypeInfo..ctor(Type type, JsonSerializerOptions options)
at System.Text.Json.JsonSerializerOptions.<InitializeForReflectionSerializer>g__CreateJsonTypeInfo|112_0(Type type, JsonSerializerOptions options)
at System.Text.Json.JsonSerializerOptions.GetClassFromContextOrCreate(Type type)
at System.Text.Json.JsonSerializerOptions.GetOrAddClass(Type type)
at System.Text.Json.JsonSerializer.GetTypeInfo(JsonSerializerOptions options, Type runtimeType)
at System.Text.Json.JsonSerializer.Deserialize[IAttributesTable](Utf8JsonReader& reader, JsonSerializerOptions options)
at NetTopologySuite.IO.Converters.StjFeatureConverter.Read(Utf8JsonReader& reader, Type objectType, JsonSerializerOptions options)
at System.Text.Json.Serialization.JsonConverter`1[[NetTopologySuite.Features.IFeature, NetTopologySuite.Features, Version=2.0.0.0, Culture=neutral, PublicKeyToken=f580a05016ebada1]].TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, IFeature& value)
at System.Text.Json.Serialization.JsonConverter`1[[NetTopologySuite.Features.IFeature, NetTopologySuite.Features, Version=2.0.0.0, Culture=neutral, PublicKeyToken=f580a05016ebada1]].ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
at System.Text.Json.Serialization.JsonConverter`1[[NetTopologySuite.Features.IFeature, NetTopologySuite.Features, Version=2.0.0.0, Culture=neutral, PublicKeyToken=f580a05016ebada1]].ReadCoreAsObject(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
at System.Text.Json.JsonSerializer.ReadFromSpan[Feature](ReadOnlySpan`1 utf8Json, JsonTypeInfo jsonTypeInfo, Nullable`1 actualByteCount)
at System.Text.Json.JsonSerializer.ReadFromSpan[Feature](ReadOnlySpan`1 json, JsonTypeInfo jsonTypeInfo)
at System.Text.Json.JsonSerializer.Deserialize[Feature](String json, JsonSerializerOptions options)
After some investigation i found the issue to be the construction of the StjAttributesTableConverter class, specifically the constructor of the StjAttributesTable class, which is instantiated for the EmptyTable property within the converter.
The call to JsonDocument.Parse("") throws this exception. What puzzles me is that the project references version 4.7.2 of the System.Text.Json library, which i verified to throw the exact same exception (tested within a blank Console Application). I first tought it could be a issue that i use .NET 6 with a 6.x version of System.Text.Json, but the behavior was the same for me with 4.7.2.
If i only deserialize the Geometry part into a Point object, the responsible converter works.
Am i doing something terribly wrong here?
Confirmed that this is an issue with the latest versions of everything.
I wanted to say that it seems that there might have been a breaking change in a newer version of the library, but even today, I can't seem to produce a scenario where the following code does not throw that error:
using System;
using System.Text.Json;
static class C
{
static void Main()
{
using (var doc = JsonDocument.Parse(""))
{
Console.WriteLine(doc.RootElement);
}
}
}I don't have the time to confirm the impact at this point, but I'll look into getting it fixed soon. Thanks for the report.
Interesting. I confirmed this on Windows, but it seems to work perfectly fine on Linux for some very strange reason (after changing Deserialize<Feature> to Deserialize<IFeature>):
I suspect that the issue will be evident in the automated tests once I switch from the Linux-only Travis-CI to our standard GitHub Actions workflow that runs the tests on all three of Windows + MacOS + Linux.
Short version: I've mostly figured out what's going on, and I think I have a fix for that which I'll push sometime later today, but I can't quite confirm 100% that it will fix whatever you saw. Read on if you want way too many details.
Coming up with the fix is easy. The hard part was figuring out why the fix is needed, in order to figure out how to write an automated test to cover whatever code path was being missed.
It is correct to note that this default constructor will always throw an exception when invoked on this line:
Since this is the only static member of the type, the CLR is allowed to leave that field uninitialized until the very last instant before it's observed... and there's just one place where it actually gets observed:
{ "type": "Feature", "properties": null }...however, this exact case is actually covered by one of our test cases:
null there.
So I THINK that what's going on is that your CLR is deciding, for some reason that I don't quite know how to identify right now, to invoke the always-broken static initializer even though it probably doesn't need to.
For that reason, the fix seems simple: delete the default constructor and static field, and change this case to return null;. So I'm going to do that regardless.
Of course, someone would see this if they were to fetch a StjAttributesTableConverter out of a GeoJsonConverterFactory and invoke its Read method directly, themselves. I can't see why anyone would do that, though...
Hi Joe,
Thanks for the fast and troughtful analysis and fix. Does this project build some alpha/beta packages to some other repository to get my hands on this fix?
https://www.myget.org/feed/nettopologysuite/package/nuget/NetTopologySuite.IO.GeoJSON4STJ
The fix is in there as of version 3.0.0-pre.191130297. If you have the time, could you please let me know if it works for you?
Thanks for the package.
It works now! I tested the following variants:
- properties with a key/value pair
- properties with an empty object
- properties with null value
- no properties key present at all
All cases are working. But in the last one without a properties key, reserializing the resulting feature adds a properties: null to the JSON-string. Is this the intended behavior and according to the spec?
Regarding the CLR: I wasn't aware about the delayed initialization behavior. I suspect the WebAssembly "port" to be the issue. Somehow the resulting engine behaves differently. When i serialize a FeatureCollection inside the backend code (Windows CLR), it serializes the properties happily with the 2.1.1 version.
Thanks for reporting back.
But in the last one without a properties key, reserializing the resulting feature adds a
properties: nullto the JSON-string. Is this the intended behavior and according to the spec?
You can ask System.Text.Json serializers to skip writing null by setting JsonSerializerOptions.DefaultIgnoreCondition to either WhenWritingDefault or WhenWritingNull. I can't confirm that we're checking that flag in 100% of the places where the spec allows us to skip writing a null, but I know that we do check it for properties, specifically, so you should be able to opt-out of at least these ones by setting that flag.
I suspect the WebAssembly "port" to be the issue.
Ahh, right, Mono (especially on WASM) is a different enough implementation of the CLR that I can totally understand seeing differences like this. OK, that's good enough for me to mark this as fixed now.