AddSystemsManager breaks other Json providers

[Renato Golia]

Description

When using AddSystemsManager after other providers based on JSON, the other provider is executed twice, throwing an exception.

Reproduction Steps

1. Create a console project with the following package references

<PackageReference Include="Amazon.Extensions.Configuration.SystemsManager" Version="2.1.0" />
<PackageReference Include="Kralizek.Extensions.Configuration.Objects" Version="1.0.1" />
<PackageReference Include="Microsoft.Extensions.Configuration" Version="5.0.0" />

2. Add the following code in Program.cs (relevant namespaces missing)

var configurationBuilder = new ConfigurationBuilder();
configurationBuilder.AddObject(new { Foo = "Bar" }, "Test");
configurationBuilder.AddSystemsManager($"/some/key/to/import");
var configuration = configurationBuilder.Build();

The following exception was thrown: System.FormatException: 'A duplicate key 'Test:Foo' was found.'

3. The same doesn't happen when:

• configurationBuilder.AddSystemsManager($"/some/key/to/import"); is commented out
• configurationBuilder.AddSystemsManager($"/some/key/to/import"); is placed before configurationBuilder.AddObject(new { Foo = "Bar" });
• two calls to AddObject are added with different objects:

configurationBuilder.AddObject(new { Foo = "Bar" }, "Test");
configurationBuilder.AddObject(new { Hello = "World" }, "Test");

Logs

System.FormatException
  HResult=0x80131537
  Message=A duplicate key 'Test:Foo' was found.
  Source=Kralizek.Extensions.Configuration.Objects
  StackTrace:
   at Kralizek.Extensions.Configuration.Internal.JsonConfigurationSerializer.VisitPrimitive(JValue data)
   at Kralizek.Extensions.Configuration.Internal.JsonConfigurationSerializer.VisitToken(JToken token)
   at Kralizek.Extensions.Configuration.Internal.JsonConfigurationSerializer.VisitJObject(JObject jObject)
   at Kralizek.Extensions.Configuration.Internal.JsonConfigurationSerializer.Serialize(Object source, String rootSectionName)
   at Kralizek.Extensions.Configuration.Internal.ObjectConfigurationProvider.Load()
   at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
   at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
   at ConfigurationSample.Program.<Main>d__0.MoveNext() in C:\Users\RenatoGolia\Development\InsightArchitectures\samples\ConfigurationSample\Program.cs:line 26
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at ConfigurationSample.Program.<Main>(String[] args)

  This exception was originally thrown at this call stack:
    Kralizek.Extensions.Configuration.Internal.JsonConfigurationSerializer.VisitPrimitive(Newtonsoft.Json.Linq.JValue)
    Kralizek.Extensions.Configuration.Internal.JsonConfigurationSerializer.VisitToken(Newtonsoft.Json.Linq.JToken)
    Kralizek.Extensions.Configuration.Internal.JsonConfigurationSerializer.VisitJObject(Newtonsoft.Json.Linq.JObject)
    Kralizek.Extensions.Configuration.Internal.JsonConfigurationSerializer.Serialize(object, string)
    Kralizek.Extensions.Configuration.Internal.ObjectConfigurationProvider.Load()
    Microsoft.Extensions.Configuration.ConfigurationRoot.ConfigurationRoot(System.Collections.Generic.IList<Microsoft.Extensions.Configuration.IConfigurationProvider>)
    Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
    ConfigurationSample.Program.Main(string[]) in Program.cs
    System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
    System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)
    ...
    [Call Stack Truncated]

I realize that the error occurs in the other library, but I created this issue because the error occurs when AddSystemsManager is added after the other provider.

Environment

• Build Version: 2.1.0
• OS Info: Windows 10
• Build Environment: dotnet run
• Targeted .NET Platform: net 5.0

Resolution

• 👋 I can/would-like-to implement a fix for this problem myself

I suspect the provider behind AddSystemsManager somehow executes other providers again but I couldn't find the source of the bug.

---

This is a 🐛 bug-report

[Ken Hundley]

Hey @Kralizek, that's correct that it does load some of the sources, depending on the situation. If you don't directly supply the AWSOptions object to the AddSystemsManager call then it'll build the previous sources in a attempt discover the necessary AWS configuration data, like profile, region, etc.

For example:

builder.AddCommandLine(args);
builder.AddEnvironmentVariables();
builder.AddSystemsManager($"/some/key/to/import");

will build the CommandLine and EnvironmentVariables sources and look for the necessary AWS configuration data.
This happens here: SystemsManagerExtensions.cs
Since the sources are executed in order they are added to the builder it always finds the ones above the AddSystemsManager call.

One way to solve this is to explicitly create the AWSOptions object and pass it to the AddSystemsManager call. Since the extenion has all that it needs it does not execute the previous sources.

builder.AddCommandLine(args);
builder.AddEnvironmentVariables();
builder.AddObject(new { Foo = "Bar" }, "Test");
var awsOptions = new AWSOptions { Profile = "test", Region = RegionEndpoint.USEast1 };
builder.AddSystemsManager($"/some/key/to/import", awsOptions);

Also, I have a suggestion for your https://github.com/Kralizek/ObjectConfigurationExtensions project. The issue is occuring because you are creating the json serializer object when the source is created and it has a class level variable in the serializer class that holds all previous data. If you were to move the seralizer creation to the load then you would also prevent this issue since the _data object wouldn't already have the previous execution.

Since the load is only executed when the configuration is built it shouldn't have any performance impact.
btw, I did a quick test with this serializer change and the issue disappeared.

[Renato Golia]

Thanks @KenHundley for your quick reply. I'll look into the change you suggested :)

[Renato Golia]

Thanks to the comment by @KenHundley I was able to fix the strange behavior that the interaction with AddSystemsManager caused.

I'm ok for closing this issue.

[Ashish Dhingra]

@Kralizek Just to mention, I'm unable to reproduce the issue using the following packages:

• Kralizek.Extensions.Configuration.Objects version 2.0.0
• Amazon.Extensions.Configuration.SystemsManager version 2.1.0

Here's my .csproj file:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.1</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Amazon.Extensions.Configuration.SystemsManager" Version="2.1.0" />
    <PackageReference Include="Kralizek.Extensions.Configuration.Objects" Version="2.0.0" />
  </ItemGroup>
</Project>

Closing this issue as you suggested.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[Renato Golia]

@ashishdhingra FYI, the version 2.0 includes the fix to the problem reported in this issue.