Race Condition in SentryMessageHandler - System.InvalidOperationException: This instance has already started one or more requests. Properties can only be modified before sending the first request.
optical opened this issue · comments
Package
Sentry
.NET Flavor
.NET
.NET Version
8.0
OS
Any (not platform specific)
SDK Version
4.8.1
Self-Hosted Sentry Version
No response
Steps to Reproduce
In
The assignment of the inner handler is not thread safe. See https://github.com/microsoft/referencesource/blob/master/System/net/System/Net/Http/DelegatingHandler.cs#L33
and
https://github.com/microsoft/referencesource/blob/51cf7850defa8a17d815b4700b67116e3fa283c2/System/net/System/Net/Http/DelegatingHandler.cs#L84
Essentially:
2 Threads enter PropagateTraceHeaders
at the same time, both decide to assign to InnerHandler as it is null. One thread gets ahead and assigns the handler and also manages to start a request. The second thread catches up and tries to perform the assign, but an exception is raised because a request has already begun, thus the inner handler cannot be changed.
You can easily reproduce this with the following application:
ar builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
app.MapGet("/", () => "Hello World!");
app.RunAsync();
while (true)
{
var httpClient = new HttpClient(new SentryHttpMessageHandler());
var tasks = Enumerable.Range(0, Environment.ProcessorCount).Select(_ => Task.Run(() => httpClient.GetAsync("https://localhost:5001"))).ToArray();
await Task.WhenAll(tasks);
Console.WriteLine("No error, trying again");
}
This should error out in a couple seconds in my testing.
This is an issue for use cases where we construct an http client and then perform bulk concurrent operations - like downloading the contents of an S3 bucket or similar.
Expected Result
Never throw an exception when using HttpClient in this way - HttpClient is intended to be thread safe
Actual Result
Unhandled exception. System.InvalidOperationException: This instance has already started one or more requests. Properties can only be modified before sending the first request.
at System.Net.Http.DelegatingHandler.CheckDisposedOrStarted()
at System.Net.Http.DelegatingHandler.set_InnerHandler(HttpMessageHandler value)
at Sentry.SentryMessageHandler.PropagateTraceHeaders(HttpRequestMessage request, String url)
at Sentry.SentryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Program.<Main>$(String[] args)
at Program.<Main>(String[] args)
Oh no, that's not good at all. Thanks for reaching out and for the details on the issue! This is straightforward to reproduce.
Hi folks,
Any idea when this will be released to Nuget?
Any idea when this will be released to Nuget?
Should be coming out in the 4.10.0 release in the next 24-48 hours.