Flaky test: WebHookAction_NoEventHeader_IsNotFound

Question

Flaky test: WebHookAction_NoEventHeader_IsNotFound

ryanbrandenburg opened this issue 6 years ago · comments

This test fails occasionally with the following error:

Microsoft.AspNetCore.Routing.Matchers.AmbiguousMatchException : The request matched multiple endpoints. Matches: 
GitHubCoreReceiver.Controllers.GitHubController.HandlerForItsPushes (GitHubCoreReceiver)
GitHubCoreReceiver.Controllers.GitHubController.HandlerForIt (GitHubCoreReceiver)
GitHubCoreReceiver.Controllers.GitHubController.HandlerForPush (GitHubCoreReceiver)
GitHubCoreReceiver.Controllers.GitHubController.GitHubHandler (GitHubCoreReceiver)
GitHubCoreReceiver.Controllers.GitHubController.FallbackHandler (GitHubCoreReceiver)
   at Microsoft.AspNetCore.Routing.EndpointConstraints.EndpointConstraintEndpointSelector.SelectAsync(HttpContext httpContext, IEndpointFeature feature, CandidateSet candidates)
   at Microsoft.AspNetCore.Routing.Matchers.DfaMatcher.MatchAsync(HttpContext httpContext, IEndpointFeature feature)
   at Microsoft.AspNetCore.Routing.GlobalRoutingMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.TestHost.HttpContextBuilder.<>c__DisplayClass10_0.<<SendAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.TestHost.ClientHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Mvc.Testing.Handlers.CookieContainerHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Mvc.Testing.Handlers.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Microsoft.AspNetCore.WebHooks.FunctionalTest.GitHubCoreReceiverTest.WebHookAction_NoEventHeader_IsNotFound() in /_/test/Microsoft.AspNetCore.WebHooks.FunctionalTest/GitHubCoreReceiverTest.cs:line 42
--- End of stack trace from previous location where exception was thrown ---

Ryan Brandenburg · Answer 1 · Thu Jul 26 2018 03:05:19 GMT+0800 (China Standard Time)

WebHookAction_NoEventHeader_IsNotFound failed again.

Ryan Brandenburg · Answer 2 · Thu Jul 26 2018 03:30:07 GMT+0800 (China Standard Time)

WebHookAction_NonPost_IsNotAllowed failed again.

Ryan Brandenburg · Answer 3 · Thu Jul 26 2018 03:32:43 GMT+0800 (China Standard Time)

WebHookAction_NonPost_IsNotAllowed failed again.

Ryan Brandenburg · Answer 4 · Thu Jul 26 2018 03:39:29 GMT+0800 (China Standard Time)

WebHookAction_NoEventHeader_IsNotFound failed again.

Eilon Lipton · Answer 5 · Thu Jul 26 2018 04:28:50 GMT+0800 (China Standard Time)

@dougbu - Please use this workflow to address this flaky test issue, including checking applicable checkboxes and filling in the applicable "TODO" entries:

Is this test failure caused by product code flakiness? (Either this product, or another product this test depends on.)
- File a bug against the product (#318, now aspnet/Routing#630)
- Is it possible to change the test to avoid the flakiness?
  - Yes? Go to the "Change the test!" section.
  - No?
    - Disable the test (PR #320, commit bedc118 -- really a workaround)
    - Wait for other bug to be resolved
    - Wait for us to get build that has the fix
    - Re-enable our test (TODO: Link to PR/commit)
    - Close this bug
Is it that the test itself is flaky? This includes external transient problems (e.g. remote server problems, file system race condition, etc.)
- Is there is a way to change our test to avoid this flakiness?
  - Yes? Change the test!
    - Change the test to avoid the flakiness, for example by using a different test strategy, or by adding retries w/ timeouts (TODO: Link to PR/commit)
    - Run the test 100 times locally as a sanity check.
    - Close this bug
  - No?
    - Delete the test because flaky tests are not useful (TODO: Link to PR/commit)

Doug Bunting · Answer 6 · Thu Jul 26 2018 04:49:31 GMT+0800 (China Standard Time)

See #318 for product bug

Ryan Brandenburg · Answer 7 · Thu Jul 26 2018 05:14:25 GMT+0800 (China Standard Time)

WebHookAction_NonPost_IsNotAllowed failed again.