Test failure: ChallengeRedirectIsIssuedCorrectly

Question

Test failure: ChallengeRedirectIsIssuedCorrectly

aspnet-hello opened this issue 6 years ago · comments

This issue was made automatically. If there is a problem contact ryanbrandenburg.

This test failed with the following error:

The redirect uri is not valid.
https://login.microsoftonline.com/common/oauth2/authorize?client_id=Test%20Id&redirect_uri=https%3A%2F%2Fexample.com%2Fsignin-oidc&response_type=id_token&scope=openid%20profile&response_mode=form_post&nonce=636748777266645870.MGYxYzJmODctMTQ3Ny00OGJkLWIxNDYtMzM1MTI2ZjFiMTRmMjA2MThkOTAtZGY1OC00ZDcyLWEzMjYtOGQ1NDY0NTMyMjk5&state=CfDJ8NOdSWsmhQpKgJsrWkOpNWmBdcel2lDp6l4ieYm3gRq153RYqt5lVWtFYXfMqD4qcZOUXov2uCL_iL02ajxnUsIjxZvoF4HKHHwKJRmfFvCvRtINMXseCeQlQsAo9_IMD97Cv99rKLiNC_1PI15srOgkGMpJzsbiD0T723v8duDe0xKPE7f9WAhKfra-XjhPBbAcs7q4t68vt-Vabi6oAerC6RAMeimjQ0W8XjsNgQPOHEnVe6tX-xOTeVCMjPztwxbFQNtl_LFYI2qKRVvjji2ISjMzl9t1gxR0hCCcX4O_c-SP9Ym7YSDjotPExv_C529OQT_E5-43j89WctLPFkw&x-client-SKU=ID_NETSTANDARD2_0&x-client-ver=5.3.0.0
Parameter x-client-SKU's expected value is 'ID_NETSTANDARD1_4' but its actual value is 'ID_NETSTANDARD2_0'
Expected: True
Actual:   False
   at Microsoft.AspNetCore.Authentication.Test.OpenIdConnect.TestSettings.ValidateRedirectCore(Uri redirectUri, OpenIdConnectRequestType requestType, String[] parametersToValidate) in /_/test/Microsoft.AspNetCore.Authentication.Test/OpenIdConnect/TestSettings.cs:line 163
   at Microsoft.AspNetCore.Authentication.Test.OpenIdConnect.TestSettings.ValidateChallengeRedirect(Uri redirectUri, String[] parametersToValidate) in /_/test/Microsoft.AspNetCore.Authentication.Test/OpenIdConnect/TestSettings.cs:line 128
   at Microsoft.AspNetCore.Authentication.Test.OpenIdConnect.OpenIdConnectChallengeTests.ChallengeRedirectIsIssuedCorrectly() in /_/test/Microsoft.AspNetCore.Authentication.Test/OpenIdConnect/OpenIdConnectChallengeTests.cs:line 40
--- End of stack trace from previous location where exception was thrown ---

Other tests within that build may have failed with a similar message, but they are not listed here. Check the link above for more info.

This test failed on 2.2.

CC @Eilon (because the bot doesn't know who else to pick)

ASP.NET Hello Bot · Answer 1 · Fri Oct 12 2018 03:29:43 GMT+0800 (China Standard Time)

This comment was made automatically. If there is a problem contact ryanbrandenburg.

Please use this workflow to address this flaky test issue, including checking applicable checkboxes and filling in the applicable "TODO" entries:

Is this actually a flaky test?
- No, this is a regular test failure, fix the test/product (TODO: Link to commit/PR)
- Yes, proceed below...
Is this test failure caused by product code flakiness? (Either this product, or another product this test depends on.)
- File a bug against the product (TODO: Link to other bug)
- Is it possible to change the test to avoid the flakiness?
  - Yes? Go to the "Change the test!" section.
  - No?
    - Disable the test (TODO: Link to PR/commit)
    - Wait for other bug to be resolved
    - Wait for us to get build that has the fix
    - Re-enable our test (TODO: Link to PR/commit)
    - Close this bug
Is it that the test itself is flaky? This includes external transient problems (e.g. remote server problems, file system race condition, etc.)
- Is there is a way to change our test to avoid this flakiness?
  - Yes? Change the test!
    - Change the test to avoid the flakiness, for example by using a different test strategy, or by adding retries w/ timeouts (TODO: Link to PR/commit)
    - Run the test 100 times locally as a sanity check.
    - Close this bug
  - No?
    - Is there any logging or extra information that we could add to make this more diagnosable when it happens again?
      - Yes?
        
        Add the logging (TODO: Link to PR/commit)
      - No?
        
        Delete the test because flaky tests are not useful (TODO: Link to PR/commit)

ASP.NET Hello Bot · Answer 2 · Fri Oct 12 2018 03:29:45 GMT+0800 (China Standard Time)

This comment was made automatically. If there is a problem contact ryanbrandenburg.

There were 1 failures with about the same error on 2.2 at 10:17:09 AM:

SignOutSettingMessage

There were 2 failures with about the same error on master at 12:14:23 PM:

ChallengeRedirectIsIssuedCorrectly
SignOutSettingMessage

There were 2 failures with about the same error on master at 12:19:57 PM:

ChallengeRedirectIsIssuedCorrectly
SignOutSettingMessage

There were 2 failures with about the same error on master at 11:39:34 AM:

SignOutSettingMessage
ChallengeRedirectIsIssuedCorrectly

There were 2 failures with about the same error on master at 1:25:59 PM:

SignOutSettingMessage
ChallengeRedirectIsIssuedCorrectly

There were 2 failures with about the same error on master at 11:51:38 AM:

ChallengeRedirectIsIssuedCorrectly
SignOutSettingMessage

There were 2 failures with about the same error on master at 11:50:13 AM:

ChallengeRedirectIsIssuedCorrectly
SignOutSettingMessage

There were 2 failures with about the same error on master at 11:57:00 AM:

SignOutSettingMessage
ChallengeRedirectIsIssuedCorrectly

There were 2 failures with about the same error on master at 1:40:25 PM:

ChallengeRedirectIsIssuedCorrectly
SignOutSettingMessage

There were 2 failures with about the same error on master at 2:10:40 PM:

ChallengeRedirectIsIssuedCorrectly
SignOutSettingMessage

Eilon Lipton · Answer 3 · Fri Oct 12 2018 04:00:42 GMT+0800 (China Standard Time)

@Tratcher can you take a look?

Chris Ross · Answer 4 · Fri Oct 12 2018 04:49:03 GMT+0800 (China Standard Time)

We need to react to aspnet/Universe@06e9531, or at least make the test less strict about these fields.

@HaoK FYI

ASP.NET Hello Bot · Answer 5 · Fri Oct 12 2018 08:22:01 GMT+0800 (China Standard Time)

This comment was made automatically. If there is a problem contact ryanbrandenburg.

There were 2 failures with about the same error on master at 2:33:08 PM:

SignOutSettingMessage
ChallengeRedirectIsIssuedCorrectly

There were 2 failures with about the same error on master at 2:33:07 PM:

ChallengeRedirectIsIssuedCorrectly
SignOutSettingMessage

There were 2 failures with about the same error on master at 2:31:34 PM:

SignOutSettingMessage
ChallengeRedirectIsIssuedCorrectly

There were 2 failures with about the same error on master at 2:45:17 PM:

SignOutSettingMessage
ChallengeRedirectIsIssuedCorrectly

There were 2 failures with about the same error on master at 11:19:38 PM:

SignOutSettingMessage
ChallengeRedirectIsIssuedCorrectly

There were 2 failures with about the same error on master at 11:15:24 PM:

ChallengeRedirectIsIssuedCorrectly
SignOutSettingMessage

There were 2 failures with about the same error on master at 11:15:23 PM:

SignOutSettingMessage
ChallengeRedirectIsIssuedCorrectly

There were 2 failures with about the same error on master at 11:15:17 PM:

ChallengeRedirectIsIssuedCorrectly
SignOutSettingMessage

There were 2 failures with about the same error on master at 11:15:24 PM:

SignOutSettingMessage
ChallengeRedirectIsIssuedCorrectly

There were 2 failures with about the same error on master at 11:15:24 PM:

ChallengeRedirectIsIssuedCorrectly
SignOutSettingMessage

There were 2 failures with about the same error on master at 11:15:24 PM:

SignOutSettingMessage
ChallengeRedirectIsIssuedCorrectly

There were 2 failures with about the same error on master at 1:07:41 AM:

SignOutSettingMessage
ChallengeRedirectIsIssuedCorrectly

There were 2 failures with about the same error on 2.2 at 11:24:35 AM:

ChallengeRedirectIsIssuedCorrectly
SignOutSettingMessage

Eilon Lipton · Answer 6 · Sat Oct 13 2018 00:16:44 GMT+0800 (China Standard Time)

@Tratcher does your fix address the 2 more recent failures?

Eilon Lipton · Answer 7 · Sat Oct 13 2018 00:16:55 GMT+0800 (China Standard Time)

Oh sorry just realized they're the same. So hopefully yes 😄