Test failure: ThrowsWhenNotOptionalAndDirectoryDoesntExist

Question

Test failure: ThrowsWhenNotOptionalAndDirectoryDoesntExist

aspnet-hello opened this issue 6 years ago · comments

This test fails occasionally with the following error:

Assert.Contains() Failure
Not found: The directory name
In value:  The path must be absolute.
Parameter name: root
   at Microsoft.Extensions.Configuration.KeyPerFile.Test.KeyPerFileTests.ThrowsWhenNotOptionalAndDirectoryDoesntExist() in /_/test/Config.KeyPerFile.Test/KeyPerFileTests.cs:line 31

Other tests within that build may have failed with a similar message, but they are not listed here. Check the link above for more info.

This test failed on master.

CC @Eilon (because the bot doesn't know who else to pick)

This issue was made automatically. If there is a problem contact ryanbrandenburg.

ASP.NET Hello Bot · Answer 1 · Thu Aug 30 2018 05:01:55 GMT+0800 (China Standard Time)

Please use this workflow to address this flaky test issue, including checking applicable checkboxes and filling in the applicable "TODO" entries:

Is this actually a flaky test?
- No, this is a regular test failure, fix the test/product (TODO: Link to commit/PR)
- Yes, proceed below...
Is this test failure caused by product code flakiness? (Either this product, or another product this test depends on.)
- File a bug against the product (TODO: Link to other bug)
- Is it possible to change the test to avoid the flakiness?
  - Yes? Go to the "Change the test!" section.
  - No?
    - Disable the test (TODO: Link to PR/commit)
    - Wait for other bug to be resolved
    - Wait for us to get build that has the fix
    - Re-enable our test (TODO: Link to PR/commit)
    - Close this bug
Is it that the test itself is flaky? This includes external transient problems (e.g. remote server problems, file system race condition, etc.)
- Is there is a way to change our test to avoid this flakiness?
  - Yes? Change the test!
    - Change the test to avoid the flakiness, for example by using a different test strategy, or by adding retries w/ timeouts (TODO: Link to PR/commit)
    - Run the test 100 times locally as a sanity check.
    - Close this bug
  - No?
    - Is there any logging or extra information that we could add to make this more diagnosable when it happens again?
      - Yes?
        
        Add the logging (TODO: Link to PR/commit)
      - No?
        
        Delete the test because flaky tests are not useful (TODO: Link to PR/commit)

This comment was made automatically. If there is a problem contact ryanbrandenburg.

Hao Kung · Answer 2 · Thu Aug 30 2018 05:05:51 GMT+0800 (China Standard Time)

Fixed by 5d87c79 waiting til fix is propagated before closing issue

ASP.NET Hello Bot · Answer 3 · Thu Aug 30 2018 06:01:53 GMT+0800 (China Standard Time)

ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

ASP.NET Hello Bot · Answer 4 · Thu Aug 30 2018 13:01:29 GMT+0800 (China Standard Time)

ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

ASP.NET Hello Bot · Answer 5 · Fri Aug 31 2018 10:03:33 GMT+0800 (China Standard Time)

ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

Nate McMaster · Answer 6 · Sat Sep 01 2018 06:02:46 GMT+0800 (China Standard Time)

Can we backport this test fix to 2.2?

ASP.NET Hello Bot · Answer 7 · Sat Sep 01 2018 08:01:24 GMT+0800 (China Standard Time)

ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

ASP.NET Hello Bot · Answer 8 · Sun Sep 02 2018 20:01:29 GMT+0800 (China Standard Time)

ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

ASP.NET Hello Bot · Answer 9 · Mon Sep 03 2018 09:01:47 GMT+0800 (China Standard Time)

ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

ASP.NET Hello Bot · Answer 10 · Tue Sep 04 2018 20:03:20 GMT+0800 (China Standard Time)

ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on 2.2.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

Hao Kung · Answer 11 · Wed Sep 05 2018 02:05:51 GMT+0800 (China Standard Time)

@natemcmaster what's the best way for me to backport? Just reapply the changes or should I try to cherrypick the single commit over?

Nate McMaster · Answer 12 · Wed Sep 05 2018 02:09:59 GMT+0800 (China Standard Time)

I usually try a soft cherry-pick to backport changes. git cherry-pick --no-commit

Hao Kung · Answer 13 · Wed Sep 05 2018 03:36:22 GMT+0800 (China Standard Time)

Fix backported, will wait a day or so to make sure no more bot reports before closing

ASP.NET Hello Bot · Answer 14 · Wed Sep 05 2018 09:02:56 GMT+0800 (China Standard Time)

ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.
ThrowsWhenNotOptionalAndDirectoryDoesntExist failed with about the same error on master.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

Hao Kung · Answer 15 · Fri Sep 07 2018 06:20:18 GMT+0800 (China Standard Time)

Closing to see if its fixed