[Bug] dotnet test will override test output
rsun-thoughtworks opened this issue · comments
Background info
I have a solution with multiple project inside it.
Action
When I execute a test within one project.
In the background, the extension is going to run a command like this
dotnet test --no-build --logger "trx;LogFileName=/Users/xx/code/result/test-explorer-K8WIVk/0.trx" --filter "FullyQualifiedName~Demo.Tests.ObjectExtensions"
Because as the extension assign the LogFileName, the dotnet test command will override the log as it process each of the projects.
Yes, each of the projects.
You can see from the output of previous command:
Test run for /Users/xxx/code/DemoManagement/DemoManagement.Api.Tests/bin/Debug/netcoreapp3.1/DemoManagement.Api.Tests.dll(.NETCoreApp,Version=v3.1)
Microsoft (R) Test Execution Command Line Tool Version 16.7.0
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Test run for /Users/xxx/code/DemoManagement/DemoManagement.Design.Tests/bin/Debug/netcoreapp3.1/DemoManagement.Design.Tests.dll(.NETCoreApp,Version=v3.1)
Test run for /Users/xxx/code/DemoManagement/DemoManagement.Domain.Tests/bin/Debug/netcoreapp3.1/DemoManagement.Domain.Tests.dll(.NETCoreApp,Version=v3.1)
Test run for /Users/xxx/code/DemoManagement/DemoManagement.IntegrationTests/bin/Debug/netcoreapp3.1/DemoManagement.IntegrationTests.dll(.NETCoreApp,Version=v3.1)
Test run for /Users/xxx/code/DemoManagement/DemoManagement.Application.Tests/bin/Debug/netcoreapp3.1/DemoManagement.Application.Tests.dll(.NETCoreApp,Version=v3.1)
Microsoft (R) Test Execution Command Line Tool Version 16.7.0
Copyright (c) Microsoft Corporation. All rights reserved.
Microsoft (R) Test Execution Command Line Tool Version 16.7.0
Copyright (c) Microsoft Corporation. All rights reserved.
Microsoft (R) Test Execution Command Line Tool Version 16.7.0
Copyright (c) Microsoft Corporation. All rights reserved.
Microsoft (R) Test Execution Command Line Tool Version 16.7.0
Copyright (c) Microsoft Corporation. All rights reserved.
No test matches the given testcase filter `FullyQualifiedName~DemoManagement.Application.Tests.ObjectExtensions` in /Users/xxx/code/DemoManagement/DemoManagement.Api.Tests/bin/Debug/netcoreapp3.1/DemoManagement.Api.Tests.dll
Starting test execution, please wait...
Starting test execution, please wait...
Starting test execution, please wait...
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
A total of 1 test files matched the specified pattern.
A total of 1 test files matched the specified pattern.
WARNING: Overwriting results file: /Users/xxx/code/result/test-explorer-K8WIVk/0.trx
Results File: /Users/xxx/code/result/test-explorer-K8WIVk/0.trx
A total of 1 test files matched the specified pattern.
No test matches the given testcase filter `FullyQualifiedName~DemoManagement.Application.Tests.ObjectExtensions` in /Users/xxx/code/DemoManagement/DemoManagement.Domain.Tests/bin/Debug/netcoreapp3.1/DemoManagement.Domain.Tests.dll
WARNING: Overwriting results file: /Users/xxx/code/result/test-explorer-K8WIVk/0.trx
Results File: /Users/xxx/code/result/test-explorer-K8WIVk/0.trx
[xUnit.net 00:00:01.15] DemoManagement.Application.Tests.ObjectExtensions.StringExtensionTests.WhiteSpace_ShouldReturnNull_WhenCalledOnWhiteSpace [FAIL]
No test matches the given testcase filter `FullyQualifiedName~DemoManagement.Application.Tests.ObjectExtensions` in /Users/xxx/code/DemoManagement/DemoManagement.Design.Tests/bin/Debug/netcoreapp3.1/DemoManagement.Design.Tests.dll
X DemoManagement.Application.Tests.ObjectExtensions.StringExtensionTests.WhiteSpace_ShouldReturnNull_WhenCalledOnWhiteSpace [3ms]
Error Message:
Assert.Null() Failure
Expected: (null)
Actual: 34
Stack Trace:
at DemoManagement.Application.Tests.ObjectExtensions.StringExtensionTests.WhiteSpace_ShouldReturnNull_WhenCalledOnWhiteSpace() in /Users/xxx/code/DemoManagement/DemoManagement.Application.Tests/ObjectExtensions/StringExtensionTests.cs:line 17
WARNING: Overwriting results file: /Users/xxx/code/result/test-explorer-K8WIVk/0.trx
Results File: /Users/xxx/code/result/test-explorer-K8WIVk/0.trx
Test Run Failed.
Total tests: 1
Failed: 1
Total time: 1.9760 Seconds
/usr/local/share/dotnet/sdk/3.1.405/Microsoft.TestPlatform.targets(32,5): error MSB4181: The "Microsoft.TestPlatform.Build.Tasks.VSTestTask" task returned false but did not log an error. [/Users/xxx/code/DemoManagement/DemoManagement.Application.Tests/DemoManagement.Application.Tests.csproj]
No test matches the given testcase filter `FullyQualifiedName~DemoManagement.Application.Tests.ObjectExtensions` in /Users/xxx/code/DemoManagement/DemoManagement.IntegrationTests/bin/Debug/netcoreapp3.1/DemoManagement.IntegrationTests.dll
WARNING: Overwriting results file: /Users/xxx/code/result/test-explorer-K8WIVk/0.trx
Results File: /Users/xxx/code/result/test-explorer-K8WIVk/0.trx
WARNING: Overwriting results file: /Users/xxx/code/result/test-explorer-K8WIVk/0.trx
Results File: /Users/xxx/code/result/test-explorer-K8WIVk/0.trx
And the result in 0.trx is the result of last project, instead of the sum of them.
Related on Stack Overflow: .net test explorer just run one of many test projects i have in solution file in vscode
This seems to be a problem within the dotnet test command, since this also occurs when executing the command from cli