Giters

microsoft / testfx

MSTest framework and adapter

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

A test method using Activator.CreateInstance that raises an exception is not failing and is showing a "Not Run" status.

pablorivassanchez opened this issue · comments

commented

Describe the bug

I'm testing a piece of code that uses Activator.CreateInstance. Although it fails, the test method status is showing as "Not Run".

Steps To Reproduce

[TestMethod]
public void CleanData_AnonymousObject()
{
    var input = new
    {
        Field1 = "StringWith\0Null",
        Field2 = "NormalString",
    };
    var cleanedObject = Activator.CreateInstance(input.GetType());
}

Expected behavior

The test fail and the status of the test is fail

Actual behavior

The test status is Not Run

Additional context

With XUnit the test fail because there is the following exception

Cannot dynamically create an instance of type '<>f__AnonymousType23[System.String,System.String,<>f__AnonymousType31[System.String]]'. Reason: No parameterless constructor defined.

commented

I can repro under vstest using dotnet test + dll,

dotnet test S:\t\mstest145\bin\Debug\net9.0\mstest145.dll
VSTest version 17.11.0-preview-24265-01 (x64)

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
  Skipped TestMethod1 [28 ms]

None     - Failed:     0, Passed:     0, Skipped:     0, Total:     0, Duration: 145 ms - mstest145.dll (net9.0)

Running with straight testing platform does not repro:

dotnet test
Restore complete (0,4s)
You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy
  mstest145 succeeded (3,5s) → bin\Debug\net9.0\mstest145.dll
  mstest145 failed with 2 error(s) (1,2s)
    C:\Users\jajares\.nuget\packages\microsoft.testing.platform.msbuild\1.1.0\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(228,5): error : Failed test 'TestMethod1', reason 'Cannot dynamically create an instance of type '<>f__AnonymousType0`2[System.String,System.String]'. Reason: No parameterless constructor defined.'
    C:\Users\jajares\.nuget\packages\microsoft.testing.platform.msbuild\1.1.0\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(228,5): error : Tests failed: 'S:\t\mstest145\bin\Debug\net9.0\TestResults\mstest145_net9.0_x64.log' [net9.0|x64]

So this looks like vstest bridge bug on a first look.

commented

The code does not check for hot reload mode, as far as I can see. And even in hot reload mode this issue would repro, because the test fails with method missing exception.

commented

If you debug this, simply set BP to the test in VS, and once you hit it, go up until you reach the offending code.

commented

Ok, thank you. So the test will not fail for now and will remain in a "Not Run" state until the next milestone, which is scheduled for June 17, correct?

commented

@pablorivassanchez We had to postpone the release of v3.4 so I will have to postpone 3.5 too (probably end of June). Please let us know if that's critical for you so we can investigate releasing a patch for v3.4

commented

The code does not check for hot reload mode, as far as I can see. And even in hot reload mode this issue would repro, because the test fails with method missing exception.

We should definitely fix the check for being in Hot Reload mode. For the other case, being already in Hot Reload, I think this is going to be tricky as we probably have no way to understand if that's expected or not.

commented

@pablorivassanchez We had to postpone the release of v3.4 so I will have to postpone 3.5 too (probably end of June). Please let us know if that's critical for you so we can investigate releasing a patch for v3.4

it's ok for my only 1 of our set of tests is affected