SimonCropp / XunitContext

Extends xUnit to expose extra context and simplify logging

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

XunitContext

Build status NuGet Status

Extends xUnit to expose extra context and simplify logging.

Redirects Trace.Write, Debug.Write, and Console.Write and Console.Error.Write to ITestOutputHelper. Also provides static access to the current ITestOutputHelper for use within testing utility methods.

Uses AsyncLocal to track state.

See Milestones for release notes.

NuGet package

https://nuget.org/packages/XunitContext/

ClassBeingTested

static class ClassBeingTested
{
    public static void Method()
    {
        Trace.WriteLine("From Trace");
        Console.WriteLine("From Console");
        Debug.WriteLine("From Debug");
        Console.Error.WriteLine("From Console Error");
    }
}

snippet source | anchor

XunitContextBase

XunitContextBase is an abstract base class for tests. It exposes logging methods for use from unit tests, and handle the flushing of logs in its Dispose method. XunitContextBase is actually a thin wrapper over XunitContext. XunitContexts Write* methods can also be use inside a test inheriting from XunitContextBase.

public class TestBaseSample(ITestOutputHelper output) :
    XunitContextBase(output)
{
    [Fact]
    public void Write_lines()
    {
        WriteLine("From Test");
        ClassBeingTested.Method();

        var logs = XunitContext.Logs;

        Assert.Contains("From Test", logs);
        Assert.Contains("From Trace", logs);
        Assert.Contains("From Debug", logs);
        Assert.Contains("From Console", logs);
        Assert.Contains("From Console Error", logs);
    }
}

snippet source | anchor

xunit Fixture

In addition to XunitContextBase class approach, one is also possible to use IContextFixture to gain access to XunitContext :

public class FixtureSample(ITestOutputHelper helper, ContextFixture ctxFixture) :
    IContextFixture
{
    Context context = ctxFixture.Start(helper);

    [Fact]
    public void Usage()
    {
        Console.WriteLine("From Test");
        Assert.Contains("From Test", context.LogMessages);
    }
}

snippet source | anchor

Logging

XunitContext provides static access to the logging state for tests. It exposes logging methods for use from unit tests, however registration of ITestOutputHelper and flushing of logs must be handled explicitly.

public class XunitLoggerSample :
    IDisposable
{
    [Fact]
    public void Usage()
    {
        XunitContext.WriteLine("From Test");

        ClassBeingTested.Method();

        var logs = XunitContext.Logs;

        Assert.Contains("From Test", logs);
        Assert.Contains("From Trace", logs);
        Assert.Contains("From Debug", logs);
        Assert.Contains("From Console", logs);
        Assert.Contains("From Console Error", logs);
    }

    public XunitLoggerSample(ITestOutputHelper testOutput) =>
        XunitContext.Register(testOutput);

    public void Dispose() =>
        XunitContext.Flush();
}

snippet source | anchor

XunitContext redirects Trace.Write, Console.Write, and Debug.Write in its static constructor.

Trace.Listeners.Clear();
Trace.Listeners.Add(new TraceListener());
#if (NETFRAMEWORK)
Debug.Listeners.Clear();
Debug.Listeners.Add(new TraceListener());
#else
DebugPoker.Overwrite(
    text =>
    {
        if (string.IsNullOrEmpty(text))
        {
            return;
        }

        if (text.EndsWith(Environment.NewLine))
        {
            WriteLine(text.TrimTrailingNewline());
            return;
        }

        Write(text);
    });
#endif
TestWriter writer = new();
Console.SetOut(writer);
Console.SetError(writer);

snippet source | anchor

These API calls are then routed to the correct xUnit ITestOutputHelper via a static AsyncLocal.

Logging Libs

Approaches to routing common logging libraries to Diagnostics.Trace:

Filters

XunitContext.Filters can be used to filter out unwanted lines:

public class FilterSample(ITestOutputHelper output) :
    XunitContextBase(output)
{
    static FilterSample() =>
        Filters.Add(_ => _ != null && !_.Contains("ignored"));

    [Fact]
    public void Write_lines()
    {
        WriteLine("first");
        WriteLine("with ignored string");
        WriteLine("last");
        var logs = XunitContext.Logs;

        Assert.Contains("first", logs);
        Assert.DoesNotContain("with ignored string", logs);
        Assert.Contains("last", logs);
    }
}

snippet source | anchor

Filters are static and shared for all tests.

Context

For every tests there is a contextual API to perform several operations.

  • Context.TestOutput: Access to ITestOutputHelper.
  • Context.Write and Context.WriteLine: Write to the current log.
  • Context.LogMessages: Access to all log message for the current test.
  • Counters: Provide access in predicable and incrementing values for the following types: Guid, Int, Long, UInt, and ULong.
  • Context.Test: Access to the current ITest.
  • Context.SourceFile: Access to the file path for the current test.
  • Context.SourceDirectory: Access to the directory path for the current test.
  • Context.SolutionDirectory: The current solution directory. Obtained by walking up the directory tree from SourceDirectory.
  • Context.TestException: Access to the exception if the current test has failed. See Test Failure.

// ReSharper disable UnusedVariable

public class ContextSample(ITestOutputHelper output) :
    XunitContextBase(output)
{
    [Fact]
    public void Usage()
    {
        Context.WriteLine("Some message");

        var currentLogMessages = Context.LogMessages;

        var testOutputHelper = Context.TestOutput;

        var currentTest = Context.Test;

        var sourceFile = Context.SourceFile;

        var sourceDirectory = Context.SourceDirectory;

        var solutionDirectory = Context.SolutionDirectory;

        var currentTestException = Context.TestException;
    }
}

snippet source | anchor

Some members are pushed down to the be accessible directly from XunitContextBase:

// ReSharper disable UnusedVariable

public class ContextPushedDownSample(ITestOutputHelper output) :
    XunitContextBase(output)
{
    [Fact]
    public void Usage()
    {
        WriteLine("Some message");

        var currentLogMessages = Logs;

        var testOutputHelper = Output;

        var sourceFile = SourceFile;

        var sourceDirectory = SourceDirectory;

        var solutionDirectory = SolutionDirectory;

        var currentTestException = TestException;
    }
}

snippet source | anchor

Context can accessed via a static API:

// ReSharper disable UnusedVariable

public class ContextStaticSample(ITestOutputHelper output) :
    XunitContextBase(output)
{
    [Fact]
    public void StaticUsage()
    {
        XunitContext.Context.WriteLine("Some message");

        var currentLogMessages = XunitContext.Context.LogMessages;

        var testOutputHelper = XunitContext.Context.TestOutput;

        var currentTest = XunitContext.Context.Test;

        var sourceFile = XunitContext.Context.SourceFile;

        var sourceDirectory = XunitContext.Context.SourceDirectory;

        var solutionDirectory = XunitContext.Context.SolutionDirectory;

        var currentTestException = XunitContext.Context.TestException;
    }
}

snippet source | anchor

Current Test

There is currently no API in xUnit to retrieve information on the current test. See issues #1359, #416, and #398.

To work around this, this project exposes the current instance of ITest via reflection.

Usage:

// ReSharper disable UnusedVariable

public class CurrentTestSample(ITestOutputHelper output) :
    XunitContextBase(output)
{
    [Fact]
    public void Usage()
    {
        var currentTest = Context.Test;
        // DisplayName will be 'CurrentTestSample.Usage'
        var displayName = currentTest.DisplayName;
    }

    [Fact]
    public void StaticUsage()
    {
        var currentTest = XunitContext.Context.Test;
        // DisplayName will be 'CurrentTestSample.StaticUsage'
        var displayName = currentTest.DisplayName;
    }
}

snippet source | anchor

Implementation:

namespace Xunit;

public partial class Context
{
    ITest? test;


    public ITest Test
    {
        get
        {
            InitTest();

            return test!;
        }
    }

    MethodInfo? methodInfo;

    public MethodInfo MethodInfo
    {
        get
        {
            InitTest();
            return methodInfo!;
        }
    }

    Type? testType;

    public Type TestType
    {
        get
        {
            InitTest();
            return testType!;
        }
    }

    void InitTest()
    {
        if (test != null)
        {
            return;
        }

        if (TestOutput == null)
        {
            throw new(MissingTestOutput);
        }

#if NET8_0_OR_GREATER
        [UnsafeAccessor(UnsafeAccessorKind.Field, Name = "test")]
        static extern ref ITest GetTest(TestOutputHelper? c);
        test = GetTest((TestOutputHelper) TestOutput);
#else
        test = (ITest) GetTestMethod(TestOutput)
            .GetValue(TestOutput)!;
#endif
        var method = (ReflectionMethodInfo) test.TestCase.TestMethod.Method;
        var type = (ReflectionTypeInfo) test.TestCase.TestMethod.TestClass.Class;
        methodInfo = method.MethodInfo;
        testType = type.Type;
    }

    public const string MissingTestOutput = "ITestOutputHelper has not been set. It is possible that the call to `XunitContext.Register()` is missing, or the current test does not inherit from `XunitContextBase`.";

#if !NET8_0_OR_GREATER
    static FieldInfo? cachedTestMember;

    static FieldInfo GetTestMethod(ITestOutputHelper testOutput)
    {
        if (cachedTestMember != null)
        {
            return cachedTestMember;
        }

        var testOutputType = testOutput.GetType();
        cachedTestMember = testOutputType.GetField("test", BindingFlags.Instance | BindingFlags.NonPublic);
        if (cachedTestMember == null)
        {
            throw new($"Unable to find 'test' field on {testOutputType.FullName}");
        }

        return cachedTestMember;
    }
#endif
}

snippet source | anchor

Test Failure

When a test fails it is expressed as an exception. The exception can be viewed by enabling exception capture, and then accessing Context.TestException. The TestException will be null if the test has passed.

One common case is to perform some logic, based on the existence of the exception, in the Dispose of a test.

// ReSharper disable UnusedVariable
public static class GlobalSetup
{
    [ModuleInitializer]
    public static void Setup() =>
        XunitContext.EnableExceptionCapture();
}

public class TestExceptionSample(ITestOutputHelper output) :
    XunitContextBase(output)
{
    [Fact(Skip = "Will fail")]
    public void Usage() =>
        //This tests will fail
        Assert.False(true);

    public override void Dispose()
    {
        var theExceptionThrownByTest = Context.TestException;
        var testDisplayName = Context.Test.DisplayName;
        var testCase = Context.Test.TestCase;
        base.Dispose();
    }
}

snippet source | anchor

Base Class

When creating a custom base class for other tests, it is necessary to pass through the source file path to XunitContextBase via the constructor.

public class CustomBase(
    ITestOutputHelper testOutput,
    [CallerFilePath] string sourceFile = "")
    :
        XunitContextBase(testOutput, sourceFile);

snippet source | anchor

Parameters

Provided the parameters passed to the current test when using a [Theory].

Use cases:

Usage:

public class ParametersSample(ITestOutputHelper output) :
    XunitContextBase(output)
{
    [Theory]
    [MemberData(nameof(GetData))]
    public void Usage(string arg)
    {
        var parameter = Context.Parameters.Single();
        var parameterInfo = parameter.Info;
        Assert.Equal("arg", parameterInfo.Name);
        Assert.Equal(arg, parameter.Value);
    }

    public static IEnumerable<object[]> GetData()
    {
        yield return ["Value1"];
        yield return ["Value2"];
    }
}

snippet source | anchor

Implementation:

static List<Parameter> GetParameters(ITestCase testCase) =>
    GetParameters(testCase, testCase.TestMethodArguments);

static List<Parameter> GetParameters(ITestCase testCase, object[] arguments)
{
    var method = testCase.TestMethod;
    var infos = method
        .Method.GetParameters()
        .ToList();
    if (arguments == null || arguments.Length == 0)
    {
        if (infos.Count == 0)
        {
            return empty;
        }

        throw NewNoArgumentsDetectedException();
    }

    List<Parameter> items = [];

    for (var index = 0; index < infos.Count; index++)
    {
        items.Add(new(infos[index], arguments[index]));
    }

    return items;
}

snippet source | anchor

Complex parameters

Only core types (string, int, DateTime etc) can use the above automated approach. If a complex type is used the following exception will be thrown

No arguments detected for method with parameters. This is most likely caused by using a parameter that Xunit cannot serialize. Instead pass in a simple type as a parameter and construct the complex object inside the test. Alternatively; override the current parameters using UseParameters() via the current test base class, or via XunitContext.Current.UseParameters().

To use complex types override the parameter resolution using XunitContextBase.UseParameters:

public class ComplexParameterSample(ITestOutputHelper output) :
    XunitContextBase(output)
{
    [Theory]
    [MemberData(nameof(GetData))]
    public void UseComplexMemberData(ComplexClass arg)
    {
        UseParameters(arg);
        var parameter = Context.Parameters.Single();
        var parameterInfo = parameter.Info;
        Assert.Equal("arg", parameterInfo.Name);
        Assert.Equal(arg, parameter.Value);
    }

    public static IEnumerable<object[]> GetData()
    {
        yield return [new ComplexClass("Value1")];
        yield return [new ComplexClass("Value2")];
    }

    public class ComplexClass(string value)
    {
        public string Value { get; } = value;
    }
}

snippet source | anchor

UniqueTestName

Provided a string that uniquely identifies a test case.

Usage:

public class UniqueTestNameSample(ITestOutputHelper output) :
    XunitContextBase(output)
{
    [Fact]
    public void Usage()
    {
        var testName = Context.UniqueTestName;

        Context.WriteLine(testName);
    }
}

snippet source | anchor

Implementation:

string GetUniqueTestName(ITestCase testCase)
{
    var method = testCase.TestMethod;
    var name = $"{method.TestClass.Class.ClassName()}.{method.Method.Name}";
    if (!Parameters.Any())
    {
        return name;
    }

    var builder = new StringBuilder($"{name}_");
    foreach (var parameter in Parameters)
    {
        builder.Append($"{parameter.Info.Name}=");
        builder.Append(string.Join(",", SplitParams(parameter.Value)));
        builder.Append('_');
    }

    builder.Length -= 1;

    return builder.ToString();
}

static IEnumerable<string> SplitParams(object? parameter)
{
    if (parameter == null)
    {
        yield return "null";
        yield break;
    }

    if (parameter is string stringValue)
    {
        yield return stringValue;
        yield break;
    }

    if (parameter is IEnumerable enumerable)
    {
        foreach (var item in enumerable)
        {
            foreach (var sub in SplitParams(item))
            {
                yield return sub;
            }
        }

        yield break;
    }

    var toString = parameter.ToString();
    if (toString == null)
    {
        yield return "null";
    }
    else
    {
        yield return toString;
    }
}

snippet source | anchor

Global Setup

Xunit has no way to run code once before any tests executing. So use one of the following:

Icon

Wolverine designed by Mike Rowe from The Noun Project.

About

Extends xUnit to expose extra context and simplify logging

License:MIT License


Languages

Language:C# 100.0%