LoggingTestSet

[notinaboat]

I came up with this as a way to include test results in my log when using TeeLogger(ConsoleLogger(), FileLogger("my.log")).

LoggingTestSet wraps DefaultTestSet and does an @error log for each failed test and an @info summary at the end.
All the usual console test output is retained.

Do you think this would be a useful addition to LoggingExtras?
I could submit a PR (or you're welcome to tweak the code below to your taste and just commit it).

using Test
using Test: DefaultTestSet, AbstractTestSet, Fail, Error, scrub_backtrace

struct LoggingTestSet <: AbstractTestSet
    ts::DefaultTestSet
    LoggingTestSet(args...; kw...) = new(DefaultTestSet(args...; kw...))
end

function Test.record(ts::LoggingTestSet, t::Union{Fail, Error})

    io = IOBuffer()

    begin # copied from: stdlib/Test/src/Test.jl https://git.io/JqZQk
        print(io, ts.ts.description, ": ")
        # don't print for interrupted tests
        if !(t isa Error) || t.test_type !== :test_interrupted
            print(io, t)
            if !isa(t, Error) # if not gets printed in the show method
                Base.show_backtrace(io, scrub_backtrace(backtrace()))
            end
            println(io)
        end
    end

    @error String(take!(io))

    Test.record(ts.ts, t)
end
Test.record(ts::LoggingTestSet, args...) = Test.record(ts.ts, args...)

function Test.finish(ts::LoggingTestSet)

    # Copied from: stdlib/Test/src/Test.jl https://git.io/JqZ70
    np, nf, ne, nb, ncp, ncf, nce, ncb = Test.get_test_counts(ts.ts)
    passes = np + ncp
    fails  = nf + ncf
    errors = ne + nce
    broken = nb + ncb

    @info "Test Summary: $(ts.ts.description)" passes fails errors broken

    try
        Test.finish(ts.ts)
    catch err
        @error err
        rethrow(err)
    end
end

[notinaboat]

https://github.com/notinaboat/LoggingTestSets.jl

[Frames White]

Cool idea, but
I don't think it is suitable for this package.
Primary, focus here is on compositional loggers.
Secondary focus is on logging sinks.
This is neither, I think a stand lone package like you have created is better.