Test Helper: TestProcessWrapper

Launch and control dotnet processes wrapped into the coverlet code coverage analyzer.

The class TestProcessWrapper is intended to launch one ore more dotnet processes for performing acceptance tests. The class captures the messages written to the Console and to Console.Error. It allows to terminate the process gracefully and forcefully. One of the processes can be wrapped by the coverlet command line tool in order to calculate code coverage.

Usage Examples

The most simple use is described by the acceptance test SmokeTests.cs.

You can find detailed usage examples in the Acceptance Test Suite (BDD).

First read a Gherkin .feature file from the Features folder. It explains why each feature exists and which use scenarios are addressed. Then read the corresponding *StepDefinition.cs file in the Steps folder. It shows how the test steps from the feature file (given, when, then) are actually implemented. Note, that some frequently used steps are implemented in the SharedStepDefinitions folder.

Attention

You can use the coverlet wrapper only once per dotnet application, because coverlet instruments the dotnet DLL. If you use coverlet with two or more instances of the same application, coverlet will report an exception after (or during) the application termination and the reported coverage will be 0.

Development and Support Standard

This project is incomplete and not production ready.

I am developing during my spare time and use this project for learning purposes. Please assume that I will need some days to answer your questions. Please keep this in mind when using this project in a production environment.

Thanks

Many thanks to JetBrains who provide an Open Source License for this project ❤️.

Development

Prerequisites

To compile, test and run this project the latest .NET SDK is required on your machine. For calculating code metrics I recommend metrix++. This requires Python. If you'd like to contribute, then please use the dotnet format command as described below.

To use the RemoteControlledProcess library and to run the unit tests you need the following tools installed:

dotnet tool install --global coverlet.console --configfile NuGet-OfficialOnly.config
dotnet tool install --global dotnet-reportgenerator-globaltool --configfile NuGet-OfficialOnly.config

Troubleshooting

If you are installing a dotnet tool for the first time, then you'll need to add the path to the dotnet tools to your system PATH. Please make sure that there is no "~" character in your PATH to coverlet.

E.g. add the following line to the end of your shell rc file (e.g. ~/.zshrc):

export PATH="$PATH:$HOME/.dotnet/tools"

Build, Test, Run

Run the following commands from the folder containing the RemoteControlledProcess.sln file in order to build, test and run the application:

Build the Solution and Run the Acceptance Tests

dotnet build

# Simply run the tests
dotnet test

# As an alternative, run the tests with coverage and produce a coverage report
rm -r RemoteControlledProcess.Acceptance.Tests/TestResults && \
  dotnet test --no-restore --verbosity normal /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput='./TestResults/coverage.cobertura.xml' && \
  reportgenerator "-reports:RemoteControlledProcess.Acceptance.Tests/TestResults/*.xml" "-targetdir:RemoteControlledProcess.Acceptance.Tests/TestResults/report" "-reporttypes:Html;lcov" "-title:RemoteControlledProcess"
open RemoteControlledProcess.Acceptance.Tests/TestResults/report/index.html

The script build.sh builds the NuGet package like the build pipeline does it. This can be helpful when debugging issues popping up in the build pipeline.

Known Issue

When you run the tests on a mac, then the tests using two TestProcessWrappers issue a crash report regarding "dotnet".

At the moment I cannot explain that behavior.

Before Creating a Pull Request ...

... apply code formatting rules

dotnet format

... and check code metrics using metrix++

# Collect metrics
metrix++ collect --std.code.complexity.cyclomatic --std.code.lines.code --std.code.todo.comments --std.code.maintindex.simple -- .

# Get an overview
metrix++ view --db-file=./metrixpp.db

# Apply thresholds
metrix++ limit --db-file=./metrixpp.db --max-limit=std.code.complexity:cyclomatic:5 --max-limit=std.code.lines:code:25:function --max-limit=std.code.todo:comments:0 --max-limit=std.code.mi:simple:1

At the time of writing, I want to stay below the following thresholds:

--max-limit=std.code.complexity:cyclomatic:5
--max-limit=std.code.lines:code:25:function
--max-limit=std.code.todo:comments:0
--max-limit=std.code.mi:simple:1

I allow generated files named *.feature.cs to exceed these thresholds.

Finally, remove all code duplication. The next section describes how to detect code duplication.

Identify Code Duplication

The tools\dupfinder.bat or tools/dupfinder.sh file calls the JetBrains dupfinder tool and creates an HTML report of duplicated code blocks in the solution directory.

In order to use the dupfinder you need to globally install the JetBrains ReSharper Command Line Tools On Unix like operating systems you also need xsltproc, which is pre-installed on macOS.

From the folder containing the .sln file run

tools\dupfinder.bat

tools/dupfinder.sh

respectively.

The report will be created as dupfinder-report.html in the current directory.

References

alex-held / RemoteControlledProcess