SpecFlow plugin for using property-based testing with FsCheck in SpecFlow scenarios.
Currently supports
- SpecFlow v2.1
- FsCheck v2.2.4 or above
License: Apache (https://github.com/gasparnagy/SpecFlow.FsCheck/blob/master/LICENSE)
NuGet: https://www.nuget.org/packages/SpecFlow.FsCheck
Check the complete example at https://github.com/gasparnagy/SpecFlow.FsCheck/tree/master/sample/AdditionSample.
Install plugin from NuGet into your SpecFlow project.
PM> Install-Package SpecFlow.FsCheck
Describe your scenarios that describe a rule instead of the example and tag them with @propertyBased tag, like this:
@propertyBased
Scenario: Addition - Identity property
a + 0 == a
Given I have entered any number into the calculator
And I have entered 0 into the calculator
When I press add
Then the result should be the first number on the screen
Create a binding class to describe the constraints for the parameters (any number, the first number) as [StepArgumentTransformation]. Inherit this class from the SpecFlow.FsCheck.ConstraintsBase class.
[Binding]
public class Constraints : ConstraintsBase
{
[StepArgumentTransformation("any number")]
public int AnyNumber()
{
return AsParam("any", Arb.Default.Int32());
//could be constrainded: AsParam("any", Gen.Choose(0, 100));
}
[StepArgumentTransformation("the first number")]
public int TheFirstNumber()
{
return AsFormula(actualParams => (int)actualParams.First());
}
}
Run the test generated from the scenario. The scenario steps will be executed multiple times with different input parameters. The random input parameters are generated by FsCheck. Once FsCheck finds a failing example, it even tries to narrow down the input space (called shrinking) to find out what is the first number that makes the test fail.
Test Name: Addition_IdentityProperty
Test Outcome: Failed
Result Message:
TestCleanup method MyCalculator.Tests.AdditionFeature.ScenarioTearDown threw exception.
System.Exception: System.Exception: Falsifiable, after 84 tests (3 shrinks) (StdGen (937639104,296219540)):
Original:
59
Shrunk:
38
with exception:
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.AreEqual failed. Expected:<38>. Actual:<76>.
...
You should expect the tests to be executed many (>50) times, so be careful with slow tests.