Rework verification error handling and logging
YevgeniyShunevych opened this issue · comments
Error handling
Verification error handling and logging was reviewed and reworked. There was a need to distinguish default Atata native assertion failure handling and the handling for NUnit. Similar to how it is done for warnings.
The new IAssertionFailureReportStrategy appeared with 2 implementations: default AtataAssertionFailureReportStrategy and NUnitAssertionFailureReportStrategy.
New methods added to AtataContextBuilder:
AtataContextBuilder UseAssertionFailureReportStrategy(IAssertionFailureReportStrategy strategy);
AtataContextBuilder UseNUnitAssertionFailureReportStrategy();UseNUnitAssertionFailureReportStrategy is called by UseAllNUnitFeatures method, so mostly there is no need for extra configuration.
Logging
Until now, assertion error is logged exactly inside "Assert" log section, before that it was logged in "Deinitialization" log section. Also all exceptions occurred in "Step", "Setup" and "Aggregate assert" sections are now logged inside these sections. Aggregate assertion exception is now not logged as an error in "Deinitialization" section to avoid duplication of failures in log, because they are logged during the test execution one by one. In all other cases unhandled exceptions will continue to be written to log in "Deinitialization" log section.
Other
A blank line added between multiple errors in aggregate assertion exception message for NUnit, as below:
Multiple failures or warnings in test:
1) Unexpected sut
Expected: should contain "x"
Actual: "abc"
2) Unexpected sut
Expected: should contain "y"
Actual: "abc"
3) Wrong sut
Expected: should contain "z"
Actual: "abc"