advancedtelematic / quickcheck-state-machine

Test monadic programs using state machine based models

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add pretty print for CounterExample and improve test output

kderme opened this issue · comments

tests print things like

	AnnotateC "Read" (PredicateC (1 :/= 2))
	PostconditionFailed "AnnotateC \"Read\" (PredicateC (6 :/= 5))" /= Ok

which may not be very helpful, or just hard to understand for the user

Also drawing for CrashAndLogic is broken for some reason.

Yes, this can be improved for sure!

I once had the idea that a nice output could be obtained by combining the initial Logic formula and the Counterexample that the formula evaluated to, e.g. something like:

prettyPrintCounterexample :: Logic -> Counterexample -> String
prettyPrintCounterexample l@(p :&& q) (Fst ce) = concat
  [ "When checking that: "
  , show l
  , " is true, "
  , "the first component is false, because: "
  , prettyPrintCounterexample p ce
  ]

eval :: Logic -> Either String ()
eval l = case logic l of
  VTrue -> Right ()
  VFalse ce -> Left (prettyPrintCounterexample l ce)

I think the tricky part is how to not be too verbose and yet precise. Perhaps underlining the component of the formula that was false? Perhaps one should check how deep the formula is and only show parts of the tree? What do other tools do?