advancedtelematic / quickcheck-state-machine

Test monadic programs using state machine based models

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Property Result in case of exceptions

kderme opened this issue · comments

At the moment runParallelCommandsNTimes and prettyParallelCommands return a passing Result even when there is some thrown exception on semantics. Maybe we should discuss which should be the default behaviour.

Also the equivalent NParallel.. commands may encounter executeCommands: impossible (for example when an exception during a MemoryReferene Create fails) because this #317 is not adopted yet.

What do you mean by "return a passing Result"?

Also the equivalent NParallel.. commands may encounter executeCommands: impossible (for example when an exception during a MemoryReferene Create fails) because this #317 is not adopted yet.

Yes, we should implement the analogue of #317 for the NParallel... functions.

What do you mean by "return a passing Result"?

The property doesn't fail

The parallel property shouldn't necessarily fail if an exception is thrown on one of the threads. It should only fail if we cannot complete the history (#317) in any way that linearises.

Arguably we should either make runParallelCommands' (which completes histories) the default way to run the parallel property and remove runParallelCommands, or keep both runParallelCommands and runParallelCommands' and have runParallelCommands fail if there's any exception (since that function doesn't complete histories).

Is that what you meant or did you mean something else?

Another way to think of it is that runParallelCommands should be used when there's no fault injection (exceptions), while runParallelCommands' should be used when there's fault injection.

Yes, this is what I meant. I personally agree a bit more with the second option: make runParallelCommands fail on exceptions, since it doesn't complete histories.

Another way to think of it is that runParallelCommands should be used when there's no fault injection (exceptions), while runParallelCommands' should be used when there's fault injection.

Yes exactly.

I personally agree a bit more with the second option...

Yeah, I think you're right -- that's probably the better option.