hedgehogqa / haskell-hedgehog

Release with confidence, state-of-the-art property testing for Haskell.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Async exceptions should be handled differently?

chris-martin opened this issue · comments

I often leave ghcid running with the --test flag to continually re-run tests. When I'm editing files more rapidly than the tests can run, there are a lot of occasions when ghcid will kill the test runner before it's finished. When the happens, I see some arbitrary hedgehog test failure pop up, with its beautiful annotated exception view, pointing at whatever property happened to be checking at the moment when the testing was aborted.

It seems to me that the fancy error output should not be printed when the exception is async, because an async exception does not actually indicate that a property failed, but merely that the user no longer wants to run the test. It should just say that the testing was aborted.

I may be wrong about this, of course - Are there actually situations where a test would legitimately fail by throwing an async exception?

This isn't a problem for me anymore, but it did create some amount of anxiety for a while before I understood what was happening, as I believed I was seeing flashes of test failures that could not reliably be reproduced.

Yes, I believe there are times. Suppose, for example, that the code being tested deadlocks on an MVar or an STM transaction.

Hmmm, I see. Okay, I suppose this is a bad idea then.

Jfyi, those aren't actually asynchronous exceptions, @treeowl! Nonetheless I think I agree that they should not be treated specially