JUnit formatter doesn't play nice with test.check
jcf opened this issue · comments
Hey James,
Because both the JUnit formatter and test.check print to test out you end up with hash maps in the XML output.
[snip]
</testcase>{:result true, :num-tests 100, :seed 1500392295920, :test-var "t-params->map-keys"}
<testcase name="t-params->map-keys" classname=[snip]
Maybe logging to a file will work from the REPL, but there's not option to log to a file when configuring lein-eftest as far as I can see.
I can maybe throw together a PR this week if you have a preferred solution in mind.
How is test.check printing these things? A cursory look at the source code suggests its doing so via the test reporter, so couldn't we add methods for the extra events that test.check generates?
That we could. I was just looking into adding something to my own codebase, but ultimately maybe that belongs in eftest?
It looks like these are the events emitted by test.check:
:clojure.test.check.clojure-test/trial:clojure.test.check.clojure-test/shrinking:clojure.test.check.clojure-test/shrunk
In theory we could extend the current Eftest reporter multimethods with support for the extra test.check events. For the junit reporter, presumably we just want to ignore them.
For the junit reporter, presumably we just want to ignore them.
I guess so. If you're using clojure.test.check.clojure-test/defspec the test failures will be reported via clojure.test itself as you can see if the XML snippet I posted above.
For the other multimethods you refer to, I'm not sure. I always use defspec but maybe others want the test.check reports to show up somewhere?
For now, what about ignoring the reports for the junit reporter, and just deferring to the default for the others (which should require no change?). That would solve the immediate problem of bad data in the junit output.
Whoops! Sorry. I'll add the test.check events to the JUnit reported via GitHub.