Option to only show a dot for green tests
Firesphere opened this issue · comments
The output is pretty long and easily overlooked. An option to only output failed tests or tests that take a long time would be a good asset.
Unfortunately there is currently no way to specify printer options in PHPUnit. I think the linked issue would need to be resolved before considering this.
Could you have a couple of pre-configured printers made available as separate classes in the module? E.g. provide ScriptFUSION\PHPUnitImmediateExceptionPrinter\TerseImmediateExceptionPrinter as well, which is a subclass of ImmediateExceptionPrinter that changes some parameters.
I agree that it would be nicer if it was fixed up-stream, but pending that, would it make sense to work with what we have?
We could definitely ship lots of specializations but that's a maintenance nightmare. One guy wants to replace success lines with just a dot. Another guy wants no output at all for successful tests. Very soon we have a class explosion and then a long list of BC breaks when we delete them all because the upstream issue is fixed. I think it should be a priority to create a PR to pass options to printers in PHPUnit so we can configure them to our heart's content and make everyone happy in a maintainable way.
As an aside, the idea to have no output at all for success cases and then to integrate that with Travis is a perilous one that I can strongly advise against because Travis has a stalled build detection feature that activates when there isn't any output within a given time limit. In the typical case where your build has no errors you will see absolutely no output which is not only a poor user experience but has the potential to crash your build due to timeouts. Ergo the dot idea was better.
Sorry to put my comment at silverstripe/silverstripe-framework#6945 (comment) in context, I meant to "print VERBOSE information on errors", not suppress all output. :)
At the moment the only information we need for success is that it passed; The current "." (green dot) is fine for this in our case.
I left a relevant comment at silverstripe/silverstripe-framework#6941 (comment)
We could definitely ship lots of specializations but that's a maintenance nightmare.
All that we need is the ability to customise the printer; A great outcome to this solution could be, rather than a pre-configured printer, maybe a simple "how to subclass and override the core printer class" with a simple example, perhaps a markdown file linked to from the root index?
On second thoughts, core documentation may not even be necessary, if you wanted to keep the focus of your module pretty well defined.
If you could post an example of how to (as I would guess would be required) customise onEndTest from https://github.com/ScriptFUSION/PHPUnit-Immediate-Exception-Printer/blob/master/src/Printer.php that would be enough for us, as well as others who look up this ticket. :)
Unfortunately there is currently [no way to specify printer options
In absence of any infrastructure provided by PHPUnit, this printer could read a config file upon first time initialization and read configuration options from there. Something like phpunit-exception-printer.(json|yaml|xml|younameit).dist etc.
I would love to use it but the point would be to "see the errors" ASAP but with large test suites the immediately get washed away with all the success responses.
In my fork (see #9 (comment) ) I actually implemented this.
Not very elegant, but at least it's configurable. I can share the pain with big test suites (talking >5k tests) which literally pages and pages of successful lines when all you really need are the failures.