Merging doesn't work with test-unit

[Ecco]

Rails 3 application. Simple tests (no rspec, no cucumber).

If I add this to my gemfile :

require 'test-unit', '2.3.0'

then merging doesn't work as unit, functional, and integration test results overwrite each-other.

Thing is, I need this for the performance tests to work.

[Ecco]

For some reason, it seems like ARGV is not passed to SimpleCov::CommandGuesser when using "test-unit 2.3.0"

[Ecco]

Ok, I figured it out : for some weird reason, the "unit-test" gem alters ARGV !
A very simple fix is to call SimpleCov.command_name (w/o any parameter) right after SimpleCov.start, so that the gem hasn't had time to touch ARGV.

[Christoph Olszowka]

Oh, ok. Haven't tried test-unit 2+ yet, but if it's the way you're saying that it messes with ARGV, that seems like a pretty bad practice to me :/

It'd be great if you could swing up a little part about test/unit 2 including your workaround as a code example for the README and phase out a pull request. Thanks!

[Christoph Olszowka]

Actually reopening this as a reminder to add something about this into the README

[Eric Wanchic]

I am also having this problem

[Pablo Fernandez]

I got this issue and I spent a couple of hours trying to figure out what was going on... are my tests not running? no, they are running, is there some old coverage data somewhere else? Until I concluded the merging wasn't working, and then upon inspecting the .yml file, realizing results were not separated by test suit. Yeah, documenting this issue seems important, and fixing it if possible too so that it doesn't happen.

[George]

I am having this issue as well. If I view the HTML file after the unit tests part, the results are different then at the end, after the functionals.

Using: $ rake test

Gemfile:
gem 'test-unit', '1.2.3'

I see the fix is: "to call SimpleCov.command_name (w/o any parameter) right after SimpleCov.start"
OK, I added SimpleCov.command_name to test_helper.rb. The first 3 lines are now:
require 'simplecov'
SimpleCov.start 'rails'
SimpleCov.command_name

But I am still seeing the same symptom. How can I put the results of both units and functionals in one report? Is it possible to create a rake task that would do it?

Also, I am running test-unit 1.2.3, so I don't know if this is an issue with just test-unit 2?

Otherwise, seems to work great. Thank you!

[George]

Still seeing this issue with 0.5.0. When using rake test, the functional tests results overwrite the unit test results.

Updated to rake 0.9.2.

OK, after working on this for 12 hours, I have narrowed it down to 2 or 3 gems. However, these gems do not always cause the problem and the problem does not disappear unless all 3 are removed:
gem 'mocha'
gem 'redgreen'
gem 'test-unit', '1.2.3'

How can you save the results from a test suite with a given name, and manually merge the results of 2 test suites?

[Fernando Guillen]

Same problem with test-unit 2.4.0

As Ecco said, it is the test-unit who is eating the ARGV content.

A fix for test-unit 2.4.0 could be:

~/Downloads/test-unit(master) $ git diff --no-prefix 
diff --git lib/test/unit/autorunner.rb lib/test/unit/autorunner.rb
index 7a31aa1..029a4b5 100644
--- lib/test/unit/autorunner.rb
+++ lib/test/unit/autorunner.rb
@@ -146,7 +146,7 @@ module Test
       def process_args(args = ARGV)
         begin
           args.unshift(*@default_arguments)
-          options.order!(args) {|arg| @to_run << arg}
+          options.order!(args.dup) {|arg| @to_run << arg}
         rescue OptionParser::ParseError => e
           puts e
           puts options

I let it here just in case it is helpful for someone, I'll try to comunicate with the test-unit maintainers but maybe I'll forget.

[Nikos Dimitrakopoulos]

Thanks @fguillen for the tip. Sent a pull request to the test-unit people (test-unit/test-unit#12).

[Christoph Olszowka]

Obviously the test-unit guys merged this into Test/Unit, I suspect by release date that v2.4.3+ should have this resolved. Please re-open if there is still trouble.

Thanks a lot for taking care of this guys!

[Michael Irwin]

This problem still exists w/ test-unit 2.4.5 and simplecov 0.5.4.

[Tony Primerano]

yeah, still broken..

The workaround is to add the following line to one of your functional test files

SimpleCov.command_name 'test:functionals' if ENV['COVERAGE']

and this line to one of the units

SimpleCov.command_name 'test:units' if ENV['COVERAGE']

Leave off the if ENV['COVERAGE'] if you're not using that variable.

[Benjamin Fleischer]

Closing as stale. Please reopen if this is still an issue. (And take a look at #340 )