Extraneous output using TestTask
trans opened this issue · comments
When using the TestTask, when a test fails or errors, the end of the output is:
rake aborted!
Command failed with status (1): [ruby -I"lib:test" -I ... ]
Is it possible to suppress this (and still get the proper exit code, i.e. 1)? I noticed Rails::TestTask manages it to do it, but I haven't figure out how.
This output comes from Rake::FileUtils#sh:
$ cat test.rf
require 'rake/testtask'
task :default do
sh 'exit 2'
end
$ rake -f test.rf
exit 2
rake aborted!
Command failed with status (2): [exit 2...]
/Users/drbrain/Work/git/rake/test.rf:4:in `block in <top (required)>'
Tasks: TOP => default
(See full trace by running task with --trace)
There is no way to suppress the output. I don't see the value in suppressing the output for Rake::TestTask as it can provide valuable information for figuring out why your test failed (syntax error vs missing require vs test failure).
You can write your own test runner that runs the tests in-process. Rake::TestTask doesn't operate this way because it is safer and more consistent to run tests in their own process.
There is no way to retrieve the exit code from the internal task. Providing this would encourage using rake in inappropriate ways (see #257).
@trans did you ever figure out a way to suppress the rake aborted! ...
backtrace?
@ddarbyson @trans I have. Rails does this by using a TestTask sub class. The code in the define
method seems to prevent the output but don't understand how. Below is a stripped down Rakefile (its all I need for a non Rails project) that leaves out the rake aborted!...
output. Hope it helps.
require 'rake/testtask'
module Rails
class TestTask < Rake::TestTask
def initialize(name = :test)
super
@libs << "test" # lib *and* test seem like a better default
end
def define
task @name do
libs = @libs - $LOAD_PATH
$LOAD_PATH.unshift(*libs)
file_list.each { |fl|
FileList[fl].to_a.each { |f| require File.expand_path f }
}
end
end
end
end
task default: :test
task :test do
Rake::Task['test:run'].invoke
end
namespace :test do
Rails::TestTask.new(:run) do |t|
t.pattern = "test/**/*_test.rb"
end
end
Reference: https://github.com/rails/rails/blob/master/railties/lib/rails/test_unit/sub_test_task.rb
@javierjulio Yes, something like that can work -- in most cases. The problem is that the tests are running in the Rake process, so you have all the Rake dependencies loaded while the tests are running. As @drbrain pointed out, it is "safer and more consistent to run tests in their own process". So there's the rub.
@dbrian, the reason that it is useful to suppress this output is for structured test output. For example, I have custom output format that produces JSON, designed to be fed into another program. Not being able to suppress extraneous output means the JSON becomes malformed. That's why this issue arose. It occurs to me that there might be a simple solution. Can that information output to stderr instead of stdout? That should do the trick. And actually it seems like it might be a good idea regardless of this issue.
Good call @trans, sending this to stderr sounds like better handling...
What I usually do in cases like this is forego use of sh() altogether.
Using systemu I'm capturing stdout and stderr separately. Usually I suppress all output when things are OK and print out everything in case of an error. But since the streams are separate the choice is yours.
Also, to suppress all rake output if it is really, really important, you could monkey patch exception handling
module Rake
class Application
def standard_exception_handling
begin
yield
rescue SystemExit => ex
# Exit silently with current status
raise
rescue OptionParser::InvalidOption => ex
# Invalid option. Exit silently, option parser has already printed error message.
exit(false)
rescue Exception => ex
# Internal failure. Exit with error message and stack trace.
$stderr.puts "Exception: #{ex.message}"
$stderr.puts ex.backtrace.join("\n")
exit(false)
end
end
end
end