rails 5 test outputs wrong location of test case
logicminds opened this issue · comments
Rails 5 now support testing a single test by specifying a line number on a file (just like rspec). However when using this gem it appears like that functionality is somehow messed up.
When running rails test test/controllers/controller_name_test.rb I expect rails to output the the line number of the failed test in order to focus on a single test.
like this bin/rails test /home/developer/my_rails_app/test/controllers/controller_name_test.rb:81
However the result is actually like this:
bin/rails test /home/developer/vendor/ruby/2.3.0/gems/minitest-spec-rails-5.4.0/lib/minitest-spec-rails/dsl.rb:27
I am not sure who is to blame here but since the response from rails is pointing to this gem I suspect this gem is doing something funky to mess up this new capability.
Thanks! I'll take a look at this today.
I'm seeing differently in my local Rails 5 project which uses this gem. Are you sure it is not some other gem you have installed? FWIW, my Gemfile.lock says I am using the latest 5.9.0 of minitest too.
$ rails test test/models/user_test.rb
Run options: --seed 3598
# Running:
.....
Finished in 0.150684s, 33.1820 runs/s, 106.1824 assertions/s.
5 runs, 16 assertions, 0 failures, 0 errors, 0 skips
$ rails test test/models/user_test.rb:20
Run options: --seed 37132
# Running:
.
Finished in 0.243915s, 4.0998 runs/s, 8.1996 assertions/s.
1 runs, 2 assertions, 0 failures, 0 errors, 0 skips
You need to make a test fail in order to see the output. In your example above nothing is failing so you don't see the new output generated by rails 5.
Sorry I missed that... here is what I am seeing (condensed) when I made a test fail on line 21 of that same test. This seems to show that I need to use line :20 which is correct.
$ rails test test/models/user_test.rb
Run options: --seed 18909
# Running:
....F
Failure:
UserTest#test_0002_.logged_in! - will return the DB user each time [/.../test/models/user_test.rb:21]:
Expected true to not be truthy.
bin/rails test test/models/user_test.rb:20
Finished in 0.120783s, 41.3965 runs/s, 124.1895 assertions/s.
5 runs, 15 assertions, 1 failures, 0 errors, 0 skips
I keep all my tests flat with no inner describes (or very few). Here is what happens when I add one just to check it out. With the following output.
describe 'foo' do
it 'bar' do
refute true
end
endFailure:
UserTest::xxx#test_0001_foo [/.../test/models/user_test.rb:23]:
Expected true to not be truthy.
bin/rails test test/models/user_test.rb:22
Can you reproduce a test case (no pun) where this is not working for you or find out why I am seeing different results.
I'm having the same issue. It seems to only occur in integration tests here, but not all from what I can see. Also I'm using poltergeist with minitest-rails-capybara if could be a cause. Will investigate further and see if I can reproduce in a fresh rails5 project.
Seems mintiest-rails-capybara has a dep on mintiest-rails. You really do not need both when using this gem.
Not sure if I follow this. I'm using minitest-rails-capybara which does indeed have a dependency on minitest-rails so both will be installed when I specify the former in the Gemfile. Which is the case. Adding minitest-rails to the Gemfile doesn't change anything.
I am saying that both mintiest-rails-capybara and mintiest-rails are not needed. I actually think mintiest-rails may even conflict with this gem. Thankfully, it is super easy to use Capybara with Minitest. Just mixin the DSL and go. I usually do something like this. A bit verbose, but thought you might find some helpful info here.
test/test_helper_integration.rb
require 'test_helper'
require 'capybara/poltergeist'
require 'capybara/rails'
Capybara.register_driver :poltergeist do |app|
options = {
window_size: [1280, 800],
js_errors: false,
phantomjs_options: ['--ignore-ssl-errors=yes', '--load-images=no'],
phantomjs: Phantomjs.path
}
Capybara::Poltergeist::Driver.new app, options
end
Capybara.default_driver = :poltergeist
Capybara.server = :puma
Capybara.default_max_wait_time = 4
module ActionDispatch
class IntegrationTest
after { capybara_reset! }
private
def capybara_reset!
Capybara.reset_sessions!
page.driver.reset!
end
end
endOkay, thanks for the info. That's pretty much what I have been using in my test_helper.rb and it was working with all the three gems on Rails 4. I also tried removing minitest-rails-capybara and I still have the same issue. (Still haven't tried to replicate this on a fresh Rails5 install..)
I'm not sure if this is related but I'm unable to run a single test in Rails5 in my project. Dong bin/rails test test/my_test.rb:50 doesn't run any tests. When removing minitest-spec-rails from the Gemfile it runs the test.
I can confirm that you can do ./bin/rails test test/my_test.rb and even with my_test.rb:12 line number just fine. If you can not do this, please remove the offending gem like mintiest-rails and the capybara gems. Less is more! :)
Yeah sorry for the poor scope of testing, no intention of putting blame here. Just thought it would be useful info for any other facing issues upgrading from rails 4 to 5 with a similar setup as mine. I also had other issues with my tests, that worked fine on Rails4, so it's probably something from the old days that not ready for v5.
so it's probably something from the old days that not ready for v5
Indeed, not be adversarial, but I have noticed from the GitHub activity that the mintiest-rails gem had a significant path to Rails 5 support. Thanks for posting!!!
I'm building a Rails 5 app and using minitest-spec-rails, and I'm also having this issue. When I remove minitest-spec-rails, I can specify the line number.
I've tried using minitest-focus, and also have this issue. minitest-focus is not working with minitest-spec-rails, and when I remove minitest-spec-rails, minitest-focus works.
Hey @andrewferk thanks for posting. Mind sharing your Gemfile.lock? I can confirm that installing mintiest-focus v1.1.2 does indeed still allow me to run ./bin/rails test test/my_test.rb:12 by line number.
My Gemfile.lock
GEM
remote: https://rubygems.org/
specs:
actioncable (5.0.0.1)
actionpack (= 5.0.0.1)
nio4r (~> 1.2)
websocket-driver (~> 0.6.1)
actionmailer (5.0.0.1)
actionpack (= 5.0.0.1)
actionview (= 5.0.0.1)
activejob (= 5.0.0.1)
mail (~> 2.5, >= 2.5.4)
rails-dom-testing (~> 2.0)
actionpack (5.0.0.1)
actionview (= 5.0.0.1)
activesupport (= 5.0.0.1)
rack (~> 2.0)
rack-test (~> 0.6.3)
rails-dom-testing (~> 2.0)
rails-html-sanitizer (~> 1.0, >= 1.0.2)
actionview (5.0.0.1)
activesupport (= 5.0.0.1)
builder (~> 3.1)
erubis (~> 2.7.0)
rails-dom-testing (~> 2.0)
rails-html-sanitizer (~> 1.0, >= 1.0.2)
activejob (5.0.0.1)
activesupport (= 5.0.0.1)
globalid (>= 0.3.6)
activemodel (5.0.0.1)
activesupport (= 5.0.0.1)
activerecord (5.0.0.1)
activemodel (= 5.0.0.1)
activesupport (= 5.0.0.1)
arel (~> 7.0)
activesupport (5.0.0.1)
concurrent-ruby (~> 1.0, >= 1.0.2)
i18n (~> 0.7)
minitest (~> 5.1)
tzinfo (~> 1.1)
arel (7.1.2)
bcrypt (3.1.11)
builder (3.2.2)
byebug (9.0.5)
concurrent-ruby (1.0.2)
devise (4.2.0)
bcrypt (~> 3.0)
orm_adapter (~> 0.1)
railties (>= 4.1.0, < 5.1)
responders
warden (~> 1.2.3)
devise_token_auth (0.1.39)
devise (> 3.5.2, <= 4.2)
rails (< 6)
erubis (2.7.0)
factory_girl (4.7.0)
activesupport (>= 3.0.0)
factory_girl_rails (4.7.0)
factory_girl (~> 4.7.0)
railties (>= 3.0.0)
ffi (1.9.14)
globalid (0.3.7)
activesupport (>= 4.1.0)
hashie (3.4.4)
i18n (0.7.0)
listen (3.0.8)
rb-fsevent (~> 0.9, >= 0.9.4)
rb-inotify (~> 0.9, >= 0.9.7)
loofah (2.0.3)
nokogiri (>= 1.5.9)
mail (2.6.4)
mime-types (>= 1.16, < 4)
method_source (0.8.2)
mime-types (3.1)
mime-types-data (~> 3.2015)
mime-types-data (3.2016.0521)
mini_portile2 (2.1.0)
minitest (5.9.0)
minitest-spec-rails (5.4.0)
minitest (~> 5.0)
rails (>= 4.1)
nio4r (1.2.1)
nokogiri (1.6.8)
mini_portile2 (~> 2.1.0)
pkg-config (~> 1.1.7)
omniauth (1.3.1)
hashie (>= 1.2, < 4)
rack (>= 1.0, < 3)
orm_adapter (0.5.0)
pkg-config (1.1.7)
puma (3.6.0)
rack (2.0.1)
rack-cors (0.4.0)
rack-test (0.6.3)
rack (>= 1.0)
rails (5.0.0.1)
actioncable (= 5.0.0.1)
actionmailer (= 5.0.0.1)
actionpack (= 5.0.0.1)
actionview (= 5.0.0.1)
activejob (= 5.0.0.1)
activemodel (= 5.0.0.1)
activerecord (= 5.0.0.1)
activesupport (= 5.0.0.1)
bundler (>= 1.3.0, < 2.0)
railties (= 5.0.0.1)
sprockets-rails (>= 2.0.0)
rails-dom-testing (2.0.1)
activesupport (>= 4.2.0, < 6.0)
nokogiri (~> 1.6.0)
rails-html-sanitizer (1.0.3)
loofah (~> 2.0)
railties (5.0.0.1)
actionpack (= 5.0.0.1)
activesupport (= 5.0.0.1)
method_source
rake (>= 0.8.7)
thor (>= 0.18.1, < 2.0)
rake (11.2.2)
rb-fsevent (0.9.7)
rb-inotify (0.9.7)
ffi (>= 0.5.0)
responders (2.3.0)
railties (>= 4.2.0, < 5.1)
spring (1.7.2)
spring-watcher-listen (2.0.0)
listen (>= 2.7, < 4.0)
spring (~> 1.2)
sprockets (3.7.0)
concurrent-ruby (~> 1.0)
rack (> 1, < 3)
sprockets-rails (3.2.0)
actionpack (>= 4.0)
activesupport (>= 4.0)
sprockets (>= 3.0.0)
sqlite3 (1.3.11)
thor (0.19.1)
thread_safe (0.3.5)
tzinfo (1.2.2)
thread_safe (~> 0.1)
warden (1.2.6)
rack (>= 1.0)
websocket-driver (0.6.4)
websocket-extensions (>= 0.1.0)
websocket-extensions (0.1.2)
PLATFORMS
ruby
DEPENDENCIES
byebug
devise (~> 4.2.0)
devise_token_auth (~> 0.1.39)
factory_girl_rails (~> 4.7.0)
listen (~> 3.0.5)
minitest-spec-rails (~> 5.4.0)
omniauth (~> 1.3.1)
puma (~> 3.0)
rack-cors (~> 0.4.0)
rails (~> 5.0.0, >= 5.0.0.1)
spring
spring-watcher-listen (~> 2.0.0)
sqlite3
tzinfo-data
BUNDLED WITH
1.13.0
Have the same issue - line number specification stopped working on a vanilla Rails 5 app when the minitest-spec-rails gem is added. However, it's syntax based. When the gem is added,
test 'bar' do; refute true; end stops working (no tests run), but it 'bar' do; refute true; end does work. I'l poke around in the source for a solution and PR, but that might be a contributing factor to what @espen and @andrewferk were seeing. Here's my fix in test_helper for now:
class ActiveSupport::TestCase
class << self
alias :test :it
end
...
end
(Great gem, BTW!)