Giters

minitest / minitest-bisect

Hunting down random test failures can be very very difficult, sometimes impossible, but minitest-bisect makes it easy.

Home Page:http://docs.seattlerb.org/minitest-bisect

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

how to get this to run with rails?

bisrael opened this issue · comments

commented

usually i run my tests using the built-in rake task that comes with rails like:

rake test

but when i try to run mintest_bisect --seed xxxx test/*.rb or the like, it just runs and does nothing.

does this gem work with rails/rake?

commented

The gem definitely works with rails... what does ls test/*.rb output? My guess is nothing. I've had better luck using a subshell command like:

mintest_bisect --seed xxxx $(find test -name \*_test.rb)
commented

This seems like a good thread to add to. So, it took me a while to get minitest_bisect running coming from rails too, but I'm glad I persisted because it's helped me track down some nasty intermittent failures.

My first problem was that I was unfamiliar with the required syntax to run the tests. I'm used to running tests like @bisrael with commands like:

rake test
rake test:units
rake test TEST=test/units/user_test.rb

So stepping away from that familiarity was a little intimidating. A note in the README hand-holding this transition would be great.

I finally got it working after a bunch of experimentation, and then @zenspider told me on Twitter that my long command could be simplified to:

minitest_bisect --seed=57712 -Itest $(find test -type f -name \*_test.rb)

This works, and runs all my tests. Fantastic! That's the command that should be in the README.

Well, it didn't quite work for me. It almost worked, and I think for 99% of rails apps it probably would.

It tells me Reproduction run passed? Aborting. Try running with MTB_VERBOSE=2 to verify. So, I try that. MTB_VERBOSE=2 minitest_bisect --seed=57712 -Itest $(find test -type f -name \*_test.rb). Then it tells me the tests didn't run, due to this error:

/Users/tim/.rvm/gems/ruby-2.3.0/gems/rake-10.5.0/lib/rake/task_manager.rb:62:in []': Don't know how to build task 'test:load_additional'

Okay, test:load_additional is a special rake task we have setup to help CircleCI load our additional test databases. It's included in the top of the test_helper.rb. When I comment that out (It's not needed in my local test env), then minitest_bisect works as expected.

So, my suggestions:

  1. Update the README to include minitest_bisect --seed=57712 -Itest $(find test -type f -name \*_test.rb) as the command to use for a typical rails app.

  2. Detect if no tests were run, and emit the error if possible, instead of saying Reproduction run passed?

  3. Me: Fix my local repo so test:load_additional doesn't break minitest_bisect.

Thanks @zenspider for your work on this, it's a truly awesome tool.

commented

@zenspider Would it make sense to have -Itest $(find test -type f -name \*_test.rb) as the default? Or have it included with a --rails flag? The find command is unwieldy.

commented

I think I'd rather pull in the code I use on flog/flay to expand directories. Or some variant thereof, since those are written funny in order to be multi-language pluggable. Wanna take a whack at it?

commented

  1. updating readme: yes. Please send PR if you have time. Comment if you don't.

  2. Yes and done.

  3. Yes. :P

commented

@timhaines any interest?

commented

No response. Closing.