Install

gem install forking_test_runner

Usage

Run folders

forking-test-runner test/
Running tests test/another_test.rb test/pollution_test.rb test/simple_test.rb
------ >>> test/another_test.rb
Run options: --seed 19151

# Running tests:

.

Finished tests in 0.002904s, 344.3526 tests/s, 344.3526 assertions/s.

1 tests, 1 assertions, 0 failures, 0 errors, 0 skips
------ <<< test/another_test.rb ---- OK

...

Results:
test/another_test.rb: OK
test/pollution_test.rb: OK
test/simple_test.rb: OK

9 assertions, 0 errors, 0 failures, 0 skips, 8 tests

Run files

forking-test-runner test/models/user_test.rb test/models/order_test.rb

Parallel

Execute in 4 parallel processes, with TEST_ENV_NUMBER set to '' / '2' / '3' / '4', see parallel_tests for setup helpers and details. Stdout is synchronized and test results (start/output/finish) are printed when a test completes.

forking-test-runner test/ --parallel 4

Parallel execution on CI

Make CI have 20 Parallel workers that each test 1 group of tests, each worker runs a hardcoded group:

forking-test-runner test/ --group 1 --groups 20

Executing multiple test groups

Helps with balancing when 1 group is slower than the others.

forking-test-runner test/ --group 1,2,3,4 --groups 20
forking-test-runner test/ --group 1,2,3,4 --groups 20 --parallel 4

Make test groups take the same time

Record test runtime (on your CI, see other modes below)

forking-test-runner test/ --group 1 --groups 20 --record-runtime amend

Will generate a download url, download the runtime info and commit it to your repo, and then run with runtime

wget -o test/files/runtime.log <url>
git add test/files/runtime.log
forking-test-runner test/ --group 1 --groups 20 --runtime-log test/files/runtime.log

Only show output of failed tests

forking-test-runner test/ --quiet

RSpec

Run with --rspec

Options

forking-test-runner folder [options]
    --rspec                      RSpec mode
    --helper                     Helper file to load before tests start
    --quiet                      Quiet
    --no-fixtures                Do not load fixtures
    --no-ar                      Disable ActiveRecord logic
    --merge-coverage             Merge base code coverage into individual files coverage and summarize coverage report
    --record-runtime=MODE        
      Record test runtime:
        simple = write to disk at --runtime-log)
        amend  = write from multiple remote workers via http://github.com/grosser/amend, needs TRAVIS_REPO_SLUG & TRAVIS_BUILD_NUMBER
    --runtime-log=FILE           File to store runtime log in or runtime.log
    --parallel=NUM               Number of parallel groups to run at once
    --group=NUM[,NUM]            What group this is (use with --groups / starts at 1)
    --groups=NUM                 How many groups there are in total (use with --group)
    --version                    Show version
    --help                       Show help

Supported CI Providers

Travis CI (TRAVIS_REPO_SLUG, TRAVIS_BUILD_NUMBER)
Buildkite (BUILDKITE_ORGANIZATION_SLUG, BUILDKITE_PIPELINE_SLUG, BUILDKITE_JOB_ID)
TODO: github action

Log aggregation

To analyze all builds try this streaming travis log analyzer it will show all failures, the failed files and failed jobs.

Development

bundle exec rake run tests
BUNDLE_GEMFILE=gemfiles/60.gemfile bundle exec rake run tests on specific gemfile
bundle exec rake bundle_all to update all Gemfiles (run on ruby 2.7 for best results)

Authors

Contributors

Michael Grosser
michael@grosser.it
License: MIT

grosser / forking_test_runner