tests always fail when files run in a particular order - Capybara::Poltergeist::StatusFailError

[John Bachir]

capybara 2.14.0
rails 5.1.0
phantomjs 2.1.1
poltergeist 1.15.0
ruby 2.4.1
minitest 5.10.1

I have two test files

foo_test.rb

class FooTest < ApplicationSystemTestCase
  test "registering from /" do
    p ">>> FooTest"
    visit "/"
    refute_text "profile"
    fill_in "user_name", with: "Sally Smith"
    fill_in "user_email", with: "[e-mail]"
    fill_in "user_password", with: "password123"
    click_on "register-button"
    assert_match(/\/stuff$/, page.current_url)
    assert_text "profile"
  end
end

bar_test.rb

class BarTest < ApplicationSystemTestCase
  test "loading front page" do
    p ">>> BarTest"
    visit "/"
    assert_text "Welcome to our product"
  end
end

• If I run the tests with a seed that puts bar first and foo second, it always succeeds and all tests pass.
• If I run the tests with a seed that puts foo first and bar second
  • as shown above: the Foo tests always succeed and then the suite hangs before starting the Bar tests. ">>> BarTest" is NOT printed, the suite just hangs for an unlimited amount of time (I've seen it up to 60 minutes on CI). If I ^C, then 1) ">>> BarTest" is immediately printed and 2) after the timeout seconds specified for Poltergeist, the suite finishes running and BarTest fails with this error: "Capybara::Poltergeist::StatusFailError: Request to 'http://127.0.0.1:31337/' failed to reach server, check DNS and/or server status - Timed out with no open resource requests"
  • with FooTest commented out starting from click_on "register-button": both tests succeed and finish

If FooTest didn't have assertions after the button click, I would suspect something is hanging/being blocked and I would go down that route. But since it has assertions, that means the webserver finished its request and the DOM rendered. So, this makes me suspect some sort of resource somewhere isn't being released.

I'm not sure if this is a capybara, Poltergeist, or PhantomJS issue. If you strongly suspect it's not Poltergeist, I can try to recreate the situation with chromium.

Let me know if I can provide any more information!

[Thomas Walpole]

Please provide a debug log and your setup/teardown blocks (if possible a reproducible sample would be excellent) without that there's nothing we can tell.

As an aside you should never be doing assert_match against current_url since it defeats the waiting/retrying behvior and button clicks are not guaranteed to be synchronous. Instead use assert_current_path

[John Bachir]

Okay, I will work on that. thanks for the note about assert_match vs. current_path -- just tried that and behavior is the same

[John Bachir]

in test_helper.rb

Capybara.register_driver :poltergeist do |app|
  Capybara::Poltergeist::Driver.new(app, {js_errors: true, phantomjs: Phantomjs.path, timeout: 4, debug: true})
end
Capybara.server_port = 31337

in application_system_test_case.rb

class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  driven_by :poltergeist, screen_size: [1400, 1400]

  setup do
    page.driver.browser.url_whitelist = %w[0.0.0.0 127.0.0.1]
  end
end

and I'll email you the output with debug logging.

[John Bachir]

This problem turned out to be caused by me using transactional tests. Transactional tests can't be used for system tests, because the test driver doesn't share state (namely, db connections)
with the application code.

for those finding this discussion in the future, here's the test/application_system_test_case.rb which works for me (after installing the database_cleaner gem). No other changes were needed:

require "test_helper"
require "capybara/poltergeist"


class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  driven_by :poltergeist, screen_size: [1400, 1400]

  self.use_transactional_tests = false

  setup do
    page.driver.browser.url_whitelist = %w[0.0.0.0 127.0.0.1]
    DatabaseCleaner.strategy = :truncation
    DatabaseCleaner.start
  end

  teardown do
    DatabaseCleaner.clean
  end

end