rb-fsevent taking up over 100% CPU

[Gavin Morrice]

I'm running guard in my Rails app and the test suite has recently stopped running smoothly.
If I'm lucky, it'll run all of the tests once, maybe twice. After that, it takes so long to respond to even one small test file being changed that
using the gem becomes futile.

When following top while the tests run, I can see there's a ruby process that's taking up over 100% of the CPU constantly. Even once all tests are run and I haven't made any changes to the file.

The ruby process is:

/Users/Bodacious/.rvm/gems/ruby-2.0.0-p247@MyApp/gems/rb-fsevent-0.9.3/bin/fsevent_watch --latency 0.1 /Users/Bodaiou/Clients/MyApp

(process 31332) in the screenshot attached.

Here's my setup:

Gemfile

group :test do
  gem 'launchy'
  gem "mocha", require: false
  gem 'database_cleaner'
  gem 'selenium-webdriver'
  gem 'capybara-webkit' # for headless javascript tests
  gem 'timecop'
  gem "minitest-focus"
end

group :development, :test do
  gem "minitest-rails"
  gem "minitest-rails-capybara"
  gem 'zeus'
  gem 'guard'
  gem 'guard-minitest'
  gem 'factory_girl_rails'
end

Guardfile

guard :minitest, all_on_start: false do
  # Rails 4
  watch(%r{^app/(.+)\.rb})                               { |m| "test/#{m[1]}_test.rb" }
  watch(%r{^app/controllers/application_controller\.rb}) { 'test/controllers' }
  watch(%r{^app/controllers/(.+)_controller\.rb})        { |m| "test/integration/#{m[1]}_test.rb" }
  watch(%r{^app/views/(.+)_mailer/.+})                   { |m| "test/mailers/#{m[1]}_mailer_test.rb" }
  watch(%r{^lib/(.+)\.rb})                               { |m| "test/lib/#{m[1]}_test.rb" }
  watch(%r{^test/.+_test\.rb})
  watch(%r{^test/test_helper\.rb})                       { 'test' }

  ignore(%r{^tmp/})

end

test_helper

ENV["RAILS_ENV"] = "test"
require File.expand_path("../../config/environment", __FILE__)

require 'rails/test_help'
require 'minitest/autorun'
require 'minitest/pride'
require "minitest/rails/capybara"

require "mocha/setup"

# Sidekiq https://github.com/mperham/sidekiq/wiki/Testing
require 'sidekiq/testing'
Sidekiq::Testing.fake!

Dir[Rails.root.join('test', 'support', '*.rb')].each do |file|
  require file
end


class MiniTest::Spec
  include FactoryGirl::Syntax::Methods
  include AllTestHelper

end


class ActiveSupport::TestCase
  include FactoryGirl::Syntax::Methods
  include AllTestHelper
end

class Capybara::Rails::TestCase
  include Rails.application.routes.url_helpers 
  include Capybara::DSL
  include Capybara::Assertions
  include IntegrationTestHelper

  # Called before each Feature spec
  before :each do
  end

  # Called after each Feature spec
  after :each do
    Capybara.reset_sessions!
  end
end

Gem Versions:

• minitest (4.7.5)
• minitest-capybara (0.5.0)
• minitest-focus (1.1.0)
• minitest-metadata (0.4.0)
• minitest-rails (0.9.2)
• minitest-rails-capybara (0.10.0)
• mobvious-rails (0.1.2)
• mocha (0.14.0)
• guard (2.2.4)
• guard-minitest (2.1.2)
• rb-fsevent (0.9.3)

Any advise on where to look to start fixing this?

[Thibaud Guillaume-Gentil]

Do you have a lot of files in your app? You could try to ignore some of them with ignore method: https://github.com/guard/guard#ignore

[Gavin Morrice]

How many files is a lot?

It's not a particularly large app:

+----------------------+-------+-------+---------+---------+-----+-------+
| Name                 | Lines |   LOC | Classes | Methods | M/C | LOC/M |
+----------------------+-------+-------+---------+---------+-----+-------+
| Controllers          |  2162 |  1526 |      67 |     282 |   4 |     3 |
| Helpers              |  1184 |   894 |       8 |     127 |  15 |     5 |
| Models               |  1729 |  1017 |      43 |     120 |   2 |     6 |
| Mailers              |   197 |   103 |       2 |      11 |   5 |     7 |
| Javascripts          |   360 |   225 |       6 |      72 |  12 |     1 |
| Libraries            |    40 |    29 |       1 |       6 |   6 |     2 |
| Controller tests     |     7 |     3 |       0 |       0 |   0 |     0 |
| Helper tests         |     9 |     6 |       0 |       0 |   0 |     0 |
| Model tests          |   860 |   622 |       0 |       0 |   0 |     0 |
| Mailer tests         |    55 |     3 |       0 |       0 |   0 |     0 |
+----------------------+-------+-------+---------+---------+-----+-------+
| Total                |  6603 |  4428 |     127 |     618 |   4 |     5 |
+----------------------+-------+-------+---------+---------+-----+-------+
  Code LOC: 3794     Test LOC: 634     Code to Test Ratio: 1:0.2

Comp stats:
Processor: 2.7 GHz Intel Core i7
Memory: 8GB

Even stripping the Guardfile back down to this shows the CPU usage at 90%+:

guard :minitest, all_on_start: false do
  # Rails 4
  watch(%r{^app/(.+)\.rb})                               { |m| "test/#{m[1]}_test.rb" }
  watch(%r{^test/.+_test\.rb})
  watch(%r{^test/test_helper\.rb})                       { 'test' }
end

[Thibaud Guillaume-Gentil]

I was more thinking about files like logs/tmp, not your app code.

[Gavin Morrice]

I thought only files defined in watch() were watched?

i.e. The others are implicitly ignored

[Thibaud Guillaume-Gentil]

No, all files not ignored by default are watched.

find . -type f | wc -l is useful to count all files in a dir.

[Gavin Morrice]

Ah - I didn't realise that. 7216 is probably a lot of files to be watching.

[Thibaud Guillaume-Gentil]

Yeah, try to ignore some.

[Gavin Morrice]

That's working a lot better now - thanks:

guard :minitest, all_on_start: false, zeus: true, bundler: false do
  # Rails 4 watch:
  watch(%r{^app/(.+)\.rb})                               { |m| "test/#{m[1]}_test.rb" }
  watch(%r{^test/.+_test\.rb})
  watch(%r{^test/test_helper\.rb})                       { 'test' }

  # Rails 4 ignores:
  ignore(%r{^bin/*})
  ignore(%r{^config/*})
  ignore(%r{^db/*})
  ignore(%r{^lib/*})
  ignore(%r{^log/*})
  ignore(%r{^public/*})
  ignore(%r{^tmp/*})
end

[Thibaud Guillaume-Gentil]

Great, you can do it like that:

 # Rails 4 ignores:
ignore([%r{^bin/*}, %r{^config/*}, %r{^db/*}, %r{^lib/*}, %r{^log/*}, %r{^public/*}, %r{^tmp/*}])

guard :minitest, all_on_start: false, zeus: true, bundler: false do
  # Rails 4 watch:
  watch(%r{^app/(.+)\.rb})                               { |m| "test/#{m[1]}_test.rb" }
  watch(%r{^test/.+_test\.rb})
  watch(%r{^test/test_helper\.rb})                       { 'test' }
end

[Gavin Morrice]

Thanks!