How to configure SimpleCov with Spring?

[Hunter]

Rails 4.2.0, RSpec 3.1, SimpleCov 0.9.2

I have the following in my rails_helper.rb file, outside of the RSpec.configure block

require 'simplecov'
SimpleCov.start 'rails' do
  add_filter '/app/models/legacy/'
  add_filter '/app/controllers/playlists_controller_legacy.rb'
  add_filter '/app/models/login.rb'
  add_filter '/app/models/login_auth.rb'
  add_filter '/app/models/login_config.rb'
end

I have read online that some people find running spring rspec returns high or even 100% coverage. It does not seem that way for our app, but I would just like to make sure. The LOC count seems a little low (short by 1000 maybe?), although again that could be from recent refactoring I have not viewed the commits for.

And my config/spring.rb file contains nothing about SimpleCov. Would I put the above configuration in the spring file itself, or under the Spring.after_fork block?

[Craig Sniffen]

I definitely have an issue with spring and simplecov:

$ bundle exec rspec
Finished in 15.98 seconds (files took 8.6 seconds to load)
87 examples, 0 failures, 5 pending

Coverage report generated for RSpec to /Users/craigsniffen/Documents/Projects/71lbs/coverage. 2372 / 7486 LOC (31.69%) covered.

$ bundle exec spring rspec
Finished in 13.9 seconds (files took 0.65848 seconds to load)
87 examples, 0 failures, 5 pending

Coverage report generated for RSpec to /Users/craigsniffen/Documents/Projects/71lbs/coverage. 18 / 27 LOC (66.67%) covered.

This is with config.eager_load = true set in the test environment, as I'd like to have all the files, even untested ones, in the coverage report. Setting this to false 'fixes' spring:

$ bundle exec rspec
Finished in 20.97 seconds (files took 11.15 seconds to load)
87 examples, 0 failures, 5 pending

Coverage report generated for RSpec to /Users/craigsniffen/Documents/Projects/71lbs/coverage. 1505 / 3530 LOC (42.63%) covered.

$ bundle exec spring rspec
Finished in 15.01 seconds (files took 0.89234 seconds to load)
87 examples, 0 failures, 5 pending

Coverage report generated for RSpec to /Users/craigsniffen/Documents/Projects/71lbs/coverage. 1188 / 2953 LOC (40.23%) covered.

The difference in LOC is troubling, however. It's also not loading all of the files as I'd want.

[Hunter]

Why is the LOC count in your first bundle exec rspec higher than the second?

I have the following settings in my test.rb file:

  # The test environment is used exclusively to run your application's
  # test suite.  You never need to work with it otherwise.  Remember that
  # your test database is "scratch space" for the test suite and is wiped
  # and recreated between test runs.  Don't rely on the data there!
  config.cache_classes = false

  # Disable Rails's static asset server (Apache or nginx will already do this)
  config.serve_static_files = false
  config.eager_load = false

When running rspec alone, versus with spring, I did not notice a change in LOC or coverage. As I mentioned in my original post, we may have truly deleted a couple thousand LOC.

[Craig Sniffen]

I'm going to assume that it's because it's loading files that weren't loaded in the other run. I have very poor test coverage.

I found the new section on the readme for dealing with spring, but after implementing it my coverage reports now claim that all code outside of methods are not covered.

[Hunter]

I am going to run some tests with eager load toggled. Thank you for the advice

UPDATE: I see what you mean. Turning eager load to true makes my coverage 0/0 LOC at 100%

[Benjamin Fleischer]

Maybe a dumb question, but have you seen #341 (in the README?)

Also, and this is just my opinion, but spring ruins everything it touches. I can't tell you how many times some bizarre error has been due to spring running old code. My bash_profile now has export DISABLE_SPRING=1 and things have been normal since. And conceptually, I don't think there's a right way to mix persisting pre-loaded code with tracking discrete runs of executed code. Maybe some of http://tenderlovemaking.com/2015/02/13/predicting-test-failues.html will make it into SimpleCov at some point which will make this feasible. Also, check out https://github.com/danmayer/coverband

[Hunter]

I did see that part of the README. My question, do I put that in my Spring.after_fork? Or completely before then? This is my my configuration now:

# config/spring.rb
Spring.after_fork do
  if Rails.env == "test"
    class ActiveRecord::Base
      mattr_accessor :shared_connection
      @@shared_connection = nil
      def self.connection
        @@shared_connection || ConnectionPool::Wrapper.new(size: 1) { retrieve_connection }
      end
    end
    ActiveRecord::Base.shared_connection = ActiveRecord::Base.connection

    RSpec.configure do |config|
      srand; config.seed = srand % 0xFFFF
    end
  end
end

[Hunter]

@bf4 As I mentioned, I do not see any strange behavior with spring right now. I just wanted to properly set it up to prevent any behavior. When I make major changes I use spring stop and start back up again. My SimpleCov config is in my rails_helper.rb, with config.eager_load = false.

I guess I should simply not fix what ain't broken?

[Benjamin Fleischer]

If you think you can improve rhe docs, please submit a pr

[Ibrahim Awwal]

As a related note, since this is one of the top search results for "SimpleCov eager_load", if you do want to eager load all your code, you could try what is mentioned here: https://stormconsultancy.co.uk/blog/development/ruby-on-rails/getting-accurate-code-coverage-metrics-from-simplecov-rails-project/

just run Rails.application.eager_load! manually after SimpleCov.start

I'll submit a PR for the docs if this seems like something that should be in the documentation!

[David Gil]

@ibrahima thanks for the suggestion! Unfortunately, I'm seeing this error spec/spec_helper.rb:6:in <top (required)>': uninitialized constant Rails (NameError) after adding Rails.application.eager_load! to my spec_helper.rb. Should I add the line to my rails_helper.rb, down the require list instead?

[David Gil]

Just to let you know, everything is working fine when I moved Rails.application.eager_load! to rails_helper.rb juts after the require "spec_helper" line. I'm also seeing improvements in the coverage, probably this fixed our problems - thanks!

[Nick Reshetnikov]

I was having this issue recently and I tried to do the suggested approach of setting config.eager_load = false in config/environments/test.rb and doing the following in spec/rails_helper.rb:

require "simplecov"
SimpleCov.start "rails" 
Rails.application.eager_load!

this did not work for me, but what did work for me is updating config/environments/test.rb like so:

require "simplecov"
SimpleCov.start "rails" 

Rails.application.configure do
  config.eager_load = true
  ...
end

so that the SimpleCov call preceded the Rails configuration step