[Feature Request] Don't require all branches to be covered twice in ensure-blocks
mak-dunkelziffer opened this issue · comments
I am currently turning on branch coverage for projects that already have full line coverage. I have stumbled across the following issue multiple times now. To get to 100% branch coverage, SimpleCov requires that each branch within an ensure-block is hit twice. Once if the ensure block is entered regularly and once if it is entered via an exception. However, in my opinion, this is no longer "branch coverage", but instead "path coverage", which is even harder to achieve and sometimes unnecessarily tricky to test.
One such example is the following:
def something_about_files
...
# stuff that might raise
...
tmpdir = Dir.mktmpdir('my_directory')
...
# stuff that might raise
...
ensure
FileUtils.rm_rf(tmpdir, secure: true) if tmpdir
end
The case that that the ensure-block is reached without exception AND tmpdir
is falsey is pretty hard to test and I don't think this adds much to the guarantees that you get from your test suite.
Could we get a config option to turn off this "double branch coverage / path coverage" in ensure-blocks (and wherever else it might happen)? What are your opinions on this? Do you think that this type of coverage adds much to your test suite? Am I wrong for perceiving SimpleCov's handling of ensure-blocks as "stronger than branch coverage"? Do you have examples, where the current behavior is more reasonable than in my example?
Setup:
- SimpleCov: 0.21.2
- test framework: RSpec
- projects: Rails projects and gems without Rails
This is how Ruby coverage is implemented. So I think you should request that feature in Ruby issue tracker before that could be implemented in simplecov.
Maybe you could still resolve it in SimpleCov if Ruby hands you sufficient meta information already. Do you know what kind of info you get for such a line in an ensure-block? If you only get then: 1, else: 1, then: 1, else: 0
this is obviously insufficient to resolve it in SimpleCov. But if these then/else
results contain more information, it could be possible to detect situations where it would be reasonable to merge them together.
This issue seems to have been resolved for current versions of Ruby already.
Issue: https://bugs.ruby-lang.org/issues/16967
I tested this on Ruby 3.1.2 and the branch coverage indeed behaves as desired.