Goveralls counts braces but "go tool cover" doesn't

[Charlie Dennison]

Hi everyone,

I've been trying to contribute to and open source project and noticed that the coveralls coverage percent differs from what I get from go tool cover. The difference is relatively small but I thought you still might want to know about it. I think it's entirely caused by goveralls counting braces but go cover not counting them (go cover approach explained here).

Possible solutions (I'd be happy to contribute toward either)

1. Don't count the braces (which as a downside will make the coveralls.io report looks weird because the first line of functions wouldn't be marked as covered - the same thing of course happens when you use go's html report go tool cover -html=profile.cov.

2. Provide a way to output the coverage from goveralls without submitting the report to coveralls.io. Basically add a flag so I can do something like goveralls --report and get something similar to go tool cover -func=profile.cov. This would actually be pretty useful because goveralls correctly outputs coverage across multiple packages (by merging the coverage reports) which go cover doesn't - so this could be used for projects not even using coveralls.io.

I'd be happy to submit a PR if you want to go in either direction above.

Here's a contrived example:
(based on this)

//config.go
package config

type Config struct {
	data map[string]map[string]string
}

func NewConfig() () {
	_ = &Config{
		data: make(map[string]map[string]string),
	}
}

func (c *Config) parse(fname string) (err error) {
	return nil
}

//config_test.go
package config

import (
	"testing"
)

func TestGet(t *testing.T) {
	NewConfig()
}

Then I run the test like this
go test -covermode=count -coverprofile=profile.cov
And check my coverage like this
go tool cover -func=profile.cov
Which give 50%.

For coveralls I get 62.5%. Here's why:

cat profile.cov 
mode: count
github.com/casbin/casbin/config/config.go:8.21,12.2 1 1
github.com/casbin/casbin/config/config.go:14.50,16.2 1 0

Go cover I believe just sees (2) blocks:
block 1: 1/1 lines covered
block 2: 0/1 lines covered
Total = 1/2 = 50%

goveralls gives this output
(https://github.com/mattn/goveralls/blob/master/gocover.go#L102)
1
2
3
4
5
6
7 1
8 1
9 1
10 1
11 1
12
13 0
14 0
15 0

Which means it counts 8 lines (instead of 2 because 6 are braces) and 3 lines without coverage (instead of 1) giving the 5/8 and 62.5%.

Also to be fair - your coverage does match the HTML report (which shows 5/8 lines covered)
go tool cover -html=profile.cov

[mattn]

I understand what you mean. But coveralls never gave 100% if using go cover.

[Charlie Dennison]

@mattn - sorry I missed your comment - I'm not sure I follow. You can get 100% coverage wth go cover - it just doesn't care about the braces. The only difference between goveralls and go cover is regarding missing coverage - the go cover percent will often be lower because it doesn't count braces. Don't get me wrong - most coverage calculators for languages other than go count braces so I think your approach makes more sense, but it's confusing if it's different than the standard library.

[gdm85]

@cdennison I think at the very least we should document this behaviour. If we were to change goveralls (either with an option or in all cases), would skipping the first and the last line of the blocks always be correct?

[Charlie Dennison]

@gdm85 - I believe you just skip all likes that only contain only closing brace "}" but there may be other differences between goveralls and go cover (I'm not an expert on either). I'm happy to create a PR that adds an experimental flag to output "go cover" coverage without closing braces to verify this theory but won't bother if no one cares.