This repository contains Overcover. Overcover is a testing tool meant
to work in conjunction with the coverage profile data generated by the
go test tool with the -coverprofile option; it computes an
overall coverage of the entire source base and emits that
information. Overcover can also optionally enforce a minimum coverage
threshold, allowing test jobs to be failed if the testing coverage has
been weakened.
To use Overcover, install and execute it like so:
% overcover --coverprofile coverage.out
This will read the indicated coverage profile and emit a message describing the overall coverage represented by the profile. A threshold can be enforced, which will cause Overcover to exit with a status code of 1 if the coverage does not reach the threshold:
% overcover --coverprofile coverage.out --threshold 75
These values can also be passed to Overcover using environment
variables named after the long option names--e.g.,
OVERCOVER_COVERPROFILE specifies a value for --coverprofile,
and OVERCOVER_THRESHOLD specifies a value for --threshold.
The go test command does not include packages that do not contain
any test files in the output coverage profile. To ensure that all
code is counted in the coverage determination, Overcover can be
provided the package specification in the same format taken by go
test; it will count the number of statements and include that data
in its calculation. For example:
% overcover --coverprofile coverage.out ./...
Build arguments may be provided to influence the selection of source
packages; these are standard build arguments to the go tools, and
are specified with a --build-arg prefix; the corresponding
environment variable, OVERCOVER_BUILD_ARG, should contain a
space-separated sequence of build arguments.
Finally, summary information on the coverage per package can be
emitted using --summary (OVERCOVER_SUMMARY), and detailed
per-file information can be emitted using --detailed
(OVERCOVER_DETAILED); these two options may be used together, in
which case the package data will be emitted prior to the file data.
The output is sorted first to place the lowest coverage at the top,
then it is sorted lexically by package or file name.
Overcover can also read a configuration file containing the target
threshold, which may be in any number of formats as long as the file
has a proper extension; e.g., for a YAML file, the configuration file
name should end with ".yaml" or ".yml". The values in this file are
named the same as the command line options, and the configuration file
may be specified to Overcover using either OVERCOVER_CONFIG or the
--config command line option:
% overcover --config config.yaml
An example of that file could be as follows:
--- threshold: 75
When using a configuration file with Overcover, the Overcover tool can
automatically update the threshold in the file, to help ensure that
coverage can never decrease. This is done using the headroom
options, --min-headroom and --max-headroom
(OVERCOVER_MIN_HEADROOM and OVERCOVER_MAX_HEADROOM for
environment variables, and min_headroom and max_headroom in
the configuration files). If the total coverage exceeds the threshold
plus the maximum headroom, then the threshold will be updated to be
the total coverage minus the minimum headroom. An example of a
configuration file set up to use this automatic update feature would
be as follows:
--- threshold: 75 min_headroom: 1 max_headroom: 2
To provide a concrete example of the automatic threshold update, let's
assume that we're using the above configuration file, and that the
actual coverage is 80%. Since max_headroom is set to 2, that
means that the coverage would be greater than threshold +
max_headroom (77%), so the threshold would be updated to the
coverage minus the min_headroom, or 79%, and this configuration
file would become:
--- threshold: 79 min_headroom: 1 max_headroom: 2
(Note that the configuration file after the update likely won't look exactly like this, as it is programmatically generated.)
Automatic threshold update only occurs when a configuration file is
specified using --config or the environment variable
OVERCOVER_CONFIG, and when max_headroom is set to a value
strictly greater than min_headroom. Defaults for all these values
are shown below.
Sometimes it is convenient to prevent an automatic update of the
configuration file, such as during a CI run. This can be done by
using the --readonly flag, or setting the OVERCOVER_READONLY
environment variable. (The variable need not have any particular
value; it need only appear in the environment.) In this scenario,
Overcover will exit with a status code of 5 to indicate that the
configuration file needs to be updated with a new threshold; it will
also emit a human-readable error message to the standard error stream
indicating what the threshold should be set to.
| Configuration | Environment Variable | Command Line Option | Default | Description |
|---|---|---|---|---|
| OVERCOVER_COVERPROFILE | --coverprofile (-p) | Required | Name of the coverage profile generated by go test. |
|
| threshold | OVERCOVER_THRESHOLD | --threshold (-t) | 0.0 | Minimum coverage threshold required. |
| min_headroom | OVERCOVER_MIN_HEADROOM | --min-headroom (-m) | 0.0 | Minimum headroom. Used to compute a new threshold. |
| max_headroom | OVERCOVER_MAX_HEADROOM | --max-headroom (-M) | 0.0 | Maximum headroom. Used to determine when the threshold must be updated. |
| OVERCOVER_CONFIG | --config (-c) | None | Specifies the name of the configuration file to use. | |
| OVERCOVER_READONLY | --readonly (-r) | Specifies that the configuration file should not be updated. | ||
| OVERCOVER_BUILD_ARG | --build-arg (-b) | None | Specifies a build argument for package selection. | |
| OVERCOVER_SUMMARY | --summary (-s) | Specifies that per-package summary information should be emitted. | ||
| OVERCOVER_DETAILED | --detailed (-d) | Specifies that per-file coverage information should be emitted. | ||
| --help (-h) | Emits help text describing how to use Overcover. |
Key:
- Required - Option must be provided.
- None - No default, not required.