parsecmacher / grunt-code-coverage-enforcer

This is a grunt task that will read the LCOV file and fails the build if it doesn't meet the specified thresholds

grunt-code-coverage-enforcer

Enforces code coverage thresholds for a project with lcov files. This means it works with a wide range of code coverage tools like blanketjs, istanbul, .....

Introduction

Most code coverage enforcers are tied to a specific code coverage tool like blanketjs or istanbul. This plugin works on the lcov file that can be generated by most test runners.

As a result you can work with any tests runners that are capable of generating the widely used LCOV file format.

Why is our plugin better

Most other coverage threshold tools fail the build only when you have a test and the test doesn't meet the threshold criteria. Most tools will actually pass the build if you dont write any tests for your project.

This plugin will check and fail if there are files that have not been covered at all.

You can always exclude the files that you specifically do not want to test for coverage. Since the policy is opt-out, less files will slip through the cracks ;-)

---

Getting Started

npm install grunt-code-coverage-enforcer --save-dev

Once the plugin has been installed, it may be enabled inside your Gruntfile with this line of JavaScript:

grunt.loadNpmTasks('grunt-code-coverage-enforcer');

The "code-coverage-enforcer" task

Enabling lcov reporter in your test runner.

You will need to enable lcov reporter in your test runner. The specific config for this may vary for each runner. Please check your specific runner for instructions.

Sample Intern config

intern:  {
   runner: {
        options: {
            config: "<intern- config>",
            runType: "runner",
            reporters: ["console", "lcov"]
        }
    }
  }

Sample Karma config

(http://karma-runner.github.io/0.8/config/coverage.html)

coverageReporter = {
  type : 'lcovonly',
  dir : 'coverage/',
  file : 'lcov.info'
}

Sample Istanbul config

(https://github.com/gotwarlost/istanbul, https://www.npmjs.org/package/grunt-istanbul-coverage, https://www.npmjs.org/package/grunt-mocha-istanbul)

Cli version of istanbul has a report parameter that you need to set to lcov or lcovonly. By default it should generate the lcov files. Most istanbul plugin allow this to be passed into their grunt config as well.

Here is one suc example for the grunt-mocha-istanbul plugin

  options: {
                    reportFormats: ['cobertura','lcovonly']
          }

Mocha

Mocha has an lcov reporter that can be foounf here https://nodejsmodules.org/pkg/mocha-lcov-reporter You can also use istanbul with Mocha using the plugin here https://github.com/pocesar/grunt-mocha-istanbul

This should typically generate an lcov.info file that will specified as input into the code-coverage-enforcer plugin.

Overview

In your project's Gruntfile, add a section named code-coverage-enforcer to the data object passed into grunt.initConfig().

grunt.initConfig({
  "code-coverage-enforcer": {
    options: {
        lcovfile: "lcov.info",
        lines: 60,
        functions: 60,
        branches: 60,
        src: "src",
        includes: ["src/**/*.js"],
        excludes: ["src/uncoveredfile.js"]
    }
  },
})

the default threshold values are

          {
            lines: 50,
            functions: 50,
            branches: 0,
            includes: ["**/*.js"],
            src: process.cwd(),
            excludes: []
        }

You can also provide multiple configurations for different features/packages of your project.

Options

lines

Type: Number Default value: 50

A Number value that is used to specify in % the line coverage required.

functions

Type: Number Default value: 50

A Number value that is used to specify in % the function coverage required.

branches

Type: Number Default value: 50

A Number value that is used to specify in % the branches coverage required.

src

Type: string Default value: src

The location of the folder within which we will search for files.

includes

Type: matcher Default value: *

The matcher required to compare files within the src location that are to be included

excludes

Type: [matcher] Default value: []

The matchers required to compare files within the src location that are to be excluded.

Usage Examples

Default Options

In this example, the default options are used

grunt.initConfig({
  "code-coverage-enforcer": {
    options: {
        lcovfile: "lcov.info",
        includes: ["src/**/*.js"],
        excludes: ["src/uncoveredfile.js"]
    }
  },
})

Custom Options

In this example, custom options are used to specify the threshold limits

grunt.initConfig({
  "code-coverage-enforcer": {
    options: {
        lcovfile: "lcov.info",
        lines: 60,
        functions: 60,
        branches: 60,
        src: "src",
        includes: ["src/**/*.js"],
        excludes: ["src/uncoveredfile.js"]
    }
  },
})

With the release of version 0.2.0, you will be able to customize the code-coverage on and individual folder/package level. The configuration options on an individual folder/package level can be specified in the following way:

grunt.initConfig({
  "code-coverage-enforcer": {
    options: {
        lcovfile: "lcov.info",
        lines: 60, //Global line coverage configuration
        functions: 60, // Global function coverage configuration
        branches: 60, // Global branch coverage configuration
        src: [{
          path:"middleware", 
          lines:90
        }, {
          path:"backend", 
          lines: 90, 
          functions: 80
        }, {
          path:"frontend", 
          lines: 90, 
          functions: 80,
          branches:70, 
          includes:["**.js"], 
          excludes:["frontend/exclude.js"]
        }
        includes: ["src/**/*.js"],
        excludes: ["src/uncoveredfile.js"]
    }
  },
})

In the above example we configured our project to have the following three coverage configuration

1. 90% line coverage, 60% function coverage, 60% branch coverage for all the files in middleware folder.
2. 90% line coverage, 80% function coverage, 60% branch coverage for all the files in backend folder.
3. 90% line coverage, 80% function coverage, 70% branch coverage for all the files in frontend folder except exclude.js file.

If any of the configuration option is not provided for a given folder/package, the default configurations are chosen.

Recommendations

We recommend using a specific task names build acceptance that will run the code-coverage-enforcer along with other build acceptance type of tasks.

Here is an example config that should go into your Gruntfile

    grunt.registerTask("build-acceptance", ["code-coverage-enforcer"]);

You can now simply invoke this from command line using

grunt build-acceptance

Contributing

The project includes jshint and jscs config as part of its dev grunt tasks. Please run grunt lint to ensure that the code passes the code conventions.

Once you have your code ready ready please make a pull request to the author.

Release History

March 17 2-15 0.2.0: Threshold configuration now supports per folder configuration in addition to one config for all.

June 19 2014 Initial Release.

License

Copyright (c) 2014 Intuit Inc.

Licensed under the MIT license.