The plugin provides an ability to perform a mutation testing and calculate a mutation coverage of a Gradle-based projects with PIT.
Add gradle-pitest-plugin to the plugins configuration in your build.gradle file:
plugins {
id 'info.solidsoft.pitest' version '1.4.0'
}Call Gradle with pitest task:
gradle pitest
After the measurements a report created by PIT will be placed in ${PROJECT_DIR}/build/reports/pitest directory.
Optionally make it depend on build:
build.dependsOn 'pitest'Note that when making pitest depend on another task, it must be referred to by name. Otherwise Gradle will resolve pitest to the configuration and not the task.
Please take into account that only versions starting with 1.1.11 are available via the Plugin Portal.
"The plugins way" has some limitations. As the primary repository for the plugin is the Central Repository (aka Maven Central) it is also possible to add the plugin to your project using "the generic way":
buildscript {
repositories {
mavenCentral()
//Needed only for SNAPSHOT versions
//maven { url 'https://oss.sonatype.org/content/repositories/snapshots/' }
}
dependencies {
classpath 'info.solidsoft.gradle.pitest:gradle-pitest-plugin:1.4.0'
}
}Apply the plugin:
apply plugin: 'info.solidsoft.pitest'For versions <1.1.0 the plugin can be applied with:
apply plugin: 'pitest'The Pitest plugin has to be configured. All the command line options are
supported. To make life easier taskClasspath, mutableCodePaths, sourceDirs, reportDir and pitestVersion are
automatically set by a plugin. In addition sourceDirs, reportDir and pitestVersion can be overridden by an user.
In the past there was one mandatory parameter - targetClasses - which points to the classes which should be mutated.
Starting from 0.32.0 it is only required if a group
for the project is not set. Otherwise value "${project.group}.*" is set by default (which can be overridden using pitest.targetClasses parameter).
In case of using not default PIT version the pitestVersion parameter should be used to override it.
The configuration in Gradle is the real Groovy code which makes all assignments very intuitive. All values expected by
PIT should be passed as a corresponding types. There is only one important difference. For the parameters where PIT expects
a coma separated list of strings in a Gradle configuration a list of strings should be used (see outputFormats in the
following example).
pitest {
targetClasses = ['our.base.package.*'] //by default "${project.group}.*"
pitestVersion = '1.1.0' //not needed when a default PIT version should be used
threads = 4
outputFormats = ['XML', 'HTML']
timestampedReports = false
}Check PIT documentation for a list of all available command line parameters. The expected parameter format in a plugin configuration can be taken from PitestPluginExtension.
There are a few parameters specific for Gradle plugin:
testSourceSets- defines test source sets which should be used by PIT (by default sourceSets.test, but allows to add integration tests located in a different source set) (since 0.30.1)mainSourceSets- defines main source sets which should be used by PIT (by default sourceSets.main) (since 0.30.1)mainProcessJvmArgs- JVM arguments to be used when launching the main PIT process; make a note that PIT itself launches another Java processes for mutation testing execution and usuallyjvmArgsshould be used to for example increase maximum memory size (since 0.33.0 - see #7);additionalMutableCodePaths- additional classes to mutate (useful for integration tests with production code in a different module - since 1.1.4 - see #25)useClasspathFile- enables passing additional classpath as a file content (useful for Windows users with lots of classpath elements, disabled by default - since 1.2.2)fileExtensionsToFilter- provides ability to filter additional file extensions from PIT classpath (since 1.2.4 - see #53)
For example:
pitest {
...
testSourceSets = [sourceSets.test, sourceSets.integrationTest]
mainSourceSets = [sourceSets.main, sourceSets.additionalMain]
jvmArgs = ['-Xmx1024m']
useClasspathFile = true //useful with bigger projects on Windows
fileExtensionsToFilter += ['xml']
}gradle-pitest-plugin can be used in multi-module projects. The gradle-pitest-plugin dependency should be added to the buildscript configuration in the root project while the plugin has to be applied in all subprojects which should be processed with PIT. A sample snippet from build.gradle located for the root project:
//in root project configuration
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath 'info.solidsoft.gradle.pitest:gradle-pitest-plugin:1.4.0'
(...)
}
}
subprojects {
...
apply plugin: 'info.solidsoft.pitest' //'pitest' for plugin versions <1.1.0
pitest {
threads = 4
if (project.name in ['module-without-any-test']) {
failWhenNoMutations = false
}
}
}Currently PIT does not provide an aggregated report for multi-module project. A report for each module has to be browsed separately. Alternatively a PIT plugin for Sonar can be used to get aggregated results.
Since gradle-pitest-plugin 1.1.4 it is possible to mutate code located in different subproject. Gradle internally does not rely on
output directory from other subproject, but builds JAR and uses classes from it. For PIT those are two different sets of class files, so
to make it work it is required to define both mainSourceSets and additionalMutableCodePaths. For example:
configure(project(':itest')) {
apply plugin: 'info.solidsoft.pitest'
dependencies {
compile project(':shared')
}
configurations { mutableCodeBase { transitive false } }
dependencies { mutableCodeBase project(':shared') }
pitest {
mainSourceSets = [project.sourceSets.main, project(':shared').sourceSets.main]
additionalMutableCodePaths = [configurations.mutableCodeBase.singleFile]
}
}The above is the way recommended by the Gradle team, but in specific cases the simpler solution should also work:
configure(project(':itest')) {
apply plugin: 'info.solidsoft.pitest'
dependencies {
compile project(':shared')
}
pitest {
mainSourceSets = [project.sourceSets.main, project(':shared').sourceSets.main]
additionalMutableCodePaths = project(':shared').jar.outputs.files.getFiles()
}
}Minimal working multi-project build is available in functional tests suite.
Test plugins are used to support different test frameworks than JUnit4. They are officially supported by gradle-pitest-plugin staring with version 1.1.4 (although it was possible to use it since 1.1.0).
To enable PIT plugins, it is enough to add it to the pitest configuration in the buildscript closure and also set the testPlugin property. For example:
buildscript {
repositories {
mavenCentral()
}
configurations.maybeCreate('pitest')
dependencies {
classpath 'info.solidsoft.gradle.pitest:gradle-pitest-plugin:1.4.0'
pitest 'org.pitest:pitest-junit5-plugin:0.8'
}
}
pitest {
testPlugin = 'junit5' //or built-in 'testng' which also has to be activated
// rest of your pitest configuration
}The minimal working example is available in functional tests suite.
Every gradle-pitest-plugin version by default uses a predefined PIT version. Usually this a the latest released version
of PIT available at the time of releasing a plugin version. It can be overridden by using pitestVersion parameter
in a pitest configuration closure.
Please be aware that in some cases there could be some issues when using non default PIT versions.
gradle-pitest-plugin 1.3.x by default uses PIT 1.3.x, 1.2.x uses PIT 1.2.x, etc.
Starting with version 1.4.0 gradle-pitest-plugin requires Gradle 4.0. Previous version provided support for Gradle 2.0+. The current version was automatically smoke tested with Gradle 4.0, 4.10.2, 5.0 and 5.1.1 under Java 8. Tests with Java 9, 10 and 11 are limited to the compatible versions of Gradle and PIT.
Java 11 is officially supported starting with gradle-pitest-plugin 1.4.0 (thanks to using PIT 1.4.3). However, please check an appropriate point in FAQ how to achieve Java 11 compatibility in gradle-pitest-plugin 1.3.0.
Due to incompatible changes in Gradle 4.0 support for older Gradle versions will be removed in one of the future versions. The latest version which supports Gradle 2 and 3 is gradle-pitest-plugin 1.3.0. Gradle 1.x (1.6+) was supported by gradle-pitest-plugin 1.1.4.
Starting with the version 1.3.0 the produced binaries require Java 8 (as a JDK used for running a Gradle build). The latest version with Java 7 compatible binaries is 1.2.4.
See changelog file for more detailed list of changes in the plugin itself.
It should be fixed in PIT 0.29.
As a workaround in older versions add `jvmArgs = '-XX:-UseSplitVerifier'` to a pitest configuration block
pitest {
...
//jvmArgs = '-XX:-UseSplitVerifier' //<0.33.0
jvmArgs = ['-XX:-UseSplitVerifier'] //>=0.33.0
}2. Why have I got GroovyCastException: Cannot cast object '-Xmx1024', '-Xms512m' with class 'java.lang.String' to class 'java.util.List' after upgrade to version 0.33.0?
To keep consistency with the new mainProcessJvmArgs configuration parameter and make an input format more predictable jvmArgs parameter type was changed from String to List<String> in gradle-pitest-plugin 0.33.0. The migration is trivial, but unfortunately I am not aware of the way to keep both parameter types active at the same time.
pitest {
...
//jvmArgs = '-Xmx1024 -Xms512m' //old format
jvmArgs = ['-Xmx1024', '-Xms512m'] //new format
}Update. Spring Boot 1.1.0 is fully compatible with gradle-pitest-plugin.
There is was an issue with the way how spring-boot-gradle-plugin (<1.1.0) handles JavaExec tasks
(including pitest task which in the version 0.33.0 became JavaExec task to resolve classpath issue with configured non default PIT version - see
issue #7).
Luckily there is a workaround which allows to run PIT 0.33 (with Java 8 support) with gradle-pitest-plugin 0.32.0:
buildscript {
(...)
dependencies {
classpath('info.solidsoft.gradle.pitest:gradle-pitest-plugin:0.32.0') {
exclude group: 'org.pitest'
}
classpath 'org.pitest:pitest-command-line:0.33'
}
}
pitest {
pitestVersion = '0.33'
}Gradle does not provide a built-in way to override plugin configuration via command line, but gradle-override-plugin can be used to do that.
After applied gradle-override-plugin in your project it is possible to do following:
./gradlew pitest -Doverride.pitest.reportDir=build/pitReport -Doverride.pitest.threads=8
Note. The mechanism should work fine for String and numeric properties, but the are limitations with support of Lists/Sets/Maps and Boolean values.
For more information see project web page.
Could not resolve all dependencies for configuration ':pitest'.
> Could not find org.pitest:pitest-command-line:1.1.0.
Required by:
:Gradle-Pitest-Example:unspecified
Starting from version 1.0.0 for multi-project builds gradle-pitest-plugin dependency should be added to the buildscript configuration in the root project. The plugin should be applied in all subprojects which should be processed with PIT.
gradle-pitest-plugin by default uses a corresponsing PIT version (with the same number). The plugin is released only if there are internal changes or
there is a need to adjust to changes in newer PIT version. There is a dedicated mechanism to allow to use the latest PIT version (e.g, a bugfix release)
or to downgrade PIT in case of detected issues. To override a defalt version it is enough to set pitestVersion property in the pitest configuration
closure.
pitest {
pitestVersion = '2.8.1-the.greatest.one'
}In case of errors detected when the latest available version of the plugin is used with newer PIT version please raise an issue.
Placing PIT reports directly in ${PROJECT_DIR}/build/reports/pitest can be enabled with timestampedReports configuration property:
pitest {
timestampedReports = false
}8. How can I debug a gradle-pitest-plugin execution or a PIT process execution itself in a Gradle build?
Ocasionally, it may be useful to debug a gradle-pitest-plugin execution or a PIT execution itself (e.g. NPE in PIT) to provide sensible error report.
The gradle-pitest-plugin execution can be remotely debugged with adding -Dorg.gradle.debug=true to the command line.
However, as PIT is started as a separate process to debug its execution the following arguments need to be added to the plugin configuration:
pitest {
mainProcessJvmArgs = ['-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005']
}Short answer is: not directly. Due to some incompatibilities between "standard" Java applications and Android Java applications in Gradle the plugin does not support the later. Luckily, there is an Android fork of the plugin maintained by Karol Wrótniak which provides a modified version supporting Android applications (but on the other hand it doesn't work with standard Java applications).
Update. gradle-pitest-plugin 1.4.0 should work with Java 11 out of the box.
The gradle-pitest-plugin 1.3.0 is smoke testing with Java 8, 9, 10 and 11. However, to run PIT sucessfully with Java 11, it is required to override a default PIT version defined in the plugin to 1.4.3 (or preferably to the latest one). It can be simply done in the project configuration with:
pitest {
pitestVersion = '1.4.3' //for Java 11 compatibility with gradle-pitest-plugin 1.3.0
}-
too verbose output from PIT
-
0.33.0+ is not compatible with Spring Boot projects due to a bug in spring-boot-gradle-plugin - see FAQ for a workaround- works with Spring Boot 1.1.0+
gradle-pitest-plugin cloned from the repository can be built using Gradle command:
./gradlew build
The easiest way to make a JAR with local changes visible in another project is to install it into the local Maven repository:
./gradlew install
There are also basic functional tests written using nebula-test which can be run with:
./gradlew funcTest
gradle-pitest-plugin was written by Marcin Zajączkowski. The author can be contacted directly via email: mszpak ATT wp DOTT pl. There is also Marcin's blog available: Solid Soft - working code is not enough.
The plugin surely has some bugs and missing features. They can be reported using an issue tracker. However it is often a better idea to send a questions to the PIT mailing list first.
The plugin is licensed under the terms of the Apache License, Version 2.0.