Slow test performance

[Jeff Lockhart]

Describe the bug
I found that Kover slows down JVM test runs, in some cases when using parallel coroutines by a lot.

Expected behavior
I was unaware that Kover was running on all JVM test runs, and not just when generating its reports. I would expect normal JVM test runs shouldn't be impacted by Kover.

I'd also expect parallel coroutines to not be significantly slower than running the same code sequentially.

Reproducer
kover-slow-test.zip

Reports
Running this test without Kover on a 16-core CPU:

0 249.901979ms
1 248.530566ms
2 254.228630ms
3 249.449091ms
4 239.983553ms
5 243.419348ms
6 243.652377ms
7 249.540971ms
8 252.376248ms
9 256.178971ms
10 252.962235ms
11 245.185620ms
12 250.231187ms
13 246.561204ms
14 251.857170ms
15 247.229061ms
Without coroutines: 4.012906210s
0 276.228992ms
3 277.375637ms
15 275.671535ms
12 277.222348ms
10 278.674351ms
1 280.761071ms
2 281.079150ms
7 281.659647ms
5 281.960807ms
11 281.052791ms
14 281.033821ms
9 282.077916ms
4 282.861293ms
8 283.675079ms
6 285.752790ms
13 286.216757ms
With coroutines: 348.880909ms

Running with Kover:

0 570.481784ms
1 495.259729ms
2 498.058526ms
3 493.501416ms
4 488.237780ms
5 486.254588ms
6 495.960375ms
7 502.377977ms
8 484.046018ms
9 503.903930ms
10 489.172675ms
11 503.375202ms
12 505.227134ms
13 490.989507ms
14 486.554507ms
15 491.134007ms
Without coroutines: 8.013552646s
0 14.861984062s
14 14.867873495s
9 14.915033116s
8 14.933522893s
11 14.955004188s
7 14.974335832s
2 14.985772431s
6 14.991468946s
10 15.022878786s
5 15.025456945s
3 15.026649970s
1 15.041704843s
4 15.044281802s
13 15.042278650s
12 15.047098948s
15 15.046905120s
With coroutines: 15.084903500s

The parallel coroutines take twice as long as running the same code sequentially. Sometimes the second parallel part of the test will take even longer. I observed up to 45 seconds.

Environment

• Kover Gradle Plugin version: tested 0.6-0.7.5
• Gradle version: tested 8.2-8.6
• Kotlin project type: Kotlin/JVM or Kotlin/Multiplatform with JVM target
• Coverage Toolset (if customized in build script): N/A
• Other context important for this bug: N/A

[Sergey Shanshin]

Hi,
class instrumentation modifies the class code to measure the execution of each line/branch of code - this increases the amount of bytecode, thereby disabling many JVM optimizations.
Saving the fact of calling each line occurs in a common memory - this can have a big side effect in the form of performance degradation when intensively calling the same methods in parallel.

It is recommended to disable instrumentation during performance testing and concurrency tests.

Due to the Gradle build cache, task settings should depend only on the build configuration, but not on the list of tasks to run.
Therefore, you can add a parameter to enable instrumentation if you need to generate a Kover report.

For example, add such code to each subproject:

kover {
    if (!hasProperty("enableKover")) {
        disable()
    }
}

and pass the parameter to the command
./gradlew koverHtmlReport -PenableKover

[Jeff Lockhart]

Thanks for the clarification and workaround. I think this would be helpful to clarify this behavior in the README documentation.

[Matt Groth]

Yes I have been doing somethig like -PenableKover for a while now for exactly this reason.