Skipped tests when parallelExecution is false

[felix-hedenstrom]

I've encountered an issue where when I run distage test kit some tests are skipped in 1.2.4. I did not manage to reproduce the issue in 1.2.3

If we look at this example

import izumi.distage.testkit.scalatest.{AssertZIO, SpecZIO}
abstract class Test extends SpecZIO with AssertZIO
class FooServiceTest extends Test {
  "FooService" should {
    "do foo" in {
      assertIO(1 == 1)
    }
  }
}
class BarServiceTest extends Test {
  "BarService" should {
    "do bar" in {
      assertIO(1 == 1)
    }
  }
}

and run it using sbt test when parallelExecution in ThisBuild := false I get

...
[info] BarServiceTest:
[info] - BarService should do bar
[info] FooService:
[info] Run completed in 6 seconds, 448 milliseconds.
[info] Total number of tests run: 1
[info] Suites: completed 2, aborted 0
[info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 7 s, completed Jan 19, 2024, 4:08:25 PM

with the notable part being Total number of tests run: 1 even though it detected both suites. If I set parallelExecution in ThisBuild := true
I get what I would expect

[info] BarServiceTest:
[info] FooServiceTest:
[info] - BarService should do bar
[info] - FooService should do foo
[info] Run completed in 5 seconds, 685 milliseconds.
[info] Total number of tests run: 2
[info] Suites: completed 2, aborted 0
[info] Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 6 s, completed Jan 19, 2024, 4:10:48 PM

I have observed similar issues in a larger real project with parallelExecution as both true and false but it is hard to reproduce it in a reasonably small example.

[felix-hedenstrom]

Maybe I'm reading your github actions incorrect but just in case I'm not I'm going to post it here anyways. I'm not that familiar with github actions and your testing flow.

Looking a test report before 1.2.4 I can see that every report has at least one passed test https://github.com/7mind/izumi/actions/runs/6739072294/job/18320219110

This is not the case for the latest build
https://github.com/7mind/izumi/runs/20581165785

[Kai]

@felix-hedenstrom Fixed in 1.2.5 release. Sorry about that! 🙏

NB: distage-teskit ignores parallelExecution, it only respects options set in TestConfig#parallelTests/Suites/Envs