helmethair-co / scalatest-junit-runner

JUnit 5 runner for Scalatest

Repository from Github https://github.comhelmethair-co/scalatest-junit-runnerRepository from Github https://github.comhelmethair-co/scalatest-junit-runner

RunAborted events are not reported

SheliakLyr opened this issue · comments

Describe the bug
One of my suites throws an exception in beforeAll method. To my surprise, this problem is not reported back to gradle and build finishes with a success.

Debugging shows that this code in Executor.java is triggered after an exception:

            } else {
                reporter.apply(runAborted(args.tracker().nextOrdinal(), e, Resources.bigProblems(e), scalasuite));
                if (!NonFatal$.MODULE$.apply(e)) {
                    throw e;
                }
            }

In my case, !NonFatal$.MODULE$.apply(e) is evaluated to false.

To Reproduce

  1. Create a gradle project that uses scala + scalatest + scalatest-junit-runner.
  2. Create a suite that throws RuntimeException in beforeAll()
  3. Run gradle test. Build should finish successfully.

Expected behavior
Build fails with a message about aborted runs.

Build/Test environment:

  • Gradle 6.6
  • Scala version 2.13.3
  • Scalatest version 3.2.0
  • scalatest-junit-runner 0.1.8

Thank you @SheliakLyr, will look into this. Can you please verify that:

  • the exception is thrown inside a beforeAll block
  • The exception is not a RuntimeException and is a subclass of Exception

As far as I see, unfortunately Junit5+Gradle handles aborted executions differently how I expected and it does not break the build.

The exception is thrown inside beforeAll() method, like this:

class SomeCodeTest extends AnyFunSpec with Matchers with BeforeAndAfterAll {
  override def beforeAll(): Unit = {
    super.beforeAll()
    throw new RuntimeException("Test exception for junit-runner")
  }

The exception is an instance of RuntimeException. The same thing happens for an instance of Exception (new Exception(...)).

I have uploaded a sample project based on your example: https://github.com/SheliakLyr/junit-runner-failure

Fixed in v0.1.9. Please allow some time for Central Repository to propagate the release. Thank you very much for the report, @SheliakLyr !