With MUnit / TestContainersForAll, containers start even if the suite is excluded via categories

[Tommaso Petrucciani]

Using MUnit and TestContainersForAll, if a test suite is skipped via filtering by category (https://scalameta.org/munit/docs/filtering.html#group-test-suites-with-categories), the containers start up anyway.

It looks like TestContainersForAll only checks that the suite has at least one test without the Ignore tag, but does not consider other ways the suite (or all tests in it) could be skipped:

https://github.com/testcontainers/testcontainers-scala/blob/master/test-framework/munit/src/main/scala/com/dimafeng/testcontainers/munit/TestContainersForAll.scala#L46

Is there some other way to skip a whole test suite and not start up the containers?

(The use case is to be able, by filtering on category, to run only unit tests in an environment that doesn't have Docker.)

[Dmitry F]

@tpetrucciani thanks for reporting it. I'm not an expert in MUnit but by looking at the documentation, I'd assume that it should work out of the box.

override def munitTests(): Seq[Test] = {
    val default = super.munitTests()
    if (!Properties.isWin) default
    else default.filter(_.tags.contains(Windows))
  }

Wouldn't it return an empty Seq that will get into this if statement - https://github.com/testcontainers/testcontainers-scala/blob/master/test-framework/munit/src/main/scala/com/dimafeng/testcontainers/munit/TestContainersForAll.scala#L46

[Yuriy Badalyants]

Maybe @psisoyev could help?

[Tommaso Petrucciani]

Hi, thanks for the quick response.

I'm not sure if I was clear: I meant specifically excluding whole test suites from running using the include/exclude categories feature in https://scalameta.org/munit/docs/filtering.html#group-test-suites-with-categories (actually a JUnit feature I think).

This is useful e.g. to distinguish unit from integration test suites and run only unit or only integration with different SBT commands (whereas I think the example with override def munitTests() is more about filtering individual test cases based on a dynamic condition, like Properties.isWin – not sure if it can be used for this purpose).

I've put a small example here: https://github.com/tpetrucciani/testcontainers-scala-issue-171.

Running it with sbt "testOnly -- --exclude-categories=issue171.Slow" starts up containers even if no tests are actually run. So it seems that excluding the category does not affect the result of munitTests.

[Yuriy Badalyants]

@tpetrucciani it looks like you have the biggest context about this problem. Maybe you could try to fix it?

[Tommaso Petrucciani]

👍 I'll try and have a look at it.

[Tommaso Petrucciani]

Sorry for never getting round to this, but anyway it's been fixed by MUnit (issue scalameta/munit#497, PR scalameta/munit#499), so this is no longer relevant for the release of MUnit supported by testcontainers-scala. I'm closing this as completed.