[Proposal] Add more detailed Scala Test Classes Request

[Kamil Podsiadło]

Proposition

Currently, mentioned request returns

export interface ScalaTestClassesResult {
  items: ScalaTestClassesItem[];

  /** An optional id of the request that triggered this result. */
  originId?: String;
}

export interface ScalaTestClassesItem {
  /** The build target that contains the test classes. */
  target: BuildTargetIdentifier;

  /** The fully qualified names of the test classes in this target */
  classes: String[];
}

and I want to focus on ScalaTestClassesItem.classes. It consists of fully qualified names, there is no additional information about e.g. test framework to which the given test class belongs. It's fine until BSP client uses other BSP endpoints to execute test such as test request or debug request. However performing some action which requires knowledge about test framework by BSP client is very challenging - client has to somehow determine test framework of the given test class.

In order to solve that issue, aforementioned request can be slightly altered in order to contain such information:

case class ScalaTestSuitesResult(
  items: List[ScalaTestSuitesItem]
)
case class ScalaTestSuitesItem(
    target: BuildTargetIdentifier,
    suites: List[ScalaTestFrameworkSuites]
)
case class ScalaTestFrameworkSuites(
    framework: String,
    // Fully qualified names of test classes
    classes: List[String]
)

Since most (if not even all) known to me BSP servers uses sbt-test-interface under the hood it is reasonable to define value of ScalaTestFrameworkSuites.framework field to be the value of Framework.name. Then BSP Client can perform some framework-specific logic based on this field value.

Motivation

Metals and test explorer are concrete example of how knowing test framewor could be used:

• it will allow more fine grained discovery of tests: e.g. search for @Test annotations for junit. Different strategy has to be applied for each framework.
• it will allow Metals to implement their own TestRunner. Currently for running tests Metals are using debug request with no breakpoints. Implementing TestRunner in Metals will decrease number of moving parts and reduce overhead related with Debug Adapter Protocol stuff which will result in a better user experience.

Implementation difficulty

I've looked at bloop, sbt and mill to estimate if this endpoint can be implemented in each of them - it's doable and it seems to not be difficult. I've even implemented this endpoint for bloop already as a PoC.

[Tomasz Godzik]

I think this is a great idea. I think Intellij could also benefit from that information, no? CC @jastice

[Adrien Piquerez]

This proposal makes perfect sense to me.

In term of implementation I would suggest a slightly different format:

case class ScalaTestSuitesResult(
  items: List[ScalaTestSuitesItem]
)
case class ScalaTestSuitesItem(
    target: BuildTargetIdentifier,
    framework: Option[String],
    suites: List[ScalaTestFrameworkSuites]
)

First because I think it is more simple to have one layer (ScalaTestSutiesItem) rather than two (ScalaTestSuitesItem and ScalaTestFrameworkSuites).

More importantly it maintains the compatibility: Newer BSP servers can respond to older clients with this format and newer clients can read this format from older servers.
The framework field would be optional for this reason only, but it should be expected in newer implementations.

[Kamil Podsiadło]

Yeah, maintaining compatibility might be easier to handle on both sides, client and server, than a brand new endpoint. Thanks @adpi2!

[Kamil Podsiadło]

closed via #300