Loss of location of assert when using smart asserts.

[Gerard Murphy]

Using:

libraryDependencies += "com.lihaoyi" %% "utest" % "0.8.1" % Test, running under IntelliJ.

Code (Scala 3.3.0):

package com.sageserpent.kineticmerge.core

import utest.*

object OddStackTraceReport extends TestSuite:
  def excised(item: Int): Unit = assert(item != item)

  override val tests: Tests = Tests:
    test("reproduction by causing a failure"):
      // Simulating a parameterised test ...
      val deferred = 0 until 10 map (item => () => excised(item))

      deferred.foreach(_.apply())

end OddStackTraceReport

Hi, this is a minimisation of a problem I'm experiencing when using uTest with a parameterised test framework.

It actually runs either standalone or integrated under JUnit, so the original problem does not use the uTest TestSuite, just the uTest assertions - nevertheless I can reproduce it using the 'standard' approach.

This test obviously fails, but the stack trace reported looks like this:

item != item
item: Int = 0
item: Int = 0
utest.AssertionError: item != item
item: Int = 0
item: Int = 0
	at utest.AssertionError$.apply(Errors.scala:15)
	at utest.asserts.Util$.makeAssertError$$anonfun$1(Util.scala:26)
	at utest.framework.StackMarker$.dropInside(StackMarker.scala:11)
	at utest.asserts.Util$.makeAssertError(Util.scala:29)
	at utest.asserts.Asserts$.assertImpl(Asserts.scala:30)
	at com.sageserpent.kineticmerge.core.OddStackTraceReport$.excised(OddStackTraceReport.scala:16)
	at com.sageserpent.kineticmerge.core.OddStackTraceReport$.$anonfun$1$$anonfun$1(OddStackTraceReport.scala:11)
	at com.sageserpent.kineticmerge.core.OddStackTraceReport$.$init$$$anonfun$1$$anonfun$1$$anonfun$1(OddStackTraceReport.scala:13)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.Vector.foreach(Vector.scala:1895)
	at com.sageserpent.kineticmerge.core.OddStackTraceReport$.$init$$$anonfun$1$$anonfun$1(OddStackTraceReport.scala:13)
	at utest.framework.StackMarker$.dropOutside(StackMarker.scala:13)
	at utest.framework.TestCallTree.run(Model.scala:45)
	at utest.framework.TestCallTree.run(Model.scala:43)
	at utest.TestRunner$.$anonfun$8$$anonfun$1(TestRunner.scala:74)

 etc...

Note how the invocation of the lambda is correctly reported at line 13, as is the helper method on line 11, but the actual assertion location is approximated by line 16, rather than being precisely line 6.

This isn't a problem here as the example is simple, but for realistic tests it is inconvenient.

Changing the import to be import utest.{TestSuite, Tests, test} uses the standard assert in Predef and fixes the stack trace:

assertion failed
java.lang.AssertionError: assertion failed
	at scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:11)
	at com.sageserpent.kineticmerge.core.OddStackTraceReport$.excised(OddStackTraceReport.scala:6)
	at com.sageserpent.kineticmerge.core.OddStackTraceReport$.$anonfun$1$$anonfun$1(OddStackTraceReport.scala:11)
	at com.sageserpent.kineticmerge.core.OddStackTraceReport$.$init$$$anonfun$1$$anonfun$1$$anonfun$1(OddStackTraceReport.scala:13)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.Vector.foreach(Vector.scala:1895)
	at com.sageserpent.kineticmerge.core.OddStackTraceReport$.$init$$$anonfun$1$$anonfun$1(OddStackTraceReport.scala:13)
	at utest.framework.StackMarker$.dropOutside(StackMarker.scala:13)
	at utest.framework.TestCallTree.run(Model.scala:45)
	at utest.framework.TestCallTree.run(Model.scala:43)
	at utest.TestRunner$.$anonfun$8$$anonfun$1(TestRunner.scala:74)

Oddly enough, if the helper excised is inlined into the lambda used to define deferred, then both approaches yield unsatisfactory stack traces, so I’m not completely sure how much of this is to do with uTest and how much is a quirk of the Scala implementation.

[Gerard Murphy]

To provide some context, what I was trying to do was to use uTest's assertions from within a JUnit test - this is because I use a parameterised test framework that integrates nicely with JUnit, but prefer uTest's assertions for Scala:

https://github.com/sageserpent-open/kineticMerge/blob/6019287ba6dc4aa272acc8e60c2f7887807debef/src/test/scala/com/sageserpent/kineticmerge/core/LongestCommonSubsequenceTest.scala#L59

[Gerard Murphy]

In the meantime, I've discovered Expecty, which as a standalone assertion library suits my purposes better.

I imagine that the majority of users of uTest will be using the whole framework to structure their tests - so a test will be defined inline to get picked up by the test macro rather than in a lambda, so this isn't likely to cause problems for them.

Feel free to close at will, but I'll leave this here just in case...