Unreadable test result
evis opened this issue · comments
Hi, do you have a link to this invocation on our cloud instance that you could share (or DM me in our Slack channel) if you're using hosted BuildBuddy? Or if not, do you mind downloading the test.xml file from that page and attaching it here? This test page is showing the results of the test.xml file so having that file would let us understand this issue a little better.
Sure, here it is: test.xml.zip
@bduffany is given test.xml helpful? Maybe, something else needed?
@evis it seems like the error was printed as is from your test.xml file. Here is how it looks redacted
> grep -B1 -A2 'tskv publish event.*offers\.DraftsManagerSpec' test.xml | less
<testcase
name="DraftsManager should write tskv publish event" classname="ru.auto.api.managers.offers.DraftsManagerSpec" time="0.014">
<failure
message="Unexpected not mocked invocation getLatestResult(car_info {&#010; mark: "AUDI"&#010; model: "A4"&#010; super_gen_id: 20637504&#010; configuration_id: 20637561&#010; tech_param_id: 20693205&#010;}&#010;color_hex: "ff0000"&#010;status: DRAFT&#010;category: MOTO&#010;section: USED&#010;availability: IN_STOCK&#010;price_info {&#010; ... Request(unknown, [empty trace]))" type="class ru.yandex.vertis.mockito.NotMockedException">
Is this what you are expecting from your test?
Is it possible to reduce the amount of data dumped from your log? 🤔
The problematic part from the snippet that Son showed is this part:
<failure
message="Unexpected not mocked invocation getLatestResult(car_info {&#010; mark: "
note that the message is quite verbose and contains entities like &#010; etc.
I don't have a ton of experience with these XML files and hoping other folks can chime in if they have more experience with this, but from reading the XML schema documentation here I think that in normal usage, the message attribute (assert message) is intended for a short summary of the error (a few words), while the text content inside the <failure> tag should contain more details such as the stack trace. It appears that in the XML doc you provided, the same full error message is included in both the message attribute and the text of the tag.
Which bazel test rules are you using? is it just java_test or something else? Also what test library are you using? What does the code for the failing assertion look like? Having this info would help us understand this a little better and possibly reproduce it.
Given @evis profile, I think it's from rules_scala with https://github.com/bazelbuild/rules_scala/blob/e2fe29c3b51647ecd07a562917c2c09fb48f9c0a/scala/support/JUnitXmlReporter.scala#L412-L437
A quick fix that I could think of is to patch rules_scala so that the message attribute will only include the first line of failure.message. That will give you a much more readable error with Junit XML and Bazel (and BuildBuddy).
@evis could you please open an issue on rules_scala to report this? I think rules_scala maintainers would have more context on how scalatest runner created these events, as well as best practices from other Scala build tools to replicate over.
Since this is XML, shouldn't the entities be translated correctly when displayed? So & should become &.
Yeah, I think it's encoded twice. Once in rules_scala for \n to become , then the second time for & into &, resulting in &#10;.
We could reverse these operations on our end. However, it does not change the fact that rules_scala is being silly, fitting a huge stack trace into the message attribute of <failure>...</failure>. The same error + stack trace is again shoved into the text body inside this tag, which duplicates the content.
I think the correct fix here is to ensure that message only uses the first line in TestFailed event of scalatest. That way there would be no new line to encode/decode. We are still displaying the stack trace from the text body below.
Ima close this as not planned. We could reopen the issue if there is a valid use case for new line characters inside these XML attributes.