MarathonLabs / marathon

Cross-platform test runner

Home Page:https://docs.marathonlabs.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

No logs and empty failure element in marathon_junit_report.xml for uncompleted tests

Twinsen81 opened this issue · comments

Describe the bug
If a test fails with the <TestRunResultsListener> uncompleted error, then:

  • No .log file is generated under marathon/logs except for the batch UUID files
  • The failure element in marathon_junit_report.xml is empty

To Reproduce
Steps to reproduce the behaviour:

  1. Have testOutputTimeoutMillis = 60000 or less in the configuration file
  2. Have a test that looks for a view with RootMatchers.isDialog() when there's no dialog on the screen
  3. Run the test with Marathon
  4. The test fails with the <TestRunResultsListener> uncompleted error

Actual behavior

  • No .log file is generated under marathon/logs
  • The batch log file (UUID-named) contains the logs of the test run with the cause of the failure (no matching root view in this case)
  • The failure element in marathon_junit_report.xml is empty

image_2023-10-25_12-34-12

<testcase name="whenClickPostOverflowMenuThenReportingOptionsShown" time="60.082" classname="com.tumblr.reporting.ReportingBlockingTest"><failure></failure></testcase>

Expected behavior

  • The failure element in marathon_junit_report.xml contains some information about the failure. At least: The test didn't complete within the testOutputTimeoutMillis timeout.
  • There's a .log file for the failed test, and it contains the logs of the test run with the cause of the failure - what I can see in the batch logs (UUID-named files)

Added the failure message in the PR. As for the logs - this is a consistency issue with how logcat works in Android, I don't think it's possible to fix currently