Support tests written using Maestro

[Tarek Belkahia]

Is your feature request related to a problem? Please describe.

We write our mobile tests using Maestro

Describe the solution you'd like

Being able to run the maestro tests on Marathon

Describe alternatives you've considered

• Rewriting the whole testsuite using espresso and XCUITests ❌
• Running the tests on Maestro's cloud offering ❌
• Running the tests on our CI pipeline by handling the creating and launching of Simulators/Emulators ✅

Additional context

This would probably require changes to maestro itself as well to get better access to the internals

[Anton Malinskiy]

At the very minimum, the maestro cli will need to support listing tests i.e. test parsing (not currently available to my knowledge).
Secondly, I'm not familiar with the nuances of writing maestro tests, but marathon will require maximum granularity in terms of test filtering. Currently, I see test-suite-name in the filters, if this is the minimal executable unit of tests then this should be fine as long as multiple suites can be specified to support batching. If there is something smaller then test suite then it should be reflected both in the test parser and filters.

➜ maestro test --help
Missing required parameter: '<flowFile>'
Usage: maestro test [-c] [--[no-]ansi] [--debug-output=<debugOutput>] [--format=<format>] [--output=<output>] [--test-suite-name=<testSuiteName>]
                    [-e=<String=String>]... [--exclude-tags=<excludeTags>[,<excludeTags>...]]... [--include-tags=<includeTags>[,<includeTags>...]]... <flowFile>
Test a Flow or set of Flows on a local iOS Simulator or Android Emulator
      <flowFile>
  -c, --continuous
      --debug-output=<debugOutput>
                          Configures the debug output in this path, instead of default
  -e, --env=<String=String>

      --exclude-tags=<excludeTags>[,<excludeTags>...]
                          List of tags that will remove the Flows containing the provided tags
      --format=<format>   Test report format (default=NOOP): JUNIT, NOOP
      --include-tags=<includeTags>[,<includeTags>...]
                          List of tags that will remove the Flows that does not have the provided tags
      --[no-]ansi, --[no-]color
                          Enable / disable colors and ansi output
      --output=<output>
      --test-suite-name=<testSuiteName>
                          Test suite name

For iOS, maestro currently uses idb which is not required by marathon at all and will likely cause additional prework on the infrastructure side outside outside of marathon.

For Android, keep in mind that marathon does not start Android emulators, so this step is outside of the responsibilities of OSS marathon.

I'd be happy to help support the development of integration here, assuming the above prerequisites are satisfied.