flank exited with status code 10: at least one matrix not finished (usually a FTL internal error) or unexpected error occurred

[Ziad Tawfeek]

Hey folks 👋

We use Flank to run iOS UI tests on Firebase Test Lab. Recently we have been facing an issue that always results on failing the build pipeline due to exit code 10 but I'm struggling to understand what is the blocker here to troubleshoot

The way we configure flank is to run on three test shards on iPhone 8, two of the test shards are successfully executed but the remaining one always fails due to exit code 10.

Here is a snapshot of the logs in a CI run

  23m  1s matrix-pdssj0rgk79aa iphone11pro-14.7 Starting results processing. Attempt: 1
  21m 54s matrix-6ig6wi6bsq4ya iphone8-14.7 FINISHED
  22m  9s matrix-6ig6wi6bsq4ya iphone11pro-14.7 FINISHED
  20m  4s matrix-2o4e0un8jas68 iphone8-14.7 FINISHED
  19m 38s matrix-2o4e0un8jas68 iphone11pro-14.7 FINISHED
  23m  6s matrix-pdssj0rgk79aa iphone8-14.7 FINISHED
  24m 25s matrix-pdssj0rgk79aa iphone11pro-14.7 FINISHED
  21m 54s matrix-6ig6wi6bsq4ya iphone8-14.7 FINISHED
  22m  9s matrix-6ig6wi6bsq4ya iphone11pro-14.7 FINISHED
  20m  4s matrix-2o4e0un8jas68 iphone8-14.7 FINISHED
  19m 38s matrix-2o4e0un8jas68 iphone11pro-14.7 FINISHED
  24m 25s matrix-pdssj0rgk79aa FINISHED
  24m 25s matrix-2o4e0un8jas68 FINISHED
  24m 25s matrix-6ig6wi6bsq4ya FINISHED

It seems that the results processing of the matrix-pdssj0rgk79aa is not returning back the results to flank but the logs states here is FINISHED and I don't think there is anything that could be done to optimize the tests here

I would appreciate if anyone could point to any solutions or troubleshooting that could be done here

[Drilon Reçica]

@ziadtawfeek I am having a similar issue, did you fix it and if yes, how?

[Ziad Tawfeek]

@drilonrecica This is what flank throws when one or more tests are failing or an infrastructure failure occurs, so the error message is not really descriptive of the issue happening