Giters

Flank / flank

:speedboat: Massively parallel Android and iOS test runner for Firebase Test Lab

Home Page:https://firebase.community/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

flank reports always as failure on macOS stack

andreschab90 opened this issue · comments

commented

Describe the bug

  • flank steps reports always as failure on macOS stack
  • ubuntu stack works fine
  • step finishes with exit code 1 always, even when all tests succeeded due to an illegal option in the command

Screenshot 2022-11-18 at 21 01 44

To Reproduce

  • run flank on macOS

Expected behavior

step should not fail when all tests were successful

Details (please complete the following information):

Have you tested on the latest Flank snapshot?
yes, same issue with snapshot

Post the output of flank --version.
version: v22.10.0
revision: 57996ac

Additional context

Add any other context about the problem here.

commented

Hi @andreschab90, is this running flank as an action in your repository? It looks like the issue might be that a different invocation of find is needed to support both macOS and Ubuntu. Is it possible to have your action point at a revision on a different branch to test? I pushed b9d9f02, which should run successfully on both operating systems, and can merge if you confirm that it works as expected.

commented

@adamvduke will try it out tomorrow for both platfomrs and let you know.
yeah flank runs as a step in one workflow in my repo

commented

@adamvduke i checked all scenarios

  • flank runs on ubuntu and all tests are passing -> step is marked as successful as expected
  • flank runs on ubuntu and tests are failing -> step is marked as successful, but failed was expected
  • flank runs on macOS and all tests are passing -> step is marked as successful
  • flank runs on macOS and all tests are failing -> step is marked as successful, but failed was expected

so the issue is that the step is always successful now, even so tests were failing. IMO the step should fail when tests are failing

commented

So, it seems like there are two issues:

  1. incorrect find usage on macOS
  2. expected exit status

We can fix the find issue so that the action runs on macOS, and file a separate issue for fixing the expected exit status?

commented

ok, separate issue sounds okay to me. thanks @adamvduke