example-recording reports wrong status if parallel job fails

[Mike McCready]

Current

The example-recording.yml workflow passes back a success status to GitHub in the parallel job whether or not the Cypress tests from the project examples/recording pass or fail.

This shows as inconsistent status reporting:

Desired

The example-recording.yml should pass back a status to GitHub from the parallel job that correctly reflects the pass or fail status from running tests against Cypress project examples/recording.

Steps to reproduce

1. Modify examples/recording/cypress/e2e/spec-a.cy.js to introduce a deliberate failure condition by adding the following to the test spec:

assert.fail('This is a deliberate error')

2. Manually trigger a run of workflows/example-recording.yml by clicking on "Run workflow".

3. After the workflow run has completed examine the logs and note that the group job has failed, whereas both parallel (1) and parallel (2) are marked with a green tick as though they had passed. In the Annotation section it shows that one of the parallel jobs failed, so the reported status is inconsistent.

Suggestion

Remove continue-on-error: true from example-recording.yml which causes the Cypress error to be ignored. (See jobs.<job_id>.steps[*].continue-on-error.)

github-action/.github/workflows/example-recording.yml

Lines 58 to 60 in 36589a7

 # Continue the build in case of an error, as we need to set the 
 # commit status in the next step, both in case of success and failure 
 continue-on-error: true 

Add if: always() to the parallel job, into the step with the name "Print Cypress Cloud URL". (See jobs.<job_id>.steps[*].if.)

[github-actions]

🎉 This issue has been resolved in version 6.6.1 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀