Tests are not skipped on specs
tssobe opened this issue · comments
Describe the bug
I am using the default configuration of cypress-fail-fast( for FAIL_FAST_STRATEGY the default is run)
Tests are being skipped only for current spec, but the next specs are running with all tests.
To Reproduce
I have 4 spec.js files.
I run cypress run --browser chrome --headed --spec cypress/tests/integration/sanity/*
test fail on flow0.spec.js and all tests are skipped in the flow0.spec.js
flow1.spec.js is running with all tests
Expected behavior
With FAIL_FAST_STRATEGY run I expect that specs (flow1,flow2,flow3) wont run if a test fails on flow0.spec.js
Logs
If applicable, add logs to help explain your problem.
** Operating system, Node.js an npm versions, or browser version (please complete the following information):**
- OS: win10
- Node.js:18.12.0
- npm: 6.14.13
- Browser: Chrome 106.0.5249.119
Additional context
Add any other context about the problem here.
Hi @tssobe ,
This is a very basic behavior that is being widely tested in the library. So, in order to be able to reproduce the issue, could you please provide your Cypress version and further details about how are you loading the plugin, the whole configuration, etc?
Hi @javierbrea
"cypress": "7.2.0"
cypress/plugins/index.js -
module.exports = (on, config) => {
require("cypress-fail-fast/plugin")(on, config);
cypress/support/index.js - import "cypress-fail-fast";
package.json - "cypress-fail-fast": "5.0.1"
Running Cypress - "cy:sanity": "cypress run --browser chrome --headed --spec cypress/tests/integration/sanity/*"
Sanity folder:
When running the tests:
Flow 0
and as you can see flow 1 runs despite flow 0 failed
@javierbrea if needed, I can set a meeting to show you the issue
Found the issue.
When running Cypress in the format: "cypress run --browser chrome --headed --spec cypress/tests/integration/sanity/*"
It wont work properly .
I have changed it to:
"cypress run --browser chrome --headed --spec cypress/tests/integration/sanity/allSanity.spec.js"
Hi @tssobe ,
I don't understand the "issue" you have fixed. In theory, you had a problem when running multiple spec files. In your solution proposal, you are filtering the spec files to be executed, so it only executes one spec. Obviously, the "issue" can't be produced in that case, am I right?
I suspect that, if there is such issue, It might be related to the usage of the "--headed" flag. Note that the plugin is intented to be used in "headless" mode, and so is specified in the docs.
Anyway, as you have found a way to make the plugin work for your specific case, I won't continue investigating.