Random test cases were failing when we tried to generate integration code coverage report with cypress-parallel having multiple threads
rk011219 opened this issue · comments
Logs and screenshots
Versions
Cypress 10.8.0
"cypress-parallel": "0.9.1",
"@cypress/code-coverage": "3.10.0",
Angular 12.1.3
os : windows 10, linux
Node v12.22.3
npm v6.14.13
application instrumented using istambul and followed below lnk
https://lukas-klement.medium.com/implementing-code-coverage-with-angular-and-cypress-6ed08ed7e617
window.coverage : yes
- Is there
.nyc_outputfolder? Is there.nyc_output/out.jsonfile.
-yes - Do you have any custom NYC settings in
package.json(nycobject) or in other [NYC config files] - yes
- {
"extends": "@istanbuljs/nyc-config-typescript",
"all": true,
"excludeAfterRemap": true,
"reporter": ["html"],
"exclude": [
"./coverage/",
"cypress/",
"./dist/",
"/.spec.ts",
"/projects//test.ts",
"/projects//.js",
"./.js",
"./.config.ts",
"/projects//*.html",
"./src/**"
]
} - Do you run Cypress tests in a Docker container?
NO
Describe the bug
I am able to generate report when there is no cypress-parallel but when I have used cypress-parallel command then code-coverage report is generating but random test cases were failing due to below errors:-
CypressError: cy.task('coverageReport') failed with the following error:
Unexpected end of JSON input
https://on.cypress.io/api/task
Because this error occurred during a after all hook we are skipping all of the remaining tests.�[0m�[90m
at (http://localhost:4210/__cypress/runner/cypress_runner.js:145679:78)
at tryCatcher (http://localhost:4210/__cypress/runner/cypress_runner.js:11318:23)
at Promise._settlePromiseFromHandler (http://localhost:4210/__cypress/runner/cypress_runner.js:9253:31)
at Promise._settlePromise (http://localhost:4210/__cypress/runner/cypress_runner.js:9310:18)
at Promise._settlePromise0 (http://localhost:4210/__cypress/runner/cypress_runner.js:9355:10)
at Promise._settlePromises (http://localhost:4210/__cypress/runner/cypress_runner.js:9431:18)
at _drainQueueStep (http://localhost:4210/__cypress/runner/cypress_runner.js:6025:12)
at _drainQueue (http://localhost:4210/__cypress/runner/cypress_runner.js:6018:9)
at ../../node_modules/bluebird/js/release/async.js.Async._drainQueues (http://localhost:4210/__cypress/runner/cypress_runner.js:6034:5)
at Async.drainQueues (http://localhost:4210/__cypress/runner/cypress_runner.js:5904:14)
From Your Spec Code:
at Context.generateReport (webpack:///./node_modules/@cypress/code-coverage/support.js:200:0)
From Node.js Internals:
SyntaxError: Unexpected end of JSON input
at JSON.parse ()
at includeAllFiles (/var/jenkins/workspace/sample_project/node_modules/@cypress/code-coverage/task-utils.js:348:28)
at coverageReport (/var/jenkins/workspace/sample_project/node_modules/@cypress/code-coverage/task.js:204:7)
at invoke (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/lib/plugins/child/run_plugins.js:234:16)
at (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/lib/plugins/util.js:59:14)
at tryCatcher (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/node_modules/bluebird/js/release/util.js:16:23)
at Function.Promise.attempt.Promise.try (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/node_modules/bluebird/js/release/method.js:39:29)
at Object.wrapChildPromise (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/lib/plugins/util.js:58:23)
at RunPlugins.taskExecute (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/lib/plugins/child/run_plugins.js:240:10)
at RunPlugins.execute (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/lib/plugins/child/run_plugins.js:160:21)
at EventEmitter. (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/lib/plugins/child/run_plugins.js:257:12)
at EventEmitter.emit (events.js:314:20)
at EventEmitter.emit (domain.js:483:12)
at process. (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/lib/plugins/util.js:33:22)
at process.emit (events.js:314:20)
at process.EventEmitter.emit (domain.js:483:12)
at process.emit.sharedData.processEmitHook.installedValue [as emit] (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@cspotcode/source-map-support/source-map-support.js:745:40)
at emit (internal/child_process.js:877:12)
at processTicksAndRejections (internal/process/task_queues.js:85:21)
Or Sometime getting this error
CypressError: cy.task('coverageReport') failed with the following error:
Unexpected end of JSON input
https://on.cypress.io/api/task
Because this error occurred during a after all hook we are skipping all of the remaining tests.
FYI , I have followed below link in order to generate cypress integration code coverage report and we need parallelism in order to speed up test execution so I have used below command in order to run test cases in parallel:-
"cypress-parallel -s cypress:run -t 8 -d cypress/e2e/testcases --verbose --reporter cypress-mochawesome-reporter -a '"--config baseUrl=[http://localhost:4210](http://localhost:4210%5C/)"' --strictMode false",
Please note we have one CI machine so we have to use cypress-parallel with multi-threading
Total test cases around 50.
Update: Seems to be related to parallel Cypress runs. Fairly consistently fails for me when running with parallelization, but succeeds when using a single (non-parallel) job.
I posted a workaround in the related tnicola/cypress-parallel#126