fix(jest): automatic checks should not be triggered for failed tests
mohanraj-r opened this issue · comments
-
Automatic check is triggered regardless of the test status which would result in the original test failure if any getting overwritten by a11y failures if any from automatic checks.
-
Tests using the sa11y jest api would get tested twice with automatic checks - once as part of the sa11y API in the test and again as part of the automatic check
- a11y issues from automatic checks would overwrite the a11y issues found by the API
- If the sa11y API has been added to the test to check specific intermediate states of the DOM, enabling automatic checks could result in missed a11y issues
Workarounds
Current
- Enable automatic checks in a parallel CI workflow and not in the main CI workflow.
Future
- With the default Jasmine test runner in Jest v26 there doesn't seem to be clean way to detect test failures from Jest's teardown hooks
- But jest-circus offers event handlers which allows binding to test failure events among others
jest-circus
is the default test runner starting Jest v27. After Jest v27 gains more ground and known issues (#65) are fixed this could be a viable option
- But even if the test failure status can be detected reliably, the question then is if the automatic checks can be triggered from within the test context - from within a CustomEnvironment that extends JSDOM env e.g. where the event handler is bound
- With custom reporter / results processor e.g., accessing DOM is not possible and the test context is no longer available (#64).