Intermittant UI CI failures
a3957273 opened this issue · comments
One of our UI tests is flaky, which is making merging new features harder. Example:
https://github.com/gchq/CyberChef/actions/runs/7770605606/job/21190991013?pr=1555
>> ✖ NightwatchAssertError
>> Timed out while waiting for element <#stale-indicator> to not be visible for 5000 milliseconds. - expected "not visible" but got: "visible" (5122ms)
>>
>> Error location:
>> /home/runner/work/CyberChef/CyberChef/tests/browser/browserUtils.js:
>> –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
>> 51 | browser
>> 52 | .click("#bake")
>> 53 | .waitForElementNotVisible("#stale-indicator", 5000)
>> 54 | .waitForElementNotVisible("#output-loader", 5000);
>> 55 | }
>> –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
FAILED: 1 assertions failed and 8 passed (5.552s)
───────────────────────────────────────────────────────────────────────────────────────────────────
️TEST FAILURE (50.092s):
- 1 other test suites were aborted;
- 1 assertions failed; 742 passed
- 1 skipped
✖ 1) 01_io
– Loading from URL (5.552s)
→ ✖ NightwatchAssertError
Timed out while waiting for element <#stale-indicator> to not be visible for 5000 milliseconds. - expected "not visible" but got: "visible" (5122ms)
Error location:
/home/runner/work/CyberChef/CyberChef/tests/browser/01_io.js:
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
657 | browser.expect.element("#output-text .eol-value").text.that.equals("PS");
658 |
659 | utils.bake(browser);
660 |
661 | browser.expect.element(`#output-text .cm-content`).to.have.property("textContent").match(/^.{44}$/);
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
SKIPPED:
- Replace input with output
Wrote HTML report file to: /home/runner/work/CyberChef/CyberChef/tests/browser/output/nightwatch-html-report/index.html
>> ✖ NightwatchAssertError
>> Timed out while waiting for element <#stale-indicator> to not be visible for 5000 milliseconds. - expected "not visible" but got: "visible" (5122ms)
>>
>> Error location:
>> /home/runner/work/CyberChef/CyberChef/tests/browser/browserUtils.js:
>> –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
>> 51 | browser
>> 52 | .click("#bake")
>> 53 | .waitForElementNotVisible("#stale-indicator", 5000)
>> 54 | .waitForElementNotVisible("#output-loader", 5000);
>> 55 | }
>> –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
>> Exited with code: 10.
>> Error executing child process: Error: Process exited with code 10.