Connection refused when accessibility tests fail
deviantintegral opened this issue · comments
When an accessibility test fails, additional errors are thrown. I created a repository with a test case that follows the installation instructions and creates a basic test: https://github.com/deviantintegral/nightwatch-accessibility-test
Using node 14:
$ node node_modules/.bin/nightwatch --env chrome test.js
[Test] Test Suite
=================
ℹ Connected to localhost on port 9515 (2913ms).
Using: chrome (96.0.4664.55) on Mac OS X platform.
✔ Running Test:
✔ Element <body> was visible after 25 milliseconds.
✔ Passed [ok]: Elements must only use allowed ARIA attributes
✔ Passed [ok]: ARIA role should be appropriate for the element
✔ Passed [ok]: aria-hidden='true' must not be present on the document body
✔ Passed [ok]: Required ARIA attributes must be provided
✔ Passed [ok]: ARIA roles used must conform to valid values
✔ Passed [ok]: ARIA attributes must conform to valid values
✔ Passed [ok]: ARIA attributes must conform to valid names
✔ Passed [ok]: Inline text spacing must be adjustable with custom stylesheets
✔ Passed [ok]: Buttons must have discernible text
✔ Passed [ok]: Elements must have sufficient color contrast
✔ Passed [ok]: IDs of active elements must be unique
✔ Passed [ok]: IDs used in ARIA and labels must be unique
✔ Passed [ok]: id attribute value must be unique
✔ Passed [ok]: Headings should not be empty
✔ Passed [ok]: Heading levels should only increase by one
✔ Passed [ok]: Links with the same name must have a similar purpose
✔ Passed [ok]: Images must have alternate text
✔ Passed [ok]: Alternative text of images should not be repeated as text
✔ Passed [ok]: Banner landmark should not be contained in another landmark
✔ Passed [ok]: Contentinfo landmark should not be contained in another landmark
✔ Passed [ok]: Main landmark should not be contained in another landmark
✔ Passed [ok]: Document should not have more than one banner landmark
✔ Passed [ok]: Document should not have more than one contentinfo landmark
✔ Passed [ok]: Document should not have more than one main landmark
✔ Passed [ok]: Ensures landmarks are unique
✔ Passed [ok]: Links must have discernible text
✔ Passed [ok]: <ul> and <ol> must only directly contain <li>, <script> or <template> elements
✔ Passed [ok]: <li> elements must be contained in a <ul> or <ol>
✔ Passed [ok]: Interactive controls must not be nested
✔ Passed [ok]: All page content should be contained by landmarks
✔ Passed [ok]: Elements should not have tabindex greater than zero
✖ Failed [fail]: (id attribute value must be unique [<div class="block block-entity-browser-block block-entity-browser-blockcontent-browser-modal" id="block-entity-browser-blockcontent-browser-modal">] https://dequeuniversity.com/rules/axe/4.3/duplicate-id?application=axeAPI)
at processTicksAndRejections (internal/process/task_queues.js:95:5)
FAILED: 1 assertions failed and 32 passed (1.87s)
_________________________________________________
TEST FAILURE: 1 assertions failed, 32 passed (5.301s)
✖ test
– Test (1.87s)
POST /timeouts/async_script - ECONNREFUSED
Error: connect ECONNREFUSED 127.0.0.1:9515
Error while running .setTimeoutsAsyncScript() protocol action: Error ECONNREFUSED: connect ECONNREFUSED 127.0.0.1:9515
POST /execute_async - ECONNREFUSED
Error: connect ECONNREFUSED 127.0.0.1:9515
Error while running .executeScriptAsync() protocol action: Error ECONNREFUSED: connect ECONNREFUSED 127.0.0.1:9515
On our actual project, we get a slightly different error, but this could be because this is running on linux inside of a Docker container, while the above was running on my macOS host:
✔ Passed [ok]: svg elements with an img role have an alternative text
✔ Passed [ok]: Elements should not have tabindex greater than zero
✖ Failed [fail]: (All page content should be contained by landmarks [<h2 id="primary-tabs-title" class="visually-hidden">Primary tabs</h2>] https://dequeuniversity.com/rules/axe/4.3/region?application=axeAPI)
at processTicksAndRejections (internal/process/task_queues.js:95:5)
FAILED: 1 assertions failed and 48 passed (3.85s)
_________________________________________________
TEST FAILURE: 1 assertions failed, 48 passed (9.02s)
✖ nightwatch/example.nightwatch
– Demo test (3.85s)
Error while running .setTimeoutsAsyncScript() protocol action: Unknown command 404 Not Found
Error while running .executeScriptAsync() protocol action: Unknown command 404 Not Found
✖ Failed [fail]: (aXe failed to execute)
at processTicksAndRejections (internal/process/task_queues.js:95:5)
Error while running .setTimeoutsAsyncScript() protocol action: Unknown command 404 Not Found
Error while running .executeScriptAsync() protocol action: Unknown command 404 Not Found
This may be a regression of #23.
please try with v2.0.0