[New API] Add `setupGlobalA11yHooks` helper

[Steve Calvert]

Summary

This issue proposes adding a new global setup helper: setupGlobalA11yHooks. This helper will allow you to, with a single call in your test-helper.js file (or wherever makes sense), wire up the a11yAudit(If) calls to specific @ember/test-helpers hooks using the new _registerHooks API.

Motivation

In order to add accessibility-specific checks in your tests, you need to explicitly import and call one of the a11yAudit(If) helpers. While the explicitness provides value - you can clearly see these audits are being called within specific tests - it's explicitness is also a drawback. Requiring engineers to remember and intentionally call these APIs can mean that these audits may fall by the wayside and be forgotten.

In order to encourage, and indeed elevate, accessibility audits as part of what is considered standard testing best practices, this PR proposes adding this additional mechanism to invoke these calls. This will allow tests to take advantage of these audits automatically.

Details

As previously mentioned, this issue proposes adding the following:

A new global helper exported from this addon called setupGlobalA11yHooks

function setupGlobalA11yHooks(invocationStrategy: () => boolean): void;

• invocationStrategy - a function that returns a boolean, which controls how the audit is invoked. This would control whether the audit occurs or not
  • should provide default invocationStrategy implementation:
    • invokeAll: invokes all audits
• should gracefully handle the case where the version of @ember/test-helpers does not contain the _registerHook API
• Call _registerHook from @ember/test-helpers to hook into test helpers to run a11yAudit(If) checks at specific times (after those helpers' invocations have completed by passing the 'end' hook as the second parameter to _registerHook)
• Register hooks (initially) for
  • visit
  • click
  • doubleClick
  • tap
• Register non-a11yAudit hooks for (helpers that don't need to run a11yAudit calls, but can benefit from additional validations, such as whether outer label elements are present, etc.
  • fillIn
  • typeIn
• Provide a mechanism to provide configuration to the individual a11yAudit(If) calls. Since engineers won't be invoking these functions themselves, we still need a way to pass options into these calls at the individual test level. We can provide a setA11yAuditOptions function that takes these options, and is cleaned up at the end of each test run.
• Ensure we can provide a reproduction URL in order to reproduce a specific test run that invoked a specific test. Prior Art, specifically around outputting a URL to reproduce the tests.
• Ensure we provide clear, actionable feedback when tests fail so users aren't confused if accessibility audits are running during 'regular' invocations of @ember/test-helpers.

Alternatives

There have been alternatives proposed, the most relevant was the proposal for adding an explicit visitAndAudit helper, which seemed like a completely reasonable proposal.

The challenge with adding an API in this manner is

1. Users will have to import and explicitly call another API
2. This API does not include other points-in-time that you may want to audit, such as after click or tap
3. If you want to employ this similar pattern to do the same for click or tap (and others), you'd need to add additional APIs for those. This could add many permutations of APIs, exploding the API surface area of this addon.

cc/ @rwjblue @MelSumner @drewlee Hoping for some specific feedback on this proposal.

[Robert Jackson]

excludeTests? - an optional array to explicitly skip a list of test names that we know will already fail the checks.

I'd prefer to make this just part of the invocationStrategy function. This would make the signature:

function setupGlobalA11yHooks(invocationStrategy: () => boolean): void;

[Steve Calvert]

Sure, we could do that. This would allow people to just supply a custom invocation strategy when they wanted to exclude tests. Seems good.

[Steve Calvert]

Current spike: https://github.com/scalvert/ember-a11y-testing/tree/setup-global-a11y-hooks

[Andrew A Lee]

This is really helpful @scalvert. Thank you!

[Steve Calvert]

Implemented in #200