Giters

ember-a11y / ember-a11y-testing

A suite of accessibility tests that can be run within the Ember testing framework

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Drop auto-run feature?

trentmwillis opened this issue · comments

commented

So, having spent the last couple days working on the addon and trying to fix issues like #60, I am of the opinion that we should drop the "auto-run" feature for tests.

Here is my reasoning:

  1. It makes adoption of the addon difficult because it will likely result in many failures for existing applications.
  2. It can be quite spammy, since some applications trigger run-loops frequently in their app code, which amplifies point #1.
  3. In addition to noise, running the audit on every turn of the run-loop can also be quite slow and costly for applications with complex UIs and many test cases.
  4. The implementation of auto-run is hacky at best and has proven difficult to maintain, since we are effectively jamming outside, asynchronous behavior into the application's run-loop.
  5. The implementation is currently QUnit-specific, which limits our ability to support those using other testing libraries.

Beyond that, I see several benefits to changing the approach to be on-demand (e.g., using a11yAudit()):

  1. Has a clear, incremental path for adopting accessibility testing.
  2. Is more explicit about when audits occur (less "magic" testing).
  3. It is much easier to reason about the story for passing options and scoping the audit.
  4. Will be much easier to maintain and test over time.

@ember-a11y/core thoughts? I'm especially curious if anyone is actually using the auto-run feature as it exists today. My hunch is that either no one is, or very few people are.

commented

I agree with this approach.
In our use case, we turned it off app-wide, and then turn it on component by component.

commented

Additional points from Slack:

@MelSumner: I think it aligns with how Ember (seems to be from my POV) - it would make it an easier tool to use even for those with limited exposure & supports use for those just starting out on this.

@rwjblue: I think its basically untenable to maintain the auto-run behavior with some of the things that we have planned in Ember

commented

This has been removed in #64 and likely to be released in 0.3.0