Assertion API Ideas

[James Shore]

What should Quixote's assertions look like? Use this issue for discussing it.

[James Shore]

Some initial thoughts:

Quixote will run inside another test runner, so the actual assertion library will be outside our control. Programmers may decide to use a fluent style: expect(foo).to.equal(bar); a classical style: assertEqual(foo, bar, "message"); or something else entirely.

So we need to provide an API that works well with existing assertion frameworks.

On the other hand, we might be able to be more user-friendly if we also provide custom assertions. For example, imagine that we provide a way for users to define a "standard style" that consists of many rules, and we had an API to check if an element matched that style. Here's several ways such an API could be implemented:

• element.matchesStandard(style) could return a simple boolean true/false
• element.assertStandard(style) could throw an exception explaining what didn't match
• element.checkStandard(style) could return a string explaining what didn't match, or null if it did match

What are your ideas?

[James Shore]

I'm leaning toward creating a constraint analyzer.

I tried writing a description of what I meant by a "constraint analyzer," but it didn't turn out well. So I'll try explaining by example.

The most common thing you'll do with Quixote is make some assertions about how an HTML element is displayed on the page. You'll do that by calling element.diff(...), which will return a string explaining the differences Quixote found. If the string is empty, there were no differences.

So a typical Quixote test will look like this: assert.equal(element.diff(myExpectations), "");.

The nice thing about this approach is that it allows Quixote to provide meaningful explanations and error messages, while still allowing users to use whichever assertion style they like. So folks who prefer an assertion library such as should.js could write element.diff(myExpectations).should.equal("") and it will work just as well.

Expectations would be written in terms of constraints. Here are some basic examples:

assert.equal(foo.diff({
  top: 13,   // top edge of 'foo' is 13 pixels from the top of the page
  backgroundColor: rgb(0, 0, 0)     // the background is black
}, "");

But a constraint analyzer could be much more sophisticated. It could allow us to express things in terms of relations:

assert.equal(foo.diff({
  top: bar.bottom,    // the top edge of 'foo' is aligned with the bottom edge of 'bar'
  center: baz.left     // the center of 'foo' is aligned with the left edge of 'baz'
}, "");

And those relations could be arbitrarily complex:

assert.equal(foo.diff({
  top: bar.bottom.plus(foo.height),   // top edge of 'foo' is below the bottom of edge of 'bar' by the height of 'foo'
  backgroundColor: bar.color.darker(10)   // background color of 'foo' is the same as foreground color of 'bar', but 10% darker
}, "");

And could relate styles and positions in interesting ways:

assert.equal(foo.diff({
  top: bar.top.plus(bar.height.fraction(1/3)).plus(bar.topLeft.borderRadius.height)
  // top edge of 'foo' is aligned with the top third of 'bar', plus the height of its curved border
}, "");

[James Shore]

Under the covers, the constraints would be read-only objects that describe the constraint, not its result. That's how we can use foo.top instead of foo.top(). Only when the constraint was analyzed would the styles be read and comparison made.

For example, foo.top might be represented with an ElementEdge object. ElementEdge would have methods like plus() and minus(), or maybe up() and down(). Those methods would return another ElementEdge object, allowing them to be strung together, perhaps using the Decorator pattern.

Similarly, bar.height could be represented with an Dimension object. Dimension would have methods like fraction() that would return another Dimension.

ElementEdge.plus() would take a Dimension, and that's how bar.top.plus(bar.height.fraction(1/3)) could be made. It would return a constraint (an ElementEdge object, in this case) that was the composite of all its operations. Calling toString() on that expression might result in "the top edge of the #bar element, plus 1/3 the height of the #bar element".

Constraints could be compared with another method; perhaps relativeTo(). So you could say foo.top.relativeTo(bar.top.plus(bar.height)).toString() and get "the top edge of the #foo element relative to the top edge of the #bar element, plus the height of the #bar element".

These comparisons could be resolved; perhaps using an is() method: foo.top.relativeTo(bar.top).is() would yield a "10px" Dimension object or something similar.

And then finally, the diff() method would bring it all together by constructing and resolving constraints as appropriate, and calling toString() on the constraints as needed to provide useful error messages.

[James Shore]

As of the 0.4 release, these ideas have been implemented, so I'm closing this issue.