[Discussion] Refuting stubbed methods?

[Nishiki (錦華)]

First off before anything, thanks for this lovely gem! I've always been fairly frustrated with mocking libraries but this one does it really nicely.

This is more of a discussion starter than anything else, but what are your thoughts on including a Mocktail.verify_not or something similar? (Not using #refute since that would clash with Minitest::Assertions#refute.)

While it's true that we can easily accomplish this with something like:

assert_raises(Mocktail::VerificationError) do
  Mocktail.verify do
    # ...
  end
end

I just think it'd just be really nice if there was a method for this, kind of in the same Ruby spirit as Minitest having #assert and #refute, Rails having things like #second, #third, and so on.

[Justin Searls]

I completely understand where you're coming from and I may consider this in the future, but after maintaining so many test double libraries in the past (and seeing how they're often abused in practice), I'm very wary of adding public API methods that will be very often used to do the wrong thing.

9 times out of 10, when I see a test that verifies that a call didn't happen, it's an example of over-specification, and not actually necessary (in this sense) for the test; rather, people tend to over-apply it in a way that would look silly if it were, say, refute'ing a dozen values that a method did not return. In general, a unit test shouldn't need to assert the infinite things that the subject under test did not do, but rather what it did.

As a result, I tend to be more explicit with this sort of thing when it does matter (syntactic vinegar, I suppose). The following should work, and it specifies the call that I expect not to happen very explicitly:

Mocktail.verify(times: 0) { dog.wag(:tail) }

Will fail if the double's wag method was called with exactly the symbol :tail.

Of course, when the arguments passed to the double are more dynamic (such as iterating over a loop or something or passed an interpolated string), you could use a matcher to loosen those arguments:

Mocktail.verify(times: 0) { |m| dog.wag(m.any) }

An equivalent thing you could do if the arguments are optional is to pass ignore_extra_args: true to verify().

Or, if you really want to specify that a method was absolutely not called, you could both ignore arity and all unspecified arguments with this very verbose assertion:

Mocktail.verify(times: 0, ignore_arity: true, ignore_extra_args: true) { dog.wag }

Of course, this is a bit verbose relative to the purpose of the test, but in practice of having written a lot of unit tests with mocks over the years, I haven't really found very many cases where it's necessary to verify a call did not occur and I had no control over the arguments that would have been passed.

A way to answer the same question is to use the Mocktail.explain method:

assert_equal 0, Mocktail.explain(dog).reference.calls { |c| c.method == :wag }.size

Though upon looking at the above, I think that Mocktail.explain(dog.method(:wag)) ought to work, and I was surprised it doesn't.

[Justin Searls]

Ok, I've released mocktail@1.1.0 which adds support for passing methods to Mocktail.explain

[Nishiki (錦華)]

Great, thanks for the detailed response and links—all makes total sense. I can definitely see how refuting could be abused like you stated. Wasn't aware of the times arg you showed above, so I may try using that if I ever come across a situation where a refutation makes sense again.