Tests sometimes fail with `** (MatchError) no match of right hand side value: false`

[Victoria]

In some our tests we have plenty of mocks and sometimes tests fail with

     ** (MatchError) no match of right hand side value: false
     code: with_mocks(mocks) do
     stacktrace:
       test/test.exs:267:  <line with `with_mock` call>
       (elixir) lib/enum.ex:1940: Enum."-reduce/3-lists^foldl/2-0-"/3

Everything is defenitely Ok with mocks itself - tests usually pass, but still it is rather annoying when you have lots of such tests and they sometimes fail for no reason :(

[Daniel Olshansky]

So is the same test failing/passing ephemerally?

Could you provide some example code so we can debug together?

[Victoria]

Here is shortened version of test that still reproduces the error, I hope It will be engouh.

Here is is sample of failed test output

  1) test #init assert 6 (MyAppRouter.ConsumerListenerTest)
     test/consumer_listener_test.exs:101
     ** (MatchError) no match of right hand side value: false
     code: with_mocks(mocks) do
     stacktrace:
       test/consumer_listener_test.exs:104: anonymous fn/2 in MyAppRouter.ConsumerListenerTest."test #init assert 6"/1
       (elixir) lib/enum.ex:1940: Enum."-reduce/3-lists^foldl/2-0-"/3
       test/consumer_listener_test.exs:104: (test)

And one more detail: Our project uses umbrella. The case with flicker failing tests reporoduces only when running tests from the root direcrory of the whole project. When running from the directory of the application with specific test - the test always pass

[Daniel Olshansky]

Hmm...

Given that you said this: When running from the directory of the application with specific test - the test always pass and looking at the test you pasted, the only theory I have is that your tests are asynchronous, so there's a weird race condition happening when you are trying to mock things.

Can you try updating this line with use ExUnit.Case, async: true and see if the error persists?

Reference: https://hexdocs.pm/ex_unit/ExUnit.Case.html

[Victoria]

No, tests are not asynchronous, if add async: true, that way they will be asynchronous and they won't pass at all but with other errors.
Sorry for late reply