Running MIRAI in cargo test

[Dominik Harz]

Issue

I was wondering if there's a way to run MIRAI on invoking cargo test. I wanted to create a test case with abstract values that would look something like this:

#[test]
fn verify_add() {
  let a = abstract_value!(2);
  let b = abstract_value!(4);
  let expected_result = abstract_value!(6);
  let result = add(a, b);
  checked_verify_eq!(result, expected_result);

However, I'm not sure how to ask MIRAI to run this test. cargo test would just run the normal tests, treating the abstract values as concrete values and the checked verify as assert_eq.

After a bit of digging in the code, I found: https://github.com/facebookexperimental/MIRAI/blob/e45acd458080be1cdbddca01280926dd68e4c600/checker/tests/integration_tests.rs#LL48C16-L48C16

It looks to me as if there's some special setup required to run tests using MIRAI. I tried to search in the docs as well but did not find an answer.

Steps to Reproduce

• Create a function add that adds two numbers.
• Create a test case as above.
• Run cargo test and see that MIRAI is not used for verification.

Expected Behavior

MIRAI being used in cargo test or invoking cargo mirai with options that would execute the tests.

Actual Results

MIRAI is not checking the abstract values.

Environment

Rust version (rustc --version)

rustc 1.68.0-nightly (b70baa4f9 2022-12-14)

[Herman Venter]

The way this is supposed to work is to run "cargo mirai --tests". It is not being tested in CI and seems to have been broken by a rust compiler change. Oops.

When it gets fixed, your example above is almost right. The problem is just that since expected_result is abstract, nothing is known about it and hence the verify check will fail. (If you removed the abstract_value! calls, the code becomes verifiable because constant folding will handle it, but you are doing no better than just running cargo test.)

If you could write an assertion that only depends on the fact that some unknown number has been added to some other unknown number, that would be statically verifiable, but coming up with such an assertion would be a bit of a challenge, unless you further constrain the input values, for example if you add assume!(a < 10) and assume!(b < 20) then you can verify that result is less than 30.