Counterexample: unknown

[acmLL]

Describe the bug

I was performing a comparison between halmos and forge test against SMTChecker. I have used an adaptation of the following code

// SPDX-License-Identifier: GPL-3.0
pragma solidity >=0.8.0;

contract Monotonic {
function f(uint x) internal pure returns (uint) {
require(x < type(uint128).max);
return x * 42;
}

function inv(uint a, uint b) public pure {
    require(b > a);
    assert(f(b) > f(a));
}

}
as found in https://docs.soliditylang.org/en/v0.8.17/smtchecker.html.

To Reproduce
I ran the command "forge init" in a folder and added this function

function f(uint x) public pure returns (uint) {
    if(x < type(uint128).max)
        return x * 42;
    else return x;
}

to the file Counter.sol and the test function

function testf(uint a, uint b) public view {
if(b > a)
assert(counter.f(b) > counter.f(a));
else
assert(true);
}
to the file Counter.t.sol. Thus I ran "forge test" and halmos and got this.

acmLL % forge test
[⠒] Compiling...
No files changed, compilation skipped
Running 3 tests for test/Counter.t.sol:CounterTest
[PASS] testIncrement() (gas: 28435)
[PASS] testSetNumber(uint256) (runs: 256, μ: 27908, ~: 28375)
[PASS] testf(uint256,uint256) (runs: 256, μ: 3601, ~: 6448)
Test result: ok. 3 passed; 0 failed; finished in 54.41ms

acmLL % halmos
Running 3 tests for test/Counter.t.sol:CounterTest
[PASS] testIncrement() (paths: 1/2, time: 0.21s, bounds: [])
[PASS] testSetNumber(uint256) (paths: 1/2, time: 0.26s, bounds: [])
[FAIL] testf(uint256,uint256) (paths: 4/9, time: 61.82s, bounds: [])
Counterexample: unknown
Symbolic test result: 2 passed; 1 failed

Environment:

• macOS 12.6.5
• Python version: 3.9.6
• Halmos and other dependency versions:
• Package Version

---

altgraph 0.17.2
cbor2 5.4.6
crytic-compile 0.3.1
future 0.18.2
halmos 0.0.7
macholib 1.15.2
packaging 23.1
pip 21.2.4
pycryptodome 3.17
setuptools 58.0.4
six 1.15.0
solc-select 1.0.3
wheel 0.37.0
z3-solver 4.12.1.0

Additional context
Initially I used the require version (instead of the adapted if-based version) as presented in https://docs.soliditylang.org/en/v0.8.17/smtchecker.html, forge test detected a revert case and halmos gave the same report.

[Daejun Park]

The assertion is no longer satisfied with your new f() function, but Halmos did not provide a counterexample because it thought the generated counterexample might be invalid. To address this issue, I have added a new option --print-potential-counterexample to force it to provide counterexamples even if they are potentially invalid. The new option is available in the latest master version (#65).

Please note that, however, Halmos does not generally perform well for tests that involve nonlinear arithmetic (e.g., division) due to the inherent limitations of underlying SMT solvers. It is generally undecidable to solve nonlinear integer arithmetic, which is why it is unavoidable to encounter counterexample generation failures in such cases.