Dump all SMT queries

[Mate Soos @ Ethereum Foundation]

Is your feature request related to a problem? Please describe.
I'd like to generate SMT queries in order to send them to the SMT competition as benchmarks to help developers improve their large-bitwidth performance :) Win for everyone! Basically, I'd like if STP/CVC5/Z3/Bituzla developers improved their SMT engines :)

I am implementing a --dumpsmt parameter to ./bench.py in eth-sc-comp/benchmarks, PR here: eth-sc-comp/benchmarks#27 HEVM dumps all queries if asked, including UNSAT.

Describe the solution you'd like
--dump-smt-queries should dump all SMT queries, not just SAT queries

Describe alternatives you've considered
Current version only seems to dump only SAT queries (?), not UNSAT I think?

Additional context
I would like to help SMT developers improve their tools :)

[karmacoma]

Just double checked, --dump-smt-queries dumps all assertion queries regardless of sat/unsat status. We don't dump path queries though, so we need at least one path leading down an sat/unsat assertion query in order to dump something.

Here is an example:

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.15;
import "forge-std/Test.sol";

contract DivisionIsHard {
    /// @notice we expect a counterexample [y = 0, x = 1]
    function prove_divRoundsDown(uint256 x, uint256 y) external {
        assert(x / y < x);
    }

    /// @notice we expect unsat (no counterexample)
    function prove_divRoundsToZero(uint256 x, uint256 y) external {
        require(x < y);
        assert(x / y == 0);
    }
}

halmos --function prove_divRoundsToZero --solver-timeout-assertion 0 --dump-smt-queries                                                          
[⠢] Compiling...
No files changed, compilation skipped

Running 1 tests for test/62_divRoundsDown.t.sol:DivisionIsHard
Generating SMT queries in /tmp/prove_divRoundsToZero.388a8564d3e5406b91f4191ec142f597
[PASS] prove_divRoundsToZero(uint256,uint256) (paths: 3, time: 26.29s, bounds: [])
Symbolic test result: 1 passed; 0 failed; time: 26.30s

[karmacoma]

Note that the query we dump to file is before refinement, so evm_bvudiv shows up there.

We should probably also dump queries after refinement -- cc @daejunpark

[Mate Soos @ Ethereum Foundation]

Ah, I see. Ok, that clears things up, thanks! Sorry for the noise. Yeah, we also dump the refinement, if it's performed. Honestly, we probably want to build some system to monitor and kill SMT processes, because sometimes they can run amok -- e.g. Z3 often doesn't respect timeouts. In there, one could add a dumper, too, which may be more accurate and easier to manage and make reproducible. Anyway, that's for another day.

Thanks again for this and cheers,

Mate