Leaner build system causes tests to fail with weird exception

Question

Leaner build system causes tests to fail with weird exception

Niols opened this issue a year ago · comments

This issue is here to track the debugging of the issue caused by #187 where:

$ cabal test pirouette-plutusir:spec
...
Test suite spec: RUNNING...
Language
  PlutusIR
    ToTerm
      With resource: resources/fromPlutusIRSpec-01.pir
        Translates the PIR program without errors: OK
        Decls contain 'long' and 'short' terms:    OK
        expandDefs produces NF terms:              OK
    Builtins
      Read, translate and typecheck programs
        Split:                                     OK
        Auction:                                   OK (0.02s)
      Can parse terms:                             OK
    Symbolic evaluation
      simple triples
        [input > 0] add 1 [result > 0] counter:    FAIL
          Exception: fd:15: hGetLine: end of file
          Use -p '/[input > 0] add 1 [result > 0] counter/' to rerun this test only.
        [input > 0] add 1 [result > 1] verified:   FAIL
          Exception: fd:18: hGetLine: end of file
          Use -p '/[input > 0] add 1 [result > 1] verified/' to rerun this test only.

2 out of 8 tests failed (0.06s)
Test suite spec: FAIL
...

Nicolas Jeannerod · Answer 1 · Wed Jan 11 2023 20:35:43 GMT+0800 (China Standard Time)

The issue could be reproduced exactly (file descriptors included):

in CI
on my local machine
on @gabrielhdt's machine
on the Tweag builder

Nicolas Jeannerod · Answer 2 · Wed Jan 11 2023 20:39:53 GMT+0800 (China Standard Time)

A bisection shows that both 07301a6 (last commit on main before @gabrielhdt's fork) and 35856cd (last commit on main at time of writing this) are exempt of this bug.

Nicolas Jeannerod · Answer 3 · Wed Jan 11 2023 20:46:08 GMT+0800 (China Standard Time)

Some trace-based debugging leads me to think that the bug is somewhere in this piece of code. To my surprise, it does not seem to take place in openAndParsePIR but rather in execIncorrectnessLogic.

Following up would involve debugging in Pirouette. Pirouette does not use the mentioned hGetLine but it probably uses some higher-level version of it.

Nicolas Jeannerod · Answer 4 · Wed Jan 11 2023 20:46:31 GMT+0800 (China Standard Time)

Trying to build gh@trythings with IOHK's compiler (instead of the one from nixpkgs) does not improve the situation.

Nicolas Jeannerod · Answer 5 · Wed Jan 11 2023 20:58:55 GMT+0800 (China Standard Time)

Bringing back LANG = "en_US.UTF-8"; and/or LOCALE_ARCHIVE = "${pkgs.glibcLocales}/lib/locale/locale-archive" does not help with the situation.

Nicolas Jeannerod · Answer 6 · Wed Jan 11 2023 21:26:36 GMT+0800 (China Standard Time)

8d14a97 seems to be the first commit with an issue. Sadly, this commit is pretty big.

Nicolas Jeannerod · Answer 7 · Wed Jan 11 2023 22:27:55 GMT+0800 (China Standard Time)

Compiling 8d14a97 with nixpkgs pinned to the version pinned by Niv in 07301a6 (NixOS/nixpkgs@74b1085) also fails in the same way.

Nicolas Jeannerod · Answer 8 · Thu Jan 12 2023 01:26:35 GMT+0800 (China Standard Time)

Compiling 8d14a97 with an old version of Pirouette (tweag/pirouette@3805638) and Z3 and CVC4 in the Shell's buildInputs works! So it probably has to do with the new integration of Z3.

Edit: welp, there it is. It was probably just CVC4 missing from the environment. SO DUMB.

Nicolas Jeannerod · Answer 9 · Thu Jan 12 2023 17:19:32 GMT+0800 (China Standard Time)

My experiments show that, actually, everything runs fine with recent commits of Pirouette. Not the most recent ones because there was an API change. It looks like tweag/pirouette@69d1eea works fine. Hence #209 (and its corresponding PR on @gabrielhdt's branch, https://github.com/gabrielhdt/plutus-libs/pull/1).

Nicolas Jeannerod · Answer 10 · Thu Jan 12 2023 17:55:56 GMT+0800 (China Standard Time)

I ran a git bisect on Pirouette to try and find where the bug was fixed. The last versions that contain the bug are:

tweag/pirouette@41fe7a5, the version currently in cabal.project.
tweag/pirouette@c511a16, the commit on which PRs tweag/pirouette#153 and tweag/pirouette#154 are based.
tweag/pirouette@c15b4a4, the merge commit of tweag/pirouette#153.

The first versions that do not contain the bug anymore are:

tweag/pirouette@748cbb6, the merge commit of tweag/pirouette#154.
tweag/pirouette@33977c2, a commit within tweag/pirouette#154.

First, I include tweag/pirouette#153 in my picture because it was merged into tweag/pirouette#154 before this one was merged back into main. Second, sadly, the very first commits of tweag/pirouette#154 do not compile, so we only know that the bug was fixed on the path:

tweag/pirouette@c511a16 (with the bug)
tweag/pirouette@c79d8bb (does not compile)
tweag/pirouette@f52cf48 (does not compile)
tweag/pirouette@64f18eb (does not compile)
tweag/pirouette@33977c2 (without the bug)

One could for instance compare what those commits change to try and understand where the bug comes from.

Nicolas Jeannerod · Answer 11 · Thu Jan 12 2023 20:44:31 GMT+0800 (China Standard Time)

The conclusion of this debugging session is STUPID. plutus-libs was following an old version of Pirouette... that was still using CVC4. Pirouette was not finding the binary and not managing to talk it and therefore failing when reading from it. It would deserve better error reporting but this point is made irrelevant by the recent changes in Pirouette that now relies on SMTLIB-backends. The improvement to error reporting can be tracked there, cf tweag/smtlib-backends#43.

In the meantime, there are two possible fixes to this:

we can add CVC4 as a dependency in the flake
or we can bump Pirouette's version and add Z3 as a dependency in the flake. This is essentially what https://github.com/gabrielhdt/plutus-libs/pull/1 does and I would rather go for this option.