KLEE missing an error when dropping an assumption

[salvadorer]

Dear all,

I detected the following issue in KLEE:
I compiled the code below with clang -I ~/klee/include -emit-llvm -c -g -O0 -Xclang -disable-O0-optnone test.c.

#include <assert.h>
#include "klee/klee.h"

int main() {
  int p1 = klee_int("val");
  int p2 = klee_int("val");
  int p3 = klee_int("val");
  int p4 = klee_int("val");
  int p5 = klee_int("val");
  int p6 = klee_int("val");
  int p7 = klee_int("val");
  int p8 = klee_int("val");
  int p9 = klee_int("val");
  int p10 = klee_int("val");
  int p11 = klee_int("val");
  int p12 = klee_int("val");
  int p13 = klee_int("val");
  int p14 = klee_int("val");
  int p15 = klee_int("val");
  int cond = klee_int("val");
  klee_assume(p12 > p14);
  klee_assume(p6 > p3);
  klee_assume(p2 > 0);
  klee_assume(p7 != 0);
  klee_assume(p11 < p14 & p15 < p13);
  klee_assume(cond > p5);
  klee_assume(0 > p4);
  assert(p2 > p11);
}

When running klee --silent-klee-assume test.bc I see 1 warning for the assertion as expected. However if I drop the assumption klee_assume(p2 > 0); and compile and run as in the previous step, KLEE gives no warnings, which is unsound.
The same behavior occurs if I drop the call to klee_assume in the compiled LLVM-Code (check the attached files initial.ll and transformed.ll).
I am working on an Ubuntu system and klee --version prints
KLEE 3.1-pre (https://klee.github.io)
Build mode: RelWithDebInfo (Asserts: ON)
Build revision: 9edf8e8

Ubuntu LLVM version 14.0.0

Optimized build.
Default target: x86_64-pc-linux-gnu
Host CPU: alderlake
llvm_attach.zip

[Frank Busse]

I commented out that line (klee_assume(p2 > 0);) and tried it in the latest KLEE docker container but cannot reproduce it (neither on my local machine). Same for transformed.ll.

[salvadorer]

Ok I think it is due to different solver backends. On my machine I only have Z3 installed. If I run it on the docker with klee --silent-klee-assume --solver-backend=z3 test.bc (where test.bc is obtained by compiling the C-Code from above without the third assumption), I see the same as on my local machine:

KLEE: output directory is "/home/klee/klee-out-14"
KLEE: Using Z3 solver backend

KLEE: done: total instructions = 268
KLEE: done: completed paths = 1
KLEE: done: partially completed paths = 0
KLEE: done: generated tests = 1

[Cristian Cadar]

Thanks for the report. I also cannot reproduce it, and I have also tried Z3.
When you say warnings, do you mean the "assertion fail" error message? It would be useful to include a detailed transcript of what you run and what the output is in each case.

Also, can you reproduce it in the mainline?
And what Z3 are you using? Have you tried different versions?

Finally, can you try to reduce the test case a bit? Many variables, such as p10, are never used: are they really needed to reproduce the error?

[Frank Busse]

Z3 was a good hint, I can reproduce it now (version 4.12).

[Cristian Cadar]

I have Z3 4.8.8, and I see the same behaviour as with STP. So perhaps a Z3 regression?