Giters

javapathfinder / jpf-core

JPF is an extensible software analysis framework for Java bytecode. jpf-core is the basis for all JPF projects; you always need to install it. It contains the basic VM and model checking infrastructure, and can be used to check for concurrency defects like deadlocks, and unhandled exceptions like NullPointerExceptions and AssertionErrors.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Getting a strange error from JPF

stevenrbrandt opened this issue · comments

commented

I'm getting this error when running JPF. Any idea what it's telling me?

java.lang.ArrayIndexOutOfBoundsException: 1
        at gov.nasa.jpf.vm.NamedFields.setDoubleValue(NamedFields.java:164)
        at gov.nasa.jpf.vm.FunctionObjectFactory.setFuncObjFields(FunctionObjectFactory.java:60)
        at gov.nasa.jpf.vm.FunctionObjectFactory.getFunctionObject(FunctionObjectFactory.java:37)
        at gov.nasa.jpf.jvm.bytecode.INVOKEDYNAMIC.execute(INVOKEDYNAMIC.java:121)
        at gov.nasa.jpf.vm.ThreadInfo.executeInstruction(ThreadInfo.java:1908)
        at gov.nasa.jpf.vm.ThreadInfo.executeTransition(ThreadInfo.java:1859)
        at gov.nasa.jpf.vm.SystemState.executeNextTransition(SystemState.java:765)
        at gov.nasa.jpf.vm.VM.forward(VM.java:1722)
        at gov.nasa.jpf.search.Search.forward(Search.java:937)
        at gov.nasa.jpf.search.DFSearch.search(DFSearch.java:79)
        at gov.nasa.jpf.JPF.run(JPF.java:613)
        at gov.nasa.jpf.JPF.start(JPF.java:189)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at gov.nasa.jpf.tool.Run.call(Run.java:85)
        at gov.nasa.jpf.tool.RunJPF.main(RunJPF.java:116)
commented

Hi,
This seems to be an internal error in JPF. Can you share a test case to reproduce this?

commented

It may not be as small a test as you'd like, but here it is...

set -e

# Need java 1.8
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export PATH="$JAVA_HOME/bin:$PATH"

# we want $jpf_root/bin/jpf to exist
export jpf_root=$PWD/jpf-core

if [ ! -d $jpf_root ]
then
    git clone https://github.com/javapathfinder/jpf-core.git $jpf_root
    pushd $jpf_root
    bash ./gradlew
    popd
fi

export experiment_root=$PWD
mkdir -p $PWD/account/source
mkdir -p $PWD/account/outputs

# We want our program called Main
cat > Main.java << EOF
import edu.lsu.cct.javalineer.sir.account.javalineer.Accounts;
import edu.lsu.cct.javalineer.MyPool;
public class Main {
    public static void main(String[] args) {
        System.out.println("Running code 1");
        MyPool mp = new MyPool(1);
        //mp.execute(()->{ System.out.println("Hello, world"); });
        Accounts.main(args);
        System.out.println("Done");
        System.exit(0);
    }
}
EOF

if [ ! -d javalineer ]
then
    git clone -b sir https://github.com/max-morris/javalineer.git
fi

javac -d account/source/ Main.java $(find javalineer -name \*.java|grep -v /test/)
cat > runJPF.sh << EOF
#!/bin/bash
RANDOM_RUNS=1 #number of random search; default is 1
TIMELIMIT=1 #let the clock tick to ensure each random search is unique

USAGE="Usage: \$0 comp_dir search_type [number_of_runs]\ncomp_dir: orig/fixed\rsearch_type: default/random\
    \nnumber_of_runs (optional): number of times JPF is invoked"

if [ "`echo "\t"`" == "\\t" ]
then
    ECHOE="echo -e"
else
    ECHOE="echo"
fi

if [ ! \${jpf_root} ]
then
    echo "jpf_root is unset in the shell"
    echo "please set jpf_root to point to your JPF installation"
    echo "see README for more information:"
    exit 2
fi

if [ ! \${experiment_root} ]
then
    echo "experiment_root is unset in the shell"
    echo "please set experiment_root to point to your experiment directory"
    echo "see README for more information:"
    exit 2
fi

if test \$# -lt 2
then
    \$ECHOE \$USAGE
    exit 0
fi

if test \$# -eq 3
then
    case \${3} in
        *[!0-9]* )
            echo "number of JPF runs is non-numeric" >&2
            \$ECHOE \$USAGE >&2
            exit 5
            ;;
    esac

    RANDOM_RUNS=\$3
fi

subject_dir=\${experiment_root}/account
cd \${subject_dir}/source

if test \$1 = "orig"
then
    if test \$2 = "default"
    then
        echo Invoking JPF
        \${jpf_root}/bin/jpf +classpath=\${subject_dir}/source \
            Main 1 1 12 2>&1 | tee \${subject_dir}/outputs/accountJPF.log
    else 
        if test \$2 = "random" 
        then
            echo Invoking JPF
             #if using expect to limit the amount of time spent on a search
             #set the TIMEOUT value below to the number of seconds
             #the search should run and uncomment
            TIMEOUT=3600
            X=0 #loop counter
            Z=\$RANDOM_RUNS #number of runs per test case
            while [ \$X -lt \$Z ]
            do
                seed=\$RANDOM
                echo "Seed: \$seed"
                  #Example of how to run JPF using expect to limit search time
                  #run JPF using expect - uncomment next four lines
		  #expect -c "set timeout \$TIMEOUT; set log_file \${subject_dir}/outputs/accountJPF\${seed}.log; spawn \
                  #\${jpf_root}/bin/jpf +classpath=\${subject_dir}/source \
                  #+cg.randomize_choices=path +cg.seed=\${seed} \
                  #Main 1 1 12 2>&1 |; log_file \${subject_dir}/outputs/accountJPF\${seed}.log; expect; log_file" \
                  #run JPF without expect
                  #comment out the following 4 lines if using expect above
                \${jpf_root}/bin/jpf +classpath=\${subject_dir}/source \
                    +cg.randomize_choices=FIXED_SEED +cg.seed=\$seed \
                    Main 1 1 12 2>&1 | \
                    tee \${subject_dir}/outputs/accountJPF\$seed.log 
                X=\$((X+1))
                  #Random run requires a clock tick to make sure a new random
                  #number is generated
	        sleep \$TIMELIMIT
            done
        fi
    fi
else
    echo orig is the only option currently available
fi
EOF
bash -x ./runJPF.sh orig random 1
commented

Thanks for the example. It is indeed a bit complex because it requires a library. Perhaps it's possible to provide a simplified example without that library, but of course creating that takes some time. I'll look into this when I have time.

commented

I'm getting the very same error message on my side. What would you recommend to do?
Is there any way to download a stable version that does not have this issue? I obtained my current version from this repo's master branch and I see no releases here. So what can we do?

FYI I'm on opejdk ver 8 and on Ubuntu 20.04

commented

I found the source of the problem. I was using java binary version 1.8 but javac one was the latest (17).
When set javac to ver 1.8 as well and recompiled my sources the error was gone.
Sorry for a false alarm

commented

OK, thanks for confirming. I'll close this issue then.

commented

Sorry, this issue should remain open. Unlike @zavalyshyn, I was using java 1.8 and got the error.

commented

@cyrille-artho can we re-open it?

commented

Correct me if I am wrong but isn't this an array index out of bound error on those below listed lines.

commented

After checking this, I found that this application might reveal more than one bug (or unsupported feature) in JPF. I'm trying to reduce them into separate unit tests. And I'm going to concentrate on the java-10-gradle branch first.

Steps to reproduce

  1. Clone and build the lib (the resulting jar file javalineer.jar will be located at javalineer/build/libs/javalineer.jar)
git clone -b sir https://github.com/max-morris/javalineer.git
cd javalineer
git checkout 38c225e03042a8bf3dd35c19e17d40dd7a17bcf7
rm -rf src/edu/lsu/cct/javalineer/test
./gradlew build
  1. Compile the Main.java with the jar file javalineer.jar
// Main.java
import edu.lsu.cct.javalineer.sir.account.javalineer.Accounts;
import edu.lsu.cct.javalineer.MyPool;
public class Main {
    public static void main(String[] args) {
        System.out.println("Running code 1");
        MyPool mp = new MyPool(1);
        mp.execute(()->{ System.out.println("Hello, world"); });
        Accounts.main(args);
        System.out.println("Done");
        System.exit(0);
    }
}
javac -cp .:./javalineer.jar Main.java
  1. Run jpf (suppose javalineer.jar and Main.class both reside in the directory /tmp/cp/ and we are in jpf's root directory)
java -Xmx1024m -ea \
  --add-opens java.base/jdk.internal.misc=ALL-UNNAMED \
  -jar ./build/RunJPF.jar \
  +jpf-core.classpath='${jpf-core.classpath};/tmp/cp;/tmp/cp/javalineer.jar' \
  Main

Running results

In java-10-gradle branch, we will get the following error

====================================================== search started: 7/28/23, 9:52 PM
[WARNING] orphan NativePeer method: jdk.internal.misc.Unsafe.getUnsafe()Lsun/misc/Unsafe;
Running code 1
[SEVERE] JPF exception, terminating: not an int[]
---------------------- JPF error stack trace ---------------------
gov.nasa.jpf.JPFException: not an int[]
        at gov.nasa.jpf.vm.ArrayFields.getIntValue(ArrayFields.java:94)
        at gov.nasa.jpf.vm.ElementInfo.get1SlotField(ElementInfo.java:1164)
        at gov.nasa.jpf.jvm.bytecode.GETFIELD.execute(GETFIELD.java:79)
        at gov.nasa.jpf.vm.ThreadInfo.executeInstruction(ThreadInfo.java:1921)
        at gov.nasa.jpf.vm.ThreadInfo.executeTransition(ThreadInfo.java:1872)
        at gov.nasa.jpf.vm.SystemState.executeNextTransition(SystemState.java:765)
        at gov.nasa.jpf.vm.VM.forward(VM.java:1721)
        at gov.nasa.jpf.search.Search.forward(Search.java:579)
        at gov.nasa.jpf.search.DFSearch.search(DFSearch.java:79)
        at gov.nasa.jpf.JPF.run(JPF.java:613)
        at gov.nasa.jpf.JPF.start(JPF.java:189)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at gov.nasa.jpf.tool.Run.call(Run.java:80)
        at gov.nasa.jpf.tool.RunJPF.main(RunJPF.java:116)
---------------------- JPF error stack trace ---------------------
gov.nasa.jpf.JPFException: not an int[]
        at gov.nasa.jpf.vm.ArrayFields.getIntValue(ArrayFields.java:94)
        at gov.nasa.jpf.vm.ElementInfo.get1SlotField(ElementInfo.java:1164)
        at gov.nasa.jpf.jvm.bytecode.GETFIELD.execute(GETFIELD.java:79)
        at gov.nasa.jpf.vm.ThreadInfo.executeInstruction(ThreadInfo.java:1921)
        at gov.nasa.jpf.vm.ThreadInfo.executeTransition(ThreadInfo.java:1872)
        at gov.nasa.jpf.vm.SystemState.executeNextTransition(SystemState.java:765)
        at gov.nasa.jpf.vm.VM.forward(VM.java:1721)
        at gov.nasa.jpf.search.Search.forward(Search.java:579)
        at gov.nasa.jpf.search.DFSearch.search(DFSearch.java:79)
        at gov.nasa.jpf.JPF.run(JPF.java:613)
        at gov.nasa.jpf.JPF.start(JPF.java:189)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at gov.nasa.jpf.tool.Run.call(Run.java:80)
        at gov.nasa.jpf.tool.RunJPF.main(RunJPF.java:116)
commented

During diagnosing this, I found a unit test that fails on java-10-gradle branch:

  interface I {
    void foo();
  }

  @Test
  public void testLocalCapture() throws Exception {
    if (verifyNoPropertyViolation()) {
      String s = "abc";
      I i = () -> { 
        assertTrue("abc".equals(s));
      };
      i.foo();
    }
  }
commented

I have another example where I ran into a similar error, in case it might help narrowing down the cause:

public class TestArrays {
  public static void main(String[] args) {
    final int[] x = {1, 2, 3, 4, 5};
    Thread t1 = new Thread(() -> {
      int[] b = new int[x.length];
    });

    t1.start();

    try {
      t1.join();
    } catch (Exception e) {}
  }
}

The error:

====================================================== search started: 8/1/23, 7:37 PM
[WARNING] orphan NativePeer method: jdk.internal.misc.Unsafe.getUnsafe()Lsun/misc/Unsafe;
[SEVERE] JPF exception, terminating: not an array: java.lang.Thread
---------------------- JPF error stack trace ---------------------
gov.nasa.jpf.JPFException: not an array: java.lang.Thread
        at gov.nasa.jpf.vm.ElementInfo.arrayLength(ElementInfo.java:1564)
        at gov.nasa.jpf.jvm.bytecode.ARRAYLENGTH.execute(ARRAYLENGTH.java:64)
        at gov.nasa.jpf.vm.ThreadInfo.executeInstruction(ThreadInfo.java:1911)
        at gov.nasa.jpf.vm.ThreadInfo.executeTransition(ThreadInfo.java:1861)
        at gov.nasa.jpf.vm.SystemState.executeNextTransition(SystemState.java:765)
        at gov.nasa.jpf.vm.VM.forward(VM.java:1721)
        at gov.nasa.jpf.search.Search.forward(Search.java:579)
        at gov.nasa.jpf.search.DFSearch.search(DFSearch.java:79)
        at gov.nasa.jpf.JPF.run(JPF.java:613)
        at gov.nasa.jpf.JPF.start(JPF.java:189)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at gov.nasa.jpf.tool.Run.call(Run.java:80)
        at gov.nasa.jpf.tool.RunJPF.main(RunJPF.java:116)
---------------------- JPF error stack trace ---------------------
gov.nasa.jpf.JPFException: not an array: java.lang.Thread
        at gov.nasa.jpf.vm.ElementInfo.arrayLength(ElementInfo.java:1564)
        at gov.nasa.jpf.jvm.bytecode.ARRAYLENGTH.execute(ARRAYLENGTH.java:64)
        at gov.nasa.jpf.vm.ThreadInfo.executeInstruction(ThreadInfo.java:1911)
        at gov.nasa.jpf.vm.ThreadInfo.executeTransition(ThreadInfo.java:1861)
        at gov.nasa.jpf.vm.SystemState.executeNextTransition(SystemState.java:765)
        at gov.nasa.jpf.vm.VM.forward(VM.java:1721)
        at gov.nasa.jpf.search.Search.forward(Search.java:579)
        at gov.nasa.jpf.search.DFSearch.search(DFSearch.java:79)
        at gov.nasa.jpf.JPF.run(JPF.java:613)
        at gov.nasa.jpf.JPF.start(JPF.java:189)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at gov.nasa.jpf.tool.Run.call(Run.java:80)
        at gov.nasa.jpf.tool.RunJPF.main(RunJPF.java:116)

Some additional info from my experiments - the error doesn't occur if I replace x.length with a constant or an int variable. The error also doesn't occur when I don't use threads.

commented

Hi @samvid25, thanks a lot for your reporting and reduction work! And your case has the same root cause with the one I mentioned above. But don't worry, I have already figured out a fix for them. I'll create corresponding issue and PR ASAP.

commented

Another failing unit test I found on java-10-gradle branch during diagnosis this issue is:

  interface I {
    void foo();
  }

  @Test
  public void testLocalCapture() throws Exception {
    if (verifyNoPropertyViolation()) {
      double d = 2.0;
      I i = () -> { 
        assertTrue(d == 2.0);
      };
      i.foo();
    }
  }
commented

I have created issues and proposed tentative fixes for them in #378 and #379.

commented

Great, thanks!

commented

Hi @cyrille-artho and @pparizek, with PR #381 and #382 (plus another simple native peer for java.util.concurrent.atomic.AtomicReference with getAndSet() implementation), the test case provided in this issue could run for a long time without crash.

It doesn't stop maybe because there are too many generated states. A partial evidence is that if we use gov.nasa.jpf.search.PathSearch instead of gov.nasa.jpf.search.DFSearch as search class, the test case runs to stop successfully (PathSearch doesn't do backtrack).

IMHO, it could be a further enhancement to add native peers for classes under java.util.concurrent.atomic package once and for all, which improves traceability. For now, maybe we could close this issue as completed.

commented

OK, I'll close this issue, as a performance issue is not the same as a crash (if someone runs into this in a real example, we can open a new issue).