What's wrong with veritesting?

[Akul Pillai]

Hi,

I was trying to solve 12_angr_veritesting, but I am not getting a solution even though I have enabled veritesting. This is the script I used:

import sys
import angr
import logging
logging.getLogger('angr').setLevel('DEBUG')

win = 0x08048686
lose = 0x08048698

proj = angr.Project("./12_angr_veritesting")
state = proj.factory.entry_state()

sm = proj.factory.simulation_manager(state, veritesting = True)
sm.explore(find=win,avoid=lose)

found=sm.found[0]

print found.posix.dumps(sys.stdin.fileno())

I get this at the the end of the output when I run it:

DEBUG   | 2018-03-21 15:59:36,528 | angr.manager | Filtering 1 states
DEBUG   | 2018-03-21 15:59:36,529 | angr.manager | ... state <SimState @ 0x80486f1> matched!
DEBUG   | 2018-03-21 15:59:36,529 | angr.manager | ... returning 1 matches and 0 non-matches
DEBUG   | 2018-03-21 15:59:36,529 | angr.manager | Filtering 0 states
DEBUG   | 2018-03-21 15:59:36,529 | angr.manager | ... returning 0 matches and 0 non-matches
INFO    | 2018-03-21 15:59:36,529 | angr.analyses.veritesting | Returning new paths: (successful: 0, deadended: 0, errored: 0, deviated: 1)
DEBUG   | 2018-03-21 15:59:36,529 | angr.manager | Out of states in stash active
DEBUG   | 2018-03-21 15:59:36,530 | angr.manager | Out of states in stash active
Traceback (most recent call last):
  File "angr12_test.py", line 15, in <module>
    found=sm.found[0]
IndexError: list index out of range

I also tried running the solution script given, which also didn't work and gave this:

DEBUG   | 2018-03-21 16:10:57,423 | angr.manager | Filtering 1 states
DEBUG   | 2018-03-21 16:10:57,423 | angr.manager | ... state <SimState @ 0x80486f1> matched!
DEBUG   | 2018-03-21 16:10:57,423 | angr.manager | ... returning 1 matches and 0 non-matches
DEBUG   | 2018-03-21 16:10:57,423 | angr.manager | Filtering 0 states
DEBUG   | 2018-03-21 16:10:57,423 | angr.manager | ... returning 0 matches and 0 non-matches
INFO    | 2018-03-21 16:10:57,424 | angr.analyses.veritesting | Returning new paths: (successful: 0, deadended: 0, errored: 0, deviated: 1)
DEBUG   | 2018-03-21 16:10:57,424 | angr.manager | Out of states in stash active
DEBUG   | 2018-03-21 16:10:57,424 | angr.manager | Out of states in stash active
Traceback (most recent call last):
  File "solve12.py", line 45, in <module>
    main(sys.argv)
  File "solve12.py", line 42, in main
    raise Exception('Could not find the solution')
Exception: Could not find the solution

Can someone explain what's going wrong here? and what does enabling veritesting really do?

[Mark Mankins]

Take a look at the CMU paper referenced on this page: https://docs.angr.io/docs/pathgroups.html to get an idea about what veritesting is all about.

It was helpful to me to run simulation.step() manually and examine where execution stopped after each step with veritesting on and again with veritesting off.

What you'll find is that angr is likely stepping over both your win and lose addresses with veritesting active. You'll need to adjust your win and lose addresses so they align with an address where angr stops execution.

I can't explain why the provided solution isn't working for you - it's working fine for me. I'm using python - are you using pypy?

[Akul Pillai]

Thanks, my script worked with these addresses:

win = 0x08048693
lose = 0x080486a2

and yes I guess something is wrong with my installation of angr, the provided solution and my corrected script only worked on a different machine, I will try reinstalling.
I will check out that paper too. Thanks again!