lifting-bits / fcd

An optimizing decompiler (modified to use remill semantics)

Home Page:http://zneak.github.io/fcd

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Enhance function return type recovery in `RemillArgumentRecovery`

surovic opened this issue · comments

As it is now, the analysis of function return types implemented in RecoverRetType() in the RemillArgumentRecovery pass is pretty limited and would benefit from some enhancing.

@pgoodman in #13 proposed the following:

I think you need to look at all paths leading to a ret. You might be able to simplify your life using some existing CFG pass that restructures the function. This may need to be applied to a clone of the function.

What I would consider looking into would be an combination inside-out and outside-in approach:

  • look for all definitions of return registers that lead to a ret, add those to stored_regs[func].
  • for every call to a func present in stored_regs[func], check if the anything from stored_regs[func] is used on a path that is reachable from the call, then do ret_reg[func][reg] += 1 to add confidence to that register.
  • If one function tail-calls another, then you may want to union together stored_regs[caller] += stored_regs[callee].
  • In descending order of confidence, type check ret_regs[func] for each function.
  • Some functions in stored_regs are never directly called, fall back to an approach like what you have, but expand it so that it looks beyond the terminal blocks.