Segmentation fault when decompiling
TambourineReindeer opened this issue · comments
This has happened with every file I've tried to decompile, mostly Windows exes, but a few elf files as well. At the decompiling stage, Boomerang will get about halfway and immediately crash around the same point every time.
The only way I've been able to circumvent this is to check "Don't decode child processes", but that greatly reduces the code output to a point that's not useful. "Enable data-flow analysis" does not make a change in whether or not it crashes.
I compiled and ran Boomerang on Linux Mint 17.2 Cinnamon 64-bit.
Here are some executables I've tried:
http://advsys.net/ken/voxlap/pnd3d.htm
https://drive.google.com/file/d/0B0Tw1fnDScRsN192NzRqYk8yckE/view
http://www.classicdosgames.com/game/Inner_Worlds.html
I may later try to debug in gdb and provide more information if possible.
Console output (sorry, won't let me attach it):
./boomerang
Not adding 0 sized section ""
Signatures directory does not exist.
Valid but undecoded instruction at "63a1f"
Valid but undecoded instruction at "63a4f"
Valid but undecoded instruction at "676b5"
Valid but undecoded instruction at "675d5"
Valid but undecoded instruction at "2387"
Valid but undecoded instruction at "25a3"
ERROR: unknown instruction INTIB at 4351, ignoring.
Valid but undecoded instruction at "4351"
ERROR: unknown instruction INTIB at 435e, ignoring.
Valid but undecoded instruction at "435e"
Valid but undecoded instruction at "404b"
Valid but undecoded instruction at "63a1f"
Valid but undecoded instruction at "63a4f"
Valid but undecoded instruction at "67735"
Valid but undecoded instruction at "678b5"
Valid but undecoded instruction at "675f5"
Valid but undecoded instruction at "67775"
Valid but undecoded instruction at "3e82"
Valid but undecoded instruction at "5e83"
Valid but undecoded instruction at "61d2"
Valid but undecoded instruction at "3ca9b"
Valid but undecoded instruction at "468b"
Valid but undecoded instruction at "ab17"
Valid but undecoded instruction at "4cddb"
Valid but undecoded instruction at "90cf"
ERROR: unknown instruction INTIB at 3fc1, ignoring.
Valid but undecoded instruction at "3fc1"
ERROR: unknown instruction INTIB at 3fce, ignoring.
Valid but undecoded instruction at "3fce"
Valid but undecoded instruction at "65d77"
Valid but undecoded instruction at "663b3"
Valid but undecoded instruction at "65fd7"
Valid but undecoded instruction at "5e127"
Valid but undecoded instruction at "677b5"
Valid but undecoded instruction at "3583"
Valid but undecoded instruction at "6053"
Valid but undecoded instruction at "3ea7"
ERROR: unknown instruction INTIB at 43a5, ignoring.
Valid but undecoded instruction at "43a5"
ERROR: unknown instruction INTIB at 43c9, ignoring.
Valid but undecoded instruction at "43c9"
Valid but undecoded instruction at "11b9f"
Valid but undecoded instruction at "6c17"
Valid but undecoded instruction at "45ff"
Valid but undecoded instruction at "13c87"
Valid but undecoded instruction at "646db"
Valid but undecoded instruction at "cd47"
Valid but undecoded instruction at "454b"
ERROR: unknown instruction INTIB at 4551, ignoring.
Valid but undecoded instruction at "4551"
ERROR: unknown instruction INTIB at 4565, ignoring.
Valid but undecoded instruction at "4565"
Valid but undecoded instruction at "4cd8b"
Valid but undecoded instruction at "846b"
Valid but undecoded instruction at "3dea"
Valid but undecoded instruction at "63c3b"
Valid but undecoded instruction at "676f5"
Valid but undecoded instruction at "3113"
Valid but undecoded instruction at "32b7"
Valid but undecoded instruction at "3457"
Valid but undecoded instruction at "17ef5"
Valid but undecoded instruction at "39a3"
Valid but undecoded instruction at "196f7"
Valid but undecoded instruction at "196be"
Valid but undecoded instruction at "1974a"
Valid but undecoded instruction at "196da"
Valid but undecoded instruction at "10ddf"
Valid but undecoded instruction at "f5df"
Valid but undecoded instruction at "157cb"
Valid but undecoded instruction at "64985"
Valid but undecoded instruction at "ba73"
Valid but undecoded instruction at "67635"
Valid but undecoded instruction at "62deb"
Valid but undecoded instruction at "63037"
Valid but undecoded instruction at "67615"
Valid but undecoded instruction at "54913"
Valid but undecoded instruction at "1107b"
Valid but undecoded instruction at "5bdb7"
Valid but undecoded instruction at "76d3"
Valid but undecoded instruction at "7737"
Valid but undecoded instruction at "409b"
Valid but undecoded instruction at "677d5"
Valid but undecoded instruction at "1a407"
Valid but undecoded instruction at "1a417"
Valid but undecoded instruction at "67bdf"
Valid but undecoded instruction at "67675"
Valid but undecoded instruction at "5f4f6"
Valid but undecoded instruction at "4ce0f"
Valid but undecoded instruction at "17acb"
Valid but undecoded instruction at "11a0f"
Valid but undecoded instruction at "67715"
Valid but undecoded instruction at "5c3fb"
Valid but undecoded instruction at "749b"
Valid but undecoded instruction at "1355f"
Valid but undecoded instruction at "48a4"
Valid but undecoded instruction at "4b57"
Valid but undecoded instruction at "1be9f"
Valid but undecoded instruction at "18467"
Valid but undecoded instruction at "3984"
Valid but undecoded instruction at "38f7"
Valid but undecoded instruction at "392d"
Valid but undecoded instruction at "19037"
Valid but undecoded instruction at "19627"
Valid but undecoded instruction at "1863b"
Valid but undecoded instruction at "5feab"
Valid but undecoded instruction at "5fc07"
Valid but undecoded instruction at "5facf"
Valid but undecoded instruction at "5fbaf"
Valid but undecoded instruction at "5fbb2"
Valid but undecoded instruction at "5fbce"
Valid but undecoded instruction at "5fb6e"
Valid but undecoded instruction at "5d4b3"
Valid but undecoded instruction at "5d88f"
Valid but undecoded instruction at "60220"
Valid but undecoded instruction at "5fa93"
Valid but undecoded instruction at "5fa4a"
Valid but undecoded instruction at "6016c"
Valid but undecoded instruction at "5eb78"
Valid but undecoded instruction at "5f0b4"
Valid but undecoded instruction at "676d5"
Valid but undecoded instruction at "648af"
Valid but undecoded instruction at "17d9a"
Valid but undecoded instruction at "67655"
Valid but undecoded instruction at "631a7"
Valid but undecoded instruction at "162cb"
Valid but undecoded instruction at "1c093"
Valid but undecoded instruction at "19237"
Valid but undecoded instruction at "185f7"
Valid but undecoded instruction at "5fa6c"
Valid but undecoded instruction at "67895"
Valid but undecoded instruction at "5c7f3"
Valid but undecoded instruction at "64b1b"
Valid but undecoded instruction at "ffdf"
Valid but undecoded instruction at "1417f"
Valid but undecoded instruction at "14347"
Valid but undecoded instruction at "1407f"
Valid but undecoded instruction at "19ea3"
Valid but undecoded instruction at "17fb7"
Valid but undecoded instruction at "191c7"
main function not found
main function not found
main function not found
Error: no entry for INT.Ib' in RTL dictionary Error: no entry forINT.Ib' in RTL dictionary
Error: no entry for INT.Ib' in RTL dictionary Error: no entry forINT.Ib' in RTL dictionary
Error: no entry for INT.Ib' in RTL dictionary Error: no entry forINT.Ib' in RTL dictionary
Error: no entry for INT.Ib' in RTL dictionary Error: no entry forINT.Ib' in RTL dictionary
considering _start
considering proc1
considering proc8
considering proc15
considering proc4
decompiling proc4
considering proc7
considering proc14
decompiling proc14
t decompiling proc7
rprrpttttr decompiling proc15
prtttr decompiling proc8
prprrrrprpttttttttttr decompiling proc1
pr considering proc2
decompiling proc2
considering proc3
considering proc9
considering proc16
considering proc86
considering proc191
considering proc315
decompiling proc315
considering proc316
considering proc440
considering proc382
decompiling proc382
decompiling proc440
rtt considering proc441
considering proc521
considering proc561
decompiling proc561
decompiling proc521
p decompiling proc441
decompiling proc316
rtt decompiling proc191
rprt considering proc192
considering proc17
considering proc87
decompiling proc87
rtt considering proc6
decompiling proc6
prt decompiling proc17
t considering proc193
considering proc317
considering proc442
decompiling proc442
r considering proc363
decompiling proc363
rt decompiling proc317
prrprrrpttttttttr decompiling proc193
rpprrrprrrrprrpprValid but undecoded instruction at "65d77"
Valid but undecoded instruction at "663b3"
Valid but undecoded instruction at "65fd7"
Valid but undecoded instruction at "67835"
Valid but undecoded instruction at "677f5"
Valid but undecoded instruction at "67815"
reconsidering proc86
considering proc576
considering proc577
considering proc583
considering proc586
decompiling proc586
considering proc587
decompiling proc587
considering proc588
decompiling proc588
considering proc589
decompiling proc589
considering proc590
decompiling proc590
considering proc591
considering proc592
decompiling proc592
decompiling proc591
rt considering proc21
considering proc5
decompiling proc5
decompiling proc21
prrtttr decompiling proc583
prttr decompiling proc577
prtt considering proc582
decompiling proc582
rprtttp considering proc203
decompiling proc203
rrpttt considering proc585
decompiling proc585
rprttttr considering proc584
decompiling proc584
prprttttt decompiling proc576
rprprtttt considering proc578
decompiling proc578
rValid but undecoded instruction at "69fc3"
Valid but undecoded instruction at "69e07"
Valid but undecoded instruction at "6a51b"
Valid but undecoded instruction at "6a5c7"
considering proc579
decompiling proc579
considering proc580
considering proc593
considering proc594
decompiling proc594
considering proc595
considering proc596
decompiling proc596
prt considering proc606
decompiling proc606
r considering proc198
considering proc600
decompiling proc600
rp considering proc599
decompiling proc599
considering proc597
considering proc603
decompiling proc603
rttp considering proc598
decompiling proc598
rprtttt decompiling proc597
prtt considering proc601
decompiling proc601
rpttrValid but undecoded instruction at "68563"
Valid but undecoded instruction at "6982b"
Valid but undecoded instruction at "6659b"
considering proc602
considering proc605
considering proc604
considering proc581
decompiling proc581
ttrprppprrpprrrrpprrrrrrpprrrrrrrpprrrrrrpprrrppprrValid but undecoded instruction at "69fc3"
Valid but undecoded instruction at "69e07"
Valid but undecoded instruction at "6a51b"
Valid but undecoded instruction at "6a5c7"
reconsidering proc593
rpprprpprrrrrpprrrrrpprrrrrrrpprrrrrrrpprrrppprrrrprprrprrrrrpprrrrrrrpprrrppprrrppppppttttttttttttttttttttttttttttttttrpprtttrtttttttpttttttttttttttttttttttttttttttttttttttttttttrrppprrtrrrrSegmentation fault
Thank you for this - free time permitting I'll look into the first two of those.
Sadly the third is a DOS executable, and boomerang does not handle those at all 😞
Leaving this open - will update if anything changes
Okay, there are a lot more, essentially all the files I've worked with, but I can't remember them all. >.<
I didn't know Boomerang didn't handle DOS executables. I actually was able to extract an elf executable from it (in my attempts at reverse engineering, using photorec to scan), and that Boomerang was able to read, but it ran into the same problem as the others with crashing partway through.
Thank you very much though :)
Here's the elf executable:
https://www.mediafire.com/?49oajbhqf39nbpr
Unfortunately the only thing that happens when I try to run it from the command line is a message to the console saying "Killed". I'm assuming since it's an old executable from the 1990s, it probably isn't runnable even if it is a Linux executable.
Pastes of GDB logs (one is the attempted decompile of the elf file I mentioned, the other is a dll file)
http://pastebin.com/sarZKyhj
http://pastebin.com/JgDWV3jb
I haven't been able to look at this too much, but I skimmed through the GDB logs again and it appears the second one involves an assertion in proc.cpp failing (line 5627, or assert(pas.second.def()); ). The first one is that it seems to have a problem with the iterators within UserProc::recursionGroupAnalysis(), or UserProc::decompile() itself which is called inside recursionGroupAnalysis().
I hope this helps and I'm sorry the GDB logs are a mess to read.
And is the console supposed to output "p" "r" and "t" or is that a bug?
The main problem seems to be with recursionGroupAnalysis and indirect control transfer processing.
At one point the dataflow and proc->cfg synchronization seems to be lost ( the size of dataflow->indices that should be exactly same as proc->getCFG()->getNumBBs() is no longer same ) - needs more investigation
@TambourineReindeer : "I'm assuming since it's an old executable from the 1990s, it probably isn't runnable even if it is a Linux executable."
... Well, not necessarily un-runnable. If you can run VirtualBox or other virtualization software, and if you can find a "free dos" or MS-DOS alternative, you may be able to run it, ... I think.
@TambourineReindeer: until Boomerang does support MS-DOS, may I humbly suggest trying your binary with the reko decompiler (https://github.com/uxmal/reko)?
Thank you all of you. @uxmal I'll look at Reko soon :3
@JustGregory I can get the executable to run in DOSBox just fine (which has a debug mode as well but I had trouble with that) and I've used it in FreeDOS as well with some success (runnable in VirtualBox but no audio so I might have been missing something). I learned later that the DOS program uses a DOS Extender called MOSS DOS extender (http://www.cs.utah.edu/flux/moss/) (http://www.bttr-software.de/forum/forum_entry.php?id=5865) and is ELF-based, hence the ELF file inside it (which decompiles under most decompilers).
@nemerle Maybe if I recompile Boomerang with -g I can get better output to help diagnose the problem.
Ah, okay, my mind must've skipped over that you said "elf-based" and failed to connect it to "Linux executable".
I'm closing this issue since Boomerang does not crash for the ELF any more, neither when using develop nor when using the capstone branch. Note that many instructions are still not recognized, so the output will not be correct. Please open a new issue if Boomerang still crashes.