# This file is part of Codemancer. # Copyright 2014-2015 Graham Shaw. # Distribution and modification are permitted within the terms of the # GNU General Public License (version 3 or any later version). ----------------------------------------------------------------------- Codemancer Version 0.0.0-rc0 (2015-01-02) ----------------------------------------------------------------------- Introduction Codemancer is ultimately intended to be an Open Source interactive disassembler for reverse engineering of executable code. Planned features include: - support for multiple instruction set architectures by means of an XML-based description language; - a database for recording information about the behaviour and purpose of code which has been disassembled; - an inference engine for augmenting the content of the database; and - an interactive front end for the user to supply information which cannot be deduced automatically. Since these goals will likely take several years to realise, features will be released incrementally as they are developed. Current status Disassembler - plain disassembly: working - semantic modelling: working, but untested and accessible only via API - control flow analysis: incomplete, experimental - data flow analysis: incomplete, experimental - interactive user interface: not started Architectures - 6500: complete, including 65C00, R65C00 and W65C00S - 6800: complete - 6805: complete - ARM: complete to armv4 but not beyond - PIC (14-bit core): complete - x86: incomplete - Z80: complete Object file formats: - ELF: working, but no relocations - COFF: working, but no relocations - AOF: working, but no relocations - more to follow - integration with disassembler: not started Building Codemancer The source code can be checked out from the public repository using git: git clone git://codemancer.org/codemancer.git The result will be placed in a newly-created directory named 'codemancer'. Subsequent changes can be fetched using the git pull command from within that directory: git pull To build the code you will need the following: - Java (tested with OpenJDK 6) - Ant (tested with version 1.7.1) Executing the 'ant' command with no arguments will build the default target, which is the Codemancer jar file: ant This is written to the pathname build/jar/codemancer.jar. You can additionally run the automated tests by building the 'test' target: ant test Running Codemancer To use the jar file it must be listed on the CLASSPATH: export CLASSPATH=build/jar/codemancer.jar The class org.codemancer.Disassemble is provided to demonstrate the ability to disassemble a raw binary file. It can be invoked from the command line as follows: java org.codemancer.Disassemble <architecture> <pathname> for example: java org.codemancer.Disassemble 6502 image.bin The resulting assembly language listing is written to stdout. Interface stability Neither the API nor the markup language should be considered stable at present. It is intended that both should become stable, but the current rate of change is such that this would seriously hamper progress if attempted now. Limitations Only a small part of the planned funcionality for Codemancer has been implemented thus far. Most notably it is only usable as a plain disassembler at present, and then only for a small number of architectures. Other functionality, to the extent that it has been implemented, should be considered experimental. Other limitations include the following: - Status flags are not properly modelled. - There is no easy way to handle registers which are part of larger registers (for example H, L and HL on the Z80). Current practice is to treat them as separate registers, which will not accurately reflect the behaviour of the program. - The <memory> element is limited to a single linear address space. - Repetition (for example, the Z80 LDIR instruction) is not modelled. - Architectures where the instruction width is not a multiple of 8 bits (for example, the PIC 12- and 14-bit architectures) could be handled more elegantly. Control and data flow analysis (experimental) Currently the analysis code is able to: - identify branches and subroutine calls to fixed addresses; - identify basic blocks, extended basic blocks and subroutines. - track the content of registers within basic blocks; - perform a partial transformation into static single-assignment form. However, it does not (yet): - analyse jump tables; - make deductions from knowledge of the calling convention (which has not been modelled yet); - track the content of stack-resident local variables; - track the content of global variables; - track the flow of data between basic blocks; - attempt to deduce the type or meaning of data values; - allow the database to be augmented by the user. In principle the algorithms implemented so far are applicable to any architecture, however they have only been tested on ARM code in an ELF object file. The algorithms will need to be made idempotent, however that has not been arranged yet. If you would like to experiment with the analysis code then the following must be accessible via the CLASSPATH: - Apache Derby (tested with 10.3.2.1); - Hibernate (tested with 3.6.9). From Derby you need both derby.jar and derbyclient.jar. From Hibernate you need hibernate3.jar, plus the dependencies found in the lib/required and lib/jpa subdirectories of the Hibernate distribution. When a binary is analysed, the results are recorded in a database. You can do this using the command: java org.codemancer.test.Analyse <projname> <architecture> <pathname> where <projname> is a project name of your choosing (used as a subdirectory name in which to store the database), <architecture> is the relevant machine architecture, and <pathname> is the binary to be analysed. Be aware that this currently quite a slow process: expect throughput of a few kilobytes per minute on mid-range hardware, so anything larger than a megabyte is likely to take many hours. When the analysis is complete, the content of the database can be listed using the command: java org.codemancer.test.ListDb <projname> The result is not expected to be useful yet, however it shows that the disassembler is able to partially understand the behaviour of the executable, and track values as they are passed from register to register. Future versions will attempt to take this analysis further, provide an interactive user interface, and improve throughput if it is feasible to do so.