saruman9 / ghidra_dev_pres

Ghidra's development plugins, scripts, contributing. Presentation

Ghidra. Dev

Table of Contents

Introduction

About Ghidra

• Integrated environment for software reverse engineering (IDE for SRE)
• Developed by NSA Research
• Written almost entirely in Java
• Some native code (C/C++)

Major parts

Programs

• ELF
• PE
• Mach-O
• COFF
• PDB
• Raw binaries
• etc

Project Manager

Tools

Plugins

Scripts

Server

• Multiple users work together on the same program(s)
• Network storage for the shared project
• Controls user access
• Allows files to be versioned (à la primitive Git)
• Others features

Why use Ghidra?

• Designed to handle large data sets (not large binaries!)
• Supports collaboration
• Highly configurable environment
• Highly extensible via plugins and scripts
• Multi-platform (Linux, Mac, Windows, FreeBSD, Haiku, etc)

IDE for developing (Eclipse IntelliJ IDEA)

Why not Eclipse

Not to my taste, but…

• GhidraDev plugin (extensions and scripts developing)
• GhidraSleighEditor plugin (Sleigh linter)

My alternatives

• Emacs (Eclipse backend)?
• VIM (Eclipse backend)?
• IntelliJ IDEA (Community Edition) ✓

Integrate with Ghidra (Linux)

Plugins

• Use default Skeleton project

  Copy Skeleton’s directory (or yours prepared extension) to needed location

• Create the project
  • File → New → Project from Existing Sources... (use Gradle model)
  • Choose the directory

• Set gradle’s environment (GHIDRA_INSTALL_DIR)
  • Set in Run/Debug Configurations (Gradle)
  • Create gradle.properties file and set the environment:

    GHIDRA_INSTALL_DIR = /home/user/ghidra
            

Point to the directory of sources of Ghidra, when it will be needed

Scripts

• Create or use existing the ghidra_scripts directory
• Use existing HelloWorldScript script as a template

  ${GHIDRA_HOME}/Ghidra/Features/Base/ghidra_scripts/HelloWorldScript.java

• Import the directory as Existing Sources without models

• Add Ghidra as module to the project:
  • Open Project Structure (F4 hotkey in Project window)
  • Select Modules item
  • Import Module (select the directory of sources of Ghidra)

    

• Wait while IntelliJ IDEA indexes Ghidra’s files
• Add module dependencies via Context Actions

  

• Also you can add templates/snippets for scripts

Debug

• Run Ghidra in Debug mode

  $ /opt/ghidra/support/ghidraDebug
      

• Attach to Process…

  

• Debug a script/plugin
• Use a recompiling for testing

Ghidra developing

• Clone the repository

  $ git clone https://github.com/NationalSecurityAgency/ghidra
      

• Build Ghidra (see DevGuide.md for details)

  $ gradle --init-script gradle/support/fetchDependencies.gradle init
  $ gradle buildGhidra
      

• Create the project from Existing Sources (Gradle model)

• For testing of patches or new features you can use features of IntelliJ IDEA or Linux tools (sed, inotify, (bsd)tar, etc)

Skip Gradle’s tasks for faster building (use -x argument):

• createJavadocs
• createJsondocs
• sleighCompile
• etc

Development

Extension Point

Every piece (Plugins, Scripts, Analyzers, Fields, Importers, Exporters, etc) of Ghidra is extensible!

Extensible components:

• ExtensionPoint interface
• List suffix in ExtensionPoint.manifest

Extension source tree

src

./src
├── main
│  ├── help
│  │  └── help
│  │     ├── shared
│  │     │  └── Frontpage.css
│  │     ├── topics
│  │     │  └── skeleton
│  │     │     └── help.html
│  │     └── TOC_Source.xml
│  ├── java
│  │  └── skeleton
│  │     ├── SkeletonAnalyzer.java
│  │     ├── SkeletonExporter.java
│  │     ├── SkeletonFileSystem.java
│  │     ├── SkeletonLoader.java
│  │     └── SkeletonPlugin.java
│  └── resources
│     └── images
└── test
   └── java

./src/main
├── java
│  └── skeleton
│     ├── SkeletonAnalyzer.java
│     ├── SkeletonExporter.java
│     ├── SkeletonFileSystem.java
│     ├── SkeletonLoader.java
│     └── SkeletonPlugin.java
└── resources
   └── images

• Packaged into a .jar file
• Hold the Java/Clojure/Kotlin/JVM-based source for this extension

./src/test
└── java

• Unit tests are not included in the .jar file

./src/main/help
└── help
   ├── shared
   │  └── Frontpage.css
   ├── topics
   │  └── skeleton
   │     └── help.html
   └── TOC_Source.xml

• Online help for this extension
• Contains the table of contents to append
• Contains the CSS and HTML components

ghidra_scripts

./ghidra_scripts

• Holds scripts for this extension
• Unpacked as source to the file system on installation
• May provide examples to use an extension’s API

data

./data
├── languages
│  ├── skel.cspec
│  ├── skel.ldefs
│  ├── skel.opinion
│  ├── skel.pspec
│  ├── skel.sinc
│  └── skel.slaspec
├── buildLanguage.xml
└── sleighArgs.txt

• Holds data files for this extension
• Will not end up inside the .jar file
• Will be present in the distribution .zip file
• Unpacked to the file system on installation

lib

./lib

• Holds external Java dependencies for this extension
• When working in IntelliJ IDEA, the contents of this directory should be added by Gradle’s task

os

./os
├── linux64
├── osx64
└── win64

• Holds native components for this extension
• NEVER EVER USE JNI! Communicate with a native process using sockets, I/O, stream, etc

Other

.
├── build.gradle
├── extension.properties
└── Module.manifest

build.gradle

Gradle’s task. No need for changes

extension.properties

name=@extname@
description=The extension description.
author=
createdOn=
version=@extversion@

Module.manifest

MODULE NAME:
MODULE DEPENDENCY:
MODULE FILE LICENSE:
EXCLUDE FROM GHIDRA JAR
DATA SEARCH IGNORE DIR:
MODULE DIR:
FAT JAR:

Build extension

$ gradle

or

$ GHIDRA_INSTALL_DIR=${GHIDRA_HOME} gradle

See ./dist/

Install extension

API

Program API

• Object-Oriented
• Very deep

• Program
  • Listing
    • Instructions
    • Data
    • Functions
      • Basic Blocks
    • Comments
  • Memory
    • Memory Blocks
    • Bytes
  • Symbol Tables
    • Symbols
• etc

• Can change from version to version

Flat API

• Flat
• Provides access to most common features
• Is not complete
• Will not change (legacy)

FlatProgramAPI

public class FlatProgramAPI {
    FlatProgramAPI(Program)

    analyze()
    clear...()
    create...()
    find...()
    get...()
    remove...()
    save()
    set...()
    to...()
}

Scripting

Script Manager

Script Category Tree

Script table

Script Directories

Default:

• ${HOME}/ghidra_scripts
• ${GHIDRA_HOME}/Features/Base/ghidra_scripts
• ${GHIDRA_MODULE}/ghidra_scripts

Custom

GhidraScriptAPI

public class GhidraScriptAPI extends FlatProgramAPI {

  ask...()
  create...()
  get...()
  print...()
  run...()
  to...()
}

Sample Script

//Writes "Hello World" to console.
//@category    Examples
//@menupath    Help.Examples.Hello World
//@keybinding  ctrl shift COMMA
//@toolbar    world.png

import ghidra.app.script.GhidraScript;

public class HelloWorldScript extends GhidraScript {
        @Override
        public void run() throws Exception {
                println("Hello World");
        }
}

Script State

currentProgram

The current active open program

currentAddress

The current address of the location of the cursor

currentLocation

The program location of the cursor

OperandFieldLocation@00400260, row=0, col=0, charOffset=8, OpRep = 47h    G,
 subOpIndex = -1, VariableOffset = null

currentSelection

The current selection or null if no selection exists

currentHighlight

The current highlight or null if no highlight exists

state

Provides place to store environment variables

monitor

• Allows script writer to inform user (messages and/or progress)
• Allows user to cancel script
• Always use inside loops

while (!monitor.isCancelled()) {
...
}

Ghidra Bundles

Dynamic modules with OSGi

Dependency

• Intra-bundle (compile time) dependency

my_ghidra_scripts/mylib/MyLibrary.java:

package mylib;

public class MyLibrary {
  public void doStuff() {
    // ...
  }
}

my_ghidra_scripts/IntraBundleExampleScript.java:

// Intra-bundle dependency example.
//@category Examples

import ghidra.app.script.GhidraScript;
import mylib.MyLibrary;

• Inter-bundle (run time) dependency

@importpackage in a script

your_ghidra_scripts/yourlib/YourLibrary.java:

package yourlib;

public class YourLibrary {
  public void doOtherStuff() {
    // ...
  }
}

my_ghidra_scripts/InterBundleExampleScript.java:

// Inter-bundle dependency example.
//@category Examples
//@importpackage yourlib

import ghidra.app.script.GhidraScript;
import yourlib.YourLibrary;

public class InterBundleExampleScript extends GhidraScript {
  @Override
  public void run() throws Exception {
    new YourLibrary().doOtherStuff();
  }
}

Cleaning bundles

$HOME/.ghidra/.ghidra-<version>/osgi/compiled-bundles/<hash>

Console

Python Interpreter

Headless scripting

• Ghidra can be run from the command line without invoking the user interface
• Can be run on one or more programs
• Any script that does not invoke the GUI can be run in headless mode

Arguments

Usage: analyzeHeadless
           <project_location> <project_name>[/<folder_path>]
             | ghidra://<server>[:<port>]/<repository_name>[/<folder_path>]
           [[-import [<directory>|<file>]+] | [-process [<project_file>]]]
           [-preScript <ScriptName>]
           [-postScript <ScriptName>]
           [-scriptPath "<path1>[;<path2>...]"]
           [-propertiesPath "<path1>[;<path2>...]"]
           [-scriptlog <path to script log file>]
           [-log <path to log file>]
           [-overwrite]
           [-recursive]

[-readOnly]
[-deleteProject]
[-noanalysis]
[-processor <languageID>]
[-cspec <compilerSpecID>]
[-analysisTimeoutPerFile <timeout in seconds>]
[-keystore <KeystorePath>]
[-connect <userID>]
[-p]
[-commit ["<comment>"]]
[-okToDelete]
[-max-cpu <max cpu cores to use>]
[-loader <desired loader name>]

Plugins

Derive from Plugin (directly or indirectly)

Services

/**
 * Plugin to for adding/deleting/editing bookmarks.
 */
//@formatter:off
@PluginInfo(
  status = PluginStatus.RELEASED,
  packageName = CorePluginPackage.NAME,
  category = PluginCategoryNames.CODE_VIEWER,
  shortDescription = "Manage Bookmarks",
  description = "This plugin allows the user to add, edit, " +
      "delete, and show bookmarks. It adds navigation markers at " +
      "addresses where bookmarks reside.",
  servicesRequired = { GoToService.class, MarkerService.class },
  servicesProvided = { BookmarkService.class },
  eventsProduced = { ProgramSelectionPluginEvent.class }
)
//@formatter:on

/**
 * Plugin to for adding/deleting/editing bookmarks.
 */
...
  servicesRequired = { GoToService.class, MarkerService.class },
  servicesProvided = { BookmarkService.class },
...

Events

/**
 * Visible Plugin to show ByteBlock data in various formats.
 */
//@formatter:off
@PluginInfo(
  ...
  shortDescription = "Displays bytes in memory",
  ...
  servicesRequired = { ProgramManager.class, GoToService.class,
    NavigationHistoryService.class, ClipboardService.class },
  eventsConsumed = {
    ProgramLocationPluginEvent.class, ProgramActivatedPluginEvent.class,
    ProgramSelectionPluginEvent.class, ProgramHighlightPluginEvent.class,
    ProgramClosedPluginEvent.class, ByteBlockChangePluginEvent.class },
  eventsProduced = { ProgramLocationPluginEvent.class,
    ProgramSelectionPluginEvent.class, ByteBlockChangePluginEvent.class }
)
//@formatter:on

/**
 * Visible Plugin to show ByteBlock data in various formats.
 */
...
  eventsConsumed = {
    ProgramLocationPluginEvent.class, ProgramActivatedPluginEvent.class,
    ProgramSelectionPluginEvent.class, ProgramHighlightPluginEvent.class,
    ProgramClosedPluginEvent.class, ByteBlockChangePluginEvent.class },
  eventsProduced = { ProgramLocationPluginEvent.class,
    ProgramSelectionPluginEvent.class, ByteBlockChangePluginEvent.class }
...

Actions

DockingAction defines a user action associated with a toolbar icon and/or menu item.

DockingActions can be invoked from the global menu, a popup menu, a toolbar, and/or a keybinding.

GUI

A plugin may supply a ComponentProvider that provides a visual component when the plugin is added to the tool

Guidelines

• All Plugin Classes MUST END IN ”Plugin”^*
• Have a @PluginInfo(...) annotation

  PluginInfo(
    status = PluginStatus.RELEASED,
    packageName = CorePluginPackage.NAME,
    category = PluginCategoryNames.COMMON,
    shortDescription = "Short description of plugin",
    description = "Longer description of plugin.",
    servicesProvided = { ServiceInterfaceThisPluginProvides.class }
    servicesRequired = { RequiredServiceInterface1.class, RequiredServiceInterface2.class },
    eventsConsumed = { SomePluginEvent.class },
    eventsProduced = { AnotherPluginEvent.class },
    isSlowInstallation = false
  )
      

• Have a constructor with exactly 1 parameter: PluginTool

  public MyPlugin(PluginTool tool) { ... }
      

• Register provided service implementations in constructor
• In init()
  • Retrieve services consumed
  • Create actions
• In dispose()
  • Release resources

Useful interfaces

• OptionsChangeListener — to receive notification when a configuration option is changed by the user
• FrontEndable — marks this Plugin as being suitable for inclusion in the FrontEnd tool
• FrontEndOnly — marks this Plugin as FrontEnd only, not usable in CodeBrowser or other tools
• ProgramaticUseOnly — marks this Plugin as special and not for user configuration

ProgramPlugin

Extends Plugin class

Base class to handle common program events:

• Program Open/Close
• Program Location
• Program Selection
• Program Highlight

GUI

Docking Windows

• Title bar
• Local toolbar
• Menu icon
• Close button
• Mouse cursor provides feedback
• Components can be stacked, docked or floating

Allow users to customize the layout of components within a tool

Not used GUI toolkit

Component Provider

Managed GUI component in the tool

myComponent = new MyComponent(...);
tool.addComponent(this, myComponent);

• Permanent — are always available in Window menu and closing just hides them (e.g., Listing)
• Transient — created on the fly and when closed are destroyed and removed from Window menu (e.g., Search Results)

Custom Components

• GTable
• GTree
• GComboBox
• etc

Provides:

• Custom filtering
• Event handling
• Threaded models
• Navigation
• Look-and-feel

Tables

GTable

• Filters
• Columns
• Export to CSV

Trees

GTree

• Filters
• Lazy loading to support large data

Actions

• Right mouse actions are context sensitive
• List of actions that appear will change based on where the cursor is located
• User can assign/override key bindings (F4)

Binary Formats

• ELF
• PE
• Mach-O
• etc

New Binary Format

What is needed for a new format?

• Data structure to parse the format
• Language, if not currently supported
• Loader for Ghidra’s importer
• Analyzers to annotate the binary format

Use Skeleton extension

Sleigh / PCode

• Used to disassemble binary to assemble
• Used to decompile assembly into C
• Decompiler optimizes
• Performs data type propagation
• etc

${GHIDRA_HOME}/Ghidra/Processors

Loaders

Must implement Loader interface (all Loader Classes MUST END IN ”Loader”)

Consider AbstractProgramLoader or AbstractLibraryLoader

public class SkeletonLoader implements Loader {
  Collection<LoadSpec> findSupportedLoadSpecs(...);
  List<DomainObject> load(...);
  ...
}

.opinion files

Must update the .opinion file for each processor supported by the format

<opinions>
  <constraint loader="Skeleton" compilerSpecID="default">
    <constraint primary="40" secondary="123"
      processor="Skel" size="16" variant="default" />
  </constraint>
  <constraint loader="MS Common Object File Format (COFF)"
    compilerSpecID="default">
    <constraint primary="61" processor="Skel" size="16"
      variant="default" />
  </constraint>
</opinions>

Analyzers

Must implement Analyzer interface (all Analyzer Classes MUST END IN ”Analyzer”)

// Display name, input type, priority
String getName();
AnalyzerType getAnalysisType();
AnalysisPriority getPriority();

// Called for changes to analyzers inputs
boolean added(...);
boolean removed(...);

// Register and react to user options
void registerOptions(...);
void optionsChanged(...);

...

AnalyzerType

• BYTES — analyze anywhere defined bytes are present (block of memory added)
• INSTRUCTIONS — analyze anywhere instructions are defined
• FUNCTIONS — analyze where a function is defined
• FUNCTION-MODIFIERS — analyze functions whose modifiers have changed
• FUNCTION-SIGNATURES — analyze functions whose signatures have changed
• DATA — analyze where data has been defined

File System Loader

Provides an alternative importer, which allows importing many programs from a single archive or image

• Drills down into file systems, including nested file systems
• Extract files
• Import binaries
• Perform static analysis
• View as text
• View as image (i.e., picture)

Examples

• Android XML
• Android APK
• Apple 8900
• GZIP
• YAFFS2
• Android DEX to JAR
• 7Zip

• COFF Archive
• CPIO
• Crushed PNG
• iOS Disk Image (DMG)
• iOS DYLD Cache Version 1
• etc

Implementation

Extensible using GFileSystem interface (all GFileSystem Sub-classes MUST END IN ”FileSystem”)

@FileSystemInfo(type = "ext4",
                description = "EXT4",
                factory = Ext4FileSystemFactory.class)
public class Ext4FileSystem implements GFileSystem {
  ...
}

Contributing to Ghidra

• Search in the sources, read issues (don’t do double work)
• Ask a question in Telegram/Matrix/Discord
• Write an issue
• Create a PR
• Wait…