Giters

UHH-ISS / binary-sight-seeing

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Our toolkit allows for binary and malware analysis. For manual analysis, the following directories are particularly important:

  • /pintool The pintool directory contains our PinTool, a utility that instruments binaries (with Intel Pin) to create detailed execution traces. These traces serve as a resource for your analytical efforts.
  • /python-puppeteer/puppeteering/poi They analyze the execution traces generated by PinTool, identifying instructions and memory addresses of searched patterns.

1. Compiling the Pintool

1.1 Compilation with Visual Studio

  1. Open the project /pintool/PuppeteeringPintool.sln in Visual Studio.
  2. Ensure the Windows Path is correctly set:
    • Right-click the PuppeteeringPintool project -> properties -> C/C++ -> Preprocessor definitions
    • Set _WINDOWS_H_PATH_ to C:/Program Files (x86)/Windows Kits/10/include/{YOUR VERSION}/um (e.g., 10.0.19041.0).

1.2 Compilation with Cygwin

The following steps assume a 32-bit Windows 7 compilation machine. Compiling on newer (and/or 64-bit) Windows Versions may be easier.

  1. Run via: .\setup-x86.exe --allow-unsupported-windows --site http://ctm.crouchingtigerhiddenfruitbat.org/pub/cygwin/circa/2022/11/23/063457
    • Install make
    • Add cygwin bin to PATH
  2. Extract Pin x.y (MSVC Version) to the root of this folder and rename the folder: pin-x.y-... -> pin (tested with: https://software.intel.com/sites/landingpage/pintool/downloads/pin-3.19-98425-gd666b2bee-msvc-windows.zip)
  3. Install Visual Studio 2019 for the MSVC Compiler (IDE is not required). Required components:
    • MSVC v142 - VS 2019 C++ x64/x86 build tools (Latest) Required for compilation
    • Windows 10 SDK (Latest) Required for Windows headers; Backward compatible with Windows 7
  4. Start the "Developer Command Prompt for VS 2019". Only the the require cl compiler is available
  5. Go into the pintool Folder and create the pintool\obj-ia32\utils subfolder.
  6. Run make - the Pintool DLL is compiled at pintool\obj-ia32\PuppetPintool.dll

2. Using the Pintool: Example with Locky on Windows 7 32-Bit VM

2.1 Required Tools

For ACSAC reviewer we provide a VM. You should have a link in the Artifact Abstract. Otherwise just write us. To make it easier we used snapshots using VMware Workstation. You can download a trial here: https://www.vmware.com/products/workstation-pro/workstation-pro-evaluation.html

Before using the Pintool, make sure the following tools are loaded onto your Windows 7 32-Bit VM:

  1. Intel Pin
  2. Pintool
  3. Procmon
  4. Process Hacker

ACSAC: After this step you are at snapshot ACSAC-Tools in the provided VM

2.2 Obtaining execution traces

  1. Start Procmon, Process Hacker, and then Locky.

  2. Allow Locky to encrypt files (observe in Procmon).

  3. Suspend Locky in Process Hacker. ACSAC: After this step you are at snapshot ACSAC-Suspend in the provided VM

  4. Attach the pintool using pin.exe -pid XXX -t t-32-r.dll -trace ransomware-locky-trace -trace_non_image 1 -ins_log ransomware-locky-ins-log

  5. Unsuspend Locky.

  6. Wait for Locky to encrypt a few (1-3) files.

  7. Suspend Locky again.

  8. The pintool should have generated two files. The tracefile ransomware-locky-trace. The instruction information file ransomware-locky-ins-log containing the instruction pointer and the instruction in ASM. ACSAC: After this step you are at snapshot ACSAC-Encrypted in the provided VM

3. Analyzing the execution trace

To analyze the execution trace, you can use any OS running Python3 along with the required packages, including 'fire'. In this example, we are using the Windows operating system.

3.1 Analyzing

In the example of Locky ransomware, you can locate all the relevant files within /python-puppeteer/locky-trace-data:

  • The execution trace archive: ransomlocky_trace.zip (unzip this file first)
  • The instruction log: ransomlocky_ins_log.1772
  • The cleartext file: test_iarg_preserve_app.cpp
  • The encrypted file: SHZIPMYC--IJMX--0TAQ--97AC0A23--BDADB310A222.osiris
  • The unpacked Locky module: locky-ent.exe_0x300000_pw_acsacinfected.zip (password protected)

  1. Identifying Search Bytes from the cleartext and encrypted File

    1. Open the encrypted file python-puppeteer/locky-trace-data/SHZIPMYC--IJMX--0TAQ--97AC0A23--BDADB310A222.osiris, and extract the first N bytes: D0912F9D.
    2. Use VM rollbacks to obtain the cleartext content of the encrypted file python-puppeteer/locky-trace-data/test_iarg_preserve_app.cpp. Choose some bytes, e.g., 466c6431 (hexadecimal for "Fld1").
    3. Note that Locky processes file content in batches of approximately 16 bytes. If you select bytes that span two different batches, you'll get no results in the next step. So, use content from the beginning of the file or different snippets.

  2. Run the python analyzer python -m puppeteering.poi.memory_pattern_extractor add $PATTERN $PATTERNNAME file ABSOLUTE_PATH_TO_TRACE_FILE

    • Pattern is the pattern to search for (byte encoded 44AB)
    • Patternname is the name you give the pattern.
    • For the example, unzip python-puppeteer\locky-trace-data\ransomlocky_trace.zip and then run from python-puppeteer python -m puppeteering.poi.memory_pattern_extractor add 466c6431 cleartext add D0912F9D Ciphertext file python-puppeteer\locky-trace-data\ransomlocky_trace\ransomlocky.1772 1 lockypois.json poi_details.log

For the locky example the result files can be found in /python-puppeteer/locky-analyzer-output

  • the found and rated POIs lockypois.json
  • the POI details log, containing the memory the POI instruction accessed (helps for further analysis) poi_details.log

3.2 Validating found POIs

3.2.1 Bare Validation

  1. The previous step should obtained some POIs. Open the instruction log ransomlocky_ins_log.1772
  2. Calculate the adresses to decimal, search for the address. See if they are instructions that are interesting, i.e., instructions that are located in a non-OS module or in no module at all. Locky example: Loading memory, xoring with something, writing memory. Might be an encryption routine 
    3153540;437;MOVDQU;movdqu xmm5, xmmword ptr [edi]; // POI Cleartext
3153544;678;PXOR;pxor xmm0, xmm5;
3153548;437;MOVDQU;movdqu xmmword ptr [edi], xmm0; // POI Ciphertext
  3. Check the output of poi_details.log what those instructions additionally accessed

3.2.2 Validation in IDA/Ghidra

  1. Obtain the binary to be analyzed. Dump the memory region using tools like ProcessHacker (right-click on locky.exe, properties, memory). Identify the region where the POIs are found (e.g., 0x300000 with size 156kB RWX). Save the memory dump (e.g., locky-ent.exe_0x300000_pw_acsacinfected.zip). Note that are not in any registered module of locky but a raw memory region.
  2. Load the dumped memory/executable into Ghidra/IDA. Pay attention to rebase the program as needed. Note that sometimes an additional offset is required. For instance, if your sample (the provided sample) has an offset of 0xC00, rebase to 0x300000 - 0xC00 = 0x2FF400.
  3. Use the provided IDA/Ghidra Python script /plugins/ida.py to import the POIs. To execute our script in IDA, go to 'File', select 'Script File', and choose the IDA script file. For GHidra, import our script through the Script Manager in the 'Window' menu. You can either add the script manually or include the folder containing it.

4. Botnet Monitoring

The proposed approach for automated botnet monitoring is outlined as follows:

  • Identification of IPs and ports accessed by a botnet using network monitoring techniques.
  • Utilization of these identified IPs and ports as inputs for the Point of Interest (POI) search.
  • Extraction of high-quality (highly rated) POIs and subsequent monitoring of their activities to uncover additional accessed resources. This process aids in the extraction of new peers (IPs, ports).

To conduct our assessment, a localized botnet environment has been established. This involved the integration of multiple virtual machines (VMs) and a router VM within a network. The virus-infected VMs were allowed to establish connections through the network. Subsequently, the malware's behavior was tracked using Intel Pin, generating a trace. This trace was then used to extract new peers.

The project encompasses a suite of scripts designed to automate various tasks, including VM setup, pausing, snapshot creation, and reset procedures on a Proxmox node. For preliminary and infrequent testing purposes, we recommend the manual setup of the local botnet, followed by the instrumentation of malware using the Pintool and analysis of the poi_details.log.

4.1 Overview

In the following, we give a high-level overview over the automated botnet analysis. It consists of the following components:

  • AnalysisPackage: Contains the botnet malware itself and the parameters needed to launch it. See /analysis-packages/ for some examples.
  • Agent:
    • Must be installed on the "infected" VM.
    • Is controlled via RPyC -- usually by the Puppeteer.
    • Dumps an interface given by the DUMPIF environment variable using dumpcap.
    • Runs an analysis package, i.e., analyzes a malware with our Pintool (PINPATH and PINTOOL environment variables must be set).
    • Communicates with the running Pintool via IPC -- this is used to get a live trace of called network functions (e.g., sendto and connect). All calls to these network functions are sent to the Puppeteer via RPyC.
    • Can dump the malware process via Process Dump.
  • Router: We have used -- by now EOL -- Zeroshell; however, any iptables based router should work. The router must ...
    • ... give the Puppeteer access to the Agent VM via RPyC (per default, listens in port 12345).
    • ... give the Agent VM access to a botnet for the given botnet malware. Can, for example, be a local botnet (see below) or real-world (via the Internet).
    • ... be reconfigurable by the Puppeteer
  • Puppeteer:
    • Controls the Agent VM via the Proxmox API, rolling it back to a known good state where the agent is running and no malware is installed at the start of each analysis.
    • Communicates via RPyC with the Agent.
    • Has a configuration file.
    • Analyzes the results (log-files) produced on the Agent VM by our Pintool.

Based on this, the basic setup is as follows:

  • Agent VM:
    • Disable Windows Defender and Firewall.
    • Install Wireshark. Folder containing dumpcap must be in PATH.
    • "Install" Process Dump and make pd32.exe available via PATH.
    • Start agent.py (install agent_requirements.txt beforehand).
    • Create live snapshot once the Agent is running.
  • Router:
    • Configure it as described above and give the Puppeteer access via SSH.
  • Analysis Machine:
    • Configure the Puppeteer to you liking (options documented in the auto_puppeteer.py).
    • Run the Puppeteer.

4.2 Small-scale Automated Botnet Analysis Tutorial

In the following, we give a short example of how the auto puppeteer can be used to analyze a local botnet. For this purpose, we deploy a Zeroaccess local botnet consisting of three bots. To allow for a simple evaluation, we rely on manual VM management. In our paper evaluation, we ran the auto puppeteer with up to 40 bots and all VMs were automatically manages through the Proxmox API (significantly complicating the setup).

For crawling, the VM management and router functionality is required. Thus, in this example we do not perform the automated crawling.

Important note: None of the VMs should have access to the host machine, local network, or Internet. They should only ever be able to access each other.

ACSAC: We provide you VMs setup according to the following description. All passwords are either user or admin (and analogous user names).

4.2.1 VM Setup

Analysis VM

  • Linux VM that runs the analysis
  • IP: 190.2.2.2
  • Install:
    • VS Code (optional)
    • Python 3 and pip
    • Python requirements: pip3 install -r binary-sight-seeing/python-puppeteer/auto_puppeteer_requirements.txt and pip3 install fire
  • Patches to the auto puppeteer (see 4_2_2.diff in this repository):
    • Removed auto starting/stopping VMs etc.
    • Removed router SSH connection (only needed for crawling)
    • Removed crawling implementation
    • Adjusted IP of agent
  • You can use /python-puppeteer/download_packages.sh to download offline copies of all packages required for the Windows 7 VM
  • ACSAC Login: user/user

Windows 7 VM

Router VM

  • Routes network traffic
  • For example: pfSense
  • Bridge interface: 190.1.1.1/8 Gateway: 190.0.0.1 (non existent)
  • Set firewall to block all outbound traffic (not 190.0.0.0/8)
  • This setup ensures that clients attempt to send data to all IPs and we can observe those attempts in Wireshark
  • ACSAC Login: admin/admin

All VMs

  • IP from 190.0.0.0/8
  • Gateway: 190.1.1.1

4.2.2 Tutorial

Using the VMs described above, perform the following steps to setup a local botnet and analyze it.

ACSAC: Import the VMs as follows.

  • All VMs must be connected to the same network bridge and the bridge must not be connected to the Internet. You may need to set the network interface after importing (e.g. in VMware: Settings -> Network Adapter -> Custom -> VMnet10)
  • Import the Analysis VM
  • Import the Router VM
  • Import the Windows 7 VM three times and set the following static IPs Windows:
    • Bot 1: with IP 190.186.175.13 (already set in the image)
    • Bot 2: with IP 190.10.10.10
    • Bot 3: with IP 190.20.20.20
    • Puppet: with IP 190.30.30.30

Start all VMs. In our images, Zeroaccess is not yet started!

Start Zeroaccess (ACSAC: Home Folder -> zeroaccess.exe) on Bot 1 and wait for 5 minutes. We recommend starting Wireshark on Bot 1 to get an overview over the botnet communication (use the filter udp.port == 16471). Then, start Zeroaccess on Bots 2 and 3. Wait for 10 minutes and create a live snapshot of all bots. At this snapshot, the local botnet is running.

On the Puppet VM in /python-puppeteer run: python agent.py Then create a snapshot. When running a new analysis, we can always rollback to this snapshot.

On the Analysis VM in /python-puppeteer run: python3 auto_puppeteer.py auto_puppeteer.ini ../analysis-packages/zeroaccess/ out

The out folder will contain the results of the analysis. This folder can be loaded into Ghidra using the /plugins/ghidra_with_puppeteer.py plugin.

About

Languages

Language:Python 67.7%Language:C++ 31.1%Language:Shell 0.5%Language:C 0.4%Language:Makefile 0.2%Language:Batchfile 0.1%