A code obfuscation method using virtual machines to protect programs
A virtual machine that simulates a CPU along with a few other hardware components, allows to perform arithmetic operations, reads and writes to memory and interacts with I/O devices. It can understand a machine language which can be used to program it. Virtual machines used in code obfuscation are completely different than common virtual machnines. They are very specific to the task of executing a few set of instructions. Each instruction is given a custom opcode (often generated at random).
- NASM [tested on 2.13.02]
- Only MacOS and Linux:
- Python3 [tested on 3.6.9]
- Tkinter [tested on 8.6]
- g++ [tested on 7.5.0]
- make [tested on 4.1]
A bash script was created for easier setup of the development environment. At the beginning the script checks and installs the necessary software. Next, copy files and run some unit tests. The bash script is compatible with Debian-based distributions and Advanced Package Tool, which handle the installation and removal of software.
Or simply clone the project to the desired directory and install required software.
After setting up the environment, the directory structure looks like in the screenshot below. There are:
- Debugger - the source code of the debugger,
- SharedCode - shared codes between debugger and VMCore,
- VMCore - the source code of the virtual machine,
- vm.inc - the file with definitions of opcodes,
- Editor - the source code of the code editor (only MacOS and Linux),
- VMPROTECT.py - start here (only MacOS and Linux) 😄
In ShareCode/global.hpp you can set the target system environment.
#ifndef _GLOBAL_VARIABLES_HPP
#define _GLOBAL_VARIABLES_HPP
#define _LINUX_DEV_ENVIRONMENT
// #define _WIN32_DEV_ENVIRONMENT
#endif
The editor was written in Python. It is a plain text editor with no code syntax highlighting. You can write programs for VMPROTECT here. The window consists of a menu, a space for entering text and an output from compiling and building the program. Additionally, the editor status is shown at the bottom.
There are two options for building a program. The first mode is to compile the program into a separate file and prepare the VirtualMachineObfuscationPoC and VirtualMachineObfuscationPoC-DEBUGGER executables. The second option differs from the previous one in that it merges the compiled code with VirtualMachineObfuscationPoC. Then VirtualMachineObfuscationPoC can be executed without passing arguments.
Remember to save the source code with the extension (e.g. asm) because the editor doesn't support compiling without the extension. Which can lead to the unexpected operation of the editor program. The following files will be created in the directory where the file, with source code, was saved (please do not confuse the exe extension with PE files for the Windows operating system):
- compiled program file
- VMPROTECT.exe
- VmprotDebugger.exe
The nasm as compiler is used for compilation a code. Remember to include the vm.inc file with definitions of opcodes in your programs. Additionally, at the beginning of the code should be included magic number 0x566d. An example program for virtual machine below.
%include "vm.inc" ; Or full path to this file!
start:
dw 0x6d56
movd r0, 0x5
advrd r0, 0x5
push r0
pxvn
ee
When debugging a program you can use a dedicated debugger for VirtualMachineObfuscationPoC. The debugger has following options:
- Execute a program on the VM.
- Step execution.
- Exit debugger and exit debug mode in the VirtualMachineObfuscationPoC.
- Set a value in a register.
- Set a flag.
- Show the stack.
- Show the code data.
- Show the data buffer.
- Show all registers.
- Write to the code data.
In case of option 10, the overwritten data will start from where the PC register points. The debugger connects to the VirtualMachineObfuscationPoC using a TCP socket, default port 9313. An example of debugging is seen in the screenshot below.
An example of debugging with the V_DEBUG option turned on is seen in the screenshot below.
In vmcpu.cpp change (uncomment #define V_DEBUG, see below code sample) to print more details.
#include "../include/vmcpu.hpp"
// #define V_DEBUG <- ****uncomment this to print debug****
// #include <bitset>
VMCPU::VMCPU()
{
The VirtualMachineObfuscationPoC can be start with no arguments but there need to be set a code to execute in protected.hpp.
VBYTE ProtectedData[] = { 0xFF }; // <- HERE PASTE A CODE TO EXECUTE BY VMCPU.
Param -m set a program mode. Possible program execution modes:
- exec - normal execution,
- debug - debugging mode, you need to run VirtualMachineObfuscationPoC-Debugger.
Param -p describe a path to a file with a compiled code to execute.
Example usage of VirtualMachineObfuscationPoC below.
VMPROTECT.exe -m exec -p ./example-SumAndPrint
# OR
VMPROTECT.exe -m debug -p ./example-SumAndPrint
The VMCore use (only in Linux) the ptrace syscall in order to implement a resistent anti debugging techniques. Patching the code with NOP's will not work out of the box either, because the offset calculation must not be destroyed in order to guarantee normal execution.
offset = value;
if (ptrace(PTRACE_TRACEME, 0, 1, 0) == 0) offset = value;
if (ptrace(PTRACE_TRACEME, 0, 1, 0) == -1) offset *= value;
if (offset != value) return 0;
The VM will simulate a fictional cpu (32-bit). It has a custom instrucion set compared to x86-64.
The VM has 51,200 memory locations, each of which stores a 8-bit value (it can store a total of 50kb). The VM has stack, which is a separate data structure. The stack has 256 memory locations, each of which stores a 32-bit value (it can store a total of 512b).
Also, there is a data buffer which has 1024 memory locations, each of which stores a 1-bit value. This buffer will store user input.
typedef uint8_t VBYTE;
typedef uint16_t VWORD;
typedef uint32_t VDWORD;
#define CODE_DATA_SIZE 51200
#define STACK_SIZE 256
#define INPUT_BUFFER_SIZE 1024
VBYTE codeData[CODE_DATA_SIZE];
VDWORD stack[STACK_SIZE];
VBYTE dataBuffer[INPUT_BUFFER_SIZE];
In addition, the VMCore has implemented memory management through memory frames. This allows the execution of programs larger than those specified by CODE_DATA_SIZE. The frames will be saved in files named .cached.x.frame where x is the frame number.
Drivers are designed to expand the VMPROTECT's capabilities.
OPCODE | Mnemonic and params | Description |
---|---|---|
60 | VMSYSBUS word | Arguments to functions pass via the stack |
A sysbus is a driver that allows access to a filesystem. Arguments to functions pass via the stack.
FUNC | CMD | CODE | Windows | Linux | MacOS |
---|---|---|---|---|---|
createDirectory | sysdircr | 1 | YES | YES | NO |
deleteDirectory | sysdirdel | 2 | YES | YES | NO |
moveDirectory | sysdirmv | 3 | YES | NO | NO |
copyDirectory | sysdircp | 4 | YES | NO | NO |
createFile | sysfilecr | 5 | YES | YES | NO |
deleteFile | sysfiledel | 6 | YES | YES | NO |
moveFile | sysfilemv | 7 | YES | YES | NO |
copyFile | sysfilecp | 8 | YES | YES | NO |
Functions' implementation:
int createDirectory(std::string, int));
int deleteDirectory(std::string);
int moveDirectory(std::string, std::string);
int copyDirectory(std::string, std::string);
int createFile(std::string, uint8_t*, int);
int deleteFile(std::string);
int moveFile(std::string, std::string);
int copyFile(std::string, std::string);
Example call function use case:
%include "vm.inc"
start:
dw 0x6d56
movd r1, 0x0
movd r2, 0x0
movd r1, path
movd r2, data
push r1
push r2
vmsysbus sysfilecr
ee
data:
db 0x01, 0x02, 0x03, 0x04, 0x3, 0xD
path:
db "/home/eaglx/file.bin", 0x3, 0xD
A register is a slot for storing value on the CPU. The VM has 10 total registers, each of which is 4 bytes (32 bits). The six of them are general purpose, one has designated role as program counter and another has role as stack pointer. The VM has also two regisers ZF (Zero Flag) and CF (Carry Flag). These two provide information about the most recently executed calculation (allows to check logical conditions such as AND).
/* General Purpose Registers R0 -> R7 */
VDWORD R[8];
struct {
/* Zero Flag
value 1 - flag is set if the result of the last comparison was zero
value 0 - flag is not set
*/
unsigned char ZF : 1;
/* Carry Flag
value 1 - flag is set the results of the last comparison was moving
value 0 - flag is not set
*/
unsigned char CF : 1;
};
/* Program Counter */
VDWORD PC;
/* Stack Pointer */
VDWORD SP;
An instruction is a command which tells the CPU (and the VM's cpu) to do some task, such compare two values. Instructions have both an opcode which indicates the kind of task to perform and a set of parameters which provide inputs to the task being performed.
ADRR R2, R1 => 22 02 01
Show full list of opcodes
OPCODE | Mnemonic and params | Description |
---|---|---|
00 | NOP | No operation |
EE | EE | End of code and end of the VM's cpu |
01 | MOV rdst, rsrc | Move from a register to a register |
02 | MOVMB rdst, addrsrc | Move and extend byte from memory to a register |
03 | MOVMW rdst, addrsrc | Move word from memory to a register |
04 | MOVB rdst, byte | Move and extend byte to a register |
05 | MOVW rdst, word | Move word to a register |
06 | MOVBM addrdst, rsrc | Move byte from a register to memory location |
07 | MOVWM addrdst, rsrc | Move word from a register to memory location |
08 | MOVMRB rdst, rsrc | Move and extend byte from memory to a register; get an address from a register |
09 | MOVMRW rdst, rsrc | Move word from memory to a register; get an address from a register |
0A | MOVMD rdst, addrsrc | Move double word from memory to a register |
0B | MOVD rdst, dword | Move double word to a register |
0C | MOVDM addrdst, rsrc | Move double word from a register to memory location |
0D | MOVMRD rdst, rsrc | Move double from memory to a register; get an address from a register |
20 | JMP addr | Unconditional jump |
21 | JZ addr | Jump if equal; it set up PC to the specified location if the ZF is set (1) |
22 | JNZ addr | Jump if not equal; it set up PC to the specified location if the ZF is not set (0) |
23 | JAE addr | Jump if above or equal; it set up PC to the specified location if the ZF is set (1) and the CF is not set (0) |
24 | JBE addr | Jump if below or equal; it set up PC to the specified location if the ZF is set (1) and the CF is set (1) |
25 | JB addr | Jump if below; it set up PC to the specified location if the ZF is not set (0) and the CF is set (1) |
26 | JA addr | Jump if above; it set up PC to the specified location if the ZF is not set (0) and the CF is not set (0) |
30 | ADVR rdst, word | Add word value to a register |
31 | ADRR rdst, rsrc | Add two registers |
32 | ADRRL rdst, rsrc | Add two registers (the low byte) |
33 | SUBVR rdst, word | Substract word value from a register |
34 | SUBRR rdst, rsrc | Substract two registers |
35 | SUBRRL rdst, rsrc | Substract two registers (the low byte) |
36 | XOR rdst, rsrc | Xor two registers |
37 | XOR rdst, rsrc | Xor two registers (the low byte) |
38 | NOT rdst | Bitwise NOT on value in a register |
39 | NOT rdst | Bitwise NOT on value in a register (the low byte) |
3A | ADVRD rdst, dword | Add double word value to a register |
3B | SUBVR rdst, dword | Substract double word value from a register |
3C | SHR rdst, countbyte | Shift the bits of the operand destination to the right, by the number of bits specified in the count operand |
3D | SHL rdst, countbyte | Shift the bits of the operand destination to the left, by the number of bits specified in the count operand |
50 | CMP rdst, rsrc | Compare two registers |
51 | CMPL rdst, rsrc | Compare two registers (the low byte) |
60 | VMSYSBUS word | Arguments to functions pass via the stack |
90 | PUSH rsrc | Push value from a register to stack |
91 | POP rdst | Pop value from stack to a register |
92 | CLST | Clear the stack |
93 | SETSP dword | Set the stack pointer to the double word value |
A0 | POC | Print char without new line, the value must be at the top of the stack |
A1 | POCN | Print char with new line, the value must be at the top of the stack |
A2 | TIB | Take input and move to the data buffer, the length of the string is stored in R[7] |
A3 | GIC rsrc | Get a specific char from input, that is stored in the data buffer, the value will be stored in R[7], pass the position of char via a some register |
A4 | PIC | Print char from input without new line, the value must be at the top of the stack |
A5 | PICN | Print char from input with new line, the value must be at the top of the stack |
A6 | PXV | Print a value in hex, the value must be at the top of the stack |
A7 | PXVN | Print a value in hex with a new line, the value must be at the top of the stack |