Lifting an obfuscated stack machine to LLVM IR

557 views Asked by At

I am researching a code virtualization solution named VMProtect, which creates a custom stack machine with a virtual instruction set, to obfuscate code. What I want to do, is lift every virtual machine instruction handler, most of which consists of stack based operations, to LLVM IR, and then apply optimization and recompile.

I will be using llvm's IRBuilder for this.

Some of my questions are:

  1. How can push/pop be represented in IR
  2. What would a handler like this look like in IR
  mov     rdx, [rbp] ; get virtual stack operand
  add     rbp, 8         ; add to stack pointer
  mov     [rax+rdi], rdx ; move to scratch space

0

There are 0 answers