A new visualization for packed and self-modifying programs

I have been working with my PhD supervisor on a dynamic typing system to detect and visualize the temporal evolution of self-modifying programs (it’s not as complicated as it sounds). The typing system works as follows:

  • each memory address has a read, write and execution level (r, w, x)
  • initially, every memory address begin with type (0, 0, 0)
  • when an address with type (r, w, x) is executed, its type becomes (r, w, w+1)
  • when an instruction with type (r1, w1, x1) reads a memory address with type (r2, w2, x2), the target address type becomes (x1, w2, x2)
  • when an instruction with type (r1, w1, x1) writes to a memory address with type (r2, w2, x2), the target address type becomes (r2, x1, x2)

With that we can get a trace from a program (with DBI, an emulator, a debugger, whatever) and see what is executed (execution level >= 1). By construction, if we have code with an execution level of 2, it means that it has been written by the program itself before being executed, therefore it is self-modifying code.

Again by construction, if we see code with an execution level k+1, it means that it has been written by code at level k. Hence we can precisely distinguish between different layers of code (in our jargon, different code waves)

Now we can detect some interesting properties based on the type of memory addresses:

  • if an address has been read, written and then executed (RWX), we label it decrypted
  • if an address has only been written and executed (WX), we label it blind write
  • if an address has been executed and then read (XR), we assume there has been an integrity check
  • if an address has been executed and then written (XW), we assume the code has been scrambled (supposedly as an anti-memory-dump technique)

Therefore we have a way to trace different layers of code, and some relations between the layers (decryption, blind writes, integrity checking and code scrambling). This gives us the following visualization for some packers:

upx-hostnamemolebox-hostnamepec2-hostnameyp-1allaplepelock-hostnameacprotect-hostnametelock-hostname

Note 1: thanks to Silvio Cesare for providing the packed samples

Note 2: we are going to present all this stuff at Malware (Montréal) with Jean-Yves Marion and Wadie Guizani, and at Deepsec (Vienna)

Advertisements

2 thoughts on “A new visualization for packed and self-modifying programs

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s