PolyUnpack: Automating the Hidden-Code Extraction of Unpack-Executing Malware

Paul Royal
Georgia Institute of Technology
USA

Mitch Halpin
Georgia Institute of Technology
USA

David Dagon
Georgia Institute of Technology
USA

Robert Edmonds
Georgia Institute of Technology
USA

Wenke Lee
Georgia Institute of Technology
USA

Modern malware often hide the malicious portion of their program code by making it appear as data at compile-time and transforming it back into executable code at run-time. This obfuscation technique poses obstacles to researchers who want to understand the malicious behavior of new or unknown malware and to practitioners who want to create models of detection and methods of recovery. In this paper we propose a technique for automating the process of extracting the hidden-code bodies of this class of malware. Our approach is based on the observation that sequences of packed or hidden code in a malware instance can be made self-identifying when its runtime execution is checked against its static code model. In deriving our technique, we formally define the unpack-executing behavior that such malware exhibits, and devise an algorithm for identifying and extracting its hidden-code. We also provide details of the implementation and evaluation of our extraction technique; the results from our experiments on several thousand malware binaries show that our approach can be used to significantly reduce the time required to analyze malware, and to improve the performance of malware detection tools.

Keywords: Malware Analysis, Automation, AntiVirus, Packing

Read Paper Read Paper (in PDF)