Wei Li
Rether Networks Inc.
USA

Lap-chung Lam
Rether Networks Inc.
USA

Tzi-cker Chiueh
Rether Networks Inc.
USA

Comparing the system call sequence of a network application against a sandboxing policy is a popular approach to detecting control-hijacking attack, in which the attacker exploits such software
vulnerabilities as buffer overflow to grab the control of a victim application and possibly the underlying machine. The main barrier to the acceptance of this system call monitoring approach is the availability of accurate sandboxing policies, especially for Windows applications whose source code is unavailable. In fact, many commercial computer security companies take advantage of this fact and fashion a business model in which their users have to pay a subscription fee to receive periodic updates on the application sandboxing policies, much like anti-virus signatures.
This paper describes the design, implementation and evaluation of a sandboxing system called BASS that can automatically extract a highly accurate application-specific sandboxing policy from a Win32/X86 binary, and enforce the extracted policy at run time with low overhead. BASS is built on a binary interpretation and analysis infrastructure called BIRD, which can handle application binaries with dynamically linked libraries, exception handlers
and multi-threading, and has been shown to work correctly for a large number of commercially distributed Windows-based network applications, including IIS and Apache. The throughput and
latency penalty of BASS for all the applicationse we tested except one is under 8%.

Keywords: Sandboxing, Behavior blocking, Zero-day attack, System call argument check

Read Paper (in PDF)