Deploying a Smart Sandbox for Unknown Threats and Zero-Day Attacks

Zero-day exploits pose some of the most serious risks to users everywhere. The absence of a patch means that it is up to users (and whatever security products they use) to protect against these attacks.

One of the tools that can be used in mitigating these attacks is advanced network detection solutions like Trend Micro Deep Discovery, which contains a sandbox that allows for on-the-fly analysis of various threats entering an organization’s network. This allows it to detect even attacks that use zero-day exploits without any updates being necessary, providing immediate protection to users.

Problems for common sandboxes

In today’s threat environment, sandboxing is necessary to defend against persistent threats. These generally rely on behavioral analysis within a virtual environment to detect various threats. As they become more commonplace, attackers will attempt to find methods to evade these sandboxes.

This means that attackers need to only exert some effort to show less behavior in a sandbox, such as anti-VM and anti-sandbox techniques. It is important for sandboxes to reflect user environments as accurately as possible; Deep Discovery’s custom sandbox can be configured by administrators.

This poses a challenge in the traditional field of file detection, which has expanded in recent years to exploits. There are several critical challenges to typical sandboxes:

• The exploit is used to not only deploy a payload, but also to conceal it.
  The malware payload is encrypted so that the sandbox cannot identify if it is an executable file. The shell code in the exploit is responsible for decrypting the payload before it can be executed. In the simplest cases, the malware payload is simply XOR-ed; however we have seen more complex algorithms used.
  Some payloads are even designed to execute in memory directly, which means you cannot get a completed PE file to execute within the sandbox. A common sandbox cannot easily detect malware that uses this evasion method.
• Exploits evades the sandbox as well.
  Typical sandboxes run specific file types such as .SWF, .JAR, .PDF, in order to check if these files contain exploit code. identify whether it’s an exploit. Attackers know all about this, however, and try to evade it. The exploit code can include lines that will check the running environment of the exploit, or parameter/function calls from HTML. The exploit code won’t run if it is opened directly, or in an incorrect context.

The Flash zero-day exploits we analyzed earlier this year used these methods to evade detection by common sandboxes. Smart sandboxes (as used by Trend Micro Deep Discovery) have the capability to deal with these evasion techniques and successfully detect zero-day exploits.

Smart sandbox

Compared to a common sandbox, a smart sandbox is capable of analyzing the behavior of multiple aspects of a threat: its scripts, its shellcode, and its payload, within a customizable sandbox.

Figure 1. Structure of a custom sandbox

Script behavior can tell us an exploit’s anomalous object usage, function calls, and heap sprays. Variables can also be analyzed for ROP/shellcode data.

Meanwhile, shellcode data can detect an exploit’s usage of stacks and heaps caused by ROP/shellcode execution, and anomalous file/registry operations in application processes. Analysis of payloads can reveal the scope of their impact on systems, such as created autorun routines, dropped files, and connections to C&C servers. This is the same kind of analysis used in traditional behavior analysis.

Why is a smart sandbox necessary? More and more exploit kits are using advanced obfuscation and evasion techniques:

Figure 2. Evasion against static scan used in popular exploit kits

The sandbox in Deep Discovery was built differently to allow our products to deal with these attacks. The sandbox contains an emulator that acts as a customized ActionScript Virtual Machine for Flash exploits, as well as script engines that run Java, JavaScript, and VBScript. This control over the execution environment improves the ability of the sandbox to gather information about any code that is tested within the sandbox.

For example, recent Adobe Flash zero-day exploits have been heavily encrypted to prevent static heuristics from successfully analyzing them. However, Deep Discovery was able to capture and identify this malicious behavior. In addition, we are able to get a more complete picture of exploits.

Taken together, this allows Deep Discovery to more quickly detect zero-day threats. Its smart sandbox is able to detect even obfuscated exploits more reliably. No update is necessary to provide protection against these attacks. This provides immediate protection against zero-day attacks; it also allows system administrators to see if they are targets and act accordingly.