Effective and Efficient Forensic Analysis via System Monitoring

Abstract

Advanced Persistent Threat (APT) attacks and data breaches are sophisticated and stealthy, plaguing many well-protected businesses (e.g., Target, Yahoo, Home Depot, eBay, Equifax, Marriott, etc.) with significant losses. To counter these advanced attacks, approaches based on ubiquitous system monitoring have emerged as an important solution for monitoring system activities from enterprise hosts and performing forensic analysis. System monitoring audits system calls at the kernel level to collect information about system activities, providing a global view of interactions among applications and system resources. Collection of system monitoring data enables security analysts to identify the root causes and the ramifications of attacks (i.e., attack investigation) and to detect the abnormal behaviors of attacks (i.e., attack detection). However, the daunting amount of system monitoring data and the complexity of advanced attacks pose significant challenges for designing solutions for effective and efficient forensic analysis.

In this thesis, we propose novel approaches for effective and efficient forensic analysis (attack investigation and attack detection) via system monitoring. First, we propose AIQL, a system that enables efficient post-mortem attack investigation via querying the historical system monitoring data. Second, we propose SAQL, a system that enables real-time abnormal system behavior detection via querying the stream of system monitoring data. Both AIQL and SAQL provide (1) domain-specific languages that uniquely integrate critical primitives for easily incorporating the domain knowledge of security experts to express a wide range of attack behaviors, and (2) query execution engines that employ novel optimizations based on the domain-specific characteristics of the system monitoring data and the semantics of the query for efficient query execution. Finally, we propose SysRep, a system that facilitates automatic attack investigation via (1) propagating reputation from seed sources (can be trusted or suspicious) along system dependency paths to infer the reputation of POI (point of interest) entities (e.g., files, network sockets), and (2) automatically reconstructing the attack sequence from POI entities. Together, AIQL, SAQL, and SysRep work seamlessly for effective and efficient forensic analysis of APT attacks.