Detection and Mitigation of Security Threats in Cloud Computing

Abstract

Infrastructure-as-a-Service (IaaS) clouds provide computation and storage services to enterprises and individuals with increased elasticity and low cost. Cloud customers rent resources in the form of virtual machines (VMs). However, these VMs may face various security threats.

This dissertation proposes a new architectural framework, CloudMonatt, to detect and mitigate potential security threats targeting customers’ VMs in cloud computing. CloudMonatt monitors the security health of VMs and attests to customers if they are getting their desired security. It takes actions to mitigate the potential threats that can compromise the security properties requested. We design cloud management and security services, and define new hardware-software modules in cloud servers to provide the underlying measurements. We define secure communications protocols to guarantee that the monitoring service takes place in an unforgeable way.

To demonstrate how CloudMonatt can enhance the VMs’ security, we consider a variety of threats and their defenses that can be integrated in CloudMonatt. We first consider threats on resource availability. We design a set of memory Denial-of-Service (DoS) attacks: an attacker VM can abuse the shared memory resources to significantly degrade a victim VM’s performance. Then we statistically monitor VMs’ resource consumption behaviors to detect these attacks, and use resource throttling to mitigate the availability threats.

Next, we consider subtle attacks on confidentiality, specifically cache side-channel attacks. An attacker VM can exploit a shared CPU cache to steal information from the victim VM. We collect VMs’ micro-architectural behaviors and use a combination of signature and anomaly detection techniques to identify the existence of various side-channel attacks. We use targeted VM migration to eliminate these confidentiality threats.

Then, we consider attacks on system integrity within a VM. We show how to protect a VM’s system integrity from malware, using Virtual Machine Introspection (VMI) to passively collect information for malware detection and also actively change the VM’s execution paths to defeat the potential malware.

In summary, CloudMonatt is a general-purpose architecture for providing VM security monitoring and protection to cloud customers. We hope CloudMonatt can be a foundation for future work on protecting VMs’ security health in cloud computing.