Bridging the Memory-Storage Gap

Abstract

The Internet has become indispensable in the developing world. It has

become an important tool for providing entertainment, for enabling and

bettering human communication, for delivering effective education, for

conducting research, for spreading news, and for organizing people

to rally to various causes. However, today, only a third of the world's

population has quality access to the Internet. The Internet has two

primary hindrances for expansion. First, the high cost of network

connectivity in developing regions and second, the high cost of

establishing new data centers to reduce the load on the existing data

centers. Fortunately, caches in various forms help address both these

problems by storing reusable content near the clients. Despite their

importance, today's caches are limited in their scale because of the

trends in the evolution of current memory-storage technologies.

The widening gap between memory and storage is limiting the

performance of applications like caches. The ever-increasing

amount of data and the need to access more of it quickly have further

magnified the gap. Limited DRAM capacity of servers makes it

difficult to obtain good in-memory hit-rates that are vital for

avoiding high-latency disk accesses. This dissertation presents two

new complementary methods (HashCache and SSDAlloc) to bridge this gap

for caches that affect the performance of many applications including

HTTP Web Proxy caches, wide area network accelerators, content

distribution networks, and file backup services. First, we will

develop HashCache, a novel method to drastically improve the memory

efficiency of caches. By reducing the amount of memory needed for

caching by up to 20 times, it reduces the cache's total cost of

ownership. While HashCache makes more effective use of limited RAM in

a system, SSDAlloc introduces a tier of new memory technology like

NAND-Flash between RAM and disk to further bridge the gap.

SSDAlloc removes the impediments to the integration of new

high-capacity memory technologies like NAND-Flash into the memory

hierarchy. SSDAlloc is a novel memory manager that helps applications

like HashCache tier data transparently and efficiently between DRAM

and NAND-Flash. With only a few modifications to an existing

application, restricted to the memory allocation portions of the code,

one can reap the benefits of new memory technologies. Additionally,

with SSDAlloc, applications can obtain 90\% of the raw performance of

NAND-Flash, while existing transparent tiering mechanisms deliver only

6-30\% of that. Furthermore, by cleverly discerning application

behavior, SSDAlloc writes up to 32 times less data to NAND-Flash when

compared to similar existing mechanisms. This greatly increases the

reliability of NAND-Flash that has a limited lifetime unlike DRAM.