Exploring Data Compression and Random-Access Reduction to Mitigate the Bandwidth Wall for Manycore Architectures

Abstract

The performance gap between computer processors and memory bandwidth is severely limiting the throughput of modern and future multi-core and manycore architectures. To handle this growing gap, commercial processors such as the Intel Xeon Phi and NVIDIA or AMD GPUs have needed to use expensive memory solutions like high-bandwidth memory (HBM) and 3D-stacked memory to satisfy the bandwidth demand of the growing core-count over each product generation. Without a scalable solution for the memory bandwidth issue, throughput-oriented computation cannot be improved. This problem is widely known as the bandwidth-wall.

Data compression and random-access reduction are promising approaches to increase bandwidth without raising costs.

This thesis makes three specific contributions to the state-of-the-art. First, to reduce cache misses, we propose an on-chip cache compression method that drastically increases compression performance and cache hit rate over prior work. Second, to improve direct compression of off-chip bandwidth and make it more scalable, we propose a novel link compression framework that exploits the on-chip caches themselves as a massive and scalable compression dictionary. Last, to overcome poor random-access performance of nonvolatile memory (NVM) and make it more attractive as a DRAM replacement with crash consistency, we propose a multi-undo logging scheme that seamlessly logs memory writes sequentially to maximize NVM I/O operations per second (IOPS).

As a common principle, this thesis seeks to overcome the bandwidth wall for manycore architectures not through expensive memory technologies but by assessing and exploiting workload behavior, and not through burdening programmers with specialized semantics but by implementing software-transparent architectural improvements.