Abstract
Cache hierarchies are increasingly non-uniform and difficult
to manage. Several techniques, such as scratchpads or reuse
hints, use static information about how programs access
data to manage the memory hierarchy. Static techniques are
effective on regular programs, but because they set fixed
policies, they are vulnerable to changes in program behavior
or available cache space. Instead, most systems rely on
dynamic caching policies that adapt to observed program
behavior. Unfortunately, dynamic policies spend significant
resources trying to learn how programs use memory, and yet
they often perform worse than a static policy.
We present Whirlpool, a novel approach that combines
static information with dynamic policies to reap the benefits
of each. Whirlpool statically classifies data into pools based
on how the program uses memory. Whirlpool then uses
dynamic policies to tune the cache to each pool. Hence, rather
than setting policies statically, Whirlpool uses static analysis
to guide dynamic policies. We present both an API that lets
programmers specify pools manually and a profiling tool that
discovers pools automatically in unmodified binaries.
We evaluate Whirlpool on a state-of-the-art NUCA cache.
Whirlpool significantly outperforms prior approaches: on
sequential programs, Whirlpool improves performance by up
to 38% and reduces data movement energy by up to 53%; on
parallel programs, Whirlpool improves performance by up to