When data the processor requests resides in a cache called a cache hit it is returned immediately to the processor,
with a latency of only one or two processor cycles. If data is not in the cache called a cache miss it must be fetched from the main memory, transported into cache, and then sent to the processor, requiring hundreds of processors cycles.
Modern processors have multiple levels of caches to bridge this “memory wall.”
Caches are probably the most critical hardware component for achieving high performance in today’s computers. Caches are shared by all the software running on a machine independent of any software isolation mechanisms used.