Download Memory Hierarchy in Cache

Memory Hierarchy in Cache-based
Systems
This article is to help the reader understand the architecture of modern
microprocessors. It introduces and explains the most common terminology and
addresses some of the performance related aspects.
This is an introductory article on caches. After reading this article you should
understand how modern microprocessors work and how a cache design impacts
performance.
This article is written for programmers and people who have a general interest in
microprocessors.
Despite improvements in technology, microprocessors are still much faster than
main memory. Memory access time is increasingly the bottleneck in overall
application performance. As a result, an application might spend a considerable
amount of time waiting for data. This not only negatively impacts the overall
performance, but the application cannot benefit much from a processor clock-speed
upgrade either.
One way to overcome this problem is to insert a small high-speed buffer memory
between the processor and main memory. Such a buffer is generally referred to as
cache memory, or cache for short.
The application can take advantage of this enhancement by fetching data from the
cache instead of main memory. Thanks to the shorter access time to the cache,
application performance is improved. Of course, there is still traffic between
memory and the cache, but it is minimal. This relatively simple concept works out
well in practice. The vast majority of applications benefit from caches.
This article describes how the basic idea of caches is implemented, what sort of
caches are found in most modern systems, and their impact on performance.
Because this article is accessible to a relatively large group of readers many
important details are omitted.
1