Last updated: 2026-06-29

Cache Hit Ratio and AMAT

A cache speeds up access to data by keeping frequently used items in fast storage, but it only helps in proportion to how often the data is actually found there. This calculator converts a cache's hit ratio, its hit time, and the penalty for a miss into the average memory access time (AMAT) — the single number that tells you how fast the system feels on average — along with the miss rate.

The AMAT formula

Every access either finds its data in the cache (a hit) or does not (a miss). A hit costs the hit time; a miss costs the hit time plus a penalty to fetch the data from slower memory. Averaging over many accesses gives:

$\text{AMAT} = t_h + m \cdot t_m$

where $t_h$ is the hit time, $t_m$ is the miss penalty, and $m$ is the miss rate. The miss rate is simply one minus the hit ratio:

$m = 1 - h$

Because every access pays $t_h$ but only the fraction $m$ pays $t_m$, a cache with a high hit ratio drives the average close to the fast hit time.

Worked example

Consider an L1 cache with a 95% hit ratio, a 1 ns hit time, and a 100 ns miss penalty to reach the next level:

Even though 95% of accesses are fast, the average access time is six times the hit time. The 5% of accesses that miss contribute most of the cost, because the miss penalty is a hundred times the hit time.

Why the miss rate matters more than it looks

The leverage in cache tuning comes from the large gap between hit time and miss penalty. Raising the hit ratio from 95% to 99% in the example above cuts AMAT from 6 ns to 2 ns — roughly a threefold improvement from a four-point change in hit ratio. This is why small improvements in locality, prefetching, or cache size can yield outsized gains, and why the miss penalty is worth attacking directly.

Extending to a hierarchy

Modern processors stack several caches (L1, L2, L3) in front of main memory. The formula nests naturally: the miss penalty of one level is the AMAT of the level below it. To analyze a hierarchy, compute the deepest level first and feed its result upward as the miss penalty of the level above. For the broader picture of how access latency combines across a request path, see the Latency Budget Calculator.

These figures assume a steady access pattern; real workloads vary, so treat AMAT as a representative average rather than a guarantee for any single access.