Cache Storage, Eviction and Capacity Limits

TL;DR: Freshness decides how long a response may be reused; storage capacity decides whether it is still there to reuse. RFC 9111 explicitly allows a cache to discard any stored response at any time, and every real cache does — browsers under disk pressure, edge nodes under working-set pressure, proxies at a configured size ceiling. Eviction is invisible in the response headers, which is why capacity problems are usually misdiagnosed as directive problems.

HTTP/2 200
Cache-Control: public, max-age=31536000, immutable
Age: 0
CF-Cache-Status: MISS

A year-long lifetime and a miss on the second request is not a directive failure. It is an eviction.

Mechanism and RFC Alignment

RFC 9111 §2 is unambiguous: a cache MAY store a response, and a stored response may be removed at any time. There is no directive that compels retention. max-age, s-maxage and immutable are all ceilings on reuse — statements of how long a response remains valid — and none of them is a floor on how long it remains present.

This asymmetry is the root of most confusion about capacity. A response marked immutable with a one-year lifetime and a response marked max-age=60 are equally evictable; the only difference is what happens if they survive. The directive vocabulary has no way to express “keep this”, and deliberately so: a cache that could be compelled to retain content would be trivially exhaustible by an adversary.

What a lifetime directive controls, and what it does notDirectives bound how long a stored response stays valid; nothing in the specification bounds how long a cache must keep it.Freshness directivesan upper bound on reusemax-age, s-maxage and Expires cap how long reuse is legalimmutable additionally suppresses reload revalidationExceeding the bound requires a grace directiveStorage retentionentirely at the cache's discretionAny entry may be discarded at any moment, for any reasonDisk pressure, working-set pressure and size ceilings all evictNo directive exists that can prevent it
Validity is contractual; retention is discretionary

Eviction policy is where implementations diverge. Least-recently-used is the classic choice and still the most common, because it is cheap and adapts automatically to changing access patterns. Least-frequently-used variants retain popular objects better but respond slowly when popularity shifts. Segmented or admission-controlled policies — where a new object must prove itself before being admitted to the main store — are increasingly common at the edge, precisely because they stop a burst of one-off requests from flushing a warm working set.

Scope and Precedence

Each tier in the cache hierarchy has a different capacity model, and a response can survive comfortably in one while being evicted immediately from another.

Capacity model by cache tierThe storage limit, eviction trigger and typical object-size ceiling at each tier a response passes through.LimitEviction triggerSize ceilingBrowser memorytens of MBtab or navigationsmallBrowser diskshared origin quotadisk pressure, LRU by origin~ quota fractionCDN edge PoPper-PoP working setLRU or admission policyvendor-configuredOrigin shieldlarger, regionalLRUusually higherNginx or Varnishconfigured max_sizeLRU at the ceilingconfigurable
Four tiers, four different reasons an entry disappears

Browser storage is the tier engineers reason about least and users hit most. Browsers do not give each origin a fixed allocation; they draw from a shared pool sized as a fraction of free disk, and when that pool comes under pressure they evict entire origins in least-recently-used order rather than individual entries. A user who has not visited your site for a fortnight, on a laptop with a nearly full disk, arrives with an empty cache no matter what lifetime you published.

Edge storage behaves differently again. A CDN PoP holds a working set proportional to its own traffic, so an object requested frequently in one region and rarely in another will be warm in the first and repeatedly evicted in the second. This is one of the strongest arguments for a shield tier: the shield’s working set is the union of every PoP’s, so an infrequently-requested object evicted from a quiet PoP is still one hop away rather than a full origin fetch.

Implementation Patterns

Size the store to the working set, not the catalogue. The quantity that matters is the volume of distinct bytes requested within one freshness lifetime. A catalogue of 4 TB whose daily requests touch 90 GB needs a store somewhat larger than 90 GB, not 4 TB. Measuring it is a log query, not a guess: count distinct URLs and sum their sizes over a window equal to your longest TTL.

Separate the hot small objects from the cold large ones. Mixing a video library and a page catalogue in one store lets a handful of large objects evict thousands of small ones. Where the cache supports it, give large media its own store with its own ceiling.

Raise the object-size ceiling deliberately. Most proxies refuse to store any response above a configured size, streaming it through instead. The default is often far below what a modern asset weighs, which silently makes the largest and most expensive objects permanently uncacheable.

Prefer admission control over a bigger disk. If a nightly crawler or a bot sweep flushes the working set, more capacity postpones the problem rather than solving it. An admission policy that requires a second request before storing removes single-request objects from the store entirely.

Edge hit ratio as store size approaches the working setHit ratio rises steeply until the store can hold the set of objects actually requested within one lifetime, then flattens — additional capacity beyond that point buys very little.20 GB store47%60 GB store78%100 GB store93%400 GB store95%capacity beyond the working set returns almost nothing
Hit ratio against store size for a 90 GB working set

Server and CDN Configuration

Nginx’s proxy cache is bounded by max_size on the zone, with an object-size ceiling controlled indirectly through the buffering configuration:

proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=assets:64m
                 max_size=100g inactive=7d use_temp_path=off;

server {
    location / {
        proxy_cache assets;
        proxy_cache_valid 200 301 302 10m;
        # Store responses up to 512 MB rather than streaming them through
        proxy_max_temp_file_size 512m;
        # Do not store an object until it has been requested twice
        proxy_cache_min_uses 2;
        add_header X-Cache-Status $upstream_cache_status always;
    }
}

inactive=7d is a second, independent eviction rule: an entry untouched for seven days is removed regardless of remaining freshness. proxy_cache_min_uses 2 is the admission policy — a single-request object never reaches the store.

Varnish sizes its store at startup and evicts by LRU when it fills:

# 64 GB of file-backed storage, plus a separate store for large media
varnishd -s main=file,/var/lib/varnish/main.bin,64G \
         -s media=file,/var/lib/varnish/media.bin,256G

sub vcl_backend_response {
    # Route large objects to their own store so they cannot evict the hot set
    if (beresp.http.Content-Length ~ "^[0-9]{8,}") {
        set beresp.storage = storage.media;
    }
}

CDN storage is not directly configurable, but two levers exist: object-size limits are usually a plan-level setting worth checking against your largest assets, and a shield tier effectively multiplies the retention of rarely-requested content without any capacity change at the PoPs.

Capacity interacts with the rest of the caching vocabulary mostly by undermining assumptions built on it.

A long max-age on a rarely-requested object is close to meaningless: the object will be evicted before it is requested again. Long lifetimes pay off on hot content and on content behind a shield, and barely at all on the cold remainder of the catalogue.

Validators change the cost of eviction dramatically. An evicted entry with no validator forces a full transfer; an evicted browser entry whose bytes are still at the edge costs only the edge round trip. This is why ETag generation strategy matters more, not less, on capacity-constrained tiers.

Vary multiplies storage consumption directly. Every additional variant is a full copy of the body, so a high-cardinality Vary dimension is simultaneously a hit-ratio problem and a capacity problem — the same configuration error, counted twice.

no-store is the only directive that touches capacity deterministically, by removing the response from the store entirely. Its cost is discussed in no-cache vs no-store.

Verification Workflow

Step 1 — establish whether misses correlate with the working set. Plot hit ratio against distinct URLs requested per hour. A hit ratio that declines as distinct-URL count rises, with no configuration change, is the signature of capacity pressure rather than a directive problem.

Step 2 — check the object-size ceiling directly. Request an object above and below the suspected limit and compare the cache status:

# Small object — expect a hit on the second request
curl -sI https://example.com/assets/app.css | grep -i 'x-cache-status\|^age'
curl -sI https://example.com/assets/app.css | grep -i 'x-cache-status\|^age'

# Large object — a persistent MISS with Age: 0 indicates it is never stored
curl -sI https://example.com/media/trailer-4k.mp4 | grep -i 'x-cache-status\|^age'
curl -sI https://example.com/media/trailer-4k.mp4 | grep -i 'x-cache-status\|^age'

Step 3 — inspect the store’s own accounting. For Nginx, the cache directory’s size against max_size tells you whether the store is at its ceiling and therefore evicting continuously:

du -sh /var/cache/nginx

For Varnish, varnishstat exposes the eviction counter directly:

varnishstat -1 -f MAIN.n_lru_nuked -f MAIN.n_object

A steadily rising n_lru_nuked means the store is full and every new object costs an old one.

Step 4 — confirm the browser tier separately. Browser eviction cannot be observed from the server. Use a profile with a constrained disk, or the browser’s own storage-inspection tooling, to confirm that the entries you expect are actually retained between sessions.

Reading Capacity Pressure in the Numbers

Capacity problems announce themselves as a shape in the data rather than as an error, and the shape is distinctive enough to recognise once you know what to look for.

The first signature is a hit ratio that moves with catalogue size rather than with traffic. If doubling requests leaves hit ratio unchanged but adding ten thousand new URLs drops it four points, the store is holding a fixed number of objects and the marginal object is evicting an existing one. A configuration problem behaves the opposite way: it produces a hit ratio that is bad at every catalogue size and does not move when the catalogue grows.

The second signature is a divergence between hit ratio measured over all requests and hit ratio measured over the top thousand URLs. A healthy store shows both figures high. A store at its ceiling shows a strong head and a collapsed remainder, because the popular objects survive every eviction pass and everything else is refetched on nearly every request. Splitting the metric by request rank takes minutes to build and immediately distinguishes the two cases.

The third signature is time-of-day structure that does not match traffic. If hit ratio drops sharply every night and recovers by morning, something is sweeping the store — a crawler, a backup job, a synthetic monitor walking every URL. That is an admission-control problem rather than a sizing one, and buying capacity to solve it simply raises the cost of the nightly flush.

None of these appear in a response header, which is why capacity is so often the last thing checked. Building the three views once — hit ratio against catalogue size, hit ratio by request rank, and hit ratio by hour — turns a class of problem that normally takes days of directive archaeology into a five-minute diagnosis.

Failure Modes and Gotchas

  1. Treating immutable as a retention guarantee. It suppresses revalidation; it does not reserve space. A year-long immutable asset is evicted under pressure exactly like anything else.

  2. A store permanently at its ceiling. Once max_size is reached, every admission evicts something. Hit ratio stabilises at a value set by capacity rather than by configuration, and no directive change will move it.

  3. The inactive timer evicting fresh content. Nginx removes entries untouched for inactive, independently of freshness. A long TTL with a short inactive window silently discards everything that is not requested regularly.

  4. Large objects never being stored at all. A response above the configured ceiling is streamed through on every request. The symptom — a permanent miss on exactly the most expensive objects — is easy to mistake for a directive error.

  5. A crawler flushing the working set nightly. A sweep that requests every URL once evicts the hot set in favour of objects nobody will request again. Admission control fixes this; capacity does not.

  6. Variant explosion consuming the store. Each Vary dimension multiplies stored copies. A store sized for one copy per URL will thrash as soon as a high-cardinality dimension is added.

  7. Assuming the shield inherits PoP capacity. The shield holds the union of every PoP’s working set and is therefore the tier most likely to be undersized when a catalogue grows. Measure its hit ratio separately from the edge’s.

Frequently Asked Questions

Can a cache discard a response that is still fresh?

Yes, and every cache does. RFC 9111 permits removal at any time for any reason. Freshness directives bound how long reuse remains legal; they say nothing about retention.

How much storage does a browser give a site?

Browsers allocate from a shared pool sized as a fraction of free disk, divided across origins, and evict whole origins least-recently-used when the disk fills. The exact numbers vary by browser and version and are not a foundation to design on — assume the browser cache may be empty on any given visit.

Why do large files miss more often than small ones?

Two effects compound: most caches refuse to store responses above a configured size ceiling, and large objects consume the space many small objects would occupy, so eviction reaches them sooner. Both push large-object hit ratio below the site average.

Does eviction show up in the response headers?

No. An evicted entry produces an ordinary miss with Age: 0, indistinguishable from a first request. Eviction is only visible statistically, or in the cache’s own counters.

How large should a proxy cache store be?

Large enough for the working set — the distinct bytes requested within one freshness lifetime — with headroom. Sizing to the full catalogue wastes capacity; sizing below the working set caps hit ratio at a number configuration cannot improve.

Guides in This Topic

Each guide below works through one concrete task or question from this topic, end to end.

Back to Core Caching Fundamentals & HTTP Lifecycle