Heuristic Freshness and Implicit Caching

TL;DR: When a response carries no max-age, no s-maxage and no Expires, a cache is still permitted to store and reuse it — inventing a freshness lifetime of its own. RFC 9111 §4.2.2 authorises this, most implementations settle on roughly 10% of the time since Last-Modified, and the result is content being cached for a duration nobody chose. The fix is not to disable caching; it is to always send an explicit lifetime, even when that lifetime is zero.

HTTP/2 200
Date: Fri, 31 Jul 2026 09:00:00 GMT
Last-Modified: Fri, 24 Jul 2026 09:00:00 GMT
ETag: "2f10-9c4"

No caching directive appears anywhere in that response, and yet a shared cache may reuse it for the next several hours.

Mechanism and RFC Alignment

RFC 9111 §4.2.2 states that a cache may assign a heuristic freshness lifetime when no explicit expiration time is available, provided the response’s status code is one the specification permits to be cached by default. The specification deliberately does not prescribe an algorithm. It offers one constraint and one suggestion: the heuristic must be conservative, and the interval since Last-Modified is a reasonable basis for it.

The convention that emerged — take the difference between Date and Last-Modified, and use 10% of it — traces back to the early Squid and Apache proxy implementations and has since been adopted almost universally. For the response above, Date minus Last-Modified is seven days, so 10% is roughly 16.8 hours. A cache applying that convention will serve the stored copy without contacting the origin for the rest of the day.

How a heuristic lifetime is derivedWith no explicit directive present, a cache measures the interval since the resource last changed and assigns a fraction of it as the freshness lifetime.Last-Modified24 Jul 09:00response ageDate31 Jul 09:00subtractInterval7 days10%Lifetime16.8 hours
Seven days since the last change becomes nearly seventeen hours of freshness

Two properties of this rule are worth pausing on. First, it is self-reinforcing: the longer a resource has gone without changing, the longer a cache will hold it. A file untouched for a year receives a heuristic lifetime of over a month. Second, it produces wildly different lifetimes for resources that feel equivalent to their author — a page edited yesterday gets minutes, the identical page edited last spring gets days.

The specification adds one further requirement that implementations honour inconsistently: when a heuristic lifetime greater than 24 hours is applied to a response, the cache should attach a warning indication. In practice almost nothing surfaces this, so the only way to discover a heuristic lifetime is to observe the Age header climbing past any value you intended.

Scope and Precedence

Heuristic freshness sits at the very bottom of the precedence order described in header stacking and directive precedence. It is consulted only when every explicit source is absent.

Where the heuristic sits in the precedence orderEvery explicit freshness source is consulted before the heuristic, and the presence of any one of them suppresses it entirely.CONSULTED FIRST1s-maxageshared caches2max-ageall caches3Expiresabsolute instant4Heuristic freshnessinvented by the cache5No storageonly when a directive forbids itONLY IF EVERYTHING ABOVE IS ABSENT
The heuristic is the last resort, not a default

The critical consequence is that max-age=0 is not the same as sending nothing. A response with max-age=0 is explicitly stale on arrival and must be revalidated; a response with no directive at all may be reused for hours. Teams frequently discover this the hard way when they remove a caching header believing it will disable caching, and the opposite happens.

Status code eligibility is the second half of the scope question. RFC 9111 §4.2.2 defers to the list of cacheable-by-default status codes in RFC 9110 §15: 200, 203, 204, 206, 300, 301, 308, 404, 405, 410, 414 and 501. Every other status — including 302, 307 and the entire 5xx range — may only be stored when explicit freshness information is present.

Status codes and heuristic eligibilityWhich response statuses a cache may store and reuse without any explicit freshness directive, and which require one.Heuristically cacheableNeeds explicit freshnessCommon surprise200 OK✓✗expected301 Moved Permanently✓✗redirect loops outlive the fix404 Not Found✓✗deploy glitch outlives the fix410 Gone✓✗intentional, but permanent302 Found✗✓often assumed cacheable500 Internal Server Error✗✓never cached by accident
The heuristic applies only to the cacheable-by-default statuses

The 301 and 404 rows account for most real incidents. A redirect published in error, or a page that briefly 404s during a deploy, becomes a stored response with an invented lifetime — and because no directive was involved, no one thinks to look at caching when the symptom appears.

Implementation Patterns

The pattern that eliminates the entire class of problem is to guarantee that no response leaves the origin without an explicit lifetime. Three approaches are common, and they compose well.

A default at the server. Configure a fallback Cache-Control on every response, then override it per route. This is the most reliable placement because it applies to responses the application never explicitly handles — error pages, redirects, static file handlers.

# Applies to every response unless a location block overrides it
add_header Cache-Control "public, max-age=60" always;

location ~* \.(?:js|css|woff2)$ {
    # Fingerprinted assets: opt out of the default entirely
    add_header Cache-Control "public, max-age=31536000, immutable" always;
}

location /api/ {
    # Dynamic responses: explicit zero rather than silence
    add_header Cache-Control "public, max-age=0, s-maxage=30" always;
}

The always flag matters: without it, Nginx omits add_header on error responses, which is precisely the case the default exists to cover.

An explicit rule for error statuses. Errors are transient by nature and should never be stored for an invented duration.

# Error responses must not outlive the condition that produced them
error_page 404 = @notfound;
location @notfound {
    add_header Cache-Control "no-store" always;
    return 404;
}

A guard at the edge. Even with the origin configured correctly, a header can be lost in a rewrite. A CDN-level rule that stamps a conservative default onto any response arriving without one closes the gap:

# Fastly VCL — in vcl_fetch
if (!beresp.http.Cache-Control && !beresp.http.Expires) {
    set beresp.http.Cache-Control = "public, max-age=0, s-maxage=60";
}

Server and CDN Configuration

Vendors differ in how aggressively they apply heuristics, and knowing your own stack’s behaviour matters more than knowing the convention.

Cloudflare does not apply a Last-Modified-derived heuristic to arbitrary responses; it caches by file extension for a default set of static types and treats everything else as uncacheable unless a page rule or Cache-Control says otherwise. That is a safer default than the classic proxy behaviour, but it means the same origin behaves differently behind Cloudflare than behind a standards-following proxy.

Fastly applies a configurable default TTL — often 3600 seconds — to any cacheable response arriving without directives, rather than deriving one from Last-Modified. Varnish behaves the same way, governed by the default_ttl parameter, which is 120 seconds out of the box.

Squid and most standards-following forward proxies implement the classic percentage heuristic, bounded by configurable minimum and maximum values.

# Apache: emit an explicit lifetime for everything, including errors
Header always set Cache-Control "public, max-age=300"


    Header always set Cache-Control "public, max-age=31536000, immutable"

The practical takeaway is that a response with no directives has no single defined behaviour. It has as many behaviours as there are caches in the path.

Heuristic freshness interacts with the rest of the caching vocabulary in ways that are mostly about suppression.

Any explicit lifetime suppresses it. That includes s-maxage, which suppresses the heuristic for shared caches while leaving browsers to apply their own — a split worth remembering when a page behaves differently at the edge and in the browser.

no-cache suppresses it in effect rather than by rule: the response may still be stored, but it can never be reused without revalidation, so no invented lifetime is ever applied. See no-cache vs no-store for the distinction between the two.

A validator changes the consequences rather than the calculation. A heuristically cached response that carries an ETag becomes cheap to correct once it expires, because revalidation returns 304 Not Modified rather than a full body. A heuristically cached response with no validator is expensive in both directions: stale for an invented duration, then a full refetch.

Age is the observable signal. A response you believed was uncached, arriving with a climbing Age value, is the clearest evidence that a heuristic is in play — see how to read and interpret the Age header.

Verification Workflow

Establishing whether heuristic caching is affecting a route takes two requests and one comparison.

Step 1 — confirm the response carries no explicit lifetime.

curl -sI https://example.com/some-page \
  | grep -iE 'cache-control|expires|last-modified|^age|^date'

If Cache-Control and Expires are both absent, the response is a heuristic candidate.

Step 2 — establish whether it is being stored.

curl -sI https://example.com/some-page | grep -i '^age'
sleep 20
curl -sI https://example.com/some-page | grep -i '^age'

A second Age value roughly 20 seconds higher than the first confirms a shared cache is holding the response. With no directive present, the lifetime it is applying was invented.

Step 3 — compute the lifetime the convention would produce.

curl -sI https://example.com/some-page \
  | awk -F': ' '/^[Dd]ate|^[Ll]ast-[Mm]odified/ {print $2}'

Subtract the two timestamps and take a tenth. For a resource last modified a week ago, expect somewhere near seventeen hours — and compare that against how long you would actually be comfortable serving the page unchanged.

Step 4 — confirm the fix suppresses it. After adding an explicit directive, repeat step 1 and verify Cache-Control is present. The heuristic is now unreachable regardless of what Last-Modified says.

Failure Modes and Gotchas

  1. Removing a header to “disable caching”. Deleting Cache-Control does not disable caching; it hands the decision to every cache in the path. To reduce caching you must say so explicitly.

  2. A deploy-time 404 stored for hours. A brief window where a URL returns 404 produces a heuristically cacheable negative response. The page is fixed at the origin and still missing at the edge. Purge after any deploy that could have produced errors, and mark error responses no-store.

  3. A mistaken 301 becoming effectively permanent. Permanent redirects are heuristically cacheable and browsers store them aggressively. A redirect published in error can persist in individual browsers long after the origin stops sending it, because nothing prompts the browser to re-ask.

  4. Responses with no Last-Modified at all. With no basis for the calculation, implementations diverge completely — some apply a fixed default, some decline to store. The behaviour is unpredictable rather than absent.

  5. API responses inheriting a static-file default. A server-wide default written for assets, applied to a JSON endpoint that omits its own header, produces exactly the stale-data incident the endpoint was never designed to tolerate.

  6. Clock skew distorting the interval. The calculation depends on Date and Last-Modified being produced by clocks that agree. An origin whose clock drifts produces heuristic lifetimes that drift with it — see handling clock skew in Last-Modified validation.

  7. Assuming the browser and the edge agree. The two tiers frequently apply different heuristics to the same response, because a CDN’s default TTL and a browser’s percentage-based guess have nothing in common. A page can be minutes old in one and hours old in the other, which produces the confusing report that a change is visible in a private window but not on a colleague’s machine, or vice versa. Whenever a staleness complaint cannot be reproduced consistently, check whether any explicit lifetime is present before investigating anything else — divergent heuristics are a far more common cause than a genuine cache bug, and they resolve the moment a directive is added.

Frequently Asked Questions

When is a cache allowed to invent a freshness lifetime?

Only when the response carries no explicit freshness information at all — no max-age, no s-maxage, no Expires — and its status code appears on the cacheable-by-default list. Any explicit lifetime, including zero, removes the heuristic from consideration entirely.

Is the 10% of Last-Modified rule part of the specification?

No. RFC 9111 §4.2.2 permits a heuristic and suggests the interval since Last-Modified as a reasonable input, but the 10% figure is a convention inherited from early proxy implementations. Each cache chooses its own algorithm, which is precisely why depending on the number is unwise.

Why did a 404 response stay cached after we fixed the page?

404 is heuristically cacheable, so a shared cache may store it and assign it an invented lifetime even though the origin sent no directives. A transient error during a deploy can therefore outlive the fix by hours. Emit no-store on error responses and purge after risky deploys.

Does a response with no Last-Modified header get cached heuristically?

Sometimes. With no interval to work from, some implementations fall back to a fixed default of a few minutes and others decline to store the response. The outcome depends on which cache is in the path, which makes it unpredictable rather than safe.

How do I stop caches guessing entirely?

Send an explicit lifetime on every response. A server-level default with per-route overrides, plus an edge-level guard for anything that slips through, guarantees the heuristic is never reached.

Guides in This Topic

Each guide below works through one concrete task or question from this topic, end to end.

Back to Core Caching Fundamentals & HTTP Lifecycle