Mapping Vary Headers to Edge Routing

TL;DR: The Vary response header tells each cache layer which request headers must match before a stored variant can be reused. At the CDN edge, every unique combination of Vary-listed values becomes a separate cache entry. Get the dimensions wrong and you either serve the wrong content variant to clients, or fragment the cache into thousands of near-identical entries and push your hit ratio toward zero.

Quick-reference header block for a static asset with compression variants:

Cache-Control: public, s-maxage=86400, max-age=3600
Vary: Accept-Encoding

Mechanism & RFC 9111 Alignment

RFC 9111 Section 4.1 defines the Vary matching algorithm. When a cache receives a request that could be satisfied by a stored response, it inspects the Vary field of that stored response and confirms that the corresponding header values in the incoming request match those of the original request that produced the stored response. If any Vary-listed header differs, the cache must not reuse the entry.

The three-phase evaluation at the edge:

1. Extract — the cache reads the declared header names from the Vary field of the stored response.
2. Normalize — CDNs canonicalize header values to lowercase and collapse whitespace before comparison. Browsers do the same in most implementations, but the normalization algorithms may diverge for complex multi-token values.
3. Variant lookup — the cache constructs a composite key from the URL and the normalized values of every Vary-listed header. On a match with a fresh entry, it serves that variant. On a miss, it fetches upstream.

RFC 9111 also mandates that multiple Vary fields in a single response are merged into a single comma-separated list and deduplicated before key construction. A response emitting two separate Vary headers is treated identically to one emitting a single combined header.

The normative quote from RFC 9111 Section 4.1: “A stored response is selected only if its Vary header field value either is absent or all of the header fields nominated by the Vary header field match.”

Note the evaluation order: variant matching via Vary occurs first. Freshness rules from s-maxage and max-age are checked only after a matching variant is located. An expired variant triggers revalidation, not a new variant lookup.

Scope & Precedence

Vary applies at every caching layer independently — browser, CDN edge, and shield nodes each maintain their own variant stores. The key behavioral differences between layers:

RFC 9111 Section 5.3 is unambiguous on Vary: *: “a cache MUST NOT use a stored response to satisfy a subsequent request for that target resource.” This applies to every compliant cache, CDN or browser. Deploy Vary: * only when every response genuinely differs per request — for example, a server-timing diagnostic endpoint.

Vary dimensions stack multiplicatively. A response with Vary: Accept-Encoding, Accept-Language where Accept-Encoding has two values (gzip, br) and Accept-Language covers 10 locales produces up to 20 distinct cache entries per URL. Adding Vary: Accept for content negotiation on the same route multiplies that further. Audit cardinality before shipping any new Vary dimension.

The Vary header overrides nothing in the Cache-Control directive hierarchy — it operates in a separate namespace. Public vs private cache scope directives still govern whether the CDN is permitted to store the response at all. A response with Cache-Control: private will not be stored regardless of Vary values.

Implementation Patterns

Pattern 1 — Static assets with compression (safe)

Cache-Control: public, s-maxage=86400, max-age=3600, immutable
Vary: Accept-Encoding

This is the most common Vary usage. The CDN caches one entry per encoding type (gzip, br). For Cloudflare with Edge Compression enabled, the CDN normalizes Accept-Encoding automatically and stores a single internal entry, transparently serving each client the appropriate encoding. See Using Vary: Accept-Encoding Without Fragmenting Cache for the full normalization workflow.

Pattern 2 — Locale-aware public pages (limited dimensions)

Cache-Control: public, s-maxage=3600, max-age=600
Vary: Accept-Encoding, Accept-Language

Safe only when Accept-Language is normalized to a small, controlled set (e.g., en, fr, de, es). If clients send raw browser language strings (en-US;q=0.9,en;q=0.8), normalize them at the edge before the cache lookup — otherwise each unique string is a separate variant.

Pattern 3 — Authenticated API responses (browser-only caching)

Cache-Control: private, no-cache
Vary: Cookie, Authorization

private prevents CDN storage. Vary: Cookie and Vary: Authorization describe the variation dimensions for the browser cache. The browser revalidates on every request due to no-cache, so fragmentation is bounded by the browser’s local store. The CDN never stores this response.

Pattern 4 — Content-type negotiation (structured API)

Cache-Control: public, s-maxage=300, max-age=60
Vary: Accept, Accept-Encoding

REST APIs serving both JSON and XML for the same URL must include Vary: Accept so the CDN does not return JSON to a client requesting XML. Keep the set of Accept values that your API actually supports small and documented — browsers and HTTP clients send complex Accept strings that create unbounded cardinality if not normalized.

Pattern 5 — Bypassing Vary for a CDN-managed single store

Cache-Control: public, s-maxage=86400

When the CDN handles all compression and the origin serves uncompressed responses, omit Vary: Accept-Encoding entirely. The CDN stores one entry and serves compressed or uncompressed responses based on client capability without fragmenting the key space.

Server & CDN Configuration

Nginx — normalize Accept-Encoding before cache lookup

# Strip raw Accept-Encoding and replace with a normalized form
# so the upstream cache sees only 'br', 'gzip', or nothing
map $http_accept_encoding $normalized_encoding {
    default          "";
    "~br"            "br";
    "~gzip"          "gzip";
}

server {
    location /assets/ {
        proxy_set_header Accept-Encoding $normalized_encoding;
        proxy_pass http://backend;
        add_header Cache-Control "public, s-maxage=86400, max-age=3600, immutable";
        add_header Vary "Accept-Encoding";
    }
}

Apache — conditional Vary emission

    Header set Cache-Control "public, s-maxage=86400, max-age=3600, immutable"
    Header set Vary "Accept-Encoding"


# Remove Vary for routes managed entirely by CDN compression

    Header unset Vary
    Header set Cache-Control "public, s-maxage=31536000, immutable"

Fastly VCL — Accept-Encoding normalization

Fastly does not automatically normalize Accept-Encoding. Implement normalization in vcl_recv to prevent fragmentation:

sub vcl_recv {
    if (req.http.Accept-Encoding) {
        if (req.http.Accept-Encoding ~ "br") {
            set req.http.Accept-Encoding = "br";
        } else if (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } else {
            unset req.http.Accept-Encoding;
        }
    }
}

This collapses the raw browser-sent string (e.g., br;q=1.0, gzip;q=0.9, *;q=0.1) into a single canonical token before the cache lookup, so all clients that support brotli share a single cache entry.

When deploying origin shielding and request collapsing, apply identical normalization at both edge and shield tiers. If the edge normalizes to br but the shield still sees raw strings, shield-to-edge variant handoffs produce mismatches — the shield stores a variant the edge cannot match.

Interaction with Related Directives

Vary and Cache-Control: no-store — no-store takes absolute precedence. When no-store is present, the cache neither stores the response nor builds a variant index. Vary is irrelevant. Use no-store for session tokens, CSRF state, and PII; see no-cache vs no-store: When to Use Each for the distinction.

Vary and s-maxage — s-maxage sets the shared-cache TTL for each variant independently. Each Vary bucket has its own freshness clock. A gzip variant can expire before the brotli variant if they were populated at different times. Monitor Age headers per encoding to detect skew.

Vary and stale-while-revalidate — stale serving and background refresh apply per variant. If the gzip variant goes stale while the brotli variant is still fresh, the CDN serves the stale gzip variant (and triggers background revalidation) while serving the fresh brotli variant directly. This is correct RFC 9111 behavior and not a bug.

Vary and cache key customization — CDNs let you augment or override the computed cache key independently of Vary via How CDN Cache Keys Are Generated. When you add query-string parameters or custom headers to the key at the CDN layer, those dimensions are additive — they combine with Vary dimensions multiplicatively.

Verification Workflow

Step 1 — Baseline capture

curl -sI https://example.com/asset.js \
  | grep -iE 'vary|cache-control|content-encoding|x-cache|cf-cache-status|age'

Confirm the Vary field in the response matches the value you expect. Note the Age header.

Step 2 — Variant comparison

# Request gzip variant
curl -sI -H "Accept-Encoding: gzip" https://example.com/asset.js \
  | grep -iE 'x-cache|cf-cache-status|content-encoding|age'

# Request brotli variant
curl -sI -H "Accept-Encoding: br" https://example.com/asset.js \
  | grep -iE 'x-cache|cf-cache-status|content-encoding|age'

Both should hit (X-Cache: HIT or CF-Cache-Status: HIT) once the CDN has populated both variants. If the brotli request always returns MISS, the CDN’s Accept-Encoding normalization may not be active, or the origin is not serving brotli.

Step 3 — Vary: * confirmation

# Any response with Vary: * must always MISS
curl -sI https://example.com/dynamic-unique \
  | grep -iE 'vary|x-cache'

Vary: * must produce X-Cache: MISS on every request. A HIT from a cache against a Vary: * response is an RFC 9111 violation.

Step 4 — Normalization audit

# Check what CDN is seeing as the resolved Vary key
# Cloudflare: CF-Cache-Status + CF-Ray correlate variant identity in logs
# Fastly: X-Served-By + X-Cache-Hits confirm per-PoP variant state
curl -sI -H "Accept-Encoding: gzip, deflate, br" https://example.com/asset.js \
  | grep -iE 'cf-cache-status|cf-ray|x-cache-hits|x-served-by'

Step 5 — DevTools cross-check

In Chrome DevTools, open the Network tab and disable local cache. Reload the page and inspect the resource:

• Response Headers > Vary — must list only the dimensions you intend.
• Size column — (disk cache) or (memory cache) indicates browser-layer caching.
• A 304 Not Modified on reload confirms no-cache revalidation is working (not a Vary issue).

Failure Modes & Gotchas

1. Vary: User-Agent in production — User-Agent strings are effectively unbounded. A single URL can accumulate tens of thousands of distinct cache entries within hours. CDN memory limits will evict entries constantly, producing persistent misses. Replace with edge routing rules that map device type to a small set of canonical experience keys before the cache lookup.

2. Vary mismatch between origin and CDN override — If you strip Vary: Accept-Encoding at the CDN level (via a page rule or transform) but the origin still emits it, the origin and CDN have inconsistent views of variant dimensions. Clients bypass the CDN and hit origin expecting separate variants; the CDN serves a single entry to all clients. Align both ends.

3. Shield normalization gap — When using Fastly, Varnish, or a custom reverse proxy as a shield, ensure the VCL/config normalizes Accept-Encoding identically to the edge tier. Mismatched normalization causes shield-to-edge cache misses for every request.

4. Vary stripping by middleware — Some application frameworks, ORMs, and API gateways strip or overwrite response headers. If Vary is absent from CDN-received responses, check whether middleware in the delivery pipeline is removing it before it reaches the edge.

5. Safari/WebKit Vary: Cookie behavior — Safari historically treats Vary: Cookie more aggressively than other browsers, sometimes bypassing its cache even when the cookie value has not changed between requests. For user-personalized but publicly derivable content, prefer Cache-Control: private, no-cache over relying on Vary: Cookie for browser-level cache partitioning.

6. Multi-value Accept-Encoding fragmentation — Raw browser Accept-Encoding values look like br;q=1.0, gzip;q=0.8, deflate;q=0.6. Without normalization, each distinct quality-factor string is a unique cache variant. This is the most common cause of unintentionally high cache entry cardinality for static assets.

7. Vary on non-forwarded headers — If a CDN strips a request header (e.g., Cookie, Authorization) before forwarding to origin, but Vary on that header was already stored, the cache cannot match the incoming request (the header value is unknown). Always verify that every Vary-listed header is actually forwarded to origin.

FAQ

Related

• Using Vary: Accept-Encoding Without Fragmenting Cache walks through the step-by-step fix for the most common Vary fragmentation scenario.
• How CDN Cache Keys Are Generated explains how Vary dimensions combine with URL normalization and query-string rules into the full composite key.
• Tag-Based Cache Invalidation Patterns covers how surrogate keys let you purge all variants of a URL simultaneously without knowing which Vary combinations exist.
• no-cache vs no-store: When to Use Each clarifies when to omit Vary entirely because the response should never be stored.

Back to CDN Architecture & Edge Routing Strategies