Request Collapsing in Varnish with VCL

When a thousand clients request the same expensive, currently-uncached URL within milliseconds of each other, a naive cache forwards all thousand requests to the backend simultaneously — a thundering herd that can take origin down. Varnish avoids this by default through request collapsing: concurrent misses for the same object are queued and served from a single backend fetch. This page covers how that mechanism actually works in VCL, how to inspect it with varnishlog, and how to deliberately opt specific requests out of it.

Prerequisite Concepts

This page assumes familiarity with:

Step-by-Step Resolution

Step 1 — Confirm the default collapsing behavior

Varnish ships with built-in VCL (builtin.vcl) that your custom vcl_recv, vcl_hash, and vcl_miss subroutines implicitly fall back to via return. The relevant default logic lives in vcl_hash and the backend-fetch bookkeeping around vcl_miss:

sub vcl_hash {
    hash_data(req.url);
    if (req.http.host) {
        hash_data(req.http.host);
    } else {
        hash_data(server.ip);
    }
    return (lookup);
}

hash_data() calls accumulate into a single object hash. When vcl_miss returns fetch, Varnish’s cache manager checks whether another thread is already fetching an object with the identical hash. If so, the new request is not dispatched to the backend — it is parked on that object’s waiting list. When the in-flight fetch completes, every parked request is served from the newly stored (or newly marked pass) object in one pass. This behavior requires no custom VCL at all; it is built into Varnish’s object and busy-object bookkeeping.

Varnish Request Collapsing Sequence Three clients request the same URL concurrently. Client A's request triggers a backend fetch; Client B and C are parked on the waiting list and served from the same fetched object once it completes. Client A Client B Client C Backend GET /report (miss, fetch dispatched) waiting list waiting list 200 OK, object stored B and C served from the same stored object

Step 2 — Inspect the waiting list with varnishlog

To see collapsing happen live, generate concurrent requests for a currently-uncached URL:

for i in 1 2 3 4 5; do curl -s -o /dev/null https://example.com/expensive-report & done
wait

While that runs, watch varnishlog filtered to the request:

varnishlog -g request -q 'ReqUrl eq "/expensive-report"'

Only one entry in the log shows a BereqURL (a real backend request line) for /expensive-report; the rest show the request being served without their own backend fetch. Cumulative counters confirm the same thing across the whole server:

varnishstat -1 | grep -E 'busy_(sleep|wakeup)'
MAIN.busy_sleep         4          0.00 Number of requests sent waiting due to busy object
MAIN.busy_wakeup        4          0.00 Number of requests woken after waiting for busy object

busy_sleep increments once per request that had to queue on the waiting list; busy_wakeup increments once each is released after the fetch completes. In the five-request burst above, four collapsed onto the fifth’s in-flight fetch.

Step 3 — Check how Vary and custom hashing affect collapsing

Collapsing only happens for requests that hash to the same object. If the backend response carries a broad Vary header, or if vcl_hash includes request data that differs per client, concurrent requests stop colliding:

sub vcl_hash {
    hash_data(req.url);
    hash_data(req.http.host);
    # Dangerous: including the full Cookie header fragments the hash per client
    if (req.http.cookie) {
        hash_data(req.http.cookie);
    }
    return (lookup);
}

With per-client cookies hashed in, five concurrent requests for the same URL produce five distinct hashes — none collapse, and the backend sees five simultaneous fetches, exactly the thundering-herd scenario collapsing is meant to prevent. If personalization genuinely requires per-user cache entries, only hash the specific cookie(s) that matter, not the entire header:

sub vcl_hash {
    hash_data(req.url);
    hash_data(req.http.host);
    if (req.http.cookie ~ "session_region=") {
        hash_data(regsub(req.http.cookie, ".*session_region=([^;]+).*", "\1"));
    }
    return (lookup);
}

This keeps the hash space small (one entry per region rather than one per user), preserving collapsing for concurrent requests that share a region.

Step 4 — Disable collapsing where it is actively harmful

Not every request should collapse. Genuinely uncacheable, per-user, or write-triggering requests should bypass the object cache entirely with return(pass), so Varnish never attempts to build a shared cache entry for them:

sub vcl_recv {
    if (req.url ~ "^/account/" || req.http.authorization) {
        return (pass);
    }
}

A pass request is fetched directly from the backend on every occurrence, with no waiting list and no attempt to store the result. This is the correct choice for authenticated, per-user responses where collapsing would either be impossible (each request differs) or dangerous (serving one user’s personalized response to another).

The subtler case is a response that looks cacheable when the request arrives, but turns out to be uncacheable once the backend responds — for example, the backend sets Set-Cookie or a short-lived error status. Left unhandled, every request queued on the waiting list still ends up with a response that cannot be reused, and the next miss triggers the same expensive backend call again. Varnish solves this with hit-for-pass objects:

sub vcl_backend_response {
    if (beresp.http.Set-Cookie || beresp.status >= 500) {
        set beresp.uncacheable = true;
        set beresp.ttl = 10s;
        return (deliver);
    }
}

Setting beresp.uncacheable = true stores a hit-for-pass object at that hash for the given TTL. Subsequent requests for the same hash see the hit-for-pass marker in vcl_hit and are converted straight to pass behavior — they skip the waiting list entirely instead of queuing behind an object that will never be cacheable.

Expected Output / Verification

  • varnishlog shows exactly one BereqURL per concurrently-requested, currently-uncached object; all other client requests for that same object show no independent backend fetch.
  • varnishstat -1 | grep busy_ shows busy_sleep and busy_wakeup incrementing together under concurrent load, confirming the waiting list is active.
  • After adding Vary: Cookie (or hashing the full cookie in vcl_hash), the same concurrency test produces one BereqURL per request instead of one — collapsing has stopped.
  • After adding the hit-for-pass rule for Set-Cookie responses, a burst of concurrent requests against an endpoint that sets cookies shows each request passed through independently with no busy_sleep increments, confirming the waiting list is being avoided rather than queuing behind an uncacheable fetch.

Edge Cases

  • Hit-for-pass TTL window — a short hit-for-pass TTL (commonly 2 seconds) means a fresh burst of concurrent misses arriving just after it expires can still momentarily thunder the backend before a new hit-for-pass object is created. Tune the TTL to the shortest value that still spans your typical burst window.
  • Long backend response times amplify queue depth — the waiting list holds requests for the full duration of the in-flight backend fetch. A slow backend (multi-second response time) combined with high concurrency can exhaust Varnish’s worker thread pool even though only one backend connection is open, because parked requests still hold a worker thread each while waiting.
  • Vary interacts with hash_data, not the other way around — Varnish does not automatically incorporate Vary into the request-side hash the way a browser cache does; you must explicitly mirror the relevant Vary dimensions in vcl_hash for correct cache/hash alignment, and every dimension you add is a dimension collapsing has to match on.
  • Collapsing does not apply across separate Varnish nodes — if you run multiple Varnish instances behind a load balancer without a shared cache, each node maintains its own waiting list. A request pattern that appears collapsed within one node’s logs can still generate one backend fetch per node.

Frequently Asked Questions

Does Varnish collapse requests by default without any custom VCL?

Yes. The built-in vcl_miss and vcl_hash logic already implements request coalescing: concurrent requests that hash to the same object queue on a waiting list while one thread fetches from the backend, and all queued requests are served from the resulting object once it lands.

What is a hit-for-pass object?

A hit-for-pass object is a short-lived marker Varnish stores when a backend response turns out to be uncacheable (beresp.uncacheable = true). It tells Varnish to treat subsequent requests for the same hash as pass requests directly, rather than queuing them on a waiting list behind an object that will never be cached.

Vary: Cookie makes the effective cache key depend on the full Cookie header. Since cookies are typically unique per client, almost every request produces a distinct hash, so concurrent requests rarely collapse onto the same waiting list — each effectively becomes its own miss.

Should I use return(pass) or a hit-for-pass object for authenticated endpoints?

Use return(pass) in vcl_recv when you can identify the request as uncacheable up front (an Authorization header, a known account path). Use a hit-for-pass object in vcl_backend_response when you can only tell after the backend responds — for example, when the backend unexpectedly sets Set-Cookie on what looked like a cacheable request.

Back to Origin Shielding and Request Collapsing