Request Collapsing in Varnish with VCL

When a thousand clients request the same expensive, currently-uncached URL within milliseconds of each other, a naive cache forwards all thousand requests to the backend simultaneously — a thundering herd that can take origin down. Varnish avoids this by default through request collapsing: concurrent misses for the same object are queued and served from a single backend fetch. This page covers how that mechanism actually works in VCL, how to inspect it with varnishlog, and how to deliberately opt specific requests out of it.

Prerequisite Concepts

This page assumes familiarity with:

Step-by-Step Resolution

When coalescing helps and when it hurtsSerialising requests behind one fetch is a large win for cacheable objects and a queue-building liability for uncacheable ones.Cacheable objectcoalescing is a large winOne backend fetch serves every queued clientOrigin load stays flat regardless of demand spikesLatency for queued clients is one fetch, not manyUncacheable objectcoalescing builds a queueEach response is unique, so the queue cannot be sharedRequests serialise behind one another and latency compoundsFix: mark the object uncacheable with a pass in vcl_recv
The same mechanism, opposite outcomes

Step 1 — Confirm the default collapsing behavior

Varnish ships with built-in VCL (builtin.vcl) that your custom vcl_recv, vcl_hash, and vcl_miss subroutines implicitly fall back to via return. The relevant default logic lives in vcl_hash and the backend-fetch bookkeeping around vcl_miss:

sub vcl_hash {
    hash_data(req.url);
    if (req.http.host) {
        hash_data(req.http.host);
    } else {
        hash_data(server.ip);
    }
    return (lookup);
}

hash_data() calls accumulate into a single object hash. When vcl_miss returns fetch, Varnish’s cache manager checks whether another thread is already fetching an object with the identical hash. If so, the new request is not dispatched to the backend — it is parked on that object’s waiting list. When the in-flight fetch completes, every parked request is served from the newly stored (or newly marked pass) object in one pass. This behavior requires no custom VCL at all; it is built into Varnish’s object and busy-object bookkeeping.

Four concurrent requests, one backend fetchVarnish places the first request on the busy object and queues the rest; when the fetch completes all four are served from the single stored copy.ClientsVarnishBackend4 concurrent GETs for one keyfirst request creates a busy objectsingle backend fetch200, object storedall four served from the stored object
Varnish coalescing: one fetch serves the whole queue

Step 2 — Inspect the waiting list with varnishlog

To see collapsing happen live, generate concurrent requests for a currently-uncached URL:

for i in 1 2 3 4 5; do curl -s -o /dev/null https://example.com/expensive-report & done
wait

While that runs, watch varnishlog filtered to the request:

varnishlog -g request -q 'ReqUrl eq "/expensive-report"'

Only one entry in the log shows a BereqURL (a real backend request line) for /expensive-report; the rest show the request being served without their own backend fetch. Cumulative counters confirm the same thing across the whole server:

varnishstat -1 | grep -E 'busy_(sleep|wakeup)'
MAIN.busy_sleep         4          0.00 Number of requests sent waiting due to busy object
MAIN.busy_wakeup        4          0.00 Number of requests woken after waiting for busy object

busy_sleep increments once per request that had to queue on the waiting list; busy_wakeup increments once each is released after the fetch completes. In the five-request burst above, four collapsed onto the fifth’s in-flight fetch.

Step 3 — Check how Vary and custom hashing affect collapsing

Collapsing only happens for requests that hash to the same object. If the backend response carries a broad Vary header, or if vcl_hash includes request data that differs per client, concurrent requests stop colliding:

sub vcl_hash {
    hash_data(req.url);
    hash_data(req.http.host);
    # Dangerous: including the full Cookie header fragments the hash per client
    if (req.http.cookie) {
        hash_data(req.http.cookie);
    }
    return (lookup);
}

With per-client cookies hashed in, five concurrent requests for the same URL produce five distinct hashes — none collapse, and the backend sees five simultaneous fetches, exactly the thundering-herd scenario collapsing is meant to prevent. If personalization genuinely requires per-user cache entries, only hash the specific cookie(s) that matter, not the entire header:

sub vcl_hash {
    hash_data(req.url);
    hash_data(req.http.host);
    if (req.http.cookie ~ "session_region=") {
        hash_data(regsub(req.http.cookie, ".*session_region=([^;]+).*", "\1"));
    }
    return (lookup);
}

This keeps the hash space small (one entry per region rather than one per user), preserving collapsing for concurrent requests that share a region.

Step 4 — Disable collapsing where it is actively harmful

Not every request should collapse. Genuinely uncacheable, per-user, or write-triggering requests should bypass the object cache entirely with return(pass), so Varnish never attempts to build a shared cache entry for them:

sub vcl_recv {
    if (req.url ~ "^/account/" || req.http.authorization) {
        return (pass);
    }
}

A pass request is fetched directly from the backend on every occurrence, with no waiting list and no attempt to store the result. This is the correct choice for authenticated, per-user responses where collapsing would either be impossible (each request differs) or dangerous (serving one user’s personalized response to another).

The subtler case is a response that looks cacheable when the request arrives, but turns out to be uncacheable once the backend responds — for example, the backend sets Set-Cookie or a short-lived error status. Left unhandled, every request queued on the waiting list still ends up with a response that cannot be reused, and the next miss triggers the same expensive backend call again. Varnish solves this with hit-for-pass objects:

sub vcl_backend_response {
    if (beresp.http.Set-Cookie || beresp.status >= 500) {
        set beresp.uncacheable = true;
        set beresp.ttl = 10s;
        return (deliver);
    }
}

Setting beresp.uncacheable = true stores a hit-for-pass object at that hash for the given TTL. Subsequent requests for the same hash see the hit-for-pass marker in vcl_hit and are converted straight to pass behavior — they skip the waiting list entirely instead of queuing behind an object that will never be cacheable.

Expected Output / Verification

  • varnishlog shows exactly one BereqURL per concurrently-requested, currently-uncached object; all other client requests for that same object show no independent backend fetch.
  • varnishstat -1 | grep busy_ shows busy_sleep and busy_wakeup incrementing together under concurrent load, confirming the waiting list is active.
  • After adding Vary: Cookie (or hashing the full cookie in vcl_hash), the same concurrency test produces one BereqURL per request instead of one — collapsing has stopped.
  • After adding the hit-for-pass rule for Set-Cookie responses, a burst of concurrent requests against an endpoint that sets cookies shows each request passed through independently with no busy_sleep increments, confirming the waiting list is being avoided rather than queuing behind an uncacheable fetch.
The VCL that separates the two casesMarking genuinely per-request responses as a pass keeps them out of the busy-object queue while leaving shared objects coalesced.vcl_recvif (req.url ~ "^/api/stream") { return (pass); }if (req.http.Authorization) { return (pass); }return (hash);Streaming responses are unique per request and must never coalesce.Authenticated responses are per-user; coalescing them would cross-serve.Everything else takes the normal path and benefits from coalescing.
Two lines that keep uncacheable traffic out of the queue

Edge Cases

  • Hit-for-pass TTL window — a short hit-for-pass TTL (commonly 2 seconds) means a fresh burst of concurrent misses arriving just after it expires can still momentarily thunder the backend before a new hit-for-pass object is created. Tune the TTL to the shortest value that still spans your typical burst window.
  • Long backend response times amplify queue depth — the waiting list holds requests for the full duration of the in-flight backend fetch. A slow backend (multi-second response time) combined with high concurrency can exhaust Varnish’s worker thread pool even though only one backend connection is open, because parked requests still hold a worker thread each while waiting.
  • Vary interacts with hash_data, not the other way around — Varnish does not automatically incorporate Vary into the request-side hash the way a browser cache does; you must explicitly mirror the relevant Vary dimensions in vcl_hash for correct cache/hash alignment, and every dimension you add is a dimension collapsing has to match on.
  • Collapsing does not apply across separate Varnish nodes — if you run multiple Varnish instances behind a load balancer without a shared cache, each node maintains its own waiting list. A request pattern that appears collapsed within one node’s logs can still generate one backend fetch per node.

Frequently Asked Questions

Does Varnish collapse requests by default without any custom VCL?

Yes. The built-in vcl_miss and vcl_hash logic already implements request coalescing: concurrent requests that hash to the same object queue on a waiting list while one thread fetches from the backend, and all queued requests are served from the resulting object once it lands.

What is a hit-for-pass object?

A hit-for-pass object is a short-lived marker Varnish stores when a backend response turns out to be uncacheable (beresp.uncacheable = true). It tells Varnish to treat subsequent requests for the same hash as pass requests directly, rather than queuing them on a waiting list behind an object that will never be cached.

Vary: Cookie makes the effective cache key depend on the full Cookie header. Since cookies are typically unique per client, almost every request produces a distinct hash, so concurrent requests rarely collapse onto the same waiting list — each effectively becomes its own miss.

Should I use return(pass) or a hit-for-pass object for authenticated endpoints?

Use return(pass) in vcl_recv when you can identify the request as uncacheable up front (an Authorization header, a known account path). Use a hit-for-pass object in vcl_backend_response when you can only tell after the backend responds — for example, when the backend unexpectedly sets Set-Cookie on what looked like a cacheable request.

Back to Origin Shielding and Request Collapsing