Why Large Objects Miss More Often at the Edge

Problem Statement

Site-wide hit ratio reads 95%, and the origin’s egress bill keeps rising. Splitting the metric by response size explains it: small assets hit almost every time, and the handful of objects that account for most of the bytes miss almost every time. Three separate mechanisms produce that shape, and they need different fixes.

Prerequisite Concepts

Step-by-Step Resolution

Step 1 — Split hit ratio by response size

A single site-wide number hides the problem completely, because it is dominated by whichever objects are most numerous:

awk '{
  size = $10; status = $NF;   # adjust field positions to your log format
  b = size < 102400 ? "small" : (size < 10485760 ? "medium" : "large");
  total[b]++; if (status ~ /HIT/) hit[b]++
} END {
  for (b in total) printf "%-7s %6.1f%% hit  (%d requests)\n", b, 100*hit[b]/total[b], total[b]
}' /var/log/nginx/access.log

A healthy result has all three buckets close together. The characteristic failure has small near 99%, medium in the eighties and large in the single digits.

Hit ratio by object size bucketSite-wide hit ratio is dominated by small objects, which hides a large-object hit ratio low enough to account for most origin egress.Under 100 KB99% 92% of requests100 KB to 10 MB84% 7% of requestsOver 10 MB6% 1% of requests, 71% of bytesthe smallest bucket by request count carries most of the egress
The same day, split by response size

Step 2 — Test for a hard size ceiling

A ceiling produces a permanent miss that does not vary with load or time. Request an object twice on each side of the suspected limit:

for url in https://example.com/assets/app.css \
           https://example.com/media/trailer-4k.mp4; do
  echo "== $url"
  for i in 1 2; do
    curl -sI "$url" | grep -iE 'x-cache|cf-cache-status|^age'
  done
done

A second request with Age: 0 and a miss, every time, on the large object only, is a ceiling rather than eviction. Eviction is probabilistic and produces occasional hits; a ceiling never does.

Which of the three mechanisms is activeThe three causes of a low large-object hit ratio produce distinguishable signatures, so a small number of tests identifies which one to fix.Is the miss permanent and load-independent?yesObject-size ceilingnoDoes hit ratio recover during quiet hours?yesEviction pressurenoAre requests arriving with a Range header?yesBlock-level storagenoCheck the directives on the object itself
Three signatures, three different fixes

Step 3 — Raise or work around the ceiling

In Nginx the relevant limit is the temp-file threshold, above which a response is streamed rather than stored:

location /media/ {
    proxy_cache media;
    # Store responses up to 512 MB instead of streaming them through
    proxy_max_temp_file_size 512m;
    proxy_cache_valid 200 206 30d;
}

At a CDN the ceiling is usually a plan attribute rather than a setting. Where it cannot be raised, the workaround is to stop producing objects above it — segmenting media, or splitting a monolithic bundle, converts one uncacheable object into many cacheable ones.

Step 4 — Separate the store

Even below the ceiling, large objects consume the capacity many small ones would occupy, so a single LRU store lets media evict everything else:

varnishd -s main=file,/var/lib/varnish/main.bin,120G \
         -s media=file,/var/lib/varnish/media.bin,400G

sub vcl_backend_response {
    if (beresp.http.Content-Type ~ "^(video|audio)/") {
        set beresp.storage = storage.media;
    }
}

Two stores make both hit ratios predictable and stop a release of large assets from flushing the page cache.

The three mechanisms, their signatures and their fixesEach cause produces a distinct pattern in the data, and applying the wrong fix leaves the hit ratio unchanged.SignatureFixSize ceilingpermanent miss, load-independentraise the limit or split the objectEviction pressurehit ratio recovers when traffic fallsseparate store, or more capacityBlock-level storageranged requests, partial residencyenable range caching or segment
Matching the fix to the mechanism

Step 5 — Enable range caching where media is served whole

If the requests arrive with Range headers, the object is not stored as a unit at all and whole-object hit ratio is the wrong metric. Enabling slicing gives the edge aligned blocks it can reuse, and the measurement should move to bytes served from cache rather than requests hit.

Expected Output / Verification

After the fix, the size buckets converge. The large bucket will usually remain a few points below the others — cold blocks and genuine demand for rarely-watched content are real — but the gap should be single digits rather than an order of magnitude:

small   99.1% hit  (1842019 requests)
medium  96.4% hit  (140228 requests)
large   91.8% hit  (21044 requests)

Confirm with the origin’s own egress figure rather than the CDN dashboard: it is the number the whole exercise exists to reduce.

Edge Cases

  • A ceiling at the shield but not the edge. Different tiers can enforce different limits, producing an object that caches at one and streams through the other. Test both tiers independently.
  • Compression pushing an object over the line. The stored size is the encoded size, so a poorly compressible large file can exceed a ceiling its uncompressed sibling clears.
  • A ceiling expressed in memory rather than bytes on disk. Some caches store small objects in memory and large ones on disk, with separate limits for each; a misconfigured memory tier can reject objects the disk tier would happily hold.
  • A CDN that reports block hits as object hits. Once range caching is enabled, a single object request can register as dozens of block-level hits, which inflates the request hit ratio for exactly the objects that were previously the problem. Read bytes served from cache alongside it, or the fix will look more effective than it is.
  • Objects that are large only occasionally. An endpoint whose response size varies with the query can cross the ceiling for some inputs, producing an intermittent miss that looks like a caching bug rather than a size limit.

Frequently Asked Questions

Is there a standard maximum cacheable object size?

No. Every implementation sets its own, and the defaults are frequently far below modern asset sizes. Nginx streams responses past a buffering threshold instead of storing them, and most CDNs impose a plan-level ceiling that is worth checking against your largest files.

Why does a large object miss even when the store has free space?

Because the ceiling is a per-object rule, not a capacity rule. A response above the limit is streamed through on every request regardless of how empty the store is, which produces a permanent miss that looks nothing like eviction.

Does splitting a large file into parts help?

Substantially. Each part becomes an ordinary object well under any ceiling, and partial demand only warms the parts anyone actually requests. This is the reasoning behind segmented media formats.

Should large objects have their own cache store?

Where the cache supports it, yes. A shared store lets a handful of large objects evict thousands of small ones, and separating them makes both hit ratios predictable.

Back to Cache Storage, Eviction and Capacity