Recovering a Cold Tile Cache Without Melting the Render Pool

A fully cold cache turns every request into a render, and a pool sized for a one per cent miss rate cannot absorb a hundred per cent one — so the recovery is about keeping the instantaneous miss rate inside what the renderer can drain, not about rendering faster.

Core Diagnostic and Recovery Workflow

The failure mode is a positive feedback loop, not a slowdown. Requests arrive faster than they can be served, the queue grows, latency rises past the client timeout, clients retry, and the retries add load to a queue that is already unbounded. Nothing in that loop self-corrects.

Four levers, in the order they should be reached for:

  1. Do not create it. Purge in stages by region so only part of the key space is cold at any moment. This is much cheaper than any recovery and it is available every time an invalidation is planned rather than forced.
  2. Coalesce duplicate misses. Concurrent requests for the same absent tile should trigger one render. On a popular tile this converts hundreds of renders into one.
  3. Throttle or shed at the edge. A request that will time out anyway should not occupy a worker. Serving a stale tile is better still where the cache can.
  4. Re-seed the top of the demand curve before restoring full traffic, using the ranking described in Seeding Tile Caches by Traffic-Weighted Priority.
Queue depth is the signal that crosses the line first Render queue depth over fifteen minutes after a full purge. Without mitigation the queue grows past the pool's drain capacity within ninety seconds and continues to climb for eleven minutes. With coalescing alone it stays near the line. With coalescing and edge throttling it remains comfortably below. A note records that response latency only becomes abnormal several minutes after the queue crosses the line. 0 2k 4k queued renders minutes after the purge 0 5 10 15 pool drain capacity no mitigation coalescing only coalescing + throttle Latency looks normal for the first two minutes of the top curve, by which point recovery is already several minutes away — which is why the alert should watch queue depth.
All three curves start identically. The one that matters diverges within ninety seconds, and by the time response times reflect it the backlog is already the problem.

Production-Ready Python Implementation

import threading


class CoalescingRenderer:
    """Ensure concurrent misses for the same tile trigger exactly one render."""

    def __init__(self, render_fn):
        self._render = render_fn
        self._lock = threading.Lock()
        self._inflight: dict = {}

    def get(self, key):
        with self._lock:
            event = self._inflight.get(key)
            if event is None:
                event = self._inflight[key] = {"done": threading.Event(),
                                               "value": None, "error": None}
                leader = True
            else:
                leader = False

        if not leader:
            event["done"].wait()
            if event["error"]:
                raise event["error"]
            return event["value"]

        try:
            event["value"] = self._render(key)
        except Exception as exc:                       # noqa: BLE001 — re-raised below
            event["error"] = exc
        finally:
            event["done"].set()
            with self._lock:
                self._inflight.pop(key, None)

        if event["error"]:
            raise event["error"]
        return event["value"]


def staged_purge_plan(regions, tiles_per_region: dict,
                      drain_rate_per_s: float, headroom: float = 0.5) -> list:
    """Order regions and give each a re-warm window the pool can actually absorb."""
    if not 0 < headroom <= 1:
        raise ValueError("headroom must be a fraction of drain capacity")

    usable = drain_rate_per_s * headroom
    plan = []
    for region in regions:
        n = tiles_per_region[region]
        plan.append({"region": region, "tiles": n,
                     "wait_seconds": round(n / usable, 1)})
    return plan

The coalescing renderer’s value is entirely in the duplicate case, and the duplicate case is what a cold cache produces most of. When a popular tile is missing, every viewer looking at that area requests it within the same second; without coalescing each request renders it independently.

staged_purge_plan deliberately leaves headroom. Purging a region and immediately purging the next, on the assumption that the pool will keep up, is how a staged purge becomes a full one.

Header-Inspection and Verification Best Practices

  • Alert on queue depth, not latency. The queue crosses the danger line minutes before response times look abnormal, and by then the recovery is longer.
  • Serve stale on miss where the data allows. A tile from before the purge is a far better answer than a timeout, and for most map data a few minutes of staleness is invisible.
  • Cap the retry storm. Client timeouts produce retries that add load to an overloaded queue. Return a fast error above a queue-depth threshold so retries arrive against a shedding server rather than a stalled one.
  • Verify the purge actually propagated. A staged purge whose second stage begins before the first has taken effect at every edge is a full purge with extra steps — the verification loop described in Debugging Stale Tiles in CDN-Backed Map Servers applies here too.
  • Rehearse it. A staged purge that has never been run will be run for the first time during an incident. Exercise it on a quiet region during a quiet hour.
What coalescing removes Renders triggered for one popular tile during a five-second burst after a purge. Without coalescing, 214 concurrent requests each trigger an independent render of the same tile. With coalescing, one render occurs and the other 213 requests wait on its result. A note records that the mechanism is a lock and a map of in-flight keys. without coalescing — one tile, five seconds … 214 renders of one tile with coalescing 1 render, 213 waiters The mechanism is a lock and a dict of in-flight keys — a dozen lines against a 200× reduction. Duplication is worst exactly when load is worst, because a cold cache and a busy moment are the same event seen from two sides.
Coalescing does nothing on a warm cache and everything on a cold one, which is why its absence is never noticed until the moment it is needed.

Integration and Next Steps

Everything here is downstream of an invalidation decision, so the cheapest improvement is upstream: versioned tile URLs, as described in Tile Cache and Invalidation, remove purging from the vocabulary entirely — a style change publishes a new key space and the old one ages out. Where purging is unavoidable, the seed estimate from Estimating Tile Seed Time and Storage Before You Run It gives the per-stage window, and the demand ranking decides which region to purge when.

Three ways to change what tiles show, ranked by what they cost afterwards Three strategies. Versioned URLs create no cold cache at all because the new key space fills gradually while the old one still serves. Staged regional purging keeps the instantaneous miss rate inside the pool's capacity. A full purge creates a hundred per cent miss rate and requires every mitigation to survive. versioned URLs — no cold cache exists the new key space fills gradually while the old one still serves staged regional purge — miss rate stays inside capacity each stage waits for the previous region to re-warm full purge — 100% miss rate, every mitigation required coalescing, throttling, stale-serving and a re-seed, all at once The top row is a design decision made months earlier; the bottom row is an incident.
Recovery technique is what remains when the invalidation strategy has already been chosen. Choosing it earlier is worth more than any of the mitigations below it.

Frequently Asked Questions

What actually breaks when a cache goes cold?

The render queue. A pool sized for a one per cent miss rate suddenly faces a hundredfold increase in work, so the queue grows faster than it drains and every request waits behind an unbounded backlog. Response times do not degrade gracefully — they degrade until clients time out, and the timeouts produce retries that add further load. It is a positive feedback loop rather than a slowdown, which is why the mitigations are all about limiting inbound work rather than speeding up rendering.

Why does request coalescing help so much?

Because a cold cache produces enormous duplication. When a popular tile is missing, every viewer looking at that area requests it within the same second, and without coalescing each of those requests renders the same tile independently. With coalescing the first request renders and the rest wait on its result. On a busy tile at a bad moment this is the difference between one render and several hundred, and the implementation is a lock and a dictionary of in-flight keys.

Is it better to shed requests or to queue them?

Shed, once the queue passes a threshold. A queued request that will time out anyway has occupied a worker slot for nothing and produced no value, while a request rejected quickly lets the client retry when capacity exists. Better still, serve a stale tile where the cache retains one: for most map data a few minutes of staleness is imperceptible, and it is unambiguously a better answer than a timeout.

How long should a staged purge take?

Long enough that each stage’s miss rate stays inside the pool’s drain capacity — which the seed estimate already quantifies. If a region holds 200 000 tiles and the pool renders 40 per second within the headroom you are willing to give live traffic, that region needs roughly 80 minutes before the next stage begins. Purging faster than the pool can refill simply reconstructs the full-purge scenario one region at a time, with none of the benefit and all of the load.

Back to Tile Seeding and Cache Warming