Tile Seeding and Cache Warming

A tile cache is a bet that a tile requested once will be requested again. Seeding is the decision to make that bet in advance — rendering tiles before anyone asks — and it is worth making only where demand is predictable enough that the render is not wasted. The arithmetic decides where that line falls, and the arithmetic is unforgiving: the tile pyramid quadruples at every zoom level, so a seed job that is trivial at zoom 10 is impossible at zoom 16.

This page covers estimating a seed job before running it, ordering the work so the first minutes deliver the most value, bounding concurrency so live traffic survives the run, and recovering from a cold cache without melting the render pool.

Prerequisites and Environment Configuration

python==3.11
requests==2.31.0
mercantile==1.2.1
redis==5.0.4
tenacity==8.2.3

Two measurements must exist before any seeding decision can be made, and both come from the running system rather than from documentation:

  • Mean render time and mean tile size, per zoom band. These vary by an order of magnitude between a sparse zoom 8 tile and a dense zoom 16 city tile, so a single average across the pyramid produces estimates that are wrong in both directions. Sample a hundred tiles per band and record both.
  • Historical request counts per tile key. Without them, seeding is ordered by geometry, which correlates poorly with demand. Access logs are usually sufficient; a week is enough to rank tiles even if it is not enough to predict absolute volumes.

The cache layers themselves — edge, origin and their headers — are covered under Tile Cache and Invalidation. Seeding writes into the origin cache; whether the edge is warmed as a side effect depends on whether the seed requests travel through it, which is a decision worth making explicitly.

Conceptual Foundation: The Pyramid Decides What Is Seedable

At zoom level z the world contains 4^z tiles. That single fact governs every seeding decision:

zoom world tiles tiles over a 250 000 km² coverage at 12 KB each
8 65 536 ~300 3.6 MB
10 1 048 576 ~4 800 58 MB
12 16 777 216 ~77 000 924 MB
14 268 435 456 ~1 230 000 14.8 GB
16 4 294 967 296 ~19 700 000 236 GB

Two conclusions follow. Seeding the low zooms completely is free and should simply be done — a few thousand tiles is minutes of work and those tiles are requested by every session. Seeding the high zooms completely is not a question of patience but of whether the storage exists, and for most services it does not.

Between those extremes sits the actual decision, and it is a demand question rather than a geometric one. Request logs from any public tile service show the same shape: a small fraction of the high-zoom key space serves the large majority of high-zoom requests, because readers look at cities, not at moorland. Seeding that fraction captures most of the benefit for a small fraction of the cost.

Demand is concentrated, so exhaustive seeding is mostly waste Cumulative request share plotted against the share of the zoom 14 key space, ordered by popularity. The top 2 per cent of tiles serve 61 per cent of requests, the top 10 per cent serve 84 per cent, and the remaining 90 per cent of the key space serves 16 per cent. A marked seeding cut-off at 10 per cent captures most of the benefit for a tenth of the render cost. 0% 50% 100% of requests share of the z14 key space, most popular first 0% 25% 50% 75% 100% seed the top 10% → 84% of requests the other 90% renders on demand The curve's shape is stable across services; only its steepness varies. Measure it from your own logs rather than assuming a figure, because the cut-off point is what the estimate depends on.
Ordering by demand rather than by geometry is the single decision that makes high-zoom seeding affordable at all.

Step-by-Step Implementation

Step 1: Estimate before running

import math


def estimate_seed(area_km2: float, zooms: range,
                  mean_tile_kb: dict, mean_render_ms: dict,
                  workers: int) -> dict:
    """Tile count, storage and wall-clock for a seed job, per zoom and total."""
    earth_km2 = 510_100_000.0
    rows = []
    for z in zooms:
        world_tiles = 4 ** z
        tiles = max(1, math.ceil(world_tiles * area_km2 / earth_km2))
        gb = tiles * mean_tile_kb[z] / 1_048_576
        hours = tiles * mean_render_ms[z] / 1000 / 3600 / workers
        rows.append({"zoom": z, "tiles": tiles, "storage_gb": round(gb, 2),
                     "hours": round(hours, 2)})

    return {"per_zoom": rows,
            "total_tiles": sum(r["tiles"] for r in rows),
            "total_gb": round(sum(r["storage_gb"] for r in rows), 2),
            "total_hours": round(sum(r["hours"] for r in rows), 2)}

Running this before a seed takes seconds and routinely changes the plan. A job that looked like “seed to zoom 16” becomes “seed to 13 exhaustively and to 16 by demand” once the totals are on screen, which is a decision that is much harder to make three days into a run.

Step 2: Order the queue by demand

def seed_order(candidate_keys, request_counts: dict):
    """Order tiles so the ones readers actually request are rendered first."""
    def rank(key):
        z, _, _ = key
        # Low zooms first regardless of demand: they are cheap and universal.
        return (z, -request_counts.get(key, 0))
    return sorted(candidate_keys, key=rank)

The zoom term dominating the sort is deliberate. A zoom 6 tile costs almost nothing and appears in every session that opens the map; putting it behind a popular zoom 15 tile optimises the wrong thing. Within a zoom band, demand decides.

Step 3: Bound concurrency and add backpressure

import time


class SeedThrottle:
    """Pause seeding when live request latency degrades."""

    def __init__(self, latency_probe, ceiling_ms: float,
                 cooldown_s: float = 30.0):
        self.probe = latency_probe
        self.ceiling = ceiling_ms
        self.cooldown = cooldown_s

    def wait_for_headroom(self) -> None:
        while self.probe() > self.ceiling:
            time.sleep(self.cooldown)

Without this, the seed and live traffic compete for the same workers and the seed wins, because it always has another request ready while live traffic arrives in bursts. A latency ceiling checked between batches, plus a seed concurrency capped well below the pool size, keeps the service responsive throughout. A seed that takes six hours instead of four and never degrades the p99 is the correct outcome.

Step 4: Verify rather than assume

import random


def verify_seed(sample_keys, fetch, sample_size: int = 200) -> dict:
    """Sample the seeded key space and report the observed hit rate."""
    sample = random.sample(list(sample_keys), min(sample_size, len(sample_keys)))
    hits = sum(1 for k in sample if fetch(k).headers.get("X-Cache") == "HIT")
    return {"sampled": len(sample), "hits": hits,
            "hit_rate": round(hits / len(sample), 3)}

A seed job that reports success has established that it issued requests, not that tiles are cached. Storage quotas, eviction policies and cache key mismatches all produce a seed that runs to completion and warms nothing — and a cache key mismatch in particular, where the seed requests a slightly different URL than the client does, warms an entirely separate key space that no reader will ever hit.

What an unbounded seed does to live requests Two timelines over one hour. Without a bound, the seed saturates all eight render workers and live p99 latency rises from 380 milliseconds to 4.2 seconds for the duration. With seed concurrency capped at three workers and a latency ceiling, live p99 stays near 420 milliseconds and the seed finishes about fifty per cent later. unbounded seed seed occupies 8 of 8 workers live: p99 4.2 s finishes in 4 h, degrades the service throughout capped at 3 workers, latency ceiling 600 ms seed 3/8 paused seed 3/8 p99 420 ms Finishes in 6 h with the service intact. The pause is the backpressure loop doing its job.
The seed always wins an unbounded contest because it never runs out of work to submit. The cap is what makes the pool's spare capacity actually spare.

Performance Optimization Patterns

Seed metatiles, not tiles. A render pass that produces a 4 × 4 metatile costs far less than sixteen separate renders, and it gives the label placer a wider view — the reasoning set out under Headless Rendering Engines. Order the seed queue by metatile and slice on write.

Skip tiles that are already fresh. A re-seed after a partial purge should issue conditional requests and skip anything still valid. On an incremental re-seed this routinely removes ninety per cent of the work.

Seed from the origin, not through the edge, unless the edge is the target. Requests through the CDN warm edge caches near the seeding host, which is rarely where the readers are. Decide which layer the seed is for and address it directly.

Parallelise by geography, not by key order. Seeding sequential tile keys concentrates all reads on one small region of the data store. Interleaving distant regions spreads the load across the store’s partitions and typically improves throughput by a third with no extra workers.

A full purge versus a staged one, measured at the render queue Two traces of render queue depth after invalidation. A full purge sends the miss rate to one hundred per cent and the queue grows without bound for eleven minutes before the service recovers. A staged purge across five regions keeps the miss rate near twenty per cent at any moment and the queue never exceeds the pool's drain rate. 0 2k 4k queued renders minutes after the purge 0 5 10 15 pool drain capacity full purge — 11 min above capacity staged, five regions The staged curve never crosses the drain line, so no request ever waits behind a growing queue. Same total work, same total time, and no window in which the service is effectively down.
Both purges do identical work. Only the staged one keeps the instantaneous miss rate inside what the render pool can absorb.

Common Pitfalls and Debugging

The seed reports success and the hit rate does not move. Almost always a cache key mismatch: the seed requested /tiles/{z}/{x}/{y}.png and clients request /tiles/{z}/{x}/{y}.png?v=7. Compare a seed request and a client request header by header.

Storage fills part-way through. The estimate was not run, or was run with a mean tile size sampled only from low zooms. Sample per zoom band; a zoom 16 city tile can be twenty times the size of a zoom 8 tile.

Live latency spikes for the duration of every seed. No concurrency bound. Cap the seed workers and add the latency ceiling.

A purge is followed by an outage. The thundering herd: every request became a render at once. Purge in stages by region, or re-seed the top demand tiles before reopening. This is the failure mode that makes staged invalidation worth the extra complexity.

Seeding never finishes for the top zoom. It was never going to. Re-run the estimate, and move the top zoom to demand-driven rendering.

Frequently Asked Questions

How do I estimate a seed job before committing to it?

Three numbers, all measurable from a small sample. Tile count comes from the coverage area as a fraction of the earth’s surface multiplied by 4^z at each zoom. Storage comes from tile count times the mean tile size for that zoom band. Wall-clock comes from tile count times mean render time divided by the number of workers you intend to give it. Sample a hundred tiles per zoom band to get the two means — they vary by an order of magnitude across the pyramid, so a single global average produces an estimate that is wrong at both ends. The whole exercise takes minutes and it frequently converts “seed everything to zoom 16” into a smaller, defensible plan.

Is it worth seeding every zoom level?

Only the low ones. Zooms up to about 10 are a few thousand tiles over a national coverage, they are requested by every session that opens the map, and seeding them exhaustively costs minutes. Above roughly zoom 13 the key space grows faster than demand does, and the request distribution becomes extremely uneven, so exhaustive seeding spends most of its render budget on tiles nobody will request. Seed low zooms completely, seed high zooms by demand rank, and let the tail render on first request. Where exactly the crossover sits depends on your own request distribution and should be read off the cumulative demand curve rather than assumed.

What happens to live traffic while a seed job runs?

It competes for the same render workers and loses, because the seed always has another request ready while live traffic arrives in bursts. An unbounded seed will saturate the pool and hold live p99 latency an order of magnitude above normal for the entire run. Cap seed concurrency well below the pool size, put live requests on a separate higher-priority queue, and add a backpressure loop that pauses the seed when live latency crosses a ceiling. A seed finishing fifty per cent later with the service intact is almost always the right trade, and it is the trade nobody makes by default.

How do I recover after purging a large cache?

Preferably by never creating the situation. A fully cold cache turns every request into a render, and a pool sized for a one per cent miss rate cannot absorb a hundred per cent one — the result is a queue that grows faster than it drains and a service that is effectively down while it is nominally up. Purge in stages by region so only part of the key space is ever cold, or re-seed the highest-demand tiles before reopening traffic. If a full purge has already happened, throttle inbound requests at the edge while the top of the demand curve is re-seeded, rather than letting readers generate the load.

Conclusion

Seeding is an economic decision dressed as an infrastructure task. The pyramid arithmetic tells you what is affordable, the demand curve tells you what is worth having, and the concurrency bound tells you what the service can spare while the job runs. Estimate first, order by demand, cap the workers, and verify the hit rate rather than the exit code — and the cache warms without anybody noticing it happened, which is the only version of this that counts as success.

Back to Server-Side Map Rendering and Tile Automation