Handling Outliers and Skew in Choropleth Class Breaks

Give a genuine outlier its own class rather than clipping it away or letting it stretch every other break, because a single extreme value under equal-interval or standard-deviation classification puts every remaining unit in the bottom class and produces a map with one coloured region and a hundred beige ones.

Core Algorithm and Workflow

Skew is measurable before any scheme is chosen, and the measurement decides the approach.

Diagnose first. Compare the mean to the median — a mean far above the median indicates a right tail — and compare the maximum to the 99th percentile. A maximum several times the 99th percentile is a single extreme value; a maximum close to it is a heavy tail with no individual culprit.

One extreme value: give it a class. A capital city holding a third of a country’s population is a fact about that city. A dedicated top class names it and frees the remaining classes to describe everywhere else.

A heavy tail with no single culprit: transform. Log or rank transforms suit quantities that are multiplicative by nature, where the interesting comparison is ratio rather than difference.

Neither: use a robust scheme. Quantile or natural breaks both handle moderate skew without additional machinery, as compared in Choosing Jenks vs Quantile Classification for Choropleths.

What one extreme value does to each scheme Three classifications of one distribution with a single extreme value. Equal interval puts 118 of 120 units in the bottom class. Quantile spreads units evenly but places the outlier in a class with 23 ordinary units, so it reads as typical. A dedicated outlier class holds the single extreme value alone and leaves four classes to describe the remaining 119 units. equal interval 118 units one map, one colour quantile outlier shares its class with 23 ordinary units dedicated outlier class the extreme value, named Only the third arrangement lets the map describe the outlier and everywhere else at once.
Equal interval loses everything except the outlier; quantile loses the outlier among its neighbours. A dedicated class is the only one that reports both.

Production-Ready Python Implementation

import numpy as np


def diagnose_skew(values: np.ndarray) -> dict:
    """Describe a distribution well enough to choose a classification approach."""
    v = np.asarray(values, dtype="float64")
    v = v[np.isfinite(v)]
    if v.size < 5:
        raise ValueError("too few finite values to classify meaningfully")

    median = float(np.median(v))
    mean = float(v.mean())
    p99 = float(np.percentile(v, 99))
    top = float(v.max())

    single_extreme = p99 > 0 and top / p99 > 2.0
    heavy_tail = median > 0 and mean / median > 1.5 and not single_extreme

    if single_extreme:
        approach = "dedicated outlier class"
    elif heavy_tail:
        approach = "transform (log or rank)"
    else:
        approach = "quantile or natural breaks"

    return {"mean_over_median": round(mean / median, 2) if median else None,
            "top_over_p99": round(top / p99, 2) if p99 else None,
            "single_extreme": single_extreme, "heavy_tail": heavy_tail,
            "approach": approach}


def breaks_with_outlier_class(values, k: int = 4, classifier=None) -> dict:
    """Classify the bulk of a distribution and hold extremes in their own class."""
    v = np.asarray(values, dtype="float64")
    v = v[np.isfinite(v)]
    cut = float(np.percentile(v, 99))
    bulk = v[v <= cut]
    if bulk.size < k:
        raise ValueError("not enough non-extreme values for the requested classes")

    if classifier is None:                      # even-count fallback
        edges = list(np.percentile(bulk, np.linspace(0, 100, k + 1)))
    else:
        edges = list(classifier(bulk, k))

    return {"breaks": [round(e, 4) for e in edges],
            "outlier_floor": round(cut, 4),
            "outliers": sorted(round(float(x), 4) for x in v[v > cut]),
            "legend_top_label": f"{cut:,.0f} and above"}

legend_top_label is returned rather than left to the caller because the open-ended label is the part that makes the arrangement honest. A dedicated top class whose legend reads like any other class tells the reader that the top band is bounded when it is not.

Performance Tuning and Cartographic Best Practices

  • Diagnose before choosing. Two ratios — mean over median, and maximum over the 99th percentile — separate the three cases and cost nothing.
  • Label in original units. A log transform should move where the breaks fall, not what the legend says. A reader should see “32 to 100”, never “1.5 to 2.0”.
  • Handle zero deliberately. Adding a constant before a log compresses the low end by an amount that depends on the constant. A square-root transform handles zero cleanly; genuinely signed data belongs on a diverging scheme with a pinned midpoint instead, as covered in Diverging Colormap Selection for Bivariate Thematic Maps.
  • Fix breaks across a series. A distribution reclassified per frame or per sheet produces change that is not in the data, which is the argument made in Data-Driven Classification.
  • Record the decision. Store the diagnosis alongside the breaks so a later reader knows why the top class is open-ended rather than assuming it was an oversight.
The transform belongs in the breaks, not in the legend Two legends for the same log-classified variable. The first labels classes with transformed values from 1.0 to 3.2, which a reader cannot map onto anything. The second labels the same classes with original units from 10 to 1 600, with an open-ended top class, and is directly usable. transformed units — unusable 1.0 – 1.5 1.5 – 2.0 2.0 – 2.6 2.6 – 3.2 the reader has to invert a logarithm original units — usable 10 – 32 32 – 100 100 – 400 400 – 1 600 1 600 and above open-ended top class stated explicitly Both legends describe identical breaks. Only one of them describes them to the reader.
The uneven class widths in the right-hand legend are the visible trace of the transform, which is exactly where a reader can interpret them.

Integration and Next Steps

The diagnosis and the resulting breaks belong in the frozen decision artefact for any series, so every sheet or frame classifies identically — the argument set out in Atlas and Map Series Automation. The open-ended top class also constrains the legend generator, which must render an unbounded band differently from a bounded one; see Dynamic Legend Generation for the handle-and-label contract that makes that possible.

Two ratios decide the approach A decision path. If the maximum divided by the 99th percentile exceeds two, there is a single extreme value and it gets its own class. Otherwise, if the mean divided by the median exceeds 1.5, the distribution has a heavy tail and a transform is appropriate. Otherwise quantile or natural breaks suffice without extra machinery. max ÷ p99 > 2 → dedicated outlier class one nameable entity dominates; name it rather than hiding it mean ÷ median > 1.5 → transform the values a heavy tail with no single culprit; ratios are the real comparison otherwise → quantile or natural breaks moderate skew needs no special handling Both ratios cost one pass over the values, and together they replace an argument with a test.
The two ratios are cheap and they separate the cases cleanly, which turns a recurring judgement call into a rule that can be applied per dataset without discussion.

Frequently Asked Questions

When is an outlier class better than a transform?

When the outlier is a real, nameable entity rather than a general property of the distribution. A capital city holding a third of a country’s population is a fact about that city, and a dedicated top class states it while leaving the other classes free to describe everywhere else. A transform suits a distribution that is multiplicative throughout — income, settlement size, river discharge — where no individual value is the story and the interesting comparison is ratio rather than difference.

Is winsorising ever acceptable?

Only when the legend says so. Clipping the top of a distribution changes what the map asserts: a region with ten times the next value renders identically to one with twice, and the reader has no way to tell. If the visual is genuinely unusable otherwise, clip and label the top class as open-ended — “1 600 and above” — so the ceiling is visible. Silent clipping is a misrepresentation rather than a styling decision.

Does a log transform make the legend unreadable?

It does if the labels show transformed values. Keep the labels in the original units and let the transform affect only where the breaks fall, so the reader sees “32 to 100” rather than “1.5 to 2.0”. The uneven class widths that result are the visible trace of the transform, and they are interpretable in a way that a logarithm in the legend is not.

What about zero and negative values under a log transform?

They break it, and the chosen workaround changes the map. Adding a constant before taking the log compresses the low end by an amount that depends entirely on the constant, which makes the map’s appearance a function of an arbitrary parameter. A square-root transform handles zero cleanly and is gentler overall. Data that genuinely spans negative and positive is a diverging variable and belongs on a diverging scheme with a pinned midpoint, not on a sequential one with a transform.

Back to Data-Driven Classification