Hampel Filter vs Grubbs' Test for SPC Outliers

Two outlier tests dominate manufacturing telemetry, and picking the wrong one quietly corrupts a control chart: the rolling Hampel filter, a robust median-and-MAD detector built for streaming data, and Grubbs' test, a parametric procedure that flags one extreme value in a small, normal batch. This guide implements both in Python, shows why each masks a real process shift differently, and gives a decision rule for choosing between them. It is a detection technique inside the outlier detection and filtering pipeline, and it assumes the raw stream is already schema-clean.

The stakes are the same for both methods. A single point sitting far from the recent mean is either measurement-system error to suppress or the first sample of an assignable cause the chart exists to catch. A test that cannot tell those apart will delete the leading edge of a tool-wear drift and report a capability the process never had.

Prerequisites

Confirm these before running either detector:

  • Python 3.10+ with pandas >= 2.0, numpy >= 1.24, and scipy >= 1.10 installed (pip install "pandas>=2.0" "numpy>=1.24" "scipy>=1.10")
  • A single measurement channel as a pd.Series with a monotonic index — sort and de-duplicate timestamps upstream via the time-series alignment pipeline first
  • Sentinel values (-999, -9999, full-scale rails) already mapped to NaN by the batch data validation gate, so neither test sees a physically impossible number
  • For Grubbs' test only: a defensible reason to believe the batch is a single stationary population that is approximately normal — Grubbs assumes both, and a real shift violates both
  • A rough expected process cycle length in samples, so the Hampel window spans about one cycle without straddling a rational-subgroup boundary
  • The intended chart type known in advance — a lone outlier moves both the individual and the moving range on an I-MR chart, so its tolerance differs from a subgrouped chart

Step 1 — Understand what each test actually estimates

Both tests measure how far a point sits from a center, scaled by a dispersion estimate; they differ in whether those estimates are robust and whether they are local or global.

Grubbs' test compares the most extreme point to the batch mean and standard deviation. Its statistic is

$$G = \frac{\max_i \lvert x_i - \bar{x} \rvert}{s},$$

where $\bar{x}$ is the sample mean and $s$ the sample standard deviation. Under a null of one normal population, $G$ has a known critical value, so Grubbs delivers a real significance level $\alpha$. The weakness is baked into the formula: $\bar{x}$ and $s$ are both non-robust. A single gross error inflates $s$ enough to hide itself (masking), and a second outlier can drag $\bar{x}$ so the first no longer looks extreme (swamping).

The Hampel filter replaces the mean with a rolling median and the standard deviation with the median absolute deviation over a window $w$:

$$\text{MAD} = \operatorname{median}\bigl(\lvert x_i - \tilde{x} \rvert\bigr), \qquad \hat{\sigma} = 1.4826 \cdot \text{MAD},$$

where $\tilde{x}$ is the window median. A point is flagged when $\lvert x_i - \tilde{x} \rvert > k\,\hat{\sigma}$. The median and MAD have a 50% breakdown point, so up to half the window can be contaminated before the estimate moves — which is exactly why a lone spike cannot inflate the fence around itself.

Step 2 — Implement Grubbs' test for a small stationary batch

Use Grubbs when a discrete batch — a rational subgroup, a coupon set, a lab replicate run — is small, believed normal, and suspected of holding exactly one gross error. The function below computes $G$, compares it to the two-sided critical value derived from the Student t distribution, and returns the offending index only if the null is rejected.

import numpy as np
import pandas as pd
from scipy import stats


def grubbs_test(values: pd.Series, alpha: float = 0.05) -> dict:
    """Two-sided Grubbs' test for a single outlier in a small normal batch.

    Parameters
    ----------
    values : pd.Series
        One stationary batch; assumed approximately normal.
    alpha : float
        Family-wise significance level for the single most extreme point.

    Returns
    -------
    dict
        G statistic, critical value, the flagged index (or None), and the
        decision. `is_outlier` is True only when G exceeds the critical value.
    """
    x = values.dropna().astype(float)
    n = x.size
    if n < 3:
        raise ValueError("Grubbs' test needs at least 3 observations")

    s = x.std(ddof=1)
    if s == 0:
        return {"G": 0.0, "critical": np.inf, "index": None, "is_outlier": False}

    abs_dev = (x - x.mean()).abs()
    g_stat = abs_dev.max() / s

    # Two-sided critical value from the t distribution (Grubbs 1969).
    t_crit = stats.t.ppf(1 - alpha / (2 * n), df=n - 2)
    g_crit = ((n - 1) / np.sqrt(n)) * np.sqrt(t_crit**2 / (n - 2 + t_crit**2))

    flagged = abs_dev.idxmax() if g_stat > g_crit else None
    return {
        "G": float(g_stat),
        "critical": float(g_crit),
        "index": flagged,
        "is_outlier": g_stat > g_crit,
    }

Grubbs flags at most one point per call. If a batch may hold several outliers, iterate: remove the flagged value and rerun, but stop the moment the test fails to reject — repeated removal on a batch that actually shifted will strip real points one by one. This iteration is the danger zone the decision guide in Step 4 exists to prevent.

Step 3 — Implement the rolling Hampel filter for a stream

Use the Hampel filter on a continuous single-channel stream where outliers arrive sporadically and the process may legitimately drift. Because it re-estimates center and scale in every window, it adapts to slow tool wear instead of flagging it, and it never needs the whole series in memory.

def hampel_filter(
    series: pd.Series,
    window: int = 15,
    k: float = 3.0,
    min_periods: int = 5,
) -> pd.Series:
    """Rolling Hampel outlier mask over a single measurement channel.

    Parameters
    ----------
    series : pd.Series
        Measurement channel with a monotonic index.
    window : int
        Rolling window in samples; set to roughly one process cycle.
    k : float
        Threshold in robust sigmas. 3.0 is the general default.
    min_periods : int
        Minimum observations before the window is trusted.

    Returns
    -------
    pd.Series
        Boolean mask aligned to `series.index`; True marks an outlier.
    """
    if window < 3:
        raise ValueError("window must be >= 3 for a stable rolling median")

    med = series.rolling(window, center=True, min_periods=min_periods).median()
    abs_dev = (series - med).abs()
    mad = abs_dev.rolling(window, center=True, min_periods=min_periods).median()
    sigma = 1.4826 * mad

    # A flat window has zero MAD; treat that as an infinite band so a
    # legitimately constant segment is never split into infinite z-scores.
    band = k * sigma
    is_outlier = abs_dev > band.where(band > 0, np.inf)
    return is_outlier.reindex(series.index).fillna(False)

Centered windows give symmetric context but need future samples, so a strictly real-time deployment must switch center=False and accept a half-window detection lag. Either way, mark flagged points — never delete them — so the sequential integrity that Nelson and Western Electric run rules depend on survives.

Step 4 — Apply the decision guide

The two tests answer different questions, and the choice follows the data shape, not preference.

Situation Use Why
Small discrete batch, believed normal, at most one gross error Grubbs' test Gives a real significance level $\alpha$ on a single most-extreme point
Continuous stream, sporadic spikes, legitimate slow drift Hampel filter Robust, local, streaming-friendly; drift survives, spikes do not
Several suspected outliers in one batch Hampel (or iterated Grubbs with care) A single non-robust $s$ is masked/swamped by multiple outliers
A suspected sustained shift, not an outlier Neither — investigate An assignable cause belongs on the chart, not in a filter

The last row is the one that matters most. An outlier is a single-sample defect in the measurement path; a shift is a change in the process itself, and it is precisely what an I-MR chart is built to reveal. Grubbs, run iteratively, will happily delete the first several points of a step change as "outliers" because each looks extreme against the pre-shift mean. The Hampel filter degrades more gracefully — its rolling median tracks the new level within a window — but a threshold set too tight still nibbles the leading edge. When the deviation persists, stop filtering and route it to rule detection. For the change-point guard that lets a filter keep spikes while preserving shifts, see filtering measurement outliers without masking real shifts.

Verification

Confirm the two behaviors that separate a safe detector from a dangerous one: the Hampel filter should quarantine a lone spike but preserve a sustained step, while iterated Grubbs on the same shifted series should over-flag. No live data is required.

import numpy as np
import pandas as pd

rng = np.random.default_rng(0)
idx = pd.date_range("2026-07-01", periods=200, freq="s")
signal = pd.Series(rng.normal(50.0, 0.4, size=200), index=idx)

signal.iloc[80] += 8.0          # transient spike (should be flagged)
signal.iloc[130:] += 3.0        # sustained step shift (must be preserved)

mask = hampel_filter(signal, window=15, k=3.0)

assert bool(mask.iloc[80]), "spike should be flagged by Hampel"
kept = 1.0 - mask.iloc[135:175].mean()
assert kept > 0.9, f"real shift was masked (only {kept:.0%} kept)"

# Grubbs on the whole shifted batch: the shift makes a shoulder point look
# 'most extreme', so a naive single call points at the transition, not noise.
result = grubbs_test(signal)
print("hampel ok; grubbs flagged index:", result["index"])

Expected output resembles hampel ok; grubbs flagged index: 2026-07-01 00:02:10, and the assertions must pass. The load-bearing check is kept > 0.9: a detector that flags the spike but also erases the step has reverted to the failure mode both methods are supposed to avoid.

Root-cause table

Symptom Cause Fix
Grubbs misses an obvious gross error The outlier inflated s, masking itself Switch to the Hampel filter, whose MAD is robust to the spike (Step 3)
Grubbs strips real points from a shifted batch Iterated removal on a batch that actually shifted, not one with outliers Stop iterating once the test fails to reject; investigate the shift instead (Step 4)
Hampel flags a long real drift as many outliers Window too short, or center=False lag plus a tight k Widen the window toward one cycle and relax k; drift should track the rolling median (Step 3)
Hampel mask is all False on a noisy segment Zero-MAD flat sub-window collapsed the band, or min_periods unmet Keep the band > 0 infinite-band guard; ensure enough warm-up samples (Step 3)
Both tests flag every point after a boundary Window or batch straddles a lot, shift, or tool change Reset the window and rerun per rational subgroup, never across the boundary (Steps 3–4)

Never delete a flagged row. Tag it with a reason code (HAMPEL or GRUBBS) and its timestamp, and publish the mask alongside the raw channel so a capability recalculation stays reproducible and the batch record remains defensible (AIAG SPC Reference Manual, 2nd ed., ch. I on data integrity; NIST/SEMATECH e-Handbook §1.3.5.17 on detection of outliers; §7.1.6 and §7.4.4 on Grubbs' procedure).

FAQ

When is Grubbs' test the right choice over the Hampel filter?

Use Grubbs when you have a small, discrete batch that you can defend as a single approximately normal population, and you suspect exactly one gross error in it — a set of lab replicates, a coupon run, one rational subgroup. Grubbs gives you a genuine significance level on the single most extreme point, which the Hampel filter does not. The moment the data is a continuous stream, may contain several outliers, or may legitimately drift, Grubbs' non-robust mean and standard deviation break down and the Hampel filter is the safer detector.

Why does Grubbs' test suffer from masking and swamping?

Both effects come from using the mean and standard deviation, which are not robust. Masking happens when a single gross error inflates the standard deviation so much that its own Grubbs statistic falls below the critical value and it escapes detection. Swamping happens when one true outlier drags the mean far enough that an adjacent good point now looks extreme and gets flagged instead. The Hampel filter avoids both because the median and MAD have a 50 percent breakdown point, so contamination up to nearly half the window barely moves the estimate.

How is an outlier different from an assignable cause on a control chart?

An outlier is a defect in the measurement path — a probe bounce, an electrical transient, a mis-keyed gauge reading — that affects a single sample and does not reflect the process. An assignable cause is a real change in the process itself, such as tool wear or a material lot change, and it usually shows up as a sustained shift or trend across several points. Filters exist to remove the former; control charts and run rules exist to catch the latter. Deleting the first points of an assignable cause as if they were outliers erases the very signal an I-MR chart is built to reveal, which is why the decision guide sends persistent deviations to rule detection rather than a filter.

Up one level: Outlier Detection and Filtering Pipelines. For the full ingestion sequence see Manufacturing Data Ingestion and Preprocessing.