Privacy Budget Exhaustion — Detection and Recovery

The production failure mode that silently voids a differential-privacy guarantee is not a bad noise draw — it is the query that ships after cumulative ε\varepsilon has already crossed its (ε,δ)(\varepsilon, \delta) ceiling. Each release debits the ledger; the danger is the release that debits past the cap and still returns an answer, because from that point every downstream aggregate carries a weaker guarantee than the one you documented and attested. This page is the detection-and-recovery deep-dive inside the broader privacy budget management guide, which covers steady-state ledger operation; here the focus is narrow and adversarial to your own pipeline — how to catch the moment before the ceiling is breached and what to do when a legitimate query cannot be answered without a breach. It sits within the Core Fundamentals & Architecture for Spatial Privacy reference and pairs directly with the spatial noise mechanisms that actually spend the budget. The governing rule is simple: an unavailable answer is an operational inconvenience, an unprotected one is a reportable breach.

Parameter Configuration and Calibration

Exhaustion detection has no free parameters — every knob below trades availability against the strength of the guarantee, and each one must resolve to a concrete number bound to a policy clause, not a vibe. The ceiling itself is the load-bearing value: tie εmax\varepsilon_{\max} to GDPR Article 25 data-minimisation by expressing it as the smallest total budget that still supports the analytical mission, then treating any query that would exceed it as a minimisation violation rather than a capacity problem.

Knob Typical value Rationale / privacy consequence
epsilon_max 3.0 per subject per window The (ε,δ)(\varepsilon,\delta) ceiling. Lower is stronger; bind it to Art. 25 minimisation as a hard cap, not a target.
delta 1e-6 Failure probability for the Gaussian/RDP path; keep δ1/N\delta \ll 1/N for a population of NN subjects.
alert_fraction 0.80 Threshold-alert trigger. Firing at 80 % spent leaves headroom to page an operator and drain in-flight queries before the hard stop.
reserve_epsilon 0.2 A withheld slice so a mandatory closing query (audit, safety) can always run even at nominal exhaustion.
window 24 h rolling / fixed epoch The accounting horizon. A rotation resets the ledger; the window length is the recovery cadence.
coarsen_factor 0.5 Per-query spend multiplier when coarsening resolution (e.g. 250 m → 500 m grid halves sensitivity Δf\Delta f, so b=Δf/εb=\Delta f/\varepsilon buys the same utility for less ε\varepsilon).

Monitor spend at three granularities simultaneously, because exhaustion is per-scope, never global: per subject (cumulative ε\varepsilon per data subject is the canonical unit), per tenant (a shared pipeline must not let one customer drain another’s budget), and per window (a rolling 24 h horizon bounds long-run linkage). A guard that only tracks a single global counter will pass a query that is fine in aggregate but catastrophic for one over-queried individual.

Reference Implementation

BudgetExhaustionGuard wraps a composition accountant and interposes a mandatory pre-flight gate between a query and the noise mechanism. The critical ordering property is structural: check(cost) runs to completion — raising if the spend would exceed the ceiling — before any noise is drawn or released. It exposes an on_threshold alert hook and a recover(strategy) method with the four production recovery paths. In deployment, back Accountant with a real Rényi (RDP) or zCDP accountant so composition is counted tightly rather than by naïve summation.

python
from __future__ import annotations

from dataclasses import dataclass, field
from enum import Enum
from typing import Callable, Optional


class BudgetExhausted(RuntimeError):
    """Raised when a query's cost would push cumulative ε past the (ε, δ) ceiling.

    Catching this is the fail-closed path: the query is refused and NO noise is
    released, so the accountant's guarantee stays intact.
    """


class RecoveryStrategy(str, Enum):
    FAIL_CLOSED = "fail_closed"   # reject the query outright
    COARSEN = "coarsen"           # lower release resolution → less ε per query
    ROTATE = "rotate"             # new epoch / cohort → fresh window
    CACHED_TILE = "cached_tile"   # serve a pre-aggregated privacy-safe tile


@dataclass
class Accountant:
    """Minimal cumulative-ε accountant. In production, delegate `spent` to an
    RDP/zCDP accountant so composition is counted tightly, not additively."""

    epsilon_max: float
    delta: float = 1e-6
    spent: float = 0.0

    def remaining(self) -> float:
        return self.epsilon_max - self.spent

    def would_exceed(self, cost: float) -> bool:
        # strict '>' — spending exactly to the ceiling is permitted
        return self.spent + cost > self.epsilon_max

    def commit(self, cost: float) -> None:
        """Debit the ledger. MUST be called only after the guard authorises."""
        self.spent += cost


@dataclass
class BudgetExhaustionGuard:
    """Pre-flight budget gate with threshold alerting and recovery strategies.

    `check` is the detection primitive: it fires the threshold alert and then
    raises BEFORE the caller is allowed to draw noise, so a would-be over-spend
    can never leak an under-noised release.
    """

    accountant: Accountant
    alert_fraction: float = 0.80
    coarsen_factor: float = 0.5
    on_threshold: Optional[Callable[[float], None]] = None
    _alerted: bool = field(default=False, repr=False)

    def check(self, cost: float) -> None:
        """Authorise (or veto) a prospective spend of `cost` ε.

        Runs to completion before any noise is drawn. Raises BudgetExhausted
        if committing `cost` would cross epsilon_max.
        """
        if cost < 0:
            raise ValueError("query cost (ε) must be non-negative")
        projected = self.accountant.spent + cost
        fraction = projected / self.accountant.epsilon_max
        # threshold alert fires once per window, at ≥ alert_fraction spent
        if fraction >= self.alert_fraction and not self._alerted:
            self._alerted = True
            if self.on_threshold is not None:
                self.on_threshold(fraction)
        if self.accountant.would_exceed(cost):
            # veto: an unavailable answer is acceptable, an unprotected one is not
            raise BudgetExhausted(
                f"spend {cost:.3f} would exceed ceiling: "
                f"{projected:.3f} > {self.accountant.epsilon_max:.3f}"
            )

    def spend(self, cost: float) -> float:
        """check() then commit(). Returns remaining budget after the debit."""
        self.check(cost)               # gate FIRST — may raise, releasing nothing
        self.accountant.commit(cost)   # only reached if authorised
        return self.accountant.remaining()

    def recover(self, strategy: RecoveryStrategy, cost: float = 0.0) -> float:
        """Apply a recovery strategy; return the ε the caller should now attempt.

        A return of 0.0 means 'no fresh budget is required' (cached or rotated).
        """
        if strategy is RecoveryStrategy.FAIL_CLOSED:
            # refuse and preserve the budget — the default safe action
            raise BudgetExhausted("fail-closed: query rejected, budget preserved")
        if strategy is RecoveryStrategy.COARSEN:
            # coarser grid lowers sensitivity Δf, so the same utility costs less ε
            return cost * self.coarsen_factor
        if strategy is RecoveryStrategy.CACHED_TILE:
            # a pre-aggregated privacy-safe tile was already paid for once
            return 0.0
        if strategy is RecoveryStrategy.ROTATE:
            # new epoch/cohort: open a fresh (ε, δ) window and zero the ledger
            self.accountant.spent = 0.0
            self._alerted = False
            return 0.0
        raise ValueError(f"unknown strategy: {strategy!r}")

Validation Checkpoint

Two invariants make this guard trustworthy, and both must run in CI rather than be reasoned about. First, a query that would exceed the cap is rejected before any noise is released — proven with a sentinel flag that is only set on the far side of the gate. Second, a coarsen recovery genuinely lowers per-query spend, so the retried query fits under the ceiling.

python
def _validate() -> None:
    acc = Accountant(epsilon_max=3.0, delta=1e-6)
    alerts: list[float] = []
    guard = BudgetExhaustionGuard(
        acc, alert_fraction=0.80, coarsen_factor=0.5,
        on_threshold=lambda f: alerts.append(f),
    )
    noise = {"released": False}

    def run_query(cost: float) -> None:
        guard.check(cost)          # 1. gate runs to completion first
        noise["released"] = True   # 2. only reachable when authorised
        acc.commit(cost)

    run_query(1.5)                 # spent 1.5 (50%)
    run_query(1.0)                 # projected 2.5 ≈ 83% → threshold alert fires
    assert alerts and alerts[0] >= 0.80, "threshold alert must fire at ≥ 80% spent"

    # A query that would cross the ceiling is rejected BEFORE any noise is drawn.
    noise["released"] = False
    try:
        run_query(1.0)             # 2.5 + 1.0 = 3.5 > 3.0
    except BudgetExhausted:
        pass
    else:
        raise AssertionError("over-budget query must be rejected")
    assert noise["released"] is False, "no noise may be released on a vetoed query"
    assert acc.spent == 2.5, "a vetoed query must not debit the ledger"

    # Coarsen recovery lowers per-query spend so the retried query now fits.
    cheaper = guard.recover(RecoveryStrategy.COARSEN, cost=1.0)
    assert cheaper < 1.0, "coarsening must reduce ε per query"
    run_query(cheaper)             # 2.5 + 0.5 = 3.0 ≤ 3.0
    assert acc.spent <= acc.epsilon_max

    # Rotate opens a fresh window: the ledger is zeroed for a new epoch.
    guard.recover(RecoveryStrategy.ROTATE)
    assert acc.spent == 0.0 and guard._alerted is False
    print("budget-exhaustion guard invariants hold")


if __name__ == "__main__":
    _validate()

The state machine below is the operational contract these tests encode: a scope moves from healthy through a warning band at 80 % spent into exhausted, from which exactly one recovery path is chosen, and only a window reset returns it to healthy.

Privacy-budget exhaustion state machine — healthy, warning at 80%, exhausted, recover, window reset A state diagram in three tiers. The top tier has three states left to right: "Healthy" (spent below 80% of the ceiling), "Warning" (spent at or above 80%, threshold alert fired), and "Exhausted" (a pre-flight check found the next query would exceed the ceiling, so queries are rejected). A rightward arrow from Healthy to Warning is labelled "pre-flight: projected ≥ 80% ε_max"; a rightward arrow from Warning to Exhausted is labelled "pre-flight: cost > remaining"; Exhausted carries a self-loop labelled "reject queries". The middle tier has three recovery states fed by three downward arrows from Exhausted, together labelled "recover(strategy)": "Coarsen release" (spend less per query), "Rotate epoch / cohort" (fresh window), and "Fail-closed" (reject, preserve budget). Below them a wide bar reads "Window reset — ε ledger zeroed, new (ε,δ) accounting window opens", fed by downward arrows from each recovery state. A path from the left of the reset bar climbs the left margin back into Healthy, labelled "reopen". Budget exhaustion — detect at 80%, veto at the ceiling, recover, reset the window reject queries pre-flight: projected ≥ 80% ε_max pre-flight: cost > remaining recover(strategy) reopen · resume at ε = 0 Healthy spent < 80% of ceiling queries authorised Warning · ≥ 80% threshold alert fired drain in-flight, page ops Exhausted next cost > remaining no unprotected release Coarsen release bigger grid cell → less ε per query Rotate epoch fresh cohort / new window Fail-closed reject query, preserve budget Window reset ε ledger zeroed · new (ε, δ) accounting window opens · cached tiles cost nothing

Incident Response and Edge Cases

Exhaustion incidents rarely announce themselves; they surface as a subtle drop in guarantee strength that no test caught. The scenarios below are the ones that reach production.

  • Silent over-spend from composition miscounting. A pipeline that sums raw ε\varepsilon across queries under-counts the tightening that RDP or zCDP composition actually gives — or, worse, over-counts and exhausts early — but the truly dangerous version is a parallel worker that debits its own local counter and never reconciles, so the true cumulative spend crosses the ceiling while the global ledger reads healthy. Detection: periodically recompute cumulative spend from the immutable release log and assert it equals the accountant’s spent; alert on any drift. Remediation: serialise check/commit behind a single accountant (or a transactional store), and reconstruct the ledger from the release log after any crash.
  • Boundary release at exactly the ceiling. A query that lands cumulative spend precisely on epsilon_max is permitted (the guard uses strict >), but the next non-zero query cannot run. Detection: the warning alert should already have fired; a spend that reaches the ceiling with in-flight queues behind it is the failure. Remediation: hold back reserve_epsilon so the very last admissible spend is a deliberate, logged decision rather than an accident of ordering.
  • Per-subject exhaustion hidden by a global counter. An over-queried individual can blow their personal budget while the tenant-wide ledger looks fine, producing an under-protected release for exactly the person most at risk. Detection: key the accountant by subject and tenant, not just globally, and monitor the tail of the per-subject spend distribution. Remediation: coarsen or fail-closed at the subject scope, and calibrate the ceiling against the risk tier from spatial noise mechanisms.
  • Rotation that resets the ledger but not the risk. Rotating to a fresh epoch zeroes the counter, but if it queries the same subjects the real linkage risk keeps accumulating across windows even though each window looks compliant. Detection: audit whether consecutive windows draw from overlapping cohorts. Remediation: rotate the cohort, not just the clock, or bound the number of windows any single subject may appear in — a control you can express against GDPR Article 25 storage-limitation.

Frequently Asked Questions

How is exhaustion detection different from ordinary ledger operation?

The parent guide covers the steady-state ledger: debiting each release, composing spend, reporting remaining budget. Detection is the adversarial layer on top — a mandatory pre-flight gate that answers "would this query breach the ceiling?" before a single noise sample is drawn, plus threshold alerting and per-scope monitoring that catch a breach in the making. The ledger tells you what you have spent; the guard stops you spending past the point where the guarantee still holds.

Why fail-closed instead of quietly returning a lower-resolution answer?

Because a silent degrade hides a compliance-relevant decision inside a query response. Fail-closed makes the refusal explicit and auditable: an unavailable answer is an operational inconvenience, an unprotected one is a reportable breach. Coarsening is a valid recovery, but it must be a deliberate, logged recover(COARSEN) step that re-runs the pre-flight check at the lower cost — never an automatic fallback that masks the exhaustion.

How do I detect silent over-spend from composition miscounting?

Never trust a running counter as the sole source of truth. Keep an immutable, append-only release log recording the epsilon cost of every release, and periodically recompute cumulative spend from that log through your RDP or zCDP accountant. Assert the recomputed value equals the live accountant's spent and alert on any drift. This catches parallel workers that debit local counters without reconciling, and lets you rebuild the ledger exactly after a crash.

Does rotating to a fresh epoch actually reset privacy risk?

It resets the accounting window, not the underlying risk. If the new window queries the same subjects, real re-identification risk keeps accumulating across windows even though each window individually looks compliant. Rotation is only a sound recovery when it rotates the cohort as well as the clock, or when a storage-limitation control bounds how many windows any one subject may appear in.

Up: Privacy Budget Management · Core Fundamentals & Architecture for Spatial Privacy