Queue Durability for Offline MPC Parties

An MPC round over parties that are never simultaneously online is really a queueing problem wearing a cryptographic hat. Shares arrive for a party that is offline, wait somewhere, and are delivered when it returns — and everything that makes queueing hard applies, with one addition: a dropped share is not a retryable message, it is a round that can never reconstruct. This page specifies the durability properties the queue must have and the ones it must not, under async routing for MPC in Secure Multi-Party Computation in Spatial Analytics.

Parameter Configuration and Calibration

  • retention — how long an undelivered share is held. Long enough to survive the longest plausible party outage, short enough that shares are not accumulating indefinitely for a party that has left. The retention is also a privacy parameter: a queued share is data at rest, and its lifetime belongs in the retention policy alongside everything else.
  • capacity — the queue depth per party. Size it from the arrival rate times the longest outage you intend to survive, with margin. Overflow behaviour matters more than the number: dropping the oldest share silently kills old rounds, and dropping the newest kills current ones. Both must be explicit and both must alert.
  • ack_semantics — at-least-once, with idempotent application. Exactly-once delivery does not exist; what exists is at-least-once delivery plus a receiver that can apply the same share twice without harm. Make the share application idempotent on (round id, party id) and the problem disappears.
  • encryption_at_rest — mandatory, and keyed to the recipient. A queued share must be readable only by its destination party. A queue that can read the shares it holds is a party in the protocol that nobody analysed.
Property Required? Consequence if absent
Durable across broker restart yes rounds silently unreconstructible
Encrypted to the recipient yes the broker becomes an unmodelled party
Idempotent application yes duplicate shares corrupt the sum
Ordered delivery no shares are commutative within a round
Retention bounded yes queued shares outlive their legal basis
Peak queue depth against outage length A rising straight line of peak queue depth against outage length. A five-minute outage peaks at about 34 queued shares and a ninety-minute one at over 600. A capacity chosen for the mean outage will drop shares during the tail, and a dropped share is a round that can never reconstruct. Peak queue depth against outage length 34 shares arriving per 5-minute interval 20 40 60 80 0 200 400 600 outage length (minutes) shares queued at the peak peak queue depth
Size from the longest outage you intend to survive, not the mean. The cost of being wrong is not latency — it is rounds that can never complete.

Reference Implementation

python
from __future__ import annotations

import hashlib
import hmac
from dataclasses import dataclass, field
from typing import Iterable, Mapping


class QueueOverflow(RuntimeError):
    """Raised when accepting a share would exceed a party's queue capacity."""


class ShareExpired(RuntimeError):
    """Raised when a share is delivered after its round's deadline."""


@dataclass(frozen=True)
class QueuedShare:
    round_id: str
    party_id: str
    ciphertext: bytes          # encrypted TO the recipient; the queue cannot read it
    enqueued_at: float
    deadline_at: float

    def key(self) -> tuple[str, str]:
        return (self.round_id, self.party_id)


@dataclass
class ShareQueue:
    """A per-party durable queue with bounded retention and idempotent delivery.

    The queue deliberately holds ciphertext it cannot decrypt: it routes and stores,
    and is not a participant in the protocol. That distinction is what keeps the
    threat model to the parties named in it.
    """

    capacity: int
    retention_s: float
    _items: dict[tuple[str, str], QueuedShare] = field(default_factory=dict)
    _delivered: set[tuple[str, str]] = field(default_factory=set)
    dropped: int = 0

    def enqueue(self, share: QueuedShare) -> None:
        if share.key() in self._delivered:
            return                              # idempotent: already applied
        if share.key() in self._items:
            return                              # idempotent: already queued
        if len(self._items) >= self.capacity:
            raise QueueOverflow(
                f"queue for {share.party_id} is full at {self.capacity}; dropping a "
                "share makes its round permanently unreconstructible — shed load "
                "upstream instead"
            )
        self._items[share.key()] = share

    def expire(self, now: float) -> list[QueuedShare]:
        """Remove shares past retention, returning them so the caller can alert."""
        gone = [s for s in self._items.values() if now - s.enqueued_at > self.retention_s]
        for s in gone:
            del self._items[s.key()]
            self.dropped += 1
        return gone

    def deliver(self, party_id: str, now: float) -> list[QueuedShare]:
        """Hand over every live share for a party that has come back online."""
        out: list[QueuedShare] = []
        for key, share in list(self._items.items()):
            if share.party_id != party_id:
                continue
            if now > share.deadline_at:
                del self._items[key]
                self.dropped += 1
                continue
            out.append(share)
            del self._items[key]
            self._delivered.add(key)
        return out

    def depth(self, party_id: str | None = None) -> int:
        if party_id is None:
            return len(self._items)
        return sum(1 for s in self._items.values() if s.party_id == party_id)


def round_is_recoverable(
    delivered: Iterable[QueuedShare], *, threshold: int, round_id: str
) -> bool:
    """A round reconstructs only once `threshold` distinct parties have their shares."""
    parties = {s.party_id for s in delivered if s.round_id == round_id}
    return len(parties) >= threshold

Validation Checkpoint

python
def _validate() -> None:
    q = ShareQueue(capacity=4, retention_s=3_600.0)

    def share(rid: str, pid: str, t: float = 0.0, deadline: float = 7_200.0) -> QueuedShare:
        payload = hashlib.blake2b(f"{rid}:{pid}".encode(), digest_size=32).digest()
        return QueuedShare(rid, pid, payload, t, deadline)

    # 1. Enqueue is idempotent: a redelivered share does not duplicate.
    q.enqueue(share("r1", "p1"))
    q.enqueue(share("r1", "p1"))
    assert q.depth() == 1

    # 2. Capacity is enforced by raising, not by silently dropping.
    for i in range(3):
        q.enqueue(share("r1", f"p{i+2}"))
    try:
        q.enqueue(share("r2", "p9"))
    except QueueOverflow as exc:
        assert "unreconstructible" in str(exc)
    else:
        raise AssertionError("overflow must raise, not drop")

    # 3. Delivery hands over a party's shares and marks them delivered.
    delivered = q.deliver("p1", now=10.0)
    assert len(delivered) == 1 and q.depth("p1") == 0

    # 4. Re-enqueueing a delivered share is a no-op, so an at-least-once broker
    #    cannot corrupt the round by redelivering.
    q.enqueue(share("r1", "p1"))
    assert q.depth("p1") == 0

    # 5. Shares past the round deadline are dropped at delivery, and counted.
    q2 = ShareQueue(capacity=10, retention_s=3_600.0)
    q2.enqueue(share("r3", "p5", t=0.0, deadline=100.0))
    assert q2.deliver("p5", now=500.0) == [] and q2.dropped == 1

    # 6. Retention expiry removes stale shares and reports them for alerting.
    q3 = ShareQueue(capacity=10, retention_s=60.0)
    q3.enqueue(share("r4", "p6", t=0.0))
    expired = q3.expire(now=120.0)
    assert len(expired) == 1 and q3.depth() == 0

    # 7. A round reconstructs only at the threshold, never before.
    shares = [share("r5", f"p{i}") for i in range(3)]
    assert not round_is_recoverable(shares[:2], threshold=3, round_id="r5")
    assert round_is_recoverable(shares, threshold=3, round_id="r5")

    print("share queue durability: all assertions passed")


_validate()
Overflow policy decides whether work is lost or postponed A grouped bar chart of three overflow policies under the same overload. Dropping the oldest queued shares permanently loses fourteen rounds; dropping the newest loses nine. Admission control loses none, deferring eleven rounds that can be retried once capacity returns. Overflow policy decides whether work is lost or postponed same overload, three responses 0 5 10 15 rounds 14 0 drop oldest 9 0 drop newest 0 11 admission control rounds permanently lost rounds deferred but recoverable
A conventional broker drops and logs a warning. Here that destroys every other party's completed work on the round — so overflow must propagate backwards as admission control.

Assertion 2 encodes the difference between this queue and an ordinary message queue. A conventional broker under pressure drops the oldest message and logs a warning; here, dropping any share of a round destroys the round for every participant, including the ones who behaved perfectly. Overflow must propagate backwards as admission control, not forwards as data loss.

Incident Response and Edge Cases

  • A party is offline past the retention window. Its shares expire and those rounds are lost. The correct response is to shorten rounds or lengthen retention deliberately — never to extend retention ad hoc during an incident, since queued shares are data at rest with a stated lifetime.
  • The queue is at capacity for one party. Stop admitting new rounds for that party rather than dropping queued shares. Admission control is the only overflow policy that does not silently destroy work already done by every other party.
  • The broker restarts and loses in-flight shares. Durability was not actually configured. Test it explicitly: kill the broker mid-round in a staging environment and assert that the round still reconstructs. Brokers routinely default to non-durable delivery.
  • Duplicate shares arrive after a network partition heals. Expected under at-least-once delivery and harmless given idempotent application. If duplicates do cause harm, the receiver is accumulating rather than assigning — a bug that will also corrupt legitimate retries.
  • The queue operator asks to inspect a stuck message. They can see routing metadata and ciphertext, which is the right answer. If they can see share values, the deployment has an unmodelled party and the threat model needs rewriting before the incident is closed.
At-least-once delivery is fine; a non-idempotent receiver is not A two-row table comparing an accumulating receiver with an assigning, idempotent one. Under duplicate delivery the accumulating receiver counts a share twice and corrupts the sum, and the same happens when a partition heals. The idempotent receiver, which assigns by round and party identifier, is unaffected in both cases. At-least-once delivery is fine; a non-idempotent receiver is not exactly-once does not exist — idempotence is the substitute duplicate delivered partition heals verdict Accumulating receiver share counted twice sum corrupted unsafe Assigning receiver (idempotent) no effect no effect required
Assign by (round, party) rather than accumulating. The same bug that corrupts a duplicate also corrupts a legitimate retry.

One operational habit is worth adopting early: alert on queue depth per party rather than on aggregate depth. A single party accumulating shares while everyone else drains normally is the signature of a party that has gone away, and it is invisible in the aggregate until the queue is nearly full — by which point the choice is between dropping shares and refusing new rounds, and both are incidents.

Frequently Asked Questions

Why is dropping a share worse than dropping a message?

Because a threshold scheme has no partial credit. Below the threshold the remaining shares reveal nothing and reconstruct nothing, so one dropped share wastes every other party's work on that round. Queue policies tuned for ordinary messaging — drop-oldest under pressure — are actively wrong here.

Does share order matter?

No. Shares within a round are commutative, so ordered delivery buys nothing and costs throughput and head-of-line blocking. What does matter is completeness: the threshold count must arrive before the round's deadline.

Can the queue be operated by a third party?

Yes, provided shares are encrypted to their recipients and the operator learns only routing metadata. That metadata is not nothing — which parties talk, how often, and when — so it should be part of the threat model, but it is a far weaker exposure than share contents.

How long should retention be?

Long enough to cover the longest outage you intend to survive and no longer. Retention is simultaneously an availability parameter and a data-retention commitment, so the number needs a privacy owner as well as an operations one.

Up one level: Async Routing for MPC · Section: Secure Multi-Party Computation in Spatial Analytics.