mosaicstack/stack
1
0
Fork 1
Code Issues 109 Pull Requests 2 Actions Packages Projects Releases 15 Wiki Activity

wake: dead-letter retention is now load-bearing for quarantine-audit provability — record it at the write site #953

New Issue
Closed
opened 2026-07-30 15:42:31 +00:00 by mos-dt-0 · 0 comments
mos-dt-0 commented 2026-07-30 15:42:31 +00:00
Contributor
Copy Link
Copy Source

Follow-up to #946 / PR #951 (review finding 2, non-gating). Owner: pepper.

Not a defect today — an unrecorded load-bearing dependency

store.sh quarantine-audit (landed in PR #951) can convict a false consumed-hashes witness row only against the dead-letter ledger (dead-letter.jsonl): a row is provably false iff a surviving dead-letter entry matches it on (kind, id, observed_seq, observed_hash) and the row's seq is ≤ consumed_seq. Rows whose dead-letter evidence is gone are unprovable and untouched — the audit never guesses. That provability bound is correct and must be kept.

The consequence: dead-letter retention became load-bearing for auditability the moment the audit landed — and nothing in the codebase records that it now is. Today this is latent: store.sh never modifies the ledger (it is history by design; PR #951 explicitly leaves even stale rows in place), and nothing prunes it.

The failure mode this guards against

The next person to add log rotation or a size cap to dead-letter.jsonl will be doing something locally correct — and will silently convert provable rows into unprovable ones, with no signal at either end (the audit's clean sweep reads identically before and after the evidence disappears). A correct change made against an unrecorded dependency is a more dangerous class than a bug.

Fix

A comment at the dead-letter write site in store.sh stating:

  • the ledger is append-only history AND the sole evidence base for quarantine-audit's conviction predicate;
  • any rotation/pruning/size-cap silently narrows the audit's provable set;
  • if retention limits ever become necessary, they must come with a loud signal (and the audit's residual-class reporting updated in the same change).

That comment is the whole fix. It should land before someone has an unrelated reason to touch that file.

Follow-up to #946 / PR #951 (review finding 2, non-gating). Owner: pepper. ## Not a defect today — an unrecorded load-bearing dependency `store.sh quarantine-audit` (landed in PR #951) can convict a false consumed-hashes witness row **only** against the dead-letter ledger (`dead-letter.jsonl`): a row is provably false iff a surviving dead-letter entry matches it on (kind, id, observed_seq, observed_hash) and the row's seq is ≤ consumed_seq. Rows whose dead-letter evidence is gone are **unprovable and untouched** — the audit never guesses. That provability bound is correct and must be kept. The consequence: **dead-letter retention became load-bearing for auditability** the moment the audit landed — and nothing in the codebase records that it now is. Today this is latent: `store.sh` never modifies the ledger (it is history by design; PR #951 explicitly leaves even stale rows in place), and nothing prunes it. ## The failure mode this guards against The next person to add log rotation or a size cap to `dead-letter.jsonl` will be doing something **locally correct** — and will silently convert provable rows into unprovable ones, with no signal at either end (the audit's clean sweep reads identically before and after the evidence disappears). A correct change made against an unrecorded dependency is a more dangerous class than a bug. ## Fix A comment at the dead-letter write site in `store.sh` stating: - the ledger is append-only history AND the sole evidence base for `quarantine-audit`'s conviction predicate; - any rotation/pruning/size-cap silently narrows the audit's provable set; - if retention limits ever become necessary, they must come with a loud signal (and the audit's residual-class reporting updated in the same change). That comment is the whole fix. It should land **before** someone has an unrelated reason to touch that file.
Mos closed this issue 2026-07-30 23:25:40 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
fleet-enhancement
Fleet-wide reliability/speed/cost improvements surfaced by the nightly enhance pass
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
jason.woltje
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mosaicstack/stack#953