wake/W4 hardening (low-pri): observed_seq incremented before store enqueue can permanently wedge the inbox #908

Closed
opened 2026-07-26 01:04:11 +00:00 by jason.woltje · 1 comment
Owner

Low-priority hardening tracked from the #907 (W4 detector) review (RoR comment 18933). Mos-ruled: file even though unreachable today, because the SEVERITY is a permanent inbox wedge.

Failure mode: detector.sh's _next_observed_seq increments the persistent monotonic counter BEFORE store.sh enqueue. If an enqueue ever fails, the detector fails loud and does NOT advance the per-watch last-observed hash (so the source re-observes with the NEXT seq) — but the BURNED seq never reaches the store's observed.set. By the §2.2 contiguous-prefix contract (triple-ratified in W2), every future consume --upto N with N beyond the burned seq then HARD-REJECTS on the interior-gap check — permanently wedging the consumer inbox.

Likelihood: unreachable on the validated call path todaystore.sh enqueue only fails on malformed args or catastrophic disk/jq failure, none of which occur on the detector's validated call. Hence LOW priority. But awareness-only rots; the wedge is permanent if it ever fires, so it is tracked.

Fix (cheap, when the file is next touched): increment observed_seq only AFTER a successful store.sh enqueue — OR, on the enqueue failure path, record the seq as observed-with-tombstone so the contiguous-prefix stays gapless. Add a red-first test simulating an enqueue failure and asserting no permanent gap.

Part of #892. Ref: #907 RoR (comment 18933).

Low-priority hardening tracked from the #907 (W4 detector) review (RoR comment 18933). Mos-ruled: file even though unreachable today, because the SEVERITY is a **permanent inbox wedge**. **Failure mode:** `detector.sh`'s `_next_observed_seq` increments the persistent monotonic counter BEFORE `store.sh enqueue`. If an enqueue ever fails, the detector fails loud and does NOT advance the per-watch last-observed hash (so the source re-observes with the NEXT seq) — but the BURNED seq never reaches the store's `observed.set`. By the §2.2 contiguous-prefix contract (triple-ratified in W2), every future `consume --upto N` with N beyond the burned seq then HARD-REJECTS on the interior-gap check — **permanently wedging the consumer inbox**. **Likelihood: unreachable on the validated call path today** — `store.sh enqueue` only fails on malformed args or catastrophic disk/jq failure, none of which occur on the detector's validated call. Hence LOW priority. But awareness-only rots; the wedge is permanent if it ever fires, so it is tracked. **Fix (cheap, when the file is next touched):** increment `observed_seq` only AFTER a successful `store.sh enqueue` — OR, on the enqueue failure path, record the seq as observed-with-tombstone so the contiguous-prefix stays gapless. Add a red-first test simulating an enqueue failure and asserting no permanent gap. Part of #892. Ref: #907 RoR (comment 18933).
Author
Owner

SCOPE EXPANDED (Mos ruling, 2026-07-26) — now the store-side allocator UNIFICATION, absorbing the §5 migration fix. A live dragon-lin pilot surfaced a THIRD arrow at the same detector-private-counter seam: §5 migration seeds the STORE cursors but NOT the detector's private observed_seq_counter → detector restarts at 0 → next deltas get seqs ≤ consumed → enqueue idempotent-no-ops (exit 0) while the watch-hash advances = PERMANENT SILENT SWALLOW, no fail-loud. Three independent arrows now: (1) this issue's burn-before-enqueue wedge; (2) the W5 reconcile aliasing repro (reconcile.sh dual-allocator note); (3) the live migration-restart swallow. RULING: unify on a SINGLE store-side allocator (store.sh enqueue allocates observed_seq atomically from its own durable cursor + returns it; delete the detector private counter) — this makes counter-seeding moot and the migration seam disappears STRUCTURALLY. Brief requires a red-first migration-restart test (adopt→migrate→restart→delta must WAKE not swallow) + preserves the W2 contiguous-prefix contract. Builder dispatched; same flow builder→indep-review→RoR→id-11. Still Closes #908 (fully resolved by the unification).

**SCOPE EXPANDED (Mos ruling, 2026-07-26) — now the store-side allocator UNIFICATION, absorbing the §5 migration fix.** A live dragon-lin pilot surfaced a THIRD arrow at the same detector-private-counter seam: §5 migration seeds the STORE cursors but NOT the detector's private `observed_seq_counter` → detector restarts at 0 → next deltas get seqs ≤ consumed → `enqueue` idempotent-no-ops (exit 0) while the watch-hash advances = **PERMANENT SILENT SWALLOW, no fail-loud**. Three independent arrows now: (1) this issue's burn-before-enqueue wedge; (2) the W5 reconcile aliasing repro (reconcile.sh dual-allocator note); (3) the live migration-restart swallow. RULING: unify on a SINGLE store-side allocator (`store.sh enqueue` allocates observed_seq atomically from its own durable cursor + returns it; delete the detector private counter) — this makes counter-seeding moot and the migration seam disappears STRUCTURALLY. Brief requires a red-first migration-restart test (adopt→migrate→restart→delta must WAKE not swallow) + preserves the W2 contiguous-prefix contract. Builder dispatched; same flow builder→indep-review→RoR→id-11. Still `Closes #908` (fully resolved by the unification).
Mos closed this issue 2026-07-26 06:02:04 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mosaicstack/stack#908