fix(wake): #932 stop reconciler re-enumerating already-CONSUMED detector state
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful

Wake-pilot finding #7 (safe-but-noisy G2a alarm-hygiene): after
consume-truncation a consumed detector-observed state matched NO accounting
record — the inbox was truncated, the reconciler's seen-ledger only covers its
OWN enumerations, and the detector hash-file is correctly DISTRUSTED — so the
reconciler treated it as UNACCOUNTED and re-enumerated it: one DUPLICATE
orientation wake + one SPURIOUS rc=1 CRITICAL per detector-active window per
cycle (functionally safe, no lost obligation, but cry-wolf erosion at fleet
scale).

Fix (Mos ruling, built exactly):
1. store.sh records the last-consumed observed_hash per (kind,id) at
   consume-truncation into a NEW store-owned durable record
   consumed-hashes.jsonl (atomic write; ADDITIVE — existing on-disk format
   unchanged/read-compatible; #908 allocator untouched).
2. reconcile.sh adds a THIRD accounting source alongside the inbox and its
   seen-ledger: a detector-observed state whose observed_hash MATCHES the
   store's recorded last-consumed hash for that (kind,id) is ACCOUNTED — not
   re-enumerated (no dup wake, no spurious CRITICAL).
3. Trust boundary (load-bearing): the 3rd check consults ONLY the
   store-written record. Its existence implies the state was durably
   enqueued+consumed, so it structurally cannot exhibit the §5
   hash-advance-without-enqueue swallow signature. Trusting DETECTOR
   hash-files STAYS REJECTED.
4. G3 not weakened: only states the store RECORDED as consumed are
   suppressed. A genuinely-unaccounted state (enqueued-but-unconsumed, still
   in the inbox, OR a real gap) still re-enumerates + alarms.

Red-first tests:
- store-ack T12: consume writes the store-owned last-consumed record
  (per-(kind,id), monotonic last-seq wins, lazily created).
- reconcile R10: a consumed state is ACCOUNTED (rc=0, UNACCOUNTED=0, no
  re-enumeration).
- reconcile R11: pilot repro — consumed state SUPPRESSED while a distinct
  unconsumed/gap state STILL re-enumerates (G3 teeth intact; no
  over-suppression).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158NZqN2n2ymKFeJAZ4GUCb
This commit is contained in:
mosaic-coder
2026-07-26 10:34:14 -05:00
parent 9becaf877f
commit 009e78a190
6 changed files with 244 additions and 5 deletions

View File

@@ -17,6 +17,8 @@
# T9 burn-before-enqueue: a failed durable write must NOT advance the
# observed_seq cursor (no burned seq / no interior gap) (#908 arrow 1)
# T10 concurrency: two concurrent enqueues get DISTINCT seqs (lock) (#908)
# T12 consume records the last-consumed observed_hash per (kind,id) into the
# store-owned record (additive; monotonic last-seq wins; lazily created) (#932)
#
# Isolated: every test runs against a fresh WAKE_STATE_HOME temp dir.
set -uo pipefail
@@ -575,6 +577,34 @@ echo "== T11: final observed_seq cursor write FAILURE — fail loud + observed.s
fi
) && ok
echo "== T12: #932 — consume records the last-consumed observed_hash per (kind,id) into the store-owned record (additive; monotonic; keyed by kind,id) =="
(
WAKE_STATE_HOME="$(fresh_state t12)"
export WAKE_STATE_HOME
unset WAKE_AGENT
rec="$WAKE_STATE_HOME/default/consumed-hashes.jsonl"
# Two source states for the SAME (kind,id): the store must record only the
# LAST-consumed (highest-seq) hash for that key; plus a distinct (kind,id).
s1="$("$STORE" enqueue --class actionable --locators '{"kind":"repo","id":"r1","observed_hash":"HASH-A"}')"
s2="$("$STORE" enqueue --class actionable --locators '{"kind":"repo","id":"r1","observed_hash":"HASH-B"}')"
s3="$("$STORE" enqueue --class actionable --locators '{"kind":"repo","id":"r2","observed_hash":"HASH-C"}')"
[ "$s1 $s2 $s3" = "1 2 3" ] || fail_msg "T12: contiguous allocation expected 1 2 3, got '$s1 $s2 $s3'"
# BEFORE consume there is no record (additive / lazily created).
[ ! -f "$rec" ] || fail_msg "T12: the last-consumed record must not exist before any consume"
"$STORE" consume --upto 3 >/dev/null 2>&1 || fail_msg "T12: CONSUMED 3 must succeed over the gapless prefix"
# AFTER consume the store-owned record exists and holds ONE entry per (kind,id).
[ -f "$rec" ] || fail_msg "T12: consume must write the store-owned last-consumed record ($rec)"
nkeys="$(jq -s '[ .[] | {kind,id} ] | unique | length' "$rec" 2>/dev/null || echo 0)"
[ "$nkeys" = "2" ] || fail_msg "T12: the record must hold exactly one entry per (kind,id) — 2 keys, got $nkeys"
# repo/r1 must record the LAST-consumed hash (HASH-B at seq 2), never HASH-A.
r1h="$(jq -sr '[ .[] | select(.kind=="repo" and .id=="r1") ] | .[0].observed_hash' "$rec" 2>/dev/null)"
[ "$r1h" = "HASH-B" ] || fail_msg "T12: repo/r1 must record the LAST-consumed hash HASH-B (seq 2), got '$r1h'"
r2h="$(jq -sr '[ .[] | select(.kind=="repo" and .id=="r2") ] | .[0].observed_hash' "$rec" 2>/dev/null)"
[ "$r2h" = "HASH-C" ] || fail_msg "T12: repo/r2 must record HASH-C, got '$r2h'"
# ADDITIVE: existing on-disk state files are unchanged/consistent post-consume.
"$STORE" cursors | grep -q 'consumed_seq=3' || fail_msg "T12: consumed_seq must be 3 after CONSUMED 3"
) && ok
echo
if [ -s "$FAILFILE" ]; then
echo "wake store/ack harness: FAILED ($(grep -c . "$FAILFILE") assertion(s))" >&2