fix(wake): #932 stop reconciler re-enumerating already-CONSUMED detector state
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
Wake-pilot finding #7 (safe-but-noisy G2a alarm-hygiene): after consume-truncation a consumed detector-observed state matched NO accounting record — the inbox was truncated, the reconciler's seen-ledger only covers its OWN enumerations, and the detector hash-file is correctly DISTRUSTED — so the reconciler treated it as UNACCOUNTED and re-enumerated it: one DUPLICATE orientation wake + one SPURIOUS rc=1 CRITICAL per detector-active window per cycle (functionally safe, no lost obligation, but cry-wolf erosion at fleet scale). Fix (Mos ruling, built exactly): 1. store.sh records the last-consumed observed_hash per (kind,id) at consume-truncation into a NEW store-owned durable record consumed-hashes.jsonl (atomic write; ADDITIVE — existing on-disk format unchanged/read-compatible; #908 allocator untouched). 2. reconcile.sh adds a THIRD accounting source alongside the inbox and its seen-ledger: a detector-observed state whose observed_hash MATCHES the store's recorded last-consumed hash for that (kind,id) is ACCOUNTED — not re-enumerated (no dup wake, no spurious CRITICAL). 3. Trust boundary (load-bearing): the 3rd check consults ONLY the store-written record. Its existence implies the state was durably enqueued+consumed, so it structurally cannot exhibit the §5 hash-advance-without-enqueue swallow signature. Trusting DETECTOR hash-files STAYS REJECTED. 4. G3 not weakened: only states the store RECORDED as consumed are suppressed. A genuinely-unaccounted state (enqueued-but-unconsumed, still in the inbox, OR a real gap) still re-enumerates + alarms. Red-first tests: - store-ack T12: consume writes the store-owned last-consumed record (per-(kind,id), monotonic last-seq wins, lazily created). - reconcile R10: a consumed state is ACCOUNTED (rc=0, UNACCOUNTED=0, no re-enumeration). - reconcile R11: pilot repro — consumed state SUPPRESSED while a distinct unconsumed/gap state STILL re-enumerates (G3 teeth intact; no over-suppression). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0158NZqN2n2ymKFeJAZ4GUCb
This commit is contained in:
@@ -47,10 +47,14 @@
|
||||
# RETIRED; --allow-enumerate / WAKE_RECONCILE_ALLOW_ENUMERATE remain accepted
|
||||
# as deprecated no-ops for caller compatibility. The G3 accounting invariants
|
||||
# (0-UNACCOUNTED, vacuous-pass prevention) are unaffected and still enforced.
|
||||
# * "accounted" is judged against observed_seq/inbox (§4/G3 wording) PLUS the
|
||||
# reconciler's OWN durable reconciled-state ledger — NOT the detector's
|
||||
# hash-file. Judging by the detector baseline would let a silent first-seen
|
||||
# baseline swallow a pre-existing obligation (the exact hole G3 closes).
|
||||
# * "accounted" is judged against THREE sources: observed_seq/inbox (§4/G3
|
||||
# wording), the reconciler's OWN durable reconciled-state ledger, and (#932)
|
||||
# the STORE's last-consumed record (consumed-hashes.jsonl, store-written at
|
||||
# consume-truncation) — NOT the detector's hash-file. Judging by the detector
|
||||
# baseline would let a silent first-seen baseline swallow a pre-existing
|
||||
# obligation (the exact hole G3 closes). The 3rd source suppresses ONLY states
|
||||
# the store RECORDED as consumed (structurally cannot be hash-advance-without-
|
||||
# enqueue), so a truly-unaccounted state still re-enumerates (G3 teeth intact).
|
||||
# * enumeration uses the non-coalescible fail-safe class `actionable` (§2.3)
|
||||
# so two distinct pre-existing obligations can never coalesce into one.
|
||||
#
|
||||
@@ -367,6 +371,35 @@ _inbox_has() {
|
||||
[ "${n:-0}" -gt 0 ]
|
||||
}
|
||||
|
||||
# _store_consumed_has KIND ID HASH — #932 THIRD accounting source. True iff the
|
||||
# STORE's last-consumed record (consumed-hashes.jsonl, written by store.sh at
|
||||
# consume-truncation) records HASH as the last-consumed observed_hash for
|
||||
# (kind,id). After consume truncates the pending prefix, a consumed state matches
|
||||
# NEITHER _inbox_has (inbox truncated) NOR the reconciler's own seen-ledger (it
|
||||
# only covers the reconciler's OWN enumerations) — so without this check the
|
||||
# reconciler re-enumerates the just-consumed state: one DUPLICATE orientation
|
||||
# wake + one SPURIOUS rc=1 CRITICAL per detector-active window per cycle (#932).
|
||||
#
|
||||
# TRUST BOUNDARY (load-bearing — do NOT violate): this consults ONLY the
|
||||
# store-written record under STATE_DIR — NEVER a detector-owned hash-file. The
|
||||
# store record is trustworthy BY CONSTRUCTION: it is written only at CONSUME of a
|
||||
# durably-enqueued entry, so its existence implies the state WAS durably enqueued,
|
||||
# and it structurally cannot exhibit the §5 swallow-hole signature (hash-advance-
|
||||
# WITHOUT-enqueue). Trusting DETECTOR hash-files STAYS REJECTED (it reopens §5).
|
||||
#
|
||||
# G3 is NOT weakened: this suppresses ONLY states the store has RECORDED as
|
||||
# consumed. A genuinely-unaccounted state (durably enqueued but not yet consumed
|
||||
# — still in the inbox — OR a real gap) has no matching consumed record, so it is
|
||||
# still UNACCOUNTED and still re-enumerates + alarms.
|
||||
_store_consumed_has() {
|
||||
local kind="$1" id="$2" h="$3" n
|
||||
[ -f "$STATE_DIR/consumed-hashes.jsonl" ] || return 1
|
||||
n="$(jq -s --arg k "$kind" --arg id "$id" --arg h "$h" \
|
||||
'[ .[] | select(.kind==$k and .id==$id and .observed_hash==$h) ] | length' \
|
||||
"$STATE_DIR/consumed-hashes.jsonl" 2>/dev/null || echo 0)"
|
||||
[ "${n:-0}" -gt 0 ]
|
||||
}
|
||||
|
||||
cmd_reconcile() {
|
||||
local enumerate=1 allow_enumerate=0
|
||||
while [ $# -gt 0 ]; do
|
||||
@@ -444,6 +477,12 @@ cmd_reconcile() {
|
||||
# Or already reconciled to this exact state before (durable ledger)?
|
||||
elif [ -f "$ledger" ] && [ "$(tr -d '[:space:]' <"$ledger")" = "$curhash" ]; then
|
||||
accounted=1
|
||||
# Or the STORE recorded this exact state as ALREADY CONSUMED (#932 — the THIRD
|
||||
# accounting source; a consume-truncated state matches neither the inbox nor
|
||||
# the seen-ledger, so without this it would spuriously re-enumerate). Consults
|
||||
# ONLY the store-written record, never a detector hash-file (§5 stays closed).
|
||||
elif _store_consumed_has "$kind" "$id" "$curhash"; then
|
||||
accounted=1
|
||||
fi
|
||||
|
||||
if [ "$accounted" -eq 1 ]; then
|
||||
|
||||
Reference in New Issue
Block a user