From 6905f848819c55b79db25698142f28779a5a9d70 Mon Sep 17 00:00:00 2001 From: mosaic-coder Date: Sun, 26 Jul 2026 03:13:30 -0500 Subject: [PATCH] fix(wake): #920 quarantine render-refused drain entry (no head-of-line block) + reconciler enumerations render orientation-tier MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Live pilot finding #6 (BLOCKING): a reconciler enumeration enqueued class=actionable with soft locators {kind,id,path,observed_hash} failed §2.1's ACTIONABLE hard-locator taxonomy, so digest render exit-4'd the ENTIRE cumulative-state drain — 4 consecutive timer failures, NOTHING delivered, including clean unrelated entries (head-of-line blocking). FIX 1 (quarantine, don't wedge): digest.sh cmd_render's fail-loud pass is now PER-ENTRY. A render-refused ACTIONABLE entry (no hard locator) is DEAD-LETTERED to $STATE_DIR/dead-letter.jsonl + a loud per-entry alarm and EXCLUDED, while the REST of the cumulative set still renders (exit 0). The bad entry is accounted-for, never silently dropped; one malformed entry can no longer wedge the whole drain. FIX 2 (AMENDED — render-tier, NOT class=digest): reconcile.sh's store class is UNCHANGED (non-coalescing) — the naive class=digest was rejected because store §2.3/T2 coalescing would silently collapse distinct enumerations (proven: reconcile R6 goes RED). Instead digest.sh's actionable-classifier now treats locators.reconciled==true (set only by reconcile.sh; not source-forgeable) as ORIENTATION-tier: gate-exempt, rendered as an orientation pointer via _locator_line's digest-class vocabulary. No exit-4, no wedge, no coalescing loss; §2.3/T2/G3-R6 intact. RED-FIRST: new test-wake-digest-quarantine.sh (openssl-independent) proves (a) malformed {kind,id,path,observed_hash} quarantines while a clean sibling still delivers, (b) a reconciled enumeration renders orientation-tier (no exit-4), (c) TWO distinct enumerations both survive as separate orientation renders (anti-collapse — the class=digest silent-drop we rejected), and the preserved per-entry actionable fail-loud. D2/D6 in test-wake-digest-hmac.sh updated from whole-digest exit-4 to per-entry quarantine. Wake manifest bumped 0.6.2 -> 0.6.3 + inventory. Closes #920 Part of #892 Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_0158NZqN2n2ymKFeJAZ4GUCb --- .../mosaic/framework/tools/wake/digest.sh | 147 +++++++++++---- .../mosaic/framework/tools/wake/manifest.txt | 26 ++- .../tools/wake/test-wake-digest-hmac.sh | 112 +++++++----- .../tools/wake/test-wake-digest-quarantine.sh | 173 ++++++++++++++++++ packages/mosaic/package.json | 2 +- 5 files changed, 378 insertions(+), 82 deletions(-) create mode 100755 packages/mosaic/framework/tools/wake/test-wake-digest-quarantine.sh diff --git a/packages/mosaic/framework/tools/wake/digest.sh b/packages/mosaic/framework/tools/wake/digest.sh index cc8f9227..26bb4a6e 100755 --- a/packages/mosaic/framework/tools/wake/digest.sh +++ b/packages/mosaic/framework/tools/wake/digest.sh @@ -26,8 +26,23 @@ # # HARD LOCATORS (§2.1): every actionable claim MUST carry a precise locator # (repo + issue#, a 40-char SHA, or file:anchor) so re-verification is ONE -# targeted call. A missing locator = malformed digest = FAIL-LOUD (non-zero -# exit, NOTHING emitted) — never a silent send. +# targeted call. A missing locator = malformed ACTIONABLE entry = FAIL-LOUD. +# +# #920 (per-entry quarantine — fail-loud WITHOUT head-of-line blocking): a +# render-refused entry (actionable-tier, no hard locator) is QUARANTINED — durably +# DEAD-LETTERED to $STATE_DIR/dead-letter.jsonl + a LOUD per-entry alarm — and +# EXCLUDED from this drain, while the REST of the cumulative set still renders +# (exit 0). The bad entry is accounted-for (never silently dropped); it can no +# longer wedge the whole drain (the live pilot defect: one malformed entry +# exit-4'd the entire cumulative-state drain, delivering NOTHING). +# +# #920 (amended FIX 2 — reconciler enumerations are ORIENTATION-tier): a +# reconciler ENUMERATION carries locators.reconciled==true (set ONLY by +# reconcile.sh). Enumerations are self-orienting STATE-POINTERS, never +# consequential claims, so they are EXEMPT from the actionable hard-locator gate +# and render as ORIENTATION pointers via _locator_line's digest-class vocabulary +# (kind/id/observed_hash/path/…). Their STORE class is left UNCHANGED (non- +# coalescing) so distinct enumerations never collapse (§2.3/T2/G3-R6 intact). # # BOUNDING / INJECTION / SECRETS (§2.1): link-not-inline (pointers + bounded # summary; raw diffs/logs stay at rest). Source free-text is NEVER instruction- @@ -71,8 +86,13 @@ Options: --max-free-text N cap for a quoted untrusted block (default: 200 chars). Exit codes: - 0 digest rendered. - 4 FAIL-LOUD: an actionable claim carried no hard locator (malformed digest). + 0 digest rendered. Any render-refused ACTIONABLE entry (no §2.1 hard locator) + is QUARANTINED — DEAD-LETTERED to $STATE_DIR/dead-letter.jsonl + a loud + per-entry alarm — and EXCLUDED; the rest of the cumulative set still renders + (#920: fail-loud is per-entry, never a whole-drain wedge). Reconciler + enumerations (locators.reconciled==true) render ORIENTATION-tier, gate-exempt. + 2 usage error. + 3 jq is required but missing. Environment: WAKE_STATE_HOME override base state dir (XDG by default). @@ -231,6 +251,68 @@ _free_text() { ' <<<"$1" } +# _actionable_tier LINE — echo "1" iff the entry is ACTIONABLE-tier (subject to +# the §2.1 hard-locator gate and rendered as a CLAIM@seq), else "0". Shared by +# the quarantine pass AND the actionable render loop so both classify identically +# (no entry can render as a CLAIM@seq without having passed the gate). +# +# Actionable-tier = class "actionable" OR any entry whose locators carry a +# `claim` OR a top-level `claim` (a consequential fact) — matching the render +# tier's `.claim // .locators.claim` precedence. +# +# #920 (amended FIX 2): a reconciler ENUMERATION carries locators.reconciled==true +# (set ONLY by reconcile.sh — the detector's locator whitelist +# {kind,id,observed_hash}+{repo,path,anchor,remote,branches} cannot express it, so +# a source cannot forge it). Enumerations are ORIENTATION-tier state-pointers, NOT +# consequential claims, so they are EXEMPT from actionable classification (hence +# from the hard-locator/quarantine gate) and render as orientation pointers via +# _locator_line. Their STORE class is untouched (non-coalescing) so distinct +# enumerations never collapse (§2.3/T2/G3-R6 intact — the class=digest silent-drop +# that was REJECTED). +_actionable_tier() { + local line="$1" loc + loc="$(jq -c '.locators // {}' <<<"$line")" + # Orientation exemption FIRST: a reconciler enumeration is never actionable-tier. + if jq -e '(.reconciled // false) == true' >/dev/null 2>&1 <<<"$loc"; then + printf '0' + return 0 + fi + if [ "$(jq -r '.class // "actionable"' <<<"$line")" = "actionable" ]; then + printf '1' + return 0 + fi + if jq -e 'has("claim") and ((.claim // "") != "")' >/dev/null 2>&1 <<<"$loc"; then + printf '1' + return 0 + fi + if jq -e 'has("claim") and ((.claim // "") != "")' >/dev/null 2>&1 <<<"$line"; then + printf '1' + return 0 + fi + printf '0' +} + +# _quarantine_entry LINE SEQ — QUARANTINE a render-refused entry (#920): append it +# verbatim to the durable dead-letter ledger ($STATE_DIR/dead-letter.jsonl, atomic +# write) and raise a LOUD per-entry fail-loud alarm on stderr. The entry is thereby +# accounted-for (never silently dropped) without wedging the rest of the +# cumulative-state drain. The append is idempotent (a still-pending malformed entry +# is re-drained every timer tick) so the ledger stays bounded. A dead-letter write +# failure is itself alarmed but never aborts the drain — the good entries MUST +# still deliver (availability is the whole point of #920). +_quarantine_entry() { + local line="$1" seq="$2" dlq="$STATE_DIR/dead-letter.jsonl" + if [ ! -f "$dlq" ] || ! grep -qxF "$line" "$dlq" 2>/dev/null; then + local existing + existing="$(cat "$dlq" 2>/dev/null || true)" + if ! printf '%s\n%s\n' "$existing" "$line" | grep -v '^[[:space:]]*$' | _atomic_write "$dlq"; then + echo "digest.sh: FAIL-LOUD QUARANTINE (#920) — malformed ACTIONABLE entry at observed_seq=$seq has no §2.1 hard locator AND the durable dead-letter write to $dlq FAILED. The entry is EXCLUDED from this digest and loudly surfaced here; investigate immediately." >&2 + return 0 + fi + fi + echo "digest.sh: FAIL-LOUD QUARANTINE (#920) — malformed ACTIONABLE entry at observed_seq=$seq has no §2.1 hard locator (repo+issue#/40-char SHA/file:anchor). DEAD-LETTERED to $dlq and EXCLUDED from this digest; the rest of the cumulative set still renders (no head-of-line block). Re-feed the source with a valid hard locator." >&2 +} + cmd_render() { _need_jq local src='store' from_file='' lane="${WAKE_LANE:-}" board_head='' \ @@ -263,35 +345,35 @@ cmd_render() { local observed consumed depth observed="$(_wake_read_int "$STATE_DIR/observed_seq" 0)" consumed="$(_wake_read_int "$STATE_DIR/consumed_seq" 0)" - depth="$(printf '%s\n' "$pending" | grep -c . || true)" - # --- FAIL-LOUD PASS FIRST (§2.1): validate every actionable claim carries a - # hard locator BEFORE emitting a single byte. A malformed digest must never be - # partially sent. Actionable-tier = class "actionable" OR any entry whose - # locators carry a `claim` OR a top-level `claim` (a consequential fact) — - # matching the render tier's `.claim // .locators.claim` precedence, so no - # entry can render as a CLAIM@seq without passing this gate. ------------- - local line class loc is_actionable + # --- QUARANTINE PASS FIRST (§2.1 fail-loud, PER-ENTRY — #920) -------------- + # Partition the cumulative set into the DELIVERABLE set (pending_ok) and the + # render-refused entries. An ACTIONABLE-tier entry (per _actionable_tier) that + # carries NO §2.1 hard locator is QUARANTINED — durably DEAD-LETTERED + a loud + # per-entry alarm — and EXCLUDED, while every OTHER entry still renders (exit 0). + # This preserves fail-loud (the bad entry is accounted-for, never silently + # dropped) WITHOUT letting one malformed entry wedge the whole drain (the live + # #920 head-of-line-blocking defect that exit-4'd the entire cumulative-state + # drain). Reconciler enumerations (locators.reconciled==true) are ORIENTATION- + # tier and gate-exempt, so they pass straight through to the deliverable set. + local line loc seq pending_ok='' quarantined=0 while IFS= read -r line; do [ -n "$line" ] || continue printf '%s' "$line" | jq -e . >/dev/null 2>&1 || continue - class="$(jq -r '.class // "actionable"' <<<"$line")" - loc="$(jq -c '.locators // {}' <<<"$line")" - is_actionable=0 - [ "$class" = "actionable" ] && is_actionable=1 - if jq -e 'has("claim") and ((.claim // "") != "")' >/dev/null 2>&1 <<<"$loc"; then - is_actionable=1 - fi - if jq -e 'has("claim") and ((.claim // "") != "")' >/dev/null 2>&1 <<<"$line"; then - is_actionable=1 - fi - if [ "$is_actionable" = "1" ] && ! _has_hard_locator "$loc"; then - local seq - seq="$(jq -r '.observed_seq // "?"' <<<"$line")" - echo "digest.sh: FAIL-LOUD — malformed digest: actionable claim at observed_seq=$seq has no hard locator (repo+issue#/40-char SHA/file:anchor). Refusing to render (§2.1)." >&2 - exit 4 + if [ "$(_actionable_tier "$line")" = "1" ]; then + loc="$(jq -c '.locators // {}' <<<"$line")" + if ! _has_hard_locator "$loc"; then + seq="$(jq -r '.observed_seq // "?"' <<<"$line")" + _quarantine_entry "$line" "$seq" + quarantined=$((quarantined + 1)) + continue + fi fi + pending_ok="$pending_ok$line"$'\n' done <<<"$pending" + # The deliverable cumulative set is the input MINUS the quarantined entries. + pending="$(printf '%s' "$pending_ok" | grep -v '^[[:space:]]*$' || true)" + depth="$(printf '%s\n' "$pending" | grep -c . || true)" # --- render (all validated) ---------------------------------------------- local n_actionable=0 @@ -327,14 +409,13 @@ cmd_render() { while IFS= read -r line; do [ -n "$line" ] || continue printf '%s' "$line" | jq -e . >/dev/null 2>&1 || continue - local aclass aloc a_is claim seq ft - aclass="$(jq -r '.class // "actionable"' <<<"$line")" + local aloc claim seq ft + # #920: _actionable_tier EXEMPTS reconciler enumerations (reconciled==true) + # so they never render as a CLAIM@seq — they are ORIENTATION-tier pointers + # only. Same classifier as the quarantine pass (single source of truth). + [ "$(_actionable_tier "$line")" = "1" ] || continue aloc="$(jq -c '.locators // {}' <<<"$line")" - a_is=0 - [ "$aclass" = "actionable" ] && a_is=1 claim="$(jq -r '.claim // (.locators.claim) // ""' <<<"$line")" - [ -n "$claim" ] && a_is=1 - [ "$a_is" = "1" ] || continue seq="$(jq -r '.observed_seq // "?"' <<<"$line")" printf ' * seq %s — CLAIM@seq (point-in-time; UNTRUSTED — VERIFY LIVE, do NOT act on this line):\n' "$seq" if [ -n "$claim" ]; then diff --git a/packages/mosaic/framework/tools/wake/manifest.txt b/packages/mosaic/framework/tools/wake/manifest.txt index 20d289f3..f5445f56 100644 --- a/packages/mosaic/framework/tools/wake/manifest.txt +++ b/packages/mosaic/framework/tools/wake/manifest.txt @@ -45,8 +45,26 @@ # pointer carries a usable (soft) locator instead of rendering # empty. Display-only: the ACTIONABLE-tier hard-locator FAIL-LOUD # gate (_has_hard_locator, exit 4) is unchanged. +# 0.6.3 #920 digest.sh drain-quarantine + reconciler-enumeration render tier +# (live wake-pilot finding #6, BLOCKING). (a) PER-ENTRY +# QUARANTINE: a render-refused ACTIONABLE entry (no §2.1 hard +# locator) is now DEAD-LETTERED to $STATE_DIR/dead-letter.jsonl + +# a loud per-entry alarm and EXCLUDED, while the REST of the +# cumulative set still renders (exit 0). Replaces the whole-digest +# exit-4 that let ONE malformed entry wedge the entire drain (head- +# of-line blocking — 4 consecutive live timer failures, nothing +# delivered). Fail-loud is preserved, now per-entry; the bad entry +# is never silently dropped. (b) Reconciler ENUMERATIONS render +# ORIENTATION-tier: an entry whose locators carry reconciled==true +# (set only by reconcile.sh) is EXEMPT from the actionable hard- +# locator gate and renders as an orientation pointer via +# _locator_line's digest-class vocabulary — a RENDER-layer change +# only. reconcile.sh's STORE class is UNCHANGED (non-coalescing), so +# distinct enumerations never collapse (§2.3/T2/G3-R6 intact); the +# rejected class=digest alternative would have silently coalesced +# them. store.sh and reconcile.sh are UNCHANGED by 0.6.3. component=wake -version=0.6.2 +version=0.6.3 # Watch-list schema this component consumes, and the INCLUSIVE range of # schema_version values it supports. A wake-watch-list.json whose schema_version @@ -60,7 +78,11 @@ schema_max=1 # store.sh A2 — three-cursor durable store + drain lib. (W2) # ack.sh A4 — RECEIVED/CONSUMED ack-wrapper (local-write + ship). (W2) # digest.sh A3 — cumulative-state digest renderer (hard locators, -# two-tier trust, injection/secret scrub). (W3) +# two-tier trust, injection/secret scrub). PER-ENTRY +# quarantine: a render-refused entry is dead-lettered + +# alarmed + excluded, the rest still renders (no head-of-line +# block); reconciler enumerations (reconciled==true) render +# ORIENTATION-tier, gate-exempt. (W3, #920) # sign.sh A5 — non-circular HMAC signer (independent wake_id, # load_credentials by-name; fills the hmac placeholder). (W3) # detector.sh A1 — per-host single-instance delta-gated detector daemon diff --git a/packages/mosaic/framework/tools/wake/test-wake-digest-hmac.sh b/packages/mosaic/framework/tools/wake/test-wake-digest-hmac.sh index d6a023b5..20997c7d 100755 --- a/packages/mosaic/framework/tools/wake/test-wake-digest-hmac.sh +++ b/packages/mosaic/framework/tools/wake/test-wake-digest-hmac.sh @@ -7,8 +7,9 @@ # that invariant regresses: # D1 CUMULATIVE-STATE: two still-pending changes both render (not just the # latest delta) — a delta would silently drop the older change. (§2.1) -# D2 HARD-LOCATOR enforcement: an actionable claim with no precise locator -# FAILS LOUD (non-zero exit, nothing emitted). (§2.1) +# D2 HARD-LOCATOR enforcement: an actionable claim with no precise locator is +# QUARANTINED — dead-lettered + alarmed + excluded — never delivered as +# valid (fail-loud PER-ENTRY, #920; the rest of the drain still renders). (§2.1) # D3 TWO-TIER trust: orientation decides the no-op case with ZERO tool calls; # an actionable fact is a CLAIM-TO-VERIFY, never auto-actioned. (§2.1) # D4 SCRUB: a secret-canary + ANSI/bidi/zero-width in SOURCE free-text is @@ -27,7 +28,8 @@ # digest-class entry (the shape detector.sh actually enqueues: # kind/id/observed_hash/remote/path, not repo/issue/sha/file) carries a # non-empty, usable locator — and the ACTIONABLE-tier hard-locator -# FAIL-LOUD (exit 4) is UNCHANGED for a malformed actionable claim. (§2.1) +# FAIL-LOUD is PRESERVED (now per-entry quarantine, #920) for a malformed +# actionable claim. (§2.1) # # Isolated: every test runs against a fresh temp state / credential file. set -uo pipefail @@ -100,60 +102,71 @@ echo "== D1: CUMULATIVE-STATE — two pending changes BOTH render (not just late echo "$out" | grep -q 'pending=2' || fail_msg "D1: cumulative pending count wrong (expected 2)" ) && ok -echo "== D2: HARD-LOCATOR enforcement — actionable claim with no locator FAILS LOUD ==" +echo "== D2: HARD-LOCATOR enforcement — a malformed actionable claim is QUARANTINED (fail-loud PER-ENTRY, #920); never delivered as valid ==" ( - WAKE_STATE_HOME="$(fresh_state d2)" + h="$(fresh_state d2)" + WAKE_STATE_HOME="$h" export WAKE_STATE_HOME unset WAKE_AGENT WAKE_LANE # An actionable claim carrying NO hard locator (no repo+issue / sha / file). "$STORE" enqueue --seq 1 --class actionable --locators '{"claim":"mergeable=true"}' >/dev/null - if out="$("$DIGEST" render 2>/dev/null)"; then - fail_msg "D2: malformed digest (actionable, no locator) must FAIL, but render succeeded" - fi + err="$TMP_ROOT/d2.err" rc=0 - "$DIGEST" render >/dev/null 2>&1 || rc=$? - [ "$rc" -eq 4 ] || fail_msg "D2: expected fail-loud exit 4, got $rc" - # Nothing emitted on stdout when malformed (no silent partial send). - [ -z "${out:-}" ] || fail_msg "D2: a malformed digest emitted body instead of failing loud [$out]" - # A present-but-imprecise sha (not 40 hex) does NOT satisfy the hard locator. - WAKE_STATE_HOME="$(fresh_state d2b)" + out="$("$DIGEST" render 2>"$err")" || rc=$? + # #920: NO longer a whole-digest exit-4 — the malformed entry is QUARANTINED + # (dead-lettered + alarmed + excluded) and the digest STILL renders (exit 0), + # but the unlocated claim is NEVER delivered as a valid CLAIM (fail-loud is + # preserved, now per-entry). The old exit-4 wedged the entire drain (#920). + [ "$rc" -eq 0 ] || fail_msg "D2: a malformed actionable must be quarantined (render exit 0, #920), got $rc" + printf '%s' "$out" | grep -q 'mergeable=true' && fail_msg "D2: an unlocated actionable claim must NOT be delivered as a valid CLAIM" + grep -q 'mergeable=true' "$h/default/dead-letter.jsonl" 2>/dev/null || fail_msg "D2: the malformed claim must be DEAD-LETTERED (fail-loud preserved, per-entry)" + grep -qi 'QUARANTINE' "$err" || fail_msg "D2: a LOUD per-entry alarm must fire for the malformed claim" + + # A present-but-imprecise sha (not 40 hex) does NOT satisfy the hard locator -> + # quarantined, not delivered. + h="$(fresh_state d2b)" + WAKE_STATE_HOME="$h" export WAKE_STATE_HOME "$STORE" enqueue --seq 1 --class actionable --locators '{"claim":"ci=green","sha":"abc123"}' >/dev/null rc=0 - "$DIGEST" render >/dev/null 2>&1 || rc=$? - [ "$rc" -eq 4 ] || fail_msg "D2: imprecise sha (not 40-hex) must not satisfy the hard-locator gate (got exit $rc)" - # The SAME claim WITH a precise 40-hex sha renders fine. - WAKE_STATE_HOME="$(fresh_state d2c)" + out="$("$DIGEST" render 2>/dev/null)" || rc=$? + [ "$rc" -eq 0 ] || fail_msg "D2: imprecise-sha entry must be quarantined (render exit 0), got $rc" + printf '%s' "$out" | grep -q 'ci=green' && fail_msg "D2: imprecise sha (not 40-hex) must not satisfy the hard-locator gate (claim must not deliver)" + grep -q 'ci=green' "$h/default/dead-letter.jsonl" 2>/dev/null || fail_msg "D2: imprecise-sha claim must be dead-lettered" + + # The SAME claim WITH a precise 40-hex sha renders fine (delivered, not quarantined). + h="$(fresh_state d2c)" + WAKE_STATE_HOME="$h" export WAKE_STATE_HOME "$STORE" enqueue --seq 1 --class actionable --locators "$(jq -cn --arg s "$SHA40" '{claim:"ci=green",sha:$s}')" >/dev/null - "$DIGEST" render >/dev/null 2>&1 || fail_msg "D2: a well-located actionable claim must render" - # --- #905: a NON-CANONICAL entry with a TOP-LEVEL `.claim` (store.sh never - # emits this shape; only .locators.claim is canonical) and EMPTY .locators, - # fed via --stdin / --from-file, must ALSO fail loud. The render tier honors - # `.claim // .locators.claim` (renders CLAIM@seq either way), so the - # fail-loud gate must cover top-level .claim too, else a hand-crafted / - # non-canonical caller could bypass the hard-locator gate entirely. - WAKE_STATE_HOME="$(fresh_state d2d)" + out="$("$DIGEST" render 2>/dev/null)" || fail_msg "D2: a well-located actionable claim must render" + printf '%s' "$out" | grep -q "$SHA40" || fail_msg "D2: a well-located actionable claim must be DELIVERED" + [ -s "$h/default/dead-letter.jsonl" ] && fail_msg "D2: a well-located claim must NOT be quarantined" + # --- #905 bypass-prevention (STILL enforced, now via quarantine): a NON- + # CANONICAL entry with a TOP-LEVEL `.claim` (store.sh never emits this shape; + # only .locators.claim is canonical) and EMPTY .locators, fed via --stdin / + # --from-file, must ALSO be gated. The classifier honors `.claim // + # .locators.claim`, so a hand-crafted caller cannot bypass the hard-locator gate + # — the entry is QUARANTINED (not delivered), not silently rendered. + h="$(fresh_state d2d)" + WAKE_STATE_HOME="$h" export WAKE_STATE_HOME - toplevel_claim_entry='{"class":"reaction","claim":"mergeable=true","locators":{}}' - out="" - if out="$(printf '%s\n' "$toplevel_claim_entry" | "$DIGEST" render --stdin 2>/dev/null)"; then - fail_msg "D2(#905): top-level .claim + empty locators must FAIL via --stdin, but render succeeded" - fi - [ -z "${out:-}" ] || fail_msg "D2(#905): --stdin top-level-.claim bypass emitted body instead of failing loud [$out]" + toplevel_claim_entry='{"observed_seq":1,"class":"reaction","claim":"mergeable=true","locators":{}}' rc=0 - printf '%s\n' "$toplevel_claim_entry" | "$DIGEST" render --stdin >/dev/null 2>&1 || rc=$? - [ "$rc" -eq 4 ] || fail_msg "D2(#905): expected fail-loud exit 4 for top-level .claim via --stdin, got $rc" + out="$(printf '%s\n' "$toplevel_claim_entry" | "$DIGEST" render --stdin 2>/dev/null)" || rc=$? + [ "$rc" -eq 0 ] || fail_msg "D2(#905): top-level .claim must be quarantined via --stdin (exit 0), got $rc" + printf '%s' "$out" | grep -q 'mergeable=true' && fail_msg "D2(#905): --stdin top-level-.claim must NOT be delivered (gate not bypassed)" + grep -q 'mergeable=true' "$h/default/dead-letter.jsonl" 2>/dev/null || fail_msg "D2(#905): --stdin top-level .claim must be dead-lettered (gate enforced)" ff="$TMP_ROOT/d2d-entry.jsonl" printf '%s\n' "$toplevel_claim_entry" >"$ff" - out2="" - if out2="$("$DIGEST" render --from-file "$ff" 2>/dev/null)"; then - fail_msg "D2(#905): top-level .claim + empty locators must FAIL via --from-file, but render succeeded" - fi - [ -z "${out2:-}" ] || fail_msg "D2(#905): --from-file top-level-.claim bypass emitted body instead of failing loud [$out2]" + h="$(fresh_state d2e)" + WAKE_STATE_HOME="$h" + export WAKE_STATE_HOME rc=0 - "$DIGEST" render --from-file "$ff" >/dev/null 2>&1 || rc=$? - [ "$rc" -eq 4 ] || fail_msg "D2(#905): expected fail-loud exit 4 for top-level .claim via --from-file, got $rc" + out2="$("$DIGEST" render --from-file "$ff" 2>/dev/null)" || rc=$? + [ "$rc" -eq 0 ] || fail_msg "D2(#905): top-level .claim must be quarantined via --from-file (exit 0), got $rc" + printf '%s' "$out2" | grep -q 'mergeable=true' && fail_msg "D2(#905): --from-file top-level-.claim must NOT be delivered (gate not bypassed)" + grep -q 'mergeable=true' "$h/default/dead-letter.jsonl" 2>/dev/null || fail_msg "D2(#905): --from-file top-level .claim must be dead-lettered (gate enforced)" ) && ok echo "== D3: TWO-TIER — orientation no-op with ZERO tool calls; actionable = claim-to-verify ==" @@ -371,14 +384,21 @@ echo "== D6 (#914b): DIGEST-CLASS LOCATOR THREADING — ORIENTATION pointer carr printf '%s' "$orientline" | grep -qE 'remote=example/repo|id=r1|kind=repo' || fail_msg "D6: digest-class ORIENTATION pointer does not carry a usable locator [$orientline]" - # --- ACTIONABLE-tier hard-locator FAIL-LOUD (exit 4) must be UNCHANGED ------ - WAKE_STATE_HOME="$(fresh_state d6b)" + # --- ACTIONABLE-tier hard-locator FAIL-LOUD must be PRESERVED (now PER-ENTRY + # quarantine, #920): a digest-class ORIENTATION pointer (above) renders soft, + # but a genuine ACTIONABLE claim with no hard locator is still fail-loud — it is + # QUARANTINED (dead-lettered + alarmed + excluded), never delivered as valid. + h="$(fresh_state d6b)" + WAKE_STATE_HOME="$h" export WAKE_STATE_HOME "$STORE" enqueue --seq 1 --class actionable --locators '{"claim":"mergeable=true"}' >/dev/null + err="$TMP_ROOT/d6b.err" rc=0 - "$DIGEST" render >/dev/null 2>&1 || rc=$? - [ "$rc" -eq 4 ] || - fail_msg "D6: actionable claim with no hard locator must still FAIL-LOUD exit 4 (got $rc) — regression in the unrelated hard-locator gate" + out="$("$DIGEST" render 2>"$err")" || rc=$? + [ "$rc" -eq 0 ] || fail_msg "D6: a malformed actionable is now per-entry quarantined (render exit 0, #920), got $rc" + printf '%s' "$out" | grep -q 'mergeable=true' && fail_msg "D6: a malformed actionable claim must NOT be delivered as valid (fail-loud preserved)" + grep -q 'mergeable=true' "$h/default/dead-letter.jsonl" 2>/dev/null || fail_msg "D6: a malformed actionable must be DEAD-LETTERED (fail-loud preserved, per-entry)" + grep -qi 'QUARANTINE' "$err" || fail_msg "D6: a malformed actionable must raise a loud per-entry alarm" ) && ok echo diff --git a/packages/mosaic/framework/tools/wake/test-wake-digest-quarantine.sh b/packages/mosaic/framework/tools/wake/test-wake-digest-quarantine.sh new file mode 100755 index 00000000..61db92d0 --- /dev/null +++ b/packages/mosaic/framework/tools/wake/test-wake-digest-quarantine.sh @@ -0,0 +1,173 @@ +#!/usr/bin/env bash +# test-wake-digest-quarantine.sh — RED-FIRST invariant harness for #920 (EPIC +# #892): per-entry QUARANTINE of a render-refused digest entry (no head-of-line +# blocking) + reconciler ENUMERATIONS render ORIENTATION-tier. +# +# Each test asserts ONE invariant and goes RED against the pre-#920 digest.sh: +# Q1 (a) QUARANTINE + REST-DELIVERS: a malformed `actionable` carrying the live +# pilot's EXACT locator shape {kind,id,path,observed_hash} (no hard +# locator, no reconciled marker) is DEAD-LETTERED + alarmed AND EXCLUDED, +# while a clean valid sibling in the SAME drain still renders (exit 0). +# RED baseline: the old whole-digest exit-4 delivered NOTHING (the live +# head-of-line-blocking wedge). (#920 FIX1) +# Q2 (b) ENUM-AS-ORIENTATION: a reconciler enumeration (locators.reconciled== +# true) renders as an ORIENTATION-tier pointer and does NOT exit-4 / is +# NOT quarantined. RED baseline: reconciled `actionable` + soft locators +# {kind,id,path,observed_hash} -> exit-4. (#920 FIX2) +# Q3 (c) TWO DISTINCT ENUMERATIONS BOTH SURVIVE: two distinct enumerations both +# render as SEPARATE orientation pointers (neither coalesced away) — the +# anti-collapse proof pinning the amended ruling (the REJECTED class=digest +# would have collapsed them to one; store class stays non-coalescing). (#920 FIX2) +# Q4 PRESERVED ACTIONABLE FAIL-LOUD: a genuine malformed ACTIONABLE claim +# (no reconciled marker, no hard locator) is STILL loudly surfaced +# (dead-letter + alarm) and NEVER delivered as if valid — fail-loud is +# preserved, now per-entry. (§2.1) +# Q5 CLAIM-PRECEDENCE GATED: a non-actionable-class entry that carries a +# `claim` (a consequential fact) with no hard locator is STILL gated +# (quarantined), and the reconciled exemption does not leak to it. (§2.1) +# +# Hermetic: feeds controlled JSONL via `digest.sh render --from-file` — NO store, +# NO network, NO openssl (so it runs identically under the CI openssl-mask). +# +# Each test runs in its own (..) subshell for env isolation; the per-subshell +# WAKE_STATE_HOME export is intentional (mirrors test-wake-reconcile.sh). +# shellcheck disable=SC2030,SC2031 +set -uo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +DIGEST="$SCRIPT_DIR/digest.sh" + +command -v jq >/dev/null 2>&1 || { + echo "SKIP: jq not available" >&2 + exit 0 +} + +TMP_ROOT="$(mktemp -d)" +trap 'rm -rf "$TMP_ROOT"' EXIT + +FAILFILE="$TMP_ROOT/failures" +: >"$FAILFILE" +pass=0 +fail_msg() { + echo " FAIL: $*" >&2 + echo "x" >>"$FAILFILE" +} +ok() { pass=$((pass + 1)); } + +# A 40-hex sha = a valid §2.1 hard locator. +SHA40="abcdef0123456789abcdef0123456789abcdef01" + +# fresh_home NAME — a fresh WAKE_STATE_HOME dir, echoed. +fresh_home() { + local d="$TMP_ROOT/$1" + rm -rf "$d" + mkdir -p "$d" + printf '%s' "$d" +} +# dlq HOME — the dead-letter path for the default agent under HOME. +dlq() { printf '%s/default/dead-letter.jsonl' "$1"; } + +echo "== Q1 (a): malformed {kind,id,path,observed_hash} QUARANTINES; clean sibling STILL delivers ==" +( + home="$(fresh_home q1)" + export WAKE_STATE_HOME="$home" + unset WAKE_AGENT + f="$TMP_ROOT/q1.jsonl" + # The live pilot's EXACT malformed shape (no reconciled marker) + a clean sibling. + { + printf '%s\n' '{"observed_seq":1,"class":"actionable","locators":{"kind":"repo","id":"MALFORMED-Q","path":"docs/x.md","observed_hash":"deadbeef"},"emit_ts":1}' + printf '{"observed_seq":2,"class":"actionable","locators":{"sha":"%s","file":"src/a.ts"},"emit_ts":1}\n' "$SHA40" + } >"$f" + err="$TMP_ROOT/q1.err" + out="$("$DIGEST" render --from-file "$f" --agent default 2>"$err")" + rc=$? + [ "$rc" -eq 0 ] || fail_msg "Q1: render must EXIT 0 (per-entry quarantine, not whole-digest exit-4), got rc=$rc" + printf '%s' "$out" | grep -q "$SHA40" || fail_msg "Q1: the clean sibling (sha $SHA40) must STILL be delivered in the same drain [head-of-line block]" + printf '%s' "$out" | grep -q 'MALFORMED-Q' && fail_msg "Q1: the quarantined entry must be EXCLUDED from the rendered digest" + [ -f "$(dlq "$home")" ] || fail_msg "Q1: a durable dead-letter file must be written" + grep -q 'MALFORMED-Q' "$(dlq "$home")" 2>/dev/null || fail_msg "Q1: the malformed entry must be DEAD-LETTERED (accounted-for, not silently dropped)" + grep -qi 'QUARANTINE' "$err" || fail_msg "Q1: a LOUD per-entry alarm must fire on stderr" + grep -q 'observed_seq=1' "$err" || fail_msg "Q1: the alarm must identify the offending entry (observed_seq=1)" +) && ok + +echo "== Q2 (b): reconciler enumeration (reconciled:true) renders ORIENTATION-tier, NO exit-4 ==" +( + home="$(fresh_home q2)" + export WAKE_STATE_HOME="$home" + unset WAKE_AGENT + f="$TMP_ROOT/q2.jsonl" + # A reconciler enumeration: store class actionable (unchanged) + reconciled marker. + printf '%s\n' '{"observed_seq":5,"class":"actionable","locators":{"kind":"repo","id":"ENUM-B","path":"BOARD.md","observed_hash":"cafe1234","reconciled":true},"emit_ts":1}' >"$f" + err="$TMP_ROOT/q2.err" + out="$("$DIGEST" render --from-file "$f" --agent default 2>"$err")" + rc=$? + [ "$rc" -eq 0 ] || fail_msg "Q2: a reconciler enumeration must NOT exit-4 (it is ORIENTATION-tier), got rc=$rc" + printf '%s' "$out" | grep -q 'id=ENUM-B' || fail_msg "Q2: the enumeration must render as an ORIENTATION pointer (id=ENUM-B via _locator_line)" + printf '%s' "$out" | grep -q 'CLAIM@seq' && fail_msg "Q2: an enumeration must NOT render as an ACTIONABLE CLAIM@seq" + [ -s "$(dlq "$home")" ] && fail_msg "Q2: an ORIENTATION-tier enumeration must NOT be quarantined/dead-lettered" + true +) && ok + +echo "== Q3 (c): TWO DISTINCT enumerations BOTH survive as SEPARATE orientation pointers (anti-collapse) ==" +( + home="$(fresh_home q3)" + export WAKE_STATE_HOME="$home" + unset WAKE_AGENT + f="$TMP_ROOT/q3.jsonl" + { + printf '%s\n' '{"observed_seq":6,"class":"actionable","locators":{"kind":"repo","id":"ENUM-C1","path":"a.md","observed_hash":"1111","reconciled":true},"emit_ts":1}' + printf '%s\n' '{"observed_seq":7,"class":"actionable","locators":{"kind":"repo","id":"ENUM-C2","path":"b.md","observed_hash":"2222","reconciled":true},"emit_ts":1}' + } >"$f" + err="$TMP_ROOT/q3.err" + out="$("$DIGEST" render --from-file "$f" --agent default 2>"$err")" + rc=$? + [ "$rc" -eq 0 ] || fail_msg "Q3: two enumerations must render (exit 0), got rc=$rc" + n="$(printf '%s\n' "$out" | grep -c 'id=ENUM-C[12]' || true)" + [ "$n" = "2" ] || fail_msg "Q3: BOTH distinct enumerations must survive as SEPARATE orientation pointers (expected 2, got $n) — neither coalesced away" + printf '%s' "$out" | grep -q 'id=ENUM-C1' || fail_msg "Q3: enumeration ENUM-C1 must be present" + printf '%s' "$out" | grep -q 'id=ENUM-C2' || fail_msg "Q3: enumeration ENUM-C2 must be present" + [ -s "$(dlq "$home")" ] && fail_msg "Q3: enumerations must NOT be quarantined" + true +) && ok + +echo "== Q4: PRESERVED — a genuine malformed ACTIONABLE claim is STILL loud (dead-letter+alarm), never delivered as valid ==" +( + home="$(fresh_home q4)" + export WAKE_STATE_HOME="$home" + unset WAKE_AGENT + f="$TMP_ROOT/q4.jsonl" + printf '%s\n' '{"observed_seq":9,"class":"actionable","locators":{"kind":"board_file","id":"CLAIM-KEEP","observed_hash":"beef"},"emit_ts":1}' >"$f" + err="$TMP_ROOT/q4.err" + out="$("$DIGEST" render --from-file "$f" --agent default 2>"$err")" + rc=$? + [ "$rc" -eq 0 ] || fail_msg "Q4: render must exit 0 (per-entry quarantine), got rc=$rc" + printf '%s' "$out" | grep -q 'CLAIM-KEEP' && fail_msg "Q4: a malformed actionable must NOT be delivered as if valid" + printf '%s' "$out" | grep -q '(none) — no consequential claims pending' || fail_msg "Q4: with the only claim quarantined, the ACTIONABLE section must show (none)" + grep -q 'CLAIM-KEEP' "$(dlq "$home")" 2>/dev/null || fail_msg "Q4: the malformed actionable must be DEAD-LETTERED (loudly surfaced, not silent)" + grep -qi 'QUARANTINE' "$err" || fail_msg "Q4: the malformed actionable must raise a LOUD alarm" +) && ok + +echo "== Q5: claim-precedence gated — a non-actionable-class entry carrying a claim (no hard locator) is STILL quarantined ==" +( + home="$(fresh_home q5)" + export WAKE_STATE_HOME="$home" + unset WAKE_AGENT + f="$TMP_ROOT/q5.jsonl" + # class=digest but locators carry a consequential `claim` -> actionable-tier by + # precedence; no hard locator + no reconciled marker -> must be quarantined. + printf '%s\n' '{"observed_seq":11,"class":"digest","locators":{"claim":"CI is green","kind":"repo","id":"CLAIMY"},"emit_ts":1}' >"$f" + err="$TMP_ROOT/q5.err" + out="$("$DIGEST" render --from-file "$f" --agent default 2>"$err")" + rc=$? + [ "$rc" -eq 0 ] || fail_msg "Q5: render must exit 0, got rc=$rc" + grep -q 'CLAIMY' "$(dlq "$home")" 2>/dev/null || fail_msg "Q5: a claim-carrying entry with no hard locator must be quarantined (claim precedence, §2.1)" + printf '%s' "$out" | grep -q 'CI is green' && fail_msg "Q5: the un-verifiable claim must NOT be delivered" + true +) && ok + +echo +if [ -s "$FAILFILE" ]; then + echo "wake digest-quarantine harness: FAILED ($(grep -c . "$FAILFILE") assertion(s))" >&2 + exit 1 +fi +echo "wake digest-quarantine harness: all invariants passed ($pass groups)" diff --git a/packages/mosaic/package.json b/packages/mosaic/package.json index 7d8e6181..981450d3 100644 --- a/packages/mosaic/package.json +++ b/packages/mosaic/package.json @@ -25,7 +25,7 @@ "lint": "eslint src", "typecheck": "tsc --noEmit", "test": "vitest run --passWithNoTests && pnpm run test:framework-shell", - "test:framework-shell": "python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-install.sh" + "test:framework-shell": "python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-install.sh" }, "dependencies": { "@mosaicstack/brain": "workspace:*",