feat(wake): W5 — synthetic-canary FN-oracle + source-parity reconciler
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful

Adds A6 (fn-oracle.sh) and A7 (reconcile.sh) of the wake/heartbeat canon
(EPIC #892, W5), building on merged W2/W3/W4 via their public APIs only
(store.sh/ack.sh/detector.sh untouched).

A6 fn-oracle.sh — synthetic-canary FN-oracle (§4 vector: FN-rate=0; §7 res5).
  Injects a KNOWN delta at the SOURCE boundary, drives the pipeline through the
  detector's public poll-once (a black box), and renders its verdict SOLELY from
  the terminal store cursor (did a CONSUMED ack cover the canary's observed_seq
  within the per-class SLO?). Off-domain / detector-independent: it never reads a
  detector hash-file or self-report, so a detector that silently DROPS changes —
  including a fully-disabled one at a perfect 0-wake rate — FAILS the oracle
  (the §4/A8 false-negative-blindspot killer). Runs in its own isolated XDG
  namespace, never the operator's live queue. No invented SLO (design law):
  --slo-seconds is required.

A7 reconcile.sh — source-parity reconciler (§4/G3).
  (i) source-coverage PARITY INVENTORY first: a declared source not covered by
      any watch, a dangling reference, an empty lane, or a required_sources
      omission FLAGS — an omitted source cannot make the §4 vector pass
      vacuously (not silently green).
  (ii) periodic full reconcile: enumerate each covered source's current state vs
      observed_seq/inbox => 0 unaccounted; any gap FLAGS. Handles the W4-ratified
      division (detector first-seen-baseline does NOT wake) by ENUMERATING
      pre-existing/startup state into the durable store; inbox-reflected state is
      accounted (no double-enumeration). Fail-loud (G2a) on adapter error.

RED-FIRST tests (wired into test:framework-shell), shellcheck clean:
  test-wake-fn-oracle.sh   O1 healthy FN-rate=0; O2 disabled detector -> FN-DETECTED
    at perfect no-op; O3 CONSUMED-but-late -> FN; O4 off-domain verdict; O5 SLO required.
  test-wake-reconcile.sh   R1 complete inventory; R2 omitted-source vacuous-pass
    prevented; R3 required_sources omission; R4 dangling; R5/R6 pre-existing ->
    flagged + enumerated -> re-run 0; R7 inbox-accounted no-double; R8 G2a fail-loud.

manifest.txt version 0.3.0 -> 0.4.0 + inventory (schema range untouched).
Operator-agnostic (XDG/env only; no operator paths/names/hosts/secrets).

Part of #892

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158NZqN2n2ymKFeJAZ4GUCb
This commit is contained in:
mosaic-coder
2026-07-25 20:29:33 -05:00
parent 5df47e735e
commit 60efd6a01e
6 changed files with 1237 additions and 4 deletions

View File

@@ -0,0 +1,503 @@
#!/usr/bin/env bash
# reconcile.sh — A7 of the wake canon (EPIC #892, W5): the source-parity
# reconciler.
#
# CONTRACT ANCHORS (docs/scratchpads/heartbeat-planning/CONVERGED-DESIGN.md):
# §4/G3 Source parity / reconciliation — the GO condition has TWO ordered
# parts:
# (i) a source-coverage PARITY INVENTORY FIRST: a lane-by-lane
# DECLARED inventory of every operational source the lane
# depends on, so an OMITTED source cannot make the §4 vector pass
# VACUOUSLY. An incomplete / undeclared inventory FLAGS (it must
# NOT silently pass).
# (ii) then a periodic full RECONCILE: enumerate declared+configured
# source state vs observed_seq/inbox => 0 UNACCOUNTED. Any source
# state not reflected in observed_seq/inbox is a gap => FLAG.
# §4 vector { source-parity inventory complete AND reconcile = 0 unaccounted }
# §7-res5 FN-oracle + reconciliation are a real, NON-OPTIONAL operating cost.
# W4 review division: the detector's first-seen-baseline does NOT wake; the
# RECONCILER is what enumerates PRE-EXISTING / startup source state into
# the durable store (at startup and periodically). Without it, state
# that already existed when the detector first saw it (baselined
# silently) would never enter the inbox and would be lost.
#
# SCOPE: calls the PUBLIC APIs of store.sh (cursors/drain/enqueue) only. It does
# NOT reimplement or modify the store, ack, digest, signer, or detector. It
# OBSERVES current source state through the SAME operator adapter contract the
# detector uses (WAKE_DETECTOR_SOURCE_CMD <kind> <id>, def on stdin), with the
# SAME fail-loud (G2a) discipline: a source error / ambiguous-empty is never
# silently "no state".
#
# CONTRACT NOTES (flagged, not silently guessed — see PR body):
# * observed_seq allocation on enumeration: observed_seq is detector-owned
# (§2.4). The store exposes no "allocate next seq" primitive, so the
# reconciler enumerates using seqs drawn from the STORE's authoritative
# observed_seq cursor (§2.4: "observed_seq is authoritative") + 1.. within a
# pass. A dual-allocator collision window with the detector's private
# counter is possible under true concurrency; in the per-host
# single-instance model reconcile and poll are serialised. Unifying on a
# single store-side allocator is tracked as a cross-wave follow-up.
# * "accounted" is judged against observed_seq/inbox (§4/G3 wording) PLUS the
# reconciler's OWN durable reconciled-state ledger — NOT the detector's
# hash-file. Judging by the detector baseline would let a silent first-seen
# baseline swallow a pre-existing obligation (the exact hole G3 closes).
# * enumeration uses the non-coalescible fail-safe class `actionable` (§2.3)
# so two distinct pre-existing obligations can never coalesce into one.
#
# Operator-agnostic: all state via XDG/env; no operator paths/names/secrets.
set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=./_wake-common.sh disable=SC1091
. "$SCRIPT_DIR/_wake-common.sh"
STORE_SH="$SCRIPT_DIR/store.sh"
MANIFEST="$SCRIPT_DIR/manifest.txt"
STATE_DIR="$(wake_state_dir)"
# Reconciler-local state (its durable reconciled-state ledger) lives in its own
# subdir under the store's STATE_DIR so it never collides with store/detector.
RECON_DIR="$STATE_DIR/reconciler"
_need_jq() {
command -v jq >/dev/null 2>&1 || {
echo "reconcile.sh: jq is required" >&2
exit 3
}
}
_hash() {
if command -v sha256sum >/dev/null 2>&1; then
sha256sum | awk '{print $1}'
elif command -v shasum >/dev/null 2>&1; then
shasum -a 256 | awk '{print $1}'
elif command -v openssl >/dev/null 2>&1; then
openssl dgst -sha256 | awk '{print $NF}'
else
echo "reconcile.sh: no sha256 tool (sha256sum/shasum/openssl) available" >&2
exit 3
fi
}
usage() {
cat >&2 <<'EOF'
Usage: reconcile.sh <command> [options]
Commands:
inventory
Part (i) — source-coverage PARITY INVENTORY. Prints a lane-by-lane
declared inventory and FLAGS any incompleteness that could let the §4
vector pass VACUOUSLY:
* a declared operational source (repos/board_files/lane_anchors) that
NO watch covers (an OMITTED source);
* a watch that references a source with no declared definition (dangling);
* a lane with zero sources; an empty watch-list;
* if a lane declares `required_sources`, any id missing from its sources.
Exit 0 IFF the inventory is complete; non-zero (loud) on any flag.
reconcile [--no-enumerate]
Part (ii) — periodic full reconcile. Enumerates each covered source's
CURRENT state and compares it against observed_seq/inbox (+ the
reconciler's reconciled-state ledger). Any source state NOT reflected =>
UNACCOUNTED => FLAG. Unless --no-enumerate, each unaccounted source is
ENUMERATED into the durable store (the startup/pre-existing obligation
path). Exit 0 IFF 0 unaccounted were FOUND this pass.
check
The full §4/G3 gate, side-effect-free: `inventory` AND
`reconcile --no-enumerate`. Exit 0 IFF the inventory is complete AND 0
unaccounted. Use this to evaluate the vector's G3 clause without writing.
Environment:
WAKE_WATCH_LIST Operator watch-list JSON (required).
WAKE_DETECTOR_SOURCE_CMD Source adapter (required for reconcile/check; same
contract as the detector: <cmd> <kind> <id>, def on
stdin; exit!=0 or empty => FAIL LOUD).
WAKE_STATE_HOME/WAKE_AGENT store namespace (see store.sh).
EOF
}
_manifest_val() {
local key="$1"
[ -f "$MANIFEST" ] || return 0
sed -n "s/^${key}=//p" "$MANIFEST" | head -n1 | tr -d '[:space:]'
}
# _load_watchlist — validate path + JSON + shape + Gate B schema range (mirrors
# the detector's fail-loud validation so the reconciler certifies the SAME
# watch-list the detector runs). Echoes validated JSON on stdout.
_load_watchlist() {
_need_jq
local wl="${WAKE_WATCH_LIST:-}"
if [ -z "$wl" ]; then
echo "reconcile.sh: WAKE_WATCH_LIST is not set (no watch-list to reconcile)" >&2
return 2
fi
if [ ! -f "$wl" ]; then
echo "reconcile.sh: watch-list not found: $wl" >&2
return 2
fi
local json
if ! json="$(jq -e . "$wl" 2>/dev/null)"; then
echo "reconcile.sh: watch-list is not valid JSON: $wl" >&2
return 2
fi
if ! printf '%s' "$json" | jq -e 'has("schema_version") and has("watches")' >/dev/null 2>&1; then
echo "reconcile.sh: watch-list missing required 'schema_version' or 'watches'" >&2
return 2
fi
local ver smin smax
ver="$(printf '%s' "$json" | jq -r '.schema_version')"
smin="$(_manifest_val schema_min)"
smax="$(_manifest_val schema_max)"
case "$ver" in
'' | *[!0-9]*)
echo "reconcile.sh: watch-list schema_version must be an integer (got '$ver')" >&2
return 2
;;
esac
if [ -z "$smin" ] || [ -z "$smax" ]; then
echo "reconcile.sh: manifest is missing schema_min/schema_max (cannot validate Gate B)" >&2
return 2
fi
if [ "$ver" -lt "$smin" ] || [ "$ver" -gt "$smax" ]; then
echo "reconcile.sh: FAIL LOUD (Gate B) — watch-list schema_version $ver is OUTSIDE the supported range [$smin, $smax]" >&2
return 2
fi
printf '%s' "$json"
}
# _coll_for_kind KIND — the top-level collection a source kind is defined in.
_coll_for_kind() {
case "$1" in
repo) printf 'repos' ;;
board_file) printf 'board_files' ;;
lane_anchor) printf 'lane_anchors' ;;
*) return 1 ;;
esac
}
# --- part (i): source-coverage parity inventory ----------------------------
cmd_inventory() {
local json
json="$(_load_watchlist)" || exit $?
local flags=0
# An empty watch-list can never certify coverage of anything.
local nwatch
nwatch="$(printf '%s' "$json" | jq -r '.watches | length')"
if [ "${nwatch:-0}" -eq 0 ]; then
echo "reconcile.sh: FLAG (G3 parity) — watch-list declares NO watches; coverage is vacuous." >&2
flags=$((flags + 1))
fi
echo "# source-coverage parity inventory (lane-by-lane)"
# Per-lane declared inventory + empty-lane + required_sources checks.
local nlanes li
nlanes="$(printf '%s' "$json" | jq -r '.watches | length')"
li=0
while [ "$li" -lt "${nlanes:-0}" ]; do
local lane nsrc
lane="$(printf '%s' "$json" | jq -r --argjson i "$li" '.watches[$i].lane // "(unnamed)"')"
nsrc="$(printf '%s' "$json" | jq -r --argjson i "$li" '.watches[$i].sources | length')"
printf ' lane %s: %s declared source(s)\n' "$lane" "${nsrc:-0}"
if [ "${nsrc:-0}" -eq 0 ]; then
echo "reconcile.sh: FLAG (G3 parity) — lane '$lane' declares ZERO sources (empty coverage)." >&2
flags=$((flags + 1))
fi
# required_sources (optional, additive; ignored by the detector). If the
# operator declares the lane's true dependency set here, every id MUST be
# covered by the lane's sources — an OMITTED dependency FLAGS.
local missing
missing="$(printf '%s' "$json" | jq -r --argjson i "$li" '
(.watches[$i].required_sources // []) as $req
| ($req - [ .watches[$i].sources[].id ]) | .[]')"
if [ -n "$missing" ]; then
local m
while IFS= read -r m; do
[ -n "$m" ] || continue
echo "reconcile.sh: FLAG (G3 parity) — lane '$lane' requires source '$m' but it is OMITTED from the lane's watched sources (vacuous-pass prevented)." >&2
flags=$((flags + 1))
done <<EOF
$missing
EOF
fi
li=$((li + 1))
done
# Dangling: every watched (kind,id) MUST have a declared definition.
local pairs kind id
pairs="$(printf '%s' "$json" | jq -r '[ .watches[].sources[] | "\(.kind)\t\(.id)" ] | unique | .[]')"
while IFS=$'\t' read -r kind id; do
[ -n "$kind" ] || continue
local coll def
if ! coll="$(_coll_for_kind "$kind")"; then
echo "reconcile.sh: FLAG (G3 parity) — unknown source kind '$kind' in a watch." >&2
flags=$((flags + 1))
continue
fi
def="$(printf '%s' "$json" | jq -c --arg c "$coll" --arg id "$id" '(.[$c] // []) | map(select(.id == $id)) | .[0] // empty')"
if [ -z "$def" ]; then
echo "reconcile.sh: FLAG (G3 parity) — watch references '$kind/$id' but no such entry is declared in '$coll' (dangling)." >&2
flags=$((flags + 1))
fi
done <<EOF
$pairs
EOF
# Omitted-from-coverage: every DECLARED operational source must be covered by
# at least one watch. A declared-but-unwatched source is exactly the OMITTED
# source that would let the vector pass VACUOUSLY (the detector never checks it).
local coll kinds
kinds="repos:repo board_files:board_file lane_anchors:lane_anchor"
for coll in $kinds; do
local cname ckind ids
cname="${coll%%:*}"
ckind="${coll##*:}"
ids="$(printf '%s' "$json" | jq -r --arg c "$cname" '(.[$c] // [])[].id')"
local sid covered
while IFS= read -r sid; do
[ -n "$sid" ] || continue
covered="$(printf '%s' "$json" | jq -r --arg k "$ckind" --arg id "$sid" \
'[ .watches[].sources[] | select(.kind==$k and .id==$id) ] | length')"
if [ "${covered:-0}" -eq 0 ]; then
echo "reconcile.sh: FLAG (G3 parity) — declared source '$ckind/$sid' is NOT covered by any watch (OMITTED from coverage; vacuous-pass prevented)." >&2
flags=$((flags + 1))
fi
done <<EOF
$ids
EOF
done
if [ "$flags" -ne 0 ]; then
echo "reconcile.sh: source-parity inventory INCOMPLETE — $flags flag(s). The §4 vector's G3 clause does NOT pass." >&2
return 1
fi
echo "reconcile.sh: source-parity inventory COMPLETE (every declared source is covered; no dangling/empty/omitted)."
return 0
}
# --- part (ii): full reconcile (0 unaccounted) -----------------------------
# _scope_anchor ANCHOR (content on stdin) — mirror of the detector's anchor
# scoping so the reconciler hashes the SAME region the detector would. Extract
# from the first line containing the anchor up to the next heading (or EOF).
_scope_anchor() {
local anchor="$1"
awk -v a="$anchor" '
index($0, a) > 0 && !inzone { inzone=1; print; next }
inzone && /^#/ { exit }
inzone { print }
'
}
# _recon_key KIND ID — stable per-source ledger key.
_recon_key() {
printf '%s\037%s' "$1" "$2" | _hash
}
# _observe_source KIND ID DEF — observe current scoped state; echo the hash on
# success (rc 0). rc 1 = FAIL LOUD (source error / ambiguous-empty), same G2a
# discipline as the detector: an error is never silently "no state".
_observe_source() {
local kind="$1" id="$2" def="$3"
local anchor raw rc deftmp
anchor="$(printf '%s' "$def" | jq -r '.anchor // empty')"
mkdir -p "$RECON_DIR"
# shellcheck disable=SC2154
deftmp="$(mktemp "$RECON_DIR/${_wake_tmp_prefix}defXXXXXX")" || return 1
printf '%s' "$def" >"$deftmp"
raw="$("$WAKE_DETECTOR_SOURCE_CMD" "$kind" "$id" <"$deftmp" 2>/dev/null)"
rc=$?
rm -f "$deftmp"
if [ "$rc" -ne 0 ]; then
echo "reconcile.sh: FAIL LOUD (G2a) — source '$kind/$id' errored (adapter exit $rc). Cannot certify parity; not treated as 'no state'." >&2
return 1
fi
if [ -z "$raw" ]; then
echo "reconcile.sh: FAIL LOUD (G2a) — source '$kind/$id' returned AMBIGUOUS-EMPTY (empty-that-might-mean-hidden is never 'no state')." >&2
return 1
fi
local scoped
if [ -n "$anchor" ]; then
scoped="$(printf '%s' "$raw" | _scope_anchor "$anchor")"
if [ -z "$scoped" ]; then
echo "reconcile.sh: FAIL LOUD (G2a) — source '$kind/$id' anchor '$anchor' not present (ambiguous section state)." >&2
return 1
fi
else
scoped="$raw"
fi
printf '%s' "$scoped" | _hash
}
# _inbox_has KIND ID HASH — true iff a pending inbox entry reflects this exact
# source state (observed_seq/inbox is the §4/G3 accounting universe).
_inbox_has() {
local kind="$1" id="$2" h="$3" n
n="$("$STORE_SH" drain 2>/dev/null | jq -s --arg k "$kind" --arg id "$id" --arg h "$h" \
'[ .[] | select(.locators.kind==$k and .locators.id==$id and .locators.observed_hash==$h) ] | length' 2>/dev/null || echo 0)"
[ "${n:-0}" -gt 0 ]
}
cmd_reconcile() {
local enumerate=1
while [ $# -gt 0 ]; do
case "$1" in
--no-enumerate)
enumerate=0
shift
;;
*)
echo "reconcile.sh reconcile: unknown option '$1'" >&2
exit 2
;;
esac
done
local json
json="$(_load_watchlist)" || exit $?
if [ -z "${WAKE_DETECTOR_SOURCE_CMD:-}" ]; then
echo "reconcile.sh: WAKE_DETECTOR_SOURCE_CMD is not set (no adapter to observe source state)" >&2
exit 2
fi
"$STORE_SH" init >/dev/null 2>&1 || true
mkdir -p "$RECON_DIR"
# Seq allocation base: the store's authoritative observed_seq (§2.4).
local obs_base offset
obs_base="$("$STORE_SH" cursors 2>/dev/null | sed -n 's/^observed_seq=//p')"
case "$obs_base" in '' | *[!0-9]*) obs_base=0 ;; esac
offset=0
local pairs kind id failed=0 unaccounted=0 enumerated=0
pairs="$(printf '%s' "$json" | jq -r '[ .watches[].sources[] | "\(.kind)\t\(.id)" ] | unique | .[]')"
if [ -z "$pairs" ]; then
echo "reconcile.sh: watch-list declares no sources under watches[].sources[]" >&2
exit 2
fi
echo "# full reconcile (declared+configured source state vs observed_seq/inbox)"
while IFS=$'\t' read -r kind id; do
[ -n "$kind" ] || continue
local coll def
if ! coll="$(_coll_for_kind "$kind")"; then
echo "reconcile.sh: FAIL LOUD — unknown source kind '$kind' in watch-list" >&2
failed=1
continue
fi
def="$(printf '%s' "$json" | jq -c --arg c "$coll" --arg id "$id" '(.[$c] // []) | map(select(.id == $id)) | .[0] // empty')"
if [ -z "$def" ]; then
echo "reconcile.sh: FAIL LOUD (G3 parity) — watch references '$kind/$id' but no such entry is declared in '$coll'" >&2
failed=1
continue
fi
local curhash key ledger accounted
if ! curhash="$(_observe_source "$kind" "$id" "$def")"; then
failed=1
continue
fi
key="$(_recon_key "$kind" "$id")"
ledger="$RECON_DIR/seen-$key"
accounted=0
# Reflected in the durable inbox (delivered, awaiting consume)?
if _inbox_has "$kind" "$id" "$curhash"; then
accounted=1
# Or already reconciled to this exact state before (durable ledger)?
elif [ -f "$ledger" ] && [ "$(tr -d '[:space:]' <"$ledger")" = "$curhash" ]; then
accounted=1
fi
if [ "$accounted" -eq 1 ]; then
printf ' %s/%s: ACCOUNTED\n' "$kind" "$id"
continue
fi
# UNACCOUNTED: a source state not reflected in observed_seq/inbox => a gap.
unaccounted=$((unaccounted + 1))
printf ' %s/%s: UNACCOUNTED (current state not reflected in observed_seq/inbox)\n' "$kind" "$id" >&2
if [ "$enumerate" -eq 1 ]; then
# ENUMERATE the pre-existing/startup obligation into the durable store.
# Allocate a seq from the store's authoritative observed_seq (see header
# contract note). Class = non-coalescible fail-safe `actionable` (§2.3) so
# distinct pre-existing obligations never collapse together.
offset=$((offset + 1))
local seq locators
seq=$((obs_base + offset))
locators="$(jq -cn --arg kind "$kind" --arg id "$id" --arg hash "$curhash" \
--argjson sdef "$def" \
'{kind:$kind, id:$id, observed_hash:$hash, reconciled:true}
+ ( $sdef | {repo, path, anchor, remote, branches} | with_entries(select(.value != null)) )')"
if "$STORE_SH" enqueue --seq "$seq" --class actionable --locators "$locators" >/dev/null 2>&1; then
printf '%s' "$curhash" | _atomic_write "$ledger"
enumerated=$((enumerated + 1))
printf ' -> enumerated into store as observed_seq=%s (class=actionable)\n' "$seq"
else
echo "reconcile.sh: enumerate FAILED for '$kind/$id' seq $seq" >&2
failed=1
fi
fi
done <<EOF
$pairs
EOF
echo "reconcile.sh: UNACCOUNTED=$unaccounted ENUMERATED=$enumerated"
if [ "$failed" -ne 0 ]; then
echo "reconcile.sh: reconcile FAILED (a source could not be observed/enumerated)." >&2
return 1
fi
if [ "$unaccounted" -ne 0 ]; then
if [ "$enumerate" -eq 1 ]; then
echo "reconcile.sh: FLAG — $unaccounted unaccounted source-state(s) found and enumerated. Re-run to confirm 0." >&2
else
echo "reconcile.sh: FLAG — $unaccounted unaccounted source-state(s) (audit mode; nothing enumerated)." >&2
fi
return 1
fi
echo "reconcile.sh: reconcile CLEAN — 0 unaccounted (every declared source state is reflected in observed_seq/inbox)."
return 0
}
cmd_check() {
# The full §4/G3 gate, side-effect-free.
local rc=0
cmd_inventory || rc=1
cmd_reconcile --no-enumerate || rc=1
if [ "$rc" -eq 0 ]; then
echo "reconcile.sh: G3 GATE PASS — inventory complete AND 0 unaccounted."
else
echo "reconcile.sh: G3 GATE FAIL — inventory incomplete OR unaccounted state present." >&2
fi
return "$rc"
}
main() {
[ $# -ge 1 ] || {
usage
exit 2
}
local cmd="$1"
shift
case "$cmd" in
inventory) cmd_inventory "$@" ;;
reconcile) cmd_reconcile "$@" ;;
check) cmd_check "$@" ;;
-h | --help | help) usage ;;
*)
echo "reconcile.sh: unknown command '$cmd'" >&2
usage
exit 2
;;
esac
}
main "$@"