feat(wake): W5 — synthetic-canary FN-oracle + source-parity reconciler
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
Adds A6 (fn-oracle.sh) and A7 (reconcile.sh) of the wake/heartbeat canon (EPIC #892, W5), building on merged W2/W3/W4 via their public APIs only (store.sh/ack.sh/detector.sh untouched). A6 fn-oracle.sh — synthetic-canary FN-oracle (§4 vector: FN-rate=0; §7 res5). Injects a KNOWN delta at the SOURCE boundary, drives the pipeline through the detector's public poll-once (a black box), and renders its verdict SOLELY from the terminal store cursor (did a CONSUMED ack cover the canary's observed_seq within the per-class SLO?). Off-domain / detector-independent: it never reads a detector hash-file or self-report, so a detector that silently DROPS changes — including a fully-disabled one at a perfect 0-wake rate — FAILS the oracle (the §4/A8 false-negative-blindspot killer). Runs in its own isolated XDG namespace, never the operator's live queue. No invented SLO (design law): --slo-seconds is required. A7 reconcile.sh — source-parity reconciler (§4/G3). (i) source-coverage PARITY INVENTORY first: a declared source not covered by any watch, a dangling reference, an empty lane, or a required_sources omission FLAGS — an omitted source cannot make the §4 vector pass vacuously (not silently green). (ii) periodic full reconcile: enumerate each covered source's current state vs observed_seq/inbox => 0 unaccounted; any gap FLAGS. Handles the W4-ratified division (detector first-seen-baseline does NOT wake) by ENUMERATING pre-existing/startup state into the durable store; inbox-reflected state is accounted (no double-enumeration). Fail-loud (G2a) on adapter error. RED-FIRST tests (wired into test:framework-shell), shellcheck clean: test-wake-fn-oracle.sh O1 healthy FN-rate=0; O2 disabled detector -> FN-DETECTED at perfect no-op; O3 CONSUMED-but-late -> FN; O4 off-domain verdict; O5 SLO required. test-wake-reconcile.sh R1 complete inventory; R2 omitted-source vacuous-pass prevented; R3 required_sources omission; R4 dangling; R5/R6 pre-existing -> flagged + enumerated -> re-run 0; R7 inbox-accounted no-double; R8 G2a fail-loud. manifest.txt version 0.3.0 -> 0.4.0 + inventory (schema range untouched). Operator-agnostic (XDG/env only; no operator paths/names/hosts/secrets). Part of #892 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0158NZqN2n2ymKFeJAZ4GUCb
This commit is contained in:
mosaic-coder
352
packages/mosaic/framework/tools/wake/fn-oracle.sh
Executable file
352
packages/mosaic/framework/tools/wake/fn-oracle.sh
Executable file
@@ -0,0 +1,352 @@
|
||||
#!/usr/bin/env bash
|
||||
# fn-oracle.sh — A6 of the wake canon (EPIC #892, W5): the FN-oracle /
|
||||
# synthetic-canary.
|
||||
#
|
||||
# CONTRACT ANCHORS (docs/scratchpads/heartbeat-planning/CONVERGED-DESIGN.md):
|
||||
# §4 vector { synthetic-canary FN-rate = 0 } — the oracle injects a KNOWN
|
||||
# synthetic-canary delta into the wake pipeline and ASSERTS it
|
||||
# reaches CONSUMED within its per-class SLO; it measures the
|
||||
# FALSE-NEGATIVE rate. FN-rate MUST be 0 for the timer to retire.
|
||||
# §4/§7-res5 The FN-oracle is a real, NON-OPTIONAL operating cost — the price
|
||||
# of being allowed to retire the fixed timer, not a free extra.
|
||||
# §1.4 Framework component, operator-agnostic (~/.config/mosaic/tools/).
|
||||
#
|
||||
# OFF-DOMAIN / DETECTOR-INDEPENDENT (the §4/A8 false-negative-blindspot killer):
|
||||
# The injection and the CONSUMED-assertion do NOT depend on the detector's
|
||||
# own hashing / poll internals. The oracle injects a delta at the SOURCE
|
||||
# boundary (the bytes an operator adapter reports), drives the pipeline
|
||||
# through the detector's PUBLIC API only (poll-once — a black box), and then
|
||||
# renders its verdict SOLELY from the terminal store/ack state (did a CONSUMED
|
||||
# ack advance the cursor to cover the canary's observed_seq?). It never reads a
|
||||
# detector hash-file, poll counter, or self-report. CONSEQUENCE: a detector
|
||||
# that silently DROPS changes — including a FULLY-DISABLED detector at a
|
||||
# "perfect" no-op rate (0 wakes/day, which looks great on the wakes metric) —
|
||||
# still FAILS the oracle, because the canary never reaches CONSUMED. That is
|
||||
# the whole point: the success metric is a VECTOR, not a wake-count.
|
||||
#
|
||||
# ISOLATION: the probe runs in its OWN XDG state namespace (WAKE_ORACLE_HOME),
|
||||
# NOT the operator's live queue, so a synthetic canary never delivers a fake
|
||||
# wake to a real agent. It exercises the REAL detector.sh / store.sh / ack.sh
|
||||
# CODE paths against isolated synthetic data — the standard synthetic-canary
|
||||
# shape (synthetic transaction, real code, isolated data).
|
||||
#
|
||||
# SCOPE: calls the PUBLIC APIs of detector.sh (poll-once), store.sh
|
||||
# (cursors/drain), and ack.sh (received/consumed). It does NOT reimplement or
|
||||
# modify any of them.
|
||||
#
|
||||
# Operator-agnostic: all state via XDG/env; no operator paths/names/secrets.
|
||||
set -uo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
# shellcheck source=./_wake-common.sh disable=SC1091
|
||||
. "$SCRIPT_DIR/_wake-common.sh"
|
||||
|
||||
DETECTOR_SH="$SCRIPT_DIR/detector.sh"
|
||||
STORE_SH="$SCRIPT_DIR/store.sh"
|
||||
ACK_SH="$SCRIPT_DIR/ack.sh"
|
||||
|
||||
# Isolated oracle state root (XDG). NEVER the operator's live wake queue.
|
||||
ORACLE_HOME="${WAKE_ORACLE_HOME:-${XDG_STATE_HOME:-$HOME/.local/state}/mosaic/wake/fn-oracle}"
|
||||
# The canary's own store namespace lives UNDER the oracle root.
|
||||
CANARY_STATE_HOME="$ORACLE_HOME/canary-store"
|
||||
CANARY_AGENT="canary"
|
||||
CANARY_SRC="$ORACLE_HOME/canary.src"
|
||||
CANARY_ADAPTER="$ORACLE_HOME/canary-adapter.sh"
|
||||
CANARY_WATCHLIST="$ORACLE_HOME/canary-watch-list.json"
|
||||
METRICS="$ORACLE_HOME/metrics.jsonl"
|
||||
|
||||
_need_jq() {
|
||||
command -v jq >/dev/null 2>&1 || {
|
||||
echo "fn-oracle.sh: jq is required" >&2
|
||||
exit 3
|
||||
}
|
||||
}
|
||||
|
||||
usage() {
|
||||
cat >&2 <<'EOF'
|
||||
Usage: fn-oracle.sh run --slo-seconds N [options]
|
||||
|
||||
Commands:
|
||||
run --slo-seconds N [--count K] [--class C]
|
||||
Run K synthetic-canary cycles (default 1). Each cycle injects a KNOWN
|
||||
delta at the source boundary, drives the pipeline through the detector's
|
||||
public poll-once, then asserts the canary reaches CONSUMED within the
|
||||
per-class SLO (N seconds). Emits per-canary PASS/FN-DETECTED, the
|
||||
FN-rate metric, and an overall VERDICT. Exit 0 IFF FN-rate == 0.
|
||||
status
|
||||
Print the tail of the recorded metrics ledger.
|
||||
|
||||
Options:
|
||||
--slo-seconds N REQUIRED for run. The per-class SLO (operator-tuned; symbolic
|
||||
tiers per §4 — no default is invented). event->CONSUMED must
|
||||
land within N seconds or the canary counts as a false-negative.
|
||||
--count K Number of canary cycles this run (default 1).
|
||||
--class C Wake class the canary flows as (default digest — the primary
|
||||
machine-wake path §4 certifies). digest|actionable|human.
|
||||
|
||||
Environment:
|
||||
WAKE_ORACLE_HOME Isolated oracle state root (XDG by default). NEVER the
|
||||
operator's live queue.
|
||||
WAKE_ORACLE_DETECTOR_CMD Command the oracle runs to advance ONE observe cycle
|
||||
of the pipeline (default: detector.sh poll-once). The
|
||||
canary env (watch-list/adapter/state) is exported to
|
||||
it. Pluggable so a disabled/dropping detector can be
|
||||
exercised (it MUST then be caught as FN-DETECTED).
|
||||
WAKE_ORACLE_SLO_SECONDS Fallback for --slo-seconds.
|
||||
EOF
|
||||
}
|
||||
|
||||
# --- one-time synthetic-canary scaffold (idempotent) -----------------------
|
||||
# The oracle SUPPLIES its own source adapter + watch-list so it needs no
|
||||
# operator config and stays fully self-contained. The adapter simply echoes the
|
||||
# current canary source bytes — the synthetic "source of truth" the oracle
|
||||
# controls end-to-end.
|
||||
_write_scaffold() {
|
||||
local class="$1"
|
||||
mkdir -p "$ORACLE_HOME"
|
||||
|
||||
cat >"$CANARY_ADAPTER" <<EOF
|
||||
#!/usr/bin/env bash
|
||||
# Synthetic-canary source adapter (fn-oracle). Reports the current canary bytes.
|
||||
set -u
|
||||
[ -f "$CANARY_SRC" ] && cat "$CANARY_SRC"
|
||||
exit 0
|
||||
EOF
|
||||
chmod +x "$CANARY_ADAPTER"
|
||||
|
||||
# A minimal, schema-valid watch-list with EXACTLY ONE synthetic source, no
|
||||
# anchor. One source in an isolated store => any enqueue is unambiguously the
|
||||
# canary. The class is carried so the canary flows on the class under test.
|
||||
jq -n --arg class "$class" '{
|
||||
schema_version: 1,
|
||||
repos: [ { id: "fn-canary", remote: "synthetic/fn-oracle-canary", class: $class } ],
|
||||
watches: [ { lane: "fn-oracle", sources: [ { kind: "repo", id: "fn-canary" } ] } ]
|
||||
}' >"$CANARY_WATCHLIST"
|
||||
}
|
||||
|
||||
# _canary_env — export the isolated canary namespace + adapter + watch-list for
|
||||
# a child detector/store/ack invocation.
|
||||
_canary_env() {
|
||||
export WAKE_STATE_HOME="$CANARY_STATE_HOME"
|
||||
export WAKE_AGENT="$CANARY_AGENT"
|
||||
export WAKE_WATCH_LIST="$CANARY_WATCHLIST"
|
||||
export WAKE_DETECTOR_SOURCE_CMD="$CANARY_ADAPTER"
|
||||
}
|
||||
|
||||
# _store_observed / _store_consumed — read the terminal cursors (public API).
|
||||
_store_observed() { "$STORE_SH" cursors 2>/dev/null | sed -n 's/^observed_seq=//p'; }
|
||||
_store_consumed() { "$STORE_SH" cursors 2>/dev/null | sed -n 's/^consumed_seq=//p'; }
|
||||
|
||||
# _drive_pipeline — advance ONE observe cycle. Black-box: whatever
|
||||
# WAKE_ORACLE_DETECTOR_CMD is (default: the real detector's public poll-once).
|
||||
# A source failure inside the detector is loud there; the oracle judges only the
|
||||
# terminal state, so a drop of ANY kind surfaces as an unmet CONSUMED.
|
||||
_drive_pipeline() {
|
||||
local cmd="${WAKE_ORACLE_DETECTOR_CMD:-"$DETECTOR_SH poll-once"}"
|
||||
sh -c "$cmd" >/dev/null 2>&1 || true
|
||||
}
|
||||
|
||||
# _seat_baseline — seat the source's first-seen baseline ONCE (detector
|
||||
# first-seen is silent by design). Every subsequent canary token is then a
|
||||
# genuine DELTA off the prior token, so no spurious baseline wakes are produced.
|
||||
_seat_baseline() {
|
||||
_canary_env
|
||||
printf '%s' "FN-ORACLE-BASELINE-$$-$(date +%s)" >"$CANARY_SRC"
|
||||
_drive_pipeline
|
||||
}
|
||||
|
||||
# --- one canary cycle -------------------------------------------------------
|
||||
# Returns 0 = PASS (reached CONSUMED within SLO), 1 = FN-DETECTED.
|
||||
# Prints a one-line verdict for the cycle.
|
||||
_run_one() {
|
||||
local idx="$1" slo="$2" class="$3"
|
||||
_canary_env
|
||||
|
||||
local nonce token inject_ts
|
||||
nonce="$$-$idx-$(date +%s%N 2>/dev/null || date +%s)"
|
||||
token="FN-ORACLE-CANARY-$nonce"
|
||||
|
||||
local obs_before
|
||||
obs_before="$(_store_observed)"
|
||||
case "$obs_before" in '' | *[!0-9]*) obs_before=0 ;; esac
|
||||
|
||||
# INJECT the known delta at the source boundary + stamp the clock.
|
||||
inject_ts="$(date +%s)"
|
||||
printf '%s' "$token" >"$CANARY_SRC"
|
||||
|
||||
# Drive the pipeline (through the detector's public API — a black box).
|
||||
_drive_pipeline
|
||||
|
||||
local obs_after
|
||||
obs_after="$(_store_observed)"
|
||||
case "$obs_after" in '' | *[!0-9]*) obs_after=0 ;; esac
|
||||
|
||||
# ---- OFF-DOMAIN verdict, part 1: was the delta OBSERVED at all? ----------
|
||||
# In an isolated single-source store, any observed_seq advance is the canary.
|
||||
# NO advance => the pipeline (a dropping / disabled detector) swallowed a KNOWN
|
||||
# real change => FALSE NEGATIVE. This is the killer: a perfect no-op rate does
|
||||
# not rescue a detector that fails to see a real delta.
|
||||
if [ "$obs_after" -le "$obs_before" ]; then
|
||||
printf ' canary %s: FN-DETECTED (injected delta never OBSERVED — pipeline dropped a known change; observed_seq %s unchanged)\n' \
|
||||
"$idx" "$obs_before" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
# The delta was observed and durably enqueued. Simulate the consumer contract
|
||||
# (§2.2): deliver (drain), RECEIVED-ack, capture, then CONSUMED-ack the
|
||||
# contiguous prefix. All via the PUBLIC ack/store API.
|
||||
local delivered
|
||||
delivered="$("$STORE_SH" drain 2>/dev/null | grep -c '.' || echo 0)"
|
||||
if [ "${delivered:-0}" -lt 1 ]; then
|
||||
printf ' canary %s: FN-DETECTED (observed but NOT delivered — drain returned nothing to consume)\n' "$idx" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
"$ACK_SH" received --wake-id "fn-oracle-$nonce" >/dev/null 2>&1 || true
|
||||
# CONSUMED over the contiguous prefix up to the canary's observed_seq.
|
||||
"$ACK_SH" consumed --upto "$obs_after" --no-sync >/dev/null 2>&1 || true
|
||||
|
||||
# ---- OFF-DOMAIN verdict, part 2: did it reach CONSUMED, within SLO? ------
|
||||
local consumed elapsed now
|
||||
consumed="$(_store_consumed)"
|
||||
case "$consumed" in '' | *[!0-9]*) consumed=0 ;; esac
|
||||
now="$(date +%s)"
|
||||
elapsed=$((now - inject_ts))
|
||||
|
||||
if [ "$consumed" -lt "$obs_after" ]; then
|
||||
printf ' canary %s: FN-DETECTED (delivered but consumed_seq=%s never reached the canary observed_seq=%s)\n' \
|
||||
"$idx" "$consumed" "$obs_after" >&2
|
||||
return 1
|
||||
fi
|
||||
if [ "$elapsed" -gt "$slo" ]; then
|
||||
printf ' canary %s: FN-DETECTED (reached CONSUMED but in %ss > per-class SLO %ss)\n' \
|
||||
"$idx" "$elapsed" "$slo" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
printf ' canary %s: PASS (event->CONSUMED in %ss <= SLO %ss; observed_seq=%s consumed_seq=%s class=%s)\n' \
|
||||
"$idx" "$elapsed" "$slo" "$obs_after" "$consumed" "$class"
|
||||
return 0
|
||||
}
|
||||
|
||||
cmd_run() {
|
||||
_need_jq
|
||||
local slo="${WAKE_ORACLE_SLO_SECONDS:-}" count=1 class="digest"
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--slo-seconds)
|
||||
slo="${2:-}"
|
||||
shift 2
|
||||
;;
|
||||
--count)
|
||||
count="${2:-}"
|
||||
shift 2
|
||||
;;
|
||||
--class)
|
||||
class="${2:-}"
|
||||
shift 2
|
||||
;;
|
||||
*)
|
||||
echo "fn-oracle.sh run: unknown option '$1'" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# No invented numeric SLO (design law): the per-class SLO MUST be supplied.
|
||||
case "$slo" in
|
||||
'' | *[!0-9]*)
|
||||
echo "fn-oracle.sh run: --slo-seconds (or WAKE_ORACLE_SLO_SECONDS) is REQUIRED and must be a non-negative integer (per-class SLO is operator-tuned; no default is invented)" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
case "$count" in
|
||||
'' | *[!0-9]* | 0)
|
||||
echo "fn-oracle.sh run: --count must be a positive integer" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
case "$class" in
|
||||
digest | actionable | human) : ;;
|
||||
*)
|
||||
echo "fn-oracle.sh run: --class must be one of digest|actionable|human" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
|
||||
# Fresh isolated probe state each run => a deterministic baseline. This is the
|
||||
# oracle's OWN synthetic store, never the operator's live queue.
|
||||
rm -rf "$CANARY_STATE_HOME"
|
||||
mkdir -p "$ORACLE_HOME"
|
||||
_write_scaffold "$class"
|
||||
|
||||
echo "fn-oracle: synthetic-canary run (count=$count, class=$class, per-class SLO=${slo}s)"
|
||||
_seat_baseline # one-time first-seen baseline (silent by design)
|
||||
local fn=0 i
|
||||
i=1
|
||||
while [ "$i" -le "$count" ]; do
|
||||
if _run_one "$i" "$slo" "$class"; then
|
||||
:
|
||||
else
|
||||
fn=$((fn + 1))
|
||||
fi
|
||||
i=$((i + 1))
|
||||
done
|
||||
|
||||
# FN-rate metric. §4 requires FN-rate == 0.
|
||||
local rate verdict rc
|
||||
if [ "$fn" -eq 0 ]; then
|
||||
verdict="PASS"
|
||||
rc=0
|
||||
else
|
||||
verdict="FN-DETECTED"
|
||||
rc=1
|
||||
fi
|
||||
# Rational FN-rate rendered without bc (portable): integer numerator/denominator
|
||||
# plus a scaled decimal.
|
||||
rate="$(awk -v f="$fn" -v n="$count" 'BEGIN { printf "%.4f", (n>0? f/n : 0) }')"
|
||||
|
||||
local rec
|
||||
rec="$(jq -cn \
|
||||
--argjson ts "$(date +%s)" \
|
||||
--arg class "$class" \
|
||||
--argjson slo "$slo" \
|
||||
--argjson count "$count" \
|
||||
--argjson fn "$fn" \
|
||||
--arg rate "$rate" \
|
||||
--arg verdict "$verdict" \
|
||||
'{ts:$ts, class:$class, slo_seconds:$slo, count:$count, fn:$fn, fn_rate:($rate|tonumber), verdict:$verdict}')"
|
||||
{ cat "$METRICS" 2>/dev/null; printf '%s\n' "$rec"; } | grep -v '^[[:space:]]*$' >"$METRICS.tmp" 2>/dev/null || true
|
||||
mv -f "$METRICS.tmp" "$METRICS" 2>/dev/null || true
|
||||
|
||||
echo "fn-oracle: FN-RATE = $fn/$count = $rate (class=$class, SLO=${slo}s)"
|
||||
echo "fn-oracle: VERDICT = $verdict"
|
||||
[ "$rc" -eq 0 ] || echo "fn-oracle: §4 vector FAILS — synthetic-canary FN-rate must be 0 to retire the timer." >&2
|
||||
return "$rc"
|
||||
}
|
||||
|
||||
cmd_status() {
|
||||
echo "# fn-oracle metrics (tail)"
|
||||
tail -n 10 "$METRICS" 2>/dev/null || echo "(no runs recorded)"
|
||||
}
|
||||
|
||||
main() {
|
||||
[ $# -ge 1 ] || {
|
||||
usage
|
||||
exit 2
|
||||
}
|
||||
local cmd="$1"
|
||||
shift
|
||||
case "$cmd" in
|
||||
run) cmd_run "$@" ;;
|
||||
status) cmd_status "$@" ;;
|
||||
-h | --help | help) usage ;;
|
||||
*)
|
||||
echo "fn-oracle.sh: unknown command '$cmd'" >&2
|
||||
usage
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Reference in New Issue
Block a user