feat(wake): W6 — off-host dead-man beacon + pluggable alarm-sink adapter
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
A8 of the wake canon (EPIC #892, W6): the independent liveness leg. beacon.sh: - MONOTONIC EMITTER (emit): strictly-increasing seq + emit_ts each cycle, shipped off-host via a pluggable sink adapter. The primitive the detector run-loop calls each poll (§1.3). - DEAD-MAN / ABSENCE alarm (record + check): the off-host monitor's receiver and absence check. A missing/stale-past-SLO beacon FIRES the alarm and ROUTES it to a human/other-host, depending on nothing the dying host does (§1.3/§4-G1). - PLUGGABLE alarm-sink / beacon-sink ADAPTER INTERFACE: the framework ships the emitter + interface; the operator owns the target endpoint, resolved BY NAME (load_credentials), never inlined (§1.4). FAIL-CLOSED: unconfigured OR unreachable target FAILS LOUD — no silent no-alarm host (§4-G2a). - DEGRADED MODES (honest, not silent): a same-host sibling is REJECTED as non-independent (shares user-manager/host/sender/socket/session); an isolated host degrades to a FLAGGED different-supervision-root beacon; capture-pane is a liveness HINT only (§1.3). detector.sh: a single minimal, opt-in W6 emit seam in the run-loop (inert unless the operator wires WAKE_BEACON_SINK_CMD; a beacon emit failure never kills the loop but is loud). No store/ack/digest/sign code touched. test-wake-beacon.sh: RED-FIRST harness (10 invariant groups), wired into test:framework-shell. manifest.txt bumped 0.4.0 -> 0.5.0 + inventory. Part of #892 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0158NZqN2n2ymKFeJAZ4GUCb
This commit is contained in:
mosaic-coder
414
packages/mosaic/framework/tools/wake/beacon.sh
Executable file
414
packages/mosaic/framework/tools/wake/beacon.sh
Executable file
@@ -0,0 +1,414 @@
|
||||
#!/usr/bin/env bash
|
||||
# beacon.sh — A8 of the wake canon (EPIC #892, W6): the off-host DEAD-MAN
|
||||
# liveness BEACON emitter + the pluggable ALARM-SINK adapter + the beacon-ABSENCE
|
||||
# alarm (the check an off-host monitor runs).
|
||||
#
|
||||
# CONTRACT ANCHORS (docs/scratchpads/heartbeat-planning/CONVERGED-DESIGN.md):
|
||||
# §1.3 Independent liveness leg — off-host dead-man beacon. Liveness is SPLIT
|
||||
# from work-triggering: the detector emits a MONOTONIC beacon each cycle
|
||||
# to existing OFF-HOST monitoring, and the alarm fires on beacon
|
||||
# ABSENCE — depending on NOTHING the dying component must actively do.
|
||||
# A same-host sibling is CONCEDED NOT independent (shares user-manager,
|
||||
# host, sender, socket, session — sol A1/A8/G9) and is REJECTED here. A
|
||||
# single ISOLATED host degrades to a weaker DIFFERENT-SUPERVISION-ROOT
|
||||
# beacon, FLAGGED as such — never silently pretending full independence.
|
||||
# `capture-pane` is a liveness HINT ONLY; readiness is proven solely by
|
||||
# the RECEIVED/CONSUMED acks (W2/W3), never by a pane scrape.
|
||||
# §4 G1 Off-host dead-man response: the beacon-absence alarm is proven to FIRE
|
||||
# AND ROUTE to a human/other-host within its SLO.
|
||||
# §4 G2a Fail-loud semantics: an unconfigured OR unreachable target FAILS LOUD
|
||||
# (no silent no-alarm host).
|
||||
# §7-res6 The only real bound on a stuck drainer is an INDEPENDENT
|
||||
# missing-RECEIVED/CONSUMED escalation through the off-host observer and
|
||||
# its SLO — the same independent leg this tool provides.
|
||||
#
|
||||
# FRAMEWORK / OPERATOR BOUNDARY (§1.4, the operator-agnostic framework/operator split):
|
||||
# FRAMEWORK (this file) ships:
|
||||
# - the monotonic beacon EMITTER (`emit`),
|
||||
# - the off-host monitor's RECEIVER + ABSENCE alarm (`record`, `check`),
|
||||
# - a pluggable ALARM-SINK / BEACON-SINK ADAPTER INTERFACE.
|
||||
# OPERATOR owns:
|
||||
# - the TARGET ENDPOINT (off-host monitor address, alarm route). It is a
|
||||
# command the operator wires; that command resolves its address/credential
|
||||
# BY NAME via `load_credentials`, NEVER inlined into this framework file.
|
||||
# W7 (installer, OUT OF SCOPE here) WIRES + install-validates the target. W6
|
||||
# provides the emitter + adapter interface + absence-alarm + the FAIL-LOUD
|
||||
# PRIMITIVE that W7's validation calls.
|
||||
#
|
||||
# ADAPTER INTERFACE (the pluggable seam — operator supplies the command):
|
||||
# WAKE_BEACON_SINK_CMD emit ships the beacon record (JSON on stdin) via
|
||||
# `sh -c "$WAKE_BEACON_SINK_CMD"`. The command is the
|
||||
# transport to the OFF-HOST monitor; it resolves its
|
||||
# endpoint by-name (load_credentials) internally. A
|
||||
# non-zero exit = UNREACHABLE target => FAIL LOUD. Unset
|
||||
# = UNCONFIGURED target => FAIL LOUD (no silent no-alarm
|
||||
# host).
|
||||
# WAKE_ALARM_SINK_CMD check routes the absence ALARM (JSON on stdin) via
|
||||
# `sh -c "$WAKE_ALARM_SINK_CMD"` to a human/other-host
|
||||
# (G1). Same fail-closed contract: unset OR non-zero
|
||||
# exit => FAIL LOUD.
|
||||
# WAKE_BEACON_INDEPENDENCE the operator DECLARES the independence class of the
|
||||
# wired sink (W7 install-validates it is truthful):
|
||||
# off-host -> full independence.
|
||||
# different-supervision-root -> DEGRADED, FLAGGED.
|
||||
# same-host-sibling -> REJECTED (not independent).
|
||||
#
|
||||
# Operator-agnostic: all state via XDG/env; no operator paths/names/secrets/hosts.
|
||||
set -uo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
# shellcheck source=./_wake-common.sh disable=SC1091
|
||||
. "$SCRIPT_DIR/_wake-common.sh"
|
||||
|
||||
STATE_DIR="$(wake_state_dir)"
|
||||
# Beacon state (emitter counter + the off-host monitor's received store) lives
|
||||
# UNDER the wake STATE_DIR in its own subdir so it never collides with the
|
||||
# store's cursor/inbox files.
|
||||
BEACON_DIR="$STATE_DIR/beacon"
|
||||
SEQ_FILE="$BEACON_DIR/seq"
|
||||
# The off-host monitor's RECEIVED store: the last (highest-seq) beacon this
|
||||
# monitor has ingested for the host. `check` reads it; `record` writes it. In a
|
||||
# real deployment this lives on the MONITOR host, fed by the sink transport; the
|
||||
# default keeps the primitive self-contained + testable.
|
||||
RECEIVED_FILE="${WAKE_BEACON_RECEIVED:-$BEACON_DIR/received.json}"
|
||||
|
||||
_need_jq() {
|
||||
command -v jq >/dev/null 2>&1 || {
|
||||
echo "beacon.sh: jq is required" >&2
|
||||
exit 3
|
||||
}
|
||||
}
|
||||
|
||||
usage() {
|
||||
cat >&2 <<'EOF'
|
||||
Usage: beacon.sh <command> [options]
|
||||
|
||||
Commands:
|
||||
emit HOST side. Mint the next MONOTONIC beacon (strictly
|
||||
increasing seq + emit_ts) and SHIP it off-host via the
|
||||
pluggable sink adapter (WAKE_BEACON_SINK_CMD). This is
|
||||
the primitive the detector's run-loop calls each poll.
|
||||
FAIL-CLOSED: an UNCONFIGURED sink (unset) or an
|
||||
UNREACHABLE sink (non-zero exit) FAILS LOUD — never a
|
||||
silent no-alarm host. A same-host-sibling independence
|
||||
declaration is REJECTED; a different-supervision-root
|
||||
one is emitted DEGRADED + FLAGGED (never as healthy).
|
||||
|
||||
record OFF-HOST MONITOR side. Ingest a received beacon (JSON
|
||||
on stdin) into the received store, keeping the highest
|
||||
seq (a stale/replayed lower-or-equal seq is ignored —
|
||||
the monotonic contract enforced at the receiver).
|
||||
|
||||
check --slo-seconds N OFF-HOST MONITOR side. The DEAD-MAN / ABSENCE check.
|
||||
Reads the last received beacon; if it is MISSING or
|
||||
STALE beyond the SLO, that ABSENCE FIRES the alarm and
|
||||
ROUTES it via WAKE_ALARM_SINK_CMD to a human/other-host
|
||||
(G1). Depends on NOTHING the dying host actively does.
|
||||
Exit 0 = alive (fresh beacon); exit 1 = absence, alarm
|
||||
fired+routed; exit 3 = fail-closed (missing SLO, or an
|
||||
unconfigured/unreachable alarm target — loudest).
|
||||
A capture-pane hint (WAKE_BEACON_PANE_HINT) is a
|
||||
liveness HINT ONLY and does NOT suppress an absence.
|
||||
|
||||
status Print the emitter counter + last received beacon.
|
||||
|
||||
Options:
|
||||
--slo-seconds N REQUIRED for check. The absence SLO (operator-tuned; symbolic
|
||||
per-class tiers per §4 — NO default is invented). A beacon
|
||||
older than N seconds is an ABSENCE.
|
||||
|
||||
Environment:
|
||||
WAKE_BEACON_SINK_CMD Pluggable transport to the off-host monitor (emit).
|
||||
WAKE_ALARM_SINK_CMD Pluggable alarm route to a human/other-host (check).
|
||||
WAKE_BEACON_INDEPENDENCE off-host | different-supervision-root |
|
||||
same-host-sibling (REQUIRED for emit; declared by
|
||||
the operator, install-validated by W7).
|
||||
WAKE_BEACON_HOST_ID Host identity carried in the beacon (default: hostname).
|
||||
WAKE_BEACON_SUPERVISION_ROOT Supervision-root label for the degraded beacon.
|
||||
WAKE_BEACON_RECEIVED Off-host monitor's received-beacon store path.
|
||||
WAKE_STATE_HOME/WAKE_AGENT wake state namespace (see store.sh).
|
||||
EOF
|
||||
}
|
||||
|
||||
# --- monotonic emitter counter (§1.3: a monotonic beacon each cycle) --------
|
||||
|
||||
# _next_seq — atomically increment + return the beacon counter. Strictly
|
||||
# increasing across emits (the monotonic contract the off-host monitor relies on
|
||||
# to distinguish a live-advancing host from a stalled one).
|
||||
_next_seq() {
|
||||
mkdir -p "$BEACON_DIR"
|
||||
local cur nxt
|
||||
cur="$(_wake_read_int "$SEQ_FILE" 0)"
|
||||
nxt=$((cur + 1))
|
||||
printf '%s' "$nxt" | _atomic_write "$SEQ_FILE"
|
||||
printf '%s' "$nxt"
|
||||
}
|
||||
|
||||
# --- independence policy (§1.3: split liveness; reject a non-independent leg) -
|
||||
|
||||
# _resolve_independence — validate the operator's declared independence class and
|
||||
# echo two space-separated tokens: "<class> <degraded 0|1>".
|
||||
# same-host-sibling -> REJECTED (fail loud): shares user-manager/host/
|
||||
# sender/socket/session, so it is NOT an independent
|
||||
# liveness leg and must NEVER stand in for one.
|
||||
# different-supervision-root -> ACCEPTED but DEGRADED=1: an isolated host with
|
||||
# no off-host monitor reachable degrades to this
|
||||
# weaker leg, FLAGGED — never silently "healthy".
|
||||
# off-host -> ACCEPTED, DEGRADED=0: full independence.
|
||||
_resolve_independence() {
|
||||
local decl="${WAKE_BEACON_INDEPENDENCE:-}"
|
||||
case "$decl" in
|
||||
off-host)
|
||||
printf 'off-host 0'
|
||||
;;
|
||||
different-supervision-root)
|
||||
printf 'different-supervision-root 1'
|
||||
;;
|
||||
same-host-sibling)
|
||||
echo "beacon.sh: FAIL LOUD (§1.3) — WAKE_BEACON_INDEPENDENCE=same-host-sibling is NOT an independent liveness leg (it shares user-manager/host/sender/socket/session with the component it would supervise). Refusing to emit a beacon that would falsely stand in for off-host supervision." >&2
|
||||
return 2
|
||||
;;
|
||||
'')
|
||||
echo "beacon.sh: FAIL LOUD — WAKE_BEACON_INDEPENDENCE is unset. Declare the wired sink's independence (off-host | different-supervision-root | same-host-sibling); liveness independence is never silently assumed." >&2
|
||||
return 2
|
||||
;;
|
||||
*)
|
||||
echo "beacon.sh: FAIL LOUD — WAKE_BEACON_INDEPENDENCE='$decl' is not a recognized class (off-host | different-supervision-root | same-host-sibling)." >&2
|
||||
return 2
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
# --- host side: emit -------------------------------------------------------
|
||||
|
||||
cmd_emit() {
|
||||
_need_jq
|
||||
|
||||
# Independence FIRST — a same-host-sibling / unknown declaration is rejected
|
||||
# before any seq is minted, so a rejected emit never advances the counter.
|
||||
local indep_class degraded parts
|
||||
parts="$(_resolve_independence)" || exit 2
|
||||
indep_class="${parts%% *}"
|
||||
degraded="${parts##* }"
|
||||
|
||||
# FAIL-CLOSED, part 1 — an UNCONFIGURED off-host target is a silent no-alarm
|
||||
# host. Refuse before minting a seq (nothing to ship it through).
|
||||
if [ -z "${WAKE_BEACON_SINK_CMD:-}" ]; then
|
||||
echo "beacon.sh: FAIL LOUD (G2a) — WAKE_BEACON_SINK_CMD is unset: no off-host beacon target is configured. A host with no beacon sink is a silent no-alarm host. Refusing to no-op." >&2
|
||||
exit 3
|
||||
fi
|
||||
|
||||
local host_id supervision_root seq emit_ts record
|
||||
host_id="${WAKE_BEACON_HOST_ID:-$(hostname 2>/dev/null || echo host)}"
|
||||
supervision_root="${WAKE_BEACON_SUPERVISION_ROOT:-}"
|
||||
seq="$(_next_seq)"
|
||||
emit_ts="$(date +%s)"
|
||||
|
||||
record="$(jq -cn \
|
||||
--argjson seq "$seq" \
|
||||
--argjson emit_ts "$emit_ts" \
|
||||
--arg host_id "$host_id" \
|
||||
--arg independence "$indep_class" \
|
||||
--argjson degraded "$([ "$degraded" -eq 1 ] && echo true || echo false)" \
|
||||
--arg supervision_root "$supervision_root" \
|
||||
'{kind:"wake-beacon", beacon_seq:$seq, emit_ts:$emit_ts, host_id:$host_id,
|
||||
independence:$independence, degraded:$degraded}
|
||||
+ (if $supervision_root == "" then {} else {supervision_root:$supervision_root} end)')"
|
||||
|
||||
# DEGRADED beacons are FLAGGED loudly (§1.3) — an isolated host must never
|
||||
# silently present a weaker different-supervision-root leg as full independence.
|
||||
if [ "$degraded" -eq 1 ]; then
|
||||
echo "beacon.sh: DEGRADED beacon (§1.3) — no off-host monitor is reachable; emitting a weaker DIFFERENT-SUPERVISION-ROOT beacon. This is FLAGGED, NOT full independence." >&2
|
||||
fi
|
||||
|
||||
# FAIL-CLOSED, part 2 — ship via the pluggable sink adapter. A non-zero exit is
|
||||
# an UNREACHABLE target => FAIL LOUD (the beacon did not reach off-host).
|
||||
if ! printf '%s\n' "$record" | sh -c "$WAKE_BEACON_SINK_CMD"; then
|
||||
echo "beacon.sh: FAIL LOUD (G2a) — beacon sink is UNREACHABLE (WAKE_BEACON_SINK_CMD exited non-zero); beacon seq $seq did NOT reach the off-host monitor." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Local record of the last emit (introspection / the detector seam). The
|
||||
# AUTHORITATIVE liveness judgement is the off-host monitor's received store,
|
||||
# never this local copy.
|
||||
printf '%s\n' "$record" | _atomic_write "$BEACON_DIR/last-emit.json"
|
||||
printf 'beacon_seq=%s emit_ts=%s independence=%s degraded=%s\n' \
|
||||
"$seq" "$emit_ts" "$indep_class" "$([ "$degraded" -eq 1 ] && echo true || echo false)"
|
||||
}
|
||||
|
||||
# --- off-host monitor side: record a received beacon -----------------------
|
||||
|
||||
cmd_record() {
|
||||
_need_jq
|
||||
local incoming
|
||||
incoming="$(cat)"
|
||||
if [ -z "$incoming" ]; then
|
||||
echo "beacon.sh record: no beacon on stdin" >&2
|
||||
exit 2
|
||||
fi
|
||||
local in_seq in_ts
|
||||
in_seq="$(printf '%s' "$incoming" | jq -r '.beacon_seq // empty' 2>/dev/null)"
|
||||
in_ts="$(printf '%s' "$incoming" | jq -r '.emit_ts // empty' 2>/dev/null)"
|
||||
case "$in_seq" in
|
||||
'' | *[!0-9]*)
|
||||
echo "beacon.sh record: FAIL LOUD — received beacon has no integer beacon_seq (malformed)" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
case "$in_ts" in
|
||||
'' | *[!0-9]*)
|
||||
echo "beacon.sh record: FAIL LOUD — received beacon has no integer emit_ts (malformed)" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
|
||||
mkdir -p "$(dirname "$RECEIVED_FILE")"
|
||||
# MONOTONIC receiver: keep the highest-seq beacon. A lower-or-equal seq is a
|
||||
# stale/replayed beacon and is IGNORED (does not roll the liveness clock back).
|
||||
local last_seq=0
|
||||
if [ -f "$RECEIVED_FILE" ]; then
|
||||
last_seq="$(jq -r '.beacon_seq // 0' "$RECEIVED_FILE" 2>/dev/null)"
|
||||
case "$last_seq" in '' | *[!0-9]*) last_seq=0 ;; esac
|
||||
fi
|
||||
if [ "$in_seq" -le "$last_seq" ]; then
|
||||
echo "beacon.sh record: ignoring stale/replayed beacon seq $in_seq (<= recorded $last_seq)" >&2
|
||||
exit 0
|
||||
fi
|
||||
printf '%s\n' "$incoming" | _atomic_write "$RECEIVED_FILE"
|
||||
}
|
||||
|
||||
# --- off-host monitor side: the DEAD-MAN / ABSENCE check -------------------
|
||||
|
||||
# _fire_alarm REASON PAYLOAD_JSON — route an absence alarm via the pluggable
|
||||
# alarm-sink adapter to a human/other-host (G1). FAIL-CLOSED: an unconfigured OR
|
||||
# unreachable alarm target is the loudest failure (a silent no-alarm host is the
|
||||
# exact condition G1 exists to prevent).
|
||||
_fire_alarm() {
|
||||
local reason="$1" payload="$2"
|
||||
if [ -z "${WAKE_ALARM_SINK_CMD:-}" ]; then
|
||||
echo "beacon.sh: FAIL LOUD (G1/G2a) — beacon ABSENCE detected ($reason) but WAKE_ALARM_SINK_CMD is UNSET: the alarm cannot route to a human/other-host. This is a silent no-alarm host — the exact failure G1 forbids." >&2
|
||||
exit 3
|
||||
fi
|
||||
if ! printf '%s\n' "$payload" | sh -c "$WAKE_ALARM_SINK_CMD"; then
|
||||
echo "beacon.sh: FAIL LOUD (G1/G2a) — beacon ABSENCE detected ($reason) but the alarm sink is UNREACHABLE (WAKE_ALARM_SINK_CMD exited non-zero): the alarm did NOT reach a human/other-host." >&2
|
||||
exit 3
|
||||
fi
|
||||
echo "beacon.sh: ALARM FIRED + ROUTED (§1.3/G1) — $reason" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
cmd_check() {
|
||||
_need_jq
|
||||
local slo=''
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--slo-seconds)
|
||||
slo="${2:-}"
|
||||
shift 2
|
||||
;;
|
||||
*)
|
||||
echo "beacon.sh check: unknown option '$1'" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# No invented SLO (design law) — the per-class absence SLO MUST be supplied.
|
||||
case "$slo" in
|
||||
'' | *[!0-9]*)
|
||||
echo "beacon.sh check: --slo-seconds is REQUIRED and must be a non-negative integer (per-class absence SLO is operator-tuned; no default is invented)." >&2
|
||||
exit 3
|
||||
;;
|
||||
esac
|
||||
|
||||
# capture-pane is a liveness HINT ONLY (§1.3): its presence NEVER suppresses an
|
||||
# absence. Readiness is proven solely by the received beacon clock (fed by the
|
||||
# RECEIVED/CONSUMED acks' independent leg), never by a pane scrape. We note the
|
||||
# hint for operators but do NOT let it gate the dead-man verdict.
|
||||
if [ -n "${WAKE_BEACON_PANE_HINT:-}" ]; then
|
||||
echo "beacon.sh check: note — a capture-pane hint is present; it is a liveness HINT ONLY and does not affect this absence verdict." >&2
|
||||
fi
|
||||
|
||||
local now
|
||||
now="$(date +%s)"
|
||||
|
||||
# ABSENCE, case 1 — NEVER received. Depends on nothing the dying host does.
|
||||
if [ ! -f "$RECEIVED_FILE" ] || [ ! -s "$RECEIVED_FILE" ]; then
|
||||
local payload
|
||||
payload="$(jq -cn --argjson now "$now" --argjson slo "$slo" \
|
||||
'{kind:"beacon-absence-alarm", reason:"no beacon ever received",
|
||||
detected_ts:$now, slo_seconds:$slo}')"
|
||||
_fire_alarm "no beacon ever received" "$payload"
|
||||
fi
|
||||
|
||||
local last_ts last_seq host_id age
|
||||
last_ts="$(jq -r '.emit_ts // empty' "$RECEIVED_FILE" 2>/dev/null)"
|
||||
last_seq="$(jq -r '.beacon_seq // empty' "$RECEIVED_FILE" 2>/dev/null)"
|
||||
host_id="$(jq -r '.host_id // "unknown"' "$RECEIVED_FILE" 2>/dev/null)"
|
||||
case "$last_ts" in
|
||||
'' | *[!0-9]*)
|
||||
local payload
|
||||
payload="$(jq -cn --argjson now "$now" --argjson slo "$slo" \
|
||||
'{kind:"beacon-absence-alarm", reason:"received beacon has no valid emit_ts (corrupt)",
|
||||
detected_ts:$now, slo_seconds:$slo}')"
|
||||
_fire_alarm "corrupt received beacon (no emit_ts)" "$payload"
|
||||
;;
|
||||
esac
|
||||
|
||||
age=$((now - last_ts))
|
||||
|
||||
# ABSENCE, case 2 — STALE beyond the SLO. A stopped emitter stops advancing the
|
||||
# received clock; once age > SLO the off-host monitor fires WITHOUT the dying
|
||||
# host lifting a finger.
|
||||
if [ "$age" -gt "$slo" ]; then
|
||||
local payload
|
||||
payload="$(jq -cn \
|
||||
--arg host_id "$host_id" \
|
||||
--argjson last_seq "${last_seq:-0}" \
|
||||
--argjson last_ts "$last_ts" \
|
||||
--argjson now "$now" \
|
||||
--argjson age "$age" \
|
||||
--argjson slo "$slo" \
|
||||
'{kind:"beacon-absence-alarm", reason:"beacon stale past SLO",
|
||||
host_id:$host_id, last_beacon_seq:$last_seq, last_emit_ts:$last_ts,
|
||||
detected_ts:$now, age_seconds:$age, slo_seconds:$slo}')"
|
||||
_fire_alarm "beacon stale (${age}s > SLO ${slo}s), host=$host_id last_seq=${last_seq:-?}" "$payload"
|
||||
fi
|
||||
|
||||
echo "beacon.sh: ALIVE — last beacon seq ${last_seq:-?} for host '$host_id' is ${age}s old (<= SLO ${slo}s)."
|
||||
}
|
||||
|
||||
cmd_status() {
|
||||
local seq
|
||||
seq="$(_wake_read_int "$SEQ_FILE" 0)"
|
||||
printf 'beacon_emitter_seq=%s\n' "$seq"
|
||||
if [ -f "$RECEIVED_FILE" ] && [ -s "$RECEIVED_FILE" ]; then
|
||||
printf 'last_received='
|
||||
cat "$RECEIVED_FILE"
|
||||
else
|
||||
printf 'last_received=(none)\n'
|
||||
fi
|
||||
}
|
||||
|
||||
main() {
|
||||
[ $# -ge 1 ] || {
|
||||
usage
|
||||
exit 2
|
||||
}
|
||||
local cmd="$1"
|
||||
shift
|
||||
case "$cmd" in
|
||||
emit) cmd_emit "$@" ;;
|
||||
record) cmd_record "$@" ;;
|
||||
check) cmd_check "$@" ;;
|
||||
status) cmd_status "$@" ;;
|
||||
-h | --help | help) usage ;;
|
||||
*)
|
||||
echo "beacon.sh: unknown command '$cmd'" >&2
|
||||
usage
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
main "$@"
|
||||
@@ -402,6 +402,18 @@ cmd_run() {
|
||||
# A single failing source must not kill the long-lived service; poll-once
|
||||
# already failed loud on stderr for it. The loop keeps serving healthy ones.
|
||||
cmd_poll_once || true
|
||||
# W6 SEAM (§1.3, off-host dead-man beacon): emit ONE monotonic liveness
|
||||
# beacon per poll cycle. This is the ONLY W6 call site in the detector — a
|
||||
# single, clean, opt-in seam. It is INERT unless the operator wires a beacon
|
||||
# sink (WAKE_BEACON_SINK_CMD); wiring + install-validating that target is W7's
|
||||
# job, not the detector's. Liveness is SPLIT from work-triggering, so a beacon
|
||||
# emit failure NEVER kills the detector loop — but it is loud (not silent),
|
||||
# and the off-host monitor's absence check (beacon.sh check) is the real
|
||||
# safety net regardless of what this dying host does.
|
||||
if [ -n "${WAKE_BEACON_SINK_CMD:-}" ]; then
|
||||
"$SCRIPT_DIR/beacon.sh" emit >/dev/null || \
|
||||
echo "detector.sh: WARN — off-host liveness beacon emit failed (see beacon.sh); the off-host absence check remains the authoritative dead-man." >&2
|
||||
fi
|
||||
[ "$once" -eq 1 ] && break
|
||||
sleep "$interval"
|
||||
done
|
||||
|
||||
@@ -15,8 +15,10 @@
|
||||
# 0.2.0 W3 — cumulative-state digest renderer + non-circular HMAC signer.
|
||||
# 0.3.0 W4 — per-host single-instance delta-gated detector daemon.
|
||||
# 0.4.0 W5 — synthetic-canary FN-oracle + source-parity reconciler.
|
||||
# 0.5.0 W6 — off-host dead-man beacon emitter + pluggable alarm-sink adapter
|
||||
# + beacon-absence alarm (fail-loud on unconfigured/unreachable).
|
||||
component=wake
|
||||
version=0.4.0
|
||||
version=0.5.0
|
||||
|
||||
# Watch-list schema this component consumes, and the INCLUSIVE range of
|
||||
# schema_version values it supports. A wake-watch-list.json whose schema_version
|
||||
@@ -45,4 +47,16 @@ schema_max=1
|
||||
# inventory (an omitted source cannot pass the vector
|
||||
# vacuously) + (ii) periodic full reconcile to 0-unaccounted,
|
||||
# enumerating pre-existing/startup state into the store. (W5)
|
||||
# Out of scope (later waves): off-host beacon (W6), installer (W7).
|
||||
# beacon.sh A8 — off-host DEAD-MAN liveness beacon: a monotonic beacon
|
||||
# EMITTER (emit — the primitive the detector run-loop calls
|
||||
# each cycle), the off-host monitor's RECEIVER + beacon-ABSENCE
|
||||
# alarm (record, check), and a pluggable alarm-sink/beacon-sink
|
||||
# ADAPTER INTERFACE. Liveness is SPLIT from work-triggering;
|
||||
# the alarm fires on ABSENCE, routing to a human/other-host
|
||||
# within its SLO (§4/G1). FAIL-CLOSED: an unconfigured OR
|
||||
# unreachable target FAILS LOUD (no silent no-alarm host).
|
||||
# A same-host sibling is REJECTED as non-independent; an
|
||||
# isolated host degrades to a FLAGGED different-supervision-root
|
||||
# beacon; capture-pane is a liveness HINT only. (W6)
|
||||
# Out of scope (later waves): installer (W7 wires + install-validates the beacon
|
||||
# target that beacon.sh's fail-loud primitive is designed for).
|
||||
|
||||
279
packages/mosaic/framework/tools/wake/test-wake-beacon.sh
Executable file
279
packages/mosaic/framework/tools/wake/test-wake-beacon.sh
Executable file
@@ -0,0 +1,279 @@
|
||||
#!/usr/bin/env bash
|
||||
# test-wake-beacon.sh — RED-FIRST invariant harness for W6 (EPIC #892): the
|
||||
# off-host dead-man beacon emitter + pluggable alarm-sink adapter (beacon.sh, A8).
|
||||
#
|
||||
# Each test asserts ONE CONVERGED-DESIGN invariant and is designed to go RED if
|
||||
# that invariant regresses:
|
||||
# B1 beacon emitted each cycle -> MONOTONIC (seq strictly increases) (§1.3)
|
||||
# B2 beacon ABSENCE past SLO -> alarm FIRES + ROUTES (stopped emitter);
|
||||
# and NEVER-received -> alarm too — depends on nothing the dying host does (§1.3/G1)
|
||||
# B3 unconfigured OR unreachable ALARM target -> FAIL LOUD (no silent
|
||||
# no-alarm host) (§4 G1/G2a)
|
||||
# B4 isolated host (no off-host monitor) -> DEGRADED different-supervision-root
|
||||
# beacon FLAGGED, never silently "healthy" (§1.3)
|
||||
# B5 same-host-sibling declaration -> REJECTED as non-independent, and the
|
||||
# monotonic seq is NOT advanced by a rejected emit (§1.3)
|
||||
# B6 the alarm/beacon target is resolved BY NAME by the operator adapter;
|
||||
# beacon.sh inlines NO endpoint/secret (§1.4)
|
||||
# B7 unconfigured OR unreachable BEACON sink on emit -> FAIL LOUD (§4 G2a)
|
||||
# B8 no invented SLO -> check --slo-seconds is REQUIRED (fail-loud) (design law)
|
||||
# B9 capture-pane hint is a liveness HINT ONLY -> does NOT suppress absence (§1.3)
|
||||
# B10 fresh beacon within SLO -> ALIVE (exit 0, no alarm) (§1.3)
|
||||
#
|
||||
# Uses per-test isolated XDG homes + pluggable sink/alarm adapter SCRIPTS so a
|
||||
# stopped/dropping emitter and an unreachable target can be exercised. No live
|
||||
# network, no operator queue touched.
|
||||
#
|
||||
# SC2030/SC2031 are DELIBERATELY disabled: each test runs in its own ( ) subshell
|
||||
# and re-exports the per-test env, so environments are isolated by design (the
|
||||
# same idiom as test-wake-fn-oracle.sh / test-wake-detector.sh).
|
||||
# shellcheck disable=SC2030,SC2031
|
||||
set -uo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
BEACON="$SCRIPT_DIR/beacon.sh"
|
||||
|
||||
command -v jq >/dev/null 2>&1 || {
|
||||
echo "SKIP: jq not available" >&2
|
||||
exit 0
|
||||
}
|
||||
|
||||
TMP_ROOT="$(mktemp -d)"
|
||||
trap 'rm -rf "$TMP_ROOT"' EXIT
|
||||
|
||||
# Failures recorded to a FILE (subshell-safe — a var counter would silently
|
||||
# swallow failures across the per-test subshells; mirrors the W2/W4/W5 harnesses).
|
||||
FAILFILE="$TMP_ROOT/failures"
|
||||
: >"$FAILFILE"
|
||||
pass=0
|
||||
fail_msg() {
|
||||
echo " FAIL: $*" >&2
|
||||
echo "x" >>"$FAILFILE"
|
||||
}
|
||||
ok() { pass=$((pass + 1)); }
|
||||
|
||||
fresh_home() {
|
||||
local d="$TMP_ROOT/$1"
|
||||
rm -rf "$d"
|
||||
mkdir -p "$d"
|
||||
printf '%s' "$d"
|
||||
}
|
||||
|
||||
# --- reusable sink/alarm adapter scripts (the pluggable operator seam) -------
|
||||
|
||||
# discard-sink: a REACHABLE off-host transport that discards (exit 0).
|
||||
DISCARD_SINK="$TMP_ROOT/discard-sink.sh"
|
||||
cat >"$DISCARD_SINK" <<'EOF'
|
||||
#!/usr/bin/env bash
|
||||
cat >/dev/null
|
||||
EOF
|
||||
chmod +x "$DISCARD_SINK"
|
||||
|
||||
# recorder-sink: a REACHABLE transport that feeds the received beacon into the
|
||||
# off-host monitor's received store via beacon.sh record (WAKE_BEACON_RECEIVED).
|
||||
RECORDER_SINK="$TMP_ROOT/recorder-sink.sh"
|
||||
cat >"$RECORDER_SINK" <<EOF
|
||||
#!/usr/bin/env bash
|
||||
exec "$BEACON" record
|
||||
EOF
|
||||
chmod +x "$RECORDER_SINK"
|
||||
|
||||
# capture-alarm: a REACHABLE alarm route that writes the routed payload to
|
||||
# \$ALARM_OUT (so a test can prove the alarm actually reached the sink).
|
||||
CAPTURE_ALARM="$TMP_ROOT/capture-alarm.sh"
|
||||
cat >"$CAPTURE_ALARM" <<'EOF'
|
||||
#!/usr/bin/env bash
|
||||
cat >"$ALARM_OUT"
|
||||
EOF
|
||||
chmod +x "$CAPTURE_ALARM"
|
||||
|
||||
# byname-alarm: resolves its TARGET endpoint BY NAME from a credential store
|
||||
# (the load_credentials shape) — never an inline literal — then "delivers" the
|
||||
# alarm to that resolved target, writing "<target>\n<payload>" to \$ALARM_OUT.
|
||||
BYNAME_ALARM="$TMP_ROOT/byname-alarm.sh"
|
||||
cat >"$BYNAME_ALARM" <<'EOF'
|
||||
#!/usr/bin/env bash
|
||||
set -u
|
||||
# The operator adapter owns the endpoint: resolved BY NAME from the cred store,
|
||||
# never passed inline by the framework.
|
||||
target="$(jq -r --arg n "$ALARM_TARGET_NAME" '.wake.beacon_alarm_targets[$n] // empty' "$CRED_FILE")"
|
||||
[ -n "$target" ] || { echo "byname-alarm: target name '$ALARM_TARGET_NAME' not found" >&2; exit 1; }
|
||||
{ printf '%s\n' "$target"; cat; } >"$ALARM_OUT"
|
||||
EOF
|
||||
chmod +x "$BYNAME_ALARM"
|
||||
|
||||
echo "== B1: beacon emitted each cycle -> MONOTONIC (seq strictly increases) =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_home b1)"
|
||||
export WAKE_STATE_HOME
|
||||
export WAKE_BEACON_INDEPENDENCE="off-host"
|
||||
export WAKE_BEACON_SINK_CMD="$DISCARD_SINK"
|
||||
s1="$("$BEACON" emit 2>/dev/null | sed -n 's/.*beacon_seq=\([0-9]*\).*/\1/p')"
|
||||
s2="$("$BEACON" emit 2>/dev/null | sed -n 's/.*beacon_seq=\([0-9]*\).*/\1/p')"
|
||||
s3="$("$BEACON" emit 2>/dev/null | sed -n 's/.*beacon_seq=\([0-9]*\).*/\1/p')"
|
||||
[ -n "$s1" ] && [ -n "$s2" ] && [ -n "$s3" ] || fail_msg "B1: emit must print a beacon_seq each cycle [$s1/$s2/$s3]"
|
||||
{ [ "$s2" -gt "$s1" ] && [ "$s3" -gt "$s2" ]; } || fail_msg "B1: beacon seq must STRICTLY increase each cycle (got $s1,$s2,$s3)"
|
||||
) && ok
|
||||
|
||||
echo "== B2: beacon ABSENCE past SLO -> alarm FIRES + ROUTES =="
|
||||
(
|
||||
H="$(fresh_home b2)"
|
||||
export ALARM_OUT="$H/alarm.json"
|
||||
export WAKE_BEACON_RECEIVED="$H/received.json"
|
||||
export WAKE_ALARM_SINK_CMD="$CAPTURE_ALARM"
|
||||
# Simulate a STOPPED emitter: the monitor's last received beacon is old.
|
||||
now="$(date +%s)"
|
||||
jq -cn --argjson ts "$((now - 100))" \
|
||||
'{kind:"wake-beacon", beacon_seq:5, emit_ts:$ts, host_id:"h", independence:"off-host", degraded:false}' \
|
||||
>"$WAKE_BEACON_RECEIVED"
|
||||
out="$("$BEACON" check --slo-seconds 5 2>&1)"
|
||||
rc=$?
|
||||
[ "$rc" -eq 1 ] || fail_msg "B2: a stale-past-SLO beacon must FIRE the absence alarm (exit 1); got $rc [$out]"
|
||||
echo "$out" | grep -qi 'ALARM FIRED' || fail_msg "B2: absence must announce the alarm fired [$out]"
|
||||
[ -s "$ALARM_OUT" ] || fail_msg "B2: the alarm must actually ROUTE to the sink (payload not written)"
|
||||
jq -e '.kind == "beacon-absence-alarm"' "$ALARM_OUT" >/dev/null 2>&1 || fail_msg "B2: routed payload must be a beacon-absence-alarm [$(cat "$ALARM_OUT" 2>/dev/null)]"
|
||||
# NEVER-received is also an absence (depends on nothing the dying host does).
|
||||
rm -f "$ALARM_OUT"
|
||||
export WAKE_BEACON_RECEIVED="$H/nonexistent.json"
|
||||
out2="$("$BEACON" check --slo-seconds 5 2>&1)"
|
||||
rc2=$?
|
||||
[ "$rc2" -eq 1 ] || fail_msg "B2: a never-received beacon must FIRE the absence alarm (exit 1); got $rc2 [$out2]"
|
||||
[ -s "$ALARM_OUT" ] || fail_msg "B2: never-received absence must route to the alarm sink"
|
||||
) && ok
|
||||
|
||||
echo "== B3: unconfigured OR unreachable ALARM target -> FAIL LOUD =="
|
||||
(
|
||||
H="$(fresh_home b3)"
|
||||
export WAKE_BEACON_RECEIVED="$H/nonexistent.json" # absence condition
|
||||
# (a) UNCONFIGURED alarm sink.
|
||||
unset WAKE_ALARM_SINK_CMD
|
||||
out="$("$BEACON" check --slo-seconds 5 2>&1)"
|
||||
rc=$?
|
||||
[ "$rc" -eq 3 ] || fail_msg "B3a: unconfigured alarm target must FAIL LOUD (exit 3); got $rc [$out]"
|
||||
echo "$out" | grep -qi 'silent no-alarm host' || fail_msg "B3a: the failure must name the silent-no-alarm-host hazard [$out]"
|
||||
# (b) UNREACHABLE alarm sink (non-zero exit).
|
||||
export WAKE_ALARM_SINK_CMD="false"
|
||||
out2="$("$BEACON" check --slo-seconds 5 2>&1)"
|
||||
rc2=$?
|
||||
[ "$rc2" -eq 3 ] || fail_msg "B3b: unreachable alarm target must FAIL LOUD (exit 3); got $rc2 [$out2]"
|
||||
echo "$out2" | grep -qi 'UNREACHABLE' || fail_msg "B3b: the failure must name the unreachable target [$out2]"
|
||||
) && ok
|
||||
|
||||
echo "== B4: isolated host -> DEGRADED different-supervision-root beacon FLAGGED =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_home b4)"
|
||||
export WAKE_STATE_HOME
|
||||
export WAKE_BEACON_INDEPENDENCE="different-supervision-root"
|
||||
export WAKE_BEACON_SINK_CMD="$DISCARD_SINK"
|
||||
out="$("$BEACON" emit 2>"$TMP_ROOT/b4.err")"
|
||||
rc=$?
|
||||
err="$(cat "$TMP_ROOT/b4.err")"
|
||||
[ "$rc" -eq 0 ] || fail_msg "B4: a different-supervision-root emit must still succeed (exit 0); got $rc [$out][$err]"
|
||||
echo "$err" | grep -qi 'DEGRADED' || fail_msg "B4: a different-supervision-root beacon must be FLAGGED degraded on stderr [$err]"
|
||||
echo "$out" | grep -q 'degraded=true' || fail_msg "B4: emit must NOT silently present a degraded beacon as healthy (degraded=true expected) [$out]"
|
||||
) && ok
|
||||
|
||||
echo "== B5: same-host-sibling -> REJECTED as non-independent, seq NOT advanced =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_home b5)"
|
||||
export WAKE_STATE_HOME
|
||||
export WAKE_BEACON_INDEPENDENCE="same-host-sibling"
|
||||
export WAKE_BEACON_SINK_CMD="$DISCARD_SINK"
|
||||
out="$("$BEACON" emit 2>&1)"
|
||||
rc=$?
|
||||
[ "$rc" -ne 0 ] || fail_msg "B5: a same-host-sibling beacon must be REJECTED (non-zero exit) [$out]"
|
||||
echo "$out" | grep -qi 'not an independent' || fail_msg "B5: the rejection must explain it is NOT an independent leg [$out]"
|
||||
# A rejected emit must not have minted a seq (rejection precedes counter bump).
|
||||
seq_after="$("$BEACON" status 2>/dev/null | sed -n 's/^beacon_emitter_seq=//p')"
|
||||
[ "${seq_after:-0}" -eq 0 ] || fail_msg "B5: a rejected emit must NOT advance the monotonic seq (got $seq_after)"
|
||||
) && ok
|
||||
|
||||
echo "== B6: alarm target resolved BY NAME; beacon.sh inlines no endpoint/secret =="
|
||||
(
|
||||
H="$(fresh_home b6)"
|
||||
export ALARM_OUT="$H/alarm.out"
|
||||
export CRED_FILE="$H/credentials.json"
|
||||
export ALARM_TARGET_NAME="primary-monitor"
|
||||
SECRET_TARGET="https://monitor.invalid/alarm/DO-NOT-INLINE-abc123"
|
||||
jq -cn --arg t "$SECRET_TARGET" '{wake:{beacon_alarm_targets:{"primary-monitor":$t}}}' >"$CRED_FILE"
|
||||
export WAKE_BEACON_RECEIVED="$H/nonexistent.json" # absence -> alarm
|
||||
export WAKE_ALARM_SINK_CMD="$BYNAME_ALARM"
|
||||
out="$("$BEACON" check --slo-seconds 5 2>&1)"
|
||||
rc=$?
|
||||
[ "$rc" -eq 1 ] || fail_msg "B6: absence via a by-name alarm adapter must fire (exit 1); got $rc [$out]"
|
||||
head -n1 "$ALARM_OUT" 2>/dev/null | grep -qF "$SECRET_TARGET" || fail_msg "B6: the adapter must resolve+route to the BY-NAME target [$(cat "$ALARM_OUT" 2>/dev/null)]"
|
||||
# The framework file must NOT inline the endpoint/secret: it only knows a NAME.
|
||||
grep -qF "$SECRET_TARGET" "$BEACON" && fail_msg "B6: beacon.sh must NOT inline the target endpoint/secret"
|
||||
grep -qF "$SECRET_TARGET" "$WAKE_ALARM_SINK_CMD" && fail_msg "B6: even the adapter must resolve by-name, not inline the secret"
|
||||
true
|
||||
) && ok
|
||||
|
||||
echo "== B7: unconfigured OR unreachable BEACON sink on emit -> FAIL LOUD =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_home b7)"
|
||||
export WAKE_STATE_HOME
|
||||
export WAKE_BEACON_INDEPENDENCE="off-host"
|
||||
# (a) UNCONFIGURED beacon sink.
|
||||
unset WAKE_BEACON_SINK_CMD
|
||||
out="$("$BEACON" emit 2>&1)"
|
||||
rc=$?
|
||||
[ "$rc" -eq 3 ] || fail_msg "B7a: unconfigured beacon sink must FAIL LOUD (exit 3); got $rc [$out]"
|
||||
echo "$out" | grep -qi 'silent no-alarm host' || fail_msg "B7a: the failure must name the silent-no-alarm-host hazard [$out]"
|
||||
# A refused emit (no sink) must not have minted a seq.
|
||||
seq_after="$("$BEACON" status 2>/dev/null | sed -n 's/^beacon_emitter_seq=//p')"
|
||||
[ "${seq_after:-0}" -eq 0 ] || fail_msg "B7a: an unconfigured-sink emit must NOT advance the seq (got $seq_after)"
|
||||
# (b) UNREACHABLE beacon sink (non-zero exit).
|
||||
export WAKE_BEACON_SINK_CMD="false"
|
||||
out2="$("$BEACON" emit 2>&1)"
|
||||
rc2=$?
|
||||
[ "$rc2" -eq 1 ] || fail_msg "B7b: unreachable beacon sink must FAIL LOUD (exit 1); got $rc2 [$out2]"
|
||||
echo "$out2" | grep -qi 'UNREACHABLE' || fail_msg "B7b: the failure must name the unreachable sink [$out2]"
|
||||
) && ok
|
||||
|
||||
echo "== B8: no invented SLO -> check --slo-seconds is REQUIRED (fail-loud) =="
|
||||
(
|
||||
H="$(fresh_home b8)"
|
||||
export WAKE_BEACON_RECEIVED="$H/nonexistent.json"
|
||||
export WAKE_ALARM_SINK_CMD="$DISCARD_SINK"
|
||||
out="$("$BEACON" check 2>&1)"
|
||||
rc=$?
|
||||
[ "$rc" -eq 3 ] || fail_msg "B8: check without an SLO must fail loud (exit 3); got $rc [$out]"
|
||||
echo "$out" | grep -qi 'slo' || fail_msg "B8: the usage error must name the missing SLO [$out]"
|
||||
) && ok
|
||||
|
||||
echo "== B9: capture-pane hint is a liveness HINT ONLY (does NOT suppress absence) =="
|
||||
(
|
||||
H="$(fresh_home b9)"
|
||||
export ALARM_OUT="$H/alarm.json"
|
||||
export WAKE_BEACON_RECEIVED="$H/nonexistent.json" # genuine absence
|
||||
export WAKE_ALARM_SINK_CMD="$CAPTURE_ALARM"
|
||||
export WAKE_BEACON_PANE_HINT="agent looks idle-at-prompt in capture-pane"
|
||||
out="$("$BEACON" check --slo-seconds 5 2>&1)"
|
||||
rc=$?
|
||||
[ "$rc" -eq 1 ] || fail_msg "B9: a capture-pane hint must NOT suppress a real absence alarm (expected exit 1); got $rc [$out]"
|
||||
[ -s "$ALARM_OUT" ] || fail_msg "B9: absence must still route despite a pane hint (readiness != pane scrape)"
|
||||
) && ok
|
||||
|
||||
echo "== B10: fresh beacon within SLO -> ALIVE (exit 0, no alarm) =="
|
||||
(
|
||||
H="$(fresh_home b10)"
|
||||
export ALARM_OUT="$H/alarm.json"
|
||||
export WAKE_BEACON_RECEIVED="$H/received.json"
|
||||
export WAKE_ALARM_SINK_CMD="$CAPTURE_ALARM"
|
||||
now="$(date +%s)"
|
||||
jq -cn --argjson ts "$now" \
|
||||
'{kind:"wake-beacon", beacon_seq:7, emit_ts:$ts, host_id:"h", independence:"off-host", degraded:false}' \
|
||||
>"$WAKE_BEACON_RECEIVED"
|
||||
out="$("$BEACON" check --slo-seconds 60 2>&1)"
|
||||
rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "B10: a fresh beacon within SLO must be ALIVE (exit 0); got $rc [$out]"
|
||||
echo "$out" | grep -qi 'ALIVE' || fail_msg "B10: a fresh beacon must report ALIVE [$out]"
|
||||
[ ! -s "$ALARM_OUT" ] || fail_msg "B10: a fresh beacon must NOT fire an alarm"
|
||||
) && ok
|
||||
|
||||
echo
|
||||
if [ -s "$FAILFILE" ]; then
|
||||
echo "wake beacon harness: FAILED ($(grep -c . "$FAILFILE") assertion(s))" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "wake beacon harness: all invariants passed ($pass groups)"
|
||||
@@ -25,7 +25,7 @@
|
||||
"lint": "eslint src",
|
||||
"typecheck": "tsc --noEmit",
|
||||
"test": "vitest run --passWithNoTests && pnpm run test:framework-shell",
|
||||
"test:framework-shell": "python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh"
|
||||
"test:framework-shell": "python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh"
|
||||
},
|
||||
"dependencies": {
|
||||
"@mosaicstack/brain": "workspace:*",
|
||||
|
||||
Reference in New Issue
Block a user