feat(wake): #958 A11 preimage.sh — durable provenance for the operator-side preimage definition
ci/woodpecker/pr/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
The operator-owned bytes every observed_hash is computed FROM (the source
adapter, the watch-list) were unversioned: a byte change was attributable
only via an agent transcript, and reconcile surfaced it as N UNACCOUNTED
sources instead of one cause.
New tool preimage.sh (wake 0.6.15 -> 0.7.0):
- Derives the preimage set from the runtime env (resolved
WAKE_DETECTOR_SOURCE_CMD file, WAKE_WATCH_LIST, WAKE_PREIMAGE_EXTRA);
per file appends {ts,path,sha256,size,mtime,prev} to an append-only
ledger and captures bytes content-addressed under
$STATE_DIR/preimage/objects/<sha256> — prior bytes + change time from
durable state alone (acceptance a).
- CREDENTIAL HARD GATE (acceptance b): byte capture is REFUSED — never
redacted — for credential-store paths, secret-shaped content (digest's
six scrub shapes as a content-deny), and files over
WAKE_PREIMAGE_MAX_BYTES; the refused file still gets its
hash/size/mtime row (captured:false) so change TIME survives.
- FIRST-CLASS CAUSE LINE (acceptance c): a change/deletion enqueues one
class=actionable entry via the store allocator with path as its §2.1
hard locator; detector poll-once and reconcile run the check as a
PRE-step so the cause line lands at a LOWER observed_seq than the
deltas/enumerations it explains.
- FAIL-LOUD (D2/#955 class): unresolvable adapter, corrupt ledger
(refuses re-baseline), failed object/ledger write, failed enqueue are
loud non-zero, never "no change"; in both integrations the pass exits
non-zero but source observation still proceeds.
Installer unchanged (Gate A auto-enumerates the new file; the recording
site is the runtime tick where the env-derived set exists). Watch-list
schema untouched ([1,1]).
Tests: test-wake-preimage.sh P1-P12 (red-first verified: P3/P11/P12 fail
with the integration edits reverted); all 9 existing wake suites green.
Refs #958
Agent: PEPPER (sb-it-1-dt)
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01NsKce8iZuSuRnu3gVMCBKB
Written-by: pepper (sb-it-1-dt)
This commit is contained in:
Jason Woltje
co-authored by
Claude Fable 5
Claude Fable 5
parent
6a7fce34bb
commit
e07943d742
@@ -414,6 +414,22 @@ cmd_poll_once() {
|
||||
_wake_clean_stale_tmp "$STATE_DIR"
|
||||
mkdir -p "$DET_DIR"
|
||||
|
||||
local failed=0
|
||||
|
||||
# #958 preimage provenance: check the OPERATOR-SIDE preimage definition (the
|
||||
# source adapter file, the watch-list, operator-declared extras) BEFORE
|
||||
# observing any source. A changed preimage re-baselines EVERY source at once;
|
||||
# running the check first means its first-class cause line is enqueued at a
|
||||
# LOWER observed_seq than the N per-source deltas it explains, so the digest
|
||||
# shows the cause, not just the flood. An infrastructure failure of the check
|
||||
# is LOUD and marks this pass failed (G2a discipline — never read as "no
|
||||
# change"), but source observation still proceeds: provenance must not be
|
||||
# able to starve wake delivery.
|
||||
if ! "$SCRIPT_DIR/preimage.sh" check --enqueue; then
|
||||
echo "detector.sh: FAIL LOUD — preimage provenance check failed (see preimage.sh above); source observation continues but this pass exits non-zero." >&2
|
||||
failed=1
|
||||
fi
|
||||
|
||||
# Iterate the DECLARED source-coverage inventory (§4/G3): only sources listed
|
||||
# in watches[].sources[] are polled. An omitted source is not observed (and so
|
||||
# cannot make anything pass vacuously); a referenced-but-undefined source is a
|
||||
@@ -427,7 +443,7 @@ cmd_poll_once() {
|
||||
exit 2
|
||||
fi
|
||||
|
||||
local failed=0 kind id def class
|
||||
local kind id def class
|
||||
while IFS=$'\t' read -r kind id; do
|
||||
[ -n "$kind" ] || continue
|
||||
# Resolve the source definition from its top-level collection by id.
|
||||
|
||||
@@ -374,8 +374,55 @@
|
||||
# Changed: store.sh, digest.sh, ack.sh
|
||||
# (+ test-wake-store-ack.sh T13-T16,
|
||||
# test-wake-digest-quarantine.sh Q12-Q16).
|
||||
# 0.7.0 #958 A11 preimage.sh — durable provenance for the OPERATOR-SIDE
|
||||
# preimage definition (wake-pilot finding: source-adapter.sh was
|
||||
# unversioned, so the operator-owned bytes every observed_hash is
|
||||
# computed FROM had thinner provenance than any hash they feed;
|
||||
# attribution required an agent transcript). NEW tool + two
|
||||
# pre-step integrations, ADDITIVE: (1) preimage.sh derives the
|
||||
# preimage set from the RUNTIME env (the resolved
|
||||
# WAKE_DETECTOR_SOURCE_CMD file, WAKE_WATCH_LIST, optional
|
||||
# WAKE_PREIMAGE_EXTRA paths) and, per file, records
|
||||
# {ts,path,sha256,size,mtime,prev} to an append-only ledger +
|
||||
# captures the bytes CONTENT-ADDRESSED under
|
||||
# $STATE_DIR/preimage/objects/<sha256> — prior bytes + change
|
||||
# time are answerable from durable state alone, no transcript
|
||||
# (acceptance a). A git-repo-in-operator-dir design was REJECTED:
|
||||
# its who/why claim is false under shared-author fleets, and
|
||||
# .gitignore is pattern-based where acceptance (b) demands
|
||||
# fail-closed. (2) CREDENTIAL HARD GATE (acceptance b): byte
|
||||
# capture is REFUSED — never redacted — for (i) credential-store
|
||||
# PATHS (mosaic-home credentials.json, tools/_lib/credentials.json,
|
||||
# credentials/**, any basename credentials.json), (ii)
|
||||
# secret-SHAPED content (digest.sh's six scrub shapes reused as a
|
||||
# content-deny grep), (iii) files over WAKE_PREIMAGE_MAX_BYTES
|
||||
# (default 1 MiB). A refused file still gets its hash/size/mtime
|
||||
# row (captured:false + refused reason) so change TIME survives
|
||||
# even when content must not. (3) FIRST-CLASS CAUSE LINE
|
||||
# (acceptance c): on a change (or deletion — ABSENT is a state,
|
||||
# not an error), one class=actionable entry is enqueued via the
|
||||
# store allocator with locators {kind:preimage, path (§2.1 hard
|
||||
# locator — never quarantined), observed_hash, prev_hash,
|
||||
# preimage:true, reason:preimage-definition-changed}. Both
|
||||
# detector.sh cmd_poll_once and reconcile.sh cmd_reconcile run
|
||||
# the check as a PRE-step, so the cause line lands at a LOWER
|
||||
# observed_seq than the N per-source deltas/enumerations it
|
||||
# explains — "preimage definition changed" reads first, not N
|
||||
# UNACCOUNTED lines. (4) FAIL-LOUD discipline (D2/#955 class):
|
||||
# an unresolvable adapter, unreadable watch-list, corrupt ledger
|
||||
# (REFUSES to compare or re-baseline over corrupt history), failed
|
||||
# object/ledger write, or failed enqueue is a loud non-zero —
|
||||
# never read as "no change"; in both integrations the pass exits
|
||||
# non-zero but source observation still proceeds (no starvation).
|
||||
# First-seen is a SILENT baseline (detector first-poll idiom).
|
||||
# The installer is UNCHANGED — Gate A auto-enumerates the new
|
||||
# file from the filesystem; the recording site is the runtime
|
||||
# tick, which is where the env-derived preimage set exists.
|
||||
# store.sh/digest.sh/beacon.sh UNCHANGED; watch-list schema
|
||||
# UNTOUCHED ([1,1]). Changed: preimage.sh (new), detector.sh,
|
||||
# reconcile.sh (+ test-wake-preimage.sh P1-P12).
|
||||
component=wake
|
||||
version=0.6.15
|
||||
version=0.7.0
|
||||
|
||||
# Watch-list schema this component consumes, and the INCLUSIVE range of
|
||||
# schema_version values it supports. A wake-watch-list.json whose schema_version
|
||||
@@ -446,6 +493,19 @@ schema_max=1
|
||||
# seed) instead of a bare source error, and LINKS the unit into
|
||||
# the user systemd search path + post-install-validates it
|
||||
# resolves (#913). (W7, #913)
|
||||
# preimage.sh A11 — operator-side PREIMAGE-DEFINITION provenance: derives the
|
||||
# preimage set from the runtime env (resolved adapter file,
|
||||
# watch-list, WAKE_PREIMAGE_EXTRA), appends
|
||||
# {ts,path,sha256,size,mtime,prev} rows to an append-only
|
||||
# ledger, captures bytes content-addressed under
|
||||
# preimage/objects/<sha256>, and enqueues a FIRST-CLASS
|
||||
# "preimage-definition-changed" actionable (path = §2.1 hard
|
||||
# locator) BEFORE the deltas it explains (detector +
|
||||
# reconcile pre-step). Credential HARD GATE: capture is
|
||||
# REFUSED (hash/mtime still recorded) for credential-store
|
||||
# paths, secret-shaped content, and oversized files —
|
||||
# refusal, never redaction. Fail-loud on any infra failure;
|
||||
# a corrupt ledger refuses re-baseline. (#958)
|
||||
# Companion (framework subtree, not under tools/wake/): systemd/user/mosaic-wake.service
|
||||
# — the long-lived detector daemon unit (per-class SLO lives in
|
||||
# the daemon, NOT a systemd interval). (W7)
|
||||
|
||||
+446
@@ -0,0 +1,446 @@
|
||||
#!/usr/bin/env bash
|
||||
# preimage.sh — A11 of the wake canon (#958): durable PROVENANCE for the
|
||||
# OPERATOR-SIDE preimage definition.
|
||||
#
|
||||
# THE GAP THIS CLOSES (#958): every observed_hash in the lane store is
|
||||
# sha256(adapter stdout) — the operator's source adapter (plus the watch-list
|
||||
# and any operator-declared preimage files) IS the preimage definition for
|
||||
# every hash the pipeline ever records. Those files live OUTSIDE the versioned
|
||||
# wake component, so a byte change to them was attributable only through agent
|
||||
# transcripts: provenance thinner than any hash it feeds. A changed preimage
|
||||
# also re-baselines EVERY source at once, which previously surfaced only as N
|
||||
# per-source deltas / UNACCOUNTED lines with no first-class cause.
|
||||
#
|
||||
# WHAT THIS TOOL DOES:
|
||||
# - Derives the PREIMAGE SET generically (no operator paths baked in):
|
||||
# * the resolved file behind WAKE_DETECTOR_SOURCE_CMD (first word),
|
||||
# * the WAKE_WATCH_LIST file,
|
||||
# * operator-declared extras via WAKE_PREIMAGE_EXTRA (colon-separated —
|
||||
# e.g. detector.env, sink/feed scripts).
|
||||
# - Records each file's (sha256, size, mtime, ts) as an APPEND-ONLY ledger
|
||||
# row in <state>/preimage/preimage-ledger.jsonl and stores the full bytes
|
||||
# CONTENT-ADDRESSED at <state>/preimage/objects/<sha256> — so a change is
|
||||
# attributable (prior bytes + change time) from durable state alone, with
|
||||
# no transcript required. Acceptance (a) of #958.
|
||||
# - On a CHANGE (not first-seen), `check --enqueue` enqueues ONE first-class
|
||||
# class=actionable entry per changed file via the store's single allocator
|
||||
# (#908), carrying a §2.1 hard locator (`path`), BEFORE the per-source
|
||||
# deltas it explains are observed — so the digest shows the cause line,
|
||||
# not just N re-baselined sources. Acceptance (c).
|
||||
#
|
||||
# CREDENTIAL HARD GATE — acceptance (b): the mechanism must be UNABLE to
|
||||
# capture credential material even by operator error. Enforced two ways, both
|
||||
# fail-closed toward NOT capturing bytes (the hash/size/mtime row is still
|
||||
# written — change-time attribution survives, content capture does not;
|
||||
# a sha256 discloses nothing about the bytes):
|
||||
# 1. PATH deny: the mosaic credential store locations
|
||||
# (<mosaic-home>/credentials.json, <mosaic-home>/tools/_lib/
|
||||
# credentials.json, anything under <mosaic-home>/credentials/, and any
|
||||
# file literally named credentials.json) are refused by resolved realpath
|
||||
# before a single byte is read into the object store.
|
||||
# 2. CONTENT deny: a candidate whose bytes match the well-known secret-token
|
||||
# shapes (same conservative set digest.sh redacts: PAT/Slack/AKIA/JWT/PEM)
|
||||
# is refused — refusal, not redaction: a redacted preimage would be a
|
||||
# false witness, and secret bytes must never land in the object store.
|
||||
# Plus a size cap (WAKE_PREIMAGE_MAX_BYTES, default 1 MiB) so a stray extra
|
||||
# pointing at a large binary cannot turn provenance into data hoovering.
|
||||
#
|
||||
# FAIL-LOUD DISCIPLINE (the D2/#955 class, both layers): an infrastructure
|
||||
# failure of THIS tool (unresolvable adapter, unreadable/corrupt ledger,
|
||||
# failed durable write, failed enqueue) exits NON-ZERO and is never read as
|
||||
# "no change". A corrupt ledger REFUSES to compare (and to re-baseline):
|
||||
# silently restarting history would erase the very attribution this exists
|
||||
# to provide.
|
||||
#
|
||||
# Operator-agnostic (framework firewall): all state via XDG/env; the deny
|
||||
# list names only framework-defined credential locations, no operator hosts/
|
||||
# names/secrets. This tool never prints file CONTENT to any stream.
|
||||
set -uo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
# shellcheck source=./_wake-common.sh disable=SC1091
|
||||
. "$SCRIPT_DIR/_wake-common.sh"
|
||||
|
||||
STATE_DIR="$(wake_state_dir)"
|
||||
STORE_SH="$SCRIPT_DIR/store.sh"
|
||||
PRE_DIR="$STATE_DIR/preimage"
|
||||
LEDGER="$PRE_DIR/preimage-ledger.jsonl"
|
||||
OBJECTS="$PRE_DIR/objects"
|
||||
|
||||
_need_jq() {
|
||||
command -v jq >/dev/null 2>&1 || {
|
||||
echo "preimage.sh: jq is required" >&2
|
||||
exit 3
|
||||
}
|
||||
}
|
||||
|
||||
# _hash_stdin — sha256 of stdin, first field only (portable, same fallback
|
||||
# chain as detector.sh).
|
||||
_hash_stdin() {
|
||||
if command -v sha256sum >/dev/null 2>&1; then
|
||||
sha256sum | awk '{print $1}'
|
||||
elif command -v shasum >/dev/null 2>&1; then
|
||||
shasum -a 256 | awk '{print $1}'
|
||||
elif command -v openssl >/dev/null 2>&1; then
|
||||
openssl dgst -sha256 | awk '{print $NF}'
|
||||
else
|
||||
echo "preimage.sh: no sha256 tool (sha256sum/shasum/openssl) available" >&2
|
||||
exit 3
|
||||
fi
|
||||
}
|
||||
|
||||
usage() {
|
||||
cat >&2 <<'EOF'
|
||||
Usage: preimage.sh <command>
|
||||
|
||||
Commands:
|
||||
check [--enqueue] Compare every preimage-set file against the ledger head.
|
||||
A changed file gets a new ledger row + captured bytes;
|
||||
with --enqueue each change (not first-seen) also enqueues
|
||||
ONE first-class class=actionable store entry (hard
|
||||
locator: path). First-seen files baseline SILENTLY (row,
|
||||
no enqueue — same idiom as the detector's first-seen).
|
||||
Exit 0 whether or not changes were found; non-zero ONLY
|
||||
on infrastructure failure (never read as "no change").
|
||||
record Alias of `check` without enqueue (baseline/refresh).
|
||||
set Print the derived preimage set, one resolved path per
|
||||
line (diagnostic).
|
||||
status Print the ledger head row for every tracked path.
|
||||
|
||||
Environment:
|
||||
WAKE_DETECTOR_SOURCE_CMD Adapter command; its resolved file joins the set.
|
||||
WAKE_WATCH_LIST Watch-list path; joins the set when set.
|
||||
WAKE_PREIMAGE_EXTRA Colon-separated additional operator preimage files
|
||||
(e.g. detector.env, sink scripts). A listed path
|
||||
that does not exist is tracked as ABSENT (deletion
|
||||
of a preimage file is a change, not an error).
|
||||
WAKE_PREIMAGE_MAX_BYTES Byte-capture cap (default 1048576). Larger files:
|
||||
hash/size/mtime recorded, bytes refused.
|
||||
WAKE_STATE_HOME/WAKE_AGENT store namespace (see store.sh).
|
||||
EOF
|
||||
}
|
||||
|
||||
# --- preimage-set derivation (generic; no operator paths baked in) ----------
|
||||
|
||||
# _resolve_cmd_file CMD — resolve the FIRST WORD of an adapter command string
|
||||
# to a real file. Non-zero (loud) if it cannot be resolved: an adapter the
|
||||
# detector will invoke but provenance cannot see is an infrastructure failure,
|
||||
# not a smaller set.
|
||||
_resolve_cmd_file() {
|
||||
local cmd="$1" word path
|
||||
# shellcheck disable=SC2086
|
||||
set -- $cmd
|
||||
word="${1:-}"
|
||||
[ -n "$word" ] || return 1
|
||||
if [ -f "$word" ]; then
|
||||
path="$word"
|
||||
else
|
||||
path="$(command -v -- "$word" 2>/dev/null)" || return 1
|
||||
[ -f "$path" ] || return 1
|
||||
fi
|
||||
# realpath so the ledger keys on the actual file, not a symlink alias.
|
||||
if command -v realpath >/dev/null 2>&1; then
|
||||
realpath -- "$path" 2>/dev/null || printf '%s' "$path"
|
||||
else
|
||||
printf '%s' "$path"
|
||||
fi
|
||||
}
|
||||
|
||||
# _preimage_set — print the derived set, one path per line. Paths from
|
||||
# WAKE_PREIMAGE_EXTRA are printed EVEN IF ABSENT (deletion is a tracked
|
||||
# state); the adapter and watch-list must resolve (loud failure otherwise —
|
||||
# see _resolve_cmd_file rationale).
|
||||
_preimage_set() {
|
||||
local failed=0
|
||||
if [ -n "${WAKE_DETECTOR_SOURCE_CMD:-}" ]; then
|
||||
local af
|
||||
if af="$(_resolve_cmd_file "$WAKE_DETECTOR_SOURCE_CMD")"; then
|
||||
printf '%s\n' "$af"
|
||||
else
|
||||
echo "preimage.sh: FAIL LOUD — WAKE_DETECTOR_SOURCE_CMD ('$WAKE_DETECTOR_SOURCE_CMD') does not resolve to a file; the preimage definition cannot be observed (NOT treated as 'no change')." >&2
|
||||
failed=1
|
||||
fi
|
||||
fi
|
||||
if [ -n "${WAKE_WATCH_LIST:-}" ]; then
|
||||
if [ -f "$WAKE_WATCH_LIST" ]; then
|
||||
if command -v realpath >/dev/null 2>&1; then
|
||||
realpath -- "$WAKE_WATCH_LIST" 2>/dev/null || printf '%s' "$WAKE_WATCH_LIST"
|
||||
else
|
||||
printf '%s' "$WAKE_WATCH_LIST"
|
||||
fi
|
||||
printf '\n'
|
||||
else
|
||||
echo "preimage.sh: FAIL LOUD — WAKE_WATCH_LIST ('$WAKE_WATCH_LIST') is not a file; the preimage definition cannot be observed." >&2
|
||||
failed=1
|
||||
fi
|
||||
fi
|
||||
if [ -n "${WAKE_PREIMAGE_EXTRA:-}" ]; then
|
||||
local IFS=':' p
|
||||
for p in $WAKE_PREIMAGE_EXTRA; do
|
||||
[ -n "$p" ] || continue
|
||||
# Extras are tracked even when absent (ABSENT is a state). Resolve the
|
||||
# realpath only when the file exists; otherwise track the literal path.
|
||||
if [ -e "$p" ] && command -v realpath >/dev/null 2>&1; then
|
||||
realpath -- "$p" 2>/dev/null || printf '%s' "$p"
|
||||
printf '\n'
|
||||
else
|
||||
printf '%s\n' "$p"
|
||||
fi
|
||||
done
|
||||
fi
|
||||
return "$failed"
|
||||
}
|
||||
|
||||
# --- credential hard gate (acceptance (b)) ----------------------------------
|
||||
|
||||
# _deny_path PATH — 0 (deny) iff PATH is a known credential-store location.
|
||||
# Framework-defined locations only (firewall): the mosaic credential store,
|
||||
# the tools/_lib credential file the framework-manifest itself carves out of
|
||||
# tools/**, the credentials/ operator subtree, and any file literally named
|
||||
# credentials.json.
|
||||
_deny_path() {
|
||||
local p="$1" home="${MOSAIC_HOME:-$HOME/.config/mosaic}"
|
||||
case "$p" in
|
||||
"$home/credentials.json") return 0 ;;
|
||||
"$home/tools/_lib/credentials.json") return 0 ;;
|
||||
"$home/credentials/"*) return 0 ;;
|
||||
esac
|
||||
case "$(basename -- "$p")" in
|
||||
credentials.json) return 0 ;;
|
||||
esac
|
||||
return 1
|
||||
}
|
||||
|
||||
# _deny_content FILE — 0 (deny) iff FILE's bytes match a well-known secret-
|
||||
# token shape. Same conservative shape set digest.sh redacts (PAT / Slack /
|
||||
# AKIA / JWT / PEM) — conservative on purpose: a 40-hex git sha never matches.
|
||||
_deny_content() {
|
||||
LC_ALL=C grep -Eq \
|
||||
-e 'gh[pousr]_[A-Za-z0-9]{16,}' \
|
||||
-e 'github_pat_[A-Za-z0-9_]{16,}' \
|
||||
-e 'xox[baprs]-[A-Za-z0-9-]{10,}' \
|
||||
-e 'AKIA[0-9A-Z]{16}' \
|
||||
-e 'eyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}' \
|
||||
-e '-----BEGIN[A-Z ]*PRIVATE KEY-----' \
|
||||
-- "$1" 2>/dev/null
|
||||
}
|
||||
|
||||
# --- ledger primitives ------------------------------------------------------
|
||||
|
||||
# _ledger_head PATH — print the LAST ledger row for PATH (empty if none).
|
||||
# Non-zero (loud) if the ledger exists but is unparseable: a corrupt ledger
|
||||
# must REFUSE to compare, never silently re-baseline (that would erase the
|
||||
# attribution history this tool exists to keep).
|
||||
_ledger_head() {
|
||||
local path="$1"
|
||||
[ -f "$LEDGER" ] || return 0
|
||||
if [ -s "$LEDGER" ] && ! jq -cn 'inputs' <"$LEDGER" >/dev/null 2>&1; then
|
||||
echo "preimage.sh: FAIL LOUD — preimage ledger $LEDGER is unparseable; REFUSING to compare or re-baseline over corrupt history. Investigate/restore the ledger." >&2
|
||||
return 1
|
||||
fi
|
||||
jq -c --arg p "$path" 'select(.path == $p)' "$LEDGER" 2>/dev/null | tail -n 1
|
||||
}
|
||||
|
||||
# _ledger_append ROW_JSON — append one row atomically (read + append + rename;
|
||||
# rows are small and the writer is serialized by the preimage lock).
|
||||
_ledger_append() {
|
||||
local row="$1"
|
||||
{
|
||||
[ -f "$LEDGER" ] && cat "$LEDGER"
|
||||
printf '%s\n' "$row"
|
||||
} | _atomic_write "$LEDGER"
|
||||
}
|
||||
|
||||
# --- the check itself -------------------------------------------------------
|
||||
|
||||
# _check_one PATH ENQUEUE — compare PATH to its ledger head; on change record
|
||||
# (+bytes unless denied) and optionally enqueue. Prints nothing on no-change.
|
||||
# Returns: 0 ok (changed or not), 1 infrastructure failure.
|
||||
_check_one() {
|
||||
local path="$1" enqueue="$2"
|
||||
local cur_sha size mtime denied="" refused=""
|
||||
|
||||
if [ -f "$path" ]; then
|
||||
cur_sha="$(_hash_stdin <"$path")" || return 1
|
||||
[ -n "$cur_sha" ] || {
|
||||
echo "preimage.sh: FAIL LOUD — could not hash $path" >&2
|
||||
return 1
|
||||
}
|
||||
size="$(wc -c <"$path" | tr -d '[:space:]')"
|
||||
# Portable mtime (GNU stat -c / BSD stat -f).
|
||||
mtime="$(stat -c %Y -- "$path" 2>/dev/null || stat -f %m -- "$path" 2>/dev/null || echo 0)"
|
||||
local cap="${WAKE_PREIMAGE_MAX_BYTES:-1048576}"
|
||||
case "$cap" in '' | *[!0-9]*) cap=1048576 ;; esac
|
||||
# Deny order matters: path first (a known credential store is refused
|
||||
# without a content probe — the hash read above is deliberate: a sha256
|
||||
# discloses nothing about the bytes), size cap second (never content-grep
|
||||
# an oversized file), content shape last.
|
||||
if _deny_path "$path"; then
|
||||
denied="credential-store path (deny list)"
|
||||
elif [ "$size" -gt "$cap" ]; then
|
||||
denied="exceeds WAKE_PREIMAGE_MAX_BYTES ($size > $cap)"
|
||||
elif _deny_content "$path"; then
|
||||
denied="secret-shaped content"
|
||||
fi
|
||||
else
|
||||
# ABSENT is a state (deletion of a preimage file is a change to record).
|
||||
cur_sha="ABSENT"
|
||||
size=0
|
||||
mtime=0
|
||||
fi
|
||||
|
||||
local head prev_sha=""
|
||||
head="$(_ledger_head "$path")" || return 1
|
||||
[ -n "$head" ] && prev_sha="$(jq -r '.sha256 // ""' <<<"$head")"
|
||||
|
||||
if [ "$cur_sha" = "$prev_sha" ]; then
|
||||
return 0 # unchanged — no row, no enqueue (idempotent)
|
||||
fi
|
||||
|
||||
# --- capture bytes (content-addressed) unless the hard gate refuses -------
|
||||
local captured=true
|
||||
if [ "$cur_sha" != "ABSENT" ]; then
|
||||
if [ -n "$denied" ]; then
|
||||
captured=false
|
||||
refused="$denied"
|
||||
echo "preimage.sh: REFUSED byte capture for $path ($denied) — hash/size/mtime recorded, content NOT stored (acceptance (b), #958)." >&2
|
||||
else
|
||||
mkdir -p "$OBJECTS" || return 1
|
||||
if [ ! -f "$OBJECTS/$cur_sha" ]; then
|
||||
if ! _atomic_write "$OBJECTS/$cur_sha" <"$path"; then
|
||||
echo "preimage.sh: FAIL LOUD — durable object write failed for $path ($OBJECTS/$cur_sha); NO ledger row recorded (a row whose bytes were never durably kept would be a false witness)." >&2
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
local ts row
|
||||
ts="$(date +%s)"
|
||||
row="$(jq -cn \
|
||||
--arg path "$path" \
|
||||
--arg sha "$cur_sha" \
|
||||
--arg prev "$prev_sha" \
|
||||
--argjson size "$size" \
|
||||
--argjson mtime "$mtime" \
|
||||
--argjson ts "$ts" \
|
||||
--argjson captured "$captured" \
|
||||
--arg refused "$refused" \
|
||||
'{ts:$ts, path:$path, sha256:$sha, size:$size, mtime:$mtime, captured:$captured}
|
||||
+ (if $prev != "" then {prev:$prev} else {} end)
|
||||
+ (if $refused != "" then {refused:$refused} else {} end)')" || return 1
|
||||
if ! _ledger_append "$row"; then
|
||||
echo "preimage.sh: FAIL LOUD — durable ledger append failed ($LEDGER)" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
if [ -z "$head" ]; then
|
||||
# First-seen: baseline SILENTLY (row only, no enqueue) — the same idiom as
|
||||
# the detector's first-seen source baseline.
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo "preimage.sh: preimage definition CHANGED — $path ($prev_sha -> $cur_sha); prior bytes: $OBJECTS/$prev_sha" >&2
|
||||
|
||||
if [ "$enqueue" = "1" ]; then
|
||||
# First-class cause line (acceptance (c)): ONE class=actionable entry per
|
||||
# changed preimage file, allocated by the store's single allocator (#908).
|
||||
# `path` is a §2.1 hard locator, so the digest renders it — never
|
||||
# quarantined. Callers run this BEFORE polling sources, so this seq is
|
||||
# LOWER than the per-source deltas the change explains.
|
||||
local locators
|
||||
locators="$(jq -cn \
|
||||
--arg path "$path" \
|
||||
--arg sha "$cur_sha" \
|
||||
--arg prev "$prev_sha" \
|
||||
'{kind:"preimage", id:$path, path:$path, observed_hash:$sha, prev_hash:$prev,
|
||||
preimage:true, reason:"preimage-definition-changed"}')"
|
||||
if ! "$STORE_SH" enqueue --class actionable --locators "$locators" --emit-ts "$(date +%s)" >/dev/null; then
|
||||
echo "preimage.sh: FAIL LOUD — store enqueue of the preimage-change entry FAILED for $path (the change IS recorded in the ledger; the first-class wake line is NOT — treat as infrastructure failure)." >&2
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
cmd_check() {
|
||||
local enqueue=0
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--enqueue)
|
||||
enqueue=1
|
||||
shift
|
||||
;;
|
||||
*)
|
||||
echo "preimage.sh check: unknown option '$1'" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
done
|
||||
_need_jq
|
||||
|
||||
local set_list
|
||||
if ! set_list="$(_preimage_set)"; then
|
||||
exit 1
|
||||
fi
|
||||
if [ -z "$set_list" ]; then
|
||||
# Nothing declared (no adapter/watch-list/extras in env) — an empty set is
|
||||
# a no-op, not an error: standalone invocations outside a wake runtime
|
||||
# must not fail loud for lacking one.
|
||||
return 0
|
||||
fi
|
||||
|
||||
mkdir -p "$PRE_DIR" || exit 1
|
||||
# Serialize concurrent checks (detector tick vs reconcile vs manual) — the
|
||||
# ledger append is read+rewrite, so two writers must not interleave.
|
||||
if ! _wake_lock_acquire "$PRE_DIR/preimage.lock"; then
|
||||
echo "preimage.sh: FAIL LOUD — cannot acquire preimage lock" >&2
|
||||
exit 1
|
||||
fi
|
||||
local failed=0 p
|
||||
while IFS= read -r p; do
|
||||
[ -n "$p" ] || continue
|
||||
_check_one "$p" "$enqueue" || failed=1
|
||||
done <<EOF
|
||||
$set_list
|
||||
EOF
|
||||
_wake_lock_release
|
||||
[ "$failed" -eq 0 ] || exit 1
|
||||
}
|
||||
|
||||
cmd_status() {
|
||||
_need_jq
|
||||
[ -f "$LEDGER" ] || {
|
||||
echo "preimage.sh: no ledger yet ($LEDGER)" >&2
|
||||
return 0
|
||||
}
|
||||
# Head row per path (last wins).
|
||||
jq -cs 'group_by(.path) | map(last) | .[]' "$LEDGER"
|
||||
}
|
||||
|
||||
cmd_set() {
|
||||
_preimage_set || exit 1
|
||||
}
|
||||
|
||||
main() {
|
||||
[ $# -ge 1 ] || {
|
||||
usage
|
||||
exit 2
|
||||
}
|
||||
local cmd="$1"
|
||||
shift
|
||||
case "$cmd" in
|
||||
check) cmd_check "$@" ;;
|
||||
record) cmd_check ;;
|
||||
set) cmd_set "$@" ;;
|
||||
status) cmd_status "$@" ;;
|
||||
-h | --help | help) usage ;;
|
||||
*)
|
||||
echo "preimage.sh: unknown command '$cmd'" >&2
|
||||
usage
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
main "$@"
|
||||
@@ -440,6 +440,17 @@ cmd_reconcile() {
|
||||
: "$allow_enumerate"
|
||||
|
||||
local pairs kind id failed=0 unaccounted=0 enumerated=0
|
||||
|
||||
# #958 preimage provenance: same pre-step as the detector's poll tick, for
|
||||
# the path where the preimage changed while the detector was down — without
|
||||
# it, a changed adapter surfaces here only as N UNACCOUNTED enumerations
|
||||
# with no first-class cause line. Run BEFORE observing any source so the
|
||||
# cause entry's observed_seq precedes the enumerations it explains. Loud
|
||||
# infrastructure failure marks the reconcile failed but does not stop it.
|
||||
if ! "$SCRIPT_DIR/preimage.sh" check --enqueue; then
|
||||
echo "reconcile.sh: FAIL LOUD — preimage provenance check failed (see preimage.sh above); reconcile continues but exits non-zero." >&2
|
||||
failed=1
|
||||
fi
|
||||
pairs="$(printf '%s' "$json" | jq -r '[ .watches[].sources[] | "\(.kind)\t\(.id)" ] | unique | .[]')"
|
||||
if [ -z "$pairs" ]; then
|
||||
echo "reconcile.sh: watch-list declares no sources under watches[].sources[]" >&2
|
||||
|
||||
+348
@@ -0,0 +1,348 @@
|
||||
#!/usr/bin/env bash
|
||||
# test-wake-preimage.sh — RED-FIRST invariant harness for A11 (#958): durable
|
||||
# provenance for the OPERATOR-SIDE preimage definition (preimage.sh).
|
||||
#
|
||||
# Each test asserts ONE #958 invariant and is designed to go RED if it
|
||||
# regresses:
|
||||
# P1 first-seen -> SILENT baseline (ledger rows, NO enqueue) (detector idiom)
|
||||
# P2 change -> attributable from durable state alone: new ledger row with
|
||||
# prev, PRIOR BYTES retrievable content-addressed, first-class actionable
|
||||
# enqueued with a §2.1 hard locator (path) (acceptance a+c)
|
||||
# P3 detector ordering: the preimage cause line is enqueued at a LOWER
|
||||
# observed_seq than the per-source delta it explains (acceptance c)
|
||||
# P4 credential-store PATH is refused — bytes NEVER captured, even when the
|
||||
# operator lists it by error (hash/mtime row still written) (acceptance b)
|
||||
# P5 secret-SHAPED content is refused — refusal, not redaction (acceptance b)
|
||||
# P6 deletion of a preimage file is a tracked CHANGE (ABSENT), not an error
|
||||
# P7 no-change is idempotent (no row growth, no enqueue)
|
||||
# P8 unresolvable adapter command -> FAIL LOUD, never "no change" (D2/#955 class)
|
||||
# P9 corrupt ledger -> REFUSE to compare/re-baseline (loud, no append)
|
||||
# P10 oversized file -> bytes refused, hash still recorded (no data hoovering)
|
||||
# P11 reconcile pre-step: a preimage change surfaces as a first-class line
|
||||
# BEFORE the UNACCOUNTED enumerations it explains (acceptance c)
|
||||
# P12 detector: preimage infra failure is LOUD (pass exits non-zero) but does
|
||||
# NOT starve source observation
|
||||
#
|
||||
# Uses FAKE/STUB sources only (no live network). Isolated per test.
|
||||
# shellcheck disable=SC2030,SC2031
|
||||
set -uo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PRE="$SCRIPT_DIR/preimage.sh"
|
||||
STORE="$SCRIPT_DIR/store.sh"
|
||||
DET="$SCRIPT_DIR/detector.sh"
|
||||
RECON="$SCRIPT_DIR/reconcile.sh"
|
||||
|
||||
command -v jq >/dev/null 2>&1 || {
|
||||
echo "SKIP: jq not available" >&2
|
||||
exit 0
|
||||
}
|
||||
|
||||
TMP_ROOT="$(mktemp -d)"
|
||||
trap 'rm -rf "$TMP_ROOT"' EXIT
|
||||
|
||||
FAILFILE="$TMP_ROOT/failures"
|
||||
: >"$FAILFILE"
|
||||
pass=0
|
||||
fail_msg() {
|
||||
echo " FAIL: $*" >&2
|
||||
echo "x" >>"$FAILFILE"
|
||||
}
|
||||
ok() { pass=$((pass + 1)); }
|
||||
|
||||
fresh_state() {
|
||||
local d="$TMP_ROOT/$1"
|
||||
rm -rf "$d"
|
||||
mkdir -p "$d"
|
||||
printf '%s' "$d"
|
||||
}
|
||||
depth() { "$STORE" cursors | sed -n 's/pending_depth=//p'; }
|
||||
state_dir() { printf '%s/default' "$WAKE_STATE_HOME"; }
|
||||
ledger_rows() {
|
||||
local f
|
||||
f="$(state_dir)/preimage/preimage-ledger.jsonl"
|
||||
[ -f "$f" ] && wc -l <"$f" | tr -d '[:space:]' || echo 0
|
||||
}
|
||||
|
||||
# make_stub DIR — a source adapter reading $DIR/<kind>_<id> (same shape as the
|
||||
# reconcile harness stub).
|
||||
make_stub() {
|
||||
local dir="$1"
|
||||
cat >"$dir/adapter.sh" <<EOF
|
||||
#!/usr/bin/env bash
|
||||
set -u
|
||||
kind="\$1"; id="\$2"; base="$dir/\${kind}_\${id}"; rc=0
|
||||
[ -f "\$base.rc" ] && rc="\$(cat "\$base.rc")"
|
||||
[ -f "\$base" ] && cat "\$base"
|
||||
exit "\$rc"
|
||||
EOF
|
||||
chmod +x "$dir/adapter.sh"
|
||||
}
|
||||
|
||||
make_wl() { # make_wl FILE — one repo source r1
|
||||
cat >"$1" <<'EOF'
|
||||
{ "schema_version": 1,
|
||||
"repos": [ { "id": "r1" } ],
|
||||
"watches": [ { "lane": "L", "sources": [ { "kind": "repo", "id": "r1" } ] } ] }
|
||||
EOF
|
||||
}
|
||||
|
||||
echo "== P1: first-seen -> SILENT baseline (rows, no enqueue) =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p1)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p1fx"; mkdir -p "$fx"
|
||||
make_stub "$fx"; make_wl "$fx/wl.json"
|
||||
printf 'extra v1\n' >"$fx/extra.env"
|
||||
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json" WAKE_PREIMAGE_EXTRA="$fx/extra.env"
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "P1: baseline check must exit 0 (rc=$rc) [$out]"
|
||||
[ "$(ledger_rows)" -eq 3 ] || fail_msg "P1: expected 3 baseline rows (adapter, watch-list, extra), got $(ledger_rows)"
|
||||
[ "$(depth)" = "0" ] || fail_msg "P1: first-seen must NOT enqueue (depth=$(depth))"
|
||||
) && ok
|
||||
|
||||
echo "== P2: change -> prior bytes + first-class hard-locator enqueue =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p2)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p2fx"; mkdir -p "$fx"
|
||||
make_stub "$fx"; make_wl "$fx/wl.json"
|
||||
printf 'adapter preimage v1\n' >"$fx/extra.sh"
|
||||
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json" WAKE_PREIMAGE_EXTRA="$fx/extra.sh"
|
||||
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P2: baseline failed"
|
||||
old_sha="$(sha256sum "$fx/extra.sh" | awk '{print $1}')"
|
||||
printf 'adapter preimage v2 CHANGED\n' >"$fx/extra.sh"
|
||||
new_sha="$(sha256sum "$fx/extra.sh" | awk '{print $1}')"
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "P2: change check must exit 0 (rc=$rc) [$out]"
|
||||
sd="$(state_dir)"
|
||||
row="$(jq -c --arg p "$(realpath "$fx/extra.sh")" 'select(.path == $p)' "$sd/preimage/preimage-ledger.jsonl" | tail -n1)"
|
||||
[ "$(jq -r '.prev // ""' <<<"$row")" = "$old_sha" ] || fail_msg "P2: new row must carry prev=<old sha> [$row]"
|
||||
[ "$(jq -r '.sha256' <<<"$row")" = "$new_sha" ] || fail_msg "P2: new row sha mismatch [$row]"
|
||||
# Acceptance (a): the PRIOR BYTES are retrievable from durable state alone.
|
||||
[ -f "$sd/preimage/objects/$old_sha" ] || fail_msg "P2: prior bytes object missing"
|
||||
[ "$(cat "$sd/preimage/objects/$old_sha")" = "adapter preimage v1" ] || fail_msg "P2: prior bytes not byte-exact"
|
||||
[ -f "$sd/preimage/objects/$new_sha" ] || fail_msg "P2: current bytes object missing"
|
||||
# Acceptance (c): ONE first-class actionable with a §2.1 hard locator (path).
|
||||
[ "$(depth)" = "1" ] || fail_msg "P2: exactly one enqueue expected (depth=$(depth))"
|
||||
ent="$(tail -n1 "$sd/pending.jsonl")"
|
||||
[ "$(jq -r '.class' <<<"$ent")" = "actionable" ] || fail_msg "P2: entry class must be actionable [$ent]"
|
||||
[ "$(jq -r '.locators.kind' <<<"$ent")" = "preimage" ] || fail_msg "P2: locator kind must be preimage [$ent]"
|
||||
[ -n "$(jq -r '.locators.path // ""' <<<"$ent")" ] || fail_msg "P2: locator must carry path (hard locator) [$ent]"
|
||||
[ "$(jq -r '.locators.observed_hash' <<<"$ent")" = "$new_sha" ] || fail_msg "P2: locator observed_hash mismatch [$ent]"
|
||||
[ "$(jq -r '.locators.prev_hash' <<<"$ent")" = "$old_sha" ] || fail_msg "P2: locator prev_hash mismatch [$ent]"
|
||||
) && ok
|
||||
|
||||
echo "== P3: detector orders the cause line BEFORE the delta it explains =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p3)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p3fx"; mkdir -p "$fx"
|
||||
make_stub "$fx"; make_wl "$fx/wl.json"
|
||||
printf 'r1 state v1\n' >"$fx/repo_r1"
|
||||
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json"
|
||||
unset WAKE_PREIMAGE_EXTRA
|
||||
"$DET" poll-once >/dev/null 2>&1 || fail_msg "P3: baseline poll failed"
|
||||
[ "$(depth)" = "0" ] || fail_msg "P3: baseline poll must enqueue nothing (depth=$(depth))"
|
||||
# Change the ADAPTER (preimage) and the source state in the same window.
|
||||
printf '# comment: adapter changed\n' >>"$fx/adapter.sh"
|
||||
printf 'r1 state v2\n' >"$fx/repo_r1"
|
||||
"$DET" poll-once >/dev/null 2>&1 || fail_msg "P3: second poll failed"
|
||||
sd="$(state_dir)"
|
||||
pre_seq="$(jq -r 'select(.locators.kind == "preimage") | .observed_seq' "$sd/pending.jsonl" | head -n1)"
|
||||
src_seq="$(jq -r 'select(.locators.kind == "repo") | .observed_seq' "$sd/pending.jsonl" | head -n1)"
|
||||
[ -n "$pre_seq" ] || fail_msg "P3: no preimage cause entry enqueued"
|
||||
[ -n "$src_seq" ] || fail_msg "P3: no source delta entry enqueued"
|
||||
if [ -n "$pre_seq" ] && [ -n "$src_seq" ]; then
|
||||
[ "$pre_seq" -lt "$src_seq" ] || fail_msg "P3: cause line must precede the delta (preimage seq=$pre_seq, source seq=$src_seq)"
|
||||
fi
|
||||
) && ok
|
||||
|
||||
echo "== P4: credential-store PATH refused — bytes never captured (acceptance b) =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p4)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p4fx"; mkdir -p "$fx"
|
||||
export MOSAIC_HOME="$fx/mosaic-home"
|
||||
mkdir -p "$MOSAIC_HOME"
|
||||
secret='cred-value-P4-do-not-capture'
|
||||
printf '{"svc":{"token":"%s"}}\n' "$secret" >"$MOSAIC_HOME/credentials.json"
|
||||
# Operator error: the credential store listed as a preimage extra.
|
||||
export WAKE_PREIMAGE_EXTRA="$MOSAIC_HOME/credentials.json"
|
||||
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "P4: refusal is not an infra failure (rc=$rc) [$out]"
|
||||
sd="$(state_dir)"
|
||||
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
|
||||
[ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P4: row must record captured=false [$row]"
|
||||
jq -r '.refused // ""' <<<"$row" | grep -qi 'path' || fail_msg "P4: refusal must name the path deny [$row]"
|
||||
# THE invariant: the secret bytes exist NOWHERE under the wake state dir.
|
||||
if grep -rq "$secret" "$sd" 2>/dev/null; then
|
||||
fail_msg "P4: credential bytes leaked into the wake state dir"
|
||||
fi
|
||||
) && ok
|
||||
|
||||
echo "== P5: secret-SHAPED content refused (refusal, not redaction) =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p5)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p5fx"; mkdir -p "$fx"
|
||||
tok='ghp_abcdefghijklmnop0123456789ABCDEF'
|
||||
printf 'export MY_TOKEN=%s\n' "$tok" >"$fx/leaky.env"
|
||||
export WAKE_PREIMAGE_EXTRA="$fx/leaky.env"
|
||||
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "P5: refusal is not an infra failure (rc=$rc) [$out]"
|
||||
sd="$(state_dir)"
|
||||
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
|
||||
[ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P5: row must record captured=false [$row]"
|
||||
jq -r '.refused // ""' <<<"$row" | grep -qi 'secret' || fail_msg "P5: refusal must name secret-shaped content [$row]"
|
||||
if grep -rq "$tok" "$sd" 2>/dev/null; then
|
||||
fail_msg "P5: secret-shaped bytes leaked into the wake state dir"
|
||||
fi
|
||||
) && ok
|
||||
|
||||
echo "== P6: deletion is a tracked CHANGE (ABSENT), not an error =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p6)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p6fx"; mkdir -p "$fx"
|
||||
printf 'to be deleted\n' >"$fx/gone.env"
|
||||
export WAKE_PREIMAGE_EXTRA="$fx/gone.env"
|
||||
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
|
||||
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P6: baseline failed"
|
||||
rm -f "$fx/gone.env"
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "P6: deletion check must exit 0 (rc=$rc) [$out]"
|
||||
sd="$(state_dir)"
|
||||
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
|
||||
[ "$(jq -r '.sha256' <<<"$row")" = "ABSENT" ] || fail_msg "P6: deletion must record sha256=ABSENT [$row]"
|
||||
[ "$(depth)" = "1" ] || fail_msg "P6: deletion is a change and must enqueue (depth=$(depth))"
|
||||
) && ok
|
||||
|
||||
echo "== P7: no-change is idempotent =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p7)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p7fx"; mkdir -p "$fx"
|
||||
printf 'stable\n' >"$fx/stable.env"
|
||||
export WAKE_PREIMAGE_EXTRA="$fx/stable.env"
|
||||
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
|
||||
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P7: baseline failed"
|
||||
r1="$(ledger_rows)"
|
||||
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P7: second check failed"
|
||||
[ "$(ledger_rows)" = "$r1" ] || fail_msg "P7: no-change must not append rows ($r1 -> $(ledger_rows))"
|
||||
[ "$(depth)" = "0" ] || fail_msg "P7: no-change must not enqueue (depth=$(depth))"
|
||||
) && ok
|
||||
|
||||
echo "== P8: unresolvable adapter -> FAIL LOUD, never 'no change' (D2 class) =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p8)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
export WAKE_DETECTOR_SOURCE_CMD="$TMP_ROOT/does-not-exist-adapter"
|
||||
unset WAKE_WATCH_LIST WAKE_PREIMAGE_EXTRA MOSAIC_HOME
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -ne 0 ] || fail_msg "P8: an unobservable preimage definition must FAIL LOUD (rc=0)"
|
||||
echo "$out" | grep -qi 'does not resolve' || fail_msg "P8: the failure must name the unresolvable adapter [$out]"
|
||||
) && ok
|
||||
|
||||
echo "== P9: corrupt ledger -> REFUSE to compare/re-baseline =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p9)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p9fx"; mkdir -p "$fx"
|
||||
printf 'v1\n' >"$fx/f.env"
|
||||
export WAKE_PREIMAGE_EXTRA="$fx/f.env"
|
||||
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
|
||||
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P9: baseline failed"
|
||||
sd="$(state_dir)"
|
||||
printf 'NOT-JSON-GARBAGE{{{\n' >>"$sd/preimage/preimage-ledger.jsonl"
|
||||
r1="$(wc -l <"$sd/preimage/preimage-ledger.jsonl")"
|
||||
printf 'v2 changed\n' >"$fx/f.env"
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -ne 0 ] || fail_msg "P9: a corrupt ledger must FAIL LOUD (rc=0)"
|
||||
echo "$out" | grep -qi 'unparseable' || fail_msg "P9: the failure must name the corrupt ledger [$out]"
|
||||
[ "$(wc -l <"$sd/preimage/preimage-ledger.jsonl")" = "$r1" ] || fail_msg "P9: nothing may be appended over corrupt history"
|
||||
) && ok
|
||||
|
||||
echo "== P10: oversized file -> bytes refused, hash still recorded =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p10)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p10fx"; mkdir -p "$fx"
|
||||
head -c 200 /dev/zero | tr '\0' 'A' >"$fx/big.bin"
|
||||
export WAKE_PREIMAGE_EXTRA="$fx/big.bin" WAKE_PREIMAGE_MAX_BYTES=64
|
||||
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "P10: size refusal is not an infra failure (rc=$rc) [$out]"
|
||||
sd="$(state_dir)"
|
||||
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
|
||||
[ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P10: row must record captured=false [$row]"
|
||||
jq -r '.refused // ""' <<<"$row" | grep -qi 'MAX_BYTES' || fail_msg "P10: refusal must name the size cap [$row]"
|
||||
sha="$(jq -r '.sha256' <<<"$row")"
|
||||
[ "$sha" = "$(sha256sum "$fx/big.bin" | awk '{print $1}')" ] || fail_msg "P10: hash must still be recorded [$row]"
|
||||
[ ! -f "$sd/preimage/objects/$sha" ] || fail_msg "P10: oversized bytes must NOT be stored"
|
||||
) && ok
|
||||
|
||||
echo "== P11: reconcile surfaces the cause line before its enumerations =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p11)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p11fx"; mkdir -p "$fx"
|
||||
make_stub "$fx"; make_wl "$fx/wl.json"
|
||||
printf 'r1 state\n' >"$fx/repo_r1"
|
||||
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json"
|
||||
unset WAKE_PREIMAGE_EXTRA MOSAIC_HOME
|
||||
# Baseline the preimage set, then change the adapter while "the detector is
|
||||
# down" — the reconcile path must still produce the first-class cause line.
|
||||
"$PRE" check >/dev/null 2>&1 || fail_msg "P11: preimage baseline failed"
|
||||
printf '# adapter changed while detector down\n' >>"$fx/adapter.sh"
|
||||
"$RECON" reconcile >/dev/null 2>&1 # rc 1 expected (unaccounted enumerated)
|
||||
sd="$(state_dir)"
|
||||
pre_seq="$(jq -r 'select(.locators.kind == "preimage") | .observed_seq' "$sd/pending.jsonl" | head -n1)"
|
||||
enum_seq="$(jq -r 'select(.locators.reconciled == true) | .observed_seq' "$sd/pending.jsonl" | head -n1)"
|
||||
[ -n "$pre_seq" ] || fail_msg "P11: reconcile must enqueue the preimage cause line"
|
||||
[ -n "$enum_seq" ] || fail_msg "P11: reconcile must still enumerate the unaccounted source"
|
||||
if [ -n "$pre_seq" ] && [ -n "$enum_seq" ]; then
|
||||
[ "$pre_seq" -lt "$enum_seq" ] || fail_msg "P11: cause line must precede the enumeration (preimage seq=$pre_seq, enum seq=$enum_seq)"
|
||||
fi
|
||||
) && ok
|
||||
|
||||
echo "== P12: detector — preimage infra failure is LOUD but does not starve observation =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p12)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p12fx"; mkdir -p "$fx"
|
||||
make_stub "$fx"; make_wl "$fx/wl.json"
|
||||
printf 'r1 state\n' >"$fx/repo_r1"
|
||||
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json"
|
||||
unset WAKE_PREIMAGE_EXTRA MOSAIC_HOME
|
||||
# Corrupt the ledger BEFORE the first poll: the preimage check fails loud,
|
||||
# but the poll must still observe + baseline the source.
|
||||
sd="$(state_dir)"
|
||||
mkdir -p "$sd/preimage"
|
||||
printf 'NOT-JSON-GARBAGE{{{\n' >"$sd/preimage/preimage-ledger.jsonl"
|
||||
out="$("$DET" poll-once 2>&1)"; rc=$?
|
||||
[ "$rc" -ne 0 ] || fail_msg "P12: the pass must exit non-zero on preimage infra failure"
|
||||
echo "$out" | grep -qi 'preimage' || fail_msg "P12: the failure must name the preimage check [$out]"
|
||||
n="$(find "$sd/detector" -name 'watch-*.hash' 2>/dev/null | wc -l | tr -d '[:space:]')"
|
||||
[ "$n" -ge 1 ] || fail_msg "P12: source observation must still proceed (no watch hash baselined)"
|
||||
) && ok
|
||||
|
||||
echo
|
||||
total=$((pass + $(wc -l <"$FAILFILE")))
|
||||
echo "== test-wake-preimage: $pass/$total passed =="
|
||||
[ -s "$FAILFILE" ] && exit 1
|
||||
exit 0
|
||||
@@ -25,7 +25,7 @@
|
||||
"lint": "eslint src",
|
||||
"typecheck": "tsc --noEmit",
|
||||
"test": "vitest run --passWithNoTests && pnpm run test:framework-shell",
|
||||
"test:framework-shell": "python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-install.sh"
|
||||
"test:framework-shell": "python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-preimage.sh && bash framework/tools/wake/test-wake-install.sh"
|
||||
},
|
||||
"dependencies": {
|
||||
"@mosaicstack/brain": "workspace:*",
|
||||
|
||||
Reference in New Issue
Block a user