feat(wake): #958 A11 preimage.sh — durable provenance for the operator-side preimage definition
ci/woodpecker/pr/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
The operator-owned bytes every observed_hash is computed FROM (the source
adapter, the watch-list) were unversioned: a byte change was attributable
only via an agent transcript, and reconcile surfaced it as N UNACCOUNTED
sources instead of one cause.
New tool preimage.sh (wake 0.6.15 -> 0.7.0):
- Derives the preimage set from the runtime env (resolved
WAKE_DETECTOR_SOURCE_CMD file, WAKE_WATCH_LIST, WAKE_PREIMAGE_EXTRA);
per file appends {ts,path,sha256,size,mtime,prev} to an append-only
ledger and captures bytes content-addressed under
$STATE_DIR/preimage/objects/<sha256> — prior bytes + change time from
durable state alone (acceptance a).
- CREDENTIAL HARD GATE (acceptance b): byte capture is REFUSED — never
redacted — for credential-store paths, secret-shaped content (digest's
six scrub shapes as a content-deny), and files over
WAKE_PREIMAGE_MAX_BYTES; the refused file still gets its
hash/size/mtime row (captured:false) so change TIME survives.
- FIRST-CLASS CAUSE LINE (acceptance c): a change/deletion enqueues one
class=actionable entry via the store allocator with path as its §2.1
hard locator; detector poll-once and reconcile run the check as a
PRE-step so the cause line lands at a LOWER observed_seq than the
deltas/enumerations it explains.
- FAIL-LOUD (D2/#955 class): unresolvable adapter, corrupt ledger
(refuses re-baseline), failed object/ledger write, failed enqueue are
loud non-zero, never "no change"; in both integrations the pass exits
non-zero but source observation still proceeds.
Installer unchanged (Gate A auto-enumerates the new file; the recording
site is the runtime tick where the env-derived set exists). Watch-list
schema untouched ([1,1]).
Tests: test-wake-preimage.sh P1-P12 (red-first verified: P3/P11/P12 fail
with the integration edits reverted); all 9 existing wake suites green.
Refs #958
Agent: PEPPER (sb-it-1-dt)
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01NsKce8iZuSuRnu3gVMCBKB
Written-by: pepper (sb-it-1-dt)
This commit is contained in:
Jason Woltje
co-authored by
Claude Fable 5
Claude Fable 5
parent
6a7fce34bb
commit
e07943d742
@@ -414,6 +414,22 @@ cmd_poll_once() {
|
|||||||
_wake_clean_stale_tmp "$STATE_DIR"
|
_wake_clean_stale_tmp "$STATE_DIR"
|
||||||
mkdir -p "$DET_DIR"
|
mkdir -p "$DET_DIR"
|
||||||
|
|
||||||
|
local failed=0
|
||||||
|
|
||||||
|
# #958 preimage provenance: check the OPERATOR-SIDE preimage definition (the
|
||||||
|
# source adapter file, the watch-list, operator-declared extras) BEFORE
|
||||||
|
# observing any source. A changed preimage re-baselines EVERY source at once;
|
||||||
|
# running the check first means its first-class cause line is enqueued at a
|
||||||
|
# LOWER observed_seq than the N per-source deltas it explains, so the digest
|
||||||
|
# shows the cause, not just the flood. An infrastructure failure of the check
|
||||||
|
# is LOUD and marks this pass failed (G2a discipline — never read as "no
|
||||||
|
# change"), but source observation still proceeds: provenance must not be
|
||||||
|
# able to starve wake delivery.
|
||||||
|
if ! "$SCRIPT_DIR/preimage.sh" check --enqueue; then
|
||||||
|
echo "detector.sh: FAIL LOUD — preimage provenance check failed (see preimage.sh above); source observation continues but this pass exits non-zero." >&2
|
||||||
|
failed=1
|
||||||
|
fi
|
||||||
|
|
||||||
# Iterate the DECLARED source-coverage inventory (§4/G3): only sources listed
|
# Iterate the DECLARED source-coverage inventory (§4/G3): only sources listed
|
||||||
# in watches[].sources[] are polled. An omitted source is not observed (and so
|
# in watches[].sources[] are polled. An omitted source is not observed (and so
|
||||||
# cannot make anything pass vacuously); a referenced-but-undefined source is a
|
# cannot make anything pass vacuously); a referenced-but-undefined source is a
|
||||||
@@ -427,7 +443,7 @@ cmd_poll_once() {
|
|||||||
exit 2
|
exit 2
|
||||||
fi
|
fi
|
||||||
|
|
||||||
local failed=0 kind id def class
|
local kind id def class
|
||||||
while IFS=$'\t' read -r kind id; do
|
while IFS=$'\t' read -r kind id; do
|
||||||
[ -n "$kind" ] || continue
|
[ -n "$kind" ] || continue
|
||||||
# Resolve the source definition from its top-level collection by id.
|
# Resolve the source definition from its top-level collection by id.
|
||||||
|
|||||||
@@ -374,8 +374,55 @@
|
|||||||
# Changed: store.sh, digest.sh, ack.sh
|
# Changed: store.sh, digest.sh, ack.sh
|
||||||
# (+ test-wake-store-ack.sh T13-T16,
|
# (+ test-wake-store-ack.sh T13-T16,
|
||||||
# test-wake-digest-quarantine.sh Q12-Q16).
|
# test-wake-digest-quarantine.sh Q12-Q16).
|
||||||
|
# 0.7.0 #958 A11 preimage.sh — durable provenance for the OPERATOR-SIDE
|
||||||
|
# preimage definition (wake-pilot finding: source-adapter.sh was
|
||||||
|
# unversioned, so the operator-owned bytes every observed_hash is
|
||||||
|
# computed FROM had thinner provenance than any hash they feed;
|
||||||
|
# attribution required an agent transcript). NEW tool + two
|
||||||
|
# pre-step integrations, ADDITIVE: (1) preimage.sh derives the
|
||||||
|
# preimage set from the RUNTIME env (the resolved
|
||||||
|
# WAKE_DETECTOR_SOURCE_CMD file, WAKE_WATCH_LIST, optional
|
||||||
|
# WAKE_PREIMAGE_EXTRA paths) and, per file, records
|
||||||
|
# {ts,path,sha256,size,mtime,prev} to an append-only ledger +
|
||||||
|
# captures the bytes CONTENT-ADDRESSED under
|
||||||
|
# $STATE_DIR/preimage/objects/<sha256> — prior bytes + change
|
||||||
|
# time are answerable from durable state alone, no transcript
|
||||||
|
# (acceptance a). A git-repo-in-operator-dir design was REJECTED:
|
||||||
|
# its who/why claim is false under shared-author fleets, and
|
||||||
|
# .gitignore is pattern-based where acceptance (b) demands
|
||||||
|
# fail-closed. (2) CREDENTIAL HARD GATE (acceptance b): byte
|
||||||
|
# capture is REFUSED — never redacted — for (i) credential-store
|
||||||
|
# PATHS (mosaic-home credentials.json, tools/_lib/credentials.json,
|
||||||
|
# credentials/**, any basename credentials.json), (ii)
|
||||||
|
# secret-SHAPED content (digest.sh's six scrub shapes reused as a
|
||||||
|
# content-deny grep), (iii) files over WAKE_PREIMAGE_MAX_BYTES
|
||||||
|
# (default 1 MiB). A refused file still gets its hash/size/mtime
|
||||||
|
# row (captured:false + refused reason) so change TIME survives
|
||||||
|
# even when content must not. (3) FIRST-CLASS CAUSE LINE
|
||||||
|
# (acceptance c): on a change (or deletion — ABSENT is a state,
|
||||||
|
# not an error), one class=actionable entry is enqueued via the
|
||||||
|
# store allocator with locators {kind:preimage, path (§2.1 hard
|
||||||
|
# locator — never quarantined), observed_hash, prev_hash,
|
||||||
|
# preimage:true, reason:preimage-definition-changed}. Both
|
||||||
|
# detector.sh cmd_poll_once and reconcile.sh cmd_reconcile run
|
||||||
|
# the check as a PRE-step, so the cause line lands at a LOWER
|
||||||
|
# observed_seq than the N per-source deltas/enumerations it
|
||||||
|
# explains — "preimage definition changed" reads first, not N
|
||||||
|
# UNACCOUNTED lines. (4) FAIL-LOUD discipline (D2/#955 class):
|
||||||
|
# an unresolvable adapter, unreadable watch-list, corrupt ledger
|
||||||
|
# (REFUSES to compare or re-baseline over corrupt history), failed
|
||||||
|
# object/ledger write, or failed enqueue is a loud non-zero —
|
||||||
|
# never read as "no change"; in both integrations the pass exits
|
||||||
|
# non-zero but source observation still proceeds (no starvation).
|
||||||
|
# First-seen is a SILENT baseline (detector first-poll idiom).
|
||||||
|
# The installer is UNCHANGED — Gate A auto-enumerates the new
|
||||||
|
# file from the filesystem; the recording site is the runtime
|
||||||
|
# tick, which is where the env-derived preimage set exists.
|
||||||
|
# store.sh/digest.sh/beacon.sh UNCHANGED; watch-list schema
|
||||||
|
# UNTOUCHED ([1,1]). Changed: preimage.sh (new), detector.sh,
|
||||||
|
# reconcile.sh (+ test-wake-preimage.sh P1-P12).
|
||||||
component=wake
|
component=wake
|
||||||
version=0.6.15
|
version=0.7.0
|
||||||
|
|
||||||
# Watch-list schema this component consumes, and the INCLUSIVE range of
|
# Watch-list schema this component consumes, and the INCLUSIVE range of
|
||||||
# schema_version values it supports. A wake-watch-list.json whose schema_version
|
# schema_version values it supports. A wake-watch-list.json whose schema_version
|
||||||
@@ -446,6 +493,19 @@ schema_max=1
|
|||||||
# seed) instead of a bare source error, and LINKS the unit into
|
# seed) instead of a bare source error, and LINKS the unit into
|
||||||
# the user systemd search path + post-install-validates it
|
# the user systemd search path + post-install-validates it
|
||||||
# resolves (#913). (W7, #913)
|
# resolves (#913). (W7, #913)
|
||||||
|
# preimage.sh A11 — operator-side PREIMAGE-DEFINITION provenance: derives the
|
||||||
|
# preimage set from the runtime env (resolved adapter file,
|
||||||
|
# watch-list, WAKE_PREIMAGE_EXTRA), appends
|
||||||
|
# {ts,path,sha256,size,mtime,prev} rows to an append-only
|
||||||
|
# ledger, captures bytes content-addressed under
|
||||||
|
# preimage/objects/<sha256>, and enqueues a FIRST-CLASS
|
||||||
|
# "preimage-definition-changed" actionable (path = §2.1 hard
|
||||||
|
# locator) BEFORE the deltas it explains (detector +
|
||||||
|
# reconcile pre-step). Credential HARD GATE: capture is
|
||||||
|
# REFUSED (hash/mtime still recorded) for credential-store
|
||||||
|
# paths, secret-shaped content, and oversized files —
|
||||||
|
# refusal, never redaction. Fail-loud on any infra failure;
|
||||||
|
# a corrupt ledger refuses re-baseline. (#958)
|
||||||
# Companion (framework subtree, not under tools/wake/): systemd/user/mosaic-wake.service
|
# Companion (framework subtree, not under tools/wake/): systemd/user/mosaic-wake.service
|
||||||
# — the long-lived detector daemon unit (per-class SLO lives in
|
# — the long-lived detector daemon unit (per-class SLO lives in
|
||||||
# the daemon, NOT a systemd interval). (W7)
|
# the daemon, NOT a systemd interval). (W7)
|
||||||
|
|||||||
+446
@@ -0,0 +1,446 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# preimage.sh — A11 of the wake canon (#958): durable PROVENANCE for the
|
||||||
|
# OPERATOR-SIDE preimage definition.
|
||||||
|
#
|
||||||
|
# THE GAP THIS CLOSES (#958): every observed_hash in the lane store is
|
||||||
|
# sha256(adapter stdout) — the operator's source adapter (plus the watch-list
|
||||||
|
# and any operator-declared preimage files) IS the preimage definition for
|
||||||
|
# every hash the pipeline ever records. Those files live OUTSIDE the versioned
|
||||||
|
# wake component, so a byte change to them was attributable only through agent
|
||||||
|
# transcripts: provenance thinner than any hash it feeds. A changed preimage
|
||||||
|
# also re-baselines EVERY source at once, which previously surfaced only as N
|
||||||
|
# per-source deltas / UNACCOUNTED lines with no first-class cause.
|
||||||
|
#
|
||||||
|
# WHAT THIS TOOL DOES:
|
||||||
|
# - Derives the PREIMAGE SET generically (no operator paths baked in):
|
||||||
|
# * the resolved file behind WAKE_DETECTOR_SOURCE_CMD (first word),
|
||||||
|
# * the WAKE_WATCH_LIST file,
|
||||||
|
# * operator-declared extras via WAKE_PREIMAGE_EXTRA (colon-separated —
|
||||||
|
# e.g. detector.env, sink/feed scripts).
|
||||||
|
# - Records each file's (sha256, size, mtime, ts) as an APPEND-ONLY ledger
|
||||||
|
# row in <state>/preimage/preimage-ledger.jsonl and stores the full bytes
|
||||||
|
# CONTENT-ADDRESSED at <state>/preimage/objects/<sha256> — so a change is
|
||||||
|
# attributable (prior bytes + change time) from durable state alone, with
|
||||||
|
# no transcript required. Acceptance (a) of #958.
|
||||||
|
# - On a CHANGE (not first-seen), `check --enqueue` enqueues ONE first-class
|
||||||
|
# class=actionable entry per changed file via the store's single allocator
|
||||||
|
# (#908), carrying a §2.1 hard locator (`path`), BEFORE the per-source
|
||||||
|
# deltas it explains are observed — so the digest shows the cause line,
|
||||||
|
# not just N re-baselined sources. Acceptance (c).
|
||||||
|
#
|
||||||
|
# CREDENTIAL HARD GATE — acceptance (b): the mechanism must be UNABLE to
|
||||||
|
# capture credential material even by operator error. Enforced two ways, both
|
||||||
|
# fail-closed toward NOT capturing bytes (the hash/size/mtime row is still
|
||||||
|
# written — change-time attribution survives, content capture does not;
|
||||||
|
# a sha256 discloses nothing about the bytes):
|
||||||
|
# 1. PATH deny: the mosaic credential store locations
|
||||||
|
# (<mosaic-home>/credentials.json, <mosaic-home>/tools/_lib/
|
||||||
|
# credentials.json, anything under <mosaic-home>/credentials/, and any
|
||||||
|
# file literally named credentials.json) are refused by resolved realpath
|
||||||
|
# before a single byte is read into the object store.
|
||||||
|
# 2. CONTENT deny: a candidate whose bytes match the well-known secret-token
|
||||||
|
# shapes (same conservative set digest.sh redacts: PAT/Slack/AKIA/JWT/PEM)
|
||||||
|
# is refused — refusal, not redaction: a redacted preimage would be a
|
||||||
|
# false witness, and secret bytes must never land in the object store.
|
||||||
|
# Plus a size cap (WAKE_PREIMAGE_MAX_BYTES, default 1 MiB) so a stray extra
|
||||||
|
# pointing at a large binary cannot turn provenance into data hoovering.
|
||||||
|
#
|
||||||
|
# FAIL-LOUD DISCIPLINE (the D2/#955 class, both layers): an infrastructure
|
||||||
|
# failure of THIS tool (unresolvable adapter, unreadable/corrupt ledger,
|
||||||
|
# failed durable write, failed enqueue) exits NON-ZERO and is never read as
|
||||||
|
# "no change". A corrupt ledger REFUSES to compare (and to re-baseline):
|
||||||
|
# silently restarting history would erase the very attribution this exists
|
||||||
|
# to provide.
|
||||||
|
#
|
||||||
|
# Operator-agnostic (framework firewall): all state via XDG/env; the deny
|
||||||
|
# list names only framework-defined credential locations, no operator hosts/
|
||||||
|
# names/secrets. This tool never prints file CONTENT to any stream.
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
# shellcheck source=./_wake-common.sh disable=SC1091
|
||||||
|
. "$SCRIPT_DIR/_wake-common.sh"
|
||||||
|
|
||||||
|
STATE_DIR="$(wake_state_dir)"
|
||||||
|
STORE_SH="$SCRIPT_DIR/store.sh"
|
||||||
|
PRE_DIR="$STATE_DIR/preimage"
|
||||||
|
LEDGER="$PRE_DIR/preimage-ledger.jsonl"
|
||||||
|
OBJECTS="$PRE_DIR/objects"
|
||||||
|
|
||||||
|
_need_jq() {
|
||||||
|
command -v jq >/dev/null 2>&1 || {
|
||||||
|
echo "preimage.sh: jq is required" >&2
|
||||||
|
exit 3
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# _hash_stdin — sha256 of stdin, first field only (portable, same fallback
|
||||||
|
# chain as detector.sh).
|
||||||
|
_hash_stdin() {
|
||||||
|
if command -v sha256sum >/dev/null 2>&1; then
|
||||||
|
sha256sum | awk '{print $1}'
|
||||||
|
elif command -v shasum >/dev/null 2>&1; then
|
||||||
|
shasum -a 256 | awk '{print $1}'
|
||||||
|
elif command -v openssl >/dev/null 2>&1; then
|
||||||
|
openssl dgst -sha256 | awk '{print $NF}'
|
||||||
|
else
|
||||||
|
echo "preimage.sh: no sha256 tool (sha256sum/shasum/openssl) available" >&2
|
||||||
|
exit 3
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat >&2 <<'EOF'
|
||||||
|
Usage: preimage.sh <command>
|
||||||
|
|
||||||
|
Commands:
|
||||||
|
check [--enqueue] Compare every preimage-set file against the ledger head.
|
||||||
|
A changed file gets a new ledger row + captured bytes;
|
||||||
|
with --enqueue each change (not first-seen) also enqueues
|
||||||
|
ONE first-class class=actionable store entry (hard
|
||||||
|
locator: path). First-seen files baseline SILENTLY (row,
|
||||||
|
no enqueue — same idiom as the detector's first-seen).
|
||||||
|
Exit 0 whether or not changes were found; non-zero ONLY
|
||||||
|
on infrastructure failure (never read as "no change").
|
||||||
|
record Alias of `check` without enqueue (baseline/refresh).
|
||||||
|
set Print the derived preimage set, one resolved path per
|
||||||
|
line (diagnostic).
|
||||||
|
status Print the ledger head row for every tracked path.
|
||||||
|
|
||||||
|
Environment:
|
||||||
|
WAKE_DETECTOR_SOURCE_CMD Adapter command; its resolved file joins the set.
|
||||||
|
WAKE_WATCH_LIST Watch-list path; joins the set when set.
|
||||||
|
WAKE_PREIMAGE_EXTRA Colon-separated additional operator preimage files
|
||||||
|
(e.g. detector.env, sink scripts). A listed path
|
||||||
|
that does not exist is tracked as ABSENT (deletion
|
||||||
|
of a preimage file is a change, not an error).
|
||||||
|
WAKE_PREIMAGE_MAX_BYTES Byte-capture cap (default 1048576). Larger files:
|
||||||
|
hash/size/mtime recorded, bytes refused.
|
||||||
|
WAKE_STATE_HOME/WAKE_AGENT store namespace (see store.sh).
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- preimage-set derivation (generic; no operator paths baked in) ----------
|
||||||
|
|
||||||
|
# _resolve_cmd_file CMD — resolve the FIRST WORD of an adapter command string
|
||||||
|
# to a real file. Non-zero (loud) if it cannot be resolved: an adapter the
|
||||||
|
# detector will invoke but provenance cannot see is an infrastructure failure,
|
||||||
|
# not a smaller set.
|
||||||
|
_resolve_cmd_file() {
|
||||||
|
local cmd="$1" word path
|
||||||
|
# shellcheck disable=SC2086
|
||||||
|
set -- $cmd
|
||||||
|
word="${1:-}"
|
||||||
|
[ -n "$word" ] || return 1
|
||||||
|
if [ -f "$word" ]; then
|
||||||
|
path="$word"
|
||||||
|
else
|
||||||
|
path="$(command -v -- "$word" 2>/dev/null)" || return 1
|
||||||
|
[ -f "$path" ] || return 1
|
||||||
|
fi
|
||||||
|
# realpath so the ledger keys on the actual file, not a symlink alias.
|
||||||
|
if command -v realpath >/dev/null 2>&1; then
|
||||||
|
realpath -- "$path" 2>/dev/null || printf '%s' "$path"
|
||||||
|
else
|
||||||
|
printf '%s' "$path"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# _preimage_set — print the derived set, one path per line. Paths from
|
||||||
|
# WAKE_PREIMAGE_EXTRA are printed EVEN IF ABSENT (deletion is a tracked
|
||||||
|
# state); the adapter and watch-list must resolve (loud failure otherwise —
|
||||||
|
# see _resolve_cmd_file rationale).
|
||||||
|
_preimage_set() {
|
||||||
|
local failed=0
|
||||||
|
if [ -n "${WAKE_DETECTOR_SOURCE_CMD:-}" ]; then
|
||||||
|
local af
|
||||||
|
if af="$(_resolve_cmd_file "$WAKE_DETECTOR_SOURCE_CMD")"; then
|
||||||
|
printf '%s\n' "$af"
|
||||||
|
else
|
||||||
|
echo "preimage.sh: FAIL LOUD — WAKE_DETECTOR_SOURCE_CMD ('$WAKE_DETECTOR_SOURCE_CMD') does not resolve to a file; the preimage definition cannot be observed (NOT treated as 'no change')." >&2
|
||||||
|
failed=1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
if [ -n "${WAKE_WATCH_LIST:-}" ]; then
|
||||||
|
if [ -f "$WAKE_WATCH_LIST" ]; then
|
||||||
|
if command -v realpath >/dev/null 2>&1; then
|
||||||
|
realpath -- "$WAKE_WATCH_LIST" 2>/dev/null || printf '%s' "$WAKE_WATCH_LIST"
|
||||||
|
else
|
||||||
|
printf '%s' "$WAKE_WATCH_LIST"
|
||||||
|
fi
|
||||||
|
printf '\n'
|
||||||
|
else
|
||||||
|
echo "preimage.sh: FAIL LOUD — WAKE_WATCH_LIST ('$WAKE_WATCH_LIST') is not a file; the preimage definition cannot be observed." >&2
|
||||||
|
failed=1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
if [ -n "${WAKE_PREIMAGE_EXTRA:-}" ]; then
|
||||||
|
local IFS=':' p
|
||||||
|
for p in $WAKE_PREIMAGE_EXTRA; do
|
||||||
|
[ -n "$p" ] || continue
|
||||||
|
# Extras are tracked even when absent (ABSENT is a state). Resolve the
|
||||||
|
# realpath only when the file exists; otherwise track the literal path.
|
||||||
|
if [ -e "$p" ] && command -v realpath >/dev/null 2>&1; then
|
||||||
|
realpath -- "$p" 2>/dev/null || printf '%s' "$p"
|
||||||
|
printf '\n'
|
||||||
|
else
|
||||||
|
printf '%s\n' "$p"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
return "$failed"
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- credential hard gate (acceptance (b)) ----------------------------------
|
||||||
|
|
||||||
|
# _deny_path PATH — 0 (deny) iff PATH is a known credential-store location.
|
||||||
|
# Framework-defined locations only (firewall): the mosaic credential store,
|
||||||
|
# the tools/_lib credential file the framework-manifest itself carves out of
|
||||||
|
# tools/**, the credentials/ operator subtree, and any file literally named
|
||||||
|
# credentials.json.
|
||||||
|
_deny_path() {
|
||||||
|
local p="$1" home="${MOSAIC_HOME:-$HOME/.config/mosaic}"
|
||||||
|
case "$p" in
|
||||||
|
"$home/credentials.json") return 0 ;;
|
||||||
|
"$home/tools/_lib/credentials.json") return 0 ;;
|
||||||
|
"$home/credentials/"*) return 0 ;;
|
||||||
|
esac
|
||||||
|
case "$(basename -- "$p")" in
|
||||||
|
credentials.json) return 0 ;;
|
||||||
|
esac
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# _deny_content FILE — 0 (deny) iff FILE's bytes match a well-known secret-
|
||||||
|
# token shape. Same conservative shape set digest.sh redacts (PAT / Slack /
|
||||||
|
# AKIA / JWT / PEM) — conservative on purpose: a 40-hex git sha never matches.
|
||||||
|
_deny_content() {
|
||||||
|
LC_ALL=C grep -Eq \
|
||||||
|
-e 'gh[pousr]_[A-Za-z0-9]{16,}' \
|
||||||
|
-e 'github_pat_[A-Za-z0-9_]{16,}' \
|
||||||
|
-e 'xox[baprs]-[A-Za-z0-9-]{10,}' \
|
||||||
|
-e 'AKIA[0-9A-Z]{16}' \
|
||||||
|
-e 'eyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}' \
|
||||||
|
-e '-----BEGIN[A-Z ]*PRIVATE KEY-----' \
|
||||||
|
-- "$1" 2>/dev/null
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- ledger primitives ------------------------------------------------------
|
||||||
|
|
||||||
|
# _ledger_head PATH — print the LAST ledger row for PATH (empty if none).
|
||||||
|
# Non-zero (loud) if the ledger exists but is unparseable: a corrupt ledger
|
||||||
|
# must REFUSE to compare, never silently re-baseline (that would erase the
|
||||||
|
# attribution history this tool exists to keep).
|
||||||
|
_ledger_head() {
|
||||||
|
local path="$1"
|
||||||
|
[ -f "$LEDGER" ] || return 0
|
||||||
|
if [ -s "$LEDGER" ] && ! jq -cn 'inputs' <"$LEDGER" >/dev/null 2>&1; then
|
||||||
|
echo "preimage.sh: FAIL LOUD — preimage ledger $LEDGER is unparseable; REFUSING to compare or re-baseline over corrupt history. Investigate/restore the ledger." >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
jq -c --arg p "$path" 'select(.path == $p)' "$LEDGER" 2>/dev/null | tail -n 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# _ledger_append ROW_JSON — append one row atomically (read + append + rename;
|
||||||
|
# rows are small and the writer is serialized by the preimage lock).
|
||||||
|
_ledger_append() {
|
||||||
|
local row="$1"
|
||||||
|
{
|
||||||
|
[ -f "$LEDGER" ] && cat "$LEDGER"
|
||||||
|
printf '%s\n' "$row"
|
||||||
|
} | _atomic_write "$LEDGER"
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- the check itself -------------------------------------------------------
|
||||||
|
|
||||||
|
# _check_one PATH ENQUEUE — compare PATH to its ledger head; on change record
|
||||||
|
# (+bytes unless denied) and optionally enqueue. Prints nothing on no-change.
|
||||||
|
# Returns: 0 ok (changed or not), 1 infrastructure failure.
|
||||||
|
_check_one() {
|
||||||
|
local path="$1" enqueue="$2"
|
||||||
|
local cur_sha size mtime denied="" refused=""
|
||||||
|
|
||||||
|
if [ -f "$path" ]; then
|
||||||
|
cur_sha="$(_hash_stdin <"$path")" || return 1
|
||||||
|
[ -n "$cur_sha" ] || {
|
||||||
|
echo "preimage.sh: FAIL LOUD — could not hash $path" >&2
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
size="$(wc -c <"$path" | tr -d '[:space:]')"
|
||||||
|
# Portable mtime (GNU stat -c / BSD stat -f).
|
||||||
|
mtime="$(stat -c %Y -- "$path" 2>/dev/null || stat -f %m -- "$path" 2>/dev/null || echo 0)"
|
||||||
|
local cap="${WAKE_PREIMAGE_MAX_BYTES:-1048576}"
|
||||||
|
case "$cap" in '' | *[!0-9]*) cap=1048576 ;; esac
|
||||||
|
# Deny order matters: path first (a known credential store is refused
|
||||||
|
# without a content probe — the hash read above is deliberate: a sha256
|
||||||
|
# discloses nothing about the bytes), size cap second (never content-grep
|
||||||
|
# an oversized file), content shape last.
|
||||||
|
if _deny_path "$path"; then
|
||||||
|
denied="credential-store path (deny list)"
|
||||||
|
elif [ "$size" -gt "$cap" ]; then
|
||||||
|
denied="exceeds WAKE_PREIMAGE_MAX_BYTES ($size > $cap)"
|
||||||
|
elif _deny_content "$path"; then
|
||||||
|
denied="secret-shaped content"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
# ABSENT is a state (deletion of a preimage file is a change to record).
|
||||||
|
cur_sha="ABSENT"
|
||||||
|
size=0
|
||||||
|
mtime=0
|
||||||
|
fi
|
||||||
|
|
||||||
|
local head prev_sha=""
|
||||||
|
head="$(_ledger_head "$path")" || return 1
|
||||||
|
[ -n "$head" ] && prev_sha="$(jq -r '.sha256 // ""' <<<"$head")"
|
||||||
|
|
||||||
|
if [ "$cur_sha" = "$prev_sha" ]; then
|
||||||
|
return 0 # unchanged — no row, no enqueue (idempotent)
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- capture bytes (content-addressed) unless the hard gate refuses -------
|
||||||
|
local captured=true
|
||||||
|
if [ "$cur_sha" != "ABSENT" ]; then
|
||||||
|
if [ -n "$denied" ]; then
|
||||||
|
captured=false
|
||||||
|
refused="$denied"
|
||||||
|
echo "preimage.sh: REFUSED byte capture for $path ($denied) — hash/size/mtime recorded, content NOT stored (acceptance (b), #958)." >&2
|
||||||
|
else
|
||||||
|
mkdir -p "$OBJECTS" || return 1
|
||||||
|
if [ ! -f "$OBJECTS/$cur_sha" ]; then
|
||||||
|
if ! _atomic_write "$OBJECTS/$cur_sha" <"$path"; then
|
||||||
|
echo "preimage.sh: FAIL LOUD — durable object write failed for $path ($OBJECTS/$cur_sha); NO ledger row recorded (a row whose bytes were never durably kept would be a false witness)." >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
local ts row
|
||||||
|
ts="$(date +%s)"
|
||||||
|
row="$(jq -cn \
|
||||||
|
--arg path "$path" \
|
||||||
|
--arg sha "$cur_sha" \
|
||||||
|
--arg prev "$prev_sha" \
|
||||||
|
--argjson size "$size" \
|
||||||
|
--argjson mtime "$mtime" \
|
||||||
|
--argjson ts "$ts" \
|
||||||
|
--argjson captured "$captured" \
|
||||||
|
--arg refused "$refused" \
|
||||||
|
'{ts:$ts, path:$path, sha256:$sha, size:$size, mtime:$mtime, captured:$captured}
|
||||||
|
+ (if $prev != "" then {prev:$prev} else {} end)
|
||||||
|
+ (if $refused != "" then {refused:$refused} else {} end)')" || return 1
|
||||||
|
if ! _ledger_append "$row"; then
|
||||||
|
echo "preimage.sh: FAIL LOUD — durable ledger append failed ($LEDGER)" >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -z "$head" ]; then
|
||||||
|
# First-seen: baseline SILENTLY (row only, no enqueue) — the same idiom as
|
||||||
|
# the detector's first-seen source baseline.
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "preimage.sh: preimage definition CHANGED — $path ($prev_sha -> $cur_sha); prior bytes: $OBJECTS/$prev_sha" >&2
|
||||||
|
|
||||||
|
if [ "$enqueue" = "1" ]; then
|
||||||
|
# First-class cause line (acceptance (c)): ONE class=actionable entry per
|
||||||
|
# changed preimage file, allocated by the store's single allocator (#908).
|
||||||
|
# `path` is a §2.1 hard locator, so the digest renders it — never
|
||||||
|
# quarantined. Callers run this BEFORE polling sources, so this seq is
|
||||||
|
# LOWER than the per-source deltas the change explains.
|
||||||
|
local locators
|
||||||
|
locators="$(jq -cn \
|
||||||
|
--arg path "$path" \
|
||||||
|
--arg sha "$cur_sha" \
|
||||||
|
--arg prev "$prev_sha" \
|
||||||
|
'{kind:"preimage", id:$path, path:$path, observed_hash:$sha, prev_hash:$prev,
|
||||||
|
preimage:true, reason:"preimage-definition-changed"}')"
|
||||||
|
if ! "$STORE_SH" enqueue --class actionable --locators "$locators" --emit-ts "$(date +%s)" >/dev/null; then
|
||||||
|
echo "preimage.sh: FAIL LOUD — store enqueue of the preimage-change entry FAILED for $path (the change IS recorded in the ledger; the first-class wake line is NOT — treat as infrastructure failure)." >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd_check() {
|
||||||
|
local enqueue=0
|
||||||
|
while [ $# -gt 0 ]; do
|
||||||
|
case "$1" in
|
||||||
|
--enqueue)
|
||||||
|
enqueue=1
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo "preimage.sh check: unknown option '$1'" >&2
|
||||||
|
exit 2
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
_need_jq
|
||||||
|
|
||||||
|
local set_list
|
||||||
|
if ! set_list="$(_preimage_set)"; then
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
if [ -z "$set_list" ]; then
|
||||||
|
# Nothing declared (no adapter/watch-list/extras in env) — an empty set is
|
||||||
|
# a no-op, not an error: standalone invocations outside a wake runtime
|
||||||
|
# must not fail loud for lacking one.
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
mkdir -p "$PRE_DIR" || exit 1
|
||||||
|
# Serialize concurrent checks (detector tick vs reconcile vs manual) — the
|
||||||
|
# ledger append is read+rewrite, so two writers must not interleave.
|
||||||
|
if ! _wake_lock_acquire "$PRE_DIR/preimage.lock"; then
|
||||||
|
echo "preimage.sh: FAIL LOUD — cannot acquire preimage lock" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
local failed=0 p
|
||||||
|
while IFS= read -r p; do
|
||||||
|
[ -n "$p" ] || continue
|
||||||
|
_check_one "$p" "$enqueue" || failed=1
|
||||||
|
done <<EOF
|
||||||
|
$set_list
|
||||||
|
EOF
|
||||||
|
_wake_lock_release
|
||||||
|
[ "$failed" -eq 0 ] || exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd_status() {
|
||||||
|
_need_jq
|
||||||
|
[ -f "$LEDGER" ] || {
|
||||||
|
echo "preimage.sh: no ledger yet ($LEDGER)" >&2
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
# Head row per path (last wins).
|
||||||
|
jq -cs 'group_by(.path) | map(last) | .[]' "$LEDGER"
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd_set() {
|
||||||
|
_preimage_set || exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
main() {
|
||||||
|
[ $# -ge 1 ] || {
|
||||||
|
usage
|
||||||
|
exit 2
|
||||||
|
}
|
||||||
|
local cmd="$1"
|
||||||
|
shift
|
||||||
|
case "$cmd" in
|
||||||
|
check) cmd_check "$@" ;;
|
||||||
|
record) cmd_check ;;
|
||||||
|
set) cmd_set "$@" ;;
|
||||||
|
status) cmd_status "$@" ;;
|
||||||
|
-h | --help | help) usage ;;
|
||||||
|
*)
|
||||||
|
echo "preimage.sh: unknown command '$cmd'" >&2
|
||||||
|
usage
|
||||||
|
exit 2
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
@@ -440,6 +440,17 @@ cmd_reconcile() {
|
|||||||
: "$allow_enumerate"
|
: "$allow_enumerate"
|
||||||
|
|
||||||
local pairs kind id failed=0 unaccounted=0 enumerated=0
|
local pairs kind id failed=0 unaccounted=0 enumerated=0
|
||||||
|
|
||||||
|
# #958 preimage provenance: same pre-step as the detector's poll tick, for
|
||||||
|
# the path where the preimage changed while the detector was down — without
|
||||||
|
# it, a changed adapter surfaces here only as N UNACCOUNTED enumerations
|
||||||
|
# with no first-class cause line. Run BEFORE observing any source so the
|
||||||
|
# cause entry's observed_seq precedes the enumerations it explains. Loud
|
||||||
|
# infrastructure failure marks the reconcile failed but does not stop it.
|
||||||
|
if ! "$SCRIPT_DIR/preimage.sh" check --enqueue; then
|
||||||
|
echo "reconcile.sh: FAIL LOUD — preimage provenance check failed (see preimage.sh above); reconcile continues but exits non-zero." >&2
|
||||||
|
failed=1
|
||||||
|
fi
|
||||||
pairs="$(printf '%s' "$json" | jq -r '[ .watches[].sources[] | "\(.kind)\t\(.id)" ] | unique | .[]')"
|
pairs="$(printf '%s' "$json" | jq -r '[ .watches[].sources[] | "\(.kind)\t\(.id)" ] | unique | .[]')"
|
||||||
if [ -z "$pairs" ]; then
|
if [ -z "$pairs" ]; then
|
||||||
echo "reconcile.sh: watch-list declares no sources under watches[].sources[]" >&2
|
echo "reconcile.sh: watch-list declares no sources under watches[].sources[]" >&2
|
||||||
|
|||||||
+348
@@ -0,0 +1,348 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# test-wake-preimage.sh — RED-FIRST invariant harness for A11 (#958): durable
|
||||||
|
# provenance for the OPERATOR-SIDE preimage definition (preimage.sh).
|
||||||
|
#
|
||||||
|
# Each test asserts ONE #958 invariant and is designed to go RED if it
|
||||||
|
# regresses:
|
||||||
|
# P1 first-seen -> SILENT baseline (ledger rows, NO enqueue) (detector idiom)
|
||||||
|
# P2 change -> attributable from durable state alone: new ledger row with
|
||||||
|
# prev, PRIOR BYTES retrievable content-addressed, first-class actionable
|
||||||
|
# enqueued with a §2.1 hard locator (path) (acceptance a+c)
|
||||||
|
# P3 detector ordering: the preimage cause line is enqueued at a LOWER
|
||||||
|
# observed_seq than the per-source delta it explains (acceptance c)
|
||||||
|
# P4 credential-store PATH is refused — bytes NEVER captured, even when the
|
||||||
|
# operator lists it by error (hash/mtime row still written) (acceptance b)
|
||||||
|
# P5 secret-SHAPED content is refused — refusal, not redaction (acceptance b)
|
||||||
|
# P6 deletion of a preimage file is a tracked CHANGE (ABSENT), not an error
|
||||||
|
# P7 no-change is idempotent (no row growth, no enqueue)
|
||||||
|
# P8 unresolvable adapter command -> FAIL LOUD, never "no change" (D2/#955 class)
|
||||||
|
# P9 corrupt ledger -> REFUSE to compare/re-baseline (loud, no append)
|
||||||
|
# P10 oversized file -> bytes refused, hash still recorded (no data hoovering)
|
||||||
|
# P11 reconcile pre-step: a preimage change surfaces as a first-class line
|
||||||
|
# BEFORE the UNACCOUNTED enumerations it explains (acceptance c)
|
||||||
|
# P12 detector: preimage infra failure is LOUD (pass exits non-zero) but does
|
||||||
|
# NOT starve source observation
|
||||||
|
#
|
||||||
|
# Uses FAKE/STUB sources only (no live network). Isolated per test.
|
||||||
|
# shellcheck disable=SC2030,SC2031
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
PRE="$SCRIPT_DIR/preimage.sh"
|
||||||
|
STORE="$SCRIPT_DIR/store.sh"
|
||||||
|
DET="$SCRIPT_DIR/detector.sh"
|
||||||
|
RECON="$SCRIPT_DIR/reconcile.sh"
|
||||||
|
|
||||||
|
command -v jq >/dev/null 2>&1 || {
|
||||||
|
echo "SKIP: jq not available" >&2
|
||||||
|
exit 0
|
||||||
|
}
|
||||||
|
|
||||||
|
TMP_ROOT="$(mktemp -d)"
|
||||||
|
trap 'rm -rf "$TMP_ROOT"' EXIT
|
||||||
|
|
||||||
|
FAILFILE="$TMP_ROOT/failures"
|
||||||
|
: >"$FAILFILE"
|
||||||
|
pass=0
|
||||||
|
fail_msg() {
|
||||||
|
echo " FAIL: $*" >&2
|
||||||
|
echo "x" >>"$FAILFILE"
|
||||||
|
}
|
||||||
|
ok() { pass=$((pass + 1)); }
|
||||||
|
|
||||||
|
fresh_state() {
|
||||||
|
local d="$TMP_ROOT/$1"
|
||||||
|
rm -rf "$d"
|
||||||
|
mkdir -p "$d"
|
||||||
|
printf '%s' "$d"
|
||||||
|
}
|
||||||
|
depth() { "$STORE" cursors | sed -n 's/pending_depth=//p'; }
|
||||||
|
state_dir() { printf '%s/default' "$WAKE_STATE_HOME"; }
|
||||||
|
ledger_rows() {
|
||||||
|
local f
|
||||||
|
f="$(state_dir)/preimage/preimage-ledger.jsonl"
|
||||||
|
[ -f "$f" ] && wc -l <"$f" | tr -d '[:space:]' || echo 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# make_stub DIR — a source adapter reading $DIR/<kind>_<id> (same shape as the
|
||||||
|
# reconcile harness stub).
|
||||||
|
make_stub() {
|
||||||
|
local dir="$1"
|
||||||
|
cat >"$dir/adapter.sh" <<EOF
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
set -u
|
||||||
|
kind="\$1"; id="\$2"; base="$dir/\${kind}_\${id}"; rc=0
|
||||||
|
[ -f "\$base.rc" ] && rc="\$(cat "\$base.rc")"
|
||||||
|
[ -f "\$base" ] && cat "\$base"
|
||||||
|
exit "\$rc"
|
||||||
|
EOF
|
||||||
|
chmod +x "$dir/adapter.sh"
|
||||||
|
}
|
||||||
|
|
||||||
|
make_wl() { # make_wl FILE — one repo source r1
|
||||||
|
cat >"$1" <<'EOF'
|
||||||
|
{ "schema_version": 1,
|
||||||
|
"repos": [ { "id": "r1" } ],
|
||||||
|
"watches": [ { "lane": "L", "sources": [ { "kind": "repo", "id": "r1" } ] } ] }
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
echo "== P1: first-seen -> SILENT baseline (rows, no enqueue) =="
|
||||||
|
(
|
||||||
|
WAKE_STATE_HOME="$(fresh_state p1)"
|
||||||
|
export WAKE_STATE_HOME
|
||||||
|
unset WAKE_AGENT
|
||||||
|
fx="$TMP_ROOT/p1fx"; mkdir -p "$fx"
|
||||||
|
make_stub "$fx"; make_wl "$fx/wl.json"
|
||||||
|
printf 'extra v1\n' >"$fx/extra.env"
|
||||||
|
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json" WAKE_PREIMAGE_EXTRA="$fx/extra.env"
|
||||||
|
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||||
|
[ "$rc" -eq 0 ] || fail_msg "P1: baseline check must exit 0 (rc=$rc) [$out]"
|
||||||
|
[ "$(ledger_rows)" -eq 3 ] || fail_msg "P1: expected 3 baseline rows (adapter, watch-list, extra), got $(ledger_rows)"
|
||||||
|
[ "$(depth)" = "0" ] || fail_msg "P1: first-seen must NOT enqueue (depth=$(depth))"
|
||||||
|
) && ok
|
||||||
|
|
||||||
|
echo "== P2: change -> prior bytes + first-class hard-locator enqueue =="
|
||||||
|
(
|
||||||
|
WAKE_STATE_HOME="$(fresh_state p2)"
|
||||||
|
export WAKE_STATE_HOME
|
||||||
|
unset WAKE_AGENT
|
||||||
|
fx="$TMP_ROOT/p2fx"; mkdir -p "$fx"
|
||||||
|
make_stub "$fx"; make_wl "$fx/wl.json"
|
||||||
|
printf 'adapter preimage v1\n' >"$fx/extra.sh"
|
||||||
|
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json" WAKE_PREIMAGE_EXTRA="$fx/extra.sh"
|
||||||
|
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P2: baseline failed"
|
||||||
|
old_sha="$(sha256sum "$fx/extra.sh" | awk '{print $1}')"
|
||||||
|
printf 'adapter preimage v2 CHANGED\n' >"$fx/extra.sh"
|
||||||
|
new_sha="$(sha256sum "$fx/extra.sh" | awk '{print $1}')"
|
||||||
|
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||||
|
[ "$rc" -eq 0 ] || fail_msg "P2: change check must exit 0 (rc=$rc) [$out]"
|
||||||
|
sd="$(state_dir)"
|
||||||
|
row="$(jq -c --arg p "$(realpath "$fx/extra.sh")" 'select(.path == $p)' "$sd/preimage/preimage-ledger.jsonl" | tail -n1)"
|
||||||
|
[ "$(jq -r '.prev // ""' <<<"$row")" = "$old_sha" ] || fail_msg "P2: new row must carry prev=<old sha> [$row]"
|
||||||
|
[ "$(jq -r '.sha256' <<<"$row")" = "$new_sha" ] || fail_msg "P2: new row sha mismatch [$row]"
|
||||||
|
# Acceptance (a): the PRIOR BYTES are retrievable from durable state alone.
|
||||||
|
[ -f "$sd/preimage/objects/$old_sha" ] || fail_msg "P2: prior bytes object missing"
|
||||||
|
[ "$(cat "$sd/preimage/objects/$old_sha")" = "adapter preimage v1" ] || fail_msg "P2: prior bytes not byte-exact"
|
||||||
|
[ -f "$sd/preimage/objects/$new_sha" ] || fail_msg "P2: current bytes object missing"
|
||||||
|
# Acceptance (c): ONE first-class actionable with a §2.1 hard locator (path).
|
||||||
|
[ "$(depth)" = "1" ] || fail_msg "P2: exactly one enqueue expected (depth=$(depth))"
|
||||||
|
ent="$(tail -n1 "$sd/pending.jsonl")"
|
||||||
|
[ "$(jq -r '.class' <<<"$ent")" = "actionable" ] || fail_msg "P2: entry class must be actionable [$ent]"
|
||||||
|
[ "$(jq -r '.locators.kind' <<<"$ent")" = "preimage" ] || fail_msg "P2: locator kind must be preimage [$ent]"
|
||||||
|
[ -n "$(jq -r '.locators.path // ""' <<<"$ent")" ] || fail_msg "P2: locator must carry path (hard locator) [$ent]"
|
||||||
|
[ "$(jq -r '.locators.observed_hash' <<<"$ent")" = "$new_sha" ] || fail_msg "P2: locator observed_hash mismatch [$ent]"
|
||||||
|
[ "$(jq -r '.locators.prev_hash' <<<"$ent")" = "$old_sha" ] || fail_msg "P2: locator prev_hash mismatch [$ent]"
|
||||||
|
) && ok
|
||||||
|
|
||||||
|
echo "== P3: detector orders the cause line BEFORE the delta it explains =="
|
||||||
|
(
|
||||||
|
WAKE_STATE_HOME="$(fresh_state p3)"
|
||||||
|
export WAKE_STATE_HOME
|
||||||
|
unset WAKE_AGENT
|
||||||
|
fx="$TMP_ROOT/p3fx"; mkdir -p "$fx"
|
||||||
|
make_stub "$fx"; make_wl "$fx/wl.json"
|
||||||
|
printf 'r1 state v1\n' >"$fx/repo_r1"
|
||||||
|
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json"
|
||||||
|
unset WAKE_PREIMAGE_EXTRA
|
||||||
|
"$DET" poll-once >/dev/null 2>&1 || fail_msg "P3: baseline poll failed"
|
||||||
|
[ "$(depth)" = "0" ] || fail_msg "P3: baseline poll must enqueue nothing (depth=$(depth))"
|
||||||
|
# Change the ADAPTER (preimage) and the source state in the same window.
|
||||||
|
printf '# comment: adapter changed\n' >>"$fx/adapter.sh"
|
||||||
|
printf 'r1 state v2\n' >"$fx/repo_r1"
|
||||||
|
"$DET" poll-once >/dev/null 2>&1 || fail_msg "P3: second poll failed"
|
||||||
|
sd="$(state_dir)"
|
||||||
|
pre_seq="$(jq -r 'select(.locators.kind == "preimage") | .observed_seq' "$sd/pending.jsonl" | head -n1)"
|
||||||
|
src_seq="$(jq -r 'select(.locators.kind == "repo") | .observed_seq' "$sd/pending.jsonl" | head -n1)"
|
||||||
|
[ -n "$pre_seq" ] || fail_msg "P3: no preimage cause entry enqueued"
|
||||||
|
[ -n "$src_seq" ] || fail_msg "P3: no source delta entry enqueued"
|
||||||
|
if [ -n "$pre_seq" ] && [ -n "$src_seq" ]; then
|
||||||
|
[ "$pre_seq" -lt "$src_seq" ] || fail_msg "P3: cause line must precede the delta (preimage seq=$pre_seq, source seq=$src_seq)"
|
||||||
|
fi
|
||||||
|
) && ok
|
||||||
|
|
||||||
|
echo "== P4: credential-store PATH refused — bytes never captured (acceptance b) =="
|
||||||
|
(
|
||||||
|
WAKE_STATE_HOME="$(fresh_state p4)"
|
||||||
|
export WAKE_STATE_HOME
|
||||||
|
unset WAKE_AGENT
|
||||||
|
fx="$TMP_ROOT/p4fx"; mkdir -p "$fx"
|
||||||
|
export MOSAIC_HOME="$fx/mosaic-home"
|
||||||
|
mkdir -p "$MOSAIC_HOME"
|
||||||
|
secret='cred-value-P4-do-not-capture'
|
||||||
|
printf '{"svc":{"token":"%s"}}\n' "$secret" >"$MOSAIC_HOME/credentials.json"
|
||||||
|
# Operator error: the credential store listed as a preimage extra.
|
||||||
|
export WAKE_PREIMAGE_EXTRA="$MOSAIC_HOME/credentials.json"
|
||||||
|
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST
|
||||||
|
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||||
|
[ "$rc" -eq 0 ] || fail_msg "P4: refusal is not an infra failure (rc=$rc) [$out]"
|
||||||
|
sd="$(state_dir)"
|
||||||
|
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
|
||||||
|
[ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P4: row must record captured=false [$row]"
|
||||||
|
jq -r '.refused // ""' <<<"$row" | grep -qi 'path' || fail_msg "P4: refusal must name the path deny [$row]"
|
||||||
|
# THE invariant: the secret bytes exist NOWHERE under the wake state dir.
|
||||||
|
if grep -rq "$secret" "$sd" 2>/dev/null; then
|
||||||
|
fail_msg "P4: credential bytes leaked into the wake state dir"
|
||||||
|
fi
|
||||||
|
) && ok
|
||||||
|
|
||||||
|
echo "== P5: secret-SHAPED content refused (refusal, not redaction) =="
|
||||||
|
(
|
||||||
|
WAKE_STATE_HOME="$(fresh_state p5)"
|
||||||
|
export WAKE_STATE_HOME
|
||||||
|
unset WAKE_AGENT
|
||||||
|
fx="$TMP_ROOT/p5fx"; mkdir -p "$fx"
|
||||||
|
tok='ghp_abcdefghijklmnop0123456789ABCDEF'
|
||||||
|
printf 'export MY_TOKEN=%s\n' "$tok" >"$fx/leaky.env"
|
||||||
|
export WAKE_PREIMAGE_EXTRA="$fx/leaky.env"
|
||||||
|
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
|
||||||
|
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||||
|
[ "$rc" -eq 0 ] || fail_msg "P5: refusal is not an infra failure (rc=$rc) [$out]"
|
||||||
|
sd="$(state_dir)"
|
||||||
|
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
|
||||||
|
[ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P5: row must record captured=false [$row]"
|
||||||
|
jq -r '.refused // ""' <<<"$row" | grep -qi 'secret' || fail_msg "P5: refusal must name secret-shaped content [$row]"
|
||||||
|
if grep -rq "$tok" "$sd" 2>/dev/null; then
|
||||||
|
fail_msg "P5: secret-shaped bytes leaked into the wake state dir"
|
||||||
|
fi
|
||||||
|
) && ok
|
||||||
|
|
||||||
|
echo "== P6: deletion is a tracked CHANGE (ABSENT), not an error =="
|
||||||
|
(
|
||||||
|
WAKE_STATE_HOME="$(fresh_state p6)"
|
||||||
|
export WAKE_STATE_HOME
|
||||||
|
unset WAKE_AGENT
|
||||||
|
fx="$TMP_ROOT/p6fx"; mkdir -p "$fx"
|
||||||
|
printf 'to be deleted\n' >"$fx/gone.env"
|
||||||
|
export WAKE_PREIMAGE_EXTRA="$fx/gone.env"
|
||||||
|
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
|
||||||
|
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P6: baseline failed"
|
||||||
|
rm -f "$fx/gone.env"
|
||||||
|
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||||
|
[ "$rc" -eq 0 ] || fail_msg "P6: deletion check must exit 0 (rc=$rc) [$out]"
|
||||||
|
sd="$(state_dir)"
|
||||||
|
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
|
||||||
|
[ "$(jq -r '.sha256' <<<"$row")" = "ABSENT" ] || fail_msg "P6: deletion must record sha256=ABSENT [$row]"
|
||||||
|
[ "$(depth)" = "1" ] || fail_msg "P6: deletion is a change and must enqueue (depth=$(depth))"
|
||||||
|
) && ok
|
||||||
|
|
||||||
|
echo "== P7: no-change is idempotent =="
|
||||||
|
(
|
||||||
|
WAKE_STATE_HOME="$(fresh_state p7)"
|
||||||
|
export WAKE_STATE_HOME
|
||||||
|
unset WAKE_AGENT
|
||||||
|
fx="$TMP_ROOT/p7fx"; mkdir -p "$fx"
|
||||||
|
printf 'stable\n' >"$fx/stable.env"
|
||||||
|
export WAKE_PREIMAGE_EXTRA="$fx/stable.env"
|
||||||
|
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
|
||||||
|
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P7: baseline failed"
|
||||||
|
r1="$(ledger_rows)"
|
||||||
|
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P7: second check failed"
|
||||||
|
[ "$(ledger_rows)" = "$r1" ] || fail_msg "P7: no-change must not append rows ($r1 -> $(ledger_rows))"
|
||||||
|
[ "$(depth)" = "0" ] || fail_msg "P7: no-change must not enqueue (depth=$(depth))"
|
||||||
|
) && ok
|
||||||
|
|
||||||
|
echo "== P8: unresolvable adapter -> FAIL LOUD, never 'no change' (D2 class) =="
|
||||||
|
(
|
||||||
|
WAKE_STATE_HOME="$(fresh_state p8)"
|
||||||
|
export WAKE_STATE_HOME
|
||||||
|
unset WAKE_AGENT
|
||||||
|
export WAKE_DETECTOR_SOURCE_CMD="$TMP_ROOT/does-not-exist-adapter"
|
||||||
|
unset WAKE_WATCH_LIST WAKE_PREIMAGE_EXTRA MOSAIC_HOME
|
||||||
|
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||||
|
[ "$rc" -ne 0 ] || fail_msg "P8: an unobservable preimage definition must FAIL LOUD (rc=0)"
|
||||||
|
echo "$out" | grep -qi 'does not resolve' || fail_msg "P8: the failure must name the unresolvable adapter [$out]"
|
||||||
|
) && ok
|
||||||
|
|
||||||
|
echo "== P9: corrupt ledger -> REFUSE to compare/re-baseline =="
|
||||||
|
(
|
||||||
|
WAKE_STATE_HOME="$(fresh_state p9)"
|
||||||
|
export WAKE_STATE_HOME
|
||||||
|
unset WAKE_AGENT
|
||||||
|
fx="$TMP_ROOT/p9fx"; mkdir -p "$fx"
|
||||||
|
printf 'v1\n' >"$fx/f.env"
|
||||||
|
export WAKE_PREIMAGE_EXTRA="$fx/f.env"
|
||||||
|
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
|
||||||
|
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P9: baseline failed"
|
||||||
|
sd="$(state_dir)"
|
||||||
|
printf 'NOT-JSON-GARBAGE{{{\n' >>"$sd/preimage/preimage-ledger.jsonl"
|
||||||
|
r1="$(wc -l <"$sd/preimage/preimage-ledger.jsonl")"
|
||||||
|
printf 'v2 changed\n' >"$fx/f.env"
|
||||||
|
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||||
|
[ "$rc" -ne 0 ] || fail_msg "P9: a corrupt ledger must FAIL LOUD (rc=0)"
|
||||||
|
echo "$out" | grep -qi 'unparseable' || fail_msg "P9: the failure must name the corrupt ledger [$out]"
|
||||||
|
[ "$(wc -l <"$sd/preimage/preimage-ledger.jsonl")" = "$r1" ] || fail_msg "P9: nothing may be appended over corrupt history"
|
||||||
|
) && ok
|
||||||
|
|
||||||
|
echo "== P10: oversized file -> bytes refused, hash still recorded =="
|
||||||
|
(
|
||||||
|
WAKE_STATE_HOME="$(fresh_state p10)"
|
||||||
|
export WAKE_STATE_HOME
|
||||||
|
unset WAKE_AGENT
|
||||||
|
fx="$TMP_ROOT/p10fx"; mkdir -p "$fx"
|
||||||
|
head -c 200 /dev/zero | tr '\0' 'A' >"$fx/big.bin"
|
||||||
|
export WAKE_PREIMAGE_EXTRA="$fx/big.bin" WAKE_PREIMAGE_MAX_BYTES=64
|
||||||
|
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
|
||||||
|
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||||
|
[ "$rc" -eq 0 ] || fail_msg "P10: size refusal is not an infra failure (rc=$rc) [$out]"
|
||||||
|
sd="$(state_dir)"
|
||||||
|
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
|
||||||
|
[ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P10: row must record captured=false [$row]"
|
||||||
|
jq -r '.refused // ""' <<<"$row" | grep -qi 'MAX_BYTES' || fail_msg "P10: refusal must name the size cap [$row]"
|
||||||
|
sha="$(jq -r '.sha256' <<<"$row")"
|
||||||
|
[ "$sha" = "$(sha256sum "$fx/big.bin" | awk '{print $1}')" ] || fail_msg "P10: hash must still be recorded [$row]"
|
||||||
|
[ ! -f "$sd/preimage/objects/$sha" ] || fail_msg "P10: oversized bytes must NOT be stored"
|
||||||
|
) && ok
|
||||||
|
|
||||||
|
echo "== P11: reconcile surfaces the cause line before its enumerations =="
|
||||||
|
(
|
||||||
|
WAKE_STATE_HOME="$(fresh_state p11)"
|
||||||
|
export WAKE_STATE_HOME
|
||||||
|
unset WAKE_AGENT
|
||||||
|
fx="$TMP_ROOT/p11fx"; mkdir -p "$fx"
|
||||||
|
make_stub "$fx"; make_wl "$fx/wl.json"
|
||||||
|
printf 'r1 state\n' >"$fx/repo_r1"
|
||||||
|
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json"
|
||||||
|
unset WAKE_PREIMAGE_EXTRA MOSAIC_HOME
|
||||||
|
# Baseline the preimage set, then change the adapter while "the detector is
|
||||||
|
# down" — the reconcile path must still produce the first-class cause line.
|
||||||
|
"$PRE" check >/dev/null 2>&1 || fail_msg "P11: preimage baseline failed"
|
||||||
|
printf '# adapter changed while detector down\n' >>"$fx/adapter.sh"
|
||||||
|
"$RECON" reconcile >/dev/null 2>&1 # rc 1 expected (unaccounted enumerated)
|
||||||
|
sd="$(state_dir)"
|
||||||
|
pre_seq="$(jq -r 'select(.locators.kind == "preimage") | .observed_seq' "$sd/pending.jsonl" | head -n1)"
|
||||||
|
enum_seq="$(jq -r 'select(.locators.reconciled == true) | .observed_seq' "$sd/pending.jsonl" | head -n1)"
|
||||||
|
[ -n "$pre_seq" ] || fail_msg "P11: reconcile must enqueue the preimage cause line"
|
||||||
|
[ -n "$enum_seq" ] || fail_msg "P11: reconcile must still enumerate the unaccounted source"
|
||||||
|
if [ -n "$pre_seq" ] && [ -n "$enum_seq" ]; then
|
||||||
|
[ "$pre_seq" -lt "$enum_seq" ] || fail_msg "P11: cause line must precede the enumeration (preimage seq=$pre_seq, enum seq=$enum_seq)"
|
||||||
|
fi
|
||||||
|
) && ok
|
||||||
|
|
||||||
|
echo "== P12: detector — preimage infra failure is LOUD but does not starve observation =="
|
||||||
|
(
|
||||||
|
WAKE_STATE_HOME="$(fresh_state p12)"
|
||||||
|
export WAKE_STATE_HOME
|
||||||
|
unset WAKE_AGENT
|
||||||
|
fx="$TMP_ROOT/p12fx"; mkdir -p "$fx"
|
||||||
|
make_stub "$fx"; make_wl "$fx/wl.json"
|
||||||
|
printf 'r1 state\n' >"$fx/repo_r1"
|
||||||
|
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json"
|
||||||
|
unset WAKE_PREIMAGE_EXTRA MOSAIC_HOME
|
||||||
|
# Corrupt the ledger BEFORE the first poll: the preimage check fails loud,
|
||||||
|
# but the poll must still observe + baseline the source.
|
||||||
|
sd="$(state_dir)"
|
||||||
|
mkdir -p "$sd/preimage"
|
||||||
|
printf 'NOT-JSON-GARBAGE{{{\n' >"$sd/preimage/preimage-ledger.jsonl"
|
||||||
|
out="$("$DET" poll-once 2>&1)"; rc=$?
|
||||||
|
[ "$rc" -ne 0 ] || fail_msg "P12: the pass must exit non-zero on preimage infra failure"
|
||||||
|
echo "$out" | grep -qi 'preimage' || fail_msg "P12: the failure must name the preimage check [$out]"
|
||||||
|
n="$(find "$sd/detector" -name 'watch-*.hash' 2>/dev/null | wc -l | tr -d '[:space:]')"
|
||||||
|
[ "$n" -ge 1 ] || fail_msg "P12: source observation must still proceed (no watch hash baselined)"
|
||||||
|
) && ok
|
||||||
|
|
||||||
|
echo
|
||||||
|
total=$((pass + $(wc -l <"$FAILFILE")))
|
||||||
|
echo "== test-wake-preimage: $pass/$total passed =="
|
||||||
|
[ -s "$FAILFILE" ] && exit 1
|
||||||
|
exit 0
|
||||||
@@ -25,7 +25,7 @@
|
|||||||
"lint": "eslint src",
|
"lint": "eslint src",
|
||||||
"typecheck": "tsc --noEmit",
|
"typecheck": "tsc --noEmit",
|
||||||
"test": "vitest run --passWithNoTests && pnpm run test:framework-shell",
|
"test": "vitest run --passWithNoTests && pnpm run test:framework-shell",
|
||||||
"test:framework-shell": "python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-install.sh"
|
"test:framework-shell": "python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-preimage.sh && bash framework/tools/wake/test-wake-install.sh"
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@mosaicstack/brain": "workspace:*",
|
"@mosaicstack/brain": "workspace:*",
|
||||||
|
|||||||
Reference in New Issue
Block a user