diff --git a/packages/mosaic/framework/tools/wake/manifest.txt b/packages/mosaic/framework/tools/wake/manifest.txt index 84efaf64..d0342b1b 100644 --- a/packages/mosaic/framework/tools/wake/manifest.txt +++ b/packages/mosaic/framework/tools/wake/manifest.txt @@ -390,15 +390,33 @@ # (acceptance a). A git-repo-in-operator-dir design was REJECTED: # its who/why claim is false under shared-author fleets, and # .gitignore is pattern-based where acceptance (b) demands -# fail-closed. (2) CREDENTIAL HARD GATE (acceptance b): byte -# capture is REFUSED — never redacted — for (i) credential-store -# PATHS (mosaic-home credentials.json, tools/_lib/credentials.json, -# credentials/**, any basename credentials.json), (ii) -# secret-SHAPED content (digest.sh's six scrub shapes reused as a -# content-deny grep), (iii) files over WAKE_PREIMAGE_MAX_BYTES -# (default 1 MiB). A refused file still gets its hash/size/mtime -# row (captured:false + refused reason) so change TIME survives -# even when content must not. (3) FIRST-CLASS CAUSE LINE +# fail-closed. (2) CREDENTIAL HARD GATE (acceptance b), FOUR +# LAYERS (#964 re-verdict: B2 cases C+D): (i) POLARITY — extras +# (WAKE_PREIMAGE_EXTRA) are RECORD-ONLY by default (ledger row + +# cause line, NO bytes); byte capture for an extra requires the +# path listed in WAKE_PREIMAGE_CAPTURE (colon-separated opt-in); +# only the core set (adapter, watch-list) is capture-eligible by +# default — a shape list can only refuse the secrets someone +# already enumerated, so safety may not rest on one (#964-D). +# (ii) PATH DENY evaluated on BOTH the raw and realpath-resolved +# candidate forms against BOTH unresolved and resolved +# mosaic-home anchors (credentials.json, +# tools/_lib/credentials.json, credentials/**, any basename +# credentials.json) — a deny list written in unresolved paths +# cannot match a path resolved before it arrived (#964-C), and a +# symlinked opt-in entry cannot smuggle a denied target (deny +# runs FIRST, independent of what the opt-in matched). (iii) +# CONTENT DENY on the opt-in path only, defense-in-depth NOT the +# safety mechanism: digest.sh's six scrub shapes + a +# named-assignment probe (key/token/secret/passw/hmac/credential/ +# bearer = unbroken value >=16 chars); probe ERROR fails TOWARD +# refusal (an error exit is not a negative result); false +# positives are acceptable — a false match only withholds byte +# capture, never tracking. (iv) size cap WAKE_PREIMAGE_MAX_BYTES +# (default 1 MiB). Deny ALWAYS wins over the opt-in. A refused +# file still gets its hash/size/mtime row (captured:false + +# refused reason) so change TIME survives even when content must +# not. (3) FIRST-CLASS CAUSE LINE # (acceptance c): on a change (or deletion — ABSENT is a state, # not an error), one class=actionable entry is enqueued via the # store allocator with locators {kind:preimage, path (§2.1 hard @@ -414,13 +432,20 @@ # object/ledger write, or failed enqueue is a loud non-zero — # never read as "no change"; in both integrations the pass exits # non-zero but source observation still proceeds (no starvation). -# First-seen is a SILENT baseline (detector first-poll idiom). +# An ABSENT/EMPTY ledger with objects/ NON-empty is a loud +# non-zero REFUSAL to re-baseline (#964-B11: objects with no +# ledger cannot be a first install — history was deleted; the +# first-install path must not silently absorb it). First-seen on +# a genuinely clean state dir is a SILENT baseline (detector +# first-poll idiom). # The installer is UNCHANGED — Gate A auto-enumerates the new # file from the filesystem; the recording site is the runtime # tick, which is where the env-derived preimage set exists. # store.sh/digest.sh/beacon.sh UNCHANGED; watch-list schema # UNTOUCHED ([1,1]). Changed: preimage.sh (new), detector.sh, -# reconcile.sh (+ test-wake-preimage.sh P1-P12). +# reconcile.sh (+ test-wake-preimage.sh P1-P17; P13-P17 are the +# #964 re-verdict regression needles: symlinked store, live +# secret value, polarity, ledger deletion, allowlist-symlink). component=wake version=0.7.0 diff --git a/packages/mosaic/framework/tools/wake/preimage.sh b/packages/mosaic/framework/tools/wake/preimage.sh index 68771cd8..40f09eae 100755 --- a/packages/mosaic/framework/tools/wake/preimage.sh +++ b/packages/mosaic/framework/tools/wake/preimage.sh @@ -16,7 +16,8 @@ # * the resolved file behind WAKE_DETECTOR_SOURCE_CMD (first word), # * the WAKE_WATCH_LIST file, # * operator-declared extras via WAKE_PREIMAGE_EXTRA (colon-separated — -# e.g. detector.env, sink/feed scripts). +# e.g. sink/feed scripts). Extras are RECORD-ONLY by default (see the +# credential hard gate below); never list key-material files. # - Records each file's (sha256, size, mtime, ts) as an APPEND-ONLY ledger # row in /preimage/preimage-ledger.jsonl and stores the full bytes # CONTENT-ADDRESSED at /preimage/objects/ — so a change is @@ -29,28 +30,48 @@ # not just N re-baselined sources. Acceptance (c). # # CREDENTIAL HARD GATE — acceptance (b): the mechanism must be UNABLE to -# capture credential material even by operator error. Enforced two ways, both +# capture credential material even by operator error. Layered, every layer # fail-closed toward NOT capturing bytes (the hash/size/mtime row is still # written — change-time attribution survives, content capture does not; # a sha256 discloses nothing about the bytes): -# 1. PATH deny: the mosaic credential store locations +# 1. POLARITY (#964 review, case D): byte capture is DENY-BY-DEFAULT for +# operator extras. WAKE_PREIMAGE_EXTRA paths are RECORD-ONLY (rows + +# first-class change entries, no bytes) unless explicitly listed in +# WAKE_PREIMAGE_CAPTURE — a shape list can only refuse the secrets +# someone already enumerated, so arbitrary operator-pointed files must +# not default to capture. The CORE set (the resolved adapter file, the +# watch-list) is capture-eligible: it IS the preimage definition this +# tool exists to snapshot, and the deny layers below still apply to it. +# 2. PATH deny: the mosaic credential store locations # (/credentials.json, /tools/_lib/ # credentials.json, anything under /credentials/, and any -# file literally named credentials.json) are refused by resolved realpath -# before a single byte is read into the object store. -# 2. CONTENT deny: a candidate whose bytes match the well-known secret-token +# file literally named credentials.json) are refused. Evaluated on BOTH +# the operator-supplied form AND the symlink-resolved form, against BOTH +# the unresolved and resolved forms of the mosaic-home anchor — a deny +# list written only in unresolved paths cannot match a path that was +# resolved before it arrived (#964 review, case C). +# 3. CONTENT deny: a candidate whose bytes match the well-known secret-token # shapes (same conservative set digest.sh redacts: PAT/Slack/AKIA/JWT/PEM) -# is refused — refusal, not redaction: a redacted preimage would be a -# false witness, and secret bytes must never land in the object store. +# OR a named-assignment secret shape (key/token/secret/passw/hmac/ +# credential/bearer = long unbroken value — catches prefix-less +# high-entropy keys, e.g. an HMAC key in an env file) is refused — +# refusal, not redaction: a redacted preimage would be a false witness, +# and secret bytes must never land in the object store. Deliberately +# conservative toward REFUSAL: a false match only withholds byte capture, +# never tracking. # Plus a size cap (WAKE_PREIMAGE_MAX_BYTES, default 1 MiB) so a stray extra # pointing at a large binary cannot turn provenance into data hoovering. +# Deny ALWAYS wins over the WAKE_PREIMAGE_CAPTURE opt-in. # # FAIL-LOUD DISCIPLINE (the D2/#955 class, both layers): an infrastructure # failure of THIS tool (unresolvable adapter, unreadable/corrupt ledger, # failed durable write, failed enqueue) exits NON-ZERO and is never read as # "no change". A corrupt ledger REFUSES to compare (and to re-baseline): # silently restarting history would erase the very attribution this exists -# to provide. +# to provide. Likewise (#964 review, B11) an ABSENT/EMPTY ledger while +# objects/ still holds prior captures is DELETED HISTORY, not a first +# install — it refuses loudly instead of re-baselining, because a silent +# restart would absorb the next real change as first-seen. # # Operator-agnostic (framework firewall): all state via XDG/env; the deny # list names only framework-defined credential locations, no operator hosts/ @@ -95,7 +116,8 @@ Usage: preimage.sh Commands: check [--enqueue] Compare every preimage-set file against the ledger head. - A changed file gets a new ledger row + captured bytes; + A changed file gets a new ledger row (+ captured bytes + when capture-eligible and not denied — see Environment); with --enqueue each change (not first-seen) also enqueues ONE first-class class=actionable store entry (hard locator: path). First-seen files baseline SILENTLY (row, @@ -111,9 +133,19 @@ Environment: WAKE_DETECTOR_SOURCE_CMD Adapter command; its resolved file joins the set. WAKE_WATCH_LIST Watch-list path; joins the set when set. WAKE_PREIMAGE_EXTRA Colon-separated additional operator preimage files - (e.g. detector.env, sink scripts). A listed path - that does not exist is tracked as ABSENT (deletion - of a preimage file is a change, not an error). + (e.g. sink/feed scripts). Extras are RECORD-ONLY + by default: changes get a hash/size/mtime row and + a first-class change entry, but their BYTES are + never captured unless the path is also listed in + WAKE_PREIMAGE_CAPTURE. A listed path that does not + exist is tracked as ABSENT (deletion of a preimage + file is a change, not an error). + WAKE_PREIMAGE_CAPTURE Colon-separated allowlist of extras whose bytes + MAY be captured. The credential deny rules ALWAYS + win over this list. NEVER list files that can hold + key material (env files with keys/HMACs, credential + stores): the deny gate is a backstop, not a + license. WAKE_PREIMAGE_MAX_BYTES Byte-capture cap (default 1048576). Larger files: hash/size/mtime recorded, bytes refused. WAKE_STATE_HOME/WAKE_AGENT store namespace (see store.sh). @@ -122,10 +154,21 @@ EOF # --- preimage-set derivation (generic; no operator paths baked in) ---------- +# _realpath_or_self PATH — resolved form when resolvable, the input otherwise. +_realpath_or_self() { + local p="$1" + if command -v realpath >/dev/null 2>&1; then + realpath -- "$p" 2>/dev/null || printf '%s' "$p" + else + printf '%s' "$p" + fi +} + # _resolve_cmd_file CMD — resolve the FIRST WORD of an adapter command string -# to a real file. Non-zero (loud) if it cannot be resolved: an adapter the -# detector will invoke but provenance cannot see is an infrastructure failure, -# not a smaller set. +# to a real file; prints `rawresolved`. Non-zero (loud) if it cannot be +# resolved: an adapter the detector will invoke but provenance cannot see is +# an infrastructure failure, not a smaller set. BOTH forms are kept: the deny +# gate must see the pre-resolution form too (#964 review, case C). _resolve_cmd_file() { local cmd="$1" word path # shellcheck disable=SC2086 @@ -139,23 +182,22 @@ _resolve_cmd_file() { [ -f "$path" ] || return 1 fi # realpath so the ledger keys on the actual file, not a symlink alias. - if command -v realpath >/dev/null 2>&1; then - realpath -- "$path" 2>/dev/null || printf '%s' "$path" - else - printf '%s' "$path" - fi + printf '%s\t%s' "$path" "$(_realpath_or_self "$path")" } -# _preimage_set — print the derived set, one path per line. Paths from -# WAKE_PREIMAGE_EXTRA are printed EVEN IF ABSENT (deletion is a tracked -# state); the adapter and watch-list must resolve (loud failure otherwise — -# see _resolve_cmd_file rationale). +# _preimage_set — print the derived set, one member per line as +# `originrawresolved` (origin: core|extra). BOTH path forms travel +# with every member so the deny gate and the capture allowlist are evaluated +# on the SAME candidate in BOTH its forms — resolving before denying is the +# #964 case-C ordering defect. Paths from WAKE_PREIMAGE_EXTRA are printed +# EVEN IF ABSENT (deletion is a tracked state); the adapter and watch-list +# must resolve (loud failure otherwise — see _resolve_cmd_file rationale). _preimage_set() { local failed=0 if [ -n "${WAKE_DETECTOR_SOURCE_CMD:-}" ]; then local af if af="$(_resolve_cmd_file "$WAKE_DETECTOR_SOURCE_CMD")"; then - printf '%s\n' "$af" + printf 'core\t%s\n' "$af" else echo "preimage.sh: FAIL LOUD — WAKE_DETECTOR_SOURCE_CMD ('$WAKE_DETECTOR_SOURCE_CMD') does not resolve to a file; the preimage definition cannot be observed (NOT treated as 'no change')." >&2 failed=1 @@ -163,12 +205,7 @@ _preimage_set() { fi if [ -n "${WAKE_WATCH_LIST:-}" ]; then if [ -f "$WAKE_WATCH_LIST" ]; then - if command -v realpath >/dev/null 2>&1; then - realpath -- "$WAKE_WATCH_LIST" 2>/dev/null || printf '%s' "$WAKE_WATCH_LIST" - else - printf '%s' "$WAKE_WATCH_LIST" - fi - printf '\n' + printf 'core\t%s\t%s\n' "$WAKE_WATCH_LIST" "$(_realpath_or_self "$WAKE_WATCH_LIST")" else echo "preimage.sh: FAIL LOUD — WAKE_WATCH_LIST ('$WAKE_WATCH_LIST') is not a file; the preimage definition cannot be observed." >&2 failed=1 @@ -180,11 +217,10 @@ _preimage_set() { [ -n "$p" ] || continue # Extras are tracked even when absent (ABSENT is a state). Resolve the # realpath only when the file exists; otherwise track the literal path. - if [ -e "$p" ] && command -v realpath >/dev/null 2>&1; then - realpath -- "$p" 2>/dev/null || printf '%s' "$p" - printf '\n' + if [ -e "$p" ]; then + printf 'extra\t%s\t%s\n' "$p" "$(_realpath_or_self "$p")" else - printf '%s\n' "$p" + printf 'extra\t%s\t%s\n' "$p" "$p" fi done fi @@ -193,28 +229,59 @@ _preimage_set() { # --- credential hard gate (acceptance (b)) ---------------------------------- -# _deny_path PATH — 0 (deny) iff PATH is a known credential-store location. -# Framework-defined locations only (firewall): the mosaic credential store, -# the tools/_lib credential file the framework-manifest itself carves out of -# tools/**, the credentials/ operator subtree, and any file literally named -# credentials.json. +# _deny_path RAW RESOLVED — 0 (deny) iff EITHER form of the candidate names a +# known credential-store location. Framework-defined locations only +# (firewall): the mosaic credential store, the tools/_lib credential file the +# framework-manifest itself carves out of tools/**, the credentials/ operator +# subtree, and any file literally named credentials.json. +# +# Evaluated on BOTH the operator-supplied (raw) form and the symlink-resolved +# form, against BOTH the unresolved and resolved forms of the mosaic-home +# anchor: a deny list written only in unresolved paths cannot match a path +# that was resolved before it arrived (#964 review, case C — a symlink whose +# target was renamed slipped every path rule because the candidate had +# already been realpath'd but the anchors never were). _deny_path() { - local p="$1" home="${MOSAIC_HOME:-$HOME/.config/mosaic}" - case "$p" in - "$home/credentials.json") return 0 ;; - "$home/tools/_lib/credentials.json") return 0 ;; - "$home/credentials/"*) return 0 ;; - esac - case "$(basename -- "$p")" in - credentials.json) return 0 ;; - esac + local raw="$1" resolved="$2" + local home="${MOSAIC_HOME:-$HOME/.config/mosaic}" rhome p a + rhome="$(_realpath_or_self "$home")" + for p in "$raw" "$resolved"; do + [ -n "$p" ] || continue + for a in "$home" "$rhome"; do + case "$p" in + "$a/credentials.json") return 0 ;; + "$a/tools/_lib/credentials.json") return 0 ;; + "$a/credentials/"*) return 0 ;; + esac + done + case "$(basename -- "$p")" in + credentials.json) return 0 ;; + esac + done return 1 } -# _deny_content FILE — 0 (deny) iff FILE's bytes match a well-known secret- -# token shape. Same conservative shape set digest.sh redacts (PAT / Slack / -# AKIA / JWT / PEM) — conservative on purpose: a 40-hex git sha never matches. +# _deny_content FILE — 0 (deny) iff FILE's bytes look secret-shaped. +# Two probes: (a) the well-known vendor-token shapes digest.sh redacts +# (PAT / Slack / AKIA / JWT / PEM — conservative: a 40-hex git sha never +# matches); (b) a named-assignment shape (key/token/secret/passw/hmac/ +# credential/bearer = long unbroken value) that catches prefix-less +# high-entropy keys, e.g. an HMAC key in an env file (#964 review, case D). +# +# (b) is defense-in-depth for the OPT-IN path only, NOT the thing that keeps +# case D safe — polarity does that (extras are record-only unless allowlisted; +# a shape list can only refuse the secrets someone already enumerated). It is +# deliberately conservative toward REFUSAL: a false match (a checksum in a +# config, a path that happens to be long and unbroken) only withholds byte +# capture — the hash/size/mtime row and change entry still land. A variable +# REFERENCE (token="$GITEA_TOKEN") never matches: `$` is not in the value +# class — only literal secret values do. +# +# FAILS TOWARD REFUSAL: if a probe itself errors (unreadable file, grep +# failure — rc >= 2), the answer is DENY, not capture. An error exit is not a +# negative result. _deny_content() { + local rc LC_ALL=C grep -Eq \ -e 'gh[pousr]_[A-Za-z0-9]{16,}' \ -e 'github_pat_[A-Za-z0-9_]{16,}' \ @@ -223,6 +290,44 @@ _deny_content() { -e 'eyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}' \ -e '-----BEGIN[A-Z ]*PRIVATE KEY-----' \ -- "$1" 2>/dev/null + rc=$? + [ "$rc" -eq 1 ] || return 0 + LC_ALL=C grep -Eiq \ + -e "[A-Za-z0-9_]*(key|token|secret|passw|hmac|credential|bearer)[A-Za-z0-9_]*[[:space:]]*[=:][[:space:]]*[\"']?[A-Za-z0-9+/=_-]{16,}" \ + -- "$1" 2>/dev/null + rc=$? + [ "$rc" -eq 1 ] || return 0 + return 1 +} + +# _capture_allowed ORIGIN RAW RESOLVED — 0 iff byte capture may even be +# ATTEMPTED for this member (the deny gate still runs after, and wins). +# POLARITY (#964 review, case D): core members (the adapter file, the +# watch-list) are capture-eligible — they ARE the preimage definition this +# tool exists to snapshot (acceptance (a)). Extras are RECORD-ONLY unless +# listed in WAKE_PREIMAGE_CAPTURE. +# +# The allowlist is matched with the SAME both-forms discipline as the deny +# list (entry raw/resolved vs candidate raw/resolved): two lists keyed on +# different strings would let a symlink alias slip between them. "Deny wins +# over opt-in" is a precedence rule, not a guarantee both lists see the same +# path — the guarantee comes from _check_one running _deny_path on both forms +# FIRST, independent of anything matched here. +_capture_allowed() { + local origin="$1" raw="$2" resolved="$3" + [ "$origin" = "core" ] && return 0 + [ -n "${WAKE_PREIMAGE_CAPTURE:-}" ] || return 1 + local IFS=':' e er + for e in $WAKE_PREIMAGE_CAPTURE; do + [ -n "$e" ] || continue + er="$e" + [ -e "$e" ] && er="$(_realpath_or_self "$e")" + if [ "$e" = "$raw" ] || [ "$e" = "$resolved" ] || + [ "$er" = "$raw" ] || [ "$er" = "$resolved" ]; then + return 0 + fi + done + return 1 } # --- ledger primitives ------------------------------------------------------ @@ -253,11 +358,12 @@ _ledger_append() { # --- the check itself ------------------------------------------------------- -# _check_one PATH ENQUEUE — compare PATH to its ledger head; on change record -# (+bytes unless denied) and optionally enqueue. Prints nothing on no-change. -# Returns: 0 ok (changed or not), 1 infrastructure failure. +# _check_one ORIGIN RAW PATH ENQUEUE — compare PATH (the resolved form; the +# ledger keys on it) to its ledger head; on change record (+bytes only if +# capture-eligible and not denied) and optionally enqueue. Prints nothing on +# no-change. Returns: 0 ok (changed or not), 1 infrastructure failure. _check_one() { - local path="$1" enqueue="$2" + local origin="$1" raw="$2" path="$3" enqueue="$4" local cur_sha size mtime denied="" refused="" if [ -f "$path" ]; then @@ -271,12 +377,18 @@ _check_one() { mtime="$(stat -c %Y -- "$path" 2>/dev/null || stat -f %m -- "$path" 2>/dev/null || echo 0)" local cap="${WAKE_PREIMAGE_MAX_BYTES:-1048576}" case "$cap" in '' | *[!0-9]*) cap=1048576 ;; esac - # Deny order matters: path first (a known credential store is refused - # without a content probe — the hash read above is deliberate: a sha256 - # discloses nothing about the bytes), size cap second (never content-grep - # an oversized file), content shape last. - if _deny_path "$path"; then + # Gate order matters: path deny FIRST, on BOTH forms, INDEPENDENT of the + # capture allowlist — so an allowlisted symlink whose target is denied + # refuses no matter what string the allowlist matched (a known credential + # store is refused without a content probe; the hash read above is + # deliberate: a sha256 discloses nothing about the bytes). Then polarity + # (record-only extras never reach the content probe — case D must be safe + # WITHOUT the shape list), then size cap (never content-grep an oversized + # file), content shape last. + if _deny_path "$raw" "$path"; then denied="credential-store path (deny list)" + elif ! _capture_allowed "$origin" "$raw" "$path"; then + denied="extra is record-only by default (byte capture requires WAKE_PREIMAGE_CAPTURE; deny rules still win)" elif [ "$size" -gt "$cap" ]; then denied="exceeds WAKE_PREIMAGE_MAX_BYTES ($size > $cap)" elif _deny_content "$path"; then @@ -303,7 +415,7 @@ _check_one() { if [ -n "$denied" ]; then captured=false refused="$denied" - echo "preimage.sh: REFUSED byte capture for $path ($denied) — hash/size/mtime recorded, content NOT stored (acceptance (b), #958)." >&2 + echo "preimage.sh: byte capture WITHHELD for $path ($denied) — hash/size/mtime recorded, content NOT stored (#958)." >&2 else mkdir -p "$OBJECTS" || return 1 if [ ! -f "$OBJECTS/$cur_sha" ]; then @@ -397,10 +509,21 @@ cmd_check() { echo "preimage.sh: FAIL LOUD — cannot acquire preimage lock" >&2 exit 1 fi - local failed=0 p - while IFS= read -r p; do - [ -n "$p" ] || continue - _check_one "$p" "$enqueue" || failed=1 + # #964 review (B11): an ABSENT/EMPTY ledger while objects/ still holds + # prior captures is DELETED HISTORY, not a first install — the asymmetry is + # locally detectable and is the whole detection. Re-baselining here would + # silently absorb the next real change as first-seen (the exact erasure + # this tool exists to prevent). ABSENT is not CORRUPT: it must not take the + # first-install path either. + if [ ! -s "$LEDGER" ] && [ -d "$OBJECTS" ] && [ -n "$(ls -A -- "$OBJECTS" 2>/dev/null)" ]; then + echo "preimage.sh: FAIL LOUD — preimage ledger $LEDGER is ABSENT/EMPTY but $OBJECTS still holds prior objects: history was deleted; REFUSING to re-baseline over it (a silent restart would absorb the next real change as first-seen). Restore the ledger, or move objects/ aside explicitly after investigation." >&2 + _wake_lock_release + exit 1 + fi + local failed=0 origin raw resolved + while IFS=$'\t' read -r origin raw resolved; do + [ -n "$resolved" ] || continue + _check_one "$origin" "$raw" "$resolved" "$enqueue" || failed=1 done <v3 change is NOT absorbed as first-seen +# P17 allowlist symmetry: an allowlisted symlink whose TARGET is denied +# refuses (deny runs first, on both forms — precedence is not path +# agreement); a symlink to an ALLOWED target still captures (the fix +# must not close case C by breaking every symlinked path) +# # Uses FAKE/STUB sources only (no live network). Isolated per test. # shellcheck disable=SC2030,SC2031 set -uo pipefail @@ -102,29 +120,31 @@ echo "== P1: first-seen -> SILENT baseline (rows, no enqueue) ==" [ "$(depth)" = "0" ] || fail_msg "P1: first-seen must NOT enqueue (depth=$(depth))" ) && ok -echo "== P2: change -> prior bytes + first-class hard-locator enqueue ==" +echo "== P2: adapter change -> prior bytes + first-class hard-locator enqueue ==" ( WAKE_STATE_HOME="$(fresh_state p2)" export WAKE_STATE_HOME unset WAKE_AGENT fx="$TMP_ROOT/p2fx"; mkdir -p "$fx" make_stub "$fx"; make_wl "$fx/wl.json" - printf 'adapter preimage v1\n' >"$fx/extra.sh" - export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json" WAKE_PREIMAGE_EXTRA="$fx/extra.sh" + # Acceptance (a) names the OPERATOR ADAPTER — a core member. Core members + # capture by default (they ARE the preimage definition); extras do not (P15). + export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json" + unset WAKE_PREIMAGE_EXTRA WAKE_PREIMAGE_CAPTURE MOSAIC_HOME "$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P2: baseline failed" - old_sha="$(sha256sum "$fx/extra.sh" | awk '{print $1}')" - printf 'adapter preimage v2 CHANGED\n' >"$fx/extra.sh" - new_sha="$(sha256sum "$fx/extra.sh" | awk '{print $1}')" + cp "$fx/adapter.sh" "$fx/adapter.v1" + old_sha="$(sha256sum "$fx/adapter.sh" | awk '{print $1}')" + printf '# adapter v2 CHANGED\n' >>"$fx/adapter.sh" + new_sha="$(sha256sum "$fx/adapter.sh" | awk '{print $1}')" out="$("$PRE" check --enqueue 2>&1)"; rc=$? [ "$rc" -eq 0 ] || fail_msg "P2: change check must exit 0 (rc=$rc) [$out]" sd="$(state_dir)" - row="$(jq -c --arg p "$(realpath "$fx/extra.sh")" 'select(.path == $p)' "$sd/preimage/preimage-ledger.jsonl" | tail -n1)" + row="$(jq -c --arg p "$(realpath "$fx/adapter.sh")" 'select(.path == $p)' "$sd/preimage/preimage-ledger.jsonl" | tail -n1)" [ "$(jq -r '.prev // ""' <<<"$row")" = "$old_sha" ] || fail_msg "P2: new row must carry prev= [$row]" [ "$(jq -r '.sha256' <<<"$row")" = "$new_sha" ] || fail_msg "P2: new row sha mismatch [$row]" # Acceptance (a): the PRIOR BYTES are retrievable from durable state alone. - [ -f "$sd/preimage/objects/$old_sha" ] || fail_msg "P2: prior bytes object missing" - [ "$(cat "$sd/preimage/objects/$old_sha")" = "adapter preimage v1" ] || fail_msg "P2: prior bytes not byte-exact" - [ -f "$sd/preimage/objects/$new_sha" ] || fail_msg "P2: current bytes object missing" + cmp -s "$sd/preimage/objects/$old_sha" "$fx/adapter.v1" || fail_msg "P2: prior bytes missing or not byte-exact" + cmp -s "$sd/preimage/objects/$new_sha" "$fx/adapter.sh" || fail_msg "P2: current bytes object missing or mismatched" # Acceptance (c): ONE first-class actionable with a §2.1 hard locator (path). [ "$(depth)" = "1" ] || fail_msg "P2: exactly one enqueue expected (depth=$(depth))" ent="$(tail -n1 "$sd/pending.jsonl")" @@ -171,8 +191,11 @@ echo "== P4: credential-store PATH refused — bytes never captured (acceptance mkdir -p "$MOSAIC_HOME" secret='cred-value-P4-do-not-capture' printf '{"svc":{"token":"%s"}}\n' "$secret" >"$MOSAIC_HOME/credentials.json" - # Operator error: the credential store listed as a preimage extra. + # Operator error: the credential store listed as a preimage extra — AND + # explicitly opted in to capture. Deny must win over the opt-in (#964: a + # path both denied and allowlisted must refuse). export WAKE_PREIMAGE_EXTRA="$MOSAIC_HOME/credentials.json" + export WAKE_PREIMAGE_CAPTURE="$MOSAIC_HOME/credentials.json" unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST out="$("$PRE" check --enqueue 2>&1)"; rc=$? [ "$rc" -eq 0 ] || fail_msg "P4: refusal is not an infra failure (rc=$rc) [$out]" @@ -194,7 +217,8 @@ echo "== P5: secret-SHAPED content refused (refusal, not redaction) ==" fx="$TMP_ROOT/p5fx"; mkdir -p "$fx" tok='ghp_abcdefghijklmnop0123456789ABCDEF' printf 'export MY_TOKEN=%s\n' "$tok" >"$fx/leaky.env" - export WAKE_PREIMAGE_EXTRA="$fx/leaky.env" + # Opted in, so the CONTENT gate (not record-only polarity) is what refuses. + export WAKE_PREIMAGE_EXTRA="$fx/leaky.env" WAKE_PREIMAGE_CAPTURE="$fx/leaky.env" unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME out="$("$PRE" check --enqueue 2>&1)"; rc=$? [ "$rc" -eq 0 ] || fail_msg "P5: refusal is not an infra failure (rc=$rc) [$out]" @@ -281,7 +305,9 @@ echo "== P10: oversized file -> bytes refused, hash still recorded ==" unset WAKE_AGENT fx="$TMP_ROOT/p10fx"; mkdir -p "$fx" head -c 200 /dev/zero | tr '\0' 'A' >"$fx/big.bin" + # Opted in, so the SIZE gate (not record-only polarity) is what refuses. export WAKE_PREIMAGE_EXTRA="$fx/big.bin" WAKE_PREIMAGE_MAX_BYTES=64 + export WAKE_PREIMAGE_CAPTURE="$fx/big.bin" unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME out="$("$PRE" check --enqueue 2>&1)"; rc=$? [ "$rc" -eq 0 ] || fail_msg "P10: size refusal is not an infra failure (rc=$rc) [$out]" @@ -341,6 +367,166 @@ echo "== P12: detector — preimage infra failure is LOUD but does not starve ob [ "$n" -ge 1 ] || fail_msg "P12: source observation must still proceed (no watch hash baselined)" ) && ok +echo "== P13: symlinked/renamed credential store refused on BOTH path forms (#964 C) ==" +( + WAKE_STATE_HOME="$(fresh_state p13)" + export WAKE_STATE_HOME + unset WAKE_AGENT + fx="$TMP_ROOT/p13fx"; mkdir -p "$fx" + # The mosaic home is ITSELF behind a symlink: a deny list written in + # unresolved paths cannot match a candidate that was realpath'd before the + # deny saw it — unless the anchors are resolved too. + mkdir -p "$fx/real-store/credentials" + ln -s "$fx/real-store" "$fx/mh" + export MOSAIC_HOME="$fx/mh" + s1='decoy-P13a-renamed-target-under-store' + printf 'x_token=%s\n' "$s1" >"$fx/mh/credentials/brain-creds.json" + # And a renamed target OUTSIDE every known store location, reached via a + # benign-looking symlink: no path rule can name it — the content gate must + # hold on the resolved file. + s2='0123456789abcdef0123456789abcdefP13b' + printf 'service_hmac_key=%s\n' "$s2" >"$fx/renamed-anywhere.cfg" + ln -s "$fx/renamed-anywhere.cfg" "$fx/link-b.env" + export WAKE_PREIMAGE_EXTRA="$fx/mh/credentials/brain-creds.json:$fx/link-b.env" + # Explicit opt-in for BOTH: deny must win over the allowlist. + export WAKE_PREIMAGE_CAPTURE="$fx/mh/credentials/brain-creds.json:$fx/link-b.env" + unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST + out="$("$PRE" check --enqueue 2>&1)"; rc=$? + [ "$rc" -eq 0 ] || fail_msg "P13: refusal is not an infra failure (rc=$rc) [$out]" + sd="$(state_dir)" + n="$(jq -r 'select(.captured == false) | .path' "$sd/preimage/preimage-ledger.jsonl" | wc -l | tr -d '[:space:]')" + [ "$n" = "2" ] || fail_msg "P13: both decoys must record captured=false (got $n)" + # THE invariant (decoy method): the planted bytes exist NOWHERE in state. + if grep -rq "$s1" "$sd" 2>/dev/null; then + fail_msg "P13: renamed-target store bytes leaked into the wake state dir" + fi + if grep -rq "$s2" "$sd" 2>/dev/null; then + fail_msg "P13: prefix-less key bytes leaked into the wake state dir" + fi +) && ok + +echo "== P14: detector.env-class key — safe WITHOUT opt-in by polarity, refused WITH opt-in by shape (#964 D) ==" +( + WAKE_STATE_HOME="$(fresh_state p14)" + export WAKE_STATE_HOME + unset WAKE_AGENT + fx="$TMP_ROOT/p14fx"; mkdir -p "$fx" + hm='9f8e7d6c5b4a39281706f5e4d3c2b1a09f8e7d6c' + printf 'WAKE_HMAC_KEY=%s\n' "$hm" >"$fx/detector.env" + export WAKE_PREIMAGE_EXTRA="$fx/detector.env" + unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME WAKE_PREIMAGE_CAPTURE + # (a) NO opt-in: polarity alone keeps the bytes out. This leg must hold even + # with the content shape list deleted — the shape is defense-in-depth for + # the opt-in path, never the thing that makes case D read safe. + out="$("$PRE" check --enqueue 2>&1)"; rc=$? + [ "$rc" -eq 0 ] || fail_msg "P14a: record-only tracking must succeed (rc=$rc) [$out]" + sd="$(state_dir)" + row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")" + [ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P14a: extra must be record-only without opt-in [$row]" + if grep -rq "$hm" "$sd" 2>/dev/null; then + fail_msg "P14a: key bytes leaked without any opt-in" + fi + # (b) The operator opts the env file in (the exact error the old usage text + # invited): the named-assignment shape must still refuse the bytes. + export WAKE_PREIMAGE_CAPTURE="$fx/detector.env" + printf 'WAKE_HMAC_KEY=%s\nrotated=1\n' "$hm" >"$fx/detector.env" + out="$("$PRE" check --enqueue 2>&1)"; rc=$? + [ "$rc" -eq 0 ] || fail_msg "P14b: refusal is not an infra failure (rc=$rc) [$out]" + row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")" + [ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P14b: opted-in key material must still refuse [$row]" + jq -r '.refused // ""' <<<"$row" | grep -qi 'secret' || fail_msg "P14b: refusal must name secret-shaped content [$row]" + if grep -rq "$hm" "$sd" 2>/dev/null; then + fail_msg "P14b: key bytes leaked despite refusal" + fi +) && ok + +echo "== P15: extras are RECORD-ONLY by default; capture requires explicit opt-in ==" +( + WAKE_STATE_HOME="$(fresh_state p15)" + export WAKE_STATE_HOME + unset WAKE_AGENT + fx="$TMP_ROOT/p15fx"; mkdir -p "$fx" + printf 'plain v1\n' >"$fx/notes.cfg" + export WAKE_PREIMAGE_EXTRA="$fx/notes.cfg" + unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME WAKE_PREIMAGE_CAPTURE + "$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P15: baseline failed" + printf 'plain v2\n' >"$fx/notes.cfg" + sha2="$(sha256sum "$fx/notes.cfg" | awk '{print $1}')" + "$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P15: change check failed" + sd="$(state_dir)" + row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")" + [ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P15: default must be record-only [$row]" + [ ! -f "$sd/preimage/objects/$sha2" ] || fail_msg "P15: bytes captured without opt-in" + [ "$(depth)" = "1" ] || fail_msg "P15: record-only must still track the change (depth=$(depth))" + # Opt in -> the next change captures. + export WAKE_PREIMAGE_CAPTURE="$fx/notes.cfg" + printf 'plain v3\n' >"$fx/notes.cfg" + sha3="$(sha256sum "$fx/notes.cfg" | awk '{print $1}')" + "$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P15: opted-in change check failed" + row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")" + [ "$(jq -r '.captured' <<<"$row")" = "true" ] || fail_msg "P15: opt-in must capture [$row]" + cmp -s "$sd/preimage/objects/$sha3" "$fx/notes.cfg" || fail_msg "P15: opted-in bytes missing or mismatched" +) && ok + +echo "== P16: deleted ledger over surviving objects/ — REFUSE to re-baseline (#964 B11) ==" +( + WAKE_STATE_HOME="$(fresh_state p16)" + export WAKE_STATE_HOME + unset WAKE_AGENT + fx="$TMP_ROOT/p16fx"; mkdir -p "$fx" + make_stub "$fx" + export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" + unset WAKE_WATCH_LIST WAKE_PREIMAGE_EXTRA WAKE_PREIMAGE_CAPTURE MOSAIC_HOME + "$PRE" check >/dev/null 2>&1 || fail_msg "P16: v1 baseline failed" + printf '# v2\n' >>"$fx/adapter.sh" + "$PRE" check >/dev/null 2>&1 || fail_msg "P16: v2 change failed" + sd="$(state_dir)" + nobj="$(ls "$sd/preimage/objects" | wc -l | tr -d '[:space:]')" + # Delete the ledger; objects/ survives. ABSENT is not CORRUPT — and it is + # not a first install either: that asymmetry is locally detectable. + rm -f "$sd/preimage/preimage-ledger.jsonl" + printf '# v3\n' >>"$fx/adapter.sh" + out="$("$PRE" check --enqueue 2>&1)"; rc=$? + [ "$rc" -ne 0 ] || fail_msg "P16: absent ledger over prior objects must FAIL LOUD (rc=0) [$out]" + echo "$out" | grep -qi 'REFUSING to re-baseline' || fail_msg "P16: the failure must name the refusal [$out]" + [ ! -e "$sd/preimage/preimage-ledger.jsonl" ] || fail_msg "P16: the ledger must NOT be silently recreated over deleted history" + [ "$(ls "$sd/preimage/objects" | wc -l | tr -d '[:space:]')" = "$nobj" ] || fail_msg "P16: prior objects must remain untouched" + [ "$(depth)" = "0" ] || fail_msg "P16: the v2->v3 change must NOT be absorbed or enqueued from an unverifiable baseline (depth=$(depth))" +) && ok + +echo "== P17: allowlist symmetry — symlink to a DENIED target refuses; symlink to an allowed target captures ==" +( + WAKE_STATE_HOME="$(fresh_state p17)" + export WAKE_STATE_HOME + unset WAKE_AGENT + fx="$TMP_ROOT/p17fx"; mkdir -p "$fx" + export MOSAIC_HOME="$fx/mh" + mkdir -p "$MOSAIC_HOME" + s='decoy-P17-cred-behind-benign-alias' + printf '{"svc":{"token":"%s"}}\n' "$s" >"$MOSAIC_HOME/credentials.json" + # A benign-looking alias whose TARGET is denied: the allowlist matches the + # alias string, but deny runs FIRST and on BOTH forms — "deny wins over + # opt-in" is a precedence rule, path agreement is what makes it hold. + ln -s "$MOSAIC_HOME/credentials.json" "$fx/settings.json" + # And a symlink to an ALLOWED target: the fix must not close case C by + # breaking every symlinked path outright. + printf 'benign payload v1\n' >"$fx/target.cfg" + ln -s "$fx/target.cfg" "$fx/alias.cfg" + export WAKE_PREIMAGE_EXTRA="$fx/settings.json:$fx/alias.cfg" + export WAKE_PREIMAGE_CAPTURE="$fx/settings.json:$fx/alias.cfg" + unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST + out="$("$PRE" check --enqueue 2>&1)"; rc=$? + [ "$rc" -eq 0 ] || fail_msg "P17: check must exit 0 (rc=$rc) [$out]" + sd="$(state_dir)" + crow="$(jq -c --arg p "$(realpath "$fx/settings.json")" 'select(.path == $p)' "$sd/preimage/preimage-ledger.jsonl" | tail -n1)" + [ "$(jq -r '.captured' <<<"$crow")" = "false" ] || fail_msg "P17: allowlisted symlink to a denied target must refuse [$crow]" + if grep -rq "$s" "$sd" 2>/dev/null; then + fail_msg "P17: credential bytes leaked via an allowlisted alias" + fi + brow="$(jq -c --arg p "$(realpath "$fx/alias.cfg")" 'select(.path == $p)' "$sd/preimage/preimage-ledger.jsonl" | tail -n1)" + [ "$(jq -r '.captured' <<<"$brow")" = "true" ] || fail_msg "P17: symlink to an allowed target must still capture [$brow]" +) && ok + echo total=$((pass + $(wc -l <"$FAILFILE"))) echo "== test-wake-preimage: $pass/$total passed =="