Compare commits

..
Author SHA1 Message Date
Jason WoltjeandClaude Fable 5 a8d3acef03 feat(wake): #958 A11 preimage.sh — durable provenance for the operator-side preimage definition
ci/woodpecker/pr/ci Pipeline is pending approval
The operator-owned bytes every observed_hash is computed FROM (the source
adapter, the watch-list) were unversioned: a byte change was attributable
only via an agent transcript, and reconcile surfaced it as N UNACCOUNTED
sources instead of one cause.

New tool preimage.sh (wake 0.6.15 -> 0.7.0):
- Derives the preimage set from the runtime env (resolved
  WAKE_DETECTOR_SOURCE_CMD file, WAKE_WATCH_LIST, WAKE_PREIMAGE_EXTRA);
  per file appends {ts,path,sha256,size,mtime,prev} to an append-only
  ledger and captures bytes content-addressed under
  $STATE_DIR/preimage/objects/<sha256> — prior bytes + change time from
  durable state alone (acceptance a).
- CREDENTIAL HARD GATE (acceptance b): byte capture is REFUSED — never
  redacted — for credential-store paths, secret-shaped content (digest's
  six scrub shapes as a content-deny), and files over
  WAKE_PREIMAGE_MAX_BYTES; the refused file still gets its
  hash/size/mtime row (captured:false) so change TIME survives.
- FIRST-CLASS CAUSE LINE (acceptance c): a change/deletion enqueues one
  class=actionable entry via the store allocator with path as its §2.1
  hard locator; detector poll-once and reconcile run the check as a
  PRE-step so the cause line lands at a LOWER observed_seq than the
  deltas/enumerations it explains.
- FAIL-LOUD (D2/#955 class): unresolvable adapter, corrupt ledger
  (refuses re-baseline), failed object/ledger write, failed enqueue are
  loud non-zero, never "no change"; in both integrations the pass exits
  non-zero but source observation still proceeds.

Installer unchanged (Gate A auto-enumerates the new file; the recording
site is the runtime tick where the env-derived set exists). Watch-list
schema untouched ([1,1]).

Tests: test-wake-preimage.sh P1-P12 (red-first verified: P3/P11/P12 fail
with the integration edits reverted); all 9 existing wake suites green.

Refs #958

Agent: PEPPER (sb-it-1-dt)

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01NsKce8iZuSuRnu3gVMCBKB
2026-07-30 15:46:12 -05:00
9 changed files with 884 additions and 146 deletions
@@ -39,7 +39,6 @@ overwritten on upgrade. (Layer model: `constitution/LAYER-MODEL.md`.)
| TypeScript strict typing | `guides/TYPESCRIPT.md` |
| QA / test strategy | `guides/QA-TESTING.md` |
| Documentation (any code/API/auth/infra change) | `guides/DOCUMENTATION.md` |
| Writing style (docs, comms, any prose) | `guides/WRITING-STYLE.md` |
| Secrets / vault usage | `guides/VAULT-SECRETS.md` |
| Tool/credential reference (service CLIs, wrappers) | `guides/TOOLS-REFERENCE.md` |
| Memory protocol (OpenBrain capture/recall) | `guides/MEMORY.md` |
@@ -27,14 +27,6 @@ Master/slave model:
- Do not perform destructive git/file actions without explicit instruction.
- Browser automation (Playwright, Cypress, Puppeteer) MUST run in headless mode. Never launch a visible browser — it collides with the user's display and active session.
### Output standards (writing + code)
- Technical documentation follows **MOS-STE** (Mosaic Simplified Technical English — an adapted ASD-STE100 profile): short sentences, one instruction per sentence, active voice, one word per meaning, one term per concept. Full rules: `~/.config/mosaic/guides/WRITING-STYLE.md`.
- Apply MOS-STE **hardest to verification artifacts** (acceptance criteria, witness predicates, gate/alarm conditions). There an ambiguous term produces a false green, not just a confused reader.
- Source code follows the **Google Style Guide** for the language.
- User-facing comms follow the user's declared `communicationStyle` in `USER.md` "Communication Preferences" (`direct` | `friendly` | `formal`, default `direct`); `guides/WRITING-STYLE.md` §5 maps each value to output. The documentation standard does not change with user preference.
- **Carve-out:** MOS-STE does NOT apply to content that must carry a specific human voice (letters, personal or marketing prose, voice-matched output). A declared voice profile wins.
### Secrets handling (HARD RULE)
- Vault is the canonical source-of-truth for every secret in every environment. No exceptions.
@@ -1,134 +0,0 @@
# Writing Style Standard — MOS-STE (MANDATORY)
This guide defines how agents write. It sets one style standard per output type.
It is written in the standard it defines, as a worked example.
**Adapted, not compliant.** MOS-STE (Mosaic Simplified Technical English) is an
adapted profile of ASD-STE100. Mosaic does not license or certify against
ASD-STE100. Mosaic uses the load-bearing rules and fits them to agent work. This
is the same stance Mosaic takes toward DO-178B/C: use the rigor, do not claim the
certification.
## Scope — which standard governs which output
| Output type | Standard |
| ------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| Technical documentation (READMEs, runbooks, PRDs, procedures, ADRs, guides, acceptance criteria, design docs) | **MOS-STE** (this guide) |
| Source code and code comments | **Google Style Guide** for the language (§4) |
| Inter-agent comms | MOS-STE by default (concise, structured) |
| User-facing comms | **Per-user style choice** — read `USER.md` "Communication Preferences" (§5) |
| End-user prose the user owns (marketing, letters, personal writing, voice-matched content) | The user's declared voice. MOS-STE does NOT apply. |
**The user-voice carve-out is absolute.** Do not apply MOS-STE to content that
must carry a specific human voice (for example a cover letter, a personal
message, or marketing copy). That content needs the user's voice. MOS-STE would
damage it. When a project declares a voice profile, that profile wins.
## 1. Why one standard
Agent documentation drifts across projects. Different agents use different terms,
sentence styles, and structures for the same concept. Readers lose time.
Assumptions hide in ambiguous prose. One standard gives agents a clear target. It
gives reviewers a clear test.
## 2. Where MOS-STE matters most — verification artifacts
Apply MOS-STE hardest to acceptance criteria, witness predicates, gate
definitions, and alarm conditions. In prose, an ambiguous term produces a
confused reader. In a verification artifact, an ambiguous term produces a false
green — a check that passes without testing the claim.
The one-term-one-concept rule (rule 9) is the guard. When one word names two
concepts in one predicate, the check can test the wrong concept and still pass.
**Worked failure.** A rename used a witness predicate with three clauses: ref A
present, ref B absent, tip committed from this host. Every clause tested the git
_ref_ (the channel). The claim under test was about a _field inside the payload_.
The word "beacon" named two concepts in one sentence. Deleting ref B was the next
scheduled step. That step flips the last clause green and certifies a state in
which the payload still names the wrong host. The predicate was one planned action
away from a false green on its normal path. The payload field was never tested.
Rule: when N failure modes share one observable, the observable is not a
diagnostic. In a verification artifact, that ambiguity does not confuse a reader —
it certifies the defect.
## 3. MOS-STE rules
### 3.1 Sentence rules
1. Keep sentences short. Use 20 words or fewer for a procedure. Use 25 words or
fewer for a description. (Reasoning and doctrine prose relaxes this limit —
see §3.4. A future lint enforces §3.1, not §3.4.)
2. Write one instruction per sentence. In a procedure, give one command per step.
3. Use the active voice. Write "Run the script." Do not write "The script should
be run."
4. Use the imperative for instructions. Start the sentence with the verb.
5. Use simple verb tenses. Prefer the present tense. Avoid the perfect and
progressive tenses when a simple tense works.
6. Do not use an `-ing` form when it makes the meaning unclear.
7. Write positive statements. State what to do, not only what to avoid.
### 3.2 Word rules
8. Use one word for one meaning. Do not use the same word in two senses.
9. Use one term for one concept. Do not use synonyms for variety. Example: choose
`secret`, `credential`, or `key` for each concept, and keep it.
10. Use articles (`a`, `the`). Do not drop words to save space.
11. Keep an approved-terms glossary per project. Add each domain noun and each
chosen verb. Technical names (for example `Vault`, `cgroup`, `systemd`) are
always allowed.
12. Define an abbreviation at its first use. Then use it consistently.
### 3.3 Structure rules
13. Use a list for parallel items or sequential steps. Do not put them in one long
sentence.
14. Use a table for data with more than two dimensions.
15. Use parallel structure in headings and steps.
16. Repeat the noun. Do not use a pronoun when the reference is unclear.
### 3.4 Adaptation notes (where MOS-STE deviates from ASD-STE100, and why)
- **No licensed dictionary.** ASD-STE100 ships a controlled dictionary under
copyright. MOS-STE uses per-project glossaries instead (rule 11).
- **Domain terms are allowed.** MOS-STE keeps every term the work needs.
- **Reasoning prose gets structure, not amputation.** Apply the sentence and word
rules to design and doctrine writing. Allow the length a subtle argument needs.
Readable-first beats rule-strict when the two conflict.
## 4. Code — Google Style Guide
Write source code to the Google Style Guide for the language (Python, TypeScript,
Shell, Go, and so on). Match the existing file when a local convention already
exists. Keep code comments to the MOS-STE sentence and word rules.
## 5. User-facing comms — a per-user choice
Mosaic is multi-user. Different users want different comms styles. The framework
already carries the selectable setting: `communicationStyle` (`direct` |
`friendly` | `formal`, default `direct`). `mosaic init` writes it, and the
builder renders it into the generated `USER.md` "Communication Preferences"
section. This guide adds the OUTPUT meaning of each value; do not invent new
values.
The builder renders the style as prose bullets, not the token name, so match on
the leading bullet the generated `USER.md` actually contains:
| `USER.md` leading bullet | Style | User-facing output |
| ----------------------------- | ------------------ | ---------------------------------------------------------------------- |
| "Direct and concise" | `direct` (default) | MOS-STE structure — short, active, defined terms, tables for overview. |
| "Warm and conversational" | `friendly` | Warmer register. Full sentences, explain reasoning, fewer tables. |
| "Professional and structured" | `formal` | Professional and structured. Thorough, with explicit recommendations. |
This setting governs **user-facing comms only**. It does not change the
documentation standard (§3), which is always MOS-STE regardless of the value.
## 6. Enforcement
- **Now:** human review only. **No mechanical prose check exists today.** The
pre-push gate runs typecheck, lint, build, and tests; it inspects no prose.
Reviewers check output against the scope table and the MOS-STE rules by hand.
- **Future:** an MOS-STE lint check (built from the §3.1 sentence rules) and a
Google-style linter in the pre-push gate. A future linter enforces §3.1, not
§3.4 — see the note at rule 1.
@@ -414,6 +414,22 @@ cmd_poll_once() {
_wake_clean_stale_tmp "$STATE_DIR"
mkdir -p "$DET_DIR"
local failed=0
# #958 preimage provenance: check the OPERATOR-SIDE preimage definition (the
# source adapter file, the watch-list, operator-declared extras) BEFORE
# observing any source. A changed preimage re-baselines EVERY source at once;
# running the check first means its first-class cause line is enqueued at a
# LOWER observed_seq than the N per-source deltas it explains, so the digest
# shows the cause, not just the flood. An infrastructure failure of the check
# is LOUD and marks this pass failed (G2a discipline — never read as "no
# change"), but source observation still proceeds: provenance must not be
# able to starve wake delivery.
if ! "$SCRIPT_DIR/preimage.sh" check --enqueue; then
echo "detector.sh: FAIL LOUD — preimage provenance check failed (see preimage.sh above); source observation continues but this pass exits non-zero." >&2
failed=1
fi
# Iterate the DECLARED source-coverage inventory (§4/G3): only sources listed
# in watches[].sources[] are polled. An omitted source is not observed (and so
# cannot make anything pass vacuously); a referenced-but-undefined source is a
@@ -427,7 +443,7 @@ cmd_poll_once() {
exit 2
fi
local failed=0 kind id def class
local kind id def class
while IFS=$'\t' read -r kind id; do
[ -n "$kind" ] || continue
# Resolve the source definition from its top-level collection by id.
@@ -374,8 +374,55 @@
# Changed: store.sh, digest.sh, ack.sh
# (+ test-wake-store-ack.sh T13-T16,
# test-wake-digest-quarantine.sh Q12-Q16).
# 0.7.0 #958 A11 preimage.sh — durable provenance for the OPERATOR-SIDE
# preimage definition (wake-pilot finding: source-adapter.sh was
# unversioned, so the operator-owned bytes every observed_hash is
# computed FROM had thinner provenance than any hash they feed;
# attribution required an agent transcript). NEW tool + two
# pre-step integrations, ADDITIVE: (1) preimage.sh derives the
# preimage set from the RUNTIME env (the resolved
# WAKE_DETECTOR_SOURCE_CMD file, WAKE_WATCH_LIST, optional
# WAKE_PREIMAGE_EXTRA paths) and, per file, records
# {ts,path,sha256,size,mtime,prev} to an append-only ledger +
# captures the bytes CONTENT-ADDRESSED under
# $STATE_DIR/preimage/objects/<sha256> — prior bytes + change
# time are answerable from durable state alone, no transcript
# (acceptance a). A git-repo-in-operator-dir design was REJECTED:
# its who/why claim is false under shared-author fleets, and
# .gitignore is pattern-based where acceptance (b) demands
# fail-closed. (2) CREDENTIAL HARD GATE (acceptance b): byte
# capture is REFUSED — never redacted — for (i) credential-store
# PATHS (mosaic-home credentials.json, tools/_lib/credentials.json,
# credentials/**, any basename credentials.json), (ii)
# secret-SHAPED content (digest.sh's six scrub shapes reused as a
# content-deny grep), (iii) files over WAKE_PREIMAGE_MAX_BYTES
# (default 1 MiB). A refused file still gets its hash/size/mtime
# row (captured:false + refused reason) so change TIME survives
# even when content must not. (3) FIRST-CLASS CAUSE LINE
# (acceptance c): on a change (or deletion — ABSENT is a state,
# not an error), one class=actionable entry is enqueued via the
# store allocator with locators {kind:preimage, path (§2.1 hard
# locator — never quarantined), observed_hash, prev_hash,
# preimage:true, reason:preimage-definition-changed}. Both
# detector.sh cmd_poll_once and reconcile.sh cmd_reconcile run
# the check as a PRE-step, so the cause line lands at a LOWER
# observed_seq than the N per-source deltas/enumerations it
# explains — "preimage definition changed" reads first, not N
# UNACCOUNTED lines. (4) FAIL-LOUD discipline (D2/#955 class):
# an unresolvable adapter, unreadable watch-list, corrupt ledger
# (REFUSES to compare or re-baseline over corrupt history), failed
# object/ledger write, or failed enqueue is a loud non-zero —
# never read as "no change"; in both integrations the pass exits
# non-zero but source observation still proceeds (no starvation).
# First-seen is a SILENT baseline (detector first-poll idiom).
# The installer is UNCHANGED — Gate A auto-enumerates the new
# file from the filesystem; the recording site is the runtime
# tick, which is where the env-derived preimage set exists.
# store.sh/digest.sh/beacon.sh UNCHANGED; watch-list schema
# UNTOUCHED ([1,1]). Changed: preimage.sh (new), detector.sh,
# reconcile.sh (+ test-wake-preimage.sh P1-P12).
component=wake
version=0.6.15
version=0.7.0
# Watch-list schema this component consumes, and the INCLUSIVE range of
# schema_version values it supports. A wake-watch-list.json whose schema_version
@@ -446,6 +493,19 @@ schema_max=1
# seed) instead of a bare source error, and LINKS the unit into
# the user systemd search path + post-install-validates it
# resolves (#913). (W7, #913)
# preimage.sh A11 — operator-side PREIMAGE-DEFINITION provenance: derives the
# preimage set from the runtime env (resolved adapter file,
# watch-list, WAKE_PREIMAGE_EXTRA), appends
# {ts,path,sha256,size,mtime,prev} rows to an append-only
# ledger, captures bytes content-addressed under
# preimage/objects/<sha256>, and enqueues a FIRST-CLASS
# "preimage-definition-changed" actionable (path = §2.1 hard
# locator) BEFORE the deltas it explains (detector +
# reconcile pre-step). Credential HARD GATE: capture is
# REFUSED (hash/mtime still recorded) for credential-store
# paths, secret-shaped content, and oversized files —
# refusal, never redaction. Fail-loud on any infra failure;
# a corrupt ledger refuses re-baseline. (#958)
# Companion (framework subtree, not under tools/wake/): systemd/user/mosaic-wake.service
# — the long-lived detector daemon unit (per-class SLO lives in
# the daemon, NOT a systemd interval). (W7)
+446
View File
@@ -0,0 +1,446 @@
#!/usr/bin/env bash
# preimage.sh — A11 of the wake canon (#958): durable PROVENANCE for the
# OPERATOR-SIDE preimage definition.
#
# THE GAP THIS CLOSES (#958): every observed_hash in the lane store is
# sha256(adapter stdout) — the operator's source adapter (plus the watch-list
# and any operator-declared preimage files) IS the preimage definition for
# every hash the pipeline ever records. Those files live OUTSIDE the versioned
# wake component, so a byte change to them was attributable only through agent
# transcripts: provenance thinner than any hash it feeds. A changed preimage
# also re-baselines EVERY source at once, which previously surfaced only as N
# per-source deltas / UNACCOUNTED lines with no first-class cause.
#
# WHAT THIS TOOL DOES:
# - Derives the PREIMAGE SET generically (no operator paths baked in):
# * the resolved file behind WAKE_DETECTOR_SOURCE_CMD (first word),
# * the WAKE_WATCH_LIST file,
# * operator-declared extras via WAKE_PREIMAGE_EXTRA (colon-separated —
# e.g. detector.env, sink/feed scripts).
# - Records each file's (sha256, size, mtime, ts) as an APPEND-ONLY ledger
# row in <state>/preimage/preimage-ledger.jsonl and stores the full bytes
# CONTENT-ADDRESSED at <state>/preimage/objects/<sha256> — so a change is
# attributable (prior bytes + change time) from durable state alone, with
# no transcript required. Acceptance (a) of #958.
# - On a CHANGE (not first-seen), `check --enqueue` enqueues ONE first-class
# class=actionable entry per changed file via the store's single allocator
# (#908), carrying a §2.1 hard locator (`path`), BEFORE the per-source
# deltas it explains are observed — so the digest shows the cause line,
# not just N re-baselined sources. Acceptance (c).
#
# CREDENTIAL HARD GATE — acceptance (b): the mechanism must be UNABLE to
# capture credential material even by operator error. Enforced two ways, both
# fail-closed toward NOT capturing bytes (the hash/size/mtime row is still
# written — change-time attribution survives, content capture does not;
# a sha256 discloses nothing about the bytes):
# 1. PATH deny: the mosaic credential store locations
# (<mosaic-home>/credentials.json, <mosaic-home>/tools/_lib/
# credentials.json, anything under <mosaic-home>/credentials/, and any
# file literally named credentials.json) are refused by resolved realpath
# before a single byte is read into the object store.
# 2. CONTENT deny: a candidate whose bytes match the well-known secret-token
# shapes (same conservative set digest.sh redacts: PAT/Slack/AKIA/JWT/PEM)
# is refused — refusal, not redaction: a redacted preimage would be a
# false witness, and secret bytes must never land in the object store.
# Plus a size cap (WAKE_PREIMAGE_MAX_BYTES, default 1 MiB) so a stray extra
# pointing at a large binary cannot turn provenance into data hoovering.
#
# FAIL-LOUD DISCIPLINE (the D2/#955 class, both layers): an infrastructure
# failure of THIS tool (unresolvable adapter, unreadable/corrupt ledger,
# failed durable write, failed enqueue) exits NON-ZERO and is never read as
# "no change". A corrupt ledger REFUSES to compare (and to re-baseline):
# silently restarting history would erase the very attribution this exists
# to provide.
#
# Operator-agnostic (framework firewall): all state via XDG/env; the deny
# list names only framework-defined credential locations, no operator hosts/
# names/secrets. This tool never prints file CONTENT to any stream.
set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=./_wake-common.sh disable=SC1091
. "$SCRIPT_DIR/_wake-common.sh"
STATE_DIR="$(wake_state_dir)"
STORE_SH="$SCRIPT_DIR/store.sh"
PRE_DIR="$STATE_DIR/preimage"
LEDGER="$PRE_DIR/preimage-ledger.jsonl"
OBJECTS="$PRE_DIR/objects"
_need_jq() {
command -v jq >/dev/null 2>&1 || {
echo "preimage.sh: jq is required" >&2
exit 3
}
}
# _hash_stdin — sha256 of stdin, first field only (portable, same fallback
# chain as detector.sh).
_hash_stdin() {
if command -v sha256sum >/dev/null 2>&1; then
sha256sum | awk '{print $1}'
elif command -v shasum >/dev/null 2>&1; then
shasum -a 256 | awk '{print $1}'
elif command -v openssl >/dev/null 2>&1; then
openssl dgst -sha256 | awk '{print $NF}'
else
echo "preimage.sh: no sha256 tool (sha256sum/shasum/openssl) available" >&2
exit 3
fi
}
usage() {
cat >&2 <<'EOF'
Usage: preimage.sh <command>
Commands:
check [--enqueue] Compare every preimage-set file against the ledger head.
A changed file gets a new ledger row + captured bytes;
with --enqueue each change (not first-seen) also enqueues
ONE first-class class=actionable store entry (hard
locator: path). First-seen files baseline SILENTLY (row,
no enqueue — same idiom as the detector's first-seen).
Exit 0 whether or not changes were found; non-zero ONLY
on infrastructure failure (never read as "no change").
record Alias of `check` without enqueue (baseline/refresh).
set Print the derived preimage set, one resolved path per
line (diagnostic).
status Print the ledger head row for every tracked path.
Environment:
WAKE_DETECTOR_SOURCE_CMD Adapter command; its resolved file joins the set.
WAKE_WATCH_LIST Watch-list path; joins the set when set.
WAKE_PREIMAGE_EXTRA Colon-separated additional operator preimage files
(e.g. detector.env, sink scripts). A listed path
that does not exist is tracked as ABSENT (deletion
of a preimage file is a change, not an error).
WAKE_PREIMAGE_MAX_BYTES Byte-capture cap (default 1048576). Larger files:
hash/size/mtime recorded, bytes refused.
WAKE_STATE_HOME/WAKE_AGENT store namespace (see store.sh).
EOF
}
# --- preimage-set derivation (generic; no operator paths baked in) ----------
# _resolve_cmd_file CMD — resolve the FIRST WORD of an adapter command string
# to a real file. Non-zero (loud) if it cannot be resolved: an adapter the
# detector will invoke but provenance cannot see is an infrastructure failure,
# not a smaller set.
_resolve_cmd_file() {
local cmd="$1" word path
# shellcheck disable=SC2086
set -- $cmd
word="${1:-}"
[ -n "$word" ] || return 1
if [ -f "$word" ]; then
path="$word"
else
path="$(command -v -- "$word" 2>/dev/null)" || return 1
[ -f "$path" ] || return 1
fi
# realpath so the ledger keys on the actual file, not a symlink alias.
if command -v realpath >/dev/null 2>&1; then
realpath -- "$path" 2>/dev/null || printf '%s' "$path"
else
printf '%s' "$path"
fi
}
# _preimage_set — print the derived set, one path per line. Paths from
# WAKE_PREIMAGE_EXTRA are printed EVEN IF ABSENT (deletion is a tracked
# state); the adapter and watch-list must resolve (loud failure otherwise —
# see _resolve_cmd_file rationale).
_preimage_set() {
local failed=0
if [ -n "${WAKE_DETECTOR_SOURCE_CMD:-}" ]; then
local af
if af="$(_resolve_cmd_file "$WAKE_DETECTOR_SOURCE_CMD")"; then
printf '%s\n' "$af"
else
echo "preimage.sh: FAIL LOUD — WAKE_DETECTOR_SOURCE_CMD ('$WAKE_DETECTOR_SOURCE_CMD') does not resolve to a file; the preimage definition cannot be observed (NOT treated as 'no change')." >&2
failed=1
fi
fi
if [ -n "${WAKE_WATCH_LIST:-}" ]; then
if [ -f "$WAKE_WATCH_LIST" ]; then
if command -v realpath >/dev/null 2>&1; then
realpath -- "$WAKE_WATCH_LIST" 2>/dev/null || printf '%s' "$WAKE_WATCH_LIST"
else
printf '%s' "$WAKE_WATCH_LIST"
fi
printf '\n'
else
echo "preimage.sh: FAIL LOUD — WAKE_WATCH_LIST ('$WAKE_WATCH_LIST') is not a file; the preimage definition cannot be observed." >&2
failed=1
fi
fi
if [ -n "${WAKE_PREIMAGE_EXTRA:-}" ]; then
local IFS=':' p
for p in $WAKE_PREIMAGE_EXTRA; do
[ -n "$p" ] || continue
# Extras are tracked even when absent (ABSENT is a state). Resolve the
# realpath only when the file exists; otherwise track the literal path.
if [ -e "$p" ] && command -v realpath >/dev/null 2>&1; then
realpath -- "$p" 2>/dev/null || printf '%s' "$p"
printf '\n'
else
printf '%s\n' "$p"
fi
done
fi
return "$failed"
}
# --- credential hard gate (acceptance (b)) ----------------------------------
# _deny_path PATH — 0 (deny) iff PATH is a known credential-store location.
# Framework-defined locations only (firewall): the mosaic credential store,
# the tools/_lib credential file the framework-manifest itself carves out of
# tools/**, the credentials/ operator subtree, and any file literally named
# credentials.json.
_deny_path() {
local p="$1" home="${MOSAIC_HOME:-$HOME/.config/mosaic}"
case "$p" in
"$home/credentials.json") return 0 ;;
"$home/tools/_lib/credentials.json") return 0 ;;
"$home/credentials/"*) return 0 ;;
esac
case "$(basename -- "$p")" in
credentials.json) return 0 ;;
esac
return 1
}
# _deny_content FILE — 0 (deny) iff FILE's bytes match a well-known secret-
# token shape. Same conservative shape set digest.sh redacts (PAT / Slack /
# AKIA / JWT / PEM) — conservative on purpose: a 40-hex git sha never matches.
_deny_content() {
LC_ALL=C grep -Eq \
-e 'gh[pousr]_[A-Za-z0-9]{16,}' \
-e 'github_pat_[A-Za-z0-9_]{16,}' \
-e 'xox[baprs]-[A-Za-z0-9-]{10,}' \
-e 'AKIA[0-9A-Z]{16}' \
-e 'eyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}' \
-e '-----BEGIN[A-Z ]*PRIVATE KEY-----' \
-- "$1" 2>/dev/null
}
# --- ledger primitives ------------------------------------------------------
# _ledger_head PATH — print the LAST ledger row for PATH (empty if none).
# Non-zero (loud) if the ledger exists but is unparseable: a corrupt ledger
# must REFUSE to compare, never silently re-baseline (that would erase the
# attribution history this tool exists to keep).
_ledger_head() {
local path="$1"
[ -f "$LEDGER" ] || return 0
if [ -s "$LEDGER" ] && ! jq -cn 'inputs' <"$LEDGER" >/dev/null 2>&1; then
echo "preimage.sh: FAIL LOUD — preimage ledger $LEDGER is unparseable; REFUSING to compare or re-baseline over corrupt history. Investigate/restore the ledger." >&2
return 1
fi
jq -c --arg p "$path" 'select(.path == $p)' "$LEDGER" 2>/dev/null | tail -n 1
}
# _ledger_append ROW_JSON — append one row atomically (read + append + rename;
# rows are small and the writer is serialized by the preimage lock).
_ledger_append() {
local row="$1"
{
[ -f "$LEDGER" ] && cat "$LEDGER"
printf '%s\n' "$row"
} | _atomic_write "$LEDGER"
}
# --- the check itself -------------------------------------------------------
# _check_one PATH ENQUEUE — compare PATH to its ledger head; on change record
# (+bytes unless denied) and optionally enqueue. Prints nothing on no-change.
# Returns: 0 ok (changed or not), 1 infrastructure failure.
_check_one() {
local path="$1" enqueue="$2"
local cur_sha size mtime denied="" refused=""
if [ -f "$path" ]; then
cur_sha="$(_hash_stdin <"$path")" || return 1
[ -n "$cur_sha" ] || {
echo "preimage.sh: FAIL LOUD — could not hash $path" >&2
return 1
}
size="$(wc -c <"$path" | tr -d '[:space:]')"
# Portable mtime (GNU stat -c / BSD stat -f).
mtime="$(stat -c %Y -- "$path" 2>/dev/null || stat -f %m -- "$path" 2>/dev/null || echo 0)"
local cap="${WAKE_PREIMAGE_MAX_BYTES:-1048576}"
case "$cap" in '' | *[!0-9]*) cap=1048576 ;; esac
# Deny order matters: path first (a known credential store is refused
# without a content probe — the hash read above is deliberate: a sha256
# discloses nothing about the bytes), size cap second (never content-grep
# an oversized file), content shape last.
if _deny_path "$path"; then
denied="credential-store path (deny list)"
elif [ "$size" -gt "$cap" ]; then
denied="exceeds WAKE_PREIMAGE_MAX_BYTES ($size > $cap)"
elif _deny_content "$path"; then
denied="secret-shaped content"
fi
else
# ABSENT is a state (deletion of a preimage file is a change to record).
cur_sha="ABSENT"
size=0
mtime=0
fi
local head prev_sha=""
head="$(_ledger_head "$path")" || return 1
[ -n "$head" ] && prev_sha="$(jq -r '.sha256 // ""' <<<"$head")"
if [ "$cur_sha" = "$prev_sha" ]; then
return 0 # unchanged — no row, no enqueue (idempotent)
fi
# --- capture bytes (content-addressed) unless the hard gate refuses -------
local captured=true
if [ "$cur_sha" != "ABSENT" ]; then
if [ -n "$denied" ]; then
captured=false
refused="$denied"
echo "preimage.sh: REFUSED byte capture for $path ($denied) — hash/size/mtime recorded, content NOT stored (acceptance (b), #958)." >&2
else
mkdir -p "$OBJECTS" || return 1
if [ ! -f "$OBJECTS/$cur_sha" ]; then
if ! _atomic_write "$OBJECTS/$cur_sha" <"$path"; then
echo "preimage.sh: FAIL LOUD — durable object write failed for $path ($OBJECTS/$cur_sha); NO ledger row recorded (a row whose bytes were never durably kept would be a false witness)." >&2
return 1
fi
fi
fi
fi
local ts row
ts="$(date +%s)"
row="$(jq -cn \
--arg path "$path" \
--arg sha "$cur_sha" \
--arg prev "$prev_sha" \
--argjson size "$size" \
--argjson mtime "$mtime" \
--argjson ts "$ts" \
--argjson captured "$captured" \
--arg refused "$refused" \
'{ts:$ts, path:$path, sha256:$sha, size:$size, mtime:$mtime, captured:$captured}
+ (if $prev != "" then {prev:$prev} else {} end)
+ (if $refused != "" then {refused:$refused} else {} end)')" || return 1
if ! _ledger_append "$row"; then
echo "preimage.sh: FAIL LOUD — durable ledger append failed ($LEDGER)" >&2
return 1
fi
if [ -z "$head" ]; then
# First-seen: baseline SILENTLY (row only, no enqueue) — the same idiom as
# the detector's first-seen source baseline.
return 0
fi
echo "preimage.sh: preimage definition CHANGED — $path ($prev_sha -> $cur_sha); prior bytes: $OBJECTS/$prev_sha" >&2
if [ "$enqueue" = "1" ]; then
# First-class cause line (acceptance (c)): ONE class=actionable entry per
# changed preimage file, allocated by the store's single allocator (#908).
# `path` is a §2.1 hard locator, so the digest renders it — never
# quarantined. Callers run this BEFORE polling sources, so this seq is
# LOWER than the per-source deltas the change explains.
local locators
locators="$(jq -cn \
--arg path "$path" \
--arg sha "$cur_sha" \
--arg prev "$prev_sha" \
'{kind:"preimage", id:$path, path:$path, observed_hash:$sha, prev_hash:$prev,
preimage:true, reason:"preimage-definition-changed"}')"
if ! "$STORE_SH" enqueue --class actionable --locators "$locators" --emit-ts "$(date +%s)" >/dev/null; then
echo "preimage.sh: FAIL LOUD — store enqueue of the preimage-change entry FAILED for $path (the change IS recorded in the ledger; the first-class wake line is NOT — treat as infrastructure failure)." >&2
return 1
fi
fi
return 0
}
cmd_check() {
local enqueue=0
while [ $# -gt 0 ]; do
case "$1" in
--enqueue)
enqueue=1
shift
;;
*)
echo "preimage.sh check: unknown option '$1'" >&2
exit 2
;;
esac
done
_need_jq
local set_list
if ! set_list="$(_preimage_set)"; then
exit 1
fi
if [ -z "$set_list" ]; then
# Nothing declared (no adapter/watch-list/extras in env) — an empty set is
# a no-op, not an error: standalone invocations outside a wake runtime
# must not fail loud for lacking one.
return 0
fi
mkdir -p "$PRE_DIR" || exit 1
# Serialize concurrent checks (detector tick vs reconcile vs manual) — the
# ledger append is read+rewrite, so two writers must not interleave.
if ! _wake_lock_acquire "$PRE_DIR/preimage.lock"; then
echo "preimage.sh: FAIL LOUD — cannot acquire preimage lock" >&2
exit 1
fi
local failed=0 p
while IFS= read -r p; do
[ -n "$p" ] || continue
_check_one "$p" "$enqueue" || failed=1
done <<EOF
$set_list
EOF
_wake_lock_release
[ "$failed" -eq 0 ] || exit 1
}
cmd_status() {
_need_jq
[ -f "$LEDGER" ] || {
echo "preimage.sh: no ledger yet ($LEDGER)" >&2
return 0
}
# Head row per path (last wins).
jq -cs 'group_by(.path) | map(last) | .[]' "$LEDGER"
}
cmd_set() {
_preimage_set || exit 1
}
main() {
[ $# -ge 1 ] || {
usage
exit 2
}
local cmd="$1"
shift
case "$cmd" in
check) cmd_check "$@" ;;
record) cmd_check ;;
set) cmd_set "$@" ;;
status) cmd_status "$@" ;;
-h | --help | help) usage ;;
*)
echo "preimage.sh: unknown command '$cmd'" >&2
usage
exit 2
;;
esac
}
main "$@"
@@ -440,6 +440,17 @@ cmd_reconcile() {
: "$allow_enumerate"
local pairs kind id failed=0 unaccounted=0 enumerated=0
# #958 preimage provenance: same pre-step as the detector's poll tick, for
# the path where the preimage changed while the detector was down — without
# it, a changed adapter surfaces here only as N UNACCOUNTED enumerations
# with no first-class cause line. Run BEFORE observing any source so the
# cause entry's observed_seq precedes the enumerations it explains. Loud
# infrastructure failure marks the reconcile failed but does not stop it.
if ! "$SCRIPT_DIR/preimage.sh" check --enqueue; then
echo "reconcile.sh: FAIL LOUD — preimage provenance check failed (see preimage.sh above); reconcile continues but exits non-zero." >&2
failed=1
fi
pairs="$(printf '%s' "$json" | jq -r '[ .watches[].sources[] | "\(.kind)\t\(.id)" ] | unique | .[]')"
if [ -z "$pairs" ]; then
echo "reconcile.sh: watch-list declares no sources under watches[].sources[]" >&2
+348
View File
@@ -0,0 +1,348 @@
#!/usr/bin/env bash
# test-wake-preimage.sh — RED-FIRST invariant harness for A11 (#958): durable
# provenance for the OPERATOR-SIDE preimage definition (preimage.sh).
#
# Each test asserts ONE #958 invariant and is designed to go RED if it
# regresses:
# P1 first-seen -> SILENT baseline (ledger rows, NO enqueue) (detector idiom)
# P2 change -> attributable from durable state alone: new ledger row with
# prev, PRIOR BYTES retrievable content-addressed, first-class actionable
# enqueued with a §2.1 hard locator (path) (acceptance a+c)
# P3 detector ordering: the preimage cause line is enqueued at a LOWER
# observed_seq than the per-source delta it explains (acceptance c)
# P4 credential-store PATH is refused — bytes NEVER captured, even when the
# operator lists it by error (hash/mtime row still written) (acceptance b)
# P5 secret-SHAPED content is refused — refusal, not redaction (acceptance b)
# P6 deletion of a preimage file is a tracked CHANGE (ABSENT), not an error
# P7 no-change is idempotent (no row growth, no enqueue)
# P8 unresolvable adapter command -> FAIL LOUD, never "no change" (D2/#955 class)
# P9 corrupt ledger -> REFUSE to compare/re-baseline (loud, no append)
# P10 oversized file -> bytes refused, hash still recorded (no data hoovering)
# P11 reconcile pre-step: a preimage change surfaces as a first-class line
# BEFORE the UNACCOUNTED enumerations it explains (acceptance c)
# P12 detector: preimage infra failure is LOUD (pass exits non-zero) but does
# NOT starve source observation
#
# Uses FAKE/STUB sources only (no live network). Isolated per test.
# shellcheck disable=SC2030,SC2031
set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PRE="$SCRIPT_DIR/preimage.sh"
STORE="$SCRIPT_DIR/store.sh"
DET="$SCRIPT_DIR/detector.sh"
RECON="$SCRIPT_DIR/reconcile.sh"
command -v jq >/dev/null 2>&1 || {
echo "SKIP: jq not available" >&2
exit 0
}
TMP_ROOT="$(mktemp -d)"
trap 'rm -rf "$TMP_ROOT"' EXIT
FAILFILE="$TMP_ROOT/failures"
: >"$FAILFILE"
pass=0
fail_msg() {
echo " FAIL: $*" >&2
echo "x" >>"$FAILFILE"
}
ok() { pass=$((pass + 1)); }
fresh_state() {
local d="$TMP_ROOT/$1"
rm -rf "$d"
mkdir -p "$d"
printf '%s' "$d"
}
depth() { "$STORE" cursors | sed -n 's/pending_depth=//p'; }
state_dir() { printf '%s/default' "$WAKE_STATE_HOME"; }
ledger_rows() {
local f
f="$(state_dir)/preimage/preimage-ledger.jsonl"
[ -f "$f" ] && wc -l <"$f" | tr -d '[:space:]' || echo 0
}
# make_stub DIR — a source adapter reading $DIR/<kind>_<id> (same shape as the
# reconcile harness stub).
make_stub() {
local dir="$1"
cat >"$dir/adapter.sh" <<EOF
#!/usr/bin/env bash
set -u
kind="\$1"; id="\$2"; base="$dir/\${kind}_\${id}"; rc=0
[ -f "\$base.rc" ] && rc="\$(cat "\$base.rc")"
[ -f "\$base" ] && cat "\$base"
exit "\$rc"
EOF
chmod +x "$dir/adapter.sh"
}
make_wl() { # make_wl FILE — one repo source r1
cat >"$1" <<'EOF'
{ "schema_version": 1,
"repos": [ { "id": "r1" } ],
"watches": [ { "lane": "L", "sources": [ { "kind": "repo", "id": "r1" } ] } ] }
EOF
}
echo "== P1: first-seen -> SILENT baseline (rows, no enqueue) =="
(
WAKE_STATE_HOME="$(fresh_state p1)"
export WAKE_STATE_HOME
unset WAKE_AGENT
fx="$TMP_ROOT/p1fx"; mkdir -p "$fx"
make_stub "$fx"; make_wl "$fx/wl.json"
printf 'extra v1\n' >"$fx/extra.env"
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json" WAKE_PREIMAGE_EXTRA="$fx/extra.env"
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
[ "$rc" -eq 0 ] || fail_msg "P1: baseline check must exit 0 (rc=$rc) [$out]"
[ "$(ledger_rows)" -eq 3 ] || fail_msg "P1: expected 3 baseline rows (adapter, watch-list, extra), got $(ledger_rows)"
[ "$(depth)" = "0" ] || fail_msg "P1: first-seen must NOT enqueue (depth=$(depth))"
) && ok
echo "== P2: change -> prior bytes + first-class hard-locator enqueue =="
(
WAKE_STATE_HOME="$(fresh_state p2)"
export WAKE_STATE_HOME
unset WAKE_AGENT
fx="$TMP_ROOT/p2fx"; mkdir -p "$fx"
make_stub "$fx"; make_wl "$fx/wl.json"
printf 'adapter preimage v1\n' >"$fx/extra.sh"
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json" WAKE_PREIMAGE_EXTRA="$fx/extra.sh"
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P2: baseline failed"
old_sha="$(sha256sum "$fx/extra.sh" | awk '{print $1}')"
printf 'adapter preimage v2 CHANGED\n' >"$fx/extra.sh"
new_sha="$(sha256sum "$fx/extra.sh" | awk '{print $1}')"
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
[ "$rc" -eq 0 ] || fail_msg "P2: change check must exit 0 (rc=$rc) [$out]"
sd="$(state_dir)"
row="$(jq -c --arg p "$(realpath "$fx/extra.sh")" 'select(.path == $p)' "$sd/preimage/preimage-ledger.jsonl" | tail -n1)"
[ "$(jq -r '.prev // ""' <<<"$row")" = "$old_sha" ] || fail_msg "P2: new row must carry prev=<old sha> [$row]"
[ "$(jq -r '.sha256' <<<"$row")" = "$new_sha" ] || fail_msg "P2: new row sha mismatch [$row]"
# Acceptance (a): the PRIOR BYTES are retrievable from durable state alone.
[ -f "$sd/preimage/objects/$old_sha" ] || fail_msg "P2: prior bytes object missing"
[ "$(cat "$sd/preimage/objects/$old_sha")" = "adapter preimage v1" ] || fail_msg "P2: prior bytes not byte-exact"
[ -f "$sd/preimage/objects/$new_sha" ] || fail_msg "P2: current bytes object missing"
# Acceptance (c): ONE first-class actionable with a §2.1 hard locator (path).
[ "$(depth)" = "1" ] || fail_msg "P2: exactly one enqueue expected (depth=$(depth))"
ent="$(tail -n1 "$sd/pending.jsonl")"
[ "$(jq -r '.class' <<<"$ent")" = "actionable" ] || fail_msg "P2: entry class must be actionable [$ent]"
[ "$(jq -r '.locators.kind' <<<"$ent")" = "preimage" ] || fail_msg "P2: locator kind must be preimage [$ent]"
[ -n "$(jq -r '.locators.path // ""' <<<"$ent")" ] || fail_msg "P2: locator must carry path (hard locator) [$ent]"
[ "$(jq -r '.locators.observed_hash' <<<"$ent")" = "$new_sha" ] || fail_msg "P2: locator observed_hash mismatch [$ent]"
[ "$(jq -r '.locators.prev_hash' <<<"$ent")" = "$old_sha" ] || fail_msg "P2: locator prev_hash mismatch [$ent]"
) && ok
echo "== P3: detector orders the cause line BEFORE the delta it explains =="
(
WAKE_STATE_HOME="$(fresh_state p3)"
export WAKE_STATE_HOME
unset WAKE_AGENT
fx="$TMP_ROOT/p3fx"; mkdir -p "$fx"
make_stub "$fx"; make_wl "$fx/wl.json"
printf 'r1 state v1\n' >"$fx/repo_r1"
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json"
unset WAKE_PREIMAGE_EXTRA
"$DET" poll-once >/dev/null 2>&1 || fail_msg "P3: baseline poll failed"
[ "$(depth)" = "0" ] || fail_msg "P3: baseline poll must enqueue nothing (depth=$(depth))"
# Change the ADAPTER (preimage) and the source state in the same window.
printf '# comment: adapter changed\n' >>"$fx/adapter.sh"
printf 'r1 state v2\n' >"$fx/repo_r1"
"$DET" poll-once >/dev/null 2>&1 || fail_msg "P3: second poll failed"
sd="$(state_dir)"
pre_seq="$(jq -r 'select(.locators.kind == "preimage") | .observed_seq' "$sd/pending.jsonl" | head -n1)"
src_seq="$(jq -r 'select(.locators.kind == "repo") | .observed_seq' "$sd/pending.jsonl" | head -n1)"
[ -n "$pre_seq" ] || fail_msg "P3: no preimage cause entry enqueued"
[ -n "$src_seq" ] || fail_msg "P3: no source delta entry enqueued"
if [ -n "$pre_seq" ] && [ -n "$src_seq" ]; then
[ "$pre_seq" -lt "$src_seq" ] || fail_msg "P3: cause line must precede the delta (preimage seq=$pre_seq, source seq=$src_seq)"
fi
) && ok
echo "== P4: credential-store PATH refused — bytes never captured (acceptance b) =="
(
WAKE_STATE_HOME="$(fresh_state p4)"
export WAKE_STATE_HOME
unset WAKE_AGENT
fx="$TMP_ROOT/p4fx"; mkdir -p "$fx"
export MOSAIC_HOME="$fx/mosaic-home"
mkdir -p "$MOSAIC_HOME"
secret='cred-value-P4-do-not-capture'
printf '{"svc":{"token":"%s"}}\n' "$secret" >"$MOSAIC_HOME/credentials.json"
# Operator error: the credential store listed as a preimage extra.
export WAKE_PREIMAGE_EXTRA="$MOSAIC_HOME/credentials.json"
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
[ "$rc" -eq 0 ] || fail_msg "P4: refusal is not an infra failure (rc=$rc) [$out]"
sd="$(state_dir)"
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
[ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P4: row must record captured=false [$row]"
jq -r '.refused // ""' <<<"$row" | grep -qi 'path' || fail_msg "P4: refusal must name the path deny [$row]"
# THE invariant: the secret bytes exist NOWHERE under the wake state dir.
if grep -rq "$secret" "$sd" 2>/dev/null; then
fail_msg "P4: credential bytes leaked into the wake state dir"
fi
) && ok
echo "== P5: secret-SHAPED content refused (refusal, not redaction) =="
(
WAKE_STATE_HOME="$(fresh_state p5)"
export WAKE_STATE_HOME
unset WAKE_AGENT
fx="$TMP_ROOT/p5fx"; mkdir -p "$fx"
tok='ghp_abcdefghijklmnop0123456789ABCDEF'
printf 'export MY_TOKEN=%s\n' "$tok" >"$fx/leaky.env"
export WAKE_PREIMAGE_EXTRA="$fx/leaky.env"
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
[ "$rc" -eq 0 ] || fail_msg "P5: refusal is not an infra failure (rc=$rc) [$out]"
sd="$(state_dir)"
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
[ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P5: row must record captured=false [$row]"
jq -r '.refused // ""' <<<"$row" | grep -qi 'secret' || fail_msg "P5: refusal must name secret-shaped content [$row]"
if grep -rq "$tok" "$sd" 2>/dev/null; then
fail_msg "P5: secret-shaped bytes leaked into the wake state dir"
fi
) && ok
echo "== P6: deletion is a tracked CHANGE (ABSENT), not an error =="
(
WAKE_STATE_HOME="$(fresh_state p6)"
export WAKE_STATE_HOME
unset WAKE_AGENT
fx="$TMP_ROOT/p6fx"; mkdir -p "$fx"
printf 'to be deleted\n' >"$fx/gone.env"
export WAKE_PREIMAGE_EXTRA="$fx/gone.env"
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P6: baseline failed"
rm -f "$fx/gone.env"
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
[ "$rc" -eq 0 ] || fail_msg "P6: deletion check must exit 0 (rc=$rc) [$out]"
sd="$(state_dir)"
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
[ "$(jq -r '.sha256' <<<"$row")" = "ABSENT" ] || fail_msg "P6: deletion must record sha256=ABSENT [$row]"
[ "$(depth)" = "1" ] || fail_msg "P6: deletion is a change and must enqueue (depth=$(depth))"
) && ok
echo "== P7: no-change is idempotent =="
(
WAKE_STATE_HOME="$(fresh_state p7)"
export WAKE_STATE_HOME
unset WAKE_AGENT
fx="$TMP_ROOT/p7fx"; mkdir -p "$fx"
printf 'stable\n' >"$fx/stable.env"
export WAKE_PREIMAGE_EXTRA="$fx/stable.env"
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P7: baseline failed"
r1="$(ledger_rows)"
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P7: second check failed"
[ "$(ledger_rows)" = "$r1" ] || fail_msg "P7: no-change must not append rows ($r1 -> $(ledger_rows))"
[ "$(depth)" = "0" ] || fail_msg "P7: no-change must not enqueue (depth=$(depth))"
) && ok
echo "== P8: unresolvable adapter -> FAIL LOUD, never 'no change' (D2 class) =="
(
WAKE_STATE_HOME="$(fresh_state p8)"
export WAKE_STATE_HOME
unset WAKE_AGENT
export WAKE_DETECTOR_SOURCE_CMD="$TMP_ROOT/does-not-exist-adapter"
unset WAKE_WATCH_LIST WAKE_PREIMAGE_EXTRA MOSAIC_HOME
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
[ "$rc" -ne 0 ] || fail_msg "P8: an unobservable preimage definition must FAIL LOUD (rc=0)"
echo "$out" | grep -qi 'does not resolve' || fail_msg "P8: the failure must name the unresolvable adapter [$out]"
) && ok
echo "== P9: corrupt ledger -> REFUSE to compare/re-baseline =="
(
WAKE_STATE_HOME="$(fresh_state p9)"
export WAKE_STATE_HOME
unset WAKE_AGENT
fx="$TMP_ROOT/p9fx"; mkdir -p "$fx"
printf 'v1\n' >"$fx/f.env"
export WAKE_PREIMAGE_EXTRA="$fx/f.env"
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P9: baseline failed"
sd="$(state_dir)"
printf 'NOT-JSON-GARBAGE{{{\n' >>"$sd/preimage/preimage-ledger.jsonl"
r1="$(wc -l <"$sd/preimage/preimage-ledger.jsonl")"
printf 'v2 changed\n' >"$fx/f.env"
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
[ "$rc" -ne 0 ] || fail_msg "P9: a corrupt ledger must FAIL LOUD (rc=0)"
echo "$out" | grep -qi 'unparseable' || fail_msg "P9: the failure must name the corrupt ledger [$out]"
[ "$(wc -l <"$sd/preimage/preimage-ledger.jsonl")" = "$r1" ] || fail_msg "P9: nothing may be appended over corrupt history"
) && ok
echo "== P10: oversized file -> bytes refused, hash still recorded =="
(
WAKE_STATE_HOME="$(fresh_state p10)"
export WAKE_STATE_HOME
unset WAKE_AGENT
fx="$TMP_ROOT/p10fx"; mkdir -p "$fx"
head -c 200 /dev/zero | tr '\0' 'A' >"$fx/big.bin"
export WAKE_PREIMAGE_EXTRA="$fx/big.bin" WAKE_PREIMAGE_MAX_BYTES=64
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
[ "$rc" -eq 0 ] || fail_msg "P10: size refusal is not an infra failure (rc=$rc) [$out]"
sd="$(state_dir)"
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
[ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P10: row must record captured=false [$row]"
jq -r '.refused // ""' <<<"$row" | grep -qi 'MAX_BYTES' || fail_msg "P10: refusal must name the size cap [$row]"
sha="$(jq -r '.sha256' <<<"$row")"
[ "$sha" = "$(sha256sum "$fx/big.bin" | awk '{print $1}')" ] || fail_msg "P10: hash must still be recorded [$row]"
[ ! -f "$sd/preimage/objects/$sha" ] || fail_msg "P10: oversized bytes must NOT be stored"
) && ok
echo "== P11: reconcile surfaces the cause line before its enumerations =="
(
WAKE_STATE_HOME="$(fresh_state p11)"
export WAKE_STATE_HOME
unset WAKE_AGENT
fx="$TMP_ROOT/p11fx"; mkdir -p "$fx"
make_stub "$fx"; make_wl "$fx/wl.json"
printf 'r1 state\n' >"$fx/repo_r1"
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json"
unset WAKE_PREIMAGE_EXTRA MOSAIC_HOME
# Baseline the preimage set, then change the adapter while "the detector is
# down" — the reconcile path must still produce the first-class cause line.
"$PRE" check >/dev/null 2>&1 || fail_msg "P11: preimage baseline failed"
printf '# adapter changed while detector down\n' >>"$fx/adapter.sh"
"$RECON" reconcile >/dev/null 2>&1 # rc 1 expected (unaccounted enumerated)
sd="$(state_dir)"
pre_seq="$(jq -r 'select(.locators.kind == "preimage") | .observed_seq' "$sd/pending.jsonl" | head -n1)"
enum_seq="$(jq -r 'select(.locators.reconciled == true) | .observed_seq' "$sd/pending.jsonl" | head -n1)"
[ -n "$pre_seq" ] || fail_msg "P11: reconcile must enqueue the preimage cause line"
[ -n "$enum_seq" ] || fail_msg "P11: reconcile must still enumerate the unaccounted source"
if [ -n "$pre_seq" ] && [ -n "$enum_seq" ]; then
[ "$pre_seq" -lt "$enum_seq" ] || fail_msg "P11: cause line must precede the enumeration (preimage seq=$pre_seq, enum seq=$enum_seq)"
fi
) && ok
echo "== P12: detector — preimage infra failure is LOUD but does not starve observation =="
(
WAKE_STATE_HOME="$(fresh_state p12)"
export WAKE_STATE_HOME
unset WAKE_AGENT
fx="$TMP_ROOT/p12fx"; mkdir -p "$fx"
make_stub "$fx"; make_wl "$fx/wl.json"
printf 'r1 state\n' >"$fx/repo_r1"
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json"
unset WAKE_PREIMAGE_EXTRA MOSAIC_HOME
# Corrupt the ledger BEFORE the first poll: the preimage check fails loud,
# but the poll must still observe + baseline the source.
sd="$(state_dir)"
mkdir -p "$sd/preimage"
printf 'NOT-JSON-GARBAGE{{{\n' >"$sd/preimage/preimage-ledger.jsonl"
out="$("$DET" poll-once 2>&1)"; rc=$?
[ "$rc" -ne 0 ] || fail_msg "P12: the pass must exit non-zero on preimage infra failure"
echo "$out" | grep -qi 'preimage' || fail_msg "P12: the failure must name the preimage check [$out]"
n="$(find "$sd/detector" -name 'watch-*.hash' 2>/dev/null | wc -l | tr -d '[:space:]')"
[ "$n" -ge 1 ] || fail_msg "P12: source observation must still proceed (no watch hash baselined)"
) && ok
echo
total=$((pass + $(wc -l <"$FAILFILE")))
echo "== test-wake-preimage: $pass/$total passed =="
[ -s "$FAILFILE" ] && exit 1
exit 0
+1 -1
View File
@@ -25,7 +25,7 @@
"lint": "eslint src",
"typecheck": "tsc --noEmit",
"test": "vitest run --passWithNoTests && pnpm run test:framework-shell",
"test:framework-shell": "python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-install.sh"
"test:framework-shell": "python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-preimage.sh && bash framework/tools/wake/test-wake-install.sh"
},
"dependencies": {
"@mosaicstack/brain": "workspace:*",