Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
  jarvisandClaude Fable 5
|
89e26ff7c2 | ||
|
jarvisandClaude Fable 5
|
caf40afc01 | ||
|
jarvisandClaude Fable 5
|
6cc093afdb |
@@ -39,6 +39,7 @@ overwritten on upgrade. (Layer model: `constitution/LAYER-MODEL.md`.)
|
||||
| TypeScript strict typing | `guides/TYPESCRIPT.md` |
|
||||
| QA / test strategy | `guides/QA-TESTING.md` |
|
||||
| Documentation (any code/API/auth/infra change) | `guides/DOCUMENTATION.md` |
|
||||
| Writing style (docs, comms, any prose) | `guides/WRITING-STYLE.md` |
|
||||
| Secrets / vault usage | `guides/VAULT-SECRETS.md` |
|
||||
| Tool/credential reference (service CLIs, wrappers) | `guides/TOOLS-REFERENCE.md` |
|
||||
| Memory protocol (OpenBrain capture/recall) | `guides/MEMORY.md` |
|
||||
|
||||
@@ -27,6 +27,14 @@ Master/slave model:
|
||||
- Do not perform destructive git/file actions without explicit instruction.
|
||||
- Browser automation (Playwright, Cypress, Puppeteer) MUST run in headless mode. Never launch a visible browser — it collides with the user's display and active session.
|
||||
|
||||
### Output standards (writing + code)
|
||||
|
||||
- Technical documentation follows **MOS-STE** (Mosaic Simplified Technical English — an adapted ASD-STE100 profile): short sentences, one instruction per sentence, active voice, one word per meaning, one term per concept. Full rules: `~/.config/mosaic/guides/WRITING-STYLE.md`.
|
||||
- Apply MOS-STE **hardest to verification artifacts** (acceptance criteria, witness predicates, gate/alarm conditions). There an ambiguous term produces a false green, not just a confused reader.
|
||||
- Source code follows the **Google Style Guide** for the language.
|
||||
- User-facing comms follow the user's declared `communicationStyle` in `USER.md` "Communication Preferences" (`direct` | `friendly` | `formal`, default `direct`); `guides/WRITING-STYLE.md` §5 maps each value to output. The documentation standard does not change with user preference.
|
||||
- **Carve-out:** MOS-STE does NOT apply to content that must carry a specific human voice (letters, personal or marketing prose, voice-matched output). A declared voice profile wins.
|
||||
|
||||
### Secrets handling (HARD RULE)
|
||||
|
||||
- Vault is the canonical source-of-truth for every secret in every environment. No exceptions.
|
||||
|
||||
@@ -0,0 +1,134 @@
|
||||
# Writing Style Standard — MOS-STE (MANDATORY)
|
||||
|
||||
This guide defines how agents write. It sets one style standard per output type.
|
||||
It is written in the standard it defines, as a worked example.
|
||||
|
||||
**Adapted, not compliant.** MOS-STE (Mosaic Simplified Technical English) is an
|
||||
adapted profile of ASD-STE100. Mosaic does not license or certify against
|
||||
ASD-STE100. Mosaic uses the load-bearing rules and fits them to agent work. This
|
||||
is the same stance Mosaic takes toward DO-178B/C: use the rigor, do not claim the
|
||||
certification.
|
||||
|
||||
## Scope — which standard governs which output
|
||||
|
||||
| Output type | Standard |
|
||||
|---|---|
|
||||
| Technical documentation (READMEs, runbooks, PRDs, procedures, ADRs, guides, acceptance criteria, design docs) | **MOS-STE** (this guide) |
|
||||
| Source code and code comments | **Google Style Guide** for the language (§4) |
|
||||
| Inter-agent comms | MOS-STE by default (concise, structured) |
|
||||
| User-facing comms | **Per-user style choice** — read `USER.md` "Communication Preferences" (§5) |
|
||||
| End-user prose the user owns (marketing, letters, personal writing, voice-matched content) | The user's declared voice. MOS-STE does NOT apply. |
|
||||
|
||||
**The user-voice carve-out is absolute.** Do not apply MOS-STE to content that
|
||||
must carry a specific human voice (for example a cover letter, a personal
|
||||
message, or marketing copy). That content needs the user's voice. MOS-STE would
|
||||
damage it. When a project declares a voice profile, that profile wins.
|
||||
|
||||
## 1. Why one standard
|
||||
|
||||
Agent documentation drifts across projects. Different agents use different terms,
|
||||
sentence styles, and structures for the same concept. Readers lose time.
|
||||
Assumptions hide in ambiguous prose. One standard gives agents a clear target. It
|
||||
gives reviewers a clear test.
|
||||
|
||||
## 2. Where MOS-STE matters most — verification artifacts
|
||||
|
||||
Apply MOS-STE hardest to acceptance criteria, witness predicates, gate
|
||||
definitions, and alarm conditions. In prose, an ambiguous term produces a
|
||||
confused reader. In a verification artifact, an ambiguous term produces a false
|
||||
green — a check that passes without testing the claim.
|
||||
|
||||
The one-term-one-concept rule (rule 9) is the guard. When one word names two
|
||||
concepts in one predicate, the check can test the wrong concept and still pass.
|
||||
|
||||
**Worked failure.** A rename used a witness predicate with three clauses: ref A
|
||||
present, ref B absent, tip committed from this host. Every clause tested the git
|
||||
*ref* (the channel). The claim under test was about a *field inside the payload*.
|
||||
The word "beacon" named two concepts in one sentence. Deleting ref B was the next
|
||||
scheduled step. That step flips the last clause green and certifies a state in
|
||||
which the payload still names the wrong host. The predicate was one planned action
|
||||
away from a false green on its normal path. The payload field was never tested.
|
||||
|
||||
Rule: when N failure modes share one observable, the observable is not a
|
||||
diagnostic. In a verification artifact, that ambiguity does not confuse a reader —
|
||||
it certifies the defect.
|
||||
|
||||
## 3. MOS-STE rules
|
||||
|
||||
### 3.1 Sentence rules
|
||||
|
||||
1. Keep sentences short. Use 20 words or fewer for a procedure. Use 25 words or
|
||||
fewer for a description. (Reasoning and doctrine prose relaxes this limit —
|
||||
see §3.4. A future lint enforces §3.1, not §3.4.)
|
||||
2. Write one instruction per sentence. In a procedure, give one command per step.
|
||||
3. Use the active voice. Write "Run the script." Do not write "The script should
|
||||
be run."
|
||||
4. Use the imperative for instructions. Start the sentence with the verb.
|
||||
5. Use simple verb tenses. Prefer the present tense. Avoid the perfect and
|
||||
progressive tenses when a simple tense works.
|
||||
6. Do not use an `-ing` form when it makes the meaning unclear.
|
||||
7. Write positive statements. State what to do, not only what to avoid.
|
||||
|
||||
### 3.2 Word rules
|
||||
|
||||
8. Use one word for one meaning. Do not use the same word in two senses.
|
||||
9. Use one term for one concept. Do not use synonyms for variety. Example: choose
|
||||
`secret`, `credential`, or `key` for each concept, and keep it.
|
||||
10. Use articles (`a`, `the`). Do not drop words to save space.
|
||||
11. Keep an approved-terms glossary per project. Add each domain noun and each
|
||||
chosen verb. Technical names (for example `Vault`, `cgroup`, `systemd`) are
|
||||
always allowed.
|
||||
12. Define an abbreviation at its first use. Then use it consistently.
|
||||
|
||||
### 3.3 Structure rules
|
||||
|
||||
13. Use a list for parallel items or sequential steps. Do not put them in one long
|
||||
sentence.
|
||||
14. Use a table for data with more than two dimensions.
|
||||
15. Use parallel structure in headings and steps.
|
||||
16. Repeat the noun. Do not use a pronoun when the reference is unclear.
|
||||
|
||||
### 3.4 Adaptation notes (where MOS-STE deviates from ASD-STE100, and why)
|
||||
|
||||
- **No licensed dictionary.** ASD-STE100 ships a controlled dictionary under
|
||||
copyright. MOS-STE uses per-project glossaries instead (rule 11).
|
||||
- **Domain terms are allowed.** MOS-STE keeps every term the work needs.
|
||||
- **Reasoning prose gets structure, not amputation.** Apply the sentence and word
|
||||
rules to design and doctrine writing. Allow the length a subtle argument needs.
|
||||
Readable-first beats rule-strict when the two conflict.
|
||||
|
||||
## 4. Code — Google Style Guide
|
||||
|
||||
Write source code to the Google Style Guide for the language (Python, TypeScript,
|
||||
Shell, Go, and so on). Match the existing file when a local convention already
|
||||
exists. Keep code comments to the MOS-STE sentence and word rules.
|
||||
|
||||
## 5. User-facing comms — a per-user choice
|
||||
|
||||
Mosaic is multi-user. Different users want different comms styles. The framework
|
||||
already carries the selectable setting: `communicationStyle` (`direct` |
|
||||
`friendly` | `formal`, default `direct`). `mosaic init` writes it, and the
|
||||
builder renders it into the generated `USER.md` "Communication Preferences"
|
||||
section. This guide adds the OUTPUT meaning of each value; do not invent new
|
||||
values.
|
||||
|
||||
The builder renders the style as prose bullets, not the token name, so match on
|
||||
the leading bullet the generated `USER.md` actually contains:
|
||||
|
||||
| `USER.md` leading bullet | Style | User-facing output |
|
||||
|---|---|---|
|
||||
| "Direct and concise" | `direct` (default) | MOS-STE structure — short, active, defined terms, tables for overview. |
|
||||
| "Warm and conversational" | `friendly` | Warmer register. Full sentences, explain reasoning, fewer tables. |
|
||||
| "Professional and structured" | `formal` | Professional and structured. Thorough, with explicit recommendations. |
|
||||
|
||||
This setting governs **user-facing comms only**. It does not change the
|
||||
documentation standard (§3), which is always MOS-STE regardless of the value.
|
||||
|
||||
## 6. Enforcement
|
||||
|
||||
- **Now:** human review only. **No mechanical prose check exists today.** The
|
||||
pre-push gate runs typecheck, lint, build, and tests; it inspects no prose.
|
||||
Reviewers check output against the scope table and the MOS-STE rules by hand.
|
||||
- **Future:** an MOS-STE lint check (built from the §3.1 sentence rules) and a
|
||||
Google-style linter in the pre-push gate. A future linter enforces §3.1, not
|
||||
§3.4 — see the note at rule 1.
|
||||
@@ -1,143 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# mutate-push-guard.sh -- regenerate the README's mutation table from MEASUREMENT.
|
||||
#
|
||||
# WHY THIS EXISTS. The README carried a hand-written mutation table quoting
|
||||
# "32/32" style results. Those numbers were true when typed and went stale in
|
||||
# silence as the suite grew. A README is what a reader trusts when the tool
|
||||
# misbehaves, so a confidently-wrong one is worse than none. Every number the
|
||||
# README prints about mutation now comes out of this script.
|
||||
#
|
||||
# TWO WAYS A MUTATION RUN LIES, AND THE GUARD FOR EACH:
|
||||
#
|
||||
# 1. THE ANCHOR NO LONGER MATCHES. The mutant is never applied, the suite is
|
||||
# green, and the report says SURVIVED -- which is the same word a real
|
||||
# coverage gap gets. Guarded by grep-before-mutate (ANCHOR MISSING).
|
||||
#
|
||||
# 2. THE ANCHOR MATCHES PROSE. This one bit me on the first run: my
|
||||
# "revert refuse-on-missing-config" mutant landed inside the usage()
|
||||
# heredoc, edited a help string, changed no behaviour, and duly reported
|
||||
# SURVIVED. I nearly recorded a documentation edit as an uncovered branch.
|
||||
# A MUTATION THAT CANNOT CHANGE BEHAVIOUR IS NOT A SURVIVING MUTANT, IT IS
|
||||
# A NON-MEASUREMENT -- and a non-measurement reported as a result is the
|
||||
# same defect as the vacuous test it is supposed to be hunting.
|
||||
# Guarded by prose_range(): anchors inside usage() are a loud refusal.
|
||||
#
|
||||
# 3. THE ANCHOR IS AMBIGUOUS. replace(...,1) silently picks the first match,
|
||||
# which may not be the branch named in the label. Guarded by an exact
|
||||
# occurrence count.
|
||||
set -uo pipefail
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
TARGET="$HERE/push-guard.sh"
|
||||
SUITE="$HERE/test-push-guard.sh"
|
||||
BAK="$(mktemp)"; cp "$TARGET" "$BAK"
|
||||
trap 'cp "$BAK" "$TARGET"; rm -f "$BAK"' EXIT
|
||||
|
||||
# --- where the prose lives: usage() { ... EOF ---------------------------------
|
||||
PROSE_LO="$(grep -n '^usage() {' "$BAK" | head -1 | cut -d: -f1)"
|
||||
PROSE_HI="$(awk -v lo="$PROSE_LO" 'NR > lo && /^EOF$/ { print NR; exit }' "$BAK")"
|
||||
if [[ -z "$PROSE_LO" || -z "$PROSE_HI" ]]; then
|
||||
echo "!! cannot locate the usage() heredoc -- the prose guard would be inert; refusing" >&2
|
||||
exit 1
|
||||
fi
|
||||
printf 'prose (usage heredoc) is lines %s-%s -- anchors there are refused, not scored\n\n' \
|
||||
"$PROSE_LO" "$PROSE_HI"
|
||||
|
||||
rc_all=0
|
||||
KILLED=0; SURVIVED=0
|
||||
declare -a ROWS=()
|
||||
|
||||
mutate() {
|
||||
local name="$1" find="$2" repl="$3"
|
||||
# Count and locate in python so MULTI-LINE anchors work. They are required:
|
||||
# two unrelated branches in this file are both the single line
|
||||
# `if (( status != 0 )); then`, and a one-line anchor cannot say which one a
|
||||
# result belongs to. Guessing would attribute a kill to the wrong branch.
|
||||
local loc; loc="$(python3 - "$BAK" "$find" <<'LOCPY'
|
||||
import sys
|
||||
s = open(sys.argv[1]).read(); find = sys.argv[2]
|
||||
n = s.count(find)
|
||||
print(n, (s[:s.index(find)].count("\n") + 1) if n else 0)
|
||||
LOCPY
|
||||
)"
|
||||
local n="${loc%% *}" ln="${loc##* }"
|
||||
if (( n == 0 )); then
|
||||
printf ' !! ANCHOR MISSING %-46s NOT APPLIED -- result would be meaningless\n' "$name"
|
||||
rc_all=1; return
|
||||
fi
|
||||
if (( n != 1 )); then
|
||||
printf ' !! ANCHOR AMBIGUOUS %-46s %d matches -- refusing to guess which branch\n' "$name" "$n"
|
||||
rc_all=1; return
|
||||
fi
|
||||
if (( ln >= PROSE_LO && ln <= PROSE_HI )); then
|
||||
printf ' !! ANCHOR IS PROSE %-46s line %d is inside usage() -- not a branch\n' "$name" "$ln"
|
||||
rc_all=1; return
|
||||
fi
|
||||
|
||||
python3 - "$BAK" "$TARGET" "$find" "$repl" <<'PY'
|
||||
import sys
|
||||
src, dst, find, repl = sys.argv[1:5]
|
||||
open(dst, "w").write(open(src).read().replace(find, repl, 1))
|
||||
PY
|
||||
local out line failed passed total
|
||||
out="$("$SUITE" 2>&1)"
|
||||
line="$(printf '%s\n' "$out" | grep -E 'needles: [0-9]+ passed' | tail -1)"
|
||||
failed="$(printf '%s\n' "$line" | sed -n 's/.*, \([0-9]*\) failed.*/\1/p')"
|
||||
passed="$(printf '%s\n' "$line" | sed -n 's/.*: \([0-9]*\) passed.*/\1/p')"
|
||||
if [[ -z "$failed" || -z "$passed" ]]; then
|
||||
printf ' !! NO TALLY %-46s suite produced no needle count\n' "$name"
|
||||
rc_all=1; cp "$BAK" "$TARGET"; return
|
||||
fi
|
||||
total=$(( passed + failed ))
|
||||
if (( failed > 0 )); then
|
||||
printf ' KILLED L%-5s %-46s %s/%s fail\n' "$ln" "$name" "$failed" "$total"
|
||||
ROWS+=("| \`$name\` (L$ln) | $failed/$total fail | killed |")
|
||||
KILLED=$(( KILLED + 1 ))
|
||||
else
|
||||
printf ' SURVIVED L%-5s %-46s 0/%s fail <-- UNCOVERED BRANCH\n' "$ln" "$name" "$total"
|
||||
ROWS+=("| \`$name\` (L$ln) | 0/$total fail | **SURVIVED** |")
|
||||
SURVIVED=$(( SURVIVED + 1 )); rc_all=1
|
||||
fi
|
||||
cp "$BAK" "$TARGET"
|
||||
}
|
||||
|
||||
echo "=== push-guard mutation run ==="
|
||||
mutate "json decision requirement bypassed" \
|
||||
' if (( ${#JSON_PATHS[@]} == 0 )); then' ' if false; then'
|
||||
mutate "opt-out accepted with no written reason" \
|
||||
'if not isinstance(reason, str) or not reason.strip():' 'if False:'
|
||||
mutate "committed re-read of the opt-out skipped" \
|
||||
' [[ "$CFG_MODE" == "none" ]] || return 0' ' return 0'
|
||||
mutate "untracked config honoured as an opt-out" \
|
||||
' if [[ -z "$rel" ]]; then' ' if false; then'
|
||||
mutate "staged-but-uncommitted opt-out honoured" \
|
||||
' if [[ -z "$cmode" ]]; then' ' if false; then'
|
||||
mutate "committed SYMLINK config honoured" \
|
||||
' if [[ "$cmode" == "120000" ]]; then' ' if false; then'
|
||||
mutate "unparseable committed config ignored" \
|
||||
' if (( cstatus != 0 )); then' ' if false; then'
|
||||
mutate "local-only opt-out (HEAD says ON) honoured" \
|
||||
' if [[ "$cmode_val" != "none" ]]; then' ' if false; then'
|
||||
mutate "empty MERGE exempted" \
|
||||
' if [[ "$all_same" == yes ]]; then' ' if false; then'
|
||||
mutate "empty ROOT exempted" \
|
||||
'if [[ -z "$(git diff-tree --root -r --name-only --no-commit-id HEAD)" ]]; then' \
|
||||
'if false; then'
|
||||
mutate "--since-head ancestry check removed" \
|
||||
'if ! git merge-base --is-ancestor "$since_head" "$head"; then' 'if false; then'
|
||||
mutate "staged-file enumeration ignores git failure" \
|
||||
"$(printf 'if (( status != 0 )); then\n local msg')" \
|
||||
"$(printf 'if false; then\n local msg')"
|
||||
mutate "malformed config degrades to absent instead of refusing" \
|
||||
"$(printf 'if (( status != 0 )); then\n fail "$EX_CONFIG"')" \
|
||||
"$(printf 'if false; then\n fail "$EX_CONFIG"')"
|
||||
|
||||
BASE="$("$SUITE" 2>&1 | grep -E 'needles: [0-9]+ passed' | tail -1)"
|
||||
printf '\nunmodified: %s\n' "$BASE"
|
||||
printf '%d killed, %d survived\n' "$KILLED" "$SURVIVED"
|
||||
|
||||
printf '\n--- README TABLE (paste verbatim) ---\n'
|
||||
printf '| mutation | suite result | verdict |\n|---|---|---|\n'
|
||||
printf '| *unmodified* | %s | baseline |\n' \
|
||||
"$(printf '%s' "$BASE" | sed 's/push-guard needles: //')"
|
||||
printf '%s\n' "${ROWS[@]}"
|
||||
exit "$rc_all"
|
||||
@@ -1,164 +0,0 @@
|
||||
# push-guard
|
||||
|
||||
Mechanical closure of one defect: **a verification that passes when the thing it verifies never happened.**
|
||||
|
||||
Three incidents in one evening, three agents, no coordination, same shape:
|
||||
|
||||
| # | Incident | Why the check passed |
|
||||
| --- | ----------------------------------------------------------------- | ------------------------------------------------------------------- |
|
||||
| 1 | `PUSH VERIFIED` reported after nothing was pushed | Commit had aborted, so local HEAD trivially equalled the remote ref |
|
||||
| 2 | An **empty commit** carrying another commit's message was pushed | A push was chained after a _failed_ commit and ran on stale state |
|
||||
| 3 | Conflict markers + invalid JSON committed into 70 generated files | Nothing asserted the generated output still parsed |
|
||||
|
||||
Each is an assertion satisfied by the null case. `push-guard.sh` asserts the **positive** fact instead: a new object exists, the remote _moved_, the content _parses_.
|
||||
|
||||
## Checks
|
||||
|
||||
| Sub-command | Asserts | Exit on failure |
|
||||
| -------------- | -------------------------------------------- | --------------- |
|
||||
| `check-staged` | index has no unmerged paths | `2` |
|
||||
| | no conflict markers in staged content | `2` |
|
||||
| | the conflict scan actually _completed_ | `2` |
|
||||
| | staged JSON under the nominated paths parses | `3` |
|
||||
| | **something is actually staged** | `5` |
|
||||
| | **an explicit JSON decision exists** | `6` |
|
||||
| `push` | HEAD advanced past `--since-head` | `5` |
|
||||
| | HEAD is a non-empty commit | `5` |
|
||||
| | remote **moved**, and now equals local HEAD | `4` |
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
push-guard.sh check-staged --json-path 'data/**/*.json'
|
||||
|
||||
BEFORE=$(git rev-parse HEAD)
|
||||
git commit -m "..."
|
||||
push-guard.sh push --remote origin --branch main --since-head "$BEFORE"
|
||||
```
|
||||
|
||||
`--since-head` is what distinguishes "committed nothing" from "committed something", and capturing the remote ref _before_ pushing is what makes `remote == local` mean anything. Both were missing from the guard that produced incident 1.
|
||||
|
||||
## Design decisions that measurement forced
|
||||
|
||||
These are the interesting part; each one was wrong in the first draft.
|
||||
|
||||
**A bare `=======` is deliberately NOT treated as a conflict marker.** It collides with reStructuredText underlines and ASCII rules. Every genuine conflict git writes also contains `<<<<<<<` and `>>>>>>>`, so requiring those loses no real detection. Measured against a real repository: **zero** false positives across the entire tracked tree.
|
||||
|
||||
**The JSON check is opt-in by path, not on-by-default.** The first draft checked every staged `*.json`, on the reasoning that broader is strictly stronger. Measured against the same repository: **17 of 119** tracked `.json` files fail a strict parse, _all legitimately_ — every `tsconfig*.json` is JSONC (comments are legal there) and the CA templates are Go templates that merely carry a `.json` suffix. An on-by-default check fires on ~14% of the repo's JSON, and a guard that cries wolf gets routed around until the bypass is habitual — at which point the bypass covers the true positives too. Broader was **weaker**.
|
||||
|
||||
**`--json-path` values are normalized to `:(glob)` magic.** Git's default pathspec matching makes `data/**/*.json` require at least one intermediate directory: it matches `data/sub/b.json` and _silently skips_ `data/a.json`. The obvious spelling would have delivered partial coverage with no warning.
|
||||
|
||||
## Found by independent review, after the needles were already green
|
||||
|
||||
An adversarial review (author ≠ reviewer) found two genuine fail-opens that 17 self-written needles, five mutation runs and a repo-wide false-positive measurement had all missed. Both were reproduced before being fixed, and both now have needles.
|
||||
|
||||
**Renamed files were invisible to every content check.** Git detects renames by default (`diff.renames=true` since 2.9), so `git mv` plus a small edit is reported as a single `R` entry — which `--diff-filter=ACM` does not match. Reproduced at 97% similarity: the guard printed `staged content OK` and exited 0 with conflict markers in the staged blob. Fixed with `--no-renames`.
|
||||
|
||||
There is a trap inside this one worth recording. With _only_ the renamed file staged, the guard exits 5 — the nothing-staged check fires because the file list came back empty. That looks like a catch and is pure accident; co-stage one ordinary file and the fail-open is total. The needle deliberately co-stages a clean file so it cannot pass for the wrong reason.
|
||||
|
||||
**`-I` skipped files marked binary in `.gitattributes`,** while the summary still counted them as scanned — a clean pass _and_ a false count. Fixed with `-a`.
|
||||
|
||||
The review also correctly identified one **vacuous control**: the positive `--since-head` case asserted only exit 0, which the push produces anyway, so deleting the entire `--since-head` block left it passing. It now asserts on output.
|
||||
|
||||
Two reported findings did **not** reproduce and were not acted on: `:(glob)` does still match a bare directory pathspec, and my first rename repro failed only because the edit dropped similarity below the detection threshold — a badly built test, not an absent bug. Re-testing properly is what confirmed it.
|
||||
|
||||
## The JSON decision is mandatory (`.push-guard.json`)
|
||||
|
||||
The first release announced `JSON check NOT REQUESTED` and continued. That was _visible_ rather than silent, which is better — but it is still an **absence**, and an absence is not reviewable. Nobody reads a line that has printed correctly ten thousand times. A repo that needed the check and never wired it up stayed unprotected forever, and nothing ever failed.
|
||||
|
||||
The decision is now required, from one of exactly two places:
|
||||
|
||||
```jsonc
|
||||
// nominate generated JSON for parse-checking
|
||||
{"json_paths": ["data/**/*.json"], "allow_invalid_json": ["fixtures/*"]}
|
||||
|
||||
// or opt out — the reason is REQUIRED and is printed on every run
|
||||
{"json_check": "none", "reason": "no generated JSON in this repo"}
|
||||
```
|
||||
|
||||
`--json-path` on the command line also satisfies it. Saying nothing is refused (exit `6`).
|
||||
|
||||
**The asymmetry is deliberate.** Turning the check _on_ is safe from anywhere, so a CLI flag suffices. Turning it _off_ is confined to a committed file, because that is the only form a human can review: you can read a reason in a diff, and you cannot review the fact that nobody typed a flag. An opt-out living in an ad-hoc command line is the old fail-open with extra steps.
|
||||
|
||||
A **malformed** config is refused outright rather than treated as absent — that fallback would mean a typo silently disables the check the file was written to enable. A config that parses but _states nothing_ (`{}`) is refused too: valid JSON that says nothing is the original defect wearing a config file as a disguise.
|
||||
|
||||
**Migration is a hard cutover, on purpose.** The tempting path is "warn for one release, then enforce" — but that warning phase _is_ the degrade-to-a-warning this tool forbids, and it leaves the fail-open open for exactly as long as the warning is ignored, which is indefinitely. Consumers add a config or the guard refuses. It breaks loudly, once. Two pre-existing cases in this repo's own harness were the first to pay that cost, which is the correct place to feel it.
|
||||
|
||||
## Known limitations — stated, not hidden
|
||||
|
||||
- `check-staged` inspects the index. Content added to the working tree _after_ it runs is not covered — it belongs in a `pre-commit` hook, where the window is smallest.
|
||||
- `push` hardcodes `HEAD:refs/heads/$branch`, so it cannot verify a commit built via plumbing on a different base. A `--sha` option would close this; it is deliberately not added without a needle.
|
||||
|
||||
## Test harness
|
||||
|
||||
`test-push-guard.sh` — 46 cases, each check in **both polarities**:
|
||||
|
||||
- **NEEDLE** — a deliberately broken fixture that must trip the guard.
|
||||
- **CONTROL** — a clean fixture that must pass.
|
||||
|
||||
Controls are not decoration. A guard that failed unconditionally would satisfy every needle and look fully covered. Several controls additionally assert on the guard's _output_ (`--out`), because exit 0 cannot distinguish "checked the files and they were fine" from "matched no files and had nothing to check" — two controls in an earlier revision were passing vacuously for exactly that reason.
|
||||
|
||||
### Mutation results — the harness was itself tested by breaking the guard
|
||||
|
||||
**Everything in this section is regenerated by `./mutate-push-guard.sh`, not typed.** The previous version of this table was hand-maintained: it was true when written, went stale as the suite grew, and ended up asserting `unmodified | 32/32 pass` against a 46-case suite. A README is what a reader consults when the tool misbehaves, so a confidently-wrong one is worse than none. Re-run the script and paste; do not edit the numbers.
|
||||
|
||||
| mutation | suite result | verdict |
|
||||
| ---------------------------------------------------------------- | ------------------- | -------- |
|
||||
| _unmodified_ | 46 passed, 0 failed | baseline |
|
||||
| `json decision requirement bypassed` (L482) | 3/46 fail | killed |
|
||||
| `opt-out accepted with no written reason` (L334) | 1/46 fail | killed |
|
||||
| `committed re-read of the opt-out skipped` (L405) | 5/46 fail | killed |
|
||||
| `untracked config honoured as an opt-out` (L409) | 1/46 fail | killed |
|
||||
| `staged-but-uncommitted opt-out honoured` (L425) | 1/46 fail | killed |
|
||||
| `committed SYMLINK config honoured` (L433) | 1/46 fail | killed |
|
||||
| `unparseable committed config ignored` (L444) | 1/46 fail | killed |
|
||||
| `local-only opt-out (HEAD says ON) honoured` (L459) | 1/46 fail | killed |
|
||||
| `empty MERGE exempted` (L704) | 1/46 fail | killed |
|
||||
| `empty ROOT exempted` (L715) | 1/46 fail | killed |
|
||||
| `--since-head ancestry check removed` (L665) | 1/46 fail | killed |
|
||||
| `staged-file enumeration ignores git failure` (L173) | 1/46 fail | killed |
|
||||
| `malformed config degrades to absent instead of refusing` (L375) | 4/46 fail | killed |
|
||||
|
||||
13 mutants, 0 survived.
|
||||
|
||||
Earlier mutants, run at the suite size of the day and kept as history rather than as a live claim: fail-open (9/16), always-fail (16/16 — controls catch it), revert `:(glob)` normalization (3/16, caught **only** by the `--out` assertions), drop the remote-did-not-move assertion (1/16), restore blanket `|| true` on the scan (1/17), revert `--no-renames` (1/19), revert `-a` to `-I` (1/19), delete the `--since-head` block (2/19, incl. the control that _was_ vacuous), stray-warning emission (1/32).
|
||||
|
||||
A guard nobody has watched fail is not a guard. The same applies to the harness: the `:(glob)` mutant would have passed silently without the output assertions, so the assertions are load-bearing rather than ornamental.
|
||||
|
||||
### Two branches were uncovered, and writing this table is what found them
|
||||
|
||||
Building the generator turned up two mutants that survived a **46-case suite reporting 46/46 green**: `staged-but-uncommitted opt-out honoured` and `unparseable committed config ignored`. Neither branch had any case at all. Both are now needled, which is why they read _killed_ above.
|
||||
|
||||
Both survivors share a shape worth naming: the mutant still exits `6`, because control falls through to a _sibling_ refusal that rejects for a different reason. **An exit-code-only assertion would have been satisfied by the wrong branch.** The new cases anchor on the distinguishing clause instead.
|
||||
|
||||
The same audit found a live instance of that defect already in the suite. Three separate branches print the headline `OPT-OUT IS NOT REVIEWABLE` — untracked, staged-not-committed, and symlink — and the untracked needle was anchored on the shared headline. Delete the untracked branch and control reaches a sibling printing those same words, so **the needle would not fail; it would re-point.** A substring anchor does not fail when its subject is removed. It is now anchored on `is not tracked in git`.
|
||||
|
||||
### The generator refuses three ways a mutation run can lie
|
||||
|
||||
1. **The anchor no longer matches.** The mutant is never applied, the suite is green, and a naive report says _survived_ — the same word a real coverage gap gets. `ANCHOR MISSING` is a loud failure instead.
|
||||
2. **The anchor matches prose.** This one landed on the first run: a mutant aimed at the refuse-on-missing-config branch matched inside the `usage()` heredoc, edited a help string, changed no behaviour, and duly reported _survived_. A documentation edit was one step from being recorded as an uncovered branch. **A mutation that cannot change behaviour is not a surviving mutant, it is a non-measurement** — and a non-measurement reported as a result is the same defect as the vacuous test the harness exists to hunt. Anchors resolving inside `usage()` are now refused, and the heredoc's bounds are located at runtime rather than hardcoded.
|
||||
3. **The anchor is ambiguous.** Two unrelated branches here are both the single line `if (( status != 0 )); then`; a first-match replace would silently attribute the kill to whichever came first. Multi-line anchors are supported and a match count `!= 1` refuses rather than guesses.
|
||||
|
||||
**A blanket `|| true` on the scan was a fail-open in the guard's own error handling.** `git grep` exits `1` for "no match" but `>=2` for a real failure. `|| true` collapsed the two, so a malformed pathspec or unreadable index would have been reported as "ok, no conflict markers" — a clean pass from a scan that never ran. The status is now discriminated, and a PATH-shim fault-injection needle proves the refusal.
|
||||
|
||||
**A `<<'PY'` heredoc inside `$( )` made bash print a warning on every run — and 30 needles plus a clean linter all missed it.** The config parser started life as an inline heredoc inside a command substitution, so every invocation emitted `warning: command substitution: 1 unterminated here-document` to stderr. It executed correctly, every needle stayed green, and shellcheck reported nothing. The harness missed it because `expect` asserts _substrings it was told to look for_ — and nobody tells you to look for output you did not know existed. It surfaced only when the guard was run against a real repository.
|
||||
|
||||
The fix moves the parser to a top-level constant. The lesson is encoded as case `z1`, which asserts the **absence of a whole output class** rather than the presence of an expected string, and is paired with a needle proving the detector can fire. A tool built to refuse quiet failures was quietly polluting stderr for its entire existence.
|
||||
|
||||
**The `-E` flag on `git grep` is load-bearing and was caught by a needle, not by review.** `git grep` defaults to _basic_ regex, in which `(`, `|` and `{7}` are literal characters. Without `-E` the patterns match nothing and the check reports a clean pass over a file full of conflict markers — the exact defect this tool exists to prevent, shipped inside the tool itself.
|
||||
|
||||
## Proposed framework path
|
||||
|
||||
```
|
||||
framework/tools/git/push-guard.sh # the guard
|
||||
framework/tools/git/test-push-guard.sh # 46 needles and controls
|
||||
framework/tools/git/mutate-push-guard.sh # regenerates the mutation table above
|
||||
framework/tools/git/verify-clean-clone.sh # proves the COMMITTED artifact runs
|
||||
framework/tools/git/test-verify-clean-clone.sh # 6 needles for the verifier itself
|
||||
```
|
||||
|
||||
Matches the existing `tools/git/test-*.sh` convention. Dependencies: bash 4.4+, git, python3 — `python3` is already an accepted dependency of `ci-queue-wait.sh`.
|
||||
|
||||
**All five must be committed mode `100755`.** They were once delivered `100644`, so a clone exited `126 Permission denied` for everyone who was not the author; `verify-clean-clone.sh` exists to make that unshippable and asserts the mode from `git ls-tree` of the source commit, never from the filesystem.
|
||||
|
||||
Operator-agnostic: no hostnames, credentials, remotes, or operator-specific paths. Clean under the framework-PR firewall.
|
||||
@@ -1,793 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# push-guard.sh - Mechanical guards against verifications that PASS when the
|
||||
# thing they verify never happened.
|
||||
#
|
||||
# WHY THIS EXISTS
|
||||
# ---------------
|
||||
# Three independent incidents, one shape:
|
||||
# 1. An agent's "PUSH VERIFIED" step compared local HEAD to the remote ref and
|
||||
# reported success. The commit had ABORTED, so local trivially equalled
|
||||
# remote. The guard could not distinguish "pushed" from "committed nothing".
|
||||
# 2. An agent chained a push after a FAILED commit, ran on stale state, and
|
||||
# pushed an EMPTY commit carrying an unrelated commit's message.
|
||||
# 3. An agent committed unresolved conflict markers into 70 generated files and
|
||||
# pushed invalid JSON to a default branch.
|
||||
#
|
||||
# All three are the same defect: a check whose success condition is satisfied by
|
||||
# the null case. This script asserts the POSITIVE fact (a new object exists, the
|
||||
# remote MOVED, the content parses) rather than the absence of an error.
|
||||
#
|
||||
# DESIGN RULES (do not relax these)
|
||||
# * Every check exits NON-ZERO and names what failed. There is deliberately no
|
||||
# --warn-only / --soft flag: a guard that can degrade to a warning is the
|
||||
# fail-open we are trying to remove.
|
||||
# * Checks are fail-CLOSED. `set -euo pipefail` means an unexpected git or
|
||||
# network error aborts non-zero rather than falling through to "OK".
|
||||
# * Nothing is skipped silently. A check with no applicable inputs says so on
|
||||
# stdout instead of contributing a quiet green.
|
||||
#
|
||||
# EXIT CODES
|
||||
# 0 all requested checks passed
|
||||
# 2 conflict markers present, or the index has unmerged paths
|
||||
# 3 staged JSON does not parse
|
||||
# 4 push verification failed (remote did not move, or moved elsewhere)
|
||||
# 5 nothing staged / empty commit — the null case, refused
|
||||
# 6 NO JSON DECISION — no --json-path and no usable .push-guard.json
|
||||
# 64 usage error
|
||||
#
|
||||
# Dependencies: bash 4.4+, git, python3.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
readonly EX_OK=0
|
||||
readonly EX_CONFLICT=2
|
||||
readonly EX_JSON=3
|
||||
readonly EX_PUSH=4
|
||||
readonly EX_NULL=5
|
||||
readonly EX_CONFIG=6
|
||||
readonly EX_USAGE=64
|
||||
|
||||
# Repo-level configuration. Its ABSENCE is refused, not defaulted — see
|
||||
# resolve_json_config() for why.
|
||||
readonly CONFIG_FILE=".push-guard.json"
|
||||
|
||||
# Conflict-marker detection.
|
||||
#
|
||||
# We match `<<<<<<<`, `>>>>>>>` and the diff3 `|||||||` marker, each anchored at
|
||||
# column 0 and required to be followed by a space or end-of-line (real markers
|
||||
# are exactly seven characters plus an optional ref label).
|
||||
#
|
||||
# We deliberately do NOT match a bare `=======` line. It is the one marker that
|
||||
# collides with ordinary prose — reStructuredText section underlines and ASCII
|
||||
# rules use it constantly — and a guard that fires on documentation gets switched
|
||||
# off, which costs more than the miss. Every genuine conflict git writes contains
|
||||
# `<<<<<<<` and `>>>>>>>` as well, so requiring those loses no real detection.
|
||||
readonly MARKER_PATTERNS=(
|
||||
-e '^<<<<<<<( |$)'
|
||||
-e '^>>>>>>>( |$)'
|
||||
-e '^[|]{7}( |$)'
|
||||
)
|
||||
|
||||
log() { printf '%s\n' "$*"; }
|
||||
fail() {
|
||||
local code="$1"; shift
|
||||
printf '\n' >&2
|
||||
printf 'PUSH-GUARD FAILED: %s\n' "$1" >&2
|
||||
shift
|
||||
local line
|
||||
for line in "$@"; do printf ' %s\n' "$line" >&2; done
|
||||
printf '\n' >&2
|
||||
exit "$code"
|
||||
}
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
push-guard.sh check-staged [--json-path <pathspec>]...
|
||||
[--allow-invalid-json <pathspec>]...
|
||||
push-guard.sh push --remote <remote> --branch <branch> [--since-head <sha>]
|
||||
[-- <extra git push args>...]
|
||||
push-guard.sh --help
|
||||
|
||||
Subcommands:
|
||||
check-staged Run pre-commit content guards against the STAGED index:
|
||||
(a) no unmerged index paths, no conflict markers
|
||||
(c) staged JSON under the nominated paths parses
|
||||
Refuses to pass when nothing is staged (exit 5).
|
||||
Check (c) requires an EXPLICIT decision — either --json-path,
|
||||
or .push-guard.json. Saying nothing is refused (exit 6).
|
||||
|
||||
push Perform a push and prove it HAPPENED:
|
||||
- HEAD is a non-empty commit (differs from its parent)
|
||||
- with --since-head: HEAD advanced past that sha
|
||||
- the remote ref MOVED, and now equals local HEAD
|
||||
Capturing the remote ref BEFORE the push is what makes
|
||||
"remote == local" meaningful; without it the assertion is
|
||||
satisfied by having pushed nothing.
|
||||
|
||||
Options:
|
||||
--json-path <pathspec> Git pathspec of generated/serialized JSON to
|
||||
parse-check, e.g. 'data/**/*.json'. Repeatable.
|
||||
Opt-in on purpose: many real .json files are
|
||||
JSONC (tsconfig) or templates and do NOT parse
|
||||
strictly. Nominate the generated ones. Satisfies
|
||||
the decision requirement on its own; overrides a
|
||||
config opt-out (loudly).
|
||||
--allow-invalid-json <pathspec> Exempt a path from the JSON parse check
|
||||
(for deliberate malformed-input fixtures).
|
||||
Exemptions are printed, never silent.
|
||||
--since-head <sha> HEAD before your commit step. Asserts a new
|
||||
commit object was actually created.
|
||||
--remote <name> Remote to push to.
|
||||
--branch <name> Branch to push.
|
||||
|
||||
Repo config (.push-guard.json, at the repository root):
|
||||
{"json_paths": ["data/**/*.json"], "allow_invalid_json": ["fixtures/*"]}
|
||||
— nominate generated JSON for parse-checking.
|
||||
{"json_check": "none", "reason": "no generated JSON in this repo"}
|
||||
— opt out. The reason is REQUIRED and is printed on every run.
|
||||
|
||||
An opt-out is only accepted from this file, never from a command-line flag: a
|
||||
committed reason can be read in a diff, an absent flag cannot be reviewed at all.
|
||||
An invalid config is refused outright rather than treated as absent.
|
||||
|
||||
There is no flag to downgrade a failure to a warning. That is intentional.
|
||||
EOF
|
||||
}
|
||||
|
||||
require_repo() {
|
||||
git rev-parse --show-toplevel >/dev/null 2>&1 \
|
||||
|| fail "$EX_USAGE" "not inside a git repository"
|
||||
cd "$(git rev-parse --show-toplevel)"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------- check-staged
|
||||
|
||||
# staged_files_z <array-name> [pathspec...]
|
||||
#
|
||||
# THE FAIL-OPEN THIS FUNCTION EXISTS TO REMOVE:
|
||||
# mapfile -d '' -t files < <(git diff ...)
|
||||
# observes MAPFILE's exit status, never the producer's. When git diff dies -- a
|
||||
# bad pathspec, a corrupt index -- mapfile cheerfully reads zero bytes and
|
||||
# succeeds, and the caller then reports that silence as "nothing to check".
|
||||
#
|
||||
# `set -o pipefail` does NOT cover this. pipefail applies to PIPELINES; a process
|
||||
# substitution is an asynchronous child whose status is never collected. That is
|
||||
# the precise reason every `cmd | cmd` in this file is safe and both `< <(cmd)`
|
||||
# constructs were not -- a distinction worth stating, because "we set pipefail"
|
||||
# reads like whole-file coverage and is not.
|
||||
#
|
||||
# Both callers now route through here, so the category is gone rather than the
|
||||
# two known instances being patched.
|
||||
staged_files_z() {
|
||||
local -n _out="$1"; shift
|
||||
local tmp err status=0
|
||||
tmp="$(mktemp)"; err="$(mktemp)"
|
||||
if (( $# )); then
|
||||
git diff --cached --name-only --diff-filter=ACM --no-renames -z \
|
||||
-- "$@" >"$tmp" 2>"$err" || status=$?
|
||||
else
|
||||
git diff --cached --name-only --diff-filter=ACM --no-renames -z \
|
||||
>"$tmp" 2>"$err" || status=$?
|
||||
fi
|
||||
if (( status != 0 )); then
|
||||
local msg; msg="$(tr '\n' ' ' <"$err")"
|
||||
rm -f "$tmp" "$err"
|
||||
fail "$EX_CONFIG" \
|
||||
"CANNOT ENUMERATE STAGED FILES — git diff exited $status" \
|
||||
"git said: ${msg:-(no message)}" \
|
||||
"" \
|
||||
"Refusing to report a result. An enumeration that failed returns no" \
|
||||
"files, which is indistinguishable from a clean tree — reporting that" \
|
||||
"as OK would be a silent pass on an unexamined index."
|
||||
fi
|
||||
_out=()
|
||||
mapfile -d '' -t _out <"$tmp"
|
||||
rm -f "$tmp" "$err"
|
||||
}
|
||||
|
||||
check_unmerged() {
|
||||
local unmerged
|
||||
unmerged="$(git ls-files --unmerged | awk '{print $4}' | sort -u)"
|
||||
if [[ -n "$unmerged" ]]; then
|
||||
mapfile -t paths <<<"$unmerged"
|
||||
fail "$EX_CONFLICT" \
|
||||
"the index has UNMERGED paths — a merge/rebase is still in progress" \
|
||||
"${paths[@]}" \
|
||||
"" \
|
||||
"Resolve the conflict and 'git add' the result. Do not commit past this."
|
||||
fi
|
||||
log " ok index has no unmerged paths"
|
||||
}
|
||||
|
||||
check_conflict_markers() {
|
||||
local -a files=("$@")
|
||||
local hits
|
||||
# git grep --cached searches the INDEX (what will actually be committed),
|
||||
# not the working tree.
|
||||
#
|
||||
# -a (treat every blob as text), NOT -I (skip binary). -I honours git's
|
||||
# binary classification, which includes an explicit `.gitattributes` `binary`
|
||||
# attribute. A repo that marks a path binary would have its staged conflict
|
||||
# markers silently skipped while the summary still counted the file as
|
||||
# scanned — a clean pass AND a false count. Verified reproducible. The cost
|
||||
# of -a is that a genuine binary blob could theoretically emit a noisy match
|
||||
# line; a false alarm is recoverable, a silent miss is not.
|
||||
# -E is load-bearing: git grep defaults to BASIC regex, in which '(', '|'
|
||||
# and '{7}' are literal characters. Without it these patterns match nothing
|
||||
# and the check reports a clean pass over a file full of markers. That
|
||||
# failure was caught by the needle, not by review.
|
||||
#
|
||||
# Exit status is discriminated, NOT swallowed. git grep returns 1 for "no
|
||||
# match" and >=2 for a real error (bad pathspec, unreadable index). A blanket
|
||||
# '|| true' would turn a broken scan into "ok, no conflict markers" — the
|
||||
# same fail-open in the guard's own error handling.
|
||||
local status=0
|
||||
hits="$(git grep --cached -a -n -E "${MARKER_PATTERNS[@]}" -- "${files[@]}")" || status=$?
|
||||
if (( status > 1 )); then
|
||||
fail "$EX_CONFLICT" \
|
||||
"git grep FAILED (exit $status) — the conflict scan did not run" \
|
||||
"Refusing to report a clean result from a scan that did not complete."
|
||||
fi
|
||||
if [[ -n "$hits" ]]; then
|
||||
mapfile -t lines <<<"$hits"
|
||||
fail "$EX_CONFLICT" \
|
||||
"staged content contains unresolved CONFLICT MARKERS" \
|
||||
"${lines[@]}" \
|
||||
"" \
|
||||
"These are staged and would be committed verbatim."
|
||||
fi
|
||||
log " ok no conflict markers in ${#files[@]} staged file(s)"
|
||||
}
|
||||
|
||||
# JSON parse check.
|
||||
#
|
||||
# SCOPE IS OPT-IN, AND THAT IS A MEASURED DECISION, NOT LAZINESS.
|
||||
# The first draft checked every staged *.json. Run against a real repository that
|
||||
# turned out to be 17 of 119 tracked .json files failing a strict parse — all of
|
||||
# them legitimate: every tsconfig*.json is JSONC (comments are legal there) and
|
||||
# the CA templates are Go templates that merely carry a .json suffix. An
|
||||
# on-by-default check would have fired on ~14% of this repo's JSON, and a guard
|
||||
# that cries wolf gets routed around until the bypass is habitual — which is
|
||||
# worse than no guard, because the bypass then also covers the true positives.
|
||||
#
|
||||
# So the caller nominates the generated/serialized paths this check is FOR, as
|
||||
# git pathspecs (git, not bash, expands them — '**' works correctly).
|
||||
#
|
||||
# WHAT SAYING NOTHING NOW COSTS YOU
|
||||
# ---------------------------------
|
||||
# The first release announced "JSON check NOT REQUESTED" and continued. That was
|
||||
# visible rather than silent, which is better — but it is still an ABSENCE, and
|
||||
# an absence is not reviewable. Nobody reads a line that has printed correctly
|
||||
# ten thousand times. A repo that needed the check and never wired it up stayed
|
||||
# unprotected forever, and nothing ever failed.
|
||||
#
|
||||
# The decision is now MANDATORY and must come from one of two places:
|
||||
# * turning the check ON — --json-path on the command line, or "json_paths"
|
||||
# in .push-guard.json
|
||||
# * turning the check OFF — ONLY "json_check": "none" in .push-guard.json,
|
||||
# which REQUIRES a non-empty "reason"
|
||||
# Saying nothing at all is refused (exit 6).
|
||||
#
|
||||
# The asymmetry is deliberate. Turning a check on is safe from anywhere. Turning
|
||||
# one OFF is confined to a committed file because that is the only form a human
|
||||
# can review: you can read a reason in a diff, and you cannot review the fact
|
||||
# that nobody typed a flag. An opt-out that lives in an ad-hoc command line is
|
||||
# just the old fail-open with extra steps.
|
||||
# The config parser, held as a string rather than an inline heredoc.
|
||||
#
|
||||
# A `<<'PY'` heredoc INSIDE a $( ) command substitution makes bash emit
|
||||
# warning: command substitution: 1 unterminated here-document
|
||||
# on every single run. It still executed, every needle stayed green and the
|
||||
# static linter stayed clean — the warning goes to stderr and broke no
|
||||
# assertion. It surfaced only when the guard was run against a real
|
||||
# repository. A tool built to refuse quiet output was quietly polluting
|
||||
# stderr; needle 'z1' now asserts the guard emits no warnings at all.
|
||||
#
|
||||
# `read -d ''` returns non-zero at EOF without finding a NUL, which is the
|
||||
# NORMAL path when slurping a heredoc — hence `|| true`. That is the one
|
||||
# shape where it does not mask a real error, unlike the `git grep || true`
|
||||
# this codebase already removed once.
|
||||
IFS='' read -r -d '' CONFIG_PARSER <<'PY' || true
|
||||
import json, sys
|
||||
|
||||
path = sys.argv[1]
|
||||
try:
|
||||
with open(path) as fh:
|
||||
cfg = json.load(fh)
|
||||
except Exception as exc:
|
||||
sys.stderr.write("does not parse: %s" % exc)
|
||||
sys.exit(2)
|
||||
|
||||
if not isinstance(cfg, dict):
|
||||
sys.stderr.write("must contain a JSON object, got %s" % type(cfg).__name__)
|
||||
sys.exit(2)
|
||||
|
||||
def clean_list(name, value):
|
||||
if not isinstance(value, list):
|
||||
sys.stderr.write('"%s" must be an array' % name)
|
||||
sys.exit(2)
|
||||
for item in value:
|
||||
if not isinstance(item, str) or not item.strip():
|
||||
sys.stderr.write('"%s" must contain only non-empty strings' % name)
|
||||
sys.exit(2)
|
||||
# Tabs/newlines would be mangled by the tab-delimited handoff below.
|
||||
# Reject them explicitly rather than silently truncating a pathspec.
|
||||
if "\t" in item or "\n" in item:
|
||||
sys.stderr.write('"%s" entry contains a tab or newline: %r' % (name, item))
|
||||
sys.exit(2)
|
||||
return value
|
||||
|
||||
mode = cfg.get("json_check")
|
||||
paths = cfg.get("json_paths")
|
||||
allow = cfg.get("allow_invalid_json", [])
|
||||
|
||||
if mode is not None and mode != "none":
|
||||
sys.stderr.write('"json_check" must be "none" if present, got %r' % (mode,))
|
||||
sys.exit(2)
|
||||
|
||||
if mode == "none":
|
||||
if paths:
|
||||
sys.stderr.write('"json_check": "none" and "json_paths" are mutually exclusive')
|
||||
sys.exit(2)
|
||||
reason = cfg.get("reason")
|
||||
if not isinstance(reason, str) or not reason.strip():
|
||||
sys.stderr.write('"json_check": "none" REQUIRES a non-empty "reason"')
|
||||
sys.exit(2)
|
||||
print("MODE\tnone")
|
||||
print("REASON\t%s" % reason.strip().replace("\t", " ").replace("\n", " "))
|
||||
sys.exit(0)
|
||||
|
||||
if paths is None:
|
||||
sys.stderr.write(
|
||||
'states no decision — needs "json_paths", '
|
||||
'or "json_check": "none" with a "reason"'
|
||||
)
|
||||
sys.exit(2)
|
||||
|
||||
clean_list("json_paths", paths)
|
||||
if not paths:
|
||||
sys.stderr.write('"json_paths" must not be empty')
|
||||
sys.exit(2)
|
||||
clean_list("allow_invalid_json", allow)
|
||||
|
||||
print("MODE\tcheck")
|
||||
for item in paths:
|
||||
print("JPATH\t%s" % item)
|
||||
for item in allow:
|
||||
print("ALLOW\t%s" % item)
|
||||
PY
|
||||
|
||||
resolve_json_config() {
|
||||
local cfg out status=0
|
||||
cfg="$(git rev-parse --show-toplevel)/$CONFIG_FILE"
|
||||
|
||||
if [[ ! -f "$cfg" ]]; then
|
||||
CFG_MODE="absent"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# A malformed config must REFUSE, never degrade to "treat as absent". That
|
||||
# fallback would rebuild precisely the fail-open this gate closes: a typo in
|
||||
# the config would silently disable the check it was written to enable.
|
||||
out="$(python3 -c "$CONFIG_PARSER" "$cfg" 2>&1)" || status=$?
|
||||
|
||||
if (( status != 0 )); then
|
||||
fail "$EX_CONFIG" \
|
||||
"$CONFIG_FILE is INVALID — $out" \
|
||||
"" \
|
||||
"Refusing to run. A config that does not parse is not the same as no" \
|
||||
"config: treating it as absent would silently disable the very check" \
|
||||
"this file was written to enable."
|
||||
fi
|
||||
|
||||
local key val
|
||||
while IFS=$'\t' read -r key val; do
|
||||
case "$key" in
|
||||
MODE) CFG_MODE="$val" ;;
|
||||
REASON) CFG_REASON="$val" ;;
|
||||
JPATH) CFG_PATHS+=("$val") ;;
|
||||
ALLOW) CFG_ALLOW+=("$val") ;;
|
||||
esac
|
||||
done <<<"$out"
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# THE ASYMMETRY, ENFORCED RATHER THAN INTENDED.
|
||||
# ON may come from anywhere: turning a check on needs no review.
|
||||
# OFF must come from a COMMITTED object, because the entire argument for
|
||||
# requiring a written reason was that a reason is reviewable in a diff
|
||||
# while an absence is not. An opt-out honoured from an untracked
|
||||
# working-tree file is not reviewable either -- it is an ad-hoc local
|
||||
# bypass wearing a config file's clothes, and it defeats the design.
|
||||
# So the OFF decision is re-read from HEAD and the working tree gets no
|
||||
# vote in it. A committed ON flipped to OFF locally is likewise refused.
|
||||
# ------------------------------------------------------------------
|
||||
[[ "$CFG_MODE" == "none" ]] || return 0
|
||||
|
||||
local rel cmode cblob cout cstatus=0
|
||||
rel="$(git ls-files --full-name --error-unmatch -- "$cfg" 2>/dev/null)" || rel=""
|
||||
if [[ -z "$rel" ]]; then
|
||||
fail "$EX_CONFIG" \
|
||||
"OPT-OUT IS NOT REVIEWABLE — $CONFIG_FILE is not tracked in git" \
|
||||
"recorded reason was: ${CFG_REASON:-(none)}" \
|
||||
"" \
|
||||
"The opt-out requires a written reason precisely so it shows up in a" \
|
||||
"diff. An untracked file shows up in nobody's review, so honouring it" \
|
||||
"would rebuild the unreviewable bypass this design exists to prevent." \
|
||||
"" \
|
||||
"Commit $CONFIG_FILE, then run again. Turning the check ON needs no" \
|
||||
"commit; only turning it OFF does."
|
||||
fi
|
||||
|
||||
# A COMMITTED SYMLINK (mode 120000) is a mutable target: the reviewed blob
|
||||
# is a path, and what it points at can change without any diff at all.
|
||||
cmode="$(git ls-tree HEAD -- "$rel" 2>/dev/null | awk '{print $1}')"
|
||||
if [[ -z "$cmode" ]]; then
|
||||
fail "$EX_CONFIG" \
|
||||
"OPT-OUT IS NOT REVIEWABLE — $CONFIG_FILE is staged but not yet in HEAD" \
|
||||
"recorded reason was: ${CFG_REASON:-(none)}" \
|
||||
"" \
|
||||
"A staged-but-uncommitted opt-out has not been through review either." \
|
||||
"Commit it first."
|
||||
fi
|
||||
if [[ "$cmode" == "120000" ]]; then
|
||||
fail "$EX_CONFIG" \
|
||||
"OPT-OUT IS NOT REVIEWABLE — $CONFIG_FILE is a committed SYMLINK" \
|
||||
"" \
|
||||
"The reviewed object would be a path, not a decision: what it points" \
|
||||
"at can be changed later without producing any diff. Commit the" \
|
||||
"config as a regular file."
|
||||
fi
|
||||
|
||||
cblob="$(git show "HEAD:$rel" 2>/dev/null)" || cblob=""
|
||||
cout="$(printf '%s' "$cblob" | python3 -c "$CONFIG_PARSER" /dev/stdin 2>&1)" || cstatus=$?
|
||||
if (( cstatus != 0 )); then
|
||||
fail "$EX_CONFIG" \
|
||||
"COMMITTED $CONFIG_FILE is INVALID — $cout" \
|
||||
"" \
|
||||
"The working tree opts out, but the committed version does not parse," \
|
||||
"so there is no reviewable decision to honour."
|
||||
fi
|
||||
local cmode_val="" creason=""
|
||||
while IFS=$'\t' read -r key val; do
|
||||
case "$key" in
|
||||
MODE) cmode_val="$val" ;;
|
||||
REASON) creason="$val" ;;
|
||||
esac
|
||||
done <<<"$cout"
|
||||
|
||||
if [[ "$cmode_val" != "none" ]]; then
|
||||
fail "$EX_CONFIG" \
|
||||
"LOCAL-ONLY OPT-OUT — the working tree turns the JSON check OFF but" \
|
||||
"HEAD does not (committed decision: $cmode_val)" \
|
||||
"working-tree reason: ${CFG_REASON:-(none)}" \
|
||||
"" \
|
||||
"An uncommitted edit that disables a committed check is exactly the" \
|
||||
"unreviewable bypass this guard refuses. Commit the change if it is" \
|
||||
"real; revert it if it was a local convenience."
|
||||
fi
|
||||
|
||||
# Print the COMMITTED reason, never the working tree's: the reason anyone
|
||||
# can actually review is the one in the object.
|
||||
CFG_REASON="$creason"
|
||||
}
|
||||
|
||||
check_staged_json() {
|
||||
local -a checked=() skipped=()
|
||||
local f err
|
||||
local -a pathspec=()
|
||||
|
||||
resolve_json_config
|
||||
|
||||
if (( ${#JSON_PATHS[@]} == 0 )); then
|
||||
case "$CFG_MODE" in
|
||||
absent)
|
||||
fail "$EX_CONFIG" \
|
||||
"NO JSON DECISION — no --json-path, and no $CONFIG_FILE" \
|
||||
"" \
|
||||
"This guard will not run without an explicit decision about" \
|
||||
"staged-JSON validation. Create $CONFIG_FILE with EITHER:" \
|
||||
"" \
|
||||
' {"json_paths": ["data/**/*.json"]}' \
|
||||
"" \
|
||||
"or, if this repo genuinely has no generated JSON:" \
|
||||
"" \
|
||||
' {"json_check": "none", "reason": "<why>"}' \
|
||||
"" \
|
||||
"The reason is mandatory so the opt-out is recorded and" \
|
||||
"reviewable in the diff, instead of being an absence nobody sees."
|
||||
;;
|
||||
none)
|
||||
log " -- JSON check OPTED OUT in $CONFIG_FILE — recorded reason: $CFG_REASON"
|
||||
return 0
|
||||
;;
|
||||
check)
|
||||
JSON_PATHS=(${CFG_PATHS[@]+"${CFG_PATHS[@]}"})
|
||||
ALLOW_INVALID_JSON+=(${CFG_ALLOW[@]+"${CFG_ALLOW[@]}"})
|
||||
log " .. JSON paths from $CONFIG_FILE: ${JSON_PATHS[*]}"
|
||||
;;
|
||||
esac
|
||||
else
|
||||
# An explicit --json-path turns the check ON, so it satisfies the decision
|
||||
# requirement by itself. If the config opted out, the nomination WINS and
|
||||
# says so loudly — resolving a contradiction toward more checking is the
|
||||
# only safe direction, but silently ignoring a committed opt-out would
|
||||
# hide a real disagreement.
|
||||
case "$CFG_MODE" in
|
||||
none)
|
||||
log " !! $CONFIG_FILE opts out of the JSON check, but --json-path was"
|
||||
log " !! given — honouring the nomination and checking anyway."
|
||||
;;
|
||||
check)
|
||||
JSON_PATHS+=(${CFG_PATHS[@]+"${CFG_PATHS[@]}"})
|
||||
ALLOW_INVALID_JSON+=(${CFG_ALLOW[@]+"${CFG_ALLOW[@]}"})
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
|
||||
# Normalize to :(glob) magic. WITHOUT it, git's default pathspec matching
|
||||
# makes 'data/**/*.json' require at least one intermediate directory, so it
|
||||
# matches data/sub/b.json but SILENTLY SKIPS data/a.json. The obvious
|
||||
# spelling would then deliver partial coverage with no warning — a
|
||||
# quiet-underscan, which is the same family of defect as a quiet pass.
|
||||
# Pathspecs that already carry explicit magic (leading ':') are left alone.
|
||||
for f in "${JSON_PATHS[@]}"; do
|
||||
[[ "$f" == :* ]] && pathspec+=("$f") || pathspec+=(":(glob)$f")
|
||||
done
|
||||
for f in ${ALLOW_INVALID_JSON[@]+"${ALLOW_INVALID_JSON[@]}"}; do
|
||||
[[ "$f" == :* ]] && pathspec+=("$f") || pathspec+=(":(exclude,glob)$f")
|
||||
skipped+=("$f")
|
||||
done
|
||||
|
||||
local -a files=()
|
||||
staged_files_z files "${pathspec[@]}"
|
||||
|
||||
for f in ${files[@]+"${files[@]}"}; do
|
||||
[[ "$f" == *.json ]] || continue
|
||||
|
||||
if ! err="$(git show ":$f" | python3 -c '
|
||||
import json, sys
|
||||
try:
|
||||
json.load(sys.stdin)
|
||||
except Exception as exc:
|
||||
sys.stderr.write(str(exc))
|
||||
sys.exit(1)
|
||||
' 2>&1)"; then
|
||||
fail "$EX_JSON" \
|
||||
"staged JSON does not parse: $f" \
|
||||
"$err" \
|
||||
"" \
|
||||
"A serialization error from a generator is a STOP, not something to commit past."
|
||||
fi
|
||||
checked+=("$f")
|
||||
done
|
||||
|
||||
for f in ${skipped[@]+"${skipped[@]}"}; do
|
||||
log " -- JSON check EXEMPTED by --allow-invalid-json: $f"
|
||||
done
|
||||
if (( ${#checked[@]} == 0 )); then
|
||||
log " -- no staged .json under ${JSON_PATHS[*]} — JSON check not applicable"
|
||||
else
|
||||
log " ok ${#checked[@]} staged .json file(s) under ${JSON_PATHS[*]} parse"
|
||||
fi
|
||||
}
|
||||
|
||||
cmd_check_staged() {
|
||||
require_repo
|
||||
log "push-guard: checking staged content"
|
||||
|
||||
check_unmerged
|
||||
|
||||
# --no-renames is load-bearing, NOT cosmetic. Git detects renames by default
|
||||
# (diff.renames=true since 2.9), so a `git mv` plus a small edit is reported
|
||||
# as ONE 'R' entry — which --diff-filter=ACM DOES NOT MATCH. The renamed file
|
||||
# becomes invisible to every content check: staged conflict markers sail
|
||||
# through and the guard prints "staged content OK". Verified reproducible at
|
||||
# 97% similarity with an ordinary co-staged file present. --no-renames
|
||||
# decomposes each rename back into D + A so the new path is always seen.
|
||||
# (Adding R to the filter also works, but --name-only -z emits two paths for
|
||||
# a rename and is ambiguous to parse — this removes the category instead.)
|
||||
#
|
||||
# SECOND INSTANCE OF THE SAME DEFECT, found by sweeping for the construct
|
||||
# rather than fixing only the one that was reported. This one failed CLOSED
|
||||
# (zero files hit "NOTHING IS STAGED"), so it was never a security hole --
|
||||
# but it reported a confident WRONG DIAGNOSIS, sending anyone debugging it
|
||||
# at their index instead of at the failing git invocation.
|
||||
local -a files=()
|
||||
staged_files_z files
|
||||
if (( ${#files[@]} == 0 )); then
|
||||
fail "$EX_NULL" \
|
||||
"NOTHING IS STAGED" \
|
||||
"About to commit, but the index is identical to HEAD." \
|
||||
"" \
|
||||
"This is the null case: a commit here is empty, and any later" \
|
||||
"'verified' step would be asserting against content that does not exist."
|
||||
fi
|
||||
|
||||
check_conflict_markers "${files[@]}"
|
||||
check_staged_json "${files[@]}"
|
||||
|
||||
log "push-guard: staged content OK"
|
||||
}
|
||||
|
||||
# ------------------------------------------------------------------------ push
|
||||
|
||||
remote_sha() {
|
||||
# Empty output means the branch does not exist on the remote yet.
|
||||
# A network/auth failure makes ls-remote non-zero, which set -e turns into an
|
||||
# abort — the guard never treats an unreachable remote as "unchanged".
|
||||
git ls-remote "$1" "refs/heads/$2" | awk 'NR==1{print $1}'
|
||||
}
|
||||
|
||||
cmd_push() {
|
||||
require_repo
|
||||
local remote="" branch="" since_head=""
|
||||
local -a extra=()
|
||||
|
||||
while (( $# )); do
|
||||
case "$1" in
|
||||
--remote) remote="${2:-}"; shift 2 ;;
|
||||
--branch) branch="${2:-}"; shift 2 ;;
|
||||
--since-head) since_head="${2:-}"; shift 2 ;;
|
||||
--) shift; extra=("$@"); break ;;
|
||||
*) fail "$EX_USAGE" "unknown argument to push: $1" ;;
|
||||
esac
|
||||
done
|
||||
[[ -n "$remote" && -n "$branch" ]] \
|
||||
|| fail "$EX_USAGE" "push requires --remote and --branch"
|
||||
|
||||
local head
|
||||
head="$(git rev-parse HEAD)"
|
||||
log "push-guard: verifying push of $head -> $remote/$branch"
|
||||
|
||||
# (1) Did a new commit actually get created?
|
||||
if [[ -n "$since_head" ]]; then
|
||||
git rev-parse --verify --quiet "${since_head}^{commit}" >/dev/null \
|
||||
|| fail "$EX_USAGE" \
|
||||
"--since-head is not a commit in this repository: $since_head" \
|
||||
"" \
|
||||
"Cannot verify advancement against a start point that does not exist."
|
||||
|
||||
if [[ "$head" == "$since_head" ]]; then
|
||||
fail "$EX_NULL" \
|
||||
"HEAD DID NOT ADVANCE — no commit was created" \
|
||||
"HEAD is still $head" \
|
||||
"" \
|
||||
"Your commit step failed and execution continued on stale state." \
|
||||
"Pushing now would publish something you did not just build."
|
||||
fi
|
||||
# INEQUALITY IS NOT ADVANCEMENT. "different from where I started" is
|
||||
# satisfied by checking out any unrelated commit -- including one that
|
||||
# existed long before this session -- which would let pre-existing work
|
||||
# be published as something just built. The claim being made is that new
|
||||
# commits were created ON TOP OF the recorded start point, and only
|
||||
# ancestry states that.
|
||||
if ! git merge-base --is-ancestor "$since_head" "$head"; then
|
||||
fail "$EX_NULL" \
|
||||
"HEAD IS NOT A DESCENDANT of the recorded start point" \
|
||||
"start: $since_head" \
|
||||
"head : $head" \
|
||||
"" \
|
||||
"HEAD differs from the start point but does not build on it, so" \
|
||||
"this is a checkout of other history rather than work you just" \
|
||||
"created. Being different is not the same as having advanced."
|
||||
fi
|
||||
log " ok HEAD advanced ${since_head:0:9} -> ${head:0:9} (descendant)"
|
||||
fi
|
||||
|
||||
# (2) Is that commit non-empty?
|
||||
# EXEMPTING BY PARENT COUNT WAS A FAIL-OPEN. An empty root commit sailed
|
||||
# through "emptiness check not applicable" and was then reported as a
|
||||
# CONFIRMED PUSH -- directly contradicting the non-empty requirement this
|
||||
# step exists to enforce. Root and merge commits do have well-defined
|
||||
# emptiness; the original code declined to define it, which is not the same
|
||||
# as it being undefined.
|
||||
local parents
|
||||
parents="$(git rev-list --parents -n 1 HEAD | wc -w)"
|
||||
if (( parents == 2 )); then # sha + exactly one parent
|
||||
if git diff --quiet HEAD^ HEAD; then
|
||||
fail "$EX_NULL" \
|
||||
"HEAD IS AN EMPTY COMMIT — it changes nothing against its parent" \
|
||||
"commit $head" \
|
||||
"" \
|
||||
"An empty commit carrying a real message is indistinguishable" \
|
||||
"from real work in the log. Refusing to push it."
|
||||
fi
|
||||
log " ok HEAD is a non-empty commit"
|
||||
elif (( parents > 2 )); then
|
||||
# A merge is empty when its tree matches EVERY parent: it then carries
|
||||
# no content of its own and no integration either.
|
||||
local p all_same=yes
|
||||
for p in $(git rev-list --parents -n 1 HEAD | cut -d' ' -f2-); do
|
||||
git diff --quiet "$p" HEAD || { all_same=no; break; }
|
||||
done
|
||||
if [[ "$all_same" == yes ]]; then
|
||||
fail "$EX_NULL" \
|
||||
"HEAD IS AN EMPTY MERGE — its tree is identical to every parent" \
|
||||
"commit $head" \
|
||||
"" \
|
||||
"It integrates nothing and introduces nothing. Refusing to push it."
|
||||
fi
|
||||
log " ok HEAD is a non-empty merge commit"
|
||||
else
|
||||
# A root commit has no parent, so emptiness is measured against the
|
||||
# empty tree: it is empty when it adds no paths at all.
|
||||
if [[ -z "$(git diff-tree --root -r --name-only --no-commit-id HEAD)" ]]; then
|
||||
fail "$EX_NULL" \
|
||||
"HEAD IS AN EMPTY ROOT COMMIT — it introduces no files" \
|
||||
"commit $head" \
|
||||
"" \
|
||||
"A root commit is empty when it adds nothing against the empty" \
|
||||
"tree. Refusing to push it."
|
||||
fi
|
||||
log " ok HEAD is a non-empty root commit"
|
||||
fi
|
||||
|
||||
# (3) Capture the remote BEFORE. This is the step whose absence made the
|
||||
# original "PUSH VERIFIED" a false positive.
|
||||
local before after
|
||||
before="$(remote_sha "$remote" "$branch")"
|
||||
log " .. remote before: ${before:-<branch does not exist>}"
|
||||
|
||||
git push "$remote" "HEAD:refs/heads/$branch" ${extra[@]+"${extra[@]}"}
|
||||
|
||||
after="$(remote_sha "$remote" "$branch")"
|
||||
log " .. remote after: ${after:-<absent>}"
|
||||
|
||||
# (4) The remote must now equal local HEAD...
|
||||
if [[ "$after" != "$head" ]]; then
|
||||
fail "$EX_PUSH" \
|
||||
"REMOTE DOES NOT MATCH LOCAL HEAD after push" \
|
||||
"local HEAD: $head" \
|
||||
"remote $branch: ${after:-<absent>}" \
|
||||
"" \
|
||||
"The push did not land what you are holding."
|
||||
fi
|
||||
# (5) ...AND it must have MOVED to get there. Without this, a push that
|
||||
# transferred nothing passes step (4) trivially.
|
||||
if [[ "$after" == "$before" ]]; then
|
||||
fail "$EX_PUSH" \
|
||||
"REMOTE DID NOT MOVE — nothing was actually pushed" \
|
||||
"remote was already at $after before this push ran" \
|
||||
"" \
|
||||
"'remote == local' is satisfied by having pushed nothing. That is" \
|
||||
"the false positive this guard exists to catch: the work you think" \
|
||||
"you just published was already there, or was never committed."
|
||||
fi
|
||||
|
||||
log "push-guard: PUSH CONFIRMED ${before:0:9}${before:+ }-> ${after:0:9}"
|
||||
}
|
||||
|
||||
# ------------------------------------------------------------------------ main
|
||||
|
||||
ALLOW_INVALID_JSON=()
|
||||
JSON_PATHS=()
|
||||
|
||||
CFG_MODE="absent"
|
||||
CFG_REASON=""
|
||||
CFG_PATHS=()
|
||||
CFG_ALLOW=()
|
||||
|
||||
main() {
|
||||
(( $# )) || { usage; exit "$EX_USAGE"; }
|
||||
local sub="$1"; shift
|
||||
case "$sub" in
|
||||
check-staged)
|
||||
while (( $# )); do
|
||||
case "$1" in
|
||||
--json-path) JSON_PATHS+=("${2:-}"); shift 2 ;;
|
||||
--allow-invalid-json) ALLOW_INVALID_JSON+=("${2:-}"); shift 2 ;;
|
||||
*) fail "$EX_USAGE" "unknown argument to check-staged: $1" ;;
|
||||
esac
|
||||
done
|
||||
cmd_check_staged
|
||||
;;
|
||||
push) cmd_push "$@" ;;
|
||||
-h|--help) usage ;;
|
||||
*) usage; exit "$EX_USAGE" ;;
|
||||
esac
|
||||
# Explicit: the only way to reach here is with every requested check passed.
|
||||
exit "$EX_OK"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
@@ -1,659 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# test-push-guard.sh - Needle harness for push-guard.sh.
|
||||
#
|
||||
# Every check gets BOTH polarities:
|
||||
# NEEDLE a deliberately-broken fixture that MUST trip the guard.
|
||||
# CONTROL a clean fixture that MUST pass.
|
||||
#
|
||||
# The controls are not decoration. A guard that failed unconditionally would
|
||||
# satisfy every needle and look fully covered — which is the same class of defect
|
||||
# (an assertion satisfied by the null case) that push-guard.sh exists to prevent.
|
||||
# A needle set without controls cannot tell "working guard" from "broken guard".
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
GUARD="$SCRIPT_DIR/push-guard.sh"
|
||||
WORK_DIR="${MOSAIC_TEST_WORK_DIR:-$SCRIPT_DIR/.work}"
|
||||
|
||||
PASS=0
|
||||
FAIL=0
|
||||
|
||||
# Fresh repo + bare "remote" for each case, so no case can inherit another's state.
|
||||
new_repo() {
|
||||
local name="$1"
|
||||
local dir="$WORK_DIR/$name"
|
||||
rm -rf "$dir"
|
||||
mkdir -p "$dir"
|
||||
git init -q -b main "$dir/repo"
|
||||
git init -q --bare "$dir/remote.git"
|
||||
git -C "$dir/repo" remote add origin "$dir/remote.git"
|
||||
git -C "$dir/repo" config user.name "Needle Harness"
|
||||
git -C "$dir/repo" config user.email "[email protected]"
|
||||
printf 'seed\n' > "$dir/repo/seed.txt"
|
||||
git -C "$dir/repo" add seed.txt
|
||||
git -C "$dir/repo" commit -q -m "seed"
|
||||
printf '%s' "$dir/repo"
|
||||
}
|
||||
|
||||
# write_config <repo> <json-text>
|
||||
#
|
||||
# THIS HELPER USED TO WRITE THE CONFIG UNTRACKED, and the comment that lived here
|
||||
# justified it ("does not need to be staged"). That made every opt-out control in
|
||||
# this file assert the FORBIDDEN provenance and pass — a control that does not
|
||||
# merely test nothing, but tests the OPPOSITE of the requirement and goes green.
|
||||
# The whole design is: ON from anywhere, OFF only from a committed reviewable
|
||||
# artifact. A harness that opts out from an untracked file was proving the bypass
|
||||
# worked. It now COMMITS, so the opt-out controls exercise the supported path.
|
||||
write_config() {
|
||||
printf '%s\n' "$2" > "$1/.push-guard.json"
|
||||
git -C "$1" add .push-guard.json
|
||||
git -C "$1" commit -q -m "push-guard config"
|
||||
}
|
||||
|
||||
# write_config_untracked <repo> <json-text>
|
||||
# Deliberately NOT committed — used only where the untracked config is the thing
|
||||
# under test and the expected outcome is a REFUSAL.
|
||||
write_config_untracked() {
|
||||
printf '%s\n' "$2" > "$1/.push-guard.json"
|
||||
}
|
||||
|
||||
# expect <kind> <expected_exit> <description> [--out <substring>] -- <command...>
|
||||
#
|
||||
# --out asserts a substring of the guard's own output. It exists because exit 0
|
||||
# alone cannot distinguish "checked the files and they were fine" from "matched
|
||||
# no files and had nothing to check". Two controls in an earlier revision of this
|
||||
# harness were passing vacuously for exactly that reason — the pathspec silently
|
||||
# matched nothing. A control that cannot fail is not a control.
|
||||
expect() {
|
||||
local kind="$1" want="$2" desc="$3"; shift 3
|
||||
local need_out=""
|
||||
while (( $# )); do
|
||||
case "$1" in
|
||||
--out) need_out="$2"; shift 2 ;;
|
||||
--) shift; break ;;
|
||||
*) break ;;
|
||||
esac
|
||||
done
|
||||
local got=0 out
|
||||
out="$("$@" 2>&1)" || got=$?
|
||||
|
||||
local why=""
|
||||
[[ "$got" == "$want" ]] || why="wanted exit $want, got $got"
|
||||
if [[ -z "$why" && -n "$need_out" && "$out" != *"$need_out"* ]]; then
|
||||
why="exit $got as expected, but output never said: $need_out"
|
||||
fi
|
||||
|
||||
if [[ -z "$why" ]]; then
|
||||
printf ' PASS [%-7s] %s (exit %s)\n' "$kind" "$desc" "$got"
|
||||
PASS=$((PASS + 1))
|
||||
else
|
||||
printf ' FAIL [%-7s] %s — %s\n' "$kind" "$desc" "$why"
|
||||
printf '%s\n' "$out" | sed 's/^/ | /'
|
||||
FAIL=$((FAIL + 1))
|
||||
fi
|
||||
}
|
||||
|
||||
rm -rf "$WORK_DIR"
|
||||
mkdir -p "$WORK_DIR"
|
||||
|
||||
echo "=== (a) conflict markers / unmerged index ==="
|
||||
|
||||
# NEEDLE: staged content carrying real conflict markers.
|
||||
R="$(new_repo a1)"
|
||||
cat > "$R/conflicted.txt" <<'EOF'
|
||||
intro line
|
||||
<<<<<<< HEAD
|
||||
ours
|
||||
=======
|
||||
theirs
|
||||
>>>>>>> feature/x
|
||||
tail line
|
||||
EOF
|
||||
git -C "$R" add conflicted.txt
|
||||
expect NEEDLE 2 "staged conflict markers are refused" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# NEEDLE: a genuine unmerged index (merge in progress).
|
||||
R="$(new_repo a2)"
|
||||
git -C "$R" checkout -q -b other
|
||||
printf 'from-other\n' > "$R/clash.txt"
|
||||
git -C "$R" add clash.txt && git -C "$R" commit -q -m other
|
||||
git -C "$R" checkout -q main
|
||||
printf 'from-main\n' > "$R/clash.txt"
|
||||
git -C "$R" add clash.txt && git -C "$R" commit -q -m main
|
||||
git -C "$R" merge other -q >/dev/null 2>&1 || true
|
||||
expect NEEDLE 2 "unmerged index paths are refused" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# CONTROL: ordinary staged content passes.
|
||||
# The config is REQUIRED as of the decision gate: this fixture has no generated
|
||||
# JSON, so it records that fact with a reason. This is the migration cost of the
|
||||
# hard cutover, paid here first — these two controls were the only pre-existing
|
||||
# cases that reached the JSON stage without nominating a path, and they are
|
||||
# exactly the "repo that never wired it up" the gate now refuses.
|
||||
R="$(new_repo a3)"
|
||||
write_config "$R" '{"json_check": "none", "reason": "fixture has no generated JSON"}'
|
||||
printf 'just some code\n' > "$R/clean.txt"
|
||||
git -C "$R" add clean.txt
|
||||
expect CONTROL 0 "clean staged content passes (1 file actually scanned)" \
|
||||
--out "no conflict markers in 1 staged file(s)" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# CONTROL (false-positive guard): prose using ======= as an underline must NOT trip.
|
||||
# This is why the marker pattern deliberately ignores a bare '======='.
|
||||
R="$(new_repo a4)"
|
||||
write_config "$R" '{"json_check": "none", "reason": "fixture has no generated JSON"}'
|
||||
cat > "$R/README.rst" <<'EOF'
|
||||
Section Title
|
||||
=============
|
||||
|
||||
Body text.
|
||||
|
||||
Another Heading
|
||||
=======
|
||||
EOF
|
||||
git -C "$R" add README.rst
|
||||
expect CONTROL 0 "prose using ======= underlines does not trip the guard" \
|
||||
--out "no conflict markers in 1 staged file(s)" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# NEEDLE (fault injection): if the scan itself ERRORS, the guard must refuse
|
||||
# rather than report a clean result. git grep exits 1 for "no match" but >=2 for
|
||||
# a real failure; a blanket '|| true' would collapse the two. We inject the
|
||||
# failure with a PATH shim rather than trying to corrupt an index.
|
||||
R="$(new_repo a5)"
|
||||
SHIM="$WORK_DIR/a5/bin"
|
||||
mkdir -p "$SHIM"
|
||||
REAL_GIT="$(command -v git)"
|
||||
cat > "$SHIM/git" <<SH
|
||||
#!/usr/bin/env bash
|
||||
if [[ "\$1" == "grep" ]]; then exit 2; fi
|
||||
exec "$REAL_GIT" "\$@"
|
||||
SH
|
||||
chmod +x "$SHIM/git"
|
||||
printf 'ordinary content\n' > "$R/thing.txt"
|
||||
git -C "$R" add thing.txt
|
||||
expect NEEDLE 2 "a conflict scan that ERRORS is refused, not reported clean" \
|
||||
--out "did not run" -- \
|
||||
bash -c "cd '$R' && export PATH=\"$SHIM:\$PATH\" && '$GUARD' check-staged"
|
||||
|
||||
# NEEDLE: RENAME + MODIFY. Git reports a `git mv` plus a small edit as a single
|
||||
# 'R' entry, which --diff-filter=ACM does not match, making the file invisible to
|
||||
# every content check. A clean file is co-staged deliberately: without it the
|
||||
# file list is empty and the nothing-staged guard fires by ACCIDENT, which would
|
||||
# make this needle pass for the wrong reason and hide the real defect.
|
||||
R="$(new_repo a6)"
|
||||
mkdir -p "$R/data"
|
||||
seq 1 200 | sed 's/^/line /' > "$R/data/canon.txt"
|
||||
printf 'original\n' > "$R/other.txt"
|
||||
git -C "$R" add -A && git -C "$R" commit -q -m seed
|
||||
git -C "$R" mv data/canon.txt data/canon2.txt
|
||||
printf '<<<<<<< HEAD\nours\n=======\ntheirs\n>>>>>>> b\n' >> "$R/data/canon2.txt"
|
||||
printf 'an ordinary innocent change\n' > "$R/other.txt"
|
||||
git -C "$R" add -A
|
||||
expect NEEDLE 2 "conflict markers in a RENAMED file are refused (rename+modify)" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# NEEDLE: a path marked binary in .gitattributes. `git grep -I` honours that
|
||||
# classification and skips the blob entirely while the summary still counts the
|
||||
# file as scanned — a clean pass AND a false count. Hence -a, not -I.
|
||||
R="$(new_repo a7)"
|
||||
printf 'notes.txt binary\n' > "$R/.gitattributes"
|
||||
printf 'x\n<<<<<<< HEAD\nours\n=======\ntheirs\n>>>>>>> b\ny\n' > "$R/notes.txt"
|
||||
git -C "$R" add -A
|
||||
expect NEEDLE 2 "conflict markers in a .gitattributes-binary file are refused" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
echo "=== (c) staged JSON parses ==="
|
||||
|
||||
# NEEDLE: malformed JSON, the shape a broken generator emits.
|
||||
R="$(new_repo c1)"
|
||||
mkdir -p "$R/data"
|
||||
printf '{"a": 1,\n' > "$R/data/broken.json"
|
||||
git -C "$R" add data/broken.json
|
||||
expect NEEDLE 3 "staged unparseable JSON under --json-path is refused" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged --json-path 'data/**/*.json'"
|
||||
|
||||
# NEEDLE: conflict markers inside a generated JSON file — the 70-file incident.
|
||||
R="$(new_repo c2)"
|
||||
mkdir -p "$R/data"
|
||||
cat > "$R/data/canon.json" <<'EOF'
|
||||
{
|
||||
<<<<<<< HEAD
|
||||
"v": 1
|
||||
=======
|
||||
"v": 2
|
||||
>>>>>>> main
|
||||
}
|
||||
EOF
|
||||
git -C "$R" add data/canon.json
|
||||
expect NEEDLE 2 "conflict markers in generated JSON are refused" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# CONTROL: valid JSON passes.
|
||||
R="$(new_repo c3)"
|
||||
mkdir -p "$R/data"
|
||||
printf '{"a": 1, "b": [2, 3]}\n' > "$R/data/good.json"
|
||||
git -C "$R" add data/good.json
|
||||
expect CONTROL 0 "valid staged JSON passes (and was actually parsed)" \
|
||||
--out "1 staged .json file(s)" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged --json-path 'data/**/*.json'"
|
||||
|
||||
# CONTROL: an explicit exemption works (and is reported, never silent).
|
||||
R="$(new_repo c4)"
|
||||
mkdir -p "$R/fixtures"
|
||||
printf 'not json at all' > "$R/fixtures/malformed.json"
|
||||
git -C "$R" add fixtures/malformed.json
|
||||
expect CONTROL 0 "deliberate malformed fixture can be exempted (exemption announced)" \
|
||||
--out "EXEMPTED by --allow-invalid-json" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged --json-path 'fixtures/*' --allow-invalid-json 'fixtures/*'"
|
||||
|
||||
# CONTROL (false-positive regression): JSONC is the common real-world case.
|
||||
# tsconfig.json legally carries comments and does NOT parse strictly. Measured on
|
||||
# a real repo: 17 of 119 tracked .json files are like this. An on-by-default
|
||||
# check would fire on all of them; scoping by --json-path is what prevents it.
|
||||
R="$(new_repo c5)"
|
||||
cat > "$R/tsconfig.json" <<'EOF'
|
||||
{
|
||||
// JSONC: comments are legal here and strict json.load() rejects them.
|
||||
"compilerOptions": { "strict": true }
|
||||
}
|
||||
EOF
|
||||
mkdir -p "$R/data"
|
||||
printf '{"generated": true}\n' > "$R/data/view.json"
|
||||
git -C "$R" add tsconfig.json data/view.json
|
||||
expect CONTROL 0 "JSONC outside --json-path does not trip the JSON check" \
|
||||
--out "1 staged .json file(s)" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged --json-path 'data/**/*.json'"
|
||||
|
||||
# NEEDLE: the same tsconfig DOES fail if someone wrongly nominates it, proving
|
||||
# the check is genuinely running and the control above is not a vacuous pass.
|
||||
expect NEEDLE 3 "the same JSONC file fails when explicitly nominated" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged --json-path 'tsconfig.json'"
|
||||
|
||||
echo "=== (d) the JSON decision is MANDATORY (.push-guard.json) ==="
|
||||
|
||||
# NEEDLE: the fail-open this gate closes. No --json-path, no config: the previous
|
||||
# release printed "JSON check NOT REQUESTED" and exited 0, leaving the repo
|
||||
# unprotected forever with nothing ever failing.
|
||||
R="$(new_repo d1)"
|
||||
printf '{"a": 1}\n' > "$R/thing.json"
|
||||
git -C "$R" add thing.json
|
||||
expect NEEDLE 6 "no --json-path and no config is REFUSED (the closed fail-open)" \
|
||||
--out "NO JSON DECISION" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# NEEDLE: config nominates paths, and the check genuinely runs off it.
|
||||
R="$(new_repo d2)"
|
||||
write_config "$R" '{"json_paths": ["data/**/*.json"]}'
|
||||
mkdir -p "$R/data"
|
||||
printf '{"a": 1,\n' > "$R/data/broken.json"
|
||||
git -C "$R" add data/broken.json
|
||||
expect NEEDLE 3 "broken JSON is caught via config-supplied json_paths" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# NEEDLE: opt-out WITHOUT a reason. An unexplained opt-out is the absence again,
|
||||
# merely relocated into a file.
|
||||
R="$(new_repo d3)"
|
||||
write_config "$R" '{"json_check": "none"}'
|
||||
printf 'code\n' > "$R/x.txt"
|
||||
git -C "$R" add x.txt
|
||||
expect NEEDLE 6 "json_check:none without a reason is refused" \
|
||||
--out "REQUIRES a non-empty" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# NEEDLE: malformed config must REFUSE, never fall back to "treat as absent".
|
||||
# That fallback would mean a typo silently disables the check the file enables.
|
||||
R="$(new_repo d4)"
|
||||
write_config "$R" '{"json_paths": ["data/**/*.json",'
|
||||
printf 'code\n' > "$R/x.txt"
|
||||
git -C "$R" add x.txt
|
||||
expect NEEDLE 6 "an unparseable config is refused, not treated as absent" \
|
||||
--out "INVALID" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# NEEDLE: a config that parses but states NO decision. Distinct from malformed —
|
||||
# this one is valid JSON and still says nothing, which is the original defect
|
||||
# wearing a config file as a disguise.
|
||||
R="$(new_repo d5)"
|
||||
write_config "$R" '{}'
|
||||
printf 'code\n' > "$R/x.txt"
|
||||
git -C "$R" add x.txt
|
||||
expect NEEDLE 6 "a valid config that states no decision is refused" \
|
||||
--out "states no decision" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# NEEDLE: a bogus json_check value must not be read as an opt-out.
|
||||
R="$(new_repo d6)"
|
||||
write_config "$R" '{"json_check": "off", "reason": "typo for none"}'
|
||||
printf 'code\n' > "$R/x.txt"
|
||||
git -C "$R" add x.txt
|
||||
expect NEEDLE 6 'json_check with an unrecognised value is refused' \
|
||||
--out 'must be "none"' -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# CONTROL: config nominates paths, JSON is clean.
|
||||
# --out is REQUIRED. Exit 0 alone cannot distinguish "read the config, resolved
|
||||
# the paths, parsed the files" from "matched nothing and had nothing to do" —
|
||||
# the vacuous-control trap that already caught this harness once.
|
||||
R="$(new_repo d7)"
|
||||
write_config "$R" '{"json_paths": ["data/**/*.json"]}'
|
||||
mkdir -p "$R/data"
|
||||
printf '{"generated": true}\n' > "$R/data/view.json"
|
||||
git -C "$R" add data/view.json
|
||||
expect CONTROL 0 "config-supplied json_paths pass and are reported as parsed" \
|
||||
--out "1 staged .json file(s)" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# CONTROL: a recorded opt-out passes AND the reason is echoed. Asserting on the
|
||||
# reason is the whole point — an opt-out nobody can see is what we just removed.
|
||||
R="$(new_repo d8)"
|
||||
write_config "$R" '{"json_check": "none", "reason": "no generated JSON in this repo"}'
|
||||
printf 'code\n' > "$R/x.txt"
|
||||
git -C "$R" add x.txt
|
||||
expect CONTROL 0 "a recorded opt-out passes and PRINTS its reason" \
|
||||
--out "recorded reason: no generated JSON in this repo" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# CONTROL: config-supplied exemptions still work through the config path.
|
||||
R="$(new_repo d9)"
|
||||
write_config "$R" '{"json_paths": ["fixtures/*"], "allow_invalid_json": ["fixtures/*"]}'
|
||||
mkdir -p "$R/fixtures"
|
||||
printf 'not json at all' > "$R/fixtures/malformed.json"
|
||||
git -C "$R" add fixtures/malformed.json
|
||||
expect CONTROL 0 "config-supplied allow_invalid_json exempts (and announces it)" \
|
||||
--out "EXEMPTED by --allow-invalid-json" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# CONTROL: --json-path alone still satisfies the decision, with no config at all.
|
||||
# This is what keeps every pre-existing caller working: nominating a path IS an
|
||||
# explicit decision. Only saying nothing is refused.
|
||||
R="$(new_repo d10)"
|
||||
mkdir -p "$R/data"
|
||||
printf '{"a": 1}\n' > "$R/data/good.json"
|
||||
git -C "$R" add data/good.json
|
||||
expect CONTROL 0 "--json-path alone satisfies the decision with no config present" \
|
||||
--out "1 staged .json file(s)" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged --json-path 'data/**/*.json'"
|
||||
|
||||
# NEEDLE: a CLI nomination must WIN over a config opt-out, loudly. Resolving a
|
||||
# contradiction toward more checking is the only safe direction, but doing it
|
||||
# silently would hide a genuine disagreement between caller and repo.
|
||||
R="$(new_repo d11)"
|
||||
write_config "$R" '{"json_check": "none", "reason": "claims no generated JSON"}'
|
||||
mkdir -p "$R/data"
|
||||
printf '{"a": 1,\n' > "$R/data/broken.json"
|
||||
git -C "$R" add data/broken.json
|
||||
expect NEEDLE 3 "--json-path overrides a config opt-out and still catches bad JSON" \
|
||||
--out "honouring the nomination" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged --json-path 'data/**/*.json'"
|
||||
|
||||
echo "=== (z) the guard itself emits no stray output ==="
|
||||
|
||||
# CONTROL: the guard must not print interpreter warnings.
|
||||
#
|
||||
# This case exists because a real one shipped: the config parser was an inline
|
||||
# `<<'PY'` heredoc inside a $( ) command substitution, so bash printed
|
||||
# warning: command substitution: 1 unterminated here-document
|
||||
# on EVERY run. Thirty needles and a clean shellcheck all missed it — the
|
||||
# warning went to stderr, and no assertion in this harness looked at output it
|
||||
# had not already been told to expect. It was found only by running the guard
|
||||
# against a real repository.
|
||||
#
|
||||
# The lesson is narrow and worth encoding: asserting on what you EXPECT to see
|
||||
# cannot detect what you never thought to look for. This asserts on the absence
|
||||
# of a whole output class instead.
|
||||
R="$(new_repo z1)"
|
||||
write_config "$R" '{"json_paths": ["data/**/*.json"]}'
|
||||
mkdir -p "$R/data"
|
||||
printf '{"a": 1}\n' > "$R/data/view.json"
|
||||
git -C "$R" add data/view.json
|
||||
z_out="$(cd "$R" && "$GUARD" check-staged 2>&1)"
|
||||
if grep -qiE 'warning:|unterminated|command substitution' <<<"$z_out"; then
|
||||
printf ' FAIL [%-7s] %s\n' "CONTROL" "guard emits interpreter warnings"
|
||||
printf '%s\n' "$z_out" | sed 's/^/ | /'
|
||||
FAIL=$((FAIL + 1))
|
||||
else
|
||||
printf ' PASS [%-7s] %s\n' "CONTROL" "guard runs without emitting any interpreter warning"
|
||||
PASS=$((PASS + 1))
|
||||
fi
|
||||
|
||||
# NEEDLE for the case above: prove the detector can actually fire, otherwise it
|
||||
# is one more assertion that passes because it looked at nothing.
|
||||
if grep -qiE 'warning:|unterminated|command substitution' \
|
||||
<<<"bash: warning: command substitution: 1 unterminated here-document"; then
|
||||
printf ' PASS [%-7s] %s\n' "NEEDLE" "the warning detector fires on a known warning string"
|
||||
PASS=$((PASS + 1))
|
||||
else
|
||||
printf ' FAIL [%-7s] %s\n' "NEEDLE" "the warning detector is dead — it cannot fire"
|
||||
FAIL=$((FAIL + 1))
|
||||
fi
|
||||
|
||||
echo "=== null case: nothing staged ==="
|
||||
|
||||
# NEEDLE: the index equals HEAD. Committing here produces the empty commit.
|
||||
R="$(new_repo n1)"
|
||||
expect NEEDLE 5 "empty index is refused before a commit happens" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
echo "=== (b) push actually happened ==="
|
||||
|
||||
# CONTROL: a real push that moves the remote.
|
||||
R="$(new_repo b1)"
|
||||
printf 'work\n' > "$R/work.txt"
|
||||
git -C "$R" add work.txt && git -C "$R" commit -q -m "real work"
|
||||
expect CONTROL 0 "a push that moves the remote is confirmed" \
|
||||
--out "PUSH CONFIRMED" -- \
|
||||
bash -c "cd '$R' && '$GUARD' push --remote origin --branch main"
|
||||
|
||||
# NEEDLE: THE ORIGINAL FALSE POSITIVE. Remote already equals local HEAD, so the
|
||||
# naive 'remote == local' assertion succeeds while nothing is transferred.
|
||||
R="$(new_repo b2)"
|
||||
printf 'work\n' > "$R/work.txt"
|
||||
git -C "$R" add work.txt && git -C "$R" commit -q -m "real work"
|
||||
git -C "$R" push -q origin HEAD:refs/heads/main
|
||||
expect NEEDLE 4 "second push transferring nothing is refused (remote did not move)" -- \
|
||||
bash -c "cd '$R' && '$GUARD' push --remote origin --branch main"
|
||||
|
||||
# NEEDLE: the commit step aborted, so HEAD never advanced.
|
||||
R="$(new_repo b3)"
|
||||
HEAD_BEFORE="$(git -C "$R" rev-parse HEAD)"
|
||||
expect NEEDLE 5 "push after an aborted commit is refused (HEAD did not advance)" -- \
|
||||
bash -c "cd '$R' && '$GUARD' push --remote origin --branch main --since-head '$HEAD_BEFORE'"
|
||||
|
||||
# NEEDLE: an empty commit carrying a real message.
|
||||
R="$(new_repo b4)"
|
||||
git -C "$R" commit -q --allow-empty -m "feat: important-sounding message"
|
||||
expect NEEDLE 5 "pushing an empty commit is refused" -- \
|
||||
bash -c "cd '$R' && '$GUARD' push --remote origin --branch main"
|
||||
|
||||
# CONTROL: --since-head is satisfied when a real commit was made.
|
||||
R="$(new_repo b5)"
|
||||
HEAD_BEFORE="$(git -C "$R" rev-parse HEAD)"
|
||||
printf 'work\n' > "$R/work.txt"
|
||||
git -C "$R" add work.txt && git -C "$R" commit -q -m "real work"
|
||||
# --out is required here. Without it this control is VACUOUS: deleting the whole
|
||||
# --since-head validation block would still yield exit 0, because the empty-commit
|
||||
# and remote-moved checks independently succeed. It would prove nothing about the
|
||||
# code path it names.
|
||||
expect CONTROL 0 "--since-head passes when a real commit was created" \
|
||||
--out "HEAD advanced" -- \
|
||||
bash -c "cd '$R' && '$GUARD' push --remote origin --branch main --since-head '$HEAD_BEFORE'"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Needles for the five blockers rev-974 found from a clean checkout. Every one
|
||||
# of these was reproduced before it was fixed; none is a hypothesis.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
echo
|
||||
echo "-- blocker 2: the enumerating producer's exit status --"
|
||||
# FAULT INJECTION. mapfile < <(git diff) reported MAPFILE's status, so a git diff
|
||||
# that died returned zero files and the guard published that silence as clean.
|
||||
R="$(new_repo e1)"
|
||||
mkdir -p "$R/data"; printf '{ not json' > "$R/data/bad.json"
|
||||
git -C "$R" add -f data/bad.json
|
||||
expect NEEDLE 6 "a failed file enumeration REFUSES instead of reporting clean" \
|
||||
--out "CANNOT ENUMERATE STAGED FILES" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged --json-path ':(this-magic-does-not-exist)data/*.json'"
|
||||
# CONTROL: the same malformed file with a WORKING pathspec must still be caught,
|
||||
# so the needle above is not passing merely because everything now refuses.
|
||||
expect CONTROL 3 "a working pathspec still catches the malformed JSON" \
|
||||
--out "data/bad.json" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged --json-path 'data/*.json'"
|
||||
|
||||
echo
|
||||
echo "-- blocker 3: OFF must come from a committed, reviewable object --"
|
||||
R="$(new_repo e2)"
|
||||
write_config_untracked "$R" '{"json_check": "none", "reason": "local unreviewed bypass"}'
|
||||
mkdir -p "$R/data"; printf '{ not json' > "$R/data/bad.json"
|
||||
git -C "$R" add -f data/bad.json
|
||||
# The anchor is the DISTINGUISHING clause, not the shared headline. Three
|
||||
# separate branches print "OPT-OUT IS NOT REVIEWABLE" — untracked, staged-not-
|
||||
# committed, and symlink. Anchoring on the headline means this needle stays green
|
||||
# if the untracked branch is deleted and control falls through to a SIBLING that
|
||||
# prints the same words: a substring anchor does not fail when its subject is
|
||||
# removed, IT RE-POINTS. Anchor on the clause only this branch can produce.
|
||||
expect NEEDLE 6 "an UNTRACKED opt-out is refused as unreviewable" \
|
||||
--out "is not tracked in git" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# STAGED BUT NOT COMMITTED. Found by mutation, not by review: `if [[ -z "$cmode" ]]`
|
||||
# survived `if false` against a 44/44 green suite, because no case ever built this
|
||||
# state. Note the mutant still exits 6 — control falls to the LOCAL-ONLY branch
|
||||
# below and refuses for a different reason. An exit-code-only assertion here would
|
||||
# be satisfied by the wrong branch, which is why --out carries the distinguishing
|
||||
# clause. `git add` puts the file in the index, so ls-files matches while HEAD
|
||||
# does not: staged review is not review.
|
||||
R="$(new_repo e2b)"
|
||||
write_config_untracked "$R" '{"json_check": "none", "reason": "staged, never committed"}'
|
||||
git -C "$R" add .push-guard.json
|
||||
mkdir -p "$R/data"; printf '{ not json' > "$R/data/bad.json"
|
||||
git -C "$R" add -f data/bad.json
|
||||
expect NEEDLE 6 "a STAGED-but-uncommitted opt-out is refused" \
|
||||
--out "staged but not yet in HEAD" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# COMMITTED CONFIG DOES NOT PARSE while the working tree opts out cleanly. Also a
|
||||
# mutation survivor. The working-tree config is valid, so the early config check
|
||||
# passes and we reach the re-read; the committed object is what fails. Without
|
||||
# this branch an opt-out could be honoured on the strength of a HEAD blob nobody
|
||||
# can actually read a decision out of.
|
||||
R="$(new_repo e2c)"
|
||||
write_config "$R" '{ this is not json'
|
||||
printf '%s\n' '{"json_check": "none", "reason": "valid here, broken in HEAD"}' > "$R/.push-guard.json"
|
||||
mkdir -p "$R/data"; printf '{ not json' > "$R/data/bad.json"
|
||||
git -C "$R" add -f data/bad.json
|
||||
expect NEEDLE 6 "an UNPARSEABLE committed config cannot authorise an opt-out" \
|
||||
--out "is INVALID" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# A committed ON silently flipped OFF in the working tree is the same bypass.
|
||||
R="$(new_repo e3)"
|
||||
write_config "$R" '{"json_paths": ["data/**/*.json"]}'
|
||||
printf '%s\n' '{"json_check": "none", "reason": "local convenience"}' > "$R/.push-guard.json"
|
||||
mkdir -p "$R/data"; printf '{ not json' > "$R/data/bad.json"
|
||||
git -C "$R" add -f data/bad.json
|
||||
expect NEEDLE 6 "a committed ON flipped OFF in the working tree is refused" \
|
||||
--out "LOCAL-ONLY OPT-OUT" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# A committed SYMLINK is a mutable target: the reviewed blob is a path, and what
|
||||
# it points at can change with no diff at all.
|
||||
R="$(new_repo e4)"
|
||||
printf '%s\n' '{"json_check": "none", "reason": "via symlink"}' > "$R/real-config.json"
|
||||
ln -s real-config.json "$R/.push-guard.json"
|
||||
git -C "$R" add real-config.json .push-guard.json
|
||||
git -C "$R" commit -q -m "symlinked config"
|
||||
mkdir -p "$R/data"; printf '{ not json' > "$R/data/bad.json"
|
||||
git -C "$R" add -f data/bad.json
|
||||
expect NEEDLE 6 "a committed SYMLINK config is refused as a mutable target" \
|
||||
--out "committed SYMLINK" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
# CONTROL: the supported path still works. Without this the whole opt-out feature
|
||||
# could be dead and every needle above would still pass — a gate that can never
|
||||
# say yes is not a gate.
|
||||
R="$(new_repo e5)"
|
||||
write_config "$R" '{"json_check": "none", "reason": "committed and reviewable"}'
|
||||
mkdir -p "$R/data"; printf '{ not json' > "$R/data/bad.json"
|
||||
git -C "$R" add -f data/bad.json
|
||||
expect CONTROL 0 "a COMMITTED opt-out is honoured and prints its committed reason" \
|
||||
--out "recorded reason: committed and reviewable" -- \
|
||||
bash -c "cd '$R' && '$GUARD' check-staged"
|
||||
|
||||
echo
|
||||
echo "-- blocker 4: advancement is ancestry, not inequality --"
|
||||
R="$(new_repo e6)"
|
||||
git -C "$R" checkout -qb other
|
||||
printf 'o\n' > "$R/o.txt"; git -C "$R" add o.txt; git -C "$R" commit -q -m o
|
||||
OTHER="$(git -C "$R" rev-parse HEAD)"
|
||||
git -C "$R" checkout -q main 2>/dev/null || git -C "$R" checkout -q master
|
||||
printf 'm\n' > "$R/m.txt"; git -C "$R" add m.txt; git -C "$R" commit -q -m m
|
||||
MAINH="$(git -C "$R" rev-parse HEAD)"
|
||||
git -C "$R" checkout -q "$OTHER"
|
||||
expect NEEDLE 5 "a checkout of pre-existing diverged history is not 'advancement'" \
|
||||
--out "NOT A DESCENDANT" -- \
|
||||
bash -c "cd '$R' && '$GUARD' push --remote origin --branch other --since-head '$MAINH'"
|
||||
|
||||
echo
|
||||
echo "-- blocker 5: root and merge commits have defined emptiness --"
|
||||
R="$(new_repo e7)"
|
||||
git -C "$R" checkout -q --orphan fresh
|
||||
git -C "$R" rm -q -rf . >/dev/null 2>&1 || true
|
||||
git -C "$R" commit -q --allow-empty -m "empty root"
|
||||
expect NEEDLE 5 "an EMPTY ROOT commit is refused, not exempted" \
|
||||
--out "EMPTY ROOT COMMIT" -- \
|
||||
bash -c "cd '$R' && '$GUARD' push --remote origin --branch fresh"
|
||||
|
||||
# CONTROL: a real root commit must still push, or the fix is just a new wall.
|
||||
R="$(new_repo e8)"
|
||||
git -C "$R" checkout -q --orphan fresh2
|
||||
git -C "$R" rm -q -rf . >/dev/null 2>&1 || true
|
||||
printf 'real\n' > "$R/real.txt"; git -C "$R" add real.txt
|
||||
git -C "$R" commit -q -m "real root"
|
||||
expect CONTROL 0 "a NON-empty root commit still pushes" \
|
||||
--out "non-empty root commit" -- \
|
||||
bash -c "cd '$R' && '$GUARD' push --remote origin --branch fresh2"
|
||||
|
||||
# THE MERGE BRANCH HAD NO NEEDLE AT ALL. It was defined, hand-verified, and
|
||||
# shipped -- and deleting the refusal left the suite at 41/41 green. Defining
|
||||
# semantics is not testing them, and an untested branch is indistinguishable
|
||||
# from an absent one to everyone downstream.
|
||||
# EMPTY MERGE: two branches that each change nothing, merged. The merge tree is
|
||||
# then identical to EVERY parent -- it integrates nothing and introduces nothing.
|
||||
R="$(new_repo e9)"
|
||||
git -C "$R" checkout -q -b mx
|
||||
git -C "$R" commit -q --allow-empty -m "x: no tree change"
|
||||
git -C "$R" checkout -q -b my HEAD~1 2>/dev/null || git -C "$R" checkout -q -b my
|
||||
git -C "$R" commit -q --allow-empty -m "y: no tree change"
|
||||
git -C "$R" checkout -q mx
|
||||
git -C "$R" merge -q --no-ff --no-edit my
|
||||
# PROVE THE FIXTURE IS ACTUALLY AN EMPTY MERGE before asserting on it, or the
|
||||
# needle passes for the wrong reason on a repo that never made a merge at all.
|
||||
np="$(git -C "$R" rev-list --parents -n 1 HEAD | wc -w)"
|
||||
if (( np > 2 )) && git -C "$R" diff --quiet HEAD^1 HEAD && git -C "$R" diff --quiet HEAD^2 HEAD; then
|
||||
printf ' ok [%-14s] fixture is a merge (%d fields) with tree identical to both parents\n' "e9-fixture" "$np"; PASS=$(( PASS + 1 ))
|
||||
else
|
||||
printf ' FAIL [%-14s] fixture is not an empty merge -- needle would be vacuous\n' "e9-fixture"; FAIL=$(( FAIL + 1 ))
|
||||
fi
|
||||
expect NEEDLE 5 "an EMPTY MERGE is refused, not exempted" \
|
||||
--out "EMPTY MERGE" -- \
|
||||
bash -c "cd '$R' && '$GUARD' push --remote origin --branch mx"
|
||||
|
||||
# CONTROL: a merge that really integrates must still push. Both sides add a
|
||||
# distinct file, so the merge tree differs from BOTH parents.
|
||||
R="$(new_repo e10)"
|
||||
git -C "$R" checkout -q -b nx
|
||||
printf 'from x\n' > "$R/x.txt"; git -C "$R" add x.txt; git -C "$R" commit -q -m "x adds a file"
|
||||
git -C "$R" checkout -q -b ny HEAD~1
|
||||
printf 'from y\n' > "$R/y.txt"; git -C "$R" add y.txt; git -C "$R" commit -q -m "y adds a file"
|
||||
git -C "$R" checkout -q nx
|
||||
git -C "$R" merge -q --no-ff --no-edit ny
|
||||
expect CONTROL 0 "a NON-empty merge commit still pushes" \
|
||||
--out "non-empty merge commit" -- \
|
||||
bash -c "cd '$R' && '$GUARD' push --remote origin --branch nx"
|
||||
|
||||
echo
|
||||
printf 'push-guard needles: %d passed, %d failed\n' "$PASS" "$FAIL"
|
||||
(( FAIL == 0 )) || exit 1
|
||||
@@ -1,95 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# test-verify-clean-clone.sh -- needles for the verifier itself.
|
||||
#
|
||||
# v1 of the verifier passed while the real repository recorded 100644, because it
|
||||
# asserted a SCRATCH repo built by cp. There was no needle that could have caught
|
||||
# that: every case ran against a tree whose modes came off my filesystem.
|
||||
# So the load-bearing needle here is w2 -- SOURCE GIT MODE 100644, DISK MODE 755.
|
||||
# That is the exact contaminated state, and v1 exits 0 on it while v2 must refuse.
|
||||
# A verifier without this needle is the same shape as the defect it verifies.
|
||||
|
||||
set -uo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
VERIFY="$HERE/verify-clean-clone.sh"
|
||||
ARTIFACTS=(push-guard.sh test-push-guard.sh verify-clean-clone.sh mutate-push-guard.sh)
|
||||
|
||||
PASS=0; FAIL=0
|
||||
TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT
|
||||
|
||||
# Build a source repo whose COMMITTED modes are 100755 and whose disk modes are
|
||||
# executable -- the honest state.
|
||||
mkrepo() {
|
||||
local d="$1"
|
||||
mkdir -p "$d"
|
||||
local f
|
||||
for f in "${ARTIFACTS[@]}"; do
|
||||
install -m 755 "$HERE/$f" "$d/$f"
|
||||
done
|
||||
git -C "$d" init -q .
|
||||
git -C "$d" config user.email "[email protected]"
|
||||
git -C "$d" config user.name "Verifier Needles"
|
||||
git -C "$d" add -A
|
||||
git -C "$d" commit -q -m "artifacts"
|
||||
}
|
||||
|
||||
run_case() {
|
||||
local name="$1" want_rc="$2" want_txt="$3" repo="$4"
|
||||
local out rc
|
||||
out="$("$VERIFY" --repo "$repo" 2>&1)"; rc=$?
|
||||
if (( rc == want_rc )) && [[ "$out" == *"$want_txt"* ]]; then
|
||||
printf ' ok [%-14s]\n' "$name"; PASS=$(( PASS + 1 ))
|
||||
else
|
||||
printf ' FAIL [%-14s] wanted rc=%d containing %q; got rc=%d\n%s\n' \
|
||||
"$name" "$want_rc" "$want_txt" "$rc" "$out"; FAIL=$(( FAIL + 1 ))
|
||||
fi
|
||||
}
|
||||
|
||||
echo "== POSITIVE CONTROL: an honestly-committed artifact must PASS =="
|
||||
# Without this, every case below could be passing because the verifier always
|
||||
# refuses -- a wall, not a gate.
|
||||
mkrepo "$TMP/good"
|
||||
run_case w1-honest 0 "the COMMITTED artifact ran and passed" "$TMP/good"
|
||||
|
||||
echo
|
||||
echo "== THE B1 NEEDLE: source git mode 100644, DISK MODE STILL 755 =="
|
||||
# This is the reviewer's exact reproduction. v1 of the verifier exits 0 here.
|
||||
mkrepo "$TMP/laundered"
|
||||
git -C "$TMP/laundered" update-index --chmod=-x push-guard.sh test-push-guard.sh
|
||||
git -C "$TMP/laundered" commit -q -m "drop exec bit in git only"
|
||||
# PROVE THE FIXTURE IS THE CONTAMINATED STATE, not merely a broken repo: git must
|
||||
# say 100644 while the filesystem still says executable. If the disk bit were
|
||||
# gone too, the needle would be testing something easier than the real defect.
|
||||
src_mode="$(git -C "$TMP/laundered" ls-tree HEAD -- push-guard.sh | awk '{print $1}')"
|
||||
disk_mode="$(stat -c '%a' "$TMP/laundered/push-guard.sh")"
|
||||
if [[ "$src_mode" == "100644" && "$disk_mode" == "755" ]]; then
|
||||
printf ' ok [%-14s] fixture is git=%s disk=%s\n' "w2-fixture" "$src_mode" "$disk_mode"; PASS=$(( PASS + 1 ))
|
||||
else
|
||||
printf ' FAIL [%-14s] fixture wrong: git=%s disk=%s -- needle would not test the defect\n' \
|
||||
"w2-fixture" "$src_mode" "$disk_mode"; FAIL=$(( FAIL + 1 ))
|
||||
fi
|
||||
run_case w2-laundered 1 "committed 100644, needs 100755" "$TMP/laundered"
|
||||
|
||||
echo
|
||||
echo "== the artifact must be COMMITTED, or there is no mode to verify =="
|
||||
mkrepo "$TMP/untracked"
|
||||
git -C "$TMP/untracked" rm -q --cached push-guard.sh
|
||||
git -C "$TMP/untracked" commit -q -m "untrack the guard"
|
||||
run_case w3-untracked 1 "NOT TRACKED" "$TMP/untracked"
|
||||
|
||||
echo
|
||||
echo "== a committed SYMLINK is a path, not the reviewed code =="
|
||||
mkrepo "$TMP/symlink"
|
||||
( cd "$TMP/symlink" && rm -f push-guard.sh && ln -s /dev/null push-guard.sh \
|
||||
&& git add push-guard.sh && git commit -q -m "symlink the guard" )
|
||||
run_case w4-symlink 1 "SYMLINK" "$TMP/symlink"
|
||||
|
||||
echo
|
||||
echo "== not a repository at all: REFUSE, never pass =="
|
||||
mkdir -p "$TMP/bare"
|
||||
for f in "${ARTIFACTS[@]}"; do install -m 755 "$HERE/$f" "$TMP/bare/$f"; done
|
||||
run_case w5-norepo 1 "not inside a git repository" "$TMP/bare"
|
||||
|
||||
echo
|
||||
printf '%d passed, %d failed\n' "$PASS" "$FAIL"
|
||||
(( FAIL == 0 ))
|
||||
@@ -1,127 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# verify-clean-clone.sh -- prove the COMMITTED artifact runs, from a clean clone.
|
||||
#
|
||||
# ================== THIS FILE IS THE SECOND VERSION. THE FIRST HAD B1. ========
|
||||
# B1 was: the delivered scripts were committed mode 100644, so the artifact
|
||||
# returned 126 Permission denied for anyone who cloned it. Two of us ran 32/32
|
||||
# because our local working copies had the exec bit set by hand at creation.
|
||||
#
|
||||
# I wrote v1 of this file to prevent exactly that. V1 COPIED THE WORKING-TREE
|
||||
# FILES INTO A SCRATCH REPOSITORY AND ASSERTED THE SCRATCH REPOSITORY'S INDEX.
|
||||
# cp preserves the local exec bit, so `git add` in the scratch repo recorded
|
||||
# 100755 NO MATTER WHAT THE REAL REPOSITORY STORED. Reviewer reproduction:
|
||||
# git update-index --chmod=-x push-guard.sh test-push-guard.sh
|
||||
# ./verify-clean-clone.sh -> EXIT 0, "CLEAN CLONE: suite ran and passed"
|
||||
# while the real index read 100644 -- the precise contaminated state B1 was.
|
||||
#
|
||||
# THE CONTROL FOR THE DEFECT INHERITED THE DEFECT, BECAUSE IT MEASURED A COPY
|
||||
# INSTEAD OF THE SUBJECT. cp LAUNDERS MODE.
|
||||
#
|
||||
# Both of v1's mechanisms were right -- assert the INDEX not the disk, execute
|
||||
# DIRECTLY not via `bash script`. They were applied to the wrong repository.
|
||||
# So v2 changes what is measured, not how:
|
||||
# * mode is read from the SOURCE repository's COMMITTED TREE (git ls-tree),
|
||||
# which no local chmod can influence;
|
||||
# * the tree under test is produced by CLONING THAT COMMIT, so every mode
|
||||
# comes out of the object store rather than off my filesystem.
|
||||
# There is no cp anywhere in this file, and that is deliberate.
|
||||
#
|
||||
# It also REFUSES rather than passes when the artifacts are untracked: an
|
||||
# artifact that is not committed has no mode to verify, and a verifier that
|
||||
# silently succeeds on nothing is the vacuous-absence failure all over again.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
readonly EX_FAIL=1
|
||||
readonly EX_USAGE=64
|
||||
|
||||
REPO=""
|
||||
REV="HEAD"
|
||||
|
||||
die() { printf 'usage error: %s\n' "$1" >&2; exit "$EX_USAGE"; }
|
||||
|
||||
while (( $# )); do
|
||||
case "$1" in
|
||||
--repo) REPO="${2:-}"; shift 2 ;;
|
||||
--rev) REV="${2:-}"; shift 2 ;;
|
||||
*) die "unknown argument: $1" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
[[ -n "$REPO" ]] || REPO="$HERE"
|
||||
|
||||
ARTIFACTS=(push-guard.sh test-push-guard.sh verify-clean-clone.sh mutate-push-guard.sh)
|
||||
SUITE="test-push-guard.sh"
|
||||
|
||||
# --- the subject must be a real repository, or there is nothing to verify -----
|
||||
if ! ROOT="$(git -C "$REPO" rev-parse --show-toplevel 2>/dev/null)"; then
|
||||
printf 'REFUSING: %s is not inside a git repository.\n' "$REPO" >&2
|
||||
printf 'This verifier checks the COMMITTED mode of the artifact. An uncommitted\n' >&2
|
||||
printf 'artifact has no committed mode, and passing here would prove nothing.\n' >&2
|
||||
exit "$EX_FAIL"
|
||||
fi
|
||||
if ! REV_SHA="$(git -C "$ROOT" rev-parse --verify --quiet "${REV}^{commit}")"; then
|
||||
printf 'REFUSING: %s does not resolve to a commit in %s\n' "$REV" "$ROOT" >&2
|
||||
exit "$EX_FAIL"
|
||||
fi
|
||||
|
||||
printf '=== subject ===\n'
|
||||
printf ' repo %s\n rev %s (%s)\n\n' "$ROOT" "$REV" "$REV_SHA"
|
||||
|
||||
# --- 1. MODE, FROM THE COMMITTED TREE OF THE SUBJECT -------------------------
|
||||
# git ls-tree reports what the COMMIT records. Nothing on my filesystem can
|
||||
# move this number -- which is the whole point, and what v1 got wrong.
|
||||
printf '=== committed modes (git ls-tree %s) ===\n' "$REV"
|
||||
rc=0
|
||||
for f in "${ARTIFACTS[@]}"; do
|
||||
entry="$(git -C "$ROOT" ls-tree "$REV_SHA" -- "$f")"
|
||||
if [[ -z "$entry" ]]; then
|
||||
printf ' FAIL %s is NOT TRACKED at %s -- nothing committed to verify\n' "$f" "$REV"
|
||||
rc=1; continue
|
||||
fi
|
||||
mode="${entry%% *}"; rest="${entry#* }"; type="${rest%% *}"
|
||||
if [[ "$type" != blob ]]; then
|
||||
printf ' FAIL %s is a %s at %s, not a regular file\n' "$f" "$type" "$REV"
|
||||
rc=1; continue
|
||||
fi
|
||||
case "$mode" in
|
||||
100755) printf ' ok %s committed %s\n' "$f" "$mode" ;;
|
||||
120000) printf ' FAIL %s is a SYMLINK (%s) -- the reviewed blob is a path, not the code\n' "$f" "$mode"; rc=1 ;;
|
||||
*) printf ' FAIL %s committed %s, needs 100755\n' "$f" "$mode"
|
||||
printf ' fix with: git update-index --chmod=+x %s && commit\n' "$f"
|
||||
rc=1 ;;
|
||||
esac
|
||||
done
|
||||
if (( rc != 0 )); then
|
||||
printf '\nREFUSING TO CONTINUE: the COMMITTED modes are wrong, whatever the disk says.\n'
|
||||
printf 'A clone of this commit would exit 126 for the next person.\n'
|
||||
exit "$EX_FAIL"
|
||||
fi
|
||||
|
||||
# --- 2. RUN FROM A CLONE OF THAT COMMIT --------------------------------------
|
||||
# Cloning materialises every file from the object store, so the modes on disk
|
||||
# are the COMMITTED modes by construction. No cp, no local state, nothing this
|
||||
# machine can contribute.
|
||||
WORK="$(mktemp -d)"
|
||||
trap 'rm -rf "$WORK"' EXIT
|
||||
|
||||
printf '\n=== cloning %s (no local mode bits survive this) ===\n' "$REV_SHA"
|
||||
git clone -q --no-local --no-hardlinks "$ROOT" "$WORK/clone"
|
||||
git -C "$WORK/clone" -c advice.detachedHead=false checkout -q "$REV_SHA"
|
||||
|
||||
for f in "${ARTIFACTS[@]}"; do
|
||||
printf ' clone disk mode: %s %s\n' "$(stat -c '%a' "$WORK/clone/$f")" "$f"
|
||||
done
|
||||
|
||||
printf '\n=== executing DIRECTLY (./%s), not via "bash %s" ===\n' "$SUITE" "$SUITE"
|
||||
# Invoking the interpreter explicitly masks a missing exec bit completely -- it
|
||||
# is how the original green was obtained. Direct execution is the thing under
|
||||
# test, so the mode has to be real for this line to succeed.
|
||||
cd "$WORK/clone"
|
||||
if ! "./$SUITE"; then
|
||||
printf '\nCLEAN-CLONE RUN FAILED.\n'
|
||||
exit "$EX_FAIL"
|
||||
fi
|
||||
|
||||
printf '\n=== CLEAN CLONE: the COMMITTED artifact ran and passed ===\n'
|
||||
@@ -414,22 +414,6 @@ cmd_poll_once() {
|
||||
_wake_clean_stale_tmp "$STATE_DIR"
|
||||
mkdir -p "$DET_DIR"
|
||||
|
||||
local failed=0
|
||||
|
||||
# #958 preimage provenance: check the OPERATOR-SIDE preimage definition (the
|
||||
# source adapter file, the watch-list, operator-declared extras) BEFORE
|
||||
# observing any source. A changed preimage re-baselines EVERY source at once;
|
||||
# running the check first means its first-class cause line is enqueued at a
|
||||
# LOWER observed_seq than the N per-source deltas it explains, so the digest
|
||||
# shows the cause, not just the flood. An infrastructure failure of the check
|
||||
# is LOUD and marks this pass failed (G2a discipline — never read as "no
|
||||
# change"), but source observation still proceeds: provenance must not be
|
||||
# able to starve wake delivery.
|
||||
if ! "$SCRIPT_DIR/preimage.sh" check --enqueue; then
|
||||
echo "detector.sh: FAIL LOUD — preimage provenance check failed (see preimage.sh above); source observation continues but this pass exits non-zero." >&2
|
||||
failed=1
|
||||
fi
|
||||
|
||||
# Iterate the DECLARED source-coverage inventory (§4/G3): only sources listed
|
||||
# in watches[].sources[] are polled. An omitted source is not observed (and so
|
||||
# cannot make anything pass vacuously); a referenced-but-undefined source is a
|
||||
@@ -443,7 +427,7 @@ cmd_poll_once() {
|
||||
exit 2
|
||||
fi
|
||||
|
||||
local kind id def class
|
||||
local failed=0 kind id def class
|
||||
while IFS=$'\t' read -r kind id; do
|
||||
[ -n "$kind" ] || continue
|
||||
# Resolve the source definition from its top-level collection by id.
|
||||
|
||||
@@ -488,19 +488,6 @@ _dlq_alarm_offhost() {
|
||||
# A dead-letter write failure is itself alarmed (both legs) but never aborts the
|
||||
# drain — the good entries MUST still deliver (availability is the whole point of
|
||||
# #920).
|
||||
#
|
||||
# RETENTION IS LOAD-BEARING (#953) — read BEFORE adding rotation/pruning/caps:
|
||||
# this ledger is append-only history AND the SOLE evidence base for
|
||||
# store.sh quarantine-audit's conviction predicate (#946): a false
|
||||
# consumed-hashes witness row is provable ONLY while its matching entry
|
||||
# survives HERE. Any rotation, pruning, or size cap — however locally correct
|
||||
# — silently converts provable rows into unprovable ones, and the audit's
|
||||
# clean sweep reads IDENTICALLY before and after the evidence disappears (no
|
||||
# signal at either end; the audit never guesses, by design). If retention
|
||||
# limits ever become genuinely necessary, they MUST ship with (a) a loud
|
||||
# signal at prune time naming what evidence is being given up, and (b) the
|
||||
# audit's residual-class reporting updated IN THE SAME CHANGE. Until then:
|
||||
# nothing prunes this file, and that is a recorded decision, not an omission.
|
||||
_quarantine_entry() {
|
||||
local line="$1" seq="$2" dlq="$STATE_DIR/dead-letter.jsonl"
|
||||
if [ ! -f "$dlq" ] || ! grep -qxF "$line" "$dlq" 2>/dev/null; then
|
||||
|
||||
@@ -374,99 +374,8 @@
|
||||
# Changed: store.sh, digest.sh, ack.sh
|
||||
# (+ test-wake-store-ack.sh T13-T16,
|
||||
# test-wake-digest-quarantine.sh Q12-Q16).
|
||||
# 0.7.0 #958 A11 preimage.sh — durable provenance for the OPERATOR-SIDE
|
||||
# preimage definition (wake-pilot finding: source-adapter.sh was
|
||||
# unversioned, so the operator-owned bytes every observed_hash is
|
||||
# computed FROM had thinner provenance than any hash they feed;
|
||||
# attribution required an agent transcript). NEW tool + two
|
||||
# pre-step integrations, ADDITIVE: (1) preimage.sh derives the
|
||||
# preimage set from the RUNTIME env (the resolved
|
||||
# WAKE_DETECTOR_SOURCE_CMD file, WAKE_WATCH_LIST, optional
|
||||
# WAKE_PREIMAGE_EXTRA paths) and, per file, records
|
||||
# {ts,path,sha256,size,mtime,prev} to an append-only ledger +
|
||||
# captures the bytes CONTENT-ADDRESSED under
|
||||
# $STATE_DIR/preimage/objects/<sha256> — prior bytes + change
|
||||
# time are answerable from durable state alone, no transcript
|
||||
# (acceptance a). A git-repo-in-operator-dir design was REJECTED:
|
||||
# its who/why claim is false under shared-author fleets, and
|
||||
# .gitignore is pattern-based where acceptance (b) demands
|
||||
# fail-closed. (2) CREDENTIAL HARD GATE (acceptance b), FOUR
|
||||
# LAYERS (#964 re-verdict: B2 cases C+D): (i) POLARITY — extras
|
||||
# (WAKE_PREIMAGE_EXTRA) are RECORD-ONLY by default (ledger row +
|
||||
# cause line, NO bytes); byte capture for an extra requires the
|
||||
# path listed in WAKE_PREIMAGE_CAPTURE (colon-separated opt-in);
|
||||
# only the core set (adapter, watch-list) is capture-eligible by
|
||||
# default — a shape list can only refuse the secrets someone
|
||||
# already enumerated, so safety may not rest on one (#964-D).
|
||||
# (ii) PATH DENY evaluated on BOTH the raw and realpath-resolved
|
||||
# candidate forms against BOTH unresolved and resolved
|
||||
# mosaic-home anchors (credentials.json,
|
||||
# tools/_lib/credentials.json, credentials/**, any basename
|
||||
# credentials.json) — a deny list written in unresolved paths
|
||||
# cannot match a path resolved before it arrived (#964-C), and a
|
||||
# symlinked opt-in entry cannot smuggle a denied target (deny
|
||||
# runs FIRST, independent of what the opt-in matched). (iii)
|
||||
# CONTENT DENY on the opt-in path only, defense-in-depth NOT the
|
||||
# safety mechanism: digest.sh's six scrub shapes + a
|
||||
# named-assignment probe (key/token/secret/passw/hmac/credential/
|
||||
# bearer = unbroken value >=16 chars); probe ERROR fails TOWARD
|
||||
# refusal (an error exit is not a negative result); false
|
||||
# positives are acceptable — a false match only withholds byte
|
||||
# capture, never tracking. (iv) size cap WAKE_PREIMAGE_MAX_BYTES
|
||||
# (default 1 MiB). Deny ALWAYS wins over the opt-in. A refused
|
||||
# file still gets its hash/size/mtime row (captured:false +
|
||||
# refused reason) so change TIME survives even when content must
|
||||
# not. (3) FIRST-CLASS CAUSE LINE
|
||||
# (acceptance c): on a change (or deletion — ABSENT is a state,
|
||||
# not an error), one class=actionable entry is enqueued via the
|
||||
# store allocator with locators {kind:preimage, path (§2.1 hard
|
||||
# locator — never quarantined), observed_hash, prev_hash,
|
||||
# preimage:true, reason:preimage-definition-changed}. Both
|
||||
# detector.sh cmd_poll_once and reconcile.sh cmd_reconcile run
|
||||
# the check as a PRE-step, so the cause line lands at a LOWER
|
||||
# observed_seq than the N per-source deltas/enumerations it
|
||||
# explains — "preimage definition changed" reads first, not N
|
||||
# UNACCOUNTED lines. (4) FAIL-LOUD discipline (D2/#955 class):
|
||||
# an unresolvable adapter, unreadable watch-list, corrupt ledger
|
||||
# (REFUSES to compare or re-baseline over corrupt history), failed
|
||||
# object/ledger write, or failed enqueue is a loud non-zero —
|
||||
# never read as "no change"; in both integrations the pass exits
|
||||
# non-zero but source observation still proceeds (no starvation).
|
||||
# An ABSENT/EMPTY ledger with objects/ NON-empty is a loud
|
||||
# non-zero REFUSAL to re-baseline (#964-B11: objects with no
|
||||
# ledger cannot be a first install — history was deleted; the
|
||||
# first-install path must not silently absorb it). First-seen on
|
||||
# a genuinely clean state dir is a SILENT baseline (detector
|
||||
# first-poll idiom).
|
||||
# The installer is UNCHANGED — Gate A auto-enumerates the new
|
||||
# file from the filesystem; the recording site is the runtime
|
||||
# tick, which is where the env-derived preimage set exists.
|
||||
# store.sh/digest.sh/beacon.sh UNCHANGED; watch-list schema
|
||||
# UNTOUCHED ([1,1]). Changed: preimage.sh (new), detector.sh,
|
||||
# reconcile.sh (+ test-wake-preimage.sh P1-P17; P13-P17 are the
|
||||
# #964 re-verdict regression needles: symlinked store, live
|
||||
# secret value, polarity, ledger deletion, allowlist-symlink).
|
||||
# 0.7.1 #952 quarantine-audit clean-sweep message named only ONE of the
|
||||
# two unprovable residual classes ("rows without surviving
|
||||
# dead-letter evidence"), so an operator reading the OK concluded
|
||||
# NO evidence exists when evidence can exist and be UNUSABLE: a
|
||||
# surviving dead-letter row whose locator extracts an empty
|
||||
# observed_hash (live specimen: mos-dt seq 13,
|
||||
# bench/malformed-locator-test, "deliberately non-conformant")
|
||||
# can never satisfy the four-field conviction match, because
|
||||
# _record_last_consumed only writes rows with a NON-empty hash.
|
||||
# WORDING-ONLY fix (measurement defect, #951 review finding 1):
|
||||
# THREE surfaces — the clean-sweep message, the PROVABILITY
|
||||
# BOUND comment, and (second commit, author-side self-catch)
|
||||
# the usage() help text — now name both classes; the
|
||||
# conviction predicate is UNCHANGED.
|
||||
# Test T17 drives the real writer flow and plants the verbatim
|
||||
# live specimen (nested .locators.*, NO observed_hash key) —
|
||||
# never a hand-built flat dead-letter row, which would make the
|
||||
# audit's correct non-conviction look exactly like the defect
|
||||
# under hunt (the #951 review's false-defect near-miss).
|
||||
component=wake
|
||||
version=0.7.1
|
||||
version=0.6.15
|
||||
|
||||
# Watch-list schema this component consumes, and the INCLUSIVE range of
|
||||
# schema_version values it supports. A wake-watch-list.json whose schema_version
|
||||
@@ -537,19 +446,6 @@ schema_max=1
|
||||
# seed) instead of a bare source error, and LINKS the unit into
|
||||
# the user systemd search path + post-install-validates it
|
||||
# resolves (#913). (W7, #913)
|
||||
# preimage.sh A11 — operator-side PREIMAGE-DEFINITION provenance: derives the
|
||||
# preimage set from the runtime env (resolved adapter file,
|
||||
# watch-list, WAKE_PREIMAGE_EXTRA), appends
|
||||
# {ts,path,sha256,size,mtime,prev} rows to an append-only
|
||||
# ledger, captures bytes content-addressed under
|
||||
# preimage/objects/<sha256>, and enqueues a FIRST-CLASS
|
||||
# "preimage-definition-changed" actionable (path = §2.1 hard
|
||||
# locator) BEFORE the deltas it explains (detector +
|
||||
# reconcile pre-step). Credential HARD GATE: capture is
|
||||
# REFUSED (hash/mtime still recorded) for credential-store
|
||||
# paths, secret-shaped content, and oversized files —
|
||||
# refusal, never redaction. Fail-loud on any infra failure;
|
||||
# a corrupt ledger refuses re-baseline. (#958)
|
||||
# Companion (framework subtree, not under tools/wake/): systemd/user/mosaic-wake.service
|
||||
# — the long-lived detector daemon unit (per-class SLO lives in
|
||||
# the daemon, NOT a systemd interval). (W7)
|
||||
|
||||
@@ -1,571 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# preimage.sh — A11 of the wake canon (#958): durable PROVENANCE for the
|
||||
# OPERATOR-SIDE preimage definition.
|
||||
#
|
||||
# THE GAP THIS CLOSES (#958): every observed_hash in the lane store is
|
||||
# sha256(adapter stdout) — the operator's source adapter (plus the watch-list
|
||||
# and any operator-declared preimage files) IS the preimage definition for
|
||||
# every hash the pipeline ever records. Those files live OUTSIDE the versioned
|
||||
# wake component, so a byte change to them was attributable only through agent
|
||||
# transcripts: provenance thinner than any hash it feeds. A changed preimage
|
||||
# also re-baselines EVERY source at once, which previously surfaced only as N
|
||||
# per-source deltas / UNACCOUNTED lines with no first-class cause.
|
||||
#
|
||||
# WHAT THIS TOOL DOES:
|
||||
# - Derives the PREIMAGE SET generically (no operator paths baked in):
|
||||
# * the resolved file behind WAKE_DETECTOR_SOURCE_CMD (first word),
|
||||
# * the WAKE_WATCH_LIST file,
|
||||
# * operator-declared extras via WAKE_PREIMAGE_EXTRA (colon-separated —
|
||||
# e.g. sink/feed scripts). Extras are RECORD-ONLY by default (see the
|
||||
# credential hard gate below); never list key-material files.
|
||||
# - Records each file's (sha256, size, mtime, ts) as an APPEND-ONLY ledger
|
||||
# row in <state>/preimage/preimage-ledger.jsonl and stores the full bytes
|
||||
# CONTENT-ADDRESSED at <state>/preimage/objects/<sha256> — so a change is
|
||||
# attributable (prior bytes + change time) from durable state alone, with
|
||||
# no transcript required. Acceptance (a) of #958.
|
||||
# - On a CHANGE (not first-seen), `check --enqueue` enqueues ONE first-class
|
||||
# class=actionable entry per changed file via the store's single allocator
|
||||
# (#908), carrying a §2.1 hard locator (`path`), BEFORE the per-source
|
||||
# deltas it explains are observed — so the digest shows the cause line,
|
||||
# not just N re-baselined sources. Acceptance (c).
|
||||
#
|
||||
# CREDENTIAL HARD GATE — acceptance (b): the mechanism must be UNABLE to
|
||||
# capture credential material even by operator error. Layered, every layer
|
||||
# fail-closed toward NOT capturing bytes (the hash/size/mtime row is still
|
||||
# written — change-time attribution survives, content capture does not;
|
||||
# a sha256 discloses nothing about the bytes):
|
||||
# 1. POLARITY (#964 review, case D): byte capture is DENY-BY-DEFAULT for
|
||||
# operator extras. WAKE_PREIMAGE_EXTRA paths are RECORD-ONLY (rows +
|
||||
# first-class change entries, no bytes) unless explicitly listed in
|
||||
# WAKE_PREIMAGE_CAPTURE — a shape list can only refuse the secrets
|
||||
# someone already enumerated, so arbitrary operator-pointed files must
|
||||
# not default to capture. The CORE set (the resolved adapter file, the
|
||||
# watch-list) is capture-eligible: it IS the preimage definition this
|
||||
# tool exists to snapshot, and the deny layers below still apply to it.
|
||||
# 2. PATH deny: the mosaic credential store locations
|
||||
# (<mosaic-home>/credentials.json, <mosaic-home>/tools/_lib/
|
||||
# credentials.json, anything under <mosaic-home>/credentials/, and any
|
||||
# file literally named credentials.json) are refused. Evaluated on BOTH
|
||||
# the operator-supplied form AND the symlink-resolved form, against BOTH
|
||||
# the unresolved and resolved forms of the mosaic-home anchor — a deny
|
||||
# list written only in unresolved paths cannot match a path that was
|
||||
# resolved before it arrived (#964 review, case C).
|
||||
# 3. CONTENT deny: a candidate whose bytes match the well-known secret-token
|
||||
# shapes (same conservative set digest.sh redacts: PAT/Slack/AKIA/JWT/PEM)
|
||||
# OR a named-assignment secret shape (key/token/secret/passw/hmac/
|
||||
# credential/bearer = long unbroken value — catches prefix-less
|
||||
# high-entropy keys, e.g. an HMAC key in an env file) is refused —
|
||||
# refusal, not redaction: a redacted preimage would be a false witness,
|
||||
# and secret bytes must never land in the object store. Deliberately
|
||||
# conservative toward REFUSAL: a false match only withholds byte capture,
|
||||
# never tracking.
|
||||
# Plus a size cap (WAKE_PREIMAGE_MAX_BYTES, default 1 MiB) so a stray extra
|
||||
# pointing at a large binary cannot turn provenance into data hoovering.
|
||||
# Deny ALWAYS wins over the WAKE_PREIMAGE_CAPTURE opt-in.
|
||||
#
|
||||
# FAIL-LOUD DISCIPLINE (the D2/#955 class, both layers): an infrastructure
|
||||
# failure of THIS tool (unresolvable adapter, unreadable/corrupt ledger,
|
||||
# failed durable write, failed enqueue) exits NON-ZERO and is never read as
|
||||
# "no change". A corrupt ledger REFUSES to compare (and to re-baseline):
|
||||
# silently restarting history would erase the very attribution this exists
|
||||
# to provide. Likewise (#964 review, B11) an ABSENT/EMPTY ledger while
|
||||
# objects/ still holds prior captures is DELETED HISTORY, not a first
|
||||
# install — it refuses loudly instead of re-baselining, because a silent
|
||||
# restart would absorb the next real change as first-seen.
|
||||
#
|
||||
# Operator-agnostic (framework firewall): all state via XDG/env; the deny
|
||||
# list names only framework-defined credential locations, no operator hosts/
|
||||
# names/secrets. This tool never prints file CONTENT to any stream.
|
||||
set -uo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
# shellcheck source=./_wake-common.sh disable=SC1091
|
||||
. "$SCRIPT_DIR/_wake-common.sh"
|
||||
|
||||
STATE_DIR="$(wake_state_dir)"
|
||||
STORE_SH="$SCRIPT_DIR/store.sh"
|
||||
PRE_DIR="$STATE_DIR/preimage"
|
||||
LEDGER="$PRE_DIR/preimage-ledger.jsonl"
|
||||
OBJECTS="$PRE_DIR/objects"
|
||||
|
||||
_need_jq() {
|
||||
command -v jq >/dev/null 2>&1 || {
|
||||
echo "preimage.sh: jq is required" >&2
|
||||
exit 3
|
||||
}
|
||||
}
|
||||
|
||||
# _hash_stdin — sha256 of stdin, first field only (portable, same fallback
|
||||
# chain as detector.sh).
|
||||
_hash_stdin() {
|
||||
if command -v sha256sum >/dev/null 2>&1; then
|
||||
sha256sum | awk '{print $1}'
|
||||
elif command -v shasum >/dev/null 2>&1; then
|
||||
shasum -a 256 | awk '{print $1}'
|
||||
elif command -v openssl >/dev/null 2>&1; then
|
||||
openssl dgst -sha256 | awk '{print $NF}'
|
||||
else
|
||||
echo "preimage.sh: no sha256 tool (sha256sum/shasum/openssl) available" >&2
|
||||
exit 3
|
||||
fi
|
||||
}
|
||||
|
||||
usage() {
|
||||
cat >&2 <<'EOF'
|
||||
Usage: preimage.sh <command>
|
||||
|
||||
Commands:
|
||||
check [--enqueue] Compare every preimage-set file against the ledger head.
|
||||
A changed file gets a new ledger row (+ captured bytes
|
||||
when capture-eligible and not denied — see Environment);
|
||||
with --enqueue each change (not first-seen) also enqueues
|
||||
ONE first-class class=actionable store entry (hard
|
||||
locator: path). First-seen files baseline SILENTLY (row,
|
||||
no enqueue — same idiom as the detector's first-seen).
|
||||
Exit 0 whether or not changes were found; non-zero ONLY
|
||||
on infrastructure failure (never read as "no change").
|
||||
record Alias of `check` without enqueue (baseline/refresh).
|
||||
set Print the derived preimage set, one resolved path per
|
||||
line (diagnostic).
|
||||
status Print the ledger head row for every tracked path.
|
||||
|
||||
Environment:
|
||||
WAKE_DETECTOR_SOURCE_CMD Adapter command; its resolved file joins the set.
|
||||
WAKE_WATCH_LIST Watch-list path; joins the set when set.
|
||||
WAKE_PREIMAGE_EXTRA Colon-separated additional operator preimage files
|
||||
(e.g. sink/feed scripts). Extras are RECORD-ONLY
|
||||
by default: changes get a hash/size/mtime row and
|
||||
a first-class change entry, but their BYTES are
|
||||
never captured unless the path is also listed in
|
||||
WAKE_PREIMAGE_CAPTURE. A listed path that does not
|
||||
exist is tracked as ABSENT (deletion of a preimage
|
||||
file is a change, not an error).
|
||||
WAKE_PREIMAGE_CAPTURE Colon-separated allowlist of extras whose bytes
|
||||
MAY be captured. The credential deny rules ALWAYS
|
||||
win over this list. NEVER list files that can hold
|
||||
key material (env files with keys/HMACs, credential
|
||||
stores): the deny gate is a backstop, not a
|
||||
license.
|
||||
WAKE_PREIMAGE_MAX_BYTES Byte-capture cap (default 1048576). Larger files:
|
||||
hash/size/mtime recorded, bytes refused.
|
||||
WAKE_STATE_HOME/WAKE_AGENT store namespace (see store.sh).
|
||||
EOF
|
||||
}
|
||||
|
||||
# --- preimage-set derivation (generic; no operator paths baked in) ----------
|
||||
|
||||
# _realpath_or_self PATH — resolved form when resolvable, the input otherwise.
|
||||
_realpath_or_self() {
|
||||
local p="$1"
|
||||
if command -v realpath >/dev/null 2>&1; then
|
||||
realpath -- "$p" 2>/dev/null || printf '%s' "$p"
|
||||
else
|
||||
printf '%s' "$p"
|
||||
fi
|
||||
}
|
||||
|
||||
# _resolve_cmd_file CMD — resolve the FIRST WORD of an adapter command string
|
||||
# to a real file; prints `raw<TAB>resolved`. Non-zero (loud) if it cannot be
|
||||
# resolved: an adapter the detector will invoke but provenance cannot see is
|
||||
# an infrastructure failure, not a smaller set. BOTH forms are kept: the deny
|
||||
# gate must see the pre-resolution form too (#964 review, case C).
|
||||
_resolve_cmd_file() {
|
||||
local cmd="$1" word path
|
||||
# shellcheck disable=SC2086
|
||||
set -- $cmd
|
||||
word="${1:-}"
|
||||
[ -n "$word" ] || return 1
|
||||
if [ -f "$word" ]; then
|
||||
path="$word"
|
||||
else
|
||||
path="$(command -v -- "$word" 2>/dev/null)" || return 1
|
||||
[ -f "$path" ] || return 1
|
||||
fi
|
||||
# realpath so the ledger keys on the actual file, not a symlink alias.
|
||||
printf '%s\t%s' "$path" "$(_realpath_or_self "$path")"
|
||||
}
|
||||
|
||||
# _preimage_set — print the derived set, one member per line as
|
||||
# `origin<TAB>raw<TAB>resolved` (origin: core|extra). BOTH path forms travel
|
||||
# with every member so the deny gate and the capture allowlist are evaluated
|
||||
# on the SAME candidate in BOTH its forms — resolving before denying is the
|
||||
# #964 case-C ordering defect. Paths from WAKE_PREIMAGE_EXTRA are printed
|
||||
# EVEN IF ABSENT (deletion is a tracked state); the adapter and watch-list
|
||||
# must resolve (loud failure otherwise — see _resolve_cmd_file rationale).
|
||||
_preimage_set() {
|
||||
local failed=0
|
||||
if [ -n "${WAKE_DETECTOR_SOURCE_CMD:-}" ]; then
|
||||
local af
|
||||
if af="$(_resolve_cmd_file "$WAKE_DETECTOR_SOURCE_CMD")"; then
|
||||
printf 'core\t%s\n' "$af"
|
||||
else
|
||||
echo "preimage.sh: FAIL LOUD — WAKE_DETECTOR_SOURCE_CMD ('$WAKE_DETECTOR_SOURCE_CMD') does not resolve to a file; the preimage definition cannot be observed (NOT treated as 'no change')." >&2
|
||||
failed=1
|
||||
fi
|
||||
fi
|
||||
if [ -n "${WAKE_WATCH_LIST:-}" ]; then
|
||||
if [ -f "$WAKE_WATCH_LIST" ]; then
|
||||
printf 'core\t%s\t%s\n' "$WAKE_WATCH_LIST" "$(_realpath_or_self "$WAKE_WATCH_LIST")"
|
||||
else
|
||||
echo "preimage.sh: FAIL LOUD — WAKE_WATCH_LIST ('$WAKE_WATCH_LIST') is not a file; the preimage definition cannot be observed." >&2
|
||||
failed=1
|
||||
fi
|
||||
fi
|
||||
if [ -n "${WAKE_PREIMAGE_EXTRA:-}" ]; then
|
||||
local IFS=':' p
|
||||
for p in $WAKE_PREIMAGE_EXTRA; do
|
||||
[ -n "$p" ] || continue
|
||||
# Extras are tracked even when absent (ABSENT is a state). Resolve the
|
||||
# realpath only when the file exists; otherwise track the literal path.
|
||||
if [ -e "$p" ]; then
|
||||
printf 'extra\t%s\t%s\n' "$p" "$(_realpath_or_self "$p")"
|
||||
else
|
||||
printf 'extra\t%s\t%s\n' "$p" "$p"
|
||||
fi
|
||||
done
|
||||
fi
|
||||
return "$failed"
|
||||
}
|
||||
|
||||
# --- credential hard gate (acceptance (b)) ----------------------------------
|
||||
|
||||
# _deny_path RAW RESOLVED — 0 (deny) iff EITHER form of the candidate names a
|
||||
# known credential-store location. Framework-defined locations only
|
||||
# (firewall): the mosaic credential store, the tools/_lib credential file the
|
||||
# framework-manifest itself carves out of tools/**, the credentials/ operator
|
||||
# subtree, and any file literally named credentials.json.
|
||||
#
|
||||
# Evaluated on BOTH the operator-supplied (raw) form and the symlink-resolved
|
||||
# form, against BOTH the unresolved and resolved forms of the mosaic-home
|
||||
# anchor: a deny list written only in unresolved paths cannot match a path
|
||||
# that was resolved before it arrived (#964 review, case C — a symlink whose
|
||||
# target was renamed slipped every path rule because the candidate had
|
||||
# already been realpath'd but the anchors never were).
|
||||
_deny_path() {
|
||||
local raw="$1" resolved="$2"
|
||||
local home="${MOSAIC_HOME:-$HOME/.config/mosaic}" rhome p a
|
||||
rhome="$(_realpath_or_self "$home")"
|
||||
for p in "$raw" "$resolved"; do
|
||||
[ -n "$p" ] || continue
|
||||
for a in "$home" "$rhome"; do
|
||||
case "$p" in
|
||||
"$a/credentials.json") return 0 ;;
|
||||
"$a/tools/_lib/credentials.json") return 0 ;;
|
||||
"$a/credentials/"*) return 0 ;;
|
||||
esac
|
||||
done
|
||||
case "$(basename -- "$p")" in
|
||||
credentials.json) return 0 ;;
|
||||
esac
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
# _deny_content FILE — 0 (deny) iff FILE's bytes look secret-shaped.
|
||||
# Two probes: (a) the well-known vendor-token shapes digest.sh redacts
|
||||
# (PAT / Slack / AKIA / JWT / PEM — conservative: a 40-hex git sha never
|
||||
# matches); (b) a named-assignment shape (key/token/secret/passw/hmac/
|
||||
# credential/bearer = long unbroken value) that catches prefix-less
|
||||
# high-entropy keys, e.g. an HMAC key in an env file (#964 review, case D).
|
||||
#
|
||||
# (b) is defense-in-depth for the OPT-IN path only, NOT the thing that keeps
|
||||
# case D safe — polarity does that (extras are record-only unless allowlisted;
|
||||
# a shape list can only refuse the secrets someone already enumerated). It is
|
||||
# deliberately conservative toward REFUSAL: a false match (a checksum in a
|
||||
# config, a path that happens to be long and unbroken) only withholds byte
|
||||
# capture — the hash/size/mtime row and change entry still land. A variable
|
||||
# REFERENCE (token="$GITEA_TOKEN") never matches: `$` is not in the value
|
||||
# class — only literal secret values do.
|
||||
#
|
||||
# FAILS TOWARD REFUSAL: if a probe itself errors (unreadable file, grep
|
||||
# failure — rc >= 2), the answer is DENY, not capture. An error exit is not a
|
||||
# negative result.
|
||||
_deny_content() {
|
||||
local rc
|
||||
LC_ALL=C grep -Eq \
|
||||
-e 'gh[pousr]_[A-Za-z0-9]{16,}' \
|
||||
-e 'github_pat_[A-Za-z0-9_]{16,}' \
|
||||
-e 'xox[baprs]-[A-Za-z0-9-]{10,}' \
|
||||
-e 'AKIA[0-9A-Z]{16}' \
|
||||
-e 'eyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}' \
|
||||
-e '-----BEGIN[A-Z ]*PRIVATE KEY-----' \
|
||||
-- "$1" 2>/dev/null
|
||||
rc=$?
|
||||
[ "$rc" -eq 1 ] || return 0
|
||||
LC_ALL=C grep -Eiq \
|
||||
-e "[A-Za-z0-9_]*(key|token|secret|passw|hmac|credential|bearer)[A-Za-z0-9_]*[[:space:]]*[=:][[:space:]]*[\"']?[A-Za-z0-9+/=_-]{16,}" \
|
||||
-- "$1" 2>/dev/null
|
||||
rc=$?
|
||||
[ "$rc" -eq 1 ] || return 0
|
||||
return 1
|
||||
}
|
||||
|
||||
# _capture_allowed ORIGIN RAW RESOLVED — 0 iff byte capture may even be
|
||||
# ATTEMPTED for this member (the deny gate still runs after, and wins).
|
||||
# POLARITY (#964 review, case D): core members (the adapter file, the
|
||||
# watch-list) are capture-eligible — they ARE the preimage definition this
|
||||
# tool exists to snapshot (acceptance (a)). Extras are RECORD-ONLY unless
|
||||
# listed in WAKE_PREIMAGE_CAPTURE.
|
||||
#
|
||||
# The allowlist is matched with the SAME both-forms discipline as the deny
|
||||
# list (entry raw/resolved vs candidate raw/resolved): two lists keyed on
|
||||
# different strings would let a symlink alias slip between them. "Deny wins
|
||||
# over opt-in" is a precedence rule, not a guarantee both lists see the same
|
||||
# path — the guarantee comes from _check_one running _deny_path on both forms
|
||||
# FIRST, independent of anything matched here.
|
||||
_capture_allowed() {
|
||||
local origin="$1" raw="$2" resolved="$3"
|
||||
[ "$origin" = "core" ] && return 0
|
||||
[ -n "${WAKE_PREIMAGE_CAPTURE:-}" ] || return 1
|
||||
local IFS=':' e er
|
||||
for e in $WAKE_PREIMAGE_CAPTURE; do
|
||||
[ -n "$e" ] || continue
|
||||
er="$e"
|
||||
[ -e "$e" ] && er="$(_realpath_or_self "$e")"
|
||||
if [ "$e" = "$raw" ] || [ "$e" = "$resolved" ] ||
|
||||
[ "$er" = "$raw" ] || [ "$er" = "$resolved" ]; then
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
# --- ledger primitives ------------------------------------------------------
|
||||
|
||||
# _ledger_head PATH — print the LAST ledger row for PATH (empty if none).
|
||||
# Non-zero (loud) if the ledger exists but is unparseable: a corrupt ledger
|
||||
# must REFUSE to compare, never silently re-baseline (that would erase the
|
||||
# attribution history this tool exists to keep).
|
||||
_ledger_head() {
|
||||
local path="$1"
|
||||
[ -f "$LEDGER" ] || return 0
|
||||
if [ -s "$LEDGER" ] && ! jq -cn 'inputs' <"$LEDGER" >/dev/null 2>&1; then
|
||||
echo "preimage.sh: FAIL LOUD — preimage ledger $LEDGER is unparseable; REFUSING to compare or re-baseline over corrupt history. Investigate/restore the ledger." >&2
|
||||
return 1
|
||||
fi
|
||||
jq -c --arg p "$path" 'select(.path == $p)' "$LEDGER" 2>/dev/null | tail -n 1
|
||||
}
|
||||
|
||||
# _ledger_append ROW_JSON — append one row atomically (read + append + rename;
|
||||
# rows are small and the writer is serialized by the preimage lock).
|
||||
_ledger_append() {
|
||||
local row="$1"
|
||||
{
|
||||
[ -f "$LEDGER" ] && cat "$LEDGER"
|
||||
printf '%s\n' "$row"
|
||||
} | _atomic_write "$LEDGER"
|
||||
}
|
||||
|
||||
# --- the check itself -------------------------------------------------------
|
||||
|
||||
# _check_one ORIGIN RAW PATH ENQUEUE — compare PATH (the resolved form; the
|
||||
# ledger keys on it) to its ledger head; on change record (+bytes only if
|
||||
# capture-eligible and not denied) and optionally enqueue. Prints nothing on
|
||||
# no-change. Returns: 0 ok (changed or not), 1 infrastructure failure.
|
||||
_check_one() {
|
||||
local origin="$1" raw="$2" path="$3" enqueue="$4"
|
||||
local cur_sha size mtime denied="" refused=""
|
||||
|
||||
if [ -f "$path" ]; then
|
||||
cur_sha="$(_hash_stdin <"$path")" || return 1
|
||||
[ -n "$cur_sha" ] || {
|
||||
echo "preimage.sh: FAIL LOUD — could not hash $path" >&2
|
||||
return 1
|
||||
}
|
||||
size="$(wc -c <"$path" | tr -d '[:space:]')"
|
||||
# Portable mtime (GNU stat -c / BSD stat -f).
|
||||
mtime="$(stat -c %Y -- "$path" 2>/dev/null || stat -f %m -- "$path" 2>/dev/null || echo 0)"
|
||||
local cap="${WAKE_PREIMAGE_MAX_BYTES:-1048576}"
|
||||
case "$cap" in '' | *[!0-9]*) cap=1048576 ;; esac
|
||||
# Gate order matters: path deny FIRST, on BOTH forms, INDEPENDENT of the
|
||||
# capture allowlist — so an allowlisted symlink whose target is denied
|
||||
# refuses no matter what string the allowlist matched (a known credential
|
||||
# store is refused without a content probe; the hash read above is
|
||||
# deliberate: a sha256 discloses nothing about the bytes). Then polarity
|
||||
# (record-only extras never reach the content probe — case D must be safe
|
||||
# WITHOUT the shape list), then size cap (never content-grep an oversized
|
||||
# file), content shape last.
|
||||
if _deny_path "$raw" "$path"; then
|
||||
denied="credential-store path (deny list)"
|
||||
elif ! _capture_allowed "$origin" "$raw" "$path"; then
|
||||
denied="extra is record-only by default (byte capture requires WAKE_PREIMAGE_CAPTURE; deny rules still win)"
|
||||
elif [ "$size" -gt "$cap" ]; then
|
||||
denied="exceeds WAKE_PREIMAGE_MAX_BYTES ($size > $cap)"
|
||||
elif _deny_content "$path"; then
|
||||
denied="secret-shaped content"
|
||||
fi
|
||||
else
|
||||
# ABSENT is a state (deletion of a preimage file is a change to record).
|
||||
cur_sha="ABSENT"
|
||||
size=0
|
||||
mtime=0
|
||||
fi
|
||||
|
||||
local head prev_sha=""
|
||||
head="$(_ledger_head "$path")" || return 1
|
||||
[ -n "$head" ] && prev_sha="$(jq -r '.sha256 // ""' <<<"$head")"
|
||||
|
||||
if [ "$cur_sha" = "$prev_sha" ]; then
|
||||
return 0 # unchanged — no row, no enqueue (idempotent)
|
||||
fi
|
||||
|
||||
# --- capture bytes (content-addressed) unless the hard gate refuses -------
|
||||
local captured=true
|
||||
if [ "$cur_sha" != "ABSENT" ]; then
|
||||
if [ -n "$denied" ]; then
|
||||
captured=false
|
||||
refused="$denied"
|
||||
echo "preimage.sh: byte capture WITHHELD for $path ($denied) — hash/size/mtime recorded, content NOT stored (#958)." >&2
|
||||
else
|
||||
mkdir -p "$OBJECTS" || return 1
|
||||
if [ ! -f "$OBJECTS/$cur_sha" ]; then
|
||||
if ! _atomic_write "$OBJECTS/$cur_sha" <"$path"; then
|
||||
echo "preimage.sh: FAIL LOUD — durable object write failed for $path ($OBJECTS/$cur_sha); NO ledger row recorded (a row whose bytes were never durably kept would be a false witness)." >&2
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
local ts row
|
||||
ts="$(date +%s)"
|
||||
row="$(jq -cn \
|
||||
--arg path "$path" \
|
||||
--arg sha "$cur_sha" \
|
||||
--arg prev "$prev_sha" \
|
||||
--argjson size "$size" \
|
||||
--argjson mtime "$mtime" \
|
||||
--argjson ts "$ts" \
|
||||
--argjson captured "$captured" \
|
||||
--arg refused "$refused" \
|
||||
'{ts:$ts, path:$path, sha256:$sha, size:$size, mtime:$mtime, captured:$captured}
|
||||
+ (if $prev != "" then {prev:$prev} else {} end)
|
||||
+ (if $refused != "" then {refused:$refused} else {} end)')" || return 1
|
||||
if ! _ledger_append "$row"; then
|
||||
echo "preimage.sh: FAIL LOUD — durable ledger append failed ($LEDGER)" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
if [ -z "$head" ]; then
|
||||
# First-seen: baseline SILENTLY (row only, no enqueue) — the same idiom as
|
||||
# the detector's first-seen source baseline.
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo "preimage.sh: preimage definition CHANGED — $path ($prev_sha -> $cur_sha); prior bytes: $OBJECTS/$prev_sha" >&2
|
||||
|
||||
if [ "$enqueue" = "1" ]; then
|
||||
# First-class cause line (acceptance (c)): ONE class=actionable entry per
|
||||
# changed preimage file, allocated by the store's single allocator (#908).
|
||||
# `path` is a §2.1 hard locator, so the digest renders it — never
|
||||
# quarantined. Callers run this BEFORE polling sources, so this seq is
|
||||
# LOWER than the per-source deltas the change explains.
|
||||
local locators
|
||||
locators="$(jq -cn \
|
||||
--arg path "$path" \
|
||||
--arg sha "$cur_sha" \
|
||||
--arg prev "$prev_sha" \
|
||||
'{kind:"preimage", id:$path, path:$path, observed_hash:$sha, prev_hash:$prev,
|
||||
preimage:true, reason:"preimage-definition-changed"}')"
|
||||
if ! "$STORE_SH" enqueue --class actionable --locators "$locators" --emit-ts "$(date +%s)" >/dev/null; then
|
||||
echo "preimage.sh: FAIL LOUD — store enqueue of the preimage-change entry FAILED for $path (the change IS recorded in the ledger; the first-class wake line is NOT — treat as infrastructure failure)." >&2
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
cmd_check() {
|
||||
local enqueue=0
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--enqueue)
|
||||
enqueue=1
|
||||
shift
|
||||
;;
|
||||
*)
|
||||
echo "preimage.sh check: unknown option '$1'" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
done
|
||||
_need_jq
|
||||
|
||||
local set_list
|
||||
if ! set_list="$(_preimage_set)"; then
|
||||
exit 1
|
||||
fi
|
||||
if [ -z "$set_list" ]; then
|
||||
# Nothing declared (no adapter/watch-list/extras in env) — an empty set is
|
||||
# a no-op, not an error: standalone invocations outside a wake runtime
|
||||
# must not fail loud for lacking one.
|
||||
return 0
|
||||
fi
|
||||
|
||||
mkdir -p "$PRE_DIR" || exit 1
|
||||
# Serialize concurrent checks (detector tick vs reconcile vs manual) — the
|
||||
# ledger append is read+rewrite, so two writers must not interleave.
|
||||
if ! _wake_lock_acquire "$PRE_DIR/preimage.lock"; then
|
||||
echo "preimage.sh: FAIL LOUD — cannot acquire preimage lock" >&2
|
||||
exit 1
|
||||
fi
|
||||
# #964 review (B11): an ABSENT/EMPTY ledger while objects/ still holds
|
||||
# prior captures is DELETED HISTORY, not a first install — the asymmetry is
|
||||
# locally detectable and is the whole detection. Re-baselining here would
|
||||
# silently absorb the next real change as first-seen (the exact erasure
|
||||
# this tool exists to prevent). ABSENT is not CORRUPT: it must not take the
|
||||
# first-install path either.
|
||||
if [ ! -s "$LEDGER" ] && [ -d "$OBJECTS" ] && [ -n "$(ls -A -- "$OBJECTS" 2>/dev/null)" ]; then
|
||||
echo "preimage.sh: FAIL LOUD — preimage ledger $LEDGER is ABSENT/EMPTY but $OBJECTS still holds prior objects: history was deleted; REFUSING to re-baseline over it (a silent restart would absorb the next real change as first-seen). Restore the ledger, or move objects/ aside explicitly after investigation." >&2
|
||||
_wake_lock_release
|
||||
exit 1
|
||||
fi
|
||||
local failed=0 origin raw resolved
|
||||
while IFS=$'\t' read -r origin raw resolved; do
|
||||
[ -n "$resolved" ] || continue
|
||||
_check_one "$origin" "$raw" "$resolved" "$enqueue" || failed=1
|
||||
done <<EOF
|
||||
$set_list
|
||||
EOF
|
||||
_wake_lock_release
|
||||
[ "$failed" -eq 0 ] || exit 1
|
||||
}
|
||||
|
||||
cmd_status() {
|
||||
_need_jq
|
||||
[ -f "$LEDGER" ] || {
|
||||
echo "preimage.sh: no ledger yet ($LEDGER)" >&2
|
||||
return 0
|
||||
}
|
||||
# Head row per path (last wins).
|
||||
jq -cs 'group_by(.path) | map(last) | .[]' "$LEDGER"
|
||||
}
|
||||
|
||||
cmd_set() {
|
||||
local s
|
||||
s="$(_preimage_set)" || exit 1
|
||||
[ -n "$s" ] && printf '%s\n' "$s" | cut -f3
|
||||
}
|
||||
|
||||
main() {
|
||||
[ $# -ge 1 ] || {
|
||||
usage
|
||||
exit 2
|
||||
}
|
||||
local cmd="$1"
|
||||
shift
|
||||
case "$cmd" in
|
||||
check) cmd_check "$@" ;;
|
||||
record) cmd_check ;;
|
||||
set) cmd_set "$@" ;;
|
||||
status) cmd_status "$@" ;;
|
||||
-h | --help | help) usage ;;
|
||||
*)
|
||||
echo "preimage.sh: unknown command '$cmd'" >&2
|
||||
usage
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
main "$@"
|
||||
@@ -440,17 +440,6 @@ cmd_reconcile() {
|
||||
: "$allow_enumerate"
|
||||
|
||||
local pairs kind id failed=0 unaccounted=0 enumerated=0
|
||||
|
||||
# #958 preimage provenance: same pre-step as the detector's poll tick, for
|
||||
# the path where the preimage changed while the detector was down — without
|
||||
# it, a changed adapter surfaces here only as N UNACCOUNTED enumerations
|
||||
# with no first-class cause line. Run BEFORE observing any source so the
|
||||
# cause entry's observed_seq precedes the enumerations it explains. Loud
|
||||
# infrastructure failure marks the reconcile failed but does not stop it.
|
||||
if ! "$SCRIPT_DIR/preimage.sh" check --enqueue; then
|
||||
echo "reconcile.sh: FAIL LOUD — preimage provenance check failed (see preimage.sh above); reconcile continues but exits non-zero." >&2
|
||||
failed=1
|
||||
fi
|
||||
pairs="$(printf '%s' "$json" | jq -r '[ .watches[].sources[] | "\(.kind)\t\(.id)" ] | unique | .[]')"
|
||||
if [ -z "$pairs" ]; then
|
||||
echo "reconcile.sh: watch-list declares no sources under watches[].sources[]" >&2
|
||||
|
||||
@@ -110,13 +110,9 @@ Commands:
|
||||
found); --repair removes exactly those
|
||||
rows (atomic, loud). The dead-letter
|
||||
ledger itself is NEVER modified (it is
|
||||
history). TWO residual classes are
|
||||
unprovable and never touched (#952):
|
||||
rows whose dead-letter evidence was
|
||||
pruned away, AND rows whose surviving
|
||||
evidence extracts an empty
|
||||
observed_hash (it can never satisfy
|
||||
the four-field conviction match).
|
||||
history). Rows whose dead-letter evidence
|
||||
was pruned are NOT provable and are
|
||||
never touched.
|
||||
cursors Print observed_seq / consumed_seq / depth.
|
||||
|
||||
Environment:
|
||||
@@ -567,17 +563,10 @@ cmd_quarantine_sync() {
|
||||
# it on (kind, id, observed_seq, observed_hash) AND row.observed_seq <=
|
||||
# consumed_seq: the per-key max_by merge means the surviving row's provenance IS
|
||||
# that quarantined entry (a healed row differs in seq/hash and never matches).
|
||||
# PROVABILITY BOUND — TWO residual classes, both unprovable (#952): (1) a row
|
||||
# whose dead-letter evidence was pruned/rotated away — no evidence to convict
|
||||
# on; (2) a row whose dead-letter evidence SURVIVES but extracts an empty
|
||||
# observed_hash (e.g. a deliberately non-conformant locator: nested
|
||||
# .locators.* with no observed_hash key) — evidence exists but can never
|
||||
# satisfy the four-field match, because _record_last_consumed only ever writes
|
||||
# rows with a NON-empty hash. Neither class is touched — this audit only ever
|
||||
# removes what the ledger can convict, and the clean-sweep message names BOTH
|
||||
# classes: "no evidence" and "evidence unusable" are different operator
|
||||
# conclusions. The dead-letter ledger itself is history and is NEVER modified
|
||||
# here.
|
||||
# PROVABILITY BOUND: a row whose dead-letter evidence was pruned/rotated away is
|
||||
# NOT provable and is never touched — this audit only ever removes what the
|
||||
# ledger can convict. The dead-letter ledger itself is history and is NEVER
|
||||
# modified here.
|
||||
cmd_quarantine_audit() {
|
||||
local repair=0
|
||||
while [ $# -gt 0 ]; do
|
||||
@@ -616,7 +605,7 @@ cmd_quarantine_audit() {
|
||||
)) | length) > 0)
|
||||
' "$rec" 2>/dev/null || true)"
|
||||
if [ -z "$false_rows" ]; then
|
||||
echo "store.sh quarantine-audit: OK — no provably-false consumed-hash rows. Two residual classes are unprovable and were NOT judged: rows whose dead-letter evidence was pruned/rotated away (no evidence to convict on), and rows whose surviving dead-letter evidence extracts an empty observed_hash (evidence exists but can never satisfy the four-field conviction match)."
|
||||
echo "store.sh quarantine-audit: OK — no provably-false consumed-hash rows (rows without surviving dead-letter evidence are not provable and were not judged)"
|
||||
return 0
|
||||
fi
|
||||
local n row
|
||||
|
||||
@@ -1,534 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# test-wake-preimage.sh — RED-FIRST invariant harness for A11 (#958): durable
|
||||
# provenance for the OPERATOR-SIDE preimage definition (preimage.sh).
|
||||
#
|
||||
# Each test asserts ONE #958 invariant and is designed to go RED if it
|
||||
# regresses:
|
||||
# P1 first-seen -> SILENT baseline (ledger rows, NO enqueue) (detector idiom)
|
||||
# P2 change -> attributable from durable state alone: new ledger row with
|
||||
# prev, PRIOR BYTES retrievable content-addressed, first-class actionable
|
||||
# enqueued with a §2.1 hard locator (path) (acceptance a+c)
|
||||
# P3 detector ordering: the preimage cause line is enqueued at a LOWER
|
||||
# observed_seq than the per-source delta it explains (acceptance c)
|
||||
# P4 credential-store PATH is refused — bytes NEVER captured, even when the
|
||||
# operator lists it by error (hash/mtime row still written) (acceptance b)
|
||||
# P5 secret-SHAPED content is refused — refusal, not redaction (acceptance b)
|
||||
# P6 deletion of a preimage file is a tracked CHANGE (ABSENT), not an error
|
||||
# P7 no-change is idempotent (no row growth, no enqueue)
|
||||
# P8 unresolvable adapter command -> FAIL LOUD, never "no change" (D2/#955 class)
|
||||
# P9 corrupt ledger -> REFUSE to compare/re-baseline (loud, no append)
|
||||
# P10 oversized file -> bytes refused, hash still recorded (no data hoovering)
|
||||
# P11 reconcile pre-step: a preimage change surfaces as a first-class line
|
||||
# BEFORE the UNACCOUNTED enumerations it explains (acceptance c)
|
||||
# P12 detector: preimage infra failure is LOUD (pass exits non-zero) but does
|
||||
# NOT starve source observation
|
||||
#
|
||||
# Regression needles from the #964 NOT-CLEAR review (decoy method: plant the
|
||||
# bytes, grep the WHOLE state dir — the decoy does not know what the code
|
||||
# believes):
|
||||
# P13 case C: symlinked/renamed credential store (mosaic-home itself behind
|
||||
# a symlink; a renamed target outside every path rule) — refused on BOTH
|
||||
# path forms / by content, bytes grep-absent (acceptance b)
|
||||
# P14 case D: prefix-less high-entropy key (detector.env class) — safe
|
||||
# WITHOUT opt-in by POLARITY alone (must hold with the shape list
|
||||
# deleted), and refused WITH opt-in by the named-assignment shape
|
||||
# P15 polarity: extras are RECORD-ONLY by default (change row + enqueue, no
|
||||
# bytes); byte capture requires explicit WAKE_PREIMAGE_CAPTURE opt-in
|
||||
# P16 case B11: ledger deleted while objects/ survives — LOUD refusal to
|
||||
# re-baseline; the v2->v3 change is NOT absorbed as first-seen
|
||||
# P17 allowlist symmetry: an allowlisted symlink whose TARGET is denied
|
||||
# refuses (deny runs first, on both forms — precedence is not path
|
||||
# agreement); a symlink to an ALLOWED target still captures (the fix
|
||||
# must not close case C by breaking every symlinked path)
|
||||
#
|
||||
# Uses FAKE/STUB sources only (no live network). Isolated per test.
|
||||
# shellcheck disable=SC2030,SC2031
|
||||
set -uo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PRE="$SCRIPT_DIR/preimage.sh"
|
||||
STORE="$SCRIPT_DIR/store.sh"
|
||||
DET="$SCRIPT_DIR/detector.sh"
|
||||
RECON="$SCRIPT_DIR/reconcile.sh"
|
||||
|
||||
command -v jq >/dev/null 2>&1 || {
|
||||
echo "SKIP: jq not available" >&2
|
||||
exit 0
|
||||
}
|
||||
|
||||
TMP_ROOT="$(mktemp -d)"
|
||||
trap 'rm -rf "$TMP_ROOT"' EXIT
|
||||
|
||||
FAILFILE="$TMP_ROOT/failures"
|
||||
: >"$FAILFILE"
|
||||
pass=0
|
||||
fail_msg() {
|
||||
echo " FAIL: $*" >&2
|
||||
echo "x" >>"$FAILFILE"
|
||||
}
|
||||
ok() { pass=$((pass + 1)); }
|
||||
|
||||
fresh_state() {
|
||||
local d="$TMP_ROOT/$1"
|
||||
rm -rf "$d"
|
||||
mkdir -p "$d"
|
||||
printf '%s' "$d"
|
||||
}
|
||||
depth() { "$STORE" cursors | sed -n 's/pending_depth=//p'; }
|
||||
state_dir() { printf '%s/default' "$WAKE_STATE_HOME"; }
|
||||
ledger_rows() {
|
||||
local f
|
||||
f="$(state_dir)/preimage/preimage-ledger.jsonl"
|
||||
[ -f "$f" ] && wc -l <"$f" | tr -d '[:space:]' || echo 0
|
||||
}
|
||||
|
||||
# make_stub DIR — a source adapter reading $DIR/<kind>_<id> (same shape as the
|
||||
# reconcile harness stub).
|
||||
make_stub() {
|
||||
local dir="$1"
|
||||
cat >"$dir/adapter.sh" <<EOF
|
||||
#!/usr/bin/env bash
|
||||
set -u
|
||||
kind="\$1"; id="\$2"; base="$dir/\${kind}_\${id}"; rc=0
|
||||
[ -f "\$base.rc" ] && rc="\$(cat "\$base.rc")"
|
||||
[ -f "\$base" ] && cat "\$base"
|
||||
exit "\$rc"
|
||||
EOF
|
||||
chmod +x "$dir/adapter.sh"
|
||||
}
|
||||
|
||||
make_wl() { # make_wl FILE — one repo source r1
|
||||
cat >"$1" <<'EOF'
|
||||
{ "schema_version": 1,
|
||||
"repos": [ { "id": "r1" } ],
|
||||
"watches": [ { "lane": "L", "sources": [ { "kind": "repo", "id": "r1" } ] } ] }
|
||||
EOF
|
||||
}
|
||||
|
||||
echo "== P1: first-seen -> SILENT baseline (rows, no enqueue) =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p1)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p1fx"; mkdir -p "$fx"
|
||||
make_stub "$fx"; make_wl "$fx/wl.json"
|
||||
printf 'extra v1\n' >"$fx/extra.env"
|
||||
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json" WAKE_PREIMAGE_EXTRA="$fx/extra.env"
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "P1: baseline check must exit 0 (rc=$rc) [$out]"
|
||||
[ "$(ledger_rows)" -eq 3 ] || fail_msg "P1: expected 3 baseline rows (adapter, watch-list, extra), got $(ledger_rows)"
|
||||
[ "$(depth)" = "0" ] || fail_msg "P1: first-seen must NOT enqueue (depth=$(depth))"
|
||||
) && ok
|
||||
|
||||
echo "== P2: adapter change -> prior bytes + first-class hard-locator enqueue =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p2)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p2fx"; mkdir -p "$fx"
|
||||
make_stub "$fx"; make_wl "$fx/wl.json"
|
||||
# Acceptance (a) names the OPERATOR ADAPTER — a core member. Core members
|
||||
# capture by default (they ARE the preimage definition); extras do not (P15).
|
||||
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json"
|
||||
unset WAKE_PREIMAGE_EXTRA WAKE_PREIMAGE_CAPTURE MOSAIC_HOME
|
||||
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P2: baseline failed"
|
||||
cp "$fx/adapter.sh" "$fx/adapter.v1"
|
||||
old_sha="$(sha256sum "$fx/adapter.sh" | awk '{print $1}')"
|
||||
printf '# adapter v2 CHANGED\n' >>"$fx/adapter.sh"
|
||||
new_sha="$(sha256sum "$fx/adapter.sh" | awk '{print $1}')"
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "P2: change check must exit 0 (rc=$rc) [$out]"
|
||||
sd="$(state_dir)"
|
||||
row="$(jq -c --arg p "$(realpath "$fx/adapter.sh")" 'select(.path == $p)' "$sd/preimage/preimage-ledger.jsonl" | tail -n1)"
|
||||
[ "$(jq -r '.prev // ""' <<<"$row")" = "$old_sha" ] || fail_msg "P2: new row must carry prev=<old sha> [$row]"
|
||||
[ "$(jq -r '.sha256' <<<"$row")" = "$new_sha" ] || fail_msg "P2: new row sha mismatch [$row]"
|
||||
# Acceptance (a): the PRIOR BYTES are retrievable from durable state alone.
|
||||
cmp -s "$sd/preimage/objects/$old_sha" "$fx/adapter.v1" || fail_msg "P2: prior bytes missing or not byte-exact"
|
||||
cmp -s "$sd/preimage/objects/$new_sha" "$fx/adapter.sh" || fail_msg "P2: current bytes object missing or mismatched"
|
||||
# Acceptance (c): ONE first-class actionable with a §2.1 hard locator (path).
|
||||
[ "$(depth)" = "1" ] || fail_msg "P2: exactly one enqueue expected (depth=$(depth))"
|
||||
ent="$(tail -n1 "$sd/pending.jsonl")"
|
||||
[ "$(jq -r '.class' <<<"$ent")" = "actionable" ] || fail_msg "P2: entry class must be actionable [$ent]"
|
||||
[ "$(jq -r '.locators.kind' <<<"$ent")" = "preimage" ] || fail_msg "P2: locator kind must be preimage [$ent]"
|
||||
[ -n "$(jq -r '.locators.path // ""' <<<"$ent")" ] || fail_msg "P2: locator must carry path (hard locator) [$ent]"
|
||||
[ "$(jq -r '.locators.observed_hash' <<<"$ent")" = "$new_sha" ] || fail_msg "P2: locator observed_hash mismatch [$ent]"
|
||||
[ "$(jq -r '.locators.prev_hash' <<<"$ent")" = "$old_sha" ] || fail_msg "P2: locator prev_hash mismatch [$ent]"
|
||||
) && ok
|
||||
|
||||
echo "== P3: detector orders the cause line BEFORE the delta it explains =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p3)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p3fx"; mkdir -p "$fx"
|
||||
make_stub "$fx"; make_wl "$fx/wl.json"
|
||||
printf 'r1 state v1\n' >"$fx/repo_r1"
|
||||
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json"
|
||||
unset WAKE_PREIMAGE_EXTRA
|
||||
"$DET" poll-once >/dev/null 2>&1 || fail_msg "P3: baseline poll failed"
|
||||
[ "$(depth)" = "0" ] || fail_msg "P3: baseline poll must enqueue nothing (depth=$(depth))"
|
||||
# Change the ADAPTER (preimage) and the source state in the same window.
|
||||
printf '# comment: adapter changed\n' >>"$fx/adapter.sh"
|
||||
printf 'r1 state v2\n' >"$fx/repo_r1"
|
||||
"$DET" poll-once >/dev/null 2>&1 || fail_msg "P3: second poll failed"
|
||||
sd="$(state_dir)"
|
||||
pre_seq="$(jq -r 'select(.locators.kind == "preimage") | .observed_seq' "$sd/pending.jsonl" | head -n1)"
|
||||
src_seq="$(jq -r 'select(.locators.kind == "repo") | .observed_seq' "$sd/pending.jsonl" | head -n1)"
|
||||
[ -n "$pre_seq" ] || fail_msg "P3: no preimage cause entry enqueued"
|
||||
[ -n "$src_seq" ] || fail_msg "P3: no source delta entry enqueued"
|
||||
if [ -n "$pre_seq" ] && [ -n "$src_seq" ]; then
|
||||
[ "$pre_seq" -lt "$src_seq" ] || fail_msg "P3: cause line must precede the delta (preimage seq=$pre_seq, source seq=$src_seq)"
|
||||
fi
|
||||
) && ok
|
||||
|
||||
echo "== P4: credential-store PATH refused — bytes never captured (acceptance b) =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p4)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p4fx"; mkdir -p "$fx"
|
||||
export MOSAIC_HOME="$fx/mosaic-home"
|
||||
mkdir -p "$MOSAIC_HOME"
|
||||
secret='cred-value-P4-do-not-capture'
|
||||
printf '{"svc":{"token":"%s"}}\n' "$secret" >"$MOSAIC_HOME/credentials.json"
|
||||
# Operator error: the credential store listed as a preimage extra — AND
|
||||
# explicitly opted in to capture. Deny must win over the opt-in (#964: a
|
||||
# path both denied and allowlisted must refuse).
|
||||
export WAKE_PREIMAGE_EXTRA="$MOSAIC_HOME/credentials.json"
|
||||
export WAKE_PREIMAGE_CAPTURE="$MOSAIC_HOME/credentials.json"
|
||||
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "P4: refusal is not an infra failure (rc=$rc) [$out]"
|
||||
sd="$(state_dir)"
|
||||
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
|
||||
[ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P4: row must record captured=false [$row]"
|
||||
jq -r '.refused // ""' <<<"$row" | grep -qi 'path' || fail_msg "P4: refusal must name the path deny [$row]"
|
||||
# THE invariant: the secret bytes exist NOWHERE under the wake state dir.
|
||||
if grep -rq "$secret" "$sd" 2>/dev/null; then
|
||||
fail_msg "P4: credential bytes leaked into the wake state dir"
|
||||
fi
|
||||
) && ok
|
||||
|
||||
echo "== P5: secret-SHAPED content refused (refusal, not redaction) =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p5)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p5fx"; mkdir -p "$fx"
|
||||
tok='ghp_abcdefghijklmnop0123456789ABCDEF'
|
||||
printf 'export MY_TOKEN=%s\n' "$tok" >"$fx/leaky.env"
|
||||
# Opted in, so the CONTENT gate (not record-only polarity) is what refuses.
|
||||
export WAKE_PREIMAGE_EXTRA="$fx/leaky.env" WAKE_PREIMAGE_CAPTURE="$fx/leaky.env"
|
||||
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "P5: refusal is not an infra failure (rc=$rc) [$out]"
|
||||
sd="$(state_dir)"
|
||||
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
|
||||
[ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P5: row must record captured=false [$row]"
|
||||
jq -r '.refused // ""' <<<"$row" | grep -qi 'secret' || fail_msg "P5: refusal must name secret-shaped content [$row]"
|
||||
if grep -rq "$tok" "$sd" 2>/dev/null; then
|
||||
fail_msg "P5: secret-shaped bytes leaked into the wake state dir"
|
||||
fi
|
||||
) && ok
|
||||
|
||||
echo "== P6: deletion is a tracked CHANGE (ABSENT), not an error =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p6)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p6fx"; mkdir -p "$fx"
|
||||
printf 'to be deleted\n' >"$fx/gone.env"
|
||||
export WAKE_PREIMAGE_EXTRA="$fx/gone.env"
|
||||
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
|
||||
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P6: baseline failed"
|
||||
rm -f "$fx/gone.env"
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "P6: deletion check must exit 0 (rc=$rc) [$out]"
|
||||
sd="$(state_dir)"
|
||||
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
|
||||
[ "$(jq -r '.sha256' <<<"$row")" = "ABSENT" ] || fail_msg "P6: deletion must record sha256=ABSENT [$row]"
|
||||
[ "$(depth)" = "1" ] || fail_msg "P6: deletion is a change and must enqueue (depth=$(depth))"
|
||||
) && ok
|
||||
|
||||
echo "== P7: no-change is idempotent =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p7)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p7fx"; mkdir -p "$fx"
|
||||
printf 'stable\n' >"$fx/stable.env"
|
||||
export WAKE_PREIMAGE_EXTRA="$fx/stable.env"
|
||||
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
|
||||
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P7: baseline failed"
|
||||
r1="$(ledger_rows)"
|
||||
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P7: second check failed"
|
||||
[ "$(ledger_rows)" = "$r1" ] || fail_msg "P7: no-change must not append rows ($r1 -> $(ledger_rows))"
|
||||
[ "$(depth)" = "0" ] || fail_msg "P7: no-change must not enqueue (depth=$(depth))"
|
||||
) && ok
|
||||
|
||||
echo "== P8: unresolvable adapter -> FAIL LOUD, never 'no change' (D2 class) =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p8)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
export WAKE_DETECTOR_SOURCE_CMD="$TMP_ROOT/does-not-exist-adapter"
|
||||
unset WAKE_WATCH_LIST WAKE_PREIMAGE_EXTRA MOSAIC_HOME
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -ne 0 ] || fail_msg "P8: an unobservable preimage definition must FAIL LOUD (rc=0)"
|
||||
echo "$out" | grep -qi 'does not resolve' || fail_msg "P8: the failure must name the unresolvable adapter [$out]"
|
||||
) && ok
|
||||
|
||||
echo "== P9: corrupt ledger -> REFUSE to compare/re-baseline =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p9)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p9fx"; mkdir -p "$fx"
|
||||
printf 'v1\n' >"$fx/f.env"
|
||||
export WAKE_PREIMAGE_EXTRA="$fx/f.env"
|
||||
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
|
||||
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P9: baseline failed"
|
||||
sd="$(state_dir)"
|
||||
printf 'NOT-JSON-GARBAGE{{{\n' >>"$sd/preimage/preimage-ledger.jsonl"
|
||||
r1="$(wc -l <"$sd/preimage/preimage-ledger.jsonl")"
|
||||
printf 'v2 changed\n' >"$fx/f.env"
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -ne 0 ] || fail_msg "P9: a corrupt ledger must FAIL LOUD (rc=0)"
|
||||
echo "$out" | grep -qi 'unparseable' || fail_msg "P9: the failure must name the corrupt ledger [$out]"
|
||||
[ "$(wc -l <"$sd/preimage/preimage-ledger.jsonl")" = "$r1" ] || fail_msg "P9: nothing may be appended over corrupt history"
|
||||
) && ok
|
||||
|
||||
echo "== P10: oversized file -> bytes refused, hash still recorded =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p10)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p10fx"; mkdir -p "$fx"
|
||||
head -c 200 /dev/zero | tr '\0' 'A' >"$fx/big.bin"
|
||||
# Opted in, so the SIZE gate (not record-only polarity) is what refuses.
|
||||
export WAKE_PREIMAGE_EXTRA="$fx/big.bin" WAKE_PREIMAGE_MAX_BYTES=64
|
||||
export WAKE_PREIMAGE_CAPTURE="$fx/big.bin"
|
||||
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "P10: size refusal is not an infra failure (rc=$rc) [$out]"
|
||||
sd="$(state_dir)"
|
||||
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
|
||||
[ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P10: row must record captured=false [$row]"
|
||||
jq -r '.refused // ""' <<<"$row" | grep -qi 'MAX_BYTES' || fail_msg "P10: refusal must name the size cap [$row]"
|
||||
sha="$(jq -r '.sha256' <<<"$row")"
|
||||
[ "$sha" = "$(sha256sum "$fx/big.bin" | awk '{print $1}')" ] || fail_msg "P10: hash must still be recorded [$row]"
|
||||
[ ! -f "$sd/preimage/objects/$sha" ] || fail_msg "P10: oversized bytes must NOT be stored"
|
||||
) && ok
|
||||
|
||||
echo "== P11: reconcile surfaces the cause line before its enumerations =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p11)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p11fx"; mkdir -p "$fx"
|
||||
make_stub "$fx"; make_wl "$fx/wl.json"
|
||||
printf 'r1 state\n' >"$fx/repo_r1"
|
||||
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json"
|
||||
unset WAKE_PREIMAGE_EXTRA MOSAIC_HOME
|
||||
# Baseline the preimage set, then change the adapter while "the detector is
|
||||
# down" — the reconcile path must still produce the first-class cause line.
|
||||
"$PRE" check >/dev/null 2>&1 || fail_msg "P11: preimage baseline failed"
|
||||
printf '# adapter changed while detector down\n' >>"$fx/adapter.sh"
|
||||
"$RECON" reconcile >/dev/null 2>&1 # rc 1 expected (unaccounted enumerated)
|
||||
sd="$(state_dir)"
|
||||
pre_seq="$(jq -r 'select(.locators.kind == "preimage") | .observed_seq' "$sd/pending.jsonl" | head -n1)"
|
||||
enum_seq="$(jq -r 'select(.locators.reconciled == true) | .observed_seq' "$sd/pending.jsonl" | head -n1)"
|
||||
[ -n "$pre_seq" ] || fail_msg "P11: reconcile must enqueue the preimage cause line"
|
||||
[ -n "$enum_seq" ] || fail_msg "P11: reconcile must still enumerate the unaccounted source"
|
||||
if [ -n "$pre_seq" ] && [ -n "$enum_seq" ]; then
|
||||
[ "$pre_seq" -lt "$enum_seq" ] || fail_msg "P11: cause line must precede the enumeration (preimage seq=$pre_seq, enum seq=$enum_seq)"
|
||||
fi
|
||||
) && ok
|
||||
|
||||
echo "== P12: detector — preimage infra failure is LOUD but does not starve observation =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p12)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p12fx"; mkdir -p "$fx"
|
||||
make_stub "$fx"; make_wl "$fx/wl.json"
|
||||
printf 'r1 state\n' >"$fx/repo_r1"
|
||||
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh" WAKE_WATCH_LIST="$fx/wl.json"
|
||||
unset WAKE_PREIMAGE_EXTRA MOSAIC_HOME
|
||||
# Corrupt the ledger BEFORE the first poll: the preimage check fails loud,
|
||||
# but the poll must still observe + baseline the source.
|
||||
sd="$(state_dir)"
|
||||
mkdir -p "$sd/preimage"
|
||||
printf 'NOT-JSON-GARBAGE{{{\n' >"$sd/preimage/preimage-ledger.jsonl"
|
||||
out="$("$DET" poll-once 2>&1)"; rc=$?
|
||||
[ "$rc" -ne 0 ] || fail_msg "P12: the pass must exit non-zero on preimage infra failure"
|
||||
echo "$out" | grep -qi 'preimage' || fail_msg "P12: the failure must name the preimage check [$out]"
|
||||
n="$(find "$sd/detector" -name 'watch-*.hash' 2>/dev/null | wc -l | tr -d '[:space:]')"
|
||||
[ "$n" -ge 1 ] || fail_msg "P12: source observation must still proceed (no watch hash baselined)"
|
||||
) && ok
|
||||
|
||||
echo "== P13: symlinked/renamed credential store refused on BOTH path forms (#964 C) =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p13)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p13fx"; mkdir -p "$fx"
|
||||
# The mosaic home is ITSELF behind a symlink: a deny list written in
|
||||
# unresolved paths cannot match a candidate that was realpath'd before the
|
||||
# deny saw it — unless the anchors are resolved too.
|
||||
mkdir -p "$fx/real-store/credentials"
|
||||
ln -s "$fx/real-store" "$fx/mh"
|
||||
export MOSAIC_HOME="$fx/mh"
|
||||
s1='decoy-P13a-renamed-target-under-store'
|
||||
printf 'x_token=%s\n' "$s1" >"$fx/mh/credentials/brain-creds.json"
|
||||
# And a renamed target OUTSIDE every known store location, reached via a
|
||||
# benign-looking symlink: no path rule can name it — the content gate must
|
||||
# hold on the resolved file.
|
||||
s2='0123456789abcdef0123456789abcdefP13b'
|
||||
printf 'service_hmac_key=%s\n' "$s2" >"$fx/renamed-anywhere.cfg"
|
||||
ln -s "$fx/renamed-anywhere.cfg" "$fx/link-b.env"
|
||||
export WAKE_PREIMAGE_EXTRA="$fx/mh/credentials/brain-creds.json:$fx/link-b.env"
|
||||
# Explicit opt-in for BOTH: deny must win over the allowlist.
|
||||
export WAKE_PREIMAGE_CAPTURE="$fx/mh/credentials/brain-creds.json:$fx/link-b.env"
|
||||
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "P13: refusal is not an infra failure (rc=$rc) [$out]"
|
||||
sd="$(state_dir)"
|
||||
n="$(jq -r 'select(.captured == false) | .path' "$sd/preimage/preimage-ledger.jsonl" | wc -l | tr -d '[:space:]')"
|
||||
[ "$n" = "2" ] || fail_msg "P13: both decoys must record captured=false (got $n)"
|
||||
# THE invariant (decoy method): the planted bytes exist NOWHERE in state.
|
||||
if grep -rq "$s1" "$sd" 2>/dev/null; then
|
||||
fail_msg "P13: renamed-target store bytes leaked into the wake state dir"
|
||||
fi
|
||||
if grep -rq "$s2" "$sd" 2>/dev/null; then
|
||||
fail_msg "P13: prefix-less key bytes leaked into the wake state dir"
|
||||
fi
|
||||
) && ok
|
||||
|
||||
echo "== P14: detector.env-class key — safe WITHOUT opt-in by polarity, refused WITH opt-in by shape (#964 D) =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p14)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p14fx"; mkdir -p "$fx"
|
||||
hm='9f8e7d6c5b4a39281706f5e4d3c2b1a09f8e7d6c'
|
||||
printf 'WAKE_HMAC_KEY=%s\n' "$hm" >"$fx/detector.env"
|
||||
export WAKE_PREIMAGE_EXTRA="$fx/detector.env"
|
||||
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME WAKE_PREIMAGE_CAPTURE
|
||||
# (a) NO opt-in: polarity alone keeps the bytes out. This leg must hold even
|
||||
# with the content shape list deleted — the shape is defense-in-depth for
|
||||
# the opt-in path, never the thing that makes case D read safe.
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "P14a: record-only tracking must succeed (rc=$rc) [$out]"
|
||||
sd="$(state_dir)"
|
||||
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
|
||||
[ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P14a: extra must be record-only without opt-in [$row]"
|
||||
if grep -rq "$hm" "$sd" 2>/dev/null; then
|
||||
fail_msg "P14a: key bytes leaked without any opt-in"
|
||||
fi
|
||||
# (b) The operator opts the env file in (the exact error the old usage text
|
||||
# invited): the named-assignment shape must still refuse the bytes.
|
||||
export WAKE_PREIMAGE_CAPTURE="$fx/detector.env"
|
||||
printf 'WAKE_HMAC_KEY=%s\nrotated=1\n' "$hm" >"$fx/detector.env"
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "P14b: refusal is not an infra failure (rc=$rc) [$out]"
|
||||
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
|
||||
[ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P14b: opted-in key material must still refuse [$row]"
|
||||
jq -r '.refused // ""' <<<"$row" | grep -qi 'secret' || fail_msg "P14b: refusal must name secret-shaped content [$row]"
|
||||
if grep -rq "$hm" "$sd" 2>/dev/null; then
|
||||
fail_msg "P14b: key bytes leaked despite refusal"
|
||||
fi
|
||||
) && ok
|
||||
|
||||
echo "== P15: extras are RECORD-ONLY by default; capture requires explicit opt-in =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p15)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p15fx"; mkdir -p "$fx"
|
||||
printf 'plain v1\n' >"$fx/notes.cfg"
|
||||
export WAKE_PREIMAGE_EXTRA="$fx/notes.cfg"
|
||||
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST MOSAIC_HOME WAKE_PREIMAGE_CAPTURE
|
||||
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P15: baseline failed"
|
||||
printf 'plain v2\n' >"$fx/notes.cfg"
|
||||
sha2="$(sha256sum "$fx/notes.cfg" | awk '{print $1}')"
|
||||
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P15: change check failed"
|
||||
sd="$(state_dir)"
|
||||
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
|
||||
[ "$(jq -r '.captured' <<<"$row")" = "false" ] || fail_msg "P15: default must be record-only [$row]"
|
||||
[ ! -f "$sd/preimage/objects/$sha2" ] || fail_msg "P15: bytes captured without opt-in"
|
||||
[ "$(depth)" = "1" ] || fail_msg "P15: record-only must still track the change (depth=$(depth))"
|
||||
# Opt in -> the next change captures.
|
||||
export WAKE_PREIMAGE_CAPTURE="$fx/notes.cfg"
|
||||
printf 'plain v3\n' >"$fx/notes.cfg"
|
||||
sha3="$(sha256sum "$fx/notes.cfg" | awk '{print $1}')"
|
||||
"$PRE" check --enqueue >/dev/null 2>&1 || fail_msg "P15: opted-in change check failed"
|
||||
row="$(tail -n1 "$sd/preimage/preimage-ledger.jsonl")"
|
||||
[ "$(jq -r '.captured' <<<"$row")" = "true" ] || fail_msg "P15: opt-in must capture [$row]"
|
||||
cmp -s "$sd/preimage/objects/$sha3" "$fx/notes.cfg" || fail_msg "P15: opted-in bytes missing or mismatched"
|
||||
) && ok
|
||||
|
||||
echo "== P16: deleted ledger over surviving objects/ — REFUSE to re-baseline (#964 B11) =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p16)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p16fx"; mkdir -p "$fx"
|
||||
make_stub "$fx"
|
||||
export WAKE_DETECTOR_SOURCE_CMD="$fx/adapter.sh"
|
||||
unset WAKE_WATCH_LIST WAKE_PREIMAGE_EXTRA WAKE_PREIMAGE_CAPTURE MOSAIC_HOME
|
||||
"$PRE" check >/dev/null 2>&1 || fail_msg "P16: v1 baseline failed"
|
||||
printf '# v2\n' >>"$fx/adapter.sh"
|
||||
"$PRE" check >/dev/null 2>&1 || fail_msg "P16: v2 change failed"
|
||||
sd="$(state_dir)"
|
||||
nobj="$(ls "$sd/preimage/objects" | wc -l | tr -d '[:space:]')"
|
||||
# Delete the ledger; objects/ survives. ABSENT is not CORRUPT — and it is
|
||||
# not a first install either: that asymmetry is locally detectable.
|
||||
rm -f "$sd/preimage/preimage-ledger.jsonl"
|
||||
printf '# v3\n' >>"$fx/adapter.sh"
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -ne 0 ] || fail_msg "P16: absent ledger over prior objects must FAIL LOUD (rc=0) [$out]"
|
||||
echo "$out" | grep -qi 'REFUSING to re-baseline' || fail_msg "P16: the failure must name the refusal [$out]"
|
||||
[ ! -e "$sd/preimage/preimage-ledger.jsonl" ] || fail_msg "P16: the ledger must NOT be silently recreated over deleted history"
|
||||
[ "$(ls "$sd/preimage/objects" | wc -l | tr -d '[:space:]')" = "$nobj" ] || fail_msg "P16: prior objects must remain untouched"
|
||||
[ "$(depth)" = "0" ] || fail_msg "P16: the v2->v3 change must NOT be absorbed or enqueued from an unverifiable baseline (depth=$(depth))"
|
||||
) && ok
|
||||
|
||||
echo "== P17: allowlist symmetry — symlink to a DENIED target refuses; symlink to an allowed target captures =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state p17)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
fx="$TMP_ROOT/p17fx"; mkdir -p "$fx"
|
||||
export MOSAIC_HOME="$fx/mh"
|
||||
mkdir -p "$MOSAIC_HOME"
|
||||
s='decoy-P17-cred-behind-benign-alias'
|
||||
printf '{"svc":{"token":"%s"}}\n' "$s" >"$MOSAIC_HOME/credentials.json"
|
||||
# A benign-looking alias whose TARGET is denied: the allowlist matches the
|
||||
# alias string, but deny runs FIRST and on BOTH forms — "deny wins over
|
||||
# opt-in" is a precedence rule, path agreement is what makes it hold.
|
||||
ln -s "$MOSAIC_HOME/credentials.json" "$fx/settings.json"
|
||||
# And a symlink to an ALLOWED target: the fix must not close case C by
|
||||
# breaking every symlinked path outright.
|
||||
printf 'benign payload v1\n' >"$fx/target.cfg"
|
||||
ln -s "$fx/target.cfg" "$fx/alias.cfg"
|
||||
export WAKE_PREIMAGE_EXTRA="$fx/settings.json:$fx/alias.cfg"
|
||||
export WAKE_PREIMAGE_CAPTURE="$fx/settings.json:$fx/alias.cfg"
|
||||
unset WAKE_DETECTOR_SOURCE_CMD WAKE_WATCH_LIST
|
||||
out="$("$PRE" check --enqueue 2>&1)"; rc=$?
|
||||
[ "$rc" -eq 0 ] || fail_msg "P17: check must exit 0 (rc=$rc) [$out]"
|
||||
sd="$(state_dir)"
|
||||
crow="$(jq -c --arg p "$(realpath "$fx/settings.json")" 'select(.path == $p)' "$sd/preimage/preimage-ledger.jsonl" | tail -n1)"
|
||||
[ "$(jq -r '.captured' <<<"$crow")" = "false" ] || fail_msg "P17: allowlisted symlink to a denied target must refuse [$crow]"
|
||||
if grep -rq "$s" "$sd" 2>/dev/null; then
|
||||
fail_msg "P17: credential bytes leaked via an allowlisted alias"
|
||||
fi
|
||||
brow="$(jq -c --arg p "$(realpath "$fx/alias.cfg")" 'select(.path == $p)' "$sd/preimage/preimage-ledger.jsonl" | tail -n1)"
|
||||
[ "$(jq -r '.captured' <<<"$brow")" = "true" ] || fail_msg "P17: symlink to an allowed target must still capture [$brow]"
|
||||
) && ok
|
||||
|
||||
echo
|
||||
total=$((pass + $(wc -l <"$FAILFILE")))
|
||||
echo "== test-wake-preimage: $pass/$total passed =="
|
||||
[ -s "$FAILFILE" ] && exit 1
|
||||
exit 0
|
||||
@@ -28,15 +28,6 @@
|
||||
# T16 #946 quarantine-audit: consumed-hashes rows provably false against the
|
||||
# dead-letter ledger are reported (exit 1) and removed only under --repair;
|
||||
# healed rows and the ledger itself are untouched
|
||||
# T17 #952 quarantine-audit clean sweep names BOTH unprovable residual
|
||||
# classes: evidence pruned (no evidence) AND evidence surviving with an
|
||||
# empty observed_hash (evidence unusable). Fixture is the VERBATIM live
|
||||
# specimen (mos-dt lane dead-letter seq 13, bench/malformed-locator-test)
|
||||
# — a nested .locators.* row with NO observed_hash key. Do NOT hand-build
|
||||
# a FLAT dead-letter fixture here: consumed-hashes rows are genuinely
|
||||
# flat, dead-letter rows are genuinely nested, and a flat fixture makes
|
||||
# the audit's CORRECT non-conviction look exactly like the defect under
|
||||
# hunt (this false-defect near-miss actually happened in the #951 review)
|
||||
#
|
||||
# Isolated: every test runs against a fresh WAKE_STATE_HOME temp dir.
|
||||
set -uo pipefail
|
||||
@@ -689,50 +680,6 @@ echo "== T16: #946 — quarantine-audit: a consumed-hash row matching a dead-let
|
||||
grep -qi 'OK' "$TMP_ROOT/t16.ok" || fail_msg "T16: a clean audit must say OK [$(cat "$TMP_ROOT/t16.ok")]"
|
||||
) && ok
|
||||
|
||||
echo "== T17: #952 — the clean-sweep message names BOTH unprovable residual classes; a surviving-but-empty-hash dead-letter row is correctly NOT convicted =="
|
||||
(
|
||||
WAKE_STATE_HOME="$(fresh_state t17)"
|
||||
export WAKE_STATE_HOME
|
||||
unset WAKE_AGENT
|
||||
STATE_DIR="$WAKE_STATE_HOME/default"
|
||||
rec="$STATE_DIR/consumed-hashes.jsonl"
|
||||
dl="$STATE_DIR/dead-letter.jsonl"
|
||||
# Drive the REAL flow so the consumed-hashes row is produced by the actual
|
||||
# writer (_record_last_consumed), not hand-built: 12 fillers, then the bench
|
||||
# key lands at observed_seq 13 — the same seq as the live specimen below.
|
||||
i=1
|
||||
while [ "$i" -le 12 ]; do
|
||||
"$STORE" enqueue --class actionable --locators "{\"kind\":\"filler\",\"id\":\"f$i\",\"observed_hash\":\"HF$i\"}" >/dev/null || fail_msg "T17: filler enqueue $i failed"
|
||||
i=$((i + 1))
|
||||
done
|
||||
"$STORE" enqueue --class actionable --locators '{"kind":"bench","id":"malformed-locator-test","observed_hash":"H-REAL-CONSUMED"}' >/dev/null
|
||||
"$STORE" consume --upto 13 >/dev/null 2>&1 || fail_msg "T17: baseline consume failed"
|
||||
jq_any "$rec" '.kind=="bench" and .id=="malformed-locator-test" and .observed_seq==13 and .observed_hash=="H-REAL-CONSUMED"' ||
|
||||
fail_msg "T17: fixture broken — the real writer did not record the bench@13 row"
|
||||
# VERBATIM live specimen: mos-dt lane dead-letter.jsonl line 1 (2026-07-26).
|
||||
# Copied byte-for-byte per the #952 fixture constraint — nested .locators.*
|
||||
# with NO observed_hash key, so the audit's extraction yields "".
|
||||
printf '%s\n' '{"observed_seq":13,"locators":{"kind":"bench","id":"malformed-locator-test","note":"deliberately non-conformant"},"class":"actionable","emit_ts":1785055610,"hmac":""}' >"$dl"
|
||||
# Guard the near-miss trap: the specimen must be genuinely NESTED (no
|
||||
# top-level kind) and must extract an EMPTY hash via the audit's own paths.
|
||||
jq -e '(has("kind") | not) and .locators.kind=="bench" and ((.locators.observed_hash // "") == "")' "$dl" >/dev/null ||
|
||||
fail_msg "T17: specimen fixture is not the genuine nested shape — rebuild it from the live lane, not the schema"
|
||||
# Evidence SURVIVES (kind/id/seq all match the row) but extracts hash "";
|
||||
# _record_last_consumed never writes an empty-hash row, so the four-field
|
||||
# match can never fire: this is unprovable class 2, NOT a false witness.
|
||||
rep="$TMP_ROOT/t17.rep"
|
||||
"$STORE" quarantine-audit >"$rep" 2>&1 || fail_msg "T17: the audit must exit 0 — nothing here is provable [$(cat "$rep")]"
|
||||
grep -q 'FALSE WITNESS' "$rep" && fail_msg "T17: the empty-hash evidence must NOT convict (the predicate is correct and must not change) [$(cat "$rep")]"
|
||||
# The wording under test (#952): BOTH residual classes, named.
|
||||
grep -qi 'pruned' "$rep" || fail_msg "T17: clean sweep must name residual class 1 — evidence pruned away [$(cat "$rep")]"
|
||||
grep -qi 'empty observed_hash' "$rep" || fail_msg "T17: clean sweep must name residual class 2 — surviving evidence with an empty observed_hash [$(cat "$rep")]"
|
||||
# --repair on a clean sweep removes nothing: the unprovable row survives.
|
||||
"$STORE" quarantine-audit --repair >"$TMP_ROOT/t17.fix" 2>&1 || fail_msg "T17: --repair on a clean sweep must exit 0 [$(cat "$TMP_ROOT/t17.fix")]"
|
||||
jq_any "$rec" '.kind=="bench" and .observed_seq==13 and .observed_hash=="H-REAL-CONSUMED"' ||
|
||||
fail_msg "T17: --repair must NOT remove the unprovable bench@13 row — the audit only removes what the ledger can convict"
|
||||
[ "$(grep -c . "$dl")" = "1" ] || fail_msg "T17: the dead-letter LEDGER must be untouched"
|
||||
) && ok
|
||||
|
||||
echo
|
||||
if [ -s "$FAILFILE" ]; then
|
||||
echo "wake store/ack harness: FAILED ($(grep -c . "$FAILFILE") assertion(s))" >&2
|
||||
|
||||
@@ -25,7 +25,7 @@
|
||||
"lint": "eslint src",
|
||||
"typecheck": "tsc --noEmit",
|
||||
"test": "vitest run --passWithNoTests && pnpm run test:framework-shell",
|
||||
"test:framework-shell": "python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-preimage.sh && bash framework/tools/wake/test-wake-install.sh"
|
||||
"test:framework-shell": "python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-install.sh"
|
||||
},
|
||||
"dependencies": {
|
||||
"@mosaicstack/brain": "workspace:*",
|
||||
|
||||
Reference in New Issue
Block a user