fix(git): push-guard harness — resolve three re-review blockers + two corrections
ci/woodpecker/pr/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
Authored by installer-7; committed by mos-claude (no credential for this remote). push-guard.sh and test-push-guard.sh are BYTE-IDENTICAL to the previously cleared versions — every change this round is in the harness, confirming the reviewer's framing that none of the three blockers was in the guard itself. B1 verify-clean-clone.sh could not verify the artifact in its real monorepo location: it resolved ROOT but kept artifacts as bare basenames, so running it in place reported all artifacts NOT TRACKED. Its own suite missed this because every fixture installed artifacts at fixture ROOT — a fixture encoding a layout the real subject does not have. PREFIX now comes from 'git rev-parse --show-prefix' and is threaded through the ls-tree pathspec, the cloned stat, and the suite cwd; the verifier PRINTS the prefix. Three needles: nested-layout pass, prefix-reported (else a green only means the prefix was ignored harmlessly), and mode-needle-still- bites-nested (a prefix threaded into the clone but not ls-tree would silently stop checking modes). B2 the generator reported full coverage and exited 0 on a RED baseline — any pre-existing failure marked every mutant killed. Now refuses unless baseline is exit 0 with zero failures, prints the actual tally on refusal, emits no table, and scores kills by NAMED DELTA rather than a raw red count. B3 the generator mutated the reviewed source in place; SIGKILL stranded a mutant and contaminated a following run. Guard and suite are now copied into a temp dir and mutations apply to that copy — no restore step to fail. The author's first control for this was itself vacuous (a 3s kill lands during baseline, before any mutation, so it passed against the unfixed mechanism too); the real control drives the kill from inside the run on the second suite invocation, plus a needle proving the reconstructed pre-fix mechanism DOES strand. Corrections: 'shellcheck clean' had been measured at -S warning and published unqualified — a filtered measurement stated as an unfiltered claim. Now clean at DEFAULT severity across six files with one scoped, documented SC2016 disable. ARTIFACTS extended five -> six so the test files no longer omit themselves. Verified before commit: six files sha256-matched to the author's hashes; shellcheck exit 0 at default severity; push-guard 46/46 and verifier 9/9 run directly; the in-place verifier resolved the real nested prefix against this tree. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01YKj59Qadrb2WBLaePvkM7H
This commit is contained in:
installer-7
co-authored by
Claude Opus 5
Claude Opus 5
parent
8310075d33
commit
8fdc8738ed
@@ -7,30 +7,81 @@
|
||||
# misbehaves, so a confidently-wrong one is worse than none. Every number the
|
||||
# README prints about mutation now comes out of this script.
|
||||
#
|
||||
# TWO WAYS A MUTATION RUN LIES, AND THE GUARD FOR EACH:
|
||||
# ============================ WHAT THIS TOOL GOT WRONG ========================
|
||||
# Three defects, all found by review, all of the same shape: A TOOL WHOSE ENTIRE
|
||||
# OUTPUT IS A COVERAGE CLAIM MUST BE HARDER TO FOOL THAN THE CODE IT MEASURES.
|
||||
#
|
||||
# 1. THE ANCHOR NO LONGER MATCHES. The mutant is never applied, the suite is
|
||||
# green, and the report says SURVIVED -- which is the same word a real
|
||||
# coverage gap gets. Guarded by grep-before-mutate (ANCHOR MISSING).
|
||||
# 1. IT REPORTED FULL COVERAGE ON A RED BASELINE. A mutant was "killed" whenever
|
||||
# the suite reported any failure at all, and the baseline was run only at the
|
||||
# END and never required to be green. So ONE pre-existing suite failure --
|
||||
# changing no guard behaviour whatsoever -- satisfied EVERY mutant: 13 killed,
|
||||
# 0 survived, a confident table generated and pasted into the README, exit 0.
|
||||
# Now: the baseline runs FIRST and must be exit-0 with zero failures, and a
|
||||
# kill requires the mutant to break a case THE BASELINE PASSED, recorded BY
|
||||
# NAME. A tally is not evidence; a named delta is.
|
||||
#
|
||||
# 2. THE ANCHOR MATCHES PROSE. This one bit me on the first run: my
|
||||
# "revert refuse-on-missing-config" mutant landed inside the usage()
|
||||
# heredoc, edited a help string, changed no behaviour, and duly reported
|
||||
# SURVIVED. I nearly recorded a documentation edit as an uncovered branch.
|
||||
# A MUTATION THAT CANNOT CHANGE BEHAVIOUR IS NOT A SURVIVING MUTANT, IT IS
|
||||
# A NON-MEASUREMENT -- and a non-measurement reported as a result is the
|
||||
# same defect as the vacuous test it is supposed to be hunting.
|
||||
# Guarded by prose_range(): anchors inside usage() are a loud refusal.
|
||||
# 2. IT MUTATED THE REVIEWED SOURCE IN PLACE. Restoration leaned on an EXIT trap.
|
||||
# A TRAP IS CLEANUP, NOT ISOLATION -- SIGKILL cannot run it. An interrupted run
|
||||
# left push-guard.sh mutated in the working tree, and the reviewer's NEXT
|
||||
# suite run silently inherited it. A verification tool that alters its subject
|
||||
# can leave the subject wrong in a way the next measurement believes.
|
||||
# Now: the subject is copied into a temp dir and only the COPY is ever
|
||||
# written to. The source is untouched BY CONSTRUCTION rather than by cleanup,
|
||||
# which is the only version of this that survives kill -9.
|
||||
#
|
||||
# 3. THE ANCHOR IS AMBIGUOUS. replace(...,1) silently picks the first match,
|
||||
# which may not be the branch named in the label. Guarded by an exact
|
||||
# occurrence count.
|
||||
# 3. ITS WORK DIR WAS SHARED. Concurrent runs interfered through the suite's
|
||||
# default .work directory. Each run now gets its own.
|
||||
#
|
||||
# (Note the deliberate asymmetry with verify-clean-clone.sh, which forbids cp:
|
||||
# there the copy LAUNDERED the property under measurement, so measuring a copy
|
||||
# was the defect. Here mutation is destructive by design, so copying is what
|
||||
# PROTECTS the subject. The rule is not "never copy" -- it is "know whether the
|
||||
# copy preserves the property you are about to measure.")
|
||||
#
|
||||
# ================== THREE WAYS A MUTATION RUN LIES, AND THE GUARD FOR EACH ====
|
||||
# A. THE ANCHOR NO LONGER MATCHES. The mutant is never applied, the suite is
|
||||
# green, and the report says SURVIVED -- the same word a real coverage gap
|
||||
# gets. Guarded: ANCHOR MISSING is a loud failure.
|
||||
# B. THE ANCHOR MATCHES PROSE. This one landed on the first run: a mutant aimed
|
||||
# at a branch matched inside the usage() heredoc, edited a help string,
|
||||
# changed no behaviour, and duly reported SURVIVED. A documentation edit was
|
||||
# one step from being recorded as an uncovered branch. A MUTATION THAT CANNOT
|
||||
# CHANGE BEHAVIOUR IS NOT A SURVIVING MUTANT, IT IS A NON-MEASUREMENT.
|
||||
# Guarded: anchors resolving inside usage() are refused.
|
||||
# C. THE ANCHOR IS AMBIGUOUS. Two unrelated branches here are both the line
|
||||
# `if (( status != 0 )); then`; a first-match replace would credit the kill
|
||||
# to the wrong branch. Guarded: a match count != 1 refuses rather than guesses.
|
||||
set -uo pipefail
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
TARGET="$HERE/push-guard.sh"
|
||||
SUITE="$HERE/test-push-guard.sh"
|
||||
BAK="$(mktemp)"; cp "$TARGET" "$BAK"
|
||||
trap 'cp "$BAK" "$TARGET"; rm -f "$BAK"' EXIT
|
||||
|
||||
SRC_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
while (( $# )); do
|
||||
case "$1" in
|
||||
# --dir exists so this tool can be pointed at a FIXTURE copy and tested.
|
||||
--dir) SRC_DIR="$(cd "$2" && pwd)"; shift 2 ;;
|
||||
*) printf 'usage error: unknown argument: %s\n' "$1" >&2; exit 64 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
SRC_TARGET="$SRC_DIR/push-guard.sh"
|
||||
SRC_SUITE="$SRC_DIR/test-push-guard.sh"
|
||||
for f in "$SRC_TARGET" "$SRC_SUITE"; do
|
||||
[[ -r "$f" ]] || { printf 'REFUSING: cannot read %s\n' "$f" >&2; exit 1; }
|
||||
done
|
||||
|
||||
# --- ISOLATION, NOT CLEANUP --------------------------------------------------
|
||||
# Everything below writes only inside WORK. The trap is a courtesy for disk
|
||||
# space; correctness does not depend on it running.
|
||||
WORK="$(mktemp -d)"
|
||||
trap 'rm -rf "$WORK"' EXIT
|
||||
install -m 755 "$SRC_TARGET" "$WORK/push-guard.sh"
|
||||
install -m 755 "$SRC_SUITE" "$WORK/test-push-guard.sh"
|
||||
TARGET="$WORK/push-guard.sh"
|
||||
SUITE="$WORK/test-push-guard.sh"
|
||||
BAK="$WORK/push-guard.sh.orig"
|
||||
cp "$TARGET" "$BAK"
|
||||
# Per-run work dir: the suite otherwise defaults to a shared .work beside itself,
|
||||
# and two concurrent runs corrupt each other's fixtures.
|
||||
export MOSAIC_TEST_WORK_DIR="$WORK/.work"
|
||||
|
||||
# --- where the prose lives: usage() { ... EOF ---------------------------------
|
||||
PROSE_LO="$(grep -n '^usage() {' "$BAK" | head -1 | cut -d: -f1)"
|
||||
@@ -39,7 +90,32 @@ if [[ -z "$PROSE_LO" || -z "$PROSE_HI" ]]; then
|
||||
echo "!! cannot locate the usage() heredoc -- the prose guard would be inert; refusing" >&2
|
||||
exit 1
|
||||
fi
|
||||
printf 'prose (usage heredoc) is lines %s-%s -- anchors there are refused, not scored\n\n' \
|
||||
|
||||
# passing_cases <output> -- names of cases that PASSED, one per line
|
||||
passing_cases() { printf '%s\n' "$1" | sed -n 's/^ PASS \[[^]]*\] \(.*\) (exit [0-9]*)$/\1/p'; }
|
||||
tally() { printf '%s\n' "$1" | grep -E 'needles: [0-9]+ passed' | tail -1; }
|
||||
|
||||
# --- THE BASELINE MUST BE GREEN, AND IT IS ESTABLISHED FIRST ------------------
|
||||
printf '=== baseline (must be exit 0 with zero failures) ===\n'
|
||||
BASE_OUT="$("$SUITE" 2>&1)"; BASE_RC=$?
|
||||
BASE_LINE="$(tally "$BASE_OUT")"
|
||||
BASE_FAILED="$(printf '%s\n' "$BASE_LINE" | sed -n 's/.*, \([0-9]*\) failed.*/\1/p')"
|
||||
if (( BASE_RC != 0 )) || [[ -z "$BASE_LINE" || "$BASE_FAILED" != "0" ]]; then
|
||||
printf 'REFUSING: baseline is not green -- exit %s, tally: %s\n' \
|
||||
"$BASE_RC" "${BASE_LINE:-<no tally emitted>}" >&2
|
||||
printf '\nEvery mutant would be scored KILLED by the pre-existing failure, and this\n' >&2
|
||||
printf 'tool would publish a confident coverage table that measured nothing. Fix the\n' >&2
|
||||
printf 'suite first. NO TABLE IS EMITTED.\n' >&2
|
||||
exit 1
|
||||
fi
|
||||
printf ' %s\n' "$BASE_LINE"
|
||||
mapfile -t BASE_PASSING < <(passing_cases "$BASE_OUT")
|
||||
printf ' %d named cases passing at baseline\n' "${#BASE_PASSING[@]}"
|
||||
if (( ${#BASE_PASSING[@]} == 0 )); then
|
||||
printf 'REFUSING: could not parse any case names -- kills could not be attributed.\n' >&2
|
||||
exit 1
|
||||
fi
|
||||
printf ' prose (usage heredoc) is lines %s-%s -- anchors there are refused, not scored\n\n' \
|
||||
"$PROSE_LO" "$PROSE_HI"
|
||||
|
||||
rc_all=0
|
||||
@@ -73,71 +149,93 @@ LOCPY
|
||||
rc_all=1; return
|
||||
fi
|
||||
|
||||
python3 - "$BAK" "$TARGET" "$find" "$repl" <<'PY'
|
||||
python3 - "$BAK" "$TARGET" "$find" "$repl" <<'MUTPY'
|
||||
import sys
|
||||
src, dst, find, repl = sys.argv[1:5]
|
||||
open(dst, "w").write(open(src).read().replace(find, repl, 1))
|
||||
PY
|
||||
local out line failed passed total
|
||||
out="$("$SUITE" 2>&1)"
|
||||
line="$(printf '%s\n' "$out" | grep -E 'needles: [0-9]+ passed' | tail -1)"
|
||||
failed="$(printf '%s\n' "$line" | sed -n 's/.*, \([0-9]*\) failed.*/\1/p')"
|
||||
passed="$(printf '%s\n' "$line" | sed -n 's/.*: \([0-9]*\) passed.*/\1/p')"
|
||||
if [[ -z "$failed" || -z "$passed" ]]; then
|
||||
MUTPY
|
||||
local out; out="$("$SUITE" 2>&1)"
|
||||
cp "$BAK" "$TARGET"
|
||||
|
||||
local line; line="$(tally "$out")"
|
||||
if [[ -z "$line" ]]; then
|
||||
printf ' !! NO TALLY %-46s suite produced no needle count\n' "$name"
|
||||
rc_all=1; cp "$BAK" "$TARGET"; return
|
||||
rc_all=1; return
|
||||
fi
|
||||
total=$(( passed + failed ))
|
||||
if (( failed > 0 )); then
|
||||
printf ' KILLED L%-5s %-46s %s/%s fail\n' "$ln" "$name" "$failed" "$total"
|
||||
ROWS+=("| \`$name\` (L$ln) | $failed/$total fail | killed |")
|
||||
# A KILL IS A NAMED DELTA, NOT A TALLY. Cases that passed at baseline and no
|
||||
# longer pass are the evidence; anything else (a case that was already
|
||||
# failing, a suite that died early) cannot be credited to this mutant.
|
||||
local now; now="$(passing_cases "$out")"
|
||||
local -a broke=()
|
||||
local c
|
||||
for c in "${BASE_PASSING[@]}"; do
|
||||
grep -qxF -- "$c" <<<"$now" || broke+=("$c")
|
||||
done
|
||||
local total="${#BASE_PASSING[@]}"
|
||||
|
||||
if (( ${#broke[@]} > 0 )); then
|
||||
printf ' KILLED L%-5s %-46s %d/%d fail\n' "$ln" "$name" "${#broke[@]}" "$total"
|
||||
printf ' by: %s\n' "${broke[0]}"
|
||||
(( ${#broke[@]} > 1 )) && printf ' +%d more\n' "$(( ${#broke[@]} - 1 ))"
|
||||
ROWS+=("| \`$name\` (L$ln) | ${#broke[@]}/$total fail | killed |")
|
||||
KILLED=$(( KILLED + 1 ))
|
||||
else
|
||||
printf ' SURVIVED L%-5s %-46s 0/%s fail <-- UNCOVERED BRANCH\n' "$ln" "$name" "$total"
|
||||
printf ' SURVIVED L%-5s %-46s 0/%d fail <-- UNCOVERED BRANCH\n' "$ln" "$name" "$total"
|
||||
ROWS+=("| \`$name\` (L$ln) | 0/$total fail | **SURVIVED** |")
|
||||
SURVIVED=$(( SURVIVED + 1 )); rc_all=1
|
||||
fi
|
||||
cp "$BAK" "$TARGET"
|
||||
}
|
||||
|
||||
echo "=== push-guard mutation run ==="
|
||||
mutate "json decision requirement bypassed" \
|
||||
' if (( ${#JSON_PATHS[@]} == 0 )); then' ' if false; then'
|
||||
mutate "opt-out accepted with no written reason" \
|
||||
'if not isinstance(reason, str) or not reason.strip():' 'if False:'
|
||||
mutate "committed re-read of the opt-out skipped" \
|
||||
' [[ "$CFG_MODE" == "none" ]] || return 0' ' return 0'
|
||||
mutate "untracked config honoured as an opt-out" \
|
||||
' if [[ -z "$rel" ]]; then' ' if false; then'
|
||||
mutate "staged-but-uncommitted opt-out honoured" \
|
||||
' if [[ -z "$cmode" ]]; then' ' if false; then'
|
||||
mutate "committed SYMLINK config honoured" \
|
||||
' if [[ "$cmode" == "120000" ]]; then' ' if false; then'
|
||||
mutate "unparseable committed config ignored" \
|
||||
' if (( cstatus != 0 )); then' ' if false; then'
|
||||
mutate "local-only opt-out (HEAD says ON) honoured" \
|
||||
' if [[ "$cmode_val" != "none" ]]; then' ' if false; then'
|
||||
mutate "empty MERGE exempted" \
|
||||
' if [[ "$all_same" == yes ]]; then' ' if false; then'
|
||||
mutate "empty ROOT exempted" \
|
||||
'if [[ -z "$(git diff-tree --root -r --name-only --no-commit-id HEAD)" ]]; then' \
|
||||
'if false; then'
|
||||
mutate "--since-head ancestry check removed" \
|
||||
'if ! git merge-base --is-ancestor "$since_head" "$head"; then' 'if false; then'
|
||||
mutate "staged-file enumeration ignores git failure" \
|
||||
"$(printf 'if (( status != 0 )); then\n local msg')" \
|
||||
"$(printf 'if false; then\n local msg')"
|
||||
mutate "malformed config degrades to absent instead of refusing" \
|
||||
"$(printf 'if (( status != 0 )); then\n fail "$EX_CONFIG"')" \
|
||||
"$(printf 'if false; then\n fail "$EX_CONFIG"')"
|
||||
# EVERY ANCHOR BELOW IS VERBATIM SOURCE TEXT OF THE GUARD, so the single quotes
|
||||
# are load-bearing: these strings must reach `mutate` as the CHARACTERS that
|
||||
# appear in push-guard.sh. Expanding them would search for THIS shell's (unset)
|
||||
# $rel, $cmode, $EX_CONFIG and match nothing -- which the anchor guards would
|
||||
# report as ANCHOR MISSING rather than silently, but the intent is still to
|
||||
# forbid expansion. The directive is scoped to this function so it cannot mask a
|
||||
# genuine unintended-literal anywhere else in the file.
|
||||
# shellcheck disable=SC2016
|
||||
run_mutants() {
|
||||
mutate "json decision requirement bypassed" \
|
||||
' if (( ${#JSON_PATHS[@]} == 0 )); then' ' if false; then'
|
||||
mutate "opt-out accepted with no written reason" \
|
||||
'if not isinstance(reason, str) or not reason.strip():' 'if False:'
|
||||
mutate "committed re-read of the opt-out skipped" \
|
||||
' [[ "$CFG_MODE" == "none" ]] || return 0' ' return 0'
|
||||
mutate "untracked config honoured as an opt-out" \
|
||||
' if [[ -z "$rel" ]]; then' ' if false; then'
|
||||
mutate "staged-but-uncommitted opt-out honoured" \
|
||||
' if [[ -z "$cmode" ]]; then' ' if false; then'
|
||||
mutate "committed SYMLINK config honoured" \
|
||||
' if [[ "$cmode" == "120000" ]]; then' ' if false; then'
|
||||
mutate "unparseable committed config ignored" \
|
||||
' if (( cstatus != 0 )); then' ' if false; then'
|
||||
mutate "local-only opt-out (HEAD says ON) honoured" \
|
||||
' if [[ "$cmode_val" != "none" ]]; then' ' if false; then'
|
||||
mutate "empty MERGE exempted" \
|
||||
' if [[ "$all_same" == yes ]]; then' ' if false; then'
|
||||
mutate "empty ROOT exempted" \
|
||||
'if [[ -z "$(git diff-tree --root -r --name-only --no-commit-id HEAD)" ]]; then' \
|
||||
'if false; then'
|
||||
mutate "--since-head ancestry check removed" \
|
||||
'if ! git merge-base --is-ancestor "$since_head" "$head"; then' 'if false; then'
|
||||
# guard's own source text, matched verbatim. Expanding them here would search for
|
||||
# this shell's (empty) $EX_CONFIG instead of the characters in the file.
|
||||
mutate "staged-file enumeration ignores git failure" \
|
||||
"$(printf 'if (( status != 0 )); then\n local msg')" \
|
||||
"$(printf 'if false; then\n local msg')"
|
||||
mutate "malformed config degrades to absent instead of refusing" \
|
||||
"$(printf 'if (( status != 0 )); then\n fail "$EX_CONFIG"')" \
|
||||
"$(printf 'if false; then\n fail "$EX_CONFIG"')"
|
||||
}
|
||||
|
||||
BASE="$("$SUITE" 2>&1 | grep -E 'needles: [0-9]+ passed' | tail -1)"
|
||||
printf '\nunmodified: %s\n' "$BASE"
|
||||
run_mutants
|
||||
|
||||
printf '\nbaseline: %s\n' "$BASE_LINE"
|
||||
printf '%d killed, %d survived\n' "$KILLED" "$SURVIVED"
|
||||
|
||||
printf '\n--- README TABLE (paste verbatim) ---\n'
|
||||
printf '| mutation | suite result | verdict |\n|---|---|---|\n'
|
||||
printf '| *unmodified* | %s | baseline |\n' \
|
||||
"$(printf '%s' "$BASE" | sed 's/push-guard needles: //')"
|
||||
printf '| *unmodified* | %s | baseline |\n' "$(printf '%s' "$BASE_LINE" | sed 's/push-guard needles: //')"
|
||||
printf '%s\n' "${ROWS[@]}"
|
||||
exit "$rc_all"
|
||||
|
||||
@@ -105,21 +105,21 @@ Controls are not decoration. A guard that failed unconditionally would satisfy e
|
||||
| mutation | suite result | verdict |
|
||||
| ---------------------------------------------------------------- | ------------------- | -------- |
|
||||
| _unmodified_ | 46 passed, 0 failed | baseline |
|
||||
| `json decision requirement bypassed` (L482) | 3/46 fail | killed |
|
||||
| `opt-out accepted with no written reason` (L334) | 1/46 fail | killed |
|
||||
| `committed re-read of the opt-out skipped` (L405) | 5/46 fail | killed |
|
||||
| `untracked config honoured as an opt-out` (L409) | 1/46 fail | killed |
|
||||
| `staged-but-uncommitted opt-out honoured` (L425) | 1/46 fail | killed |
|
||||
| `committed SYMLINK config honoured` (L433) | 1/46 fail | killed |
|
||||
| `unparseable committed config ignored` (L444) | 1/46 fail | killed |
|
||||
| `local-only opt-out (HEAD says ON) honoured` (L459) | 1/46 fail | killed |
|
||||
| `empty MERGE exempted` (L704) | 1/46 fail | killed |
|
||||
| `empty ROOT exempted` (L715) | 1/46 fail | killed |
|
||||
| `--since-head ancestry check removed` (L665) | 1/46 fail | killed |
|
||||
| `staged-file enumeration ignores git failure` (L173) | 1/46 fail | killed |
|
||||
| `malformed config degrades to absent instead of refusing` (L375) | 4/46 fail | killed |
|
||||
| `json decision requirement bypassed` (L482) | 3/43 fail | killed |
|
||||
| `opt-out accepted with no written reason` (L334) | 1/43 fail | killed |
|
||||
| `committed re-read of the opt-out skipped` (L405) | 5/43 fail | killed |
|
||||
| `untracked config honoured as an opt-out` (L409) | 1/43 fail | killed |
|
||||
| `staged-but-uncommitted opt-out honoured` (L425) | 1/43 fail | killed |
|
||||
| `committed SYMLINK config honoured` (L433) | 1/43 fail | killed |
|
||||
| `unparseable committed config ignored` (L444) | 1/43 fail | killed |
|
||||
| `local-only opt-out (HEAD says ON) honoured` (L459) | 1/43 fail | killed |
|
||||
| `empty MERGE exempted` (L704) | 1/43 fail | killed |
|
||||
| `empty ROOT exempted` (L715) | 1/43 fail | killed |
|
||||
| `--since-head ancestry check removed` (L665) | 1/43 fail | killed |
|
||||
| `staged-file enumeration ignores git failure` (L173) | 1/43 fail | killed |
|
||||
| `malformed config degrades to absent instead of refusing` (L375) | 4/43 fail | killed |
|
||||
|
||||
13 mutants, 0 survived.
|
||||
13 mutants, 0 survived. The denominator is the 43 cases the suite reports BY NAME; the three fixture-assertion lines (`w2-fixture`, `e9-fixture`, `g1-fixture`) print in a different format and are excluded from attribution, which is why 46 pass but 43 are attributable.
|
||||
|
||||
Earlier mutants, run at the suite size of the day and kept as history rather than as a live claim: fail-open (9/16), always-fail (16/16 — controls catch it), revert `:(glob)` normalization (3/16, caught **only** by the `--out` assertions), drop the remote-did-not-move assertion (1/16), restore blanket `|| true` on the scan (1/17), revert `--no-renames` (1/19), revert `-a` to `-I` (1/19), delete the `--since-head` block (2/19, incl. the control that _was_ vacuous), stray-warning emission (1/32).
|
||||
|
||||
@@ -147,18 +147,46 @@ The fix moves the parser to a top-level constant. The lesson is encoded as case
|
||||
|
||||
**The `-E` flag on `git grep` is load-bearing and was caught by a needle, not by review.** `git grep` defaults to _basic_ regex, in which `(`, `|` and `{7}` are literal characters. Without `-E` the patterns match nothing and the check reports a clean pass over a file full of conflict markers — the exact defect this tool exists to prevent, shipped inside the tool itself.
|
||||
|
||||
### Then the review turned on the harness, and found three more
|
||||
|
||||
A second independent review ran everything from a fresh clone and reproduced three defects — **all of them in the tools written to prevent defects.** None was in `push-guard.sh`.
|
||||
|
||||
**1. The clean-clone verifier could not verify the tree that ships it.** `verify-clean-clone.sh` resolved the repository top level correctly but then passed **bare basenames** to `git ls-tree`, which is a root-relative pathspec. These files really live at `packages/mosaic/framework/tools/git/`, so in place every artifact came back `NOT TRACKED at HEAD` and the verifier exited 1 without ever running. The tool built to stop packaging false-greens was unusable against its own packaging.
|
||||
|
||||
Its suite could not see it, because **every fixture installed the artifacts at the fixture repository root.** 6/6 green proved flat-layout operation and said nothing about the deployed path. That is the third time on this tool that a control validated a _model_ instead of the _subject_: v1 of the verifier measured a `cp`'d scratch repo, and then v2's own tests measured a layout that does not exist. **A fixture is a claim about the world; an untested fixture is an unreviewed one.** The committed prefix now comes from `git rev-parse --show-prefix` and is threaded through `ls-tree`, the cloned `stat`, and the suite's working directory; `w6-nested` builds the real nested layout, `w6-prefix` asserts the verifier _reports_ that prefix (so a green `w6` cannot mean the prefix was harmlessly ignored), and `w7-nested-mode` proves the mode needle still bites down there.
|
||||
|
||||
**2. Any pre-existing suite failure satisfied every mutant.** The generator called a mutant `KILLED` whenever `failed > 0`, and never required a green baseline. Inject one always-failing case that changes no guard behaviour whatsoever and the run reports _13 killed, 0 survived_, emits the table above, and exits 0. **A tally is not evidence; a named delta is.** The generator now refuses outright unless the unmodified baseline is exit-0 with zero failures — and emits no table when it refuses — then scores each mutant by the _named cases_ that stopped passing, printing the first one (`by: <case>`) beside every kill.
|
||||
|
||||
**3. An interrupted run stranded a mutated `push-guard.sh` in the reviewed tree.** Restoration leaned on an `EXIT` trap. **A trap is cleanup, not isolation, and SIGKILL cannot run it.** It happened to the reviewer twice and contaminated the following suite run until the clone was discarded. Isolation is now by construction: the guard and suite are `install`ed into a temp dir and every mutation is applied to _that_ copy, so the reviewed file is never opened for writing at all.
|
||||
|
||||
Note the deliberate asymmetry with `verify-clean-clone.sh`, which forbids `cp` anywhere in the file. The rule is not "never copy" — it is **know whether the copy preserves the property you are about to measure.** `cp` launders mode, so the verifier must not copy; mutation is destructive by design, so the generator must.
|
||||
|
||||
`test-mutate-push-guard.sh` (8 cases) now covers all three: `g1-*` proves a red baseline is refused with no table, `g2-*` is the positive control plus an assertion that kills are attributed by name, and `g3-*` kills the generator mid-mutation and asserts the subject is byte-identical afterwards.
|
||||
|
||||
That last one was **vacuous on its first attempt.** `timeout -s KILL 3` looked convincing and proved nothing: at three seconds the generator is still running its baseline, so no mutation has been applied and the subject is trivially unchanged — for the _unfixed_ in-place generator too, which I confirmed by rebuilding it and running it. The kill is now driven from inside the run (the fixture's suite counts its own invocations and kills the generator on the second, when mutant #1 is applied), and `g3-needle-bites` puts the reconstructed pre-fix mechanism through the identical kill to prove it _does_ strand a mutated file. A control written to close a blocker was itself a member of the vacuous family.
|
||||
|
||||
Two smaller things fell out of building those cases, both worth recording because both read as the opposite of what they were:
|
||||
|
||||
- **`grep -q` under `pipefail` turns a successful match into a failed assertion.** `grep -q` exits at the first match, the producer dies of SIGPIPE, and `pipefail` reports 141. This cost a red `w6-prefix` against a verifier that was printing the right prefix all along. Capture into a variable and test the variable.
|
||||
- **`$PPID` inside `$( )` is the subshell, not the caller.** Killing it merely ends the command substitution; the parent carries on and exits 0. The fixture uses `kill -9 0` (the process group) with the generator launched under `setsid --wait`.
|
||||
|
||||
**Linting is measured at default severity, and the earlier claim was not.** "shellcheck clean on all five" was published on the strength of `shellcheck -S warning`, which exited 0 — while the default severity exited 1 with twelve `SC2016` findings. A filtered measurement reported as an unfiltered claim is the same shape as everything else on this page. Those literals genuinely must not expand, so `run_mutants()` carries one scoped, documented `SC2016` suppression; all six files are now clean at **default** severity. (A documented "this literal is intentionally unexpanded" is a different thing from a comment asserting a safety property nobody rechecks.)
|
||||
|
||||
**Not independently reproduced here:** blocker 1's original repro ran against the real PR checkout, and this session has no credential for that remote. The `w6`/`w7` fixtures replicate the layout at the exact deployed prefix instead, which is a reconstruction, not the original observation. Stated rather than glossed.
|
||||
|
||||
## Proposed framework path
|
||||
|
||||
```
|
||||
framework/tools/git/push-guard.sh # the guard
|
||||
framework/tools/git/test-push-guard.sh # 46 needles and controls
|
||||
framework/tools/git/mutate-push-guard.sh # regenerates the mutation table above
|
||||
framework/tools/git/verify-clean-clone.sh # proves the COMMITTED artifact runs
|
||||
framework/tools/git/test-verify-clean-clone.sh # 6 needles for the verifier itself
|
||||
framework/tools/git/push-guard.sh # the guard
|
||||
framework/tools/git/test-push-guard.sh # 46 needles and controls
|
||||
framework/tools/git/mutate-push-guard.sh # regenerates the mutation table above
|
||||
framework/tools/git/test-mutate-push-guard.sh # 8 needles for the generator
|
||||
framework/tools/git/verify-clean-clone.sh # proves the COMMITTED artifact runs
|
||||
framework/tools/git/test-verify-clean-clone.sh # 9 needles for the verifier
|
||||
```
|
||||
|
||||
Matches the existing `tools/git/test-*.sh` convention. Dependencies: bash 4.4+, git, python3 — `python3` is already an accepted dependency of `ci-queue-wait.sh`.
|
||||
|
||||
**All five must be committed mode `100755`.** They were once delivered `100644`, so a clone exited `126 Permission denied` for everyone who was not the author; `verify-clean-clone.sh` exists to make that unshippable and asserts the mode from `git ls-tree` of the source commit, never from the filesystem.
|
||||
**All six must be committed mode `100755`.** They were once delivered `100644`, so a clone exited `126 Permission denied` for everyone who was not the author; `verify-clean-clone.sh` exists to make that unshippable and asserts the mode from `git ls-tree` of the source commit, never from the filesystem.
|
||||
|
||||
Operator-agnostic: no hostnames, credentials, remotes, or operator-specific paths. Clean under the framework-PR firewall.
|
||||
|
||||
@@ -0,0 +1,178 @@
|
||||
#!/usr/bin/env bash
|
||||
# test-mutate-push-guard.sh -- needles for the mutation generator.
|
||||
#
|
||||
# The generator's entire output is a COVERAGE CLAIM, and the README publishes it.
|
||||
# That makes it the most dangerous file here: when it is wrong it does not fail,
|
||||
# it reassures. Two of its three defects were found by review rather than by any
|
||||
# test, so these are the cases that had to exist.
|
||||
#
|
||||
# g1 a RED BASELINE must be refused before any mutant runs. Previously ONE
|
||||
# pre-existing suite failure -- changing no guard behaviour at all --
|
||||
# satisfied every mutant: 13 killed, 0 survived, table emitted, exit 0.
|
||||
# g3 an INTERRUPTED run must leave the subject byte-identical. Restoration
|
||||
# used to lean on an EXIT trap, and A TRAP IS CLEANUP, NOT ISOLATION:
|
||||
# SIGKILL cannot run it, so an interrupted run stranded a mutated
|
||||
# push-guard.sh that the next suite run silently inherited.
|
||||
# g2 is the positive control. Without it every case here could be passing
|
||||
# because the generator refuses unconditionally, which is a wall, not a gate.
|
||||
|
||||
set -uo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
GEN="$HERE/mutate-push-guard.sh"
|
||||
PASS=0; FAIL=0
|
||||
TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT
|
||||
|
||||
ok() { printf ' ok [%-16s] %s\n' "$1" "${2:-}"; PASS=$(( PASS + 1 )); }
|
||||
bad() { printf ' FAIL [%-16s] %s\n' "$1" "$2"; FAIL=$(( FAIL + 1 )); }
|
||||
|
||||
# fixture <name> -- a directory holding an independent copy of guard + suite
|
||||
fixture() {
|
||||
local d="$TMP/$1"; mkdir -p "$d"
|
||||
install -m 755 "$HERE/push-guard.sh" "$d/push-guard.sh"
|
||||
install -m 755 "$HERE/test-push-guard.sh" "$d/test-push-guard.sh"
|
||||
printf '%s\n' "$d"
|
||||
}
|
||||
|
||||
echo "== g1: a RED BASELINE must be refused, with NO table emitted =="
|
||||
# The injected case asserts exit 99 from `true`. It fails always, and it changes
|
||||
# NO guard behaviour -- which is the whole point: a defect anywhere in the suite
|
||||
# used to be enough to certify every branch as covered.
|
||||
g1="$(fixture red)"
|
||||
python3 - "$g1/test-push-guard.sh" <<'PY'
|
||||
import sys
|
||||
p = sys.argv[1]; s = open(p).read()
|
||||
anchor = 'mkdir -p "$WORK_DIR"\n'
|
||||
assert s.count(anchor) == 1, "injection anchor not unique -- fixture would not be the red baseline"
|
||||
s = s.replace(anchor, anchor + '\nexpect NEEDLE 99 "injected always-failing case" -- true\n', 1)
|
||||
open(p, "w").write(s)
|
||||
PY
|
||||
# Prove the fixture really IS red before asserting the generator notices, or g1
|
||||
# could pass against a green suite and test nothing.
|
||||
if "$g1/test-push-guard.sh" >/dev/null 2>&1; then
|
||||
bad g1-fixture "injected suite still passes -- fixture is not a red baseline"
|
||||
else
|
||||
ok g1-fixture "fixture suite is red, as required"
|
||||
fi
|
||||
out="$("$GEN" --dir "$g1" 2>&1)"; rc=$?
|
||||
if (( rc != 0 )) && [[ "$out" == *"REFUSING: baseline is not green"* ]]; then
|
||||
ok g1-refused "exit $rc, refused before mutating"
|
||||
else
|
||||
bad g1-refused "wanted nonzero + refusal; got rc=$rc"
|
||||
fi
|
||||
if [[ "$out" != *"README TABLE"* && "$out" != *"killed,"* ]]; then
|
||||
ok g1-no-table "no coverage table emitted from a red baseline"
|
||||
else
|
||||
bad g1-no-table "a table or kill count was published despite the red baseline"
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "== g3: an INTERRUPTED run must not alter the subject =="
|
||||
# THE KILL MUST LAND INSIDE A MUTATION WINDOW OR THIS CASE PROVES NOTHING.
|
||||
# First attempt used `timeout -s KILL 3`. At three seconds the generator is still
|
||||
# running its BASELINE, so no mutation has been applied yet and the subject is
|
||||
# trivially unchanged -- for the ORIGINAL in-place generator too, which I
|
||||
# confirmed by running it. The control passed for a reason unrelated to the fix:
|
||||
# a vacuous control, in the control written for blocker 3.
|
||||
#
|
||||
# So the kill is now driven from INSIDE the run. The fixture's suite counts its
|
||||
# own invocations and SIGKILLs the generator on the second one -- invocation 1 is
|
||||
# the baseline, invocation 2 happens with mutant #1 APPLIED. That is exactly the
|
||||
# window where an in-place generator strands a mutated subject.
|
||||
instrument_kill_at_second_run() {
|
||||
python3 - "$1" <<'PY'
|
||||
import sys
|
||||
p = sys.argv[1]; s = open(p).read()
|
||||
anchor = 'PASS=0\nFAIL=0\n'
|
||||
assert s.count(anchor) == 1, "instrumentation anchor not unique"
|
||||
inject = anchor + '''
|
||||
if [[ -n "${G3_COUNTER:-}" ]]; then
|
||||
n=$(( $(cat "$G3_COUNTER" 2>/dev/null || echo 0) + 1 ))
|
||||
printf '%s' "$n" > "$G3_COUNTER"
|
||||
# Invocation 2 = first mutant applied. Kill the generator where it hurts.
|
||||
# `kill -9 0` targets the whole PROCESS GROUP, not $PPID: the generator runs
|
||||
# the suite inside $( ), which forks, so $PPID is that subshell and killing
|
||||
# it merely ends the command substitution -- the generator carries on and
|
||||
# exits 0. The caller puts the generator in its OWN group via setsid, so the
|
||||
# group is exactly the generator and its children, and this harness is not
|
||||
# in it.
|
||||
(( n == 2 )) && kill -9 0
|
||||
fi
|
||||
'''
|
||||
open(p, "w").write(s.replace(anchor, inject, 1))
|
||||
PY
|
||||
}
|
||||
|
||||
# run_killed <dir> -> echoes "<rc> <before> <after>"
|
||||
run_killed() {
|
||||
local d="$1" gen="$2" before after rc
|
||||
instrument_kill_at_second_run "$d/test-push-guard.sh"
|
||||
before="$(sha256sum "$d/push-guard.sh" | cut -d' ' -f1)"
|
||||
G3_COUNTER="$d/.count" setsid --wait "$gen" --dir "$d" >/dev/null 2>&1; rc=$?
|
||||
after="$(sha256sum "$d/push-guard.sh" | cut -d' ' -f1)"
|
||||
printf '%s %s %s\n' "$rc" "$before" "$after"
|
||||
}
|
||||
|
||||
g3="$(fixture killed)"
|
||||
read -r krc before after <<<"$(run_killed "$g3" "$GEN")"
|
||||
if (( krc != 0 )); then
|
||||
ok g3-was-killed "generator died mid-mutation (exit $krc)"
|
||||
else
|
||||
bad g3-was-killed "generator exited 0 -- the kill never landed, so the check below is vacuous"
|
||||
fi
|
||||
if [[ "$before" == "$after" ]]; then
|
||||
ok g3-subject-intact "push-guard.sh byte-identical after the kill"
|
||||
else
|
||||
bad g3-subject-intact "SUBJECT MUTATED AND STRANDED: $before -> $after"
|
||||
fi
|
||||
|
||||
# PROVE THE NEEDLE BITES. Reconstruct the pre-fix mechanism -- mutate the source
|
||||
# in place, restore from an EXIT trap -- and put it through the identical kill.
|
||||
# If this does NOT strand a mutated file, g3-subject-intact is measuring nothing
|
||||
# and the isolation fix is unevidenced.
|
||||
g3o="$(fixture killed-inplace)"
|
||||
python3 - "$GEN" "$g3o/gen-inplace.sh" <<'PY'
|
||||
import sys
|
||||
s = open(sys.argv[1]).read()
|
||||
old = '''install -m 755 "$SRC_TARGET" "$WORK/push-guard.sh"
|
||||
install -m 755 "$SRC_SUITE" "$WORK/test-push-guard.sh"
|
||||
TARGET="$WORK/push-guard.sh"
|
||||
SUITE="$WORK/test-push-guard.sh"
|
||||
BAK="$WORK/push-guard.sh.orig"
|
||||
cp "$TARGET" "$BAK"'''
|
||||
new = '''TARGET="$SRC_TARGET"
|
||||
SUITE="$SRC_SUITE"
|
||||
BAK="$WORK/push-guard.sh.orig"
|
||||
cp "$TARGET" "$BAK"
|
||||
trap 'cp "$BAK" "$TARGET"; rm -rf "$WORK"' EXIT'''
|
||||
assert s.count(old) == 1, "cannot reconstruct the in-place mechanism -- bite proof would be fake"
|
||||
open(sys.argv[2], "w").write(s.replace(old, new, 1))
|
||||
PY
|
||||
chmod +x "$g3o/gen-inplace.sh"
|
||||
read -r _orc obefore oafter <<<"$(run_killed "$g3o" "$g3o/gen-inplace.sh")"
|
||||
if [[ "$obefore" != "$oafter" ]]; then
|
||||
ok g3-needle-bites "in-place generator strands a mutated subject, as it must"
|
||||
else
|
||||
bad g3-needle-bites "the OLD mechanism also left the subject intact -- g3 is vacuous"
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "== g2: POSITIVE CONTROL -- a clean subject must produce a full green run =="
|
||||
g2="$(fixture clean)"
|
||||
out2="$("$GEN" --dir "$g2" 2>&1)"; rc2=$?
|
||||
if (( rc2 == 0 )) && [[ "$out2" == *"0 survived"* ]]; then
|
||||
ok g2-control "$(printf '%s' "$out2" | grep -E '^[0-9]+ killed')"
|
||||
else
|
||||
bad g2-control "wanted exit 0 with 0 survived; got rc=$rc2"
|
||||
fi
|
||||
# A kill must be attributed to a NAMED case, not to a bare tally -- that is the
|
||||
# fix for blocker 2 and it needs its own assertion.
|
||||
if [[ "$out2" == *" by: "* ]]; then
|
||||
ok g2-attributed "kills name the case they broke"
|
||||
else
|
||||
bad g2-attributed "no per-mutant case attribution in the output"
|
||||
fi
|
||||
|
||||
echo
|
||||
printf '%d passed, %d failed\n' "$PASS" "$FAIL"
|
||||
(( FAIL == 0 ))
|
||||
@@ -12,25 +12,41 @@ set -uo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
VERIFY="$HERE/verify-clean-clone.sh"
|
||||
ARTIFACTS=(push-guard.sh test-push-guard.sh verify-clean-clone.sh mutate-push-guard.sh)
|
||||
ARTIFACTS=(push-guard.sh test-push-guard.sh verify-clean-clone.sh mutate-push-guard.sh
|
||||
test-mutate-push-guard.sh
|
||||
test-verify-clean-clone.sh)
|
||||
|
||||
# THE REAL DEPLOYED LAYOUT. Every fixture in the first version of this file
|
||||
# installed the artifacts at the fixture ROOT, so 6/6 green proved flat-layout
|
||||
# operation and said NOTHING about the path these files actually ship at. The
|
||||
# verifier was unusable in place and the suite could not see it, because THE
|
||||
# FIXTURE ENCODED A LAYOUT THAT DOES NOT EXIST. A fixture is a claim about the
|
||||
# world; an untested fixture is an unreviewed one.
|
||||
readonly REAL_PREFIX="packages/mosaic/framework/tools/git"
|
||||
|
||||
PASS=0; FAIL=0
|
||||
TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT
|
||||
|
||||
# Build a source repo whose COMMITTED modes are 100755 and whose disk modes are
|
||||
# executable -- the honest state.
|
||||
# mkrepo <repo-root> [subdir]
|
||||
# With a subdir the artifacts are committed at repo/<subdir>/, which is how they
|
||||
# really ship. Echoes the directory the artifacts landed in -- that is what gets
|
||||
# passed as --repo, so the caller never has to reconstruct the path.
|
||||
mkrepo() {
|
||||
local d="$1"
|
||||
mkdir -p "$d"
|
||||
local d="$1" sub="${2:-}"
|
||||
local dest="$d${sub:+/$sub}"
|
||||
mkdir -p "$dest"
|
||||
local f
|
||||
for f in "${ARTIFACTS[@]}"; do
|
||||
install -m 755 "$HERE/$f" "$d/$f"
|
||||
install -m 755 "$HERE/$f" "$dest/$f"
|
||||
done
|
||||
git -C "$d" init -q .
|
||||
git -C "$d" config user.email "[email protected]"
|
||||
git -C "$d" config user.name "Verifier Needles"
|
||||
git -C "$d" add -A
|
||||
git -C "$d" commit -q -m "artifacts"
|
||||
printf '%s\n' "$dest"
|
||||
}
|
||||
|
||||
run_case() {
|
||||
@@ -48,13 +64,46 @@ run_case() {
|
||||
echo "== POSITIVE CONTROL: an honestly-committed artifact must PASS =="
|
||||
# Without this, every case below could be passing because the verifier always
|
||||
# refuses -- a wall, not a gate.
|
||||
mkrepo "$TMP/good"
|
||||
mkrepo "$TMP/good" >/dev/null
|
||||
run_case w1-honest 0 "the COMMITTED artifact ran and passed" "$TMP/good"
|
||||
|
||||
echo
|
||||
echo "== THE DEPLOYED LAYOUT: artifacts NESTED, not at the repository root =="
|
||||
# The blocker: ls-tree was given a bare basename, which is a ROOT-relative
|
||||
# pathspec, so in the real tree every artifact came back NOT TRACKED and the
|
||||
# verifier refused to run at all. This control fails against that version and
|
||||
# passes only when the committed PREFIX is threaded through ls-tree, the cloned
|
||||
# stat, and the suite's working directory.
|
||||
w6dir="$(mkrepo "$TMP/nested" "$REAL_PREFIX")"
|
||||
run_case w6-nested 0 "the COMMITTED artifact ran and passed" "$w6dir"
|
||||
# ...and prove the run actually happened DOWN THERE rather than at the root, or
|
||||
# a passing w6 would only mean the prefix was ignored harmlessly.
|
||||
#
|
||||
# Captured into a variable rather than piped into `grep -q` ON PURPOSE. grep -q
|
||||
# exits at the FIRST match, the verifier then dies of SIGPIPE, and under
|
||||
# `pipefail` the pipeline reports that 141 -- so A SUCCESSFUL MATCH READS AS A
|
||||
# FAILED ASSERTION. This cost me a red w6-prefix against a verifier that was
|
||||
# printing the right prefix all along. Same family as the pipefail/process-
|
||||
# substitution note already on record: the exit status being consulted is not
|
||||
# the status of the thing being asserted.
|
||||
w6out="$("$VERIFY" --repo "$w6dir" 2>&1)"
|
||||
if [[ "$w6out" == *"prefix $REAL_PREFIX/"* ]]; then
|
||||
printf ' ok [%-14s] verifier reported prefix %s/\n' "w6-prefix" "$REAL_PREFIX"; PASS=$(( PASS + 1 ))
|
||||
else
|
||||
printf ' FAIL [%-14s] verifier did not report the nested prefix -- it may have\n' "w6-prefix"
|
||||
printf ' passed by looking at the root, which is the blocker unfixed\n'; FAIL=$(( FAIL + 1 ))
|
||||
fi
|
||||
# And the mode needle must ALSO bite in the nested layout: a prefix threaded
|
||||
# into the clone but not into ls-tree would silently stop checking modes.
|
||||
w7dir="$(mkrepo "$TMP/nested-laundered" "$REAL_PREFIX")"
|
||||
git -C "$TMP/nested-laundered" update-index --chmod=-x "$REAL_PREFIX/push-guard.sh"
|
||||
git -C "$TMP/nested-laundered" commit -q -m "drop exec bit in git only"
|
||||
run_case w7-nested-mode 1 "committed 100644, needs 100755" "$w7dir"
|
||||
|
||||
echo
|
||||
echo "== THE B1 NEEDLE: source git mode 100644, DISK MODE STILL 755 =="
|
||||
# This is the reviewer's exact reproduction. v1 of the verifier exits 0 here.
|
||||
mkrepo "$TMP/laundered"
|
||||
mkrepo "$TMP/laundered" >/dev/null
|
||||
git -C "$TMP/laundered" update-index --chmod=-x push-guard.sh test-push-guard.sh
|
||||
git -C "$TMP/laundered" commit -q -m "drop exec bit in git only"
|
||||
# PROVE THE FIXTURE IS THE CONTAMINATED STATE, not merely a broken repo: git must
|
||||
@@ -72,14 +121,14 @@ run_case w2-laundered 1 "committed 100644, needs 100755" "$TMP/laundered"
|
||||
|
||||
echo
|
||||
echo "== the artifact must be COMMITTED, or there is no mode to verify =="
|
||||
mkrepo "$TMP/untracked"
|
||||
mkrepo "$TMP/untracked" >/dev/null
|
||||
git -C "$TMP/untracked" rm -q --cached push-guard.sh
|
||||
git -C "$TMP/untracked" commit -q -m "untrack the guard"
|
||||
run_case w3-untracked 1 "NOT TRACKED" "$TMP/untracked"
|
||||
|
||||
echo
|
||||
echo "== a committed SYMLINK is a path, not the reviewed code =="
|
||||
mkrepo "$TMP/symlink"
|
||||
mkrepo "$TMP/symlink" >/dev/null
|
||||
( cd "$TMP/symlink" && rm -f push-guard.sh && ln -s /dev/null push-guard.sh \
|
||||
&& git add push-guard.sh && git commit -q -m "symlink the guard" )
|
||||
run_case w4-symlink 1 "SYMLINK" "$TMP/symlink"
|
||||
|
||||
@@ -51,7 +51,9 @@ done
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
[[ -n "$REPO" ]] || REPO="$HERE"
|
||||
|
||||
ARTIFACTS=(push-guard.sh test-push-guard.sh verify-clean-clone.sh mutate-push-guard.sh)
|
||||
ARTIFACTS=(push-guard.sh test-push-guard.sh verify-clean-clone.sh mutate-push-guard.sh
|
||||
test-mutate-push-guard.sh
|
||||
test-verify-clean-clone.sh)
|
||||
SUITE="test-push-guard.sh"
|
||||
|
||||
# --- the subject must be a real repository, or there is nothing to verify -----
|
||||
@@ -66,8 +68,26 @@ if ! REV_SHA="$(git -C "$ROOT" rev-parse --verify --quiet "${REV}^{commit}")"; t
|
||||
exit "$EX_FAIL"
|
||||
fi
|
||||
|
||||
# --- THE ARTIFACT'S COMMITTED PATH, NOT ITS BASENAME -------------------------
|
||||
# v2 asserted `git ls-tree HEAD -- push-guard.sh`, which is a ROOT-RELATIVE
|
||||
# pathspec. These files really live several directories down. Run in place and
|
||||
# every artifact came back "NOT TRACKED" -- the verifier built to stop packaging
|
||||
# false-greens could not verify the tree that ships it.
|
||||
#
|
||||
# The tests missed it because EVERY FIXTURE INSTALLED THE ARTIFACTS AT THE
|
||||
# FIXTURE ROOT. 6/6 green proved flat-layout operation and nothing about the
|
||||
# deployed path. THAT IS THE THIRD TIME ON THIS TOOL THAT A CONTROL VALIDATED A
|
||||
# MODEL INSTEAD OF THE SUBJECT: v1 measured a cp'd scratch repo, and then v2's
|
||||
# own tests measured a layout that does not exist. A fixture is a claim about
|
||||
# the world, and an untested fixture is an unreviewed one.
|
||||
#
|
||||
# --show-prefix answers "where is this directory inside its repository", so the
|
||||
# same prefix drives ls-tree, the cloned stat, and the suite's working dir.
|
||||
PREFIX="$(git -C "$REPO" rev-parse --show-prefix)"
|
||||
|
||||
printf '=== subject ===\n'
|
||||
printf ' repo %s\n rev %s (%s)\n\n' "$ROOT" "$REV" "$REV_SHA"
|
||||
printf ' repo %s\n rev %s (%s)\n prefix %s\n\n' \
|
||||
"$ROOT" "$REV" "$REV_SHA" "${PREFIX:-<repository root>}"
|
||||
|
||||
# --- 1. MODE, FROM THE COMMITTED TREE OF THE SUBJECT -------------------------
|
||||
# git ls-tree reports what the COMMIT records. Nothing on my filesystem can
|
||||
@@ -75,7 +95,7 @@ printf ' repo %s\n rev %s (%s)\n\n' "$ROOT" "$REV" "$REV_SHA"
|
||||
printf '=== committed modes (git ls-tree %s) ===\n' "$REV"
|
||||
rc=0
|
||||
for f in "${ARTIFACTS[@]}"; do
|
||||
entry="$(git -C "$ROOT" ls-tree "$REV_SHA" -- "$f")"
|
||||
entry="$(git -C "$ROOT" ls-tree "$REV_SHA" -- "${PREFIX}${f}")"
|
||||
if [[ -z "$entry" ]]; then
|
||||
printf ' FAIL %s is NOT TRACKED at %s -- nothing committed to verify\n' "$f" "$REV"
|
||||
rc=1; continue
|
||||
@@ -111,14 +131,14 @@ git clone -q --no-local --no-hardlinks "$ROOT" "$WORK/clone"
|
||||
git -C "$WORK/clone" -c advice.detachedHead=false checkout -q "$REV_SHA"
|
||||
|
||||
for f in "${ARTIFACTS[@]}"; do
|
||||
printf ' clone disk mode: %s %s\n' "$(stat -c '%a' "$WORK/clone/$f")" "$f"
|
||||
printf ' clone disk mode: %s %s\n' "$(stat -c '%a' "$WORK/clone/${PREFIX}${f}")" "${PREFIX}${f}"
|
||||
done
|
||||
|
||||
printf '\n=== executing DIRECTLY (./%s), not via "bash %s" ===\n' "$SUITE" "$SUITE"
|
||||
# Invoking the interpreter explicitly masks a missing exec bit completely -- it
|
||||
# is how the original green was obtained. Direct execution is the thing under
|
||||
# test, so the mode has to be real for this line to succeed.
|
||||
cd "$WORK/clone"
|
||||
cd "$WORK/clone/${PREFIX}"
|
||||
if ! "./$SUITE"; then
|
||||
printf '\nCLEAN-CLONE RUN FAILED.\n'
|
||||
exit "$EX_FAIL"
|
||||
|
||||
Reference in New Issue
Block a user