Compare commits

..
Author SHA1 Message Date
Jason Woltje 2fa6bcd576 fix(git): #1007 — test-issue-comment-readback is a FIFTH affected suite (second census correction)
ci/woodpecker/pr/ci Pipeline was successful
My previous commit said four. It is five. `test-issue-comment-readback.sh` has
the same defect and is fixed the same way, and I had already looked straight at
it and filed it as an *unrelated* silent failure. Correcting that here rather
than folding it in quietly.

WHY IT WAS MISSED — the general lesson, not the excuse. `run_comment()` sends
the wrapper's stdout AND stderr to `$OUTPUT_FILE`, and the `EXIT` trap deletes
`$WORK_DIR`. The suite therefore exits 1 with ZERO bytes on stdout and stderr,
and the one line that says what went wrong —

    Error: Gitea authenticated-identity read failed with HTTP 401

— lives only inside a directory that no longer exists when anyone looks. Every
oracle I had swept the family with greps for a SYMPTOM in surviving output, so
against this suite all of them returned "nothing found", which I read as "clean"
in the first sweep and as "unrelated pre-existing failure" in the second. A
suite that discards or deletes its own evidence converts a post-hoc assay into a
non-measurement, and I wrote that sentence into the previous commit while it was
already false about a file in the same directory.

HOW IT WAS ACTUALLY FOUND. Intercept the identity read at its SOURCE instead of
grepping for its consequence: a PATH shim over `git` that logs every
`mosaic.gitIdentity` read — args, rc, and resolved value — to a file OUTSIDE any
suite's work dir, then execs the real git. Deletion-proof by construction, and
it measures the defect's cause rather than one of its symptoms. Sweeping all 16
suites with it under an ordinary invocation:

  resolves a REAL identity (`mos-dt-0`) before the fix:
    test-issue-comment-readback          1 read   rc=1 (RED on every seat)
    test-pr-review-repo-host-override    6 reads  rc=0
    test-ci-queue-wait-branch-absent     3 reads  rc=0
  the four fixed in the previous commit now read empty; the rest never read at all.

The latter two are NOT affected and are deliberately left alone: under a seat
replica (identity set, no per-slot token) neither reaches `get_gitea_token`'s
fail-loud branch, and under a canary HOME neither carries the canary credential
into any surviving artifact. They read the identity and never enter a credential
path. That residual is structural and belongs to the wrapper half of #1007 —
scoping the read with `git -C "$repo"` removes it for everyone at once.

An earlier version of that sweep reported the four fixed suites as still
resolving a real identity. That was my grep, not the suites: `value=\[..*\]` is
satisfied by `value=[] args=[…]`, because `.*` runs past the empty pair and
matches the closing bracket of the NEXT one. `value=\[[^]]` is the correct test.
Recorded because the wrong pattern failed in the direction that would have sent
me re-fixing four already-correct files.

VERIFICATION of this suite, four HOME arms, all rc=0 with zero non-empty
identity reads and the pass line on stdout: real HOME, seat replica, canary
HOME, and an empty HOME with no identity at all. Full 16-suite sweep after the
change: every suite rc=0.

CONSEQUENCE FOR THE FINDING LIST IN THE PREVIOUS COMMIT: item 2 there — the
"silently red, unrelated to #1007" suite — is withdrawn. It was #1007 all along.
Item 1 (`pr-metadata.sh:89-92`, the anonymous fallback that reports an HTTP 200
carrying valid JSON as "unknown API error") stands and is still unfixed here.

Refs #1007
2026-07-31 07:27:10 -05:00
Jason Woltje 1afe2b36dc fix(git): #1007 suite hermeticity — pin repo-local mosaic.gitIdentity in four test suites
CENSUS CORRECTION: FOUR suites, not the three my own #1007 audit named. The
fourth (test-pr-metadata-gitea.sh) was outside the candidate set that audit
worked from and was found only by sweeping the discriminator across all 16
tools/git/test-*.sh suites. Recording that as a correction to my finding, not
as part of the original claim.

THE DEFECT. get_gitea_token() (detect-platform.sh:502-599) resolves a per-agent
identity at STEP 0, from `git config --get mosaic.gitIdentity`, BEFORE both the
Mosaic credential loader (step 1) and the GITEA_TOKEN env check (step 2). On a
provisioned agent seat that value is set GLOBALLY in ~/.gitconfig and is
inherited by any freshly-`git init`ed repo, so step 0 reads a REAL per-slot
token out of $HOME and returns it without ever consulting the suite's own
MOSAIC_CREDENTIALS_FILE / GITEA_TOKEN fixtures. The suites were running against
production credentials, and the fixture credential each one carefully
constructs was inert.

THE FIX: an empty repo-local `mosaic.gitIdentity`. An empty local value shadows
the global one and reads back empty at rc=0, so step 0 declines. The env route
does NOT work: detect-platform.sh reads "${MOSAIC_GIT_IDENTITY:-}", and `:-`
treats set-but-empty identically to unset.

OPERATIVE vs CONTAINMENT — the two mechanisms are not interchangeable and the
comment in each suite says so. The pin is operative: it prevents the resolution.
The sandboxed HOME each suite now also gets is containment: it bounds a failure
the pin should already have prevented. Conflating them is how this class stays
invisible, because a decoy HOME REMOVES the trigger (~/.gitconfig is where the
global identity lives), so any suite audited under one reads clean however
vulnerable it is. To MEASURE, replicate a seat: a decoy HOME whose .gitconfig
sets mosaic.gitIdentity with no per-slot token, so step 0 reaches its fail-loud
branch. That note is in each file for the next auditor.

SECOND, INDEPENDENT DEFECT in test-pr-metadata-gitea.sh. Applying the pin alone
turned that suite RED — and a control at baseline 826a8b3 under a plain HOME
reproduced the same failure, so it is pre-existing, not introduced. Its
`GITEA_TOKEN="stub-token"` / `GITEA_URL="https://git.example.test"` pair can
never satisfy step 2, because step 2 accepts GITEA_TOKEN only when GITEA_URL
matches the remote host and this repo's origin is git.uscllc.com. The suite had
therefore only ever passed by resolving a REAL credential — step 0 on a seat, or
step 1 from the operator's own credentials.json. A MOSAIC_CREDENTIALS_FILE
fixture is added rather than leaning on the sandboxed HOME making step 1 find
nothing: a test that passes because production configuration is ABSENT fails the
moment it is present. Shipping the pin without this would have moved the failure
rather than removed it.

NO CI ARM. .woodpecker/ci.yml does not run these suites; packages/mosaic/
package.json:28 (test:framework-shell) runs an ENUMERATED list that excludes all
four. They run only by hand — i.e. exclusively on a provisioned seat, the one
environment where the defect is live. "Passes in CI, fails on a seat" does not
apply here; there is no CI observation at all.

VERIFICATION (seat replica = decoy HOME with mosaic.gitIdentity set, no per-slot
token; canary = same plus a marked non-credential at both per-slot paths; plain
= empty HOME; real = ordinary invocation):
  - bash -n clean on all four.
  - Sweep of all 16 suites at baseline 826a8b3 under the seat replica:
    test-gitea-login-resolution rc=1 REACHES-STEP0; test-issue-create-
    interactive-auth rc=1 REACHES-STEP0; test-pr-merge-gitea-empty-uid rc=1
    REACHES-STEP0; test-pr-metadata-gitea rc=1 REACHES-STEP0.
  - Same sweep after: every row rc=0 with step0 absent.
  - test-gitea-token-identity flags REACHES-STEP0 in BOTH arms and is NOT a
    defect: it runs under `env -i HOME="$FAKE_HOME"` (line 77) and its hit is
    its own deliberate assert_failloud fixtures (lines 158-171). The fail-loud
    grep matches the intended behaviour as well as the defect, so it needs the
    second discriminator; recorded here so the next sweep does not re-file it.
  - Durable-argv assay (a PATH shim that tees argv out of each suite's own mock
    curl, because test-pr-merge-gitea-empty-uid truncates its log between phases
    and its EXIT trap removes the sandbox — a post-hoc read of that suite is a
    non-measurement, and "no trace" there is not a clearance):
      test-pr-merge-gitea-empty-uid  before: canary token in argv, fixture never
        used. after: fixture token in argv, canary absent. 5 curl calls both arms.
      test-pr-metadata-gitea         before: canary in argv. after: both calls
        carry the fixture token against git.uscllc.com.
  - test-pr-metadata-gitea across seat/canary/plain HOMEs after the fix: rc=0,
    rc=0, rc=0.
  - All four under the real HOME: rc=0. No regression to ordinary invocation.

The comment block is duplicated across the four files rather than pointing at a
shared note. Deliberate, and matching the merged #1006 precedent
(test-pr-review-gitea-comment.sh:87-95): the reader who needs it is auditing one
file.

TWO FINDINGS DELIBERATELY NOT FIXED HERE (out of this branch's scope, to be
filed):
  1. pr-metadata.sh:89-92 — the anonymous curl fallback does not check ^2, so an
     HTTP 200 carrying valid JSON is reported as "unknown API error" at rc=1.
  2. test-issue-comment-readback.sh exits 1 with ZERO bytes on stdout AND
     stderr, dying at its first seed_state python3 heredoc. Reproduces at
     baseline 826a8b3 under both a seat replica and the real HOME. Silently red
     at main for everyone; unrelated to #1007.

Refs #1007
2026-07-31 07:13:13 -05:00
13 changed files with 213 additions and 623 deletions
-5
View File
@@ -41,11 +41,6 @@ steps:
# (Constitution + dispatcher + each RUNTIME.md slice). See DESIGN §7 / R9.
- bash packages/mosaic/framework/tools/quality/scripts/check-resident-budget.sh --self-test
- bash packages/mosaic/framework/tools/quality/scripts/check-resident-budget.sh
# Test-membership guard (#1017): also first link of test:framework-shell.
# Invoked from BOTH surfaces it audits (F2, PR #1018) — the guard is link
# [0] of the pnpm chain, so severing that chain would silence it together
# with everything it guards; this direct line keeps one instrument running.
- bash packages/mosaic/framework/tools/quality/scripts/check-test-enumeration.sh
# Blocking gate (#791): a framework upgrade must never write or delete an
# operator-owned path. The HARD GATE proves an unanticipated operator sentinel
@@ -34,18 +34,12 @@ EOF
# get_remote_host and get_gitea_token are provided by detect-platform.sh
get_state_from_status_json() {
# Capture piped JSON BEFORE invoking `python3 - <<PY`. The heredoc binds
# stdin to the Python program text — so json.load(sys.stdin) inside would
# try to re-read stdin after `-` already consumed it for the program,
# yielding EOF and returning "unknown" every time. Pass payload via env.
local payload
payload=$(cat)
CI_QUEUE_STATUS_JSON="$payload" python3 - <<'PY'
python3 - <<'PY'
import json
import os
import sys
try:
payload = json.loads(os.environ.get("CI_QUEUE_STATUS_JSON", ""))
payload = json.load(sys.stdin)
except Exception:
print("unknown")
raise SystemExit(0)
@@ -89,15 +83,12 @@ PY
}
print_pending_contexts() {
# Same stdin hazard as get_state_from_status_json above — pass payload via env.
local payload
payload=$(cat)
CI_QUEUE_STATUS_JSON="$payload" python3 - <<'PY'
python3 - <<'PY'
import json
import os
import sys
try:
payload = json.loads(os.environ.get("CI_QUEUE_STATUS_JSON", ""))
payload = json.load(sys.stdin)
except Exception:
print("[ci-queue-wait] unable to decode status payload")
raise SystemExit(0)
@@ -1,205 +0,0 @@
#!/usr/bin/env bash
# Regression suite for #1019: ci-queue-wait.sh's status parser must return a
# state DERIVED FROM ITS INPUT.
#
# The defect this pins: both python sites piped a payload into `python3 - <<'PY'`.
# The heredoc binds stdin to the Python program text, so json.load(sys.stdin) saw
# EOF, the bare `except` fired, and the function returned "unknown" for every input
# — success, pending and failure alike. "unknown" then reaches an `exit 0` arm, so
# the gate-6 queue guard passed unconditionally on both the gitea and github paths.
#
# Every assertion below is on the RETURNED STATE STRING. A test that asserted only
# `rc=0` would have passed against the broken build, which is why this bug survived.
#
# The functions are extracted from the shipped wrapper rather than reimplemented, so
# this suite measures the code that actually runs.
set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# Overridable so a mutation test can point the suite at a deliberately-broken copy
# without touching the tracked file. The earlier version of this suite required
# `git stash` + `git checkout` to do that, which left a repo-global stash entry
# that any sibling worktree could have popped onto an unrelated branch.
WRAPPER="${CI_QUEUE_WAIT_WRAPPER:-$SCRIPT_DIR/ci-queue-wait.sh}"
PASS=0
FAIL=0
pass() { printf ' PASS %s\n' "$1"; PASS=$((PASS + 1)); }
fail() { printf ' FAIL %s\n' "$1"; FAIL=$((FAIL + 1)); }
if [[ ! -f "$WRAPPER" ]]; then
printf 'FATAL: wrapper not found at %s\n' "$WRAPPER" >&2
exit 1
fi
TMPDIR_T="$(mktemp -d)"
trap 'rm -rf "$TMPDIR_T"' EXIT
extract_fn() {
# $1 = function name, $2 = source file, $3 = destination
sed -n "/^$1()/,/^}/p" "$2" > "$3"
[[ -s "$3" ]] || { printf 'FATAL: could not extract %s from %s\n' "$1" "$2" >&2; exit 1; }
}
extract_fn get_state_from_status_json "$WRAPPER" "$TMPDIR_T/state.sh"
extract_fn print_pending_contexts "$WRAPPER" "$TMPDIR_T/contexts.sh"
state_of() {
# shellcheck disable=SC1091
( source "$TMPDIR_T/state.sh"; printf '%s' "$1" | get_state_from_status_json )
}
expect_state() {
local label="$1" payload="$2" want="$3" got
got="$(state_of "$payload")"
if [[ "$got" == "$want" ]]; then
pass "$label -> $want"
else
fail "$label -> got '$got', want '$want'"
fi
}
echo "=== parser returns a state derived from its input (#1019) ==="
expect_state "success payload" \
'{"state":"success","statuses":[{"status":"success","context":"ci/build"}]}' \
'terminal-success'
expect_state "pending payload" \
'{"state":"pending","statuses":[{"status":"pending","context":"ci/build"}]}' \
'pending'
expect_state "failure payload" \
'{"state":"failure","statuses":[{"status":"failure","context":"ci/build"}]}' \
'terminal-failure'
expect_state "mixed success+pending is pending" \
'{"state":"pending","statuses":[{"status":"success"},{"status":"pending"}]}' \
'pending'
expect_state "running counts as pending" \
'{"state":"pending","statuses":[{"status":"running"}]}' \
'pending'
expect_state "error counts as failure" \
'{"state":"failure","statuses":[{"status":"error"}]}' \
'terminal-failure'
expect_state "empty status set is no-status" \
'{"state":"","statuses":[]}' \
'no-status'
# Control. This is the ONE input for which "unknown" is correct. Without it, a
# regression that hardcoded "unknown" again would still fail the cases above but
# the suite would give no signal that "unknown" remains reachable when it should be.
expect_state "undecodable payload stays unknown" \
'not json at all' \
'unknown'
expect_state "unrecognised status vocabulary is unknown" \
'{"state":"weird","statuses":[{"status":"weird"}]}' \
'unknown'
echo "=== pending contexts are reported to the operator ==="
contexts_of() {
# shellcheck disable=SC1091
( source "$TMPDIR_T/contexts.sh"; printf '%s' "$1" | print_pending_contexts )
}
out="$(contexts_of '{"statuses":[{"status":"pending","context":"ci/alpha"},{"status":"pending","context":"ci/beta"}]}')"
if grep -q 'ci/alpha' <<<"$out" && grep -q 'ci/beta' <<<"$out"; then
pass "both pending contexts emitted"
else
fail "pending contexts not emitted; got: $out"
fi
out="$(contexts_of '{"statuses":[{"status":"success","context":"ci/alpha"}]}')"
# Assert the POSITIVE message, not merely the absence of the context name. Absence
# alone is satisfied by total silence — and the pre-fix build was silent, so an
# absence-only assertion passed against the very defect this suite exists to catch.
if grep -q 'ci/alpha' <<<"$out"; then
fail "a non-pending context was emitted; got: $out"
elif grep -q 'no pending contexts' <<<"$out"; then
pass "non-pending context suppressed, and reported as 'no pending contexts'"
else
fail "expected an explicit 'no pending contexts' report; got: $out"
fi
echo "=== needle: the broken construct is caught, not merely absent today ==="
# Rebuild the pre-fix form and assert this suite would have failed against it.
# Without this, the suite proves the current file is correct but not that it can
# detect the defect returning.
BROKEN="$TMPDIR_T/broken.sh"
cat > "$BROKEN" <<'BROKEN_EOF'
get_state_from_status_json() {
python3 - <<'PY'
import json
import sys
try:
payload = json.load(sys.stdin)
except Exception:
print("unknown")
raise SystemExit(0)
print("terminal-success" if (payload.get("state") or "") == "success" else "pending")
PY
}
BROKEN_EOF
broken_got="$( ( source "$BROKEN"; printf '%s' '{"state":"success","statuses":[]}' | get_state_from_status_json ) )"
if [[ "$broken_got" == "unknown" ]]; then
pass "[NEEDLE ] pre-fix construct reproduces the defect (returns 'unknown' for a success payload)"
else
fail "[NEEDLE ] pre-fix construct did NOT reproduce the defect; got '$broken_got' — the needle no longer pins anything"
fi
# And assert the shipped wrapper does not contain that construct.
#
# The check must have HEREDOC SEMANTICS, not merely match the text. `json.load(sys.stdin)`
# is perfectly correct under `python3 -c '...'` — there the program comes from argv, so
# stdin really is the payload, and ci-queue-wait.sh uses that form legitimately in
# gitea_get_branch_head_sha. Only `python3 - <<DELIM` is defective, because `-` has
# already bound stdin to the program text.
#
# A flat `grep json\.load\(sys\.stdin\)` therefore fails on correct code. It did: the
# first version of this needle flagged the innocent `python3 -c` site. Same shape as
# #1018 F1 (a regex over raw text has no syntax semantics) reproduced inside the needle
# written to pin a different instance of it.
heredoc_stdin_sites() {
awk '
/python3[[:space:]]+-[[:space:]]*<</ {
d = $0
sub(/.*<<[[:space:]]*/, "", d)
gsub(/['"'"'"]/, "", d)
delim = d; inhd = 1; next
}
inhd && $0 == delim { inhd = 0; next }
inhd {
line = $0
sub(/[[:space:]]*#.*$/, "", line)
if (line ~ /sys\.stdin/) printf "%d: %s\n", FNR, $0
}
' "$1"
}
# Positive control FIRST. An empty result from a broken scanner looks exactly like a
# clean file, so the scanner is proven to fire before its silence is trusted.
if [[ -n "$(heredoc_stdin_sites "$BROKEN")" ]]; then
pass "[NEEDLE ] scanner detects a stdin read inside a python3-heredoc"
else
fail "[NEEDLE ] scanner did NOT fire on the known-broken form — its silence proves nothing"
fi
sites="$(heredoc_stdin_sites "$WRAPPER")"
if [[ -n "$sites" ]]; then
fail "[NEEDLE ] wrapper reads stdin inside a python3-heredoc at: $sites"
else
pass "[NEEDLE ] wrapper has no stdin read inside any python3-heredoc"
fi
printf '\nci-queue-wait parse: %d passed, %d failed\n' "$PASS" "$FAIL"
[[ "$FAIL" -eq 0 ]] || exit 1
@@ -7,14 +7,40 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
WORK_DIR="${MOSAIC_TEST_WORK_DIR:-$PWD/.mosaic-test-work/gitea-login-resolution}"
REPO_DIR="$WORK_DIR/repo"
BIN_DIR="$WORK_DIR/bin"
HOME_DIR="$WORK_DIR/home"
LOG_FILE="$WORK_DIR/calls.log"
CREDENTIALS_FILE="$WORK_DIR/credentials.json"
rm -rf "$WORK_DIR"
mkdir -p "$REPO_DIR" "$BIN_DIR"
mkdir -p "$REPO_DIR" "$BIN_DIR" "$HOME_DIR"
git -C "$REPO_DIR" init -q
git -C "$REPO_DIR" remote add origin https://git.uscllc.com/USC/uconnect.git
# HERMETICITY (#1007) — TWO mechanisms with DIFFERENT jobs; do not conflate them.
#
# OPERATIVE: the empty repo-local `mosaic.gitIdentity` below. get_gitea_token()
# step 0 resolves a per-agent identity from `git config --get mosaic.gitIdentity`,
# which on a provisioned agent seat is set GLOBALLY and so leaks into this fresh
# repo. It then reads a REAL per-slot token from $HOME and returns it WITHOUT ever
# consulting MOSAIC_CREDENTIALS_FILE, so the fixture credentials below are silently
# ignored. This suite is the one where the consequence is not subtle: it FAILS
# outright on a provisioned seat (rc=1 bare, rc=0 with $HOME sandboxed, one
# variable changed) and passes everywhere else, including CI, which has no
# per-agent token to leak.
#
# CONTAINMENT: the sandboxed HOME in the four run helpers below. It only has to
# bound a failure that the pin should already have prevented.
#
# NOTE FOR ANYONE AUDITING THIS SUITE: the sandboxed HOME is containment, NOT an
# assay. Running a suite under a decoy HOME to test for this defect REMOVES the
# trigger — ~/.gitconfig is where the global identity lives, so step 0 is skipped
# by construction and every suite reads clean however vulnerable it is. To measure,
# REPLICATE a seat (a decoy HOME whose .gitconfig sets mosaic.gitIdentity, with no
# per-slot token) so step 0 reaches its fail-loud branch.
#
# Note the env-var route does NOT work: detect-platform.sh reads
# "${MOSAIC_GIT_IDENTITY:-}", and `:-` treats set-but-empty identically to unset.
git -C "$REPO_DIR" config mosaic.gitIdentity ""
cat > "$CREDENTIALS_FILE" <<'JSON'
{
@@ -86,6 +112,7 @@ run_in_repo() {
(
cd "$REPO_DIR"
PATH="$BIN_DIR:$PATH" \
HOME="$HOME_DIR" \
MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \
MOSAIC_TEST_LOG="$LOG_FILE" \
"$@"
@@ -283,6 +310,7 @@ run_in_repo2() {
(
cd "$REPO_DIR"
PATH="$BIN_DIR2:$PATH" \
HOME="$HOME_DIR" \
MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \
MOSAIC_TEST_LOG="$LOG_FILE" \
"$@"
@@ -343,7 +371,7 @@ write_fixture() { printf '%s' "$1" > "$FIXTURE_XDG/tea/config.yml"; }
token_fallback() {
(
cd "$REPO_DIR"
XDG_CONFIG_HOME="$FIXTURE_XDG" PYTHONPATH="$NOYAML_DIR" bash -c '
HOME="$HOME_DIR" XDG_CONFIG_HOME="$FIXTURE_XDG" PYTHONPATH="$NOYAML_DIR" bash -c '
source "'"$SCRIPT_DIR"'/detect-platform.sh"
get_gitea_token_for_login "$1" "$2"
' _ "$1" "$2"
@@ -354,7 +382,7 @@ token_fallback() {
token_pyyaml() {
(
cd "$REPO_DIR"
XDG_CONFIG_HOME="$FIXTURE_XDG" bash -c '
HOME="$HOME_DIR" XDG_CONFIG_HOME="$FIXTURE_XDG" bash -c '
source "'"$SCRIPT_DIR"'/detect-platform.sh"
get_gitea_token_for_login "$1" "$2"
' _ "$1" "$2"
@@ -61,15 +61,54 @@ STATE_FILE="$WORK_DIR/comments.json"
# A dedicated scratch dir the wrapper is pointed at via TMPDIR, so the leak
# check can assert every POST/GET body + metadata temp file is cleaned up.
TMP_SCRATCH="$WORK_DIR/scratch"
HOME_DIR="$WORK_DIR/home"
cleanup() {
rm -rf "$WORK_DIR"
}
trap cleanup EXIT
mkdir -p "$REPO_DIR" "$BIN_DIR" "$XDG_DIR" "$TMP_SCRATCH"
mkdir -p "$REPO_DIR" "$BIN_DIR" "$XDG_DIR" "$TMP_SCRATCH" "$HOME_DIR"
git -C "$REPO_DIR" init -q
git -C "$REPO_DIR" remote add origin https://git.mosaicstack.dev/mosaicstack/stack.git
# HERMETICITY (#1007) — TWO mechanisms with DIFFERENT jobs; do not conflate them.
#
# OPERATIVE: the empty repo-local `mosaic.gitIdentity` below. get_gitea_token()
# step 0 resolves a per-agent identity from `git config --get mosaic.gitIdentity`,
# which on a provisioned agent seat is set GLOBALLY and so leaks into this fresh
# repo. It then reads a REAL per-slot token from $HOME and returns it WITHOUT ever
# consulting MOSAIC_CREDENTIALS_FILE, so the fixture credential below is silently
# ignored. The stub curl then rejects the unrecognised bearer, and this suite
# fails at its FIRST case with `Gitea authenticated-identity read failed with
# HTTP 401`. An empty repo-local value shadows the global one and reads back
# empty at rc=0. Measured: without this pin the suite is RED on every seat.
#
# CONTAINMENT: the sandboxed HOME in run_comment(). It only has to bound a
# failure that the pin should already have prevented.
#
# THIS SUITE WAS THE HARDEST OF THE FIVE TO SEE, and the reason is worth stating
# because it generalises: run_comment() sends the wrapper's stdout AND stderr to
# $OUTPUT_FILE, and the EXIT trap above deletes $WORK_DIR. So the 401 — the only
# thing that says what went wrong — exists only inside a directory that is gone
# by the time anyone looks. The suite exits 1 with ZERO bytes on stdout and
# stderr. A suite that discards or deletes its own evidence turns any post-hoc
# assay into a non-measurement: "nothing found" there means "no surviving
# trace", never "clean". It was found by intercepting the identity read at its
# SOURCE (a PATH shim over `git` logging every `mosaic.gitIdentity` read to a
# file outside $WORK_DIR), which is deletion-proof by construction, rather than
# by grepping for the symptom.
#
# NOTE FOR ANYONE AUDITING THIS SUITE: the sandboxed HOME is containment, NOT an
# assay. Running a suite under a decoy HOME to test for this defect REMOVES the
# trigger — ~/.gitconfig is where the global identity lives, so step 0 is skipped
# by construction and every suite reads clean however vulnerable it is. To
# measure, REPLICATE a seat (a decoy HOME whose .gitconfig sets
# mosaic.gitIdentity, with no per-slot token) so step 0 reaches its fail-loud
# branch — or intercept the read as described above.
#
# Note the env-var route does NOT work: detect-platform.sh reads
# "${MOSAIC_GIT_IDENTITY:-}", and `:-` treats set-but-empty identically to unset.
git -C "$REPO_DIR" config mosaic.gitIdentity ""
ISSUE_NUMBER=7
REPO_SLUG="mosaicstack/stack"
@@ -366,6 +405,7 @@ run_comment() {
cd "$REPO_DIR"
PATH="$BIN_DIR:$PATH" \
TMPDIR="$TMP_SCRATCH" \
HOME="$HOME_DIR" \
XDG_CONFIG_HOME="$XDG_DIR" \
MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \
ISSUE_COMMENT_TEA_LOG="$TEA_LOG" \
@@ -7,13 +7,38 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
WORK_DIR="${MOSAIC_TEST_WORK_DIR:-$PWD/.mosaic-test-work/issue-create-interactive-auth}"
REPO_DIR="$WORK_DIR/repo"
BIN_DIR="$WORK_DIR/bin"
HOME_DIR="$WORK_DIR/home"
LOG_FILE="$WORK_DIR/calls.log"
CREDENTIALS_FILE="$WORK_DIR/credentials.json"
rm -rf "$WORK_DIR"
mkdir -p "$REPO_DIR" "$BIN_DIR"
mkdir -p "$REPO_DIR" "$BIN_DIR" "$HOME_DIR"
git -C "$REPO_DIR" init -q
git -C "$REPO_DIR" remote add origin https://git.mosaicstack.dev/mosaicstack/stack.git
# HERMETICITY (#1007) — TWO mechanisms with DIFFERENT jobs; do not conflate them.
#
# OPERATIVE: the empty repo-local `mosaic.gitIdentity` below. get_gitea_token()
# step 0 resolves a per-agent identity from `git config --get mosaic.gitIdentity`,
# which on a provisioned agent seat is set GLOBALLY and so leaks into this fresh
# repo. It then reads a REAL per-slot token from $HOME and returns it WITHOUT ever
# consulting MOSAIC_CREDENTIALS_FILE, so the fixture credential below is silently
# ignored and the suite runs against a production credential. An empty repo-local
# value shadows the global one and reads back empty at rc=0. Measured: this suite
# resolves a per-slot token without it.
#
# CONTAINMENT: the sandboxed HOME in run_wrapper(). It only has to bound a failure
# that the pin should already have prevented.
#
# NOTE FOR ANYONE AUDITING THIS SUITE: the sandboxed HOME is containment, NOT an
# assay. Running a suite under a decoy HOME to test for this defect REMOVES the
# trigger — ~/.gitconfig is where the global identity lives, so step 0 is skipped
# by construction and every suite reads clean however vulnerable it is. To measure,
# REPLICATE a seat (a decoy HOME whose .gitconfig sets mosaic.gitIdentity, with no
# per-slot token) so step 0 reaches its fail-loud branch.
#
# Note the env-var route does NOT work: detect-platform.sh reads
# "${MOSAIC_GIT_IDENTITY:-}", and `:-` treats set-but-empty identically to unset.
git -C "$REPO_DIR" config mosaic.gitIdentity ""
cat > "$CREDENTIALS_FILE" <<'JSON'
{"gitea":{"mosaicstack":{"url":"https://git.mosaicstack.dev","token":"test-token"}}}
@@ -50,6 +75,7 @@ run_wrapper() {
(
cd "$REPO_DIR"
PATH="$BIN_DIR:$PATH" \
HOME="$HOME_DIR" \
MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \
MOSAIC_TEST_LOG="$LOG_FILE" \
"$@"
@@ -8,6 +8,7 @@ WORK_ROOT="${AGENT_WORK_ROOT:-${HOME:-/tmp}/mosaic/agent-work}"
SANDBOX="$WORK_ROOT/pr-merge-empty-uid-test-$$"
MOCK_BIN="$SANDBOX/bin"
REPO_DIR="$SANDBOX/repo"
HOME_DIR="$SANDBOX/home"
LOG_FILE="$SANDBOX/mock.log"
cleanup() {
@@ -15,7 +16,7 @@ cleanup() {
}
trap cleanup EXIT
mkdir -p "$MOCK_BIN" "$REPO_DIR"
mkdir -p "$MOCK_BIN" "$REPO_DIR" "$HOME_DIR"
: > "$LOG_FILE"
cat > "$MOCK_BIN/tea" <<'EOF'
@@ -99,7 +100,48 @@ chmod +x "$MOCK_BIN/curl"
cd "$REPO_DIR"
git init -q
git remote add origin https://git.mosaicstack.dev/mosaicstack/stack.git
# HERMETICITY (#1007) — TWO mechanisms with DIFFERENT jobs; do not conflate them.
#
# OPERATIVE: the empty repo-local `mosaic.gitIdentity` below. get_gitea_token()
# step 0 resolves a per-agent identity from `git config --get mosaic.gitIdentity`,
# which on a provisioned agent seat is set GLOBALLY and so leaks into this fresh
# repo. Step 0 runs BEFORE the credential loader AND before the GITEA_TOKEN env
# check, so the `GITEA_TOKEN=redacted-test-token` exported below is silently
# overridden and a REAL per-slot token from $HOME is what flows through the
# wrapper. Measured on a provisioned seat before this pin: all 5 mock-curl calls
# carried the real per-slot token in argv and the fixture token was never used at
# ALL. Three consequences specific to this suite:
# 1. pr-merge.sh passes the token as `-H "Authorization: token $token"` and the
# mock curl logs full argv, so the real credential is written to $LOG_FILE
# on disk — transiently: the suite truncates that file between phases and
# the EXIT trap removes $SANDBOX, so it leaves NO post-hoc trace. That is
# why this suite was the hardest of the three to detect; observing it needs
# an instrument that captures argv while the run is live.
# 2. Every failure path dumps $OUTPUT/$LOG_FILE to stderr through
# `sed 's/redacted-test-token/***REDACTED***/g'` — a redaction pattern that
# is the literal fixture string and therefore CANNOT match the token
# actually in use.
# 3. The leak assertion at "Token leaked to pr-merge.sh output" greps for that
# same fixture string, so on a provisioned seat it passes vacuously: it is
# searching for a value the run never used.
# An empty repo-local value shadows the global one and reads back empty at rc=0.
#
# CONTAINMENT: the sandboxed HOME exported below. It only has to bound a failure
# that the pin should already have prevented.
#
# NOTE FOR ANYONE AUDITING THIS SUITE: the sandboxed HOME is containment, NOT an
# assay. Running a suite under a decoy HOME to test for this defect REMOVES the
# trigger — ~/.gitconfig is where the global identity lives, so step 0 is skipped
# by construction and every suite reads clean however vulnerable it is. To measure,
# REPLICATE a seat (a decoy HOME whose .gitconfig sets mosaic.gitIdentity, with no
# per-slot token) so step 0 reaches its fail-loud branch.
#
# Note the env-var route does NOT work: detect-platform.sh reads
# "${MOSAIC_GIT_IDENTITY:-}", and `:-` treats set-but-empty identically to unset.
git -C "$REPO_DIR" config mosaic.gitIdentity ""
# $SANDBOX/$HOME_DIR were derived from the real $HOME above, before this export.
export HOME="$HOME_DIR"
export PATH="$MOCK_BIN:$PATH"
export PR_MERGE_TEST_LOG="$LOG_FILE"
export GITEA_LOGIN="git.mosaicstack.dev"
@@ -8,12 +8,68 @@ WORK_DIR="${MOSAIC_TEST_WORK_DIR:-$PWD/.mosaic-test-work/pr-metadata-gitea}"
REPO_DIR="$WORK_DIR/repo"
FIXTURE_DIR="$WORK_DIR/fixtures"
STUB_DIR="$WORK_DIR/stubs"
HOME_DIR="$WORK_DIR/home"
CREDENTIALS_FILE="$WORK_DIR/credentials.json"
rm -rf "$WORK_DIR"
mkdir -p "$REPO_DIR" "$FIXTURE_DIR" "$STUB_DIR"
mkdir -p "$REPO_DIR" "$FIXTURE_DIR" "$STUB_DIR" "$HOME_DIR"
git -C "$REPO_DIR" init -q
git -C "$REPO_DIR" remote add origin https://git.uscllc.com/USC/uconnect.git
# HERMETICITY (#1007) — TWO mechanisms with DIFFERENT jobs; do not conflate them.
#
# OPERATIVE: the empty repo-local `mosaic.gitIdentity` below. get_gitea_token()
# step 0 resolves a per-agent identity from `git config --get mosaic.gitIdentity`,
# which on a provisioned agent seat is set GLOBALLY and so leaks into this fresh
# repo. Step 0 runs BEFORE the credential loader AND before the GITEA_TOKEN env
# check, so the `GITEA_TOKEN="stub-token"` set in the run helpers below is
# silently overridden and a REAL per-slot token from $HOME is what reaches curl.
# Measured on a provisioned seat before this pin: both stub-curl calls carried
# the real token in argv. An empty repo-local value shadows the global one and
# reads back empty at rc=0.
#
# CONTAINMENT: the sandboxed HOME in the three run helpers below. It only has to
# bound a failure that the pin should already have prevented.
#
# NOTE FOR ANYONE AUDITING THIS SUITE: the sandboxed HOME is containment, NOT an
# assay. Running a suite under a decoy HOME to test for this defect REMOVES the
# trigger — ~/.gitconfig is where the global identity lives, so step 0 is skipped
# by construction and every suite reads clean however vulnerable it is. To measure,
# REPLICATE a seat (a decoy HOME whose .gitconfig sets mosaic.gitIdentity, with no
# per-slot token) so step 0 reaches its fail-loud branch. See
# test-gitea-token-identity.sh for the stronger `env -i HOME=…` form used where a
# suite's whole subject IS identity resolution.
#
# Note the env-var route does NOT work: detect-platform.sh reads
# "${MOSAIC_GIT_IDENTITY:-}", and `:-` treats set-but-empty identically to unset.
git -C "$REPO_DIR" config mosaic.gitIdentity ""
# The pin above removes step 0, but this suite has a SECOND, independent
# dependency on operator state, and closing only the first would leave the suite
# red on any hermetic environment. The `GITEA_TOKEN="stub-token"` /
# `GITEA_URL="https://git.example.test"` pair the run helpers set is INERT: step 2
# of get_gitea_token accepts GITEA_TOKEN only when GITEA_URL matches the remote
# host, and this repo's origin is git.uscllc.com, so that pair can never satisfy
# it. Before this fixture the only credential that could reach the authenticated
# curl branch was a REAL one — from step 0 on an agent seat, or from step 1
# reading the operator's own ~/.config/mosaic/credentials.json. That is why the
# "curl success path" case passed: not because the stub credential worked, but
# because a production credential was available.
#
# A fixture is used rather than relying on the sandboxed HOME making step 1 find
# nothing: a test that passes because production configuration is ABSENT fails
# the moment it is present. Step 1 now resolves deterministically to a value that
# is a fixture on every machine.
cat > "$CREDENTIALS_FILE" <<'JSON'
{
"gitea": {
"usc": {
"url": "https://git.uscllc.com",
"token": "stub-token"
}
}
}
JSON
cat > "$FIXTURE_DIR/gitea-standard.json" <<'JSON'
{
@@ -131,6 +187,8 @@ run_curl_success_case() {
set +e
output=$(cd "$REPO_DIR" && \
PATH="$STUB_DIR:$PATH" \
HOME="$HOME_DIR" \
MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \
TMPDIR="$tmpdir" \
GITEA_TOKEN="stub-token" \
GITEA_URL="https://git.example.test" \
@@ -170,6 +228,8 @@ run_curl_early_exit_cleanup_case() {
set +e
output=$(cd "$REPO_DIR" && \
PATH="$STUB_DIR:$PATH" \
HOME="$HOME_DIR" \
MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \
TMPDIR="$tmpdir" \
GITEA_TOKEN="stub-token" \
GITEA_URL="https://git.example.test" \
@@ -204,7 +264,8 @@ run_curl_early_exit_cleanup_case() {
run_case() {
local fixture="$1" expected_number="$2" expected_head="$3"
local output
output=$(cd "$REPO_DIR" && MOSAIC_GITEA_PR_METADATA_RAW_FILE="$fixture" "$SCRIPT_DIR/pr-metadata.sh" -n "$expected_number")
output=$(cd "$REPO_DIR" && HOME="$HOME_DIR" MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \
MOSAIC_GITEA_PR_METADATA_RAW_FILE="$fixture" "$SCRIPT_DIR/pr-metadata.sh" -n "$expected_number")
PR_METADATA_OUTPUT="$output" python3 - "$expected_number" "$expected_head" <<'PY'
import json
import os
@@ -1,165 +0,0 @@
#!/usr/bin/env bash
# check-test-enumeration.sh — CI test-membership guard (#1017).
#
# CI reaches shell suites through two hand-enumerated surfaces:
# S1 packages/mosaic/package.json scripts."test:framework-shell"
# S2 .woodpecker/ci.yml direct `bash packages/mosaic/framework/tools/...` commands
#
# A hand-enumerated allowlist re-arms its own gap: a new suite never auto-joins,
# so the list silently under-runs the disk (17 of 39 suites were invisible when
# #1017 was filed). This guard makes that under-run impossible to do silently:
#
# FAIL when a suite-shaped file exists on disk and is neither enumerated on
# the UNION of both surfaces nor listed in the exclusions file.
# ("Enumerated", deliberately — F1/F2 on PR #1018 proved this guard sees
# NAMING, not reachability, and its words must not claim otherwise.)
# FAIL when either surface names a path that does not exist on disk
# (a rename manufactures a stale entry silently — checked BOTH directions).
# FAIL when an exclusion entry has no reason, names a path that is gone,
# names a path that is also enumerated (contradiction), or names a path
# outside the population (dead weight that looks like coverage).
#
# POPULATION PATTERN — a deliberate decision, stated per #1017's record:
# basename matches *test*.sh (contains "test", ends ".sh"). Deliberately BROAD:
# the strict `test-*.sh` prefix cannot even name three real boundary files
# (tmux/agent-send.test.sh — CI-run; orchestrator/smoke-test.sh;
# wake/validate-973/microtest-wake-assert.sh), and three independent censuses
# handled that last file three different ways with no trace of the judgement.
# The broad pattern makes such files MEMBERS, so their disposition must be a
# signed exclusion, not an accident of the glob. The SAME pattern is applied to
# both sides of the comparison (disk and enumeration) — a comparison globbed two
# ways runs on two different populations. Scripts outside the pattern on both
# sides symmetrically (e.g. check-resident-budget.sh, verify-sanitized.sh) are
# check-scripts, not suites; their existence is still verified via the
# both-directions rule because every surface-named path must exist on disk.
#
# The surfaces are PARSED, never line-ranged: three seats independently
# mis-scoped hand-written line ranges against these files (#1017 thread). S1 is
# read via JSON + command-chain tokenization; S2 by extracting every
# packages/mosaic/framework/tools/ token wherever it appears in the file.
#
# Exclusions file format (framework/tools/quality/test-enumeration-exclusions.txt):
# <repo-relative-path> | <non-empty reason>
# Lines starting with # and blank lines are ignored. An exclusion is a recorded
# decision someone signed, not an omission nobody made.
set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "$SCRIPT_DIR/../../../../../.." && pwd)"
while (( $# )); do
case "$1" in
--root) ROOT="$(cd "$2" && pwd)"; shift 2 ;;
*) echo "usage: check-test-enumeration.sh [--root <repo-root>]" >&2; exit 2 ;;
esac
done
PKG_JSON="$ROOT/packages/mosaic/package.json"
CI_YML="$ROOT/.woodpecker/ci.yml"
TOOLS_DIR="$ROOT/packages/mosaic/framework/tools"
EXCLUSIONS="$TOOLS_DIR/quality/test-enumeration-exclusions.txt"
for f in "$PKG_JSON" "$CI_YML"; do
[[ -f "$f" ]] || { echo "FAIL: required surface file missing: $f" >&2; exit 2; }
done
[[ -d "$TOOLS_DIR" ]] || { echo "FAIL: tools dir missing: $TOOLS_DIR" >&2; exit 2; }
fail_count=0
fail() { printf 'FAIL %s\n' "$1"; fail_count=$(( fail_count + 1 )); }
# in_population <repo-relative path> — the single pattern, used for BOTH sides.
in_population() {
local base; base="$(basename "$1")"
[[ "$base" == *test*.sh ]]
}
# --- Surface 1: package.json test:framework-shell, parsed, repo-relative -----
# Tokens are script paths iff they contain "/" and end .sh/.py; interpreter
# names and flags are skipped. Paths are relative to packages/mosaic/.
mapfile -t S1 < <(python3 - "$PKG_JSON" <<'PY'
import json, shlex, sys
cmd = json.load(open(sys.argv[1]))["scripts"].get("test:framework-shell", "")
seen = []
for seg in cmd.split("&&"):
for tok in shlex.split(seg):
if "/" in tok and (tok.endswith(".sh") or tok.endswith(".py")):
path = "packages/mosaic/" + tok
if path not in seen:
seen.append(path)
print("\n".join(seen))
PY
)
# --- Surface 2: ci.yml, every framework/tools token wherever it appears ------
# Comment lines (first non-whitespace char is #) are skipped BEFORE matching:
# commenting an invocation out is the most common way a suite actually gets
# disabled, and a raw-text regex would keep calling it enumerated (F1, 20155 on
# PR #1018 — demonstrated, not argued). Known residual limit: a path named only
# in a TRAILING comment on a live line still matches; no such line exists today
# and full fidelity would need a YAML parser the CI image does not ship.
mapfile -t S2 < <(grep -vE '^[[:space:]]*#' "$CI_YML" \
| grep -oE 'packages/mosaic/framework/tools/[A-Za-z0-9_./-]+\.(sh|py)' | sort -u)
# --- Union, and its population-restricted view -------------------------------
declare -A ENUM=() ENUM_POP=()
for p in "${S1[@]:-}" "${S2[@]:-}"; do
[[ -n "$p" ]] || continue
ENUM["$p"]=1
in_population "$p" && ENUM_POP["$p"]=1
done
# --- Direction B: every surface-named path must exist on disk ----------------
for p in "${!ENUM[@]}"; do
[[ -f "$ROOT/$p" ]] || fail "STALE ENUMERATION: surfaces name '$p' but it does not exist on disk"
done
# --- Exclusions: parsed with the same rigor the enumeration gets -------------
declare -A EXCLUDED=()
if [[ -f "$EXCLUSIONS" ]]; then
lineno=0
while IFS= read -r line; do
lineno=$(( lineno + 1 ))
[[ "$line" =~ ^[[:space:]]*(#|$) ]] && continue
path="${line%%|*}"; reason="${line#*|}"
path="$(echo "$path" | sed 's/^[[:space:]]*//; s/[[:space:]]*$//')"
reason="$(echo "$reason" | sed 's/^[[:space:]]*//; s/[[:space:]]*$//')"
if [[ "$line" != *"|"* || -z "$reason" ]]; then
fail "EXCLUSION MISSING REASON: line $lineno ('$path') — an exclusion is a recorded decision someone signed"
continue
fi
if [[ ! -f "$ROOT/$path" ]]; then
fail "STALE EXCLUSION: line $lineno excludes '$path' which does not exist on disk"
continue
fi
if ! in_population "$path"; then
fail "EXCLUSION OUTSIDE POPULATION: line $lineno excludes '$path' which the population pattern does not name — dead weight that reads as coverage"
continue
fi
if [[ -n "${ENUM[$path]:-}" ]]; then
fail "CONTRADICTORY EXCLUSION: line $lineno excludes '$path' which the surfaces already enumerate"
continue
fi
EXCLUDED["$path"]=1
done < "$EXCLUSIONS"
fi
# --- Direction A: disk population must be enumerated or signed-excluded ------
disk_total=0
unlisted=0
while IFS= read -r f; do
rel="${f#"$ROOT"/}"
in_population "$rel" || continue
disk_total=$(( disk_total + 1 ))
if [[ -z "${ENUM_POP[$rel]:-}" && -z "${EXCLUDED[$rel]:-}" ]]; then
fail "UNENUMERATED: '$rel' exists on disk but is neither enumerated on any CI surface nor signed in the exclusions file"
unlisted=$(( unlisted + 1 ))
fi
done < <(find "$TOOLS_DIR" -type f -name '*.sh' | sort)
if (( fail_count > 0 )); then
printf 'enumeration guard: %d failure(s) — population %d, enumerated (in-population) %d, excluded %d\n' \
"$fail_count" "$disk_total" "${#ENUM_POP[@]}" "${#EXCLUDED[@]}"
exit 1
fi
printf 'enumeration guard: OK — population %d, enumerated (in-population) %d, excluded (signed) %d, surfaces name %d path(s), all present on disk\n' \
"$disk_total" "${#ENUM_POP[@]}" "${#EXCLUDED[@]}" "${#ENUM[@]}"
@@ -1,166 +0,0 @@
#!/usr/bin/env bash
# test-check-test-enumeration.sh — needles for the enumeration guard (#1017).
#
# Every failure mode the guard promises gets BOTH polarities:
# NEEDLE a fixture that MUST trip the guard, asserted on the guard's OWN
# words (--out) — exit 1 alone cannot distinguish "caught the rogue
# file" from "choked on the fixture".
# CONTROL a fixture that MUST pass. A guard that failed unconditionally
# would satisfy every needle here — the null-case defect the guard's
# own subject matter (#1017) exists to make impossible.
#
# The needles encode the specific errors that produced #1017's thread:
# n6 is the 20124 boundary file (a suite the strict prefix cannot name);
# n2b proves surface 2 is PARSED, not line-ranged (three seats mis-scoped
# hand-written ranges against ci.yml);
# n5/n7 keep the exclusions file honest so it cannot become the next silent cap;
# n8/c4 are F1 (20155): a commented-out ci.yml line is NOT enumeration —
# commenting-out is the most common way a suite actually gets disabled,
# and it must fail loud in one direction without false-staling the other.
set -uo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
GUARD="$HERE/check-test-enumeration.sh"
TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT
PASS=0; FAIL=0
# fixture <name> — a minimal repo root the guard accepts via --root:
# one suite enumerated on S1 (plus a naming-outlier suite, so the S1 parser's
# handling of non-prefix names is always exercised), one on S2, one check-script
# named on S2 that is outside the population, and an empty exclusions file.
fixture() {
local r="$TMP/$1"
mkdir -p "$r/packages/mosaic/framework/tools/git" \
"$r/packages/mosaic/framework/tools/tmux" \
"$r/packages/mosaic/framework/tools/quality/scripts" \
"$r/.woodpecker"
printf '#!/usr/bin/env bash\nexit 0\n' > "$r/packages/mosaic/framework/tools/git/test-a.sh"
printf '#!/usr/bin/env bash\nexit 0\n' > "$r/packages/mosaic/framework/tools/tmux/outlier.test.sh"
printf '#!/usr/bin/env bash\nexit 0\n' > "$r/packages/mosaic/framework/tools/quality/scripts/test-ci.sh"
printf '#!/usr/bin/env bash\nexit 0\n' > "$r/packages/mosaic/framework/tools/quality/scripts/verify-thing.sh"
cat > "$r/packages/mosaic/package.json" <<'JSON'
{"scripts": {"test:framework-shell": "bash framework/tools/git/test-a.sh && bash framework/tools/tmux/outlier.test.sh"}}
JSON
cat > "$r/.woodpecker/ci.yml" <<'YML'
steps:
sanitize:
commands:
- bash packages/mosaic/framework/tools/quality/scripts/verify-thing.sh
guard:
commands:
- bash packages/mosaic/framework/tools/quality/scripts/test-ci.sh
YML
: > "$r/packages/mosaic/framework/tools/quality/test-enumeration-exclusions.txt"
printf '%s' "$r"
}
# expect <kind> <want-exit> <desc> [--out <substring>] -- <root>
expect() {
local kind="$1" want="$2" desc="$3"; shift 3
local need_out=""
while (( $# )); do
case "$1" in
--out) need_out="$2"; shift 2 ;;
--) shift; break ;;
esac
done
local root="$1" got=0 out
out="$(bash "$GUARD" --root "$root" 2>&1)" || got=$?
local why=""
[[ "$got" == "$want" ]] || why="wanted exit $want, got $got"
if [[ -z "$why" && -n "$need_out" && "$out" != *"$need_out"* ]]; then
why="exit $got as expected, but output never said: $need_out"
fi
if [[ -z "$why" ]]; then
printf ' PASS [%-7s] %s (exit %s)\n' "$kind" "$desc" "$got"
PASS=$(( PASS + 1 ))
else
printf ' FAIL [%-7s] %s — %s\n' "$kind" "$desc" "$why"
printf '%s\n' "$out" | sed 's/^/ | /'
FAIL=$(( FAIL + 1 ))
fi
}
excl() { printf '%s\n' "$2" >> "$1/packages/mosaic/framework/tools/quality/test-enumeration-exclusions.txt"; }
echo "=== c1: a fully consistent fixture passes ==="
R="$(fixture c1)"
expect CONTROL 0 "consistent tree: both surfaces enumerated, nothing unlisted" \
--out "enumeration guard: OK" -- "$R"
echo "=== n1: an on-disk suite reachable from no surface must fail ==="
R="$(fixture n1)"
printf '#!/usr/bin/env bash\nexit 0\n' > "$R/packages/mosaic/framework/tools/git/test-rogue.sh"
expect NEEDLE 1 "unlisted suite is named in the failure" \
--out "UNENUMERATED: 'packages/mosaic/framework/tools/git/test-rogue.sh'" -- "$R"
echo "=== n6: the 20124 boundary file — a suite the strict prefix cannot name ==="
R="$(fixture n6)"
printf '#!/usr/bin/env bash\nexit 0\n' > "$R/packages/mosaic/framework/tools/git/rogue.test.sh"
expect NEEDLE 1 "naming-outlier suite (*.test.sh) is a population member, not invisible" \
--out "UNENUMERATED: 'packages/mosaic/framework/tools/git/rogue.test.sh'" -- "$R"
echo "=== c3: a non-suite script outside the pattern is outside it on BOTH sides ==="
R="$(fixture c3)"
printf '#!/usr/bin/env bash\nexit 0\n' > "$R/packages/mosaic/framework/tools/git/check-unrelated.sh"
expect CONTROL 0 "check-script on disk, unlisted, outside population: not the guard's business" \
--out "enumeration guard: OK" -- "$R"
echo "=== n2/n2b: a surface naming a path absent from disk must fail — both surfaces ==="
R="$(fixture n2)"
rm "$R/packages/mosaic/framework/tools/git/test-a.sh"
expect NEEDLE 1 "S1 (package.json) stale entry" \
--out "STALE ENUMERATION: surfaces name 'packages/mosaic/framework/tools/git/test-a.sh'" -- "$R"
R="$(fixture n2b)"
rm "$R/packages/mosaic/framework/tools/quality/scripts/test-ci.sh"
expect NEEDLE 1 "S2 (ci.yml) stale entry — proves ci.yml is parsed, not line-ranged" \
--out "STALE ENUMERATION: surfaces name 'packages/mosaic/framework/tools/quality/scripts/test-ci.sh'" -- "$R"
echo "=== c2: a rogue suite with a SIGNED exclusion passes, and is counted ==="
R="$(fixture c2)"
printf '#!/usr/bin/env bash\nexit 0\n' > "$R/packages/mosaic/framework/tools/git/test-rogue.sh"
excl "$R" "packages/mosaic/framework/tools/git/test-rogue.sh | non-hermetic pending fixture work (needle-suite specimen)"
expect CONTROL 0 "signed exclusion is honoured and visible in the summary" \
--out "excluded (signed) 1" -- "$R"
echo "=== n3: an exclusion with no reason is not a decision ==="
R="$(fixture n3)"
printf '#!/usr/bin/env bash\nexit 0\n' > "$R/packages/mosaic/framework/tools/git/test-rogue.sh"
excl "$R" "packages/mosaic/framework/tools/git/test-rogue.sh | "
expect NEEDLE 1 "empty reason rejected" --out "EXCLUSION MISSING REASON" -- "$R"
R="$(fixture n3b)"
printf '#!/usr/bin/env bash\nexit 0\n' > "$R/packages/mosaic/framework/tools/git/test-rogue.sh"
excl "$R" "packages/mosaic/framework/tools/git/test-rogue.sh"
expect NEEDLE 1 "missing separator rejected (the path alone is not a signature)" \
--out "EXCLUSION MISSING REASON" -- "$R"
echo "=== n4: an exclusion whose path is gone is stale, not satisfied ==="
R="$(fixture n4)"
excl "$R" "packages/mosaic/framework/tools/git/test-vanished.sh | was excluded once, then deleted"
expect NEEDLE 1 "stale exclusion rejected" --out "STALE EXCLUSION" -- "$R"
echo "=== n5: excluding an enumerated suite is a contradiction, not belt-and-braces ==="
R="$(fixture n5)"
excl "$R" "packages/mosaic/framework/tools/git/test-a.sh | already in CI but excluded anyway"
expect NEEDLE 1 "contradictory exclusion rejected" --out "CONTRADICTORY EXCLUSION" -- "$R"
echo "=== n8/c4: a commented-out ci.yml line is not enumeration (F1, 20155) ==="
R="$(fixture n8)"
printf '#!/usr/bin/env bash\nexit 0\n' > "$R/packages/mosaic/framework/tools/git/test-disabled.sh"
printf ' # - bash packages/mosaic/framework/tools/git/test-disabled.sh\n' >> "$R/.woodpecker/ci.yml"
expect NEEDLE 1 "suite named only in a commented-out invocation is UNENUMERATED" \
--out "UNENUMERATED: 'packages/mosaic/framework/tools/git/test-disabled.sh'" -- "$R"
R="$(fixture c4)"
printf ' # - bash packages/mosaic/framework/tools/git/test-vanished.sh\n' >> "$R/.woodpecker/ci.yml"
expect CONTROL 0 "comment naming an absent path raises no false stale-enumeration" \
--out "enumeration guard: OK" -- "$R"
echo "=== n7: excluding a file outside the population is dead weight, not coverage ==="
R="$(fixture n7)"
excl "$R" "packages/mosaic/framework/tools/quality/scripts/verify-thing.sh | not a suite but signing it anyway"
expect NEEDLE 1 "out-of-population exclusion rejected" --out "EXCLUSION OUTSIDE POPULATION" -- "$R"
echo
printf 'enumeration-guard needles: %d passed, %d failed\n' "$PASS" "$FAIL"
(( FAIL == 0 ))
@@ -1,45 +0,0 @@
# test-enumeration-exclusions.txt — signed exclusions for check-test-enumeration.sh (#1017).
#
# Every entry is a recorded decision: a suite-shaped file that exists on disk,
# is NOT reachable from any CI surface, and carries the reason someone signed
# for that. The guard FAILS on an entry with no reason, a stale path, or a path
# the surfaces already enumerate. Burning an entry down = making it CI-reachable
# (package.json test:framework-shell or a ci.yml step) and deleting its line.
#
# Format: <repo-relative path> | <reason>
# All entries below were signed at #1017's filing base (main 826a8b3b, 2026-07-31)
# by pepper (sb-it-1-dt); measurements cited are one-run assertions from that seat.
# --- tools/git: the #1007 five — non-hermetic, resolve real credentials ---
packages/mosaic/framework/tools/git/test-pr-merge-gitea-empty-uid.sh | resolves real credentials (#1007 census); joins CI after the wrapper-half hermeticity fix (git -C scoping)
packages/mosaic/framework/tools/git/test-issue-create-interactive-auth.sh | resolves real credentials (#1007 census); joins CI after the wrapper-half hermeticity fix
packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh | resolves real credentials (#1007 census); joins CI after the wrapper-half hermeticity fix
packages/mosaic/framework/tools/git/test-pr-metadata-gitea.sh | resolves real credentials (#1007 census, fourth entry via family-grep); joins CI after the wrapper-half hermeticity fix
packages/mosaic/framework/tools/git/test-issue-comment-readback.sh | resolves real credentials (#1007 census, fifth entry); joins CI after the wrapper-half hermeticity fix
# --- tools/git: push guards — measured green locally, CI-image fitness unverified ---
packages/mosaic/framework/tools/git/test-push-guard.sh | measured green at 826a8b3b (46 passed / 0 failed, one run, 2026-07-31); CI-image fitness unverified; #1017 burndown
packages/mosaic/framework/tools/git/test-mutate-push-guard.sh | measured green at 826a8b3b (8/0, 13 mutants killed 0 survived, one run, 2026-07-31); requires setsid (util-linux), absent from the alpine base image; #1017 burndown
packages/mosaic/framework/tools/git/test-issue-create-body-safety.sh | hermeticity unaudited — the unprotected suite in #1007's protected/unprotected split; audit before CI; #1017 burndown
# --- tools/git: unmeasured ---
packages/mosaic/framework/tools/git/test-verify-clean-clone.sh | unmeasured in CI image; asserts git file-mode (100644/755) semantics that need verification on the CI filesystem first; #1017 burndown
packages/mosaic/framework/tools/git/test-help-exit-code.sh | unmeasured in CI image; stub-based (#701 regression harness), likely CI-fit; #1017 burndown
packages/mosaic/framework/tools/git/test-lane-brief-pr-linkage.sh | unmeasured in CI image; fixture-based (#546/#547 regression harness), likely CI-fit; #1017 burndown
# --- tools/tmux: require a live tmux server ---
packages/mosaic/framework/tools/tmux/test-send-message-socket.sh | requires a real tmux server on a throwaway socket; CI image ships no tmux; #1017 burndown (needs tmux in image or a signed permanent exclusion)
packages/mosaic/framework/tools/tmux/test-send-message-verdict.sh | requires real tmux-pane fixtures on a throwaway socket; CI image ships no tmux; #1017 burndown (same condition as its sibling)
# --- single-suite directories: unmeasured in CI ---
packages/mosaic/framework/tools/fleet/test-start-agent-session.sh | unmeasured in CI image; stubs tmux via a fake bin dir, likely CI-fit; #1017 burndown
packages/mosaic/framework/tools/glpi/test-list-http-status.sh | unmeasured in CI image; stub-based (#807 regression harness), likely CI-fit; #1017 burndown
packages/mosaic/framework/tools/orchestrator/test-board-roll.sh | unmeasured in CI image; file-fixture based, likely CI-fit; #1017 burndown
packages/mosaic/framework/tools/woodpecker/test-ci-wait-exit-matrix.sh | unmeasured in CI image; drives ci-wait.sh against a stub pipeline-status.sh, likely CI-fit; #1017 burndown
# --- naming-boundary files the strict test-*.sh prefix cannot even name ---
# (#1017: three independent censuses handled the microtest file three different
# ways — editorial drop, structural exclusion, accidental inclusion — with no
# recorded judgement. These lines ARE that judgement, signed.)
packages/mosaic/framework/tools/orchestrator/smoke-test.sh | behavior smoke checks for coord continue/run workflows, run manually by orchestrator seats; unmeasured in CI; #1017 burndown
packages/mosaic/framework/tools/wake/validate-973/microtest-wake-assert.sh | #973 instrument self-test, run as a precondition of the validate-973 evidence procedure rather than as a standing CI suite; #1017 burndown candidate
@@ -529,19 +529,7 @@ cmd_run() {
echo "detector.sh: WARN — off-host liveness beacon emit failed (see beacon.sh); the off-host absence check remains the authoritative dead-man." >&2
fi
[ "$once" -eq 1 ] && break
# Close the detector lock fd in the sleep child; otherwise an orphaned sleep
# keeps the single-instance flock (fd 9, taken at exec 9> above) alive after
# the detector parent dies. The lock is non-blocking (`flock -n`, above), so
# for as long as that sleep survives a replacement instance is REFUSED and
# exits rather than queueing. This particular hold is BOUNDED by one poll
# interval (WAKE_DETECTOR_INTERVAL, default 30s): when the orphaned sleep
# exits its copy of fd 9 closes, ending this bounded sleep-child hold. It
# does NOT follow that the next start succeeds — other inheritors of fd 9
# (the M1 adapter, M2 sink grandchildren) are outside this patch's scope and
# can keep holding the flock. The cost this removes is a restart window in
# which every supervisor retry fails on the sleep child's account.
# `9>&-` closes ONLY the child's copy — the parent's lock is unaffected.
sleep "$interval" 9>&-
sleep "$interval"
done
}
+1 -1
View File
@@ -25,7 +25,7 @@
"lint": "eslint src",
"typecheck": "tsc --noEmit",
"test": "vitest run --passWithNoTests && pnpm run test:framework-shell",
"test:framework-shell": "bash framework/tools/quality/scripts/check-test-enumeration.sh && bash framework/tools/quality/scripts/test-check-test-enumeration.sh && python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-ci-queue-wait-parse.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-preimage.sh && bash framework/tools/wake/test-wake-install.sh"
"test:framework-shell": "python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-preimage.sh && bash framework/tools/wake/test-wake-install.sh"
},
"dependencies": {
"@mosaicstack/brain": "workspace:*",