quality: scan the framework, not just tools/, in the enumeration guard (#1017)
ci/woodpecker/pr/ci Pipeline failed

The guard's population is 'basename matches *test*.sh'. That is not a tools/
property, but Direction A scanned only framework/tools/, so a suite in a SIBLING
of tools/ was structurally invisible to the guard whose stated purpose is making
that impossible.

The live specimen, found by @scooby on origin/next:
framework/systemd/user/test-fleet-units.sh — a population member by the guard's
own definition, named on neither package.json nor ci.yml, and in no exclusion.
It is the only such file today. Same shape as #1017's original 17-of-39,
reintroduced for one subtree.

Three symmetric changes. The scan root moves to the framework. The S2 ci.yml
regex widens with it — not cosmetic: S1 is already general, S2 was the only
tools-scoped surface, so without it 'enumerate the file' would not be a
reachable disposition and an exclusion would be the only way to green. And the
directory-exists precondition follows the root it guards.

Disposition for the specimen: enumerated, not excluded. Its tmux block
self-skips on `command -v tmux && cc`; the rest — the unit-file assertions and
`systemd-analyze verify --user` — is real structural coverage that runs in CI
today. Measured in a CI shape (/usr/bin minus tmux, cc, systemd-analyze): rc=0.
That is strictly better than the two send-message siblings at exclusion lines
31-32, which have no CI-valuable tmux-free half.

Needles n9/c5 added, since a widening with no needle is the same silence one
layer up. n9 fails against the original guard. c5's scope is narrower than it
looks and the comment records the measurement: it passes vacuously on the
original and discriminates against the half-patch (scan wide, S2 narrow), which
is the realistic future regression.

    guard, real tree:  OK — population 53 (was 52), enumerated 38, excluded 15
    needles:           16 passed, 0 failed
This commit is contained in:
2026-08-16 15:07:41 -05:00
parent 476db12b92
commit 6497f79966
3 changed files with 39 additions and 5 deletions
@@ -161,6 +161,34 @@ R="$(fixture n7)"
excl "$R" "packages/mosaic/framework/tools/quality/scripts/verify-thing.sh | not a suite but signing it anyway"
expect NEEDLE 1 "out-of-population exclusion rejected" --out "EXCLUSION OUTSIDE POPULATION" -- "$R"
echo "=== n9/c5: a suite in a SIBLING of tools/ is in the population (@scooby, 2026-08-16) ==="
# Every other fixture here lives under framework/tools/, which is how the guard came to
# scan TOOLS_DIR while claiming a population defined by basename alone. The real specimen
# was framework/systemd/user/test-fleet-units.sh: a member by the guard's own definition,
# on no CI surface, in no exclusion, and structurally unreachable by the scan. n9 is that
# blind spot; without it a future narrowing back to TOOLS_DIR passes all fourteen needles.
R="$(fixture n9)"
mkdir -p "$R/packages/mosaic/framework/systemd/user"
printf '#!/usr/bin/env bash\nexit 0\n' > "$R/packages/mosaic/framework/systemd/user/test-sibling.sh"
expect NEEDLE 1 "suite outside tools/ but inside framework/ is enumerable, not invisible" \
--out "UNENUMERATED: 'packages/mosaic/framework/systemd/user/test-sibling.sh'" -- "$R"
# c5 is why the S2 regex had to widen WITH the scan: detecting the file is useless if the
# fix for it cannot be recognised. Enumerating a sibling-directory suite on ci.yml must
# clear the finding — under a tools/-scoped S2 it stays UNENUMERATED forever and the only
# reachable disposition is an exclusion.
#
# Measured scope of what c5 catches, because it is narrower than it looks: against the
# ORIGINAL guard (both hunks absent) c5 passes vacuously — the scan never sees the file
# and S2 never matches it, so nothing is asserted. It discriminates against the HALF-patch
# — scan widened, S2 narrowed back — which is the realistic future regression, and it was
# confirmed red in exactly that state. n9 is the one that fails on the original.
R="$(fixture c5)"
mkdir -p "$R/packages/mosaic/framework/systemd/user"
printf '#!/usr/bin/env bash\nexit 0\n' > "$R/packages/mosaic/framework/systemd/user/test-sibling.sh"
printf ' - bash packages/mosaic/framework/systemd/user/test-sibling.sh\n' >> "$R/.woodpecker/ci.yml"
expect CONTROL 0 "enumerating a sibling-directory suite on ci.yml actually clears it" \
--out "enumeration guard: OK" -- "$R"
echo
printf 'enumeration-guard needles: %d passed, %d failed\n' "$PASS" "$FAIL"
(( FAIL == 0 ))