quality: scan the framework, not just tools/, in the enumeration guard (#1017)
ci/woodpecker/pr/ci Pipeline failed
ci/woodpecker/pr/ci Pipeline failed
The guard's population is 'basename matches *test*.sh'. That is not a tools/ property, but Direction A scanned only framework/tools/, so a suite in a SIBLING of tools/ was structurally invisible to the guard whose stated purpose is making that impossible. The live specimen, found by @scooby on origin/next: framework/systemd/user/test-fleet-units.sh — a population member by the guard's own definition, named on neither package.json nor ci.yml, and in no exclusion. It is the only such file today. Same shape as #1017's original 17-of-39, reintroduced for one subtree. Three symmetric changes. The scan root moves to the framework. The S2 ci.yml regex widens with it — not cosmetic: S1 is already general, S2 was the only tools-scoped surface, so without it 'enumerate the file' would not be a reachable disposition and an exclusion would be the only way to green. And the directory-exists precondition follows the root it guards. Disposition for the specimen: enumerated, not excluded. Its tmux block self-skips on `command -v tmux && cc`; the rest — the unit-file assertions and `systemd-analyze verify --user` — is real structural coverage that runs in CI today. Measured in a CI shape (/usr/bin minus tmux, cc, systemd-analyze): rc=0. That is strictly better than the two send-message siblings at exclusion lines 31-32, which have no CI-valuable tmux-free half. Needles n9/c5 added, since a widening with no needle is the same silence one layer up. n9 fails against the original guard. c5's scope is narrower than it looks and the comment records the measurement: it passes vacuously on the original and discriminates against the half-patch (scan wide, S2 narrow), which is the realistic future regression. guard, real tree: OK — population 53 (was 52), enumerated 38, excluded 15 needles: 16 passed, 0 failed
This commit is contained in:
@@ -57,12 +57,18 @@ done
|
||||
PKG_JSON="$ROOT/packages/mosaic/package.json"
|
||||
CI_YML="$ROOT/.woodpecker/ci.yml"
|
||||
TOOLS_DIR="$ROOT/packages/mosaic/framework/tools"
|
||||
# The population is "basename matches *test*.sh", which is not a tools/ property.
|
||||
# Direction A used to scan TOOLS_DIR, so a suite in a SIBLING of tools/ was invisible
|
||||
# to the guard whose whole purpose is making that impossible — measured on origin/next
|
||||
# as systemd/user/test-fleet-units.sh, on no CI surface and in no exclusion (@scooby).
|
||||
# Scan the framework, so the scanned surface matches the claimed property.
|
||||
FRAMEWORK_DIR="$ROOT/packages/mosaic/framework"
|
||||
EXCLUSIONS="$TOOLS_DIR/quality/test-enumeration-exclusions.txt"
|
||||
|
||||
for f in "$PKG_JSON" "$CI_YML"; do
|
||||
[[ -f "$f" ]] || { echo "FAIL: required surface file missing: $f" >&2; exit 2; }
|
||||
done
|
||||
[[ -d "$TOOLS_DIR" ]] || { echo "FAIL: tools dir missing: $TOOLS_DIR" >&2; exit 2; }
|
||||
[[ -d "$FRAMEWORK_DIR" ]] || { echo "FAIL: framework dir missing: $FRAMEWORK_DIR" >&2; exit 2; }
|
||||
|
||||
fail_count=0
|
||||
fail() { printf 'FAIL %s\n' "$1"; fail_count=$(( fail_count + 1 )); }
|
||||
@@ -90,7 +96,7 @@ print("\n".join(seen))
|
||||
PY
|
||||
)
|
||||
|
||||
# --- Surface 2: ci.yml, every framework/tools token wherever it appears ------
|
||||
# --- Surface 2: ci.yml, every framework token wherever it appears ------------
|
||||
# Comment lines (first non-whitespace char is #) are skipped BEFORE matching:
|
||||
# commenting an invocation out is the most common way a suite actually gets
|
||||
# disabled, and a raw-text regex would keep calling it enumerated (F1, 20155 on
|
||||
@@ -98,7 +104,7 @@ PY
|
||||
# in a TRAILING comment on a live line still matches; no such line exists today
|
||||
# and full fidelity would need a YAML parser the CI image does not ship.
|
||||
mapfile -t S2 < <(grep -vE '^[[:space:]]*#' "$CI_YML" \
|
||||
| grep -oE 'packages/mosaic/framework/tools/[A-Za-z0-9_./-]+\.(sh|py)' | sort -u)
|
||||
| grep -oE 'packages/mosaic/framework/[A-Za-z0-9_./-]+\.(sh|py)' | sort -u)
|
||||
|
||||
# --- Union, and its population-restricted view -------------------------------
|
||||
declare -A ENUM=() ENUM_POP=()
|
||||
@@ -154,7 +160,7 @@ while IFS= read -r f; do
|
||||
fail "UNENUMERATED: '$rel' exists on disk but is neither enumerated on any CI surface nor signed in the exclusions file"
|
||||
unlisted=$(( unlisted + 1 ))
|
||||
fi
|
||||
done < <(find "$TOOLS_DIR" -type f -name '*.sh' | sort)
|
||||
done < <(find "$FRAMEWORK_DIR" -type f -name '*.sh' | sort)
|
||||
|
||||
if (( fail_count > 0 )); then
|
||||
printf 'enumeration guard: %d failure(s) — population %d, enumerated (in-population) %d, excluded %d\n' \
|
||||
|
||||
@@ -161,6 +161,34 @@ R="$(fixture n7)"
|
||||
excl "$R" "packages/mosaic/framework/tools/quality/scripts/verify-thing.sh | not a suite but signing it anyway"
|
||||
expect NEEDLE 1 "out-of-population exclusion rejected" --out "EXCLUSION OUTSIDE POPULATION" -- "$R"
|
||||
|
||||
echo "=== n9/c5: a suite in a SIBLING of tools/ is in the population (@scooby, 2026-08-16) ==="
|
||||
# Every other fixture here lives under framework/tools/, which is how the guard came to
|
||||
# scan TOOLS_DIR while claiming a population defined by basename alone. The real specimen
|
||||
# was framework/systemd/user/test-fleet-units.sh: a member by the guard's own definition,
|
||||
# on no CI surface, in no exclusion, and structurally unreachable by the scan. n9 is that
|
||||
# blind spot; without it a future narrowing back to TOOLS_DIR passes all fourteen needles.
|
||||
R="$(fixture n9)"
|
||||
mkdir -p "$R/packages/mosaic/framework/systemd/user"
|
||||
printf '#!/usr/bin/env bash\nexit 0\n' > "$R/packages/mosaic/framework/systemd/user/test-sibling.sh"
|
||||
expect NEEDLE 1 "suite outside tools/ but inside framework/ is enumerable, not invisible" \
|
||||
--out "UNENUMERATED: 'packages/mosaic/framework/systemd/user/test-sibling.sh'" -- "$R"
|
||||
# c5 is why the S2 regex had to widen WITH the scan: detecting the file is useless if the
|
||||
# fix for it cannot be recognised. Enumerating a sibling-directory suite on ci.yml must
|
||||
# clear the finding — under a tools/-scoped S2 it stays UNENUMERATED forever and the only
|
||||
# reachable disposition is an exclusion.
|
||||
#
|
||||
# Measured scope of what c5 catches, because it is narrower than it looks: against the
|
||||
# ORIGINAL guard (both hunks absent) c5 passes vacuously — the scan never sees the file
|
||||
# and S2 never matches it, so nothing is asserted. It discriminates against the HALF-patch
|
||||
# — scan widened, S2 narrowed back — which is the realistic future regression, and it was
|
||||
# confirmed red in exactly that state. n9 is the one that fails on the original.
|
||||
R="$(fixture c5)"
|
||||
mkdir -p "$R/packages/mosaic/framework/systemd/user"
|
||||
printf '#!/usr/bin/env bash\nexit 0\n' > "$R/packages/mosaic/framework/systemd/user/test-sibling.sh"
|
||||
printf ' - bash packages/mosaic/framework/systemd/user/test-sibling.sh\n' >> "$R/.woodpecker/ci.yml"
|
||||
expect CONTROL 0 "enumerating a sibling-directory suite on ci.yml actually clears it" \
|
||||
--out "enumeration guard: OK" -- "$R"
|
||||
|
||||
echo
|
||||
printf 'enumeration-guard needles: %d passed, %d failed\n' "$PASS" "$FAIL"
|
||||
(( FAIL == 0 ))
|
||||
|
||||
Reference in New Issue
Block a user