fix(quality): enumeration guard sees naming, not comments — F1/F2/F3 from 20155

F1 (demonstrated defect): surface 2 was a regex over raw YAML text, so a
commented-out ci.yml invocation still counted as enumerated — #1017's own
failure mode reproduced inside the fix, in the most common disable path
there is. Comment lines are now skipped before matching; needles n8/c4
pin both polarities (a comment is not enumeration; a comment naming an
absent path raises no false stale-enumeration). The residual trailing-
comment limit is documented where the grep lives.

F2 (structural): the guard was link [0] of the six-hop pnpm chain it
guards, so severing that chain silenced guard and guarded together.
ci.yml now invokes it directly from the sanitization step too — reachable
from both surfaces it audits, so either path alone keeps one instrument
running.

F3 (vocabulary): failure line and header said 'reachable'; F1/F2 prove
the guard measures NAMING. It now says 'enumerated' everywhere, matching
the success line.

Needles: 14 passed / 0 failed. Real tree: population 44, enumerated 25,
excluded (signed) 19, surfaces 38, unchanged — F2's line adds redundancy,
not membership. F1 demo replayed against the fix: commenting out
test-install-migration.sh now fails loud (UNENUMERATED, rc=1).

Written-by: pepper (sb-it-1-dt)
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01NsKce8iZuSuRnu3gVMCBKB

.woodpecker/ci.yml

@@ -41,6 +41,11 @@ steps:
       # (Constitution + dispatcher + each RUNTIME.md slice). See DESIGN §7 / R9.
       - bash packages/mosaic/framework/tools/quality/scripts/check-resident-budget.sh --self-test
       - bash packages/mosaic/framework/tools/quality/scripts/check-resident-budget.sh
+      # Test-membership guard (#1017): also first link of test:framework-shell.
+      # Invoked from BOTH surfaces it audits (F2, PR #1018) — the guard is link
+      # [0] of the pnpm chain, so severing that chain would silence it together
+      # with everything it guards; this direct line keeps one instrument running.
+      - bash packages/mosaic/framework/tools/quality/scripts/check-test-enumeration.sh
 
   # Blocking gate (#791): a framework upgrade must never write or delete an
   # operator-owned path. The HARD GATE proves an unanticipated operator sentinel

packages/mosaic/framework/tools/quality/scripts/check-test-enumeration.sh

@@ -9,8 +9,10 @@
 # so the list silently under-runs the disk (17 of 39 suites were invisible when
 # #1017 was filed). This guard makes that under-run impossible to do silently:
 #
-#   FAIL when a suite-shaped file exists on disk and is neither reachable from
+#   FAIL when a suite-shaped file exists on disk and is neither enumerated on
 #        the UNION of both surfaces nor listed in the exclusions file.
+#        ("Enumerated", deliberately — F1/F2 on PR #1018 proved this guard sees
+#        NAMING, not reachability, and its words must not claim otherwise.)
 #   FAIL when either surface names a path that does not exist on disk
 #        (a rename manufactures a stale entry silently — checked BOTH directions).
 #   FAIL when an exclusion entry has no reason, names a path that is gone,
@@ -89,7 +91,14 @@ PY
 )
 
 # --- Surface 2: ci.yml, every framework/tools token wherever it appears ------
-mapfile -t S2 < <(grep -oE 'packages/mosaic/framework/tools/[A-Za-z0-9_./-]+\.(sh|py)' "$CI_YML" | sort -u)
+# Comment lines (first non-whitespace char is #) are skipped BEFORE matching:
+# commenting an invocation out is the most common way a suite actually gets
+# disabled, and a raw-text regex would keep calling it enumerated (F1, 20155 on
+# PR #1018 — demonstrated, not argued). Known residual limit: a path named only
+# in a TRAILING comment on a live line still matches; no such line exists today
+# and full fidelity would need a YAML parser the CI image does not ship.
+mapfile -t S2 < <(grep -vE '^[[:space:]]*#' "$CI_YML" \
+    | grep -oE 'packages/mosaic/framework/tools/[A-Za-z0-9_./-]+\.(sh|py)' | sort -u)
 
 # --- Union, and its population-restricted view -------------------------------
 declare -A ENUM=() ENUM_POP=()
@@ -142,7 +151,7 @@ while IFS= read -r f; do
     in_population "$rel" || continue
     disk_total=$(( disk_total + 1 ))
     if [[ -z "${ENUM_POP[$rel]:-}" && -z "${EXCLUDED[$rel]:-}" ]]; then
-        fail "UNENUMERATED: '$rel' exists on disk but is neither reachable from any CI surface nor signed in the exclusions file"
+        fail "UNENUMERATED: '$rel' exists on disk but is neither enumerated on any CI surface nor signed in the exclusions file"
         unlisted=$(( unlisted + 1 ))
     fi
 done < <(find "$TOOLS_DIR" -type f -name '*.sh' | sort)

packages/mosaic/framework/tools/quality/scripts/test-check-test-enumeration.sh

@@ -13,7 +13,10 @@
 #   n6 is the 20124 boundary file (a suite the strict prefix cannot name);
 #   n2b proves surface 2 is PARSED, not line-ranged (three seats mis-scoped
 #       hand-written ranges against ci.yml);
-#   n5/n7 keep the exclusions file honest so it cannot become the next silent cap.
+#   n5/n7 keep the exclusions file honest so it cannot become the next silent cap;
+#   n8/c4 are F1 (20155): a commented-out ci.yml line is NOT enumeration —
+#       commenting-out is the most common way a suite actually gets disabled,
+#       and it must fail loud in one direction without false-staling the other.
 
 set -uo pipefail
 
@@ -142,6 +145,17 @@ R="$(fixture n5)"
 excl "$R" "packages/mosaic/framework/tools/git/test-a.sh | already in CI but excluded anyway"
 expect NEEDLE 1 "contradictory exclusion rejected" --out "CONTRADICTORY EXCLUSION" -- "$R"
 
+echo "=== n8/c4: a commented-out ci.yml line is not enumeration (F1, 20155) ==="
+R="$(fixture n8)"
+printf '#!/usr/bin/env bash\nexit 0\n' > "$R/packages/mosaic/framework/tools/git/test-disabled.sh"
+printf '      # - bash packages/mosaic/framework/tools/git/test-disabled.sh\n' >> "$R/.woodpecker/ci.yml"
+expect NEEDLE 1 "suite named only in a commented-out invocation is UNENUMERATED" \
+    --out "UNENUMERATED: 'packages/mosaic/framework/tools/git/test-disabled.sh'" -- "$R"
+R="$(fixture c4)"
+printf '      # - bash packages/mosaic/framework/tools/git/test-vanished.sh\n' >> "$R/.woodpecker/ci.yml"
+expect CONTROL 0 "comment naming an absent path raises no false stale-enumeration" \
+    --out "enumeration guard: OK" -- "$R"
+
 echo "=== n7: excluding a file outside the population is dead weight, not coverage ==="
 R="$(fixture n7)"
 excl "$R" "packages/mosaic/framework/tools/quality/scripts/verify-thing.sh | not a suite but signing it anyway"