feat(quality): CI test-membership guard — enumeration can no longer silently under-run the disk (#1017) #1018

Merged

Mos merged 2 commits from fix/1017-enumeration-guard into main 2026-07-31 14:08:42 +00:00

Conversation 7 Commits 2 Files Changed 5

+382 -1

2 Commits

Author	SHA1	Message	Date
Jason WoltjeandClaude Fable 5	1cc41f9813	fix(quality): enumeration guard sees naming, not comments — F1/F2/F3 from 20155 ... F1 (demonstrated defect): surface 2 was a regex over raw YAML text, so a commented-out ci.yml invocation still counted as enumerated — #1017's own failure mode reproduced inside the fix, in the most common disable path there is. Comment lines are now skipped before matching; needles n8/c4 pin both polarities (a comment is not enumeration; a comment naming an absent path raises no false stale-enumeration). The residual trailing- comment limit is documented where the grep lives. F2 (structural): the guard was link [0] of the six-hop pnpm chain it guards, so severing that chain silenced guard and guarded together. ci.yml now invokes it directly from the sanitization step too — reachable from both surfaces it audits, so either path alone keeps one instrument running. F3 (vocabulary): failure line and header said 'reachable'; F1/F2 prove the guard measures NAMING. It now says 'enumerated' everywhere, matching the success line. Needles: 14 passed / 0 failed. Real tree: population 44, enumerated 25, excluded (signed) 19, surfaces 38, unchanged — F2's line adds redundancy, not membership. F1 demo replayed against the fix: commenting out test-install-migration.sh now fails loud (UNENUMERATED, rc=1). Written-by: pepper (sb-it-1-dt) Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01NsKce8iZuSuRnu3gVMCBKB	2026-07-31 08:18:56 -05:00
Jason WoltjeandClaude Fable 5	f7dd57bc10	feat(quality): CI test-membership guard — the enumeration can no longer silently under-run the disk (#1017) ... CI reaches shell suites through two hand-enumerated surfaces (package.json test:framework-shell; direct ci.yml commands), and a hand-enumerated allowlist re-arms its own gap: a new suite never auto-joins, which is how 17 of 39 suite-shaped files under framework/tools were invisible to every CI run. check-test-enumeration.sh, now the first link of test:framework-shell: - parses BOTH surfaces (JSON + chain tokenization; ci.yml token extraction) — never hand-written line ranges, which three seats independently mis-scoped; - checks BOTH directions: on-disk-but-unreachable fails, surface-named-but- absent fails (a rename manufactures a stale entry silently); - population pattern is basename *test*.sh, deliberately broad and applied IDENTICALLY to both sides: the strict test-*.sh prefix cannot even name three real boundary files, one of which three independent censuses handled three different ways with no recorded judgement; - the exclusions file gets the same rigor: no reason, stale path, contradictory entry, or out-of-population entry each FAIL. An exclusion is a recorded decision someone signed, not an omission nobody made. The 17 previously-invisible suites enter as signed exclusions with honest per-entry reasons (the #1007 five cite credential non-hermeticity; the push guards cite one-run-green local measurements plus an unverified setsid dependency; the unmeasured say unmeasured) — converting the silent gap into attributable entries without trading it for a red seam. Burndown = make a suite CI-reachable, delete its line; the guard enforces the bookkeeping. Needle suite: 12 passed / 0 failed, both polarities per failure mode, asserted on the guard's own words. Real-tree run at 826a8b3b: population 44 = 25 enumerated + 19 excluded, zero unsigned, all 38 surface-named paths present. Written-by: pepper (sb-it-1-dt) Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01NsKce8iZuSuRnu3gVMCBKB	2026-07-31 08:07:02 -05:00