feat(quality): CI test-membership guard — the enumeration can no longer silently under-run the disk (#1017)
ci/woodpecker/pr/ci Pipeline was successful

CI reaches shell suites through two hand-enumerated surfaces (package.json
test:framework-shell; direct ci.yml commands), and a hand-enumerated allowlist
re-arms its own gap: a new suite never auto-joins, which is how 17 of 39
suite-shaped files under framework/tools were invisible to every CI run.

check-test-enumeration.sh, now the first link of test:framework-shell:
- parses BOTH surfaces (JSON + chain tokenization; ci.yml token extraction) —
  never hand-written line ranges, which three seats independently mis-scoped;
- checks BOTH directions: on-disk-but-unreachable fails, surface-named-but-
  absent fails (a rename manufactures a stale entry silently);
- population pattern is basename *test*.sh, deliberately broad and applied
  IDENTICALLY to both sides: the strict test-*.sh prefix cannot even name
  three real boundary files, one of which three independent censuses handled
  three different ways with no recorded judgement;
- the exclusions file gets the same rigor: no reason, stale path, contradictory
  entry, or out-of-population entry each FAIL. An exclusion is a recorded
  decision someone signed, not an omission nobody made.

The 17 previously-invisible suites enter as signed exclusions with honest
per-entry reasons (the #1007 five cite credential non-hermeticity; the push
guards cite one-run-green local measurements plus an unverified setsid
dependency; the unmeasured say unmeasured) — converting the silent gap into
attributable entries without trading it for a red seam. Burndown = make a
suite CI-reachable, delete its line; the guard enforces the bookkeeping.

Needle suite: 12 passed / 0 failed, both polarities per failure mode, asserted
on the guard's own words. Real-tree run at 826a8b3b: population 44 =
25 enumerated + 19 excluded, zero unsigned, all 38 surface-named paths present.

Written-by: pepper (sb-it-1-dt)
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01NsKce8iZuSuRnu3gVMCBKB
This commit is contained in:
Jason Woltje
2026-07-31 08:07:02 -05:00
co-authored by Claude Fable 5
parent 826a8b3b26
commit f7dd57bc10
4 changed files with 354 additions and 1 deletions
+1 -1
View File
@@ -25,7 +25,7 @@
"lint": "eslint src",
"typecheck": "tsc --noEmit",
"test": "vitest run --passWithNoTests && pnpm run test:framework-shell",
"test:framework-shell": "python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-preimage.sh && bash framework/tools/wake/test-wake-install.sh"
"test:framework-shell": "bash framework/tools/quality/scripts/check-test-enumeration.sh && bash framework/tools/quality/scripts/test-check-test-enumeration.sh && python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-preimage.sh && bash framework/tools/wake/test-wake-install.sh"
},
"dependencies": {
"@mosaicstack/brain": "workspace:*",