test(#1017): wire in four CI-fit shell suites, drop their signed exclusions #1252

2026-08-16T06:14:34Z

fred commented

2026-08-16 06:14:34 +00:00

Burns down four of the #1017 signed exclusions. Zero production code touched.

What was excluded, and why it no longer needs to be

test-enumeration-exclusions.txt carried four suites under
# --- single-suite directories: unmeasured in CI ---, each self-graded
"unmeasured in CI image; …; likely CI-fit; #1017 burndown":

framework/tools/fleet/test-start-agent-session.sh
framework/tools/glpi/test-list-http-status.sh
framework/tools/orchestrator/test-board-roll.sh
framework/tools/woodpecker/test-ci-wait-exit-matrix.sh

"Likely CI-fit" was a guess written at signing time. I measured it instead.

All four pass standing alone, and — the part that actually matters, since the exclusion
reason is the CI image — all four still pass with tmux removed from PATH entirely:

tmux visible in sandbox PATH? NO
fleet/test-start-agent-session.sh          rc=0  ok - start-agent-session generated environment boundary
glpi/test-list-http-status.sh              rc=0  ALL PASS: test-list-http-status.sh
orchestrator/test-board-roll.sh            rc=0  board-roll regression passed (8 groups)
woodpecker/test-ci-wait-exit-matrix.sh     rc=0  ALL PASS: test-ci-wait-exit-matrix.sh

(Sandbox built by symlinking all of /usr/bin and /bin except tmux.)

test-start-agent-session.sh was the one worth checking hardest, since it is the fleet one and
the CI image ships no tmux. It writes its own tmux shim into a mktemp -d fake bin dir, so it
never depended on the real binary. Its exclusion was precautionary, not necessary.

Red-first

Removing the four exclusion lines makes the guard fail with exactly four failures, naming exactly
those four files:

FAIL  UNENUMERATED: 'packages/mosaic/framework/tools/fleet/test-start-agent-session.sh' …
FAIL  UNENUMERATED: 'packages/mosaic/framework/tools/glpi/test-list-http-status.sh' …
FAIL  UNENUMERATED: 'packages/mosaic/framework/tools/orchestrator/test-board-roll.sh' …
FAIL  UNENUMERATED: 'packages/mosaic/framework/tools/woodpecker/test-ci-wait-exit-matrix.sh' …
enumeration guard: 4 failure(s) — population 51, enumerated (in-population) 32, excluded 15

Appending the four to test:framework-shell returns it to green:

enumeration guard: OK — population 51, enumerated (in-population) 36, excluded (signed) 15,
surfaces name 53 path(s), all present on disk

In-population enumerated 32 → 36; signed exclusions 19 → 15.

Diff

Two files, +2 / −6. The package.json change is the single test:framework-shell line; the
exclusions change is four deleted lines. Nothing else.

What this does not do

This is scooby's task 1 only. It does not touch task 2, which is the harder and more
interesting half:

systemd/user/test-fleet-units.sh sits outside the guard's population entirely.
check-test-enumeration.sh sets TOOLS_DIR="$ROOT/packages/mosaic/framework/tools" (:59),
in_population() (:71) matches only packages/mosaic/framework/tools/….(sh|py), and the scan
at :157 is find "$TOOLS_DIR". So that file appears zero times in the exclusions census and is
named nowhere in ci.yml — the guard that exists to prevent un-run tests cannot see it, and
it is not signed, it is invisible. PR #1244 added +8 lines to it.

Task 2 is therefore population-scope first (widen past tools/, or relocate the file), and only
then the tmux question — at which point it inherits the disposition of
tools/tmux/test-send-message-{socket,verdict}.sh, which genuinely need a real tmux server and
are correctly excluded pending "tmux in image or a signed permanent exclusion".

Credit to @scooby for catching that scope blindness — I had originally reported both fleet
launcher harnesses as signed exclusions, and only one of them is.

Refs #1017

Burns down four of the `#1017` signed exclusions. Zero production code touched. ## What was excluded, and why it no longer needs to be `test-enumeration-exclusions.txt` carried four suites under `# --- single-suite directories: unmeasured in CI ---`, each self-graded *"unmeasured in CI image; …; likely CI-fit; #1017 burndown"*: - `framework/tools/fleet/test-start-agent-session.sh` - `framework/tools/glpi/test-list-http-status.sh` - `framework/tools/orchestrator/test-board-roll.sh` - `framework/tools/woodpecker/test-ci-wait-exit-matrix.sh` "Likely CI-fit" was a guess written at signing time. I measured it instead. **All four pass standing alone**, and — the part that actually matters, since the exclusion reason is the CI *image* — **all four still pass with `tmux` removed from `PATH` entirely**: ``` tmux visible in sandbox PATH? NO fleet/test-start-agent-session.sh rc=0 ok - start-agent-session generated environment boundary glpi/test-list-http-status.sh rc=0 ALL PASS: test-list-http-status.sh orchestrator/test-board-roll.sh rc=0 board-roll regression passed (8 groups) woodpecker/test-ci-wait-exit-matrix.sh rc=0 ALL PASS: test-ci-wait-exit-matrix.sh ``` (Sandbox built by symlinking all of `/usr/bin` and `/bin` except `tmux`.) `test-start-agent-session.sh` was the one worth checking hardest, since it is the fleet one and the CI image ships no tmux. It writes its own tmux shim into a `mktemp -d` fake bin dir, so it never depended on the real binary. Its exclusion was precautionary, not necessary. ## Red-first Removing the four exclusion lines makes the guard fail with exactly four failures, naming exactly those four files: ``` FAIL UNENUMERATED: 'packages/mosaic/framework/tools/fleet/test-start-agent-session.sh' … FAIL UNENUMERATED: 'packages/mosaic/framework/tools/glpi/test-list-http-status.sh' … FAIL UNENUMERATED: 'packages/mosaic/framework/tools/orchestrator/test-board-roll.sh' … FAIL UNENUMERATED: 'packages/mosaic/framework/tools/woodpecker/test-ci-wait-exit-matrix.sh' … enumeration guard: 4 failure(s) — population 51, enumerated (in-population) 32, excluded 15 ``` Appending the four to `test:framework-shell` returns it to green: ``` enumeration guard: OK — population 51, enumerated (in-population) 36, excluded (signed) 15, surfaces name 53 path(s), all present on disk ``` In-population enumerated **32 → 36**; signed exclusions **19 → 15**. ## Diff Two files, +2 / −6. The `package.json` change is the single `test:framework-shell` line; the exclusions change is four deleted lines. Nothing else. ## What this does not do This is scooby's **task 1** only. It does not touch **task 2**, which is the harder and more interesting half: `systemd/user/test-fleet-units.sh` sits **outside** the guard's population entirely. `check-test-enumeration.sh` sets `TOOLS_DIR="$ROOT/packages/mosaic/framework/tools"` (:59), `in_population()` (:71) matches only `packages/mosaic/framework/tools/….(sh|py)`, and the scan at :157 is `find "$TOOLS_DIR"`. So that file appears zero times in the exclusions census and is named nowhere in `ci.yml` — the guard that exists to prevent un-run tests **cannot see it**, and it is not signed, it is invisible. PR #1244 added +8 lines to it. Task 2 is therefore population-scope first (widen past `tools/`, or relocate the file), and only then the tmux question — at which point it inherits the disposition of `tools/tmux/test-send-message-{socket,verdict}.sh`, which genuinely need a real tmux server and are correctly excluded pending *"tmux in image or a signed permanent exclusion"*. Credit to @scooby for catching that scope blindness — I had originally reported both fleet launcher harnesses as signed exclusions, and only one of them is. Refs #1017

fred added 1 commit 2026-08-16 06:14:35 +00:00

test(#1017 ): wire in four CI-fit shell suites, drop their signed exclusions

ci/woodpecker/pr/ci Pipeline failed

Details

c56483eb1b

check-test-enumeration.sh signed four suites as 'likely CI-fit; #1017 burndown'.
Measured all four: each passes standing alone, and each still passes with tmux
removed from PATH entirely (test-start-agent-session.sh writes its own tmux shim
into a fake bin dir, so it never needed the real binary).

Red-first: removing the four exclusion lines makes the guard report exactly four
UNENUMERATED failures. Appending the four to test:framework-shell returns it to
OK, with in-population enumerated going 32 -> 36 and signed exclusions 19 -> 15.

Refs #1017

fred merged commit 19ac0a02d7 into next

2026-08-16 18:06:54 +00:00

fred referenced this issue from a commit

2026-08-16 18:11:34 +00:00

merge next into fix/1240-fleet-transport-check

fred referenced this issue from a commit

2026-08-16 18:12:36 +00:00

Merge pull request 'fix(fleet): tell the operator when the fleet transport is missing (#1240)' (#1245) from fix/1240-fleet-transport-check into next

~~fred referenced this pull request 2026-08-16 18:13:24 +00:00~~

[INVALID] pr-merge.sh dry-run identity — measured a stale local framework copy, already fixed on next #1253

Sign in to join this conversation.