The guard's population is 'basename matches *test*.sh'. That is not a tools/
property, but Direction A scanned only framework/tools/, so a suite in a SIBLING
of tools/ was structurally invisible to the guard whose stated purpose is making
that impossible.
The live specimen, found by @scooby on origin/next:
framework/systemd/user/test-fleet-units.sh — a population member by the guard's
own definition, named on neither package.json nor ci.yml, and in no exclusion.
It is the only such file today. Same shape as #1017's original 17-of-39,
reintroduced for one subtree.
Three symmetric changes. The scan root moves to the framework. The S2 ci.yml
regex widens with it — not cosmetic: S1 is already general, S2 was the only
tools-scoped surface, so without it 'enumerate the file' would not be a
reachable disposition and an exclusion would be the only way to green. And the
directory-exists precondition follows the root it guards.
Disposition for the specimen: enumerated, not excluded. Its tmux block
self-skips on `command -v tmux && cc`; the rest — the unit-file assertions and
`systemd-analyze verify --user` — is real structural coverage that runs in CI
today. Measured in a CI shape (/usr/bin minus tmux, cc, systemd-analyze): rc=0.
That is strictly better than the two send-message siblings at exclusion lines
31-32, which have no CI-valuable tmux-free half.
Needles n9/c5 added, since a widening with no needle is the same silence one
layer up. n9 fails against the original guard. c5's scope is narrower than it
looks and the comment records the measurement: it passes vacuously on the
original and discriminates against the half-patch (scan wide, S2 narrow), which
is the realistic future regression.
guard, real tree: OK — population 53 (was 52), enumerated 38, excluded 15
needles: 16 passed, 0 failed
The first PR for this change was filed under the retired mos-dt-0 principal
(pr-create.sh has no --login flag and find_tea_login_for_host returns the first
host match) and was closed and refiled as #1270. That left in-tree references
pointing at a closed duplicate PR rather than at the burn-down issue, which is
the wrong target for them anyway: the open design question belongs on #1271.
The `test` step has failed on every `next` pipeline since #1017 on exactly one
assertion, and it is the same one on unrelated PRs:
FAIL: host provides 'pi' in the system path; missing-binary cases are not
measurable here (framework/tools/fleet/test-start-agent-session.sh:103)
Measured 2026-08-16 across pipelines 2444 (#1256), 2438 (#1240) and 2441
(#1017-quality): exactly one FAIL line in each full log, identical, this line.
Control `zzz-not-present-zzz` -> 0 on all three.
Cause. #1241 (5c35a250) added the guard: the suite shims fake mosaic/pi/npm into
$FAKE_BIN, but the constructed PANE_PATH always ends in the real system path, so
on a host that installs those binaries the missing-binary cases cannot be
measured and a green run would mean nothing. The guard says so instead of
passing. Its own pipeline 2430 was green only because the suite was CI-excluded
at the time, so the guard had never run in CI. #1017 (c56483eb) then enumerated
it and dropped the exclusion. The CI image installs
@earendil-works/[email protected].1 on purpose, so the precondition is
unsatisfiable there. Both commits are mine.
The guard is correct and is not being softened. A check that cannot measure its
property and reports success is the failure mode this repo has been cataloguing
all week; the error was wiring the suite into an image that violates its
precondition, so the wiring is what gets reverted.
Second effect, which is the reason this cost a day rather than an hour:
test:framework-shell is one && chain and this sat at position 44 of 48, so
glpi/test-list-http-status.sh, orchestrator/test-board-roll.sh,
woodpecker/test-ci-wait-exit-matrix.sh and _scripts/test-fleet-transport-check.sh
have not run at all since the merge. The pipeline reported one failure, never
"one failure plus four unrun". All four are green when run directly on
sb-it-1-dt, so the mask hid nothing broken -- but that is a local result on one
host, not a CI-image result.
Verification, with controls:
- enumeration guard OK (population 52, enumerated 36, signed-excluded 16).
- control A, exclusion line removed while unwired -> FAIL UNENUMERATED.
- control B, exclusion line kept while rewired -> FAIL CONTRADICTORY EXCLUSION.
The gate discriminates in both directions, so its OK is load-bearing.
- the four formerly-masked suites: rc=0 each, run directly.
- the full chain cannot be run to completion on sb-it-1-dt: it stops earlier, at
the lease-broker Invariant R test, because this host carries the quarantined
operator-global pi 0.84.2 against a measured 0.84.1. That is host-specific and
out of scope here -- CI pins 0.84.1, and the single FAIL line in those three
pipelines proves positions 1-43 passed there.
Burn-down is to control the tail of PANE_PATH inside the test, not to remove pi
from the image. Recorded in the exclusion reason and in #1269.
check-test-enumeration.sh signed four suites as 'likely CI-fit; #1017 burndown'.
Measured all four: each passes standing alone, and each still passes with tmux
removed from PATH entirely (test-start-agent-session.sh writes its own tmux shim
into a fake bin dir, so it never needed the real binary).
Red-first: removing the four exclusion lines makes the guard report exactly four
UNENUMERATED failures. Appending the four to test:framework-shell returns it to
OK, with in-population enumerated going 32 -> 36 and signed exclusions 19 -> 15.
Refs #1017
fredandops-03
ops-01
fred
jason.woltje