fred
93c1de51e1
ci/woodpecker/pr/ci Pipeline was canceled
The `test` step has failed on every `next` pipeline since #1017 on exactly one assertion, and it is the same one on unrelated PRs: FAIL: host provides 'pi' in the system path; missing-binary cases are not measurable here (framework/tools/fleet/test-start-agent-session.sh:103) Measured 2026-08-16 across pipelines 2444 (#1256), 2438 (#1240) and 2441 (#1017-quality): exactly one FAIL line in each full log, identical, this line. Control `zzz-not-present-zzz` -> 0 on all three. Cause. #1241 (5c35a250) added the guard: the suite shims fake mosaic/pi/npm into $FAKE_BIN, but the constructed PANE_PATH always ends in the real system path, so on a host that installs those binaries the missing-binary cases cannot be measured and a green run would mean nothing. The guard says so instead of passing. Its own pipeline 2430 was green only because the suite was CI-excluded at the time, so the guard had never run in CI. #1017 (c56483eb) then enumerated it and dropped the exclusion. The CI image installs @earendil-works/[email protected].1 on purpose, so the precondition is unsatisfiable there. Both commits are mine. The guard is correct and is not being softened. A check that cannot measure its property and reports success is the failure mode this repo has been cataloguing all week; the error was wiring the suite into an image that violates its precondition, so the wiring is what gets reverted. Second effect, which is the reason this cost a day rather than an hour: test:framework-shell is one && chain and this sat at position 44 of 48, so glpi/test-list-http-status.sh, orchestrator/test-board-roll.sh, woodpecker/test-ci-wait-exit-matrix.sh and _scripts/test-fleet-transport-check.sh have not run at all since the merge. The pipeline reported one failure, never "one failure plus four unrun". All four are green when run directly on sb-it-1-dt, so the mask hid nothing broken -- but that is a local result on one host, not a CI-image result. Verification, with controls: - enumeration guard OK (population 52, enumerated 36, signed-excluded 16). - control A, exclusion line removed while unwired -> FAIL UNENUMERATED. - control B, exclusion line kept while rewired -> FAIL CONTRADICTORY EXCLUSION. The gate discriminates in both directions, so its OK is load-bearing. - the four formerly-masked suites: rc=0 each, run directly. - the full chain cannot be run to completion on sb-it-1-dt: it stops earlier, at the lease-broker Invariant R test, because this host carries the quarantined operator-global pi 0.84.2 against a measured 0.84.1. That is host-specific and out of scope here -- CI pins 0.84.1, and the single FAIL line in those three pipelines proves positions 1-43 passed there. Burn-down is to control the tail of PANE_PATH inside the test, not to remove pi from the image. Recorded in the exclusion reason and in #1269.
58 lines
5.6 KiB
Plaintext
58 lines
5.6 KiB
Plaintext
# test-enumeration-exclusions.txt — signed exclusions for check-test-enumeration.sh (#1017).
|
|
#
|
|
# Every entry is a recorded decision: a suite-shaped file that exists on disk,
|
|
# is NOT reachable from any CI surface, and carries the reason someone signed
|
|
# for that. The guard FAILS on an entry with no reason, a stale path, or a path
|
|
# the surfaces already enumerate. Burning an entry down = making it CI-reachable
|
|
# (package.json test:framework-shell or a ci.yml step) and deleting its line.
|
|
#
|
|
# Format: <repo-relative path> | <reason>
|
|
# All entries below were signed at #1017's filing base (main 826a8b3b, 2026-07-31)
|
|
# by pepper (sb-it-1-dt); measurements cited are one-run assertions from that seat.
|
|
|
|
# --- tools/git: the #1007 five — non-hermetic, resolve real credentials ---
|
|
packages/mosaic/framework/tools/git/test-pr-merge-gitea-empty-uid.sh | resolves real credentials (#1007 census); joins CI after the wrapper-half hermeticity fix (git -C scoping)
|
|
packages/mosaic/framework/tools/git/test-issue-create-interactive-auth.sh | resolves real credentials (#1007 census); joins CI after the wrapper-half hermeticity fix
|
|
packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh | resolves real credentials (#1007 census); joins CI after the wrapper-half hermeticity fix
|
|
packages/mosaic/framework/tools/git/test-pr-metadata-gitea.sh | resolves real credentials (#1007 census, fourth entry via family-grep); joins CI after the wrapper-half hermeticity fix
|
|
packages/mosaic/framework/tools/git/test-issue-comment-readback.sh | resolves real credentials (#1007 census, fifth entry); joins CI after the wrapper-half hermeticity fix
|
|
|
|
# --- tools/git: push guards — measured green locally, CI-image fitness unverified ---
|
|
packages/mosaic/framework/tools/git/test-push-guard.sh | measured green at 826a8b3b (46 passed / 0 failed, one run, 2026-07-31); CI-image fitness unverified; #1017 burndown
|
|
packages/mosaic/framework/tools/git/test-mutate-push-guard.sh | measured green at 826a8b3b (8/0, 13 mutants killed 0 survived, one run, 2026-07-31); requires setsid (util-linux), absent from the alpine base image; #1017 burndown
|
|
packages/mosaic/framework/tools/git/test-issue-create-body-safety.sh | hermeticity unaudited — the unprotected suite in #1007's protected/unprotected split; audit before CI; #1017 burndown
|
|
|
|
# --- tools/git: unmeasured ---
|
|
packages/mosaic/framework/tools/git/test-verify-clean-clone.sh | unmeasured in CI image; asserts git file-mode (100644/755) semantics that need verification on the CI filesystem first; #1017 burndown
|
|
packages/mosaic/framework/tools/git/test-help-exit-code.sh | unmeasured in CI image; stub-based (#701 regression harness), likely CI-fit; #1017 burndown
|
|
packages/mosaic/framework/tools/git/test-lane-brief-pr-linkage.sh | unmeasured in CI image; fixture-based (#546/#547 regression harness), likely CI-fit; #1017 burndown
|
|
|
|
# --- tools/tmux: require a live tmux server ---
|
|
packages/mosaic/framework/tools/tmux/test-send-message-socket.sh | requires a real tmux server on a throwaway socket; CI image ships no tmux; #1017 burndown (needs tmux in image or a signed permanent exclusion)
|
|
packages/mosaic/framework/tools/tmux/test-send-message-verdict.sh | requires real tmux-pane fixtures on a throwaway socket; CI image ships no tmux; #1017 burndown (same condition as its sibling)
|
|
|
|
# --- single-suite directories: unmeasured in CI ---
|
|
|
|
# --- naming-boundary files the strict test-*.sh prefix cannot even name ---
|
|
# (#1017: three independent censuses handled the microtest file three different
|
|
# ways — editorial drop, structural exclusion, accidental inclusion — with no
|
|
# recorded judgement. These lines ARE that judgement, signed.)
|
|
packages/mosaic/framework/tools/orchestrator/smoke-test.sh | behavior smoke checks for coord continue/run workflows, run manually by orchestrator seats; unmeasured in CI; #1017 burndown
|
|
packages/mosaic/framework/tools/wake/validate-973/microtest-wake-assert.sh | #973 instrument self-test, run as a precondition of the validate-973 evidence procedure rather than as a standing CI suite; #1017 burndown candidate
|
|
|
|
# --- tools/fleet: precondition is unsatisfiable in the CI image (#1269) ---
|
|
# Signed by fred (sb-it-1-dt, 2026-08-16) at origin/next 476db12.
|
|
# This suite asserts the launcher's behaviour when `mosaic` and `pi` are MISSING.
|
|
# It shims fakes into $FAKE_BIN, but the constructed PANE_PATH always ends in the
|
|
# real system path, so on a host that installs those binaries the missing-binary
|
|
# cases cannot be measured at all. The suite's own guard (line 103) says so and
|
|
# fails rather than reporting a pass it cannot back. That guard is correct.
|
|
# The error was wiring the suite into CI: #1017 (c56483eb) enumerated it and
|
|
# dropped this exclusion, and the CI image provides `pi` in the system path, so
|
|
# it has failed on every pipeline since. Measured 2026-08-16 across pipelines
|
|
# 2444 (#1256), 2438 (#1240) and 2441 (#1017-quality): exactly one FAIL line in
|
|
# each full log, identical, this assertion; control `zzz-not-present-zzz` -> 0.
|
|
# Because test:framework-shell is one && chain and this sat at position 44 of 48,
|
|
# the four suites after it had not run at all since the merge.
|
|
packages/mosaic/framework/tools/fleet/test-start-agent-session.sh | precondition unsatisfiable in the CI image: asserts missing-binary behaviour, but PANE_PATH always ends in the system path and the image provides `pi` there; guard at line 103 fails by design rather than passing unmeasured. Burn down by controlling the tail of PANE_PATH inside the test. NOT by removing `pi` from the image: the CI image installs @earendil-works/[email protected] deliberately (measured in pipeline 2444's test-step log), and other suites depend on that pin. See #1269
|