fix(fleet): lease-broker activation, symlink-safe unit placement, named launch refusal — Wall 6 (#1292) #1297
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 fargo
|
04a87262c8 |
ci(#1297): compose the preflight live-socket path instead of mktemp -u
ci/woodpecker/pr/ci Pipeline was successful
Pipeline 2562: the shell chain now reaches the preflight suite and dies at 'mktemp: : Invalid argument' — the CI image's mktemp dialect rejects 'mktemp -u /tmp/...XXXXXX.sock' (GNU accepts it locally, which is why the suite passed on dev hosts only). The path is now composed (/tmp/mosaic-preflight-$RANDOM-$$.sock): short enough for the 108-byte AF_UNIX bind limit (a repo-deep path fails with 'path too long', measured), unique per run, no pre-existing file, cleaned by the existing trap. Verified locally: suite passes, no /tmp residue. |
||
|
fargo
|
e68650a8e7 |
test(#1297): make broker-dependent tests host-independent (CI root + no live broker)
Pipeline 2561 failed 5 tests that pass on a dev host, two环境 classes: 1. Un-seamed success-path start/apply tests answered the HOST's broker state: this dev machine has a live lease-broker socket at the default XDG path, so the real probe (correctly, since #1297 F3) returned true locally; CI has no broker, so the new start re-check refused broker-absent. Fixed by injecting a deterministic present-broker seam in exactly the tests whose property is ordering/targeting (reconciler start-ordering, two apply-running tests, and the CLI command spec's shared helper). Refusal and real-probe tests keep explicit false seams / explicit socket paths. 2. The placeUnitFile unlink-failure spec relies on EACCES from a 0500 directory; CI steps run as root (apk add succeeds unprivileged-free) and CAP_DAC_OVERRIDE makes unlink succeed, so the abort path cannot be triggered there. Now it.skip under uid 0 with the reason stated; exercised on every non-root dev host. The remaining unseamed apply tests use the stopped-agent path (no broker interaction) and passed CI unchanged — verified against the 2561 log, not assumed. |
||
|
fargo
|
c7c5bfc036 |
ci(#1297): wire the broker-preflight suite into test:framework-shell; track the env contract
Pipeline 2468/2556 failed the CI 'sanitization' step; the log (2556, step 60166) shows verify-sanitized.sh PASSED both times — the failure was the test-enumeration guard bundled in that step: the new tools/fleet/test-agent-session-broker-preflight.sh existed on disk but was not enumerated on any CI surface. Wired as the last link of test:framework-shell (guard: population 62, enumerated 47, excluded 16). Freshened onto current next, which also grew a required generated-env key (MOSAIC_GIT_IDENTITY, = agent name): the suite's hermetic env.generated fixture now declares it, keeping the suite CI-fit against the current start-agent-session.sh contract rather than the one it was written for. |
||
|
fargo
|
bdc23da04e |
fix(#1297 review): real socket probe, honest placement abort, observable broker state
rev-security-03 REQUEST_CHANGES (review note on the brain), all four findings verified by measurement before fixing: F1 BLOCKER — brokerSocketPresent used access(path, S_IFSOCK). S_IFSOCK (0xC000) is a file-type constant, not an access() mode (0-7): on node 24.18.0 the call throws ERR_OUT_OF_RANGE, so the probe could NEVER return true and the swallow-catch made every un-seamed call report the broker absent — on this host (roster v1) 'mosaic fleet start' would have refused broker-absent forever. Now stat().isSocket(), matching the bash side's [ -S ]. New spec exercises the REAL probe against a REAL unix socket (true), a regular file (false), and an absent path (false) — no seam, so no seam can hide this again. F2 — placeUnitFile's single catch conflated destination-absent with unlink-FAILED; on unlink failure it copied through the still-live symlink (copy-through overwrite measured by the reviewer). Now: ENOENT is the only swallowed lstat outcome; unlink failure aborts with a named UnitPlacementError BEFORE any copy. New spec proves the residue target's bytes survive an unlink failure and the destination link is untouched. F3 — observeBroker defaulted unit/socket to false when seams unset, and production injects none: plan/status/doctor reported a healthy broker as absent. Seams still take precedence; with no seam the real stat() probes run. The v2 start path (and apply-with-running) now re-check the socket after enable+start with a NAMED lifecycle-precondition-failed refusal — previously only the v1 path had that protection, and the refusal was masked into the generic recoverable result. Acceptance fixtures gain hermetic brokerSocketEnv paths (the old fixture asserted the lying default). F4 (note) — unlink race stays inside the user-owned dir; no action. Local gates on this tree: vitest 1566/1566, typecheck, lint, prettier 3.8.1, verify-sanitized all pass. CI 2468's sanitization failure did not reproduce locally on the identical tree; watching the fresh pipeline. |
||
|
fargo
|
c2db6728b4 |
fix(fleet): activate the lease broker at install/start, place units through symlinks safely, refuse doomed launches (#1292)
Wall 6: no documented path ever enabled or started the shipped
mosaic-lease-broker.service — every gated runtime died ~4s in at lease
registration while fleet start reported rc0, and a broker not in the
reconciler plan could not be reported as drifted.
Activation lands in the control plane, not the launcher:
- fleet install places ALL FOUR units through placeUnitFile — a placement
helper that unlinks any by-path-enable symlink at the destination
BEFORE copying (Node copyFile follows the link and overwrites the SEED
template; measured on a throwaway systemd user instance 2026-08-17,
with both cp and fs.copyFile), removes a stale wants-symlink pointing
outside the active dir (readlink — readFile returns the target's
content, not the link path), then copies and daemon-reloads. The same
measurement showed systemctl enable <name> does NOT rewrite an existing
by-path wants-symlink — reconciliation must be explicit. Idempotent:
second install on by-path residue converges to the identical state.
Until now the copy block named three units and omitted the broker, and
the residue set / copy set were disjoint only by accident (fomo-lin
survived copy-through because its one symlink was the one unit not
copied); adding the broker made them intersect on first run. See the
SET-INDEPENDENCE note on the helper before adding a fifth unit.
- enableFleetUnits enables the broker first, alongside the holder.
- fleet start / reconciler start the broker BEFORE any holder/agent
lifecycle effect, then RE-CHECK the socket (not unit state) and exit
nonzero with a named code if it did not appear. Re-probed on every
invocation — a RemainAfterExit=yes dead-looking-active unit can never
make retry look like repair (the sticky-retry check).
- The reconciler plan carries broker {unitInstalled, socketPresent} as a
first-class member; the socket is the signal (enabled-but-dead units
report socketPresent=false).
- start-agent-session.sh preflights the broker socket BEFORE any tmux
effect (moved ahead of the ownership probe): absent -> exit 75
(EX_TEMPFAIL), named refusal with socket path and remedy, no doomed
pane. The agent@ unit is Type=oneshot with no Restart=, so the message
survives instead of looping. The preflight detects and refuses; it
never starts the broker.
- mosaic doctor's lease check names one convention-neutral remedy:
'mosaic fleet install (it reconciles either enable convention)' —
written from the measurement; teaching a manual systemctl line could
leave a host with competing wants-symlinks.
Tests: fleet-place-unit.spec.ts (8: clean-host negative control,
by-path residue -> seed bytes AND mtime unchanged [the finding-2 check],
wants-residue cleared, idempotence single + double-install convergence);
fleet.spec.ts broker-first enable ordering, refused start emits no
holder/agent calls, second-start re-probe; reconciler broker plan member
(enabled-but-dead shape) + broker-before-agent ordering in both command
and apply paths; test-agent-session-broker-preflight.sh (CI-fit: fake
tmux, real unix socket at a short /tmp path — AF_UNIX caps at 108 bytes,
hermetic env; absent -> exit 75 + no tmux session, live socket passes,
explicit env wins, --stop not fenced). 1563/1563 vitest, lint, root
build 25/25, root typecheck 45/45.
Sabotage controls: placement unlink removed -> exactly the seed-integrity
test reddens (1/8); socket re-check disabled -> exactly the two preflight
specs redden; shell preflight removed -> the bash suite reddens (6 FAIL
assertions, rc=1). All restored byte-identically (sha256-verified), all
green again.
Test 6 (greenfield 1124, seat alive 2min + second fleet start) runs on
sandbox after daphne's baseline, coordinated with fred.
Note: the preflight uses exit 75 measured against the unit's Restart=
policy (oneshot, none) — no restart loop.
|