fargo
9d3e22b1c1
ci/woodpecker/pr/ci Pipeline failed
Wall 6: no documented path ever enabled or started the shipped
mosaic-lease-broker.service — every gated runtime died ~4s in at lease
registration while fleet start reported rc0, and a broker not in the
reconciler plan could not be reported as drifted.
Activation lands in the control plane, not the launcher:
- fleet install places ALL FOUR units through placeUnitFile — a placement
helper that unlinks any by-path-enable symlink at the destination
BEFORE copying (Node copyFile follows the link and overwrites the SEED
template; measured on a throwaway systemd user instance 2026-08-17,
with both cp and fs.copyFile), removes a stale wants-symlink pointing
outside the active dir (readlink — readFile returns the target's
content, not the link path), then copies and daemon-reloads. The same
measurement showed systemctl enable <name> does NOT rewrite an existing
by-path wants-symlink — reconciliation must be explicit. Idempotent:
second install on by-path residue converges to the identical state.
Until now the copy block named three units and omitted the broker, and
the residue set / copy set were disjoint only by accident (fomo-lin
survived copy-through because its one symlink was the one unit not
copied); adding the broker made them intersect on first run. See the
SET-INDEPENDENCE note on the helper before adding a fifth unit.
- enableFleetUnits enables the broker first, alongside the holder.
- fleet start / reconciler start the broker BEFORE any holder/agent
lifecycle effect, then RE-CHECK the socket (not unit state) and exit
nonzero with a named code if it did not appear. Re-probed on every
invocation — a RemainAfterExit=yes dead-looking-active unit can never
make retry look like repair (the sticky-retry check).
- The reconciler plan carries broker {unitInstalled, socketPresent} as a
first-class member; the socket is the signal (enabled-but-dead units
report socketPresent=false).
- start-agent-session.sh preflights the broker socket BEFORE any tmux
effect (moved ahead of the ownership probe): absent -> exit 75
(EX_TEMPFAIL), named refusal with socket path and remedy, no doomed
pane. The agent@ unit is Type=oneshot with no Restart=, so the message
survives instead of looping. The preflight detects and refuses; it
never starts the broker.
- mosaic doctor's lease check names one convention-neutral remedy:
'mosaic fleet install (it reconciles either enable convention)' —
written from the measurement; teaching a manual systemctl line could
leave a host with competing wants-symlinks.
Tests: fleet-place-unit.spec.ts (8: clean-host negative control,
by-path residue -> seed bytes AND mtime unchanged [the finding-2 check],
wants-residue cleared, idempotence single + double-install convergence);
fleet.spec.ts broker-first enable ordering, refused start emits no
holder/agent calls, second-start re-probe; reconciler broker plan member
(enabled-but-dead shape) + broker-before-agent ordering in both command
and apply paths; test-agent-session-broker-preflight.sh (CI-fit: fake
tmux, real unix socket at a short /tmp path — AF_UNIX caps at 108 bytes,
hermetic env; absent -> exit 75 + no tmux session, live socket passes,
explicit env wins, --stop not fenced). 1563/1563 vitest, lint, root
build 25/25, root typecheck 45/45.
Sabotage controls: placement unlink removed -> exactly the seed-integrity
test reddens (1/8); socket re-check disabled -> exactly the two preflight
specs redden; shell preflight removed -> the bash suite reddens (6 FAIL
assertions, rc=1). All restored byte-identically (sha256-verified), all
green again.
Test 6 (greenfield 1124, seat alive 2min + second fleet start) runs on
sandbox after daphne's baseline, coordinated with fred.
Note: the preflight uses exit 75 measured against the unit's Restart=
policy (oneshot, none) — no restart loop.