Reviewed by scooby via git comms (no mosaicstack principal on fomo-lin; review is the comms record, terminal ACK 32d986). Merge directed by Jason 2026-08-16. Part of the five-PR greenfield composite verified E2E on two independent bare boxes.
`mosaic fleet start` returned 0 over three dead panes. The launcher knew,
and said the wrong thing at the wrong severity to the wrong layer.
The pane runs `mosaic yolo <runtime>` under PANE_PATH with a cleared
environment. When that binary is absent the pane dies in under a second,
tmux destroys the session, and the diagnostic goes with it. The launcher
then found no PANE_PID, printed a WARNING about the *heartbeat sidecar*,
and exited 0 — so systemd logged "Finished ... successfully" and
`fleet start` reported success. `fleet ps` was the only component telling
the truth.
Two changes, both in start-agent-session.sh:
1. Before any effect, resolve `mosaic` and the roster's runtime against
PANE_PATH — the pane's own view of the path, not the launcher's.
`mosaic yolo <runtime>` calls checkRuntime(runtime) and looks for a
binary named exactly like the runtime, so this asks the same question
the pane will ask a moment later, while an operator can still see the
answer. Absent binary -> exit 69, code=missing-binary, no session
created.
2. Replace the dead-pane WARNING+exit-0. An absent session one second
after new-session is a runtime that died on startup, not a heartbeat
problem -> exit 69, code=pane-did-not-survive, with the command to run
by hand to see why. A present session with no pane PID after five
attempts -> code=pane-pid-unresolved. Neither branch kills the
session; destroying a possibly-live pane on a guess is worse than
leaving it for inspection.
Exit 69 (EX_UNAVAILABLE) is deliberate: the 64s already in this file mean
the projection was bad, and here the data is fine and the host is not
ready. Callers separate the cases by `code=`, the same way fail_env's
codes share 64.
This propagates for free. `fleet start` calls runChecked() for the holder
and each agent, and runChecked throws on non-zero, so layers 4 and 5 stop
lying without a TypeScript change. Two adjacent defects are left for a
follow-up issue rather than widened into this diff: the per-agent loop
aborts on the first failure instead of attempting all and reporting an
aggregate, and runChecked's bare throw surfaces the launcher's message
under a Node unhandled-rejection stack trace because program.parse() is
synchronous.
Tests:
- test-start-agent-session.sh gains three cases: `mosaic` absent from the
pane path, the runtime absent from the pane path, and a pane that does
not survive. Each was verified individually red against the unmodified
origin/next launcher.
- The two cases asserting a valid launch now supply a pane PID. Until now
the suite's one success path was itself a dead pane the launcher
reported as fine.
- The harness fakes `npm` so PANE_PATH stops depending on whatever the
host has installed, and fails loudly if the host provides `mosaic` or
`pi` in the system path, where the missing-binary cases would not be
measurable at all.
- test-fleet-units.sh gains a `pi` shim in its runtime bin. The real-tmux
harness named `pi` in its roster and never installed it; the new
preflight caught it.
Refs #1241
Self-inflicted and worth recording rather than quietly amending.
To prove the new tests were red without the fix I ran
`git checkout origin/next -- <fleet.ts> <[email protected]>`. That writes
the *index*, not just the working tree. Copying my versions back afterwards
restored the working tree only, so the unit file sat staged-as-origin/next and
modified-in-tree, and the next commit (67f5014c) committed the index — silently
removing the ConditionPathExists line that 463745e3 had added.
Nothing caught it. The spec reads the file from the working tree, so it stayed
10/10 green against a HEAD that no longer had the guard. Found by reading
`git status` after the push, not by any gate.
Verified by content, not by assumption:
origin/next 0 occurrences
463745e3 1
67f5014c 0 <- the regression
this commit 1
Refs #1237
Two follow-ups from the canary red->green run and scooby's review.
1. The v2 refusal in `add`/`remove` was a bare `throw`, which reaches the CLI
top level uncaught and prints the guidance under a Node stack trace. The
message *is* the point of the refusal, so it now goes through
`command.error()` — the same clean path the roster-config error uses.
Caught on canary, not in review: the unit tests asserted the message text
and passed either way.
2. The unit-template test asserted only that ConditionPathExists is present.
Presence is not effect. Added two tests for the parts that can drift in
code while that assertion still passes: the condition resolving to exactly
the file the fleet writes (%h/%i rendered against a real install), and the
launcher genuinely failing on an absent generated env (exit 64,
`missing-file`) — which is what makes the condition load-bearing rather
than decorative.
systemd is not available in the suite, so the effect itself was measured on
canary (2026-08-16), roster v2 generation 3:
with the condition: start rc=0, Result=success, ConditionResult=no,
journal "skipped, unmet condition check"
condition removed by
drop-in, nothing else: start rc=1, Result=exit-code, ExecMainStatus=64,
unit failed, "agent environment rejected: missing-file"
Canary red->green for the three commands, same v2 roster, side by side:
fleet ps 0.0.50-next.2413 rc=1 -> branch rc=0 (3 agents listed)
fleet install 0.0.50-next.2413 rc=1 -> branch rc=0
fleet remove <name> 0.0.50-next.2413 rc=1 -> branch rc=1, refusal naming
delete + apply
All three previously failed with "Fleet roster has unknown field(s):
generation." The #791 negative was measured too: the six existing
*.env.generated files were untouched by `install` (mtimes 20+ minutes older
than the run).
Gates: typecheck 0, eslint 0, prettier clean, fleet specs 382 passed, new spec
10/10 with the fix and 9/10 red against origin/next (the 10th passes there for
an unrelated reason and is annotated as such). Full suite: only
mutator-gate.acceptance.spec.ts fails, pre-existing on origin/next.
Still true and still worth saying: a correct fix here shows install rc=0 and
start rc=0 and STILL no live seat. #1240 (tmux absent) is upstream, #1241
(start reports lifecycle-complete over dead panes) and the missing agent
runtime are downstream.
Refs #1237
Reviewed-by: scooby (by git comms; cannot file a Gitea review from fomo-lin)
On a roster-v2 fleet, `ps`, `install`, `install-systemd`, `add` and `remove`
all failed in the v1 parser. The consequence was that a greenfield v2 box could
never get its unit templates placed, so nothing downstream could start.
The read-only commands get a narrow version-agnostic view of the roster
(version, socket name, holder session, and per agent name/alias/runtime).
This is deliberately not a v2 -> v1 downshift. A downshifted FleetRoster would
be accepted by generateAgentEnvValues, which would make a third writer of
fleet/agents/<name>.env.generated through the v1 mapping and break the #791
single-SSOT invariant that projectRosterV2AgentGeneratedEnv is documented to
hold. The view is too small to write a roster or an env file back from, so that
misuse is unavailable rather than merely discouraged.
So on a v2 roster `install` places the tool files and the unit templates,
enables the units, and writes no generated env at all. Env belongs to `apply`
and `regen`, both already v2-native.
That change alone would have traded an init-time failure for a boot-time one.
`install` enables mosaic-agent@<name>.service (WantedBy=default.target) without
starting it, so a reboot between `install` and the first `apply` would run
ExecStart against an absent env file and fail every seat unit, further from its
cause. The unit template now carries
ConditionPathExists=%h/.config/mosaic/fleet/agents/%i.env.generated
which skips an enabled-but-unconfigured unit cleanly and starts it on the next
start once the reconciler has written env. On v1 it is a no-op, since v1
`install` writes env itself. Found in review by scooby.
`add` and `remove` are not routed to `create` and `delete`. They are different
operations: the v1 pair edits the roster and drives systemd, the v2 pair is
documented as changing desired state without runtime actions. `add` also
collects four fields where a v2 agent requires eleven, so routing it would mean
inventing an operator's provider, alias, reasoning and tool policy. On v2 both
now fail with the real two-step sequence instead.
Tests: 8 new, 7 of which are red before this change. Includes the greenfield
case scooby asked for — `ps` on a fresh v2 install with nothing running is rc=0
and lists every agent stopped, since that is the command an operator runs to
find out why there is no seat.
Note for anyone verifying this: a correct fix here shows `install` rc=0 and
`start` rc=0 and still no live seat. #1240 (tmux absent) is upstream, #1241
(start reports lifecycle-complete over dead panes) and the missing agent
runtime are downstream. A dead pane after this change is not a regression here.
Refs #1237, #791, #1240, #1241
Part of #869
Mos (id-11) Gate-16 merge: independent APPROVE @75235ef8 (9/9, no live host mutation), author id2 != approver id11, CI green wp1971.
Co-authored-by: jason.woltje <[email protected]>
Co-committed-by: jason.woltje <[email protected]>
tiny
fred
jason.woltjeandMos