fix(fleet): fail the agent launcher when the pane cannot survive (#1241) #1244

2026-08-16T04:59:40Z

fred commented

2026-08-16 04:59:40 +00:00

Fixes the third link in the chain between a green mosaic fleet install and an actually-live seat.

What was wrong

On canary, mosaic fleet start returned success over three dead panes. Measured directly, before this change:

$ start-agent-session.sh generalist
WARNING: could not resolve pane PID for generalist — heartbeat sidecar not started
rc=0
$ tmux -L mosaic-fleet ls
_holder: 1 windows      # no generalist session — the pane died and tmux destroyed it

The reporting chain, layer by layer:

start-agent-session.sh:119 validates the runtime name against claude|codex|opencode|pi. That is a spelling check. There is no command -v anywhere in the file.
The session is created anyway. The pane runs mosaic yolo <runtime> under PANE_PATH with a cleared environment; the binary is absent, so it dies in under a second and tmux destroys the session, taking the diagnostic with it.
PANE_PID comes back empty. The launcher prints a WARNING about the heartbeat sidecar and exits 0.
systemd logs Finished [email protected] — Mosaic tmux fleet agent generalist.
mosaic fleet start returns rc=0 with no output.

mosaic fleet ps was the only component in the stack telling the truth.

To be clear about what is not the bug: active (exited) is correct. These units are Type=oneshot with RemainAfterExit=yes; they spawn into tmux and exit by design. The bug is the dead pane and the silence above it.

What this changes

Two edits, both in start-agent-session.sh.

1. Resolve the pane's binaries before any effect. Immediately after PANE_PATH is computed — before _ensure_claude_workdir_trusted, before mkdir, before new-session — resolve mosaic and the roster's runtime against PANE_PATH, which is the pane's own view of the path and not the launcher's. mosaic yolo <runtime> calls checkRuntime(runtime) (launch.ts:896, all four runtimes) and the binary it looks for is named exactly like the runtime, so resolving the runtime name here asks the same question the pane will ask a second later — while an operator can still see the answer.

2. The dead-pane branch stops reporting a heartbeat problem. tmux destroys a session when its pane command exits, so an absent session one second after new-session means the runtime died on startup. That is now code=pane-did-not-survive, and it carries the command to run by hand to find out why. A session that is present but yields no pane PID after five attempts is code=pane-pid-unresolved — a different fact, so a different code.

Neither branch kills the session. Destroying a pane that might be alive for a transient reason is a destructive act on a guess; leaving it for inspection is not.

Exit 69 (EX_UNAVAILABLE), deliberately. The fail/fail_env rejections already in this file exit 64 (EX_USAGE) and mean the projection itself was bad. Here the data is fine and the host is not ready. Callers separate the individual cases by code=, exactly the way fail_env's many codes already share exit 64.

Measured on canary (`mosaic-sbx-canary`, real tmux, real systemd, no runtime installed)

After:

$ start-agent-session.sh generalist
ERROR: agent launch aborted: code=missing-binary agent=generalist 'pi' is not on the pane PATH (/home/mosaic/.npm-global/bin:/usr/local/bin:/usr/bin:/bin)
rc=69

$ systemctl --user start [email protected]
Job for [email protected] failed because the control process exited with error code.
  ActiveState=failed  Result=exit-code  ExecMainStatus=69

$ journalctl --user -u [email protected]
ERROR: agent launch aborted: code=missing-binary agent=generalist 'pi' is not on the pane PATH (...)
[email protected]: Main process exited, code=exited, status=69/UNAVAILABLE
Failed to start [email protected].

And with a runtime that exists but exits instantly (fake pi that returns 0), which exercises the other branch:

ERROR: agent launch aborted: code=pane-did-not-survive agent=generalist the pane exited immediately and tmux destroyed the session; run 'mosaic yolo pi' in /home/mosaic/src to see why
rc=69

Layer 5 (mosaic fleet start itself) could not be measured on canary because the installed CLI still rejects that box's v2 roster with lifecycle-precondition-failed — that is #1237, fixed separately in #1243. It should propagate without further change: fleet start calls runChecked() for the holder and each agent (fleet.ts:1606-1609), and runChecked throws on non-zero. Worth re-measuring once #1243 lands.

Tests

test-start-agent-session.sh gains three cases — mosaic absent from the pane path, the runtime absent from the pane path, and a pane that does not survive. Each was verified individually red against the unmodified origin/next launcher (swapped in with git show <ref>:<path> > <path>, not git checkout, which writes the index).

Three supporting changes to that harness are worth calling out:

The two cases asserting a valid launch now supply a pane PID. They did not before, which means the suite's one success path was itself a dead pane the launcher reported as fine. The suite was reproducing the bug and asserting it was correct.
The harness fakes npm. PANE_PATH is derived partly from npm config get prefix, so left to the real npm it splices whatever the host has installed into the path under test, and the missing-binary cases would pass or fail by accident of the machine.
It fails loudly if the host provides mosaic or pi in the system path, where those cases are not measurable at all. A green run that measured nothing is worse than a red one.

test-fleet-units.sh gains a pi shim in its runtime bin. That harness named pi in its roster and never installed it; the new preflight caught it, which is the preflight working.

Gates run: bash -n on all three scripts (rc=0), root pnpm typecheck (rc=0, 45/45), fleet specs (215 passed), test-fleet-units.sh under real tmux (rc=0), test-start-agent-session.sh (rc=0). prettier --check cannot parse .sh (rc=2, "No parser could be inferred") and shellcheck is not installed on this host, so shell changes currently pass through the quality gate unlinted — a separate finding I will file.

Deliberately not in this diff

Two adjacent defects, both real, both wider than this fix:

fleet start's per-agent loop aborts on the first failing agent instead of attempting all and reporting an aggregate. With this change, one missing runtime now stops the rest of the fleet from being attempted.
runChecked's bare throw surfaces the launcher's message under a Node unhandled-rejection stack trace, because program.parse() (cli.ts:622) is synchronous. The message is the point, and a stack trace buries it. Same family as the pre-existing bare throw new Error("Invalid runtime …") in this path.

Follow-up issue to come for both.

The runtime-provisioning question behind all of this — should the installer provide a runtime, should fleet init stop scaffolding rosters that name absent ones, should fleet install refuse — is a design decision, not a bug fix, and I am not deciding it unilaterally. Same posture as #1237 Piece B.

Refs #1241. Chain context: #1240 (tmux absent, neither installer nor doctor mentions it) is upstream; #1243 fixes #1237 Piece A; the missing agent runtime is downstream of all three.

Fixes the third link in the chain between a green `mosaic fleet install` and an actually-live seat. ## What was wrong On canary, `mosaic fleet start` returned success over three dead panes. Measured directly, before this change: ``` $ start-agent-session.sh generalist WARNING: could not resolve pane PID for generalist — heartbeat sidecar not started rc=0 $ tmux -L mosaic-fleet ls _holder: 1 windows # no generalist session — the pane died and tmux destroyed it ``` The reporting chain, layer by layer: 1. `start-agent-session.sh:119` validates the runtime *name* against `claude|codex|opencode|pi`. That is a spelling check. There is no `command -v` anywhere in the file. 2. The session is created anyway. The pane runs `mosaic yolo <runtime>` under `PANE_PATH` with a cleared environment; the binary is absent, so it dies in under a second and tmux destroys the session, taking the diagnostic with it. 3. `PANE_PID` comes back empty. The launcher prints a WARNING about the **heartbeat sidecar** and exits 0. 4. systemd logs `Finished [email protected] — Mosaic tmux fleet agent generalist.` 5. `mosaic fleet start` returns rc=0 with no output. `mosaic fleet ps` was the only component in the stack telling the truth. To be clear about what is *not* the bug: `active (exited)` is correct. These units are `Type=oneshot` with `RemainAfterExit=yes`; they spawn into tmux and exit by design. The bug is the dead pane and the silence above it. ## What this changes Two edits, both in `start-agent-session.sh`. **1. Resolve the pane's binaries before any effect.** Immediately after `PANE_PATH` is computed — before `_ensure_claude_workdir_trusted`, before `mkdir`, before `new-session` — resolve `mosaic` and the roster's runtime against `PANE_PATH`, which is the pane's own view of the path and not the launcher's. `mosaic yolo <runtime>` calls `checkRuntime(runtime)` (`launch.ts:896`, all four runtimes) and the binary it looks for is named exactly like the runtime, so resolving the runtime name here asks the same question the pane will ask a second later — while an operator can still see the answer. **2. The dead-pane branch stops reporting a heartbeat problem.** tmux destroys a session when its pane command exits, so an absent session one second after `new-session` means the runtime died on startup. That is now `code=pane-did-not-survive`, and it carries the command to run by hand to find out why. A session that *is* present but yields no pane PID after five attempts is `code=pane-pid-unresolved` — a different fact, so a different code. Neither branch kills the session. Destroying a pane that might be alive for a transient reason is a destructive act on a guess; leaving it for inspection is not. **Exit 69 (`EX_UNAVAILABLE`), deliberately.** The `fail`/`fail_env` rejections already in this file exit 64 (`EX_USAGE`) and mean *the projection itself was bad*. Here the data is fine and the host is not ready. Callers separate the individual cases by `code=`, exactly the way `fail_env`'s many codes already share exit 64. ## Measured on canary (`mosaic-sbx-canary`, real tmux, real systemd, no runtime installed) After: ``` $ start-agent-session.sh generalist ERROR: agent launch aborted: code=missing-binary agent=generalist 'pi' is not on the pane PATH (/home/mosaic/.npm-global/bin:/usr/local/bin:/usr/bin:/bin) rc=69 $ systemctl --user start [email protected] Job for [email protected] failed because the control process exited with error code. ActiveState=failed Result=exit-code ExecMainStatus=69 $ journalctl --user -u [email protected] ERROR: agent launch aborted: code=missing-binary agent=generalist 'pi' is not on the pane PATH (...) [email protected]: Main process exited, code=exited, status=69/UNAVAILABLE Failed to start [email protected]. ``` And with a runtime that exists but exits instantly (fake `pi` that returns 0), which exercises the other branch: ``` ERROR: agent launch aborted: code=pane-did-not-survive agent=generalist the pane exited immediately and tmux destroyed the session; run 'mosaic yolo pi' in /home/mosaic/src to see why rc=69 ``` Layer 5 (`mosaic fleet start` itself) could not be measured on canary because the installed CLI still rejects that box's v2 roster with `lifecycle-precondition-failed` — that is #1237, fixed separately in #1243. It should propagate without further change: `fleet start` calls `runChecked()` for the holder and each agent (`fleet.ts:1606-1609`), and `runChecked` throws on non-zero. Worth re-measuring once #1243 lands. ## Tests `test-start-agent-session.sh` gains three cases — `mosaic` absent from the pane path, the runtime absent from the pane path, and a pane that does not survive. Each was verified **individually red** against the unmodified `origin/next` launcher (swapped in with `git show <ref>:<path> > <path>`, not `git checkout`, which writes the index). Three supporting changes to that harness are worth calling out: - **The two cases asserting a valid launch now supply a pane PID.** They did not before, which means the suite's one success path was itself a dead pane the launcher reported as fine. The suite was reproducing the bug and asserting it was correct. - **The harness fakes `npm`.** `PANE_PATH` is derived partly from `npm config get prefix`, so left to the real npm it splices whatever the host has installed into the path under test, and the missing-binary cases would pass or fail by accident of the machine. - **It fails loudly if the host provides `mosaic` or `pi` in the system path**, where those cases are not measurable at all. A green run that measured nothing is worse than a red one. `test-fleet-units.sh` gains a `pi` shim in its runtime bin. That harness named `pi` in its roster and never installed it; the new preflight caught it, which is the preflight working. Gates run: `bash -n` on all three scripts (rc=0), root `pnpm typecheck` (rc=0, 45/45), fleet specs (215 passed), `test-fleet-units.sh` under real tmux (rc=0), `test-start-agent-session.sh` (rc=0). `prettier --check` cannot parse `.sh` (rc=2, "No parser could be inferred") and `shellcheck` is not installed on this host, so **shell changes currently pass through the quality gate unlinted** — a separate finding I will file. ## Deliberately not in this diff Two adjacent defects, both real, both wider than this fix: - `fleet start`'s per-agent loop **aborts on the first failing agent** instead of attempting all and reporting an aggregate. With this change, one missing runtime now stops the rest of the fleet from being attempted. - `runChecked`'s bare `throw` surfaces the launcher's message under a Node unhandled-rejection stack trace, because `program.parse()` (`cli.ts:622`) is synchronous. The message is the point, and a stack trace buries it. Same family as the pre-existing bare `throw new Error("Invalid runtime …")` in this path. Follow-up issue to come for both. The runtime-provisioning question behind all of this — should the installer provide a runtime, should `fleet init` stop scaffolding rosters that name absent ones, should `fleet install` refuse — is a design decision, not a bug fix, and I am not deciding it unilaterally. Same posture as #1237 Piece B. Refs #1241. Chain context: #1240 (tmux absent, neither installer nor `doctor` mentions it) is upstream; #1243 fixes #1237 Piece A; the missing agent runtime is downstream of all three.

fred added 1 commit 2026-08-16 04:59:40 +00:00

fix(fleet): fail the agent launcher when the pane cannot survive (#1241 )

ci/woodpecker/pr/ci Pipeline was successful

Details

61a907a12f

`mosaic fleet start` returned 0 over three dead panes. The launcher knew,
and said the wrong thing at the wrong severity to the wrong layer.

The pane runs `mosaic yolo <runtime>` under PANE_PATH with a cleared
environment. When that binary is absent the pane dies in under a second,
tmux destroys the session, and the diagnostic goes with it. The launcher
then found no PANE_PID, printed a WARNING about the *heartbeat sidecar*,
and exited 0 — so systemd logged "Finished ... successfully" and
`fleet start` reported success. `fleet ps` was the only component telling
the truth.

Two changes, both in start-agent-session.sh:

1. Before any effect, resolve `mosaic` and the roster's runtime against
   PANE_PATH — the pane's own view of the path, not the launcher's.
   `mosaic yolo <runtime>` calls checkRuntime(runtime) and looks for a
   binary named exactly like the runtime, so this asks the same question
   the pane will ask a moment later, while an operator can still see the
   answer. Absent binary -> exit 69, code=missing-binary, no session
   created.

2. Replace the dead-pane WARNING+exit-0. An absent session one second
   after new-session is a runtime that died on startup, not a heartbeat
   problem -> exit 69, code=pane-did-not-survive, with the command to run
   by hand to see why. A present session with no pane PID after five
   attempts -> code=pane-pid-unresolved. Neither branch kills the
   session; destroying a possibly-live pane on a guess is worse than
   leaving it for inspection.

Exit 69 (EX_UNAVAILABLE) is deliberate: the 64s already in this file mean
the projection was bad, and here the data is fine and the host is not
ready. Callers separate the cases by `code=`, the same way fail_env's
codes share 64.

This propagates for free. `fleet start` calls runChecked() for the holder
and each agent, and runChecked throws on non-zero, so layers 4 and 5 stop
lying without a TypeScript change. Two adjacent defects are left for a
follow-up issue rather than widened into this diff: the per-agent loop
aborts on the first failure instead of attempting all and reporting an
aggregate, and runChecked's bare throw surfaces the launcher's message
under a Node unhandled-rejection stack trace because program.parse() is
synchronous.

Tests:

- test-start-agent-session.sh gains three cases: `mosaic` absent from the
  pane path, the runtime absent from the pane path, and a pane that does
  not survive. Each was verified individually red against the unmodified
  origin/next launcher.
- The two cases asserting a valid launch now supply a pane PID. Until now
  the suite's one success path was itself a dead pane the launcher
  reported as fine.
- The harness fakes `npm` so PANE_PATH stops depending on whatever the
  host has installed, and fails loudly if the host provides `mosaic` or
  `pi` in the system path, where the missing-binary cases would not be
  measurable at all.
- test-fleet-units.sh gains a `pi` shim in its runtime bin. The real-tmux
  harness named `pi` in its roster and never installed it; the new
  preflight caught it.

Refs #1241

mos-dt-0 referenced this pull request

2026-08-16 05:00:03 +00:00

fleet start exits 0 with every agent pane dead: launcher detects the failed launch, logs it as a heartbeat warning, and succeeds #1241

fred referenced this pull request

2026-08-16 05:16:40 +00:00

fix(fleet): tell the operator when the fleet transport is missing (#1240) #1245

mos-dt-0 added 1 commit 2026-08-16 05:19:48 +00:00

test(fleet): cover the pane-pid-unresolved branch this PR shipped (#1241 )

ci/woodpecker/pr/ci Pipeline was canceled

Details

10a1f82031

Review finding from scooby: this PR added a failure branch the harness
structurally could not reach. The fake tmux answered `has-session` only for
`=_holder:0.0`, so every non-holder agent landed in the session-is-gone branch
no matter what — the `elif` (tmux still reports the session, no pane PID after
the retries) had zero coverage and no way to get any.

That is the same shape as the bug this PR exists to fix, one layer down: a code
path shipped green where the gate that should measure it cannot. Less severe,
because the branch fails closed at exit 69 rather than reporting success — but
"the harness can't reach it" is the sentence that precedes the next silent
regression, so it gets closed here rather than filed.

`MOSAIC_TEST_HELD_SESSIONS` lets a case name targets the shim should also
answer for. It answers them only AFTER `new-session`, and that detail is the
whole trick: the launcher asks `has-session` about the same name twice — once
at line 255 where a yes means "already running, exit 0", and once at 417 where
a yes means "the session survived". A shim answering yes to both short-circuits
at the first and never reaches the branch under test. It would have looked like
coverage while measuring the idempotency path.

Both failure modes were measured, not reasoned about:
- toggle absent (the old shim): `code=pane-did-not-survive` — the case lands on
  the wrong branch, which is exactly the unreachability being reported.
- toggle answering unconditionally: launcher exits 0 via the idempotency
  short-circuit — "launcher reported success over a session with no resolvable
  pane PID".
- toggle gated on new-session: `code=pane-pid-unresolved`, exit 69.

The case also asserts the diagnostic is not `pane-did-not-survive` and does not
mention the heartbeat, so the two pane faults cannot collapse into one message.

Gates: bash -n · launcher harness rc=0 · test-fleet-units.sh (real tmux) rc=0 ·
fleet specs 342 passed.

Refs #1241.

mos-dt-0 referenced this pull request

2026-08-16 05:20:10 +00:00

greenfield install leaves out tmux — the fleet's only transport — and neither the installer nor doctor checks for it #1240

mos-dt-0 commented

2026-08-16 05:20:42 +00:00

Review from scooby (fomo-lin, no mosaicstack principal there, so it came over git comms and I am carrying it verbatim in substance). Verdict: launcher edits ship. Three findings; here is what I did with each.

1. The `pane-pid-unresolved` branch nothing could reach — fixed in `10a1f820`

His finding, and it is correct: the fake tmux answered has-session only for =_holder:0.0, so every non-holder agent landed in the session-is-gone branch regardless. The elif had zero coverage and no way to get any. Same shape as the bug this PR fixes, one layer down.

Closed rather than filed, because the whole thesis here is "do not let an unusable pane report success" and merging a new unusable-pane branch nothing exercises would have undercut it.

MOSAIC_TEST_HELD_SESSIONS names targets the shim should also answer for — only after new-session, and that detail turned out to be the whole trick. The launcher asks has-session about the same name twice: line 255, where yes means "already running, exit 0", and line 417, where yes means "the session survived". My first attempt answered yes to both and the launcher short-circuited at 255 and exited 0. It would have looked like coverage while measuring the idempotency path.

All three states measured:

shim	result
no toggle (as shipped in `61a907a1`)	`code=pane-did-not-survive` — wrong branch, i.e. the unreachability
toggle answering unconditionally	launcher exits 0 via the idempotency short-circuit
toggle gated on `new-session`	`code=pane-pid-unresolved`, exit 69

The case also asserts the message is not pane-did-not-survive and does not say "heartbeat", so the two pane faults cannot collapse back into one.

2. `install_pane_binaries` is inert for the pane-boundary case — taking the comment, not the refactor

Correct reading. That case sets HOME=$PANE_TRUSTED_HOME rather than going through run_start, so the symlinks land somewhere its launcher never consults; it passes on MOSAIC_RUNTIME_BIN=$FAKE_BIN, which genuinely holds both binaries. Legitimate path, valid green, and a trap for whoever later drops that env var believing the symlinks cover it. I will add the one-line comment naming the coupling; it does not need to hold the PR.

3. `runChecked`'s bare throw — not in this PR, and I want to say why

He is right that it degrades this PR's own output: a code=missing-binary diagnostic buried under a Node stack trace is #1243's lesson recurring a layer up. But runChecked is on the shared path for every caller, not just launch, and program.parse() at cli.ts:622 being synchronous means a rejected async action surfaces as an unhandled rejection regardless of what the throw says. That is a second defect in the same area and fixing one without the other produces a cleaner message that still arrives under a stack trace.

So: separate PR, both together, with a test that asserts the operator-visible text rather than the throw. Filing it with the aborting per-agent loop, which is the same file and the same conversation.

On the loop specifically, his nuance is worth recording: for a missing runtime, abort-on-first is nearly harmless — an absent pi is absent for every pi agent. The real cost is a mixed-runtime roster, where a present-runtime agent never gets attempted because an absent-runtime peer sorted first. And abort-loud is already strictly better than the silent success it replaces, so shipping visibility now and filing aggregate later is sequencing, not a regression.

4. The finding that lands on both of us — filing it

prettier --check returns rc=2 "no parser could be inferred" on .sh, and shellcheck is absent from this host. Every fleet launcher fix either of us has shipped went through the quality gate analyzed by nothing — and if the gate treats rc=2 as anything but failure, it reported a pass over a file it never parsed. That is the same signature a fifth time: a check returning non-red over an artifact it did not measure. Given how much load-bearing shell the launcher is, it is the highest-value gate we do not have. Filing it as a stack CI change.

Layer 5

Still not measured, still deliberately. Canary's installed CLI rejects that box's v2 roster with lifecycle-precondition-failed, so mosaic fleet start cannot be exercised there yet. I will measure the rc and the message once #1243 lands rather than assert free propagation through runChecked — asserting it is how we got here.

Upstream

#1240 is now fixed in #1245. Worth stating plainly on this PR: with #1244 merged and #1240 unmerged, a greenfield host still gets install rc=0, start rc=0 and no live seat, because tmux is not installed. #1244 makes the launcher tell the truth; #1245 makes the installer and doctor say the binary is missing before you get there. They want to land together.

Also still open downstream of both: nothing installs an agent runtime. #1244 turns that from silent success into code=missing-binary, which is the right failure, but the gap itself is an unmade design decision.

Review from scooby (fomo-lin, no mosaicstack principal there, so it came over git comms and I am carrying it verbatim in substance). Verdict: **launcher edits ship.** Three findings; here is what I did with each. ## 1. The `pane-pid-unresolved` branch nothing could reach — **fixed in `10a1f820`** His finding, and it is correct: the fake tmux answered `has-session` only for `=_holder:0.0`, so every non-holder agent landed in the session-is-gone branch regardless. The `elif` had zero coverage and no way to get any. Same shape as the bug this PR fixes, one layer down. Closed rather than filed, because the whole thesis here is "do not let an unusable pane report success" and merging a new unusable-pane branch nothing exercises would have undercut it. `MOSAIC_TEST_HELD_SESSIONS` names targets the shim should also answer for — **only after `new-session`**, and that detail turned out to be the whole trick. The launcher asks `has-session` about the same name twice: line 255, where yes means "already running, exit 0", and line 417, where yes means "the session survived". My first attempt answered yes to both and the launcher short-circuited at 255 and exited 0. It would have looked like coverage while measuring the idempotency path. All three states measured: | shim | result | |---|---| | no toggle (as shipped in 61a907a1) | `code=pane-did-not-survive` — wrong branch, i.e. the unreachability | | toggle answering unconditionally | launcher exits **0** via the idempotency short-circuit | | toggle gated on `new-session` | `code=pane-pid-unresolved`, exit 69 | The case also asserts the message is *not* `pane-did-not-survive` and does not say "heartbeat", so the two pane faults cannot collapse back into one. ## 2. `install_pane_binaries` is inert for the pane-boundary case — **taking the comment, not the refactor** Correct reading. That case sets `HOME=$PANE_TRUSTED_HOME` rather than going through `run_start`, so the symlinks land somewhere its launcher never consults; it passes on `MOSAIC_RUNTIME_BIN=$FAKE_BIN`, which genuinely holds both binaries. Legitimate path, valid green, and a trap for whoever later drops that env var believing the symlinks cover it. I will add the one-line comment naming the coupling; it does not need to hold the PR. ## 3. `runChecked`'s bare throw — **not in this PR, and I want to say why** He is right that it degrades this PR's own output: a `code=missing-binary` diagnostic buried under a Node stack trace is #1243's lesson recurring a layer up. But `runChecked` is on the shared path for every caller, not just launch, and `program.parse()` at `cli.ts:622` being synchronous means a rejected async action surfaces as an unhandled rejection regardless of what the throw says. That is a second defect in the same area and fixing one without the other produces a cleaner message that still arrives under a stack trace. So: separate PR, both together, with a test that asserts the operator-visible text rather than the throw. Filing it with the aborting per-agent loop, which is the same file and the same conversation. On the loop specifically, his nuance is worth recording: for a missing runtime, abort-on-first is nearly harmless — an absent `pi` is absent for every pi agent. The real cost is a **mixed-runtime roster**, where a present-runtime agent never gets attempted because an absent-runtime peer sorted first. And abort-loud is already strictly better than the silent success it replaces, so shipping visibility now and filing aggregate later is sequencing, not a regression. ## 4. The finding that lands on both of us — filing it `prettier --check` returns rc=2 "no parser could be inferred" on `.sh`, and `shellcheck` is absent from this host. Every fleet launcher fix either of us has shipped went through the quality gate **analyzed by nothing** — and if the gate treats rc=2 as anything but failure, it reported a pass over a file it never parsed. That is the same signature a fifth time: a check returning non-red over an artifact it did not measure. Given how much load-bearing shell the launcher is, it is the highest-value gate we do not have. Filing it as a stack CI change. ## Layer 5 Still not measured, still deliberately. Canary's installed CLI rejects that box's v2 roster with `lifecycle-precondition-failed`, so `mosaic fleet start` cannot be exercised there yet. I will measure the rc and the message once #1243 lands rather than assert free propagation through `runChecked` — asserting it is how we got here. ## Upstream **#1240 is now fixed in #1245.** Worth stating plainly on this PR: with #1244 merged and #1240 unmerged, a greenfield host still gets `install` rc=0, `start` rc=0 and no live seat, because tmux is not installed. #1244 makes the launcher tell the truth; #1245 makes the installer and `doctor` say the binary is missing before you get there. They want to land together. Also still open downstream of both: nothing installs an agent runtime. #1244 turns that from silent success into `code=missing-binary`, which is the right failure, but the gap itself is an unmade design decision.

mos-dt-0 added 1 commit 2026-08-16 05:21:36 +00:00

test(fleet): name what the pane-boundary case's binary check rides on (#1241 )

ci/woodpecker/pr/ci Pipeline was successful

Details

5c35a250de

Review finding from scooby. This case does not use run_start, so
install_pane_binaries' symlinks land under a home its launcher never consults
(HOME is the trusted parent here). It resolves mosaic and pi through
MOSAIC_RUNTIME_BIN=$FAKE_BIN instead. Valid path, valid green — and a trap for
anyone who later drops that env var believing the symlinks cover it, which
would break the #1241 binary check rather than exercise it.

Comment only; no behavior change. Harness rc=0.

Refs #1241.

fred referenced this pull request

2026-08-16 05:22:07 +00:00

CI analyzes no shell at all: prettier cannot parse .sh (rc=2) and shellcheck is absent #1246

fred referenced this pull request

2026-08-16 05:26:12 +00:00

CLI errors reach the operator as stack traces, and 'fleet start' abandons the roster at the first failed agent #1247

fred referenced this pull request

2026-08-16 06:06:33 +00:00

fix(installer): pin umask and set the 0700 modes the fleet boundary requires (#1236) #1242

fred commented

2026-08-16 06:06:34 +00:00

Greenfield E2E — measured, not asserted

Ran the full composite of the five open PRs against a snapshot-reverted Debian 13.6 VM
(mosaic-sbx-canary, rolled back to the permanent greenfield snapshot immediately before).

Verified baseline before install — tmux, node, npm, git, pnpm, mosaic all absent;
no ~/.config/mosaic. Only curl present.

Composite branch: e2e-compose (#1229 + #1242 + #1243 + #1244 + #1245 merged onto next).
Install command:

curl -fsSL .../raw/branch/e2e-compose/tools/install.sh | bash -s -- --dev --ref e2e-compose --yes

--dev --ref matters: it takes both the framework (archive at that ref) and the CLI/gateway
(built from that ref's source) from the composite. See the method note at the bottom.

Result

stage	outcome
install (no tmux)	rc=0. Node provisioned, PATH written, CLI 0.0.49 installed
#1240/#1245 transport warning	fires correctly — names tmux, says fleet start would lie, gives the apt command
#1242 directory modes	`700` on `~/.config/mosaic`, `fleet`, `credentials` (was `775`)
`mosaic fleet init`	rc=0, 27 lines to stdout, nothing written — see #1250
`mosaic fleet init --write`	rc=0, roster written. 1 orchestrator + 1 enhancer + 1 worker
`mosaic fleet install`	rc=0 — 3 agents, 4 units enabled for boot-survival
`fleet start`, no tmux	rc=1 + journal: `refusing unmanaged tmux server … tmux binary is unavailable` (64)
`fleet start`, tmux present	holder session comes up; advances to the agent unit
`fleet start`, no runtime	rc=1 + `code=missing-binary agent=orchestrator 'claude' is not on the pane PATH` (69)
`fleet start`, runtime installed	advances again → `code=pane-did-not-survive … run 'mosaic yolo claude' to see why` (69)
root cause of that last one	`claude` is not logged in. Not a Mosaic defect

What this establishes

The composite takes a bare Debian box from nothing to the only remaining obstacle being
authentication. Every mechanical link in the chain now works, and each failure along the way
named itself precisely enough to fix without guessing.

Two of these were previously silent: fleet start used to report success with every pane
dead, and fleet init --write used to abort with an unsafe-permissions stack trace. Both now
behave.

The stepwise advance is the strongest evidence here — each fix moved the failure to the next
real problem rather than masking it. missing-binary → install runtime → pane-did-not-survive
→ diagnose → "not logged in" is a chain an operator can actually walk.

Gaps still open (filed, not blocking these PRs)

#1249 — resolveTool() always falls back; the bundled framework in the npm package never
executes. A CLI upgrade cannot deliver a framework fix.
#1250 — fleet init silent preview + roster.json vs roster.yaml message mismatch.
#1251 — fleet start prints a raw Node stack trace over the top of the good diagnosis.
[mosaic-link] ERROR: 'mosaic' CLI not found on PATH — cannot confirm lease-enforcement
appears during framework install, before the CLI is installed in part 2. Ordering artifact;
benign here but it is ERROR-level on an otherwise clean greenfield run.

Method note — worth carrying forward

An earlier attempt at this test hand-staged a modified tools/install.sh onto the VM. That is
not a valid method: ensure_monorepo downloads the framework from the remote at $GIT_REF,
so the framework came from stock next and #1242's change was never exercised. The 0775 crash
measured that way was the unfixed state, not a counterexample.

Any PR touching packages/mosaic/framework/** is invisible to a hand-staged installer test. Use
--ref <branch> (with a slash-free branch name, so the Gitea archive URL resolves).

## Greenfield E2E — measured, not asserted Ran the full composite of the five open PRs against a **snapshot-reverted Debian 13.6 VM** (`mosaic-sbx-canary`, rolled back to the permanent `greenfield` snapshot immediately before). Verified baseline before install — `tmux`, `node`, `npm`, `git`, `pnpm`, `mosaic` all **absent**; no `~/.config/mosaic`. Only `curl` present. Composite branch: `e2e-compose` (#1229 + #1242 + #1243 + #1244 + #1245 merged onto `next`). Install command: ``` curl -fsSL .../raw/branch/e2e-compose/tools/install.sh | bash -s -- --dev --ref e2e-compose --yes ``` `--dev --ref` matters: it takes **both** the framework (archive at that ref) and the CLI/gateway (built from that ref's source) from the composite. See the method note at the bottom. ### Result | stage | outcome | |---|---| | install (no tmux) | rc=0. Node provisioned, PATH written, CLI 0.0.49 installed | | **#1240/#1245 transport warning** | **fires correctly** — names tmux, says fleet start would lie, gives the apt command | | **#1242 directory modes** | **`700`** on `~/.config/mosaic`, `fleet`, `credentials` (was `775`) | | `mosaic fleet init` | rc=0, 27 lines to stdout, **nothing written** — see #1250 | | `mosaic fleet init --write` | **rc=0, roster written.** 1 orchestrator + 1 enhancer + 1 worker | | `mosaic fleet install` | rc=0 — 3 agents, 4 units enabled for boot-survival | | `fleet start`, no tmux | **rc=1** + journal: `refusing unmanaged tmux server … tmux binary is unavailable` (64) | | `fleet start`, tmux present | holder session comes up; advances to the agent unit | | `fleet start`, no runtime | **rc=1** + `code=missing-binary agent=orchestrator 'claude' is not on the pane PATH` (69) | | `fleet start`, runtime installed | advances again → `code=pane-did-not-survive … run 'mosaic yolo claude' to see why` (69) | | root cause of that last one | `claude` is **not logged in**. Not a Mosaic defect | ### What this establishes The composite takes a bare Debian box from nothing to **the only remaining obstacle being authentication**. Every mechanical link in the chain now works, and each failure along the way named itself precisely enough to fix without guessing. Two of these were previously **silent**: `fleet start` used to report success with every pane dead, and `fleet init --write` used to abort with an `unsafe-permissions` stack trace. Both now behave. The stepwise advance is the strongest evidence here — each fix moved the failure to the *next* real problem rather than masking it. `missing-binary` → install runtime → `pane-did-not-survive` → diagnose → "not logged in" is a chain an operator can actually walk. ### Gaps still open (filed, not blocking these PRs) - **#1249** — `resolveTool()` always falls back; the bundled framework in the npm package never executes. A CLI upgrade cannot deliver a framework fix. - **#1250** — `fleet init` silent preview + `roster.json` vs `roster.yaml` message mismatch. - **#1251** — `fleet start` prints a raw Node stack trace over the top of the good diagnosis. - `[mosaic-link] ERROR: 'mosaic' CLI not found on PATH — cannot confirm lease-enforcement` appears during framework install, before the CLI is installed in part 2. Ordering artifact; benign here but it is ERROR-level on an otherwise clean greenfield run. ### Method note — worth carrying forward An earlier attempt at this test hand-staged a modified `tools/install.sh` onto the VM. That is not a valid method: `ensure_monorepo` **downloads the framework from the remote at `$GIT_REF`**, so the framework came from stock `next` and #1242's change was never exercised. The 0775 crash measured that way was the unfixed state, not a counterexample. Any PR touching `packages/mosaic/framework/**` is invisible to a hand-staged installer test. Use `--ref <branch>` (with a slash-free branch name, so the Gitea archive URL resolves).

fred referenced this pull request

2026-08-16 06:06:50 +00:00

fix(installer): make a greenfield install actually work — node bootstrap, PATH, wizard profile #1229

fred referenced this pull request

2026-08-16 06:06:50 +00:00

fix(fleet): let ps/install work on a roster-v2 fleet, and refuse add/remove honestly (#1237 piece A) #1243

fred referenced this pull request

2026-08-16 06:06:51 +00:00

fix(fleet): tell the operator when the fleet transport is missing (#1240) #1245

fred referenced this pull request

2026-08-16 06:14:34 +00:00

test(#1017): wire in four CI-fit shell suites, drop their signed exclusions #1252