Compare commits

...
Author SHA1 Message Date
fred a972249a2b fleet: put the bootstrapped Node on PANE_PATH (#1256)
ci/woodpecker/pr/ci Pipeline failed
On a host with no system Node, tools/install.sh bootstraps one into
~/.mosaic/node/current/bin and records it in ~/.profile. The fleet unit runs
`env -i ... bash --noprofile --norc`, so ~/.profile is never read — that is
deliberate — and _build_runtime_bin_prefix did not name the directory itself.
Runtime binaries are `#!/usr/bin/env node`, so the pane resolved `mosaic` and
then died on `env: 'node': No such file or directory` after an install that
reported success. Measured on a greenfield VM.

The existing `npm config get prefix` branch cannot cover it: that reports a
package prefix (~/.npm-global), never a Node runtime directory.

The test case asserts the property rather than the string — it runs the pane for
real with a Node-shebang `mosaic` and requires the pane to have executed. A PATH
substring check would pass on a fix that put the directory in the wrong position.
Red without the launcher change, green with it, rest of the suite unaffected.
2026-08-16 15:02:04 -05:00
fred 476db12b92 Merge pull request 'fix(fleet): tell the operator when the fleet transport is missing (#1240)' (#1245) from fix/1240-fleet-transport-check into next
ci/woodpecker/push/publish Pipeline failed
Reviewed by scooby via git comms (terminal ACK 32d986). Merge directed by Jason 2026-08-16. Conflicts from #1229 and #1252 resolved on the branch by a pi seat; pre-merge gate verified: tools/install.sh=5d28f773, framework/install.sh=1578c33b, test:framework-shell=48 links with both #1252 and #1245 suites present.
2026-08-16 18:12:33 +00:00
fred 5198c3f198 merge next into fix/1240-fleet-transport-check
ci/woodpecker/pr/ci Pipeline failed
Resolves conflicts from #1229 (tools/install.sh node provisioning) and #1252
(package.json test:framework-shell). tools/install.sh resolved to the reviewed
composite blob 5d28f773; package.json resolved as a union so both #1252's four
suites and #1245's transport-check suite run (48 links).
2026-08-16 13:11:31 -05:00
fred 19ac0a02d7 Merge pull request 'test(#1017): wire in four CI-fit shell suites, drop their signed exclusions' (#1252) from fix/1017-wire-start-agent-session into next
ci/woodpecker/push/publish Pipeline was canceled
Reviewed by scooby via git comms (no mosaicstack principal on fomo-lin; review is the comms record, terminal ACK 32d986). Merge directed by Jason 2026-08-16. Part of the five-PR greenfield composite verified E2E on two independent bare boxes.
2026-08-16 18:06:53 +00:00
fred 6d9387c857 Merge pull request 'fix(fleet): fail the agent launcher when the pane cannot survive (#1241)' (#1244) from fix/1241-launch-failure-visible into next
ci/woodpecker/push/publish Pipeline was canceled
Reviewed by scooby via git comms (no mosaicstack principal on fomo-lin; review is the comms record, terminal ACK 32d986). Merge directed by Jason 2026-08-16. Part of the five-PR greenfield composite verified E2E on two independent bare boxes.
2026-08-16 18:06:50 +00:00
fred 14cb9c6a1e Merge pull request 'fix(fleet): let ps/install work on a roster-v2 fleet, and refuse add/remove honestly (#1237 piece A)' (#1243) from fix/1237-fleet-v2-dispatch into next
ci/woodpecker/push/publish Pipeline was canceled
Reviewed by scooby via git comms (no mosaicstack principal on fomo-lin; review is the comms record, terminal ACK 32d986). Merge directed by Jason 2026-08-16. Part of the five-PR greenfield composite verified E2E on two independent bare boxes.
2026-08-16 18:06:35 +00:00
fred b5b322f80d Merge pull request 'fix(installer): pin umask and set the 0700 modes the fleet boundary requires (#1236)' (#1242) from fix/1236-installer-dir-modes into next
ci/woodpecker/push/publish Pipeline was canceled
Reviewed by scooby via git comms (no mosaicstack principal on fomo-lin; review is the comms record, terminal ACK 32d986). Merge directed by Jason 2026-08-16. Part of the five-PR greenfield composite verified E2E on two independent bare boxes.
2026-08-16 18:06:21 +00:00
jason.woltje e4674709be Merge pull request 'fix(installer): make a greenfield install actually work — node bootstrap, PATH, wizard profile' (#1229) from fix/installer-path-and-node into next
ci/woodpecker/push/publish Pipeline was canceled
Reviewed-on: #1229
2026-08-16 18:01:07 +00:00
fred c56483eb1b test(#1017): wire in four CI-fit shell suites, drop their signed exclusions
ci/woodpecker/pr/ci Pipeline failed
check-test-enumeration.sh signed four suites as 'likely CI-fit; #1017 burndown'.
Measured all four: each passes standing alone, and each still passes with tmux
removed from PATH entirely (test-start-agent-session.sh writes its own tmux shim
into a fake bin dir, so it never needed the real binary).

Red-first: removing the four exclusion lines makes the guard report exactly four
UNENUMERATED failures. Appending the four to test:framework-shell returns it to
OK, with in-population enumerated going 32 -> 36 and signed exclusions 19 -> 15.

Refs #1017
2026-08-16 01:14:03 -05:00
fred 5c35a250de test(fleet): name what the pane-boundary case's binary check rides on (#1241)
ci/woodpecker/pr/ci Pipeline was successful
Review finding from scooby. This case does not use run_start, so
install_pane_binaries' symlinks land under a home its launcher never consults
(HOME is the trusted parent here). It resolves mosaic and pi through
MOSAIC_RUNTIME_BIN=$FAKE_BIN instead. Valid path, valid green — and a trap for
anyone who later drops that env var believing the symlinks cover it, which
would break the #1241 binary check rather than exercise it.

Comment only; no behavior change. Harness rc=0.

Refs #1241.
2026-08-16 00:21:13 -05:00
fred 10a1f82031 test(fleet): cover the pane-pid-unresolved branch this PR shipped (#1241)
ci/woodpecker/pr/ci Pipeline was canceled
Review finding from scooby: this PR added a failure branch the harness
structurally could not reach. The fake tmux answered `has-session` only for
`=_holder:0.0`, so every non-holder agent landed in the session-is-gone branch
no matter what — the `elif` (tmux still reports the session, no pane PID after
the retries) had zero coverage and no way to get any.

That is the same shape as the bug this PR exists to fix, one layer down: a code
path shipped green where the gate that should measure it cannot. Less severe,
because the branch fails closed at exit 69 rather than reporting success — but
"the harness can't reach it" is the sentence that precedes the next silent
regression, so it gets closed here rather than filed.

`MOSAIC_TEST_HELD_SESSIONS` lets a case name targets the shim should also
answer for. It answers them only AFTER `new-session`, and that detail is the
whole trick: the launcher asks `has-session` about the same name twice — once
at line 255 where a yes means "already running, exit 0", and once at 417 where
a yes means "the session survived". A shim answering yes to both short-circuits
at the first and never reaches the branch under test. It would have looked like
coverage while measuring the idempotency path.

Both failure modes were measured, not reasoned about:
- toggle absent (the old shim): `code=pane-did-not-survive` — the case lands on
  the wrong branch, which is exactly the unreachability being reported.
- toggle answering unconditionally: launcher exits 0 via the idempotency
  short-circuit — "launcher reported success over a session with no resolvable
  pane PID".
- toggle gated on new-session: `code=pane-pid-unresolved`, exit 69.

The case also asserts the diagnostic is not `pane-did-not-survive` and does not
mention the heartbeat, so the two pane faults cannot collapse into one message.

Gates: bash -n · launcher harness rc=0 · test-fleet-units.sh (real tmux) rc=0 ·
fleet specs 342 passed.

Refs #1241.
2026-08-16 00:19:25 -05:00
fred b61789fe26 fix(fleet): tell the operator when the fleet transport is missing (#1240)
ci/woodpecker/pr/ci Pipeline was successful
`mosaic fleet --help` reads "Manage the local Mosaic tmux fleet" and every
roster the CLI scaffolds sets `transport: tmux`, but neither `tools/install.sh`
nor `tools/_scripts/mosaic-doctor` contained the string "tmux" at all. A
greenfield host therefore came out of the installer able to install a fleet,
start a fleet, and run no seat, with `mosaic fleet ps` as the operator's first
and only signal.

Measured on mosaic-sbx-dev (Debian, no tmux, framework installed): `mosaic-doctor`
reported 11 warnings and not one of them named the reason no seat could launch.

The installer gets a warning, not a `require_cmd` hard failure: tmux is required
by the fleet, not by mosaic. Hosts that install this to run `mosaic claude` and
never scaffold a roster are common, and failing their install over a binary they
do not need would be wrong. The check runs in `--check` mode too — "what is the
state of this host" is the question `--check` is asked.

Both checks read the roster's own `transport:` rather than assuming tmux, so a
host declaring something else is pointed at the binary it actually needs instead
of at the wrong package.

The two implementations are deliberately parallel and each carries a comment
pointing at the other. They are separate because the installer must answer this
before the framework's own scripts are guaranteed to be on disk. One harness
drives BOTH from the shipped text — the functions are extracted from the scripts
by awk rather than copied — so the pair cannot drift silently, and the test
cannot keep passing after the shipped copy changes.

The harness is wired into `test:framework-shell`. Without that it would have
tripped the #1017 enumeration guard as UNENUMERATED, which is the guard doing
its job: a check nothing runs is not a check.

Evidence:
- red: the harness fails against origin/next ("could not extract
  fleet_declared_transport"); `grep -ci tmux` on both files at origin/next = 0.
- green on real hosts, all four branches:
  - dev (no tmux, no roster)  -> WARN naming tmux, points at `mosaic fleet init`
  - dev (no tmux, v2 roster)  -> WARN naming the roster, points at `mosaic fleet start`
  - dev installer --check     -> WARN saying start "reports success and no seat comes up"
  - canary (tmux present)     -> `[OK] Fleet transport available: tmux` under --verbose,
                                 silent by default (pass() is verbose-gated), installer silent
- harness green on node:24-alpine/busybox, the CI base image.
- `bash -n` x3, `pnpm typecheck` 45/45, fleet specs 342 passed,
  enumeration guard OK, its self-test OK, prettier clean.

Refs #1240. Upstream of #1237/#1243 and #1241/#1244: a correct fix for either of
those still leaves this host with no live seat.
2026-08-16 00:15:50 -05:00
fred 61a907a12f fix(fleet): fail the agent launcher when the pane cannot survive (#1241)
ci/woodpecker/pr/ci Pipeline was successful
`mosaic fleet start` returned 0 over three dead panes. The launcher knew,
and said the wrong thing at the wrong severity to the wrong layer.

The pane runs `mosaic yolo <runtime>` under PANE_PATH with a cleared
environment. When that binary is absent the pane dies in under a second,
tmux destroys the session, and the diagnostic goes with it. The launcher
then found no PANE_PID, printed a WARNING about the *heartbeat sidecar*,
and exited 0 — so systemd logged "Finished ... successfully" and
`fleet start` reported success. `fleet ps` was the only component telling
the truth.

Two changes, both in start-agent-session.sh:

1. Before any effect, resolve `mosaic` and the roster's runtime against
   PANE_PATH — the pane's own view of the path, not the launcher's.
   `mosaic yolo <runtime>` calls checkRuntime(runtime) and looks for a
   binary named exactly like the runtime, so this asks the same question
   the pane will ask a moment later, while an operator can still see the
   answer. Absent binary -> exit 69, code=missing-binary, no session
   created.

2. Replace the dead-pane WARNING+exit-0. An absent session one second
   after new-session is a runtime that died on startup, not a heartbeat
   problem -> exit 69, code=pane-did-not-survive, with the command to run
   by hand to see why. A present session with no pane PID after five
   attempts -> code=pane-pid-unresolved. Neither branch kills the
   session; destroying a possibly-live pane on a guess is worse than
   leaving it for inspection.

Exit 69 (EX_UNAVAILABLE) is deliberate: the 64s already in this file mean
the projection was bad, and here the data is fine and the host is not
ready. Callers separate the cases by `code=`, the same way fail_env's
codes share 64.

This propagates for free. `fleet start` calls runChecked() for the holder
and each agent, and runChecked throws on non-zero, so layers 4 and 5 stop
lying without a TypeScript change. Two adjacent defects are left for a
follow-up issue rather than widened into this diff: the per-agent loop
aborts on the first failure instead of attempting all and reporting an
aggregate, and runChecked's bare throw surfaces the launcher's message
under a Node unhandled-rejection stack trace because program.parse() is
synchronous.

Tests:

- test-start-agent-session.sh gains three cases: `mosaic` absent from the
  pane path, the runtime absent from the pane path, and a pane that does
  not survive. Each was verified individually red against the unmodified
  origin/next launcher.
- The two cases asserting a valid launch now supply a pane PID. Until now
  the suite's one success path was itself a dead pane the launcher
  reported as fine.
- The harness fakes `npm` so PANE_PATH stops depending on whatever the
  host has installed, and fails loudly if the host provides `mosaic` or
  `pi` in the system path, where the missing-binary cases would not be
  measurable at all.
- test-fleet-units.sh gains a `pi` shim in its runtime bin. The real-tmux
  harness named `pi` in its roster and never installed it; the new
  preflight caught it.

Refs #1241
2026-08-15 23:56:53 -05:00
fred 6f5b4c3dc1 fix(fleet): restore ConditionPathExists dropped by my own red-check
ci/woodpecker/pr/ci Pipeline was successful
Self-inflicted and worth recording rather than quietly amending.

To prove the new tests were red without the fix I ran
`git checkout origin/next -- <fleet.ts> <[email protected]>`. That writes
the *index*, not just the working tree. Copying my versions back afterwards
restored the working tree only, so the unit file sat staged-as-origin/next and
modified-in-tree, and the next commit (67f5014c) committed the index — silently
removing the ConditionPathExists line that 463745e3 had added.

Nothing caught it. The spec reads the file from the working tree, so it stayed
10/10 green against a HEAD that no longer had the guard. Found by reading
`git status` after the push, not by any gate.

Verified by content, not by assumption:
  origin/next  0 occurrences
  463745e3     1
  67f5014c     0   <- the regression
  this commit  1

Refs #1237
2026-08-15 23:35:54 -05:00
fred 67f5014cc0 fix(fleet): refuse v2 add/remove cleanly, and pin the Condition's effect
Two follow-ups from the canary red->green run and scooby's review.

1. The v2 refusal in `add`/`remove` was a bare `throw`, which reaches the CLI
   top level uncaught and prints the guidance under a Node stack trace. The
   message *is* the point of the refusal, so it now goes through
   `command.error()` — the same clean path the roster-config error uses.
   Caught on canary, not in review: the unit tests asserted the message text
   and passed either way.

2. The unit-template test asserted only that ConditionPathExists is present.
   Presence is not effect. Added two tests for the parts that can drift in
   code while that assertion still passes: the condition resolving to exactly
   the file the fleet writes (%h/%i rendered against a real install), and the
   launcher genuinely failing on an absent generated env (exit 64,
   `missing-file`) — which is what makes the condition load-bearing rather
   than decorative.

systemd is not available in the suite, so the effect itself was measured on
canary (2026-08-16), roster v2 generation 3:

  with the condition:    start rc=0, Result=success, ConditionResult=no,
                         journal "skipped, unmet condition check"
  condition removed by
  drop-in, nothing else: start rc=1, Result=exit-code, ExecMainStatus=64,
                         unit failed, "agent environment rejected: missing-file"

Canary red->green for the three commands, same v2 roster, side by side:

  fleet ps               0.0.50-next.2413 rc=1  ->  branch rc=0 (3 agents listed)
  fleet install          0.0.50-next.2413 rc=1  ->  branch rc=0
  fleet remove <name>    0.0.50-next.2413 rc=1  ->  branch rc=1, refusal naming
                                                    delete + apply

All three previously failed with "Fleet roster has unknown field(s):
generation." The #791 negative was measured too: the six existing
*.env.generated files were untouched by `install` (mtimes 20+ minutes older
than the run).

Gates: typecheck 0, eslint 0, prettier clean, fleet specs 382 passed, new spec
10/10 with the fix and 9/10 red against origin/next (the 10th passes there for
an unrelated reason and is annotated as such). Full suite: only
mutator-gate.acceptance.spec.ts fails, pre-existing on origin/next.

Still true and still worth saying: a correct fix here shows install rc=0 and
start rc=0 and STILL no live seat. #1240 (tmux absent) is upstream, #1241
(start reports lifecycle-complete over dead panes) and the missing agent
runtime are downstream.

Refs #1237
Reviewed-by: scooby (by git comms; cannot file a Gitea review from fomo-lin)
2026-08-15 23:34:35 -05:00
fred 463745e314 fix(#1237): let ps/install work on a roster-v2 fleet, and refuse add/remove honestly
On a roster-v2 fleet, `ps`, `install`, `install-systemd`, `add` and `remove`
all failed in the v1 parser. The consequence was that a greenfield v2 box could
never get its unit templates placed, so nothing downstream could start.

The read-only commands get a narrow version-agnostic view of the roster
(version, socket name, holder session, and per agent name/alias/runtime).
This is deliberately not a v2 -> v1 downshift. A downshifted FleetRoster would
be accepted by generateAgentEnvValues, which would make a third writer of
fleet/agents/<name>.env.generated through the v1 mapping and break the #791
single-SSOT invariant that projectRosterV2AgentGeneratedEnv is documented to
hold. The view is too small to write a roster or an env file back from, so that
misuse is unavailable rather than merely discouraged.

So on a v2 roster `install` places the tool files and the unit templates,
enables the units, and writes no generated env at all. Env belongs to `apply`
and `regen`, both already v2-native.

That change alone would have traded an init-time failure for a boot-time one.
`install` enables mosaic-agent@<name>.service (WantedBy=default.target) without
starting it, so a reboot between `install` and the first `apply` would run
ExecStart against an absent env file and fail every seat unit, further from its
cause. The unit template now carries

  ConditionPathExists=%h/.config/mosaic/fleet/agents/%i.env.generated

which skips an enabled-but-unconfigured unit cleanly and starts it on the next
start once the reconciler has written env. On v1 it is a no-op, since v1
`install` writes env itself. Found in review by scooby.

`add` and `remove` are not routed to `create` and `delete`. They are different
operations: the v1 pair edits the roster and drives systemd, the v2 pair is
documented as changing desired state without runtime actions. `add` also
collects four fields where a v2 agent requires eleven, so routing it would mean
inventing an operator's provider, alias, reasoning and tool policy. On v2 both
now fail with the real two-step sequence instead.

Tests: 8 new, 7 of which are red before this change. Includes the greenfield
case scooby asked for — `ps` on a fresh v2 install with nothing running is rc=0
and lists every agent stopped, since that is the command an operator runs to
find out why there is no seat.

Note for anyone verifying this: a correct fix here shows `install` rc=0 and
`start` rc=0 and still no live seat. #1240 (tmux absent) is upstream, #1241
(start reports lifecycle-complete over dead panes) and the missing agent
runtime are downstream. A dead pane after this change is not a regression here.

Refs #1237, #791, #1240, #1241
2026-08-15 23:24:24 -05:00
fredandClaude Opus 5 03eda02c20 fix(installer): warn on a failed credentials/ chmod instead of swallowing it
ci/woodpecker/pr/ci Pipeline was successful
scooby's review flag 1 on #1242. The other three chmods warn; this one was
`|| true`. It is the one directory holding secrets, so a chmod that fails
silently there is the failure most worth a line in the output.

Comment-and-warn only. No behaviour change on the success path.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WYgWocp36goy8hj2ui6ps1
2026-08-15 22:40:47 -05:00
fredandClaude Opus 5 3b4055017e fix(installer): pin umask and set the 0700 modes the fleet boundary requires (#1236)
ci/woodpecker/pr/ci Pipeline was canceled
A greenfield install cannot run `mosaic fleet init --write`. It fails with
`unsafe-permissions` on an unnamed `(directory)` and an unhandled Node throw,
and every mutating `mosaic fleet` command fails the same way. Measured on a
reverted-to-greenfield sandbox VM at CLI 0.0.50-next.2413: `~/.config/mosaic`,
`fleet/` and `credentials/` all land at 0775, and 1735 directories under the
framework root carry `mode & 022`.

Two independent causes, and fixing either one alone leaves it broken.

1. The installer inherited the caller's umask. Debian/Ubuntu ship 002, so every
   `mkdir -p` produced 0775. Fedora/RHEL ship 022 and produced 0755. The
   product therefore worked or did not depending on the operator's login shell,
   with nothing in the install output distinguishing the two. 022 is already
   what this script assumes it produces — `make_durable_snapshot` restores the
   ambient umask specifically so "every later sync copy and new framework dir"
   gets 0644/0755 — so pin it rather than inherit it.

2. Even at a correct 0755, three directories are rejected. The fleet code
   guards its managed paths with two masks in two languages:
   `assertPrivateManagedDirectory` (fleet-reconciler.js, `mode & 0o077`) covers
   MOSAIC_HOME and `fleet/` and runs before the roster lock is taken;
   `assert_private_directory` (tools/fleet/start-agent-session.sh, `mode & 077`)
   covers `fleet/agents` and runs before a pane is spawned. Their laxer
   siblings use `mode & 0o022` and accept 0755. The strict mask wins, so the
   installer states 0700 outright instead of hoping a umask implies it.

The `find -perm /022 -exec chmod go-w` sweep repairs a tree installed before
this change, which the umask alone cannot reach. It strips group/other WRITE
only — never read or execute — and is scoped to directories, so it corrects the
boundary violation without changing who may traverse or read anything. It is
not sufficient for `fleet/agents`: stripping write from 0755 yields 0750 and
`mode & 077` is still non-zero, which is why that path gets its own chmod.

Reported as #1236. The `fleet/agents` half was found by scooby reading
start-agent-session.sh; the umask framing is theirs too — my first report
blamed the distro rather than the umask.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WYgWocp36goy8hj2ui6ps1
2026-08-15 22:29:48 -05:00
12 changed files with 1092 additions and 15 deletions
+58
View File
@@ -35,6 +35,18 @@ SOURCE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
TARGET_DIR="${MOSAIC_HOME:-$HOME/.config/mosaic}"
INSTALL_MODE="${MOSAIC_INSTALL_MODE:-prompt}"
# Normalize the ambient umask so directory modes are a property of the installer
# and not of whatever shell invoked it (#1236). Debian/Ubuntu ship umask 002, so
# every `mkdir -p` below yielded 0775 — and the fleet env boundary rejects any
# managed directory with `mode & 0o022`, which made `mosaic fleet init --write`
# impossible on a stock install of those distros. Fedora/RHEL ship 022 and did
# not trip it, so the product worked or did not depending on the operator's
# login shell. 022 is what this script already assumes it produces: see the
# umask note in make_durable_snapshot, which restores to the ambient value
# precisely so "every later sync copy and new framework dir" gets 0644/0755.
# Now that value is 022 rather than whatever was inherited.
umask 022
# Deliberately parsed from "$@" (a real, explicit, per-invocation argument) —
# never an environment variable — so this opt-out can never sit silently
# inherited in a shell profile. See #869 Point-1 C2.
@@ -696,6 +708,52 @@ sync_framework
mkdir -p "$TARGET_DIR/memory"
mkdir -p "$TARGET_DIR/credentials"
# Three directories must be 0700, not merely not-group-writable (#1236).
# The fleet code guards them with two different masks in two different
# languages, and the strict one wins:
#
# assertPrivateManagedDirectory (fleet-reconciler.js, `mode & 0o077`)
# -> MOSAIC_HOME and MOSAIC_HOME/fleet, checked before the roster lock is
# taken, so every mutating `mosaic fleet` command dies at 0755.
# assert_private_directory (tools/fleet/start-agent-session.sh, `mode & 077`)
# -> MOSAIC_HOME/fleet/agents, checked before a pane is ever spawned.
#
# Their laxer siblings (`mode & 0o022`) accept 0755, which is why normalizing
# the umask above is necessary and not sufficient — a correct umask-022 install
# still produces 0755 and still cannot run `mosaic fleet init --write`. Say the
# strict modes outright rather than inferring them from a umask.
#
# Only these. The rest of the tree is content, stays 0755, and is only ever
# reached by the 0o022 checks, which 0755 satisfies.
chmod 700 "$TARGET_DIR" 2>/dev/null || \
warn "Could not set 0700 on $TARGET_DIR — 'mosaic fleet' mutations will fail as unsafe-permissions."
if [[ -d "$TARGET_DIR/fleet" ]]; then
chmod 700 "$TARGET_DIR/fleet" 2>/dev/null || \
warn "Could not set 0700 on $TARGET_DIR/fleet — 'mosaic fleet' mutations will fail as unsafe-permissions."
fi
# fleet/agents does not exist on a first install — the CLI creates it 0700 on
# demand. It is chmod'd here for the UPGRADE case: a tree built under umask 002
# has it at 0775, and the repair sweep below cannot rescue it, because stripping
# group/other write from 0755 leaves 0750 and `mode & 077` is still non-zero.
if [[ -d "$TARGET_DIR/fleet/agents" ]]; then
chmod 700 "$TARGET_DIR/fleet/agents" 2>/dev/null || \
warn "Could not set 0700 on $TARGET_DIR/fleet/agents — agent sessions will fail to start as unsafe-permissions."
fi
# credentials/ holds secrets and was never meant to be group-readable either.
# It is not on the fleet boundary, so a failure here breaks nothing — but it is
# the one directory where a silently-failed chmod leaves secrets group-readable,
# which is precisely the failure worth a line in the output.
chmod 700 "$TARGET_DIR/credentials" 2>/dev/null || \
warn "Could not set 0700 on $TARGET_DIR/credentials — stored secrets may be readable by other users on this host."
# Repair an existing tree. The umask above only governs directories this run
# creates, so a host installed under umask 002 before this fix keeps its 0775
# dirs through every upgrade and stays broken. Strips group/other WRITE only —
# never read or execute — so it can repair the boundary violation without
# changing who can traverse or read anything. Scoped to directories: file modes
# are the manifest's business, not this fix's.
find "$TARGET_DIR" -type d -perm /022 -exec chmod go-w {} + 2>/dev/null || true
# Reconcile contract files from defaults/ into the framework root: framework-owned
# files (CONSTITUTION/AGENTS/STANDARDS) are overwritten every upgrade (a divergent
# copy is backed up once); user-seeded files (TOOLS) are written on first install only.
@@ -4,6 +4,14 @@ Documentation=https://git.mosaicstack.dev/mosaicstack/stack
Requires=mosaic-tmux-holder.service
After=mosaic-tmux-holder.service
PartOf=mosaic-tmux-holder.service
# Do not attempt a seat before its generated env exists. `install` enables this
# unit (WantedBy=default.target) but on a roster-v2 fleet the reconciler owns the
# generated env, so between `install` and the first `apply`/`regen --write` there
# is a boot window where ExecStart would run against an absent env file and the
# launcher would fail the unit. A skipped unit is the honest state for "enabled
# but not yet configured"; systemd re-evaluates the condition on every start, so
# the seat comes up on the next start once the reconciler has written env.
ConditionPathExists=%h/.config/mosaic/fleet/agents/%i.env.generated
[Service]
Type=oneshot
@@ -128,6 +128,14 @@ EOF
sleep 30
EOF
chmod 700 "$AGENT_BIN/mosaic"
# The launcher resolves the roster's runtime against PANE_PATH before it
# spawns anything (#1241), so the runtime this projection names has to be
# present here even though the fake `mosaic` above never execs it.
cat > "$AGENT_BIN/pi" <<'EOF'
#!/bin/sh
sleep 30
EOF
chmod 700 "$AGENT_BIN/pi"
server_environment_before=$(tmux -L "$TEST_SOCKET" show-environment -g | sort)
server_sessions_before=$(tmux -L "$TEST_SOCKET" list-sessions | sort)
if /usr/bin/env -i HOME="$HOLDER_HOME" PATH=/usr/bin:/bin MOSAIC_HOME="$AGENT_HOME" \
@@ -225,6 +225,54 @@ else
warn "mosaic-ensure-sequential-thinking helper missing"
fi
# Fleet transport binary (#1240).
#
# `mosaic fleet --help` reads "Manage the local Mosaic tmux fleet" and every
# roster the CLI scaffolds sets `transport: tmux`, but nothing in the install
# path provides tmux and, until now, nothing here noticed it was absent. On a
# greenfield host that produced a fleet which installed clean, started clean,
# and had no live seat; `mosaic fleet ps` was the operator's first and only
# signal that anything was wrong.
#
# The roster's own `transport:` is read rather than assumed, so a host that
# declares something other than tmux is told about the binary it actually
# needs. Absent a roster the check still runs — `mosaic fleet init` will
# scaffold a tmux fleet on this host, and finding out beforehand is the point.
#
# `tools/install.sh` carries a deliberately parallel check at the end of its
# summary. The two are separate because the installer must be able to say this
# before the framework's own scripts are guaranteed to be on disk; keep their
# wording in step.
fleet_declared_transport() {
local roster="$MOSAIC_HOME/fleet/roster.yaml"
local declared=""
if [[ -f "$roster" ]]; then
declared="$(sed -n 's/^[[:space:]]*transport:[[:space:]]*//p' "$roster" | head -1 |
tr -d '"'\''' | tr -d '\r' | awk '{print $1}')"
fi
printf '%s\n' "${declared:-tmux}"
}
check_fleet_transport() {
local transport
transport="$(fleet_declared_transport)"
if command -v "$transport" >/dev/null 2>&1; then
pass "Fleet transport available: $transport"
return
fi
if [[ -f "$MOSAIC_HOME/fleet/roster.yaml" ]]; then
warn "Fleet transport '$transport' is not installed — this host has a roster and no seat can launch. Install it (e.g. sudo apt-get install -y $transport), then 'mosaic fleet start'."
else
warn "Fleet transport '$transport' is not installed — 'mosaic fleet' cannot run seats here. Install it (e.g. sudo apt-get install -y $transport) before 'mosaic fleet init'."
fi
}
check_fleet_transport
# Legacy migration surfaces should no longer contain symlink trees.
legacy_paths=(
"$HOME/.claude/agent-guides"
@@ -0,0 +1,215 @@
#!/usr/bin/env bash
# Covers the #1240 fleet-transport checks in `mosaic-doctor` and in
# `tools/install.sh`.
#
# Both checks answer the same question — "can a seat actually launch on this
# host?" — from two different places, because the installer has to be able to
# answer it before the framework's own scripts are guaranteed to be on disk.
# Two implementations of one rule is exactly the shape that drifts, so this
# harness drives BOTH, in one file, from the same table of cases.
#
# The functions are extracted from the shipped scripts rather than copied here.
# A test that carries its own copy of the logic is a test that keeps passing
# after the shipped copy changes — the failure mode this whole change is about.
# Extraction is by exact function header and a closing brace in column one; if
# either script is reshaped so that stops matching, the extraction yields
# nothing and this fails loudly instead of silently measuring an empty string.
set -euo pipefail
SCRIPT_DIR=$(cd -- "$(dirname -- "$0")" && pwd)
DOCTOR="$SCRIPT_DIR/mosaic-doctor"
# framework/tools/_scripts -> framework/tools -> framework -> mosaic -> packages -> repo
INSTALLER=$(cd -- "$SCRIPT_DIR/../../../../.." && pwd)/tools/install.sh
fail() {
echo "FAIL: $*" >&2
exit 1
}
[ -f "$DOCTOR" ] || fail "missing mosaic-doctor at $DOCTOR"
[ -f "$INSTALLER" ] || fail "missing install.sh at $INSTALLER"
ROOT=$(mktemp -d)
trap 'rm -rf "$ROOT"' EXIT
# The cases below run with PATH set to a directory that deliberately does not
# contain a shell, and a PATH assignment on a command also governs how that
# command is looked up — so bash has to be named absolutely or it becomes the
# thing that is missing.
BASH_BIN=$(command -v bash) || fail "host is missing 'bash'"
# A PATH containing exactly the utilities these functions use and nothing else.
# The absent-transport cases are only meaningful on a PATH where the transport
# is genuinely unresolvable, and this host (like most) has tmux in /usr/bin —
# so the system path cannot be part of the path under test.
FAKE_BIN="$ROOT/bin"
mkdir -p "$FAKE_BIN"
for utility in sed head tr awk; do
utility_path=$(command -v "$utility") || fail "host is missing '$utility'"
ln -s "$utility_path" "$FAKE_BIN/$utility"
done
if PATH="$FAKE_BIN" command -v tmux >/dev/null 2>&1; then
fail "'tmux' is resolvable on the minimal test path; absent-transport cases are not measurable"
fi
# Extract a function by its exact header, up to a closing brace in column one.
extract_function() {
local source_file="$1"
local function_name="$2"
local destination="$3"
awk -v name="$function_name" '
$0 == name "() {" { collecting = 1 }
collecting { print }
collecting && $0 == "}" { exit }
' "$source_file" > "$destination"
grep -qF "$function_name() {" "$destination" ||
fail "could not extract '$function_name' from $source_file — has it been renamed or reshaped?"
# An unterminated extraction would be a syntax error the moment it is sourced,
# but saying so here names the cause instead of leaving a bash parse error.
bash -n "$destination" ||
fail "extracted '$function_name' does not parse; the closing brace was probably not found"
}
extract_function "$DOCTOR" fleet_declared_transport "$ROOT/doctor-declared.sh"
extract_function "$DOCTOR" check_fleet_transport "$ROOT/doctor-check.sh"
extract_function "$INSTALLER" check_fleet_transport "$ROOT/installer-check.sh"
# Build a MOSAIC_HOME, optionally with a roster declaring a transport.
make_home() {
local home="$ROOT/$1"
local declared="${2-}"
rm -rf "$home"
mkdir -p "$home"
if [ -n "$declared" ]; then
mkdir -p "$home/fleet"
cat > "$home/fleet/roster.yaml" <<EOF
version: 2
generation: 1
transport: $declared
agents: []
EOF
fi
printf '%s\n' "$home"
}
# Run the doctor's check against a given home and path, capturing which
# reporter the check chose. The real `pass` prints only under `--verbose` and
# the real `warn` always prints; these stubs make both unconditional on
# purpose, because what is under test is the severity the check selects, not
# whether the default verbosity happens to show it. A check that warned where
# it should pass would otherwise be invisible here.
run_doctor_check() {
local home="$1"
local path="$2"
MOSAIC_HOME="$home" PATH="$path" "$BASH_BIN" --noprofile --norc -c '
set -euo pipefail
warn() { echo "[WARN] $*"; }
pass() { echo "[OK] $*"; }
MOSAIC_HOME="$1"
source "$2"
source "$3"
check_fleet_transport
' _ "$home" "$ROOT/doctor-declared.sh" "$ROOT/doctor-check.sh" 2>&1
}
run_installer_check() {
local home="$1"
local path="$2"
MOSAIC_HOME="$home" PATH="$path" "$BASH_BIN" --noprofile --norc -c '
set -euo pipefail
warn() { echo "[WARN] $*"; }
C="" RESET=""
MOSAIC_HOME="$1"
source "$2"
check_fleet_transport
' _ "$home" "$ROOT/installer-check.sh" 2>&1
}
# A transport that exists. Named tmux because that is what the default roster
# declares; the binary never runs, it only has to resolve.
PRESENT_BIN="$ROOT/present-bin"
mkdir -p "$PRESENT_BIN"
printf '#!/usr/bin/env bash\nexit 0\n' > "$PRESENT_BIN/tmux"
chmod +x "$PRESENT_BIN/tmux"
PATH_WITH_TMUX="$PRESENT_BIN:$FAKE_BIN"
# ── absent, no roster ────────────────────────────────────────────────────────
# Nothing has been configured yet, so the honest thing to point at is `init`.
home=$(make_home no-roster)
output=$(run_doctor_check "$home" "$FAKE_BIN")
echo "$output" | grep -qF '[WARN]' || fail "doctor did not warn when tmux was absent"
echo "$output" | grep -qF 'tmux' || fail "doctor warning did not name the transport"
echo "$output" | grep -qF 'mosaic fleet init' || fail "doctor did not point a rosterless host at init"
output=$(run_installer_check "$home" "$FAKE_BIN")
echo "$output" | grep -qF '[WARN]' || fail "installer did not warn when tmux was absent"
echo "$output" | grep -qF 'reports success and no seat comes up' ||
fail "installer warning did not say what the missing transport actually breaks"
# ── absent, roster present ───────────────────────────────────────────────────
# A configured fleet that cannot launch is a stronger statement than a
# hypothetical one, and the message says so.
home=$(make_home with-roster tmux)
output=$(run_doctor_check "$home" "$FAKE_BIN")
echo "$output" | grep -qF '[WARN]' || fail "doctor did not warn with a roster present and tmux absent"
echo "$output" | grep -qF 'roster' || fail "doctor did not mention the roster it found"
echo "$output" | grep -qF 'mosaic fleet start' || fail "doctor did not point a configured host at start"
# ── present ──────────────────────────────────────────────────────────────────
# Silence from the installer, and a pass (not a warning) from the audit.
for home_name in no-roster with-roster; do
home="$ROOT/$home_name"
output=$(run_doctor_check "$home" "$PATH_WITH_TMUX")
if echo "$output" | grep -qF '[WARN]'; then
fail "doctor warned about the transport while tmux was present ($home_name)"
fi
echo "$output" | grep -qF '[OK]' || fail "doctor did not record a pass with tmux present ($home_name)"
output=$(run_installer_check "$home" "$PATH_WITH_TMUX")
if [ -n "$output" ]; then
fail "installer was not silent with tmux present ($home_name): $output"
fi
done
# ── the roster declares something other than tmux ────────────────────────────
# The roster is read, not assumed. A host that declares a different transport
# is told about the binary it actually needs, and never about tmux — being sent
# to install the wrong package is worse than no advice at all.
home=$(make_home other-transport zellij)
output=$(run_doctor_check "$home" "$PATH_WITH_TMUX")
echo "$output" | grep -qF 'zellij' || fail "doctor ignored the roster's declared transport"
if echo "$output" | grep -qF 'tmux'; then
fail "doctor named tmux for a host whose roster declares zellij"
fi
output=$(run_installer_check "$home" "$PATH_WITH_TMUX")
echo "$output" | grep -qF 'zellij' || fail "installer ignored the roster's declared transport"
if echo "$output" | grep -qF 'tmux'; then
fail "installer named tmux for a host whose roster declares zellij"
fi
# ── a quoted or trailing-comment transport value ─────────────────────────────
# YAML permits both and neither is exotic; a check that installs `tmux"` or
# reads `tmux # default` as a binary name would send the operator nowhere.
home=$(make_home quoted-transport '"tmux" # the only transport today')
output=$(run_doctor_check "$home" "$PATH_WITH_TMUX")
echo "$output" | grep -qF '[OK] Fleet transport available: tmux' ||
fail "doctor did not parse a quoted/commented transport value: $output"
output=$(run_installer_check "$home" "$PATH_WITH_TMUX")
if [ -n "$output" ]; then
fail "installer did not parse a quoted/commented transport value: $output"
fi
echo "ok - fleet transport checks (mosaic-doctor + install.sh)"
@@ -268,6 +268,15 @@ esac
_build_runtime_bin_prefix() {
local candidates=()
if [ -n "$MOSAIC_RUNTIME_BIN" ]; then candidates+=("$MOSAIC_RUNTIME_BIN"); fi
# A host with no system Node gets one bootstrapped here by tools/install.sh, which
# records it in ~/.profile. The fleet unit runs `env -i ... bash --noprofile --norc`
# by design, so ~/.profile is never read and the directory has to be named here.
# The npm probe below cannot cover this: it reports a package prefix
# (~/.npm-global), never a Node runtime directory. It is first so the bootstrapped
# runtime wins on a host that has both — that is the one the installer verified.
# Runtime binaries are `#!/usr/bin/env node`, so without this the pane resolves the
# binary and then dies on `env: 'node': No such file or directory`.
candidates+=("$PANE_HOME/.mosaic/node/current/bin")
if command -v npm >/dev/null 2>&1; then
local npm_prefix
npm_prefix=$(npm config get prefix 2>/dev/null) || true
@@ -286,6 +295,36 @@ _build_runtime_bin_prefix() {
MOSAIC_RUNTIME_BIN_PREFIX=$(_build_runtime_bin_prefix)
PANE_PATH=${MOSAIC_RUNTIME_BIN_PREFIX:+${MOSAIC_RUNTIME_BIN_PREFIX}:}/usr/local/bin:/usr/bin:/bin
# #1241. The pane runs `mosaic yolo <runtime>` under PANE_PATH with a cleared
# environment. A binary missing from *that* path is a pane that dies in under a
# second, inside a session nobody is attached to, with its diagnostic scrolled
# into a pane tmux then destroys. Resolve both here, before any effect, where
# the failure is still attributable to the thing that caused it.
#
# `mosaic yolo <runtime>` runs checkRuntime(runtime) and the binary it looks for
# is named exactly like the runtime, so resolving the runtime name is the same
# question the pane will ask a moment later — asked while an operator can still
# see the answer.
_resolve_in_pane_path() {
PATH="$PANE_PATH" command -v -- "$1" 2>/dev/null
}
# Exit 69 (EX_UNAVAILABLE): the seat cannot be provided. Distinguished from the
# 64 (EX_USAGE) rejections above, which mean the projection itself was bad —
# here the data is fine and the host is not ready. Callers tell the individual
# cases apart by `code=`, the same way fail_env's many codes share exit 64.
fail_launch() {
local code="$1"
shift
echo "ERROR: agent launch aborted: code=${code} agent=${AGENT_NAME} $*" >&2
exit 69
}
for required_binary in mosaic "$MOSAIC_AGENT_RUNTIME"; do
_resolve_in_pane_path "$required_binary" >/dev/null ||
fail_launch missing-binary "'${required_binary}' is not on the pane PATH (${PANE_PATH})"
done
_ensure_claude_workdir_trusted() {
local workdir="$1"
local resolved
@@ -384,6 +423,19 @@ if [ -n "$PANE_PID" ]; then
_start_heartbeat_sidecar "$AGENT_NAME" "$PANE_PID" \
"$MOSAIC_HEARTBEAT_RUN_DIR" "$MOSAIC_HEARTBEAT_INTERVAL" || \
echo "WARNING: heartbeat sidecar could not be started for $AGENT_NAME" >&2
elif _tmux has-session -t "=${AGENT_NAME}:0.0" 2>/dev/null; then
# #1241. Session present, no pane PID after a second of retries. Whatever this
# is, it is not a seat an operator can use, so it is not a success either.
fail_launch pane-pid-unresolved \
"tmux reports the session but no pane PID after 5 attempts"
else
echo "WARNING: could not resolve pane PID for $AGENT_NAME — heartbeat sidecar not started" >&2
# #1241. This branch used to print a WARNING about the heartbeat sidecar and
# exit 0. It is not a heartbeat problem: tmux destroys a session when its pane
# command exits, so an absent session one second after new-session means the
# runtime died on startup. Reporting it as success is what let `fleet start`
# return 0 over three dead panes — the launcher knew, and said the wrong thing
# at the wrong severity to the wrong layer.
fail_launch pane-did-not-survive \
"the pane exited immediately and tmux destroyed the session;" \
"run 'mosaic yolo ${MOSAIC_AGENT_RUNTIME}' in ${MOSAIC_AGENT_WORKDIR} to see why"
fi
@@ -23,8 +23,26 @@ index=0
if [ "${args[0]:-}" = -L ]; then index=2; fi
case "${args[$index]:-}" in
has-session)
# The holder always answers. MOSAIC_TEST_HELD_SESSIONS lets a case add
# other targets that should answer too — without it there is no way to
# model "tmux still reports the session" for a non-holder agent, and the
# launcher's pane-pid-unresolved branch is unreachable from this harness.
#
# A listed target answers only AFTER new-session, because the launcher asks
# this question twice about the same name: once before launching, where a
# yes means "already running, nothing to do, exit 0", and once after, where
# a yes means "the session survived". A shim that answered yes to both
# would short-circuit at the first and never reach the branch under test —
# it would look like coverage and measure the idempotency path instead.
for argument in "${args[@]}"; do
[ "$argument" = '=_holder:0.0' ] && exit 0
case " ${MOSAIC_TEST_HELD_SESSIONS:-} " in
*" $argument "*)
if tr '\0' '\n' < "${MOSAIC_TEST_TMUX_CALLS:?}" | grep -qxF new-session; then
exit 0
fi
;;
esac
done
exit 1
;;
@@ -62,6 +80,30 @@ env -0 > "${MOSAIC_HOME:?}/fleet/pane-environment"
SHIM
chmod +x "$FAKE_BIN/mosaic"
# The runtime the rosters below name. The launcher resolves it against PANE_PATH
# before spawning (#1241), so it has to exist somewhere the pane would find it —
# not merely on the launcher's own PATH.
printf '#!/usr/bin/env bash\nexit 0\n' > "$FAKE_BIN/pi"
chmod +x "$FAKE_BIN/pi"
# PANE_PATH is derived partly from `npm config get prefix`. Left to the real npm
# it would splice whatever the host has installed into the path under test, and
# the missing-binary cases below would pass or fail by accident of the machine.
cat > "$FAKE_BIN/npm" <<'SHIM'
#!/usr/bin/env bash
printf '%s\n' "${MOSAIC_TEST_NPM_PREFIX:-/nonexistent}"
SHIM
chmod +x "$FAKE_BIN/npm"
# PANE_PATH always ends in the system path. A host that installs these there can
# not measure the missing-binary cases at all, and a green run would mean
# nothing — so say so instead of passing.
for host_binary in mosaic pi; do
if PATH=/usr/local/bin:/usr/bin:/bin command -v "$host_binary" >/dev/null 2>&1; then
fail "host provides '$host_binary' in the system path; missing-binary cases are not measurable here"
fi
done
write_generated() {
local home="$1"
local agent="$2"
@@ -81,6 +123,19 @@ MOSAIC_TMUX_SOCKET=mosaic-test
EOF
chmod 600 "$home/fleet/agents/$agent.env.generated"
mkdir -p "$home/work"
install_pane_binaries "$home"
}
# `$PANE_HOME/.npm-global/bin` is one of the prefixes the launcher folds into
# PANE_PATH, so this is the pane's own view of "installed", distinct from the
# launcher's PATH. Tests that need a binary *absent* remove it from here.
install_pane_binaries() {
local pane_home="$1"
mkdir -p "$pane_home/.npm-global/bin"
local binary
for binary in mosaic pi; do
ln -sf "$FAKE_BIN/$binary" "$pane_home/.npm-global/bin/$binary"
done
}
run_start() {
@@ -88,6 +143,7 @@ run_start() {
local agent="$2"
HOME="$home" PATH="$FAKE_BIN:$PATH" MOSAIC_TEST_TMUX_CALLS="$TMUX_CALLS" \
MOSAIC_TEST_PANE_PID="${MOSAIC_TEST_PANE_PID:-}" \
MOSAIC_TEST_HELD_SESSIONS="${MOSAIC_TEST_HELD_SESSIONS:-}" \
MOSAIC_TEST_HOME="$home" \
MOSAIC_TEST_FLEET_OWNER=123e4567-e89b-12d3-a456-426614174000 \
MOSAIC_HOME="$home" "$START" "$agent"
@@ -98,7 +154,10 @@ run_start() {
HOME_VALID="$ROOT/valid"
AGENT_VALID="coder0"
write_generated "$HOME_VALID" "$AGENT_VALID"
run_start "$HOME_VALID" "$AGENT_VALID"
# A live pane PID is part of what "valid launch" means. Until #1241 this case
# ran with none, so the suite's one success path was itself a dead pane the
# launcher reported as fine.
MOSAIC_TEST_PANE_PID=$$ run_start "$HOME_VALID" "$AGENT_VALID"
valid_args=$(tr '\0' '\n' < "$TMUX_CALLS")
echo "$valid_args" | grep -qF new-session || fail "valid generated projection did not reach tmux"
echo "$valid_args" | grep -qF 'mosaic' || fail "fixed mosaic launcher command missing"
@@ -245,6 +304,13 @@ PANE_BASH_ENV="$ROOT/pane-boundary.bash-env"
printf 'MOSAIC_RUNTIME_BIN=%s\n' "$FAKE_BIN" > \
"$HOME_PANE_BOUNDARY/fleet/agents/coder-pane-boundary.env.local"
chmod 600 "$HOME_PANE_BOUNDARY/fleet/agents/coder-pane-boundary.env.local"
# This case does not go through run_start, so its pane binaries come from
# MOSAIC_RUNTIME_BIN=$FAKE_BIN in the env.local written above — not from the
# symlinks install_pane_binaries planted under the generated home, which this
# launcher never consults because HOME here is the trusted parent. That is a
# legitimate resolution path, but it means dropping MOSAIC_RUNTIME_BIN from
# this case on the belief that the symlinks cover it would break the #1241
# binary check rather than exercise it.
LD_PRELOAD='/not/loaded/by-clean-bootstrap.so' \
BASH_ENV="$PANE_BASH_ENV" \
MOSAIC_UNTRUSTED_SENTINEL='must-not-reach-pane' \
@@ -258,6 +324,7 @@ PATH="$PANE_STALE_PATH" \
"MOSAIC_TEST_HOME=$PANE_TRUSTED_HOME" \
MOSAIC_TEST_FLEET_OWNER=123e4567-e89b-12d3-a456-426614174000 \
MOSAIC_TEST_EXECUTE_PANE=1 \
"MOSAIC_TEST_PANE_PID=$$" \
"$START" coder-pane-boundary
pane_args=$(tr '\0' '\n' < "$TMUX_CALLS")
echo "$pane_args" | grep -qxF "HOME=$PANE_TRUSTED_HOME" || \
@@ -281,6 +348,89 @@ for blocked in LD_PRELOAD= BASH_ENV= MOSAIC_UNTRUSTED_SENTINEL=; do
echo "$pane_environment" | grep -qF "$blocked" && fail "runtime pane received $blocked"
done
# #1256. On a host with no system Node, tools/install.sh bootstraps one into
# ~/.mosaic/node/ and writes that directory to ~/.profile. The fleet unit runs
# `env -i ... bash --noprofile --norc`, so ~/.profile is never read — correctly, by
# design — and _build_runtime_bin_prefix does not list the bootstrap directory. Its
# `npm config get prefix` branch cannot cover the gap either: the installer points
# npm's prefix at ~/.npm-global, so that branch contributes the npm-global directory
# and never the Node one, however it resolves.
#
# The property under test is not "the string is in PATH". It is that the pane can
# EXECUTE a Node-shebang runtime binary — which is what `mosaic` is
# (`#!/usr/bin/env node`) and what actually failed: measured on a greenfield VM as
# `env: 'node': No such file or directory` after a clean install that reported success.
#
# So this case runs the pane for real and requires it to have run. A PATH-substring
# assertion would pass on a fix that put the directory in the wrong position, and it
# would keep passing if the pane later stopped running for some unrelated reason.
: > "$TMUX_CALLS"
HOME_NODE="$ROOT/bootstrap-node/.config/mosaic"
write_generated "$HOME_NODE" "coder-node"
NODE_PANE_HOME="${HOME_NODE%/.config/mosaic}"
NODE_BOOTSTRAP_BIN="$NODE_PANE_HOME/.mosaic/node/current/bin"
mkdir -p "$NODE_BOOTSTRAP_BIN"
# The bootstrapped runtime. It records that it ran, which is the evidence this case
# turns on: no node reachable from the pane means no marker.
cat > "$NODE_BOOTSTRAP_BIN/node" <<'SHIM'
#!/usr/bin/env bash
set -euo pipefail
env -0 > "${MOSAIC_HOME:?}/fleet/pane-environment"
SHIM
chmod +x "$NODE_BOOTSTRAP_BIN/node"
# write_generated plants its symlinks under the MOSAIC_HOME it is given; here the
# pane's HOME is the trusted parent, so the pane's view of "installed" is this
# directory instead. `pi` is what #1241 resolves against PANE_PATH; `mosaic` is what
# the pane then executes, and it is a Node script — not a bash script that would run
# anywhere and quietly hide the defect.
mkdir -p "$NODE_PANE_HOME/.npm-global/bin"
ln -sf "$FAKE_BIN/pi" "$NODE_PANE_HOME/.npm-global/bin/pi"
printf '#!/usr/bin/env node\n' > "$NODE_PANE_HOME/.npm-global/bin/mosaic"
chmod +x "$NODE_PANE_HOME/.npm-global/bin/mosaic"
# The npm branch is modelled ALIVE and still cannot close the gap, which is the
# stronger statement. An earlier draft of this case tried to model npm as absent —
# true on a real bootstrap host, where npm lives only in the Node directory — and it
# refused to run anywhere npm is in the system path, i.e. most machines. It was also
# the weaker claim: it would have proven only that a dead branch supplies nothing.
#
# On a bootstrap host the installer sets npm's prefix to ~/.npm-global. So even with
# `command -v npm` true and the branch executing, `npm config get prefix` yields the
# npm-global directory and never the Node one. The gap does not depend on whether
# that branch runs.
NODE_LAUNCHER_BIN="$ROOT/bootstrap-node-launcher-bin"
mkdir -p "$NODE_LAUNCHER_BIN"
ln -sf "$FAKE_BIN/tmux" "$NODE_LAUNCHER_BIN/tmux"
ln -sf "$FAKE_BIN/npm" "$NODE_LAUNCHER_BIN/npm"
/usr/bin/env -i \
"HOME=$NODE_PANE_HOME" \
"PATH=$NODE_LAUNCHER_BIN:/usr/bin:/bin" \
"MOSAIC_HOME=$HOME_NODE" \
"MOSAIC_TEST_TMUX_CALLS=$TMUX_CALLS" \
"MOSAIC_TEST_HOME=$NODE_PANE_HOME" \
"MOSAIC_TEST_NPM_PREFIX=$NODE_PANE_HOME/.npm-global" \
MOSAIC_TEST_FLEET_OWNER=123e4567-e89b-12d3-a456-426614174000 \
MOSAIC_TEST_EXECUTE_PANE=1 \
"MOSAIC_TEST_PANE_PID=$$" \
"$START" coder-node
[ -f "$HOME_NODE/fleet/pane-environment" ] || \
fail "pane could not execute a Node-shebang runtime: $NODE_BOOTSTRAP_BIN is absent from PANE_PATH (#1256)"
node_pane_environment=$(tr '\0' '\n' < "$HOME_NODE/fleet/pane-environment")
# Colon-pad and match a whole element. A regex with `(^|:)` after `.*` looks like it
# does this and does not: an anchor cannot match mid-pattern, so it silently requires
# a leading colon and rejects the directory in FIRST position — which is exactly where
# the fix puts it. That produced a failure reading "pane ran but PANE_PATH does not
# carry <dir>" against a PATH whose first element was that dir.
node_pane_path=":$(printf '%s\n' "$node_pane_environment" | sed -n 's/^PATH=//p' | head -1):"
case "$node_pane_path" in
*":$NODE_BOOTSTRAP_BIN:"*) ;;
*) fail "pane ran but PANE_PATH does not carry $NODE_BOOTSTRAP_BIN (PATH=$node_pane_path)" ;;
esac
write_interaction_generated() {
local home="$1"
local agent="$2"
@@ -392,6 +542,75 @@ echo "$interaction_policy_args" | grep -qF 'new-session' && \
echo "$output" | grep -qF 'operator interaction service requires runtime pi' || \
fail "interaction pinned-policy check did not follow strict parsing"
# #1241. The pane runs `mosaic yolo <runtime>` against PANE_PATH. A binary
# missing from that path is a launch failure, and it has to be named before the
# session is created — after it, the diagnostic dies with the pane.
assert_missing_pane_binary_rejected() {
local binary="$1"
local home="$ROOT/missing-$binary"
local agent="coder-missing-$binary"
write_generated "$home" "$agent"
rm -f "$home/.npm-global/bin/$binary"
: > "$TMUX_CALLS"
local output
if output=$(MOSAIC_TEST_PANE_PID=$$ run_start "$home" "$agent" 2>&1); then
fail "launch succeeded with '$binary' absent from the pane PATH"
fi
echo "$output" | grep -qF 'code=missing-binary' || fail "missing '$binary' diagnostic missing"
echo "$output" | grep -qF "'$binary'" || fail "missing-binary diagnostic did not name $binary"
if tr '\0' '\n' < "$TMUX_CALLS" | grep -qF new-session; then
fail "launcher created a session it knew would die ($binary absent)"
fi
}
assert_missing_pane_binary_rejected mosaic
assert_missing_pane_binary_rejected pi
# #1241. tmux destroys a session when its pane command exits, so no pane PID a
# second after new-session means the runtime died on startup. This used to be a
# WARNING about the heartbeat sidecar followed by exit 0 — three layers above it
# then reported a fleet that was not running.
: > "$TMUX_CALLS"
HOME_DEAD_PANE="$ROOT/dead-pane"
write_generated "$HOME_DEAD_PANE" "coder-dead-pane"
if output=$(MOSAIC_TEST_PANE_PID='' run_start "$HOME_DEAD_PANE" coder-dead-pane 2>&1); then
fail "launcher reported success over a pane that did not survive"
fi
echo "$output" | grep -qF 'code=pane-did-not-survive' || fail "dead-pane diagnostic missing"
if echo "$output" | grep -qiF 'heartbeat'; then
fail "dead pane is still being reported as a heartbeat-sidecar problem"
fi
tr '\0' '\n' < "$TMUX_CALLS" | grep -qF new-session || \
fail "dead-pane case did not reach the launch it is measuring"
# #1241, the other way a pane fails. Above, tmux destroyed the session and
# has-session said so. Here the session is still there and no PID comes back
# after the retries — a different fault (the pane is alive but unusable, or
# tmux is answering inconsistently) that an operator has to be told apart from
# a runtime that died on startup.
#
# This case exists because the branch that handles it shipped with nothing able
# to reach it: the shim answered has-session only for the holder, so every
# non-holder agent landed in the session-is-gone branch no matter what. A
# defensive branch nothing exercises is the same shape as the bug this whole
# change is about, one layer down.
: > "$TMUX_CALLS"
HOME_NO_PID="$ROOT/pane-no-pid"
write_generated "$HOME_NO_PID" "coder-no-pid"
if output=$(MOSAIC_TEST_PANE_PID='' MOSAIC_TEST_HELD_SESSIONS='=coder-no-pid:0.0' \
run_start "$HOME_NO_PID" coder-no-pid 2>&1); then
fail "launcher reported success over a session with no resolvable pane PID"
fi
echo "$output" | grep -qF 'code=pane-pid-unresolved' || \
fail "session-present/no-PID was not reported as pane-pid-unresolved: $output"
if echo "$output" | grep -qF 'code=pane-did-not-survive'; then
fail "a session tmux still reports was diagnosed as a destroyed session"
fi
if echo "$output" | grep -qiF 'heartbeat'; then
fail "an unresolvable pane PID is still being reported as a heartbeat-sidecar problem"
fi
# Exact stop derives the socket exclusively from the validated generated
# projection and ignores an ambient socket supplied by the caller.
: > "$TMUX_CALLS"
@@ -32,10 +32,6 @@ packages/mosaic/framework/tools/tmux/test-send-message-socket.sh | requires a re
packages/mosaic/framework/tools/tmux/test-send-message-verdict.sh | requires real tmux-pane fixtures on a throwaway socket; CI image ships no tmux; #1017 burndown (same condition as its sibling)
# --- single-suite directories: unmeasured in CI ---
packages/mosaic/framework/tools/fleet/test-start-agent-session.sh | unmeasured in CI image; stubs tmux via a fake bin dir, likely CI-fit; #1017 burndown
packages/mosaic/framework/tools/glpi/test-list-http-status.sh | unmeasured in CI image; stub-based (#807 regression harness), likely CI-fit; #1017 burndown
packages/mosaic/framework/tools/orchestrator/test-board-roll.sh | unmeasured in CI image; file-fixture based, likely CI-fit; #1017 burndown
packages/mosaic/framework/tools/woodpecker/test-ci-wait-exit-matrix.sh | unmeasured in CI image; drives ci-wait.sh against a stub pipeline-status.sh, likely CI-fit; #1017 burndown
# --- naming-boundary files the strict test-*.sh prefix cannot even name ---
# (#1017: three independent censuses handled the microtest file three different
+1 -1
View File
@@ -25,7 +25,7 @@
"lint": "eslint src",
"typecheck": "tsc --noEmit",
"test": "vitest run --passWithNoTests && pnpm run test:framework-shell",
"test:framework-shell": "bash framework/tools/quality/scripts/check-test-enumeration.sh && bash framework/tools/quality/scripts/test-check-test-enumeration.sh && python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/promotion_binding_unittest.py && python3 src/lease-broker/promotion_trigger_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/receipt_observer_client_unittest.py && python3 src/lease-broker/invariant_r_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-ci-queue-wait-tristate.sh && bash framework/tools/git/test-ci-queue-wait-github-checks.sh && bash framework/tools/git/test-pr-merge-queue-branch.sh && bash framework/tools/git/test-pr-merge-head-pin.sh && bash framework/tools/git/test-pr-merge-message-field.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/woodpecker/test-terminal-green-contract.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/_scripts/test-mosaic-init-rce.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-preimage.sh && bash framework/tools/wake/test-wake-install.sh"
"test:framework-shell": "bash framework/tools/quality/scripts/check-test-enumeration.sh && bash framework/tools/quality/scripts/test-check-test-enumeration.sh && python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/promotion_binding_unittest.py && python3 src/lease-broker/promotion_trigger_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/receipt_observer_client_unittest.py && python3 src/lease-broker/invariant_r_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-ci-queue-wait-tristate.sh && bash framework/tools/git/test-ci-queue-wait-github-checks.sh && bash framework/tools/git/test-pr-merge-queue-branch.sh && bash framework/tools/git/test-pr-merge-head-pin.sh && bash framework/tools/git/test-pr-merge-message-field.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/woodpecker/test-terminal-green-contract.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/_scripts/test-mosaic-init-rce.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-preimage.sh && bash framework/tools/wake/test-wake-install.sh && bash framework/tools/fleet/test-start-agent-session.sh && bash framework/tools/glpi/test-list-http-status.sh && bash framework/tools/orchestrator/test-board-roll.sh && bash framework/tools/woodpecker/test-ci-wait-exit-matrix.sh && bash framework/tools/_scripts/test-fleet-transport-check.sh"
},
"dependencies": {
"@mosaicstack/brain": "workspace:*",
@@ -0,0 +1,323 @@
import { execFile } from 'node:child_process';
import { mkdir, mkdtemp, readFile, readdir, rm, stat, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import { Command } from 'commander';
import { afterEach, describe, expect, it, vi } from 'vitest';
import { registerFleetCommand, type CommandResult, type CommandRunner } from './fleet.js';
/**
* #1237: the v1-only commands (`ps`, `install`, `install-systemd`, `add`,
* `remove`) rejected a roster-v2 fleet outright, so a greenfield v2 box could
* never get its units placed. These tests pin the three behaviours that fix
* gives it, and the two it deliberately does NOT give it.
*
* The load-bearing negative is that `install` on v2 writes no generated env:
* the reconciler owns that file through projectRosterV2AgentGeneratedEnv, and a
* second writer here — necessarily through the v1 mapping — is exactly the
* drift the #791 single-SSOT invariant exists to prevent.
*/
const rosterV2 = `
version: 2
generation: 4
transport: tmux
tmux:
socket_name: mosaic-fleet
holder_session: _holder
defaults:
working_directory: /srv/mosaic
runtime: pi
runtimes:
pi:
reset_command: /new
agents:
- name: coder0
alias: Coder 0
class: code
runtime: pi
provider: openai
model: gpt-5.6-sol
reasoning: high
tool_policy: code
working_directory: /srv/mosaic
persistent_persona: false
reset_between_tasks: true
lifecycle:
enabled: true
desired_state: stopped
launch:
yolo: true
- name: coder1
alias: Coder 1
class: code
runtime: pi
provider: openai
model: gpt-5.6-sol
reasoning: medium
tool_policy: code
working_directory: /srv/other
persistent_persona: false
reset_between_tasks: true
lifecycle:
enabled: true
desired_state: stopped
launch:
yolo: true
`;
let tempHome: string | undefined;
const savedHome = process.env.HOME;
const savedMosaicHome = process.env.MOSAIC_HOME;
afterEach(async (): Promise<void> => {
vi.restoreAllMocks();
process.exitCode = undefined;
if (savedHome === undefined) delete process.env.HOME;
else process.env.HOME = savedHome;
if (savedMosaicHome === undefined) delete process.env.MOSAIC_HOME;
else process.env.MOSAIC_HOME = savedMosaicHome;
if (tempHome) await rm(tempHome, { recursive: true, force: true });
tempHome = undefined;
});
/**
* A HOME with a roster-v2 fleet and nothing else — the greenfield shape, before
* anything has been installed, applied or started.
*/
async function v2Home(): Promise<string> {
tempHome = await mkdtemp(join(tmpdir(), 'mosaic-fleet-v2-dispatch-'));
process.env.HOME = tempHome;
delete process.env.MOSAIC_HOME;
const mosaicHome = join(tempHome, '.config', 'mosaic');
for (const directory of ['fleet', 'fleet/agents', 'fleet/roles']) {
await mkdir(join(mosaicHome, directory), { recursive: true, mode: 0o700 });
}
await writeFile(join(mosaicHome, 'fleet', 'roster.yaml'), rosterV2, { mode: 0o600 });
await writeFile(join(mosaicHome, 'fleet', 'roles', 'code.md'), '`class: code`\n\n# code\n', {
mode: 0o600,
});
return mosaicHome;
}
/**
* Stands in for a box where nothing is running: every systemctl and tmux probe
* fails the way it does before the holder has ever started. `ps` must survive
* this — it is the command an operator reaches for to find out *why* there is
* no seat, so it has to report the emptiness rather than fail on it.
*/
const greenfieldRunner: CommandRunner = async (command): Promise<CommandResult> => {
if (command === 'tmux') {
return { stdout: '', stderr: 'no server running on /tmp/tmux-1000/mosaic-fleet', exitCode: 1 };
}
return { stdout: '', stderr: '', exitCode: 1 };
};
function program(runner: CommandRunner = greenfieldRunner): Command {
const result = new Command();
result.exitOverride();
registerFleetCommand(result, { runner, frameworkRoot: resolve(process.cwd(), 'framework') });
return result;
}
function capture(): string[] {
const lines: string[] = [];
vi.spyOn(console, 'log').mockImplementation((value: string): void => {
lines.push(value);
});
return lines;
}
async function exists(path: string): Promise<boolean> {
try {
await stat(path);
return true;
} catch {
return false;
}
}
describe('mosaic fleet ps — roster v2', (): void => {
it('lists every v2 agent on a greenfield box with nothing running, and does not throw', async (): Promise<void> => {
await v2Home();
const lines = capture();
await expect(
program().parseAsync(['node', 'mosaic', 'fleet', 'ps', '--json']),
).resolves.toBeDefined();
const rows = JSON.parse(lines.join('\n')) as {
name: string;
runtime: string;
alias?: string;
paneAlive: boolean;
source: string;
}[];
expect(rows.map((row) => row.name).sort()).toEqual(['coder0', 'coder1']);
// The v2 roster's per-agent fields must survive the read model, not be
// flattened into defaults.
expect(rows.every((row) => row.runtime === 'pi')).toBe(true);
expect(rows.find((row) => row.name === 'coder0')?.alias).toBe('Coder 0');
// Nothing is running, and that is a report, not an error.
expect(rows.every((row) => row.paneAlive === false)).toBe(true);
expect(rows.every((row) => row.source === 'roster')).toBe(true);
expect(process.exitCode ?? 0).toBe(0);
});
});
describe('mosaic fleet install — roster v2', (): void => {
it('places the tool files and unit templates', async (): Promise<void> => {
const mosaicHome = await v2Home();
capture();
await expect(
program().parseAsync(['node', 'mosaic', 'fleet', 'install', '--no-enable']),
).resolves.toBeDefined();
// Units live in the systemd user dir, not under the Mosaic home.
const systemdUserDir = join(tempHome!, '.config', 'systemd', 'user');
for (const unit of [
'mosaic-tmux-holder.service',
'[email protected]',
'[email protected]',
]) {
expect(await exists(join(systemdUserDir, unit))).toBe(true);
}
const launcher = join(mosaicHome, 'tools', 'fleet', 'start-agent-session.sh');
expect(await exists(launcher)).toBe(true);
expect((await stat(launcher)).mode & 0o777).toBe(0o755);
});
it('writes NO generated env — that file belongs to the reconciler (#791)', async (): Promise<void> => {
const mosaicHome = await v2Home();
capture();
await program().parseAsync(['node', 'mosaic', 'fleet', 'install', '--no-enable']);
const agentDir = join(mosaicHome, 'fleet', 'agents');
expect(await readdir(agentDir)).toEqual([]);
});
it('tells the operator which command does own the env', async (): Promise<void> => {
await v2Home();
const lines = capture();
await program().parseAsync(['node', 'mosaic', 'fleet', 'install', '--no-enable']);
expect(lines.join('\n')).toContain('mosaic fleet apply');
});
});
describe('[email protected]', (): void => {
const unitPath = resolve(process.cwd(), 'framework', 'systemd', 'user', '[email protected]');
/** The single `ConditionPathExists=` value declared by the unit template. */
async function conditionPath(): Promise<string> {
const unit = await readFile(unitPath, 'utf8');
const matches = unit.match(/^ConditionPathExists=(.+)$/gm) ?? [];
expect(matches).toHaveLength(1);
return matches[0]!.slice('ConditionPathExists='.length).trim();
}
it('will not attempt a seat before the reconciler has written its env', async (): Promise<void> => {
// The pairing that makes "install writes no env" safe: install enables the
// unit (WantedBy=default.target) but does not start it, so without this
// condition a reboot between `install` and the first `apply` would run
// ExecStart against an absent env file and fail every seat unit.
expect(await conditionPath()).toBe('%h/.config/mosaic/fleet/agents/%i.env.generated');
});
/**
* The two halves of the guard's *effect*, which no assertion on the literal
* string can cover on its own.
*
* Measured end to end on a real box (canary, 2026-08-16) rather than inferred:
* with the condition, `systemctl --user start mosaic-agent@<name>` on an agent
* with no generated env returns rc=0, `Result=success`, `ConditionResult=no`,
* and journals "skipped, unmet condition check". With the condition removed by
* drop-in and nothing else changed, the same start returns rc=1,
* `Result=exit-code`, `ExecMainStatus=64`, and the unit enters `failed`.
*
* systemd is not available in this suite, so these two tests pin the parts
* that can drift in code: the condition naming a *different* file than the one
* the fleet actually writes, and the launcher quietly becoming tolerant of an
* absent env — either of which turns the condition into decoration while the
* literal-string assertion above still passes.
*/
it('guards exactly the file the fleet writes, so the two cannot drift apart', async (): Promise<void> => {
const mosaicHome = await v2Home();
const rendered = (await conditionPath()).replace('%h', tempHome!).replace('%i', 'coder0');
// The path an installed fleet actually places for this agent.
expect(rendered).toBe(join(mosaicHome, 'fleet', 'agents', 'coder0.env.generated'));
});
it('guards a real failure — the launcher rejects an absent generated env', async (): Promise<void> => {
await v2Home();
await program().parseAsync(['node', 'mosaic', 'fleet', 'install', '--no-enable']);
// Exactly what ExecStart runs, against the state the condition exists to
// catch: unit enabled, reconciler has not written env yet.
const launched = await new Promise<{ code: number | null; stderr: string }>((settle) => {
const child = execFile(
'/bin/bash',
[
'--noprofile',
'--norc',
join(tempHome!, '.config', 'mosaic', 'tools', 'fleet', 'start-agent-session.sh'),
'coder0',
],
{ env: { HOME: tempHome!, MOSAIC_AGENT_NAME: 'coder0', PATH: '/usr/bin:/bin' } },
(_error, _stdout, stderr) => {
settle({ code: child.exitCode, stderr });
},
);
});
expect(launched.code).not.toBe(0);
expect(launched.stderr).toContain('missing-file');
});
});
describe('mosaic fleet add / remove — roster v2', (): void => {
it('add refuses, and names the two-step v2 sequence instead of inventing defaults', async (): Promise<void> => {
await v2Home();
await expect(
program().parseAsync([
'node',
'mosaic',
'fleet',
'add',
'coder2',
'--runtime',
'pi',
'--class',
'code',
]),
).rejects.toThrow(/mosaic fleet create[\s\S]*mosaic fleet apply/);
});
it('remove refuses, and names delete plus apply', async (): Promise<void> => {
await v2Home();
await expect(
program().parseAsync(['node', 'mosaic', 'fleet', 'remove', 'coder1']),
).rejects.toThrow(/mosaic fleet delete coder1[\s\S]*mosaic fleet apply/);
});
// Note: this one passes on the unmodified tree too — there `remove` throws in
// the v1 parser, before it can touch anything. It is a regression guard on the
// ordering of the new guard clause, not evidence that the fix works.
it('refuses BEFORE mutating the roster', async (): Promise<void> => {
const mosaicHome = await v2Home();
const rosterPath = join(mosaicHome, 'fleet', 'roster.yaml');
const before = await readFile(rosterPath, 'utf8');
await expect(
program().parseAsync(['node', 'mosaic', 'fleet', 'remove', 'coder1']),
).rejects.toThrow();
expect(await readFile(rosterPath, 'utf8')).toBe(before);
});
});
+116 -8
View File
@@ -34,6 +34,7 @@ export {
resolveInstalledFleetRosterPath,
} from '../fleet/fleet-roster-v1.js';
export type { FleetAgent, FleetRoster } from '../fleet/fleet-roster-v1.js';
import { parseRosterV2 } from '../fleet/roster-v2.js';
import {
registerFleetAgentCrudCommands,
type FleetAgentCrudCommandDeps,
@@ -820,7 +821,7 @@ export function buildEnableLingerCommand(user: string): string[] {
*/
export async function enableFleetUnits(
runner: CommandRunner,
roster: FleetRoster,
roster: { readonly agents: readonly { readonly name: string }[] },
opts: { enable?: boolean },
): Promise<void> {
if (opts.enable === false) {
@@ -1527,7 +1528,8 @@ export function registerFleetCommand(program: Command, deps: FleetCommandDeps =
.option('--no-enable', 'Skip enabling units for boot-survival')
.action(async (opts: { enable?: boolean }) => {
await installFleet(cmd, frameworkRoot);
const roster = await loadRosterForCommand(cmd);
// Unit enablement needs agent names only, so it reads either version.
const roster = await loadRosterReadModel(cmd);
await enableFleetUnits(runner, roster, opts);
});
@@ -1537,7 +1539,8 @@ export function registerFleetCommand(program: Command, deps: FleetCommandDeps =
.option('--no-enable', 'Skip enabling units for boot-survival')
.action(async (opts: { enable?: boolean }) => {
await installFleet(cmd, frameworkRoot);
const roster = await loadRosterForCommand(cmd);
// Unit enablement needs agent names only, so it reads either version.
const roster = await loadRosterReadModel(cmd);
await enableFleetUnits(runner, roster, opts);
});
@@ -1688,7 +1691,9 @@ export function registerFleetCommand(program: Command, deps: FleetCommandDeps =
.action(async (opts: { json?: boolean }) => {
const commandOpts = cmd.opts<{ mosaicHome: string; roster?: string }>();
const activePaths = resolveFleetPaths(commandOpts.mosaicHome);
const roster = await loadRosterForCommand(cmd);
// ps only reads, so it takes the version-agnostic read model rather than
// the v1 parser, which rejects a v2 roster outright.
const roster = await loadRosterReadModel(cmd);
const { tenant_id, host } = getDefaultTenantAndHost();
const nowMs = Date.now();
@@ -1908,6 +1913,16 @@ export function registerFleetCommand(program: Command, deps: FleetCommandDeps =
start: boolean;
},
) => {
if (await usesRosterV2ControlPlane(cmd)) {
// command.error, not a bare throw: this is operator guidance, and a
// bare throw reaches the top level uncaught and prints it under a Node
// stack trace. Measured on canary — the message is the whole point of
// the refusal, so it has to arrive readable.
cmd.error(rosterV2MutationGuidance('add', 'create', name), {
code: 'fleet.roster-v2',
exitCode: 1,
});
}
if (!VALID_FLEET_RUNTIMES.includes(opts.runtime)) {
throw new Error(
`Invalid runtime "${opts.runtime}". Valid runtimes: ${VALID_FLEET_RUNTIMES.join(', ')}.`,
@@ -1973,6 +1988,12 @@ export function registerFleetCommand(program: Command, deps: FleetCommandDeps =
.description('Remove an agent from the fleet roster')
.option('--keep-files', 'Skip deleting env and heartbeat files')
.action(async (name: string, opts: { keepFiles?: boolean }) => {
if (await usesRosterV2ControlPlane(cmd)) {
cmd.error(rosterV2MutationGuidance('remove', 'delete', name), {
code: 'fleet.roster-v2',
exitCode: 1,
});
}
const commandOpts = cmd.opts<{ mosaicHome: string; roster?: string }>();
const activePaths = resolveFleetPaths(commandOpts.mosaicHome);
const rosterPath = await resolveRosterPath(commandOpts.mosaicHome, commandOpts.roster);
@@ -2331,7 +2352,9 @@ export function registerFleetAgentCommands(
async function installFleet(cmd: Command, frameworkRoot: string): Promise<void> {
const activePaths = resolveFleetPaths(cmd.opts<{ mosaicHome: string }>().mosaicHome);
assertDefaultMosaicHomeForSystemd(activePaths.mosaicHome);
const roster = await loadRosterForCommand(cmd);
// Read model first: every file this function places is roster-independent, and
// the v1 parser would reject a v2 roster before any of them were written.
const roster = await loadRosterReadModel(cmd);
await ensureFleetHolderIdentity(activePaths.mosaicHome);
await mkdir(activePaths.fleetToolsDir, { recursive: true });
await mkdir(activePaths.tmuxToolsDir, { recursive: true });
@@ -2391,16 +2414,30 @@ async function installFleet(cmd: Command, frameworkRoot: string): Promise<void>
join(activePaths.systemdUserDir, '[email protected]'),
);
for (const agent of roster.agents) {
// On roster v2 the reconciler owns the generated env: `apply` writes it and
// `regen` rebuilds it, both from projectRosterV2AgentGeneratedEnv. Writing it
// here too — necessarily through the v1 mapping — would be the third writer of
// one file and would break the #791 single-SSOT invariant. So v2 gets the tool
// files and the units, and nothing else.
if (roster.version === 2) {
console.log(
`Installed fleet tools and systemd units for ${roster.agents.length} agent(s). ` +
`Generated env is owned by the reconciler on roster v2 — run: mosaic fleet apply --expected-generation <n>`,
);
return;
}
const v1Roster = await loadRosterForCommand(cmd);
for (const agent of v1Roster.agents) {
await writeAgentEnvironmentProjection({
mosaicHome: activePaths.mosaicHome,
agentEnvDir: activePaths.agentEnvDir,
agentName: agent.name,
generated: generateAgentEnvValues(roster, agent),
generated: generateAgentEnvValues(v1Roster, agent),
});
}
console.log(`Installed fleet files for ${roster.agents.length} agent(s).`);
console.log(`Installed fleet files for ${v1Roster.agents.length} agent(s).`);
}
async function loadRosterForCommand(cmd: Command): Promise<FleetRoster> {
@@ -2427,6 +2464,77 @@ async function usesRosterV2ControlPlane(cmd: Command): Promise<boolean> {
);
}
/**
* `add`/`remove` and `create`/`delete` are not two spellings of one operation.
* The v1 pair edits the roster *and* drives systemd; the v2 pair is documented
* as changing desired state "without runtime actions", leaving convergence to
* `apply`. `add` also collects four fields where a v2 agent requires eleven, so
* routing it to `create` would mean inventing provider, alias, reasoning and
* tool-policy defaults on the operator's behalf. Refusing with the real command
* is honest; silently guessing an agent's provider is not.
*/
function rosterV2MutationGuidance(
v1Command: 'add' | 'remove',
v2Command: 'create' | 'delete',
name: string,
): string {
const target = v2Command === 'delete' ? ` ${name}` : '';
return (
`mosaic fleet ${v1Command} does not operate on a roster-v2 fleet. ` +
`Roster v2 separates desired state from convergence:\n` +
` 1. mosaic fleet ${v2Command}${target} --expected-generation <current> ` +
`${v2Command === 'create' ? "--agent '<json>' " : ''}` +
`(edits the roster only)\n` +
` 2. mosaic fleet apply --expected-generation <new> (converges systemd and tmux)\n` +
`Read the current generation with: mosaic fleet status`
);
}
/**
* The read-only fields shared by roster v1 and v2, for the commands that only
* ever *read* the roster (`ps`, and unit enablement inside `install`).
*
* This is deliberately NOT a v2→v1 downshift. A downshifted `FleetRoster` would
* be accepted by `generateAgentEnvValues`, and that would make a third writer of
* `fleet/agents/<name>.env.generated` — through the v1 mapping — breaking the
* #791 single-SSOT invariant that {@link projectRosterV2AgentGeneratedEnv} is
* documented to hold. Keeping the read model this small makes that misuse
* impossible: there is nothing here to write a roster or an env file back from.
*/
interface FleetRosterReadModel {
readonly version: 1 | 2;
readonly tmux: { readonly socketName: string; readonly holderSession: string };
readonly agents: readonly {
readonly name: string;
readonly alias?: string;
readonly runtime: string;
}[];
}
/** Reads either roster version into the shared read-only view. */
async function loadRosterReadModel(cmd: Command): Promise<FleetRosterReadModel> {
const opts = cmd.opts<{ mosaicHome: string; roster?: string }>();
const path = await resolveRosterPath(opts.mosaicHome, opts.roster);
if (!(await usesRosterV2ControlPlane(cmd))) {
const v1 = await loadRosterAtPath(cmd, path);
return {
version: 1,
tmux: { socketName: v1.tmux.socketName, holderSession: v1.tmux.holderSession },
agents: v1.agents,
};
}
try {
const v2 = parseRosterV2(await readFleetRosterText(path), 'yaml');
return {
version: 2,
tmux: { socketName: v2.tmux.socketName, holderSession: v2.tmux.holderSession },
agents: v2.agents,
};
} catch (error) {
reportFleetRosterConfigurationError(cmd, error);
}
}
async function loadRosterFromAgentCommand(
command: Command,
mosaicHomeOverride?: string,
+42
View File
@@ -390,6 +390,43 @@ ensure_prefix_on_path() {
dim " Run: export PATH=\"$PREFIX/bin:\$PATH\" (or start a new login shell)"
}
# Fleet transport binary (#1240).
#
# `mosaic fleet --help` reads "Manage the local Mosaic tmux fleet" and every
# roster the CLI scaffolds sets `transport: tmux`, but nothing in this script
# provides tmux and, until now, nothing in it mentioned tmux at all. A
# greenfield host came out of this installer able to install a fleet, start a
# fleet, and run no seat — the operator's first signal was `mosaic fleet ps`.
#
# Not a `require_cmd`: tmux is required by the fleet, not by mosaic. Plenty of
# hosts install this to run `mosaic claude` and will never scaffold a roster,
# and failing their install over a binary they do not need would be wrong. It
# is a warning that names precisely what it blocks.
#
# `tools/_scripts/mosaic-doctor` carries a deliberately parallel check, so the
# same host state gets the same answer from an audit as from an install. They
# are separate implementations because this one has to work before the
# framework's scripts are guaranteed to be on disk; keep their wording in step.
check_fleet_transport() {
local transport=tmux
local roster="$MOSAIC_HOME/fleet/roster.yaml"
local declared=""
if [[ -f "$roster" ]]; then
declared="$(sed -n 's/^[[:space:]]*transport:[[:space:]]*//p' "$roster" | head -1 |
tr -d '"'\''' | tr -d '\r' | awk '{print $1}')"
[[ -n "$declared" ]] && transport="$declared"
fi
command -v "$transport" &>/dev/null && return 0
warn "Fleet transport '$transport' is not installed."
echo " The Mosaic fleet runs its agent seats inside $transport. Without it,"
echo " ${C}mosaic fleet start${RESET} reports success and no seat comes up."
echo " Install it before using the fleet, e.g. ${C}sudo apt-get install -y $transport${RESET}"
echo " (this does not affect ${C}mosaic claude${RESET} or the other single-runtime commands)."
}
installed_cli_version() {
local json
json="$(npm ls -g --depth=0 --json --prefix="$PREFIX" 2>/dev/null)" || true
@@ -1106,6 +1143,11 @@ if [[ "$FLAG_CHECK" == "false" ]]; then
ok "Done."
fi
# Fleet readiness (#1240). Runs in both normal and --check mode: "what is the
# state of this host" is exactly the question --check is asked, and a host that
# cannot run a seat should not have to discover it from `fleet ps`.
check_fleet_transport
} # end main
main "$@"