Compare commits

..
Author SHA1 Message Date
fred 5c35a250de test(fleet): name what the pane-boundary case's binary check rides on (#1241)
ci/woodpecker/pr/ci Pipeline was successful
Review finding from scooby. This case does not use run_start, so
install_pane_binaries' symlinks land under a home its launcher never consults
(HOME is the trusted parent here). It resolves mosaic and pi through
MOSAIC_RUNTIME_BIN=$FAKE_BIN instead. Valid path, valid green — and a trap for
anyone who later drops that env var believing the symlinks cover it, which
would break the #1241 binary check rather than exercise it.

Comment only; no behavior change. Harness rc=0.

Refs #1241.
2026-08-16 00:21:13 -05:00
fred 10a1f82031 test(fleet): cover the pane-pid-unresolved branch this PR shipped (#1241)
ci/woodpecker/pr/ci Pipeline was canceled
Review finding from scooby: this PR added a failure branch the harness
structurally could not reach. The fake tmux answered `has-session` only for
`=_holder:0.0`, so every non-holder agent landed in the session-is-gone branch
no matter what — the `elif` (tmux still reports the session, no pane PID after
the retries) had zero coverage and no way to get any.

That is the same shape as the bug this PR exists to fix, one layer down: a code
path shipped green where the gate that should measure it cannot. Less severe,
because the branch fails closed at exit 69 rather than reporting success — but
"the harness can't reach it" is the sentence that precedes the next silent
regression, so it gets closed here rather than filed.

`MOSAIC_TEST_HELD_SESSIONS` lets a case name targets the shim should also
answer for. It answers them only AFTER `new-session`, and that detail is the
whole trick: the launcher asks `has-session` about the same name twice — once
at line 255 where a yes means "already running, exit 0", and once at 417 where
a yes means "the session survived". A shim answering yes to both short-circuits
at the first and never reaches the branch under test. It would have looked like
coverage while measuring the idempotency path.

Both failure modes were measured, not reasoned about:
- toggle absent (the old shim): `code=pane-did-not-survive` — the case lands on
  the wrong branch, which is exactly the unreachability being reported.
- toggle answering unconditionally: launcher exits 0 via the idempotency
  short-circuit — "launcher reported success over a session with no resolvable
  pane PID".
- toggle gated on new-session: `code=pane-pid-unresolved`, exit 69.

The case also asserts the diagnostic is not `pane-did-not-survive` and does not
mention the heartbeat, so the two pane faults cannot collapse into one message.

Gates: bash -n · launcher harness rc=0 · test-fleet-units.sh (real tmux) rc=0 ·
fleet specs 342 passed.

Refs #1241.
2026-08-16 00:19:25 -05:00
fred 61a907a12f fix(fleet): fail the agent launcher when the pane cannot survive (#1241)
ci/woodpecker/pr/ci Pipeline was successful
`mosaic fleet start` returned 0 over three dead panes. The launcher knew,
and said the wrong thing at the wrong severity to the wrong layer.

The pane runs `mosaic yolo <runtime>` under PANE_PATH with a cleared
environment. When that binary is absent the pane dies in under a second,
tmux destroys the session, and the diagnostic goes with it. The launcher
then found no PANE_PID, printed a WARNING about the *heartbeat sidecar*,
and exited 0 — so systemd logged "Finished ... successfully" and
`fleet start` reported success. `fleet ps` was the only component telling
the truth.

Two changes, both in start-agent-session.sh:

1. Before any effect, resolve `mosaic` and the roster's runtime against
   PANE_PATH — the pane's own view of the path, not the launcher's.
   `mosaic yolo <runtime>` calls checkRuntime(runtime) and looks for a
   binary named exactly like the runtime, so this asks the same question
   the pane will ask a moment later, while an operator can still see the
   answer. Absent binary -> exit 69, code=missing-binary, no session
   created.

2. Replace the dead-pane WARNING+exit-0. An absent session one second
   after new-session is a runtime that died on startup, not a heartbeat
   problem -> exit 69, code=pane-did-not-survive, with the command to run
   by hand to see why. A present session with no pane PID after five
   attempts -> code=pane-pid-unresolved. Neither branch kills the
   session; destroying a possibly-live pane on a guess is worse than
   leaving it for inspection.

Exit 69 (EX_UNAVAILABLE) is deliberate: the 64s already in this file mean
the projection was bad, and here the data is fine and the host is not
ready. Callers separate the cases by `code=`, the same way fail_env's
codes share 64.

This propagates for free. `fleet start` calls runChecked() for the holder
and each agent, and runChecked throws on non-zero, so layers 4 and 5 stop
lying without a TypeScript change. Two adjacent defects are left for a
follow-up issue rather than widened into this diff: the per-agent loop
aborts on the first failure instead of attempting all and reporting an
aggregate, and runChecked's bare throw surfaces the launcher's message
under a Node unhandled-rejection stack trace because program.parse() is
synchronous.

Tests:

- test-start-agent-session.sh gains three cases: `mosaic` absent from the
  pane path, the runtime absent from the pane path, and a pane that does
  not survive. Each was verified individually red against the unmodified
  origin/next launcher.
- The two cases asserting a valid launch now supply a pane PID. Until now
  the suite's one success path was itself a dead pane the launcher
  reported as fine.
- The harness fakes `npm` so PANE_PATH stops depending on whatever the
  host has installed, and fails loudly if the host provides `mosaic` or
  `pi` in the system path, where the missing-binary cases would not be
  measurable at all.
- test-fleet-units.sh gains a `pi` shim in its runtime bin. The real-tmux
  harness named `pi` in its roster and never installed it; the new
  preflight caught it.

Refs #1241
2026-08-15 23:56:53 -05:00
7 changed files with 190 additions and 308 deletions
@@ -128,6 +128,14 @@ EOF
sleep 30
EOF
chmod 700 "$AGENT_BIN/mosaic"
# The launcher resolves the roster's runtime against PANE_PATH before it
# spawns anything (#1241), so the runtime this projection names has to be
# present here even though the fake `mosaic` above never execs it.
cat > "$AGENT_BIN/pi" <<'EOF'
#!/bin/sh
sleep 30
EOF
chmod 700 "$AGENT_BIN/pi"
server_environment_before=$(tmux -L "$TEST_SOCKET" show-environment -g | sort)
server_sessions_before=$(tmux -L "$TEST_SOCKET" list-sessions | sort)
if /usr/bin/env -i HOME="$HOLDER_HOME" PATH=/usr/bin:/bin MOSAIC_HOME="$AGENT_HOME" \
@@ -225,54 +225,6 @@ else
warn "mosaic-ensure-sequential-thinking helper missing"
fi
# Fleet transport binary (#1240).
#
# `mosaic fleet --help` reads "Manage the local Mosaic tmux fleet" and every
# roster the CLI scaffolds sets `transport: tmux`, but nothing in the install
# path provides tmux and, until now, nothing here noticed it was absent. On a
# greenfield host that produced a fleet which installed clean, started clean,
# and had no live seat; `mosaic fleet ps` was the operator's first and only
# signal that anything was wrong.
#
# The roster's own `transport:` is read rather than assumed, so a host that
# declares something other than tmux is told about the binary it actually
# needs. Absent a roster the check still runs — `mosaic fleet init` will
# scaffold a tmux fleet on this host, and finding out beforehand is the point.
#
# `tools/install.sh` carries a deliberately parallel check at the end of its
# summary. The two are separate because the installer must be able to say this
# before the framework's own scripts are guaranteed to be on disk; keep their
# wording in step.
fleet_declared_transport() {
local roster="$MOSAIC_HOME/fleet/roster.yaml"
local declared=""
if [[ -f "$roster" ]]; then
declared="$(sed -n 's/^[[:space:]]*transport:[[:space:]]*//p' "$roster" | head -1 |
tr -d '"'\''' | tr -d '\r' | awk '{print $1}')"
fi
printf '%s\n' "${declared:-tmux}"
}
check_fleet_transport() {
local transport
transport="$(fleet_declared_transport)"
if command -v "$transport" >/dev/null 2>&1; then
pass "Fleet transport available: $transport"
return
fi
if [[ -f "$MOSAIC_HOME/fleet/roster.yaml" ]]; then
warn "Fleet transport '$transport' is not installed — this host has a roster and no seat can launch. Install it (e.g. sudo apt-get install -y $transport), then 'mosaic fleet start'."
else
warn "Fleet transport '$transport' is not installed — 'mosaic fleet' cannot run seats here. Install it (e.g. sudo apt-get install -y $transport) before 'mosaic fleet init'."
fi
}
check_fleet_transport
# Legacy migration surfaces should no longer contain symlink trees.
legacy_paths=(
"$HOME/.claude/agent-guides"
@@ -1,215 +0,0 @@
#!/usr/bin/env bash
# Covers the #1240 fleet-transport checks in `mosaic-doctor` and in
# `tools/install.sh`.
#
# Both checks answer the same question — "can a seat actually launch on this
# host?" — from two different places, because the installer has to be able to
# answer it before the framework's own scripts are guaranteed to be on disk.
# Two implementations of one rule is exactly the shape that drifts, so this
# harness drives BOTH, in one file, from the same table of cases.
#
# The functions are extracted from the shipped scripts rather than copied here.
# A test that carries its own copy of the logic is a test that keeps passing
# after the shipped copy changes — the failure mode this whole change is about.
# Extraction is by exact function header and a closing brace in column one; if
# either script is reshaped so that stops matching, the extraction yields
# nothing and this fails loudly instead of silently measuring an empty string.
set -euo pipefail
SCRIPT_DIR=$(cd -- "$(dirname -- "$0")" && pwd)
DOCTOR="$SCRIPT_DIR/mosaic-doctor"
# framework/tools/_scripts -> framework/tools -> framework -> mosaic -> packages -> repo
INSTALLER=$(cd -- "$SCRIPT_DIR/../../../../.." && pwd)/tools/install.sh
fail() {
echo "FAIL: $*" >&2
exit 1
}
[ -f "$DOCTOR" ] || fail "missing mosaic-doctor at $DOCTOR"
[ -f "$INSTALLER" ] || fail "missing install.sh at $INSTALLER"
ROOT=$(mktemp -d)
trap 'rm -rf "$ROOT"' EXIT
# The cases below run with PATH set to a directory that deliberately does not
# contain a shell, and a PATH assignment on a command also governs how that
# command is looked up — so bash has to be named absolutely or it becomes the
# thing that is missing.
BASH_BIN=$(command -v bash) || fail "host is missing 'bash'"
# A PATH containing exactly the utilities these functions use and nothing else.
# The absent-transport cases are only meaningful on a PATH where the transport
# is genuinely unresolvable, and this host (like most) has tmux in /usr/bin —
# so the system path cannot be part of the path under test.
FAKE_BIN="$ROOT/bin"
mkdir -p "$FAKE_BIN"
for utility in sed head tr awk; do
utility_path=$(command -v "$utility") || fail "host is missing '$utility'"
ln -s "$utility_path" "$FAKE_BIN/$utility"
done
if PATH="$FAKE_BIN" command -v tmux >/dev/null 2>&1; then
fail "'tmux' is resolvable on the minimal test path; absent-transport cases are not measurable"
fi
# Extract a function by its exact header, up to a closing brace in column one.
extract_function() {
local source_file="$1"
local function_name="$2"
local destination="$3"
awk -v name="$function_name" '
$0 == name "() {" { collecting = 1 }
collecting { print }
collecting && $0 == "}" { exit }
' "$source_file" > "$destination"
grep -qF "$function_name() {" "$destination" ||
fail "could not extract '$function_name' from $source_file — has it been renamed or reshaped?"
# An unterminated extraction would be a syntax error the moment it is sourced,
# but saying so here names the cause instead of leaving a bash parse error.
bash -n "$destination" ||
fail "extracted '$function_name' does not parse; the closing brace was probably not found"
}
extract_function "$DOCTOR" fleet_declared_transport "$ROOT/doctor-declared.sh"
extract_function "$DOCTOR" check_fleet_transport "$ROOT/doctor-check.sh"
extract_function "$INSTALLER" check_fleet_transport "$ROOT/installer-check.sh"
# Build a MOSAIC_HOME, optionally with a roster declaring a transport.
make_home() {
local home="$ROOT/$1"
local declared="${2-}"
rm -rf "$home"
mkdir -p "$home"
if [ -n "$declared" ]; then
mkdir -p "$home/fleet"
cat > "$home/fleet/roster.yaml" <<EOF
version: 2
generation: 1
transport: $declared
agents: []
EOF
fi
printf '%s\n' "$home"
}
# Run the doctor's check against a given home and path, capturing which
# reporter the check chose. The real `pass` prints only under `--verbose` and
# the real `warn` always prints; these stubs make both unconditional on
# purpose, because what is under test is the severity the check selects, not
# whether the default verbosity happens to show it. A check that warned where
# it should pass would otherwise be invisible here.
run_doctor_check() {
local home="$1"
local path="$2"
MOSAIC_HOME="$home" PATH="$path" "$BASH_BIN" --noprofile --norc -c '
set -euo pipefail
warn() { echo "[WARN] $*"; }
pass() { echo "[OK] $*"; }
MOSAIC_HOME="$1"
source "$2"
source "$3"
check_fleet_transport
' _ "$home" "$ROOT/doctor-declared.sh" "$ROOT/doctor-check.sh" 2>&1
}
run_installer_check() {
local home="$1"
local path="$2"
MOSAIC_HOME="$home" PATH="$path" "$BASH_BIN" --noprofile --norc -c '
set -euo pipefail
warn() { echo "[WARN] $*"; }
C="" RESET=""
MOSAIC_HOME="$1"
source "$2"
check_fleet_transport
' _ "$home" "$ROOT/installer-check.sh" 2>&1
}
# A transport that exists. Named tmux because that is what the default roster
# declares; the binary never runs, it only has to resolve.
PRESENT_BIN="$ROOT/present-bin"
mkdir -p "$PRESENT_BIN"
printf '#!/usr/bin/env bash\nexit 0\n' > "$PRESENT_BIN/tmux"
chmod +x "$PRESENT_BIN/tmux"
PATH_WITH_TMUX="$PRESENT_BIN:$FAKE_BIN"
# ── absent, no roster ────────────────────────────────────────────────────────
# Nothing has been configured yet, so the honest thing to point at is `init`.
home=$(make_home no-roster)
output=$(run_doctor_check "$home" "$FAKE_BIN")
echo "$output" | grep -qF '[WARN]' || fail "doctor did not warn when tmux was absent"
echo "$output" | grep -qF 'tmux' || fail "doctor warning did not name the transport"
echo "$output" | grep -qF 'mosaic fleet init' || fail "doctor did not point a rosterless host at init"
output=$(run_installer_check "$home" "$FAKE_BIN")
echo "$output" | grep -qF '[WARN]' || fail "installer did not warn when tmux was absent"
echo "$output" | grep -qF 'reports success and no seat comes up' ||
fail "installer warning did not say what the missing transport actually breaks"
# ── absent, roster present ───────────────────────────────────────────────────
# A configured fleet that cannot launch is a stronger statement than a
# hypothetical one, and the message says so.
home=$(make_home with-roster tmux)
output=$(run_doctor_check "$home" "$FAKE_BIN")
echo "$output" | grep -qF '[WARN]' || fail "doctor did not warn with a roster present and tmux absent"
echo "$output" | grep -qF 'roster' || fail "doctor did not mention the roster it found"
echo "$output" | grep -qF 'mosaic fleet start' || fail "doctor did not point a configured host at start"
# ── present ──────────────────────────────────────────────────────────────────
# Silence from the installer, and a pass (not a warning) from the audit.
for home_name in no-roster with-roster; do
home="$ROOT/$home_name"
output=$(run_doctor_check "$home" "$PATH_WITH_TMUX")
if echo "$output" | grep -qF '[WARN]'; then
fail "doctor warned about the transport while tmux was present ($home_name)"
fi
echo "$output" | grep -qF '[OK]' || fail "doctor did not record a pass with tmux present ($home_name)"
output=$(run_installer_check "$home" "$PATH_WITH_TMUX")
if [ -n "$output" ]; then
fail "installer was not silent with tmux present ($home_name): $output"
fi
done
# ── the roster declares something other than tmux ────────────────────────────
# The roster is read, not assumed. A host that declares a different transport
# is told about the binary it actually needs, and never about tmux — being sent
# to install the wrong package is worse than no advice at all.
home=$(make_home other-transport zellij)
output=$(run_doctor_check "$home" "$PATH_WITH_TMUX")
echo "$output" | grep -qF 'zellij' || fail "doctor ignored the roster's declared transport"
if echo "$output" | grep -qF 'tmux'; then
fail "doctor named tmux for a host whose roster declares zellij"
fi
output=$(run_installer_check "$home" "$PATH_WITH_TMUX")
echo "$output" | grep -qF 'zellij' || fail "installer ignored the roster's declared transport"
if echo "$output" | grep -qF 'tmux'; then
fail "installer named tmux for a host whose roster declares zellij"
fi
# ── a quoted or trailing-comment transport value ─────────────────────────────
# YAML permits both and neither is exotic; a check that installs `tmux"` or
# reads `tmux # default` as a binary name would send the operator nowhere.
home=$(make_home quoted-transport '"tmux" # the only transport today')
output=$(run_doctor_check "$home" "$PATH_WITH_TMUX")
echo "$output" | grep -qF '[OK] Fleet transport available: tmux' ||
fail "doctor did not parse a quoted/commented transport value: $output"
output=$(run_installer_check "$home" "$PATH_WITH_TMUX")
if [ -n "$output" ]; then
fail "installer did not parse a quoted/commented transport value: $output"
fi
echo "ok - fleet transport checks (mosaic-doctor + install.sh)"
@@ -286,6 +286,36 @@ _build_runtime_bin_prefix() {
MOSAIC_RUNTIME_BIN_PREFIX=$(_build_runtime_bin_prefix)
PANE_PATH=${MOSAIC_RUNTIME_BIN_PREFIX:+${MOSAIC_RUNTIME_BIN_PREFIX}:}/usr/local/bin:/usr/bin:/bin
# #1241. The pane runs `mosaic yolo <runtime>` under PANE_PATH with a cleared
# environment. A binary missing from *that* path is a pane that dies in under a
# second, inside a session nobody is attached to, with its diagnostic scrolled
# into a pane tmux then destroys. Resolve both here, before any effect, where
# the failure is still attributable to the thing that caused it.
#
# `mosaic yolo <runtime>` runs checkRuntime(runtime) and the binary it looks for
# is named exactly like the runtime, so resolving the runtime name is the same
# question the pane will ask a moment later — asked while an operator can still
# see the answer.
_resolve_in_pane_path() {
PATH="$PANE_PATH" command -v -- "$1" 2>/dev/null
}
# Exit 69 (EX_UNAVAILABLE): the seat cannot be provided. Distinguished from the
# 64 (EX_USAGE) rejections above, which mean the projection itself was bad —
# here the data is fine and the host is not ready. Callers tell the individual
# cases apart by `code=`, the same way fail_env's many codes share exit 64.
fail_launch() {
local code="$1"
shift
echo "ERROR: agent launch aborted: code=${code} agent=${AGENT_NAME} $*" >&2
exit 69
}
for required_binary in mosaic "$MOSAIC_AGENT_RUNTIME"; do
_resolve_in_pane_path "$required_binary" >/dev/null ||
fail_launch missing-binary "'${required_binary}' is not on the pane PATH (${PANE_PATH})"
done
_ensure_claude_workdir_trusted() {
local workdir="$1"
local resolved
@@ -384,6 +414,19 @@ if [ -n "$PANE_PID" ]; then
_start_heartbeat_sidecar "$AGENT_NAME" "$PANE_PID" \
"$MOSAIC_HEARTBEAT_RUN_DIR" "$MOSAIC_HEARTBEAT_INTERVAL" || \
echo "WARNING: heartbeat sidecar could not be started for $AGENT_NAME" >&2
elif _tmux has-session -t "=${AGENT_NAME}:0.0" 2>/dev/null; then
# #1241. Session present, no pane PID after a second of retries. Whatever this
# is, it is not a seat an operator can use, so it is not a success either.
fail_launch pane-pid-unresolved \
"tmux reports the session but no pane PID after 5 attempts"
else
echo "WARNING: could not resolve pane PID for $AGENT_NAME — heartbeat sidecar not started" >&2
# #1241. This branch used to print a WARNING about the heartbeat sidecar and
# exit 0. It is not a heartbeat problem: tmux destroys a session when its pane
# command exits, so an absent session one second after new-session means the
# runtime died on startup. Reporting it as success is what let `fleet start`
# return 0 over three dead panes — the launcher knew, and said the wrong thing
# at the wrong severity to the wrong layer.
fail_launch pane-did-not-survive \
"the pane exited immediately and tmux destroyed the session;" \
"run 'mosaic yolo ${MOSAIC_AGENT_RUNTIME}' in ${MOSAIC_AGENT_WORKDIR} to see why"
fi
@@ -23,8 +23,26 @@ index=0
if [ "${args[0]:-}" = -L ]; then index=2; fi
case "${args[$index]:-}" in
has-session)
# The holder always answers. MOSAIC_TEST_HELD_SESSIONS lets a case add
# other targets that should answer too — without it there is no way to
# model "tmux still reports the session" for a non-holder agent, and the
# launcher's pane-pid-unresolved branch is unreachable from this harness.
#
# A listed target answers only AFTER new-session, because the launcher asks
# this question twice about the same name: once before launching, where a
# yes means "already running, nothing to do, exit 0", and once after, where
# a yes means "the session survived". A shim that answered yes to both
# would short-circuit at the first and never reach the branch under test —
# it would look like coverage and measure the idempotency path instead.
for argument in "${args[@]}"; do
[ "$argument" = '=_holder:0.0' ] && exit 0
case " ${MOSAIC_TEST_HELD_SESSIONS:-} " in
*" $argument "*)
if tr '\0' '\n' < "${MOSAIC_TEST_TMUX_CALLS:?}" | grep -qxF new-session; then
exit 0
fi
;;
esac
done
exit 1
;;
@@ -62,6 +80,30 @@ env -0 > "${MOSAIC_HOME:?}/fleet/pane-environment"
SHIM
chmod +x "$FAKE_BIN/mosaic"
# The runtime the rosters below name. The launcher resolves it against PANE_PATH
# before spawning (#1241), so it has to exist somewhere the pane would find it —
# not merely on the launcher's own PATH.
printf '#!/usr/bin/env bash\nexit 0\n' > "$FAKE_BIN/pi"
chmod +x "$FAKE_BIN/pi"
# PANE_PATH is derived partly from `npm config get prefix`. Left to the real npm
# it would splice whatever the host has installed into the path under test, and
# the missing-binary cases below would pass or fail by accident of the machine.
cat > "$FAKE_BIN/npm" <<'SHIM'
#!/usr/bin/env bash
printf '%s\n' "${MOSAIC_TEST_NPM_PREFIX:-/nonexistent}"
SHIM
chmod +x "$FAKE_BIN/npm"
# PANE_PATH always ends in the system path. A host that installs these there can
# not measure the missing-binary cases at all, and a green run would mean
# nothing — so say so instead of passing.
for host_binary in mosaic pi; do
if PATH=/usr/local/bin:/usr/bin:/bin command -v "$host_binary" >/dev/null 2>&1; then
fail "host provides '$host_binary' in the system path; missing-binary cases are not measurable here"
fi
done
write_generated() {
local home="$1"
local agent="$2"
@@ -81,6 +123,19 @@ MOSAIC_TMUX_SOCKET=mosaic-test
EOF
chmod 600 "$home/fleet/agents/$agent.env.generated"
mkdir -p "$home/work"
install_pane_binaries "$home"
}
# `$PANE_HOME/.npm-global/bin` is one of the prefixes the launcher folds into
# PANE_PATH, so this is the pane's own view of "installed", distinct from the
# launcher's PATH. Tests that need a binary *absent* remove it from here.
install_pane_binaries() {
local pane_home="$1"
mkdir -p "$pane_home/.npm-global/bin"
local binary
for binary in mosaic pi; do
ln -sf "$FAKE_BIN/$binary" "$pane_home/.npm-global/bin/$binary"
done
}
run_start() {
@@ -88,6 +143,7 @@ run_start() {
local agent="$2"
HOME="$home" PATH="$FAKE_BIN:$PATH" MOSAIC_TEST_TMUX_CALLS="$TMUX_CALLS" \
MOSAIC_TEST_PANE_PID="${MOSAIC_TEST_PANE_PID:-}" \
MOSAIC_TEST_HELD_SESSIONS="${MOSAIC_TEST_HELD_SESSIONS:-}" \
MOSAIC_TEST_HOME="$home" \
MOSAIC_TEST_FLEET_OWNER=123e4567-e89b-12d3-a456-426614174000 \
MOSAIC_HOME="$home" "$START" "$agent"
@@ -98,7 +154,10 @@ run_start() {
HOME_VALID="$ROOT/valid"
AGENT_VALID="coder0"
write_generated "$HOME_VALID" "$AGENT_VALID"
run_start "$HOME_VALID" "$AGENT_VALID"
# A live pane PID is part of what "valid launch" means. Until #1241 this case
# ran with none, so the suite's one success path was itself a dead pane the
# launcher reported as fine.
MOSAIC_TEST_PANE_PID=$$ run_start "$HOME_VALID" "$AGENT_VALID"
valid_args=$(tr '\0' '\n' < "$TMUX_CALLS")
echo "$valid_args" | grep -qF new-session || fail "valid generated projection did not reach tmux"
echo "$valid_args" | grep -qF 'mosaic' || fail "fixed mosaic launcher command missing"
@@ -245,6 +304,13 @@ PANE_BASH_ENV="$ROOT/pane-boundary.bash-env"
printf 'MOSAIC_RUNTIME_BIN=%s\n' "$FAKE_BIN" > \
"$HOME_PANE_BOUNDARY/fleet/agents/coder-pane-boundary.env.local"
chmod 600 "$HOME_PANE_BOUNDARY/fleet/agents/coder-pane-boundary.env.local"
# This case does not go through run_start, so its pane binaries come from
# MOSAIC_RUNTIME_BIN=$FAKE_BIN in the env.local written above — not from the
# symlinks install_pane_binaries planted under the generated home, which this
# launcher never consults because HOME here is the trusted parent. That is a
# legitimate resolution path, but it means dropping MOSAIC_RUNTIME_BIN from
# this case on the belief that the symlinks cover it would break the #1241
# binary check rather than exercise it.
LD_PRELOAD='/not/loaded/by-clean-bootstrap.so' \
BASH_ENV="$PANE_BASH_ENV" \
MOSAIC_UNTRUSTED_SENTINEL='must-not-reach-pane' \
@@ -258,6 +324,7 @@ PATH="$PANE_STALE_PATH" \
"MOSAIC_TEST_HOME=$PANE_TRUSTED_HOME" \
MOSAIC_TEST_FLEET_OWNER=123e4567-e89b-12d3-a456-426614174000 \
MOSAIC_TEST_EXECUTE_PANE=1 \
"MOSAIC_TEST_PANE_PID=$$" \
"$START" coder-pane-boundary
pane_args=$(tr '\0' '\n' < "$TMUX_CALLS")
echo "$pane_args" | grep -qxF "HOME=$PANE_TRUSTED_HOME" || \
@@ -392,6 +459,75 @@ echo "$interaction_policy_args" | grep -qF 'new-session' && \
echo "$output" | grep -qF 'operator interaction service requires runtime pi' || \
fail "interaction pinned-policy check did not follow strict parsing"
# #1241. The pane runs `mosaic yolo <runtime>` against PANE_PATH. A binary
# missing from that path is a launch failure, and it has to be named before the
# session is created — after it, the diagnostic dies with the pane.
assert_missing_pane_binary_rejected() {
local binary="$1"
local home="$ROOT/missing-$binary"
local agent="coder-missing-$binary"
write_generated "$home" "$agent"
rm -f "$home/.npm-global/bin/$binary"
: > "$TMUX_CALLS"
local output
if output=$(MOSAIC_TEST_PANE_PID=$$ run_start "$home" "$agent" 2>&1); then
fail "launch succeeded with '$binary' absent from the pane PATH"
fi
echo "$output" | grep -qF 'code=missing-binary' || fail "missing '$binary' diagnostic missing"
echo "$output" | grep -qF "'$binary'" || fail "missing-binary diagnostic did not name $binary"
if tr '\0' '\n' < "$TMUX_CALLS" | grep -qF new-session; then
fail "launcher created a session it knew would die ($binary absent)"
fi
}
assert_missing_pane_binary_rejected mosaic
assert_missing_pane_binary_rejected pi
# #1241. tmux destroys a session when its pane command exits, so no pane PID a
# second after new-session means the runtime died on startup. This used to be a
# WARNING about the heartbeat sidecar followed by exit 0 — three layers above it
# then reported a fleet that was not running.
: > "$TMUX_CALLS"
HOME_DEAD_PANE="$ROOT/dead-pane"
write_generated "$HOME_DEAD_PANE" "coder-dead-pane"
if output=$(MOSAIC_TEST_PANE_PID='' run_start "$HOME_DEAD_PANE" coder-dead-pane 2>&1); then
fail "launcher reported success over a pane that did not survive"
fi
echo "$output" | grep -qF 'code=pane-did-not-survive' || fail "dead-pane diagnostic missing"
if echo "$output" | grep -qiF 'heartbeat'; then
fail "dead pane is still being reported as a heartbeat-sidecar problem"
fi
tr '\0' '\n' < "$TMUX_CALLS" | grep -qF new-session || \
fail "dead-pane case did not reach the launch it is measuring"
# #1241, the other way a pane fails. Above, tmux destroyed the session and
# has-session said so. Here the session is still there and no PID comes back
# after the retries — a different fault (the pane is alive but unusable, or
# tmux is answering inconsistently) that an operator has to be told apart from
# a runtime that died on startup.
#
# This case exists because the branch that handles it shipped with nothing able
# to reach it: the shim answered has-session only for the holder, so every
# non-holder agent landed in the session-is-gone branch no matter what. A
# defensive branch nothing exercises is the same shape as the bug this whole
# change is about, one layer down.
: > "$TMUX_CALLS"
HOME_NO_PID="$ROOT/pane-no-pid"
write_generated "$HOME_NO_PID" "coder-no-pid"
if output=$(MOSAIC_TEST_PANE_PID='' MOSAIC_TEST_HELD_SESSIONS='=coder-no-pid:0.0' \
run_start "$HOME_NO_PID" coder-no-pid 2>&1); then
fail "launcher reported success over a session with no resolvable pane PID"
fi
echo "$output" | grep -qF 'code=pane-pid-unresolved' || \
fail "session-present/no-PID was not reported as pane-pid-unresolved: $output"
if echo "$output" | grep -qF 'code=pane-did-not-survive'; then
fail "a session tmux still reports was diagnosed as a destroyed session"
fi
if echo "$output" | grep -qiF 'heartbeat'; then
fail "an unresolvable pane PID is still being reported as a heartbeat-sidecar problem"
fi
# Exact stop derives the socket exclusively from the validated generated
# projection and ignores an ambient socket supplied by the caller.
: > "$TMUX_CALLS"
+1 -1
View File
@@ -25,7 +25,7 @@
"lint": "eslint src",
"typecheck": "tsc --noEmit",
"test": "vitest run --passWithNoTests && pnpm run test:framework-shell",
"test:framework-shell": "bash framework/tools/quality/scripts/check-test-enumeration.sh && bash framework/tools/quality/scripts/test-check-test-enumeration.sh && python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/promotion_binding_unittest.py && python3 src/lease-broker/promotion_trigger_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/receipt_observer_client_unittest.py && python3 src/lease-broker/invariant_r_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-ci-queue-wait-tristate.sh && bash framework/tools/git/test-ci-queue-wait-github-checks.sh && bash framework/tools/git/test-pr-merge-queue-branch.sh && bash framework/tools/git/test-pr-merge-head-pin.sh && bash framework/tools/git/test-pr-merge-message-field.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/woodpecker/test-terminal-green-contract.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/_scripts/test-mosaic-init-rce.sh && bash framework/tools/_scripts/test-fleet-transport-check.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-preimage.sh && bash framework/tools/wake/test-wake-install.sh"
"test:framework-shell": "bash framework/tools/quality/scripts/check-test-enumeration.sh && bash framework/tools/quality/scripts/test-check-test-enumeration.sh && python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/promotion_binding_unittest.py && python3 src/lease-broker/promotion_trigger_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/receipt_observer_client_unittest.py && python3 src/lease-broker/invariant_r_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-ci-queue-wait-tristate.sh && bash framework/tools/git/test-ci-queue-wait-github-checks.sh && bash framework/tools/git/test-pr-merge-queue-branch.sh && bash framework/tools/git/test-pr-merge-head-pin.sh && bash framework/tools/git/test-pr-merge-message-field.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/woodpecker/test-terminal-green-contract.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/_scripts/test-mosaic-init-rce.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-preimage.sh && bash framework/tools/wake/test-wake-install.sh"
},
"dependencies": {
"@mosaicstack/brain": "workspace:*",
-42
View File
@@ -309,43 +309,6 @@ require_cmd() {
fi
}
# Fleet transport binary (#1240).
#
# `mosaic fleet --help` reads "Manage the local Mosaic tmux fleet" and every
# roster the CLI scaffolds sets `transport: tmux`, but nothing in this script
# provides tmux and, until now, nothing in it mentioned tmux at all. A
# greenfield host came out of this installer able to install a fleet, start a
# fleet, and run no seat — the operator's first signal was `mosaic fleet ps`.
#
# Not a `require_cmd`: tmux is required by the fleet, not by mosaic. Plenty of
# hosts install this to run `mosaic claude` and will never scaffold a roster,
# and failing their install over a binary they do not need would be wrong. It
# is a warning that names precisely what it blocks.
#
# `tools/_scripts/mosaic-doctor` carries a deliberately parallel check, so the
# same host state gets the same answer from an audit as from an install. They
# are separate implementations because this one has to work before the
# framework's scripts are guaranteed to be on disk; keep their wording in step.
check_fleet_transport() {
local transport=tmux
local roster="$MOSAIC_HOME/fleet/roster.yaml"
local declared=""
if [[ -f "$roster" ]]; then
declared="$(sed -n 's/^[[:space:]]*transport:[[:space:]]*//p' "$roster" | head -1 |
tr -d '"'\''' | tr -d '\r' | awk '{print $1}')"
[[ -n "$declared" ]] && transport="$declared"
fi
command -v "$transport" &>/dev/null && return 0
warn "Fleet transport '$transport' is not installed."
echo " The Mosaic fleet runs its agent seats inside $transport. Without it,"
echo " ${C}mosaic fleet start${RESET} reports success and no seat comes up."
echo " Install it before using the fleet, e.g. ${C}sudo apt-get install -y $transport${RESET}"
echo " (this does not affect ${C}mosaic claude${RESET} or the other single-runtime commands)."
}
installed_cli_version() {
local json
json="$(npm ls -g --depth=0 --json --prefix="$PREFIX" 2>/dev/null)" || true
@@ -907,11 +870,6 @@ if [[ "$FLAG_CHECK" == "false" ]]; then
ok "Done."
fi
# Fleet readiness (#1240). Runs in both normal and --check mode: "what is the
# state of this host" is exactly the question --check is asked, and a host that
# cannot run a seat should not have to discover it from `fleet ps`.
check_fleet_transport
} # end main
main "$@"