Files
stack/docs/guides/fleet-local-canary.md
jason.woltje b5c1381e45
All checks were successful
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/push/publish Pipeline was successful
fix(fleet): harden operator sends for release (#565)
2026-06-20 20:41:11 +00:00

145 lines
4.8 KiB
Markdown

# Local Fleet Canary
The local fleet canary runs a small tmux-backed Mosaic agent fleet on an
isolated tmux socket. The default socket is `mosaic-factory`; the commands do
not use or stop the default tmux server.
## Files
Product-owned defaults:
- `packages/mosaic/framework/fleet/roster.schema.json`
- `packages/mosaic/framework/fleet/examples/minimal.yaml`
- `packages/mosaic/framework/fleet/examples/local-canary.yaml`
- `packages/mosaic/framework/systemd/user/mosaic-tmux-holder.service`
- `packages/mosaic/framework/systemd/user/mosaic-agent@.service`
- `packages/mosaic/framework/tools/fleet/start-agent-session.sh`
- `packages/mosaic/framework/tools/tmux/agent-send.sh`
- `packages/mosaic/framework/tools/tmux/send-message.sh`
These files are published through `packages/mosaic/package.json`, whose `files`
allowlist includes `framework` along with `dist`.
Site-owned local roster:
```text
~/.config/mosaic/fleet/roster.yaml
```
Do not put a host-specific full roster into product defaults. Start from an
example and edit the local roster after `mosaic fleet init --write`.
## Install
Minimal canary:
```bash
mosaic fleet init --profile minimal --write
# If a site-owned roster already exists, inspect it first; overwrite only explicitly:
# mosaic fleet init --profile minimal --write --force
mosaic fleet install-systemd
systemctl --user daemon-reload
mosaic fleet start
mosaic fleet verify
```
Small dogfood roster:
```bash
mosaic fleet init --profile local-canary --write
# Use --force only after preserving any site-owned roster changes.
mosaic fleet install-systemd
systemctl --user daemon-reload
mosaic fleet start
mosaic fleet status
```
## Agent Operations
```bash
mosaic agent roster
mosaic agent status
mosaic agent status canary-pi
mosaic agent send canary-pi --message "status check"
mosaic agent reset canary-pi --new
mosaic agent tail canary-pi -n 80
```
These commands read the roster and target the configured tmux socket. The
generated systemd agent services use `start-agent-session.sh`; message delivery
uses the tmux send tools with `-L mosaic-factory`.
`mosaic agent send` is operator-origin traffic unless a caller explicitly says
otherwise. The CLI always passes a deterministic source label to
`agent-send.sh` with `-S`, defaulting to `<hostname>:operator`, so it does not
query the target tmux socket and accidentally identify as an active agent pane.
Use `--source-label <label>` or `--source <label>` only when deliberately
impersonating a known handoff lane. The lower-level inter-agent wrapper
`agent-send.sh -S <label>` remains the explicit source override for scripts.
## Verification
Use these checks before expanding the roster:
```bash
tmux -L mosaic-factory ls
tmux ls
mosaic fleet verify
systemctl --user status mosaic-tmux-holder.service
```
Expected results:
- `tmux -L mosaic-factory ls` shows `_holder` and roster agent sessions.
- `tmux ls` shows only the default tmux server sessions and is not changed by
fleet start/stop operations.
- `mosaic fleet verify` checks exact session targets on the isolated socket.
- `systemctl --user status ...` may show `active (exited)` for oneshot units;
that means the unit ran, not that an agent pane is live. Treat tmux
`has-session`, `list-panes`, process tree, and logs as the liveness evidence.
## Release Preflight
Run this checklist before cutting or dogfooding a fleet release:
- Real AI dogfood: send at least one task through `mosaic agent send`, then
confirm the agent accepted/responded using pane, process, or log evidence.
- Restart/stop/idempotency: run `mosaic fleet start`, `restart`, `stop`, and a
repeated `start` against the named socket; verify the default tmux server is
unchanged.
- Liveness verification: run `mosaic fleet verify` and confirm roster sessions
with `tmux -L mosaic-factory ls` or exact `has-session` checks.
- Package dry-run: run `npm pack --dry-run --json` from `packages/mosaic` and
confirm `framework/fleet`, `framework/systemd/user`,
`framework/tools/fleet`, and `framework/tools/tmux` assets are included.
- Mosaic update test: install or upgrade from the packed artifact in a temporary
Mosaic home and confirm `mosaic update` or the release upgrade path does not
remove local roster/config files.
## Rollback
Stop the local canary:
```bash
mosaic fleet stop
systemctl --user disable mosaic-agent@canary-pi.service
systemctl --user disable mosaic-tmux-holder.service
systemctl --user daemon-reload
```
For a full local cleanup of generated canary files:
```bash
rm -f ~/.config/systemd/user/mosaic-agent@.service
rm -f ~/.config/systemd/user/mosaic-tmux-holder.service
rm -rf ~/.config/mosaic/fleet
rm -rf ~/.config/mosaic/tools/fleet
```
This rollback leaves the default tmux server untouched. If a canary session is
still present after service stop, remove only the isolated socket server:
```bash
tmux -L mosaic-factory kill-server
```