docs(fleet): add operator configuration guide
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Jarvis
2026-07-16 09:27:43 -05:00
parent 9745bc3f29
commit 0aee2c0981
25 changed files with 768 additions and 77 deletions

View File

@@ -1,6 +1,25 @@
# Fleet Control-Plane CLI
The local roster-v2 control plane is `mosaic fleet`.
The local desired-state surface is `mosaic fleet`. It is distinct from the gateway-backed `mosaic agent` catalog and from legacy compatibility commands that act on roster v1.
## Roster-v2 desired-state commands
| Command | Effect | Generation | Output |
| ---------------------------------------------------------- | ---------------------------------------------- | ---------- | -------------------------- |
| `mosaic fleet get <name>` | Read one authoritative agent | no | One JSON object |
| `mosaic fleet plan <create\|update\|delete> ...` | Validate proposed CRUD and projections | required | One JSON object; no writes |
| `mosaic fleet create ... [--dry-run] [--persisted-start]` | Add desired state; default enabled/stopped | required | One JSON object |
| `mosaic fleet update <name> ... [--dry-run]` | Replace mutable agent fields | required | One JSON object |
| `mosaic fleet delete <name> ... [--dry-run]` | Remove roster member/generated projection | required | One JSON object |
| `mosaic fleet apply ... [--dry-run]` | Plan or converge projections/lifecycle | required | One JSON object |
| `mosaic fleet reconcile ... [--dry-run]` | Alias of the same convergence contract | required | One JSON object |
| `mosaic fleet start\|stop\|restart [name] ... [--dry-run]` | Exact one-shot lifecycle action | required | One JSON object |
| `mosaic fleet status [name]` | Observe desired/managed/runtime state | no | One JSON object |
| `mosaic fleet verify` | Strict observational drift/ownership gate | no | One JSON object |
| `mosaic fleet doctor` | Classify local drift and recovery context | no | One JSON object |
| `mosaic fleet migrate-v1 preview ...` | Non-mutating field-complete migration evidence | no | One JSON object |
CRUD syntax and full payload shape are documented in [agent mutations](agent-mutations.md). Reconciliation syntax:
```text
mosaic fleet apply --expected-generation <n> [--dry-run]
@@ -14,14 +33,14 @@ mosaic fleet doctor
mosaic fleet migrate-v1 preview --source <path> --decisions <path> --observations <path>
```
`migrate-v1 preview` is non-mutating: it emits value-free v1 inventory, a canonical semantically
validated v2 candidate when ready, sanitized environment dispositions, and non-executable recovery
evidence. It has no write, apply, canary, or rollback option. Missing preview inputs also return one stable
blocked JSON object and a non-zero exit, rather than Commander text. See
[the migration preview contract](../migration/v1-to-v2.md).
`get` is the read/show operation for one v2 agent. Full roster parsing and semantic validation occur on every v2 mutation/reconcile path; there is no separate mutable “config store.” The executable JSON Schema and validated example provide offline structural evidence.
`apply` and `reconcile` use roster desired state. `start`, `stop`, and `restart` are exact local one-shot lifecycle effects and never persist a desired-state edit. `status`, `verify`, and `doctor` are observational.
## JSON and exit behavior
Commands emit one JSON object. Handled precondition errors emit `{ "error": { "code": "..." } }` and exit non-zero. Partial derived/lifecycle effects use explicit `authoritativeRoster`, `projections`, `lifecycle`, and bounded `recovery` fields; they never claim rollback. Any additive `cleanup` diagnostic also exits non-zero, even where known effects are complete: it is not a clean completion and the lock requires inspection before retry.
Roster-v2 CRUD and reconciler precondition failures emit `{ "error": { "code": "..." } }` and exit non-zero. Migration preview has its own result envelope: a non-ready preview emits `{ "status": "blocked", "blockers": [...] }` and exits non-zero rather than using the CRUD/reconciler error object. Use both exit status and command-specific state fields. A partial reconciliation result distinguishes `authoritativeRoster`, `projections`, `lifecycle`, `recovery`, and optional `cleanup`; it never claims automatic rollback. `verify` exits non-zero for drift, ownership failure, or unmanaged sessions. Sensitive legacy values, credentials, and rejected command text are never printed.
This control plane is separate from the gateway-backed `mosaic agent` catalog. It is local-only and rejects remote/connector lifecycle mutation, arbitrary command/channel/secret input, and unproven tmux ownership.
## Compatibility and scope
Roster-v1 initialization, provisioning, profiles/personas, and historical `fleet add/remove` remain compatibility surfaces, not roster-v2 CRUD aliases. New v2 automation should use the table above. `migrate-v1 preview` writes nothing and has no cutover, canary, or rollback option.
`mosaic agent` is a separate catalog/transport surface; it does not own `<MOSAIC_HOME>/fleet/roster.yaml` desired state. Remote/SSH reconciliation, connector mutation, arbitrary commands/channels, secret references, and gateway convergence are rejected or outside scope.

View File

@@ -1,6 +1,6 @@
# Fleet Generated Environment Boundary
**Card:** FCM-M2-001 · **Issue:** #758 · **Status:** unreleased/card-local
**Card:** FCM-M2-001 · **Issue:** #758 · **Status:** merged contract
The local fleet roster is the desired-state authority. A launch reads a deterministic,
roster-derived generated projection and an optional strictly data-only local file; neither file is
@@ -78,12 +78,12 @@ This card does not add a USC site file, write a USC roster, or run a site canary
consolidated downstream interface packet. Status is deliberately separated from checkout presence: no
product release version has been evidenced for this interface set.
| Interface | Canonical public path and version | Tracker/release status | Downstream limit |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| M1 structural compiler | `parseRosterV2` in `packages/mosaic/src/fleet/roster-v2.ts`; schema `docs/fleet/reference/roster-v2.schema.json`; roster `version: 2` | FCM-M1-001 is recorded done, merged as #764 (`aa5b43b`); no released product version is asserted here. | Parse YAML/JSON and canonicalize a supplied v2 site roster without writes. |
| M1 semantic resolver | `validateRosterV2Semantics` in `packages/mosaic/src/fleet/roster-v2.ts`; baseline `framework/fleet/roles/` plus `roles.local/` | FCM-M1-002 remains `in-progress` in `docs/TASKS.md`; unreleased. | Reuse the shared resolver only; no parallel role resolver or lifecycle action. |
| M1 disposition evidence | `packages/mosaic/src/fleet/example-profile-dispositions.ts`; `docs/fleet/migration/example-profile-disposition.md`; retained fixture `version: 1` | FCM-M1-003 remains `not-started` in `docs/TASKS.md`; unreleased even though these checkout artifacts are inspectable. | Inspect fixture/profile/service disposition evidence only; it is not migration authorization. |
| M2 generated boundary | `packages/mosaic/src/fleet/generated-env-boundary.ts`; generated projection contract in this document | FCM-M2-001 card-local and uncommitted; unreleased. | Render/write a roster-derived projection; local input is never authority. |
| Interface | Canonical public path and version | Tracker/release status | Downstream limit |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| M1 structural compiler | `parseRosterV2` in `packages/mosaic/src/fleet/roster-v2.ts`; schema `docs/fleet/reference/roster-v2.schema.json`; roster `version: 2` | FCM-M1-001 is recorded done, merged as #764 (`aa5b43b`); no released product version is asserted here. | Parse YAML/JSON and canonicalize a supplied v2 site roster without writes. |
| M1 semantic resolver | `validateRosterV2Semantics` in `packages/mosaic/src/fleet/roster-v2.ts`; baseline `framework/fleet/roles/` plus `roles.local/` | FCM-M1-002 merged as #768 (`a5e8e55`); no released product version is asserted here. | Reuse the shared resolver only; no parallel role resolver or lifecycle action. |
| M1 disposition evidence | `packages/mosaic/src/fleet/example-profile-dispositions.ts`; `docs/fleet/migration/example-profile-disposition.md`; retained fixture `version: 1` | FCM-M1-003 merged as #770 (`e9c4aa3`); checkout evidence remains validation, not migration authorization. | Inspect fixture/profile/service disposition evidence only; it is not migration authorization. |
| M2 generated boundary | `packages/mosaic/src/fleet/generated-env-boundary.ts`; generated projection contract in this document | FCM-M2-001 merged as #772 (`191efae`); no released product version is asserted here. | Render/write a roster-derived projection; local input is never authority. |
The canonical source remains `<MOSAIC_HOME>/fleet/roster.yaml` for the current local fleet path.
Generated environment data is a rebuildable projection, not an operator-editable source of membership,

View File

@@ -1,14 +1,21 @@
# Local Fleet Lifecycle Transitions
FCM-M3-001 uses the roster-v2 `lifecycle.enabled` and `lifecycle.desired_state` fields as the only desired-state authority. Systemd, tmux, generated environment files, and heartbeats are derived or observed state.
Roster-v2 `lifecycle.enabled` and `lifecycle.desired_state` are the only persisted lifecycle authority. Systemd, tmux, generated environment, and heartbeat state are derived or observed.
| Command | Desired-state write | Runtime effect | Preconditions |
| --------------------------------- | ------------------- | -------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `fleet apply` / `fleet reconcile` | Never | Rebuilds projections, then starts only enabled agents desired `running`; stops disabled or desired-`stopped` roster agents | Current generation; private managed paths; valid projections; proven holder ownership; no unmanaged named-socket sessions |
| `fleet start <name>` | Never | One-shot exact `mosaic-agent@<name>.service` start | Current generation; exact enabled roster name; proven ownership |
| `fleet stop <name>` | Never | One-shot exact service stop | Current generation; exact roster name; proven ownership |
| `fleet restart <name>` | Never | One-shot exact service restart | Current generation; exact roster name; proven ownership |
| Event | Desired-state write | Runtime effect | Safety boundary |
| --------------------------- | -------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `fleet create` | Adds enabled/stopped by default; `--persisted-start` records running | None | Generation-guarded; validates full roster/projections. |
| `fleet update` | May change enabled/desired state in complete payload | None | Generation-guarded; stable name immutable. |
| `fleet delete` | Removes exact roster member | None | Removes only generated projection; retains local/quarantine evidence. |
| `fleet apply` / `reconcile` | Never | Rebuilds projections; starts only enabled/running; stops disabled or stopped roster members | Current generation, private lock/paths, semantic validity, holder ownership, no unmanaged named-socket sessions. |
| `fleet start <name>` | Never | One-shot exact service start | Exact enabled roster name and proven ownership. |
| `fleet stop <name>` | Never | One-shot exact service stop | Exact roster name and proven ownership. |
| `fleet restart <name>` | Never | One-shot exact service restart | Exact enabled roster name and proven ownership. |
| Reboot/service activation | Never | Current installation may activate enabled units without honoring roster lifecycle | **Held for FCM-M3-002:** boot preservation for stopped/disabled agents is not yet proven; inspect/disable units rather than assuming lifecycle-safe reboot. |
| v1 migration preview | Never | None | Observed active+present maps running; inactive+missing maps stopped; ambiguity blocks. |
| Cutover/canary | Held for FCM-M4-002 | Not implemented by preview | Must preserve every observed stopped state. |
| Rollback | Held for FCM-M4-002 | Not implemented | Must restore selected authority/projections without surprise starts or unmanaged targeting. |
A stopped roster agent is never started by `apply` or `reconcile`. Direct lifecycle commands are explicit one-shot actions and do not change persisted desired state. Use roster CRUD with the explicit persisted-start option to change that desired state.
Explicit apply/reconcile never starts a stopped roster agent. Direct lifecycle commands are explicit one-shot actions and do not persist intent. Change durable lifecycle only through generation-guarded CRUD and then review reconciliation. Reboot preservation for stopped/disabled agents is not yet guaranteed because current enabled units and launcher projections do not carry the persisted lifecycle fence; that acceptance evidence remains FCM-M3-002.
All mutations require `--expected-generation <n>` and acquire one private roster-adjacent reconciliation lock before projection or lifecycle effects. Missing or stale generations and concurrent writers fail before effects; the lock is released after success, partial failure, or thrown lifecycle failure. Stale, ownership, unmanaged-session, unsupported-runtime, path, projection, and lifecycle-precondition failures return stable redacted JSON errors and a non-zero exit. No command targets a fuzzy tmux name, arbitrary socket, arbitrary command, channel, secret, or generated file as authority.
Missing/stale generation, concurrent writer, unsafe path, ownership mismatch, unmanaged session, unsupported runtime, invalid projection, and lifecycle precondition failures return stable redacted JSON and non-zero status. No command targets fuzzy names, arbitrary sockets/commands/channels/secrets, or generated files as authority. Legacy sensitive values are never printed.

View File

@@ -50,18 +50,20 @@ agents:
## Root fields
| Field | Required | Constraint | Meaning |
| ------------ | -------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| `version` | yes | integer constant `2` | Identifies this contract. Version `1` is explicitly rejected by this compiler and remains on the existing v1 path until M4 migration. |
| `generation` | yes | positive safe integer | Desired-state generation. M2 uses it for mutation guards; M1 does not mutate it. |
| `transport` | yes | constant `tmux` | M1M5 support local tmux only. |
| `tmux` | yes | strict object | Explicit local socket and holder-session configuration. |
| `defaults` | yes | strict object | Default work directory and one supported local runtime. |
| `runtimes` | yes | non-empty object | Declared local runtime reset policy map. |
| `agents` | yes | non-empty array | Local fleet entries. Duplicate stable names are rejected. |
| Field | Required | Default | Constraint | Meaning |
| ------------ | -------- | ------- | --------------------- | ------------------------------------------------------------------------------------------------- |
| `version` | yes | none | integer constant `2` | Identifies this contract. Version `1` stays on the compatibility path pending explicit migration. |
| `generation` | yes | none | positive safe integer | Desired-state generation and mutation/reconcile concurrency fence. |
| `transport` | yes | none | constant `tmux` | M1M5 support local tmux only. |
| `tmux` | yes | none | strict object | Explicit local socket and holder-session configuration. |
| `defaults` | yes | none | strict object | Default work directory and one supported local runtime. |
| `runtimes` | yes | none | non-empty object | Declared local runtime reset policy map. |
| `agents` | yes | none | non-empty array | Local fleet entries. Duplicate stable names are rejected. |
## Nested fields
All nested fields in the v2 schema are required and have no implicit default. CRUD `create` is the only higher-level convenience: it records `lifecycle.enabled: true` and `desired_state: stopped` unless `--persisted-start` explicitly records running. That convenience still performs no runtime action.
| Path | Required | Constraint |
| ---------------------------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------- |
| `tmux.socket_name` | yes | `[A-Za-z0-9_.-]*`; empty string means the literal default tmux server, while a non-empty value names a socket |

View File

@@ -1,13 +1,25 @@
# Local Fleet Status and Drift
`mosaic fleet status [name]`, `verify`, and `doctor` are observational roster-v2 commands. They emit one JSON result and do not write projections, change desired state, start services, stop services, restart services, or mutate tmux.
`mosaic fleet status [name]`, `verify`, and `doctor` are observational roster-v2 commands. They emit one JSON result and do not write projections, mutate desired state, operate lifecycle, or change tmux.
The report distinguishes:
## State dimensions
- `missing-session`: an enabled agent desired `running` has no exact roster-named tmux session.
- `unexpected-session`: a desired-`stopped` agent still has its exact session.
- `disabled-running`: a disabled roster agent has its exact session.
- `unmanagedSessions`: sessions on the configured named socket that are neither the exact holder nor an exact roster agent.
- `holder`: `owned`, `missing`, or `ownership-mismatch` after exact holder, private install identity, and complete global tmux environment validation.
- **Desired:** roster membership, generation, enabled flag, and persisted running/stopped target.
- **Managed/derived:** generated environment and expected exact service/session topology.
- **Observed by current roster-v2 commands:** systemd active state, tmux presence, exact holder ownership, and unmanaged sessions.
`doctor` and `status` classify rather than adopt, destroy, or repair unmanaged state. `verify` is observational too, but exits non-zero if ownership cannot be proven, unmanaged sessions exist, or drift is present. Reconciliation fails closed under those conditions and never kills or adopts an unmanaged session.
Implemented drift classifications include:
- `missing-session`: enabled/desired-running agent lacks its exact session;
- `unexpected-session`: desired-stopped agent has its exact session;
- `disabled-running`: disabled roster agent has its exact session;
- `unmanagedSessions`: named-socket sessions that are neither exact holder nor roster agent;
- `holder`: `owned`, `missing`, or `ownership-mismatch` after private identity and global environment checks.
Generated projection failures/staleness are surfaced by plan/apply preparation and bounded recovery fields rather than adopted as configuration. Heartbeat remains wider-fleet observational evidence, never desired state, but the current roster-v2 `status`, `doctor`, and `verify` commands do not read heartbeat files. A provable removed-agent projection may be treated as stale derived state during deletion, but general projection-orphan classification and installed source-versus-asset revision mismatch remain FCM-M4-002/M5-002 holds; current commands must not claim those future checks.
## Command behavior
`status` and `doctor` classify rather than adopt, destroy, or repair. `verify` is observational too, but exits non-zero if ownership cannot be proven, unmanaged sessions exist, or drift is present. Reconciliation fails closed under those conditions and never kills or adopts an unmanaged session.
Doctor/error output uses stable codes and bounded recovery context. Migration, quarantine, lifecycle, status, and troubleshooting output never prints a legacy sensitive value, credential, or privileged command text.