feat(fleet): reconcile local roster state (#785)
All checks were successful
ci/woodpecker/push/publish Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful

This commit was merged in pull request #785.
This commit is contained in:
2026-07-15 15:03:31 +00:00
parent c593a15ef8
commit 499090508e
12 changed files with 1888 additions and 45 deletions

View File

@@ -0,0 +1,39 @@
# FCM-M3-001 — Local roster-owned reconciliation and lifecycle
- **Task / issue:** FCM-M3-001 / #758
- **Branch / base:** `feat/758-local-reconciler` from `origin/main` `bc5e73629e92c56a80fa6a769ebad17c0177f504`
- **Base tree:** `1b9ebe4fa1a90734b6f81118e120bae5290cd350`
- **Scope:** source, isolated fake-adapter tests, and card documentation only. No live fleet/systemd/tmux action.
## Objective
Provide local roster-v2 `apply`/`reconcile` and lifecycle/status contracts. The roster remains desired-state authority; projections and runtime observations are derived state.
## Red-first evidence
The initial focused reconciler test failed because `fleet-reconciler.ts` did not exist. The initial new-worktree test invocation also exposed absent dependencies; `pnpm install --frozen-lockfile --store-dir /home/jarvis/.local/share/pnpm/store` restored local workspace dependencies without changing source.
## Design
- A new `fleet-reconciler.ts` accepts only typed roster-v2 input plus injected command and projection adapters.
- It targets only exact `mosaic-agent@<roster-name>.service` units and the exact configured tmux socket/session.
- It classifies unowned/unmanaged state and fails mutation closed rather than adopting or killing it.
- It validates the private install-derived holder identity and complete expected tmux global environment before mutating lifecycle state.
- REVIEW-1 remediation: RED review evidence found service-level apply could omit generation and had no mutation lock. Every non-observational command now requires an expected generation; a private exclusive roster-adjacent lock is acquired before effects and released on success or partial failure. Tests cover missing/stale values, concurrent denial, no effects, release, and lock-free observation.
- REVIEW-2 remediation: lock acquisition now validates private real `MOSAIC_HOME`/`fleet` ancestors, rejects symlink or unsafe leaves, distinguishes `EEXIST` concurrency from other I/O, and binds release to the created inode plus random ownership token. A replacement lock is retained and reported, not unlinked. A crash may leave a stale lock for inspection; no stale-lock break is claimed.
- REVIEW-3 remediation: a lock cleanup failure now adds bounded `cleanup` diagnostics to a known successful or partial effect result without replacing its projection/lifecycle/recovery truth. Cleanup is not claimed as complete, and the retained lock requires inspection before retry.
- REVIEW-4 remediation: command JSON with an additive cleanup diagnostic now exits non-zero even where known effects completed; clean effect and observational JSON remain zero-exit.
- REVIEW-5 remediation: mutating operations acquire the private lock before rereading canonical `roster.yaml`; the fenced reread, not a caller snapshot, supplies generation validation, plan, projection, and lifecycle authority.
- `apply` starts only enabled agents whose persisted desired state is `running`; stopped/default agents are never started by reconciliation.
- Observational commands produce JSON classification only. Partial projection or lifecycle effects report explicit recovery without values.
## Boundaries
Excluded: live host actions, remote/SSH/connector lifecycle mutation, migrations, canaries, deployment, gateway changes, arbitrary command/channel/secret inputs, `docs/TASKS.md`, and orchestration ledgers.
## Verification
- Focused reconciler/Commander/CRUD/fleet tests: 4 files / 229 tests passed.
- Full `@mosaicstack/mosaic` suite: 56 files / 820 tests passed after REVIEW-5 canonical roster fencing remediation.
- Package and root typecheck/lint, root format check, and `git diff --check`: passed.
- Isolated launcher and systemd template harnesses passed; they use fixtures only. No live fleet, systemd, tmux, session, remote, connector, or runtime action occurred.