test(fleet): cover reconciler lifecycle gates

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

docs/scratchpads/758-fcm-m3-002-reconciler-lifecycle-gates.md

@@ -0,0 +1,83 @@
+# FCM-M3-002 — Reconciler lifecycle acceptance gates
+
+- **Task / issue:** FCM-M3-002 / mosaicstack/stack#758
+- **Branch:** `test/758-reconciler-lifecycle-gates`
+- **Required starting head:** `499090508ef1d768660e4d54e7934cbcf13cb1cd`
+- **Required starting tree:** `2f1bb7fed48291f3f7ba8b21c2b52491aa14fe2b`
+- **Scope:** isolated acceptance coverage and card-required evidence/tracking only; no live fleet, systemd, tmux, session, site, migration, canary, deployment, runtime, connector, or remote action.
+- **Budget:** use the task estimate of 25K as the working cap; keep the delta to one coherent acceptance suite plus required task/scratchpad evidence. No production change unless a failing reproducer proves an in-scope defect.
+
+## Intake evidence
+
+- Clean exact local branch/head/tree verified before editing.
+- `origin/test/758-reconciler-lifecycle-gates` fetched and verified at the same required head.
+- The Mosaic PR wrapper reported no open pull requests, so no open-PR branch collision exists.
+- Parent issue #758 is open and remains intentionally open through M5.
+- Requirements loaded from `docs/PRD.md` FCM requirements and `AC-FCM-05`, `docs/TASKS.md` FCM DAG, the M3 rows in `docs/fleet/FLEET-CONFIG-DOCS-IA-CHECKLIST.md`, and the FCM-M3-001 implementation scratchpad.
+
+## Objective
+
+Add broad, behavior-oriented acceptance evidence around the shipped local reconciler contracts. Exercise only injected fake systemd/tmux adapters and temporary filesystem fixtures. Prove exact ownership/targeting, persisted stopped-state safety, truthful partial-failure recovery, rollback behavior, and stable command JSON/exit outcomes without touching live services or sessions.
+
+## Acceptance mapping and evidence
+
+| FCM-M3-002 acceptance concern | Delivered isolated evidence |
+| --- | --- |
+| Systemd/tmux lifecycle | `fleet-reconciler.acceptance.spec.ts` drives apply, reconcile, stop, restart, status, and recovery reconcile through one stateful injected fake host. The fake models exact systemd effects and tmux session observations; no host commands run. |
+| Drift | Named/default socket matrix classifies `missing-session`, `unexpected-session`, and `disabled-running`, including combined drift, while asserting observation emits no lifecycle mutation. |
+| Exact default/named socket targeting | Matrix asserts named calls use exactly `-L mosaic-fleet` and default-socket calls contain no `-L`; recorded calls cannot target an adjacent socket. |
+| Unmanaged-session classification | Stateful fixtures report sorted `coder0-shadow`/`unmanaged` sessions, then prove an exact roster stop leaves both sessions and the near-collision service intact. |
+| Crash/partial failure | Injected restart failure is applied after the fake effect to model crash/partial truth: result is `lifecycle: incomplete`, the roster is unchanged, and observed runtime may be active. |
+| Rollback/recovery semantics | M3 has no rollback command and explicitly does not claim automatic rollback. The acceptance workflow proves the bounded recovery contract: inspect, then exact reconcile restores the persisted stopped target without a start or fuzzy effect. M4 migration/canary rollback remains outside this card. |
+| Stopped-state preservation | Stateful apply and reconcile both stop an initially running observed agent whose persisted target is stopped; failed explicit restart leaves desired state stopped; recovery reconcile restores stopped state. No start call is emitted. |
+| Zero fuzzy destructive targeting | Near-collision `coder0-shadow` service/session plus `unmanaged` session remain untouched. The recorded destructive calls contain only exact `mosaic-agent@coder0.service`; no tmux kill action is emitted. |
+| Stable JSON/exit behavior | Temporary canonical roster fixture invokes the CLI boundary and asserts exactly one JSON line, exact partial-result shape, and exit code 1. Existing focused command specs continue to cover clean zero-exit and stable error JSON. |
+| Redacted truthful recovery | Fake stderr includes `PASSWORD=acceptance-secret`; exact CLI JSON contains only bounded recovery metadata and excludes the key, value, and raw diagnostic. |
+
+## Plan
+
+1. Inventory existing reconciler and command specs against the table above; avoid duplicating narrow assertions already present.
+2. Add one acceptance-level spec using only fake/injected adapters and temporary files.
+3. If a real defect is exposed, preserve the failing reproducer and make only the smallest FCM-M3-002-required fix; otherwise leave production unchanged.
+4. Reconcile `docs/TASKS.md` only for delivered M1/M2/M3-001 truth and mark FCM-M3-002 in progress.
+5. Run focused tests, full `@mosaicstack/mosaic` tests, package/root typecheck and lint, Prettier/format and diff checks, plus adversarial fake-runner cases.
+6. Record exact evidence and leave the tree uncommitted for independent synthetic-tree review.
+
+## TDD decision
+
+This card adds acceptance coverage to already-delivered behavior. Test-first applies to any product defect discovered: retain a failing reproducer before an in-scope fix. If the shipped behavior already satisfies the acceptance contract, no production code will be changed and the acceptance suite itself is the deliverable.
+
+## Progress
+
+- Intake and immutable baseline verification complete.
+- Existing coverage inventory confirmed strong unit coverage but no stateful cross-command lifecycle acceptance harness.
+- Added `packages/mosaic/src/fleet/fleet-reconciler.acceptance.spec.ts`: one injected stateful fake systemd/tmux host, temporary canonical roster fixture, five acceptance tests (six matrix cases).
+- Production source is unchanged; no product defect requiring an FCM-M3-002 fix was found.
+- `docs/TASKS.md` reconciles only merged M1/M2/M3-001 truth and marks FCM-M3-002 in progress.
+
+## Verification evidence
+
+All commands ran from `/home/jarvis/src/mosaic-stack-local-reconciler` and passed unless explicitly noted.
+
+- `pnpm --filter @mosaicstack/mosaic exec vitest run src/fleet/fleet-reconciler.acceptance.spec.ts` — 1 file, 5 tests passed (the socket matrix executes six acceptance cases).
+- `pnpm --filter @mosaicstack/mosaic exec vitest run src/fleet/fleet-reconciler.acceptance.spec.ts src/fleet/fleet-reconciler.spec.ts src/commands/fleet-reconciler-command.spec.ts src/commands/fleet-agent-crud-command.spec.ts src/fleet/fleet-agent-crud.spec.ts src/commands/fleet.spec.ts` — 6 files, 256 tests passed.
+- `pnpm --filter @mosaicstack/mosaic test` — final run: 57 files, 825 tests passed.
+- `pnpm --filter @mosaicstack/mosaic typecheck` — passed.
+- `pnpm --filter @mosaicstack/mosaic lint` — passed.
+- `pnpm typecheck` — 42/42 Turbo tasks successful.
+- `pnpm lint` — 23/23 Turbo tasks successful.
+- `pnpm format:check` — all matched files use Prettier style.
+- `git diff --check` — passed with no output.
+- Initial scoped Prettier check found style drift in the new spec and tracking table; `pnpm exec prettier --write ...` remediated it before all final gates above.
+- No live fleet, systemctl, tmux, process, site, migration, canary, deploy, runtime, connector, or remote command was invoked. The sole tmux action after documentation/final evidence is the user-required completion `wait-for` signal.
+
+## Review boundary
+
+This is an author handoff. No self-review is represented as reviewer-of-record. The uncommitted synthetic tree is intended for independent review.
+
+## Risks / blockers
+
+- M3 truthfully reports incomplete lifecycle effects and bounded recovery; it does not implement or claim an automatic rollback command. This suite proves stopped-state restoration by the documented exact recovery reconcile. M4 retains migration/canary rollback ownership.
+- The fake host models only the public systemd/tmux runner contract and temporary roster filesystem boundary. This is intentional under the no-live-effects hold.
+- Parent issue closure, commit, push, PR, merge, deployment, and branch cleanup remain explicit holds.
+- No residual implementation blocker.