mosaicstack/stack
1
0
Fork 0
Code Issues 57 Pull Requests Actions Packages Projects Releases 15 Wiki Activity

test(fleet): cover reconciler lifecycle gates
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
Details

Browse Source 
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Jarvis
2026-07-15 10:43:13 -05:00
parent 499090508e
commit 307b07d027
3 changed files with 496 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
docs/TASKS.md
View File

@@ -52,20 +52,20 @@ Active workstream is **W1 — Federation v1**. Workers should:
> the repository quality gates, independent code and security review, terminal-green CI, and
> the applicable acceptance evidence before merge. Issue #758 remains open until M5 closes.
| id | status | description | issue | agent | repo | branch | depends_on | estimate | notes |
| ---------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- | ------------- | ----------------- | --------------------------------------- | ---------------------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------ |
| FCM-M0-001 | done | Publish normative PRD requirements/acceptance criteria, this M0–M5 DAG, docs-IA checklist, and legacy example/profile disposition inventory; no implementation changes | #758 | sonnet | mosaicstack/stack | `docs/758-fleet-config-management` | — | 18K | Merged via #760 (`c32d85a`); parent #758 intentionally remains open through M5 |
| FCM-M1-001 | done | Implement narrow local-tmux v2 roster structural contract/compiler with YAML/JSON canonicalization and schema/parser parity tests | #758 | coder0 | mosaicstack/stack | `feat/758-roster-v2-compiler` | FCM-M0-001 | 30K | #764 squash `aa5b43b`; exact-head RoR and PR/main terminal-green CI; no lifecycle or live mutation |
| FCM-M1-002 | in-progress | Reuse existing profile/persona/provision resolver for roster semantics; add canonical class/authority validation and approved aliases | #758 | native-sonnet | mosaicstack/stack | `feat/758-shared-role-resolution` | FCM-M0-001 | 25K | Started 2026-07-14 from `aa5b43b`; one shared resolver only; validator certificate-only; merge-gate sole merge authority |
| FCM-M1-003 | not-started | Convert the M0 legacy inventory into executable example/profile/service-preset validation and explicit v1-version/retirement checks | #758 | codex | mosaicstack/stack | `test/758-example-profile-dispositions` | FCM-M1-001, FCM-M1-002 | 20K | Every shipped artifact must validate, be versioned v1, or be retired with replacement |
| FCM-M2-001 | not-started | Migrate generic launch chain to deterministic `.env.generated` plus strict data-only `.env.local`; quarantine forbidden legacy keys | #758 | codex | mosaicstack/stack | `feat/758-generated-env-boundary` | FCM-M1-001, FCM-M1-002 | 30K | No arbitrary command compatibility path; diagnostics expose key names/hashes only |
| FCM-M2-002 | not-started | Add generation-guarded local fleet agent create/get/update/delete mutations with plan/dry-run, atomic roster writes, and recovery output | #758 | codex | mosaicstack/stack | `feat/758-fleet-agent-crud` | FCM-M1-001, FCM-M2-001 | 30K | Fresh create persists stopped unless explicit persisted start |
| FCM-M3-001 | not-started | Implement local roster-owned reconcile/apply plus lifecycle/status/verify/doctor contracts and stable JSON/exit codes | #758 | codex | mosaicstack/stack | `feat/758-local-reconciler` | FCM-M2-001, FCM-M2-002 | 35K | Exact systemd/tmux ownership; remote/schema-only entries are inventory only |
| FCM-M3-002 | not-started | Add isolated systemd/tmux lifecycle, drift, socket, unmanaged-session, crash, and rollback acceptance coverage | #758 | sonnet | mosaicstack/stack | `test/758-reconciler-lifecycle-gates` | FCM-M3-001 | 25K | Proves stopped-state preservation and zero fuzzy destructive targeting |
| FCM-M4-001 | not-started | Implement field-complete v1-to-v2 inventory/preview/migrator with alias, lifecycle, env-quarantine, and remote/connector disposition evidence | #758 | codex | mosaicstack/stack | `feat/758-v1-v2-migrator` | FCM-M1-003, FCM-M3-001 | 35K | Preview first; no unreviewed lifecycle inference |
| FCM-M4-002 | not-started | Add reversible canary migration, rollback, stale-projection/orphan classification, and current-host 9-managed/3-unmanaged fixture coverage | #758 | sonnet | mosaicstack/stack | `test/758-migration-rollback-gates` | FCM-M4-001, FCM-M3-002 | 25K | Never starts a previously stopped agent or kills an unproven unmanaged session |
| FCM-M5-001 | not-started | Deliver the accepted fleet documentation IA, how-to/operations/migration references, and link/example validation | #758 | haiku | mosaicstack/stack | `docs/758-fleet-config-operator-docs` | FCM-M1-003, FCM-M2-002, FCM-M3-001, FCM-M4-001 | 24K | Must close every checklist item or record an approved deferral |
| FCM-M5-002 | not-started | Package/update asset-drift checks, rolling local canary, independent validation certificate, and release evidence | #758 | sonnet | mosaicstack/stack | `feat/758-fleet-config-release-gate` | FCM-M3-002, FCM-M4-002, FCM-M5-001 | 30K | Final #758 gate: quality, independent code/security review, validator certificate, merge-gate approval, green CI |
| id | status | description | issue | agent | repo | branch | depends_on | estimate | notes |
| ---------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- | ------------- | ----------------- | --------------------------------------- | ---------------------------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------- |
| FCM-M0-001 | done | Publish normative PRD requirements/acceptance criteria, this M0–M5 DAG, docs-IA checklist, and legacy example/profile disposition inventory; no implementation changes | #758 | sonnet | mosaicstack/stack | `docs/758-fleet-config-management` | — | 18K | Merged via #760 (`c32d85a`); parent #758 intentionally remains open through M5 |
| FCM-M1-001 | done | Implement narrow local-tmux v2 roster structural contract/compiler with YAML/JSON canonicalization and schema/parser parity tests | #758 | coder0 | mosaicstack/stack | `feat/758-roster-v2-compiler` | FCM-M0-001 | 30K | #764 squash `aa5b43b`; exact-head RoR and PR/main terminal-green CI; no lifecycle or live mutation |
| FCM-M1-002 | done | Reuse existing profile/persona/provision resolver for roster semantics; add canonical class/authority validation and approved aliases | #758 | native-sonnet | mosaicstack/stack | `feat/758-shared-role-resolution` | FCM-M0-001 | 25K | #768 squash `a5e8e55`; shared resolver and canonical authority/alias validation delivered |
| FCM-M1-003 | done | Convert the M0 legacy inventory into executable example/profile/service-preset validation and explicit v1-version/retirement checks | #758 | codex | mosaicstack/stack | `test/758-example-profile-dispositions` | FCM-M1-001, FCM-M1-002 | 20K | #770 squash `e9c4aa3`; shipped artifact disposition validation delivered |
| FCM-M2-001 | done | Migrate generic launch chain to deterministic `.env.generated` plus strict data-only `.env.local`; quarantine forbidden legacy keys | #758 | codex | mosaicstack/stack | `feat/758-generated-env-boundary` | FCM-M1-001, FCM-M1-002 | 30K | #772 squash `191efae`; generated/local boundary and private quarantine delivered |
| FCM-M2-002 | done | Add generation-guarded local fleet agent create/get/update/delete mutations with plan/dry-run, atomic roster writes, and recovery output | #758 | codex | mosaicstack/stack | `feat/758-fleet-agent-crud` | FCM-M1-001, FCM-M2-001 | 30K | #773 squash `bc5e736`; generation-guarded atomic CRUD and recovery contracts delivered |
| FCM-M3-001 | done | Implement local roster-owned reconcile/apply plus lifecycle/status/verify/doctor contracts and stable JSON/exit codes | #758 | codex | mosaicstack/stack | `feat/758-local-reconciler` | FCM-M2-001, FCM-M2-002 | 35K | #785 squash `4990905`; exact roster-owned systemd/tmux reconcile and lifecycle contracts delivered |
| FCM-M3-002 | in-progress | Add isolated systemd/tmux lifecycle, drift, socket, unmanaged-session, crash, and rollback acceptance coverage | #758 | sonnet | mosaicstack/stack | `test/758-reconciler-lifecycle-gates` | FCM-M3-001 | 25K | Started from `4990905`; fake adapters/temp fixtures only; no live lifecycle actions |
| FCM-M4-001 | not-started | Implement field-complete v1-to-v2 inventory/preview/migrator with alias, lifecycle, env-quarantine, and remote/connector disposition evidence | #758 | codex | mosaicstack/stack | `feat/758-v1-v2-migrator` | FCM-M1-003, FCM-M3-001 | 35K | Preview first; no unreviewed lifecycle inference |
| FCM-M4-002 | not-started | Add reversible canary migration, rollback, stale-projection/orphan classification, and current-host 9-managed/3-unmanaged fixture coverage | #758 | sonnet | mosaicstack/stack | `test/758-migration-rollback-gates` | FCM-M4-001, FCM-M3-002 | 25K | Never starts a previously stopped agent or kills an unproven unmanaged session |
| FCM-M5-001 | not-started | Deliver the accepted fleet documentation IA, how-to/operations/migration references, and link/example validation | #758 | haiku | mosaicstack/stack | `docs/758-fleet-config-operator-docs` | FCM-M1-003, FCM-M2-002, FCM-M3-001, FCM-M4-001 | 24K | Must close every checklist item or record an approved deferral |
| FCM-M5-002 | not-started | Package/update asset-drift checks, rolling local canary, independent validation certificate, and release evidence | #758 | sonnet | mosaicstack/stack | `feat/758-fleet-config-release-gate` | FCM-M3-002, FCM-M4-002, FCM-M5-001 | 30K | Final #758 gate: quality, independent code/security review, validator certificate, merge-gate approval, green CI |
## Thin-core prompt diet (#528) — feat/contract-thin-core

83
docs/scratchpads/758-fcm-m3-002-reconciler-lifecycle-gates.md Normal file
View File

@@ -0,0 +1,83 @@
# FCM-M3-002 — Reconciler lifecycle acceptance gates
- **Task / issue:** FCM-M3-002 / mosaicstack/stack#758
- **Branch:** `test/758-reconciler-lifecycle-gates`
- **Required starting head:** `499090508ef1d768660e4d54e7934cbcf13cb1cd`
- **Required starting tree:** `2f1bb7fed48291f3f7ba8b21c2b52491aa14fe2b`
- **Scope:** isolated acceptance coverage and card-required evidence/tracking only; no live fleet, systemd, tmux, session, site, migration, canary, deployment, runtime, connector, or remote action.
- **Budget:** use the task estimate of 25K as the working cap; keep the delta to one coherent acceptance suite plus required task/scratchpad evidence. No production change unless a failing reproducer proves an in-scope defect.
## Intake evidence
- Clean exact local branch/head/tree verified before editing.
- `origin/test/758-reconciler-lifecycle-gates` fetched and verified at the same required head.
- The Mosaic PR wrapper reported no open pull requests, so no open-PR branch collision exists.
- Parent issue #758 is open and remains intentionally open through M5.
- Requirements loaded from `docs/PRD.md` FCM requirements and `AC-FCM-05`, `docs/TASKS.md` FCM DAG, the M3 rows in `docs/fleet/FLEET-CONFIG-DOCS-IA-CHECKLIST.md`, and the FCM-M3-001 implementation scratchpad.
## Objective
Add broad, behavior-oriented acceptance evidence around the shipped local reconciler contracts. Exercise only injected fake systemd/tmux adapters and temporary filesystem fixtures. Prove exact ownership/targeting, persisted stopped-state safety, truthful partial-failure recovery, rollback behavior, and stable command JSON/exit outcomes without touching live services or sessions.
## Acceptance mapping and evidence
| FCM-M3-002 acceptance concern | Delivered isolated evidence |
| --- | --- |
| Systemd/tmux lifecycle | `fleet-reconciler.acceptance.spec.ts` drives apply, reconcile, stop, restart, status, and recovery reconcile through one stateful injected fake host. The fake models exact systemd effects and tmux session observations; no host commands run. |
| Drift | Named/default socket matrix classifies `missing-session`, `unexpected-session`, and `disabled-running`, including combined drift, while asserting observation emits no lifecycle mutation. |
| Exact default/named socket targeting | Matrix asserts named calls use exactly `-L mosaic-fleet` and default-socket calls contain no `-L`; recorded calls cannot target an adjacent socket. |
| Unmanaged-session classification | Stateful fixtures report sorted `coder0-shadow`/`unmanaged` sessions, then prove an exact roster stop leaves both sessions and the near-collision service intact. |
| Crash/partial failure | Injected restart failure is applied after the fake effect to model crash/partial truth: result is `lifecycle: incomplete`, the roster is unchanged, and observed runtime may be active. |
| Rollback/recovery semantics | M3 has no rollback command and explicitly does not claim automatic rollback. The acceptance workflow proves the bounded recovery contract: inspect, then exact reconcile restores the persisted stopped target without a start or fuzzy effect. M4 migration/canary rollback remains outside this card. |
| Stopped-state preservation | Stateful apply and reconcile both stop an initially running observed agent whose persisted target is stopped; failed explicit restart leaves desired state stopped; recovery reconcile restores stopped state. No start call is emitted. |
| Zero fuzzy destructive targeting | Near-collision `coder0-shadow` service/session plus `unmanaged` session remain untouched. The recorded destructive calls contain only exact `mosaic-agent@coder0.service`; no tmux kill action is emitted. |
| Stable JSON/exit behavior | Temporary canonical roster fixture invokes the CLI boundary and asserts exactly one JSON line, exact partial-result shape, and exit code 1. Existing focused command specs continue to cover clean zero-exit and stable error JSON. |
| Redacted truthful recovery | Fake stderr includes `PASSWORD=acceptance-secret`; exact CLI JSON contains only bounded recovery metadata and excludes the key, value, and raw diagnostic. |
## Plan
1. Inventory existing reconciler and command specs against the table above; avoid duplicating narrow assertions already present.
2. Add one acceptance-level spec using only fake/injected adapters and temporary files.
3. If a real defect is exposed, preserve the failing reproducer and make only the smallest FCM-M3-002-required fix; otherwise leave production unchanged.
4. Reconcile `docs/TASKS.md` only for delivered M1/M2/M3-001 truth and mark FCM-M3-002 in progress.
5. Run focused tests, full `@mosaicstack/mosaic` tests, package/root typecheck and lint, Prettier/format and diff checks, plus adversarial fake-runner cases.
6. Record exact evidence and leave the tree uncommitted for independent synthetic-tree review.
## TDD decision
This card adds acceptance coverage to already-delivered behavior. Test-first applies to any product defect discovered: retain a failing reproducer before an in-scope fix. If the shipped behavior already satisfies the acceptance contract, no production code will be changed and the acceptance suite itself is the deliverable.
## Progress
- Intake and immutable baseline verification complete.
- Existing coverage inventory confirmed strong unit coverage but no stateful cross-command lifecycle acceptance harness.
- Added `packages/mosaic/src/fleet/fleet-reconciler.acceptance.spec.ts`: one injected stateful fake systemd/tmux host, temporary canonical roster fixture, five acceptance tests (six matrix cases).
- Production source is unchanged; no product defect requiring an FCM-M3-002 fix was found.
- `docs/TASKS.md` reconciles only merged M1/M2/M3-001 truth and marks FCM-M3-002 in progress.
## Verification evidence
All commands ran from `/home/jarvis/src/mosaic-stack-local-reconciler` and passed unless explicitly noted.
- `pnpm --filter @mosaicstack/mosaic exec vitest run src/fleet/fleet-reconciler.acceptance.spec.ts` — 1 file, 5 tests passed (the socket matrix executes six acceptance cases).
- `pnpm --filter @mosaicstack/mosaic exec vitest run src/fleet/fleet-reconciler.acceptance.spec.ts src/fleet/fleet-reconciler.spec.ts src/commands/fleet-reconciler-command.spec.ts src/commands/fleet-agent-crud-command.spec.ts src/fleet/fleet-agent-crud.spec.ts src/commands/fleet.spec.ts` — 6 files, 256 tests passed.
- `pnpm --filter @mosaicstack/mosaic test` — final run: 57 files, 825 tests passed.
- `pnpm --filter @mosaicstack/mosaic typecheck` — passed.
- `pnpm --filter @mosaicstack/mosaic lint` — passed.
- `pnpm typecheck` — 42/42 Turbo tasks successful.
- `pnpm lint` — 23/23 Turbo tasks successful.
- `pnpm format:check` — all matched files use Prettier style.
- `git diff --check` — passed with no output.
- Initial scoped Prettier check found style drift in the new spec and tracking table; `pnpm exec prettier --write ...` remediated it before all final gates above.
- No live fleet, systemctl, tmux, process, site, migration, canary, deploy, runtime, connector, or remote command was invoked. The sole tmux action after documentation/final evidence is the user-required completion `wait-for` signal.
## Review boundary
This is an author handoff. No self-review is represented as reviewer-of-record. The uncommitted synthetic tree is intended for independent review.
## Risks / blockers
- M3 truthfully reports incomplete lifecycle effects and bounded recovery; it does not implement or claim an automatic rollback command. This suite proves stopped-state restoration by the documented exact recovery reconcile. M4 retains migration/canary rollback ownership.
- The fake host models only the public systemd/tmux runner contract and temporary roster filesystem boundary. This is intentional under the no-live-effects hold.
- Parent issue closure, commit, push, PR, merge, deployment, and branch cleanup remain explicit holds.
- No residual implementation blocker.