mosaicstack/stack
20
0
Fork 1
Code Issues 335 Pull Requests 15 Actions Packages Projects Releases 15 Wiki Activity

feat(fleet): reconcile local roster state #785

Merged
jason.woltje merged 2 commits from feat/758-local-reconciler into main 2026-07-15 15:03:32 +00:00
Conversation 2 Commits 2 Files Changed 12
+1888 -45
jason.woltje commented 2026-07-15 14:24:23 +00:00
Owner
Copy Link
Copy Source

Implements FCM-M3-001 local roster-owned reconciliation and lifecycle control through the existing mosaic fleet control plane.

  • Adds plan/apply/reconcile/start/stop/restart/status/verify/doctor with stable JSON and exit behavior.
  • Fences every mutation with mandatory expected generation, private exclusive locking, and canonical roster reread under lock.
  • Preserves stopped desired state, exact systemd/tmux targets, private holder ownership, and truthful partial/cleanup recovery.
  • Keeps remote/connectors inventory-only and observational commands effect-free.
  • Adds fake-adapter/filesystem/Commander coverage and operator/reference documentation.

Independent synthetic-tree REVIEW-6: VERIFIED APPROVE on exact tree 7dc35b8881f07a60de44d2328a37a27ef6c46f64. Commit head: b48f22b50b1beef5190818461df42773d4ae3076. Full package suite: 56 files / 820 tests passed. No live fleet/systemd/tmux actions were performed.

Relates to #758. Parent #758 remains open through M5.

🤖 Generated with Claude Code

Implements FCM-M3-001 local roster-owned reconciliation and lifecycle control through the existing `mosaic fleet` control plane. - Adds plan/apply/reconcile/start/stop/restart/status/verify/doctor with stable JSON and exit behavior. - Fences every mutation with mandatory expected generation, private exclusive locking, and canonical roster reread under lock. - Preserves stopped desired state, exact systemd/tmux targets, private holder ownership, and truthful partial/cleanup recovery. - Keeps remote/connectors inventory-only and observational commands effect-free. - Adds fake-adapter/filesystem/Commander coverage and operator/reference documentation. Independent synthetic-tree REVIEW-6: VERIFIED APPROVE on exact tree `7dc35b8881f07a60de44d2328a37a27ef6c46f64`. Commit head: `b48f22b50b1beef5190818461df42773d4ae3076`. Full package suite: 56 files / 820 tests passed. No live fleet/systemd/tmux actions were performed. Relates to #758. Parent #758 remains open through M5. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
jason.woltje added 1 commit 2026-07-15 14:24:23 +00:00
feat(fleet): reconcile local roster state
ci/woodpecker/pr/ci Pipeline failed
Details
b48f22b50b
jason.woltje commented 2026-07-15 14:35:51 +00:00
Author
Owner
Copy Link
Copy Source

REQUEST CHANGES — reviewer-of-record for PR #785 head b48f22b50b

I verified the exact committed head and reviewed the 12-file scope against FCM-M3-001 / #758.

Remote/state evidence:

  • PR #785 is open; head branch feat/758-local-reconciler; base main.
  • PR body says Relates to #758. Parent #758 remains open through M5. and does not use Fixes #758.
  • Parent #758 is open.
  • git ls-remote origin refs/heads/feat/758-local-reconciler refs/pull/785/head => b48f22b50b1beef5190818461df42773d4ae3076 for both.
  • Commit tree: 7dc35b8881f07a60de44d2328a37a27ef6c46f64.
  • Parent/base recorded by commit: bc5e73629e92c56a80fa6a769ebad17c0177f504.
  • Exact scope: 12 files, 1865 insertions / 38 deletions.

Code/security review result:

  • Reconfirmed the REVIEW-6 invariants in the committed head: mandatory expected generation for mutating paths; private lock before canonical roster reread; locked canonical roster drives generation/plan/projections/targets/lifecycle for effects; stale/missing generation effect-free; observational status/doctor/verify lock-free/effect-free; safe private ancestors/leaf; EEXIST-only concurrency; inode/token-bound release; replacement-lock retention; additive cleanup truth with non-zero CLI exit; stopped-state preservation; exact systemd/tmux targets; holder ownership/global environment proof; unmanaged/remote rejection; redacted stable JSON/exit behavior.
  • No live fleet/systemd/tmux/session/site/migration/canary/deploy/runtime/remote/connector action was performed; tests used fake runners/temp files only.
  • Current origin/main has advanced to c593a15ef88b78d91cdd62916bcfcdec36040768; diff against the recorded base did not touch the 12 scoped files, and the FCM-M3-001 behavior review remains valid.

Local gates passed:

  • git diff --check — pass.
  • pnpm --filter @mosaicstack/mosaic test -- src/fleet/fleet-reconciler.spec.ts src/commands/fleet-reconciler-command.spec.ts src/commands/fleet.spec.ts — pass, 3 files / 222 tests.
  • pnpm --filter @mosaicstack/mosaic test — pass, 56 files / 820 tests.
  • pnpm --filter @mosaicstack/mosaic typecheck — pass.
  • pnpm --filter @mosaicstack/mosaic lint — pass.
  • pnpm typecheck — pass, 42/42 turbo tasks cached.
  • pnpm lint — pass, 23/23 turbo tasks cached.
  • Prettier check on exact 12-file scope — pass.
  • Adversarial committed-head fixtures passed for stale-generation under-lock fencing, mandatory generation before observe/effects, doctor lock-free/effect-free, cleanup truth/non-zero, and primary-error preservation.

Blocking finding:

  • Required PR CI is red for this exact head. ~/.config/mosaic/tools/git/pr-ci-wait.sh -n 785 -t 1200 -i 15 reported failure for PR #785 head b48f22b50b1beef5190818461df42773d4ae3076.
  • Woodpecker pipeline 1834 status is failure; steps: ci-postgres, install, sanitization, typecheck, lint, and format passed; test failed with exit 1.

Because the exact committed head has failing required CI, this cannot receive verified approval yet even though the local review and local gates passed. Please remediate or rerun to terminal green CI, then request another reviewer-of-record pass on the unchanged or new exact head.

REQUEST CHANGES — reviewer-of-record for PR #785 head b48f22b50b1beef5190818461df42773d4ae3076 I verified the exact committed head and reviewed the 12-file scope against FCM-M3-001 / #758. Remote/state evidence: - PR #785 is open; head branch `feat/758-local-reconciler`; base `main`. - PR body says `Relates to #758. Parent #758 remains open through M5.` and does not use `Fixes #758`. - Parent #758 is open. - `git ls-remote origin refs/heads/feat/758-local-reconciler refs/pull/785/head` => `b48f22b50b1beef5190818461df42773d4ae3076` for both. - Commit tree: `7dc35b8881f07a60de44d2328a37a27ef6c46f64`. - Parent/base recorded by commit: `bc5e73629e92c56a80fa6a769ebad17c0177f504`. - Exact scope: 12 files, 1865 insertions / 38 deletions. Code/security review result: - Reconfirmed the REVIEW-6 invariants in the committed head: mandatory expected generation for mutating paths; private lock before canonical roster reread; locked canonical roster drives generation/plan/projections/targets/lifecycle for effects; stale/missing generation effect-free; observational status/doctor/verify lock-free/effect-free; safe private ancestors/leaf; EEXIST-only concurrency; inode/token-bound release; replacement-lock retention; additive cleanup truth with non-zero CLI exit; stopped-state preservation; exact systemd/tmux targets; holder ownership/global environment proof; unmanaged/remote rejection; redacted stable JSON/exit behavior. - No live fleet/systemd/tmux/session/site/migration/canary/deploy/runtime/remote/connector action was performed; tests used fake runners/temp files only. - Current `origin/main` has advanced to `c593a15ef88b78d91cdd62916bcfcdec36040768`; diff against the recorded base did not touch the 12 scoped files, and the FCM-M3-001 behavior review remains valid. Local gates passed: - `git diff --check` — pass. - `pnpm --filter @mosaicstack/mosaic test -- src/fleet/fleet-reconciler.spec.ts src/commands/fleet-reconciler-command.spec.ts src/commands/fleet.spec.ts` — pass, 3 files / 222 tests. - `pnpm --filter @mosaicstack/mosaic test` — pass, 56 files / 820 tests. - `pnpm --filter @mosaicstack/mosaic typecheck` — pass. - `pnpm --filter @mosaicstack/mosaic lint` — pass. - `pnpm typecheck` — pass, 42/42 turbo tasks cached. - `pnpm lint` — pass, 23/23 turbo tasks cached. - Prettier check on exact 12-file scope — pass. - Adversarial committed-head fixtures passed for stale-generation under-lock fencing, mandatory generation before observe/effects, doctor lock-free/effect-free, cleanup truth/non-zero, and primary-error preservation. Blocking finding: - Required PR CI is red for this exact head. `~/.config/mosaic/tools/git/pr-ci-wait.sh -n 785 -t 1200 -i 15` reported failure for PR #785 head `b48f22b50b1beef5190818461df42773d4ae3076`. - Woodpecker pipeline `1834` status is `failure`; steps: `ci-postgres`, `install`, `sanitization`, `typecheck`, `lint`, and `format` passed; `test` failed with exit 1. Because the exact committed head has failing required CI, this cannot receive verified approval yet even though the local review and local gates passed. Please remediate or rerun to terminal green CI, then request another reviewer-of-record pass on the unchanged or new exact head.
jason.woltje added 1 commit 2026-07-15 14:42:51 +00:00
test(fleet): stabilize reconciler CI fixtures
ci/woodpecker/pr/ci Pipeline was successful
Details
38de24787d
jason.woltje commented 2026-07-15 15:02:02 +00:00
Author
Owner
Copy Link
Copy Source

VERIFIED APPROVE — reviewer-of-record for PR #785 head 38de24787d

Exact identity and remote state verified:

  • PR/source/local head: 38de24787d2b1fdca880e782ac37dbea3dd36346.
  • Commit tree: 6b02c257b68db1af2448474f911f1ab3ba047fe7.
  • Parent commit: b48f22b50b1beef5190818461df42773d4ae3076.
  • Original reviewed implementation head/tree: b48f22b50b1beef5190818461df42773d4ae3076 / 7dc35b8881f07a60de44d2328a37a27ef6c46f64.
  • Merge-base: bc5e73629e92c56a80fa6a769ebad17c0177f504.
  • git ls-remote confirms both refs/heads/feat/758-local-reconciler and refs/pull/785/head point to 38de24787d2b1fdca880e782ac37dbea3dd36346.
  • PR #785 is open; only one open PR is listed for this branch/head.
  • Parent #758 is open. PR body says Relates to #758. Parent #758 remains open through M5.
  • Previous REQUEST CHANGES comment 17581 applied to superseded head b48f22b50b1beef5190818461df42773d4ae3076 and is not a verdict on this head.

Reviewed scope:

  • Cumulative PR scope from bc5e73629e92c56a80fa6a769ebad17c0177f504: exact 12 files, 1888 insertions / 45 deletions.
  • CI-remediation commit scope from b48f22b50b1beef5190818461df42773d4ae3076: exact 3 files, 29 insertions / 13 deletions:
    • packages/mosaic/src/commands/fleet.spec.ts
    • packages/mosaic/src/fleet/fleet-reconciler.spec.ts
    • packages/mosaic/src/fleet/fleet-reconciler.ts

Review result:

  • Pipeline 1834 failures are closed without weakening production boundaries.
  • Legacy Commander status tests now use explicit isolated v1 roster/mosaicHome fixtures with cleanup.
  • v2 missing/malformed canonical roster paths still fail closed before runner/effects.
  • Root-portable injected EIO proves lock-io-failed deterministically.
  • acquirePrivateReconcileLock defaults to node:fs/promises open; injection is a direct test seam, not CLI/runtime input.
  • Managed-path validation still precedes open; only EEXIST maps to concurrency after existing-leaf validation; unsafe leaves reject as unsafe-lock.
  • Token/inode-bound cleanup and replacement-lock retention remain intact.
  • Generation lock, under-lock canonical roster reread, exact systemd/tmux targeting, private holder identity, redacted partial failure truth, observation-only commands, unmanaged/remote rejection, and no adoption/destruction remain intact.

CI and local gates:

  • Woodpecker pipeline 1836: terminal success for head 38de24787d2b1fdca880e782ac37dbea3dd36346; steps ci-postgres, install, sanitization, typecheck, lint, format, test all OK.
  • ~/.config/mosaic/tools/git/pr-ci-wait.sh -n 785 -t 60 -i 5 — green for PR #785 exact head.
  • git diff --check — pass.
  • Focused tests: pnpm --filter @mosaicstack/mosaic test -- src/fleet/fleet-reconciler.spec.ts src/commands/fleet-reconciler-command.spec.ts src/commands/fleet.spec.ts — pass, 3 files / 222 tests.
  • Full package tests: pnpm --filter @mosaicstack/mosaic test — pass, 56 files / 820 tests.
  • Package typecheck and lint — pass.
  • Root pnpm typecheck — pass, 42/42 turbo tasks cached.
  • Root pnpm lint — pass, 23/23 turbo tasks cached.
  • Prettier check on remediation scope — pass.
  • Adversarial exact-head fixture passed for injected EEXIST/EIO/unexpected open failures, pre-open managed-path validation, unsafe existing leaves, and v2 missing/malformed fail-closed behavior.

Residual risks / boundaries:

  • No live fleet/systemd/tmux/session/site/migration/canary/deploy/runtime/remote/connector action was performed.
  • This is reviewer-of-record approval only; no merge was performed.
  • Parent #758 remains open through M5.
VERIFIED APPROVE — reviewer-of-record for PR #785 head 38de24787d2b1fdca880e782ac37dbea3dd36346 Exact identity and remote state verified: - PR/source/local head: `38de24787d2b1fdca880e782ac37dbea3dd36346`. - Commit tree: `6b02c257b68db1af2448474f911f1ab3ba047fe7`. - Parent commit: `b48f22b50b1beef5190818461df42773d4ae3076`. - Original reviewed implementation head/tree: `b48f22b50b1beef5190818461df42773d4ae3076` / `7dc35b8881f07a60de44d2328a37a27ef6c46f64`. - Merge-base: `bc5e73629e92c56a80fa6a769ebad17c0177f504`. - `git ls-remote` confirms both `refs/heads/feat/758-local-reconciler` and `refs/pull/785/head` point to `38de24787d2b1fdca880e782ac37dbea3dd36346`. - PR #785 is open; only one open PR is listed for this branch/head. - Parent #758 is open. PR body says `Relates to #758. Parent #758 remains open through M5.` - Previous REQUEST CHANGES comment `17581` applied to superseded head `b48f22b50b1beef5190818461df42773d4ae3076` and is not a verdict on this head. Reviewed scope: - Cumulative PR scope from `bc5e73629e92c56a80fa6a769ebad17c0177f504`: exact 12 files, 1888 insertions / 45 deletions. - CI-remediation commit scope from `b48f22b50b1beef5190818461df42773d4ae3076`: exact 3 files, 29 insertions / 13 deletions: - `packages/mosaic/src/commands/fleet.spec.ts` - `packages/mosaic/src/fleet/fleet-reconciler.spec.ts` - `packages/mosaic/src/fleet/fleet-reconciler.ts` Review result: - Pipeline 1834 failures are closed without weakening production boundaries. - Legacy Commander status tests now use explicit isolated v1 roster/mosaicHome fixtures with cleanup. - v2 missing/malformed canonical roster paths still fail closed before runner/effects. - Root-portable injected `EIO` proves `lock-io-failed` deterministically. - `acquirePrivateReconcileLock` defaults to `node:fs/promises` `open`; injection is a direct test seam, not CLI/runtime input. - Managed-path validation still precedes open; only `EEXIST` maps to concurrency after existing-leaf validation; unsafe leaves reject as `unsafe-lock`. - Token/inode-bound cleanup and replacement-lock retention remain intact. - Generation lock, under-lock canonical roster reread, exact systemd/tmux targeting, private holder identity, redacted partial failure truth, observation-only commands, unmanaged/remote rejection, and no adoption/destruction remain intact. CI and local gates: - Woodpecker pipeline `1836`: terminal `success` for head `38de24787d2b1fdca880e782ac37dbea3dd36346`; steps `ci-postgres`, `install`, `sanitization`, `typecheck`, `lint`, `format`, `test` all OK. - `~/.config/mosaic/tools/git/pr-ci-wait.sh -n 785 -t 60 -i 5` — green for PR #785 exact head. - `git diff --check` — pass. - Focused tests: `pnpm --filter @mosaicstack/mosaic test -- src/fleet/fleet-reconciler.spec.ts src/commands/fleet-reconciler-command.spec.ts src/commands/fleet.spec.ts` — pass, 3 files / 222 tests. - Full package tests: `pnpm --filter @mosaicstack/mosaic test` — pass, 56 files / 820 tests. - Package `typecheck` and `lint` — pass. - Root `pnpm typecheck` — pass, 42/42 turbo tasks cached. - Root `pnpm lint` — pass, 23/23 turbo tasks cached. - Prettier check on remediation scope — pass. - Adversarial exact-head fixture passed for injected `EEXIST`/`EIO`/unexpected open failures, pre-open managed-path validation, unsafe existing leaves, and v2 missing/malformed fail-closed behavior. Residual risks / boundaries: - No live fleet/systemd/tmux/session/site/migration/canary/deploy/runtime/remote/connector action was performed. - This is reviewer-of-record approval only; no merge was performed. - Parent #758 remains open through M5.
jason.woltje merged commit 499090508e into main 2026-07-15 15:03:32 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
bug
Defect in framework tooling or delivered code
 fleet-enhancement
Fleet-wide reliability/speed/cost improvements surfaced by the nightly enhance pass
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
code-be-01 (Mosaic fleet seat code-be-01) code-be-02 (Mosaic fleet seat code-be-02) code-infra-01 (Mosaic fleet seat code-infra-01) fargo fred gate-merge-01 (Mosaic fleet seat gate-merge-01) happy jason.woltje (Jason Woltje) marcie merge-gate ops-01 (Mosaic fleet seat ops-01) ops-02 (Mosaic fleet seat ops-02) ops-03 (Mosaic fleet seat ops-03) ops-ci-01 (Mosaic fleet seat ops-ci-01) ops-deploy-01 (Mosaic fleet seat ops-deploy-01) orch-01 (Mosaic fleet seat orch-01) pepper resume rev-code-01 rev-code-02 rev-security-01 rev-security-02 rev-security-03 (Mosaic fleet seat rev-security-03) sanity scooby (Scooby) scrappy shaggy tiny topher (Mosaic fleet seat topher) velma veronica (Mosaic fleet seat veronica) vision woodpecker
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mosaicstack/stack#785