feat(installer): add transactional P0-P9 state machine

This commit is contained in:
2026-08-05 17:46:58 -05:00
parent 4904d4553c
commit 049982d30e
13 changed files with 2123 additions and 136 deletions
+1 -1
View File
@@ -1379,7 +1379,7 @@ A from-zero install can report success while leaving the target host unusable be
### Normative requirements
1. The installer SHALL implement the canonical P0P9 numbering from the greenfield-install PRD v2: P0 Resolve context; P1 Preflight; P2 Acquire artifacts; P3 Install CLI; P4 Install framework + skills; P5 Identity; P6 Runtime linking / activation; P7 Services; P8 Shell discoverability; P9 Verify + commit.
1. The installer SHALL implement the canonical P0P9 numbering from the greenfield-install PRD v2: P0 Resolve context; P1 Preflight; P2 Acquire artifacts; P3 Install CLI; P4 Install framework + skills; P5 Identity; P6 Runtime linking / activation; P7 Services; P8 Shell discoverability; P9 Verify + commit. P2 is scoped to installer-distribution artifacts and SHALL NOT foreclose credentialed downstream acquisition. P5 owns validating any credential capability required by requested downstream work; P7 may provision credential-dependent resources only after that P5 postcondition commits.
2. Every phase SHALL declare preconditions, action, committed postconditions, and rollback. An unverifiable postcondition SHALL fail the install non-zero with the named phase and a remediation line; no best-effort failure may still certify success. P1's required-tool closure includes tools invoked by later phases, including `git`; a downstream prerequisite may not remain undeclared and degrade silently.
3. A durable mutation journal SHALL open before the first mutation and commit at P9. Fallible command output needed to diagnose a phase SHALL be journaled and surfaced, never discarded.
4. `--check` SHALL run exactly the P0P8 postcondition predicates without mutation, report each phase PASS/FAIL, and exit non-zero if any predicate fails.
+5
View File
@@ -9,6 +9,11 @@
- [Whole mutator-class gate](architecture/mutator-class-gate.md) — default-deny policy, revoke-first/promote-last state machine, TTL, runtime adapters, and T-B/T-C assurance boundary.
- [Compaction revocation lifecycle](architecture/compaction-revocation.md) — Claude/Pi observer matrix, same-PID generation rollover, failure fencing, and the named bounded residual stale window.
## Installation and upgrades
- [Installer state machine and recovery](guides/installer-state-machine.md) — canonical P0P9 phases, side-effect-free checks, durable journal states, rollback/remediation, and the Debian greenfield CI gate.
- [Upgrade safety and recovery](guides/upgrade-safety-and-recovery.md) — framework ownership, durable operator snapshots, verify net, and projection regeneration.
## CLI and skill management
- [Skill registration user guide](guides/user-guide.md#claude-code-skill-registration) — register, unregister, list statuses, automatic install/update reconciliation, and Claude reload behavior.
+99
View File
@@ -0,0 +1,99 @@
# Installer State Machine and Recovery
The unified installer uses a transactional P0P9 model. It may report success only after P9 reasserts every applicable committed postcondition. Internal phases invoke the CLI by P3's absolute path; shell discovery is checked only at P8.
## Canonical phases
| Phase | Responsibility | Failure disposition |
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
| P0 Resolve context | State target user, HOME, shell, privilege mode, architecture, libc, Node, and npm | Fail before mutation |
| P1 Preflight | Validate downstream tool closure (including `git` and `python3`), writable prefix, registry lane, disk/inodes, and exclusive lock | Fail before target mutation |
| P2 Acquire artifacts | Resolve exact registry versions and an immutable framework commit; record lane and SHA-256 | Discard temporary work |
| P3 Install CLI | Install at the configured absolute prefix and require exact resolved version | Restore the prior prefix/npmrc snapshot |
| P4 Install framework + skills | Sync framework and consume a checkout-free, lane/versioned shipped-skill declaration | Restore prior framework/runtime trees |
| P5 Identity | Validate SOUL/USER content, owner, and mode; establish any credential capability requested downstream | Restore generated identity/credential binding |
| P6 Runtime linking / activation | Evaluate activation honestly; never treat dead enforcement hooks as active readiness | Restore runtime activation files |
| P7 Services | Provision only requested services/resources after any required P5 credential commits | Stop and restore requested services/resources |
| P8 Shell discoverability | Require fresh login and non-login shells of the actual target shell to resolve P3's path | Restore shell profiles |
| P9 Verify + commit | Re-run P0P8, commit the manifest, and seal the journal | Leave an honestly reported resumable failure or restore the pre-install snapshot |
The phase numbers are a cross-workstream contract and must not be renumbered.
## Side-effect-free check
```bash
bash tools/install.sh --check # stable/latest lane
bash tools/install.sh --check --next # prerelease lane
```
`--check`:
- emits exactly one `[P0]` through `[P8]` PASS/FAIL row;
- exits non-zero if any predicate fails;
- does not create the npm prefix, lock, journal, manifest, shell profile, or runtime file;
- uses temporary npm observation storage outside the target HOME and removes it before exit.
P4 currently fails as `NOT-MEASURED / UNDECLARED` until the installer publishes `~/.config/mosaic/.install-shipped-skills.json`. C1 deliberately does not select among the conflicting candidate populations; C5 owns publishing and fulfilling that declaration. Once present, the P4 predicate requires the declaration's lane/version to match the resolved install and every named skill to remain contained under `skills/<name>/SKILL.md` with matching loadable frontmatter.
## Durable journal
Each mutating run creates a private transaction directory:
```text
${XDG_STATE_HOME:-~/.local/state}/mosaic/install/
active.json
<UTC-run-id>/
journal.ndjson
journal.ndjson.sha256 # committed runs only
commands.log
snapshot/
```
Before each mutation scope is touched, `journal.ndjson` records:
- phase and path;
- whether prior state existed and where its snapshot lives;
- the reversal action;
- the captured command-output location and command status.
Journal, action-status, manifest, or command-log write/sync failure is fatal. An unrecorded mutation is not allowed. Successful P9 runs append a seal event, write the SHA-256 sidecar, and make the journal and sidecar read-only. Required P4/P6 action failures are persisted in the manifest so a later `--check` cannot turn a failed action into a false pass.
Rollback roots must be non-overlapping, non-symlinked, target-user-owned strict descendants of canonical `HOME`; unsafe custom `MOSAIC_HOME`/`MOSAIC_PREFIX` values fail at P0. The same validation runs again immediately before recursive rollback. The OS lock is concurrency authority: if a process dies while `active.json` still says `in-progress`, a retry that acquires the free lock preserves the stale projection as `prior-active.json` and proceeds from the honestly retained partial state.
`active.json` is the current projection:
- `in-progress`: incomplete/open transaction;
- `rolled-back`: a fault restored the snapshot;
- `rollback-failed`: restoration failed or refused a replaced/unsafe target and requires manual recovery;
- `failed-resumable`: named postconditions failed and the recorded partial state remains for remediation;
- `committed`: P9 passed and the journal is sealed.
## Failure recovery
1. Read the named phase and remediation line from installer stderr.
2. Inspect `active.json`, then the referenced `journal.ndjson` and `commands.log`. Command output needed to diagnose a failure is preserved and surfaced; it is not redirected away.
3. For `rolled-back`, verify the target paths match their pre-install state before retrying.
4. For `failed-resumable`, repair the named phase owner requirement, then run `install.sh --check` before retrying the installer.
5. Do not activate the #869 enforcement hooks merely to turn P6 green. A broker-less host with those hooks is a failed P6 state.
## Greenfield CI gate
`.woodpecker/greenfield-install.yml` runs `tools/e2e-install-test.sh` from zero in Debian/glibc as a non-root uid with `env -i`. No host HOME, npm cache, credentials, or bind mount enters the target process. Checkout mode packages the complete current checkout into an archive, pins its SHA-256 through an internal fixture seam, and copies the self-contained fixture into the container; framework-installer changes in the PR are therefore exercised rather than fetched from an older remote branch.
The C1 gate intentionally validates an attributable RED while C2C5 remain open:
- `git` present: P1 and strict P3 pass; P4/P5/P6/P8 fail for their own reasons; P9 refuses success.
- `git` absent: P1 fails before target mutation and the installer emits no `Done.`.
The fixture is lane-parametric:
```bash
bash tools/e2e-install-test.sh --lane next --git present
bash tools/e2e-install-test.sh --lane main --git present
```
CI exercises both lane parameters as expected-RED structural checks. The authoritative main-lane promotion acceptance and issue closure remain owned by #1037.
## Source trust boundary
Remote source mode pins the resolved commit, records the archive SHA-256, limits compressed/expanded size and entry count, and rejects traversal, links, devices, and special files before extraction. This provides immutable run provenance and archive safety, not an independent authenticity root. Signed artifact metadata/provenance is explicitly deferred by the canonical greenfield PRD; C1 does not invent a signing system. The checkout CI seam does verify an expected digest supplied independently by the fixture.
@@ -12,6 +12,20 @@ with no snapshot to fall back to.
Protection is layered. Each layer is independent; a later layer catches what an
earlier one misses.
## Layer 0 — Transaction journal (install-wide recovery)
The unified installer opens a private journal under
`${XDG_STATE_HOME:-~/.local/state}/mosaic/install/` before the first target
mutation. Every mutation scope records its path, prior snapshot, and reversal
instructions before it is touched. Journal write/sync failure is fatal, and P9
seals successful journals with a SHA-256 sidecar. See
[Installer state machine and recovery](./installer-state-machine.md).
This transaction journal is distinct from the retained operator-only backup
below. The transaction journal is required for correctness and rollback;
Layer 2's durable backup remains a separately stated, fail-open recovery bonus
for a manifest bug that the normal transaction did not detect.
## Layer 1 — Manifest-owned sync (prevention)
The single source of truth for ownership is
@@ -6,7 +6,7 @@ Implement C1 from the canonical greenfield-install PRD v2: a transactional P0
## Authority and scope
- Canonical requirements: `jason.woltje/jarvis-brain` `docs/plans/2026-08-04-greenfield-install-blockers-PRD-v2.md`, read from local `origin/main` object `b2b6ed41f5aff5ea964e69b7c701cb45718742fa`; remote currency is **unestablished** because authenticated fetch returned repository-not-found.
- Canonical requirements: `jason.woltje/jarvis-brain` `docs/plans/2026-08-04-greenfield-install-blockers-PRD-v2.md`. Currency was re-derived after compaction: authenticated fetch resolved `origin/main` to `cb23e5fbc8a282fa967b93d7a134fa48d11b4bb1`; the PRD and charters are byte-identical to the previously read remote copies.
- Tracking: `mosaicstack/stack#1050` on `git.mosaicstack.dev` (author read back as `be-coder-05`).
- Base: `origin/next` `4df478cdd150fdf8d52ea109f02ade5d85017acd`.
- Out of scope: PATH, skills, headless wizard/identity, activation remediation, #869 wiring, RM-02, main promotion.
@@ -40,23 +40,29 @@ Implement C1 from the canonical greenfield-install PRD v2: a transactional P0
## Progress
- [x] Charter, doctrine, delivery/CI/QA/docs guides read.
- [x] Canonical PRD v2 and charters read from local origin object; numbering reconciles with the TL spec. No numbering conflict found. TL additions (early durable journal and INV-C) are additive, not contradictory.
- [x] Charter, doctrine, delivery/CI/QA/docs guides read and re-anchored after compaction.
- [x] Canonical PRD v2/v3 addenda and charters read from fetched `origin/main`; numbering reconciles with the TL spec. No numbering conflict found. INV-B/C/D are binding and implemented without renumbering.
- [x] Target base reachability verified with `merge-base --is-ancestor`.
- [x] Issue #1050 created and provider author read back.
- [x] Initial RED captured; TL rejected P4's repo-root count as a false RED. Four populations disagree (framework payload 1, repo root 13, sync store 101 in the fixture, W-jarvis observation 7), so C1 now requires a checkout-free declared shipped-set artifact and reports P4 `NOT-MEASURED / UNDECLARED` until C5 supplies it.
- [x] P6 strengthens #869: the two dead enforcement hooks reproduce from zero on a clean broker-less container. C1 asserts the breach but neither wires nor unwires it.
- [x] P1 false pass identified from the P4 evidence row: `git` is absent from the Debian base and was undeclared even though skill sync shells out to it. C1 adds `git` to P1; the fixture matrix preserves absent/present controls. The prior claim that web1's missing runtime skills reproduce this greenfield mechanism is withdrawn by the TL and is not carried here.
- [ ] Corrected RED transcript captured and reported.
- [ ] State machine implemented.
- [ ] Reviews complete.
- [x] Corrected RED transcript captured and reported, including the git-present/absent controls and strict P3 PASS.
- [x] State-machine implementation complete: private pre-mutation journal/snapshot, P0P8 `--check`, P2P8 fault seam, rollback, durable manifest/journal seal, action-status persistence, safe rollback roots, and stale-projection recovery.
- [x] Debian/glibc checkout fixture now packages the complete current checkout, verifies its digest in-container, and reaches the expected attributable RED without host inheritance.
- [ ] Reviews complete. Automated review defects around Bash conditional errexit, explicit exits, P4/P6 persisted action status, dev/offline source resolution, stale locks, checkout coverage, and rollback path safety were remediated. Remaining automated objections are the charter-mandated expected RED/C5 boundary and signed provenance, which the canonical PRD explicitly defers; independent informed review is still required.
## Risks / blockers
- The deployed create wrappers do not expose `--dry-run`; identity preflight was performed through `pr-merge.sh --dry-run` on the same HOMELAB repo, which resolved `git.mosaicstack.dev` + `be-coder-05`. The issue create then fell back from tea to the API but provider read-back confirmed author `be-coder-05`.
- `next` is an integration lane; `main` promotion remains #1037-owned.
- #869 must remain staged and inactive.
- Late sequencing input MB-BRAIN-01 is accommodated without implementation or renumbering: P2 covers installer distribution only; P5 owns requested credential capability; P7 leaves an ordered seam for credential-dependent resource provisioning after P5.
## Verification log
(To be updated with exact commands and resulting objects.)
- `bash -n` and ShellCheck pass for all changed shell surfaces; `git diff --check` passes.
- `bash tools/install-state-machine.test.sh` passes, including exact P0P8 rows, good/bad discrimination, persisted P4/P6 action failures, P2P8 rollback, unsafe/overlapping/symlink roots, stale `active.json`, and fatal journal initialization.
- `bash tools/install-next-lane.test.sh` passes, including exact `@next` versions, immutable source fallback, source-build/archive-failure rollback, offline `--dev`, explicit refs, and prerelease suffix mismatch.
- `bash tools/e2e-install-test.sh --lane next --source checkout --git present` returns the required expected RED in clean Debian/glibc as uid 1001: installer P0/P1/P2/P3/P7 PASS; P4/P5/P6/P8 and P9 blocking; no `Done.` claim; checkout archive digest pinned and current framework installer exercised.
- Earlier repository gates passed: `pnpm typecheck`, `pnpm lint`, `pnpm format:check`, `pnpm test:installer`, upgrade manifest/rollback/durable-snapshot/migration suites, and focused `@mosaicstack/mosaic` tests with an isolated npm prefix. Full rerun is required after final edits.