stack/docs/scratchpads/1050-install-state-machine-red-fixture.md

# #1050 — Installer P0–P9 state machine and red-first fixture

## Objective

Implement C1 from the canonical greenfield-install PRD v2: a transactional P0–P9 installer spine, a side-effect-free P0–P8 `--check`, and a lane-parametric Debian/glibc non-root from-zero fixture. The acceptance milestone is an attributable RED on the pre-C1 installer while preserving P3 PASS.

## Authority and scope

- Canonical requirements: `jason.woltje/jarvis-brain` `docs/plans/2026-08-04-greenfield-install-blockers-PRD-v2.md`. Currency was re-derived after compaction: authenticated fetch resolved `origin/main` to `cb23e5fbc8a282fa967b93d7a134fa48d11b4bb1`; the PRD and charters are byte-identical to the previously read remote copies.
- Tracking: `mosaicstack/stack#1050` on `git.mosaicstack.dev` (author read back as `be-coder-05`).
- Historical implementation base: `origin/next` `4df478cdd150fdf8d52ea109f02ade5d85017acd`. Delivery PR #1054 targets `main` under L0's trunk-only rule; `next` remains a non-merging integration lane.
- Out of scope: PATH, skills, headless wizard/identity, activation remediation, #869 wiring, RM-02, main promotion.
- `docs/TASKS.md` is orchestrator-single-writer and is not modified by this worker.

## Plan

1. Pre-register the canonical phase/output/side-effect-free/fault-injection checks and observe RED against the base installer.
2. Commit the immutable red-first acceptance fixture before implementation.
3. Add the state-machine/journal/postcondition spine without repairing P4/P5/P8 symptoms.
4. Wire the expected-RED from-zero fixture into Woodpecker using Debian/glibc and a non-root target user.
5. Run shell/static baselines, situational container validation, code review, security review, then deliver through a PR to `next` under the coordinator-owned merge path.

## Budget

- Working estimate: 32K reasoning/output tokens.
- Hard external cap: none stated.
- Adaptation: keep implementation in shell surfaces already in scope; no package dependency install unless repository gates require it.

## Pre-registered acceptance checks

| ID | Exact case | Expected pre-fix result |
|---|---|---|
| C1-R1 | `tools/e2e-install-test.sh --lane next` in a clean Debian 12 container as uid 1001 | non-zero; P3 PASS; P4 `NOT-MEASURED / UNDECLARED`; P5/P6/P8 FAIL with own reasons |
| C1-R2 | `tools/install-state-machine.test.sh` phase table case | RED because base installer does not enumerate canonical P0–P9 contracts |
| C1-R3 | side-effect-free `--check` case over a fingerprinted HOME | RED because base `--check` is version-only rather than P0–P8 predicates |
| C1-R4 | fault injection after each P2…P8 | RED because base installer has no injectable durable journal/rollback state |
| C1-R5 | Docker unavailable | base harness incorrectly exits 0; replacement must fail non-zero |
| C1-R6 | lane resolution | bare checkout is forbidden; fixture must pass `--next` and assert the resolved prerelease version |
| C1-R7 | same Debian fixture with `git` absent vs present | absent: P1 FAIL while legacy installer exits 0 and sync degrades; present: P1 PASS and observed store/runtime containment 101/101 |

## Progress

- [x] Charter, doctrine, delivery/CI/QA/docs guides read and re-anchored after compaction.
- [x] Canonical PRD v2/v3 addenda and charters read from fetched `origin/main`; numbering reconciles with the TL spec. No numbering conflict found. INV-B/C/D are binding and implemented without renumbering.
- [x] Target base reachability verified with `merge-base --is-ancestor`.
- [x] Issue #1050 created and provider author read back.
- [x] Initial RED captured; TL rejected P4's repo-root count as a false RED. Four populations disagree (framework payload 1, repo root 13, sync store 101 in the fixture, W-jarvis observation 7), so C1 now requires a checkout-free declared shipped-set artifact and reports P4 `NOT-MEASURED / UNDECLARED` until C5 supplies it.
- [x] P6 strengthens #869: the two dead enforcement hooks reproduce from zero on a clean broker-less container. C1 asserts the breach but neither wires nor unwires it.
- [x] P1 false pass identified from the P4 evidence row: `git` is absent from the Debian base and was undeclared even though skill sync shells out to it. C1 adds `git` to P1; the fixture matrix preserves absent/present controls. The prior claim that web1's missing runtime skills reproduce this greenfield mechanism is withdrawn by the TL and is not carried here.
- [x] Corrected RED transcript captured and reported, including the git-present/absent controls and strict P3 PASS.
- [x] State-machine implementation complete: private pre-mutation journal/snapshot, P0–P8 `--check`, P2–P8 fault seam, rollback, durable manifest/journal seal, action-status persistence, safe rollback roots, and stale-projection recovery.
- [x] Debian/glibc checkout fixture now packages the complete current checkout, verifies its digest in-container, and reaches the expected attributable RED without host inheritance. CI compares its exact final phase map/reasons to `tools/fixtures/greenfield-expected-red.tsv`; the fixture remains red while the detector job is green only on an exact match.
- [ ] Reviews complete. Reviews 80 (`rev-security-02`) and 81 (`rev-974`) requested changes at `3934e03f`; their eight non-overlapping detector findings are being remediated red-first. Current remediation adds canonical-image portability, absolute P3 CLI propagation, exact expected-RED schema/cardinality, passwd-HOME binding, created-path owner/mode policy, real-action P2–P8 fault injection, verified non-empty remote installer execution, and seeded secret-canary/redacted diagnostics. Both old verdicts become void when the remediation head moves and require fresh independent review.
- [x] Successor remediation for review 90 is RED-first and recorded in `docs/reports/verification/1050-successor-remediation/`: the manifest now binds the complete supported final P0 reason; P4 rejects an incomplete created-path walk instead of discarding `find` failure; and the TERM no-exit control is independent of filesystem enumeration order while retaining a proven RED mutation. Pipeline 2224's 32/2 result was a path-order-sensitive control, not evidence that the resume bug's premise became stale. The enumeration-class sweep additionally replaced order-dependent `find | head -1` source-root selection with a checked complete inventory requiring exactly one extracted root.
- [x] C1 fix round for reviews 92/93: Blocker B was completed first. RED was reproduced before implementation: a planted `{"status":"in-progress"}` record makes the complete real walk fail the suite at P2, while the same planted defect beneath a target-owned mode-0100 directory makes real `find` fail and the frozen suite falsely exit 0 with `installer next lane tests passed`. One shared helper now captures and checks the complete NUL-delimited population before testing absence across the primary and copied harness sites; grep no-match is distinct from read failure. The same attack child drives the full suite RED with a named enumeration error, while ordinary native and root/container runs pass. Blocker A then regenerated the exact installer sidecar; provider-fetch validation remains the required post-push bar. Working estimate: 16K tokens; no external hard cap; 60% context is the stop/report gate.

## Risks / blockers

- The deployed create wrappers do not expose `--dry-run`; identity preflight was performed through `pr-merge.sh --dry-run` on the same HOMELAB repo, which resolved `git.mosaicstack.dev` + `be-coder-05`. The issue create then fell back from tea to the API but provider read-back confirmed author `be-coder-05`.
- `next` is a non-merging integration lane; PR #1054 targets `main`. The old “pending promotion to main” caution dissolved when the base moved. #1050 remains open after merge and closes only after Jarvis validates the greenfield behavior.
- #869 must remain staged and inactive.
- Late sequencing input MB-BRAIN-01 is accommodated without implementation or renumbering: P2 covers installer distribution only; P5 owns requested credential capability; P7 leaves an ordered seam for credential-dependent resource provisioning after P5.

## Remediation review controls

- B1 RED: the next-lane harness failed immediately under `ci-base:latest` as root/musl; it now models uid 1001/glibc explicitly and uses Python tree fingerprints instead of GNU `find -printf`.
- B2 RED: framework/runtime linking consumed bare `mosaic` from PATH after P3 had committed an absolute path. The unified installer now exports/passes `MOSAIC_CLI_PATH`; the linker invokes that absolute artifact, and wizard auto-launch has no stale-PATH fallback.
- B3 RED: a one-row manifest (`exit=1`) certified any exit-1 log. Full-manifest validation now requires the exact three cases, one exit and P0–P9 row each, pinned require/forbid populations, and rejects malformed/duplicate/unknown rows; shrink is a negative control.
- B4 RED: uid 1001 with a passwd HOME different from ambient HOME produced P0 PASS. P0 now binds uid, username, passwd HOME and shell and explicitly rejects root and sudo-with-inherited-HOME controls.
- B5 RED: mode-0777 CLI, mode-0644 identity, and mode-0755 credential storage passed. P3/P4/P5 now apply target owner/group plus executable/shared/private policies; framework credential storage is created 0700.
- B6 RED: fault injection only wrote `.selftest-*` files. The synthetic path was removed; the P2–P8 matrix enters the normal action flow, proves an action observation occurred, injects after each real phase, and fingerprints rollback.
- B7 RED: an HTTP-200 empty body exits zero when piped to Bash. The fetched installer must now be non-empty, digest-equal to `tools/install.sh.sha256`, and that exact file is executed; failed/empty/mismatch controls are blocking and CI has a remote immutable-commit arm.
- B8 RED: raw combined command output was duplicated to terminal and `commands.log`. Both capture layers now redact before output/persistence; a seeded canary is positively emitted by the fake credential-capable registry and must remain absent from terminal, command log, npmrc, generated files and observed argv. The real greenfield fixture also scans those populations.
- Advisory code review findings are fixed: URL userinfo redaction now handles raw `@`, token-only and percent-encoded forms, repeated `:`, multiple URLs, Authorization/Basic, npm `_auth`, and Cookie headers in both capture layers; the real greenfield path positively emits its canary through `state_run_captured`; verified-fetch removes its temporary body after successful execution; and plaintext diagnostics exist only in process-substitution pipes rather than interruptible temporary files.
- Advisory security review's independent trust-root finding is **DEFERRED by canonical PRD v2 §3**, which explicitly excludes signed provenance. README now states precisely that the same-origin sidecar detects empty/corrupt/inconsistent publication but cannot authenticate against repository/server compromise; no stronger claim remains.
- The web1 no-manifest representativeness observation is recorded but intentionally not acted on: it is explicitly outside these eight blockers. This remediation does not weaken or otherwise change P9's manifest-presence assertion.

## Verification log

- `bash -n` and ShellCheck pass for all changed shell surfaces; `git diff --check` passes.
- `bash tools/install-state-machine.test.sh` passes, including exact P0–P8 rows, passwd-HOME/privilege discrimination, owner/group/mode attacks, persisted P4/P6 action failures, no synthetic fault implementation, unsafe/overlapping/symlink roots, and fatal journal initialization.
- `bash tools/install-next-lane.test.sh` passes inside `ci-base:latest`, including exact `@next` versions, immutable source fallback, source-build/archive-failure rollback, offline `--dev`, explicit refs, prerelease suffix mismatch, absolute P3 CLI propagation, secret redaction, real-action P2–P8 rollback, and stale projection recovery.
- Comparator controls pass for verdict drift, unexpected exit, manifest shrink, missing phases, duplicate rows, unknown cases, and unknown kinds. Verified-fetch controls pass for successful execution and failed/empty/digest-mismatch rejection.
- `bash tools/e2e-install-test.sh --lane next --source checkout --git present` returns the required expected RED in clean Debian/glibc as uid 1001: installer P0/P1/P2/P3/P7 PASS; P4/P5/P6/P8 and P9 blocking; no `Done.` claim; checkout archive digest pinned and current framework installer exercised. `tools/verify-greenfield-expected-red.sh` converts that expected detector result into a green CI assertion and fails on any unreviewed verdict drift.
- Earlier repository gates passed: `pnpm typecheck`, `pnpm lint`, `pnpm format:check`, upgrade manifest/rollback/durable-snapshot/migration suites, and focused `@mosaicstack/mosaic` tests with an isolated npm prefix. Full exact-remediation rerun is required before push.
- Review-93 RED evidence at frozen `378bc1a`: isolated positive-control full-suite exit `1` with `P2 left an in-progress transaction`; isolated permission-failure attack full-suite exit `0` with final `installer next lane tests passed`. No binary shadowing or PATH interception was used; the failure came from a real target-owned mode-0100 directory.
- Fix-round GREEN: `pnpm test:installer`, native next-lane, root/`ci-base:latest` next-lane, state-machine, verified-fetch, Bash syntax, ShellCheck, `pnpm typecheck`, `pnpm lint`, `pnpm format:check`, and `git diff --check` pass. The direct attack child exits 1 and names `P2 fault-state enumeration failed`. All 81 Mosaic Vitest files / 1508 tests pass under an isolated npm prefix; the wider framework-shell chain reaches the pre-existing #973 Bash-line-number gate and exits 97, matching the known host-specific condition rather than this delta. Codex code review approved at 0.93 confidence with zero findings; Codex security review reported no risk at 0.96 confidence with zero findings.
- Sidecar GREEN locally: `sha256sum -c tools/install.sh.sha256` and the workflow's expected/actual comparison both resolve `e59cb441a2f37ae9150f8eae470238e9d858a1816df93343d9784a6796676096`. This is not substituted for the required immutable provider-fetch arm at the pushed head.