fix(installer): harden greenfield detector contracts
This commit is contained in:
+8
-2
@@ -1387,14 +1387,20 @@ A from-zero install can report success while leaving the target host unusable be
|
||||
6. The from-zero fixture SHALL be lane-parametric, use Debian/glibc, run the documented install command as a non-root target user with an isolated HOME, and inherit no host credentials, npm cache, home directory, or runtime configuration.
|
||||
7. The fixture SHALL select `next` with `--next` or `MOSAIC_NEXT=1` and assert the resolved lane version. Internal predicates use P3's absolute CLI path; shell discoverability is tested only at P8.
|
||||
8. Fault injection after each P2–P8 phase SHALL prove either clean rollback or a durable, honestly reported resumable partial state, with no journal incorrectly left in progress.
|
||||
9. Unsupported musl/Alpine and unavailable Docker SHALL fail loudly rather than skip as pass.
|
||||
9. Unsupported musl/Alpine and unavailable Docker SHALL fail loudly rather than skip as pass. The repository's installer tests SHALL nevertheless run in the canonical Alpine CI image by explicitly modeling a supported non-root/glibc target and using portable filesystem enumeration.
|
||||
10. P0 SHALL bind the effective uid and username to the authoritative passwd HOME and shell and state/reject unsafe root or sudo-with-inherited-HOME privilege contexts.
|
||||
11. Created paths SHALL satisfy phase-specific target owner/group and mode policy: P3 executables are not group/world writable, framework/runtime trees are not group/world writable, and identity/credential material is private.
|
||||
12. The expected-RED comparator SHALL validate the complete manifest before selecting a case: exact case population, one exit and P0–P9 disposition per case, pinned require/forbid classes, and no malformed, duplicate, or unknown rows.
|
||||
13. The published installer contract SHALL reject failed fetches, HTTP-success empty bodies, and digest mismatch, then execute the exact digest-verified body. The remote CI arm SHALL bind that body to the immutable CI commit.
|
||||
14. Phase diagnostics SHALL be redacted before terminal or durable-log output. A seeded positive-control canary SHALL remain absent from observed argv, output, command logs, npm configuration, generated files, and shell history.
|
||||
|
||||
### C1 acceptance criteria
|
||||
|
||||
1. The pre-C1 from-zero matrix records both discriminating controls: with `git` absent, the legacy installer still exits zero while P1 fails and skill sync degrades; with `git` present, P1 passes and the observed sync store/runtime links are 101/101. The C1 installer must fail at P1 before mutation when `git` is absent.
|
||||
2. The discriminating P3 row passes: the binary exists at the expected absolute path and reports exactly the resolved `next` lane version, while P4, P5, and P8 fail.
|
||||
3. The `--check` mutation negative control proves host fingerprints are byte-identical before and after observation.
|
||||
4. Woodpecker executes and validates the expected RED fixture; C1 does not repair P4/P5/P8 or activate #869.
|
||||
4. Woodpecker executes and validates the expected RED fixture plus the immutable remote-installer contract; C1 does not repair P4/P5/P8 or activate #869.
|
||||
5. Negative controls prove manifest shrink/duplicates/unknown rows fail, unsafe P0/P3/P4/P5 contexts fail, the P2–P8 fault seam enters real actions rather than synthetic writes, empty/mismatched fetched bodies fail, and a deliberately emitted secret canary is redacted from every persisted/output population.
|
||||
|
||||
### Explicit exclusions and dependencies
|
||||
|
||||
|
||||
@@ -6,12 +6,12 @@ The unified installer uses a transactional P0–P9 model. It may report success
|
||||
|
||||
| Phase | Responsibility | Failure disposition |
|
||||
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
|
||||
| P0 Resolve context | State target user, HOME, shell, privilege mode, architecture, libc, Node, and npm | Fail before mutation |
|
||||
| P0 Resolve context | Bind uid/username to the authoritative passwd HOME/shell, state privilege mode, architecture, libc, Node, and npm | Fail before mutation |
|
||||
| P1 Preflight | Validate downstream tool closure (including `git` and `python3`), writable prefix, registry lane, disk/inodes, and exclusive lock | Fail before target mutation |
|
||||
| P2 Acquire artifacts | Resolve exact registry versions and an immutable framework commit; record lane and SHA-256 | Discard temporary work |
|
||||
| P3 Install CLI | Install at the configured absolute prefix and require exact resolved version | Restore the prior prefix/npmrc snapshot |
|
||||
| P3 Install CLI | Install at the configured absolute prefix; require exact version plus target owner/group and non-writable executable mode | Restore the prior prefix/npmrc snapshot |
|
||||
| P4 Install framework + skills | Sync framework and consume a checkout-free, lane/versioned shipped-skill declaration | Restore prior framework/runtime trees |
|
||||
| P5 Identity | Validate SOUL/USER content, owner, and mode; establish any credential capability requested downstream | Restore generated identity/credential binding |
|
||||
| P5 Identity | Validate SOUL/USER content and private modes; require private credential storage and target owner/group | Restore generated identity/credential binding |
|
||||
| P6 Runtime linking / activation | Evaluate activation honestly; never treat dead enforcement hooks as active readiness | Restore runtime activation files |
|
||||
| P7 Services | Provision only requested services/resources after any required P5 credential commits | Stop and restore requested services/resources |
|
||||
| P8 Shell discoverability | Require fresh login and non-login shells of the actual target shell to resolve P3's path | Restore shell profiles |
|
||||
@@ -56,7 +56,7 @@ Before each mutation scope is touched, `journal.ndjson` records:
|
||||
- the reversal action;
|
||||
- the captured command-output location and command status.
|
||||
|
||||
Journal, action-status, manifest, or command-log write/sync failure is fatal. An unrecorded mutation is not allowed. Successful P9 runs append a seal event, write the SHA-256 sidecar, and make the journal and sidecar read-only. Required P4/P6 action failures are persisted in the manifest so a later `--check` cannot turn a failed action into a false pass.
|
||||
Journal, action-status, manifest, or command-log write/sync failure is fatal. An unrecorded mutation is not allowed. Command diagnostics are redacted before terminal output or durable logging; credential-shaped environment values, bearer values, auth tokens, and credentialed URLs are never deliberately persisted. Successful P9 runs append a seal event, write the SHA-256 sidecar, and make the journal and sidecar read-only. Required P4/P6 action failures are persisted in the manifest so a later `--check` cannot turn a failed action into a false pass.
|
||||
|
||||
Rollback roots must be non-overlapping, non-symlinked, target-user-owned strict descendants of canonical `HOME`; unsafe custom `MOSAIC_HOME`/`MOSAIC_PREFIX` values fail at P0. The same validation runs again immediately before recursive rollback. The OS lock is concurrency authority: if a process dies while `active.json` still says `in-progress`, a retry that acquires the free lock preserves the stale projection as `prior-active.json` and proceeds from the honestly retained partial state.
|
||||
|
||||
@@ -80,7 +80,7 @@ Rollback roots must be non-overlapping, non-symlinked, target-user-owned strict
|
||||
|
||||
`.woodpecker/greenfield-install.yml` runs `tools/e2e-install-test.sh` from zero in Debian/glibc as a non-root uid with `env -i`. No host HOME, npm cache, credentials, or bind mount enters the target process. Checkout mode packages the complete current checkout into an archive, pins its SHA-256 through an internal fixture seam, and copies the self-contained fixture into the container; framework-installer changes in the PR are therefore exercised rather than fetched from an older remote branch.
|
||||
|
||||
The C1 fixture intentionally returns an attributable RED while C2–C5 remain open. CI itself remains green only when the fixture's final P0–P9 verdicts, required discriminator rows, and non-zero exit match the versioned contract in `tools/fixtures/greenfield-expected-red.tsv`. Any later remediation that changes an observed verdict makes CI red until the owning lane deliberately updates that manifest:
|
||||
The C1 fixture intentionally returns an attributable RED while C2–C5 remain open. CI itself remains green only when the fixture's final P0–P9 verdicts, required discriminator rows, seeded secret-canary scan, and non-zero exit match the versioned contract in `tools/fixtures/greenfield-expected-red.tsv`. The comparator validates the complete three-case schema before selecting a case: exactly one exit and P0–P9 disposition per case, pinned require/forbid populations, and no duplicate or unknown rows. Any later remediation that changes an observed verdict makes CI red until the owning lane deliberately updates that manifest:
|
||||
|
||||
- `git` present: P1 and strict P3 pass; P4/P5/P6/P8 fail for their own reasons; P9 refuses success.
|
||||
- `git` absent: P1 fails before target mutation and the installer emits no `Done.`.
|
||||
@@ -92,8 +92,8 @@ bash tools/e2e-install-test.sh --lane next --git present
|
||||
bash tools/e2e-install-test.sh --lane main --git present
|
||||
```
|
||||
|
||||
CI exercises both lane parameters as expected-RED structural checks. Delivery targets `main` under the trunk-only merge rule; `next` remains a non-merging integration lane. The linked installer issue stays open after merge and closes only after Jarvis independently validates the greenfield behavior.
|
||||
CI exercises both lane parameters as expected-RED structural checks. A separate remote-contract arm fetches the installer at the immutable CI commit, rejects failed or empty HTTP-success bodies, compares it to the reviewed `tools/install.sh.sha256`, and executes that exact fetched artifact. The P2–P8 fault matrix runs the real phase actions (including the P3 npm path, P4 framework path, and wizard path) rather than synthetic representative writes, then compares the complete target tree to its pre-install fingerprint. Delivery targets `main` under the trunk-only merge rule; `next` remains a non-merging integration lane. The linked installer issue stays open after merge and closes only after Jarvis independently validates the greenfield behavior.
|
||||
|
||||
## Source trust boundary
|
||||
|
||||
Remote source mode pins the resolved commit, records the archive SHA-256, limits compressed/expanded size and entry count, and rejects traversal, links, devices, and special files before extraction. This provides immutable run provenance and archive safety, not an independent authenticity root. Signed artifact metadata/provenance is explicitly deferred by the canonical greenfield PRD; C1 does not invent a signing system. The checkout CI seam does verify an expected digest supplied independently by the fixture.
|
||||
Remote installer mode requires a non-empty body and an expected SHA-256 before execution. Remote source-archive mode separately pins the resolved commit, records the archive SHA-256, limits compressed/expanded size and entry count, and rejects traversal, links, devices, and special files before extraction. These controls provide immutable run provenance and archive safety, not an independent signing root. Signed artifact metadata/provenance is explicitly deferred by the canonical greenfield PRD; C1 does not invent a signing system. The checkout and remote CI seams verify reviewed digests before executing their artifacts.
|
||||
|
||||
@@ -50,7 +50,7 @@ Implement C1 from the canonical greenfield-install PRD v2: a transactional P0–
|
||||
- [x] Corrected RED transcript captured and reported, including the git-present/absent controls and strict P3 PASS.
|
||||
- [x] State-machine implementation complete: private pre-mutation journal/snapshot, P0–P8 `--check`, P2–P8 fault seam, rollback, durable manifest/journal seal, action-status persistence, safe rollback roots, and stale-projection recovery.
|
||||
- [x] Debian/glibc checkout fixture now packages the complete current checkout, verifies its digest in-container, and reaches the expected attributable RED without host inheritance. CI compares its exact final phase map/reasons to `tools/fixtures/greenfield-expected-red.tsv`; the fixture remains red while the detector job is green only on an exact match.
|
||||
- [ ] Reviews complete. Automated review defects around Bash conditional errexit, explicit exits, P4/P6 persisted action status, dev/offline source resolution, stale locks, checkout coverage, and rollback path safety were remediated. Remaining automated objections are the charter-mandated expected RED/C5 boundary and signed provenance, which the canonical PRD explicitly defers; independent informed review is still required.
|
||||
- [ ] Reviews complete. Reviews 80 (`rev-security-02`) and 81 (`rev-974`) requested changes at `3934e03f`; their eight non-overlapping detector findings are being remediated red-first. Current remediation adds canonical-image portability, absolute P3 CLI propagation, exact expected-RED schema/cardinality, passwd-HOME binding, created-path owner/mode policy, real-action P2–P8 fault injection, verified non-empty remote installer execution, and seeded secret-canary/redacted diagnostics. Both old verdicts become void when the remediation head moves and require fresh independent review.
|
||||
|
||||
## Risks / blockers
|
||||
|
||||
@@ -59,10 +59,25 @@ Implement C1 from the canonical greenfield-install PRD v2: a transactional P0–
|
||||
- #869 must remain staged and inactive.
|
||||
- Late sequencing input MB-BRAIN-01 is accommodated without implementation or renumbering: P2 covers installer distribution only; P5 owns requested credential capability; P7 leaves an ordered seam for credential-dependent resource provisioning after P5.
|
||||
|
||||
## Remediation review controls
|
||||
|
||||
- B1 RED: the next-lane harness failed immediately under `ci-base:latest` as root/musl; it now models uid 1001/glibc explicitly and uses Python tree fingerprints instead of GNU `find -printf`.
|
||||
- B2 RED: framework/runtime linking consumed bare `mosaic` from PATH after P3 had committed an absolute path. The unified installer now exports/passes `MOSAIC_CLI_PATH`; the linker invokes that absolute artifact, and wizard auto-launch has no stale-PATH fallback.
|
||||
- B3 RED: a one-row manifest (`exit=1`) certified any exit-1 log. Full-manifest validation now requires the exact three cases, one exit and P0–P9 row each, pinned require/forbid populations, and rejects malformed/duplicate/unknown rows; shrink is a negative control.
|
||||
- B4 RED: uid 1001 with a passwd HOME different from ambient HOME produced P0 PASS. P0 now binds uid, username, passwd HOME and shell and explicitly rejects root and sudo-with-inherited-HOME controls.
|
||||
- B5 RED: mode-0777 CLI, mode-0644 identity, and mode-0755 credential storage passed. P3/P4/P5 now apply target owner/group plus executable/shared/private policies; framework credential storage is created 0700.
|
||||
- B6 RED: fault injection only wrote `.selftest-*` files. The synthetic path was removed; the P2–P8 matrix enters the normal action flow, proves an action observation occurred, injects after each real phase, and fingerprints rollback.
|
||||
- B7 RED: an HTTP-200 empty body exits zero when piped to Bash. The fetched installer must now be non-empty, digest-equal to `tools/install.sh.sha256`, and that exact file is executed; failed/empty/mismatch controls are blocking and CI has a remote immutable-commit arm.
|
||||
- B8 RED: raw combined command output was duplicated to terminal and `commands.log`. Both capture layers now redact before output/persistence; a seeded canary is positively emitted by the fake credential-capable registry and must remain absent from terminal, command log, npmrc, generated files and observed argv. The real greenfield fixture also scans those populations.
|
||||
- Advisory code review findings are fixed: URL userinfo redaction now handles raw `@`, repeated `:`, percent encoding and multiple URLs in both capture layers; the real greenfield path positively emits its canary through `state_run_captured`; and verified-fetch removes its temporary body after successful execution.
|
||||
- Advisory security review's independent trust-root finding is **DEFERRED by canonical PRD v2 §3**, which explicitly excludes signed provenance. README now states precisely that the same-origin sidecar detects empty/corrupt/inconsistent publication but cannot authenticate against repository/server compromise; no stronger claim remains.
|
||||
- The web1 no-manifest representativeness observation is recorded but intentionally not acted on: it is explicitly outside these eight blockers. This remediation does not weaken or otherwise change P9's manifest-presence assertion.
|
||||
|
||||
## Verification log
|
||||
|
||||
- `bash -n` and ShellCheck pass for all changed shell surfaces; `git diff --check` passes.
|
||||
- `bash tools/install-state-machine.test.sh` passes, including exact P0–P8 rows, good/bad discrimination, persisted P4/P6 action failures, P2–P8 rollback, unsafe/overlapping/symlink roots, stale `active.json`, and fatal journal initialization.
|
||||
- `bash tools/install-next-lane.test.sh` passes, including exact `@next` versions, immutable source fallback, source-build/archive-failure rollback, offline `--dev`, explicit refs, and prerelease suffix mismatch.
|
||||
- `bash tools/install-state-machine.test.sh` passes, including exact P0–P8 rows, passwd-HOME/privilege discrimination, owner/group/mode attacks, persisted P4/P6 action failures, no synthetic fault implementation, unsafe/overlapping/symlink roots, and fatal journal initialization.
|
||||
- `bash tools/install-next-lane.test.sh` passes inside `ci-base:latest`, including exact `@next` versions, immutable source fallback, source-build/archive-failure rollback, offline `--dev`, explicit refs, prerelease suffix mismatch, absolute P3 CLI propagation, secret redaction, real-action P2–P8 rollback, and stale projection recovery.
|
||||
- Comparator controls pass for verdict drift, unexpected exit, manifest shrink, missing phases, duplicate rows, unknown cases, and unknown kinds. Verified-fetch controls pass for successful execution and failed/empty/digest-mismatch rejection.
|
||||
- `bash tools/e2e-install-test.sh --lane next --source checkout --git present` returns the required expected RED in clean Debian/glibc as uid 1001: installer P0/P1/P2/P3/P7 PASS; P4/P5/P6/P8 and P9 blocking; no `Done.` claim; checkout archive digest pinned and current framework installer exercised. `tools/verify-greenfield-expected-red.sh` converts that expected detector result into a green CI assertion and fails on any unreviewed verdict drift.
|
||||
- Earlier repository gates passed: `pnpm typecheck`, `pnpm lint`, `pnpm format:check`, `pnpm test:installer`, upgrade manifest/rollback/durable-snapshot/migration suites, and focused `@mosaicstack/mosaic` tests with an isolated npm prefix. Full rerun is required after final edits.
|
||||
- Earlier repository gates passed: `pnpm typecheck`, `pnpm lint`, `pnpm format:check`, upgrade manifest/rollback/durable-snapshot/migration suites, and focused `@mosaicstack/mosaic` tests with an isolated npm prefix. Full exact-remediation rerun is required before push.
|
||||
|
||||
Reference in New Issue
Block a user