ci(installer): cover all greenfield execution arms
This commit is contained in:
+2
-2
@@ -1393,14 +1393,14 @@ A from-zero install can report success while leaving the target host unusable be
|
||||
12. The expected-RED comparator SHALL validate the complete manifest before selecting a case: exact case population, one exit and P0–P9 disposition per case, pinned require/forbid classes, and no malformed, duplicate, or unknown rows.
|
||||
13. The published installer contract SHALL reject failed fetches, HTTP-success empty bodies, and digest mismatch, then execute the exact digest-verified body. The remote CI arm SHALL enumerate every payload-acquisition path and report a bound/found denominator. It SHALL bind both the installer body and the downstream stack framework/source it consumes to the same immutable CI commit, while retaining `--next` as the lane selector. Source resolution/acquisition SHALL fail closed, and the arm SHALL verify the realised source commit and archive digest after installation. Any out-of-scope unpinned sibling SHALL be named and counted rather than silently included in a broader pinning claim. `ASSUMPTION:` the configured repository's authenticated exact-commit endpoint is trusted to map that commit ID to the returned archive bytes; independent signed provenance/authenticity against repository or TLS trust-root compromise remains excluded by canonical greenfield-install PRD v2 §3.
|
||||
14. Phase diagnostics SHALL be redacted before terminal or durable-log output. A seeded positive-control canary SHALL remain absent from observed argv, output, command logs, npm configuration, generated files, and shell history.
|
||||
15. The CI fixture SHALL publish pipeline-level execution coverage for the unconditional case set derived from the expected-RED manifest. Per-case success markers SHALL be run-scoped and written only after that case's verifier passes. A final step SHALL depend on the complete fixture matrix, run after prior success or failure, emit `cases_defined=N cases_executed=M`, and fail unless the expected and executed case-name sets are exactly equal; missing, unexpected, stale, or newly added unexecuted cases SHALL fail closed.
|
||||
15. The CI fixture SHALL publish pipeline-level execution coverage for both the unconditional case set derived from the expected-RED manifest and the explicitly declared required pipeline-arm set, including the immutable remote-installer contract. Per-case and per-arm success markers SHALL be run-scoped, stored beneath an existing checkout-archive exclusion, and written only after that arm's verifier passes. A final step SHALL depend on the complete fixture matrix, run after prior success or failure, emit both `cases_defined=N cases_executed=M` and `arms_defined=N arms_executed=M`, and fail unless both expected/executed name sets are exactly equal; missing, unexpected, stale, newly added unexecuted, or checkout-contaminating state SHALL fail closed.
|
||||
|
||||
### C1 acceptance criteria
|
||||
|
||||
1. The pre-C1 from-zero matrix records both discriminating controls: with `git` absent, the legacy installer still exits zero while P1 fails and skill sync degrades; with `git` present, P1 passes and the observed sync store/runtime links are 101/101. The C1 installer must fail at P1 before mutation when `git` is absent.
|
||||
2. The discriminating P3 row passes: the binary exists at the expected absolute path and reports exactly the resolved `next` lane version, while P4, P5, and P8 fail.
|
||||
3. The `--check` mutation negative control proves host fingerprints are byte-identical before and after observation.
|
||||
4. Woodpecker executes and validates the expected RED fixture plus the immutable remote-installer contract; its pipeline-level case-coverage gate reports exact manifest-derived set equality and a skipped-arm control proves non-execution is red. C1 does not repair P4/P5/P8 or activate #869.
|
||||
4. Woodpecker executes and validates the expected RED fixture plus the immutable remote-installer contract; its pipeline-level coverage gate reports exact equality for the manifest-derived case set and the declared arm set, while skipped-case and skipped-remote-arm controls prove non-execution is red. C1 does not repair P4/P5/P8 or activate #869.
|
||||
5. Negative controls prove manifest shrink/duplicates/unknown rows fail, unsafe P0/P3/P4/P5 contexts fail, the P2–P8 fault seam enters real actions rather than synthetic writes, empty/mismatched fetched bodies fail, and a deliberately emitted secret canary is redacted from every persisted/output population.
|
||||
|
||||
### Explicit exclusions and dependencies
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
#1050 C1 ROUND 5 — ARM COVERAGE + CHECKOUT PURITY
|
||||
|
||||
IDENTITY / SCOPE
|
||||
- predecessor head: f33bd0da96bee3ce518744d6888d018bebb1c841
|
||||
- branch: feat/1050-install-state-machine-red-fixture
|
||||
- review inputs: be-coder-06 review 110; rev-security-02 review 111
|
||||
- in scope: exact required pipeline-arm set including greenfield-remote-installer-contract; success-only arm marks; skipped-remote RED control; coverage-state exclusion from checkout source archives
|
||||
- unchanged/fenced: tools/fixtures/greenfield-expected-red.tsv, tools/verify-greenfield-expected-red.sh, tools/e2e-install-test.sh, #869, and all per-case dispositions
|
||||
- status: believed-fixed, pending jarvis validation
|
||||
|
||||
RED-FIRST — FOUR ARMS WERE REPRESENTED BY THREE CASES
|
||||
The f33bd0d gate represented only the three expected-RED case identities. With all three case names marked and no remote-arm identity available, the production checker returned:
|
||||
SKIPPED_REMOTE_ARM_CURRENT_GATE_RC=0
|
||||
[fixture-suite] cases_defined=3 cases_executed=3
|
||||
The fourth required arm, greenfield-remote-installer-contract, reused next-git-present and therefore could skip without changing case-set equality.
|
||||
|
||||
RED-FIRST — COVERAGE STATE CONTAMINATED CHECKOUT ARCHIVES
|
||||
The f33bd0d workflow created .greenfield-case-state in the shared repository before the checkout-mode fixture arms. An independent reproduction using the exact e2e-install-test.sh tar exclusions created .greenfield-case-state/remote-arm.ran and measured:
|
||||
OLD_STATE_ARCHIVE_MATCHES=1
|
||||
The state instrument was therefore part of the checkout source payload it was intended only to observe.
|
||||
|
||||
IMPLEMENTATION
|
||||
- Renamed the checker to tools/verify-greenfield-execution-coverage.sh and generalized it over two validated dimensions: cases and arms.
|
||||
- Added tools/fixtures/greenfield-expected-arms.txt as the explicit required arm set:
|
||||
greenfield-git-present
|
||||
greenfield-main-git-present
|
||||
greenfield-remote-installer-contract
|
||||
greenfield-git-absent
|
||||
- Expected cases remain derived from the unchanged expected-RED TSV. Expected arms are derived from the new one-name-per-line arm declaration. Both inputs allow comments/blanks and validate every resulting name.
|
||||
- Each of the four fixture arms writes its unique arm marker only after its exact per-case verifier succeeds. The three distinct expected-RED cases continue to write case markers after successful verification.
|
||||
- The final step checks both dimensions even when the first check fails, emits both denominators, and fails unless both exact name sets match.
|
||||
- Coverage state moved to .mosaic-test-work/greenfield-execution-coverage/${CI_PIPELINE_NUMBER}-${CI_WORKFLOW_NUMBER}/{cases,arms}. The checkout archive already excludes every */.mosaic-test-work subtree.
|
||||
- The expected-RED manifest, per-case verifier, and detector are untouched.
|
||||
|
||||
MEASURED SKIPPED-REMOTE CONTROL
|
||||
With all three cases marked and only the three non-remote arms marked:
|
||||
case gate rc=0
|
||||
[fixture-suite] cases_defined=3 cases_executed=3
|
||||
arm gate rc=1
|
||||
[fixture-suite] arms_defined=4 arms_executed=3
|
||||
[fixture-suite] missing_arm=greenfield-remote-installer-contract
|
||||
This is the discriminating control: the case denominator remains green while the arm denominator catches the exact previously invisible skip.
|
||||
|
||||
OTHER DYNAMIC CONTROLS
|
||||
- missing case: cases 3/2 => rc1
|
||||
- case count inflation (one missing + one unexpected): cases 3/3 => rc1
|
||||
- stale case markers after re-init: cases 3/0 => rc1
|
||||
- exact case set: cases 3/3 => rc0
|
||||
- future manifest case: cases 4/3 => rc1
|
||||
- arm count inflation (remote missing + unexpected): arms 4/4 => rc1
|
||||
- exact arm set: arms 4/4 => rc0
|
||||
- production expected-RED manifest comments/blanks: accepted; exact set 3/3 => rc0
|
||||
- archive selector firing control: a non-excluded root marker is present in the archive
|
||||
- archive purity control: no .mosaic-test-work path is present in the archive
|
||||
|
||||
STATIC / STRUCTURAL CONTROLS
|
||||
- every case mark follows that arm's successful per-case verifier
|
||||
- every arm mark, including the remote contract, follows that arm's successful per-case verifier
|
||||
- all four arms depend on coverage initialization
|
||||
- final denominator depends on all four arms and is eligible after success or failure
|
||||
- final denominator runs both checks and aggregates their statuses
|
||||
- workflow contains the already-excluded .mosaic-test-work state root and no .greenfield-case-state reference
|
||||
|
||||
LOCAL BASELINES
|
||||
- bash -n + ShellCheck on generalized checker/test: PASS
|
||||
- pnpm test:installer: PASS, including execution-coverage controls
|
||||
- pnpm typecheck: PASS, 45/45
|
||||
- pnpm lint: PASS, 25/25
|
||||
- pnpm format:check: PASS
|
||||
- woodpecker-cli lint --strict: PASS
|
||||
- git diff --check: PASS
|
||||
- /home free before/after: 11G
|
||||
|
||||
INDEPENDENT REVIEW
|
||||
- Codex code review: APPROVE, confidence 0.93, zero findings. It confirmed exact arm coverage, success-only marking, dual-status aggregation, excluded state, checkout-purity controls, and PRD alignment.
|
||||
- Codex security review: risk NONE, confidence 0.96, zero findings. CI/path identifiers are constrained, expansions quoted, expected sets treated as data, and checks fail closed.
|
||||
- Review sandboxes could not execute their own dynamic suite/ShellCheck because their filesystem is read-only; the local measured baselines above provide that evidence.
|
||||
|
||||
PENDING
|
||||
- commit, queue guard, one push, attribution read-back
|
||||
- announce the new full SHA; no merge or closure
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## Required artifacts
|
||||
|
||||
- [x] `docs/PRD.md` exists; #1050 C1 requirement 13 binds the complete in-scope remote stack source chain, and requirement 15 binds pipeline-level exact case-set execution coverage.
|
||||
- [x] `docs/PRD.md` exists; #1050 C1 requirement 13 binds the complete in-scope remote stack source chain, and requirement 15 binds pipeline-level exact case+arm execution coverage without checkout-state contamination.
|
||||
- [x] User guide: not applicable; no user-facing production installer behavior changed.
|
||||
- [x] Admin guide: not applicable; no operator procedure or deployment behavior changed.
|
||||
- [x] Developer guide: existing `docs/guides/installer-state-machine.md` defines the fail-closed installer model; the C1-specific remote-arm contract and evidence live in the PRD, scratchpad, and scoped verification report.
|
||||
@@ -21,4 +21,5 @@
|
||||
- [x] Trust boundary states both halves: the digest proves fetched bytes remain identical through execution/verify-after; it does not authenticate authorship against provider/TLS compromise. Independent provenance is the inherited PRD v2 §3 deferral.
|
||||
- [x] Round-3 Codex code review APPROVE (confidence 0.92, zero findings) and security re-review risk NONE (confidence 0.96, zero findings). The initial HIGH trust-root finding remains documented as the canonical signed-provenance deferral, not hidden.
|
||||
- [x] Round-4 review cycle closed: two blocking Codex findings (missing dependency ordering; production manifest comments misparsed) were accepted and fixed. Final code review APPROVE (confidence 0.94, zero findings) and security risk NONE (confidence 0.96, zero findings).
|
||||
- [x] Round-5 exact arm coverage and archive-purity controls reviewed: Codex code APPROVE (confidence 0.93, zero findings) and security risk NONE (confidence 0.96, zero findings).
|
||||
- [x] Canonical evidence remains in-repo. No external publishing action was requested or performed.
|
||||
|
||||
@@ -151,3 +151,12 @@ Denominator corrections were sent to and accepted by the lane lead before implem
|
||||
- Requirement 5 adds a pipeline-level instrument above the three per-case invocations. Expected names are derived from the manifest, markers are scoped by pipeline+workflow run, and each marker is written only after that exact per-case verifier succeeds. Every producer depends on initialization; the final step depends on the complete case/contract matrix, runs after success or failure, and requires exact expected/actual set equality rather than count equality.
|
||||
- Measured firing controls: skipped arm `3/2` rc1; count inflation `3/3` rc1 with one missing and one unexpected name; stale re-initialized run `3/0` rc1; exact set `3/3` rc0; future manifest case `4/3` rc1. Codex review found and blocked two independent defects: first the missing `depends_on` graph, then comment lines being parsed as case names. Both were accepted; dependency edges are regression-asserted, blank/comments are excluded, and the focused test now consumes the production manifest directly. Final Codex code re-review APPROVE confidence 0.94 and security re-review risk NONE confidence 0.96, both with zero findings. Woodpecker strict lint, Bash syntax, ShellCheck, focused/full installer tests, typecheck, lint, Prettier, and diff check pass.
|
||||
- Full evidence: `docs/reports/verification/1050-c1-fix-round/10-round4-case-coverage.txt`.
|
||||
|
||||
## Round 5 — arm coverage and checkout purity
|
||||
|
||||
- Reviews 110/111 identified two blockers at `f33bd0da`: the three-case set could not represent the fourth `greenfield-remote-installer-contract` arm, and the root `.greenfield-case-state` directory was included in checkout fixture archives. The expected-RED manifest, per-case verifier, detector, and #869 remain fenced.
|
||||
- RED controls: with all three cases complete and no remote-arm identity, the old gate returned rc0 at `cases_defined=3 cases_executed=3`; the exact fixture tar selector archived `.greenfield-case-state/remote-arm.ran` once.
|
||||
- The checker is generalized across exact `cases` and `arms` dimensions. A new explicit four-arm declaration includes the remote contract; all four arms mark only after successful per-case verification. With the remote omitted, cases remain 3/3 rc0 while arms report 4/3 rc1 and name `greenfield-remote-installer-contract` as missing.
|
||||
- State moved beneath `.mosaic-test-work/greenfield-execution-coverage`, which the existing checkout selector excludes. The regression control proves a non-excluded root marker is archived while no `.mosaic-test-work` path is archived.
|
||||
- Final review: Codex code APPROVE confidence 0.93 and security risk NONE confidence 0.96, both with zero findings. Full installer tests, typecheck 45/45, lint 25/25, format, Bash syntax, ShellCheck, Woodpecker strict lint, and diff check pass with 11G free before/after.
|
||||
- Full evidence: `docs/reports/verification/1050-c1-fix-round/11-round5-arm-coverage-and-purity.txt`.
|
||||
|
||||
Reference in New Issue
Block a user