fix(ci): assert pinned greenfield expected red
This commit is contained in:
@@ -1,6 +1,5 @@
|
||||
# C1 expected-RED gate. The fixture must execute from zero and discriminate the
|
||||
# known failed postconditions; this step is green only when the fixture itself
|
||||
# returns the expected non-zero and the named evidence rows are present.
|
||||
# C1 detector gate. The fixture itself is intentionally RED; CI is green only
|
||||
# when its exact phase verdicts/reasons match the versioned expected-RED manifest.
|
||||
when:
|
||||
- event: [pull_request, manual]
|
||||
- event: push
|
||||
@@ -18,15 +17,8 @@ steps:
|
||||
fixture_status=$?
|
||||
set -e
|
||||
cat /tmp/greenfield-git-present.log
|
||||
test "$fixture_status" -eq 1
|
||||
grep -Eq '^\[fixture\] resolved lane=next .*version=[0-9]+\.[0-9]+\.[0-9]+-next\.' /tmp/greenfield-git-present.log
|
||||
grep -q '^\[P1\] PASS: required tools present (including downstream git)' /tmp/greenfield-git-present.log
|
||||
grep -q '^\[P3\] PASS: absolute_path=.* version=.* equals resolved lane version' /tmp/greenfield-git-present.log
|
||||
grep -q '^\[P4\] FAIL: NOT-MEASURED / UNDECLARED:' /tmp/greenfield-git-present.log
|
||||
grep -q '^\[P5\] FAIL:' /tmp/greenfield-git-present.log
|
||||
grep -q '^\[P6\] FAIL:' /tmp/greenfield-git-present.log
|
||||
grep -q '^\[P8\] FAIL:' /tmp/greenfield-git-present.log
|
||||
grep -q '^\[P9\] FAIL:' /tmp/greenfield-git-present.log
|
||||
bash tools/verify-greenfield-expected-red.sh \
|
||||
next-git-present /tmp/greenfield-git-present.log "$fixture_status"
|
||||
|
||||
greenfield-main-git-present:
|
||||
image: node:22-bookworm-slim
|
||||
@@ -39,15 +31,8 @@ steps:
|
||||
fixture_status=$?
|
||||
set -e
|
||||
cat /tmp/greenfield-main-git-present.log
|
||||
test "$fixture_status" -eq 1
|
||||
grep -Eq '^\[fixture\] resolved lane=main .*version=[0-9]+\.[0-9]+\.[0-9]+' /tmp/greenfield-main-git-present.log
|
||||
grep -q '^\[P1\] PASS: required tools present (including downstream git)' /tmp/greenfield-main-git-present.log
|
||||
grep -q '^\[P3\] PASS: absolute_path=.* version=.* equals resolved lane version' /tmp/greenfield-main-git-present.log
|
||||
grep -q '^\[P4\] FAIL: NOT-MEASURED / UNDECLARED:' /tmp/greenfield-main-git-present.log
|
||||
grep -q '^\[P5\] FAIL:' /tmp/greenfield-main-git-present.log
|
||||
grep -q '^\[P6\] FAIL:' /tmp/greenfield-main-git-present.log
|
||||
grep -q '^\[P8\] FAIL:' /tmp/greenfield-main-git-present.log
|
||||
grep -q '^\[P9\] FAIL:' /tmp/greenfield-main-git-present.log
|
||||
bash tools/verify-greenfield-expected-red.sh \
|
||||
main-git-present /tmp/greenfield-main-git-present.log "$fixture_status"
|
||||
|
||||
greenfield-git-absent:
|
||||
image: node:22-bookworm-slim
|
||||
@@ -60,11 +45,5 @@ steps:
|
||||
fixture_status=$?
|
||||
set -e
|
||||
cat /tmp/greenfield-git-absent.log
|
||||
test "$fixture_status" -eq 1
|
||||
grep -q '^\[fixture\] installer_exit=1 done_claims=0' /tmp/greenfield-git-absent.log
|
||||
grep -q '^\[P1\] FAIL: undeclared/missing prerequisite(s)=git;' /tmp/greenfield-git-absent.log
|
||||
grep -q '^\[P3\] FAIL: .*executable=no' /tmp/greenfield-git-absent.log
|
||||
if grep -q 'Done\.' /tmp/greenfield-git-absent.log; then
|
||||
echo 'git-absent state-machine run falsely certified Done' >&2
|
||||
exit 1
|
||||
fi
|
||||
bash tools/verify-greenfield-expected-red.sh \
|
||||
next-git-absent /tmp/greenfield-git-absent.log "$fixture_status"
|
||||
|
||||
@@ -80,7 +80,7 @@ Rollback roots must be non-overlapping, non-symlinked, target-user-owned strict
|
||||
|
||||
`.woodpecker/greenfield-install.yml` runs `tools/e2e-install-test.sh` from zero in Debian/glibc as a non-root uid with `env -i`. No host HOME, npm cache, credentials, or bind mount enters the target process. Checkout mode packages the complete current checkout into an archive, pins its SHA-256 through an internal fixture seam, and copies the self-contained fixture into the container; framework-installer changes in the PR are therefore exercised rather than fetched from an older remote branch.
|
||||
|
||||
The C1 gate intentionally validates an attributable RED while C2–C5 remain open:
|
||||
The C1 fixture intentionally returns an attributable RED while C2–C5 remain open. CI itself remains green only when the fixture's final P0–P9 verdicts, required discriminator rows, and non-zero exit match the versioned contract in `tools/fixtures/greenfield-expected-red.tsv`. Any later remediation that changes an observed verdict makes CI red until the owning lane deliberately updates that manifest:
|
||||
|
||||
- `git` present: P1 and strict P3 pass; P4/P5/P6/P8 fail for their own reasons; P9 refuses success.
|
||||
- `git` absent: P1 fails before target mutation and the installer emits no `Done.`.
|
||||
@@ -92,7 +92,7 @@ bash tools/e2e-install-test.sh --lane next --git present
|
||||
bash tools/e2e-install-test.sh --lane main --git present
|
||||
```
|
||||
|
||||
CI exercises both lane parameters as expected-RED structural checks. The authoritative main-lane promotion acceptance and issue closure remain owned by #1037.
|
||||
CI exercises both lane parameters as expected-RED structural checks. Delivery targets `main` under the trunk-only merge rule; `next` remains a non-merging integration lane. The linked installer issue stays open after merge and closes only after Jarvis independently validates the greenfield behavior.
|
||||
|
||||
## Source trust boundary
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ Implement C1 from the canonical greenfield-install PRD v2: a transactional P0–
|
||||
|
||||
- Canonical requirements: `jason.woltje/jarvis-brain` `docs/plans/2026-08-04-greenfield-install-blockers-PRD-v2.md`. Currency was re-derived after compaction: authenticated fetch resolved `origin/main` to `cb23e5fbc8a282fa967b93d7a134fa48d11b4bb1`; the PRD and charters are byte-identical to the previously read remote copies.
|
||||
- Tracking: `mosaicstack/stack#1050` on `git.mosaicstack.dev` (author read back as `be-coder-05`).
|
||||
- Base: `origin/next` `4df478cdd150fdf8d52ea109f02ade5d85017acd`.
|
||||
- Historical implementation base: `origin/next` `4df478cdd150fdf8d52ea109f02ade5d85017acd`. Delivery PR #1054 targets `main` under L0's trunk-only rule; `next` remains a non-merging integration lane.
|
||||
- Out of scope: PATH, skills, headless wizard/identity, activation remediation, #869 wiring, RM-02, main promotion.
|
||||
- `docs/TASKS.md` is orchestrator-single-writer and is not modified by this worker.
|
||||
|
||||
@@ -49,13 +49,13 @@ Implement C1 from the canonical greenfield-install PRD v2: a transactional P0–
|
||||
- [x] P1 false pass identified from the P4 evidence row: `git` is absent from the Debian base and was undeclared even though skill sync shells out to it. C1 adds `git` to P1; the fixture matrix preserves absent/present controls. The prior claim that web1's missing runtime skills reproduce this greenfield mechanism is withdrawn by the TL and is not carried here.
|
||||
- [x] Corrected RED transcript captured and reported, including the git-present/absent controls and strict P3 PASS.
|
||||
- [x] State-machine implementation complete: private pre-mutation journal/snapshot, P0–P8 `--check`, P2–P8 fault seam, rollback, durable manifest/journal seal, action-status persistence, safe rollback roots, and stale-projection recovery.
|
||||
- [x] Debian/glibc checkout fixture now packages the complete current checkout, verifies its digest in-container, and reaches the expected attributable RED without host inheritance.
|
||||
- [x] Debian/glibc checkout fixture now packages the complete current checkout, verifies its digest in-container, and reaches the expected attributable RED without host inheritance. CI compares its exact final phase map/reasons to `tools/fixtures/greenfield-expected-red.tsv`; the fixture remains red while the detector job is green only on an exact match.
|
||||
- [ ] Reviews complete. Automated review defects around Bash conditional errexit, explicit exits, P4/P6 persisted action status, dev/offline source resolution, stale locks, checkout coverage, and rollback path safety were remediated. Remaining automated objections are the charter-mandated expected RED/C5 boundary and signed provenance, which the canonical PRD explicitly defers; independent informed review is still required.
|
||||
|
||||
## Risks / blockers
|
||||
|
||||
- The deployed create wrappers do not expose `--dry-run`; identity preflight was performed through `pr-merge.sh --dry-run` on the same HOMELAB repo, which resolved `git.mosaicstack.dev` + `be-coder-05`. The issue create then fell back from tea to the API but provider read-back confirmed author `be-coder-05`.
|
||||
- `next` is an integration lane; `main` promotion remains #1037-owned.
|
||||
- `next` is a non-merging integration lane; PR #1054 targets `main`. The old “pending promotion to main” caution dissolved when the base moved. #1050 remains open after merge and closes only after Jarvis validates the greenfield behavior.
|
||||
- #869 must remain staged and inactive.
|
||||
- Late sequencing input MB-BRAIN-01 is accommodated without implementation or renumbering: P2 covers installer distribution only; P5 owns requested credential capability; P7 leaves an ordered seam for credential-dependent resource provisioning after P5.
|
||||
|
||||
@@ -64,5 +64,5 @@ Implement C1 from the canonical greenfield-install PRD v2: a transactional P0–
|
||||
- `bash -n` and ShellCheck pass for all changed shell surfaces; `git diff --check` passes.
|
||||
- `bash tools/install-state-machine.test.sh` passes, including exact P0–P8 rows, good/bad discrimination, persisted P4/P6 action failures, P2–P8 rollback, unsafe/overlapping/symlink roots, stale `active.json`, and fatal journal initialization.
|
||||
- `bash tools/install-next-lane.test.sh` passes, including exact `@next` versions, immutable source fallback, source-build/archive-failure rollback, offline `--dev`, explicit refs, and prerelease suffix mismatch.
|
||||
- `bash tools/e2e-install-test.sh --lane next --source checkout --git present` returns the required expected RED in clean Debian/glibc as uid 1001: installer P0/P1/P2/P3/P7 PASS; P4/P5/P6/P8 and P9 blocking; no `Done.` claim; checkout archive digest pinned and current framework installer exercised.
|
||||
- `bash tools/e2e-install-test.sh --lane next --source checkout --git present` returns the required expected RED in clean Debian/glibc as uid 1001: installer P0/P1/P2/P3/P7 PASS; P4/P5/P6/P8 and P9 blocking; no `Done.` claim; checkout archive digest pinned and current framework installer exercised. `tools/verify-greenfield-expected-red.sh` converts that expected detector result into a green CI assertion and fails on any unreviewed verdict drift.
|
||||
- Earlier repository gates passed: `pnpm typecheck`, `pnpm lint`, `pnpm format:check`, `pnpm test:installer`, upgrade manifest/rollback/durable-snapshot/migration suites, and focused `@mosaicstack/mosaic` tests with an isolated npm prefix. Full rerun is required after final edits.
|
||||
|
||||
+1
-1
@@ -11,7 +11,7 @@
|
||||
"typecheck": "pnpm preflight && turbo run typecheck",
|
||||
"test:checkout": "node --test scripts/*.test.mjs",
|
||||
"test": "pnpm test:checkout && turbo run test && pnpm run test:installer",
|
||||
"test:installer": "bash tools/install-state-machine.test.sh && bash tools/install-next-lane.test.sh",
|
||||
"test:installer": "bash tools/install-state-machine.test.sh && bash tools/install-next-lane.test.sh && bash tools/verify-greenfield-expected-red.test.sh",
|
||||
"format": "prettier --write \"**/*.{ts,tsx,js,jsx,json,md}\"",
|
||||
"format:check": "prettier --check \"**/*.{ts,tsx,js,jsx,json,md}\"",
|
||||
"prepare": "node scripts/install-hooks.mjs"
|
||||
|
||||
@@ -146,7 +146,7 @@ printf '[fixture] installer_exit=%d done_claims=%s\n' \
|
||||
# P0 Resolve context
|
||||
shell="$(getent passwd "$(id -u)" | cut -d: -f7)"
|
||||
if [[ "$(id -u)" -ne 0 && "$home" == "/home/mosaic" && "$shell" == "/bin/bash" ]] \
|
||||
&& ldd --version 2>&1 | grep -qi 'glibc\|gnu libc' \
|
||||
&& ldd --version 2>&1 | grep -i 'glibc\|gnu libc' >/dev/null \
|
||||
&& [[ "$(node -p 'Number(process.versions.node.split(".")[0])')" -ge 20 ]]; then
|
||||
phase_pass P0 "target=mosaic uid=$(id -u) HOME=$home shell=$shell libc=glibc node=$(node --version)"
|
||||
else
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
# Pinned C1 expected-RED contract. Updating a verdict/reason requires review by the owning remediation lane.
|
||||
# case kind key/value
|
||||
next-git-present exit 1
|
||||
next-git-present phase P0=PASS
|
||||
next-git-present phase P1=PASS
|
||||
next-git-present phase P2=PASS
|
||||
next-git-present phase P3=PASS
|
||||
next-git-present phase P4=FAIL
|
||||
next-git-present phase P5=FAIL
|
||||
next-git-present phase P6=FAIL
|
||||
next-git-present phase P7=PASS
|
||||
next-git-present phase P8=FAIL
|
||||
next-git-present phase P9=FAIL
|
||||
next-git-present require ^\[fixture\] resolved lane=next .*version=[0-9]+\.[0-9]+\.[0-9]+-next\.
|
||||
next-git-present require ^\[fixture\] installer_exit=1 done_claims=0$
|
||||
next-git-present require ^\[P3\] PASS: absolute_path=.* version=.* equals resolved lane version$
|
||||
next-git-present require ^\[P4\] FAIL: NOT-MEASURED / UNDECLARED:
|
||||
next-git-present require ^\[P6\] FAIL:
|
||||
next-git-present forbid Done\.
|
||||
main-git-present exit 1
|
||||
main-git-present phase P0=PASS
|
||||
main-git-present phase P1=PASS
|
||||
main-git-present phase P2=PASS
|
||||
main-git-present phase P3=PASS
|
||||
main-git-present phase P4=FAIL
|
||||
main-git-present phase P5=FAIL
|
||||
main-git-present phase P6=FAIL
|
||||
main-git-present phase P7=PASS
|
||||
main-git-present phase P8=FAIL
|
||||
main-git-present phase P9=FAIL
|
||||
main-git-present require ^\[fixture\] resolved lane=main .*version=[0-9]+\.[0-9]+\.[0-9]+$
|
||||
main-git-present require ^\[fixture\] installer_exit=1 done_claims=0$
|
||||
main-git-present require ^\[P3\] PASS: absolute_path=.* version=.* equals resolved lane version$
|
||||
main-git-present require ^\[P4\] FAIL: NOT-MEASURED / UNDECLARED:
|
||||
main-git-present require ^\[P6\] FAIL:
|
||||
main-git-present forbid Done\.
|
||||
next-git-absent exit 1
|
||||
next-git-absent phase P0=PASS
|
||||
next-git-absent phase P1=FAIL
|
||||
next-git-absent phase P2=FAIL
|
||||
next-git-absent phase P3=FAIL
|
||||
next-git-absent phase P4=FAIL
|
||||
next-git-absent phase P5=FAIL
|
||||
next-git-absent phase P6=PASS
|
||||
next-git-absent phase P7=PASS
|
||||
next-git-absent phase P8=FAIL
|
||||
next-git-absent phase P9=FAIL
|
||||
next-git-absent require ^\[fixture\] installer_exit=1 done_claims=0$
|
||||
next-git-absent require ^\[P1\] FAIL: undeclared/missing prerequisite\(s\)=git;
|
||||
next-git-absent require ^\[P3\] FAIL: .*executable=no
|
||||
next-git-absent forbid Done\.
|
||||
|
+1
-1
@@ -606,7 +606,7 @@ state_predicate() {
|
||||
npm_major="$(npm --version 2>/dev/null | cut -d. -f1 || echo 0)"
|
||||
privilege_mode="$([[ "$(id -u)" -eq 0 ]] && echo root-without-explicit-target || echo user)"
|
||||
if [[ -n "$HOME" && -n "$shell" && "$privilege_mode" == "user" && "$(uname -s)" == "Linux" ]] \
|
||||
&& ldd --version 2>&1 | grep -qi 'glibc\|gnu libc' \
|
||||
&& ldd --version 2>&1 | grep -i 'glibc\|gnu libc' >/dev/null \
|
||||
&& [[ "$(uname -m)" == "x86_64" ]] && [[ "$node_major" -ge 20 ]] && [[ "$npm_major" -ge 9 ]] \
|
||||
&& state_validate_target_paths; then
|
||||
STATE_REASON="target=$(id -un) uid=$(id -u) HOME=$HOME shell=$shell privilege=$privilege_mode arch=x86_64 libc=glibc node=$(node --version) npm=$(npm --version)"
|
||||
|
||||
Executable
+63
@@ -0,0 +1,63 @@
|
||||
#!/usr/bin/env bash
|
||||
# Verify that the detector found exactly the pinned C1 phase verdicts. The
|
||||
# fixture is expected to exit non-zero; this verifier is the green CI contract.
|
||||
set -euo pipefail
|
||||
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
MANIFEST="${MOSAIC_EXPECTED_RED_MANIFEST:-$ROOT/tools/fixtures/greenfield-expected-red.tsv}"
|
||||
CASE="${1:?usage: verify-greenfield-expected-red.sh <case> <log> <fixture-exit>}"
|
||||
LOG="${2:?usage: verify-greenfield-expected-red.sh <case> <log> <fixture-exit>}"
|
||||
FIXTURE_EXIT="${3:?usage: verify-greenfield-expected-red.sh <case> <log> <fixture-exit>}"
|
||||
|
||||
[[ -r "$MANIFEST" ]] || { echo "expected-RED manifest is unreadable: $MANIFEST" >&2; exit 2; }
|
||||
[[ -r "$LOG" ]] || { echo "fixture log is unreadable: $LOG" >&2; exit 2; }
|
||||
[[ "$FIXTURE_EXIT" =~ ^[0-9]+$ ]] || { echo "fixture exit is not numeric: $FIXTURE_EXIT" >&2; exit 2; }
|
||||
|
||||
checks=0
|
||||
failures=0
|
||||
while IFS=$'\t' read -r case_name kind expectation; do
|
||||
[[ -n "$case_name" && "${case_name:0:1}" != "#" ]] || continue
|
||||
[[ "$case_name" == "$CASE" ]] || continue
|
||||
checks=$((checks + 1))
|
||||
case "$kind" in
|
||||
exit)
|
||||
if [[ "$FIXTURE_EXIT" != "$expectation" ]]; then
|
||||
echo "expected-RED mismatch: case=$CASE fixture_exit=$FIXTURE_EXIT expected=$expectation" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
;;
|
||||
phase)
|
||||
phase="${expectation%%=*}"
|
||||
expected_verdict="${expectation#*=}"
|
||||
last_row="$(grep -E "^\[$phase\] (PASS|FAIL):" "$LOG" | tail -n 1 || true)"
|
||||
actual_verdict="$(printf '%s\n' "$last_row" | sed -n "s/^\[$phase\] \(PASS\|FAIL\):.*/\1/p")"
|
||||
if [[ "$actual_verdict" != "$expected_verdict" ]]; then
|
||||
echo "expected-RED mismatch: case=$CASE phase=$phase got=${actual_verdict:-missing} expected=$expected_verdict" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
;;
|
||||
require)
|
||||
if ! grep -Eq -- "$expectation" "$LOG"; then
|
||||
echo "expected-RED missing required evidence: case=$CASE regex=$expectation" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
;;
|
||||
forbid)
|
||||
if grep -Eq -- "$expectation" "$LOG"; then
|
||||
echo "expected-RED found forbidden evidence: case=$CASE regex=$expectation" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
;;
|
||||
*)
|
||||
echo "invalid expected-RED manifest kind: case=$case_name kind=$kind" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
done < "$MANIFEST"
|
||||
|
||||
[[ "$checks" -gt 0 ]] || { echo "expected-RED manifest has no checks for case=$CASE" >&2; exit 2; }
|
||||
if [[ "$failures" -ne 0 ]]; then
|
||||
echo "expected-RED verification failed: case=$CASE failures=$failures checks=$checks" >&2
|
||||
exit 1
|
||||
fi
|
||||
printf 'expected-RED verification passed: case=%s checks=%d\n' "$CASE" "$checks"
|
||||
Executable
+36
@@ -0,0 +1,36 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
TMP="$(mktemp -d "${TMPDIR:-/tmp}/mosaic-expected-red-test.XXXXXX")"
|
||||
trap 'rm -rf "$TMP"' EXIT
|
||||
|
||||
cat > "$TMP/match.log" <<'LOG'
|
||||
[fixture] resolved lane=next package=@mosaicstack/mosaic@next version=0.0.50-next.999
|
||||
[fixture] installer_exit=1 done_claims=0
|
||||
[P0] PASS: supported context
|
||||
[P1] PASS: preflight complete
|
||||
[P2] PASS: pinned artifact
|
||||
[P3] PASS: absolute_path=/home/test/.npm-global/bin/mosaic version=0.0.50-next.999 equals resolved lane version
|
||||
[P4] FAIL: NOT-MEASURED / UNDECLARED: declaration absent
|
||||
[P5] FAIL: identity absent
|
||||
[P6] FAIL: activation unavailable
|
||||
[P7] PASS: no services requested
|
||||
[P8] FAIL: shell path absent
|
||||
[P9] FAIL: aggregate refusal
|
||||
LOG
|
||||
|
||||
bash "$ROOT/tools/verify-greenfield-expected-red.sh" next-git-present "$TMP/match.log" 1 >/dev/null
|
||||
printf '[test] PASS: matching detector findings make the CI verifier green\n'
|
||||
|
||||
sed 's/^\[P4\] FAIL:/[P4] PASS:/' "$TMP/match.log" > "$TMP/drift.log"
|
||||
if bash "$ROOT/tools/verify-greenfield-expected-red.sh" next-git-present "$TMP/drift.log" 1 >/dev/null 2>&1; then
|
||||
echo '[test] FAIL: changed P4 verdict did not invalidate the pinned manifest' >&2
|
||||
exit 1
|
||||
fi
|
||||
printf '[test] PASS: changed phase verdict requires a deliberate manifest update\n'
|
||||
|
||||
if bash "$ROOT/tools/verify-greenfield-expected-red.sh" next-git-present "$TMP/match.log" 0 >/dev/null 2>&1; then
|
||||
echo '[test] FAIL: unexpected fixture exit did not invalidate the pinned manifest' >&2
|
||||
exit 1
|
||||
fi
|
||||
printf '[test] PASS: unexpected fixture exit remains blocking\n'
|
||||
Reference in New Issue
Block a user