diff --git a/docs/scratchpads/1000-rm-61-ci-contract-exemption.md b/docs/scratchpads/1000-rm-61-ci-contract-exemption.md index 0e0a96d7..4825f082 100644 --- a/docs/scratchpads/1000-rm-61-ci-contract-exemption.md +++ b/docs/scratchpads/1000-rm-61-ci-contract-exemption.md @@ -88,7 +88,7 @@ The nine-case contract harness was written before the verifier. First execution - Exact #2188 record: exit 0, 10 children, 9 success + 1 named exemption. - Historical set: #2158/#2167/#2184/#2186 pass with no exemption; #2170/#2175/#2182/#2187/#2188 pass with one named exemption; #2180/#2181 remain red because independent failures exist. - Provider controls: #2189 and #2191 both exit 1 under the verifier; neither is exempted. -- Unit harness: 9/9 cases pass after the red-first checkpoint. +- Unit harness: initial 9/9 cases passed after the red-first checkpoint; review remediation expands this to 12 cases with expected-head match/missing/mismatch coverage. - Test-membership guard: PASS, population 45; 26 enumerated, 19 signed exclusions; all 39 surface paths present. - Python compile: PASS. - `pnpm typecheck`: PASS, 45/45 tasks. @@ -96,6 +96,13 @@ The nine-case contract harness was written before the verifier. First execution - `pnpm format:check`: PASS after moving local evidence outside the repository tree. - `test:framework-shell`: RM-61 and all preceding suites passed, then the pre-existing wake assertion aborted with exit 97 because this host's Bash 5.2.15 reports `BASH_LINENO [3 5]` where that suite requires `[3 4]`. RM-61 does not modify the wake suite; the command is not fully runnable on this host as written and no substitute result is claimed. +## Independent review + +- Review 67 / comment 20403 at exact head `e7b29219e11efd0a19395156ac0b154bec0c3a73`: **REQUEST CHANGES**. +- Blocker: the verifier echoed the pipeline commit but did not bind it to the current PR head; mutating only #2188's commit still returned terminal-green. +- Remediation: require `--expect-commit `, add a pipeline anomaly on missing/mismatched record commits, emit expected and observed values, wire both CI documentation and the merge-gate baseline to pass provider PR head, and add match/missing/mismatch tests. +- This binding is not prohibited head-based clustering policy: it proves the evidence belongs to the commit under verdict. Runner/node/time/head correlation remains excluded from the teardown signature itself. + ## Documentation checklist - [x] CI contract documented in the canonical framework CI/CD guide. diff --git a/packages/mosaic/framework/fleet/roles/merge-gate.md b/packages/mosaic/framework/fleet/roles/merge-gate.md index 77b0759b..3f6f67b9 100644 --- a/packages/mosaic/framework/fleet/roles/merge-gate.md +++ b/packages/mosaic/framework/fleet/roles/merge-gate.md @@ -14,8 +14,10 @@ It is a **gate** role: the one and only merge path. **exclusively** by calling **`pr-merge.sh`** (the merge action, which carries the authoritative forbidden-path guard) and **`pr-ci-wait.sh`** (to wait for green CI before merging). Before issuing a verdict, scan the full JSON/API child-step - record (including `clone`) with **`verify-terminal-green.py`** and record its - exact step count, anomalies, and named exemptions. The verifier's sole interim + record (including `clone`) with **`verify-terminal-green.py --expect-commit +`** and record the equal expected/observed full-40 + commits, exact step count, anomalies, and named exemptions. Missing or mismatched + commit binding is a hard refusal. The verifier's sole interim exemption is `WP-K8S-1000-CI-POSTGRES-TEARDOWN`; it is signature-scoped, tracked by #1000, and retires when #1000 is fixed. These scripts are the _only_ sanctioned merge path. diff --git a/packages/mosaic/framework/guides/CI-CD-PIPELINES.md b/packages/mosaic/framework/guides/CI-CD-PIPELINES.md index 91802c13..035954c9 100644 --- a/packages/mosaic/framework/guides/CI-CD-PIPELINES.md +++ b/packages/mosaic/framework/guides/CI-CD-PIPELINES.md @@ -873,12 +873,16 @@ steps: A successful pipeline summary is not sufficient: verification MUST consume the full JSON/API child-step record, including `clone`. ```bash +PR_HEAD= ~/.config/mosaic/tools/woodpecker/pipeline-status.sh \ -r mosaicstack/stack -n -f json \ - | ~/.config/mosaic/tools/woodpecker/verify-terminal-green.py - + | ~/.config/mosaic/tools/woodpecker/verify-terminal-green.py \ + --expect-commit "$PR_HEAD" - ``` -The verifier reports the total step count, state counts, anomalies, and any applied exemption. Exit `0` means the record satisfies the contract; exit `1` means at least one pipeline, workflow, or child-step state blocks terminal-green; exit `2` means the JSON input could not be verified. +`PR_HEAD` MUST come from the current provider PR metadata and MUST be the full 40-hex head, not a local branch guess. The verifier fails if the argument is missing, malformed, absent from the pipeline record, or differs from that record. + +The verifier reports the expected and observed commits, total step count, state counts, anomalies, and any applied exemption. Exit `0` means the record satisfies the contract; exit `1` means the commit binding or at least one pipeline, workflow, or child-step state blocks terminal-green; exit `2` means the invocation or JSON input could not be verified. ### Named interim exemption: `WP-K8S-1000-CI-POSTGRES-TEARDOWN` diff --git a/packages/mosaic/framework/tools/woodpecker/README.md b/packages/mosaic/framework/tools/woodpecker/README.md index 80a4813a..ea163121 100644 --- a/packages/mosaic/framework/tools/woodpecker/README.md +++ b/packages/mosaic/framework/tools/woodpecker/README.md @@ -62,6 +62,7 @@ A Woodpecker API token is required. To configure: ~/.config/mosaic/tools/woodpecker/ci-wait.sh -r usc/uconnect -n 3917 -n 3918 # Verify the full JSON child-step record; do not use the text summary for this gate +PR_HEAD= ~/.config/mosaic/tools/woodpecker/pipeline-status.sh -r mosaicstack/stack -n 2188 -f json \ - | ~/.config/mosaic/tools/woodpecker/verify-terminal-green.py - + | ~/.config/mosaic/tools/woodpecker/verify-terminal-green.py --expect-commit "$PR_HEAD" - ``` diff --git a/packages/mosaic/framework/tools/woodpecker/test-terminal-green-contract.sh b/packages/mosaic/framework/tools/woodpecker/test-terminal-green-contract.sh index 3fcdd806..81e96880 100755 --- a/packages/mosaic/framework/tools/woodpecker/test-terminal-green-contract.sh +++ b/packages/mosaic/framework/tools/woodpecker/test-terminal-green-contract.sh @@ -4,6 +4,7 @@ set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" VERIFIER="$SCRIPT_DIR/verify-terminal-green.py" +EXPECTED_COMMIT=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT @@ -27,14 +28,13 @@ PY } expect_exit() { - local expected="$1" label="$2" file="$3" - shift 3 + local expected_exit="$1" label="$2" file="$3" expected_commit="${4:-$EXPECTED_COMMIT}" set +e - output=$(python3 "$VERIFIER" "$file" "$@" 2>&1) + output=$(python3 "$VERIFIER" --expect-commit "$expected_commit" "$file" 2>&1) actual=$? set -e - if [[ "$actual" -ne "$expected" ]]; then - printf 'FAIL %s: expected exit %s, got %s\n%s\n' "$label" "$expected" "$actual" "$output" >&2 + if [[ "$actual" -ne "$expected_exit" ]]; then + printf 'FAIL %s: expected exit %s, got %s\n%s\n' "$label" "$expected_exit" "$actual" "$output" >&2 exit 1 fi printf 'PASS %s\n' "$label" @@ -74,4 +74,23 @@ expect_exit 1 artifact-plus-real-failure "$TMP/artifact-plus-failure.json" >/dev write_fixture "$TMP/skipped.json" success success 0 '' skipped expect_exit 1 skipped-step "$TMP/skipped.json" >/dev/null -printf 'terminal-green contract harness: PASS (9 cases)\n' +# The scanned pipeline must be bound to an explicit, full PR-head commit. +set +e +missing_output=$(python3 "$VERIFIER" "$TMP/artifact.json" 2>&1) +missing_rc=$? +set -e +if [[ "$missing_rc" -ne 2 ]] || ! grep -q -- '--expect-commit' <<<"$missing_output"; then + printf 'FAIL missing-expected-commit: expected usage exit 2\n%s\n' "$missing_output" >&2 + exit 1 +fi +expect_exit 1 mismatched-expected-commit "$TMP/artifact.json" bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb >/dev/null + +python3 - "$TMP/artifact.json" "$TMP/missing-record-commit.json" <<'PY' +import json, sys +record = json.load(open(sys.argv[1])) +record.pop("commit") +json.dump(record, open(sys.argv[2], "w")) +PY +expect_exit 1 missing-record-commit "$TMP/missing-record-commit.json" >/dev/null + +printf 'terminal-green contract harness: PASS (12 cases)\n' diff --git a/packages/mosaic/framework/tools/woodpecker/verify-terminal-green.py b/packages/mosaic/framework/tools/woodpecker/verify-terminal-green.py index f5677aff..a16b133c 100755 --- a/packages/mosaic/framework/tools/woodpecker/verify-terminal-green.py +++ b/packages/mosaic/framework/tools/woodpecker/verify-terminal-green.py @@ -9,6 +9,7 @@ It does not fetch, retry, or re-trigger pipelines. from __future__ import annotations +import argparse import json import re import sys @@ -51,12 +52,24 @@ def is_issue_1000_artifact(step: dict[str, Any]) -> bool: ) -def verify(record: dict[str, Any]) -> tuple[int, dict[str, Any]]: +def verify(record: dict[str, Any], expected_commit: str) -> tuple[int, dict[str, Any]]: anomalies: list[dict[str, Any]] = [] candidates: list[dict[str, Any]] = [] steps: list[dict[str, Any]] = [] pipeline_status = record.get("status") + actual_commit = record.get("commit") + if actual_commit != expected_commit: + anomalies.append( + { + "scope": "pipeline", + "name": str(record.get("number", "unknown")), + "state": pipeline_status, + "reason": "pipeline commit does not equal the expected PR head", + "expected_commit": expected_commit, + "actual_commit": actual_commit, + } + ) if pipeline_status != "success": anomalies.append( { @@ -156,7 +169,8 @@ def verify(record: dict[str, Any]) -> tuple[int, dict[str, Any]]: "schema_version": "mosaic-terminal-green/v1", "verdict": "terminal-green" if not anomalies else "not-terminal-green", "pipeline_number": record.get("number"), - "commit": record.get("commit"), + "commit": actual_commit, + "expected_commit": expected_commit, "pipeline_status": pipeline_status, "total_steps": len(steps), "state_counts": dict(sorted(state_counts.items())), @@ -180,15 +194,31 @@ def verify(record: dict[str, Any]) -> tuple[int, dict[str, Any]]: return (0 if not anomalies else 1), result +def parse_arguments() -> argparse.Namespace: + parser = argparse.ArgumentParser( + description="verify the full Woodpecker terminal-green child-step contract" + ) + parser.add_argument( + "--expect-commit", + required=True, + metavar="FULL_SHA", + help="full 40-hex PR-head commit that the pipeline record must match", + ) + parser.add_argument("record", nargs="?", default="-", help="pipeline JSON file or -") + arguments = parser.parse_args() + if re.fullmatch(r"[0-9a-fA-F]{40}", arguments.expect_commit) is None: + parser.error("--expect-commit must be a full 40-hex commit") + arguments.expect_commit = arguments.expect_commit.lower() + return arguments + + def main() -> int: - if len(sys.argv) > 2 or (len(sys.argv) == 2 and sys.argv[1] in {"-h", "--help"}): - print(f"usage: {Path(sys.argv[0]).name} [pipeline.json|-]", file=sys.stderr) - return 0 if len(sys.argv) == 2 else 2 + arguments = parse_arguments() try: - record = load_record(sys.argv[1] if len(sys.argv) == 2 else None) + record = load_record(arguments.record) except (OSError, ValueError, json.JSONDecodeError) as error: return fail_usage(str(error)) - code, result = verify(record) + code, result = verify(record, arguments.expect_commit) print(json.dumps(result, indent=2, sort_keys=True)) return code