This commit is contained in:
@@ -88,7 +88,7 @@ The nine-case contract harness was written before the verifier. First execution
|
|||||||
- Exact #2188 record: exit 0, 10 children, 9 success + 1 named exemption.
|
- Exact #2188 record: exit 0, 10 children, 9 success + 1 named exemption.
|
||||||
- Historical set: #2158/#2167/#2184/#2186 pass with no exemption; #2170/#2175/#2182/#2187/#2188 pass with one named exemption; #2180/#2181 remain red because independent failures exist.
|
- Historical set: #2158/#2167/#2184/#2186 pass with no exemption; #2170/#2175/#2182/#2187/#2188 pass with one named exemption; #2180/#2181 remain red because independent failures exist.
|
||||||
- Provider controls: #2189 and #2191 both exit 1 under the verifier; neither is exempted.
|
- Provider controls: #2189 and #2191 both exit 1 under the verifier; neither is exempted.
|
||||||
- Unit harness: 9/9 cases pass after the red-first checkpoint.
|
- Unit harness: initial 9/9 cases passed after the red-first checkpoint; review remediation expands this to 12 cases with expected-head match/missing/mismatch coverage.
|
||||||
- Test-membership guard: PASS, population 45; 26 enumerated, 19 signed exclusions; all 39 surface paths present.
|
- Test-membership guard: PASS, population 45; 26 enumerated, 19 signed exclusions; all 39 surface paths present.
|
||||||
- Python compile: PASS.
|
- Python compile: PASS.
|
||||||
- `pnpm typecheck`: PASS, 45/45 tasks.
|
- `pnpm typecheck`: PASS, 45/45 tasks.
|
||||||
@@ -96,6 +96,13 @@ The nine-case contract harness was written before the verifier. First execution
|
|||||||
- `pnpm format:check`: PASS after moving local evidence outside the repository tree.
|
- `pnpm format:check`: PASS after moving local evidence outside the repository tree.
|
||||||
- `test:framework-shell`: RM-61 and all preceding suites passed, then the pre-existing wake assertion aborted with exit 97 because this host's Bash 5.2.15 reports `BASH_LINENO [3 5]` where that suite requires `[3 4]`. RM-61 does not modify the wake suite; the command is not fully runnable on this host as written and no substitute result is claimed.
|
- `test:framework-shell`: RM-61 and all preceding suites passed, then the pre-existing wake assertion aborted with exit 97 because this host's Bash 5.2.15 reports `BASH_LINENO [3 5]` where that suite requires `[3 4]`. RM-61 does not modify the wake suite; the command is not fully runnable on this host as written and no substitute result is claimed.
|
||||||
|
|
||||||
|
## Independent review
|
||||||
|
|
||||||
|
- Review 67 / comment 20403 at exact head `e7b29219e11efd0a19395156ac0b154bec0c3a73`: **REQUEST CHANGES**.
|
||||||
|
- Blocker: the verifier echoed the pipeline commit but did not bind it to the current PR head; mutating only #2188's commit still returned terminal-green.
|
||||||
|
- Remediation: require `--expect-commit <full-40>`, add a pipeline anomaly on missing/mismatched record commits, emit expected and observed values, wire both CI documentation and the merge-gate baseline to pass provider PR head, and add match/missing/mismatch tests.
|
||||||
|
- This binding is not prohibited head-based clustering policy: it proves the evidence belongs to the commit under verdict. Runner/node/time/head correlation remains excluded from the teardown signature itself.
|
||||||
|
|
||||||
## Documentation checklist
|
## Documentation checklist
|
||||||
|
|
||||||
- [x] CI contract documented in the canonical framework CI/CD guide.
|
- [x] CI contract documented in the canonical framework CI/CD guide.
|
||||||
|
|||||||
@@ -14,8 +14,10 @@ It is a **gate** role: the one and only merge path.
|
|||||||
**exclusively** by calling **`pr-merge.sh`** (the merge action, which carries the
|
**exclusively** by calling **`pr-merge.sh`** (the merge action, which carries the
|
||||||
authoritative forbidden-path guard) and **`pr-ci-wait.sh`** (to wait for green
|
authoritative forbidden-path guard) and **`pr-ci-wait.sh`** (to wait for green
|
||||||
CI before merging). Before issuing a verdict, scan the full JSON/API child-step
|
CI before merging). Before issuing a verdict, scan the full JSON/API child-step
|
||||||
record (including `clone`) with **`verify-terminal-green.py`** and record its
|
record (including `clone`) with **`verify-terminal-green.py --expect-commit
|
||||||
exact step count, anomalies, and named exemptions. The verifier's sole interim
|
<current-provider-PR-head>`** and record the equal expected/observed full-40
|
||||||
|
commits, exact step count, anomalies, and named exemptions. Missing or mismatched
|
||||||
|
commit binding is a hard refusal. The verifier's sole interim
|
||||||
exemption is `WP-K8S-1000-CI-POSTGRES-TEARDOWN`; it is signature-scoped, tracked
|
exemption is `WP-K8S-1000-CI-POSTGRES-TEARDOWN`; it is signature-scoped, tracked
|
||||||
by #1000, and retires when #1000 is fixed. These scripts are the _only_
|
by #1000, and retires when #1000 is fixed. These scripts are the _only_
|
||||||
sanctioned merge path.
|
sanctioned merge path.
|
||||||
|
|||||||
@@ -873,12 +873,16 @@ steps:
|
|||||||
A successful pipeline summary is not sufficient: verification MUST consume the full JSON/API child-step record, including `clone`.
|
A successful pipeline summary is not sufficient: verification MUST consume the full JSON/API child-step record, including `clone`.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
PR_HEAD=<full-40-hex-provider-head>
|
||||||
~/.config/mosaic/tools/woodpecker/pipeline-status.sh \
|
~/.config/mosaic/tools/woodpecker/pipeline-status.sh \
|
||||||
-r mosaicstack/stack -n <pipeline-number> -f json \
|
-r mosaicstack/stack -n <pipeline-number> -f json \
|
||||||
| ~/.config/mosaic/tools/woodpecker/verify-terminal-green.py -
|
| ~/.config/mosaic/tools/woodpecker/verify-terminal-green.py \
|
||||||
|
--expect-commit "$PR_HEAD" -
|
||||||
```
|
```
|
||||||
|
|
||||||
The verifier reports the total step count, state counts, anomalies, and any applied exemption. Exit `0` means the record satisfies the contract; exit `1` means at least one pipeline, workflow, or child-step state blocks terminal-green; exit `2` means the JSON input could not be verified.
|
`PR_HEAD` MUST come from the current provider PR metadata and MUST be the full 40-hex head, not a local branch guess. The verifier fails if the argument is missing, malformed, absent from the pipeline record, or differs from that record.
|
||||||
|
|
||||||
|
The verifier reports the expected and observed commits, total step count, state counts, anomalies, and any applied exemption. Exit `0` means the record satisfies the contract; exit `1` means the commit binding or at least one pipeline, workflow, or child-step state blocks terminal-green; exit `2` means the invocation or JSON input could not be verified.
|
||||||
|
|
||||||
### Named interim exemption: `WP-K8S-1000-CI-POSTGRES-TEARDOWN`
|
### Named interim exemption: `WP-K8S-1000-CI-POSTGRES-TEARDOWN`
|
||||||
|
|
||||||
|
|||||||
@@ -62,6 +62,7 @@ A Woodpecker API token is required. To configure:
|
|||||||
~/.config/mosaic/tools/woodpecker/ci-wait.sh -r usc/uconnect -n 3917 -n 3918
|
~/.config/mosaic/tools/woodpecker/ci-wait.sh -r usc/uconnect -n 3917 -n 3918
|
||||||
|
|
||||||
# Verify the full JSON child-step record; do not use the text summary for this gate
|
# Verify the full JSON child-step record; do not use the text summary for this gate
|
||||||
|
PR_HEAD=<full-40-hex-provider-head>
|
||||||
~/.config/mosaic/tools/woodpecker/pipeline-status.sh -r mosaicstack/stack -n 2188 -f json \
|
~/.config/mosaic/tools/woodpecker/pipeline-status.sh -r mosaicstack/stack -n 2188 -f json \
|
||||||
| ~/.config/mosaic/tools/woodpecker/verify-terminal-green.py -
|
| ~/.config/mosaic/tools/woodpecker/verify-terminal-green.py --expect-commit "$PR_HEAD" -
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ set -euo pipefail
|
|||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
VERIFIER="$SCRIPT_DIR/verify-terminal-green.py"
|
VERIFIER="$SCRIPT_DIR/verify-terminal-green.py"
|
||||||
|
EXPECTED_COMMIT=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
|
||||||
TMP=$(mktemp -d)
|
TMP=$(mktemp -d)
|
||||||
trap 'rm -rf "$TMP"' EXIT
|
trap 'rm -rf "$TMP"' EXIT
|
||||||
|
|
||||||
@@ -27,14 +28,13 @@ PY
|
|||||||
}
|
}
|
||||||
|
|
||||||
expect_exit() {
|
expect_exit() {
|
||||||
local expected="$1" label="$2" file="$3"
|
local expected_exit="$1" label="$2" file="$3" expected_commit="${4:-$EXPECTED_COMMIT}"
|
||||||
shift 3
|
|
||||||
set +e
|
set +e
|
||||||
output=$(python3 "$VERIFIER" "$file" "$@" 2>&1)
|
output=$(python3 "$VERIFIER" --expect-commit "$expected_commit" "$file" 2>&1)
|
||||||
actual=$?
|
actual=$?
|
||||||
set -e
|
set -e
|
||||||
if [[ "$actual" -ne "$expected" ]]; then
|
if [[ "$actual" -ne "$expected_exit" ]]; then
|
||||||
printf 'FAIL %s: expected exit %s, got %s\n%s\n' "$label" "$expected" "$actual" "$output" >&2
|
printf 'FAIL %s: expected exit %s, got %s\n%s\n' "$label" "$expected_exit" "$actual" "$output" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
printf 'PASS %s\n' "$label"
|
printf 'PASS %s\n' "$label"
|
||||||
@@ -74,4 +74,23 @@ expect_exit 1 artifact-plus-real-failure "$TMP/artifact-plus-failure.json" >/dev
|
|||||||
write_fixture "$TMP/skipped.json" success success 0 '' skipped
|
write_fixture "$TMP/skipped.json" success success 0 '' skipped
|
||||||
expect_exit 1 skipped-step "$TMP/skipped.json" >/dev/null
|
expect_exit 1 skipped-step "$TMP/skipped.json" >/dev/null
|
||||||
|
|
||||||
printf 'terminal-green contract harness: PASS (9 cases)\n'
|
# The scanned pipeline must be bound to an explicit, full PR-head commit.
|
||||||
|
set +e
|
||||||
|
missing_output=$(python3 "$VERIFIER" "$TMP/artifact.json" 2>&1)
|
||||||
|
missing_rc=$?
|
||||||
|
set -e
|
||||||
|
if [[ "$missing_rc" -ne 2 ]] || ! grep -q -- '--expect-commit' <<<"$missing_output"; then
|
||||||
|
printf 'FAIL missing-expected-commit: expected usage exit 2\n%s\n' "$missing_output" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
expect_exit 1 mismatched-expected-commit "$TMP/artifact.json" bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb >/dev/null
|
||||||
|
|
||||||
|
python3 - "$TMP/artifact.json" "$TMP/missing-record-commit.json" <<'PY'
|
||||||
|
import json, sys
|
||||||
|
record = json.load(open(sys.argv[1]))
|
||||||
|
record.pop("commit")
|
||||||
|
json.dump(record, open(sys.argv[2], "w"))
|
||||||
|
PY
|
||||||
|
expect_exit 1 missing-record-commit "$TMP/missing-record-commit.json" >/dev/null
|
||||||
|
|
||||||
|
printf 'terminal-green contract harness: PASS (12 cases)\n'
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ It does not fetch, retry, or re-trigger pipelines.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
import json
|
import json
|
||||||
import re
|
import re
|
||||||
import sys
|
import sys
|
||||||
@@ -51,12 +52,24 @@ def is_issue_1000_artifact(step: dict[str, Any]) -> bool:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def verify(record: dict[str, Any]) -> tuple[int, dict[str, Any]]:
|
def verify(record: dict[str, Any], expected_commit: str) -> tuple[int, dict[str, Any]]:
|
||||||
anomalies: list[dict[str, Any]] = []
|
anomalies: list[dict[str, Any]] = []
|
||||||
candidates: list[dict[str, Any]] = []
|
candidates: list[dict[str, Any]] = []
|
||||||
steps: list[dict[str, Any]] = []
|
steps: list[dict[str, Any]] = []
|
||||||
|
|
||||||
pipeline_status = record.get("status")
|
pipeline_status = record.get("status")
|
||||||
|
actual_commit = record.get("commit")
|
||||||
|
if actual_commit != expected_commit:
|
||||||
|
anomalies.append(
|
||||||
|
{
|
||||||
|
"scope": "pipeline",
|
||||||
|
"name": str(record.get("number", "unknown")),
|
||||||
|
"state": pipeline_status,
|
||||||
|
"reason": "pipeline commit does not equal the expected PR head",
|
||||||
|
"expected_commit": expected_commit,
|
||||||
|
"actual_commit": actual_commit,
|
||||||
|
}
|
||||||
|
)
|
||||||
if pipeline_status != "success":
|
if pipeline_status != "success":
|
||||||
anomalies.append(
|
anomalies.append(
|
||||||
{
|
{
|
||||||
@@ -156,7 +169,8 @@ def verify(record: dict[str, Any]) -> tuple[int, dict[str, Any]]:
|
|||||||
"schema_version": "mosaic-terminal-green/v1",
|
"schema_version": "mosaic-terminal-green/v1",
|
||||||
"verdict": "terminal-green" if not anomalies else "not-terminal-green",
|
"verdict": "terminal-green" if not anomalies else "not-terminal-green",
|
||||||
"pipeline_number": record.get("number"),
|
"pipeline_number": record.get("number"),
|
||||||
"commit": record.get("commit"),
|
"commit": actual_commit,
|
||||||
|
"expected_commit": expected_commit,
|
||||||
"pipeline_status": pipeline_status,
|
"pipeline_status": pipeline_status,
|
||||||
"total_steps": len(steps),
|
"total_steps": len(steps),
|
||||||
"state_counts": dict(sorted(state_counts.items())),
|
"state_counts": dict(sorted(state_counts.items())),
|
||||||
@@ -180,15 +194,31 @@ def verify(record: dict[str, Any]) -> tuple[int, dict[str, Any]]:
|
|||||||
return (0 if not anomalies else 1), result
|
return (0 if not anomalies else 1), result
|
||||||
|
|
||||||
|
|
||||||
|
def parse_arguments() -> argparse.Namespace:
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
description="verify the full Woodpecker terminal-green child-step contract"
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--expect-commit",
|
||||||
|
required=True,
|
||||||
|
metavar="FULL_SHA",
|
||||||
|
help="full 40-hex PR-head commit that the pipeline record must match",
|
||||||
|
)
|
||||||
|
parser.add_argument("record", nargs="?", default="-", help="pipeline JSON file or -")
|
||||||
|
arguments = parser.parse_args()
|
||||||
|
if re.fullmatch(r"[0-9a-fA-F]{40}", arguments.expect_commit) is None:
|
||||||
|
parser.error("--expect-commit must be a full 40-hex commit")
|
||||||
|
arguments.expect_commit = arguments.expect_commit.lower()
|
||||||
|
return arguments
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
def main() -> int:
|
||||||
if len(sys.argv) > 2 or (len(sys.argv) == 2 and sys.argv[1] in {"-h", "--help"}):
|
arguments = parse_arguments()
|
||||||
print(f"usage: {Path(sys.argv[0]).name} [pipeline.json|-]", file=sys.stderr)
|
|
||||||
return 0 if len(sys.argv) == 2 else 2
|
|
||||||
try:
|
try:
|
||||||
record = load_record(sys.argv[1] if len(sys.argv) == 2 else None)
|
record = load_record(arguments.record)
|
||||||
except (OSError, ValueError, json.JSONDecodeError) as error:
|
except (OSError, ValueError, json.JSONDecodeError) as error:
|
||||||
return fail_usage(str(error))
|
return fail_usage(str(error))
|
||||||
code, result = verify(record)
|
code, result = verify(record, arguments.expect_commit)
|
||||||
print(json.dumps(result, indent=2, sort_keys=True))
|
print(json.dumps(result, indent=2, sort_keys=True))
|
||||||
return code
|
return code
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user