Compare commits
7
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f9746b23c8 | ||
|
|
38f1b249cc | ||
|
|
9b4d4beb0e | ||
|
|
e4c30a33e8 | ||
|
|
ada0cbe60e | ||
|
|
bd603da4b0 | ||
|
|
444662e79a |
+17
-3
@@ -68,15 +68,29 @@ steps:
|
||||
- bash packages/mosaic/framework/tools/quality/scripts/test-upgrade-durable-snapshot.sh
|
||||
- bash packages/mosaic/framework/tools/quality/scripts/test-install-migration.sh
|
||||
|
||||
# Anti-inert-gate registry. Deliberately unconditional: no path filter and no
|
||||
# step-level `when`, because a gate can be disabled by changes outside its own path.
|
||||
gate-verify:
|
||||
image: *node_image
|
||||
# Woodpecker's shallow marker makes merge-base reject even present parents;
|
||||
# full history is required for activation ancestry and manifest provenance.
|
||||
commands:
|
||||
- *enable_pnpm
|
||||
- apk add --no-cache bubblewrap
|
||||
- if [ -f .git/shallow ]; then git fetch --unshallow --no-tags origin; fi
|
||||
- pnpm gate:verify
|
||||
depends_on:
|
||||
- install
|
||||
- sanitization
|
||||
- upgrade-guard
|
||||
|
||||
typecheck:
|
||||
image: *node_image
|
||||
commands:
|
||||
- *enable_pnpm
|
||||
- pnpm typecheck
|
||||
depends_on:
|
||||
- install
|
||||
- sanitization
|
||||
- upgrade-guard
|
||||
- gate-verify
|
||||
|
||||
# lint, format, and test are independent — run in parallel after typecheck
|
||||
lint:
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
# Administrator Guide
|
||||
|
||||
- [Gate registry operations](quality-gate-registry.md)
|
||||
@@ -0,0 +1,35 @@
|
||||
# Gate Registry Operations
|
||||
|
||||
## Routine verification
|
||||
|
||||
Run `pnpm gate:verify` from a dependency-installed checkout. Exit zero means registry observations matched their declared `actual` values; it does **not** assert required-behavior conformance while deltas remain. Open `DEFECT` records are checked descriptions of current behavior with tracked owners, never successful gate outcomes.
|
||||
|
||||
Investigate any of these immediately:
|
||||
|
||||
- `GATE VERIFY FAILED` — registry structure, observed behavior, provenance, claim binding, source/deployment identity, or negative-control detection changed. The verifier aggregates independent phase failures, so repair the responsible stable-ID diagnostics as well as any accompanying stale-fixture error; do not treat the generic error as a substitute.
|
||||
- `unregistered gate` — an executable appeared under a declared gate root without a registry entry.
|
||||
- `no negative control` — a gate has no must-fail case.
|
||||
- `DEPLOYED IDENTITY UNAVAILABLE` — the runner cannot reach the installed enforcing copy. The pinned observation is checked, but live equality is not asserted.
|
||||
- `PROVIDER EVIDENCE ... ABSENT` — retained external history was unavailable; do not infer merge-time success.
|
||||
|
||||
## Updating a gate
|
||||
|
||||
1. Add or change the criterion, its exact criterion-side `caseRefs`, and matching case-side `criterionIds`.
|
||||
2. Observe the must-fail case fail for its own stated reason; moving the binding to any undeclared case must fail verification.
|
||||
3. Declare an exact inerting mutation and observe the verifier detect it.
|
||||
4. If required and actual behavior differ, add a tracked remediation owner and justification.
|
||||
5. If meaning changed, append provenance; never replace the original silently.
|
||||
6. For an installed counterpart, verify live byte identity and update the observed digest only from measured evidence.
|
||||
7. Run focused verifier tests, `pnpm gate:verify`, and the repository baseline gates.
|
||||
|
||||
Do not add an ownerless exception or describe an open delta as pass/green/OK.
|
||||
|
||||
## CI behavior
|
||||
|
||||
Woodpecker runs `gate-verify` on every pull request and protected-main push without path filtering. This is deliberate: changes outside gate files can make a gate inert. The step unshallows the checkout so activation ancestry and historical manifest provenance can be checked; a shallow boundary must never be interpreted as non-ancestry.
|
||||
|
||||
Provider evidence input is an optional JSON array of normalized pipeline records containing `commit`, unique integer pipeline `number`, pipeline `status`, and a `gate-verify` step status. The highest numbered rerun is authoritative; ambiguous duplicates fail. Its retention window is provider-controlled and is not overstated by this repository.
|
||||
|
||||
PR CI executes current-tree verification only, unprivileged and fail-closed. It does not execute isolated own-tree replay: RM-60 must provide a protected launcher or runner-level rootless sandbox before any PR-controlled executable/configuration is evaluated. Repo-only code cannot safely grant itself the capability intended to contain itself.
|
||||
|
||||
The deferred replay implementation remains hard-fail when its sandbox cannot be established; it is not silently skipped as a successful replay. Tests recognize unavailability only from parent-generated Bubblewrap-launch provenance combined with proof that the sandbox entry command did not run. A denial-looking string from child-controlled output is not evidence. When RM-60 activates replay under protected authority, it uses frozen own-tree dependencies, namespace/environment isolation, and archived-file identity checks. A post-merge failure triggers quarantine and revert. This is detection, not pre-merge prevention.
|
||||
@@ -0,0 +1,3 @@
|
||||
# Developer Guide
|
||||
|
||||
- [Gate registry and negative controls](quality-gate-registry.md)
|
||||
@@ -0,0 +1,45 @@
|
||||
# Gate Registry and Negative Controls
|
||||
|
||||
`gates/gates.manifest.json` is the machine-readable registry for the initial RM-02 gate slice. Run:
|
||||
|
||||
```bash
|
||||
pnpm gate:verify
|
||||
```
|
||||
|
||||
## Registered slice
|
||||
|
||||
The registry covers root typecheck, lint, and format checks; RM-01 checkout preflight; the Mosaic CI queue guard; and root Husky pre-commit/pre-push hooks. It does not imply repository-wide coverage. Framework scripts, package-local build/test scripts, templates, and deployment/release scripts remain assigned to RM-54.
|
||||
|
||||
Every gate declares exact invocations, observed and required outcomes, criterion bindings, and a single exact inerting mutation. Every must-fail case requires a non-empty reason diagnostic. The verifier rejects a stale, ambiguous, crashing, or ineffective mutation. Fixture and mutation writes reject path traversal and final-component symlinks. Independent validation phases collect labeled failures instead of letting one thrown fixture, claim, discovery, deployment, mutation, or compatibility error mask already-known stable-ID diagnostics. This proves detection of the **declared** inerting mutation, not every possible semantic weakening.
|
||||
|
||||
## Required versus actual
|
||||
|
||||
A case may record different `required` and `actual` outcomes only with a tracked owner. The verifier checks current reality against `actual`, prints each difference as `DEFECT (owner: ...)`, and fails when behavior changes without a matching registry update. A defect is never described as passing, green, or OK.
|
||||
|
||||
The queue guard currently has RM-03-owned deltas. In particular, its stdin/heredoc classifier does not consume piped status JSON, so terminal-success, no-status, and terminal-failure payloads become `unknown`; unknown and malformed states exit zero; push purpose defaults to `main`. RM-02 records these observations and does not edit the guard.
|
||||
|
||||
## Criteria, prose, and compatibility
|
||||
|
||||
Each criterion declares exact `caseRefs`; the verifier compares those semantic declarations bidirectionally with case-side `criterionIds` and requires at least one must-fail case. Moving a criterion ID to an unrelated case therefore fails as both a missing declared exercising case and an undeclared binding. Registered meta-negative controls misbind a criterion, remove meaning provenance, and misbind a prose claim, and each must make structure verification red for its stated reason.
|
||||
|
||||
Designated governing prose uses `GATE-CLAIM:<id>` markers. Each claim also names the exact must-fail `caseRef` that exercises its criterion; unknown, positive-only, unrelated, unbound, or registered-but-missing claims fail. Orchestrator-owned claims from `TASKS.md` are bound through `docs/remediation/GATE-CLAIMS.md`, which records source headings and anchored text without changing task tracking. Marker completeness still requires RM-54 review because arbitrary English claims cannot be inferred safely.
|
||||
|
||||
Compatibility checks detect direct contradictions in declared finite constructions. The verifier combines referenced case fixtures and environments in one isolated tree, rejects conflicting fixture/environment values, executes the construction's exact invocation, and checks its exact outcome. They do not prove semantic consistency of arbitrary natural language.
|
||||
|
||||
Restatements preserve original text, current text, reason, finding/task, and date.
|
||||
|
||||
## Source and deployed identity
|
||||
|
||||
A gate with an external installed counterpart declares it explicitly. When the installed queue guard is reachable, its bytes must equal repository source and an internal drift control is observed red. In CI the operator-home installation may be outside the container; the verifier checks the pinned observed source digest, reports `DEPLOYED IDENTITY UNAVAILABLE (owner: RM-04)`, and does not infer live equality.
|
||||
|
||||
## Commit and provider boundary
|
||||
|
||||
**DOES:** Every PR evaluates the current checkout's registered gates and declared inerting mutations directly, unprivileged and fail-closed.
|
||||
|
||||
**DOES NOT:** Repository-controlled PR CI does not execute a commit's own verifier in an isolated replay. Doing so safely would require granting namespace capability before PR-controlled configuration or code runs; that same PR could consume the capability directly. This is an absent trust boundary, not unfinished hardening. RM-60 owns a runner-level rootless sandbox or protected immutable launcher; RM-59 owns the parallel artifact-integrity anchor.
|
||||
|
||||
The replay implementation and abuse-case tests remain fail-closed: when invoked by a future protected authority, inability to establish Bubblewrap is terminal nonzero; controls are never omitted or treated as replay success. On an unprivileged CI runner, sandbox integration tests pass only when the result carries parent-generated Bubblewrap-launch provenance and proves the sandbox entry command never ran. Child-controlled text that merely reproduces a Bubblewrap denial is not accepted. Capable local/protected environments exercise the full abuse cases. Historical installs use frozen lockfiles, isolated network/PID/IPC/UTS and environment/home boundaries, and authoritative-file snapshots that detect lifecycle rewrites.
|
||||
|
||||
Retained provider evidence can assert terminal-success **current-tree** records for prior commits when supplied through `GATE_PROVIDER_EVIDENCE_FILE`. Each normalized record contains `commit`, unique integer pipeline `number`, pipeline `status`, and exactly one `gate-verify` step; the highest-numbered rerun is authoritative. Ambiguous duplicates fail. Absent, expired, or currently-running evidence is reported explicitly and never inferred as success.
|
||||
|
||||
Once RM-60 supplies the external pre-execution anchor, protected post-merge/main replay is detection, not pre-merge prevention. A failed replay requires quarantine of the affected result and revert of the offending merge. It must never be represented as proof that CI blocked that merge. RM-25 tracks provider enforcement.
|
||||
+52
@@ -14,6 +14,58 @@
|
||||
|
||||
---
|
||||
|
||||
## RM-02 Gate Registry and Negative-Control Verifier (#1029)
|
||||
|
||||
### Problem and objective
|
||||
|
||||
Existing deterministic gates can return success without enforcing their stated property. RM-02 introduces a machine-readable registry and an unconditional CI verifier that distinguishes required behavior from observed behavior, proves every registered negative control can detect its own failure reason, and makes source/deployment drift visible.
|
||||
|
||||
### Scope
|
||||
|
||||
**In scope:** root typecheck, lint, and format gates; RM-01 checkout preflight; the Mosaic CI queue guard; root Husky pre-commit and pre-push hooks; criterion bindings; modeled compatibility; meaning-change provenance; security/integrity prose claim markers; source-versus-deployed identity; unprivileged current-tree PR verification; retained provider CI evidence where available; and explicit deferral of isolated per-commit replay to RM-60's protected external authority.
|
||||
|
||||
**Out of scope:** fixing the queue guard (RM-03); exhaustive registration of every repository executable (RM-54); semantic proof that arbitrary English criteria are mutually satisfiable (RM-54/RM-55); a same-authority trust anchor for repository-authored evidence (RM-25/RM-59).
|
||||
|
||||
### Normative requirements
|
||||
|
||||
1. `RM02-REQ-01`: The JSON registry SHALL give every gate and criterion a stable ID and SHALL declare exact invocation, input classes, cases, exact observed and required exit codes, reason diagnostics, and criterion bindings.
|
||||
2. `RM02-REQ-02`: Every gate SHALL have at least one observed-red must-fail case. The verifier SHALL reject missing, stale, ambiguous, or ineffective declared inert mutations and SHALL name an externally inerted gate.
|
||||
3. `RM02-REQ-03`: Every acceptance criterion SHALL declare exact criterion-side `caseRefs` that match case-side `criterionIds` bidirectionally and include a case that can fail for that criterion's stated reason. Moving a binding to an unrelated case SHALL fail verification. Security/integrity prose claims in the designated governing documents SHALL carry bound `GATE-CLAIM:<id>` markers and declare the exact must-fail case exercising the claim criterion.
|
||||
4. `RM02-REQ-04`: Declared finite compatibility scenarios SHALL execute together and direct modeled contradictions SHALL fail. This does not claim semantic consistency of arbitrary English.
|
||||
5. `RM02-REQ-05`: Restated criteria SHALL retain original text, current text, reason, finding/task, and dated meaning-change history.
|
||||
6. `RM02-REQ-06`: An observed behavior differing from required behavior SHALL be reported as `DEFECT` with a tracked owner; an ownerless delta SHALL fail verification. Such a gate SHALL never be described as passing, green, or OK.
|
||||
7. `RM02-REQ-07`: Executables under declared gate roots SHALL fail with `unregistered gate` when absent from the registry. The initial coverage boundary SHALL explicitly list exclusions and bind the broader inventory to RM-54.
|
||||
8. `RM02-REQ-08`: Every gate with a deployed counterpart SHALL register source/deployed byte identity and a must-fail drift control. Gates without a deployed counterpart SHALL say so explicitly.
|
||||
9. `RM02-REQ-09`: CI SHALL run `pnpm gate:verify` on every pull request without path filtering and on protected-main pushes.
|
||||
10. `RM02-REQ-10` (restated): PR CI SHALL perform unprivileged, fail-closed current-tree verification only. Isolated per-commit replay SHALL remain deferred to RM-60's protected post-merge/main authority, cross-referenced with RM-59. That future replay is detection with a quarantine/revert response, not pre-merge prevention; inability to establish its sandbox is terminal nonzero, never skip/pass. Retained provider evidence SHALL remain distinct and SHALL never be inferred when absent.
|
||||
|
||||
#### RM02-REQ-10 meaning-change provenance
|
||||
|
||||
- **Original:** “assert that every merged commit passed every required gate, evaluated AGAINST THAT COMMIT'S OWN TREE — not against current main.”
|
||||
- **Restatement:** PR CI performs unprivileged, fail-closed current-tree verification only. Isolated per-commit replay is deferred to a protected post-merge/main authority, where it is detection with a defined quarantine/revert response — explicitly not a pre-merge gate. Inability to establish the sandbox is hard nonzero, never a skip.
|
||||
- **Reason:** Isolated replay on PR CI would require granting namespace capability to PR-controlled configuration, which the same PR could use directly before containment. The trust boundary is impossible at the repository layer, not merely expensive. RM-60 owns the external pre-execution anchor; RM-59 tracks the corresponding artifact-integrity anchor.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
1. `RM02-AC-01`: A healthy current tree returns zero while prominently reporting the queue guard's required-versus-actual `DEFECT (owner: RM-03)` delta.
|
||||
2. `RM02-AC-02`: Externally mutate any registered gate at its declared inerting point so its failure path succeeds; verification returns nonzero and names that gate. The verifier's internal meta-control is observed red before its healthy result is trusted.
|
||||
3. `RM02-AC-03`: An executable added under a declared gate root without an entry returns nonzero and includes `unregistered gate`.
|
||||
4. `RM02-AC-04`: A gate with zero must-fail cases returns nonzero and includes `no negative control`.
|
||||
5. `RM02-AC-05`: Unbound or semantically misbound criteria, prose claims bound to unrelated cases, unbound governing prose markers, ownerless behavior deltas, stale mutations, source/deployed drift, and modeled compatibility conflicts each return nonzero with the responsible stable ID. A simultaneous stale fixture or independent phase error SHALL NOT mask responsible stable-ID diagnostics. Registered meta-negative controls move a criterion binding, remove meaning provenance, and redirect a prose claim to an unrelated case; each is observed red for its stated reason.
|
||||
6. `RM02-AC-06`: CI configuration invokes the verifier unconditionally on every pull request.
|
||||
7. `RM02-AC-07`: PR output states adjacent `DOES`/`DOES NOT` boundaries: current-tree gates and inerting mutations execute unprivileged and fail-closed; isolated own-tree replay does not execute in repository-controlled PR CI. RM-60/RM-59 are named, retained provider evidence is never inferred, and future protected post-merge detection specifies quarantine/revert rather than claiming pre-merge prevention.
|
||||
|
||||
### Risks, dependencies, and verification boundary
|
||||
|
||||
- The repository verifier proves declared controls, modeled scenarios, bidirectional declared criterion/case relationships, source/deployed equality at execution time, and unprivileged current-tree behavior. It does **not** infer arbitrary-English semantics, execute isolated per-commit replay, or defend against an actor able to rewrite the gate, registry, verifier, and sandbox entry consistently.
|
||||
- Sandbox refusal tests require parent-generated Bubblewrap-launch provenance and proof that the sandbox entry command never ran; child-controlled denial-looking text alone cannot establish unavailability.
|
||||
- Repo-only code cannot both grant namespace capability to PR configuration and prevent that same PR from using the capability directly. RM-60 owns a runner/provider-controlled pre-execution boundary; RM-59 owns the parallel artifact-integrity anchor.
|
||||
- Protected post-merge replay, once RM-60 exists, is detection only. Failure requires immediate quarantine of the affected result and revert of the offending merge; it is not equivalent to a pre-merge gate.
|
||||
- External branch protection and provider CI history supply merge-time current-tree evidence where retained. RM-25 tracks provider-side enforcement.
|
||||
- `ASSUMPTION:` RM-54 is the owner for expanding registration and prose-marker coverage beyond this approved seven-gate slice; rationale: the remediation task graph already assigns the fleet-wide inert-gate audit there.
|
||||
|
||||
---
|
||||
|
||||
## Problem Statement
|
||||
|
||||
Jarvis (v0.2.0) is a self-hosted AI assistant with a Python FastAPI backend and Next.js frontend. It handles chat, projects, tasks, and LLM routing but lacks orchestration depth, agent coordination, shared memory, and remote access. The Mosaic framework (`~/.config/mosaic`) provides agent guides, shell-based orchestration tools, and quality rails — but these are loose scripts, not an integrated platform. The `@mosaicstack/*` packages in mosaic-mono-v0 began consolidating these into TypeScript packages (brain, queue, coord, cli, prdy, quality-rails) but have no UI, no auth, and no agent runtime integration.
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# Documentation Sitemap
|
||||
|
||||
## Gate verification
|
||||
|
||||
- [Developer gate registry guide](DEVELOPER-GUIDE/quality-gate-registry.md) — manifest schema, negative controls, defect deltas, modeled boundaries, and commit/provider evidence.
|
||||
- [Gate registry operations](ADMIN-GUIDE/quality-gate-registry.md) — routine verification, failure interpretation, registry updates, and unconditional CI behavior.
|
||||
- [RM-02 governing claim index](remediation/GATE-CLAIMS.md) — marker bindings for orchestrator-owned remediation claims without modifying task tracking.
|
||||
|
||||
## Compaction refresh lease broker
|
||||
|
||||
- [Internal broker protocol](architecture/lease-broker-protocol.md) — kernel identity, ancestry and generation invariants, framed requests, responses, and persisted cycle bindings.
|
||||
|
||||
@@ -0,0 +1,94 @@
|
||||
# RM-02 Gate Registry Implementation Plan
|
||||
|
||||
> **For Pi:** Use test-driven development and execute each task RED → GREEN → refactor.
|
||||
|
||||
**Goal:** Build a machine-readable seven-gate registry and an unconditional CI verifier that detects inert gates, binds criteria to observed negative controls, records defects honestly, and verifies the current PR tree unprivileged and fail-closed.
|
||||
|
||||
**Architecture:** A dependency-free Node CLI reads `gates/gates.manifest.json`, validates its closed schema and references, then runs typed cases in isolated main-disk fixtures. Gate-specific fixture setup remains declarative; exact invocations and exact observed/required exits stay in JSON. A separate history module checks activation/manifest provenance and retained external current-tree CI evidence without inferring missing evidence. Isolated own-tree execution remains fail-closed code for RM-60's future protected authority; repository-controlled PR CI does not invoke it.
|
||||
|
||||
**Tech Stack:** Node.js ESM, `node:test`, JSON, shell gates, pnpm, Woodpecker CI.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Meta-negative-control kernel
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `scripts/gate-verify.test.mjs`
|
||||
- Create: `scripts/gate-verify.mjs`
|
||||
|
||||
1. Write a black-box fixture whose gate failure branch has already been changed to success.
|
||||
2. Run `node --test scripts/gate-verify.test.mjs`; require failure because the absent verifier does not name the inert gate.
|
||||
3. Implement manifest loading, exact process execution, and named mismatch reporting only.
|
||||
4. Re-run and require the external inert mutation to produce verifier nonzero while the test passes.
|
||||
5. Add an internally applied declared mutation and require a healthy fixture to report its negative control armed.
|
||||
|
||||
### Task 2: Registry structural clauses
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `scripts/gate-verify.test.mjs`
|
||||
- Modify: `scripts/gate-verify.mjs`
|
||||
|
||||
Add failing tests, one behavior at a time, for: `unregistered gate`; `no negative control`; unbound criterion; unbound `GATE-CLAIM`; ownerless required/actual delta; stale/ambiguous/ineffective mutation; direct modeled conflict; missing meaning-change provenance. Implement the minimum validator after each observed RED.
|
||||
|
||||
### Task 3: Gate fixtures and seven-gate manifest
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `gates/gates.manifest.json`
|
||||
- Create: `gates/fixtures/quality-case.mjs`
|
||||
- Create: `gates/fixtures/preflight-case.mjs`
|
||||
- Create: `gates/fixtures/queue-case.sh`
|
||||
- Create: `gates/fixtures/hook-case.sh`
|
||||
- Modify: `scripts/gate-verify.test.mjs`
|
||||
|
||||
Register typecheck, lint, format, RM-01 preflight, queue guard, pre-commit, and pre-push. For each, first run its known-bad case against an intentionally inert fixture and observe verifier RED; then restore the real gate behavior and require its exact observed exit/reason. Record queue defects as required/actual deltas owned by RM-03, never as pass/green/OK.
|
||||
|
||||
### Task 4: Source-versus-deployed identity
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `gates/gates.manifest.json`
|
||||
- Modify: `scripts/gate-verify.test.mjs`
|
||||
- Modify: `scripts/gate-verify.mjs`
|
||||
|
||||
Write and observe a failing test with a byte-mutated deployed counterpart. Implement byte equality and internal drift mutation controls. Declare `none` explicitly for gates without deployed counterparts.
|
||||
|
||||
### Task 5: Prose claims and compatibility constructions
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/remediation/MISSION.md`
|
||||
- Modify: `docs/remediation/TASKS.md` only if coordinator authorization overrides the worker prohibition; otherwise place markers in an RM-02 claim index that references immutable source anchors.
|
||||
- Modify: `gates/gates.manifest.json`
|
||||
- Modify: verifier tests/implementation.
|
||||
|
||||
Enumerate current security/integrity claims, bind each marker/id to a negative case, and reject unbound markers. Execute finite compatibility scenarios and clearly document that arbitrary English consistency is outside the model.
|
||||
|
||||
### Task 6: Current-tree boundary, deferred replay, and provider evidence
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `scripts/gate-history.mjs`
|
||||
- Create: `scripts/gate-history.test.mjs`
|
||||
- Modify: `scripts/gate-verify.mjs`
|
||||
- Modify: `gates/gates.manifest.json`
|
||||
|
||||
Test with a synthetic git repository containing two commits whose manifests differ. PR verification must state adjacent `DOES`/`DOES NOT` boundaries and must not execute the intermediate commit's verifier. Preserve isolated replay as a direct fail-closed primitive for RM-60's future protected pre-execution authority; sandbox failure remains nonzero. Add bounded Gitea/Woodpecker current-tree status lookup for prior commits when credentials/history are available. Missing, expired, and currently-running evidence must be explicit states, never inferred success. Protected post-merge replay is detection with quarantine/revert, never pre-merge prevention.
|
||||
|
||||
### Task 7: CI and documentation
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `package.json`
|
||||
- Modify: `.woodpecker/ci.yml`
|
||||
- Create/update: `docs/DEVELOPER-GUIDE/quality-gate-registry.md`
|
||||
- Create/update: `docs/ADMIN-GUIDE/quality-gate-registry.md`
|
||||
- Modify: `docs/SITEMAP.md`
|
||||
|
||||
Add `gate:verify`; run it in an unconditional PR/main CI step. Document invocation, defect semantics, coverage/exclusions, marker syntax, replay/provider boundaries, and source/deployed identity.
|
||||
|
||||
### Task 8: Verification and delivery
|
||||
|
||||
Run focused tests, `pnpm gate:verify`, checkout tests, typecheck, lint, format, and applicable integration tests. Run Codex code and security review; remediate and re-review. Verify authorship, commit, execute queue guard before push, push, create PR via Mosaic wrapper, and send exact-head review request to rev-974 through the coordinator. Do not merge without coordinator authorization.
|
||||
@@ -0,0 +1,32 @@
|
||||
# RM-02 Governing Claim Index
|
||||
|
||||
This index binds remediation claims that live in orchestrator-owned `TASKS.md` without modifying that file. The source heading and quoted text are the reviewable anchor; `gate:verify` enforces each marker-to-criterion binding. Marker completeness beyond this approved slice remains RM-54.
|
||||
|
||||
## Prose is an enforceable claim
|
||||
|
||||
<!-- GATE-CLAIM:PROSE-IS-A-CLAIM -->
|
||||
|
||||
- Source: `docs/remediation/TASKS.md`, heading `D-20 — the orchestrator's own documentation overclaimed, and a reviewer disproved it empirically`.
|
||||
- Anchored text: “The defect was not in the code — it was in this file.”
|
||||
|
||||
## Generated-state verification scope
|
||||
|
||||
<!-- GATE-CLAIM:GENERATED-STATE-SCOPE -->
|
||||
|
||||
- Source: `docs/remediation/TASKS.md`, heading `D-19 — an integrity property that cannot exist at the layer it was specified`.
|
||||
- Anchored text: “a verifier that cannot detect the attack is not a verifier.”
|
||||
|
||||
## Execution trust boundary
|
||||
|
||||
<!-- GATE-CLAIM:EXECUTION-TRUST-BOUNDARY -->
|
||||
|
||||
- Source: RM-02 ruling recorded under `docs/remediation/TASKS.md`, RM-02 clause 4, D-25.
|
||||
- Anchored text: “SELF-VERIFICATION BY THE AUDITED PARTY IS NOT VERIFICATION.”
|
||||
- Dependency: RM-60/#1031, cross-referenced with RM-59.
|
||||
|
||||
## Criterion restatement provenance
|
||||
|
||||
<!-- GATE-CLAIM:CRITERION-RESTATEMENT -->
|
||||
|
||||
- Source: `docs/remediation/TASKS.md`, heading `D-18 — two pre-registered criteria were mutually unsatisfiable, discoverable only at implementation`.
|
||||
- Anchored text: “Both criteria were pre-registered. They cannot both be satisfied.”
|
||||
@@ -12,6 +12,8 @@ gate/program; the LLM handles only genuine judgment.
|
||||
|
||||
### First-class principle — observe the property, not the exit code
|
||||
|
||||
<!-- GATE-CLAIM:OBSERVE-PROPERTY -->
|
||||
|
||||
> **No write is done until the requested PROPERTY is observed. A success exit code is not evidence.**
|
||||
>
|
||||
> **Success output is designed to be believed.** That is the whole reason the inert-gate class exists
|
||||
@@ -30,6 +32,8 @@ gate/program; the LLM handles only genuine judgment.
|
||||
|
||||
### First-class principle — pre-registration prevents retrofitting, and nothing else
|
||||
|
||||
<!-- GATE-CLAIM:PREREGISTRATION-BOUNDARY -->
|
||||
|
||||
> **A pre-registered check set can fail in three distinct ways:**
|
||||
>
|
||||
> | mode | the set is… | found as |
|
||||
@@ -56,6 +60,8 @@ gate/program; the LLM handles only genuine judgment.
|
||||
|
||||
### Corollary — never ship an integrity claim dressed as a property
|
||||
|
||||
<!-- GATE-CLAIM:ARTIFACT-INTEGRITY-BOUNDARY -->
|
||||
|
||||
> A verification artifact that can be forged by whoever it is meant to catch verifies nothing. If a
|
||||
> manifest, marker, ledger, or receipt is writable by the same actor whose behaviour it certifies, it
|
||||
> **certifies the attack.** Such an artifact must sit inside the integrity envelope it belongs to,
|
||||
@@ -68,6 +74,8 @@ gate/program; the LLM handles only genuine judgment.
|
||||
|
||||
### First-class principle — when a property cannot exist at the layer it was specified
|
||||
|
||||
<!-- GATE-CLAIM:IMPOSSIBLE-LAYER-BOUNDARY -->
|
||||
|
||||
> Some required properties are **impossible at the layer that asked for them** — not hard, impossible.
|
||||
> A local check cannot defend against an actor who can rewrite the check itself. When that happens,
|
||||
> there are exactly three honest moves, and all three are mandatory:
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
# RM-02 Gate Registry Scratchpad (#1029)
|
||||
|
||||
## Objective
|
||||
|
||||
Deliver the seven-gate registry and RED-first anti-inert verifier on `feat/rm-02-gate-registry`, preserving required-versus-actual defects without laundering them as success.
|
||||
|
||||
## Constraints and boundaries
|
||||
|
||||
- Do not modify `ci-queue-wait.sh`; RM-03 owns that fix.
|
||||
- Do not modify `docs/remediation/TASKS.md`; workers cannot edit orchestrator tracking.
|
||||
- Every declared behavior must be observed, not inferred from an exit code.
|
||||
- Repository-local evidence does not establish same-authority tamper resistance; RM-25/RM-59 own the external trust anchor.
|
||||
- Coverage is seven logical gates; broader inventory is RM-54.
|
||||
- Budget assumption: no explicit token ceiling was supplied. Keep implementation dependency-free (stock Node), avoid repeated full monorepo installs, and keep generated test artifacts under the worktree/main disk.
|
||||
|
||||
## Plan
|
||||
|
||||
1. Update PRD and tracking references.
|
||||
2. Write black-box meta-negative-control first and observe it fail for the missing verifier behavior.
|
||||
3. Implement minimal manifest parser/case runner/mutation detector; observe meta-control succeed.
|
||||
4. Add schema, coverage, provenance, prose marker, compatibility, discovery, deployment identity, and defect-delta tests RED-first.
|
||||
5. Register seven gates with exact cases and run each case.
|
||||
6. Add prospective per-commit replay and bounded provider-evidence reporting.
|
||||
7. Wire unconditional Woodpecker CI and update developer/admin documentation.
|
||||
8. Run situational and baseline verification, independent Codex review, remediate, commit, queue guard, push, PR, coordinator/reviewer handoff.
|
||||
|
||||
## Progress
|
||||
|
||||
- 2026-08-01: Design approved by `mos-remediation`; rulings A-E and source/deployed identity addition incorporated.
|
||||
- 2026-08-01: Isolated worktree created from `origin/main` f65e9ea6; RM-01 f58b3699 verified as ancestor.
|
||||
- 2026-08-01: Git author set to `f10-coder <[email protected]>`; provider issue #1029 created with `MOSAIC_GIT_IDENTITY=f10-coder`.
|
||||
- 2026-08-01: Source/deployed queue-guard SHA-256 observed equal (`19cda2f...`); this observation is not yet an enforced property.
|
||||
|
||||
## Tests and RED-first evidence
|
||||
|
||||
- Meta-negative RED first: `node --test scripts/gate-verify.test.mjs` failed because the absent verifier did not name externally inerted `meta-fixture`.
|
||||
- Structural RED: nine tests failed before implementation for internal mutation, unregistered executable, missing negative control, ownerless delta, unbound criterion/claim, modeled conflict, missing provenance, and deployment drift.
|
||||
- Clause hardening RED: positive-only security criterion and missing registered claim marker both passed incorrectly before validation was added.
|
||||
- CI wiring RED: package script and unconditional Woodpecker step tests both failed before wiring.
|
||||
- History RED: history test failed with missing module before own-tree manifest selection/provider classification was implemented.
|
||||
- `pnpm gate:verify`: exit 0; seven gates each reported `META-NEGATIVE-CONTROL ... observed red`; queue source/deployed drift control observed red; six queue behavior deltas printed as `DEFECT (owner: RM-03)`.
|
||||
- Focused Node tests: 38/38 pass after review hardening (27 verifier/wiring plus 11 history/provider tests).
|
||||
- `pnpm typecheck`: pass (45/45 Turbo tasks).
|
||||
- `pnpm lint`: pass (25/25 Turbo tasks).
|
||||
- `pnpm format:check`: pass.
|
||||
- `pnpm test`: repository suites reached 45/46 Turbo tasks; all application/package tests shown passed, then the known host-specific wake assertion aborted exit 97 (`BASH_LINENO convention violated`, #973/D-16). No test was edited or bypassed. CI remains the authoritative full-suite environment.
|
||||
|
||||
## Self-surfaced defect
|
||||
|
||||
The queue guard's `get_state_from_status_json` runs `python3 - <<'PY'` while provider JSON is piped to the shell function. The heredoc owns stdin, so Python never reads provider JSON. Terminal success, no-status, terminal failure, and malformed payloads all classify as `unknown`; the associated fail-open outcomes are recorded under RM-03. No queue-guard source was modified.
|
||||
|
||||
## Independent review remediation
|
||||
|
||||
- Final pre-commit Codex code review found three blockers: a tautological deployment drift control, independently observed rather than combined compatibility cases, and unauthorized edits to orchestrator-owned `TASKS.md`.
|
||||
- Deployment identity now uses one shared file comparator for both live equality and a temporary drifted deployed copy; a test makes that comparator inert and proves the meta-control fails.
|
||||
- Compatibility scenarios now merge referenced fixtures/environments into one isolated construction and execute an exact scenario invocation; a test proves two independently valid fixtures coexist in the combined run.
|
||||
- `TASKS.md` changes were reverted. `docs/remediation/GATE-CLAIMS.md` binds source headings and anchored text without editing orchestrator tracking.
|
||||
- Codex security review found the Bubblewrap replay shared the runner PID namespace. Replay now unshares PID, IPC, and UTS namespaces, and an abuse-case test proves a sibling runner PID is invisible.
|
||||
- Second review found empty reason diagnostics, final-symlink fixture writes, and lifecycle-script mutation of authoritative history files. Must-fail cases now require a reason pattern; writes use no-follow semantics; and replay snapshots every archived file before install and rejects any changed, deleted, or type/mode-shifted source before executing the verifier. Dedicated negative tests cover all three.
|
||||
- Third code review found ambiguous duplicate provider steps and order-sensitive JSON outcome comparison. Provider evidence now requires exactly one `gate-verify` step in the authoritative rerun, and structural equality normalizes object keys. Both regressions have RED-first tests. Third security review reported no findings.
|
||||
- Initial PR pipeline #2177 exposed Woodpecker's shallow boundary: the activation parent object was present but marked shallow, so `merge-base --is-ancestor` correctly refused to infer ancestry. The unconditional gate step now unshallows before ancestry/provenance checks; its wiring test was observed RED before the CI fix.
|
||||
- Pipeline #2178 then proved the unprivileged Docker runner cannot establish Bubblewrap namespaces. A privileged experiment remained uncommitted and was rejected after Codex correctly rated it CRITICAL: PR-controlled code executes before an in-repository sandbox and could directly use the granted capability.
|
||||
- `mos-remediation` and `rev-974` independently ruled Option C. RM02-REQ-10 now retains its original text, restatement, and reason: PR CI verifies only the current tree, unprivileged and fail-closed; isolated own-tree replay is deferred to RM-60/#1031's external pre-execution authority, cross-referenced with RM-59. Future protected post-merge replay is detection with quarantine/revert, never pre-merge prevention.
|
||||
- RED-first boundary test proved the old path executed an inert intermediate verifier. The revised path states adjacent `DOES`/`DOES NOT` claims, validates historical manifest provenance without executing it, and infers no replay success. Direct sandbox tests remain hard-fail; unprivileged CI asserts terminal refusal instead of treating replay as success. Pipelines #2179/#2180/#2181 exposed two runner refusal forms: namespace denial as `spawnSync bwrap` with `error.code=EPERM`, and a test image without Bubblewrap as `error.code=ENOENT`. The replay diagnostic now preserves spawn errors. Parent-generated launcher/entry metadata distinguishes refusal before sandbox entry from child-controlled output; the detector recognizes exact `spawnSync bwrap` provenance for `EPERM`/`EACCES`/`ENOENT` and known namespace-refusal text only when the entry command provably did not run. Focused negative assertions reject unrelated `spawnSync git EPERM`, verifier output that merely says `bwrap ENOENT`, and exact namespace-denial impersonation without provenance or after sandbox entry.
|
||||
- Exact-head independent review at `38f1b249` found one valid diagnostic-masking blocker: canonical verification knew four stable-ID rebinding failures but a thrown stale fixture replacement reached the outer catch first and emitted only the generic error. RED-first reproduction confirmed the canonical path omitted both responsible criterion IDs. Verification now collects labeled failures independently across claims, discovery, deployment, case execution/outcome checks, mutation, and compatibility, preserving structural stable-ID failures alongside the stale-fixture signal. The canonical regression test requires both missing-binding IDs and the generic fixture error.
|
||||
- Exact-head independent review at `9b4d4beb` found two valid blockers. RED-first controls reproduced both: denial-looking child stderr was accepted as sandbox unavailability, and moving meaning/prose criterion IDs to an unrelated type-error case left `gate:verify` green. Bubblewrap execution now emits a parent-generated random entry marker and returns parent-owned launcher/entry metadata; unavailability requires Bubblewrap launcher provenance plus proof entry never ran, so exact denial impersonation from plain or entered-child results is rejected. Criterion objects now declare exact `caseRefs`, checked bidirectionally against case-side `criterionIds`; prose claims declare an exact must-fail `caseRef`. Registered must-fail cases move a criterion binding, remove meaning provenance, and redirect a prose claim, each producing its stable reason. The review freeze was deliberately lifted before remediation.
|
||||
- Option C security review reported no findings. Code review rejected an initial unrelated typecheck binding for the new security criterion. It was replaced with a dedicated registered `privileged-pr-gate` case: the fixture injects a privilege key into the gate step, the wiring control rejects it for that exact reason, and `gate:verify` observes the boundary negative control. Follow-up hardening uses a closed exact gate-step construction, rejects privilege across the entire pipeline, rejects non-canonical/merged YAML keys, and pins the unrestricted PR/main trigger block; quoted/escaped/alias/merge/duplicate/filter bypass tests pass. Final Codex code review approved with no findings.
|
||||
|
||||
## Documentation checklist
|
||||
|
||||
- PRD, developer guide, admin guide, governing claim index, sitemap, plan, and scratchpad updated.
|
||||
- User/API documentation not applicable: no user workflow or API changed.
|
||||
- Independent review documentation check pending rev-974 at the revised exact head.
|
||||
- Canonical documentation remains in-repository; no external publication requested.
|
||||
|
||||
## Risks/blockers
|
||||
|
||||
- Current queue guard intentionally has required-versus-actual deltas owned by RM-03.
|
||||
- Provider CI cannot report the currently executing pipeline as terminal success; current-commit evidence must be labeled pending and becomes historical current-tree evidence only after provider completion.
|
||||
- Isolated per-commit execution requires RM-60/#1031. Until that external authority exists, no replay success is claimed. A future protected post-merge failure requires quarantine/revert.
|
||||
- CI containers may not expose the operator-home deployed queue guard. In that layer the verifier checks the pinned observed digest and reports live identity unavailable under RM-04; it does not infer live equality.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -6,6 +6,7 @@
|
||||
"build": "turbo run build",
|
||||
"dev": "turbo run dev",
|
||||
"lint": "turbo run lint",
|
||||
"gate:verify": "node scripts/gate-verify.mjs",
|
||||
"preflight": "node scripts/preflight.mjs",
|
||||
"clean:generated": "node scripts/clean-generated.mjs",
|
||||
"typecheck": "pnpm preflight && turbo run typecheck",
|
||||
|
||||
@@ -0,0 +1,359 @@
|
||||
import { existsSync } from 'node:fs';
|
||||
import { createHash, randomUUID } from 'node:crypto';
|
||||
import { access, lstat, mkdir, mkdtemp, readFile, readdir, readlink, rm } from 'node:fs/promises';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import path from 'node:path';
|
||||
|
||||
function git(root, args, { allowFailure = false } = {}) {
|
||||
const result = spawnSync('git', args, { cwd: root, encoding: 'utf8' });
|
||||
if (result.status !== 0 && !allowFailure) {
|
||||
throw new Error(`git ${args.join(' ')} failed: ${(result.stderr || result.stdout).trim()}`);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
export async function listProspectiveCommits(root, activationCommit, head = 'HEAD') {
|
||||
const result = git(root, [
|
||||
'rev-list',
|
||||
'--first-parent',
|
||||
'--reverse',
|
||||
`${activationCommit}..${head}`,
|
||||
]);
|
||||
return result.stdout.trim() ? result.stdout.trim().split('\n') : [];
|
||||
}
|
||||
|
||||
export async function readManifestAtCommit(root, commit) {
|
||||
const result = git(root, ['show', `${commit}:gates/gates.manifest.json`]);
|
||||
return JSON.parse(result.stdout);
|
||||
}
|
||||
|
||||
async function snapshotAuthoritativeTree(root) {
|
||||
const snapshot = new Map();
|
||||
async function walk(current) {
|
||||
for (const child of await readdir(current, { withFileTypes: true })) {
|
||||
if (['.git', '.home', 'node_modules'].includes(child.name)) continue;
|
||||
const absolute = path.join(current, child.name);
|
||||
const relative = path.relative(root, absolute).split(path.sep).join('/');
|
||||
const stats = await lstat(absolute);
|
||||
if (stats.isDirectory()) {
|
||||
await walk(absolute);
|
||||
} else if (stats.isSymbolicLink()) {
|
||||
snapshot.set(relative, `symlink:${stats.mode}:${await readlink(absolute)}`);
|
||||
} else if (stats.isFile()) {
|
||||
const digest = createHash('sha256')
|
||||
.update(await readFile(absolute))
|
||||
.digest('hex');
|
||||
snapshot.set(relative, `file:${stats.mode}:${digest}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
await walk(root);
|
||||
return snapshot;
|
||||
}
|
||||
|
||||
async function authoritativeTreeChanges(root, snapshot) {
|
||||
const changes = [];
|
||||
for (const [relative, expected] of snapshot) {
|
||||
const absolute = path.join(root, relative);
|
||||
let actual;
|
||||
try {
|
||||
const stats = await lstat(absolute);
|
||||
if (stats.isSymbolicLink()) {
|
||||
actual = `symlink:${stats.mode}:${await readlink(absolute)}`;
|
||||
} else if (stats.isFile()) {
|
||||
const digest = createHash('sha256')
|
||||
.update(await readFile(absolute))
|
||||
.digest('hex');
|
||||
actual = `file:${stats.mode}:${digest}`;
|
||||
} else {
|
||||
actual = `other:${stats.mode}`;
|
||||
}
|
||||
} catch (error) {
|
||||
if (error.code !== 'ENOENT') throw error;
|
||||
actual = 'missing';
|
||||
}
|
||||
if (actual !== expected) changes.push(relative);
|
||||
}
|
||||
return changes;
|
||||
}
|
||||
|
||||
function bubblewrap(root, command, args, { storePath, timeout = 300_000 } = {}) {
|
||||
const sandboxArgs = [
|
||||
'--unshare-net',
|
||||
'--unshare-pid',
|
||||
'--unshare-ipc',
|
||||
'--unshare-uts',
|
||||
'--die-with-parent',
|
||||
'--new-session',
|
||||
'--clearenv',
|
||||
];
|
||||
for (const systemPath of ['/usr', '/bin', '/lib', '/lib64', '/etc']) {
|
||||
if (existsSync(systemPath)) sandboxArgs.push('--ro-bind', systemPath, systemPath);
|
||||
}
|
||||
sandboxArgs.push('--dev', '/dev', '--proc', '/proc', '--tmpfs', '/tmp', '--bind', root, '/work');
|
||||
if (storePath) sandboxArgs.push('--ro-bind', storePath, '/pnpm-store');
|
||||
const corepackHome = path.join(process.env.HOME ?? '', '.cache', 'node', 'corepack');
|
||||
if (existsSync(corepackHome)) sandboxArgs.push('--ro-bind', corepackHome, '/corepack');
|
||||
sandboxArgs.push(
|
||||
'--chdir',
|
||||
'/work',
|
||||
'--setenv',
|
||||
'HOME',
|
||||
'/work/.home',
|
||||
'--setenv',
|
||||
'PATH',
|
||||
'/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin',
|
||||
'--setenv',
|
||||
'LANG',
|
||||
'C.UTF-8',
|
||||
'--setenv',
|
||||
'CI',
|
||||
'true',
|
||||
);
|
||||
if (storePath) sandboxArgs.push('--setenv', 'NPM_CONFIG_STORE_DIR', '/pnpm-store');
|
||||
if (existsSync(corepackHome)) sandboxArgs.push('--setenv', 'COREPACK_HOME', '/corepack');
|
||||
const enteredMarker = `__MOSAIC_BWRAP_ENTERED_${randomUUID()}__`;
|
||||
sandboxArgs.push(
|
||||
'/bin/sh',
|
||||
'-c',
|
||||
'printf "%s\\n" "$1"; shift; exec "$@"',
|
||||
'mosaic-bwrap-entry',
|
||||
enteredMarker,
|
||||
command,
|
||||
...args,
|
||||
);
|
||||
const result = spawnSync('bwrap', sandboxArgs, { encoding: 'utf8', timeout });
|
||||
const sandboxEntered = result.stdout?.includes(enteredMarker) === true;
|
||||
return {
|
||||
...result,
|
||||
stdout: (result.stdout ?? '').replace(`${enteredMarker}\n`, ''),
|
||||
sandboxLauncher: 'bwrap',
|
||||
sandboxEntered,
|
||||
};
|
||||
}
|
||||
|
||||
export async function replayCommit(root, commit) {
|
||||
const replayRoot = await mkdtemp(
|
||||
path.join(path.dirname(root), `.gate-history-${commit.slice(0, 12)}-`),
|
||||
);
|
||||
const archive = `${replayRoot}.tar`;
|
||||
try {
|
||||
git(root, ['archive', '--format=tar', `--output=${archive}`, commit]);
|
||||
const extract = spawnSync('tar', ['-xf', archive, '-C', replayRoot], { encoding: 'utf8' });
|
||||
if (extract.status !== 0) {
|
||||
return { status: extract.status, stdout: extract.stdout, stderr: extract.stderr };
|
||||
}
|
||||
const authoritativeSnapshot = await snapshotAuthoritativeTree(replayRoot);
|
||||
await mkdir(path.join(replayRoot, '.home'), { recursive: true });
|
||||
let storePath;
|
||||
try {
|
||||
await access(path.join(replayRoot, 'package.json'));
|
||||
const init = spawnSync('git', ['init', '--quiet', replayRoot], { encoding: 'utf8' });
|
||||
if (init.status !== 0) return init;
|
||||
const store = spawnSync('pnpm', ['store', 'path'], { encoding: 'utf8' });
|
||||
if (store.status !== 0) return store;
|
||||
storePath = store.stdout.trim();
|
||||
const install = bubblewrap(
|
||||
replayRoot,
|
||||
'pnpm',
|
||||
['install', '--frozen-lockfile', '--offline'],
|
||||
{ storePath, timeout: 600_000 },
|
||||
);
|
||||
if (install.status !== 0 || install.error || install.signal) {
|
||||
return {
|
||||
...install,
|
||||
stderr: `historical frozen dependency install failed: ${install.error?.message || install.stderr || install.stdout || ''}`,
|
||||
};
|
||||
}
|
||||
const authoritativeChanges = await authoritativeTreeChanges(
|
||||
replayRoot,
|
||||
authoritativeSnapshot,
|
||||
);
|
||||
if (authoritativeChanges.length > 0) {
|
||||
return {
|
||||
status: 1,
|
||||
stdout: '',
|
||||
stderr: `authoritative archived file changed during historical install: ${authoritativeChanges.join(', ')}`,
|
||||
};
|
||||
}
|
||||
} catch (error) {
|
||||
if (error.code !== 'ENOENT') throw error;
|
||||
}
|
||||
return bubblewrap(
|
||||
replayRoot,
|
||||
process.execPath,
|
||||
[
|
||||
'/work/scripts/gate-verify.mjs',
|
||||
'--root',
|
||||
'/work',
|
||||
'--manifest',
|
||||
'gates/gates.manifest.json',
|
||||
'--skip-history',
|
||||
],
|
||||
{ storePath },
|
||||
);
|
||||
} finally {
|
||||
await rm(archive, { force: true });
|
||||
await rm(replayRoot, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
export function assessProviderEvidence(commit, pipelines) {
|
||||
const matches = pipelines.filter((candidate) => candidate.commit === commit);
|
||||
if (matches.length === 0) {
|
||||
return {
|
||||
state: 'absent',
|
||||
detail:
|
||||
'no retained provider record was supplied; retention expiry and never-ran are not inferred',
|
||||
};
|
||||
}
|
||||
const pipelineStates = new Set(['success', 'failure', 'error', 'pending', 'running', 'queued']);
|
||||
const stepStates = new Set([
|
||||
'success',
|
||||
'failure',
|
||||
'error',
|
||||
'pending',
|
||||
'running',
|
||||
'queued',
|
||||
'skipped',
|
||||
]);
|
||||
const numbers = matches.map((candidate) => candidate.number);
|
||||
const malformed = matches.some(
|
||||
(candidate) =>
|
||||
typeof candidate.commit !== 'string' ||
|
||||
candidate.commit.length === 0 ||
|
||||
!Number.isInteger(candidate.number) ||
|
||||
!pipelineStates.has(candidate.status) ||
|
||||
!Array.isArray(candidate.steps) ||
|
||||
candidate.steps.some(
|
||||
(step) =>
|
||||
typeof step?.name !== 'string' ||
|
||||
typeof step?.status !== 'string' ||
|
||||
!stepStates.has(step.status),
|
||||
),
|
||||
);
|
||||
if (malformed || new Set(numbers).size !== numbers.length) {
|
||||
return {
|
||||
state: 'terminal-failure',
|
||||
detail: 'provider records are malformed or have ambiguous pipeline numbers',
|
||||
};
|
||||
}
|
||||
const pipeline = [...matches].sort((left, right) => right.number - left.number)[0];
|
||||
const gateSteps = (pipeline.steps ?? []).filter((step) => step.name === 'gate-verify');
|
||||
if (gateSteps.length !== 1) {
|
||||
return {
|
||||
state: 'terminal-failure',
|
||||
detail: `provider record has ambiguous gate-verify step count ${gateSteps.length}`,
|
||||
};
|
||||
}
|
||||
const [gateStep] = gateSteps;
|
||||
if (pipeline.status === 'success' && gateStep.status === 'success') {
|
||||
return { state: 'terminal-success', detail: 'pipeline and gate-verify step succeeded' };
|
||||
}
|
||||
if (['pending', 'running', 'queued'].includes(pipeline.status)) {
|
||||
return { state: 'current-running', detail: `pipeline is ${pipeline.status}` };
|
||||
}
|
||||
return {
|
||||
state: 'terminal-failure',
|
||||
detail: `pipeline=${pipeline.status ?? 'unknown'}, gate-verify=${gateStep?.status ?? 'absent'}`,
|
||||
};
|
||||
}
|
||||
|
||||
async function loadProviderEvidence() {
|
||||
const evidenceFile = process.env.GATE_PROVIDER_EVIDENCE_FILE;
|
||||
if (!evidenceFile) return [];
|
||||
const parsed = JSON.parse(await readFile(evidenceFile, 'utf8'));
|
||||
if (!Array.isArray(parsed)) throw new Error('provider evidence file must contain a JSON array');
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function isMainCommit(root, head) {
|
||||
if (process.env.CI_COMMIT_BRANCH === 'main') return true;
|
||||
const result = git(root, ['merge-base', '--is-ancestor', head, 'refs/remotes/origin/main'], {
|
||||
allowFailure: true,
|
||||
});
|
||||
return result.status === 0;
|
||||
}
|
||||
|
||||
export async function verifyHistory({ root, manifest }) {
|
||||
const failures = [];
|
||||
const observations = [];
|
||||
const head = git(root, ['rev-parse', 'HEAD']).stdout.trim();
|
||||
if (!manifest.activationCommit) {
|
||||
failures.push('history activationCommit is missing');
|
||||
return { failures, observations };
|
||||
}
|
||||
const activationCheck = git(
|
||||
root,
|
||||
['merge-base', '--is-ancestor', manifest.activationCommit, head],
|
||||
{ allowFailure: true },
|
||||
);
|
||||
if (activationCheck.status !== 0) {
|
||||
failures.push(
|
||||
`history activation commit ${manifest.activationCommit} is not an ancestor of ${head}`,
|
||||
);
|
||||
return { failures, observations };
|
||||
}
|
||||
const onMain = isMainCommit(root, head);
|
||||
// RM-02 execution boundary (RM-60, cross-reference RM-59), kept adjacent in both directions:
|
||||
// DOES: run every registered current-tree gate and declared inerting mutation on PR CI,
|
||||
// unprivileged and fail-closed.
|
||||
// DOES NOT: execute a commit's own verifier in an isolated PR replay. PR-controlled code would
|
||||
// otherwise need the namespace capability intended to contain that same code. That external
|
||||
// trust boundary must be runner/provider-owned before any PR executable or config is evaluated.
|
||||
observations.push(
|
||||
`RM-02 EXECUTION BOUNDARY ${head}: DOES: verify the current tree and declared inerting mutations on every PR, unprivileged and fail-closed; DOES NOT: execute isolated per-commit verifier replay in repository-controlled CI; owner RM-60, cross-reference RM-59`,
|
||||
);
|
||||
if (!onMain) {
|
||||
observations.push(
|
||||
`PROVIDER ASSERTION DEFERRED ${head}: commit is not yet on main; retained provider evidence starts after merge and no replay success is inferred`,
|
||||
);
|
||||
}
|
||||
|
||||
const pipelines = onMain ? await loadProviderEvidence() : [];
|
||||
const commits = await listProspectiveCommits(root, manifest.activationCommit, head);
|
||||
for (const commit of commits) {
|
||||
let commitManifest;
|
||||
try {
|
||||
commitManifest = await readManifestAtCommit(root, commit);
|
||||
} catch (error) {
|
||||
failures.push(`${commit}: own-tree registry cannot be read: ${error.message}`);
|
||||
continue;
|
||||
}
|
||||
if (commitManifest.schemaVersion !== manifest.schemaVersion) {
|
||||
failures.push(`${commit}: own-tree registry schema is not supported`);
|
||||
continue;
|
||||
}
|
||||
const evidence = assessProviderEvidence(commit, pipelines);
|
||||
if (commit === head) {
|
||||
observations.push(
|
||||
`CURRENT TREE EVALUATED ${commit}: all registered cases ran from this checkout; provider evidence=${evidence.state} (${evidence.detail})`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
observations.push(
|
||||
`INTERMEDIATE REPLAY DEFERRED ${commit}: isolated own-tree execution is not performed by repository-controlled CI; owner RM-60, cross-reference RM-59; no success is inferred`,
|
||||
);
|
||||
if (!onMain) {
|
||||
observations.push(
|
||||
`PROVIDER EVIDENCE ${commit}: DEFERRED until the commit is on main; no success is inferred`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
observations.push(
|
||||
`POST-MERGE DETECTION BOUNDARY ${commit}: protected isolated replay awaits RM-60; when available, a failure requires quarantine/revert and is detection, not pre-merge prevention`,
|
||||
);
|
||||
if (evidence.state === 'terminal-failure') {
|
||||
failures.push(
|
||||
`${commit}: retained provider evidence is not terminal-success (${evidence.detail})`,
|
||||
);
|
||||
} else if (evidence.state === 'terminal-success') {
|
||||
observations.push(`PROVIDER EVIDENCE ${commit}: terminal-success (${evidence.detail})`);
|
||||
} else {
|
||||
observations.push(
|
||||
`PROVIDER EVIDENCE ${commit}: ${evidence.state.toUpperCase()} (${evidence.detail}); no merge-time success is inferred`,
|
||||
);
|
||||
}
|
||||
}
|
||||
return { failures, observations };
|
||||
}
|
||||
@@ -0,0 +1,405 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { mkdir, rm, writeFile } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import { spawn, spawnSync } from 'node:child_process';
|
||||
import test from 'node:test';
|
||||
|
||||
import {
|
||||
assessProviderEvidence,
|
||||
listProspectiveCommits,
|
||||
readManifestAtCommit,
|
||||
replayCommit,
|
||||
verifyHistory,
|
||||
} from './gate-history.mjs';
|
||||
|
||||
const fixtureRoot = path.join(process.cwd(), '.mosaic-test-work', `gate-history-${process.pid}`);
|
||||
|
||||
function sandboxUnavailable(result) {
|
||||
if (result.sandboxLauncher !== 'bwrap' || result.sandboxEntered === true) return false;
|
||||
const detail = `${result.stdout ?? ''}${result.stderr ?? ''}${result.error?.message ?? ''}`;
|
||||
const bubblewrapSpawnDenied =
|
||||
['EPERM', 'EACCES', 'ENOENT'].includes(result.error?.code) &&
|
||||
/spawnSync bwrap/i.test(result.error?.message ?? '');
|
||||
if (
|
||||
!bubblewrapSpawnDenied &&
|
||||
!/bwrap:.*(?:Operation not permitted|Creating new namespace failed)/i.test(detail)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
assert.notEqual(result.status, 0, 'sandbox unavailability must remain terminal nonzero');
|
||||
return true;
|
||||
}
|
||||
|
||||
function git(root, ...args) {
|
||||
const result = spawnSync('git', args, { cwd: root, encoding: 'utf8' });
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
return result.stdout.trim();
|
||||
}
|
||||
|
||||
async function commitManifest(root, marker) {
|
||||
await mkdir(path.join(root, 'gates'), { recursive: true });
|
||||
await writeFile(
|
||||
path.join(root, 'gates', 'gates.manifest.json'),
|
||||
`${JSON.stringify({ schemaVersion: 1, marker })}\n`,
|
||||
);
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', marker);
|
||||
return git(root, 'rev-parse', 'HEAD');
|
||||
}
|
||||
|
||||
test.after(async () => {
|
||||
await rm(fixtureRoot, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('sandbox refusal classification requires Bubblewrap provenance', () => {
|
||||
for (const code of ['EPERM', 'EACCES', 'ENOENT']) {
|
||||
assert.equal(
|
||||
sandboxUnavailable({
|
||||
status: null,
|
||||
error: { code, message: `spawnSync bwrap ${code}` },
|
||||
sandboxLauncher: 'bwrap',
|
||||
sandboxEntered: false,
|
||||
}),
|
||||
true,
|
||||
);
|
||||
}
|
||||
assert.equal(
|
||||
sandboxUnavailable({
|
||||
status: null,
|
||||
error: { code: 'EPERM', message: 'spawnSync git EPERM' },
|
||||
}),
|
||||
false,
|
||||
);
|
||||
assert.equal(
|
||||
sandboxUnavailable({ status: 1, stderr: 'historical verifier said bwrap ENOENT' }),
|
||||
false,
|
||||
);
|
||||
assert.equal(
|
||||
sandboxUnavailable({
|
||||
status: 1,
|
||||
stderr: 'bwrap: Creating new namespace failed: Operation not permitted',
|
||||
sandboxLauncher: 'bwrap',
|
||||
sandboxEntered: false,
|
||||
}),
|
||||
true,
|
||||
);
|
||||
assert.equal(
|
||||
sandboxUnavailable({
|
||||
status: 1,
|
||||
stderr: 'bwrap: Creating new namespace failed: Operation not permitted',
|
||||
}),
|
||||
false,
|
||||
);
|
||||
assert.equal(
|
||||
sandboxUnavailable({
|
||||
status: 1,
|
||||
stderr: 'bwrap: Creating new namespace failed: Operation not permitted',
|
||||
sandboxLauncher: 'bwrap',
|
||||
sandboxEntered: true,
|
||||
}),
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
test('prospective history reads each commit own manifest rather than the current tree', async () => {
|
||||
await rm(fixtureRoot, { recursive: true, force: true });
|
||||
await mkdir(fixtureRoot, { recursive: true });
|
||||
git(fixtureRoot, 'init', '-q');
|
||||
git(fixtureRoot, 'config', 'user.name', 'gate-test');
|
||||
git(fixtureRoot, 'config', 'user.email', '[email protected]');
|
||||
await writeFile(path.join(fixtureRoot, 'activation.txt'), 'activation\n');
|
||||
git(fixtureRoot, 'add', '.');
|
||||
git(fixtureRoot, 'commit', '-m', 'activation');
|
||||
const activation = git(fixtureRoot, 'rev-parse', 'HEAD');
|
||||
const first = await commitManifest(fixtureRoot, 'FIRST');
|
||||
const second = await commitManifest(fixtureRoot, 'SECOND');
|
||||
|
||||
assert.deepEqual(await listProspectiveCommits(fixtureRoot, activation, second), [first, second]);
|
||||
assert.equal((await readManifestAtCommit(fixtureRoot, first)).marker, 'FIRST');
|
||||
assert.equal((await readManifestAtCommit(fixtureRoot, second)).marker, 'SECOND');
|
||||
});
|
||||
|
||||
test('historical replay executes each selected commit verifier from that commit tree', async () => {
|
||||
const root = `${fixtureRoot}-replay`;
|
||||
await rm(root, { recursive: true, force: true });
|
||||
await mkdir(path.join(root, 'scripts'), { recursive: true });
|
||||
await mkdir(path.join(root, 'gates'), { recursive: true });
|
||||
git(root, 'init', '-q');
|
||||
git(root, 'config', 'user.name', 'gate-test');
|
||||
git(root, 'config', 'user.email', '[email protected]');
|
||||
await writeFile(path.join(root, 'gates', 'gates.manifest.json'), '{"schemaVersion":1}\n');
|
||||
await writeFile(
|
||||
path.join(root, 'scripts', 'gate-verify.mjs'),
|
||||
"process.stderr.write('OLD TREE INERT\\n'); process.exitCode = 1;\n",
|
||||
);
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', 'inert historical verifier');
|
||||
const inert = git(root, 'rev-parse', 'HEAD');
|
||||
await writeFile(
|
||||
path.join(root, 'scripts', 'gate-verify.mjs'),
|
||||
"process.stdout.write('NEW TREE VERIFIED\\n');\n",
|
||||
);
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', 'fixed historical verifier');
|
||||
const fixed = git(root, 'rev-parse', 'HEAD');
|
||||
|
||||
const inertResult = await replayCommit(root, inert);
|
||||
const fixedResult = await replayCommit(root, fixed);
|
||||
if (sandboxUnavailable(inertResult) || sandboxUnavailable(fixedResult)) return;
|
||||
assert.notEqual(inertResult.status, 0);
|
||||
assert.match(inertResult.stderr, /OLD TREE INERT/);
|
||||
assert.equal(fixedResult.status, 0);
|
||||
assert.match(fixedResult.stdout, /NEW TREE VERIFIED/);
|
||||
});
|
||||
|
||||
test('historical install lifecycle cannot replace an authoritative verifier', async () => {
|
||||
const root = `${fixtureRoot}-install-tamper`;
|
||||
await rm(root, { recursive: true, force: true });
|
||||
await mkdir(path.join(root, 'scripts'), { recursive: true });
|
||||
await mkdir(path.join(root, 'gates'), { recursive: true });
|
||||
git(root, 'init', '-q');
|
||||
git(root, 'config', 'user.name', 'gate-test');
|
||||
git(root, 'config', 'user.email', '[email protected]');
|
||||
await writeFile(path.join(root, 'gates', 'gates.manifest.json'), '{"schemaVersion":1}\n');
|
||||
await writeFile(
|
||||
path.join(root, 'scripts', 'gate-verify.mjs'),
|
||||
"process.stderr.write('ORIGINAL VERIFIER RAN\\n'); process.exitCode = 7;\n",
|
||||
);
|
||||
await writeFile(path.join(root, 'forged.mjs'), "process.stdout.write('FORGED SUCCESS\\n');\n");
|
||||
await writeFile(
|
||||
path.join(root, 'package.json'),
|
||||
`${JSON.stringify({
|
||||
name: 'historical-install-tamper',
|
||||
version: '1.0.0',
|
||||
scripts: { postinstall: 'cp forged.mjs scripts/gate-verify.mjs' },
|
||||
})}\n`,
|
||||
);
|
||||
await writeFile(
|
||||
path.join(root, 'pnpm-lock.yaml'),
|
||||
"lockfileVersion: '9.0'\nsettings:\n autoInstallPeers: true\n excludeLinksFromLockfile: false\nimporters:\n .: {}\n",
|
||||
);
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', 'tampering lifecycle fixture');
|
||||
const commit = git(root, 'rev-parse', 'HEAD');
|
||||
|
||||
const result = await replayCommit(root, commit);
|
||||
assert.notEqual(result.status, 0);
|
||||
if (sandboxUnavailable(result)) return;
|
||||
assert.match(result.stderr, /authoritative archived file changed.*scripts\/gate-verify\.mjs/i);
|
||||
assert.doesNotMatch(result.stdout, /FORGED SUCCESS/);
|
||||
});
|
||||
|
||||
test('historical verifier receives no current-process secret environment', async () => {
|
||||
const root = `${fixtureRoot}-secretless`;
|
||||
await rm(root, { recursive: true, force: true });
|
||||
await mkdir(path.join(root, 'scripts'), { recursive: true });
|
||||
await mkdir(path.join(root, 'gates'), { recursive: true });
|
||||
git(root, 'init', '-q');
|
||||
git(root, 'config', 'user.name', 'gate-test');
|
||||
git(root, 'config', 'user.email', '[email protected]');
|
||||
await writeFile(path.join(root, 'gates', 'gates.manifest.json'), '{"schemaVersion":1}\n');
|
||||
await writeFile(
|
||||
path.join(root, 'scripts', 'gate-verify.mjs'),
|
||||
"if (process.env.REPLAY_SENTINEL) { process.stderr.write('SECRET LEAKED\\n'); process.exitCode = 9; } else { process.stdout.write('SECRETLESS\\n'); }\n",
|
||||
);
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', 'secretless replay fixture');
|
||||
const commit = git(root, 'rev-parse', 'HEAD');
|
||||
|
||||
process.env.REPLAY_SENTINEL = 'must-not-cross-boundary';
|
||||
try {
|
||||
const result = await replayCommit(root, commit);
|
||||
if (sandboxUnavailable(result)) return;
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
assert.match(result.stdout, /SECRETLESS/);
|
||||
assert.doesNotMatch(
|
||||
`${result.stdout}${result.stderr}`,
|
||||
/SECRET LEAKED|must-not-cross-boundary/,
|
||||
);
|
||||
} finally {
|
||||
delete process.env.REPLAY_SENTINEL;
|
||||
}
|
||||
});
|
||||
|
||||
test('historical replay cannot observe a sibling process in the runner PID namespace', async () => {
|
||||
const root = `${fixtureRoot}-pidless`;
|
||||
await rm(root, { recursive: true, force: true });
|
||||
await mkdir(path.join(root, 'scripts'), { recursive: true });
|
||||
await mkdir(path.join(root, 'gates'), { recursive: true });
|
||||
git(root, 'init', '-q');
|
||||
git(root, 'config', 'user.name', 'gate-test');
|
||||
git(root, 'config', 'user.email', '[email protected]');
|
||||
await writeFile(path.join(root, 'gates', 'gates.manifest.json'), '{"schemaVersion":1}\n');
|
||||
|
||||
const sleeper = spawn('sleep', ['30'], {
|
||||
env: { ...process.env, REPLAY_PID_SENTINEL: 'must-not-be-visible' },
|
||||
});
|
||||
try {
|
||||
await writeFile(
|
||||
path.join(root, 'scripts', 'gate-verify.mjs'),
|
||||
`import { existsSync } from 'node:fs';\nif (existsSync('/proc/${sleeper.pid}/environ')) { process.stderr.write('HOST PID VISIBLE\\n'); process.exitCode = 9; } else { process.stdout.write('PIDLESS\\n'); }\n`,
|
||||
);
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', 'pid-isolated replay fixture');
|
||||
const commit = git(root, 'rev-parse', 'HEAD');
|
||||
const result = await replayCommit(root, commit);
|
||||
if (sandboxUnavailable(result)) return;
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
assert.match(result.stdout, /PIDLESS/);
|
||||
assert.doesNotMatch(`${result.stdout}${result.stderr}`, /HOST PID VISIBLE/);
|
||||
} finally {
|
||||
sleeper.kill('SIGTERM');
|
||||
}
|
||||
});
|
||||
|
||||
test('PR verification states the RM-60 boundary without executing an intermediate verifier', async () => {
|
||||
const root = `${fixtureRoot}-feature`;
|
||||
await rm(root, { recursive: true, force: true });
|
||||
await mkdir(root, { recursive: true });
|
||||
git(root, 'init', '-q');
|
||||
git(root, 'config', 'user.name', 'gate-test');
|
||||
git(root, 'config', 'user.email', '[email protected]');
|
||||
await writeFile(path.join(root, 'activation.txt'), 'activation\n');
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', 'activation');
|
||||
const activation = git(root, 'rev-parse', 'HEAD');
|
||||
git(root, 'update-ref', 'refs/remotes/origin/main', activation);
|
||||
await mkdir(path.join(root, 'scripts'), { recursive: true });
|
||||
await mkdir(path.join(root, 'gates'), { recursive: true });
|
||||
await writeFile(path.join(root, 'gates', 'gates.manifest.json'), '{"schemaVersion":1}\n');
|
||||
await writeFile(
|
||||
path.join(root, 'scripts', 'gate-verify.mjs'),
|
||||
"process.stderr.write('INTERMEDIATE INERT\\n'); process.exitCode = 1;\n",
|
||||
);
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', 'inert intermediate');
|
||||
await writeFile(
|
||||
path.join(root, 'scripts', 'gate-verify.mjs'),
|
||||
"process.stdout.write('HEAD HEALTHY\\n');\n",
|
||||
);
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', 'healthy head');
|
||||
|
||||
const previousBranch = process.env.CI_COMMIT_BRANCH;
|
||||
process.env.CI_COMMIT_BRANCH = 'feature/rm-02';
|
||||
try {
|
||||
const result = await verifyHistory({
|
||||
root,
|
||||
manifest: { schemaVersion: 1, activationCommit: activation },
|
||||
});
|
||||
assert.deepEqual(result.failures, []);
|
||||
assert.ok(
|
||||
result.observations.some((observation) =>
|
||||
/DOES:.*current tree.*DOES NOT:.*isolated.*RM-60.*RM-59/i.test(observation),
|
||||
),
|
||||
);
|
||||
assert.ok(
|
||||
result.observations.some((observation) =>
|
||||
/INTERMEDIATE REPLAY DEFERRED.*RM-60.*no success is inferred/i.test(observation),
|
||||
),
|
||||
);
|
||||
assert.ok(result.observations.every((observation) => !/INTERMEDIATE INERT/.test(observation)));
|
||||
} finally {
|
||||
if (previousBranch === undefined) delete process.env.CI_COMMIT_BRANCH;
|
||||
else process.env.CI_COMMIT_BRANCH = previousBranch;
|
||||
}
|
||||
});
|
||||
|
||||
test('provider evidence distinguishes retained success, failure, and absent history', () => {
|
||||
const pipelines = [
|
||||
{
|
||||
commit: 'aaa',
|
||||
number: 1,
|
||||
status: 'success',
|
||||
steps: [{ name: 'gate-verify', status: 'success' }],
|
||||
},
|
||||
{
|
||||
commit: 'bbb',
|
||||
number: 2,
|
||||
status: 'failure',
|
||||
steps: [{ name: 'gate-verify', status: 'failure' }],
|
||||
},
|
||||
];
|
||||
assert.deepEqual(assessProviderEvidence('aaa', pipelines), {
|
||||
state: 'terminal-success',
|
||||
detail: 'pipeline and gate-verify step succeeded',
|
||||
});
|
||||
assert.equal(assessProviderEvidence('bbb', pipelines).state, 'terminal-failure');
|
||||
assert.equal(assessProviderEvidence('ccc', pipelines).state, 'absent');
|
||||
});
|
||||
|
||||
test('duplicate gate-verify steps cannot establish provider success', () => {
|
||||
const result = assessProviderEvidence('aaa', [
|
||||
{
|
||||
commit: 'aaa',
|
||||
number: 7,
|
||||
status: 'success',
|
||||
steps: [
|
||||
{ name: 'gate-verify', status: 'success' },
|
||||
{ name: 'gate-verify', status: 'failure' },
|
||||
],
|
||||
},
|
||||
]);
|
||||
assert.equal(result.state, 'terminal-failure');
|
||||
assert.match(result.detail, /ambiguous.*gate-verify/i);
|
||||
});
|
||||
|
||||
test('a malformed single provider record cannot establish success', () => {
|
||||
assert.equal(
|
||||
assessProviderEvidence('aaa', [
|
||||
{ commit: 'aaa', status: 'success', steps: [{ name: 'gate-verify', status: 'success' }] },
|
||||
]).state,
|
||||
'terminal-failure',
|
||||
);
|
||||
assert.equal(
|
||||
assessProviderEvidence('bbb', [
|
||||
{
|
||||
commit: 'bbb',
|
||||
number: 1,
|
||||
status: 'surprising',
|
||||
steps: [{ name: 'gate-verify', status: 'success' }],
|
||||
},
|
||||
]).state,
|
||||
'terminal-failure',
|
||||
);
|
||||
});
|
||||
|
||||
test('provider evidence selects the highest numbered rerun deterministically', () => {
|
||||
const failedThenSucceeded = [
|
||||
{
|
||||
commit: 'aaa',
|
||||
number: 10,
|
||||
status: 'failure',
|
||||
steps: [{ name: 'gate-verify', status: 'failure' }],
|
||||
},
|
||||
{
|
||||
commit: 'aaa',
|
||||
number: 11,
|
||||
status: 'success',
|
||||
steps: [{ name: 'gate-verify', status: 'success' }],
|
||||
},
|
||||
];
|
||||
const succeededThenFailed = [
|
||||
{
|
||||
commit: 'bbb',
|
||||
number: 21,
|
||||
status: 'success',
|
||||
steps: [{ name: 'gate-verify', status: 'success' }],
|
||||
},
|
||||
{
|
||||
commit: 'bbb',
|
||||
number: 22,
|
||||
status: 'failure',
|
||||
steps: [{ name: 'gate-verify', status: 'failure' }],
|
||||
},
|
||||
];
|
||||
assert.equal(assessProviderEvidence('aaa', failedThenSucceeded).state, 'terminal-success');
|
||||
assert.equal(assessProviderEvidence('bbb', succeededThenFailed).state, 'terminal-failure');
|
||||
assert.equal(
|
||||
assessProviderEvidence('ccc', [
|
||||
{ commit: 'ccc', status: 'success', steps: [{ name: 'gate-verify', status: 'success' }] },
|
||||
{ commit: 'ccc', status: 'failure', steps: [{ name: 'gate-verify', status: 'failure' }] },
|
||||
]).state,
|
||||
'terminal-failure',
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,866 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { constants } from 'node:fs';
|
||||
import { createHash } from 'node:crypto';
|
||||
import {
|
||||
access,
|
||||
chmod,
|
||||
copyFile,
|
||||
lstat,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
open,
|
||||
readFile,
|
||||
readdir,
|
||||
readlink,
|
||||
rm,
|
||||
symlink,
|
||||
writeFile,
|
||||
} from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
|
||||
import { verifyHistory } from './gate-history.mjs';
|
||||
|
||||
const COPY_SKIP = new Set(['.git', '.mosaic-test-work', '.next', '.turbo', 'coverage', 'dist']);
|
||||
|
||||
function parseArgs(argv) {
|
||||
const options = {
|
||||
root: process.cwd(),
|
||||
manifest: 'gates/gates.manifest.json',
|
||||
skipHistory: false,
|
||||
structureOnly: false,
|
||||
};
|
||||
for (let index = 0; index < argv.length; index += 1) {
|
||||
const value = argv[index];
|
||||
if (value === '--root') options.root = path.resolve(argv[++index]);
|
||||
else if (value === '--manifest') options.manifest = argv[++index];
|
||||
else if (value === '--skip-history') options.skipHistory = true;
|
||||
else if (value === '--structure-only') options.structureOnly = true;
|
||||
else throw new Error(`unknown option: ${value}`);
|
||||
}
|
||||
return options;
|
||||
}
|
||||
|
||||
function runInvocation(root, invocation, extraEnvironment = {}) {
|
||||
if (!Array.isArray(invocation) || invocation.length === 0) {
|
||||
return { status: null, stdout: '', stderr: 'missing invocation' };
|
||||
}
|
||||
return spawnSync(invocation[0], invocation.slice(1), {
|
||||
cwd: root,
|
||||
encoding: 'utf8',
|
||||
env: { ...process.env, GATE_VERIFY: '1', ...extraEnvironment },
|
||||
timeout: 300_000,
|
||||
});
|
||||
}
|
||||
|
||||
async function sandboxPath(root, relativePath, label) {
|
||||
if (typeof relativePath !== 'string' || path.isAbsolute(relativePath)) {
|
||||
throw new Error(`${label}: path escapes sandbox (${String(relativePath)})`);
|
||||
}
|
||||
const resolvedRoot = path.resolve(root);
|
||||
const target = path.resolve(resolvedRoot, relativePath);
|
||||
if (target !== resolvedRoot && !target.startsWith(`${resolvedRoot}${path.sep}`)) {
|
||||
throw new Error(`${label}: path escapes sandbox (${relativePath})`);
|
||||
}
|
||||
const parts = path.relative(resolvedRoot, path.dirname(target)).split(path.sep).filter(Boolean);
|
||||
let current = resolvedRoot;
|
||||
for (const part of parts) {
|
||||
current = path.join(current, part);
|
||||
try {
|
||||
if ((await lstat(current)).isSymbolicLink()) {
|
||||
throw new Error(`${label}: path crosses symbolic link (${relativePath})`);
|
||||
}
|
||||
} catch (error) {
|
||||
if (error.code === 'ENOENT') break;
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
return target;
|
||||
}
|
||||
|
||||
function expand(value, root) {
|
||||
return String(value)
|
||||
.replaceAll('${ROOT}', root)
|
||||
.replaceAll('${HOME}', process.env.HOME ?? '')
|
||||
.replaceAll('${PATH}', process.env.PATH ?? '');
|
||||
}
|
||||
|
||||
function normalizedJson(value) {
|
||||
if (Array.isArray(value)) return value.map(normalizedJson);
|
||||
if (value && typeof value === 'object') {
|
||||
return Object.fromEntries(
|
||||
Object.keys(value)
|
||||
.sort()
|
||||
.map((key) => [key, normalizedJson(value[key])]),
|
||||
);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function structuredValuesEqual(left, right) {
|
||||
return JSON.stringify(normalizedJson(left)) === JSON.stringify(normalizedJson(right));
|
||||
}
|
||||
|
||||
function outcomeMatches(outcome, result) {
|
||||
const combined = `${result.stdout ?? ''}\n${result.stderr ?? ''}`;
|
||||
return (
|
||||
result.status === outcome?.exitCode &&
|
||||
(!outcome?.outputPattern || new RegExp(outcome.outputPattern, 'm').test(combined)) &&
|
||||
(!outcome?.notOutputPattern || !new RegExp(outcome.notOutputPattern, 'm').test(combined))
|
||||
);
|
||||
}
|
||||
|
||||
function resultMatches(gateCase, result) {
|
||||
const combined = `${result.stdout ?? ''}\n${result.stderr ?? ''}`;
|
||||
return (
|
||||
outcomeMatches(gateCase.actual, result) &&
|
||||
(!gateCase.reasonPattern || new RegExp(gateCase.reasonPattern, 'm').test(combined))
|
||||
);
|
||||
}
|
||||
|
||||
async function safeSandboxWrite(root, relativePath, contents, label) {
|
||||
const target = await sandboxPath(root, relativePath, label);
|
||||
await mkdir(path.dirname(target), { recursive: true });
|
||||
try {
|
||||
if ((await lstat(target)).isSymbolicLink()) {
|
||||
throw new Error(`${label}: target is a symbolic link (${relativePath})`);
|
||||
}
|
||||
} catch (error) {
|
||||
if (error.code !== 'ENOENT') throw error;
|
||||
}
|
||||
let handle;
|
||||
try {
|
||||
handle = await open(
|
||||
target,
|
||||
constants.O_WRONLY | constants.O_CREAT | constants.O_TRUNC | constants.O_NOFOLLOW,
|
||||
0o666,
|
||||
);
|
||||
await handle.writeFile(contents);
|
||||
} catch (error) {
|
||||
if (error.code === 'ELOOP') {
|
||||
throw new Error(`${label}: target is a symbolic link (${relativePath})`);
|
||||
}
|
||||
throw error;
|
||||
} finally {
|
||||
await handle?.close();
|
||||
}
|
||||
return target;
|
||||
}
|
||||
|
||||
async function applyFixture(root, fixture = {}) {
|
||||
for (const entry of fixture.writeFiles ?? []) {
|
||||
const target = await safeSandboxWrite(root, entry.path, entry.content, 'fixture write');
|
||||
if (entry.mode !== undefined) await chmod(target, entry.mode);
|
||||
}
|
||||
for (const entry of fixture.replaceFiles ?? []) {
|
||||
const target = await sandboxPath(root, entry.path, 'fixture replace');
|
||||
if ((await lstat(target)).isSymbolicLink()) {
|
||||
throw new Error(`fixture replace: target is a symbolic link (${entry.path})`);
|
||||
}
|
||||
const source = await readFile(target, 'utf8');
|
||||
const occurrences = source.split(entry.find).length - 1;
|
||||
if (occurrences !== 1) {
|
||||
throw new Error(
|
||||
`fixture replace: stale or ambiguous match for ${entry.path} (${occurrences} matches)`,
|
||||
);
|
||||
}
|
||||
await safeSandboxWrite(
|
||||
root,
|
||||
entry.path,
|
||||
source.replace(entry.find, entry.replace),
|
||||
'fixture replace',
|
||||
);
|
||||
}
|
||||
for (const relativePath of fixture.removePaths ?? []) {
|
||||
await rm(await sandboxPath(root, relativePath, 'fixture remove'), {
|
||||
recursive: true,
|
||||
force: true,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
async function copySandbox(root, destination, selectedPaths) {
|
||||
if (!selectedPaths?.length) {
|
||||
await copyTree(root, destination);
|
||||
return;
|
||||
}
|
||||
await mkdir(destination, { recursive: true });
|
||||
for (const relativePath of selectedPaths) {
|
||||
await copyTree(
|
||||
await sandboxPath(root, relativePath, 'sandbox copy source'),
|
||||
await sandboxPath(destination, relativePath, 'sandbox copy destination'),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async function runCase(root, gate, gateCase) {
|
||||
let caseRoot = root;
|
||||
if (gateCase.fixture) {
|
||||
caseRoot = await mkdtemp(path.join(path.dirname(root), `.gate-case-${gate.id}-`));
|
||||
await copySandbox(root, caseRoot, gateCase.fixture.copyPaths);
|
||||
await applyFixture(caseRoot, gateCase.fixture);
|
||||
}
|
||||
try {
|
||||
const environment = Object.fromEntries(
|
||||
Object.entries(gateCase.environment ?? {}).map(([key, value]) => [
|
||||
key,
|
||||
expand(value, caseRoot),
|
||||
]),
|
||||
);
|
||||
return runInvocation(caseRoot, gateCase.invocation ?? gate.invocation, environment);
|
||||
} finally {
|
||||
if (caseRoot !== root) await rm(caseRoot, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
async function copyTree(source, destination) {
|
||||
const stats = await lstat(source);
|
||||
if (stats.isSymbolicLink()) {
|
||||
await symlink(await readlink(source), destination);
|
||||
return;
|
||||
}
|
||||
if (stats.isFile()) {
|
||||
await mkdir(path.dirname(destination), { recursive: true });
|
||||
await copyFile(source, destination);
|
||||
await chmod(destination, stats.mode);
|
||||
return;
|
||||
}
|
||||
if (!stats.isDirectory()) return;
|
||||
await mkdir(destination, { recursive: true });
|
||||
for (const child of await readdir(source, { withFileTypes: true })) {
|
||||
if (COPY_SKIP.has(child.name)) continue;
|
||||
const childSource = path.join(source, child.name);
|
||||
const childDestination = path.join(destination, child.name);
|
||||
if (child.name === 'node_modules') {
|
||||
await symlink(childSource, childDestination, 'dir');
|
||||
continue;
|
||||
}
|
||||
await copyTree(childSource, childDestination);
|
||||
}
|
||||
}
|
||||
|
||||
async function executableFiles(root, relativeRoot) {
|
||||
const base = await sandboxPath(root, relativeRoot, 'gate root');
|
||||
const found = [];
|
||||
async function walk(current) {
|
||||
for (const child of await readdir(current, { withFileTypes: true })) {
|
||||
const target = path.join(current, child.name);
|
||||
if (child.isDirectory()) await walk(target);
|
||||
else if (child.isFile()) {
|
||||
try {
|
||||
await access(target, constants.X_OK);
|
||||
found.push(path.relative(root, target).split(path.sep).join('/'));
|
||||
} catch {
|
||||
// Non-executable files are not gates for discovery purposes.
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
try {
|
||||
await walk(base);
|
||||
} catch (error) {
|
||||
if (error.code !== 'ENOENT') throw error;
|
||||
}
|
||||
return found;
|
||||
}
|
||||
|
||||
function rejectUnknownKeys(value, allowed, label, failures) {
|
||||
if (!value || typeof value !== 'object' || Array.isArray(value)) {
|
||||
failures.push(`${label}: expected an object`);
|
||||
return;
|
||||
}
|
||||
for (const key of Object.keys(value)) {
|
||||
if (!allowed.has(key)) failures.push(`${label}: unknown field ${key}`);
|
||||
}
|
||||
}
|
||||
|
||||
function rejectDuplicateIds(values, label, failures) {
|
||||
const seen = new Set();
|
||||
for (const value of values ?? []) {
|
||||
if (typeof value?.id !== 'string' || value.id.length === 0) {
|
||||
failures.push(`${label}: missing stable id`);
|
||||
} else if (seen.has(value.id)) {
|
||||
failures.push(`duplicate ${label} id ${value.id}`);
|
||||
} else {
|
||||
seen.add(value.id);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function validateClosedSchema(manifest, failures) {
|
||||
if (manifest.schemaVersion !== 1)
|
||||
failures.push(`unsupported schemaVersion ${String(manifest.schemaVersion)}`);
|
||||
rejectUnknownKeys(
|
||||
manifest,
|
||||
new Set([
|
||||
'schemaVersion',
|
||||
'activationCommit',
|
||||
'gateRoots',
|
||||
'governingClaimFiles',
|
||||
'coverageBoundary',
|
||||
'criteria',
|
||||
'proseClaims',
|
||||
'compatibilityScenarios',
|
||||
'mergeAssertions',
|
||||
'gates',
|
||||
]),
|
||||
'manifest',
|
||||
failures,
|
||||
);
|
||||
rejectDuplicateIds(manifest.criteria, 'criterion', failures);
|
||||
for (const criterion of manifest.criteria ?? []) {
|
||||
rejectUnknownKeys(
|
||||
criterion,
|
||||
new Set([
|
||||
'id',
|
||||
'originalText',
|
||||
'currentText',
|
||||
'claimType',
|
||||
'source',
|
||||
'meaningChanges',
|
||||
'caseRefs',
|
||||
]),
|
||||
`criterion ${criterion.id}`,
|
||||
failures,
|
||||
);
|
||||
if (!Array.isArray(criterion.caseRefs) || criterion.caseRefs.length === 0) {
|
||||
failures.push(`${criterion.id}: no declared exercising cases`);
|
||||
} else {
|
||||
if (criterion.caseRefs.some((caseRef) => typeof caseRef !== 'string' || !caseRef)) {
|
||||
failures.push(`${criterion.id}: declared exercising cases must be non-empty strings`);
|
||||
}
|
||||
if (new Set(criterion.caseRefs).size !== criterion.caseRefs.length) {
|
||||
failures.push(`${criterion.id}: duplicate declared exercising case`);
|
||||
}
|
||||
}
|
||||
}
|
||||
rejectDuplicateIds(manifest.proseClaims, 'prose claim', failures);
|
||||
for (const claim of manifest.proseClaims ?? []) {
|
||||
rejectUnknownKeys(
|
||||
claim,
|
||||
new Set(['id', 'criterionId', 'caseRef']),
|
||||
`prose claim ${claim.id}`,
|
||||
failures,
|
||||
);
|
||||
if (typeof claim.caseRef !== 'string' || claim.caseRef.length === 0) {
|
||||
failures.push(`GATE-CLAIM:${claim.id} has no declared exercising case`);
|
||||
}
|
||||
}
|
||||
rejectDuplicateIds(manifest.compatibilityScenarios, 'compatibility scenario', failures);
|
||||
for (const scenario of manifest.compatibilityScenarios ?? []) {
|
||||
rejectUnknownKeys(
|
||||
scenario,
|
||||
new Set([
|
||||
'id',
|
||||
'construction',
|
||||
'caseRefs',
|
||||
'invocation',
|
||||
'expected',
|
||||
'environment',
|
||||
'fixture',
|
||||
]),
|
||||
`compatibility scenario ${scenario.id}`,
|
||||
failures,
|
||||
);
|
||||
if (!Array.isArray(scenario.caseRefs) || scenario.caseRefs.length === 0) {
|
||||
failures.push(`${scenario.id}: compatibility construction has no referenced conditions`);
|
||||
}
|
||||
if (!Array.isArray(scenario.invocation) || scenario.invocation.length === 0) {
|
||||
failures.push(`${scenario.id}: compatibility construction invocation is missing`);
|
||||
}
|
||||
if (typeof scenario.expected?.exitCode !== 'number') {
|
||||
failures.push(`${scenario.id}: compatibility construction exact expected exit is missing`);
|
||||
}
|
||||
}
|
||||
rejectDuplicateIds(manifest.gates, 'gate', failures);
|
||||
for (const gate of manifest.gates ?? []) {
|
||||
rejectUnknownKeys(
|
||||
gate,
|
||||
new Set([
|
||||
'id',
|
||||
'source',
|
||||
'invocation',
|
||||
'deployment',
|
||||
'inertMutation',
|
||||
'cases',
|
||||
'discoveryAliases',
|
||||
]),
|
||||
`gate ${gate.id}`,
|
||||
failures,
|
||||
);
|
||||
rejectDuplicateIds(gate.cases, `case in gate ${gate.id}`, failures);
|
||||
if (!Array.isArray(gate.invocation) || gate.invocation.length === 0) {
|
||||
failures.push(`${gate.id}: exact invocation is missing`);
|
||||
}
|
||||
if (!['none', 'file'].includes(gate.deployment?.kind)) {
|
||||
failures.push(`${gate.id}: unsupported deployment kind ${String(gate.deployment?.kind)}`);
|
||||
}
|
||||
for (const gateCase of gate.cases ?? []) {
|
||||
rejectUnknownKeys(
|
||||
gateCase,
|
||||
new Set([
|
||||
'id',
|
||||
'criterionIds',
|
||||
'mustFail',
|
||||
'invocation',
|
||||
'required',
|
||||
'actual',
|
||||
'reasonPattern',
|
||||
'environment',
|
||||
'fixture',
|
||||
'defect',
|
||||
]),
|
||||
`${gate.id}/${gateCase.id}`,
|
||||
failures,
|
||||
);
|
||||
if (
|
||||
typeof gateCase.required?.exitCode !== 'number' ||
|
||||
typeof gateCase.actual?.exitCode !== 'number'
|
||||
) {
|
||||
failures.push(
|
||||
`${gate.id}/${gateCase.id}: required and actual exact exit codes are mandatory`,
|
||||
);
|
||||
}
|
||||
if (
|
||||
gateCase.invocation &&
|
||||
(!Array.isArray(gateCase.invocation) || gateCase.invocation.length === 0)
|
||||
) {
|
||||
failures.push(`${gate.id}/${gateCase.id}: case invocation must be non-empty`);
|
||||
}
|
||||
if (gateCase.mustFail === true && !gateCase.reasonPattern?.trim()) {
|
||||
failures.push(
|
||||
`${gate.id}/${gateCase.id}: must-fail case requires a non-empty reasonPattern`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function validateStructure(manifest, failures) {
|
||||
validateClosedSchema(manifest, failures);
|
||||
const criteria = new Map((manifest.criteria ?? []).map((criterion) => [criterion.id, criterion]));
|
||||
const boundCriteria = new Set();
|
||||
const negativeBoundCriteria = new Set();
|
||||
const observedCaseRefs = new Map();
|
||||
|
||||
for (const gate of manifest.gates ?? []) {
|
||||
const negativeCases = (gate.cases ?? []).filter((gateCase) => gateCase.mustFail === true);
|
||||
if (negativeCases.length === 0) failures.push(`${gate.id}: no negative control`);
|
||||
if (!gate.deployment?.kind) failures.push(`${gate.id}: deployment identity is not declared`);
|
||||
|
||||
for (const gateCase of gate.cases ?? []) {
|
||||
for (const criterionId of gateCase.criterionIds ?? []) {
|
||||
if (!criteria.has(criterionId)) {
|
||||
failures.push(`${gate.id}/${gateCase.id}: unknown criterion ${criterionId}`);
|
||||
}
|
||||
boundCriteria.add(criterionId);
|
||||
const caseRef = `${gate.id}/${gateCase.id}`;
|
||||
if (!observedCaseRefs.has(criterionId)) observedCaseRefs.set(criterionId, new Set());
|
||||
observedCaseRefs.get(criterionId).add(caseRef);
|
||||
if (gateCase.mustFail === true) negativeBoundCriteria.add(criterionId);
|
||||
}
|
||||
if (!structuredValuesEqual(gateCase.required, gateCase.actual) && !gateCase.defect?.owner) {
|
||||
failures.push(`${gate.id}/${gateCase.id}: behavior delta requires a tracked owner`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (const claim of manifest.proseClaims ?? []) {
|
||||
if (!criteria.has(claim.criterionId)) {
|
||||
failures.push(`GATE-CLAIM:${claim.id} references unknown criterion ${claim.criterionId}`);
|
||||
continue;
|
||||
}
|
||||
const found = findGateCase(manifest, claim.caseRef ?? '');
|
||||
if (!found) {
|
||||
failures.push(`GATE-CLAIM:${claim.id} references missing exercising case ${claim.caseRef}`);
|
||||
} else if (
|
||||
found.gateCase.mustFail !== true ||
|
||||
!found.gateCase.criterionIds?.includes(claim.criterionId)
|
||||
) {
|
||||
failures.push(
|
||||
`GATE-CLAIM:${claim.id} exercising case ${claim.caseRef} does not exercise criterion ${claim.criterionId}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
for (const criterion of criteria.values()) {
|
||||
if (!boundCriteria.has(criterion.id)) failures.push(`${criterion.id}: no bound case`);
|
||||
else if (!negativeBoundCriteria.has(criterion.id)) {
|
||||
failures.push(`${criterion.id}: no must-fail case exercises this criterion`);
|
||||
}
|
||||
const declaredRefs = new Set(criterion.caseRefs ?? []);
|
||||
const actualRefs = observedCaseRefs.get(criterion.id) ?? new Set();
|
||||
for (const caseRef of declaredRefs) {
|
||||
const found = findGateCase(manifest, caseRef);
|
||||
if (!found) failures.push(`${criterion.id}: declared exercising case ${caseRef} is missing`);
|
||||
else if (!actualRefs.has(caseRef)) {
|
||||
failures.push(`${criterion.id}: declared exercising case ${caseRef} is not bound`);
|
||||
}
|
||||
}
|
||||
for (const caseRef of actualRefs) {
|
||||
if (!declaredRefs.has(caseRef)) {
|
||||
failures.push(`${criterion.id}: bound to undeclared exercising case ${caseRef}`);
|
||||
}
|
||||
}
|
||||
if (
|
||||
criterion.originalText !== criterion.currentText &&
|
||||
(!Array.isArray(criterion.meaningChanges) || criterion.meaningChanges.length === 0)
|
||||
) {
|
||||
failures.push(`${criterion.id}: missing meaning-change provenance`);
|
||||
}
|
||||
}
|
||||
|
||||
const byConstruction = new Map();
|
||||
for (const scenario of manifest.compatibilityScenarios ?? []) {
|
||||
const previous = byConstruction.get(scenario.construction);
|
||||
if (previous && !structuredValuesEqual(previous.expected, scenario.expected)) {
|
||||
failures.push(`${previous.id} and ${scenario.id}: modeled compatibility conflict`);
|
||||
} else {
|
||||
byConstruction.set(scenario.construction, scenario);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function validateClaims(root, manifest, failures) {
|
||||
const registered = new Set((manifest.proseClaims ?? []).map((claim) => claim.id));
|
||||
const observed = new Set();
|
||||
for (const relativeFile of manifest.governingClaimFiles ?? []) {
|
||||
const contents = await readFile(
|
||||
await sandboxPath(root, relativeFile, 'governing claim file'),
|
||||
'utf8',
|
||||
);
|
||||
for (const match of contents.matchAll(/GATE-CLAIM:([A-Z0-9][A-Z0-9-]*)/g)) {
|
||||
observed.add(match[1]);
|
||||
if (!registered.has(match[1])) {
|
||||
failures.push(`GATE-CLAIM:${match[1]} is unbound in ${relativeFile}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
for (const claimId of registered) {
|
||||
if (!observed.has(claimId)) failures.push(`GATE-CLAIM:${claimId} marker is missing`);
|
||||
}
|
||||
}
|
||||
|
||||
async function validateDiscovery(root, manifest, failures) {
|
||||
const registered = new Set(
|
||||
(manifest.gates ?? []).flatMap((gate) => [gate.source, ...(gate.discoveryAliases ?? [])]),
|
||||
);
|
||||
for (const gateRoot of manifest.gateRoots ?? []) {
|
||||
for (const executable of await executableFiles(root, gateRoot)) {
|
||||
if (!registered.has(executable)) failures.push(`unregistered gate: ${executable}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function deploymentFilesEqual(sourcePath, deployedPath) {
|
||||
const [source, deployed] = await Promise.all([readFile(sourcePath), readFile(deployedPath)]);
|
||||
return source.equals(deployed);
|
||||
}
|
||||
|
||||
async function validateDeployment(root, gate, failures, observations) {
|
||||
if (gate.deployment?.kind !== 'file') return;
|
||||
const sourcePath = await sandboxPath(
|
||||
root,
|
||||
gate.deployment.source ?? gate.source,
|
||||
`${gate.id} deployment source`,
|
||||
);
|
||||
const deployedPath = path.isAbsolute(expand(gate.deployment.path, root))
|
||||
? expand(gate.deployment.path, root)
|
||||
: path.resolve(root, expand(gate.deployment.path, root));
|
||||
const source = await readFile(sourcePath);
|
||||
let deployed;
|
||||
try {
|
||||
deployed = await readFile(deployedPath);
|
||||
} catch (error) {
|
||||
if (error.code !== 'ENOENT') throw error;
|
||||
const observedHash = createHash('sha256').update(source).digest('hex');
|
||||
if (
|
||||
!gate.deployment.unavailableOwner ||
|
||||
!gate.deployment.observedSha256 ||
|
||||
gate.deployment.observedSha256 !== observedHash
|
||||
) {
|
||||
failures.push(
|
||||
`${gate.id}: deployed counterpart is unavailable and no current tracked observation matches source`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
observations.push(
|
||||
`DEPLOYED IDENTITY UNAVAILABLE (owner: ${gate.deployment.unavailableOwner}) ${gate.id}: ${deployedPath} is outside this runner; source matches pinned observation ${observedHash}, live equality is not inferred`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (!(await deploymentFilesEqual(sourcePath, deployedPath))) {
|
||||
failures.push(`${gate.id}: source versus deployed copy drift`);
|
||||
return;
|
||||
}
|
||||
|
||||
const controlRoot = await mkdtemp(
|
||||
path.join(path.dirname(root), `.gate-deployment-control-${gate.id}-`),
|
||||
);
|
||||
try {
|
||||
const alteredPath = path.join(controlRoot, 'deployed-copy');
|
||||
await writeFile(
|
||||
alteredPath,
|
||||
Buffer.concat([deployed, Buffer.from('\n# gate deployment drift control\n')]),
|
||||
);
|
||||
if (await deploymentFilesEqual(sourcePath, alteredPath)) {
|
||||
failures.push(`${gate.id}: deployed-copy drift negative control was ineffective`);
|
||||
} else {
|
||||
observations.push(`DEPLOYMENT-NEGATIVE-CONTROL ${gate.id}: observed red`);
|
||||
}
|
||||
} finally {
|
||||
await rm(controlRoot, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
async function validateMutation(root, gate, failures, observations) {
|
||||
const mutation = gate.inertMutation;
|
||||
if (
|
||||
!mutation?.file ||
|
||||
typeof mutation.find !== 'string' ||
|
||||
typeof mutation.replace !== 'string' ||
|
||||
typeof mutation.expected?.exitCode !== 'number'
|
||||
) {
|
||||
failures.push(`${gate.id}: declared inert mutation is incomplete`);
|
||||
return;
|
||||
}
|
||||
const mutationPath = await sandboxPath(root, mutation.file, `${gate.id} mutation source`);
|
||||
let source;
|
||||
try {
|
||||
source = await readFile(mutationPath, 'utf8');
|
||||
} catch (error) {
|
||||
failures.push(`${gate.id}: mutation source unreadable: ${error.message}`);
|
||||
return;
|
||||
}
|
||||
const occurrences = source.split(mutation.find).length - 1;
|
||||
if (occurrences !== 1) {
|
||||
failures.push(
|
||||
`${gate.id}: declared inert mutation is stale or ambiguous (${occurrences} matches)`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
const targetCase = mutation.caseId
|
||||
? (gate.cases ?? []).find((gateCase) => gateCase.id === mutation.caseId)
|
||||
: (gate.cases ?? []).find((gateCase) => gateCase.mustFail === true);
|
||||
if (!targetCase) {
|
||||
failures.push(
|
||||
mutation.caseId
|
||||
? `${gate.id}: declared mutation case ${mutation.caseId} was not found`
|
||||
: `${gate.id}: declared mutation has no must-fail case`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
const sandbox = await mkdtemp(path.join(path.dirname(root), `.gate-sandbox-${gate.id}-`));
|
||||
try {
|
||||
await copySandbox(root, sandbox, mutation.sandboxFiles);
|
||||
await safeSandboxWrite(
|
||||
sandbox,
|
||||
mutation.file,
|
||||
source.replace(mutation.find, mutation.replace),
|
||||
`${gate.id} sandbox mutation write`,
|
||||
);
|
||||
await applyFixture(sandbox, targetCase.fixture);
|
||||
const environment = Object.fromEntries(
|
||||
Object.entries(targetCase.environment ?? {}).map(([key, value]) => [
|
||||
key,
|
||||
expand(value, sandbox),
|
||||
]),
|
||||
);
|
||||
const result = runInvocation(sandbox, targetCase.invocation ?? gate.invocation, environment);
|
||||
if (result.error || result.signal) {
|
||||
failures.push(
|
||||
`${gate.id}: mutation execution crashed${result.signal ? ` with ${result.signal}` : ''}${result.error ? `: ${result.error.message}` : ''}`,
|
||||
);
|
||||
} else if (!outcomeMatches(mutation.expected, result)) {
|
||||
failures.push(
|
||||
`${gate.id}: mutation produced unexpected outcome ${String(result.status)}; expected ${JSON.stringify(mutation.expected)}`,
|
||||
);
|
||||
} else if (resultMatches(targetCase, result)) {
|
||||
failures.push(`${gate.id}: declared inert mutation was ineffective`);
|
||||
} else {
|
||||
observations.push(`META-NEGATIVE-CONTROL ${gate.id}: observed red`);
|
||||
}
|
||||
} finally {
|
||||
await rm(sandbox, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
function findGateCase(manifest, caseRef) {
|
||||
if (typeof caseRef !== 'string') return undefined;
|
||||
const separator = caseRef.indexOf('/');
|
||||
if (separator < 1) return undefined;
|
||||
const gateId = caseRef.slice(0, separator);
|
||||
const caseId = caseRef.slice(separator + 1);
|
||||
const gate = (manifest.gates ?? []).find((candidate) => candidate.id === gateId);
|
||||
const gateCase = (gate?.cases ?? []).find((candidate) => candidate.id === caseId);
|
||||
return gate && gateCase ? { gate, gateCase } : undefined;
|
||||
}
|
||||
|
||||
async function runCompatibilityScenario(root, manifest, scenario, failures, observations) {
|
||||
const referenced = [];
|
||||
for (const caseRef of scenario.caseRefs ?? []) {
|
||||
const found = findGateCase(manifest, caseRef);
|
||||
if (!found) {
|
||||
failures.push(
|
||||
`${scenario.id}: compatibility construction references missing case ${caseRef}`,
|
||||
);
|
||||
} else {
|
||||
referenced.push({ caseRef, ...found });
|
||||
}
|
||||
}
|
||||
if (referenced.length !== (scenario.caseRefs ?? []).length) return;
|
||||
|
||||
const sandbox = await mkdtemp(path.join(path.dirname(root), `.gate-compat-${scenario.id}-`));
|
||||
try {
|
||||
await copySandbox(root, sandbox);
|
||||
const environment = {};
|
||||
const writeSignatures = new Map();
|
||||
const removedPaths = new Set();
|
||||
const fixtures = [...referenced.map(({ gateCase }) => gateCase.fixture), scenario.fixture];
|
||||
for (const fixture of fixtures.filter(Boolean)) {
|
||||
for (const entry of fixture.writeFiles ?? []) {
|
||||
const signature = JSON.stringify({ content: entry.content, mode: entry.mode });
|
||||
const previous = writeSignatures.get(entry.path);
|
||||
if (previous !== undefined && previous !== signature) {
|
||||
failures.push(`${scenario.id}: incompatible fixture writes for ${entry.path}`);
|
||||
return;
|
||||
}
|
||||
if (removedPaths.has(entry.path)) {
|
||||
failures.push(`${scenario.id}: fixture both writes and removes ${entry.path}`);
|
||||
return;
|
||||
}
|
||||
writeSignatures.set(entry.path, signature);
|
||||
}
|
||||
for (const removed of fixture.removePaths ?? []) {
|
||||
if (writeSignatures.has(removed)) {
|
||||
failures.push(`${scenario.id}: fixture both writes and removes ${removed}`);
|
||||
return;
|
||||
}
|
||||
removedPaths.add(removed);
|
||||
}
|
||||
await applyFixture(sandbox, fixture);
|
||||
}
|
||||
for (const source of [
|
||||
...referenced.map(({ gateCase }) => gateCase.environment),
|
||||
scenario.environment,
|
||||
]) {
|
||||
for (const [key, value] of Object.entries(source ?? {})) {
|
||||
if (environment[key] !== undefined && environment[key] !== value) {
|
||||
failures.push(`${scenario.id}: incompatible environment values for ${key}`);
|
||||
return;
|
||||
}
|
||||
environment[key] = value;
|
||||
}
|
||||
}
|
||||
const expandedEnvironment = Object.fromEntries(
|
||||
Object.entries(environment).map(([key, value]) => [key, expand(value, sandbox)]),
|
||||
);
|
||||
const result = runInvocation(sandbox, scenario.invocation, expandedEnvironment);
|
||||
if (result.error || result.signal || !outcomeMatches(scenario.expected, result)) {
|
||||
failures.push(
|
||||
`${scenario.id}: combined compatibility construction ${scenario.construction} observed ${String(result.status)}${result.signal ? ` signal ${result.signal}` : ''}${result.error ? ` error ${result.error.message}` : ''}; expected ${JSON.stringify(scenario.expected)}`,
|
||||
);
|
||||
} else {
|
||||
observations.push(`COMPATIBILITY ${scenario.id}: observed expected outcome`);
|
||||
}
|
||||
} finally {
|
||||
await rm(sandbox, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
export async function verifyRegistry(options) {
|
||||
const failures = [];
|
||||
const observations = [];
|
||||
const manifestPath = path.resolve(options.root, options.manifest);
|
||||
const manifest = JSON.parse(await readFile(manifestPath, 'utf8'));
|
||||
|
||||
validateStructure(manifest, failures);
|
||||
if (options.structureOnly) return { failures, manifest, observations };
|
||||
|
||||
async function collectPhaseFailure(label, action) {
|
||||
try {
|
||||
return await action();
|
||||
} catch (error) {
|
||||
failures.push(`${label}: ${error.message}`);
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
await collectPhaseFailure('governing claim validation failed', () =>
|
||||
validateClaims(options.root, manifest, failures),
|
||||
);
|
||||
await collectPhaseFailure('gate discovery failed', () =>
|
||||
validateDiscovery(options.root, manifest, failures),
|
||||
);
|
||||
|
||||
for (const gate of manifest.gates ?? []) {
|
||||
await collectPhaseFailure(`${gate.id}: deployment validation failed`, () =>
|
||||
validateDeployment(options.root, gate, failures, observations),
|
||||
);
|
||||
for (const gateCase of gate.cases ?? []) {
|
||||
const result = await collectPhaseFailure(
|
||||
`${gate.id}/${gateCase.id}: case execution failed`,
|
||||
() => runCase(options.root, gate, gateCase),
|
||||
);
|
||||
if (!result) continue;
|
||||
const combined = `${result.stdout ?? ''}\n${result.stderr ?? ''}`;
|
||||
try {
|
||||
if (!outcomeMatches(gateCase.actual, result)) {
|
||||
failures.push(
|
||||
`${gate.id}/${gateCase.id}: observed exit ${String(result.status)}${result.signal ? ` signal ${result.signal}` : ''}${result.error ? ` error ${result.error.message}` : ''} or output disagrees with registry actual ${JSON.stringify(gateCase.actual)}`,
|
||||
);
|
||||
}
|
||||
if (gateCase.reasonPattern && !new RegExp(gateCase.reasonPattern, 'm').test(combined)) {
|
||||
failures.push(`${gate.id}/${gateCase.id}: did not fail for its stated reason`);
|
||||
}
|
||||
} catch (error) {
|
||||
failures.push(`${gate.id}/${gateCase.id}: outcome validation failed: ${error.message}`);
|
||||
}
|
||||
if (!structuredValuesEqual(gateCase.required, gateCase.actual) && gateCase.defect?.owner) {
|
||||
observations.push(
|
||||
`DEFECT (owner: ${gateCase.defect.owner}) ${gate.id}/${gateCase.id}: required ${JSON.stringify(gateCase.required)}, actual ${JSON.stringify(gateCase.actual)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
await collectPhaseFailure(`${gate.id}: mutation validation failed`, () =>
|
||||
validateMutation(options.root, gate, failures, observations),
|
||||
);
|
||||
}
|
||||
|
||||
for (const scenario of manifest.compatibilityScenarios ?? []) {
|
||||
await collectPhaseFailure(`${scenario.id}: compatibility validation failed`, () =>
|
||||
runCompatibilityScenario(options.root, manifest, scenario, failures, observations),
|
||||
);
|
||||
}
|
||||
|
||||
return { failures, manifest, observations };
|
||||
}
|
||||
|
||||
async function main() {
|
||||
try {
|
||||
const options = parseArgs(process.argv.slice(2));
|
||||
const { failures, observations, manifest } = await verifyRegistry(options);
|
||||
if (!options.skipHistory && failures.length === 0) {
|
||||
const history = await verifyHistory({ root: options.root, manifest });
|
||||
failures.push(...history.failures);
|
||||
observations.push(...history.observations);
|
||||
}
|
||||
for (const observation of observations) process.stdout.write(`${observation}\n`);
|
||||
if (failures.length > 0) {
|
||||
for (const failure of failures) process.stderr.write(`GATE VERIFY FAILED: ${failure}\n`);
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
const defects = observations.filter((line) => line.startsWith('DEFECT ')).length;
|
||||
process.stdout.write(
|
||||
`registry observations matched; open behavior deltas: ${defects}; required-behavior conformance is not asserted while deltas remain\n`,
|
||||
);
|
||||
} catch (error) {
|
||||
process.stderr.write(`GATE VERIFY FAILED: ${error.message}\n`);
|
||||
process.exitCode = 1;
|
||||
}
|
||||
}
|
||||
|
||||
if (import.meta.url === `file://${process.argv[1]}`) await main();
|
||||
@@ -0,0 +1,568 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { chmod, copyFile, mkdir, readFile, rm, symlink, writeFile } from 'node:fs/promises';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import test from 'node:test';
|
||||
|
||||
const verifier = path.join(process.cwd(), 'scripts', 'gate-verify.mjs');
|
||||
const fixtureBase = path.join(process.cwd(), '.mosaic-test-work', `gate-verify-${process.pid}`);
|
||||
|
||||
async function fixture(name = 'case') {
|
||||
const root = path.join(fixtureBase, name);
|
||||
await rm(root, { recursive: true, force: true });
|
||||
await mkdir(path.join(root, 'gates'), { recursive: true });
|
||||
return root;
|
||||
}
|
||||
|
||||
function baseManifest() {
|
||||
return {
|
||||
schemaVersion: 1,
|
||||
activationCommit: null,
|
||||
gateRoots: ['gates'],
|
||||
governingClaimFiles: [],
|
||||
coverageBoundary: { included: ['meta fixture'], excluded: [], trackedBy: 'RM-54' },
|
||||
criteria: [
|
||||
{
|
||||
id: 'META-CRIT-1',
|
||||
originalText: 'The fixture rejects its bad input.',
|
||||
currentText: 'The fixture rejects its bad input.',
|
||||
claimType: 'integrity',
|
||||
source: 'fixture',
|
||||
meaningChanges: [],
|
||||
caseRefs: ['meta-fixture/rejects-bad-input'],
|
||||
},
|
||||
],
|
||||
compatibilityScenarios: [],
|
||||
proseClaims: [],
|
||||
gates: [
|
||||
{
|
||||
id: 'meta-fixture',
|
||||
source: 'gates/meta-fixture.sh',
|
||||
invocation: ['gates/meta-fixture.sh'],
|
||||
deployment: { kind: 'none', reason: 'test fixture only' },
|
||||
inertMutation: {
|
||||
file: 'gates/meta-fixture.sh',
|
||||
find: 'exit 7',
|
||||
replace: 'exit 0',
|
||||
expected: { exitCode: 0 },
|
||||
},
|
||||
cases: [
|
||||
{
|
||||
id: 'rejects-bad-input',
|
||||
criterionIds: ['META-CRIT-1'],
|
||||
mustFail: true,
|
||||
invocation: ['gates/meta-fixture.sh'],
|
||||
required: { exitCode: 7 },
|
||||
actual: { exitCode: 7 },
|
||||
reasonPattern: 'META_REJECT',
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
async function writeGate(root, contents = '#!/bin/sh\necho META_REJECT >&2\nexit 7\n') {
|
||||
const target = path.join(root, 'gates', 'meta-fixture.sh');
|
||||
await writeFile(target, contents);
|
||||
await chmod(target, 0o755);
|
||||
}
|
||||
|
||||
async function writeManifest(root, manifest) {
|
||||
await writeFile(path.join(root, 'gates', 'gates.manifest.json'), `${JSON.stringify(manifest)}\n`);
|
||||
}
|
||||
|
||||
function verify(root, extraArgs = []) {
|
||||
return spawnSync(
|
||||
process.execPath,
|
||||
[
|
||||
verifier,
|
||||
'--root',
|
||||
root,
|
||||
'--manifest',
|
||||
'gates/gates.manifest.json',
|
||||
'--skip-history',
|
||||
...extraArgs,
|
||||
],
|
||||
{ cwd: root, encoding: 'utf8', env: { ...process.env, HOME: os.homedir() } },
|
||||
);
|
||||
}
|
||||
|
||||
function output(result) {
|
||||
return `${result.stdout}\n${result.stderr}`;
|
||||
}
|
||||
|
||||
test.after(async () => {
|
||||
await rm(fixtureBase, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('an externally inerted failure branch makes verification nonzero and names the gate', async () => {
|
||||
const root = await fixture('external-inert');
|
||||
await writeGate(root, '#!/bin/sh\nexit 0\n');
|
||||
await writeManifest(root, baseManifest());
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture/);
|
||||
});
|
||||
|
||||
test('the verifier applies the declared inert mutation and observes its own control red', async () => {
|
||||
const root = await fixture('internal-meta');
|
||||
await writeGate(root);
|
||||
await writeManifest(root, baseManifest());
|
||||
|
||||
const result = verify(root);
|
||||
assert.equal(result.status, 0, output(result));
|
||||
assert.match(output(result), /META-NEGATIVE-CONTROL.*meta-fixture.*observed red/i);
|
||||
});
|
||||
|
||||
test('a mutation crash is rejected instead of counted as an observed-red control', async () => {
|
||||
const root = await fixture('mutation-crash');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].inertMutation.replace = 'this is not valid shell (';
|
||||
manifest.gates[0].inertMutation.expected = { exitCode: 0 };
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*mutation.*unexpected outcome/i);
|
||||
assert.doesNotMatch(output(result), /META-NEGATIVE-CONTROL.*observed red/i);
|
||||
});
|
||||
|
||||
test('a stale declared mutation case id is rejected instead of falling back', async () => {
|
||||
const root = await fixture('stale-case-id');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].inertMutation.caseId = 'case-that-does-not-exist';
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*case-that-does-not-exist.*not found/i);
|
||||
});
|
||||
|
||||
test('duplicate stable ids and unsupported schema versions are rejected', async () => {
|
||||
const root = await fixture('closed-schema');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.schemaVersion = 99;
|
||||
manifest.criteria.push({ ...manifest.criteria[0] });
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /unsupported schemaVersion 99/i);
|
||||
assert.match(output(result), /duplicate criterion id META-CRIT-1/i);
|
||||
});
|
||||
|
||||
test('manifest-controlled fixture paths cannot escape the sandbox', async () => {
|
||||
const root = await fixture('path-traversal');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases[0].fixture = {
|
||||
writeFiles: [{ path: '../../escaped-by-manifest', content: 'bad' }],
|
||||
};
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /path escapes sandbox/i);
|
||||
});
|
||||
|
||||
test('fixture writes reject a final symlink and preserve its outside target', async () => {
|
||||
const root = await fixture('final-symlink');
|
||||
await writeGate(root);
|
||||
const outside = path.join(fixtureBase, 'outside-sentinel');
|
||||
await writeFile(outside, 'preserve-me\n');
|
||||
const linked = path.join(root, 'linked-sentinel');
|
||||
await symlink(outside, linked);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases[0].fixture = {
|
||||
writeFiles: [{ path: 'linked-sentinel', content: 'overwritten\n' }],
|
||||
};
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /fixture write.*symbolic link/i);
|
||||
assert.equal(await readFile(outside, 'utf8'), 'preserve-me\n');
|
||||
});
|
||||
|
||||
test('an executable below a gate root without an entry is rejected', async () => {
|
||||
const root = await fixture('unregistered');
|
||||
await writeGate(root);
|
||||
const extra = path.join(root, 'gates', 'forgotten.sh');
|
||||
await writeFile(extra, '#!/bin/sh\nexit 1\n');
|
||||
await chmod(extra, 0o755);
|
||||
await writeManifest(root, baseManifest());
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /unregistered gate.*forgotten\.sh/i);
|
||||
});
|
||||
|
||||
test('a must-fail case without a reason diagnostic is rejected', async () => {
|
||||
const root = await fixture('missing-reason');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases[0].reasonPattern = '';
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*rejects-bad-input.*reasonPattern/i);
|
||||
});
|
||||
|
||||
test('a gate with zero must-fail cases is rejected', async () => {
|
||||
const root = await fixture('no-negative');
|
||||
await writeGate(root, '#!/bin/sh\nexit 0\n');
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].inertMutation = {
|
||||
file: 'gates/meta-fixture.sh',
|
||||
find: 'exit 0',
|
||||
replace: 'exit 1',
|
||||
};
|
||||
manifest.gates[0].cases = [
|
||||
{
|
||||
id: 'positive',
|
||||
criterionIds: ['META-CRIT-1'],
|
||||
mustFail: false,
|
||||
invocation: ['gates/meta-fixture.sh'],
|
||||
required: { exitCode: 0 },
|
||||
actual: { exitCode: 0 },
|
||||
reasonPattern: '',
|
||||
},
|
||||
];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*no negative control/i);
|
||||
});
|
||||
|
||||
test('reordered but equivalent outcome fields do not create a false behavior delta', async () => {
|
||||
const root = await fixture('reordered-outcomes');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases[0].required = { exitCode: 7, outputPattern: 'META_REJECT' };
|
||||
manifest.gates[0].cases[0].actual = { outputPattern: 'META_REJECT', exitCode: 7 };
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.equal(result.status, 0, output(result));
|
||||
assert.doesNotMatch(output(result), /behavior delta requires|DEFECT \(owner:/);
|
||||
});
|
||||
|
||||
test('a required-versus-actual delta without a tracked owner is rejected', async () => {
|
||||
const root = await fixture('ownerless-delta');
|
||||
await writeGate(root, '#!/bin/sh\nexit 0\n');
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].inertMutation.find = 'exit 0';
|
||||
manifest.gates[0].inertMutation.replace = 'exit 7';
|
||||
manifest.gates[0].cases[0].actual.exitCode = 0;
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*delta.*tracked owner/i);
|
||||
});
|
||||
|
||||
test('moving criterion bindings to unrelated cases is rejected', async () => {
|
||||
const root = await fixture('semantic-misbinding');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.criteria.push({
|
||||
id: 'META-CRIT-2',
|
||||
originalText: 'The fixture reports the second rejection reason.',
|
||||
currentText: 'The fixture reports the second rejection reason.',
|
||||
claimType: 'integrity',
|
||||
source: 'fixture',
|
||||
meaningChanges: [],
|
||||
caseRefs: ['meta-fixture/rejects-second-input'],
|
||||
});
|
||||
manifest.gates[0].cases.push({
|
||||
...manifest.gates[0].cases[0],
|
||||
id: 'rejects-second-input',
|
||||
criterionIds: ['META-CRIT-1'],
|
||||
});
|
||||
manifest.gates[0].cases[0].criterionIds = ['META-CRIT-2'];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /META-CRIT-1.*rejects-bad-input.*not bound/i);
|
||||
assert.match(output(result), /META-CRIT-2.*rejects-second-input.*not bound/i);
|
||||
});
|
||||
|
||||
test('canonical verification preserves binding diagnostics when a case fixture is also stale', async () => {
|
||||
const root = await fixture('misbinding-plus-stale-fixture');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.criteria.push({
|
||||
id: 'META-CRIT-2',
|
||||
originalText: 'The second case rejects its own bad input.',
|
||||
currentText: 'The second case rejects its own bad input.',
|
||||
claimType: 'integrity',
|
||||
source: 'fixture',
|
||||
meaningChanges: [],
|
||||
caseRefs: ['meta-fixture/rejects-second-input'],
|
||||
});
|
||||
manifest.gates[0].cases.push({
|
||||
...manifest.gates[0].cases[0],
|
||||
id: 'rejects-second-input',
|
||||
criterionIds: ['META-CRIT-1'],
|
||||
});
|
||||
manifest.gates[0].cases[0].criterionIds = ['META-CRIT-2'];
|
||||
manifest.gates[0].cases[0].fixture = {
|
||||
replaceFiles: [
|
||||
{
|
||||
path: 'gates/meta-fixture.sh',
|
||||
find: 'text that is not present',
|
||||
replace: 'irrelevant',
|
||||
},
|
||||
],
|
||||
};
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /META-CRIT-1.*rejects-bad-input.*not bound/i);
|
||||
assert.match(output(result), /META-CRIT-2.*rejects-second-input.*not bound/i);
|
||||
assert.match(output(result), /fixture replace.*stale or ambiguous/i);
|
||||
});
|
||||
|
||||
test('moving meaning and prose criteria to an unrelated type error is rejected', async () => {
|
||||
const root = await fixture('real-manifest-misbinding');
|
||||
const manifest = JSON.parse(
|
||||
await readFile(path.join(process.cwd(), 'gates', 'gates.manifest.json'), 'utf8'),
|
||||
);
|
||||
for (const gate of manifest.gates) {
|
||||
for (const gateCase of gate.cases) {
|
||||
gateCase.criterionIds = gateCase.criterionIds.filter(
|
||||
(id) => !['RM02-MEANING-PROVENANCE', 'RM02-PROSE-CONTROL'].includes(id),
|
||||
);
|
||||
}
|
||||
}
|
||||
const typeError = manifest.gates
|
||||
.find((gate) => gate.id === 'quality-typecheck')
|
||||
.cases.find((gateCase) => gateCase.id === 'type-error');
|
||||
typeError.criterionIds.push('RM02-MEANING-PROVENANCE', 'RM02-PROSE-CONTROL');
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root, ['--structure-only']);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /RM02-MEANING-PROVENANCE.*missing-meaning-provenance.*not bound/i);
|
||||
assert.match(output(result), /RM02-PROSE-CONTROL.*prose-claim-misbinding.*not bound/i);
|
||||
assert.match(
|
||||
output(result),
|
||||
/RM02-(?:MEANING-PROVENANCE|PROSE-CONTROL).*undeclared exercising case quality-typecheck\/type-error/i,
|
||||
);
|
||||
});
|
||||
|
||||
test('a criterion with no bound case is rejected', async () => {
|
||||
const root = await fixture('unbound-criterion');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.criteria.push({
|
||||
id: 'ORPHAN',
|
||||
originalText: 'This criterion is not exercised.',
|
||||
currentText: 'This criterion is not exercised.',
|
||||
claimType: 'quality',
|
||||
source: 'fixture',
|
||||
meaningChanges: [],
|
||||
});
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /ORPHAN.*no bound case/i);
|
||||
});
|
||||
|
||||
test('an unbound governing prose marker is rejected', async () => {
|
||||
const root = await fixture('unbound-prose');
|
||||
await writeGate(root);
|
||||
await mkdir(path.join(root, 'docs'), { recursive: true });
|
||||
await writeFile(path.join(root, 'docs', 'governing.md'), 'GATE-CLAIM:UNBOUND\n');
|
||||
const manifest = baseManifest();
|
||||
manifest.governingClaimFiles = ['docs/governing.md'];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /GATE-CLAIM:UNBOUND.*unbound/i);
|
||||
});
|
||||
|
||||
test('directly contradictory modeled scenarios are rejected', async () => {
|
||||
const root = await fixture('conflict');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.compatibilityScenarios = [
|
||||
{
|
||||
id: 'ONE',
|
||||
construction: 'same-input',
|
||||
caseRefs: ['meta-fixture/rejects-bad-input'],
|
||||
invocation: ['sh', '-c', 'exit 0'],
|
||||
expected: { exitCode: 0 },
|
||||
},
|
||||
{
|
||||
id: 'TWO',
|
||||
construction: 'same-input',
|
||||
caseRefs: ['meta-fixture/rejects-bad-input'],
|
||||
invocation: ['sh', '-c', 'exit 1'],
|
||||
expected: { exitCode: 1 },
|
||||
},
|
||||
];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /ONE.*TWO.*conflict/i);
|
||||
});
|
||||
|
||||
test('compatibility scenarios execute referenced conditions as one construction', async () => {
|
||||
const root = await fixture('combined-compatibility');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases.push({
|
||||
id: 'second-condition',
|
||||
criterionIds: ['META-CRIT-1'],
|
||||
mustFail: true,
|
||||
invocation: ['sh', '-c', 'echo "$SECOND_REASON" >&2; exit 7'],
|
||||
fixture: { writeFiles: [{ path: 'conditions/second', content: 'present\n' }] },
|
||||
required: { exitCode: 7 },
|
||||
actual: { exitCode: 7 },
|
||||
reasonPattern: 'SECOND_REASON',
|
||||
environment: { SECOND_REASON: 'SECOND_REASON' },
|
||||
});
|
||||
manifest.criteria[0].caseRefs.push('meta-fixture/second-condition');
|
||||
manifest.gates[0].cases[0].fixture = {
|
||||
writeFiles: [{ path: 'conditions/first', content: 'present\n' }],
|
||||
};
|
||||
manifest.compatibilityScenarios = [
|
||||
{
|
||||
id: 'BOTH-CONDITIONS',
|
||||
construction: 'both-fixtures',
|
||||
caseRefs: ['meta-fixture/rejects-bad-input', 'meta-fixture/second-condition'],
|
||||
invocation: ['sh', '-c', 'test -f conditions/first && test -f conditions/second'],
|
||||
expected: { exitCode: 0 },
|
||||
},
|
||||
];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.equal(result.status, 0, output(result));
|
||||
assert.match(output(result), /COMPATIBILITY BOTH-CONDITIONS: observed expected outcome/i);
|
||||
});
|
||||
|
||||
test('a restated criterion without provenance is rejected', async () => {
|
||||
const root = await fixture('provenance');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.criteria[0].currentText = 'The fixture rejects only malformed input.';
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /META-CRIT-1.*meaning-change provenance/i);
|
||||
});
|
||||
|
||||
test('a security criterion bound only to a positive case is unregistered in substance', async () => {
|
||||
const root = await fixture('positive-only-criterion');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.criteria.push({
|
||||
id: 'POSITIVE-ONLY',
|
||||
originalText: 'A security property.',
|
||||
currentText: 'A security property.',
|
||||
claimType: 'security',
|
||||
source: 'fixture',
|
||||
meaningChanges: [],
|
||||
});
|
||||
manifest.gates[0].cases.push({
|
||||
id: 'positive-only',
|
||||
criterionIds: ['POSITIVE-ONLY'],
|
||||
mustFail: false,
|
||||
invocation: ['gates/meta-fixture.sh'],
|
||||
required: { exitCode: 7 },
|
||||
actual: { exitCode: 7 },
|
||||
reasonPattern: '',
|
||||
});
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /POSITIVE-ONLY.*no must-fail case/i);
|
||||
});
|
||||
|
||||
test('a registered prose claim whose marker is absent is rejected', async () => {
|
||||
const root = await fixture('missing-prose-marker');
|
||||
await writeGate(root);
|
||||
await mkdir(path.join(root, 'docs'), { recursive: true });
|
||||
await writeFile(path.join(root, 'docs', 'governing.md'), 'No marker here.\n');
|
||||
const manifest = baseManifest();
|
||||
manifest.governingClaimFiles = ['docs/governing.md'];
|
||||
manifest.proseClaims = [{ id: 'MISSING-MARKER', criterionId: 'META-CRIT-1' }];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /GATE-CLAIM:MISSING-MARKER.*missing/i);
|
||||
});
|
||||
|
||||
test('source-versus-deployed byte drift is rejected and names the gate', async () => {
|
||||
const root = await fixture('deployment-drift');
|
||||
await writeGate(root);
|
||||
await mkdir(path.join(root, 'deployed'), { recursive: true });
|
||||
await writeFile(path.join(root, 'deployed', 'meta-fixture.sh'), '#!/bin/sh\nexit 0\n');
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].deployment = { kind: 'file', path: 'deployed/meta-fixture.sh' };
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*source.*deployed.*drift/i);
|
||||
});
|
||||
|
||||
test('deployment drift meta-control fails if the shared comparator is made inert', async () => {
|
||||
const root = await fixture('deployment-comparator-inert');
|
||||
await writeGate(root);
|
||||
await mkdir(path.join(root, 'deployed'), { recursive: true });
|
||||
await copyFile(
|
||||
path.join(root, 'gates', 'meta-fixture.sh'),
|
||||
path.join(root, 'deployed', 'meta-fixture.sh'),
|
||||
);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].deployment = { kind: 'file', path: 'deployed/meta-fixture.sh' };
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const alteredScripts = path.join(root, 'verifier-scripts');
|
||||
await mkdir(alteredScripts, { recursive: true });
|
||||
const verifierSource = await readFile(verifier, 'utf8');
|
||||
const inertSource = verifierSource.replace(
|
||||
'return source.equals(deployed);',
|
||||
'return true; // deliberate test-only inert comparator',
|
||||
);
|
||||
assert.notEqual(inertSource, verifierSource, 'shared deployment comparator mutation went stale');
|
||||
await writeFile(path.join(alteredScripts, 'gate-verify.mjs'), inertSource);
|
||||
await copyFile(
|
||||
path.join(process.cwd(), 'scripts', 'gate-history.mjs'),
|
||||
path.join(alteredScripts, 'gate-history.mjs'),
|
||||
);
|
||||
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[
|
||||
path.join(alteredScripts, 'gate-verify.mjs'),
|
||||
'--root',
|
||||
root,
|
||||
'--manifest',
|
||||
'gates/gates.manifest.json',
|
||||
'--skip-history',
|
||||
],
|
||||
{ cwd: root, encoding: 'utf8', env: { ...process.env, HOME: os.homedir() } },
|
||||
);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*drift negative control was ineffective/i);
|
||||
});
|
||||
@@ -0,0 +1,104 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import test from 'node:test';
|
||||
|
||||
const root = process.cwd();
|
||||
const expectedTriggers = `when:
|
||||
# PR + manual CI run on any branch — the pull_request pipeline is the merge gate.
|
||||
# push CI is restricted to protected branches (main) so a feature-branch push no
|
||||
# longer fires a redundant SECOND pipeline alongside its PR pipeline. This ~halves
|
||||
# CI load on the storage-constrained runner with zero loss of gating (branch
|
||||
# protection requires no push/ci status context; main still gets full push CI).
|
||||
- event: [pull_request, manual]
|
||||
- event: push
|
||||
branch: main`;
|
||||
const expectedGateStep = ` image: *node_image
|
||||
# Woodpecker's shallow marker makes merge-base reject even present parents;
|
||||
# full history is required for activation ancestry and manifest provenance.
|
||||
commands:
|
||||
- *enable_pnpm
|
||||
- apk add --no-cache bubblewrap
|
||||
- if [ -f .git/shallow ]; then git fetch --unshallow --no-tags origin; fi
|
||||
- pnpm gate:verify
|
||||
depends_on:
|
||||
- install
|
||||
- sanitization
|
||||
- upgrade-guard`;
|
||||
|
||||
export function assertUnprivilegedGateStep(pipeline) {
|
||||
assert.doesNotMatch(
|
||||
pipeline,
|
||||
/privileged/i,
|
||||
'no pull-request pipeline step may declare privilege',
|
||||
);
|
||||
for (const line of pipeline.split('\n')) {
|
||||
const candidate = line.trimStart().replace(/^-\s+/, '');
|
||||
if (/^(?:["'!<].*|[A-Za-z_][A-Za-z0-9_-]*\s+):(?:\s|$)/.test(candidate)) {
|
||||
assert.fail(`non-canonical or merged YAML key is forbidden: ${candidate}`);
|
||||
}
|
||||
}
|
||||
const triggerMatches = [...pipeline.matchAll(/^when:\n([\s\S]*?)(?=\n\n)/gm)];
|
||||
assert.equal(triggerMatches.length, 1, 'exactly one top-level trigger is required');
|
||||
assert.equal(
|
||||
`when:\n${triggerMatches[0][1].trimEnd()}`,
|
||||
expectedTriggers,
|
||||
'top-level triggers must match closed PR/main construction',
|
||||
);
|
||||
|
||||
const matches = [
|
||||
...pipeline.matchAll(/\n gate-verify:\n([\s\S]*?)(?=\n [a-z][a-z0-9-]+:|\nservices:|$)/g),
|
||||
];
|
||||
assert.equal(matches.length, 1, 'exactly one gate-verify step is required');
|
||||
// Closed textual construction by design: accepting arbitrary YAML syntax here
|
||||
// would require a duplicate-key-preserving parser. Exact equality rejects all
|
||||
// extra keys, quoted/escaped key spellings, aliases, and mapping merges.
|
||||
assert.equal(matches[0][1].trimEnd(), expectedGateStep, 'gate-verify step must match closed unprivileged construction');
|
||||
}
|
||||
|
||||
test('package.json exposes the canonical gate:verify command', async () => {
|
||||
const packageJson = JSON.parse(await readFile(`${root}/package.json`, 'utf8'));
|
||||
assert.equal(packageJson.scripts['gate:verify'], 'node scripts/gate-verify.mjs');
|
||||
});
|
||||
|
||||
test('Woodpecker runs the closed unprivileged gate construction on every pipeline', async () => {
|
||||
const pipeline = await readFile(`${root}/.woodpecker/ci.yml`, 'utf8');
|
||||
assertUnprivilegedGateStep(pipeline);
|
||||
});
|
||||
|
||||
test('gate wiring rejects privilege syntax, merges, duplicate keys, and trigger narrowing', async () => {
|
||||
const pipeline = await readFile(`${root}/.woodpecker/ci.yml`, 'utf8');
|
||||
const additions = [
|
||||
' privileged: *enabled\n',
|
||||
' "privileged": true\n',
|
||||
" 'privileged': true\n",
|
||||
' privileged : true\n',
|
||||
' "priv\\u0069leged": true\n',
|
||||
' <<: *privileged-step\n',
|
||||
' "<<": *privileged-step\n',
|
||||
];
|
||||
for (const addition of additions) {
|
||||
const changed = pipeline.replace(' gate-verify:\n image:', ` gate-verify:\n${addition} image:`);
|
||||
assert.throws(() => assertUnprivilegedGateStep(changed));
|
||||
}
|
||||
const privilegedInstall = pipeline.replace(
|
||||
' install:\n image:',
|
||||
' install:\n privileged: true\n image:',
|
||||
);
|
||||
const duplicate = `${pipeline}\n gate-verify:\n image: *node_image\n`;
|
||||
const noPullRequest = pipeline.replace(
|
||||
' - event: [pull_request, manual]',
|
||||
' - event: manual',
|
||||
);
|
||||
const filteredPullRequest = pipeline.replace(
|
||||
' - event: [pull_request, manual]',
|
||||
' - event: [pull_request, manual]\n path: [scripts/**]',
|
||||
);
|
||||
|
||||
assert.throws(() => assertUnprivilegedGateStep(privilegedInstall), /privilege/i);
|
||||
assert.throws(() => assertUnprivilegedGateStep(duplicate), /exactly one gate-verify/);
|
||||
assert.throws(() => assertUnprivilegedGateStep(noPullRequest), /closed PR\/main construction/);
|
||||
assert.throws(
|
||||
() => assertUnprivilegedGateStep(filteredPullRequest),
|
||||
/closed PR\/main construction/,
|
||||
);
|
||||
});
|
||||
Reference in New Issue
Block a user