RM-02: enforce anti-inert gate registry #1030
+13
-3
@@ -68,15 +68,25 @@ steps:
|
||||
- bash packages/mosaic/framework/tools/quality/scripts/test-upgrade-durable-snapshot.sh
|
||||
- bash packages/mosaic/framework/tools/quality/scripts/test-install-migration.sh
|
||||
|
||||
# Anti-inert-gate registry. Deliberately unconditional: no path filter and no
|
||||
# step-level `when`, because a gate can be disabled by changes outside its own path.
|
||||
gate-verify:
|
||||
image: *node_image
|
||||
commands:
|
||||
- *enable_pnpm
|
||||
- pnpm gate:verify
|
||||
depends_on:
|
||||
- install
|
||||
- sanitization
|
||||
- upgrade-guard
|
||||
|
||||
typecheck:
|
||||
image: *node_image
|
||||
commands:
|
||||
- *enable_pnpm
|
||||
- pnpm typecheck
|
||||
depends_on:
|
||||
- install
|
||||
- sanitization
|
||||
- upgrade-guard
|
||||
- gate-verify
|
||||
|
||||
# lint, format, and test are independent — run in parallel after typecheck
|
||||
lint:
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
# Administrator Guide
|
||||
|
||||
- [Gate registry operations](quality-gate-registry.md)
|
||||
@@ -0,0 +1,35 @@
|
||||
# Gate Registry Operations
|
||||
|
||||
## Routine verification
|
||||
|
||||
Run `pnpm gate:verify` from a dependency-installed checkout. Exit zero means registry observations matched their declared `actual` values; it does **not** assert required-behavior conformance while deltas remain. Open `DEFECT` records are checked descriptions of current behavior with tracked owners, never successful gate outcomes.
|
||||
|
||||
Investigate any of these immediately:
|
||||
|
||||
- `GATE VERIFY FAILED` — registry structure, observed behavior, provenance, claim binding, source/deployment identity, or negative-control detection changed. The verifier aggregates independent phase failures, so repair the responsible stable-ID diagnostics as well as any accompanying stale-fixture error; do not treat the generic error as a substitute.
|
||||
- `unregistered gate` — an executable appeared under a declared gate root without a registry entry.
|
||||
- `no negative control` — a gate has no must-fail case.
|
||||
- `DEPLOYED IDENTITY UNAVAILABLE` — the runner cannot reach the installed enforcing copy. The pinned observation is checked, but live equality is not asserted.
|
||||
- `HISTORY_PROVENANCE_FORBIDDEN` — a history assertion path was reintroduced at the repository layer; remove it and keep RM-60 as the tracked external-boundary owner.
|
||||
|
||||
## Updating a gate
|
||||
|
||||
1. Add or change the criterion, its exact criterion-side `caseRefs`, and matching case-side `criterionIds`. Preserve recursive closed-schema validation for every nested object; new fields require explicit key and type handling.
|
||||
2. Observe the must-fail case fail for its own stated reason; moving the binding to any undeclared case must fail verification.
|
||||
3. Declare an exact inerting mutation and observe the verifier detect it.
|
||||
4. If required and actual behavior differ, add a tracked remediation owner and justification.
|
||||
5. If meaning changed, append provenance; never replace the original silently.
|
||||
6. For an installed counterpart, verify live byte identity and update the observed digest only from measured evidence.
|
||||
7. Run focused verifier tests, `pnpm gate:verify`, and the repository baseline gates.
|
||||
|
||||
Do not add an ownerless exception or describe an open delta as pass/green/OK.
|
||||
|
||||
## CI behavior
|
||||
|
||||
Woodpecker runs `gate-verify` on every pull request and protected-main push without path filtering. This is deliberate: changes outside gate files can make a gate inert. The step needs no local history preparation because RM-02 asserts no history-provenance property.
|
||||
|
||||
**DOES:** PR CI executes current-tree verification unprivileged and fail-closed. It compares the manifest and verifier inventory separately with `gates/required-gates.baseline.json`, and consumed case evidence carries a subject checked against its gate definition. **Detects accidental and incompetent inventory drift within a checkout; does NOT survive an adversary who rewrites baseline, manifest, and verifier consistently — that guarantee requires RM-60's external boundary.** The registered shrink-both, overclaim, and per-gate evidence-subject controls must remain red for their stated reasons.
|
||||
|
||||
**DOES NOT:** No local git state in the PR checkout is trustworthy as a history anchor because PR-controlled lifecycle code executes before the gate. The verifier has no history-verification path, and its closed current-tree observation renderer has no history/ancestry/lineage success class. Do not add a local ref, config, remote URL, source constant, or author-positioned path as a replacement anchor.
|
||||
|
||||
`scripts/gate-history-exclusion-control.mjs` enforces the output incapacity by testing alternate success wording and production renderer wiring; its registered fixture proves adding a prohibited success class goes red. RM-60 is the tracked owner of the provider-controlled/protected pre-execution boundary. If a bootstrap override is ever proposed before RM-60, it requires a separate explicit, loud, audited, retiring, negative-controlled design; RM-02 contains no override.
|
||||
@@ -0,0 +1,3 @@
|
||||
# Developer Guide
|
||||
|
||||
- [Gate registry and negative controls](quality-gate-registry.md)
|
||||
@@ -0,0 +1,43 @@
|
||||
# Gate Registry and Negative Controls
|
||||
|
||||
`gates/gates.manifest.json` is the machine-readable registry for the initial RM-02 gate slice. Run:
|
||||
|
||||
```bash
|
||||
pnpm gate:verify
|
||||
```
|
||||
|
||||
## Registered slice
|
||||
|
||||
The registry covers root typecheck, lint, and format checks; RM-01 checkout preflight; the Mosaic CI queue guard; and root Husky pre-commit/pre-push hooks. It does not imply repository-wide coverage. Framework scripts, package-local build/test scripts, templates, and deployment/release scripts remain assigned to RM-54.
|
||||
|
||||
Every gate declares exact invocations, observed and required outcomes, criterion bindings, and a single exact inerting mutation. Every case result carries an evidence-side subject that is compared with the gate definition when the result is consumed; the population control mutates that evidence-side subject independently for every required gate. Every must-fail case requires a non-empty reason diagnostic. The verifier rejects a stale, ambiguous, crashing, or ineffective mutation. Fixture and mutation writes reject path traversal and final-component symlinks. Every nested manifest object used by outcomes, mutations, fixtures, deployments, defects, compatibility, provenance, coverage, and merge assertions has closed keys and strict field types; a misspelling cannot silently turn a required comparison into an absent optional field. Independent validation phases collect labeled failures instead of letting one thrown fixture, claim, discovery, deployment, mutation, or compatibility error mask already-known stable-ID diagnostics. This proves detection of the **declared** inerting mutation, not every possible semantic weakening.
|
||||
|
||||
## Required versus actual
|
||||
|
||||
A case may record different `required` and `actual` outcomes only with a tracked owner. The verifier checks current reality against `actual`, prints each difference as `DEFECT (owner: ...)`, and fails when behavior changes without a matching registry update. A defect is never described as passing, green, or OK.
|
||||
|
||||
The queue guard currently has RM-03-owned deltas. In particular, its stdin/heredoc classifier does not consume piped status JSON, so terminal-success, no-status, and terminal-failure payloads become `unknown`; unknown and malformed states exit zero; push purpose defaults to `main`. RM-02 records these observations and does not edit the guard.
|
||||
|
||||
## Criteria, prose, and compatibility
|
||||
|
||||
Each criterion declares exact `caseRefs`; the verifier compares those semantic declarations bidirectionally with case-side `criterionIds` and requires at least one must-fail case. Moving a criterion ID to an unrelated case therefore fails as both a missing declared exercising case and an undeclared binding. Registered meta-negative controls misbind a criterion, remove meaning provenance, and misbind a prose claim, and each must make structure verification red for its stated reason.
|
||||
|
||||
Designated governing prose uses `GATE-CLAIM:<id>` markers. Each claim also names the exact must-fail `caseRef` that exercises its criterion; unknown, positive-only, unrelated, unbound, or registered-but-missing claims fail. Orchestrator-owned claims from `TASKS.md` are bound through `docs/remediation/GATE-CLAIMS.md`, which records source headings and anchored text without changing task tracking. Marker completeness still requires RM-54 review because arbitrary English claims cannot be inferred safely.
|
||||
|
||||
Compatibility checks detect direct contradictions in declared finite constructions. The verifier combines referenced case fixtures and environments in one isolated tree, rejects conflicting fixture/environment values, executes the construction's exact invocation, and checks its exact outcome. They do not prove semantic consistency of arbitrary natural language.
|
||||
|
||||
Restatements preserve original text, current text, reason, finding/task, and date.
|
||||
|
||||
## Source and deployed identity
|
||||
|
||||
A gate with an external installed counterpart declares it explicitly. When the installed queue guard is reachable, its bytes must equal repository source and an internal drift control is observed red. In CI the operator-home installation may be outside the container; the verifier checks the pinned observed source digest, reports `DEPLOYED IDENTITY UNAVAILABLE (owner: RM-04)`, and does not infer live equality.
|
||||
|
||||
## Current-tree and history-provenance boundary
|
||||
|
||||
**DOES:** Every PR evaluates the current checkout's registered gates and declared inerting mutations directly, unprivileged and fail-closed. The seven-gate population, verifier inventory, and `gates/required-gates.baseline.json` are compared inside the checkout. Evidence-side subjects are consumed and compared with gate definitions for every gate. **Detects accidental and incompetent inventory drift within a checkout; does NOT survive an adversary who rewrites baseline, manifest, and verifier consistently — that guarantee requires RM-60's external boundary.**
|
||||
|
||||
**DOES NOT:** This repository layer establishes history provenance at all. `pnpm install` executes PR-controlled lifecycle code before `gate:verify`, so no local ref, git config, remote URL, constant, or author-positioned path in the checkout can anchor a history claim. An observation saying “unverifiable” while returning zero would be a green wearing a disclaimer, so the claim and history verifier path are removed.
|
||||
|
||||
The production output path uses a closed current-tree observation renderer; history, ancestry, and provider-lineage success are not representable observation classes. `scripts/gate-history-exclusion-control.mjs` exercises alternate success wording and production renderer wiring. Its registered must-fail case turns red if a prohibited success class is added or renderer consumption is bypassed. RM-60 owns the provider-controlled/protected pre-execution boundary needed before history provenance can be asserted. No bootstrap override exists in RM-02.
|
||||
|
||||
Universal checks first prove populations non-empty. The production profile reads the same-checkout baseline before comparing the verifier inventory and manifest separately, while `gateRefs` on the D-38/D-40 criteria must exactly span every registered gate. Population controls mutate evidence-side subjects and type-strict comparison inputs one gate at a time and require rejection across the complete inventory. A registered prose-claim control fails when the same-checkout mechanism is described as an adversarial protection.
|
||||
+57
@@ -14,6 +14,63 @@
|
||||
|
||||
---
|
||||
|
||||
## RM-02 Gate Registry and Negative-Control Verifier (#1029)
|
||||
|
||||
### Problem and objective
|
||||
|
||||
Existing deterministic gates can return success without enforcing their stated property. RM-02 introduces a machine-readable registry and an unconditional CI verifier that distinguishes required behavior from observed behavior, proves every registered negative control can detect its own failure reason, and makes source/deployment drift visible.
|
||||
|
||||
### Scope
|
||||
|
||||
**In scope:** root typecheck, lint, and format gates; RM-01 checkout preflight; the Mosaic CI queue guard; root Husky pre-commit and pre-push hooks; criterion bindings; modeled compatibility; meaning-change provenance; security/integrity prose claim markers; source-versus-deployed identity; unprivileged current-tree PR verification; independent required-inventory comparison; evidence-side subject consumption; and enforced structural exclusion of history provenance until RM-60 provides protected external authority.
|
||||
|
||||
**Out of scope:** fixing the queue guard (RM-03); exhaustive registration of every repository executable (RM-54); semantic proof that arbitrary English criteria are mutually satisfiable (RM-54/RM-55); history provenance before RM-60's provider-controlled/protected execution boundary; and a same-authority trust anchor for repository-authored evidence (RM-25/RM-59).
|
||||
|
||||
### Normative requirements
|
||||
|
||||
1. `RM02-REQ-01`: The JSON registry SHALL give every gate and criterion a stable ID and SHALL declare exact invocation, input classes, cases, exact observed and required exit codes, reason diagnostics, and criterion bindings.
|
||||
2. `RM02-REQ-02`: Every gate SHALL have at least one observed-red must-fail case. The verifier SHALL reject missing, stale, ambiguous, or ineffective declared inert mutations and SHALL name an externally inerted gate.
|
||||
3. `RM02-REQ-03`: Every acceptance criterion SHALL declare exact criterion-side `caseRefs` that match case-side `criterionIds` bidirectionally and include a case that can fail for that criterion's stated reason. Moving a binding to an unrelated case SHALL fail verification. Security/integrity prose claims in the designated governing documents SHALL carry bound `GATE-CLAIM:<id>` markers and declare the exact must-fail case exercising the claim criterion.
|
||||
4. `RM02-REQ-04`: Declared finite compatibility scenarios SHALL execute together and direct modeled contradictions SHALL fail. This does not claim semantic consistency of arbitrary English.
|
||||
5. `RM02-REQ-05`: Restated criteria SHALL retain original text, current text, reason, finding/task, and dated meaning-change history.
|
||||
6. `RM02-REQ-06`: An observed behavior differing from required behavior SHALL be reported as `DEFECT` with a tracked owner; an ownerless delta SHALL fail verification. Such a gate SHALL never be described as passing, green, or OK.
|
||||
7. `RM02-REQ-07`: Executables under declared gate roots SHALL fail with `unregistered gate` when absent from the registry. The initial coverage boundary SHALL explicitly list exclusions and bind the broader inventory to RM-54.
|
||||
8. `RM02-REQ-08`: Every gate with a deployed counterpart SHALL register source/deployed byte identity and a must-fail drift control. Gates without a deployed counterpart SHALL say so explicitly.
|
||||
9. `RM02-REQ-09`: CI SHALL run `pnpm gate:verify` on every pull request without path filtering and on protected-main pushes.
|
||||
10. `RM02-REQ-10` (restated): PR CI SHALL perform unprivileged, fail-closed current-tree verification only. The production observation renderer SHALL be a closed current-tree-only output type with no representable history-provenance success state. A registered must-fail control SHALL turn red if history/ancestry/lineage success is added to that renderer or if production output bypasses the renderer. RM-60 owns the provider-controlled/protected pre-execution boundary required to establish history provenance.
|
||||
11. `RM02-REQ-11` (`D-46`): The required seven-gate inventory SHALL be read from a same-checkout baseline and compared separately with both the verifier inventory and manifest. Shrinking the verifier inventory and manifest together while leaving the baseline intact SHALL fail for the removed gate. **Detects accidental and incompetent inventory drift within a checkout; does NOT survive an adversary who rewrites baseline, manifest, and verifier consistently — that guarantee requires RM-60's external boundary.**
|
||||
12. `RM02-REQ-12` (`D-38`): Every consumed case result SHALL carry an evidence-side subject independently declared from the gate definition. The verifier SHALL compare that evidence subject with the gate definition when consuming the result. A population control SHALL mutate the evidence-side subject independently for every required gate and observe rejection for every gate.
|
||||
13. `RM02-REQ-13` (`D-40`): For every gate listed in the required inventory baseline, each discriminator and comparison input SHALL have a recursively closed, type-strict schema. Unknown, misspelled, wrong-type, or present-but-empty nested assertion fields SHALL fail rather than disabling an assertion.
|
||||
14. `RM02-REQ-14` (`D-46`): No universally quantified registry check SHALL run until its population is proven non-empty and compared with the same-checkout baseline. The required seven-gate inventory, criteria, prose claims, and compatibility scenarios SHALL reject empty populations before reporting that all registered cases ran.
|
||||
|
||||
#### RM02-REQ-10 meaning-change provenance
|
||||
|
||||
- **Original:** “assert that every merged commit passed every required gate, evaluated AGAINST THAT COMMIT'S OWN TREE — not against current main.”
|
||||
- **Restatement:** The repository verifier performs current-tree verification and is structurally incapable of asserting history provenance. RM-60 supplies the provider-controlled/protected pre-execution boundary before any repository-controlled lifecycle code executes.
|
||||
- **Reason:** `pnpm install` executes PR-controlled lifecycle code before `gate:verify`; therefore no local ref, git config, remote URL, constant, or author-positioned path in the checkout is a trustworthy history anchor. A local disclaimer would be a green wearing a note, not a property. The history-provenance claim is removed rather than weakened.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
1. `RM02-AC-01`: A healthy current tree returns zero while prominently reporting the queue guard's required-versus-actual `DEFECT (owner: RM-03)` delta.
|
||||
2. `RM02-AC-02`: Externally mutate any registered gate at its declared inerting point so its failure path succeeds; verification returns nonzero and names that gate. The verifier's internal meta-control is observed red before its healthy result is trusted.
|
||||
3. `RM02-AC-03`: An executable added under a declared gate root without an entry returns nonzero and includes `unregistered gate`.
|
||||
4. `RM02-AC-04`: A gate with zero must-fail cases returns nonzero and includes `no negative control`.
|
||||
5. `RM02-AC-05`: Unbound or semantically misbound criteria, prose claims bound to unrelated cases, unbound governing prose markers, ownerless behavior deltas, stale mutations, source/deployed drift, and modeled compatibility conflicts each return nonzero with the responsible stable ID. A simultaneous stale fixture or independent phase error SHALL NOT mask responsible stable-ID diagnostics. Registered meta-negative controls move a criterion binding, remove meaning provenance, and redirect a prose claim to an unrelated case; each is observed red for its stated reason.
|
||||
6. `RM02-AC-06`: CI configuration invokes the verifier unconditionally on every pull request.
|
||||
7. `RM02-AC-07`: PR output states adjacent `DOES`/`DOES NOT` boundaries: the repository layer verifies current-tree registered cases, same-checkout inventory drift, and evidence-side subject consumption; it does not establish history provenance at all. **Detects accidental and incompetent inventory drift within a checkout; does NOT survive an adversary who rewrites baseline, manifest, and verifier consistently — that guarantee requires RM-60's external boundary.**
|
||||
8. `RM02-AC-08`: Registered must-fail controls reject an emptied registry; shrinking the verifier inventory and manifest together; overstating the same-checkout inventory boundary; adding history/ancestry/lineage success to the closed observation renderer or bypassing renderer consumption; a misspelled nested outcome field; a wrong outcome field type; and a present-but-empty outcome pattern.
|
||||
9. `RM02-AC-09`: Population controls iterate every gate listed in the baseline and prove consumed evidence-subject mismatch and wrong-type comparison input are rejected for each gate. `RM02-EVIDENCE-SUBJECT-BINDING`, `RM02-TYPE-STRICT-SCHEMA`, `RM02-HISTORY-PROVENANCE-EXCLUDED`, and `RM02-NONEMPTY-ANCHORED-QUANTIFICATION` are bidirectionally bound to their must-fail controls.
|
||||
|
||||
### Risks, dependencies, and verification boundary
|
||||
|
||||
- **DOES:** The repository verifier proves declared current-tree controls, modeled scenarios, bidirectional criterion/case relationships, source/deployed equality at execution time, a same-checkout seven-gate baseline comparison, and evidence-side subject consumption.
|
||||
- **DOES NOT:** This layer establishes no history provenance. PR-controlled lifecycle code executes before the gate, so no local git state in the checkout is trustworthy as a history anchor. RM-60 owns the provider-controlled/protected pre-execution boundary. The production verifier has no history verifier path, and its closed current-tree observation renderer cannot represent a history-provenance success state.
|
||||
- **Detects accidental and incompetent inventory drift within a checkout; does NOT survive an adversary who rewrites baseline, manifest, and verifier consistently — that guarantee requires RM-60's external boundary.** A registered claim control fails if the checked artifacts overstate this boundary.
|
||||
- The verifier does **not** infer arbitrary-English semantics or defend against an actor able to rewrite the gate, registry, verifier, controls, and baseline consistently.
|
||||
- `ASSUMPTION:` RM-54 is the owner for expanding registration and prose-marker coverage beyond this approved seven-gate slice; rationale: the remediation task graph already assigns the fleet-wide inert-gate audit there.
|
||||
|
||||
---
|
||||
|
||||
## Problem Statement
|
||||
|
||||
Jarvis (v0.2.0) is a self-hosted AI assistant with a Python FastAPI backend and Next.js frontend. It handles chat, projects, tasks, and LLM routing but lacks orchestration depth, agent coordination, shared memory, and remote access. The Mosaic framework (`~/.config/mosaic`) provides agent guides, shell-based orchestration tools, and quality rails — but these are loose scripts, not an integrated platform. The `@mosaicstack/*` packages in mosaic-mono-v0 began consolidating these into TypeScript packages (brain, queue, coord, cli, prdy, quality-rails) but have no UI, no auth, and no agent runtime integration.
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# Documentation Sitemap
|
||||
|
||||
## Gate verification
|
||||
|
||||
- [Developer gate registry guide](DEVELOPER-GUIDE/quality-gate-registry.md) — manifest schema, negative controls, evidence-side subjects, and the enforced RM-60 history-provenance exclusion. Detects accidental and incompetent inventory drift within a checkout; does NOT survive an adversary who rewrites baseline, manifest, and verifier consistently — that guarantee requires RM-60's external boundary.
|
||||
- [Gate registry operations](ADMIN-GUIDE/quality-gate-registry.md) — routine verification, failure interpretation, registry updates, and unconditional CI behavior.
|
||||
- [RM-02 governing claim index](remediation/GATE-CLAIMS.md) — marker bindings for orchestrator-owned remediation claims without modifying task tracking.
|
||||
|
||||
## Compaction refresh lease broker
|
||||
|
||||
- [Internal broker protocol](architecture/lease-broker-protocol.md) — kernel identity, ancestry and generation invariants, framed requests, responses, and persisted cycle bindings.
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
# RM-02 Gate Registry Implementation Plan
|
||||
|
||||
> **For Pi:** Use test-driven development and execute each task RED → GREEN → refactor.
|
||||
|
||||
**Goal:** Build a machine-readable seven-gate registry and an unconditional CI verifier that detects inert gates, binds criteria to observed negative controls, records defects honestly, and verifies the current PR tree unprivileged and fail-closed.
|
||||
|
||||
**Architecture:** A dependency-free Node CLI reads `gates/gates.manifest.json` and the same-checkout `gates/required-gates.baseline.json`, validates closed schemas and references, then runs typed cases in isolated main-disk fixtures. Detects accidental and incompetent inventory drift within a checkout; does NOT survive an adversary who rewrites baseline, manifest, and verifier consistently — that guarantee requires RM-60's external boundary. Gate-specific fixture setup remains declarative; exact invocations, evidence-side subjects, and exact observed/required exits stay in JSON. The production verifier is structurally incapable of asserting history provenance; RM-60 owns the provider-controlled/protected pre-execution boundary.
|
||||
|
||||
**Tech Stack:** Node.js ESM, `node:test`, JSON, shell gates, pnpm, Woodpecker CI.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Meta-negative-control kernel
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `scripts/gate-verify.test.mjs`
|
||||
- Create: `scripts/gate-verify.mjs`
|
||||
|
||||
1. Write a black-box fixture whose gate failure branch has already been changed to success.
|
||||
2. Run `node --test scripts/gate-verify.test.mjs`; require failure because the absent verifier does not name the inert gate.
|
||||
3. Implement manifest loading, exact process execution, and named mismatch reporting only.
|
||||
4. Re-run and require the external inert mutation to produce verifier nonzero while the test passes.
|
||||
5. Add an internally applied declared mutation and require a healthy fixture to report its negative control armed.
|
||||
|
||||
### Task 2: Registry structural clauses
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `scripts/gate-verify.test.mjs`
|
||||
- Modify: `scripts/gate-verify.mjs`
|
||||
|
||||
Add failing tests, one behavior at a time, for: `unregistered gate`; `no negative control`; unbound criterion; unbound `GATE-CLAIM`; ownerless required/actual delta; stale/ambiguous/ineffective mutation; direct modeled conflict; missing meaning-change provenance. Implement the minimum validator after each observed RED.
|
||||
|
||||
### Task 3: Gate fixtures and seven-gate manifest
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `gates/gates.manifest.json`
|
||||
- Create: `gates/fixtures/quality-case.mjs`
|
||||
- Create: `gates/fixtures/preflight-case.mjs`
|
||||
- Create: `gates/fixtures/queue-case.sh`
|
||||
- Create: `gates/fixtures/hook-case.sh`
|
||||
- Modify: `scripts/gate-verify.test.mjs`
|
||||
|
||||
Register typecheck, lint, format, RM-01 preflight, queue guard, pre-commit, and pre-push. For each, first run its known-bad case against an intentionally inert fixture and observe verifier RED; then restore the real gate behavior and require its exact observed exit/reason. Record queue defects as required/actual deltas owned by RM-03, never as pass/green/OK.
|
||||
|
||||
### Task 4: Source-versus-deployed identity
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `gates/gates.manifest.json`
|
||||
- Modify: `scripts/gate-verify.test.mjs`
|
||||
- Modify: `scripts/gate-verify.mjs`
|
||||
|
||||
Write and observe a failing test with a byte-mutated deployed counterpart. Implement byte equality and internal drift mutation controls. Declare `none` explicitly for gates without deployed counterparts.
|
||||
|
||||
### Task 5: Prose claims and compatibility constructions
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/remediation/MISSION.md`
|
||||
- Modify: `docs/remediation/TASKS.md` only if coordinator authorization overrides the worker prohibition; otherwise place markers in an RM-02 claim index that references immutable source anchors.
|
||||
- Modify: `gates/gates.manifest.json`
|
||||
- Modify: verifier tests/implementation.
|
||||
|
||||
Enumerate current security/integrity claims, bind each marker/id to a negative case, and reject unbound markers. Execute finite compatibility scenarios and clearly document that arbitrary English consistency is outside the model.
|
||||
|
||||
### Task 6: Current-tree boundary and enforced history-provenance exclusion
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `gates/required-gates.baseline.json`
|
||||
- Create: `scripts/gate-history-exclusion-control.mjs`
|
||||
- Create: `scripts/gate-inventory-shrink-control.mjs`
|
||||
- Modify: `scripts/gate-verify.mjs`
|
||||
- Modify: `gates/gates.manifest.json`
|
||||
|
||||
Verify current-tree behavior only. Remove the anchor-dependent history verifier and provider-history consumption because PR-controlled lifecycle code executes before the gate and makes every local git anchor untrustworthy. Use a closed current-tree observation renderer and register a must-fail control that turns red if history/ancestry/lineage success becomes representable or production output bypasses renderer consumption. Compare the verifier inventory and manifest separately against the same-checkout baseline, register must-fail attacks that shrink either paired representation, and register an overclaim control. Detects accidental and incompetent inventory drift within a checkout; does NOT survive an adversary who rewrites baseline, manifest, and verifier consistently — that guarantee requires RM-60's external boundary. State both directions: current-tree controls, inventory shape, and evidence subjects are enforced here; history provenance is not established until RM-60 provides a provider-controlled/protected pre-execution boundary.
|
||||
|
||||
### Task 7: CI and documentation
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `package.json`
|
||||
- Modify: `.woodpecker/ci.yml`
|
||||
- Create/update: `docs/DEVELOPER-GUIDE/quality-gate-registry.md`
|
||||
- Create/update: `docs/ADMIN-GUIDE/quality-gate-registry.md`
|
||||
- Modify: `docs/SITEMAP.md`
|
||||
|
||||
Add `gate:verify`; run it in an unconditional PR/main CI step. Document invocation, defect semantics, coverage/exclusions, marker syntax, replay/provider boundaries, and source/deployed identity.
|
||||
|
||||
### Task 8: Verification and delivery
|
||||
|
||||
Run focused tests, `pnpm gate:verify`, checkout tests, typecheck, lint, format, and applicable integration tests. Run Codex code and security review; remediate and re-review. Verify authorship, commit, execute queue guard before push, push, create PR via Mosaic wrapper, and send exact-head review request to rev-974 through the coordinator. Do not merge without coordinator authorization.
|
||||
@@ -0,0 +1,40 @@
|
||||
# RM-02 Governing Claim Index
|
||||
|
||||
This index binds remediation claims that live in orchestrator-owned `TASKS.md` without modifying that file. The source heading and quoted text are the reviewable anchor; `gate:verify` enforces each marker-to-criterion binding. Marker completeness beyond this approved slice remains RM-54.
|
||||
|
||||
## Prose is an enforceable claim
|
||||
|
||||
<!-- GATE-CLAIM:PROSE-IS-A-CLAIM -->
|
||||
|
||||
- Source: `docs/remediation/TASKS.md`, heading `D-20 — the orchestrator's own documentation overclaimed, and a reviewer disproved it empirically`.
|
||||
- Anchored text: “The defect was not in the code — it was in this file.”
|
||||
|
||||
## Generated-state verification scope
|
||||
|
||||
<!-- GATE-CLAIM:GENERATED-STATE-SCOPE -->
|
||||
|
||||
- Source: `docs/remediation/TASKS.md`, heading `D-19 — an integrity property that cannot exist at the layer it was specified`.
|
||||
- Anchored text: “a verifier that cannot detect the attack is not a verifier.”
|
||||
|
||||
## Execution trust boundary
|
||||
|
||||
<!-- GATE-CLAIM:EXECUTION-TRUST-BOUNDARY -->
|
||||
|
||||
- Source: RM-02 ruling recorded under `docs/remediation/TASKS.md`, RM-02 clause 4, D-25.
|
||||
- Anchored text: “SELF-VERIFICATION BY THE AUDITED PARTY IS NOT VERIFICATION.”
|
||||
- Dependency: RM-60/#1031, cross-referenced with RM-59.
|
||||
|
||||
## Same-checkout inventory drift boundary
|
||||
|
||||
<!-- GATE-CLAIM:INVENTORY-DRIFT-BOUNDARY -->
|
||||
|
||||
- Source: RM-02 ruling after D-48/CWE-353 reproduced against the round-4 baseline.
|
||||
- Boundary: Detects accidental and incompetent inventory drift within a checkout; does NOT survive an adversary who rewrites baseline, manifest, and verifier consistently — that guarantee requires RM-60's external boundary.
|
||||
- Negative control: `checkout-preflight/inventory-claim-overstatement` rewrites the boundary as protection and must go red.
|
||||
|
||||
## Criterion restatement provenance
|
||||
|
||||
<!-- GATE-CLAIM:CRITERION-RESTATEMENT -->
|
||||
|
||||
- Source: `docs/remediation/TASKS.md`, heading `D-18 — two pre-registered criteria were mutually unsatisfiable, discoverable only at implementation`.
|
||||
- Anchored text: “Both criteria were pre-registered. They cannot both be satisfied.”
|
||||
@@ -12,6 +12,8 @@ gate/program; the LLM handles only genuine judgment.
|
||||
|
||||
### First-class principle — observe the property, not the exit code
|
||||
|
||||
<!-- GATE-CLAIM:OBSERVE-PROPERTY -->
|
||||
|
||||
> **No write is done until the requested PROPERTY is observed. A success exit code is not evidence.**
|
||||
>
|
||||
> **Success output is designed to be believed.** That is the whole reason the inert-gate class exists
|
||||
@@ -30,6 +32,8 @@ gate/program; the LLM handles only genuine judgment.
|
||||
|
||||
### First-class principle — pre-registration prevents retrofitting, and nothing else
|
||||
|
||||
<!-- GATE-CLAIM:PREREGISTRATION-BOUNDARY -->
|
||||
|
||||
> **A pre-registered check set can fail in three distinct ways:**
|
||||
>
|
||||
> | mode | the set is… | found as |
|
||||
@@ -56,6 +60,8 @@ gate/program; the LLM handles only genuine judgment.
|
||||
|
||||
### Corollary — never ship an integrity claim dressed as a property
|
||||
|
||||
<!-- GATE-CLAIM:ARTIFACT-INTEGRITY-BOUNDARY -->
|
||||
|
||||
> A verification artifact that can be forged by whoever it is meant to catch verifies nothing. If a
|
||||
> manifest, marker, ledger, or receipt is writable by the same actor whose behaviour it certifies, it
|
||||
> **certifies the attack.** Such an artifact must sit inside the integrity envelope it belongs to,
|
||||
@@ -68,6 +74,8 @@ gate/program; the LLM handles only genuine judgment.
|
||||
|
||||
### First-class principle — when a property cannot exist at the layer it was specified
|
||||
|
||||
<!-- GATE-CLAIM:IMPOSSIBLE-LAYER-BOUNDARY -->
|
||||
|
||||
> Some required properties are **impossible at the layer that asked for them** — not hard, impossible.
|
||||
> A local check cannot defend against an actor who can rewrite the check itself. When that happens,
|
||||
> there are exactly three honest moves, and all three are mandatory:
|
||||
|
||||
@@ -0,0 +1,107 @@
|
||||
# RM-02 Gate Registry Scratchpad (#1029)
|
||||
|
||||
## Objective
|
||||
|
||||
Deliver the seven-gate registry and RED-first anti-inert verifier on `feat/rm-02-gate-registry`, preserving required-versus-actual defects without laundering them as success.
|
||||
|
||||
## Constraints and boundaries
|
||||
|
||||
- Do not modify `ci-queue-wait.sh`; RM-03 owns that fix.
|
||||
- Do not modify `docs/remediation/TASKS.md`; workers cannot edit orchestrator tracking.
|
||||
- Every declared behavior must be observed, not inferred from an exit code.
|
||||
- Repository-local evidence does not establish same-authority tamper resistance; RM-25/RM-59 own the external trust anchor.
|
||||
- Coverage is seven logical gates; broader inventory is RM-54.
|
||||
- Budget assumption: no explicit token ceiling was supplied. Keep implementation dependency-free (stock Node), avoid repeated full monorepo installs, and keep generated test artifacts under the worktree/main disk.
|
||||
|
||||
## Plan
|
||||
|
||||
1. Update PRD and tracking references.
|
||||
2. Write black-box meta-negative-control first and observe it fail for the missing verifier behavior.
|
||||
3. Implement minimal manifest parser/case runner/mutation detector; observe meta-control succeed.
|
||||
4. Add schema, coverage, provenance, prose marker, compatibility, discovery, deployment identity, and defect-delta tests RED-first.
|
||||
5. Register seven gates with exact cases and run each case.
|
||||
6. Add prospective per-commit replay and bounded provider-evidence reporting.
|
||||
7. Wire unconditional Woodpecker CI and update developer/admin documentation.
|
||||
8. Run situational and baseline verification, independent Codex review, remediate, commit, queue guard, push, PR, coordinator/reviewer handoff.
|
||||
|
||||
## Progress
|
||||
|
||||
- 2026-08-01: Design approved by `mos-remediation`; rulings A-E and source/deployed identity addition incorporated.
|
||||
- 2026-08-01: Isolated worktree created from `origin/main` f65e9ea6; RM-01 f58b3699 verified as ancestor.
|
||||
- 2026-08-01: Git author set to `f10-coder <[email protected]>`; provider issue #1029 created with `MOSAIC_GIT_IDENTITY=f10-coder`.
|
||||
- 2026-08-01: Source/deployed queue-guard SHA-256 observed equal (`19cda2f...`); this observation is not yet an enforced property.
|
||||
|
||||
## Tests and RED-first evidence
|
||||
|
||||
- Meta-negative RED first: `node --test scripts/gate-verify.test.mjs` failed because the absent verifier did not name externally inerted `meta-fixture`.
|
||||
- Structural RED: nine tests failed before implementation for internal mutation, unregistered executable, missing negative control, ownerless delta, unbound criterion/claim, modeled conflict, missing provenance, and deployment drift.
|
||||
- Clause hardening RED: positive-only security criterion and missing registered claim marker both passed incorrectly before validation was added.
|
||||
- CI wiring RED: package script and unconditional Woodpecker step tests both failed before wiring.
|
||||
- History RED: history test failed with missing module before own-tree manifest selection/provider classification was implemented.
|
||||
- `pnpm gate:verify`: exit 0; seven gates each reported `META-NEGATIVE-CONTROL ... observed red`; queue source/deployed drift control observed red; six queue behavior deltas printed as `DEFECT (owner: RM-03)`.
|
||||
- Focused Node tests: 54/54 pass after third-round hardening (36 verifier/wiring plus 18 history/provider tests).
|
||||
- `pnpm typecheck`: pass (45/45 Turbo tasks).
|
||||
- `pnpm lint`: pass (25/25 Turbo tasks).
|
||||
- `pnpm format:check`: pass.
|
||||
- `pnpm test`: repository suites reached 45/46 Turbo tasks; all application/package tests shown passed, then the known host-specific wake assertion aborted exit 97 (`BASH_LINENO convention violated`, #973/D-16). No test was edited or bypassed. CI remains the authoritative full-suite environment.
|
||||
|
||||
## Self-surfaced defect
|
||||
|
||||
The queue guard's `get_state_from_status_json` runs `python3 - <<'PY'` while provider JSON is piped to the shell function. The heredoc owns stdin, so Python never reads provider JSON. Terminal success, no-status, terminal failure, and malformed payloads all classify as `unknown`; the associated fail-open outcomes are recorded under RM-03. No queue-guard source was modified.
|
||||
|
||||
## Independent review remediation
|
||||
|
||||
- Final pre-commit Codex code review found three blockers: a tautological deployment drift control, independently observed rather than combined compatibility cases, and unauthorized edits to orchestrator-owned `TASKS.md`.
|
||||
- Deployment identity now uses one shared file comparator for both live equality and a temporary drifted deployed copy; a test makes that comparator inert and proves the meta-control fails.
|
||||
- Compatibility scenarios now merge referenced fixtures/environments into one isolated construction and execute an exact scenario invocation; a test proves two independently valid fixtures coexist in the combined run.
|
||||
- `TASKS.md` changes were reverted. `docs/remediation/GATE-CLAIMS.md` binds source headings and anchored text without editing orchestrator tracking.
|
||||
- Codex security review found the Bubblewrap replay shared the runner PID namespace. Replay now unshares PID, IPC, and UTS namespaces, and an abuse-case test proves a sibling runner PID is invisible.
|
||||
- Second review found empty reason diagnostics, final-symlink fixture writes, and lifecycle-script mutation of authoritative history files. Must-fail cases now require a reason pattern; writes use no-follow semantics; and replay snapshots every archived file before install and rejects any changed, deleted, or type/mode-shifted source before executing the verifier. Dedicated negative tests cover all three.
|
||||
- Third code review found ambiguous duplicate provider steps and order-sensitive JSON outcome comparison. Provider evidence now requires exactly one `gate-verify` step in the authoritative rerun, and structural equality normalizes object keys. Both regressions have RED-first tests. Third security review reported no findings.
|
||||
- Initial PR pipeline #2177 exposed Woodpecker's shallow boundary: the activation parent object was present but marked shallow, so `merge-base --is-ancestor` correctly refused to infer ancestry. The unconditional gate step now unshallows before ancestry/provenance checks; its wiring test was observed RED before the CI fix.
|
||||
- Pipeline #2178 then proved the unprivileged Docker runner cannot establish Bubblewrap namespaces. A privileged experiment remained uncommitted and was rejected after Codex correctly rated it CRITICAL: PR-controlled code executes before an in-repository sandbox and could directly use the granted capability.
|
||||
- `mos-remediation` and `rev-974` independently ruled Option C. RM02-REQ-10 now retains its original text, restatement, and reason: PR CI verifies only the current tree, unprivileged and fail-closed; isolated own-tree replay is deferred to RM-60/#1031's external pre-execution authority, cross-referenced with RM-59. Future protected post-merge replay is detection with quarantine/revert, never pre-merge prevention.
|
||||
- RED-first boundary test proved the old path executed an inert intermediate verifier. The revised path states adjacent `DOES`/`DOES NOT` claims, validates historical manifest provenance without executing it, and infers no replay success. Direct sandbox tests remain hard-fail; unprivileged CI asserts terminal refusal instead of treating replay as success. Pipelines #2179/#2180/#2181 exposed two runner refusal forms: namespace denial as `spawnSync bwrap` with `error.code=EPERM`, and a test image without Bubblewrap as `error.code=ENOENT`. The replay diagnostic now preserves spawn errors. Parent-generated launcher/entry metadata distinguishes refusal before sandbox entry from child-controlled output; the detector recognizes exact `spawnSync bwrap` provenance for `EPERM`/`EACCES`/`ENOENT` and known namespace-refusal text only when the entry command provably did not run. Focused negative assertions reject unrelated `spawnSync git EPERM`, verifier output that merely says `bwrap ENOENT`, and exact namespace-denial impersonation without provenance or after sandbox entry.
|
||||
- Exact-head independent review at `38f1b249` found one valid diagnostic-masking blocker: canonical verification knew four stable-ID rebinding failures but a thrown stale fixture replacement reached the outer catch first and emitted only the generic error. RED-first reproduction confirmed the canonical path omitted both responsible criterion IDs. Verification now collects labeled failures independently across claims, discovery, deployment, case execution/outcome checks, mutation, and compatibility, preserving structural stable-ID failures alongside the stale-fixture signal. The canonical regression test requires both missing-binding IDs and the generic fixture error.
|
||||
- Exact-head independent review at `9b4d4beb` found two valid blockers. RED-first controls reproduced both: denial-looking child stderr was accepted as sandbox unavailability, and moving meaning/prose criterion IDs to an unrelated type-error case left `gate:verify` green. Bubblewrap execution now emits a parent-generated random entry marker and returns parent-owned launcher/entry metadata; unavailability requires Bubblewrap launcher provenance plus proof entry never ran, so exact denial impersonation from plain or entered-child results is rejected. Criterion objects now declare exact `caseRefs`, checked bidirectionally against case-side `criterionIds`; prose claims declare an exact must-fail `caseRef`. Registered must-fail cases move a criterion binding, remove meaning provenance, and redirect a prose claim, each producing its stable reason. The review freeze was deliberately lifted before remediation.
|
||||
- Option C security review reported no findings. Code review rejected an initial unrelated typecheck binding for the new security criterion. It was replaced with a dedicated registered `privileged-pr-gate` case: the fixture injects a privilege key into the gate step, the wiring control rejects it for that exact reason, and `gate:verify` observes the boundary negative control. Follow-up hardening uses a closed exact gate-step construction, rejects privilege across the entire pipeline, rejects non-canonical/merged YAML keys, and pins the unrestricted PR/main trigger block; quoted/escaped/alias/merge/duplicate/filter bypass tests pass. Final Codex code review approved with no findings.
|
||||
|
||||
- Exact-head review at `83d2ecb2` found four silent-defeat paths. Genuine RED-first tests on the pre-fix code proved: author-controlled HEAD/parent/introduction seams produced no boundary failure; cross-commit duplicate pipeline number 7 returned terminal-success; and misspelling `outputPattern` as `outputPatern` in required/actual left canonical verification green. Fixes derive the seam from Git, validate provider identity globally before subject filtering, and recursively close/type-check nested schemas. New registry criteria `RM02-EVIDENCE-SUBJECT-BINDING`, `RM02-TYPE-STRICT-SCHEMA`, and `RM02-HISTORY-BOUNDARY` are bidirectionally bound to eight registered must-fail controls. The broad nested-object typo table and exact derived-boundary test are regression guards added after implementation, not claimed as RED-first. Pre-commit Codex review then found that global evidence failure was only observed—not failed—when HEAD was the sole prospective commit, and that non-object collection entries threw before normalization. Both were reproduced RED-first, then fixed by one collection validator used before iteration and by per-commit assessment. Follow-up review found collection validation still omitted exactly-one-gate-step cardinality for unrelated subjects; a focused test reproduced terminal-success RED-first, and collection validation now rejects that ambiguity globally. Security review then found present-but-empty outcome patterns were truthy-optional assertion bypasses; a reason-specific test reproduced that they lacked the required schema diagnostic, and present pattern fields now require non-whitespace content.
|
||||
|
||||
- Exact-head review at `32b490a7` found three population-level blockers. Genuine RED-first tests proved a gate change before author-delayed registry introduction fell outside the range, and empty criteria/gates/prose/scenario populations returned zero. History now anchors at the provider target merge-base; delayed introduction is a registered must-fail control. Production verification requires non-empty populations and a hardcoded seven-gate ID/source inventory before quantified checks. D-38/D-40 criteria now quantify over `gateRefs` exactly spanning every registered gate; each gate declares its evidence subject, and population controls mutate evidence subject and comparison type for every gate. The history record states both bootstrap directions: sound against a branch author unable to rewrite main, not sound against compromised/rewritten main, with residual owned by Builds 1-2. Pre-commit review rejected an initial production CLI `--fixture-profile` test relaxation as a vacuity bypass. That flag was removed; synthetic fixtures now use a non-executable test-support runner, while regression tests prove the shipped CLI rejects the flag and production population checks remain mandatory. Follow-up review then proved deleting `gateRefs` skipped population validation; a RED-first loop reproduced all three deletions, and the three general criterion IDs now require the field before exact-span validation.
|
||||
|
||||
## Documentation checklist
|
||||
|
||||
- PRD, developer guide, admin guide, governing claim index, sitemap, plan, and scratchpad updated.
|
||||
- User/API documentation not applicable: no user workflow or API changed.
|
||||
- Independent review documentation check pending rev-974 at the revised exact head.
|
||||
- Canonical documentation remains in-repository; no external publication requested.
|
||||
|
||||
## Rebase onto RM-61
|
||||
|
||||
- Rebasing `f9746b23` onto main `f4fd5967` completed mechanically with no conflicts.
|
||||
- The first post-rebase `pnpm gate:verify` correctly failed because the old activation seam `f65e9ea6` made the newly merged pre-registry RM-61 commit part of prospective history even though that commit predates the registry. An initial manifest update to `f4fd5967` had the right value but retained an author-controlled mechanism. Independent mutation proved HEAD, HEAD's parent, and the introduction commit could each make history coverage vacuous or partial. The manifest field is now forbidden; the verifier derives the boundary as the parent of the first first-parent registry-introduction commit, and registered must-fail controls reject all three unsafe candidates.
|
||||
|
||||
## Risks/blockers
|
||||
|
||||
- Current queue guard intentionally has required-versus-actual deltas owned by RM-03.
|
||||
- Provider CI cannot report the currently executing pipeline as terminal success; current-commit evidence must be labeled pending and becomes historical current-tree evidence only after provider completion.
|
||||
- Isolated per-commit execution requires RM-60/#1031. Until that external authority exists, no replay success is claimed. A future protected post-merge failure requires quarantine/revert.
|
||||
- CI containers may not expose the operator-home deployed queue guard. In that layer the verifier checks the pinned observed digest and reports live identity unavailable under RM-04; it does not infer live equality.
|
||||
|
||||
## coder-mos2 remediation — fourth round
|
||||
|
||||
- Coordinator correction loaded: Blocker 2 requires a genuine shrink-both control against an independent inventory baseline; Blocker 3 requires evidence-side subjects consumed against gate definitions for every gate; anchor-dependent history provenance must be removed rather than relabelled because PR lifecycle code makes all local git state untrustworthy before verification.
|
||||
- Boundary to preserve in both directions: this repository layer DOES verify current-tree registered cases, independently anchored inventory shape, evidence subject consumption, and an enforced absence of history-provenance claims. It DOES NOT establish history provenance at all. RM-60 owns the provider-controlled/protected pre-execution boundary required for that property; no local ref, config, URL, constant, or author-positioned path is treated as an anchor.
|
||||
- TDD plan: add three genuine red-first controls before implementation: shrink verifier inventory plus manifest together; mutate evidence-side subject for every gate; and reject the currently enabled history verifier/import/report path. Existing controls retained as regression guards and labelled honestly.
|
||||
- Identity observation before first commit: `git var GIT_AUTHOR_IDENT` returned `coder-mos2 <[email protected]>` using process-scoped author/committer variables; shared repository config was not modified.
|
||||
- Genuine RED-first evidence: `node --test scripts/gate-remediation.test.mjs` produced 0/3 passing on frozen head `fbb61912`: source+manifest shrink exited zero, no evidence-side case subjects existed, and the production verifier still imported/invoked history verification. These were failures for the three stated blocker reasons, not regression guards.
|
||||
- Sixth mutation found before review: after the first baseline implementation, shrinking the new baseline and manifest together while leaving the verifier inventory unchanged still exited zero. A new test observed that attack RED first. Inventory equality diagnostics are now bidirectional, and the registered shrink control attacks both source+manifest and baseline+manifest pairs.
|
||||
- Regression guards retained and honestly labelled: manifest-only shrink, exact `gateRefs` span, production fixture-profile rejection, and broad nested schema checks already blocked before this round.
|
||||
- Current focused evidence before independent review: remediation controls 4/4; verifier/wiring/remediation suite 39/39; canonical `pnpm gate:verify` exits zero while reporting six RM-03-owned `DEFECT` deltas and all seven gate meta-negative controls observed red.
|
||||
- Independent Codex code review requested changes on two valid blockers. First, the evidence population control called the subject helper directly rather than traversing production result consumption. It now creates one lightweight executed fixture per required gate, invokes the real `verifyRegistry` path, and fails to observe rejection if the production consumer is removed; a regression mutation proves that coupling. Second, a lexical history blacklist overclaimed structural incapacity. Production output now passes through a closed current-tree observation renderer with no history/ancestry/lineage success class; the exclusion control tests three alternate success wordings plus exact production renderer wiring, and the registered must-fail fixture adds a prohibited class.
|
||||
- Codex review test attempts were unrunnable in its read-only sandbox (`EROFS`/`EPERM`); the reviewer disclosed this rather than substituting a passing variant. Local writable-worktree tests remain the runnable evidence.
|
||||
- Renderer remediation: every non-error production observation and final summary now routes through the closed current-tree output renderer. The exclusion control additionally requires one stdout sink, exactly two `GATE VERIFY FAILED` stderr sinks, no console sinks, and renderer use for both per-observation and final-summary paths. Regression attacks cover allowlisted history/ancestry/lineage wording, writer assertion removal, direct final-success stdout, and direct success stderr; focused suite 41/41, `pnpm gate:verify` green with six declared RM-03 deltas, and format check green.
|
||||
- Coordinator ruling after CWE-353: retain the same-checkout inventory comparison but narrow its claim. Canonical current boundary: “Detects accidental and incompetent inventory drift within a checkout; does NOT survive an adversary who rewrites baseline, manifest, and verifier consistently — that guarantee requires RM-60's external boundary.” This supersedes the earlier “independent/anchored” wording in the historical entries above; those entries remain as append-only evidence, not current claims.
|
||||
- Genuine RED-first overclaim control: before claim remediation, `node scripts/gate-inventory-claim-control.mjs` exited 84 and named every artifact missing the narrowed boundary plus each current overclaim. The registered `inventory-claim-overstatement` case now rewrites the baseline purpose to a protection claim and must exit 84 with `INVENTORY_CLAIM_OVERSTATED`. Existing shrink-pair controls remain regression guards for the same-checkout drift property, not adversarial-integrity claims.
|
||||
- Follow-up code review found a genuine propagation/control gap: `docs/PRD.md` still said “independent seven-gate baseline,” and the first forbidden regex did not match the intervening qualifier. The reviewer observed the control green against that overclaim. The PRD now says same-checkout, the detector rejects `independent … baseline` across bounded same-line qualifiers, the registered mutation uses the exact qualified wording, and a focused behavioral test requires exit 84 for it. Follow-up security review passed with risk `none` and explicitly accepted the narrowed RM-60 boundary.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"purpose": "Detects accidental and incompetent inventory drift within a checkout; does NOT survive an adversary who rewrites baseline, manifest, and verifier consistently — that guarantee requires RM-60's external boundary.",
|
||||
"gates": [
|
||||
{
|
||||
"id": "quality-typecheck",
|
||||
"source": "package.json"
|
||||
},
|
||||
{
|
||||
"id": "quality-lint",
|
||||
"source": "package.json"
|
||||
},
|
||||
{
|
||||
"id": "quality-format",
|
||||
"source": "package.json"
|
||||
},
|
||||
{
|
||||
"id": "checkout-preflight",
|
||||
"source": "scripts/preflight.mjs"
|
||||
},
|
||||
{
|
||||
"id": "ci-queue-wait",
|
||||
"source": "packages/mosaic/framework/tools/git/ci-queue-wait.sh"
|
||||
},
|
||||
{
|
||||
"id": "hook-pre-commit",
|
||||
"source": ".husky/pre-commit"
|
||||
},
|
||||
{
|
||||
"id": "hook-pre-push",
|
||||
"source": ".husky/pre-push"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -6,6 +6,7 @@
|
||||
"build": "turbo run build",
|
||||
"dev": "turbo run dev",
|
||||
"lint": "turbo run lint",
|
||||
"gate:verify": "node scripts/gate-verify.mjs",
|
||||
"preflight": "node scripts/preflight.mjs",
|
||||
"clean:generated": "node scripts/clean-generated.mjs",
|
||||
"typecheck": "pnpm preflight && turbo run typecheck",
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
|
||||
import { verifyRegistry } from './gate-verify.mjs';
|
||||
|
||||
const root = process.cwd();
|
||||
const manifest = JSON.parse(await readFile(path.join(root, 'gates/gates.manifest.json'), 'utf8'));
|
||||
manifest.gateRoots = ['.mosaic-empty-gate-root'];
|
||||
manifest.criteria = [];
|
||||
manifest.gates = [];
|
||||
manifest.proseClaims = [];
|
||||
manifest.compatibilityScenarios = [];
|
||||
const directory = await mkdtemp(path.join(os.tmpdir(), 'gate-empty-population-'));
|
||||
try {
|
||||
const manifestPath = path.join(directory, 'manifest.json');
|
||||
await writeFile(manifestPath, `${JSON.stringify(manifest)}\n`);
|
||||
const result = await verifyRegistry({
|
||||
root,
|
||||
manifest: manifestPath,
|
||||
structureOnly: true,
|
||||
fixtureProfile: false,
|
||||
});
|
||||
const required = ['criteria', 'gates', 'proseClaims', 'compatibilityScenarios'];
|
||||
const missing = required.filter(
|
||||
(population) =>
|
||||
!result.failures.some((failure) =>
|
||||
failure.includes(`${population} population must be non-empty and anchored`),
|
||||
),
|
||||
);
|
||||
if (missing.length > 0) {
|
||||
process.stdout.write(`empty populations were not rejected: ${missing.join(', ')}\n`);
|
||||
process.exitCode = 0;
|
||||
} else {
|
||||
process.stderr.write(
|
||||
'empty universally quantified registry populations rejected before evaluation\n',
|
||||
);
|
||||
process.exitCode = 1;
|
||||
}
|
||||
} finally {
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
|
||||
import { assertCurrentTreeObservation } from './gate-verify.mjs';
|
||||
|
||||
const prohibitedReports = [
|
||||
'HISTORY PROVENANCE VERIFIED local-ref',
|
||||
'COMMIT ANCESTRY VERIFIED origin/main',
|
||||
'PROVIDER LINEAGE SUCCESS pipeline-7',
|
||||
];
|
||||
const accepted = prohibitedReports.filter((report) => {
|
||||
try {
|
||||
assertCurrentTreeObservation(report);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
});
|
||||
|
||||
const verifier = await readFile(path.join(process.cwd(), 'scripts', 'gate-verify.mjs'), 'utf8');
|
||||
const stdoutWrites = [...verifier.matchAll(/process\.stdout\.write\s*\(/g)].length;
|
||||
const stderrWrites = [...verifier.matchAll(/process\.stderr\.write\s*\(/g)].length;
|
||||
const gatedErrorWrites = [...verifier.matchAll(/process\.stderr\.write\(`GATE VERIFY FAILED:/g)]
|
||||
.length;
|
||||
const currentTreeWriterCalls = [...verifier.matchAll(/writeCurrentTreeOutput\s*\(/g)].length;
|
||||
const productionRendererWired =
|
||||
stdoutWrites === 1 &&
|
||||
stderrWrites === 2 &&
|
||||
gatedErrorWrites === 2 &&
|
||||
currentTreeWriterCalls === 3 &&
|
||||
/function writeCurrentTreeOutput\(output\) \{\s*assertCurrentTreeObservation\(output\);\s*process\.stdout\.write/s.test(
|
||||
verifier,
|
||||
) &&
|
||||
/for \(const observation of observations\) \{\s*writeCurrentTreeOutput\(observation\);\s*\}/s.test(
|
||||
verifier,
|
||||
) &&
|
||||
/writeCurrentTreeOutput\(\s*`REGISTRY SUMMARY open behavior deltas:/s.test(verifier) &&
|
||||
!/\bconsole\.(?:log|info|debug|error|warn)\s*\(/.test(verifier);
|
||||
|
||||
if (accepted.length > 0 || !productionRendererWired) {
|
||||
process.stderr.write(
|
||||
`HISTORY_PROVENANCE_FORBIDDEN: closed current-tree observation renderer rejected=${prohibitedReports.length - accepted.length}/${prohibitedReports.length} production-wired=${productionRendererWired}\n`,
|
||||
);
|
||||
process.exit(79);
|
||||
}
|
||||
process.stdout.write('history provenance reporting capability is absent; owner RM-60\n');
|
||||
@@ -0,0 +1,44 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
|
||||
const boundary =
|
||||
"Detects accidental and incompetent inventory drift within a checkout; does NOT survive an adversary who rewrites baseline, manifest, and verifier consistently — that guarantee requires RM-60's external boundary.";
|
||||
const artifacts = [
|
||||
'gates/required-gates.baseline.json',
|
||||
'gates/gates.manifest.json',
|
||||
'docs/PRD.md',
|
||||
'docs/ADMIN-GUIDE/quality-gate-registry.md',
|
||||
'docs/DEVELOPER-GUIDE/quality-gate-registry.md',
|
||||
'docs/remediation/GATE-CLAIMS.md',
|
||||
'docs/plans/2026-08-01-rm-02-gate-registry.md',
|
||||
'docs/SITEMAP.md',
|
||||
];
|
||||
const forbidden = [
|
||||
/\bindependent\b[^\n.]{0,80}\bbaseline\b/i,
|
||||
/independently baselined/i,
|
||||
/independently anchored inventory/i,
|
||||
/protected (?:inventory )?(?:anchor|baseline)/i,
|
||||
/inventory (?:anchor|anchored)/i,
|
||||
];
|
||||
const failures = [];
|
||||
for (const relativePath of artifacts) {
|
||||
const contents = await readFile(path.join(process.cwd(), relativePath), 'utf8');
|
||||
const claims =
|
||||
relativePath === 'gates/gates.manifest.json'
|
||||
? JSON.parse(contents)
|
||||
.criteria.map((criterion) => criterion.currentText)
|
||||
.join('\n')
|
||||
: contents;
|
||||
if (!claims.includes(boundary)) failures.push(`${relativePath}: narrowed boundary is missing`);
|
||||
const overclaim = forbidden.find((pattern) => pattern.test(claims));
|
||||
if (overclaim) failures.push(`${relativePath}: inventory protection is overstated`);
|
||||
}
|
||||
if (failures.length > 0) {
|
||||
for (const failure of failures) {
|
||||
process.stderr.write(`INVENTORY_CLAIM_OVERSTATED: ${failure}\n`);
|
||||
}
|
||||
process.exit(84);
|
||||
}
|
||||
process.stdout.write('inventory drift boundary is stated in both directions; owner RM-60\n');
|
||||
@@ -0,0 +1,119 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import assert from 'node:assert/strict';
|
||||
import { copyFile, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
|
||||
const root = process.cwd();
|
||||
const removedGateId = 'hook-pre-push';
|
||||
const inventoryEntry = " ['hook-pre-push', '.husky/pre-push'],\n";
|
||||
|
||||
function shrinkManifest(manifest) {
|
||||
const removedGate = manifest.gates.find((gate) => gate.id === removedGateId);
|
||||
assert.ok(removedGate);
|
||||
const removedCaseRefs = new Set(
|
||||
removedGate.cases.map((gateCase) => `${removedGateId}/${gateCase.id}`),
|
||||
);
|
||||
const removedCriterionIds = new Set(
|
||||
manifest.criteria
|
||||
.filter(
|
||||
(criterion) =>
|
||||
criterion.caseRefs.length > 0 &&
|
||||
criterion.caseRefs.every((caseRef) => removedCaseRefs.has(caseRef)),
|
||||
)
|
||||
.map((criterion) => criterion.id),
|
||||
);
|
||||
manifest.gates = manifest.gates.filter((gate) => gate.id !== removedGateId);
|
||||
manifest.criteria = manifest.criteria
|
||||
.filter((criterion) => !removedCriterionIds.has(criterion.id))
|
||||
.map((criterion) => ({
|
||||
...criterion,
|
||||
caseRefs: criterion.caseRefs.filter((caseRef) => !removedCaseRefs.has(caseRef)),
|
||||
...(criterion.gateRefs
|
||||
? { gateRefs: criterion.gateRefs.filter((gateId) => gateId !== removedGateId) }
|
||||
: {}),
|
||||
}));
|
||||
manifest.proseClaims = manifest.proseClaims.filter(
|
||||
(claim) => !removedCriterionIds.has(claim.criterionId) && !removedCaseRefs.has(claim.caseRef),
|
||||
);
|
||||
manifest.compatibilityScenarios = manifest.compatibilityScenarios
|
||||
.map((scenario) => ({
|
||||
...scenario,
|
||||
caseRefs: scenario.caseRefs.filter((caseRef) => !removedCaseRefs.has(caseRef)),
|
||||
}))
|
||||
.filter((scenario) => scenario.caseRefs.length > 0);
|
||||
for (const gate of manifest.gates) {
|
||||
for (const gateCase of gate.cases) {
|
||||
gateCase.criterionIds = gateCase.criterionIds.filter(
|
||||
(criterionId) => !removedCriterionIds.has(criterionId),
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function attack(mode) {
|
||||
const fixture = await mkdtemp(path.join(os.tmpdir(), `gate-inventory-${mode}-`));
|
||||
try {
|
||||
await mkdir(path.join(fixture, 'scripts'), { recursive: true });
|
||||
await mkdir(path.join(fixture, 'gates'), { recursive: true });
|
||||
const source = await readFile(path.join(root, 'scripts', 'gate-verify.mjs'), 'utf8');
|
||||
if (source.split(inventoryEntry).length - 1 !== 1) {
|
||||
throw new Error('source inventory fixture drifted');
|
||||
}
|
||||
await writeFile(
|
||||
path.join(fixture, 'scripts', 'gate-verify.mjs'),
|
||||
mode === 'source-manifest' ? source.replace(inventoryEntry, '') : source,
|
||||
);
|
||||
const baseline = JSON.parse(
|
||||
await readFile(path.join(root, 'gates', 'required-gates.baseline.json'), 'utf8'),
|
||||
);
|
||||
if (mode === 'baseline-manifest') {
|
||||
baseline.gates = baseline.gates.filter((gate) => gate.id !== removedGateId);
|
||||
}
|
||||
await writeFile(
|
||||
path.join(fixture, 'gates', 'required-gates.baseline.json'),
|
||||
`${JSON.stringify(baseline)}\n`,
|
||||
);
|
||||
const manifest = JSON.parse(
|
||||
await readFile(path.join(root, 'gates', 'gates.manifest.json'), 'utf8'),
|
||||
);
|
||||
shrinkManifest(manifest);
|
||||
await writeFile(
|
||||
path.join(fixture, 'gates', 'gates.manifest.json'),
|
||||
`${JSON.stringify(manifest)}\n`,
|
||||
);
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[
|
||||
path.join(fixture, 'scripts', 'gate-verify.mjs'),
|
||||
'--root',
|
||||
fixture,
|
||||
'--manifest',
|
||||
'gates/gates.manifest.json',
|
||||
'--structure-only',
|
||||
],
|
||||
{ cwd: fixture, encoding: 'utf8' },
|
||||
);
|
||||
const combined = `${result.stdout ?? ''}\n${result.stderr ?? ''}`;
|
||||
return (
|
||||
result.status !== 0 &&
|
||||
new RegExp(`(?:baseline|verifier inventory).*${removedGateId}`, 'i').test(combined)
|
||||
);
|
||||
} finally {
|
||||
await rm(fixture, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
const sourceManifestRejected = await attack('source-manifest');
|
||||
const baselineManifestRejected = await attack('baseline-manifest');
|
||||
if (sourceManifestRejected && baselineManifestRejected) {
|
||||
process.stderr.write(
|
||||
'INVENTORY_SHRINK_REJECTED: source+manifest and baseline+manifest shrink attacks detected\n',
|
||||
);
|
||||
process.exit(83);
|
||||
}
|
||||
process.stdout.write(
|
||||
`inventory shrink attack escaped: source-manifest=${sourceManifestRejected} baseline-manifest=${baselineManifestRejected}\n`,
|
||||
);
|
||||
@@ -0,0 +1,138 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { chmod, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
|
||||
import { verifyRegistry } from './gate-verify.mjs';
|
||||
|
||||
const mode = process.argv[2];
|
||||
const root = process.cwd();
|
||||
const source = JSON.parse(await readFile(path.join(root, 'gates/gates.manifest.json'), 'utf8'));
|
||||
const expectedGateIds = source.gates.map((gate) => gate.id);
|
||||
if (expectedGateIds.length === 0) {
|
||||
process.stderr.write('gate population control requires a non-empty anchored inventory\n');
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
async function rejectedForEveryGate(mutate, diagnostic) {
|
||||
for (const gateId of expectedGateIds) {
|
||||
const manifest = structuredClone(source);
|
||||
const gate = manifest.gates.find((candidate) => candidate.id === gateId);
|
||||
mutate(gate);
|
||||
const directory = await mkdtemp(path.join(os.tmpdir(), 'gate-population-control-'));
|
||||
const manifestPath = path.join(directory, 'manifest.json');
|
||||
try {
|
||||
await writeFile(manifestPath, `${JSON.stringify(manifest)}\n`);
|
||||
const result = await verifyRegistry({
|
||||
root,
|
||||
manifest: manifestPath,
|
||||
structureOnly: true,
|
||||
fixtureProfile: false,
|
||||
});
|
||||
if (!result.failures.some((failure) => diagnostic(failure, gateId))) return false;
|
||||
} finally {
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
let rejected;
|
||||
if (mode === 'evidence-subject') {
|
||||
rejected = true;
|
||||
for (const gateId of expectedGateIds) {
|
||||
const directory = await mkdtemp(path.join(os.tmpdir(), 'gate-evidence-consumption-'));
|
||||
try {
|
||||
await mkdir(path.join(directory, 'gates'), { recursive: true });
|
||||
const probe = path.join(directory, 'gates', 'probe.sh');
|
||||
await writeFile(probe, '#!/bin/sh\necho EVIDENCE_PROBE >&2\nexit 7\n');
|
||||
await chmod(probe, 0o755);
|
||||
const manifest = {
|
||||
schemaVersion: 1,
|
||||
gateRoots: ['gates'],
|
||||
governingClaimFiles: [],
|
||||
coverageBoundary: { included: ['evidence fixture'], excluded: [], trackedBy: 'RM-02' },
|
||||
criteria: [
|
||||
{
|
||||
id: 'EVIDENCE-CONSUMPTION',
|
||||
originalText: 'Consumed evidence stays bound to its gate.',
|
||||
currentText: 'Consumed evidence stays bound to its gate.',
|
||||
claimType: 'integrity',
|
||||
source: 'gate-population-control',
|
||||
meaningChanges: [],
|
||||
caseRefs: [`${gateId}/probe`],
|
||||
},
|
||||
],
|
||||
proseClaims: [],
|
||||
compatibilityScenarios: [],
|
||||
gates: [
|
||||
{
|
||||
id: gateId,
|
||||
source: 'gates/probe.sh',
|
||||
invocation: ['gates/probe.sh'],
|
||||
deployment: { kind: 'none', reason: 'population fixture' },
|
||||
inertMutation: {
|
||||
file: 'gates/probe.sh',
|
||||
find: 'exit 7',
|
||||
replace: 'exit 0',
|
||||
caseId: 'probe',
|
||||
expected: { exitCode: 0 },
|
||||
},
|
||||
cases: [
|
||||
{
|
||||
id: 'probe',
|
||||
criterionIds: ['EVIDENCE-CONSUMPTION'],
|
||||
mustFail: true,
|
||||
required: { exitCode: 7 },
|
||||
actual: { exitCode: 7 },
|
||||
evidence: { subject: 'different-gate-subject' },
|
||||
reasonPattern: 'EVIDENCE_PROBE',
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
const manifestPath = path.join(directory, 'gates', 'gates.manifest.json');
|
||||
await writeFile(manifestPath, `${JSON.stringify(manifest)}\n`);
|
||||
const result = await verifyRegistry({
|
||||
root: directory,
|
||||
manifest: manifestPath,
|
||||
structureOnly: false,
|
||||
fixtureProfile: true,
|
||||
});
|
||||
if (
|
||||
!result.failures.some(
|
||||
(failure) =>
|
||||
failure.includes(`gate ${gateId}: consumed evidence subject`) &&
|
||||
failure.includes('does not match gate definition'),
|
||||
)
|
||||
) {
|
||||
rejected = false;
|
||||
break;
|
||||
}
|
||||
} finally {
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
} else if (mode === 'type-strict') {
|
||||
rejected = await rejectedForEveryGate(
|
||||
(gate) => {
|
||||
gate.cases[0].actual.exitCode = '0';
|
||||
},
|
||||
(failure, gateId) =>
|
||||
failure.includes(
|
||||
`${gateId}/${source.gates.find((gate) => gate.id === gateId).cases[0].id}.actual.exitCode`,
|
||||
) && failure.includes('expected an integer'),
|
||||
);
|
||||
} else {
|
||||
process.stderr.write(`unknown gate population control ${String(mode)}\n`);
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
if (!rejected) {
|
||||
process.stdout.write(`${mode} population control did not reject every registered gate\n`);
|
||||
process.exit(0);
|
||||
}
|
||||
process.stderr.write(`${mode} population control rejected every registered gate\n`);
|
||||
process.exit(1);
|
||||
@@ -0,0 +1,283 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { copyFile, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import test from 'node:test';
|
||||
|
||||
const root = process.cwd();
|
||||
const verifierPath = path.join(root, 'scripts', 'gate-verify.mjs');
|
||||
const manifestPath = path.join(root, 'gates', 'gates.manifest.json');
|
||||
const requiredGateId = 'hook-pre-push';
|
||||
|
||||
function output(result) {
|
||||
return `${result.stdout ?? ''}\n${result.stderr ?? ''}`;
|
||||
}
|
||||
|
||||
function shrinkManifest(manifest, removedGateId) {
|
||||
const removedGate = manifest.gates.find((gate) => gate.id === removedGateId);
|
||||
assert.ok(removedGate, `fixture gate ${removedGateId} must exist`);
|
||||
const removedCaseRefs = new Set(
|
||||
removedGate.cases.map((gateCase) => `${removedGateId}/${gateCase.id}`),
|
||||
);
|
||||
const removedCriterionIds = new Set(
|
||||
manifest.criteria
|
||||
.filter(
|
||||
(criterion) =>
|
||||
criterion.caseRefs.length > 0 &&
|
||||
criterion.caseRefs.every((caseRef) => removedCaseRefs.has(caseRef)),
|
||||
)
|
||||
.map((criterion) => criterion.id),
|
||||
);
|
||||
|
||||
manifest.gates = manifest.gates.filter((gate) => gate.id !== removedGateId);
|
||||
manifest.criteria = manifest.criteria
|
||||
.filter((criterion) => !removedCriterionIds.has(criterion.id))
|
||||
.map((criterion) => ({
|
||||
...criterion,
|
||||
caseRefs: criterion.caseRefs.filter((caseRef) => !removedCaseRefs.has(caseRef)),
|
||||
...(criterion.gateRefs
|
||||
? { gateRefs: criterion.gateRefs.filter((gateId) => gateId !== removedGateId) }
|
||||
: {}),
|
||||
}));
|
||||
manifest.proseClaims = manifest.proseClaims.filter(
|
||||
(claim) => !removedCriterionIds.has(claim.criterionId) && !removedCaseRefs.has(claim.caseRef),
|
||||
);
|
||||
manifest.compatibilityScenarios = manifest.compatibilityScenarios
|
||||
.map((scenario) => ({
|
||||
...scenario,
|
||||
caseRefs: scenario.caseRefs.filter((caseRef) => !removedCaseRefs.has(caseRef)),
|
||||
}))
|
||||
.filter((scenario) => scenario.caseRefs.length > 0);
|
||||
for (const gate of manifest.gates) {
|
||||
for (const gateCase of gate.cases) {
|
||||
gateCase.criterionIds = gateCase.criterionIds.filter(
|
||||
(criterionId) => !removedCriterionIds.has(criterionId),
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
test('shrinking the verifier inventory and manifest together is rejected by a same-checkout baseline', async () => {
|
||||
const fixture = await mkdtemp(path.join(os.tmpdir(), 'rm02-shrink-both-'));
|
||||
try {
|
||||
await mkdir(path.join(fixture, 'scripts'), { recursive: true });
|
||||
await mkdir(path.join(fixture, 'gates'), { recursive: true });
|
||||
const verifier = await readFile(verifierPath, 'utf8');
|
||||
const inventoryEntry = " ['hook-pre-push', '.husky/pre-push'],\n";
|
||||
assert.equal(verifier.split(inventoryEntry).length - 1, 1, 'source inventory fixture drifted');
|
||||
await writeFile(
|
||||
path.join(fixture, 'scripts', 'gate-verify.mjs'),
|
||||
verifier.replace(inventoryEntry, ''),
|
||||
);
|
||||
await copyFile(
|
||||
path.join(root, 'gates', 'required-gates.baseline.json'),
|
||||
path.join(fixture, 'gates', 'required-gates.baseline.json'),
|
||||
);
|
||||
const manifest = JSON.parse(await readFile(manifestPath, 'utf8'));
|
||||
shrinkManifest(manifest, requiredGateId);
|
||||
await writeFile(
|
||||
path.join(fixture, 'gates', 'gates.manifest.json'),
|
||||
`${JSON.stringify(manifest)}\n`,
|
||||
);
|
||||
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[
|
||||
path.join(fixture, 'scripts', 'gate-verify.mjs'),
|
||||
'--root',
|
||||
fixture,
|
||||
'--manifest',
|
||||
'gates/gates.manifest.json',
|
||||
'--structure-only',
|
||||
],
|
||||
{ cwd: fixture, encoding: 'utf8' },
|
||||
);
|
||||
assert.notEqual(result.status, 0, 'shrinking source anchor and manifest together must go red');
|
||||
assert.match(output(result), /same-checkout required-gate baseline.*hook-pre-push/i);
|
||||
} finally {
|
||||
await rm(fixture, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('shrinking the same-checkout baseline and manifest together is rejected by verifier inventory', async () => {
|
||||
const fixture = await mkdtemp(path.join(os.tmpdir(), 'rm02-shrink-baseline-manifest-'));
|
||||
try {
|
||||
await mkdir(path.join(fixture, 'scripts'), { recursive: true });
|
||||
await mkdir(path.join(fixture, 'gates'), { recursive: true });
|
||||
await copyFile(verifierPath, path.join(fixture, 'scripts', 'gate-verify.mjs'));
|
||||
const baseline = JSON.parse(
|
||||
await readFile(path.join(root, 'gates', 'required-gates.baseline.json'), 'utf8'),
|
||||
);
|
||||
baseline.gates = baseline.gates.filter((gate) => gate.id !== requiredGateId);
|
||||
await writeFile(
|
||||
path.join(fixture, 'gates', 'required-gates.baseline.json'),
|
||||
`${JSON.stringify(baseline)}\n`,
|
||||
);
|
||||
const manifest = JSON.parse(await readFile(manifestPath, 'utf8'));
|
||||
shrinkManifest(manifest, requiredGateId);
|
||||
await writeFile(
|
||||
path.join(fixture, 'gates', 'gates.manifest.json'),
|
||||
`${JSON.stringify(manifest)}\n`,
|
||||
);
|
||||
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[
|
||||
path.join(fixture, 'scripts', 'gate-verify.mjs'),
|
||||
'--root',
|
||||
fixture,
|
||||
'--manifest',
|
||||
'gates/gates.manifest.json',
|
||||
'--structure-only',
|
||||
],
|
||||
{ cwd: fixture, encoding: 'utf8' },
|
||||
);
|
||||
assert.notEqual(result.status, 0, 'shrinking baseline and manifest together must go red');
|
||||
assert.match(output(result), /verifier inventory.*hook-pre-push.*baseline/i);
|
||||
} finally {
|
||||
await rm(fixture, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('every gate carries an evidence-side subject distinct from its definition', async () => {
|
||||
const manifest = JSON.parse(await readFile(manifestPath, 'utf8'));
|
||||
for (const gate of manifest.gates) {
|
||||
assert.ok(gate.cases.length > 0, `${gate.id} must have consumable evidence`);
|
||||
for (const gateCase of gate.cases) {
|
||||
assert.equal(
|
||||
gateCase.evidence?.subject,
|
||||
gate.id,
|
||||
`${gate.id}/${gateCase.id} must source its subject from the evidence record`,
|
||||
);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test('inventory claim control rejects qualified independence wording', async () => {
|
||||
const fixture = await mkdtemp(path.join(os.tmpdir(), 'rm02-inventory-overclaim-'));
|
||||
const artifacts = [
|
||||
'gates/required-gates.baseline.json',
|
||||
'gates/gates.manifest.json',
|
||||
'docs/PRD.md',
|
||||
'docs/ADMIN-GUIDE/quality-gate-registry.md',
|
||||
'docs/DEVELOPER-GUIDE/quality-gate-registry.md',
|
||||
'docs/remediation/GATE-CLAIMS.md',
|
||||
'docs/plans/2026-08-01-rm-02-gate-registry.md',
|
||||
'docs/SITEMAP.md',
|
||||
];
|
||||
try {
|
||||
await mkdir(path.join(fixture, 'scripts'), { recursive: true });
|
||||
await copyFile(
|
||||
path.join(root, 'scripts', 'gate-inventory-claim-control.mjs'),
|
||||
path.join(fixture, 'scripts', 'gate-inventory-claim-control.mjs'),
|
||||
);
|
||||
for (const relativePath of artifacts) {
|
||||
const target = path.join(fixture, relativePath);
|
||||
await mkdir(path.dirname(target), { recursive: true });
|
||||
await copyFile(path.join(root, relativePath), target);
|
||||
}
|
||||
const prd = path.join(fixture, 'docs', 'PRD.md');
|
||||
await writeFile(
|
||||
prd,
|
||||
`${await readFile(prd, 'utf8')}\nThis provides an independent seven-gate baseline comparison.\n`,
|
||||
);
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[path.join(fixture, 'scripts', 'gate-inventory-claim-control.mjs')],
|
||||
{ cwd: fixture, encoding: 'utf8' },
|
||||
);
|
||||
assert.equal(result.status, 84, output(result));
|
||||
assert.match(output(result), /INVENTORY_CLAIM_OVERSTATED.*docs\/PRD\.md/i);
|
||||
} finally {
|
||||
await rm(fixture, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('evidence population control depends on production result consumption wiring', async () => {
|
||||
const fixture = await mkdtemp(path.join(os.tmpdir(), 'rm02-evidence-consumer-inert-'));
|
||||
try {
|
||||
await mkdir(path.join(fixture, 'scripts'), { recursive: true });
|
||||
await mkdir(path.join(fixture, 'gates'), { recursive: true });
|
||||
const verifier = await readFile(verifierPath, 'utf8');
|
||||
const consumer = ` const subjectFailure = consumeEvidenceSubject(gate, result.evidence);\n if (subjectFailure) failures.push(subjectFailure);\n`;
|
||||
assert.equal(verifier.split(consumer).length - 1, 1, 'consumer fixture drifted');
|
||||
await writeFile(
|
||||
path.join(fixture, 'scripts', 'gate-verify.mjs'),
|
||||
verifier.replace(consumer, ''),
|
||||
);
|
||||
await copyFile(
|
||||
path.join(root, 'scripts', 'gate-population-control.mjs'),
|
||||
path.join(fixture, 'scripts', 'gate-population-control.mjs'),
|
||||
);
|
||||
await copyFile(manifestPath, path.join(fixture, 'gates', 'gates.manifest.json'));
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[path.join(fixture, 'scripts', 'gate-population-control.mjs'), 'evidence-subject'],
|
||||
{ cwd: fixture, encoding: 'utf8' },
|
||||
);
|
||||
assert.equal(result.status, 0, output(result));
|
||||
assert.match(output(result), /did not reject every registered gate/i);
|
||||
} finally {
|
||||
await rm(fixture, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('production verification has a closed current-tree observation renderer', async () => {
|
||||
const verifier = await readFile(verifierPath, 'utf8');
|
||||
assert.doesNotMatch(verifier, /from ['"]\.\/gate-history\.mjs['"]/);
|
||||
assert.doesNotMatch(verifier, /\bverifyHistory\s*\(/);
|
||||
assert.doesNotMatch(verifier, /history[-_ ]?provenance/i);
|
||||
const attacks = [
|
||||
[
|
||||
'history wording',
|
||||
' /^META-NEGATIVE-CONTROL /,',
|
||||
' /^HISTORY PROVENANCE VERIFIED /,\n /^META-NEGATIVE-CONTROL /,',
|
||||
],
|
||||
[
|
||||
'renamed ancestry wording',
|
||||
' /^META-NEGATIVE-CONTROL /,',
|
||||
' /^COMMIT ANCESTRY VERIFIED /,\n /^META-NEGATIVE-CONTROL /,',
|
||||
],
|
||||
[
|
||||
'provider lineage wording',
|
||||
' /^META-NEGATIVE-CONTROL /,',
|
||||
' /^PROVIDER LINEAGE SUCCESS /,\n /^META-NEGATIVE-CONTROL /,',
|
||||
],
|
||||
['writer assertion bypass', ' assertCurrentTreeObservation(output);\n', ''],
|
||||
[
|
||||
'final success stdout bypass',
|
||||
' writeCurrentTreeOutput(\n `REGISTRY SUMMARY open behavior deltas: ${defects}; required-behavior conformance is not asserted while deltas remain`,\n );',
|
||||
" process.stdout.write('COMMIT ANCESTRY VERIFIED\\n');",
|
||||
],
|
||||
[
|
||||
'direct success stderr bypass',
|
||||
" const defects = observations.filter((line) => line.startsWith('DEFECT ')).length;",
|
||||
" process.stderr.write('PROVIDER LINEAGE SUCCESS\\n');\n const defects = observations.filter((line) => line.startsWith('DEFECT ')).length;",
|
||||
],
|
||||
];
|
||||
for (const [name, find, replace] of attacks) {
|
||||
const fixture = await mkdtemp(path.join(os.tmpdir(), 'rm02-history-renderer-'));
|
||||
try {
|
||||
await mkdir(path.join(fixture, 'scripts'), { recursive: true });
|
||||
assert.equal(verifier.split(find).length - 1, 1, `${name}: fixture drifted`);
|
||||
await writeFile(
|
||||
path.join(fixture, 'scripts', 'gate-verify.mjs'),
|
||||
verifier.replace(find, replace),
|
||||
);
|
||||
await copyFile(
|
||||
path.join(root, 'scripts', 'gate-history-exclusion-control.mjs'),
|
||||
path.join(fixture, 'scripts', 'gate-history-exclusion-control.mjs'),
|
||||
);
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[path.join(fixture, 'scripts', 'gate-history-exclusion-control.mjs')],
|
||||
{ cwd: fixture, encoding: 'utf8' },
|
||||
);
|
||||
assert.equal(result.status, 79, `${name}: ${output(result)}`);
|
||||
assert.match(output(result), /HISTORY_PROVENANCE_FORBIDDEN/);
|
||||
} finally {
|
||||
await rm(fixture, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
});
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,785 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { chmod, copyFile, mkdir, readFile, rm, symlink, writeFile } from 'node:fs/promises';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import test from 'node:test';
|
||||
|
||||
const verifier = path.join(process.cwd(), 'scripts', 'gate-verify.mjs');
|
||||
const fixtureRunner = path.join(
|
||||
process.cwd(),
|
||||
'scripts',
|
||||
'test-support',
|
||||
'gate-verify-fixture-runner.mjs',
|
||||
);
|
||||
const fixtureBase = path.join(process.cwd(), '.mosaic-test-work', `gate-verify-${process.pid}`);
|
||||
|
||||
async function fixture(name = 'case') {
|
||||
const root = path.join(fixtureBase, name);
|
||||
await rm(root, { recursive: true, force: true });
|
||||
await mkdir(path.join(root, 'gates'), { recursive: true });
|
||||
return root;
|
||||
}
|
||||
|
||||
function baseManifest() {
|
||||
return {
|
||||
schemaVersion: 1,
|
||||
gateRoots: ['gates'],
|
||||
governingClaimFiles: [],
|
||||
coverageBoundary: { included: ['meta fixture'], excluded: [], trackedBy: 'RM-54' },
|
||||
criteria: [
|
||||
{
|
||||
id: 'META-CRIT-1',
|
||||
originalText: 'The fixture rejects its bad input.',
|
||||
currentText: 'The fixture rejects its bad input.',
|
||||
claimType: 'integrity',
|
||||
source: 'fixture',
|
||||
meaningChanges: [],
|
||||
caseRefs: ['meta-fixture/rejects-bad-input'],
|
||||
},
|
||||
],
|
||||
compatibilityScenarios: [],
|
||||
proseClaims: [],
|
||||
gates: [
|
||||
{
|
||||
id: 'meta-fixture',
|
||||
source: 'gates/meta-fixture.sh',
|
||||
invocation: ['gates/meta-fixture.sh'],
|
||||
deployment: { kind: 'none', reason: 'test fixture only' },
|
||||
inertMutation: {
|
||||
file: 'gates/meta-fixture.sh',
|
||||
find: 'exit 7',
|
||||
replace: 'exit 0',
|
||||
expected: { exitCode: 0 },
|
||||
},
|
||||
cases: [
|
||||
{
|
||||
id: 'rejects-bad-input',
|
||||
criterionIds: ['META-CRIT-1'],
|
||||
mustFail: true,
|
||||
invocation: ['gates/meta-fixture.sh'],
|
||||
required: { exitCode: 7 },
|
||||
actual: { exitCode: 7 },
|
||||
evidence: { subject: 'meta-fixture' },
|
||||
reasonPattern: 'META_REJECT',
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
async function writeGate(root, contents = '#!/bin/sh\necho META_REJECT >&2\nexit 7\n') {
|
||||
const target = path.join(root, 'gates', 'meta-fixture.sh');
|
||||
await writeFile(target, contents);
|
||||
await chmod(target, 0o755);
|
||||
}
|
||||
|
||||
async function writeManifest(root, manifest) {
|
||||
await writeFile(path.join(root, 'gates', 'gates.manifest.json'), `${JSON.stringify(manifest)}\n`);
|
||||
}
|
||||
|
||||
function verify(root, extraArgs = []) {
|
||||
return spawnSync(
|
||||
process.execPath,
|
||||
[fixtureRunner, '--root', root, '--manifest', 'gates/gates.manifest.json', ...extraArgs],
|
||||
{ cwd: root, encoding: 'utf8', env: { ...process.env, HOME: os.homedir() } },
|
||||
);
|
||||
}
|
||||
|
||||
function verifyProductionStructure(root, extraArgs = []) {
|
||||
return spawnSync(
|
||||
process.execPath,
|
||||
[
|
||||
verifier,
|
||||
'--root',
|
||||
root,
|
||||
'--manifest',
|
||||
'gates/gates.manifest.json',
|
||||
'--structure-only',
|
||||
...extraArgs,
|
||||
],
|
||||
{ cwd: root, encoding: 'utf8', env: { ...process.env, HOME: os.homedir() } },
|
||||
);
|
||||
}
|
||||
|
||||
function output(result) {
|
||||
return `${result.stdout}\n${result.stderr}`;
|
||||
}
|
||||
|
||||
test.after(async () => {
|
||||
await rm(fixtureBase, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('universally quantified registry checks reject empty populations before evaluation', async () => {
|
||||
const root = await fixture('empty-registry-populations');
|
||||
await mkdir(path.join(root, 'empty-gate-root'), { recursive: true });
|
||||
const manifest = baseManifest();
|
||||
manifest.gateRoots = ['empty-gate-root'];
|
||||
manifest.criteria = [];
|
||||
manifest.proseClaims = [];
|
||||
manifest.compatibilityScenarios = [];
|
||||
manifest.gates = [];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verifyProductionStructure(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /criteria population.*non-empty.*anchored/i);
|
||||
assert.match(output(result), /gates population.*non-empty.*anchored/i);
|
||||
assert.match(output(result), /proseClaims population.*non-empty.*anchored/i);
|
||||
assert.match(output(result), /compatibilityScenarios population.*non-empty.*anchored/i);
|
||||
});
|
||||
|
||||
test('production verifier exposes no fixture-profile population bypass', async () => {
|
||||
const root = await fixture('no-production-fixture-profile');
|
||||
const result = verifyProductionStructure(root, ['--fixture-profile']);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /unknown option: --fixture-profile/i);
|
||||
});
|
||||
|
||||
test('anchored gate inventory and population criteria cannot shrink together', async () => {
|
||||
const source = JSON.parse(
|
||||
await readFile(path.join(process.cwd(), 'gates', 'gates.manifest.json'), 'utf8'),
|
||||
);
|
||||
const root = await fixture('shrunken-gate-population');
|
||||
await copyFile(
|
||||
path.join(process.cwd(), 'gates', 'required-gates.baseline.json'),
|
||||
path.join(root, 'gates', 'required-gates.baseline.json'),
|
||||
);
|
||||
source.gates = source.gates.filter((gate) => gate.id !== 'hook-pre-push');
|
||||
for (const criterion of source.criteria) {
|
||||
if (criterion.gateRefs) {
|
||||
criterion.gateRefs = criterion.gateRefs.filter((gateId) => gateId !== 'hook-pre-push');
|
||||
}
|
||||
criterion.caseRefs = criterion.caseRefs.filter(
|
||||
(caseRef) => !caseRef.startsWith('hook-pre-push/'),
|
||||
);
|
||||
}
|
||||
await writeManifest(root, source);
|
||||
|
||||
const result = verifyProductionStructure(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(
|
||||
output(result),
|
||||
/same-checkout required-gate baseline rejects manifest drift at hook-pre-push/i,
|
||||
);
|
||||
});
|
||||
|
||||
test('general population criteria cannot delete their gateRefs binding', async () => {
|
||||
const requiredCriteria = [
|
||||
'RM02-EVIDENCE-SUBJECT-BINDING',
|
||||
'RM02-TYPE-STRICT-SCHEMA',
|
||||
'RM02-NONEMPTY-ANCHORED-QUANTIFICATION',
|
||||
];
|
||||
for (const criterionId of requiredCriteria) {
|
||||
const source = JSON.parse(
|
||||
await readFile(path.join(process.cwd(), 'gates', 'gates.manifest.json'), 'utf8'),
|
||||
);
|
||||
const root = await fixture(`missing-gate-refs-${criterionId}`);
|
||||
delete source.criteria.find((criterion) => criterion.id === criterionId).gateRefs;
|
||||
await writeManifest(root, source);
|
||||
const result = verifyProductionStructure(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(
|
||||
output(result),
|
||||
new RegExp(`${criterionId}.*gateRefs.*required`, 'i'),
|
||||
criterionId,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('general population criteria must span every registered gate', async () => {
|
||||
const source = JSON.parse(
|
||||
await readFile(path.join(process.cwd(), 'gates', 'gates.manifest.json'), 'utf8'),
|
||||
);
|
||||
const root = await fixture('incomplete-gate-refs');
|
||||
source.criteria.find((criterion) => criterion.id === 'RM02-EVIDENCE-SUBJECT-BINDING').gateRefs =
|
||||
source.criteria
|
||||
.find((criterion) => criterion.id === 'RM02-EVIDENCE-SUBJECT-BINDING')
|
||||
.gateRefs.filter((gateId) => gateId !== 'quality-lint');
|
||||
await writeManifest(root, source);
|
||||
|
||||
const result = verifyProductionStructure(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(
|
||||
output(result),
|
||||
/RM02-EVIDENCE-SUBJECT-BINDING.*gate population binding is missing quality-lint/i,
|
||||
);
|
||||
});
|
||||
|
||||
test('an externally inerted failure branch makes verification nonzero and names the gate', async () => {
|
||||
const root = await fixture('external-inert');
|
||||
await writeGate(root, '#!/bin/sh\nexit 0\n');
|
||||
await writeManifest(root, baseManifest());
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture/);
|
||||
});
|
||||
|
||||
test('the verifier applies the declared inert mutation and observes its own control red', async () => {
|
||||
const root = await fixture('internal-meta');
|
||||
await writeGate(root);
|
||||
await writeManifest(root, baseManifest());
|
||||
|
||||
const result = verify(root);
|
||||
assert.equal(result.status, 0, output(result));
|
||||
assert.match(output(result), /META-NEGATIVE-CONTROL.*meta-fixture.*observed red/i);
|
||||
});
|
||||
|
||||
test('a mutation crash is rejected instead of counted as an observed-red control', async () => {
|
||||
const root = await fixture('mutation-crash');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].inertMutation.replace = 'this is not valid shell (';
|
||||
manifest.gates[0].inertMutation.expected = { exitCode: 0 };
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*mutation.*unexpected outcome/i);
|
||||
assert.doesNotMatch(output(result), /META-NEGATIVE-CONTROL.*observed red/i);
|
||||
});
|
||||
|
||||
test('a stale declared mutation case id is rejected instead of falling back', async () => {
|
||||
const root = await fixture('stale-case-id');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].inertMutation.caseId = 'case-that-does-not-exist';
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*case-that-does-not-exist.*not found/i);
|
||||
});
|
||||
|
||||
test('duplicate stable ids and unsupported schema versions are rejected', async () => {
|
||||
const root = await fixture('closed-schema');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.schemaVersion = 99;
|
||||
manifest.criteria.push({ ...manifest.criteria[0] });
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /unsupported schemaVersion 99/i);
|
||||
assert.match(output(result), /duplicate criterion id META-CRIT-1/i);
|
||||
});
|
||||
|
||||
test('misspelled nested assertion fields are rejected instead of becoming optional', async () => {
|
||||
const root = await fixture('nested-schema-typo');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases[0].required.outputPatern = 'META_REJECT';
|
||||
manifest.gates[0].cases[0].actual.outputPatern = 'META_REJECT';
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /required.*unknown field outputPatern/i);
|
||||
assert.match(output(result), /actual.*unknown field outputPatern/i);
|
||||
});
|
||||
|
||||
test('present outcome patterns cannot be empty assertion bypasses', async () => {
|
||||
const root = await fixture('nested-schema-empty-patterns');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases[0].required.outputPattern = '';
|
||||
manifest.gates[0].cases[0].actual.notOutputPattern = ' ';
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root, ['--structure-only']);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /required.outputPattern: expected a non-empty pattern/i);
|
||||
assert.match(output(result), /actual.notOutputPattern: expected a non-empty pattern/i);
|
||||
});
|
||||
|
||||
test('nested discriminator and comparison fields reject wrong types', async () => {
|
||||
const root = await fixture('nested-schema-types');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases[0].mustFail = 'true';
|
||||
manifest.gates[0].cases[0].required.exitCode = '7';
|
||||
manifest.gates[0].cases[0].actual.outputPattern = 7;
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root, ['--structure-only']);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /mustFail: expected a boolean/i);
|
||||
assert.match(output(result), /required.exitCode: expected an integer/i);
|
||||
assert.match(output(result), /actual.outputPattern: expected a string/i);
|
||||
});
|
||||
|
||||
test('recursive closed-schema guards reject unknown fields in every nested assertion object', async () => {
|
||||
const source = JSON.parse(
|
||||
await readFile(path.join(process.cwd(), 'gates', 'gates.manifest.json'), 'utf8'),
|
||||
);
|
||||
const checkout = source.gates.find((gate) => gate.id === 'checkout-preflight');
|
||||
const stale = checkout.cases.find((gateCase) => gateCase.id === 'stale-build-lock');
|
||||
const queue = source.gates.find((gate) => gate.id === 'ci-queue-wait');
|
||||
const queueCase = queue.cases.find((gateCase) => gateCase.id === 'terminal-success');
|
||||
const targets = [
|
||||
['coverageBoundary', (manifest) => manifest.coverageBoundary],
|
||||
['mergeAssertions', (manifest) => manifest.mergeAssertions],
|
||||
[
|
||||
'meaningChanges',
|
||||
(manifest) =>
|
||||
manifest.criteria.find((criterion) => criterion.meaningChanges.length).meaningChanges[0],
|
||||
],
|
||||
['proseClaims', (manifest) => manifest.proseClaims[0]],
|
||||
['compatibility expected', (manifest) => manifest.compatibilityScenarios[0].expected],
|
||||
[
|
||||
'deployment',
|
||||
(manifest) => manifest.gates.find((gate) => gate.id === 'ci-queue-wait').deployment,
|
||||
],
|
||||
['inertMutation', (manifest) => manifest.gates[0].inertMutation],
|
||||
['inert expected', (manifest) => manifest.gates[0].inertMutation.expected],
|
||||
['required', (manifest) => manifest.gates[0].cases[0].required],
|
||||
['actual', (manifest) => manifest.gates[0].cases[0].actual],
|
||||
[
|
||||
'fixture',
|
||||
(manifest) =>
|
||||
manifest.gates
|
||||
.find((gate) => gate.id === 'checkout-preflight')
|
||||
.cases.find((gateCase) => gateCase.id === 'stale-build-lock').fixture,
|
||||
],
|
||||
[
|
||||
'write entry',
|
||||
(manifest) =>
|
||||
manifest.gates
|
||||
.find((gate) => gate.id === 'checkout-preflight')
|
||||
.cases.find((gateCase) => gateCase.id === 'stale-build-lock').fixture.writeFiles[0],
|
||||
],
|
||||
[
|
||||
'replace entry',
|
||||
(manifest) =>
|
||||
manifest.gates
|
||||
.find((gate) => gate.id === 'checkout-preflight')
|
||||
.cases.find((gateCase) => gateCase.id === 'criterion-misbinding').fixture.replaceFiles[0],
|
||||
],
|
||||
[
|
||||
'defect',
|
||||
(manifest) =>
|
||||
manifest.gates
|
||||
.find((gate) => gate.id === 'ci-queue-wait')
|
||||
.cases.find((gateCase) => gateCase.id === 'terminal-success').defect,
|
||||
],
|
||||
];
|
||||
assert.ok(stale.fixture && queueCase.defect);
|
||||
for (const [name, select] of targets) {
|
||||
const root = await fixture(`recursive-${name.replaceAll(' ', '-')}`);
|
||||
const manifest = structuredClone(source);
|
||||
select(manifest).unexpectedNestedField = true;
|
||||
await writeManifest(root, manifest);
|
||||
const result = verify(root, ['--structure-only']);
|
||||
assert.notEqual(result.status, 0, `${name}: ${output(result)}`);
|
||||
assert.match(output(result), /unknown field unexpectedNestedField/i, name);
|
||||
}
|
||||
});
|
||||
|
||||
test('manifest-controlled fixture paths cannot escape the sandbox', async () => {
|
||||
const root = await fixture('path-traversal');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases[0].fixture = {
|
||||
writeFiles: [{ path: '../../escaped-by-manifest', content: 'bad' }],
|
||||
};
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /path escapes sandbox/i);
|
||||
});
|
||||
|
||||
test('fixture writes reject a final symlink and preserve its outside target', async () => {
|
||||
const root = await fixture('final-symlink');
|
||||
await writeGate(root);
|
||||
const outside = path.join(fixtureBase, 'outside-sentinel');
|
||||
await writeFile(outside, 'preserve-me\n');
|
||||
const linked = path.join(root, 'linked-sentinel');
|
||||
await symlink(outside, linked);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases[0].fixture = {
|
||||
writeFiles: [{ path: 'linked-sentinel', content: 'overwritten\n' }],
|
||||
};
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /fixture write.*symbolic link/i);
|
||||
assert.equal(await readFile(outside, 'utf8'), 'preserve-me\n');
|
||||
});
|
||||
|
||||
test('an executable below a gate root without an entry is rejected', async () => {
|
||||
const root = await fixture('unregistered');
|
||||
await writeGate(root);
|
||||
const extra = path.join(root, 'gates', 'forgotten.sh');
|
||||
await writeFile(extra, '#!/bin/sh\nexit 1\n');
|
||||
await chmod(extra, 0o755);
|
||||
await writeManifest(root, baseManifest());
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /unregistered gate.*forgotten\.sh/i);
|
||||
});
|
||||
|
||||
test('a must-fail case without a reason diagnostic is rejected', async () => {
|
||||
const root = await fixture('missing-reason');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases[0].reasonPattern = '';
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*rejects-bad-input.*reasonPattern/i);
|
||||
});
|
||||
|
||||
test('a gate with zero must-fail cases is rejected', async () => {
|
||||
const root = await fixture('no-negative');
|
||||
await writeGate(root, '#!/bin/sh\nexit 0\n');
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].inertMutation = {
|
||||
file: 'gates/meta-fixture.sh',
|
||||
find: 'exit 0',
|
||||
replace: 'exit 1',
|
||||
};
|
||||
manifest.gates[0].cases = [
|
||||
{
|
||||
id: 'positive',
|
||||
criterionIds: ['META-CRIT-1'],
|
||||
mustFail: false,
|
||||
invocation: ['gates/meta-fixture.sh'],
|
||||
required: { exitCode: 0 },
|
||||
actual: { exitCode: 0 },
|
||||
reasonPattern: '',
|
||||
},
|
||||
];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*no negative control/i);
|
||||
});
|
||||
|
||||
test('reordered but equivalent outcome fields do not create a false behavior delta', async () => {
|
||||
const root = await fixture('reordered-outcomes');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases[0].required = { exitCode: 7, outputPattern: 'META_REJECT' };
|
||||
manifest.gates[0].cases[0].actual = { outputPattern: 'META_REJECT', exitCode: 7 };
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.equal(result.status, 0, output(result));
|
||||
assert.doesNotMatch(output(result), /behavior delta requires|DEFECT \(owner:/);
|
||||
});
|
||||
|
||||
test('a required-versus-actual delta without a tracked owner is rejected', async () => {
|
||||
const root = await fixture('ownerless-delta');
|
||||
await writeGate(root, '#!/bin/sh\nexit 0\n');
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].inertMutation.find = 'exit 0';
|
||||
manifest.gates[0].inertMutation.replace = 'exit 7';
|
||||
manifest.gates[0].cases[0].actual.exitCode = 0;
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*delta.*tracked owner/i);
|
||||
});
|
||||
|
||||
test('moving criterion bindings to unrelated cases is rejected', async () => {
|
||||
const root = await fixture('semantic-misbinding');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.criteria.push({
|
||||
id: 'META-CRIT-2',
|
||||
originalText: 'The fixture reports the second rejection reason.',
|
||||
currentText: 'The fixture reports the second rejection reason.',
|
||||
claimType: 'integrity',
|
||||
source: 'fixture',
|
||||
meaningChanges: [],
|
||||
caseRefs: ['meta-fixture/rejects-second-input'],
|
||||
});
|
||||
manifest.gates[0].cases.push({
|
||||
...manifest.gates[0].cases[0],
|
||||
id: 'rejects-second-input',
|
||||
criterionIds: ['META-CRIT-1'],
|
||||
});
|
||||
manifest.gates[0].cases[0].criterionIds = ['META-CRIT-2'];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /META-CRIT-1.*rejects-bad-input.*not bound/i);
|
||||
assert.match(output(result), /META-CRIT-2.*rejects-second-input.*not bound/i);
|
||||
});
|
||||
|
||||
test('canonical verification preserves binding diagnostics when a case fixture is also stale', async () => {
|
||||
const root = await fixture('misbinding-plus-stale-fixture');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.criteria.push({
|
||||
id: 'META-CRIT-2',
|
||||
originalText: 'The second case rejects its own bad input.',
|
||||
currentText: 'The second case rejects its own bad input.',
|
||||
claimType: 'integrity',
|
||||
source: 'fixture',
|
||||
meaningChanges: [],
|
||||
caseRefs: ['meta-fixture/rejects-second-input'],
|
||||
});
|
||||
manifest.gates[0].cases.push({
|
||||
...manifest.gates[0].cases[0],
|
||||
id: 'rejects-second-input',
|
||||
criterionIds: ['META-CRIT-1'],
|
||||
});
|
||||
manifest.gates[0].cases[0].criterionIds = ['META-CRIT-2'];
|
||||
manifest.gates[0].cases[0].fixture = {
|
||||
replaceFiles: [
|
||||
{
|
||||
path: 'gates/meta-fixture.sh',
|
||||
find: 'text that is not present',
|
||||
replace: 'irrelevant',
|
||||
},
|
||||
],
|
||||
};
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /META-CRIT-1.*rejects-bad-input.*not bound/i);
|
||||
assert.match(output(result), /META-CRIT-2.*rejects-second-input.*not bound/i);
|
||||
assert.match(output(result), /fixture replace.*stale or ambiguous/i);
|
||||
});
|
||||
|
||||
test('moving meaning and prose criteria to an unrelated type error is rejected', async () => {
|
||||
const root = await fixture('real-manifest-misbinding');
|
||||
const manifest = JSON.parse(
|
||||
await readFile(path.join(process.cwd(), 'gates', 'gates.manifest.json'), 'utf8'),
|
||||
);
|
||||
for (const gate of manifest.gates) {
|
||||
for (const gateCase of gate.cases) {
|
||||
gateCase.criterionIds = gateCase.criterionIds.filter(
|
||||
(id) => !['RM02-MEANING-PROVENANCE', 'RM02-PROSE-CONTROL'].includes(id),
|
||||
);
|
||||
}
|
||||
}
|
||||
const typeError = manifest.gates
|
||||
.find((gate) => gate.id === 'quality-typecheck')
|
||||
.cases.find((gateCase) => gateCase.id === 'type-error');
|
||||
typeError.criterionIds.push('RM02-MEANING-PROVENANCE', 'RM02-PROSE-CONTROL');
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root, ['--structure-only']);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /RM02-MEANING-PROVENANCE.*missing-meaning-provenance.*not bound/i);
|
||||
assert.match(output(result), /RM02-PROSE-CONTROL.*prose-claim-misbinding.*not bound/i);
|
||||
assert.match(
|
||||
output(result),
|
||||
/RM02-(?:MEANING-PROVENANCE|PROSE-CONTROL).*undeclared exercising case quality-typecheck\/type-error/i,
|
||||
);
|
||||
});
|
||||
|
||||
test('a criterion with no bound case is rejected', async () => {
|
||||
const root = await fixture('unbound-criterion');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.criteria.push({
|
||||
id: 'ORPHAN',
|
||||
originalText: 'This criterion is not exercised.',
|
||||
currentText: 'This criterion is not exercised.',
|
||||
claimType: 'quality',
|
||||
source: 'fixture',
|
||||
meaningChanges: [],
|
||||
});
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /ORPHAN.*no bound case/i);
|
||||
});
|
||||
|
||||
test('an unbound governing prose marker is rejected', async () => {
|
||||
const root = await fixture('unbound-prose');
|
||||
await writeGate(root);
|
||||
await mkdir(path.join(root, 'docs'), { recursive: true });
|
||||
await writeFile(path.join(root, 'docs', 'governing.md'), 'GATE-CLAIM:UNBOUND\n');
|
||||
const manifest = baseManifest();
|
||||
manifest.governingClaimFiles = ['docs/governing.md'];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /GATE-CLAIM:UNBOUND.*unbound/i);
|
||||
});
|
||||
|
||||
test('directly contradictory modeled scenarios are rejected', async () => {
|
||||
const root = await fixture('conflict');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.compatibilityScenarios = [
|
||||
{
|
||||
id: 'ONE',
|
||||
construction: 'same-input',
|
||||
caseRefs: ['meta-fixture/rejects-bad-input'],
|
||||
invocation: ['sh', '-c', 'exit 0'],
|
||||
expected: { exitCode: 0 },
|
||||
},
|
||||
{
|
||||
id: 'TWO',
|
||||
construction: 'same-input',
|
||||
caseRefs: ['meta-fixture/rejects-bad-input'],
|
||||
invocation: ['sh', '-c', 'exit 1'],
|
||||
expected: { exitCode: 1 },
|
||||
},
|
||||
];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /ONE.*TWO.*conflict/i);
|
||||
});
|
||||
|
||||
test('compatibility scenarios execute referenced conditions as one construction', async () => {
|
||||
const root = await fixture('combined-compatibility');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases.push({
|
||||
id: 'second-condition',
|
||||
criterionIds: ['META-CRIT-1'],
|
||||
mustFail: true,
|
||||
invocation: ['sh', '-c', 'echo "$SECOND_REASON" >&2; exit 7'],
|
||||
fixture: { writeFiles: [{ path: 'conditions/second', content: 'present\n' }] },
|
||||
required: { exitCode: 7 },
|
||||
actual: { exitCode: 7 },
|
||||
evidence: { subject: 'meta-fixture' },
|
||||
reasonPattern: 'SECOND_REASON',
|
||||
environment: { SECOND_REASON: 'SECOND_REASON' },
|
||||
});
|
||||
manifest.criteria[0].caseRefs.push('meta-fixture/second-condition');
|
||||
manifest.gates[0].cases[0].fixture = {
|
||||
writeFiles: [{ path: 'conditions/first', content: 'present\n' }],
|
||||
};
|
||||
manifest.compatibilityScenarios = [
|
||||
{
|
||||
id: 'BOTH-CONDITIONS',
|
||||
construction: 'both-fixtures',
|
||||
caseRefs: ['meta-fixture/rejects-bad-input', 'meta-fixture/second-condition'],
|
||||
invocation: ['sh', '-c', 'test -f conditions/first && test -f conditions/second'],
|
||||
expected: { exitCode: 0 },
|
||||
},
|
||||
];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.equal(result.status, 0, output(result));
|
||||
assert.match(output(result), /COMPATIBILITY BOTH-CONDITIONS: observed expected outcome/i);
|
||||
});
|
||||
|
||||
test('a restated criterion without provenance is rejected', async () => {
|
||||
const root = await fixture('provenance');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.criteria[0].currentText = 'The fixture rejects only malformed input.';
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /META-CRIT-1.*meaning-change provenance/i);
|
||||
});
|
||||
|
||||
test('a security criterion bound only to a positive case is unregistered in substance', async () => {
|
||||
const root = await fixture('positive-only-criterion');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.criteria.push({
|
||||
id: 'POSITIVE-ONLY',
|
||||
originalText: 'A security property.',
|
||||
currentText: 'A security property.',
|
||||
claimType: 'security',
|
||||
source: 'fixture',
|
||||
meaningChanges: [],
|
||||
});
|
||||
manifest.gates[0].cases.push({
|
||||
id: 'positive-only',
|
||||
criterionIds: ['POSITIVE-ONLY'],
|
||||
mustFail: false,
|
||||
invocation: ['gates/meta-fixture.sh'],
|
||||
required: { exitCode: 7 },
|
||||
actual: { exitCode: 7 },
|
||||
reasonPattern: '',
|
||||
});
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /POSITIVE-ONLY.*no must-fail case/i);
|
||||
});
|
||||
|
||||
test('a registered prose claim whose marker is absent is rejected', async () => {
|
||||
const root = await fixture('missing-prose-marker');
|
||||
await writeGate(root);
|
||||
await mkdir(path.join(root, 'docs'), { recursive: true });
|
||||
await writeFile(path.join(root, 'docs', 'governing.md'), 'No marker here.\n');
|
||||
const manifest = baseManifest();
|
||||
manifest.governingClaimFiles = ['docs/governing.md'];
|
||||
manifest.proseClaims = [{ id: 'MISSING-MARKER', criterionId: 'META-CRIT-1' }];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /GATE-CLAIM:MISSING-MARKER.*missing/i);
|
||||
});
|
||||
|
||||
test('source-versus-deployed byte drift is rejected and names the gate', async () => {
|
||||
const root = await fixture('deployment-drift');
|
||||
await writeGate(root);
|
||||
await mkdir(path.join(root, 'deployed'), { recursive: true });
|
||||
await writeFile(path.join(root, 'deployed', 'meta-fixture.sh'), '#!/bin/sh\nexit 0\n');
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].deployment = { kind: 'file', path: 'deployed/meta-fixture.sh' };
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*source.*deployed.*drift/i);
|
||||
});
|
||||
|
||||
test('deployment drift meta-control fails if the shared comparator is made inert', async () => {
|
||||
const root = await fixture('deployment-comparator-inert');
|
||||
await writeGate(root);
|
||||
await mkdir(path.join(root, 'deployed'), { recursive: true });
|
||||
await copyFile(
|
||||
path.join(root, 'gates', 'meta-fixture.sh'),
|
||||
path.join(root, 'deployed', 'meta-fixture.sh'),
|
||||
);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].deployment = { kind: 'file', path: 'deployed/meta-fixture.sh' };
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const alteredScripts = path.join(root, 'verifier-scripts');
|
||||
await mkdir(alteredScripts, { recursive: true });
|
||||
const verifierSource = await readFile(verifier, 'utf8');
|
||||
const inertSource = verifierSource.replace(
|
||||
'return source.equals(deployed);',
|
||||
'return true; // deliberate test-only inert comparator',
|
||||
);
|
||||
assert.notEqual(inertSource, verifierSource, 'shared deployment comparator mutation went stale');
|
||||
await writeFile(path.join(alteredScripts, 'gate-verify.mjs'), inertSource);
|
||||
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[
|
||||
path.join(alteredScripts, 'gate-verify.mjs'),
|
||||
'--root',
|
||||
root,
|
||||
'--manifest',
|
||||
'gates/gates.manifest.json',
|
||||
],
|
||||
{ cwd: root, encoding: 'utf8', env: { ...process.env, HOME: os.homedir() } },
|
||||
);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*drift negative control was ineffective/i);
|
||||
});
|
||||
@@ -0,0 +1,104 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import test from 'node:test';
|
||||
|
||||
const root = process.cwd();
|
||||
const expectedTriggers = `when:
|
||||
# PR + manual CI run on any branch — the pull_request pipeline is the merge gate.
|
||||
# push CI is restricted to protected branches (main) so a feature-branch push no
|
||||
# longer fires a redundant SECOND pipeline alongside its PR pipeline. This ~halves
|
||||
# CI load on the storage-constrained runner with zero loss of gating (branch
|
||||
# protection requires no push/ci status context; main still gets full push CI).
|
||||
- event: [pull_request, manual]
|
||||
- event: push
|
||||
branch: main`;
|
||||
const expectedGateStep = ` image: *node_image
|
||||
commands:
|
||||
- *enable_pnpm
|
||||
- pnpm gate:verify
|
||||
depends_on:
|
||||
- install
|
||||
- sanitization
|
||||
- upgrade-guard`;
|
||||
|
||||
export function assertUnprivilegedGateStep(pipeline) {
|
||||
assert.doesNotMatch(
|
||||
pipeline,
|
||||
/privileged/i,
|
||||
'no pull-request pipeline step may declare privilege',
|
||||
);
|
||||
for (const line of pipeline.split('\n')) {
|
||||
const candidate = line.trimStart().replace(/^-\s+/, '');
|
||||
if (/^(?:["'!<].*|[A-Za-z_][A-Za-z0-9_-]*\s+):(?:\s|$)/.test(candidate)) {
|
||||
assert.fail(`non-canonical or merged YAML key is forbidden: ${candidate}`);
|
||||
}
|
||||
}
|
||||
const triggerMatches = [...pipeline.matchAll(/^when:\n([\s\S]*?)(?=\n\n)/gm)];
|
||||
assert.equal(triggerMatches.length, 1, 'exactly one top-level trigger is required');
|
||||
assert.equal(
|
||||
`when:\n${triggerMatches[0][1].trimEnd()}`,
|
||||
expectedTriggers,
|
||||
'top-level triggers must match closed PR/main construction',
|
||||
);
|
||||
|
||||
const matches = [
|
||||
...pipeline.matchAll(/\n gate-verify:\n([\s\S]*?)(?=\n [a-z][a-z0-9-]+:|\nservices:|$)/g),
|
||||
];
|
||||
assert.equal(matches.length, 1, 'exactly one gate-verify step is required');
|
||||
// Closed textual construction by design: accepting arbitrary YAML syntax here
|
||||
// would require a duplicate-key-preserving parser. Exact equality rejects all
|
||||
// extra keys, quoted/escaped key spellings, aliases, and mapping merges.
|
||||
assert.equal(
|
||||
matches[0][1].trimEnd(),
|
||||
expectedGateStep,
|
||||
'gate-verify step must match closed unprivileged construction',
|
||||
);
|
||||
}
|
||||
|
||||
test('package.json exposes the canonical gate:verify command', async () => {
|
||||
const packageJson = JSON.parse(await readFile(`${root}/package.json`, 'utf8'));
|
||||
assert.equal(packageJson.scripts['gate:verify'], 'node scripts/gate-verify.mjs');
|
||||
});
|
||||
|
||||
test('Woodpecker runs the closed unprivileged gate construction on every pipeline', async () => {
|
||||
const pipeline = await readFile(`${root}/.woodpecker/ci.yml`, 'utf8');
|
||||
assertUnprivilegedGateStep(pipeline);
|
||||
});
|
||||
|
||||
test('gate wiring rejects privilege syntax, merges, duplicate keys, and trigger narrowing', async () => {
|
||||
const pipeline = await readFile(`${root}/.woodpecker/ci.yml`, 'utf8');
|
||||
const additions = [
|
||||
' privileged: *enabled\n',
|
||||
' "privileged": true\n',
|
||||
" 'privileged': true\n",
|
||||
' privileged : true\n',
|
||||
' "priv\\u0069leged": true\n',
|
||||
' <<: *privileged-step\n',
|
||||
' "<<": *privileged-step\n',
|
||||
];
|
||||
for (const addition of additions) {
|
||||
const changed = pipeline.replace(
|
||||
' gate-verify:\n image:',
|
||||
` gate-verify:\n${addition} image:`,
|
||||
);
|
||||
assert.throws(() => assertUnprivilegedGateStep(changed));
|
||||
}
|
||||
const privilegedInstall = pipeline.replace(
|
||||
' install:\n image:',
|
||||
' install:\n privileged: true\n image:',
|
||||
);
|
||||
const duplicate = `${pipeline}\n gate-verify:\n image: *node_image\n`;
|
||||
const noPullRequest = pipeline.replace(' - event: [pull_request, manual]', ' - event: manual');
|
||||
const filteredPullRequest = pipeline.replace(
|
||||
' - event: [pull_request, manual]',
|
||||
' - event: [pull_request, manual]\n path: [scripts/**]',
|
||||
);
|
||||
|
||||
assert.throws(() => assertUnprivilegedGateStep(privilegedInstall), /privilege/i);
|
||||
assert.throws(() => assertUnprivilegedGateStep(duplicate), /exactly one gate-verify/);
|
||||
assert.throws(() => assertUnprivilegedGateStep(noPullRequest), /closed PR\/main construction/);
|
||||
assert.throws(
|
||||
() => assertUnprivilegedGateStep(filteredPullRequest),
|
||||
/closed PR\/main construction/,
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,27 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import path from 'node:path';
|
||||
|
||||
import { verifyRegistry } from '../gate-verify.mjs';
|
||||
|
||||
let root;
|
||||
let manifest = 'gates/gates.manifest.json';
|
||||
let structureOnly = false;
|
||||
for (let index = 0; index < process.argv.slice(2).length; index += 1) {
|
||||
const args = process.argv.slice(2);
|
||||
const value = args[index];
|
||||
if (value === '--root') root = path.resolve(args[++index]);
|
||||
else if (value === '--manifest') manifest = args[++index];
|
||||
else if (value === '--structure-only') structureOnly = true;
|
||||
else throw new Error(`unknown fixture-runner option: ${value}`);
|
||||
}
|
||||
if (!root) throw new Error('fixture runner requires --root');
|
||||
const { failures, observations } = await verifyRegistry({
|
||||
root,
|
||||
manifest,
|
||||
structureOnly,
|
||||
fixtureProfile: true,
|
||||
});
|
||||
for (const observation of observations) process.stdout.write(`${observation}\n`);
|
||||
for (const failure of failures) process.stderr.write(`GATE VERIFY FAILED: ${failure}\n`);
|
||||
if (failures.length > 0) process.exitCode = 1;
|
||||
Reference in New Issue
Block a user