fix(quality): close registry silent-defeat paths
ci/woodpecker/pr/ci Pipeline was successful

This commit is contained in:
2026-08-01 11:28:28 -05:00
parent 83d2ecb224
commit 32b490a712
11 changed files with 889 additions and 52 deletions
+4 -2
View File
@@ -14,7 +14,7 @@ Investigate any of these immediately:
## Updating a gate
1. Add or change the criterion, its exact criterion-side `caseRefs`, and matching case-side `criterionIds`.
1. Add or change the criterion, its exact criterion-side `caseRefs`, and matching case-side `criterionIds`. Preserve recursive closed-schema validation for every nested object; new fields require explicit key and type handling.
2. Observe the must-fail case fail for its own stated reason; moving the binding to any undeclared case must fail verification.
3. Declare an exact inerting mutation and observe the verifier detect it.
4. If required and actual behavior differ, add a tracked remediation owner and justification.
@@ -28,7 +28,9 @@ Do not add an ownerless exception or describe an open delta as pass/green/OK.
Woodpecker runs `gate-verify` on every pull request and protected-main push without path filtering. This is deliberate: changes outside gate files can make a gate inert. The step unshallows the checkout so activation ancestry and historical manifest provenance can be checked; a shallow boundary must never be interpreted as non-ancestry.
Provider evidence input is an optional JSON array of normalized pipeline records containing `commit`, unique integer pipeline `number`, pipeline `status`, and a `gate-verify` step status. The highest numbered rerun is authoritative; ambiguous duplicates fail. Its retention window is provider-controlled and is not overstated by this repository.
Provider evidence input is an optional JSON array of normalized pipeline records containing `commit`, globally unique integer pipeline `number`, pipeline `status`, and a `gate-verify` step status. Collection-wide identity/type validation occurs before commit filtering; a duplicate number across two commits fails subject binding. The highest numbered valid rerun for one commit is authoritative. Its retention window is provider-controlled and is not overstated by this repository.
Do not add or update an activation SHA in the manifest. The verifier derives the immutable history boundary from Git as the parent of the first first-parent registry-introduction commit. HEAD, HEAD's parent, and the introduction commit are registered rejected seam candidates.
PR CI executes current-tree verification only, unprivileged and fail-closed. It does not execute isolated own-tree replay: RM-60 must provide a protected launcher or runner-level rootless sandbox before any PR-controlled executable/configuration is evaluated. Repo-only code cannot safely grant itself the capability intended to contain itself.
@@ -10,7 +10,7 @@ pnpm gate:verify
The registry covers root typecheck, lint, and format checks; RM-01 checkout preflight; the Mosaic CI queue guard; and root Husky pre-commit/pre-push hooks. It does not imply repository-wide coverage. Framework scripts, package-local build/test scripts, templates, and deployment/release scripts remain assigned to RM-54.
Every gate declares exact invocations, observed and required outcomes, criterion bindings, and a single exact inerting mutation. Every must-fail case requires a non-empty reason diagnostic. The verifier rejects a stale, ambiguous, crashing, or ineffective mutation. Fixture and mutation writes reject path traversal and final-component symlinks. Independent validation phases collect labeled failures instead of letting one thrown fixture, claim, discovery, deployment, mutation, or compatibility error mask already-known stable-ID diagnostics. This proves detection of the **declared** inerting mutation, not every possible semantic weakening.
Every gate declares exact invocations, observed and required outcomes, criterion bindings, and a single exact inerting mutation. Every must-fail case requires a non-empty reason diagnostic. The verifier rejects a stale, ambiguous, crashing, or ineffective mutation. Fixture and mutation writes reject path traversal and final-component symlinks. Every nested manifest object used by outcomes, mutations, fixtures, deployments, defects, compatibility, provenance, coverage, and merge assertions has closed keys and strict field types; a misspelling cannot silently turn a required comparison into an absent optional field. Independent validation phases collect labeled failures instead of letting one thrown fixture, claim, discovery, deployment, mutation, or compatibility error mask already-known stable-ID diagnostics. This proves detection of the **declared** inerting mutation, not every possible semantic weakening.
## Required versus actual
@@ -40,6 +40,8 @@ A gate with an external installed counterpart declares it explicitly. When the i
The replay implementation and abuse-case tests remain fail-closed: when invoked by a future protected authority, inability to establish Bubblewrap is terminal nonzero; controls are never omitted or treated as replay success. On an unprivileged CI runner, sandbox integration tests pass only when the result carries parent-generated Bubblewrap-launch provenance and proves the sandbox entry command never ran. Child-controlled text that merely reproduces a Bubblewrap denial is not accepted. Capable local/protected environments exercise the full abuse cases. Historical installs use frozen lockfiles, isolated network/PID/IPC/UTS and environment/home boundaries, and authoritative-file snapshots that detect lifecycle rewrites.
Retained provider evidence can assert terminal-success **current-tree** records for prior commits when supplied through `GATE_PROVIDER_EVIDENCE_FILE`. Each normalized record contains `commit`, unique integer pipeline `number`, pipeline `status`, and exactly one `gate-verify` step; the highest-numbered rerun is authoritative. Ambiguous duplicates fail. Absent, expired, or currently-running evidence is reported explicitly and never inferred as success.
Retained provider evidence can assert terminal-success **current-tree** records for prior commits when supplied through `GATE_PROVIDER_EVIDENCE_FILE`. Each normalized record contains `commit`, globally unique integer pipeline `number`, pipeline `status`, and exactly one `gate-verify` step; the highest-numbered rerun for a commit is authoritative. The full collection is validated before commit filtering, so one pipeline identity cannot certify two commit subjects. Ambiguous duplicates fail. Absent, expired, or currently-running evidence is reported explicitly and never inferred as success.
The history seam is not a manifest field. `gate-history.mjs` derives it from Git as the parent of the first first-parent commit adding `gates/gates.manifest.json`, which necessarily includes the introduction commit and all later first-parent registry commits. Registered controls reject HEAD, HEAD's parent, and the introduction commit as candidate seams.
Once RM-60 supplies the external pre-execution anchor, protected post-merge/main replay is detection, not pre-merge prevention. A failed replay requires quarantine of the affected result and revert of the offending merge. It must never be represented as proof that CI blocked that merge. RM-25 tracks provider enforcement.
+4
View File
@@ -38,6 +38,9 @@ Existing deterministic gates can return success without enforcing their stated p
8. `RM02-REQ-08`: Every gate with a deployed counterpart SHALL register source/deployed byte identity and a must-fail drift control. Gates without a deployed counterpart SHALL say so explicitly.
9. `RM02-REQ-09`: CI SHALL run `pnpm gate:verify` on every pull request without path filtering and on protected-main pushes.
10. `RM02-REQ-10` (restated): PR CI SHALL perform unprivileged, fail-closed current-tree verification only. Isolated per-commit replay SHALL remain deferred to RM-60's protected post-merge/main authority, cross-referenced with RM-59. That future replay is detection with a quarantine/revert response, not pre-merge prevention; inability to establish its sandbox is terminal nonzero, never skip/pass. Retained provider evidence SHALL remain distinct and SHALL never be inferred when absent.
11. `RM02-REQ-11`: The history activation boundary SHALL be derived, not manifest-authored, as the parent of the first first-parent commit introducing `gates/gates.manifest.json`. The introduction commit SHALL be included. Author-controlled candidates equal to HEAD, HEAD's parent, or the introduction commit SHALL be rejected by registered must-fail controls.
12. `RM02-REQ-12` (`D-38`): Evidence SHALL be bound to exactly one subject. Provider pipeline identity uniqueness SHALL be validated over the full evidence collection before commit filtering; a duplicate pipeline number across different commits SHALL fail.
13. `RM02-REQ-13` (`D-40`): Every registry object used for discrimination, comparison, mutation, fixture construction, provenance, deployment, or outcome assessment SHALL have a recursively closed, type-strict schema. Unknown or misspelled nested fields SHALL fail rather than becoming absent optional assertions.
#### RM02-REQ-10 meaning-change provenance
@@ -54,6 +57,7 @@ Existing deterministic gates can return success without enforcing their stated p
5. `RM02-AC-05`: Unbound or semantically misbound criteria, prose claims bound to unrelated cases, unbound governing prose markers, ownerless behavior deltas, stale mutations, source/deployed drift, and modeled compatibility conflicts each return nonzero with the responsible stable ID. A simultaneous stale fixture or independent phase error SHALL NOT mask responsible stable-ID diagnostics. Registered meta-negative controls move a criterion binding, remove meaning provenance, and redirect a prose claim to an unrelated case; each is observed red for its stated reason.
6. `RM02-AC-06`: CI configuration invokes the verifier unconditionally on every pull request.
7. `RM02-AC-07`: PR output states adjacent `DOES`/`DOES NOT` boundaries: current-tree gates and inerting mutations execute unprivileged and fail-closed; isolated own-tree replay does not execute in repository-controlled PR CI. RM-60/RM-59 are named, retained provider evidence is never inferred, and future protected post-merge detection specifies quarantine/revert rather than claiming pre-merge prevention.
8. `RM02-AC-08`: Registered must-fail controls reject history seam candidates at HEAD, HEAD's parent, and the registry introduction; a cross-commit duplicate provider pipeline identity; a misspelled nested outcome field; a wrong outcome field type; and a present-but-empty outcome pattern. The `RM02-EVIDENCE-SUBJECT-BINDING` and `RM02-TYPE-STRICT-SCHEMA` criteria are bidirectionally bound to their controls.
### Risks, dependencies, and verification boundary
+4 -2
View File
@@ -39,7 +39,7 @@ Deliver the seven-gate registry and RED-first anti-inert verifier on `feat/rm-02
- CI wiring RED: package script and unconditional Woodpecker step tests both failed before wiring.
- History RED: history test failed with missing module before own-tree manifest selection/provider classification was implemented.
- `pnpm gate:verify`: exit 0; seven gates each reported `META-NEGATIVE-CONTROL ... observed red`; queue source/deployed drift control observed red; six queue behavior deltas printed as `DEFECT (owner: RM-03)`.
- Focused Node tests: 38/38 pass after review hardening (27 verifier/wiring plus 11 history/provider tests).
- Focused Node tests: 48/48 pass after review hardening (31 verifier/wiring plus 17 history/provider tests).
- `pnpm typecheck`: pass (45/45 Turbo tasks).
- `pnpm lint`: pass (25/25 Turbo tasks).
- `pnpm format:check`: pass.
@@ -66,6 +66,8 @@ The queue guard's `get_state_from_status_json` runs `python3 - <<'PY'` while pro
- Exact-head independent review at `9b4d4beb` found two valid blockers. RED-first controls reproduced both: denial-looking child stderr was accepted as sandbox unavailability, and moving meaning/prose criterion IDs to an unrelated type-error case left `gate:verify` green. Bubblewrap execution now emits a parent-generated random entry marker and returns parent-owned launcher/entry metadata; unavailability requires Bubblewrap launcher provenance plus proof entry never ran, so exact denial impersonation from plain or entered-child results is rejected. Criterion objects now declare exact `caseRefs`, checked bidirectionally against case-side `criterionIds`; prose claims declare an exact must-fail `caseRef`. Registered must-fail cases move a criterion binding, remove meaning provenance, and redirect a prose claim, each producing its stable reason. The review freeze was deliberately lifted before remediation.
- Option C security review reported no findings. Code review rejected an initial unrelated typecheck binding for the new security criterion. It was replaced with a dedicated registered `privileged-pr-gate` case: the fixture injects a privilege key into the gate step, the wiring control rejects it for that exact reason, and `gate:verify` observes the boundary negative control. Follow-up hardening uses a closed exact gate-step construction, rejects privilege across the entire pipeline, rejects non-canonical/merged YAML keys, and pins the unrestricted PR/main trigger block; quoted/escaped/alias/merge/duplicate/filter bypass tests pass. Final Codex code review approved with no findings.
- Exact-head review at `83d2ecb2` found four silent-defeat paths. Genuine RED-first tests on the pre-fix code proved: author-controlled HEAD/parent/introduction seams produced no boundary failure; cross-commit duplicate pipeline number 7 returned terminal-success; and misspelling `outputPattern` as `outputPatern` in required/actual left canonical verification green. Fixes derive the seam from Git, validate provider identity globally before subject filtering, and recursively close/type-check nested schemas. New registry criteria `RM02-EVIDENCE-SUBJECT-BINDING`, `RM02-TYPE-STRICT-SCHEMA`, and `RM02-HISTORY-BOUNDARY` are bidirectionally bound to eight registered must-fail controls. The broad nested-object typo table and exact derived-boundary test are regression guards added after implementation, not claimed as RED-first. Pre-commit Codex review then found that global evidence failure was only observed—not failed—when HEAD was the sole prospective commit, and that non-object collection entries threw before normalization. Both were reproduced RED-first, then fixed by one collection validator used before iteration and by per-commit assessment. Follow-up review found collection validation still omitted exactly-one-gate-step cardinality for unrelated subjects; a focused test reproduced terminal-success RED-first, and collection validation now rejects that ambiguity globally. Security review then found present-but-empty outcome patterns were truthy-optional assertion bypasses; a reason-specific test reproduced that they lacked the required schema diagnostic, and present pattern fields now require non-whitespace content.
## Documentation checklist
- PRD, developer guide, admin guide, governing claim index, sitemap, plan, and scratchpad updated.
@@ -76,7 +78,7 @@ The queue guard's `get_state_from_status_json` runs `python3 - <<'PY'` while pro
## Rebase onto RM-61
- Rebasing `f9746b23` onto main `f4fd5967` completed mechanically with no conflicts.
- The first post-rebase `pnpm gate:verify` correctly failed because the old activation seam `f65e9ea6` made the newly merged pre-registry RM-61 commit part of prospective history even though that commit predates the registry. The activation commit was advanced to the new main parent `f4fd5967`, so own-tree provenance starts with this branch's first registry commit rather than demanding a manifest from an unactivated baseline commit. No gate case or acceptance rule was weakened.
- The first post-rebase `pnpm gate:verify` correctly failed because the old activation seam `f65e9ea6` made the newly merged pre-registry RM-61 commit part of prospective history even though that commit predates the registry. An initial manifest update to `f4fd5967` had the right value but retained an author-controlled mechanism. Independent mutation proved HEAD, HEAD's parent, and the introduction commit could each make history coverage vacuous or partial. The manifest field is now forbidden; the verifier derives the boundary as the parent of the first first-parent registry-introduction commit, and registered must-fail controls reject all three unsafe candidates.
## Risks/blockers
+213 -1
View File
@@ -1,6 +1,5 @@
{
"schemaVersion": 1,
"activationCommit": "f4fd5967fc5d4cbc72d680b199d88224aa855131",
"gateRoots": ["gates"],
"governingClaimFiles": ["docs/remediation/MISSION.md", "docs/remediation/GATE-CLAIMS.md"],
"coverageBoundary": {
@@ -34,6 +33,13 @@
"checkout-preflight/criterion-misbinding",
"checkout-preflight/missing-meaning-provenance",
"checkout-preflight/prose-claim-misbinding",
"checkout-preflight/history-seam-head",
"checkout-preflight/history-seam-parent",
"checkout-preflight/history-seam-introduction",
"checkout-preflight/provider-cross-commit-duplicate",
"checkout-preflight/misspelled-outcome-field",
"checkout-preflight/wrong-outcome-field-type",
"checkout-preflight/empty-outcome-pattern",
"ci-queue-wait/no-status-required",
"ci-queue-wait/unknown-option",
"hook-pre-commit/lint-staged-failure",
@@ -225,6 +231,41 @@
}
],
"caseRefs": ["ci-queue-wait/terminal-success", "ci-queue-wait/unknown-option"]
},
{
"id": "RM02-HISTORY-BOUNDARY",
"originalText": "The history activation boundary is derived as the parent of the first first-parent commit that introduces the registry.",
"currentText": "The history activation boundary is derived as the parent of the first first-parent commit that introduces the registry.",
"claimType": "integrity",
"source": "docs/remediation/TASKS.md#d-17",
"meaningChanges": [],
"caseRefs": [
"checkout-preflight/history-seam-head",
"checkout-preflight/history-seam-parent",
"checkout-preflight/history-seam-introduction"
]
},
{
"id": "RM02-EVIDENCE-SUBJECT-BINDING",
"originalText": "A provider evidence identity cannot establish success for more than one commit subject.",
"currentText": "A provider evidence identity cannot establish success for more than one commit subject.",
"claimType": "integrity",
"source": "docs/remediation/TASKS.md#d-38",
"meaningChanges": [],
"caseRefs": ["checkout-preflight/provider-cross-commit-duplicate"]
},
{
"id": "RM02-TYPE-STRICT-SCHEMA",
"originalText": "Every registry discriminator and comparison field is recursively closed and type-strict so a misspelling cannot disable an assertion.",
"currentText": "Every registry discriminator and comparison field is recursively closed and type-strict so a misspelling cannot disable an assertion.",
"claimType": "integrity",
"source": "docs/remediation/TASKS.md#d-40",
"meaningChanges": [],
"caseRefs": [
"checkout-preflight/misspelled-outcome-field",
"checkout-preflight/wrong-outcome-field-type",
"checkout-preflight/empty-outcome-pattern"
]
}
],
"proseClaims": [
@@ -678,6 +719,177 @@
}
]
}
},
{
"id": "history-seam-head",
"criterionIds": ["RM02-CHECK-RIGHT", "RM02-HISTORY-BOUNDARY"],
"mustFail": true,
"invocation": ["node", "scripts/gate-history-boundary-control.mjs", "head"],
"required": {
"exitCode": 1,
"outputPattern": "history boundary candidate head rejected"
},
"actual": {
"exitCode": 1,
"outputPattern": "history boundary candidate head rejected"
},
"reasonPattern": "derived activation is parent of registry introduction"
},
{
"id": "history-seam-parent",
"criterionIds": ["RM02-CHECK-RIGHT", "RM02-HISTORY-BOUNDARY"],
"mustFail": true,
"invocation": ["node", "scripts/gate-history-boundary-control.mjs", "parent"],
"required": {
"exitCode": 1,
"outputPattern": "history boundary candidate parent rejected"
},
"actual": {
"exitCode": 1,
"outputPattern": "history boundary candidate parent rejected"
},
"reasonPattern": "derived activation is parent of registry introduction"
},
{
"id": "history-seam-introduction",
"criterionIds": ["RM02-CHECK-RIGHT", "RM02-HISTORY-BOUNDARY"],
"mustFail": true,
"invocation": ["node", "scripts/gate-history-boundary-control.mjs", "introduction"],
"required": {
"exitCode": 1,
"outputPattern": "history boundary candidate introduction rejected"
},
"actual": {
"exitCode": 1,
"outputPattern": "history boundary candidate introduction rejected"
},
"reasonPattern": "derived activation is parent of registry introduction"
},
{
"id": "provider-cross-commit-duplicate",
"criterionIds": ["RM02-CHECK-RIGHT", "RM02-EVIDENCE-SUBJECT-BINDING"],
"mustFail": true,
"invocation": ["node", "scripts/gate-provider-binding-control.mjs"],
"required": {
"exitCode": 1,
"outputPattern": "duplicate pipeline identity across commits"
},
"actual": {
"exitCode": 1,
"outputPattern": "duplicate pipeline identity across commits"
},
"reasonPattern": "duplicate pipeline identity across commits"
},
{
"id": "misspelled-outcome-field",
"criterionIds": ["RM02-CHECK-RIGHT", "RM02-TYPE-STRICT-SCHEMA"],
"mustFail": true,
"invocation": [
"node",
"scripts/gate-verify.mjs",
"--root",
".",
"--manifest",
"gates/gates.manifest.json",
"--structure-only"
],
"required": {
"exitCode": 1,
"outputPattern": "required: unknown field outputPatern"
},
"actual": {
"exitCode": 1,
"outputPattern": "required: unknown field outputPatern"
},
"reasonPattern": "required: unknown field outputPatern",
"fixture": {
"copyPaths": [
"gates/gates.manifest.json",
"scripts/gate-verify.mjs",
"scripts/gate-history.mjs"
],
"replaceFiles": [
{
"path": "gates/gates.manifest.json",
"find": "\"required\": {\n \"exitCode\": 0,\n \"outputPattern\": \"checkout preflight passed\"\n },\n \"actual\":",
"replace": "\"required\": {\n \"exitCode\": 0,\n \"outputPatern\": \"checkout preflight passed\"\n },\n \"actual\":"
}
]
}
},
{
"id": "wrong-outcome-field-type",
"criterionIds": ["RM02-CHECK-RIGHT", "RM02-TYPE-STRICT-SCHEMA"],
"mustFail": true,
"invocation": [
"node",
"scripts/gate-verify.mjs",
"--root",
".",
"--manifest",
"gates/gates.manifest.json",
"--structure-only"
],
"required": {
"exitCode": 1,
"outputPattern": "required.exitCode: expected an integer"
},
"actual": {
"exitCode": 1,
"outputPattern": "required.exitCode: expected an integer"
},
"reasonPattern": "required.exitCode: expected an integer",
"fixture": {
"copyPaths": [
"gates/gates.manifest.json",
"scripts/gate-verify.mjs",
"scripts/gate-history.mjs"
],
"replaceFiles": [
{
"path": "gates/gates.manifest.json",
"find": "\"required\": {\n \"exitCode\": 0,\n \"outputPattern\": \"checkout preflight passed\"\n },\n \"actual\":",
"replace": "\"required\": {\n \"exitCode\": \"0\",\n \"outputPattern\": \"checkout preflight passed\"\n },\n \"actual\":"
}
]
}
},
{
"id": "empty-outcome-pattern",
"criterionIds": ["RM02-CHECK-RIGHT", "RM02-TYPE-STRICT-SCHEMA"],
"mustFail": true,
"invocation": [
"node",
"scripts/gate-verify.mjs",
"--root",
".",
"--manifest",
"gates/gates.manifest.json",
"--structure-only"
],
"required": {
"exitCode": 1,
"outputPattern": "required.outputPattern: expected a non-empty pattern"
},
"actual": {
"exitCode": 1,
"outputPattern": "required.outputPattern: expected a non-empty pattern"
},
"reasonPattern": "required.outputPattern: expected a non-empty pattern",
"fixture": {
"copyPaths": [
"gates/gates.manifest.json",
"scripts/gate-verify.mjs",
"scripts/gate-history.mjs"
],
"replaceFiles": [
{
"path": "gates/gates.manifest.json",
"find": "\"required\": {\n \"exitCode\": 0,\n \"outputPattern\": \"checkout preflight passed\"\n },\n \"actual\":",
"replace": "\"required\": {\n \"exitCode\": 0,\n \"outputPattern\": \" \"\n },\n \"actual\":"
}
]
}
}
]
},
+40
View File
@@ -0,0 +1,40 @@
#!/usr/bin/env node
import { spawnSync } from 'node:child_process';
import { deriveHistoryBoundary } from './gate-history.mjs';
const candidateKind = process.argv[2];
const root = process.cwd();
const boundary = deriveHistoryBoundary(root);
function revParse(revision) {
const result = spawnSync('git', ['rev-parse', revision], {
cwd: root,
encoding: 'utf8',
});
if (result.status !== 0) {
process.stderr.write(`history boundary control could not resolve ${revision}\n`);
process.exit(2);
}
return result.stdout.trim();
}
const candidates = {
head: revParse('HEAD'),
parent: revParse('HEAD^'),
introduction: boundary.introductionCommit,
};
if (!Object.hasOwn(candidates, candidateKind)) {
process.stderr.write(`unknown history boundary candidate ${String(candidateKind)}\n`);
process.exit(2);
}
const candidate = candidates[candidateKind];
if (candidate === boundary.activationCommit) {
process.stdout.write(`history boundary candidate ${candidateKind} matched derived activation\n`);
process.exit(0);
}
process.stderr.write(
`history boundary candidate ${candidateKind} rejected: derived activation is parent of registry introduction\n`,
);
process.exit(1);
+78 -27
View File
@@ -22,6 +22,33 @@ export async function listProspectiveCommits(root, activationCommit, head = 'HEA
return result.stdout.trim() ? result.stdout.trim().split('\n') : [];
}
export function deriveHistoryBoundary(root, head = 'HEAD') {
const introductions = git(root, [
'log',
'--first-parent',
'--diff-filter=A',
'--format=%H',
'--reverse',
head,
'--',
'gates/gates.manifest.json',
])
.stdout.trim()
.split('\n')
.filter(Boolean);
if (introductions.length === 0) {
throw new Error('history boundary cannot be derived: registry introduction is absent');
}
const introductionCommit = introductions[0];
const parent = git(root, ['rev-parse', `${introductionCommit}^`], { allowFailure: true });
if (parent.status !== 0 || !parent.stdout.trim()) {
throw new Error(
`history boundary cannot be derived: registry introduction ${introductionCommit} has no parent`,
);
}
return { activationCommit: parent.stdout.trim(), introductionCommit };
}
export async function readManifestAtCommit(root, commit) {
const result = git(root, ['show', `${commit}:gates/gates.manifest.json`]);
return JSON.parse(result.stdout);
@@ -198,15 +225,8 @@ export async function replayCommit(root, commit) {
}
}
export function assessProviderEvidence(commit, pipelines) {
const matches = pipelines.filter((candidate) => candidate.commit === commit);
if (matches.length === 0) {
return {
state: 'absent',
detail:
'no retained provider record was supplied; retention expiry and never-ran are not inferred',
};
}
export function validateProviderEvidenceCollection(pipelines) {
if (!Array.isArray(pipelines)) return 'provider evidence collection is malformed';
const pipelineStates = new Set(['success', 'failure', 'error', 'pending', 'running', 'queued']);
const stepStates = new Set([
'success',
@@ -217,9 +237,11 @@ export function assessProviderEvidence(commit, pipelines) {
'queued',
'skipped',
]);
const numbers = matches.map((candidate) => candidate.number);
const malformed = matches.some(
const malformed = pipelines.some(
(candidate) =>
!candidate ||
typeof candidate !== 'object' ||
Array.isArray(candidate) ||
typeof candidate.commit !== 'string' ||
candidate.commit.length === 0 ||
!Number.isInteger(candidate.number) ||
@@ -227,15 +249,43 @@ export function assessProviderEvidence(commit, pipelines) {
!Array.isArray(candidate.steps) ||
candidate.steps.some(
(step) =>
typeof step?.name !== 'string' ||
typeof step?.status !== 'string' ||
!step ||
typeof step !== 'object' ||
Array.isArray(step) ||
typeof step.name !== 'string' ||
typeof step.status !== 'string' ||
!stepStates.has(step.status),
),
);
if (malformed || new Set(numbers).size !== numbers.length) {
if (malformed) return 'provider records are malformed';
const ambiguousGateRecord = pipelines.find(
(candidate) => candidate.steps.filter((step) => step.name === 'gate-verify').length !== 1,
);
if (ambiguousGateRecord) {
const count = ambiguousGateRecord.steps.filter((step) => step.name === 'gate-verify').length;
return `provider record ${ambiguousGateRecord.number} has ambiguous gate-verify step count ${count}`;
}
const numbers = pipelines.map((candidate) => candidate.number);
if (new Set(numbers).size !== numbers.length) {
return 'duplicate pipeline identity across commits in provider evidence collection';
}
return undefined;
}
export function assessProviderEvidence(commit, pipelines) {
const collectionFailure = validateProviderEvidenceCollection(pipelines);
if (collectionFailure) {
return {
state: 'terminal-failure',
detail: 'provider records are malformed or have ambiguous pipeline numbers',
detail: collectionFailure,
};
}
const matches = pipelines.filter((candidate) => candidate.commit === commit);
if (matches.length === 0) {
return {
state: 'absent',
detail:
'no retained provider record was supplied; retention expiry and never-ran are not inferred',
};
}
const pipeline = [...matches].sort((left, right) => right.number - left.number)[0];
@@ -279,19 +329,16 @@ export async function verifyHistory({ root, manifest }) {
const failures = [];
const observations = [];
const head = git(root, ['rev-parse', 'HEAD']).stdout.trim();
if (!manifest.activationCommit) {
failures.push('history activationCommit is missing');
return { failures, observations };
}
const activationCheck = git(
root,
['merge-base', '--is-ancestor', manifest.activationCommit, head],
{ allowFailure: true },
);
if (activationCheck.status !== 0) {
if (Object.hasOwn(manifest, 'activationCommit')) {
failures.push(
`history activation commit ${manifest.activationCommit} is not an ancestor of ${head}`,
'author-controlled activationCommit is forbidden; history boundary is derived from the registry introduction',
);
}
let boundary;
try {
boundary = deriveHistoryBoundary(root, head);
} catch (error) {
failures.push(error.message);
return { failures, observations };
}
const onMain = isMainCommit(root, head);
@@ -311,7 +358,11 @@ export async function verifyHistory({ root, manifest }) {
}
const pipelines = onMain ? await loadProviderEvidence() : [];
const commits = await listProspectiveCommits(root, manifest.activationCommit, head);
const collectionFailure = validateProviderEvidenceCollection(pipelines);
if (collectionFailure) {
failures.push(`provider evidence collection invalid: ${collectionFailure}`);
}
const commits = await listProspectiveCommits(root, boundary.activationCommit, head);
for (const commit of commits) {
let commitManifest;
try {
+165 -1
View File
@@ -6,6 +6,7 @@ import test from 'node:test';
import {
assessProviderEvidence,
deriveHistoryBoundary,
listProspectiveCommits,
readManifestAtCommit,
replayCommit,
@@ -285,7 +286,7 @@ test('PR verification states the RM-60 boundary without executing an intermediat
try {
const result = await verifyHistory({
root,
manifest: { schemaVersion: 1, activationCommit: activation },
manifest: { schemaVersion: 1 },
});
assert.deepEqual(result.failures, []);
assert.ok(
@@ -305,6 +306,169 @@ test('PR verification states the RM-60 boundary without executing an intermediat
}
});
test('derived history boundary includes the registry-introduction commit', async () => {
const root = `${fixtureRoot}-derived-boundary`;
await rm(root, { recursive: true, force: true });
await mkdir(root, { recursive: true });
git(root, 'init', '-q');
git(root, 'config', 'user.name', 'gate-test');
git(root, 'config', 'user.email', '[email protected]');
await writeFile(path.join(root, 'baseline.txt'), 'baseline\n');
git(root, 'add', '.');
git(root, 'commit', '-m', 'baseline');
const baseline = git(root, 'rev-parse', 'HEAD');
await mkdir(path.join(root, 'gates'), { recursive: true });
await writeFile(path.join(root, 'gates', 'gates.manifest.json'), '{"schemaVersion":1}\n');
git(root, 'add', '.');
git(root, 'commit', '-m', 'registry introduction');
const introduction = git(root, 'rev-parse', 'HEAD');
await writeFile(path.join(root, 'later.txt'), 'later\n');
git(root, 'add', '.');
git(root, 'commit', '-m', 'later');
const head = git(root, 'rev-parse', 'HEAD');
assert.deepEqual(deriveHistoryBoundary(root, head), {
activationCommit: baseline,
introductionCommit: introduction,
});
assert.deepEqual(await listProspectiveCommits(root, baseline, head), [introduction, head]);
});
test('author-controlled activation seams cannot omit registry-era history', async () => {
const root = `${fixtureRoot}-activation-seam`;
await rm(root, { recursive: true, force: true });
await mkdir(path.join(root, 'gates'), { recursive: true });
git(root, 'init', '-q');
git(root, 'config', 'user.name', 'gate-test');
git(root, 'config', 'user.email', '[email protected]');
await writeFile(path.join(root, 'gates', 'gates.manifest.json'), '{"schemaVersion":1}\n');
git(root, 'add', '.');
git(root, 'commit', '-m', 'registry introduction');
const introduction = git(root, 'rev-parse', 'HEAD');
await writeFile(path.join(root, 'one.txt'), 'one\n');
git(root, 'add', '.');
git(root, 'commit', '-m', 'one');
await writeFile(path.join(root, 'two.txt'), 'two\n');
git(root, 'add', '.');
git(root, 'commit', '-m', 'two');
const head = git(root, 'rev-parse', 'HEAD');
const parent = git(root, 'rev-parse', 'HEAD^');
const previousBranch = process.env.CI_COMMIT_BRANCH;
process.env.CI_COMMIT_BRANCH = 'feature/activation-seam';
try {
for (const candidate of [head, parent, introduction]) {
const result = await verifyHistory({
root,
manifest: { schemaVersion: 1, activationCommit: candidate },
});
assert.match(result.failures.join('\n'), /author-controlled activationCommit.*forbidden/i);
}
} finally {
if (previousBranch === undefined) delete process.env.CI_COMMIT_BRANCH;
else process.env.CI_COMMIT_BRANCH = previousBranch;
}
});
test('globally invalid provider evidence fails when HEAD is the only prospective commit', async () => {
const root = `${fixtureRoot}-head-only-evidence`;
await rm(root, { recursive: true, force: true });
await mkdir(root, { recursive: true });
git(root, 'init', '-q');
git(root, 'config', 'user.name', 'gate-test');
git(root, 'config', 'user.email', '[email protected]');
await writeFile(path.join(root, 'baseline.txt'), 'baseline\n');
git(root, 'add', '.');
git(root, 'commit', '-m', 'baseline');
await mkdir(path.join(root, 'gates'), { recursive: true });
await writeFile(path.join(root, 'gates', 'gates.manifest.json'), '{"schemaVersion":1}\n');
git(root, 'add', '.');
git(root, 'commit', '-m', 'registry introduction');
const head = git(root, 'rev-parse', 'HEAD');
const evidenceFile = path.join(root, 'provider-evidence.json');
await writeFile(
evidenceFile,
JSON.stringify([
{
commit: head,
number: 7,
status: 'success',
steps: [{ name: 'gate-verify', status: 'success' }],
},
{
commit: 'other-subject',
number: 7,
status: 'success',
steps: [{ name: 'gate-verify', status: 'success' }],
},
]),
);
const previousBranch = process.env.CI_COMMIT_BRANCH;
const previousEvidence = process.env.GATE_PROVIDER_EVIDENCE_FILE;
process.env.CI_COMMIT_BRANCH = 'main';
process.env.GATE_PROVIDER_EVIDENCE_FILE = evidenceFile;
try {
const result = await verifyHistory({ root, manifest: { schemaVersion: 1 } });
assert.match(result.failures.join('\n'), /duplicate pipeline identity across commits/i);
} finally {
if (previousBranch === undefined) delete process.env.CI_COMMIT_BRANCH;
else process.env.CI_COMMIT_BRANCH = previousBranch;
if (previousEvidence === undefined) delete process.env.GATE_PROVIDER_EVIDENCE_FILE;
else process.env.GATE_PROVIDER_EVIDENCE_FILE = previousEvidence;
}
});
test('collection-wide validation rejects ambiguous gate steps on unrelated commits', () => {
const records = [
{
commit: 'target',
number: 7,
status: 'success',
steps: [{ name: 'gate-verify', status: 'success' }],
},
{
commit: 'unrelated',
number: 8,
status: 'success',
steps: [
{ name: 'gate-verify', status: 'success' },
{ name: 'gate-verify', status: 'failure' },
],
},
];
const result = assessProviderEvidence('target', records);
assert.equal(result.state, 'terminal-failure');
assert.match(result.detail, /ambiguous gate-verify step count/i);
});
test('provider evidence rejects non-object collection entries without crashing', () => {
for (const record of [null, [], 'text', 42]) {
const result = assessProviderEvidence('aaa', [record]);
assert.equal(result.state, 'terminal-failure');
assert.match(result.detail, /malformed/i);
}
});
test('provider evidence rejects duplicate pipeline identity across commits', () => {
const records = [
{
commit: 'aaa',
number: 7,
status: 'success',
steps: [{ name: 'gate-verify', status: 'success' }],
},
{
commit: 'bbb',
number: 7,
status: 'success',
steps: [{ name: 'gate-verify', status: 'success' }],
},
];
const result = assessProviderEvidence('aaa', records);
assert.equal(result.state, 'terminal-failure');
assert.match(result.detail, /duplicate pipeline.*across.*commit|global.*pipeline.*identity/i);
});
test('provider evidence distinguishes retained success, failure, and absent history', () => {
const pipelines = [
{
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env node
import { assessProviderEvidence } from './gate-history.mjs';
const records = [
{
commit: 'subject-a',
number: 7,
status: 'success',
steps: [{ name: 'gate-verify', status: 'success' }],
},
{
commit: 'subject-b',
number: 7,
status: 'success',
steps: [{ name: 'gate-verify', status: 'success' }],
},
];
const result = assessProviderEvidence('subject-a', records);
if (
result.state === 'terminal-failure' &&
/duplicate pipeline identity across commits/i.test(result.detail)
) {
process.stderr.write(`${result.detail}\n`);
process.exit(1);
}
process.stdout.write(
`cross-commit duplicate was not rejected: ${result.state} (${result.detail})\n`,
);
process.exit(0);
+236 -16
View File
@@ -288,6 +288,100 @@ function rejectDuplicateIds(values, label, failures) {
}
}
function requireString(value, label, failures, { allowEmpty = false } = {}) {
if (typeof value !== 'string' || (!allowEmpty && value.length === 0)) {
failures.push(`${label}: expected ${allowEmpty ? 'a string' : 'a non-empty string'}`);
}
}
function validateStringArray(value, label, failures, { allowEmpty = false } = {}) {
if (!Array.isArray(value) || (!allowEmpty && value.length === 0)) {
failures.push(`${label}: expected ${allowEmpty ? 'an array' : 'a non-empty array'}`);
return;
}
if (value.some((entry) => typeof entry !== 'string' || entry.length === 0)) {
failures.push(`${label}: entries must be non-empty strings`);
}
}
function validateOutcome(value, label, failures) {
rejectUnknownKeys(
value,
new Set(['exitCode', 'outputPattern', 'notOutputPattern']),
label,
failures,
);
if (typeof value?.exitCode !== 'number' || !Number.isInteger(value.exitCode)) {
failures.push(`${label}.exitCode: expected an integer`);
}
for (const key of ['outputPattern', 'notOutputPattern']) {
if (value?.[key] !== undefined && typeof value[key] !== 'string') {
failures.push(`${label}.${key}: expected a string`);
} else if (typeof value?.[key] === 'string' && value[key].trim().length === 0) {
failures.push(`${label}.${key}: expected a non-empty pattern`);
}
}
}
function validateFixture(fixture, label, failures) {
if (fixture === undefined) return;
rejectUnknownKeys(
fixture,
new Set(['writeFiles', 'replaceFiles', 'removePaths', 'copyPaths']),
label,
failures,
);
const writeFiles = Array.isArray(fixture?.writeFiles) ? fixture.writeFiles : [];
const replaceFiles = Array.isArray(fixture?.replaceFiles) ? fixture.replaceFiles : [];
for (const [index, entry] of writeFiles.entries()) {
rejectUnknownKeys(
entry,
new Set(['path', 'content', 'mode']),
`${label}.writeFiles[${index}]`,
failures,
);
requireString(entry?.path, `${label}.writeFiles[${index}].path`, failures);
if (typeof entry?.content !== 'string')
failures.push(`${label}.writeFiles[${index}].content: expected a string`);
if (entry?.mode !== undefined && (!Number.isInteger(entry.mode) || entry.mode < 0)) {
failures.push(`${label}.writeFiles[${index}].mode: expected a non-negative integer`);
}
}
for (const [index, entry] of replaceFiles.entries()) {
rejectUnknownKeys(
entry,
new Set(['path', 'find', 'replace']),
`${label}.replaceFiles[${index}]`,
failures,
);
requireString(entry?.path, `${label}.replaceFiles[${index}].path`, failures);
if (typeof entry?.find !== 'string')
failures.push(`${label}.replaceFiles[${index}].find: expected a string`);
if (typeof entry?.replace !== 'string')
failures.push(`${label}.replaceFiles[${index}].replace: expected a string`);
}
for (const key of ['removePaths', 'copyPaths']) {
if (fixture?.[key] !== undefined)
validateStringArray(fixture[key], `${label}.${key}`, failures, { allowEmpty: true });
}
for (const key of ['writeFiles', 'replaceFiles']) {
if (fixture?.[key] !== undefined && !Array.isArray(fixture[key])) {
failures.push(`${label}.${key}: expected an array`);
}
}
}
function validateEnvironment(environment, label, failures) {
if (environment === undefined) return;
if (!environment || typeof environment !== 'object' || Array.isArray(environment)) {
failures.push(`${label}: expected an object`);
return;
}
for (const [key, value] of Object.entries(environment)) {
if (!key || typeof value !== 'string') failures.push(`${label}.${key}: expected a string`);
}
}
function validateClosedSchema(manifest, failures) {
if (manifest.schemaVersion !== 1)
failures.push(`unsupported schemaVersion ${String(manifest.schemaVersion)}`);
@@ -295,7 +389,6 @@ function validateClosedSchema(manifest, failures) {
manifest,
new Set([
'schemaVersion',
'activationCommit',
'gateRoots',
'governingClaimFiles',
'coverageBoundary',
@@ -308,6 +401,43 @@ function validateClosedSchema(manifest, failures) {
'manifest',
failures,
);
validateStringArray(manifest.gateRoots, 'manifest.gateRoots', failures);
validateStringArray(manifest.governingClaimFiles, 'manifest.governingClaimFiles', failures, {
allowEmpty: true,
});
rejectUnknownKeys(
manifest.coverageBoundary,
new Set(['included', 'excluded', 'trackedBy']),
'coverageBoundary',
failures,
);
validateStringArray(manifest.coverageBoundary?.included, 'coverageBoundary.included', failures);
validateStringArray(manifest.coverageBoundary?.excluded, 'coverageBoundary.excluded', failures, {
allowEmpty: true,
});
requireString(manifest.coverageBoundary?.trackedBy, 'coverageBoundary.trackedBy', failures);
if (manifest.mergeAssertions !== undefined) {
rejectUnknownKeys(
manifest.mergeAssertions,
new Set([
'mode',
'providerEvidence',
'deferredReplayOwner',
'trustDependencies',
'postMergeResponse',
]),
'mergeAssertions',
failures,
);
for (const key of ['mode', 'providerEvidence', 'deferredReplayOwner', 'postMergeResponse']) {
requireString(manifest.mergeAssertions?.[key], `mergeAssertions.${key}`, failures);
}
validateStringArray(
manifest.mergeAssertions?.trustDependencies,
'mergeAssertions.trustDependencies',
failures,
);
}
rejectDuplicateIds(manifest.criteria, 'criterion', failures);
for (const criterion of manifest.criteria ?? []) {
rejectUnknownKeys(
@@ -324,6 +454,27 @@ function validateClosedSchema(manifest, failures) {
`criterion ${criterion.id}`,
failures,
);
for (const key of ['id', 'originalText', 'currentText', 'claimType', 'source']) {
requireString(criterion[key], `criterion ${criterion.id}.${key}`, failures);
}
if (!Array.isArray(criterion.meaningChanges)) {
failures.push(`criterion ${criterion.id}.meaningChanges: expected an array`);
}
for (const [index, change] of (criterion.meaningChanges ?? []).entries()) {
rejectUnknownKeys(
change,
new Set(['originalText', 'restatement', 'reason', 'finding', 'task', 'date']),
`criterion ${criterion.id}.meaningChanges[${index}]`,
failures,
);
for (const key of ['originalText', 'restatement', 'reason', 'finding', 'task', 'date']) {
requireString(
change?.[key],
`criterion ${criterion.id}.meaningChanges[${index}].${key}`,
failures,
);
}
}
if (!Array.isArray(criterion.caseRefs) || criterion.caseRefs.length === 0) {
failures.push(`${criterion.id}: no declared exercising cases`);
} else {
@@ -343,6 +494,9 @@ function validateClosedSchema(manifest, failures) {
`prose claim ${claim.id}`,
failures,
);
for (const key of ['id', 'criterionId', 'caseRef']) {
requireString(claim[key], `GATE-CLAIM:${claim.id}.${key}`, failures);
}
if (typeof claim.caseRef !== 'string' || claim.caseRef.length === 0) {
failures.push(`GATE-CLAIM:${claim.id} has no declared exercising case`);
}
@@ -363,15 +517,22 @@ function validateClosedSchema(manifest, failures) {
`compatibility scenario ${scenario.id}`,
failures,
);
for (const key of ['id', 'construction']) {
requireString(scenario[key], `compatibility scenario ${scenario.id}.${key}`, failures);
}
if (!Array.isArray(scenario.caseRefs) || scenario.caseRefs.length === 0) {
failures.push(`${scenario.id}: compatibility construction has no referenced conditions`);
} else {
validateStringArray(scenario.caseRefs, `${scenario.id}.caseRefs`, failures);
}
if (!Array.isArray(scenario.invocation) || scenario.invocation.length === 0) {
failures.push(`${scenario.id}: compatibility construction invocation is missing`);
} else {
validateStringArray(scenario.invocation, `${scenario.id}.invocation`, failures);
}
if (typeof scenario.expected?.exitCode !== 'number') {
failures.push(`${scenario.id}: compatibility construction exact expected exit is missing`);
}
validateOutcome(scenario.expected, `${scenario.id}.expected`, failures);
validateEnvironment(scenario.environment, `${scenario.id}.environment`, failures);
validateFixture(scenario.fixture, `${scenario.id}.fixture`, failures);
}
rejectDuplicateIds(manifest.gates, 'gate', failures);
for (const gate of manifest.gates ?? []) {
@@ -389,13 +550,57 @@ function validateClosedSchema(manifest, failures) {
`gate ${gate.id}`,
failures,
);
requireString(gate.id, `gate ${gate.id}.id`, failures);
requireString(gate.source, `gate ${gate.id}.source`, failures);
rejectDuplicateIds(gate.cases, `case in gate ${gate.id}`, failures);
if (!Array.isArray(gate.invocation) || gate.invocation.length === 0) {
failures.push(`${gate.id}: exact invocation is missing`);
} else {
validateStringArray(gate.invocation, `${gate.id}.invocation`, failures);
}
if (gate.discoveryAliases !== undefined) {
validateStringArray(gate.discoveryAliases, `${gate.id}.discoveryAliases`, failures, {
allowEmpty: true,
});
}
rejectUnknownKeys(
gate.deployment,
new Set(['kind', 'reason', 'source', 'path', 'unavailableOwner', 'observedSha256']),
`${gate.id}.deployment`,
failures,
);
if (!['none', 'file'].includes(gate.deployment?.kind)) {
failures.push(`${gate.id}: unsupported deployment kind ${String(gate.deployment?.kind)}`);
} else if (gate.deployment.kind === 'none') {
requireString(gate.deployment.reason, `${gate.id}.deployment.reason`, failures);
} else {
for (const key of ['source', 'path', 'unavailableOwner', 'observedSha256']) {
requireString(gate.deployment[key], `${gate.id}.deployment.${key}`, failures);
}
}
rejectUnknownKeys(
gate.inertMutation,
new Set(['file', 'find', 'replace', 'caseId', 'expected', 'sandboxFiles']),
`${gate.id}.inertMutation`,
failures,
);
for (const key of ['file', 'find', 'replace']) {
if (typeof gate.inertMutation?.[key] !== 'string') {
failures.push(`${gate.id}.inertMutation.${key}: expected a string`);
}
}
if (gate.inertMutation?.caseId !== undefined) {
requireString(gate.inertMutation.caseId, `${gate.id}.inertMutation.caseId`, failures);
}
if (gate.inertMutation?.sandboxFiles !== undefined) {
validateStringArray(
gate.inertMutation.sandboxFiles,
`${gate.id}.inertMutation.sandboxFiles`,
failures,
{ allowEmpty: true },
);
}
validateOutcome(gate.inertMutation?.expected, `${gate.id}.inertMutation.expected`, failures);
for (const gateCase of gate.cases ?? []) {
rejectUnknownKeys(
gateCase,
@@ -414,25 +619,40 @@ function validateClosedSchema(manifest, failures) {
`${gate.id}/${gateCase.id}`,
failures,
);
if (
typeof gateCase.required?.exitCode !== 'number' ||
typeof gateCase.actual?.exitCode !== 'number'
) {
failures.push(
`${gate.id}/${gateCase.id}: required and actual exact exit codes are mandatory`,
);
requireString(gateCase.id, `${gate.id}/${gateCase.id}.id`, failures);
validateStringArray(
gateCase.criterionIds,
`${gate.id}/${gateCase.id}.criterionIds`,
failures,
);
if (typeof gateCase.mustFail !== 'boolean') {
failures.push(`${gate.id}/${gateCase.id}.mustFail: expected a boolean`);
}
if (
gateCase.invocation &&
(!Array.isArray(gateCase.invocation) || gateCase.invocation.length === 0)
) {
failures.push(`${gate.id}/${gateCase.id}: case invocation must be non-empty`);
validateOutcome(gateCase.required, `${gate.id}/${gateCase.id}.required`, failures);
validateOutcome(gateCase.actual, `${gate.id}/${gateCase.id}.actual`, failures);
if (gateCase.invocation !== undefined) {
validateStringArray(gateCase.invocation, `${gate.id}/${gateCase.id}.invocation`, failures);
}
if (typeof gateCase.reasonPattern !== 'string') {
failures.push(`${gate.id}/${gateCase.id}.reasonPattern: expected a string`);
}
if (gateCase.mustFail === true && !gateCase.reasonPattern?.trim()) {
failures.push(
`${gate.id}/${gateCase.id}: must-fail case requires a non-empty reasonPattern`,
);
}
validateEnvironment(gateCase.environment, `${gate.id}/${gateCase.id}.environment`, failures);
validateFixture(gateCase.fixture, `${gate.id}/${gateCase.id}.fixture`, failures);
if (gateCase.defect !== undefined) {
rejectUnknownKeys(
gateCase.defect,
new Set(['owner', 'reason']),
`${gate.id}/${gateCase.id}.defect`,
failures,
);
requireString(gateCase.defect.owner, `${gate.id}/${gateCase.id}.defect.owner`, failures);
requireString(gateCase.defect.reason, `${gate.id}/${gateCase.id}.defect.reason`, failures);
}
}
}
}
+111 -1
View File
@@ -18,7 +18,6 @@ async function fixture(name = 'case') {
function baseManifest() {
return {
schemaVersion: 1,
activationCommit: null,
gateRoots: ['gates'],
governingClaimFiles: [],
coverageBoundary: { included: ['meta fixture'], excluded: [], trackedBy: 'RM-54' },
@@ -157,6 +156,117 @@ test('duplicate stable ids and unsupported schema versions are rejected', async
assert.match(output(result), /duplicate criterion id META-CRIT-1/i);
});
test('misspelled nested assertion fields are rejected instead of becoming optional', async () => {
const root = await fixture('nested-schema-typo');
await writeGate(root);
const manifest = baseManifest();
manifest.gates[0].cases[0].required.outputPatern = 'META_REJECT';
manifest.gates[0].cases[0].actual.outputPatern = 'META_REJECT';
await writeManifest(root, manifest);
const result = verify(root);
assert.notEqual(result.status, 0);
assert.match(output(result), /required.*unknown field outputPatern/i);
assert.match(output(result), /actual.*unknown field outputPatern/i);
});
test('present outcome patterns cannot be empty assertion bypasses', async () => {
const root = await fixture('nested-schema-empty-patterns');
await writeGate(root);
const manifest = baseManifest();
manifest.gates[0].cases[0].required.outputPattern = '';
manifest.gates[0].cases[0].actual.notOutputPattern = ' ';
await writeManifest(root, manifest);
const result = verify(root, ['--structure-only']);
assert.notEqual(result.status, 0);
assert.match(output(result), /required.outputPattern: expected a non-empty pattern/i);
assert.match(output(result), /actual.notOutputPattern: expected a non-empty pattern/i);
});
test('nested discriminator and comparison fields reject wrong types', async () => {
const root = await fixture('nested-schema-types');
await writeGate(root);
const manifest = baseManifest();
manifest.gates[0].cases[0].mustFail = 'true';
manifest.gates[0].cases[0].required.exitCode = '7';
manifest.gates[0].cases[0].actual.outputPattern = 7;
await writeManifest(root, manifest);
const result = verify(root, ['--structure-only']);
assert.notEqual(result.status, 0);
assert.match(output(result), /mustFail: expected a boolean/i);
assert.match(output(result), /required.exitCode: expected an integer/i);
assert.match(output(result), /actual.outputPattern: expected a string/i);
});
test('recursive closed-schema guards reject unknown fields in every nested assertion object', async () => {
const source = JSON.parse(
await readFile(path.join(process.cwd(), 'gates', 'gates.manifest.json'), 'utf8'),
);
const checkout = source.gates.find((gate) => gate.id === 'checkout-preflight');
const stale = checkout.cases.find((gateCase) => gateCase.id === 'stale-build-lock');
const queue = source.gates.find((gate) => gate.id === 'ci-queue-wait');
const queueCase = queue.cases.find((gateCase) => gateCase.id === 'terminal-success');
const targets = [
['coverageBoundary', (manifest) => manifest.coverageBoundary],
['mergeAssertions', (manifest) => manifest.mergeAssertions],
[
'meaningChanges',
(manifest) =>
manifest.criteria.find((criterion) => criterion.meaningChanges.length).meaningChanges[0],
],
['proseClaims', (manifest) => manifest.proseClaims[0]],
['compatibility expected', (manifest) => manifest.compatibilityScenarios[0].expected],
[
'deployment',
(manifest) => manifest.gates.find((gate) => gate.id === 'ci-queue-wait').deployment,
],
['inertMutation', (manifest) => manifest.gates[0].inertMutation],
['inert expected', (manifest) => manifest.gates[0].inertMutation.expected],
['required', (manifest) => manifest.gates[0].cases[0].required],
['actual', (manifest) => manifest.gates[0].cases[0].actual],
[
'fixture',
(manifest) =>
manifest.gates
.find((gate) => gate.id === 'checkout-preflight')
.cases.find((gateCase) => gateCase.id === 'stale-build-lock').fixture,
],
[
'write entry',
(manifest) =>
manifest.gates
.find((gate) => gate.id === 'checkout-preflight')
.cases.find((gateCase) => gateCase.id === 'stale-build-lock').fixture.writeFiles[0],
],
[
'replace entry',
(manifest) =>
manifest.gates
.find((gate) => gate.id === 'checkout-preflight')
.cases.find((gateCase) => gateCase.id === 'criterion-misbinding').fixture.replaceFiles[0],
],
[
'defect',
(manifest) =>
manifest.gates
.find((gate) => gate.id === 'ci-queue-wait')
.cases.find((gateCase) => gateCase.id === 'terminal-success').defect,
],
];
assert.ok(stale.fixture && queueCase.defect);
for (const [name, select] of targets) {
const root = await fixture(`recursive-${name.replaceAll(' ', '-')}`);
const manifest = structuredClone(source);
select(manifest).unexpectedNestedField = true;
await writeManifest(root, manifest);
const result = verify(root, ['--structure-only']);
assert.notEqual(result.status, 0, `${name}: ${output(result)}`);
assert.match(output(result), /unknown field unexpectedNestedField/i, name);
}
});
test('manifest-controlled fixture paths cannot escape the sandbox', async () => {
const root = await fixture('path-traversal');
await writeGate(root);