Compare commits
9
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f9746b23c8 | ||
|
|
38f1b249cc | ||
|
|
9b4d4beb0e | ||
|
|
e4c30a33e8 | ||
|
|
ada0cbe60e | ||
|
|
bd603da4b0 | ||
|
|
444662e79a | ||
|
|
f65e9ea656 | ||
|
|
f58b3699a6 |
+17
-3
@@ -68,15 +68,29 @@ steps:
|
||||
- bash packages/mosaic/framework/tools/quality/scripts/test-upgrade-durable-snapshot.sh
|
||||
- bash packages/mosaic/framework/tools/quality/scripts/test-install-migration.sh
|
||||
|
||||
# Anti-inert-gate registry. Deliberately unconditional: no path filter and no
|
||||
# step-level `when`, because a gate can be disabled by changes outside its own path.
|
||||
gate-verify:
|
||||
image: *node_image
|
||||
# Woodpecker's shallow marker makes merge-base reject even present parents;
|
||||
# full history is required for activation ancestry and manifest provenance.
|
||||
commands:
|
||||
- *enable_pnpm
|
||||
- apk add --no-cache bubblewrap
|
||||
- if [ -f .git/shallow ]; then git fetch --unshallow --no-tags origin; fi
|
||||
- pnpm gate:verify
|
||||
depends_on:
|
||||
- install
|
||||
- sanitization
|
||||
- upgrade-guard
|
||||
|
||||
typecheck:
|
||||
image: *node_image
|
||||
commands:
|
||||
- *enable_pnpm
|
||||
- pnpm typecheck
|
||||
depends_on:
|
||||
- install
|
||||
- sanitization
|
||||
- upgrade-guard
|
||||
- gate-verify
|
||||
|
||||
# lint, format, and test are independent — run in parallel after typecheck
|
||||
lint:
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
# Administrator Guide
|
||||
|
||||
- [Gate registry operations](quality-gate-registry.md)
|
||||
@@ -0,0 +1,35 @@
|
||||
# Gate Registry Operations
|
||||
|
||||
## Routine verification
|
||||
|
||||
Run `pnpm gate:verify` from a dependency-installed checkout. Exit zero means registry observations matched their declared `actual` values; it does **not** assert required-behavior conformance while deltas remain. Open `DEFECT` records are checked descriptions of current behavior with tracked owners, never successful gate outcomes.
|
||||
|
||||
Investigate any of these immediately:
|
||||
|
||||
- `GATE VERIFY FAILED` — registry structure, observed behavior, provenance, claim binding, source/deployment identity, or negative-control detection changed. The verifier aggregates independent phase failures, so repair the responsible stable-ID diagnostics as well as any accompanying stale-fixture error; do not treat the generic error as a substitute.
|
||||
- `unregistered gate` — an executable appeared under a declared gate root without a registry entry.
|
||||
- `no negative control` — a gate has no must-fail case.
|
||||
- `DEPLOYED IDENTITY UNAVAILABLE` — the runner cannot reach the installed enforcing copy. The pinned observation is checked, but live equality is not asserted.
|
||||
- `PROVIDER EVIDENCE ... ABSENT` — retained external history was unavailable; do not infer merge-time success.
|
||||
|
||||
## Updating a gate
|
||||
|
||||
1. Add or change the criterion, its exact criterion-side `caseRefs`, and matching case-side `criterionIds`.
|
||||
2. Observe the must-fail case fail for its own stated reason; moving the binding to any undeclared case must fail verification.
|
||||
3. Declare an exact inerting mutation and observe the verifier detect it.
|
||||
4. If required and actual behavior differ, add a tracked remediation owner and justification.
|
||||
5. If meaning changed, append provenance; never replace the original silently.
|
||||
6. For an installed counterpart, verify live byte identity and update the observed digest only from measured evidence.
|
||||
7. Run focused verifier tests, `pnpm gate:verify`, and the repository baseline gates.
|
||||
|
||||
Do not add an ownerless exception or describe an open delta as pass/green/OK.
|
||||
|
||||
## CI behavior
|
||||
|
||||
Woodpecker runs `gate-verify` on every pull request and protected-main push without path filtering. This is deliberate: changes outside gate files can make a gate inert. The step unshallows the checkout so activation ancestry and historical manifest provenance can be checked; a shallow boundary must never be interpreted as non-ancestry.
|
||||
|
||||
Provider evidence input is an optional JSON array of normalized pipeline records containing `commit`, unique integer pipeline `number`, pipeline `status`, and a `gate-verify` step status. The highest numbered rerun is authoritative; ambiguous duplicates fail. Its retention window is provider-controlled and is not overstated by this repository.
|
||||
|
||||
PR CI executes current-tree verification only, unprivileged and fail-closed. It does not execute isolated own-tree replay: RM-60 must provide a protected launcher or runner-level rootless sandbox before any PR-controlled executable/configuration is evaluated. Repo-only code cannot safely grant itself the capability intended to contain itself.
|
||||
|
||||
The deferred replay implementation remains hard-fail when its sandbox cannot be established; it is not silently skipped as a successful replay. Tests recognize unavailability only from parent-generated Bubblewrap-launch provenance combined with proof that the sandbox entry command did not run. A denial-looking string from child-controlled output is not evidence. When RM-60 activates replay under protected authority, it uses frozen own-tree dependencies, namespace/environment isolation, and archived-file identity checks. A post-merge failure triggers quarantine and revert. This is detection, not pre-merge prevention.
|
||||
@@ -0,0 +1,3 @@
|
||||
# Developer Guide
|
||||
|
||||
- [Gate registry and negative controls](quality-gate-registry.md)
|
||||
@@ -0,0 +1,45 @@
|
||||
# Gate Registry and Negative Controls
|
||||
|
||||
`gates/gates.manifest.json` is the machine-readable registry for the initial RM-02 gate slice. Run:
|
||||
|
||||
```bash
|
||||
pnpm gate:verify
|
||||
```
|
||||
|
||||
## Registered slice
|
||||
|
||||
The registry covers root typecheck, lint, and format checks; RM-01 checkout preflight; the Mosaic CI queue guard; and root Husky pre-commit/pre-push hooks. It does not imply repository-wide coverage. Framework scripts, package-local build/test scripts, templates, and deployment/release scripts remain assigned to RM-54.
|
||||
|
||||
Every gate declares exact invocations, observed and required outcomes, criterion bindings, and a single exact inerting mutation. Every must-fail case requires a non-empty reason diagnostic. The verifier rejects a stale, ambiguous, crashing, or ineffective mutation. Fixture and mutation writes reject path traversal and final-component symlinks. Independent validation phases collect labeled failures instead of letting one thrown fixture, claim, discovery, deployment, mutation, or compatibility error mask already-known stable-ID diagnostics. This proves detection of the **declared** inerting mutation, not every possible semantic weakening.
|
||||
|
||||
## Required versus actual
|
||||
|
||||
A case may record different `required` and `actual` outcomes only with a tracked owner. The verifier checks current reality against `actual`, prints each difference as `DEFECT (owner: ...)`, and fails when behavior changes without a matching registry update. A defect is never described as passing, green, or OK.
|
||||
|
||||
The queue guard currently has RM-03-owned deltas. In particular, its stdin/heredoc classifier does not consume piped status JSON, so terminal-success, no-status, and terminal-failure payloads become `unknown`; unknown and malformed states exit zero; push purpose defaults to `main`. RM-02 records these observations and does not edit the guard.
|
||||
|
||||
## Criteria, prose, and compatibility
|
||||
|
||||
Each criterion declares exact `caseRefs`; the verifier compares those semantic declarations bidirectionally with case-side `criterionIds` and requires at least one must-fail case. Moving a criterion ID to an unrelated case therefore fails as both a missing declared exercising case and an undeclared binding. Registered meta-negative controls misbind a criterion, remove meaning provenance, and misbind a prose claim, and each must make structure verification red for its stated reason.
|
||||
|
||||
Designated governing prose uses `GATE-CLAIM:<id>` markers. Each claim also names the exact must-fail `caseRef` that exercises its criterion; unknown, positive-only, unrelated, unbound, or registered-but-missing claims fail. Orchestrator-owned claims from `TASKS.md` are bound through `docs/remediation/GATE-CLAIMS.md`, which records source headings and anchored text without changing task tracking. Marker completeness still requires RM-54 review because arbitrary English claims cannot be inferred safely.
|
||||
|
||||
Compatibility checks detect direct contradictions in declared finite constructions. The verifier combines referenced case fixtures and environments in one isolated tree, rejects conflicting fixture/environment values, executes the construction's exact invocation, and checks its exact outcome. They do not prove semantic consistency of arbitrary natural language.
|
||||
|
||||
Restatements preserve original text, current text, reason, finding/task, and date.
|
||||
|
||||
## Source and deployed identity
|
||||
|
||||
A gate with an external installed counterpart declares it explicitly. When the installed queue guard is reachable, its bytes must equal repository source and an internal drift control is observed red. In CI the operator-home installation may be outside the container; the verifier checks the pinned observed source digest, reports `DEPLOYED IDENTITY UNAVAILABLE (owner: RM-04)`, and does not infer live equality.
|
||||
|
||||
## Commit and provider boundary
|
||||
|
||||
**DOES:** Every PR evaluates the current checkout's registered gates and declared inerting mutations directly, unprivileged and fail-closed.
|
||||
|
||||
**DOES NOT:** Repository-controlled PR CI does not execute a commit's own verifier in an isolated replay. Doing so safely would require granting namespace capability before PR-controlled configuration or code runs; that same PR could consume the capability directly. This is an absent trust boundary, not unfinished hardening. RM-60 owns a runner-level rootless sandbox or protected immutable launcher; RM-59 owns the parallel artifact-integrity anchor.
|
||||
|
||||
The replay implementation and abuse-case tests remain fail-closed: when invoked by a future protected authority, inability to establish Bubblewrap is terminal nonzero; controls are never omitted or treated as replay success. On an unprivileged CI runner, sandbox integration tests pass only when the result carries parent-generated Bubblewrap-launch provenance and proves the sandbox entry command never ran. Child-controlled text that merely reproduces a Bubblewrap denial is not accepted. Capable local/protected environments exercise the full abuse cases. Historical installs use frozen lockfiles, isolated network/PID/IPC/UTS and environment/home boundaries, and authoritative-file snapshots that detect lifecycle rewrites.
|
||||
|
||||
Retained provider evidence can assert terminal-success **current-tree** records for prior commits when supplied through `GATE_PROVIDER_EVIDENCE_FILE`. Each normalized record contains `commit`, unique integer pipeline `number`, pipeline `status`, and exactly one `gate-verify` step; the highest-numbered rerun is authoritative. Ambiguous duplicates fail. Absent, expired, or currently-running evidence is reported explicitly and never inferred as success.
|
||||
|
||||
Once RM-60 supplies the external pre-execution anchor, protected post-merge/main replay is detection, not pre-merge prevention. A failed replay requires quarantine of the affected result and revert of the offending merge. It must never be represented as proof that CI blocked that merge. RM-25 tracks provider enforcement.
|
||||
+52
@@ -14,6 +14,58 @@
|
||||
|
||||
---
|
||||
|
||||
## RM-02 Gate Registry and Negative-Control Verifier (#1029)
|
||||
|
||||
### Problem and objective
|
||||
|
||||
Existing deterministic gates can return success without enforcing their stated property. RM-02 introduces a machine-readable registry and an unconditional CI verifier that distinguishes required behavior from observed behavior, proves every registered negative control can detect its own failure reason, and makes source/deployment drift visible.
|
||||
|
||||
### Scope
|
||||
|
||||
**In scope:** root typecheck, lint, and format gates; RM-01 checkout preflight; the Mosaic CI queue guard; root Husky pre-commit and pre-push hooks; criterion bindings; modeled compatibility; meaning-change provenance; security/integrity prose claim markers; source-versus-deployed identity; unprivileged current-tree PR verification; retained provider CI evidence where available; and explicit deferral of isolated per-commit replay to RM-60's protected external authority.
|
||||
|
||||
**Out of scope:** fixing the queue guard (RM-03); exhaustive registration of every repository executable (RM-54); semantic proof that arbitrary English criteria are mutually satisfiable (RM-54/RM-55); a same-authority trust anchor for repository-authored evidence (RM-25/RM-59).
|
||||
|
||||
### Normative requirements
|
||||
|
||||
1. `RM02-REQ-01`: The JSON registry SHALL give every gate and criterion a stable ID and SHALL declare exact invocation, input classes, cases, exact observed and required exit codes, reason diagnostics, and criterion bindings.
|
||||
2. `RM02-REQ-02`: Every gate SHALL have at least one observed-red must-fail case. The verifier SHALL reject missing, stale, ambiguous, or ineffective declared inert mutations and SHALL name an externally inerted gate.
|
||||
3. `RM02-REQ-03`: Every acceptance criterion SHALL declare exact criterion-side `caseRefs` that match case-side `criterionIds` bidirectionally and include a case that can fail for that criterion's stated reason. Moving a binding to an unrelated case SHALL fail verification. Security/integrity prose claims in the designated governing documents SHALL carry bound `GATE-CLAIM:<id>` markers and declare the exact must-fail case exercising the claim criterion.
|
||||
4. `RM02-REQ-04`: Declared finite compatibility scenarios SHALL execute together and direct modeled contradictions SHALL fail. This does not claim semantic consistency of arbitrary English.
|
||||
5. `RM02-REQ-05`: Restated criteria SHALL retain original text, current text, reason, finding/task, and dated meaning-change history.
|
||||
6. `RM02-REQ-06`: An observed behavior differing from required behavior SHALL be reported as `DEFECT` with a tracked owner; an ownerless delta SHALL fail verification. Such a gate SHALL never be described as passing, green, or OK.
|
||||
7. `RM02-REQ-07`: Executables under declared gate roots SHALL fail with `unregistered gate` when absent from the registry. The initial coverage boundary SHALL explicitly list exclusions and bind the broader inventory to RM-54.
|
||||
8. `RM02-REQ-08`: Every gate with a deployed counterpart SHALL register source/deployed byte identity and a must-fail drift control. Gates without a deployed counterpart SHALL say so explicitly.
|
||||
9. `RM02-REQ-09`: CI SHALL run `pnpm gate:verify` on every pull request without path filtering and on protected-main pushes.
|
||||
10. `RM02-REQ-10` (restated): PR CI SHALL perform unprivileged, fail-closed current-tree verification only. Isolated per-commit replay SHALL remain deferred to RM-60's protected post-merge/main authority, cross-referenced with RM-59. That future replay is detection with a quarantine/revert response, not pre-merge prevention; inability to establish its sandbox is terminal nonzero, never skip/pass. Retained provider evidence SHALL remain distinct and SHALL never be inferred when absent.
|
||||
|
||||
#### RM02-REQ-10 meaning-change provenance
|
||||
|
||||
- **Original:** “assert that every merged commit passed every required gate, evaluated AGAINST THAT COMMIT'S OWN TREE — not against current main.”
|
||||
- **Restatement:** PR CI performs unprivileged, fail-closed current-tree verification only. Isolated per-commit replay is deferred to a protected post-merge/main authority, where it is detection with a defined quarantine/revert response — explicitly not a pre-merge gate. Inability to establish the sandbox is hard nonzero, never a skip.
|
||||
- **Reason:** Isolated replay on PR CI would require granting namespace capability to PR-controlled configuration, which the same PR could use directly before containment. The trust boundary is impossible at the repository layer, not merely expensive. RM-60 owns the external pre-execution anchor; RM-59 tracks the corresponding artifact-integrity anchor.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
1. `RM02-AC-01`: A healthy current tree returns zero while prominently reporting the queue guard's required-versus-actual `DEFECT (owner: RM-03)` delta.
|
||||
2. `RM02-AC-02`: Externally mutate any registered gate at its declared inerting point so its failure path succeeds; verification returns nonzero and names that gate. The verifier's internal meta-control is observed red before its healthy result is trusted.
|
||||
3. `RM02-AC-03`: An executable added under a declared gate root without an entry returns nonzero and includes `unregistered gate`.
|
||||
4. `RM02-AC-04`: A gate with zero must-fail cases returns nonzero and includes `no negative control`.
|
||||
5. `RM02-AC-05`: Unbound or semantically misbound criteria, prose claims bound to unrelated cases, unbound governing prose markers, ownerless behavior deltas, stale mutations, source/deployed drift, and modeled compatibility conflicts each return nonzero with the responsible stable ID. A simultaneous stale fixture or independent phase error SHALL NOT mask responsible stable-ID diagnostics. Registered meta-negative controls move a criterion binding, remove meaning provenance, and redirect a prose claim to an unrelated case; each is observed red for its stated reason.
|
||||
6. `RM02-AC-06`: CI configuration invokes the verifier unconditionally on every pull request.
|
||||
7. `RM02-AC-07`: PR output states adjacent `DOES`/`DOES NOT` boundaries: current-tree gates and inerting mutations execute unprivileged and fail-closed; isolated own-tree replay does not execute in repository-controlled PR CI. RM-60/RM-59 are named, retained provider evidence is never inferred, and future protected post-merge detection specifies quarantine/revert rather than claiming pre-merge prevention.
|
||||
|
||||
### Risks, dependencies, and verification boundary
|
||||
|
||||
- The repository verifier proves declared controls, modeled scenarios, bidirectional declared criterion/case relationships, source/deployed equality at execution time, and unprivileged current-tree behavior. It does **not** infer arbitrary-English semantics, execute isolated per-commit replay, or defend against an actor able to rewrite the gate, registry, verifier, and sandbox entry consistently.
|
||||
- Sandbox refusal tests require parent-generated Bubblewrap-launch provenance and proof that the sandbox entry command never ran; child-controlled denial-looking text alone cannot establish unavailability.
|
||||
- Repo-only code cannot both grant namespace capability to PR configuration and prevent that same PR from using the capability directly. RM-60 owns a runner/provider-controlled pre-execution boundary; RM-59 owns the parallel artifact-integrity anchor.
|
||||
- Protected post-merge replay, once RM-60 exists, is detection only. Failure requires immediate quarantine of the affected result and revert of the offending merge; it is not equivalent to a pre-merge gate.
|
||||
- External branch protection and provider CI history supply merge-time current-tree evidence where retained. RM-25 tracks provider-side enforcement.
|
||||
- `ASSUMPTION:` RM-54 is the owner for expanding registration and prose-marker coverage beyond this approved seven-gate slice; rationale: the remediation task graph already assigns the fleet-wide inert-gate audit there.
|
||||
|
||||
---
|
||||
|
||||
## Problem Statement
|
||||
|
||||
Jarvis (v0.2.0) is a self-hosted AI assistant with a Python FastAPI backend and Next.js frontend. It handles chat, projects, tasks, and LLM routing but lacks orchestration depth, agent coordination, shared memory, and remote access. The Mosaic framework (`~/.config/mosaic`) provides agent guides, shell-based orchestration tools, and quality rails — but these are loose scripts, not an integrated platform. The `@mosaicstack/*` packages in mosaic-mono-v0 began consolidating these into TypeScript packages (brain, queue, coord, cli, prdy, quality-rails) but have no UI, no auth, and no agent runtime integration.
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# Documentation Sitemap
|
||||
|
||||
## Gate verification
|
||||
|
||||
- [Developer gate registry guide](DEVELOPER-GUIDE/quality-gate-registry.md) — manifest schema, negative controls, defect deltas, modeled boundaries, and commit/provider evidence.
|
||||
- [Gate registry operations](ADMIN-GUIDE/quality-gate-registry.md) — routine verification, failure interpretation, registry updates, and unconditional CI behavior.
|
||||
- [RM-02 governing claim index](remediation/GATE-CLAIMS.md) — marker bindings for orchestrator-owned remediation claims without modifying task tracking.
|
||||
|
||||
## Compaction refresh lease broker
|
||||
|
||||
- [Internal broker protocol](architecture/lease-broker-protocol.md) — kernel identity, ancestry and generation invariants, framed requests, responses, and persisted cycle bindings.
|
||||
|
||||
@@ -0,0 +1,94 @@
|
||||
# RM-02 Gate Registry Implementation Plan
|
||||
|
||||
> **For Pi:** Use test-driven development and execute each task RED → GREEN → refactor.
|
||||
|
||||
**Goal:** Build a machine-readable seven-gate registry and an unconditional CI verifier that detects inert gates, binds criteria to observed negative controls, records defects honestly, and verifies the current PR tree unprivileged and fail-closed.
|
||||
|
||||
**Architecture:** A dependency-free Node CLI reads `gates/gates.manifest.json`, validates its closed schema and references, then runs typed cases in isolated main-disk fixtures. Gate-specific fixture setup remains declarative; exact invocations and exact observed/required exits stay in JSON. A separate history module checks activation/manifest provenance and retained external current-tree CI evidence without inferring missing evidence. Isolated own-tree execution remains fail-closed code for RM-60's future protected authority; repository-controlled PR CI does not invoke it.
|
||||
|
||||
**Tech Stack:** Node.js ESM, `node:test`, JSON, shell gates, pnpm, Woodpecker CI.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Meta-negative-control kernel
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `scripts/gate-verify.test.mjs`
|
||||
- Create: `scripts/gate-verify.mjs`
|
||||
|
||||
1. Write a black-box fixture whose gate failure branch has already been changed to success.
|
||||
2. Run `node --test scripts/gate-verify.test.mjs`; require failure because the absent verifier does not name the inert gate.
|
||||
3. Implement manifest loading, exact process execution, and named mismatch reporting only.
|
||||
4. Re-run and require the external inert mutation to produce verifier nonzero while the test passes.
|
||||
5. Add an internally applied declared mutation and require a healthy fixture to report its negative control armed.
|
||||
|
||||
### Task 2: Registry structural clauses
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `scripts/gate-verify.test.mjs`
|
||||
- Modify: `scripts/gate-verify.mjs`
|
||||
|
||||
Add failing tests, one behavior at a time, for: `unregistered gate`; `no negative control`; unbound criterion; unbound `GATE-CLAIM`; ownerless required/actual delta; stale/ambiguous/ineffective mutation; direct modeled conflict; missing meaning-change provenance. Implement the minimum validator after each observed RED.
|
||||
|
||||
### Task 3: Gate fixtures and seven-gate manifest
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `gates/gates.manifest.json`
|
||||
- Create: `gates/fixtures/quality-case.mjs`
|
||||
- Create: `gates/fixtures/preflight-case.mjs`
|
||||
- Create: `gates/fixtures/queue-case.sh`
|
||||
- Create: `gates/fixtures/hook-case.sh`
|
||||
- Modify: `scripts/gate-verify.test.mjs`
|
||||
|
||||
Register typecheck, lint, format, RM-01 preflight, queue guard, pre-commit, and pre-push. For each, first run its known-bad case against an intentionally inert fixture and observe verifier RED; then restore the real gate behavior and require its exact observed exit/reason. Record queue defects as required/actual deltas owned by RM-03, never as pass/green/OK.
|
||||
|
||||
### Task 4: Source-versus-deployed identity
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `gates/gates.manifest.json`
|
||||
- Modify: `scripts/gate-verify.test.mjs`
|
||||
- Modify: `scripts/gate-verify.mjs`
|
||||
|
||||
Write and observe a failing test with a byte-mutated deployed counterpart. Implement byte equality and internal drift mutation controls. Declare `none` explicitly for gates without deployed counterparts.
|
||||
|
||||
### Task 5: Prose claims and compatibility constructions
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/remediation/MISSION.md`
|
||||
- Modify: `docs/remediation/TASKS.md` only if coordinator authorization overrides the worker prohibition; otherwise place markers in an RM-02 claim index that references immutable source anchors.
|
||||
- Modify: `gates/gates.manifest.json`
|
||||
- Modify: verifier tests/implementation.
|
||||
|
||||
Enumerate current security/integrity claims, bind each marker/id to a negative case, and reject unbound markers. Execute finite compatibility scenarios and clearly document that arbitrary English consistency is outside the model.
|
||||
|
||||
### Task 6: Current-tree boundary, deferred replay, and provider evidence
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `scripts/gate-history.mjs`
|
||||
- Create: `scripts/gate-history.test.mjs`
|
||||
- Modify: `scripts/gate-verify.mjs`
|
||||
- Modify: `gates/gates.manifest.json`
|
||||
|
||||
Test with a synthetic git repository containing two commits whose manifests differ. PR verification must state adjacent `DOES`/`DOES NOT` boundaries and must not execute the intermediate commit's verifier. Preserve isolated replay as a direct fail-closed primitive for RM-60's future protected pre-execution authority; sandbox failure remains nonzero. Add bounded Gitea/Woodpecker current-tree status lookup for prior commits when credentials/history are available. Missing, expired, and currently-running evidence must be explicit states, never inferred success. Protected post-merge replay is detection with quarantine/revert, never pre-merge prevention.
|
||||
|
||||
### Task 7: CI and documentation
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `package.json`
|
||||
- Modify: `.woodpecker/ci.yml`
|
||||
- Create/update: `docs/DEVELOPER-GUIDE/quality-gate-registry.md`
|
||||
- Create/update: `docs/ADMIN-GUIDE/quality-gate-registry.md`
|
||||
- Modify: `docs/SITEMAP.md`
|
||||
|
||||
Add `gate:verify`; run it in an unconditional PR/main CI step. Document invocation, defect semantics, coverage/exclusions, marker syntax, replay/provider boundaries, and source/deployed identity.
|
||||
|
||||
### Task 8: Verification and delivery
|
||||
|
||||
Run focused tests, `pnpm gate:verify`, checkout tests, typecheck, lint, format, and applicable integration tests. Run Codex code and security review; remediate and re-review. Verify authorship, commit, execute queue guard before push, push, create PR via Mosaic wrapper, and send exact-head review request to rev-974 through the coordinator. Do not merge without coordinator authorization.
|
||||
@@ -13,3 +13,13 @@ Correct for the CI container (runs as root), fatal for EVERY non-root local chec
|
||||
### **D-2 / hygiene — husky `prepare` fails `EPERM` copying into root-owned `.husky/_/`.** Repo working
|
||||
|
||||
tree has root-owned dirs (`.husky/`, repo root) under a non-root agent. Worked around with the intended `HUSKY=0` escape hatch (does NOT disable the existing pre-commit/pre-push hooks).
|
||||
|
||||
<!-- board-roll: 2 entries rolled from BOARD.md -->
|
||||
|
||||
### **D-5 / P-QUEUE-001 + P-CONFORMANCE-001 — KEYSTONE: an inert gate that erased its own evidence.**
|
||||
|
||||
Merged PR #868 (`b79336a8`) shipped a file that FAILS `pnpm format:check` ⇒ the CI format gate did not block. An unrelated later PR (#872) then reformatted that file via its own `lint-staged`, so `main` went green again and nobody learned the gate had failed to fire. Verified blob-level under the repo's own config. **Detection must be per-merge-commit against that commit's own tree** — a "is main green today" check reports all-clear on this exact defect. Binding on RM-02/RM-55. Full chain in `TASKS.md` §1a. NOT quiet-patched, by Mos's ruling: patching the symptom destroys the signal.
|
||||
|
||||
### **D-4 / P-LIFECYCLE + hygiene — a dispatched agent silently IGNORED an in-message context reset.**
|
||||
|
||||
planner-sol was at 64.3%/372k; the brief asked it to reset first; it began work on dirty context anyway. Only an out-of-band `/new` driven by the orchestrator guaranteed clean state. Confirms the postmortem thesis: **instructions are not enforcement.** Reset must be a mechanical pre-dispatch step, not a request.
|
||||
|
||||
+16
-17
@@ -1,6 +1,6 @@
|
||||
# mos-remediation — LIVE BOARD (keep < 8 KB)
|
||||
|
||||
**Phase:** EXECUTING — first PR merged; RM-01 in flight; all 3 decisions ruled.
|
||||
**Phase:** EXECUTING — P0 open. RM-01 MERGED; RM-02 (keystone gate registry) is next.
|
||||
**Updated:** 2026-07-31 (mos-remediation orchestrator; seat active on `mosaic-fleet`).
|
||||
|
||||
## Head
|
||||
@@ -16,14 +16,13 @@
|
||||
|
||||
## In-flight
|
||||
|
||||
| Task | Owner | State |
|
||||
| ------------------------------------------------- | --------------- | ------------------------------------------------------------------ |
|
||||
| PR #1026 docs (mission record + backlog) | mos-remediation | OPEN, retargeted to main, rebased; diff verified docs-only |
|
||||
| PR #1025 hygiene | — | **MERGED** 52414605; rev-974 APPROVE + CI #2158 8/8 terminal-green |
|
||||
| DECISION-1 wire-in point (charter change) | Mos / Jason | ESCALATED — both planners reject the charter's target |
|
||||
| DECISION-2 rollback artifact + availability trade | Jason | ESCALATED |
|
||||
| DECISION-3 RM-03 vs parked PR #1023 ownership | Mos | ESCALATED |
|
||||
| RM-01 reproducible checkout (unblocks everything) | unassigned | READY TO DISPATCH |
|
||||
| Task | Owner | State |
|
||||
| ----------------------------------- | --------------- | ------------------------------------------------------------------------- |
|
||||
| RM-01 reproducible checkout | — | **MERGED** `f58b3699` (PR #1027) — rev-974 APPROVE + CI #2172 8/8 green |
|
||||
| RM-02 gate registry ★keystone | unassigned | **READY** — depends only on RM-01; not held by RM-03 |
|
||||
| RM-03 queue guard (3 defects) | — | HOLD — #1023 SUPERSEDED-PENDING-JASON |
|
||||
| RM-59 close D-19 residual risk | — | BLOCKED by RM-12/RM-21/RM-25 (spine + executor) — tracked edge, not prose |
|
||||
| `remediation/state` snapshot → main | mos-remediation | opening at this mission seam |
|
||||
|
||||
## Fleet seats
|
||||
|
||||
@@ -39,6 +38,14 @@
|
||||
- Freeze: LIFTED for this workstream only.
|
||||
- Git identity: `MOSAIC_GIT_IDENTITY=mos-dt-0` INTERIM. Mos ruled gate-16 HOLDS (author≠reviewer is what
|
||||
gate-16 protects; rev-974 reviews, mos-dt-0 never self-reviews). Dedicated identity TRACKED, Mos provisions.
|
||||
- Capability check (D-11b): before dispatching seat X to provider Y, verify
|
||||
`~/.config/mosaic/secrets/gitea-tokens/gitea-<Y>-<X>.token` exists. Token-file set = authoritative
|
||||
capability registry. Mos owns provisioning; escalate missing pairs to him.
|
||||
- Seat identity (D-11a): token identity AND `git config user.name`/`user.email` must BOTH be set and
|
||||
agree. Exporting `MOSAIC_GIT_IDENTITY` alone does NOT fix commit authorship.
|
||||
- Standing worker-brief doctrine (accreted, mandatory in every brief): don't weaken a RED test to make
|
||||
it pass; if a check is unrunnable as written SAY SO, never silently substitute; `agent-send -f` never
|
||||
`-m`; heavy artifacts off shared `/tmp`.
|
||||
- Remote control: native `/remote-control` NOT wired in this runtime. Path is **Mos-relay**
|
||||
(Jason ↔ mos-claude via Discord ↔ mos-remediation via agent-send). Not a blocker.
|
||||
|
||||
@@ -72,14 +79,6 @@ PR #1025 AC2's fixture `mkdir -p apps/*/venv/lib` creates a literal `apps/*/venv
|
||||
|
||||
Running the required `ci-queue-wait.sh --purpose push` before pushing produced `state=unknown ... exit 0` — the exact defect at `ci-queue-wait.sh:282-288` that PR #1023 is parked on. It also evaluated `branch=main` rather than the branch being pushed. The mission's own required pre-push gate passed me on an indeterminate result. Third independent live instance of the class.
|
||||
|
||||
### **D-5 / P-QUEUE-001 + P-CONFORMANCE-001 — KEYSTONE: an inert gate that erased its own evidence.**
|
||||
|
||||
Merged PR #868 (`b79336a8`) shipped a file that FAILS `pnpm format:check` ⇒ the CI format gate did not block. An unrelated later PR (#872) then reformatted that file via its own `lint-staged`, so `main` went green again and nobody learned the gate had failed to fire. Verified blob-level under the repo's own config. **Detection must be per-merge-commit against that commit's own tree** — a "is main green today" check reports all-clear on this exact defect. Binding on RM-02/RM-55. Full chain in `TASKS.md` §1a. NOT quiet-patched, by Mos's ruling: patching the symptom destroys the signal.
|
||||
|
||||
### **D-4 / P-LIFECYCLE + hygiene — a dispatched agent silently IGNORED an in-message context reset.**
|
||||
|
||||
planner-sol was at 64.3%/372k; the brief asked it to reset first; it began work on dirty context anyway. Only an out-of-band `/new` driven by the orchestrator guaranteed clean state. Confirms the postmortem thesis: **instructions are not enforcement.** Reset must be a mechanical pre-dispatch step, not a request.
|
||||
|
||||
<!-- BOARD-ROLL:END -->
|
||||
|
||||
## Decisions log
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
# RM-02 Governing Claim Index
|
||||
|
||||
This index binds remediation claims that live in orchestrator-owned `TASKS.md` without modifying that file. The source heading and quoted text are the reviewable anchor; `gate:verify` enforces each marker-to-criterion binding. Marker completeness beyond this approved slice remains RM-54.
|
||||
|
||||
## Prose is an enforceable claim
|
||||
|
||||
<!-- GATE-CLAIM:PROSE-IS-A-CLAIM -->
|
||||
|
||||
- Source: `docs/remediation/TASKS.md`, heading `D-20 — the orchestrator's own documentation overclaimed, and a reviewer disproved it empirically`.
|
||||
- Anchored text: “The defect was not in the code — it was in this file.”
|
||||
|
||||
## Generated-state verification scope
|
||||
|
||||
<!-- GATE-CLAIM:GENERATED-STATE-SCOPE -->
|
||||
|
||||
- Source: `docs/remediation/TASKS.md`, heading `D-19 — an integrity property that cannot exist at the layer it was specified`.
|
||||
- Anchored text: “a verifier that cannot detect the attack is not a verifier.”
|
||||
|
||||
## Execution trust boundary
|
||||
|
||||
<!-- GATE-CLAIM:EXECUTION-TRUST-BOUNDARY -->
|
||||
|
||||
- Source: RM-02 ruling recorded under `docs/remediation/TASKS.md`, RM-02 clause 4, D-25.
|
||||
- Anchored text: “SELF-VERIFICATION BY THE AUDITED PARTY IS NOT VERIFICATION.”
|
||||
- Dependency: RM-60/#1031, cross-referenced with RM-59.
|
||||
|
||||
## Criterion restatement provenance
|
||||
|
||||
<!-- GATE-CLAIM:CRITERION-RESTATEMENT -->
|
||||
|
||||
- Source: `docs/remediation/TASKS.md`, heading `D-18 — two pre-registered criteria were mutually unsatisfiable, discoverable only at implementation`.
|
||||
- Anchored text: “Both criteria were pre-registered. They cannot both be satisfied.”
|
||||
@@ -7,10 +7,15 @@ mechanically until Build 3 (rotation) makes it automatic.
|
||||
|
||||
## On resume (do in order, before any orchestration action)
|
||||
|
||||
1. `cd /src/mosaic-stack` and confirm you are on the remediation working branch.
|
||||
1. `cd /src/mosaic-stack`, then **`git fetch origin remediation/state`**.
|
||||
⚠ **The live board is on the rolling branch `remediation/state`, NOT on `main`.** `main` carries only
|
||||
periodic snapshots, so reading the board from `main` will silently give you a STALE tick. Read the
|
||||
live files at `origin/remediation/state` (e.g. `git show origin/remediation/state:docs/remediation/BOARD.md`),
|
||||
or check that branch out. Every tick is pushed there immediately, so its HEAD is always the newest state.
|
||||
2. Read `docs/remediation/MISSION.md` — the charter (goal, 4 builds, 15 decisions, sequencing, directives).
|
||||
3. Read `docs/remediation/BOARD.md` — the LIVE state: current phase, in-flight tasks, fleet seat assignments,
|
||||
gate status. This is your single source of in-flight truth (kept small).
|
||||
3. Read `docs/remediation/BOARD.md` **at `origin/remediation/state`** — the LIVE state: current phase,
|
||||
in-flight tasks, fleet seat assignments, gate status. Single source of in-flight truth (kept < 8 KB;
|
||||
older entries roll to `BOARD-LEDGER.md` via `board-roll.sh`).
|
||||
4. Read the discussion checkpoint for full rationale if needed:
|
||||
`../jarvis-brain/docs/scratchpads/postmortem/REMEDIATION-DISCUSSION-STATE.md` (or the jarvis-brain repo path).
|
||||
5. **Residency attestation (fail-closed):** restate from the reloaded files — (a) the goal in one line, (b) the
|
||||
|
||||
+104
-9
@@ -1,7 +1,7 @@
|
||||
# Mosaic Stack Remediation — Mission Charter
|
||||
|
||||
**Owner:** project orchestrator `mos-remediation` (Claude, launched in `/src/mosaic-stack`).
|
||||
**Origin:** 2026-07-16..31 fleet lifecycle postmortem. **Status:** PLANNING (task decomposition).
|
||||
**Origin:** 2026-07-16..31 fleet lifecycle postmortem. **Status:** EXECUTING (planning complete; RM-01 in flight).
|
||||
**HOLD lifted** for this workstream by Jason, 2026-07-31 — "begin full mosaic fleet operation on this."
|
||||
|
||||
## Goal
|
||||
@@ -10,23 +10,112 @@ Convert the 15 accepted postmortem remediation proposals into a working, **dogfo
|
||||
**North star:** anything with a deterministic right answer moves OUT of the LLM into a deterministic
|
||||
gate/program; the LLM handles only genuine judgment.
|
||||
|
||||
### First-class principle — observe the property, not the exit code
|
||||
|
||||
<!-- GATE-CLAIM:OBSERVE-PROPERTY -->
|
||||
|
||||
> **No write is done until the requested PROPERTY is observed. A success exit code is not evidence.**
|
||||
>
|
||||
> **Success output is designed to be believed.** That is the whole reason the inert-gate class exists
|
||||
> and why P-WRAPPER-001's tri-state (`verified` / `written-unverified` / `failed`) is not optional. The
|
||||
> failure is not carelessness — a green is _engineered_ to be trusted, so trusting it is the default
|
||||
> behaviour of a competent operator, not a lapse.
|
||||
>
|
||||
> Promoted to the charter by Mos (2026-07-31) after the orchestrator committed this exact error: a
|
||||
> `--draft` flag was silently dropped by a wrapper fallback that still exited 0, and the PR was reported
|
||||
> as a draft on the strength of the exit code rather than an observed `draft: true` (D-12). Twelve
|
||||
> failure instances were banked in that session; **three of them were the orchestrator's own.** That
|
||||
> ratio is the point — the mechanism must catch the mechanic too, or it is not a mechanism.
|
||||
>
|
||||
> Operationally: after any write, read back the property you required. Applies to gates, wrappers, PR
|
||||
> flags, commit authorship, file installs, and message delivery alike.
|
||||
|
||||
### First-class principle — pre-registration prevents retrofitting, and nothing else
|
||||
|
||||
<!-- GATE-CLAIM:PREREGISTRATION-BOUNDARY -->
|
||||
|
||||
> **A pre-registered check set can fail in three distinct ways:**
|
||||
>
|
||||
> | mode | the set is… | found as |
|
||||
> | --------------------------- | ------------------------------------------------- | -------- |
|
||||
> | **WRONG** | a check does not test what it claims | D-8 |
|
||||
> | **INCOMPLETE** | green while a criterion's requirement is untested | D-17 |
|
||||
> | **INTERNALLY INCONSISTENT** | two criteria cannot both hold | D-18 |
|
||||
>
|
||||
> **Pre-registration protects against exactly one thing: retrofitting a check to fit the implementation
|
||||
> it is supposed to judge.** It confers neither correctness, nor coverage, nor consistency. "We
|
||||
> pre-registered the checks" has been treated as though it settled the question — it settles one of
|
||||
> three.
|
||||
>
|
||||
> Promoted to the charter by Mos (2026-07-31). All three modes were found on this mission's own **first
|
||||
> delivery**, by the machinery applied to its own work — not by inspection, and not by looking for them.
|
||||
>
|
||||
> **Enforceable form — RM-02's four clauses.** The registry must establish that: (1) each check is
|
||||
> **right** — proven red for its own stated reason before its green counts; (2) the set **covers** —
|
||||
> every criterion bound to a case that actually exercises it; (3) no two criteria **conflict** —
|
||||
> mutual unsatisfiability is a registry defect discoverable by construction; (4) when a criterion's
|
||||
> meaning changes, the registry **retains original text, restatement, and reason**, so evolution stays
|
||||
> auditable. A criterion with no case that can fail for its own reason is unregistered in substance,
|
||||
> however it reads in the manifest.
|
||||
|
||||
### Corollary — never ship an integrity claim dressed as a property
|
||||
|
||||
<!-- GATE-CLAIM:ARTIFACT-INTEGRITY-BOUNDARY -->
|
||||
|
||||
> A verification artifact that can be forged by whoever it is meant to catch verifies nothing. If a
|
||||
> manifest, marker, ledger, or receipt is writable by the same actor whose behaviour it certifies, it
|
||||
> **certifies the attack.** Such an artifact must sit inside the integrity envelope it belongs to,
|
||||
> publish atomically, and carry a **tamper negative-control observed red** — otherwise its integrity is
|
||||
> a _claim_, not a _property_.
|
||||
>
|
||||
> **If it cannot be made tamper-evident, say so and reconsider the approach.** Laundering foreign
|
||||
> content as certified is the only unacceptable outcome; an honest "this cannot be verified" is always
|
||||
> available and always preferable.
|
||||
|
||||
### First-class principle — when a property cannot exist at the layer it was specified
|
||||
|
||||
<!-- GATE-CLAIM:IMPOSSIBLE-LAYER-BOUNDARY -->
|
||||
|
||||
> Some required properties are **impossible at the layer that asked for them** — not hard, impossible.
|
||||
> A local check cannot defend against an actor who can rewrite the check itself. When that happens,
|
||||
> there are exactly three honest moves, and all three are mandatory:
|
||||
>
|
||||
> 1. **Implement what the layer _can_ guarantee.** Partial protection against the class it was actually
|
||||
> born from is worth having.
|
||||
> 2. **State the boundary precisely, in BOTH directions.** What it does _not_ defend, **and** beside it
|
||||
> what it _does_. A reader who sees only the negative dismisses the check as worthless; one who sees
|
||||
> only the positive over-trusts it. **Both together is the honest artifact** — either alone misleads.
|
||||
> 3. **Record where the real guarantee will come from — as a TRACKED DEPENDENCY, not prose.** It must
|
||||
> name a task that someone must close. _A documented gap with no owner becomes a permanent gap that
|
||||
> reads as intentional._
|
||||
>
|
||||
> **A written-down gap is acceptable engineering. An implied-fixed gap is this mission's core failure in
|
||||
> a new costume** — a verification artifact that verifies nothing, with a green to prove it.
|
||||
>
|
||||
> Promoted to the charter by Mos (2026-07-31) from D-19. Origin: the RM-01 symlink manifest could not be
|
||||
> made tamper-evident against a same-UID actor (CWE-345), because the manifest and its marker share one
|
||||
> writable tree. The implementing seat **escalated rather than relabelling self-authentication as
|
||||
> tamper-resistance** — the corollary above firing on its first real adversarial test, on the cheapest
|
||||
> seat in the loop. Residual risk bound to **RM-59** (`depends_on: RM-12, RM-21, RM-25`), where the
|
||||
> choke-point executor and spine verify from _outside_ the worktree's authority.
|
||||
|
||||
## Decision record (authoritative, immutable)
|
||||
|
||||
- **15/15 proposals decided: 13 accept, 2 modify (P-AUTHORITY-001, P-INBOX-001), 0 reject.**
|
||||
- Site + `annotations.json`: `jarvis-brain/docs/postmortem-spec/site/` (committed, origin/main).
|
||||
- Discussion checkpoint (rich rationale per proposal): `jarvis-brain/docs/scratchpads/postmortem/REMEDIATION-DISCUSSION-STATE.md`.
|
||||
- Postmortem report: mosaicstack/stack PR #107 (merged 88f4ee04).
|
||||
- MACP wiring scout (verdict c=STRANDED): `/tmp/macp-wiring-investigation.md` (copy into this dir — see TODO).
|
||||
- MACP wiring scout (verdict c=STRANDED): [`MACP-WIRING-SCOUT.md`](./MACP-WIRING-SCOUT.md) (copied into this dir; TODO discharged). Its findings are sound; its _recommended wire-in point_ is superseded by DECISION-1.
|
||||
|
||||
## The plan — 15 proposals collapse to 4 builds + hygiene
|
||||
|
||||
| Build | Absorbs | What it is |
|
||||
| ------------------------------------------------------------ | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **1. One choke-point service** (mechanical enforcer) | MISSION, STATE, AUDIT, WRAPPER, QUEUE | Deterministic program every task/data mutation flows through. **Wire the stranded `@mosaicstack/macp`** in at `mosaic_orchestrator.py::run_single_task` — typed tasks, gate-runner, event ledger, credential binding, tri-state write outcomes. |
|
||||
| **2. One durable spine + hot path** | (storage under everything) | **PG system-of-record + Redis hot queue** (transactional-outbox). Mission/tasks/state-claims/audit-ledger/comms-inbox all land here. |
|
||||
| **3. Rotation lifecycle** (finish the Mission Control Plane) | LIFECYCLE, CONTRACT, GUIDE, RECOVERY | Coordinator daemon: contract-hash binding, compaction-detected → rotate-not-compact, checkpoint→fresh-session→rehydrate, broker-independent recovery. Deterministic, not an LLM. Reuse `packages/coord`; existing PRD at `docs/mission-control/`. |
|
||||
| **4. Comms service** | AUTHORITY, INBOX (+ versioning roadmap) | Envelope (comms/v1) → sole-path service → PG/Redis → pluggable adapters (tmux→Matrix/Discord/Slack/Telegram). Version the protocol, not participants. |
|
||||
| **+ Hygiene & proof** | FLEET, WORKFLOW, CONFORMANCE | One roster-owned socket/host + stale GC; allowlist auto-sync; the conformance harness that fault-injects the failure classes and proves builds 1–4 hold. |
|
||||
| Build | Absorbs | What it is |
|
||||
| ------------------------------------------------------------ | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| **1. One choke-point service** (mechanical enforcer) | MISSION, STATE, AUDIT, WRAPPER, QUEUE | Deterministic program every task/data mutation flows through. **Wire the stranded `@mosaicstack/macp`** — typed tasks, gate-runner, event ledger, credential binding, tri-state write outcomes. ⚠ **Target CORRECTED 2026-07-31 (DECISION-1, Mos):** a new production Node `TaskExecutor` on the **live** dispatch path (`packages/mosaic` launch + `packages/coord`), which Coord/Forge/live-dispatch submit through. **NOT** `mosaic_orchestrator.py::run_single_task` — that controller is `"enabled": false` and references a dispatcher absent from this checkout; wiring it would strand the executor, reproducing this mission's own disease. The Python rail is **deleted**, not ported. Both planners reached this independently. |
|
||||
| **2. One durable spine + hot path** | (storage under everything) | **PG system-of-record + Redis hot queue** (transactional-outbox). Mission/tasks/state-claims/audit-ledger/comms-inbox all land here. |
|
||||
| **3. Rotation lifecycle** (finish the Mission Control Plane) | LIFECYCLE, CONTRACT, GUIDE, RECOVERY | Coordinator daemon: contract-hash binding, compaction-detected → rotate-not-compact, checkpoint→fresh-session→rehydrate, broker-independent recovery. Deterministic, not an LLM. Reuse `packages/coord`; existing PRD at `docs/mission-control/`. |
|
||||
| **4. Comms service** | AUTHORITY, INBOX (+ versioning roadmap) | Envelope (comms/v1) → sole-path service → PG/Redis → pluggable adapters (tmux→Matrix/Discord/Slack/Telegram). Version the protocol, not participants. |
|
||||
| **+ Hygiene & proof** | FLEET, WORKFLOW, CONFORMANCE | One roster-owned socket/host + stale GC; allowlist auto-sync; the conformance harness that fault-injects the failure classes and proves builds 1–4 hold. |
|
||||
|
||||
## The finding that sets the cost
|
||||
|
||||
@@ -37,6 +126,7 @@ orphaned context loader, a fail-open bypass. **Work = wire + consolidate + retir
|
||||
## Sequencing (skeleton — adversarial decomposition refines this)
|
||||
|
||||
1. **Spine + choke-point service** (builds 1+2) — foundation; unlocks MISSION/STATE/AUDIT/WRAPPER/QUEUE at one integration point.
|
||||
(Per DECISION-1, a P0 phase of provable-gate + activation work precedes this; see `TASKS.md` §3.)
|
||||
2. **Rotation daemon** (build 3) on that spine — the drift fix proper.
|
||||
3. **Comms service** (build 4) — envelope → service → PG/Redis → adapters; retire direct-tmux.
|
||||
4. **Hygiene + conformance** (build 5) — fleet convergence, allowlist sync, dogfood harness.
|
||||
@@ -50,6 +140,11 @@ orphaned context loader, a fail-open bypass. **Work = wire + consolidate + retir
|
||||
(WRAPPER), auto-sync sweep (WORKFLOW), #1018 stale-consumed (INBOX). The fleet is its own test bed.
|
||||
- **Orchestration tracking → DB**, hard cutover ("rip off the bandaid"), NO flat-file interim. jarvis-brain
|
||||
PDA flat-files untouched. Current flat-file tracking runs as-is/unhardened until DB tracking is real, then one clean replace.
|
||||
- ⚠ **QUALIFIED 2026-07-31 (DECISION-2, Mos):** the DB spine **must NOT be a single-point hard-stop.**
|
||||
A broker-independent / degraded mode **and** a rehearsed rollback artifact are **design requirements**
|
||||
(P-RECOVERY-001), binding now on RM-12, RM-13, RM-23, RM-36 and RM-53. This **supersedes** the earlier
|
||||
orchestrator recommendation to pre-commit "no DB ⇒ the fleet stops" — that answer is _not_ on record.
|
||||
Only the specific availability _target_ remains open, queued for Jason; it does **not** block current work.
|
||||
|
||||
## The 15 decisions (one-line; full rationale in the checkpoint)
|
||||
|
||||
|
||||
+526
-11
@@ -93,6 +93,520 @@ and must not be cited as merge evidence. Rely on reviewer clearance + real CI.
|
||||
Three independent live instances in a single session — format gate, agent context reset, queue guard —
|
||||
is the class confirmed, not anecdote.
|
||||
|
||||
### D-20 — the orchestrator's own documentation overclaimed, and a reviewer disproved it empirically
|
||||
|
||||
`rev-974` blocked PR #1027 a second time. **The defect was not in the code — it was in this file**, at
|
||||
D-18's entry, written by the orchestrator.
|
||||
|
||||
Two faults, both mine:
|
||||
|
||||
1. **D-18's AC2 restatement omitted the scope clause** that D-19 later established as mandatory
|
||||
("within an accidental/independent-mutation threat model").
|
||||
2. **D-18 asserted that the tampered-manifest control turns integrity "from a claim into a property."**
|
||||
It does not, and _cannot_. That sentence was written **before** D-19 proved the property impossible
|
||||
at this layer, and was never revised when D-19 landed.
|
||||
|
||||
**The reviewer did not merely read it — it disproved it.** It performed a **same-UID consistent
|
||||
manifest + marker rewrite**, and **preflight passed**. My documented claim was falsified by experiment.
|
||||
Code, README, scratchpad and PR body all stated both threat-model directions correctly; **this file was
|
||||
the only place still overclaiming.**
|
||||
|
||||
**This is two banked findings firing on the orchestrator at once:**
|
||||
|
||||
- **The integrity-claim corollary** — I wrote an integrity _claim_ in the voice of an integrity
|
||||
_property_, in the very document that defines the rule against doing so.
|
||||
- **D-14 (propagation)** — D-19 superseded D-18's assertion. I propagated the consequence into the
|
||||
charter and the delivery conditions, but **not back into D-18 itself.** A ruling that fails to
|
||||
propagate _backwards_ into the finding it supersedes is the same defect as one that fails to
|
||||
propagate forwards, and I did not audit for that direction.
|
||||
|
||||
**Corrected in place**, with the original wording quoted and the empirical disproof recorded, rather
|
||||
than silently rewritten — the same standard demanded of any restated criterion.
|
||||
|
||||
**Requirement on RM-02 (fifth clause).** Documentation asserting a _security or integrity_ property is
|
||||
itself a claim requiring a negative control. Where a document states "X is guaranteed", the registry
|
||||
must hold a case that **fails if X is not guaranteed** — and that case must have been observed red.
|
||||
**Prose is not exempt from the mission's own evidentiary standard**, and prose in the _governing_
|
||||
document least of all: it is the artifact most likely to be quoted as authority long after the code has
|
||||
moved on.
|
||||
|
||||
**Reviewer credit.** rev-974 was briefed that its highest-priority check was "confirm the PR claims no
|
||||
more than it can deliver, and a softened or omitted boundary is a finding even though the code works."
|
||||
It applied that instruction **to the orchestrator's own governing document** and produced an experiment
|
||||
to settle it. That is the review standard this mission is trying to make ordinary.
|
||||
|
||||
### D-19 — an integrity property that cannot exist at the layer it was specified
|
||||
|
||||
Implementing D-18's manifest, the seat + a Codex security review reached **CWE-345**: the symlink
|
||||
manifest and the source-hash marker both live in the **same same-UID writable generated tree**, so an
|
||||
actor with that UID can plant a rogue link, regenerate _both_, retain the fingerprint, and pass. **No
|
||||
local cryptographic construction fixes self-authentication** without a key outside that actor's
|
||||
authority; relocating the marker changes the path, not the authority.
|
||||
|
||||
The seat **escalated rather than describing self-authentication as tamper-resistant** — the explicit
|
||||
failure mode the charter corollary demands. That is the corollary working, on its first real test.
|
||||
|
||||
**Ruling — Option A: scope AC2 to accidental / independent / stale mutation; retain the design.**
|
||||
Rationale, recorded so it can be challenged:
|
||||
|
||||
1. **The undefendable boundary is not the weak link.** An actor with same-UID write can already edit the
|
||||
source, the tests, `scripts/preflight.mjs` itself, and `.husky/*`. If they have that, _nothing_ in the
|
||||
local checkout is trustworthy — hardening the manifest buys no real security while **implying
|
||||
protection that does not exist**, which is worse than the gap.
|
||||
2. **What AC2 is actually for.** These checks exist because a five-month-stale `.next` produced 19
|
||||
phantom `TS2307` errors indistinguishable from real ones (**D-5**). That is staleness, drift and
|
||||
foreign residue — and against that class the design demonstrably works.
|
||||
3. **A real trust anchor arrives later, from this mission's own architecture.** An anchor must live
|
||||
outside the actor's authority; for a fleet running as one user that means a separate service —
|
||||
precisely the **choke-point executor + PG spine** of Builds 1–2, which verify outside the worktree's
|
||||
authority. Hand-rolling key distribution for a local preflight now would duplicate that work badly.
|
||||
4. Option C (structural policy, no manifest) is strictly worse — it cannot detect a **removed** expected link.
|
||||
|
||||
**Option A is acceptable only with honest labelling**, or it becomes the disease it is meant to cure.
|
||||
Conditions (last two added/sharpened by Mos):
|
||||
|
||||
- Threat model stated verbatim in the code **and** the PR; the words _tamper-proof / tamper-evident /
|
||||
secure_ **barred** from that context; the scope carried in AC2's restatement; every control kept
|
||||
RED-first including manifest-only tamper.
|
||||
- **State the boundary in BOTH directions.** Not only what it does _not_ defend (same-UID write; no
|
||||
local construction can) but, beside it, what it **does** defend: accidental / independent / stale /
|
||||
foreign-residue mutation — the **D-5** class it was born from (the five-month `.next` and its 19
|
||||
phantom `TS2307`s). _A reader who sees only the negative dismisses the check as worthless; one who
|
||||
sees only the positive over-trusts it. Both together is the honest artifact._
|
||||
- **The residual risk is a HARD TRACKED DEPENDENCY EDGE, not a comment.** It is **RM-59**, owned by the
|
||||
choke-point executor + spine work (`depends_on: RM-12, RM-21, RM-25`), and the AC2 scope note must
|
||||
cite that id. _"Record where the real guarantee comes from" only holds if the record is a live
|
||||
dependency someone must close._ **A documented gap with no owner becomes a permanent gap that reads
|
||||
as intentional.**
|
||||
|
||||
**The generalizable rule.** When a required property **cannot exist at the layer where it was
|
||||
specified**, the honest moves are: implement what the layer _can_ guarantee, **state the boundary
|
||||
precisely**, and record where the real guarantee will come from. **A known gap that is written down is
|
||||
acceptable; a gap that is implied fixed is not.** Silence here would have shipped a verification
|
||||
artifact that verifies nothing — with a green to prove it.
|
||||
|
||||
### D-18 — two pre-registered criteria were mutually unsatisfiable, discoverable only at implementation
|
||||
|
||||
Implementing D-17's fix surfaced a conflict **between** pre-registered criteria:
|
||||
|
||||
- **AC2** (as written) — reject symlinked generated state.
|
||||
- **AC4** — the canonical `pnpm -w build` succeeds and leaves no residue.
|
||||
|
||||
Verified independently rather than taken on report: `apps/web/next.config.ts:4` sets
|
||||
`output: 'standalone'`, and the built tree contains **42 legitimate pnpm dependency symlinks** under
|
||||
`.next/standalone/node_modules`. A blanket descendant-symlink rejection makes the canonical build fail
|
||||
its own preflight with exit 43. **AC2 read literally is unsatisfiable alongside AC4 under this
|
||||
configuration**, and nothing short of building the tree would have revealed it.
|
||||
|
||||
**Third distinct failure mode of a pre-registered check set**, completing the chain:
|
||||
|
||||
| finding | a pre-registered check set can be… |
|
||||
| ------- | ------------------------------------------------------------------ |
|
||||
| D-8 | **wrong** — a check that does not test what it claims |
|
||||
| D-17 | **incomplete** — green while a criterion's requirement is untested |
|
||||
| D-18 | **internally inconsistent** — two criteria that cannot both hold |
|
||||
|
||||
The implementing seat escalated instead of silently picking a winner. That matters: **quietly resolving
|
||||
a conflict between pre-registered criteria destroys the point of pre-registering them** — the registration
|
||||
exists so that changes of meaning are auditable rather than absorbed.
|
||||
|
||||
**Resolution (orchestrator ruling).** Approved a **build-certified symlink manifest**: `.next` itself is
|
||||
still rejected as a symlink; descendants are rejected unless _exactly_ certified by a manifest the build
|
||||
publishes atomically. Strictly **stronger** than blanket rejection — it also catches a **retargeted**
|
||||
symlink, which blanket rejection cannot distinguish from a legitimate one.
|
||||
|
||||
**AC2 restated (recorded, not absorbed).** _Generated state must reject `.next` itself being a symlink
|
||||
or non-directory, and must reject any descendant symlink not exactly certified by the build manifest —
|
||||
added, removed, retargeted, or manifest-only-tampered all fail with exit 43 — **within an accidental /
|
||||
independent-mutation threat model.**_
|
||||
|
||||
> ⚠ **This entry is superseded in part by [D-19](#d-19). Do not read D-18 standalone.** The scope clause
|
||||
> above is load-bearing: the design **cannot** defend against a same-UID actor, which can rewrite the
|
||||
> manifest and the marker consistently (CWE-345). D-18 was written **before** that impossibility was
|
||||
> established.
|
||||
|
||||
**Hardening required before this counts.** The manifest is itself generated state, so **a manifest
|
||||
writable by whoever plants a rogue symlink certifies the attack** — that is the one way this design
|
||||
fails. It must sit inside the same ownership/fingerprint envelope, published atomically via the existing
|
||||
marker mechanism, with negative controls **observed red first** for: added, removed, retargeted,
|
||||
**manifest-only-tampered**, plus a positive control that the canonical build passes.
|
||||
|
||||
> ⚠ **CORRECTED (D-20).** This paragraph originally ended: _"without it, integrity is a claim rather
|
||||
> than a property."_ **That overclaimed**, by implying the control makes integrity a _property_. It does
|
||||
> not, and cannot. The manifest-only-tamper control detects **independent** mutation of the manifest;
|
||||
> it confers **no authenticity** against an actor who rewrites manifest _and_ marker together.
|
||||
> `rev-974` disproved the original wording empirically — a same-UID consistent manifest+marker rewrite
|
||||
> **passed preflight**. Integrity here remains a scoped **drift-detection** property, never an
|
||||
> authenticity one. See D-19 and the charter principle on properties that cannot exist at their
|
||||
> specified layer.
|
||||
|
||||
**Requirement on RM-02 (fourth clause).** The registry must detect **conflicts between registered
|
||||
criteria**, not only wrongness and coverage. Two criteria that cannot simultaneously hold is a registry
|
||||
defect discoverable by construction — and when a criterion is restated, the registry must retain the
|
||||
original text, the restatement, and the reason, so the evolution stays auditable.
|
||||
|
||||
### D-17 — a pre-registered criterion passed a green suite without being satisfied
|
||||
|
||||
`rev-974` returned **CHANGES REQUESTED** on PR #1027 with one blocking finding, and it is the sharpest
|
||||
instance of the session's theme because it occurred **inside our own verification machinery**.
|
||||
|
||||
**AC2** was pre-registered before any code was written, and explicitly required that **symlinked
|
||||
generated state be rejected**. The implementation does not do it:
|
||||
|
||||
```sh
|
||||
ln -s /etc/hosts apps/web/.next/reviewer-symlink
|
||||
pnpm preflight # → "checkout preflight passed", exit 0
|
||||
# → required: generated-state exit 43
|
||||
```
|
||||
|
||||
The acceptance suite was **21/21 green** throughout. Confirmed independently rather than relayed:
|
||||
`scripts/preflight.mjs:82-92` rejects symlinks on the **source** path; `:28` merely _skips_ symlinked
|
||||
directories rather than rejecting them; and the **generated-state** path at `:141-163` `lstat`s and
|
||||
checks `uid` (ownership) but **never** calls `isSymbolicLink()`. The suite's only symlink cases
|
||||
(`preflight.test.mjs:59`, `:115`) cover the turbo binary and a _source_ file. No generated-state case
|
||||
exists anywhere.
|
||||
|
||||
**So: criterion pre-registered, suite green, requirement unmet.** Nobody was careless — the coverage gap
|
||||
is _invisible from a green_, which is the entire problem.
|
||||
|
||||
**This sharpens D-8 rather than repeating it.** D-8 established that pre-registration does not confer
|
||||
_correctness_ (a check can be wrong when written). D-17 establishes the adjacent failure:
|
||||
**pre-registration does not confer _coverage_** — a suite can be green, and every registered criterion
|
||||
can appear satisfied, while a criterion's actual requirement is untested. The two together mean a
|
||||
registry of checks needs **two** properties, not one: each check must be _right_, and the set must
|
||||
_actually exercise_ what it claims.
|
||||
|
||||
**Requirement on RM-02 (third clause).** The registry must bind each acceptance criterion to the
|
||||
**specific case that exercises it**, and prove that case red before trusting its green. A criterion with
|
||||
no case that can fail for _that criterion's stated reason_ is unregistered in substance however it
|
||||
appears in the manifest. This is mutation testing pointed at the **criterion-to-case mapping**, not
|
||||
merely at the gate.
|
||||
|
||||
**Credit where due:** the reviewer also declined to re-run AC8, stating plainly that the PR carried it
|
||||
forward with no runnable command rather than silently substituting a different boundary test. That is
|
||||
the D-8 clause working a second time, in the same review that produced D-17.
|
||||
|
||||
### D-16 — the local test gate and the CI test gate disagree by environment
|
||||
|
||||
Mos flagged a shape worth chasing: if `pnpm test` exits non-zero on a _pre-existing_ guard, then either
|
||||
`main` is red and merges step around it (the #868 shape again), or CI does not run that path. **Both
|
||||
branches turned out wrong, and the truth is a third thing.** Established by running it, not by asking:
|
||||
|
||||
CI runs **exactly** `pnpm test` (`.woodpecker/ci.yml`, `test` step) — the same command. So the path _is_
|
||||
exercised. Yet:
|
||||
|
||||
| environment | result |
|
||||
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| CI container | `test` step **green** (#2158, #2167) |
|
||||
| this host, clean worktree | **exit 97** — `WAKE-ASSERT INIT ABORT: BASH_LINENO convention violated on bash 5.2.15(1)-release … expected [3 4], probe reported [3 5] (#973)`, after `PASS=18 FAIL=0` |
|
||||
| this host, main checkout | **exit 1** — a _different_, second defect (below) |
|
||||
|
||||
The guard is **environment-dependent**: it aborts on this host's bash and not in CI's container. `git
|
||||
diff origin/main...` confirms PR #1027 touches **zero** files under `packages/mosaic`, so the guard is
|
||||
genuinely pre-existing and unrelated — **f10-coder's report was accurate in every particular**, and
|
||||
`main` is equally affected on this host.
|
||||
|
||||
**So it is not "merges step around a red" — it is worse in one specific way: the local gate and the CI
|
||||
gate do not agree about what passing means.** No agent on this host can obtain a green `pnpm test` at
|
||||
all, on any branch, including `main`. A gate an operator cannot run is a gate that only CI enforces,
|
||||
and a gate only CI enforces cannot be a pre-push gate. This is the hermeticity/portability class
|
||||
already live as #1007 (PR #1024).
|
||||
|
||||
**Second, independent defect found while establishing the above.** In the main checkout the same
|
||||
package fails differently — exit 1 — because a test **scans the working tree** and asserts on files it
|
||||
finds, picking up `apps/coordinator/venv/**` (third-party `site-packages`: `pi = math.pi` in `rich`,
|
||||
`setuptools`, `mypy`). **A test whose result depends on untracked files present in the tree is not
|
||||
hermetic.** This is the _same_ contamination source that made `pnpm format:check` unpassable (D-1/D-7
|
||||
hygiene) — one untracked foreign tree silently breaking two independent gates.
|
||||
|
||||
**Requirements.** RM-01/RM-04: a gate must produce the same verdict on a developer host and in CI, or
|
||||
declare loudly that it cannot run here — never diverge silently. RM-02 registers both as cases: the
|
||||
environment-divergence guard, and a hermeticity control asserting a suite's verdict is unchanged by the
|
||||
presence of untracked directories. Coordinate with #1007/#1024 rather than opening a third lane.
|
||||
|
||||
**Ownership (Mos, 2026-07-31).** The hermeticity fix **is** PR #1024, which sits in **Jason's parked
|
||||
delivery stack** — so, like #1023, its disposition is Jason's. Marked `SUPERSEDED-PENDING-JASON`
|
||||
alongside #1023. D-16 strengthens the urgency but does not transfer ownership: **we do not open a third
|
||||
lane on a parked PR.** The one-line escalation for Jason: _two independent gates
|
||||
(`format:check`, `pnpm test`) were broken by a single untracked directory, and a third
|
||||
(`pnpm test`) disagrees between host and CI — non-hermetic gates make every green host-dependent._
|
||||
|
||||
**Sharpened statement of the class (Mos).** A pre-push gate an operator _cannot run locally_ is a gate
|
||||
only CI enforces — so pointing `.husky/pre-push` at it **misrepresents where the gate lives**. Combined
|
||||
with the shared root cause across two gates, the finding is: **non-hermetic gates make every green
|
||||
host-dependent.** A gate that only appears to pass depending on which host runs it is this mission's
|
||||
exact subject, one meta-level up.
|
||||
|
||||
**#1027 disposition (Mos):** proceeds on **CI-green**. CI is the authoritative gate; the local exit-97
|
||||
is a known host-specific guard abort, irrelevant to the merge decision.
|
||||
|
||||
### D-15 — token scope is not repository permission (a THIRD capability layer)
|
||||
|
||||
`f10-coder` was provisioned with `gitea-mosaicstack-f10-coder.token`, scopes `write:repository` +
|
||||
`write:issue`, and the mint was verified by "repo access returns 200". It then failed to push:
|
||||
|
||||
```
|
||||
remote: error: User permission denied for writing.
|
||||
remote: error: pre-receive hook declined
|
||||
```
|
||||
|
||||
Verified objectively rather than inferred (per the charter principle):
|
||||
|
||||
| probe | result |
|
||||
| ------------------------------------------------------ | ------------------------------------------- |
|
||||
| `GET /repos/mosaicstack/stack/collaborators/f10-coder` | **404** — not a collaborator |
|
||||
| repo permissions as seen by **its own token** | `admin: false`, `push: false`, `pull: true` |
|
||||
|
||||
**Capability has at least three independent layers, and satisfying two proves nothing about the third:**
|
||||
|
||||
1. **Token file exists** → raw-API authentication works (D-11b).
|
||||
2. **`tea` login exists** → tea-dependent wrapper paths work (D-13).
|
||||
3. **Repository permission granted** (collaborator/team membership) → _writes_ are actually authorised.
|
||||
|
||||
A token can carry `write:repository` scope and still be refused, because **scope bounds what a token
|
||||
may attempt; repository permission decides what the user may do.** They are different systems.
|
||||
|
||||
**This is the charter principle failing on the very check meant to confirm capability.** The mint was
|
||||
validated by an HTTP 200 on a _read_. A 200 proves reachability; it does not prove the property that
|
||||
was required, which was **write**. Both the provisioner and I accepted it — the same
|
||||
`written-unverified` treated as `verified` as D-12, one layer up, on a check whose entire purpose was
|
||||
verification.
|
||||
|
||||
**Requirements.** RM-50's pre-dispatch capability check must probe the **effective permission for the
|
||||
operation intended** — for push authority, assert `permissions.push == true` as that seat, not token
|
||||
existence and not a 200 on a read. RM-04's registry reconciliation covers all three layers, with a
|
||||
must-fail control for each. A capability check that cannot fail on a seat lacking write permission is
|
||||
itself an inert gate.
|
||||
|
||||
### D-14 — a ruled decision did not propagate to the authoritative record
|
||||
|
||||
DECISION-1 (the corrected choke-point wire-in target) was ruled by the coordinator and applied to
|
||||
`TASKS.md`. **`MISSION.md` — the charter, the document a cold-starting seat reads first — kept the
|
||||
superseded target for hours.** It was flagged `CONTESTED` in a board note, then the ruling landed and
|
||||
nobody edited the charter. A seat resuming from the charter would have read the _rejected_ target as
|
||||
authoritative and wired the choke point into a disabled rail — the precise failure the ruling existed
|
||||
to prevent.
|
||||
|
||||
Caught by hand, during an unrelated edit. Nothing would have caught it otherwise.
|
||||
|
||||
**This is P-MISSION-001 turned on ourselves.** The mission's own thesis is that convention exists and
|
||||
_enforcement_ is the gap: a decision that lives in a chat ruling and a board note, but not in the
|
||||
source of truth, has not actually been made — it has been _agreed_. The two are different, and the
|
||||
difference is exactly what this mission is about.
|
||||
|
||||
> ⚠ **AMENDED by D-20 — propagation is BIDIRECTIONAL.** As first written, this requirement was read by
|
||||
> both the orchestrator and the coordinator as _forward_ propagation only: a ruling reaches the
|
||||
> documents that state the new rule. **D-20 proved that insufficient.** When D-19 superseded part of
|
||||
> D-18, the consequence propagated forward into the charter and the delivery conditions but **never
|
||||
> backward into D-18 itself**, which went on asserting a withdrawn claim — and a reviewer disproved it
|
||||
> by experiment. **A supersession must update BOTH the documents that render the new rule AND the
|
||||
> finding it retires, with the retired wording quoted rather than deleted.** Backward propagation is
|
||||
> the same defect as forward; neither of us audited that direction until it bit.
|
||||
|
||||
**Requirement (not merely a fix).** A ruled decision must propagate **mechanically** to the
|
||||
authoritative record; it must not depend on someone remembering to edit a second file. Concretely, once
|
||||
mission state is DB-backed (RM-53 / the P-MISSION cutover):
|
||||
|
||||
- a decision is a **record**, not prose duplicated across documents;
|
||||
- documents _render_ decisions rather than restating them, so there is one place to be wrong;
|
||||
- and where duplication is unavoidable, a check asserts the authoritative record and the derived
|
||||
document agree — with a must-fail control proving divergence is detected.
|
||||
|
||||
Until then, the interim rule: **the same commit that records a ruling updates every document that
|
||||
states it — and every finding it supersedes.** Interim rules are exactly what the DB cutover exists to replace.
|
||||
|
||||
**The rule found a second instance within minutes of being written.** Auditing the charter against all
|
||||
rulings to date (rather than waiting to be bitten again) surfaced that **DECISION-2 had also not
|
||||
propagated**: `MISSION.md`'s standing directives still stated the DB hard-cutover with no mention of
|
||||
Mos's binding qualification that _the spine must not be a single-point hard-stop_ (degraded mode +
|
||||
rollback artifact required). A seat reading the charter would have designed toward an availability
|
||||
posture the coordinator had explicitly rejected — and would have found the superseded
|
||||
"no DB ⇒ the fleet stops" recommendation nowhere contradicted. Now corrected in place.
|
||||
|
||||
**Two un-propagated rulings out of two rulings that touched charter text.** The propagation gap is not
|
||||
an oversight that happened once; without a mechanism it is the _default outcome_. That is the argument
|
||||
for making this a requirement rather than a discipline.
|
||||
|
||||
### D-13 — two credential registries that can disagree (why the `--draft` fallback fired at all)
|
||||
|
||||
Diagnosing D-12's root cause surfaced a distinct defect. There are **two parallel credential
|
||||
registries**, and capability in one does not imply capability in the other:
|
||||
|
||||
| registry | contents for identity `mos-dt-0` on `mosaicstack` |
|
||||
| ------------------------------------------------------ | -------------------------------------------------------------------------------------------- |
|
||||
| token files — `~/.config/mosaic/secrets/gitea-tokens/` | `gitea-mosaicstack-mos-dt-0.token` **EXISTS** |
|
||||
| `tea login list` | **NO** `mosaicstack` login for `mos-dt-0` (only `mosaicstack-mos` and `mosaicstack-rev-974`) |
|
||||
|
||||
So `get_gitea_token` succeeds and every raw-API path works, while every **tea-dependent** wrapper path
|
||||
fails its login validation and silently degrades to the API fallback — which is exactly what dropped
|
||||
`--draft`. **tea is not "stale"; the login simply does not exist for that identity.**
|
||||
|
||||
This matters beyond one flag: capability was declared authoritative by the token-file set (D-11b), but
|
||||
that registry does not govern the tea path. A seat can be _fully provisioned_ by the authoritative
|
||||
registry and still lose functionality with no error — only a warning, and only on the degraded path.
|
||||
|
||||
**Requirements.** RM-04 (activation coherence): the two registries must be reconciled — one source of
|
||||
truth, or a startup check asserting they agree, with a must-fail control proving disagreement is
|
||||
detected. RM-50: the pre-dispatch capability check must verify capability for the **path actually
|
||||
used**, not merely token-file presence.
|
||||
|
||||
**Confirmed working despite the gap** (so this is degradation, not outage): pushes, `pr-merge.sh`,
|
||||
PR/issue creation via API fallback, comment posting, and all reads. Impact is confined to
|
||||
tea-only features — `--draft`, `--labels`, `--milestone`.
|
||||
|
||||
**Reconciliation run by Mos (the manual form of RM-04's assert-agreement, done once by hand).** For
|
||||
`git.mosaicstack.dev`, the token-file registry holds **six** seats; `tea` holds logins for **two**:
|
||||
|
||||
| state | seats |
|
||||
| ------------------------------------------------ | -------------------------------------------------------- |
|
||||
| token file present, **no** mosaicstack tea login | `f10-coder`, `jarvis`, `mos-admin`, `mos-dt-0`, `pepper` |
|
||||
| token file present **and** tea login present | `rev-974` (only) |
|
||||
|
||||
**Five of six provisioned seats are silently degraded on tea-only features.** This is _systemic_, not
|
||||
a one-off — which is why the fix is registry reconciliation (RM-04) and not a per-seat mint. Minting
|
||||
one seat would clear a symptom and leave the class live.
|
||||
|
||||
Mos deliberately deferred the mint: it is not on RM-01's critical path, and additively editing shared
|
||||
`tea` config underneath running work is a change he declined to make without cause. Full remediation —
|
||||
mint the five missing logins **and** wire the startup must-fail assertion that _detects_ disagreement —
|
||||
lands as RM-04 at a non-critical seam, or immediately if any seat needs a tea-only feature to progress.
|
||||
|
||||
**Correction of record:** this supersedes D-11(b)'s claim that the token-file set is _the_ authoritative
|
||||
capability registry. It is **necessary but not sufficient**. Capability is **per-path**: the token file
|
||||
governs the raw-API path, the tea login governs the tea path, and the two can disagree silently.
|
||||
|
||||
### D-12 — a requested SAFETY flag was silently degraded, and I did not check
|
||||
|
||||
I created PR #1027 with `pr-create.sh ... -d` (draft) because it carries **partial, unproven work**.
|
||||
`tea` authentication was stale, so the wrapper fell back to its raw-API path — which cannot set draft —
|
||||
and emitted:
|
||||
|
||||
```
|
||||
Warning: API fallback applies title/body/head/base only; labels/milestone/draft require authenticated tea setup.
|
||||
```
|
||||
|
||||
The PR was created **not-draft**. I read the success output, saw the PR number, and moved on. I then
|
||||
reported to the coordinator that the PR was "opened as draft". **It was open, mergeable, and marked
|
||||
ready for ~25 minutes**, protected only by the words "DRAFT" and "do not merge" in its title and body —
|
||||
i.e. by prose a human might read, not by the platform control I asked for. Detected only because a
|
||||
watcher polled `draft:` and the value disagreed with my belief. Corrected by setting the `WIP:` title
|
||||
prefix (Gitea's draft mechanism); `draft: True` verified after.
|
||||
|
||||
**Three distinct failures, and the third is mine:**
|
||||
|
||||
1. **Silent degradation of a safety flag.** The fallback path dropped `--draft` and still exited 0. A
|
||||
fallback that cannot honour a _safety_ argument must fail, not proceed — degrading `--labels` is a
|
||||
nuisance; degrading `--draft` publishes unproven work as ready to merge.
|
||||
2. **The warning went to stderr and nothing consumed it.** It was correct, specific, and ignored — a
|
||||
warning nobody acts on is indistinguishable from no warning.
|
||||
3. **I did not verify the flag took effect.** I checked that the PR existed, not that it had the
|
||||
property I required. This is the mission's own thesis turned on me: **I trusted a success exit code
|
||||
over an observed state**, on exactly the class of tool this mission exists to distrust.
|
||||
|
||||
**Requirements.** RM-02: a wrapper that cannot honour a safety-relevant argument must exit non-zero —
|
||||
registered with a must-fail control asserting `--draft` on a degraded path fails rather than proceeds.
|
||||
RM-24 (tri-state write outcomes): this is precisely `written-unverified` being treated as `verified` —
|
||||
the PR write succeeded, the _requested property_ was never confirmed, and no one looked.
|
||||
|
||||
### D-11 — seat identity did not survive into git, and seat capability is invisible at dispatch
|
||||
|
||||
Two defects, one dispatch (RM-01 → `f10-coder`):
|
||||
|
||||
**(a) Identity drift — P-WRAPPER-001, reproduced on our own delivery.** The seat's commits are
|
||||
authored `mosaic-coder <[email protected]>` — the generic fallback. **You cannot tell from
|
||||
git history which seat did this work.** Recorded, not rewritten: the drift is the evidence.
|
||||
|
||||
> **Mechanism, corrected (Mos).** My original framing here was wrong, and the error was in the brief
|
||||
> before it was in the finding. `MOSAIC_GIT_IDENTITY` resolves the **token** (which per-slot credential
|
||||
> the wrappers act with). The **commit author** comes from `git config user.name` / `user.email`, which
|
||||
> is a **separate setting** — it fell back to the generic value because nothing set it. Exporting the
|
||||
> identity could never have fixed authorship. **My worker brief instructed only the export, so the
|
||||
> seat did exactly what it was told and the commits were still mis-attributed.**
|
||||
>
|
||||
> **The requirement is coherence: token and authorship must agree.** A seat acting with
|
||||
> `gitea-mosaicstack-f10-coder` must also commit as `f10-coder <[email protected]>`.
|
||||
> Either half alone is identity drift — one produces the right credential with the wrong author, the
|
||||
> other the reverse. That coherence _is_ P-WRAPPER-001, and it belongs in seat setup, not in prose
|
||||
> instructions a seat may follow correctly and still end up wrong.
|
||||
|
||||
**(b) Capability opacity.** Nothing at dispatch time revealed that `f10-coder` had no credential for
|
||||
the target provider. Per-slot tokens live at `~/.config/mosaic/secrets/gitea-tokens/`; the seat holds
|
||||
`gitea-usc-f10-coder` but not `gitea-mosaicstack-f10-coder`. This surfaced only when the seat failed
|
||||
**mid-task, after ~$9 and 69% of its context.** The orchestrator (me) selected a seat without any way
|
||||
to check it could act on the target repo — and there was no way to check.
|
||||
|
||||
`get_gitea_token` behaved **correctly**: it refused to fall through and borrow another slot's token,
|
||||
failing loud precisely to protect gate-16 attribution. The tooling was right; the _dispatch-time
|
||||
information_ did not exist.
|
||||
|
||||
**This is P-RECOVERY-001's "honest capability labeling" applied to seats rather than services.** A seat
|
||||
should declare what it can actually do — which providers, which repos, which credentials — and that
|
||||
declaration must be **checkable before dispatch**, not discovered by failure after the budget is spent.
|
||||
|
||||
**Requirements.**
|
||||
|
||||
- **RM-04 (activation coherence)** gains the identity-binding half: seat setup must set **both** the
|
||||
token identity **and** `git config user.name`/`user.email`, coherently. Verified by an
|
||||
exit-asserting test that makes a commit and asserts its author — never assumed from an instruction
|
||||
in a brief.
|
||||
- **RM-50 (roster ownership)** gains per-seat capability declaration plus a **pre-dispatch capability
|
||||
check**. Mos (who owns provisioning) confirms the check is mechanically trivial: **capability is
|
||||
token-file existence.** Before dispatching seat `X` to provider `Y`, test that
|
||||
`~/.config/mosaic/secrets/gitea-tokens/gitea-<Y>-<X>.token` exists; if absent, provision it or pick a
|
||||
provisioned seat. **The token-file set is the authoritative capability registry.** A one-second check
|
||||
would have replaced a mid-task failure that cost ~$9 and 69% of a seat's context.
|
||||
|
||||
### D-10 — the queue guard's failure modes are exactly backwards
|
||||
|
||||
`ci-queue-wait.sh` — a **required** pre-push/pre-merge gate — was observed this session doing both of
|
||||
these:
|
||||
|
||||
- **Fails OPEN on an unknown result.** `state=unknown ⇒ exit 0`, five times, during real pushes and
|
||||
real merges. It also evaluates `branch=main` rather than the branch being acted on.
|
||||
- **Fails CLOSED on credential resolution.** In a worker seat it aborted with
|
||||
`Gitea token not found`, hard-blocking a legitimate push of completed, tested work. The worker
|
||||
correctly stopped (Constitution gate 8). The identical command run from that worker's _own worktree_
|
||||
in another shell succeeded, so the checkout and remote were fine — the difference was the worker's
|
||||
process environment.
|
||||
|
||||
**A gate that waves through work it never checked, and blocks work that is ready, has its failure
|
||||
modes inverted.** Availability failures (cannot reach the provider, cannot resolve a credential)
|
||||
should degrade to a loud, auditable _inability to assert_ — never to a hard stop on delivery, and
|
||||
never to a silent pass. Correctness failures (unknown, malformed, terminal-failure) are what must
|
||||
block.
|
||||
|
||||
This is also the **Pi-brick shape** (P-RECOVERY-001): a gate whose own unavailability prevents the
|
||||
work needed to recover from it.
|
||||
|
||||
**Requirement on RM-03, extending its existing two defects:** the guard must distinguish
|
||||
`CANNOT_ASSERT` (credential/transport/provider unavailable — loud, audited, does not silently pass and
|
||||
does not permanently block) from `ASSERTED_NOT_READY` (a real non-green CI state — blocks). Both are
|
||||
registered R-002 cases with must-fail controls; neither may exit 0 silently.
|
||||
|
||||
### D-9 — the comms path shell-interprets message bodies (injection-shaped, found by accident)
|
||||
|
||||
Sending a status message with `agent-send.sh -m "...`backticks`..."` caused bash to **execute** the
|
||||
backticked text as command substitution. The recipient received a mangled body plus a
|
||||
`No such file or directory` error; the intended sentence never arrived. The message was reported as
|
||||
delivered.
|
||||
|
||||
This is the **same class** as the already-noted `pr-create.sh` backtick-quoting bug (M2 scratchpad):
|
||||
**two tools in the comms path treat a message body as shell input.** A body that can execute on the
|
||||
sender is a _correctness_ bug before it is ever a security one — and note the failure mode: the
|
||||
send reported success while silently transmitting something other than what was written. Silent
|
||||
corruption with a success receipt is precisely the pattern this mission exists to eliminate.
|
||||
|
||||
**Requirement on RM-40 / RM-42 (comms/v1), hardened by Mos.** The envelope must carry its payload
|
||||
**verbatim** and must not be subject to shell interpretation at **any** hop — sender, transport, or
|
||||
adapter. Concretely: **file/stdin transport, never argv interpolation.**
|
||||
|
||||
**Standing interim rule, effective now (Mos).** Until the envelope lands, use `agent-send.sh -f
|
||||
<file>` for any message body containing special characters — **never `-m`**. Passing a file sidesteps
|
||||
argv interpolation entirely. **This rule is mandatory in every worker brief this mission issues**,
|
||||
alongside the D-8 "if a check is unrunnable, say so" clause. Round-trip fidelity (send a body containing backticks, `$(…)`, quotes, and newlines; assert
|
||||
byte-identical receipt) is a required registered test case under RM-02, including a must-fail control
|
||||
proving the assertion can detect corruption.
|
||||
|
||||
### D-8 — a PRE-REGISTERED acceptance check that was not runnable as written
|
||||
|
||||
On PR #1025 the author (me) pre-registered AC2 with the fixture snippet `mkdir -p apps/*/venv/lib`.
|
||||
@@ -297,17 +811,18 @@ spread is itself information, and X1 says we calibrate on real merged PRs.
|
||||
|
||||
### P5 — Retirements, hygiene, conformance
|
||||
|
||||
| id | task | src | depends_on | est (S/O) | tier |
|
||||
| ----- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- | -------------------------- | ---------------- | ------ |
|
||||
| RM-50 | One roster-owned socket/host; quarantine unmanaged; **deterministic reaper for stale sessions AND dead-session disk scratch** (D-7) | O+S+live | RM-04 | 14K / 150K | sonnet |
|
||||
| RM-51 | Auto-sync **allowlist** (never auto-stage unknown paths) + worktree/lease isolation | O+S | RM-02 | 8K / 110K | sonnet |
|
||||
| RM-52 | Retire the Python controller + duplicate MACP islands (3 → 1) | O+S | RM-26, RM-27, RM-25, RM-28 | 14K / 110K | codex |
|
||||
| RM-53 | Flat-file orchestration → DB hard cutover, with rehearsed rollback artifact | O+S | RM-27, RM-30, RM-34, RM-29 | (in S-10) / 200K | opus |
|
||||
| RM-54 | Fleet-wide inert-gate audit against the RM-02 registry | O | RM-02 | — / 120K | sonnet |
|
||||
| RM-55 | **Conformance harness:** fault-inject the live failure classes on real artifacts | O+S | RM-35, RM-41, RM-53 | 18K / 260K | opus |
|
||||
| RM-56 | Retirement proof: CI asserts all three retirements are complete **and stay complete** | O | RM-52, RM-45, RM-53 | — / 90K | codex |
|
||||
| RM-57 | Operator cutover docs + activation proof; map all 15 decisions to evidence | S | RM-04, RM-36, RM-45, RM-55 | 6K / — | codex |
|
||||
| RM-58 | **Mechanical pre-dispatch context reset** — the orchestrator resets a seat out-of-band and verifies it, rather than asking the agent to reset itself | mos-remediation (D-4) | RM-31, RM-50 | 8K | sonnet |
|
||||
| id | task | src | depends_on | est (S/O) | tier |
|
||||
| ----- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------- | -------------------------- | ---------------- | ------ |
|
||||
| RM-50 | One roster-owned socket/host; quarantine unmanaged; **deterministic reaper for stale sessions AND dead-session disk scratch** (D-7) | O+S+live | RM-04 | 14K / 150K | sonnet |
|
||||
| RM-51 | Auto-sync **allowlist** (never auto-stage unknown paths) + worktree/lease isolation | O+S | RM-02 | 8K / 110K | sonnet |
|
||||
| RM-52 | Retire the Python controller + duplicate MACP islands (3 → 1) | O+S | RM-26, RM-27, RM-25, RM-28 | 14K / 110K | codex |
|
||||
| RM-53 | Flat-file orchestration → DB hard cutover, with rehearsed rollback artifact | O+S | RM-27, RM-30, RM-34, RM-29 | (in S-10) / 200K | opus |
|
||||
| RM-54 | Fleet-wide inert-gate audit against the RM-02 registry | O | RM-02 | — / 120K | sonnet |
|
||||
| RM-55 | **Conformance harness:** fault-inject the live failure classes on real artifacts | O+S | RM-35, RM-41, RM-53 | 18K / 260K | opus |
|
||||
| RM-56 | Retirement proof: CI asserts all three retirements are complete **and stay complete** | O | RM-52, RM-45, RM-53 | — / 90K | codex |
|
||||
| RM-57 | Operator cutover docs + activation proof; map all 15 decisions to evidence | S | RM-04, RM-36, RM-45, RM-55 | 6K / — | codex |
|
||||
| RM-59 | **Close the D-19 residual risk** — generated-state verification anchored **outside** the worktree's authority (executor/spine-side attestation), retiring the same-UID self-authentication gap | mos-remediation (D-19) | RM-12, RM-21, RM-25 | 20K | opus |
|
||||
| RM-58 | **Mechanical pre-dispatch context reset** — the orchestrator resets a seat out-of-band and verifies it, rather than asking the agent to reset itself | mos-remediation (D-4) | RM-31, RM-50 | 8K | sonnet |
|
||||
|
||||
**Critical path:** `RM-01 → RM-02 → RM-10 → RM-11 → RM-12 → RM-21 → RM-23 → RM-31 → RM-33 → RM-34 → RM-53 → RM-55`.
|
||||
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
# RM-02 Gate Registry Scratchpad (#1029)
|
||||
|
||||
## Objective
|
||||
|
||||
Deliver the seven-gate registry and RED-first anti-inert verifier on `feat/rm-02-gate-registry`, preserving required-versus-actual defects without laundering them as success.
|
||||
|
||||
## Constraints and boundaries
|
||||
|
||||
- Do not modify `ci-queue-wait.sh`; RM-03 owns that fix.
|
||||
- Do not modify `docs/remediation/TASKS.md`; workers cannot edit orchestrator tracking.
|
||||
- Every declared behavior must be observed, not inferred from an exit code.
|
||||
- Repository-local evidence does not establish same-authority tamper resistance; RM-25/RM-59 own the external trust anchor.
|
||||
- Coverage is seven logical gates; broader inventory is RM-54.
|
||||
- Budget assumption: no explicit token ceiling was supplied. Keep implementation dependency-free (stock Node), avoid repeated full monorepo installs, and keep generated test artifacts under the worktree/main disk.
|
||||
|
||||
## Plan
|
||||
|
||||
1. Update PRD and tracking references.
|
||||
2. Write black-box meta-negative-control first and observe it fail for the missing verifier behavior.
|
||||
3. Implement minimal manifest parser/case runner/mutation detector; observe meta-control succeed.
|
||||
4. Add schema, coverage, provenance, prose marker, compatibility, discovery, deployment identity, and defect-delta tests RED-first.
|
||||
5. Register seven gates with exact cases and run each case.
|
||||
6. Add prospective per-commit replay and bounded provider-evidence reporting.
|
||||
7. Wire unconditional Woodpecker CI and update developer/admin documentation.
|
||||
8. Run situational and baseline verification, independent Codex review, remediate, commit, queue guard, push, PR, coordinator/reviewer handoff.
|
||||
|
||||
## Progress
|
||||
|
||||
- 2026-08-01: Design approved by `mos-remediation`; rulings A-E and source/deployed identity addition incorporated.
|
||||
- 2026-08-01: Isolated worktree created from `origin/main` f65e9ea6; RM-01 f58b3699 verified as ancestor.
|
||||
- 2026-08-01: Git author set to `f10-coder <[email protected]>`; provider issue #1029 created with `MOSAIC_GIT_IDENTITY=f10-coder`.
|
||||
- 2026-08-01: Source/deployed queue-guard SHA-256 observed equal (`19cda2f...`); this observation is not yet an enforced property.
|
||||
|
||||
## Tests and RED-first evidence
|
||||
|
||||
- Meta-negative RED first: `node --test scripts/gate-verify.test.mjs` failed because the absent verifier did not name externally inerted `meta-fixture`.
|
||||
- Structural RED: nine tests failed before implementation for internal mutation, unregistered executable, missing negative control, ownerless delta, unbound criterion/claim, modeled conflict, missing provenance, and deployment drift.
|
||||
- Clause hardening RED: positive-only security criterion and missing registered claim marker both passed incorrectly before validation was added.
|
||||
- CI wiring RED: package script and unconditional Woodpecker step tests both failed before wiring.
|
||||
- History RED: history test failed with missing module before own-tree manifest selection/provider classification was implemented.
|
||||
- `pnpm gate:verify`: exit 0; seven gates each reported `META-NEGATIVE-CONTROL ... observed red`; queue source/deployed drift control observed red; six queue behavior deltas printed as `DEFECT (owner: RM-03)`.
|
||||
- Focused Node tests: 38/38 pass after review hardening (27 verifier/wiring plus 11 history/provider tests).
|
||||
- `pnpm typecheck`: pass (45/45 Turbo tasks).
|
||||
- `pnpm lint`: pass (25/25 Turbo tasks).
|
||||
- `pnpm format:check`: pass.
|
||||
- `pnpm test`: repository suites reached 45/46 Turbo tasks; all application/package tests shown passed, then the known host-specific wake assertion aborted exit 97 (`BASH_LINENO convention violated`, #973/D-16). No test was edited or bypassed. CI remains the authoritative full-suite environment.
|
||||
|
||||
## Self-surfaced defect
|
||||
|
||||
The queue guard's `get_state_from_status_json` runs `python3 - <<'PY'` while provider JSON is piped to the shell function. The heredoc owns stdin, so Python never reads provider JSON. Terminal success, no-status, terminal failure, and malformed payloads all classify as `unknown`; the associated fail-open outcomes are recorded under RM-03. No queue-guard source was modified.
|
||||
|
||||
## Independent review remediation
|
||||
|
||||
- Final pre-commit Codex code review found three blockers: a tautological deployment drift control, independently observed rather than combined compatibility cases, and unauthorized edits to orchestrator-owned `TASKS.md`.
|
||||
- Deployment identity now uses one shared file comparator for both live equality and a temporary drifted deployed copy; a test makes that comparator inert and proves the meta-control fails.
|
||||
- Compatibility scenarios now merge referenced fixtures/environments into one isolated construction and execute an exact scenario invocation; a test proves two independently valid fixtures coexist in the combined run.
|
||||
- `TASKS.md` changes were reverted. `docs/remediation/GATE-CLAIMS.md` binds source headings and anchored text without editing orchestrator tracking.
|
||||
- Codex security review found the Bubblewrap replay shared the runner PID namespace. Replay now unshares PID, IPC, and UTS namespaces, and an abuse-case test proves a sibling runner PID is invisible.
|
||||
- Second review found empty reason diagnostics, final-symlink fixture writes, and lifecycle-script mutation of authoritative history files. Must-fail cases now require a reason pattern; writes use no-follow semantics; and replay snapshots every archived file before install and rejects any changed, deleted, or type/mode-shifted source before executing the verifier. Dedicated negative tests cover all three.
|
||||
- Third code review found ambiguous duplicate provider steps and order-sensitive JSON outcome comparison. Provider evidence now requires exactly one `gate-verify` step in the authoritative rerun, and structural equality normalizes object keys. Both regressions have RED-first tests. Third security review reported no findings.
|
||||
- Initial PR pipeline #2177 exposed Woodpecker's shallow boundary: the activation parent object was present but marked shallow, so `merge-base --is-ancestor` correctly refused to infer ancestry. The unconditional gate step now unshallows before ancestry/provenance checks; its wiring test was observed RED before the CI fix.
|
||||
- Pipeline #2178 then proved the unprivileged Docker runner cannot establish Bubblewrap namespaces. A privileged experiment remained uncommitted and was rejected after Codex correctly rated it CRITICAL: PR-controlled code executes before an in-repository sandbox and could directly use the granted capability.
|
||||
- `mos-remediation` and `rev-974` independently ruled Option C. RM02-REQ-10 now retains its original text, restatement, and reason: PR CI verifies only the current tree, unprivileged and fail-closed; isolated own-tree replay is deferred to RM-60/#1031's external pre-execution authority, cross-referenced with RM-59. Future protected post-merge replay is detection with quarantine/revert, never pre-merge prevention.
|
||||
- RED-first boundary test proved the old path executed an inert intermediate verifier. The revised path states adjacent `DOES`/`DOES NOT` claims, validates historical manifest provenance without executing it, and infers no replay success. Direct sandbox tests remain hard-fail; unprivileged CI asserts terminal refusal instead of treating replay as success. Pipelines #2179/#2180/#2181 exposed two runner refusal forms: namespace denial as `spawnSync bwrap` with `error.code=EPERM`, and a test image without Bubblewrap as `error.code=ENOENT`. The replay diagnostic now preserves spawn errors. Parent-generated launcher/entry metadata distinguishes refusal before sandbox entry from child-controlled output; the detector recognizes exact `spawnSync bwrap` provenance for `EPERM`/`EACCES`/`ENOENT` and known namespace-refusal text only when the entry command provably did not run. Focused negative assertions reject unrelated `spawnSync git EPERM`, verifier output that merely says `bwrap ENOENT`, and exact namespace-denial impersonation without provenance or after sandbox entry.
|
||||
- Exact-head independent review at `38f1b249` found one valid diagnostic-masking blocker: canonical verification knew four stable-ID rebinding failures but a thrown stale fixture replacement reached the outer catch first and emitted only the generic error. RED-first reproduction confirmed the canonical path omitted both responsible criterion IDs. Verification now collects labeled failures independently across claims, discovery, deployment, case execution/outcome checks, mutation, and compatibility, preserving structural stable-ID failures alongside the stale-fixture signal. The canonical regression test requires both missing-binding IDs and the generic fixture error.
|
||||
- Exact-head independent review at `9b4d4beb` found two valid blockers. RED-first controls reproduced both: denial-looking child stderr was accepted as sandbox unavailability, and moving meaning/prose criterion IDs to an unrelated type-error case left `gate:verify` green. Bubblewrap execution now emits a parent-generated random entry marker and returns parent-owned launcher/entry metadata; unavailability requires Bubblewrap launcher provenance plus proof entry never ran, so exact denial impersonation from plain or entered-child results is rejected. Criterion objects now declare exact `caseRefs`, checked bidirectionally against case-side `criterionIds`; prose claims declare an exact must-fail `caseRef`. Registered must-fail cases move a criterion binding, remove meaning provenance, and redirect a prose claim, each producing its stable reason. The review freeze was deliberately lifted before remediation.
|
||||
- Option C security review reported no findings. Code review rejected an initial unrelated typecheck binding for the new security criterion. It was replaced with a dedicated registered `privileged-pr-gate` case: the fixture injects a privilege key into the gate step, the wiring control rejects it for that exact reason, and `gate:verify` observes the boundary negative control. Follow-up hardening uses a closed exact gate-step construction, rejects privilege across the entire pipeline, rejects non-canonical/merged YAML keys, and pins the unrestricted PR/main trigger block; quoted/escaped/alias/merge/duplicate/filter bypass tests pass. Final Codex code review approved with no findings.
|
||||
|
||||
## Documentation checklist
|
||||
|
||||
- PRD, developer guide, admin guide, governing claim index, sitemap, plan, and scratchpad updated.
|
||||
- User/API documentation not applicable: no user workflow or API changed.
|
||||
- Independent review documentation check pending rev-974 at the revised exact head.
|
||||
- Canonical documentation remains in-repository; no external publication requested.
|
||||
|
||||
## Risks/blockers
|
||||
|
||||
- Current queue guard intentionally has required-versus-actual deltas owned by RM-03.
|
||||
- Provider CI cannot report the currently executing pipeline as terminal success; current-commit evidence must be labeled pending and becomes historical current-tree evidence only after provider completion.
|
||||
- Isolated per-commit execution requires RM-60/#1031. Until that external authority exists, no replay success is claimed. A future protected post-merge failure requires quarantine/revert.
|
||||
- CI containers may not expose the operator-home deployed queue guard. In that layer the verifier checks the pinned observed digest and reports live identity unavailable under RM-04; it does not infer live equality.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -6,6 +6,7 @@
|
||||
"build": "turbo run build",
|
||||
"dev": "turbo run dev",
|
||||
"lint": "turbo run lint",
|
||||
"gate:verify": "node scripts/gate-verify.mjs",
|
||||
"preflight": "node scripts/preflight.mjs",
|
||||
"clean:generated": "node scripts/clean-generated.mjs",
|
||||
"typecheck": "pnpm preflight && turbo run typecheck",
|
||||
|
||||
@@ -0,0 +1,359 @@
|
||||
import { existsSync } from 'node:fs';
|
||||
import { createHash, randomUUID } from 'node:crypto';
|
||||
import { access, lstat, mkdir, mkdtemp, readFile, readdir, readlink, rm } from 'node:fs/promises';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import path from 'node:path';
|
||||
|
||||
function git(root, args, { allowFailure = false } = {}) {
|
||||
const result = spawnSync('git', args, { cwd: root, encoding: 'utf8' });
|
||||
if (result.status !== 0 && !allowFailure) {
|
||||
throw new Error(`git ${args.join(' ')} failed: ${(result.stderr || result.stdout).trim()}`);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
export async function listProspectiveCommits(root, activationCommit, head = 'HEAD') {
|
||||
const result = git(root, [
|
||||
'rev-list',
|
||||
'--first-parent',
|
||||
'--reverse',
|
||||
`${activationCommit}..${head}`,
|
||||
]);
|
||||
return result.stdout.trim() ? result.stdout.trim().split('\n') : [];
|
||||
}
|
||||
|
||||
export async function readManifestAtCommit(root, commit) {
|
||||
const result = git(root, ['show', `${commit}:gates/gates.manifest.json`]);
|
||||
return JSON.parse(result.stdout);
|
||||
}
|
||||
|
||||
async function snapshotAuthoritativeTree(root) {
|
||||
const snapshot = new Map();
|
||||
async function walk(current) {
|
||||
for (const child of await readdir(current, { withFileTypes: true })) {
|
||||
if (['.git', '.home', 'node_modules'].includes(child.name)) continue;
|
||||
const absolute = path.join(current, child.name);
|
||||
const relative = path.relative(root, absolute).split(path.sep).join('/');
|
||||
const stats = await lstat(absolute);
|
||||
if (stats.isDirectory()) {
|
||||
await walk(absolute);
|
||||
} else if (stats.isSymbolicLink()) {
|
||||
snapshot.set(relative, `symlink:${stats.mode}:${await readlink(absolute)}`);
|
||||
} else if (stats.isFile()) {
|
||||
const digest = createHash('sha256')
|
||||
.update(await readFile(absolute))
|
||||
.digest('hex');
|
||||
snapshot.set(relative, `file:${stats.mode}:${digest}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
await walk(root);
|
||||
return snapshot;
|
||||
}
|
||||
|
||||
async function authoritativeTreeChanges(root, snapshot) {
|
||||
const changes = [];
|
||||
for (const [relative, expected] of snapshot) {
|
||||
const absolute = path.join(root, relative);
|
||||
let actual;
|
||||
try {
|
||||
const stats = await lstat(absolute);
|
||||
if (stats.isSymbolicLink()) {
|
||||
actual = `symlink:${stats.mode}:${await readlink(absolute)}`;
|
||||
} else if (stats.isFile()) {
|
||||
const digest = createHash('sha256')
|
||||
.update(await readFile(absolute))
|
||||
.digest('hex');
|
||||
actual = `file:${stats.mode}:${digest}`;
|
||||
} else {
|
||||
actual = `other:${stats.mode}`;
|
||||
}
|
||||
} catch (error) {
|
||||
if (error.code !== 'ENOENT') throw error;
|
||||
actual = 'missing';
|
||||
}
|
||||
if (actual !== expected) changes.push(relative);
|
||||
}
|
||||
return changes;
|
||||
}
|
||||
|
||||
function bubblewrap(root, command, args, { storePath, timeout = 300_000 } = {}) {
|
||||
const sandboxArgs = [
|
||||
'--unshare-net',
|
||||
'--unshare-pid',
|
||||
'--unshare-ipc',
|
||||
'--unshare-uts',
|
||||
'--die-with-parent',
|
||||
'--new-session',
|
||||
'--clearenv',
|
||||
];
|
||||
for (const systemPath of ['/usr', '/bin', '/lib', '/lib64', '/etc']) {
|
||||
if (existsSync(systemPath)) sandboxArgs.push('--ro-bind', systemPath, systemPath);
|
||||
}
|
||||
sandboxArgs.push('--dev', '/dev', '--proc', '/proc', '--tmpfs', '/tmp', '--bind', root, '/work');
|
||||
if (storePath) sandboxArgs.push('--ro-bind', storePath, '/pnpm-store');
|
||||
const corepackHome = path.join(process.env.HOME ?? '', '.cache', 'node', 'corepack');
|
||||
if (existsSync(corepackHome)) sandboxArgs.push('--ro-bind', corepackHome, '/corepack');
|
||||
sandboxArgs.push(
|
||||
'--chdir',
|
||||
'/work',
|
||||
'--setenv',
|
||||
'HOME',
|
||||
'/work/.home',
|
||||
'--setenv',
|
||||
'PATH',
|
||||
'/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin',
|
||||
'--setenv',
|
||||
'LANG',
|
||||
'C.UTF-8',
|
||||
'--setenv',
|
||||
'CI',
|
||||
'true',
|
||||
);
|
||||
if (storePath) sandboxArgs.push('--setenv', 'NPM_CONFIG_STORE_DIR', '/pnpm-store');
|
||||
if (existsSync(corepackHome)) sandboxArgs.push('--setenv', 'COREPACK_HOME', '/corepack');
|
||||
const enteredMarker = `__MOSAIC_BWRAP_ENTERED_${randomUUID()}__`;
|
||||
sandboxArgs.push(
|
||||
'/bin/sh',
|
||||
'-c',
|
||||
'printf "%s\\n" "$1"; shift; exec "$@"',
|
||||
'mosaic-bwrap-entry',
|
||||
enteredMarker,
|
||||
command,
|
||||
...args,
|
||||
);
|
||||
const result = spawnSync('bwrap', sandboxArgs, { encoding: 'utf8', timeout });
|
||||
const sandboxEntered = result.stdout?.includes(enteredMarker) === true;
|
||||
return {
|
||||
...result,
|
||||
stdout: (result.stdout ?? '').replace(`${enteredMarker}\n`, ''),
|
||||
sandboxLauncher: 'bwrap',
|
||||
sandboxEntered,
|
||||
};
|
||||
}
|
||||
|
||||
export async function replayCommit(root, commit) {
|
||||
const replayRoot = await mkdtemp(
|
||||
path.join(path.dirname(root), `.gate-history-${commit.slice(0, 12)}-`),
|
||||
);
|
||||
const archive = `${replayRoot}.tar`;
|
||||
try {
|
||||
git(root, ['archive', '--format=tar', `--output=${archive}`, commit]);
|
||||
const extract = spawnSync('tar', ['-xf', archive, '-C', replayRoot], { encoding: 'utf8' });
|
||||
if (extract.status !== 0) {
|
||||
return { status: extract.status, stdout: extract.stdout, stderr: extract.stderr };
|
||||
}
|
||||
const authoritativeSnapshot = await snapshotAuthoritativeTree(replayRoot);
|
||||
await mkdir(path.join(replayRoot, '.home'), { recursive: true });
|
||||
let storePath;
|
||||
try {
|
||||
await access(path.join(replayRoot, 'package.json'));
|
||||
const init = spawnSync('git', ['init', '--quiet', replayRoot], { encoding: 'utf8' });
|
||||
if (init.status !== 0) return init;
|
||||
const store = spawnSync('pnpm', ['store', 'path'], { encoding: 'utf8' });
|
||||
if (store.status !== 0) return store;
|
||||
storePath = store.stdout.trim();
|
||||
const install = bubblewrap(
|
||||
replayRoot,
|
||||
'pnpm',
|
||||
['install', '--frozen-lockfile', '--offline'],
|
||||
{ storePath, timeout: 600_000 },
|
||||
);
|
||||
if (install.status !== 0 || install.error || install.signal) {
|
||||
return {
|
||||
...install,
|
||||
stderr: `historical frozen dependency install failed: ${install.error?.message || install.stderr || install.stdout || ''}`,
|
||||
};
|
||||
}
|
||||
const authoritativeChanges = await authoritativeTreeChanges(
|
||||
replayRoot,
|
||||
authoritativeSnapshot,
|
||||
);
|
||||
if (authoritativeChanges.length > 0) {
|
||||
return {
|
||||
status: 1,
|
||||
stdout: '',
|
||||
stderr: `authoritative archived file changed during historical install: ${authoritativeChanges.join(', ')}`,
|
||||
};
|
||||
}
|
||||
} catch (error) {
|
||||
if (error.code !== 'ENOENT') throw error;
|
||||
}
|
||||
return bubblewrap(
|
||||
replayRoot,
|
||||
process.execPath,
|
||||
[
|
||||
'/work/scripts/gate-verify.mjs',
|
||||
'--root',
|
||||
'/work',
|
||||
'--manifest',
|
||||
'gates/gates.manifest.json',
|
||||
'--skip-history',
|
||||
],
|
||||
{ storePath },
|
||||
);
|
||||
} finally {
|
||||
await rm(archive, { force: true });
|
||||
await rm(replayRoot, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
export function assessProviderEvidence(commit, pipelines) {
|
||||
const matches = pipelines.filter((candidate) => candidate.commit === commit);
|
||||
if (matches.length === 0) {
|
||||
return {
|
||||
state: 'absent',
|
||||
detail:
|
||||
'no retained provider record was supplied; retention expiry and never-ran are not inferred',
|
||||
};
|
||||
}
|
||||
const pipelineStates = new Set(['success', 'failure', 'error', 'pending', 'running', 'queued']);
|
||||
const stepStates = new Set([
|
||||
'success',
|
||||
'failure',
|
||||
'error',
|
||||
'pending',
|
||||
'running',
|
||||
'queued',
|
||||
'skipped',
|
||||
]);
|
||||
const numbers = matches.map((candidate) => candidate.number);
|
||||
const malformed = matches.some(
|
||||
(candidate) =>
|
||||
typeof candidate.commit !== 'string' ||
|
||||
candidate.commit.length === 0 ||
|
||||
!Number.isInteger(candidate.number) ||
|
||||
!pipelineStates.has(candidate.status) ||
|
||||
!Array.isArray(candidate.steps) ||
|
||||
candidate.steps.some(
|
||||
(step) =>
|
||||
typeof step?.name !== 'string' ||
|
||||
typeof step?.status !== 'string' ||
|
||||
!stepStates.has(step.status),
|
||||
),
|
||||
);
|
||||
if (malformed || new Set(numbers).size !== numbers.length) {
|
||||
return {
|
||||
state: 'terminal-failure',
|
||||
detail: 'provider records are malformed or have ambiguous pipeline numbers',
|
||||
};
|
||||
}
|
||||
const pipeline = [...matches].sort((left, right) => right.number - left.number)[0];
|
||||
const gateSteps = (pipeline.steps ?? []).filter((step) => step.name === 'gate-verify');
|
||||
if (gateSteps.length !== 1) {
|
||||
return {
|
||||
state: 'terminal-failure',
|
||||
detail: `provider record has ambiguous gate-verify step count ${gateSteps.length}`,
|
||||
};
|
||||
}
|
||||
const [gateStep] = gateSteps;
|
||||
if (pipeline.status === 'success' && gateStep.status === 'success') {
|
||||
return { state: 'terminal-success', detail: 'pipeline and gate-verify step succeeded' };
|
||||
}
|
||||
if (['pending', 'running', 'queued'].includes(pipeline.status)) {
|
||||
return { state: 'current-running', detail: `pipeline is ${pipeline.status}` };
|
||||
}
|
||||
return {
|
||||
state: 'terminal-failure',
|
||||
detail: `pipeline=${pipeline.status ?? 'unknown'}, gate-verify=${gateStep?.status ?? 'absent'}`,
|
||||
};
|
||||
}
|
||||
|
||||
async function loadProviderEvidence() {
|
||||
const evidenceFile = process.env.GATE_PROVIDER_EVIDENCE_FILE;
|
||||
if (!evidenceFile) return [];
|
||||
const parsed = JSON.parse(await readFile(evidenceFile, 'utf8'));
|
||||
if (!Array.isArray(parsed)) throw new Error('provider evidence file must contain a JSON array');
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function isMainCommit(root, head) {
|
||||
if (process.env.CI_COMMIT_BRANCH === 'main') return true;
|
||||
const result = git(root, ['merge-base', '--is-ancestor', head, 'refs/remotes/origin/main'], {
|
||||
allowFailure: true,
|
||||
});
|
||||
return result.status === 0;
|
||||
}
|
||||
|
||||
export async function verifyHistory({ root, manifest }) {
|
||||
const failures = [];
|
||||
const observations = [];
|
||||
const head = git(root, ['rev-parse', 'HEAD']).stdout.trim();
|
||||
if (!manifest.activationCommit) {
|
||||
failures.push('history activationCommit is missing');
|
||||
return { failures, observations };
|
||||
}
|
||||
const activationCheck = git(
|
||||
root,
|
||||
['merge-base', '--is-ancestor', manifest.activationCommit, head],
|
||||
{ allowFailure: true },
|
||||
);
|
||||
if (activationCheck.status !== 0) {
|
||||
failures.push(
|
||||
`history activation commit ${manifest.activationCommit} is not an ancestor of ${head}`,
|
||||
);
|
||||
return { failures, observations };
|
||||
}
|
||||
const onMain = isMainCommit(root, head);
|
||||
// RM-02 execution boundary (RM-60, cross-reference RM-59), kept adjacent in both directions:
|
||||
// DOES: run every registered current-tree gate and declared inerting mutation on PR CI,
|
||||
// unprivileged and fail-closed.
|
||||
// DOES NOT: execute a commit's own verifier in an isolated PR replay. PR-controlled code would
|
||||
// otherwise need the namespace capability intended to contain that same code. That external
|
||||
// trust boundary must be runner/provider-owned before any PR executable or config is evaluated.
|
||||
observations.push(
|
||||
`RM-02 EXECUTION BOUNDARY ${head}: DOES: verify the current tree and declared inerting mutations on every PR, unprivileged and fail-closed; DOES NOT: execute isolated per-commit verifier replay in repository-controlled CI; owner RM-60, cross-reference RM-59`,
|
||||
);
|
||||
if (!onMain) {
|
||||
observations.push(
|
||||
`PROVIDER ASSERTION DEFERRED ${head}: commit is not yet on main; retained provider evidence starts after merge and no replay success is inferred`,
|
||||
);
|
||||
}
|
||||
|
||||
const pipelines = onMain ? await loadProviderEvidence() : [];
|
||||
const commits = await listProspectiveCommits(root, manifest.activationCommit, head);
|
||||
for (const commit of commits) {
|
||||
let commitManifest;
|
||||
try {
|
||||
commitManifest = await readManifestAtCommit(root, commit);
|
||||
} catch (error) {
|
||||
failures.push(`${commit}: own-tree registry cannot be read: ${error.message}`);
|
||||
continue;
|
||||
}
|
||||
if (commitManifest.schemaVersion !== manifest.schemaVersion) {
|
||||
failures.push(`${commit}: own-tree registry schema is not supported`);
|
||||
continue;
|
||||
}
|
||||
const evidence = assessProviderEvidence(commit, pipelines);
|
||||
if (commit === head) {
|
||||
observations.push(
|
||||
`CURRENT TREE EVALUATED ${commit}: all registered cases ran from this checkout; provider evidence=${evidence.state} (${evidence.detail})`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
observations.push(
|
||||
`INTERMEDIATE REPLAY DEFERRED ${commit}: isolated own-tree execution is not performed by repository-controlled CI; owner RM-60, cross-reference RM-59; no success is inferred`,
|
||||
);
|
||||
if (!onMain) {
|
||||
observations.push(
|
||||
`PROVIDER EVIDENCE ${commit}: DEFERRED until the commit is on main; no success is inferred`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
observations.push(
|
||||
`POST-MERGE DETECTION BOUNDARY ${commit}: protected isolated replay awaits RM-60; when available, a failure requires quarantine/revert and is detection, not pre-merge prevention`,
|
||||
);
|
||||
if (evidence.state === 'terminal-failure') {
|
||||
failures.push(
|
||||
`${commit}: retained provider evidence is not terminal-success (${evidence.detail})`,
|
||||
);
|
||||
} else if (evidence.state === 'terminal-success') {
|
||||
observations.push(`PROVIDER EVIDENCE ${commit}: terminal-success (${evidence.detail})`);
|
||||
} else {
|
||||
observations.push(
|
||||
`PROVIDER EVIDENCE ${commit}: ${evidence.state.toUpperCase()} (${evidence.detail}); no merge-time success is inferred`,
|
||||
);
|
||||
}
|
||||
}
|
||||
return { failures, observations };
|
||||
}
|
||||
@@ -0,0 +1,405 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { mkdir, rm, writeFile } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import { spawn, spawnSync } from 'node:child_process';
|
||||
import test from 'node:test';
|
||||
|
||||
import {
|
||||
assessProviderEvidence,
|
||||
listProspectiveCommits,
|
||||
readManifestAtCommit,
|
||||
replayCommit,
|
||||
verifyHistory,
|
||||
} from './gate-history.mjs';
|
||||
|
||||
const fixtureRoot = path.join(process.cwd(), '.mosaic-test-work', `gate-history-${process.pid}`);
|
||||
|
||||
function sandboxUnavailable(result) {
|
||||
if (result.sandboxLauncher !== 'bwrap' || result.sandboxEntered === true) return false;
|
||||
const detail = `${result.stdout ?? ''}${result.stderr ?? ''}${result.error?.message ?? ''}`;
|
||||
const bubblewrapSpawnDenied =
|
||||
['EPERM', 'EACCES', 'ENOENT'].includes(result.error?.code) &&
|
||||
/spawnSync bwrap/i.test(result.error?.message ?? '');
|
||||
if (
|
||||
!bubblewrapSpawnDenied &&
|
||||
!/bwrap:.*(?:Operation not permitted|Creating new namespace failed)/i.test(detail)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
assert.notEqual(result.status, 0, 'sandbox unavailability must remain terminal nonzero');
|
||||
return true;
|
||||
}
|
||||
|
||||
function git(root, ...args) {
|
||||
const result = spawnSync('git', args, { cwd: root, encoding: 'utf8' });
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
return result.stdout.trim();
|
||||
}
|
||||
|
||||
async function commitManifest(root, marker) {
|
||||
await mkdir(path.join(root, 'gates'), { recursive: true });
|
||||
await writeFile(
|
||||
path.join(root, 'gates', 'gates.manifest.json'),
|
||||
`${JSON.stringify({ schemaVersion: 1, marker })}\n`,
|
||||
);
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', marker);
|
||||
return git(root, 'rev-parse', 'HEAD');
|
||||
}
|
||||
|
||||
test.after(async () => {
|
||||
await rm(fixtureRoot, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('sandbox refusal classification requires Bubblewrap provenance', () => {
|
||||
for (const code of ['EPERM', 'EACCES', 'ENOENT']) {
|
||||
assert.equal(
|
||||
sandboxUnavailable({
|
||||
status: null,
|
||||
error: { code, message: `spawnSync bwrap ${code}` },
|
||||
sandboxLauncher: 'bwrap',
|
||||
sandboxEntered: false,
|
||||
}),
|
||||
true,
|
||||
);
|
||||
}
|
||||
assert.equal(
|
||||
sandboxUnavailable({
|
||||
status: null,
|
||||
error: { code: 'EPERM', message: 'spawnSync git EPERM' },
|
||||
}),
|
||||
false,
|
||||
);
|
||||
assert.equal(
|
||||
sandboxUnavailable({ status: 1, stderr: 'historical verifier said bwrap ENOENT' }),
|
||||
false,
|
||||
);
|
||||
assert.equal(
|
||||
sandboxUnavailable({
|
||||
status: 1,
|
||||
stderr: 'bwrap: Creating new namespace failed: Operation not permitted',
|
||||
sandboxLauncher: 'bwrap',
|
||||
sandboxEntered: false,
|
||||
}),
|
||||
true,
|
||||
);
|
||||
assert.equal(
|
||||
sandboxUnavailable({
|
||||
status: 1,
|
||||
stderr: 'bwrap: Creating new namespace failed: Operation not permitted',
|
||||
}),
|
||||
false,
|
||||
);
|
||||
assert.equal(
|
||||
sandboxUnavailable({
|
||||
status: 1,
|
||||
stderr: 'bwrap: Creating new namespace failed: Operation not permitted',
|
||||
sandboxLauncher: 'bwrap',
|
||||
sandboxEntered: true,
|
||||
}),
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
test('prospective history reads each commit own manifest rather than the current tree', async () => {
|
||||
await rm(fixtureRoot, { recursive: true, force: true });
|
||||
await mkdir(fixtureRoot, { recursive: true });
|
||||
git(fixtureRoot, 'init', '-q');
|
||||
git(fixtureRoot, 'config', 'user.name', 'gate-test');
|
||||
git(fixtureRoot, 'config', 'user.email', '[email protected]');
|
||||
await writeFile(path.join(fixtureRoot, 'activation.txt'), 'activation\n');
|
||||
git(fixtureRoot, 'add', '.');
|
||||
git(fixtureRoot, 'commit', '-m', 'activation');
|
||||
const activation = git(fixtureRoot, 'rev-parse', 'HEAD');
|
||||
const first = await commitManifest(fixtureRoot, 'FIRST');
|
||||
const second = await commitManifest(fixtureRoot, 'SECOND');
|
||||
|
||||
assert.deepEqual(await listProspectiveCommits(fixtureRoot, activation, second), [first, second]);
|
||||
assert.equal((await readManifestAtCommit(fixtureRoot, first)).marker, 'FIRST');
|
||||
assert.equal((await readManifestAtCommit(fixtureRoot, second)).marker, 'SECOND');
|
||||
});
|
||||
|
||||
test('historical replay executes each selected commit verifier from that commit tree', async () => {
|
||||
const root = `${fixtureRoot}-replay`;
|
||||
await rm(root, { recursive: true, force: true });
|
||||
await mkdir(path.join(root, 'scripts'), { recursive: true });
|
||||
await mkdir(path.join(root, 'gates'), { recursive: true });
|
||||
git(root, 'init', '-q');
|
||||
git(root, 'config', 'user.name', 'gate-test');
|
||||
git(root, 'config', 'user.email', '[email protected]');
|
||||
await writeFile(path.join(root, 'gates', 'gates.manifest.json'), '{"schemaVersion":1}\n');
|
||||
await writeFile(
|
||||
path.join(root, 'scripts', 'gate-verify.mjs'),
|
||||
"process.stderr.write('OLD TREE INERT\\n'); process.exitCode = 1;\n",
|
||||
);
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', 'inert historical verifier');
|
||||
const inert = git(root, 'rev-parse', 'HEAD');
|
||||
await writeFile(
|
||||
path.join(root, 'scripts', 'gate-verify.mjs'),
|
||||
"process.stdout.write('NEW TREE VERIFIED\\n');\n",
|
||||
);
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', 'fixed historical verifier');
|
||||
const fixed = git(root, 'rev-parse', 'HEAD');
|
||||
|
||||
const inertResult = await replayCommit(root, inert);
|
||||
const fixedResult = await replayCommit(root, fixed);
|
||||
if (sandboxUnavailable(inertResult) || sandboxUnavailable(fixedResult)) return;
|
||||
assert.notEqual(inertResult.status, 0);
|
||||
assert.match(inertResult.stderr, /OLD TREE INERT/);
|
||||
assert.equal(fixedResult.status, 0);
|
||||
assert.match(fixedResult.stdout, /NEW TREE VERIFIED/);
|
||||
});
|
||||
|
||||
test('historical install lifecycle cannot replace an authoritative verifier', async () => {
|
||||
const root = `${fixtureRoot}-install-tamper`;
|
||||
await rm(root, { recursive: true, force: true });
|
||||
await mkdir(path.join(root, 'scripts'), { recursive: true });
|
||||
await mkdir(path.join(root, 'gates'), { recursive: true });
|
||||
git(root, 'init', '-q');
|
||||
git(root, 'config', 'user.name', 'gate-test');
|
||||
git(root, 'config', 'user.email', '[email protected]');
|
||||
await writeFile(path.join(root, 'gates', 'gates.manifest.json'), '{"schemaVersion":1}\n');
|
||||
await writeFile(
|
||||
path.join(root, 'scripts', 'gate-verify.mjs'),
|
||||
"process.stderr.write('ORIGINAL VERIFIER RAN\\n'); process.exitCode = 7;\n",
|
||||
);
|
||||
await writeFile(path.join(root, 'forged.mjs'), "process.stdout.write('FORGED SUCCESS\\n');\n");
|
||||
await writeFile(
|
||||
path.join(root, 'package.json'),
|
||||
`${JSON.stringify({
|
||||
name: 'historical-install-tamper',
|
||||
version: '1.0.0',
|
||||
scripts: { postinstall: 'cp forged.mjs scripts/gate-verify.mjs' },
|
||||
})}\n`,
|
||||
);
|
||||
await writeFile(
|
||||
path.join(root, 'pnpm-lock.yaml'),
|
||||
"lockfileVersion: '9.0'\nsettings:\n autoInstallPeers: true\n excludeLinksFromLockfile: false\nimporters:\n .: {}\n",
|
||||
);
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', 'tampering lifecycle fixture');
|
||||
const commit = git(root, 'rev-parse', 'HEAD');
|
||||
|
||||
const result = await replayCommit(root, commit);
|
||||
assert.notEqual(result.status, 0);
|
||||
if (sandboxUnavailable(result)) return;
|
||||
assert.match(result.stderr, /authoritative archived file changed.*scripts\/gate-verify\.mjs/i);
|
||||
assert.doesNotMatch(result.stdout, /FORGED SUCCESS/);
|
||||
});
|
||||
|
||||
test('historical verifier receives no current-process secret environment', async () => {
|
||||
const root = `${fixtureRoot}-secretless`;
|
||||
await rm(root, { recursive: true, force: true });
|
||||
await mkdir(path.join(root, 'scripts'), { recursive: true });
|
||||
await mkdir(path.join(root, 'gates'), { recursive: true });
|
||||
git(root, 'init', '-q');
|
||||
git(root, 'config', 'user.name', 'gate-test');
|
||||
git(root, 'config', 'user.email', '[email protected]');
|
||||
await writeFile(path.join(root, 'gates', 'gates.manifest.json'), '{"schemaVersion":1}\n');
|
||||
await writeFile(
|
||||
path.join(root, 'scripts', 'gate-verify.mjs'),
|
||||
"if (process.env.REPLAY_SENTINEL) { process.stderr.write('SECRET LEAKED\\n'); process.exitCode = 9; } else { process.stdout.write('SECRETLESS\\n'); }\n",
|
||||
);
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', 'secretless replay fixture');
|
||||
const commit = git(root, 'rev-parse', 'HEAD');
|
||||
|
||||
process.env.REPLAY_SENTINEL = 'must-not-cross-boundary';
|
||||
try {
|
||||
const result = await replayCommit(root, commit);
|
||||
if (sandboxUnavailable(result)) return;
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
assert.match(result.stdout, /SECRETLESS/);
|
||||
assert.doesNotMatch(
|
||||
`${result.stdout}${result.stderr}`,
|
||||
/SECRET LEAKED|must-not-cross-boundary/,
|
||||
);
|
||||
} finally {
|
||||
delete process.env.REPLAY_SENTINEL;
|
||||
}
|
||||
});
|
||||
|
||||
test('historical replay cannot observe a sibling process in the runner PID namespace', async () => {
|
||||
const root = `${fixtureRoot}-pidless`;
|
||||
await rm(root, { recursive: true, force: true });
|
||||
await mkdir(path.join(root, 'scripts'), { recursive: true });
|
||||
await mkdir(path.join(root, 'gates'), { recursive: true });
|
||||
git(root, 'init', '-q');
|
||||
git(root, 'config', 'user.name', 'gate-test');
|
||||
git(root, 'config', 'user.email', '[email protected]');
|
||||
await writeFile(path.join(root, 'gates', 'gates.manifest.json'), '{"schemaVersion":1}\n');
|
||||
|
||||
const sleeper = spawn('sleep', ['30'], {
|
||||
env: { ...process.env, REPLAY_PID_SENTINEL: 'must-not-be-visible' },
|
||||
});
|
||||
try {
|
||||
await writeFile(
|
||||
path.join(root, 'scripts', 'gate-verify.mjs'),
|
||||
`import { existsSync } from 'node:fs';\nif (existsSync('/proc/${sleeper.pid}/environ')) { process.stderr.write('HOST PID VISIBLE\\n'); process.exitCode = 9; } else { process.stdout.write('PIDLESS\\n'); }\n`,
|
||||
);
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', 'pid-isolated replay fixture');
|
||||
const commit = git(root, 'rev-parse', 'HEAD');
|
||||
const result = await replayCommit(root, commit);
|
||||
if (sandboxUnavailable(result)) return;
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
assert.match(result.stdout, /PIDLESS/);
|
||||
assert.doesNotMatch(`${result.stdout}${result.stderr}`, /HOST PID VISIBLE/);
|
||||
} finally {
|
||||
sleeper.kill('SIGTERM');
|
||||
}
|
||||
});
|
||||
|
||||
test('PR verification states the RM-60 boundary without executing an intermediate verifier', async () => {
|
||||
const root = `${fixtureRoot}-feature`;
|
||||
await rm(root, { recursive: true, force: true });
|
||||
await mkdir(root, { recursive: true });
|
||||
git(root, 'init', '-q');
|
||||
git(root, 'config', 'user.name', 'gate-test');
|
||||
git(root, 'config', 'user.email', '[email protected]');
|
||||
await writeFile(path.join(root, 'activation.txt'), 'activation\n');
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', 'activation');
|
||||
const activation = git(root, 'rev-parse', 'HEAD');
|
||||
git(root, 'update-ref', 'refs/remotes/origin/main', activation);
|
||||
await mkdir(path.join(root, 'scripts'), { recursive: true });
|
||||
await mkdir(path.join(root, 'gates'), { recursive: true });
|
||||
await writeFile(path.join(root, 'gates', 'gates.manifest.json'), '{"schemaVersion":1}\n');
|
||||
await writeFile(
|
||||
path.join(root, 'scripts', 'gate-verify.mjs'),
|
||||
"process.stderr.write('INTERMEDIATE INERT\\n'); process.exitCode = 1;\n",
|
||||
);
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', 'inert intermediate');
|
||||
await writeFile(
|
||||
path.join(root, 'scripts', 'gate-verify.mjs'),
|
||||
"process.stdout.write('HEAD HEALTHY\\n');\n",
|
||||
);
|
||||
git(root, 'add', '.');
|
||||
git(root, 'commit', '-m', 'healthy head');
|
||||
|
||||
const previousBranch = process.env.CI_COMMIT_BRANCH;
|
||||
process.env.CI_COMMIT_BRANCH = 'feature/rm-02';
|
||||
try {
|
||||
const result = await verifyHistory({
|
||||
root,
|
||||
manifest: { schemaVersion: 1, activationCommit: activation },
|
||||
});
|
||||
assert.deepEqual(result.failures, []);
|
||||
assert.ok(
|
||||
result.observations.some((observation) =>
|
||||
/DOES:.*current tree.*DOES NOT:.*isolated.*RM-60.*RM-59/i.test(observation),
|
||||
),
|
||||
);
|
||||
assert.ok(
|
||||
result.observations.some((observation) =>
|
||||
/INTERMEDIATE REPLAY DEFERRED.*RM-60.*no success is inferred/i.test(observation),
|
||||
),
|
||||
);
|
||||
assert.ok(result.observations.every((observation) => !/INTERMEDIATE INERT/.test(observation)));
|
||||
} finally {
|
||||
if (previousBranch === undefined) delete process.env.CI_COMMIT_BRANCH;
|
||||
else process.env.CI_COMMIT_BRANCH = previousBranch;
|
||||
}
|
||||
});
|
||||
|
||||
test('provider evidence distinguishes retained success, failure, and absent history', () => {
|
||||
const pipelines = [
|
||||
{
|
||||
commit: 'aaa',
|
||||
number: 1,
|
||||
status: 'success',
|
||||
steps: [{ name: 'gate-verify', status: 'success' }],
|
||||
},
|
||||
{
|
||||
commit: 'bbb',
|
||||
number: 2,
|
||||
status: 'failure',
|
||||
steps: [{ name: 'gate-verify', status: 'failure' }],
|
||||
},
|
||||
];
|
||||
assert.deepEqual(assessProviderEvidence('aaa', pipelines), {
|
||||
state: 'terminal-success',
|
||||
detail: 'pipeline and gate-verify step succeeded',
|
||||
});
|
||||
assert.equal(assessProviderEvidence('bbb', pipelines).state, 'terminal-failure');
|
||||
assert.equal(assessProviderEvidence('ccc', pipelines).state, 'absent');
|
||||
});
|
||||
|
||||
test('duplicate gate-verify steps cannot establish provider success', () => {
|
||||
const result = assessProviderEvidence('aaa', [
|
||||
{
|
||||
commit: 'aaa',
|
||||
number: 7,
|
||||
status: 'success',
|
||||
steps: [
|
||||
{ name: 'gate-verify', status: 'success' },
|
||||
{ name: 'gate-verify', status: 'failure' },
|
||||
],
|
||||
},
|
||||
]);
|
||||
assert.equal(result.state, 'terminal-failure');
|
||||
assert.match(result.detail, /ambiguous.*gate-verify/i);
|
||||
});
|
||||
|
||||
test('a malformed single provider record cannot establish success', () => {
|
||||
assert.equal(
|
||||
assessProviderEvidence('aaa', [
|
||||
{ commit: 'aaa', status: 'success', steps: [{ name: 'gate-verify', status: 'success' }] },
|
||||
]).state,
|
||||
'terminal-failure',
|
||||
);
|
||||
assert.equal(
|
||||
assessProviderEvidence('bbb', [
|
||||
{
|
||||
commit: 'bbb',
|
||||
number: 1,
|
||||
status: 'surprising',
|
||||
steps: [{ name: 'gate-verify', status: 'success' }],
|
||||
},
|
||||
]).state,
|
||||
'terminal-failure',
|
||||
);
|
||||
});
|
||||
|
||||
test('provider evidence selects the highest numbered rerun deterministically', () => {
|
||||
const failedThenSucceeded = [
|
||||
{
|
||||
commit: 'aaa',
|
||||
number: 10,
|
||||
status: 'failure',
|
||||
steps: [{ name: 'gate-verify', status: 'failure' }],
|
||||
},
|
||||
{
|
||||
commit: 'aaa',
|
||||
number: 11,
|
||||
status: 'success',
|
||||
steps: [{ name: 'gate-verify', status: 'success' }],
|
||||
},
|
||||
];
|
||||
const succeededThenFailed = [
|
||||
{
|
||||
commit: 'bbb',
|
||||
number: 21,
|
||||
status: 'success',
|
||||
steps: [{ name: 'gate-verify', status: 'success' }],
|
||||
},
|
||||
{
|
||||
commit: 'bbb',
|
||||
number: 22,
|
||||
status: 'failure',
|
||||
steps: [{ name: 'gate-verify', status: 'failure' }],
|
||||
},
|
||||
];
|
||||
assert.equal(assessProviderEvidence('aaa', failedThenSucceeded).state, 'terminal-success');
|
||||
assert.equal(assessProviderEvidence('bbb', succeededThenFailed).state, 'terminal-failure');
|
||||
assert.equal(
|
||||
assessProviderEvidence('ccc', [
|
||||
{ commit: 'ccc', status: 'success', steps: [{ name: 'gate-verify', status: 'success' }] },
|
||||
{ commit: 'ccc', status: 'failure', steps: [{ name: 'gate-verify', status: 'failure' }] },
|
||||
]).state,
|
||||
'terminal-failure',
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,866 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { constants } from 'node:fs';
|
||||
import { createHash } from 'node:crypto';
|
||||
import {
|
||||
access,
|
||||
chmod,
|
||||
copyFile,
|
||||
lstat,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
open,
|
||||
readFile,
|
||||
readdir,
|
||||
readlink,
|
||||
rm,
|
||||
symlink,
|
||||
writeFile,
|
||||
} from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
|
||||
import { verifyHistory } from './gate-history.mjs';
|
||||
|
||||
const COPY_SKIP = new Set(['.git', '.mosaic-test-work', '.next', '.turbo', 'coverage', 'dist']);
|
||||
|
||||
function parseArgs(argv) {
|
||||
const options = {
|
||||
root: process.cwd(),
|
||||
manifest: 'gates/gates.manifest.json',
|
||||
skipHistory: false,
|
||||
structureOnly: false,
|
||||
};
|
||||
for (let index = 0; index < argv.length; index += 1) {
|
||||
const value = argv[index];
|
||||
if (value === '--root') options.root = path.resolve(argv[++index]);
|
||||
else if (value === '--manifest') options.manifest = argv[++index];
|
||||
else if (value === '--skip-history') options.skipHistory = true;
|
||||
else if (value === '--structure-only') options.structureOnly = true;
|
||||
else throw new Error(`unknown option: ${value}`);
|
||||
}
|
||||
return options;
|
||||
}
|
||||
|
||||
function runInvocation(root, invocation, extraEnvironment = {}) {
|
||||
if (!Array.isArray(invocation) || invocation.length === 0) {
|
||||
return { status: null, stdout: '', stderr: 'missing invocation' };
|
||||
}
|
||||
return spawnSync(invocation[0], invocation.slice(1), {
|
||||
cwd: root,
|
||||
encoding: 'utf8',
|
||||
env: { ...process.env, GATE_VERIFY: '1', ...extraEnvironment },
|
||||
timeout: 300_000,
|
||||
});
|
||||
}
|
||||
|
||||
async function sandboxPath(root, relativePath, label) {
|
||||
if (typeof relativePath !== 'string' || path.isAbsolute(relativePath)) {
|
||||
throw new Error(`${label}: path escapes sandbox (${String(relativePath)})`);
|
||||
}
|
||||
const resolvedRoot = path.resolve(root);
|
||||
const target = path.resolve(resolvedRoot, relativePath);
|
||||
if (target !== resolvedRoot && !target.startsWith(`${resolvedRoot}${path.sep}`)) {
|
||||
throw new Error(`${label}: path escapes sandbox (${relativePath})`);
|
||||
}
|
||||
const parts = path.relative(resolvedRoot, path.dirname(target)).split(path.sep).filter(Boolean);
|
||||
let current = resolvedRoot;
|
||||
for (const part of parts) {
|
||||
current = path.join(current, part);
|
||||
try {
|
||||
if ((await lstat(current)).isSymbolicLink()) {
|
||||
throw new Error(`${label}: path crosses symbolic link (${relativePath})`);
|
||||
}
|
||||
} catch (error) {
|
||||
if (error.code === 'ENOENT') break;
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
return target;
|
||||
}
|
||||
|
||||
function expand(value, root) {
|
||||
return String(value)
|
||||
.replaceAll('${ROOT}', root)
|
||||
.replaceAll('${HOME}', process.env.HOME ?? '')
|
||||
.replaceAll('${PATH}', process.env.PATH ?? '');
|
||||
}
|
||||
|
||||
function normalizedJson(value) {
|
||||
if (Array.isArray(value)) return value.map(normalizedJson);
|
||||
if (value && typeof value === 'object') {
|
||||
return Object.fromEntries(
|
||||
Object.keys(value)
|
||||
.sort()
|
||||
.map((key) => [key, normalizedJson(value[key])]),
|
||||
);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function structuredValuesEqual(left, right) {
|
||||
return JSON.stringify(normalizedJson(left)) === JSON.stringify(normalizedJson(right));
|
||||
}
|
||||
|
||||
function outcomeMatches(outcome, result) {
|
||||
const combined = `${result.stdout ?? ''}\n${result.stderr ?? ''}`;
|
||||
return (
|
||||
result.status === outcome?.exitCode &&
|
||||
(!outcome?.outputPattern || new RegExp(outcome.outputPattern, 'm').test(combined)) &&
|
||||
(!outcome?.notOutputPattern || !new RegExp(outcome.notOutputPattern, 'm').test(combined))
|
||||
);
|
||||
}
|
||||
|
||||
function resultMatches(gateCase, result) {
|
||||
const combined = `${result.stdout ?? ''}\n${result.stderr ?? ''}`;
|
||||
return (
|
||||
outcomeMatches(gateCase.actual, result) &&
|
||||
(!gateCase.reasonPattern || new RegExp(gateCase.reasonPattern, 'm').test(combined))
|
||||
);
|
||||
}
|
||||
|
||||
async function safeSandboxWrite(root, relativePath, contents, label) {
|
||||
const target = await sandboxPath(root, relativePath, label);
|
||||
await mkdir(path.dirname(target), { recursive: true });
|
||||
try {
|
||||
if ((await lstat(target)).isSymbolicLink()) {
|
||||
throw new Error(`${label}: target is a symbolic link (${relativePath})`);
|
||||
}
|
||||
} catch (error) {
|
||||
if (error.code !== 'ENOENT') throw error;
|
||||
}
|
||||
let handle;
|
||||
try {
|
||||
handle = await open(
|
||||
target,
|
||||
constants.O_WRONLY | constants.O_CREAT | constants.O_TRUNC | constants.O_NOFOLLOW,
|
||||
0o666,
|
||||
);
|
||||
await handle.writeFile(contents);
|
||||
} catch (error) {
|
||||
if (error.code === 'ELOOP') {
|
||||
throw new Error(`${label}: target is a symbolic link (${relativePath})`);
|
||||
}
|
||||
throw error;
|
||||
} finally {
|
||||
await handle?.close();
|
||||
}
|
||||
return target;
|
||||
}
|
||||
|
||||
async function applyFixture(root, fixture = {}) {
|
||||
for (const entry of fixture.writeFiles ?? []) {
|
||||
const target = await safeSandboxWrite(root, entry.path, entry.content, 'fixture write');
|
||||
if (entry.mode !== undefined) await chmod(target, entry.mode);
|
||||
}
|
||||
for (const entry of fixture.replaceFiles ?? []) {
|
||||
const target = await sandboxPath(root, entry.path, 'fixture replace');
|
||||
if ((await lstat(target)).isSymbolicLink()) {
|
||||
throw new Error(`fixture replace: target is a symbolic link (${entry.path})`);
|
||||
}
|
||||
const source = await readFile(target, 'utf8');
|
||||
const occurrences = source.split(entry.find).length - 1;
|
||||
if (occurrences !== 1) {
|
||||
throw new Error(
|
||||
`fixture replace: stale or ambiguous match for ${entry.path} (${occurrences} matches)`,
|
||||
);
|
||||
}
|
||||
await safeSandboxWrite(
|
||||
root,
|
||||
entry.path,
|
||||
source.replace(entry.find, entry.replace),
|
||||
'fixture replace',
|
||||
);
|
||||
}
|
||||
for (const relativePath of fixture.removePaths ?? []) {
|
||||
await rm(await sandboxPath(root, relativePath, 'fixture remove'), {
|
||||
recursive: true,
|
||||
force: true,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
async function copySandbox(root, destination, selectedPaths) {
|
||||
if (!selectedPaths?.length) {
|
||||
await copyTree(root, destination);
|
||||
return;
|
||||
}
|
||||
await mkdir(destination, { recursive: true });
|
||||
for (const relativePath of selectedPaths) {
|
||||
await copyTree(
|
||||
await sandboxPath(root, relativePath, 'sandbox copy source'),
|
||||
await sandboxPath(destination, relativePath, 'sandbox copy destination'),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async function runCase(root, gate, gateCase) {
|
||||
let caseRoot = root;
|
||||
if (gateCase.fixture) {
|
||||
caseRoot = await mkdtemp(path.join(path.dirname(root), `.gate-case-${gate.id}-`));
|
||||
await copySandbox(root, caseRoot, gateCase.fixture.copyPaths);
|
||||
await applyFixture(caseRoot, gateCase.fixture);
|
||||
}
|
||||
try {
|
||||
const environment = Object.fromEntries(
|
||||
Object.entries(gateCase.environment ?? {}).map(([key, value]) => [
|
||||
key,
|
||||
expand(value, caseRoot),
|
||||
]),
|
||||
);
|
||||
return runInvocation(caseRoot, gateCase.invocation ?? gate.invocation, environment);
|
||||
} finally {
|
||||
if (caseRoot !== root) await rm(caseRoot, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
async function copyTree(source, destination) {
|
||||
const stats = await lstat(source);
|
||||
if (stats.isSymbolicLink()) {
|
||||
await symlink(await readlink(source), destination);
|
||||
return;
|
||||
}
|
||||
if (stats.isFile()) {
|
||||
await mkdir(path.dirname(destination), { recursive: true });
|
||||
await copyFile(source, destination);
|
||||
await chmod(destination, stats.mode);
|
||||
return;
|
||||
}
|
||||
if (!stats.isDirectory()) return;
|
||||
await mkdir(destination, { recursive: true });
|
||||
for (const child of await readdir(source, { withFileTypes: true })) {
|
||||
if (COPY_SKIP.has(child.name)) continue;
|
||||
const childSource = path.join(source, child.name);
|
||||
const childDestination = path.join(destination, child.name);
|
||||
if (child.name === 'node_modules') {
|
||||
await symlink(childSource, childDestination, 'dir');
|
||||
continue;
|
||||
}
|
||||
await copyTree(childSource, childDestination);
|
||||
}
|
||||
}
|
||||
|
||||
async function executableFiles(root, relativeRoot) {
|
||||
const base = await sandboxPath(root, relativeRoot, 'gate root');
|
||||
const found = [];
|
||||
async function walk(current) {
|
||||
for (const child of await readdir(current, { withFileTypes: true })) {
|
||||
const target = path.join(current, child.name);
|
||||
if (child.isDirectory()) await walk(target);
|
||||
else if (child.isFile()) {
|
||||
try {
|
||||
await access(target, constants.X_OK);
|
||||
found.push(path.relative(root, target).split(path.sep).join('/'));
|
||||
} catch {
|
||||
// Non-executable files are not gates for discovery purposes.
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
try {
|
||||
await walk(base);
|
||||
} catch (error) {
|
||||
if (error.code !== 'ENOENT') throw error;
|
||||
}
|
||||
return found;
|
||||
}
|
||||
|
||||
function rejectUnknownKeys(value, allowed, label, failures) {
|
||||
if (!value || typeof value !== 'object' || Array.isArray(value)) {
|
||||
failures.push(`${label}: expected an object`);
|
||||
return;
|
||||
}
|
||||
for (const key of Object.keys(value)) {
|
||||
if (!allowed.has(key)) failures.push(`${label}: unknown field ${key}`);
|
||||
}
|
||||
}
|
||||
|
||||
function rejectDuplicateIds(values, label, failures) {
|
||||
const seen = new Set();
|
||||
for (const value of values ?? []) {
|
||||
if (typeof value?.id !== 'string' || value.id.length === 0) {
|
||||
failures.push(`${label}: missing stable id`);
|
||||
} else if (seen.has(value.id)) {
|
||||
failures.push(`duplicate ${label} id ${value.id}`);
|
||||
} else {
|
||||
seen.add(value.id);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function validateClosedSchema(manifest, failures) {
|
||||
if (manifest.schemaVersion !== 1)
|
||||
failures.push(`unsupported schemaVersion ${String(manifest.schemaVersion)}`);
|
||||
rejectUnknownKeys(
|
||||
manifest,
|
||||
new Set([
|
||||
'schemaVersion',
|
||||
'activationCommit',
|
||||
'gateRoots',
|
||||
'governingClaimFiles',
|
||||
'coverageBoundary',
|
||||
'criteria',
|
||||
'proseClaims',
|
||||
'compatibilityScenarios',
|
||||
'mergeAssertions',
|
||||
'gates',
|
||||
]),
|
||||
'manifest',
|
||||
failures,
|
||||
);
|
||||
rejectDuplicateIds(manifest.criteria, 'criterion', failures);
|
||||
for (const criterion of manifest.criteria ?? []) {
|
||||
rejectUnknownKeys(
|
||||
criterion,
|
||||
new Set([
|
||||
'id',
|
||||
'originalText',
|
||||
'currentText',
|
||||
'claimType',
|
||||
'source',
|
||||
'meaningChanges',
|
||||
'caseRefs',
|
||||
]),
|
||||
`criterion ${criterion.id}`,
|
||||
failures,
|
||||
);
|
||||
if (!Array.isArray(criterion.caseRefs) || criterion.caseRefs.length === 0) {
|
||||
failures.push(`${criterion.id}: no declared exercising cases`);
|
||||
} else {
|
||||
if (criterion.caseRefs.some((caseRef) => typeof caseRef !== 'string' || !caseRef)) {
|
||||
failures.push(`${criterion.id}: declared exercising cases must be non-empty strings`);
|
||||
}
|
||||
if (new Set(criterion.caseRefs).size !== criterion.caseRefs.length) {
|
||||
failures.push(`${criterion.id}: duplicate declared exercising case`);
|
||||
}
|
||||
}
|
||||
}
|
||||
rejectDuplicateIds(manifest.proseClaims, 'prose claim', failures);
|
||||
for (const claim of manifest.proseClaims ?? []) {
|
||||
rejectUnknownKeys(
|
||||
claim,
|
||||
new Set(['id', 'criterionId', 'caseRef']),
|
||||
`prose claim ${claim.id}`,
|
||||
failures,
|
||||
);
|
||||
if (typeof claim.caseRef !== 'string' || claim.caseRef.length === 0) {
|
||||
failures.push(`GATE-CLAIM:${claim.id} has no declared exercising case`);
|
||||
}
|
||||
}
|
||||
rejectDuplicateIds(manifest.compatibilityScenarios, 'compatibility scenario', failures);
|
||||
for (const scenario of manifest.compatibilityScenarios ?? []) {
|
||||
rejectUnknownKeys(
|
||||
scenario,
|
||||
new Set([
|
||||
'id',
|
||||
'construction',
|
||||
'caseRefs',
|
||||
'invocation',
|
||||
'expected',
|
||||
'environment',
|
||||
'fixture',
|
||||
]),
|
||||
`compatibility scenario ${scenario.id}`,
|
||||
failures,
|
||||
);
|
||||
if (!Array.isArray(scenario.caseRefs) || scenario.caseRefs.length === 0) {
|
||||
failures.push(`${scenario.id}: compatibility construction has no referenced conditions`);
|
||||
}
|
||||
if (!Array.isArray(scenario.invocation) || scenario.invocation.length === 0) {
|
||||
failures.push(`${scenario.id}: compatibility construction invocation is missing`);
|
||||
}
|
||||
if (typeof scenario.expected?.exitCode !== 'number') {
|
||||
failures.push(`${scenario.id}: compatibility construction exact expected exit is missing`);
|
||||
}
|
||||
}
|
||||
rejectDuplicateIds(manifest.gates, 'gate', failures);
|
||||
for (const gate of manifest.gates ?? []) {
|
||||
rejectUnknownKeys(
|
||||
gate,
|
||||
new Set([
|
||||
'id',
|
||||
'source',
|
||||
'invocation',
|
||||
'deployment',
|
||||
'inertMutation',
|
||||
'cases',
|
||||
'discoveryAliases',
|
||||
]),
|
||||
`gate ${gate.id}`,
|
||||
failures,
|
||||
);
|
||||
rejectDuplicateIds(gate.cases, `case in gate ${gate.id}`, failures);
|
||||
if (!Array.isArray(gate.invocation) || gate.invocation.length === 0) {
|
||||
failures.push(`${gate.id}: exact invocation is missing`);
|
||||
}
|
||||
if (!['none', 'file'].includes(gate.deployment?.kind)) {
|
||||
failures.push(`${gate.id}: unsupported deployment kind ${String(gate.deployment?.kind)}`);
|
||||
}
|
||||
for (const gateCase of gate.cases ?? []) {
|
||||
rejectUnknownKeys(
|
||||
gateCase,
|
||||
new Set([
|
||||
'id',
|
||||
'criterionIds',
|
||||
'mustFail',
|
||||
'invocation',
|
||||
'required',
|
||||
'actual',
|
||||
'reasonPattern',
|
||||
'environment',
|
||||
'fixture',
|
||||
'defect',
|
||||
]),
|
||||
`${gate.id}/${gateCase.id}`,
|
||||
failures,
|
||||
);
|
||||
if (
|
||||
typeof gateCase.required?.exitCode !== 'number' ||
|
||||
typeof gateCase.actual?.exitCode !== 'number'
|
||||
) {
|
||||
failures.push(
|
||||
`${gate.id}/${gateCase.id}: required and actual exact exit codes are mandatory`,
|
||||
);
|
||||
}
|
||||
if (
|
||||
gateCase.invocation &&
|
||||
(!Array.isArray(gateCase.invocation) || gateCase.invocation.length === 0)
|
||||
) {
|
||||
failures.push(`${gate.id}/${gateCase.id}: case invocation must be non-empty`);
|
||||
}
|
||||
if (gateCase.mustFail === true && !gateCase.reasonPattern?.trim()) {
|
||||
failures.push(
|
||||
`${gate.id}/${gateCase.id}: must-fail case requires a non-empty reasonPattern`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function validateStructure(manifest, failures) {
|
||||
validateClosedSchema(manifest, failures);
|
||||
const criteria = new Map((manifest.criteria ?? []).map((criterion) => [criterion.id, criterion]));
|
||||
const boundCriteria = new Set();
|
||||
const negativeBoundCriteria = new Set();
|
||||
const observedCaseRefs = new Map();
|
||||
|
||||
for (const gate of manifest.gates ?? []) {
|
||||
const negativeCases = (gate.cases ?? []).filter((gateCase) => gateCase.mustFail === true);
|
||||
if (negativeCases.length === 0) failures.push(`${gate.id}: no negative control`);
|
||||
if (!gate.deployment?.kind) failures.push(`${gate.id}: deployment identity is not declared`);
|
||||
|
||||
for (const gateCase of gate.cases ?? []) {
|
||||
for (const criterionId of gateCase.criterionIds ?? []) {
|
||||
if (!criteria.has(criterionId)) {
|
||||
failures.push(`${gate.id}/${gateCase.id}: unknown criterion ${criterionId}`);
|
||||
}
|
||||
boundCriteria.add(criterionId);
|
||||
const caseRef = `${gate.id}/${gateCase.id}`;
|
||||
if (!observedCaseRefs.has(criterionId)) observedCaseRefs.set(criterionId, new Set());
|
||||
observedCaseRefs.get(criterionId).add(caseRef);
|
||||
if (gateCase.mustFail === true) negativeBoundCriteria.add(criterionId);
|
||||
}
|
||||
if (!structuredValuesEqual(gateCase.required, gateCase.actual) && !gateCase.defect?.owner) {
|
||||
failures.push(`${gate.id}/${gateCase.id}: behavior delta requires a tracked owner`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (const claim of manifest.proseClaims ?? []) {
|
||||
if (!criteria.has(claim.criterionId)) {
|
||||
failures.push(`GATE-CLAIM:${claim.id} references unknown criterion ${claim.criterionId}`);
|
||||
continue;
|
||||
}
|
||||
const found = findGateCase(manifest, claim.caseRef ?? '');
|
||||
if (!found) {
|
||||
failures.push(`GATE-CLAIM:${claim.id} references missing exercising case ${claim.caseRef}`);
|
||||
} else if (
|
||||
found.gateCase.mustFail !== true ||
|
||||
!found.gateCase.criterionIds?.includes(claim.criterionId)
|
||||
) {
|
||||
failures.push(
|
||||
`GATE-CLAIM:${claim.id} exercising case ${claim.caseRef} does not exercise criterion ${claim.criterionId}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
for (const criterion of criteria.values()) {
|
||||
if (!boundCriteria.has(criterion.id)) failures.push(`${criterion.id}: no bound case`);
|
||||
else if (!negativeBoundCriteria.has(criterion.id)) {
|
||||
failures.push(`${criterion.id}: no must-fail case exercises this criterion`);
|
||||
}
|
||||
const declaredRefs = new Set(criterion.caseRefs ?? []);
|
||||
const actualRefs = observedCaseRefs.get(criterion.id) ?? new Set();
|
||||
for (const caseRef of declaredRefs) {
|
||||
const found = findGateCase(manifest, caseRef);
|
||||
if (!found) failures.push(`${criterion.id}: declared exercising case ${caseRef} is missing`);
|
||||
else if (!actualRefs.has(caseRef)) {
|
||||
failures.push(`${criterion.id}: declared exercising case ${caseRef} is not bound`);
|
||||
}
|
||||
}
|
||||
for (const caseRef of actualRefs) {
|
||||
if (!declaredRefs.has(caseRef)) {
|
||||
failures.push(`${criterion.id}: bound to undeclared exercising case ${caseRef}`);
|
||||
}
|
||||
}
|
||||
if (
|
||||
criterion.originalText !== criterion.currentText &&
|
||||
(!Array.isArray(criterion.meaningChanges) || criterion.meaningChanges.length === 0)
|
||||
) {
|
||||
failures.push(`${criterion.id}: missing meaning-change provenance`);
|
||||
}
|
||||
}
|
||||
|
||||
const byConstruction = new Map();
|
||||
for (const scenario of manifest.compatibilityScenarios ?? []) {
|
||||
const previous = byConstruction.get(scenario.construction);
|
||||
if (previous && !structuredValuesEqual(previous.expected, scenario.expected)) {
|
||||
failures.push(`${previous.id} and ${scenario.id}: modeled compatibility conflict`);
|
||||
} else {
|
||||
byConstruction.set(scenario.construction, scenario);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function validateClaims(root, manifest, failures) {
|
||||
const registered = new Set((manifest.proseClaims ?? []).map((claim) => claim.id));
|
||||
const observed = new Set();
|
||||
for (const relativeFile of manifest.governingClaimFiles ?? []) {
|
||||
const contents = await readFile(
|
||||
await sandboxPath(root, relativeFile, 'governing claim file'),
|
||||
'utf8',
|
||||
);
|
||||
for (const match of contents.matchAll(/GATE-CLAIM:([A-Z0-9][A-Z0-9-]*)/g)) {
|
||||
observed.add(match[1]);
|
||||
if (!registered.has(match[1])) {
|
||||
failures.push(`GATE-CLAIM:${match[1]} is unbound in ${relativeFile}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
for (const claimId of registered) {
|
||||
if (!observed.has(claimId)) failures.push(`GATE-CLAIM:${claimId} marker is missing`);
|
||||
}
|
||||
}
|
||||
|
||||
async function validateDiscovery(root, manifest, failures) {
|
||||
const registered = new Set(
|
||||
(manifest.gates ?? []).flatMap((gate) => [gate.source, ...(gate.discoveryAliases ?? [])]),
|
||||
);
|
||||
for (const gateRoot of manifest.gateRoots ?? []) {
|
||||
for (const executable of await executableFiles(root, gateRoot)) {
|
||||
if (!registered.has(executable)) failures.push(`unregistered gate: ${executable}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function deploymentFilesEqual(sourcePath, deployedPath) {
|
||||
const [source, deployed] = await Promise.all([readFile(sourcePath), readFile(deployedPath)]);
|
||||
return source.equals(deployed);
|
||||
}
|
||||
|
||||
async function validateDeployment(root, gate, failures, observations) {
|
||||
if (gate.deployment?.kind !== 'file') return;
|
||||
const sourcePath = await sandboxPath(
|
||||
root,
|
||||
gate.deployment.source ?? gate.source,
|
||||
`${gate.id} deployment source`,
|
||||
);
|
||||
const deployedPath = path.isAbsolute(expand(gate.deployment.path, root))
|
||||
? expand(gate.deployment.path, root)
|
||||
: path.resolve(root, expand(gate.deployment.path, root));
|
||||
const source = await readFile(sourcePath);
|
||||
let deployed;
|
||||
try {
|
||||
deployed = await readFile(deployedPath);
|
||||
} catch (error) {
|
||||
if (error.code !== 'ENOENT') throw error;
|
||||
const observedHash = createHash('sha256').update(source).digest('hex');
|
||||
if (
|
||||
!gate.deployment.unavailableOwner ||
|
||||
!gate.deployment.observedSha256 ||
|
||||
gate.deployment.observedSha256 !== observedHash
|
||||
) {
|
||||
failures.push(
|
||||
`${gate.id}: deployed counterpart is unavailable and no current tracked observation matches source`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
observations.push(
|
||||
`DEPLOYED IDENTITY UNAVAILABLE (owner: ${gate.deployment.unavailableOwner}) ${gate.id}: ${deployedPath} is outside this runner; source matches pinned observation ${observedHash}, live equality is not inferred`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (!(await deploymentFilesEqual(sourcePath, deployedPath))) {
|
||||
failures.push(`${gate.id}: source versus deployed copy drift`);
|
||||
return;
|
||||
}
|
||||
|
||||
const controlRoot = await mkdtemp(
|
||||
path.join(path.dirname(root), `.gate-deployment-control-${gate.id}-`),
|
||||
);
|
||||
try {
|
||||
const alteredPath = path.join(controlRoot, 'deployed-copy');
|
||||
await writeFile(
|
||||
alteredPath,
|
||||
Buffer.concat([deployed, Buffer.from('\n# gate deployment drift control\n')]),
|
||||
);
|
||||
if (await deploymentFilesEqual(sourcePath, alteredPath)) {
|
||||
failures.push(`${gate.id}: deployed-copy drift negative control was ineffective`);
|
||||
} else {
|
||||
observations.push(`DEPLOYMENT-NEGATIVE-CONTROL ${gate.id}: observed red`);
|
||||
}
|
||||
} finally {
|
||||
await rm(controlRoot, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
async function validateMutation(root, gate, failures, observations) {
|
||||
const mutation = gate.inertMutation;
|
||||
if (
|
||||
!mutation?.file ||
|
||||
typeof mutation.find !== 'string' ||
|
||||
typeof mutation.replace !== 'string' ||
|
||||
typeof mutation.expected?.exitCode !== 'number'
|
||||
) {
|
||||
failures.push(`${gate.id}: declared inert mutation is incomplete`);
|
||||
return;
|
||||
}
|
||||
const mutationPath = await sandboxPath(root, mutation.file, `${gate.id} mutation source`);
|
||||
let source;
|
||||
try {
|
||||
source = await readFile(mutationPath, 'utf8');
|
||||
} catch (error) {
|
||||
failures.push(`${gate.id}: mutation source unreadable: ${error.message}`);
|
||||
return;
|
||||
}
|
||||
const occurrences = source.split(mutation.find).length - 1;
|
||||
if (occurrences !== 1) {
|
||||
failures.push(
|
||||
`${gate.id}: declared inert mutation is stale or ambiguous (${occurrences} matches)`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
const targetCase = mutation.caseId
|
||||
? (gate.cases ?? []).find((gateCase) => gateCase.id === mutation.caseId)
|
||||
: (gate.cases ?? []).find((gateCase) => gateCase.mustFail === true);
|
||||
if (!targetCase) {
|
||||
failures.push(
|
||||
mutation.caseId
|
||||
? `${gate.id}: declared mutation case ${mutation.caseId} was not found`
|
||||
: `${gate.id}: declared mutation has no must-fail case`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
const sandbox = await mkdtemp(path.join(path.dirname(root), `.gate-sandbox-${gate.id}-`));
|
||||
try {
|
||||
await copySandbox(root, sandbox, mutation.sandboxFiles);
|
||||
await safeSandboxWrite(
|
||||
sandbox,
|
||||
mutation.file,
|
||||
source.replace(mutation.find, mutation.replace),
|
||||
`${gate.id} sandbox mutation write`,
|
||||
);
|
||||
await applyFixture(sandbox, targetCase.fixture);
|
||||
const environment = Object.fromEntries(
|
||||
Object.entries(targetCase.environment ?? {}).map(([key, value]) => [
|
||||
key,
|
||||
expand(value, sandbox),
|
||||
]),
|
||||
);
|
||||
const result = runInvocation(sandbox, targetCase.invocation ?? gate.invocation, environment);
|
||||
if (result.error || result.signal) {
|
||||
failures.push(
|
||||
`${gate.id}: mutation execution crashed${result.signal ? ` with ${result.signal}` : ''}${result.error ? `: ${result.error.message}` : ''}`,
|
||||
);
|
||||
} else if (!outcomeMatches(mutation.expected, result)) {
|
||||
failures.push(
|
||||
`${gate.id}: mutation produced unexpected outcome ${String(result.status)}; expected ${JSON.stringify(mutation.expected)}`,
|
||||
);
|
||||
} else if (resultMatches(targetCase, result)) {
|
||||
failures.push(`${gate.id}: declared inert mutation was ineffective`);
|
||||
} else {
|
||||
observations.push(`META-NEGATIVE-CONTROL ${gate.id}: observed red`);
|
||||
}
|
||||
} finally {
|
||||
await rm(sandbox, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
function findGateCase(manifest, caseRef) {
|
||||
if (typeof caseRef !== 'string') return undefined;
|
||||
const separator = caseRef.indexOf('/');
|
||||
if (separator < 1) return undefined;
|
||||
const gateId = caseRef.slice(0, separator);
|
||||
const caseId = caseRef.slice(separator + 1);
|
||||
const gate = (manifest.gates ?? []).find((candidate) => candidate.id === gateId);
|
||||
const gateCase = (gate?.cases ?? []).find((candidate) => candidate.id === caseId);
|
||||
return gate && gateCase ? { gate, gateCase } : undefined;
|
||||
}
|
||||
|
||||
async function runCompatibilityScenario(root, manifest, scenario, failures, observations) {
|
||||
const referenced = [];
|
||||
for (const caseRef of scenario.caseRefs ?? []) {
|
||||
const found = findGateCase(manifest, caseRef);
|
||||
if (!found) {
|
||||
failures.push(
|
||||
`${scenario.id}: compatibility construction references missing case ${caseRef}`,
|
||||
);
|
||||
} else {
|
||||
referenced.push({ caseRef, ...found });
|
||||
}
|
||||
}
|
||||
if (referenced.length !== (scenario.caseRefs ?? []).length) return;
|
||||
|
||||
const sandbox = await mkdtemp(path.join(path.dirname(root), `.gate-compat-${scenario.id}-`));
|
||||
try {
|
||||
await copySandbox(root, sandbox);
|
||||
const environment = {};
|
||||
const writeSignatures = new Map();
|
||||
const removedPaths = new Set();
|
||||
const fixtures = [...referenced.map(({ gateCase }) => gateCase.fixture), scenario.fixture];
|
||||
for (const fixture of fixtures.filter(Boolean)) {
|
||||
for (const entry of fixture.writeFiles ?? []) {
|
||||
const signature = JSON.stringify({ content: entry.content, mode: entry.mode });
|
||||
const previous = writeSignatures.get(entry.path);
|
||||
if (previous !== undefined && previous !== signature) {
|
||||
failures.push(`${scenario.id}: incompatible fixture writes for ${entry.path}`);
|
||||
return;
|
||||
}
|
||||
if (removedPaths.has(entry.path)) {
|
||||
failures.push(`${scenario.id}: fixture both writes and removes ${entry.path}`);
|
||||
return;
|
||||
}
|
||||
writeSignatures.set(entry.path, signature);
|
||||
}
|
||||
for (const removed of fixture.removePaths ?? []) {
|
||||
if (writeSignatures.has(removed)) {
|
||||
failures.push(`${scenario.id}: fixture both writes and removes ${removed}`);
|
||||
return;
|
||||
}
|
||||
removedPaths.add(removed);
|
||||
}
|
||||
await applyFixture(sandbox, fixture);
|
||||
}
|
||||
for (const source of [
|
||||
...referenced.map(({ gateCase }) => gateCase.environment),
|
||||
scenario.environment,
|
||||
]) {
|
||||
for (const [key, value] of Object.entries(source ?? {})) {
|
||||
if (environment[key] !== undefined && environment[key] !== value) {
|
||||
failures.push(`${scenario.id}: incompatible environment values for ${key}`);
|
||||
return;
|
||||
}
|
||||
environment[key] = value;
|
||||
}
|
||||
}
|
||||
const expandedEnvironment = Object.fromEntries(
|
||||
Object.entries(environment).map(([key, value]) => [key, expand(value, sandbox)]),
|
||||
);
|
||||
const result = runInvocation(sandbox, scenario.invocation, expandedEnvironment);
|
||||
if (result.error || result.signal || !outcomeMatches(scenario.expected, result)) {
|
||||
failures.push(
|
||||
`${scenario.id}: combined compatibility construction ${scenario.construction} observed ${String(result.status)}${result.signal ? ` signal ${result.signal}` : ''}${result.error ? ` error ${result.error.message}` : ''}; expected ${JSON.stringify(scenario.expected)}`,
|
||||
);
|
||||
} else {
|
||||
observations.push(`COMPATIBILITY ${scenario.id}: observed expected outcome`);
|
||||
}
|
||||
} finally {
|
||||
await rm(sandbox, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
export async function verifyRegistry(options) {
|
||||
const failures = [];
|
||||
const observations = [];
|
||||
const manifestPath = path.resolve(options.root, options.manifest);
|
||||
const manifest = JSON.parse(await readFile(manifestPath, 'utf8'));
|
||||
|
||||
validateStructure(manifest, failures);
|
||||
if (options.structureOnly) return { failures, manifest, observations };
|
||||
|
||||
async function collectPhaseFailure(label, action) {
|
||||
try {
|
||||
return await action();
|
||||
} catch (error) {
|
||||
failures.push(`${label}: ${error.message}`);
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
await collectPhaseFailure('governing claim validation failed', () =>
|
||||
validateClaims(options.root, manifest, failures),
|
||||
);
|
||||
await collectPhaseFailure('gate discovery failed', () =>
|
||||
validateDiscovery(options.root, manifest, failures),
|
||||
);
|
||||
|
||||
for (const gate of manifest.gates ?? []) {
|
||||
await collectPhaseFailure(`${gate.id}: deployment validation failed`, () =>
|
||||
validateDeployment(options.root, gate, failures, observations),
|
||||
);
|
||||
for (const gateCase of gate.cases ?? []) {
|
||||
const result = await collectPhaseFailure(
|
||||
`${gate.id}/${gateCase.id}: case execution failed`,
|
||||
() => runCase(options.root, gate, gateCase),
|
||||
);
|
||||
if (!result) continue;
|
||||
const combined = `${result.stdout ?? ''}\n${result.stderr ?? ''}`;
|
||||
try {
|
||||
if (!outcomeMatches(gateCase.actual, result)) {
|
||||
failures.push(
|
||||
`${gate.id}/${gateCase.id}: observed exit ${String(result.status)}${result.signal ? ` signal ${result.signal}` : ''}${result.error ? ` error ${result.error.message}` : ''} or output disagrees with registry actual ${JSON.stringify(gateCase.actual)}`,
|
||||
);
|
||||
}
|
||||
if (gateCase.reasonPattern && !new RegExp(gateCase.reasonPattern, 'm').test(combined)) {
|
||||
failures.push(`${gate.id}/${gateCase.id}: did not fail for its stated reason`);
|
||||
}
|
||||
} catch (error) {
|
||||
failures.push(`${gate.id}/${gateCase.id}: outcome validation failed: ${error.message}`);
|
||||
}
|
||||
if (!structuredValuesEqual(gateCase.required, gateCase.actual) && gateCase.defect?.owner) {
|
||||
observations.push(
|
||||
`DEFECT (owner: ${gateCase.defect.owner}) ${gate.id}/${gateCase.id}: required ${JSON.stringify(gateCase.required)}, actual ${JSON.stringify(gateCase.actual)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
await collectPhaseFailure(`${gate.id}: mutation validation failed`, () =>
|
||||
validateMutation(options.root, gate, failures, observations),
|
||||
);
|
||||
}
|
||||
|
||||
for (const scenario of manifest.compatibilityScenarios ?? []) {
|
||||
await collectPhaseFailure(`${scenario.id}: compatibility validation failed`, () =>
|
||||
runCompatibilityScenario(options.root, manifest, scenario, failures, observations),
|
||||
);
|
||||
}
|
||||
|
||||
return { failures, manifest, observations };
|
||||
}
|
||||
|
||||
async function main() {
|
||||
try {
|
||||
const options = parseArgs(process.argv.slice(2));
|
||||
const { failures, observations, manifest } = await verifyRegistry(options);
|
||||
if (!options.skipHistory && failures.length === 0) {
|
||||
const history = await verifyHistory({ root: options.root, manifest });
|
||||
failures.push(...history.failures);
|
||||
observations.push(...history.observations);
|
||||
}
|
||||
for (const observation of observations) process.stdout.write(`${observation}\n`);
|
||||
if (failures.length > 0) {
|
||||
for (const failure of failures) process.stderr.write(`GATE VERIFY FAILED: ${failure}\n`);
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
const defects = observations.filter((line) => line.startsWith('DEFECT ')).length;
|
||||
process.stdout.write(
|
||||
`registry observations matched; open behavior deltas: ${defects}; required-behavior conformance is not asserted while deltas remain\n`,
|
||||
);
|
||||
} catch (error) {
|
||||
process.stderr.write(`GATE VERIFY FAILED: ${error.message}\n`);
|
||||
process.exitCode = 1;
|
||||
}
|
||||
}
|
||||
|
||||
if (import.meta.url === `file://${process.argv[1]}`) await main();
|
||||
@@ -0,0 +1,568 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { chmod, copyFile, mkdir, readFile, rm, symlink, writeFile } from 'node:fs/promises';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import test from 'node:test';
|
||||
|
||||
const verifier = path.join(process.cwd(), 'scripts', 'gate-verify.mjs');
|
||||
const fixtureBase = path.join(process.cwd(), '.mosaic-test-work', `gate-verify-${process.pid}`);
|
||||
|
||||
async function fixture(name = 'case') {
|
||||
const root = path.join(fixtureBase, name);
|
||||
await rm(root, { recursive: true, force: true });
|
||||
await mkdir(path.join(root, 'gates'), { recursive: true });
|
||||
return root;
|
||||
}
|
||||
|
||||
function baseManifest() {
|
||||
return {
|
||||
schemaVersion: 1,
|
||||
activationCommit: null,
|
||||
gateRoots: ['gates'],
|
||||
governingClaimFiles: [],
|
||||
coverageBoundary: { included: ['meta fixture'], excluded: [], trackedBy: 'RM-54' },
|
||||
criteria: [
|
||||
{
|
||||
id: 'META-CRIT-1',
|
||||
originalText: 'The fixture rejects its bad input.',
|
||||
currentText: 'The fixture rejects its bad input.',
|
||||
claimType: 'integrity',
|
||||
source: 'fixture',
|
||||
meaningChanges: [],
|
||||
caseRefs: ['meta-fixture/rejects-bad-input'],
|
||||
},
|
||||
],
|
||||
compatibilityScenarios: [],
|
||||
proseClaims: [],
|
||||
gates: [
|
||||
{
|
||||
id: 'meta-fixture',
|
||||
source: 'gates/meta-fixture.sh',
|
||||
invocation: ['gates/meta-fixture.sh'],
|
||||
deployment: { kind: 'none', reason: 'test fixture only' },
|
||||
inertMutation: {
|
||||
file: 'gates/meta-fixture.sh',
|
||||
find: 'exit 7',
|
||||
replace: 'exit 0',
|
||||
expected: { exitCode: 0 },
|
||||
},
|
||||
cases: [
|
||||
{
|
||||
id: 'rejects-bad-input',
|
||||
criterionIds: ['META-CRIT-1'],
|
||||
mustFail: true,
|
||||
invocation: ['gates/meta-fixture.sh'],
|
||||
required: { exitCode: 7 },
|
||||
actual: { exitCode: 7 },
|
||||
reasonPattern: 'META_REJECT',
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
async function writeGate(root, contents = '#!/bin/sh\necho META_REJECT >&2\nexit 7\n') {
|
||||
const target = path.join(root, 'gates', 'meta-fixture.sh');
|
||||
await writeFile(target, contents);
|
||||
await chmod(target, 0o755);
|
||||
}
|
||||
|
||||
async function writeManifest(root, manifest) {
|
||||
await writeFile(path.join(root, 'gates', 'gates.manifest.json'), `${JSON.stringify(manifest)}\n`);
|
||||
}
|
||||
|
||||
function verify(root, extraArgs = []) {
|
||||
return spawnSync(
|
||||
process.execPath,
|
||||
[
|
||||
verifier,
|
||||
'--root',
|
||||
root,
|
||||
'--manifest',
|
||||
'gates/gates.manifest.json',
|
||||
'--skip-history',
|
||||
...extraArgs,
|
||||
],
|
||||
{ cwd: root, encoding: 'utf8', env: { ...process.env, HOME: os.homedir() } },
|
||||
);
|
||||
}
|
||||
|
||||
function output(result) {
|
||||
return `${result.stdout}\n${result.stderr}`;
|
||||
}
|
||||
|
||||
test.after(async () => {
|
||||
await rm(fixtureBase, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('an externally inerted failure branch makes verification nonzero and names the gate', async () => {
|
||||
const root = await fixture('external-inert');
|
||||
await writeGate(root, '#!/bin/sh\nexit 0\n');
|
||||
await writeManifest(root, baseManifest());
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture/);
|
||||
});
|
||||
|
||||
test('the verifier applies the declared inert mutation and observes its own control red', async () => {
|
||||
const root = await fixture('internal-meta');
|
||||
await writeGate(root);
|
||||
await writeManifest(root, baseManifest());
|
||||
|
||||
const result = verify(root);
|
||||
assert.equal(result.status, 0, output(result));
|
||||
assert.match(output(result), /META-NEGATIVE-CONTROL.*meta-fixture.*observed red/i);
|
||||
});
|
||||
|
||||
test('a mutation crash is rejected instead of counted as an observed-red control', async () => {
|
||||
const root = await fixture('mutation-crash');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].inertMutation.replace = 'this is not valid shell (';
|
||||
manifest.gates[0].inertMutation.expected = { exitCode: 0 };
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*mutation.*unexpected outcome/i);
|
||||
assert.doesNotMatch(output(result), /META-NEGATIVE-CONTROL.*observed red/i);
|
||||
});
|
||||
|
||||
test('a stale declared mutation case id is rejected instead of falling back', async () => {
|
||||
const root = await fixture('stale-case-id');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].inertMutation.caseId = 'case-that-does-not-exist';
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*case-that-does-not-exist.*not found/i);
|
||||
});
|
||||
|
||||
test('duplicate stable ids and unsupported schema versions are rejected', async () => {
|
||||
const root = await fixture('closed-schema');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.schemaVersion = 99;
|
||||
manifest.criteria.push({ ...manifest.criteria[0] });
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /unsupported schemaVersion 99/i);
|
||||
assert.match(output(result), /duplicate criterion id META-CRIT-1/i);
|
||||
});
|
||||
|
||||
test('manifest-controlled fixture paths cannot escape the sandbox', async () => {
|
||||
const root = await fixture('path-traversal');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases[0].fixture = {
|
||||
writeFiles: [{ path: '../../escaped-by-manifest', content: 'bad' }],
|
||||
};
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /path escapes sandbox/i);
|
||||
});
|
||||
|
||||
test('fixture writes reject a final symlink and preserve its outside target', async () => {
|
||||
const root = await fixture('final-symlink');
|
||||
await writeGate(root);
|
||||
const outside = path.join(fixtureBase, 'outside-sentinel');
|
||||
await writeFile(outside, 'preserve-me\n');
|
||||
const linked = path.join(root, 'linked-sentinel');
|
||||
await symlink(outside, linked);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases[0].fixture = {
|
||||
writeFiles: [{ path: 'linked-sentinel', content: 'overwritten\n' }],
|
||||
};
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /fixture write.*symbolic link/i);
|
||||
assert.equal(await readFile(outside, 'utf8'), 'preserve-me\n');
|
||||
});
|
||||
|
||||
test('an executable below a gate root without an entry is rejected', async () => {
|
||||
const root = await fixture('unregistered');
|
||||
await writeGate(root);
|
||||
const extra = path.join(root, 'gates', 'forgotten.sh');
|
||||
await writeFile(extra, '#!/bin/sh\nexit 1\n');
|
||||
await chmod(extra, 0o755);
|
||||
await writeManifest(root, baseManifest());
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /unregistered gate.*forgotten\.sh/i);
|
||||
});
|
||||
|
||||
test('a must-fail case without a reason diagnostic is rejected', async () => {
|
||||
const root = await fixture('missing-reason');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases[0].reasonPattern = '';
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*rejects-bad-input.*reasonPattern/i);
|
||||
});
|
||||
|
||||
test('a gate with zero must-fail cases is rejected', async () => {
|
||||
const root = await fixture('no-negative');
|
||||
await writeGate(root, '#!/bin/sh\nexit 0\n');
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].inertMutation = {
|
||||
file: 'gates/meta-fixture.sh',
|
||||
find: 'exit 0',
|
||||
replace: 'exit 1',
|
||||
};
|
||||
manifest.gates[0].cases = [
|
||||
{
|
||||
id: 'positive',
|
||||
criterionIds: ['META-CRIT-1'],
|
||||
mustFail: false,
|
||||
invocation: ['gates/meta-fixture.sh'],
|
||||
required: { exitCode: 0 },
|
||||
actual: { exitCode: 0 },
|
||||
reasonPattern: '',
|
||||
},
|
||||
];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*no negative control/i);
|
||||
});
|
||||
|
||||
test('reordered but equivalent outcome fields do not create a false behavior delta', async () => {
|
||||
const root = await fixture('reordered-outcomes');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases[0].required = { exitCode: 7, outputPattern: 'META_REJECT' };
|
||||
manifest.gates[0].cases[0].actual = { outputPattern: 'META_REJECT', exitCode: 7 };
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.equal(result.status, 0, output(result));
|
||||
assert.doesNotMatch(output(result), /behavior delta requires|DEFECT \(owner:/);
|
||||
});
|
||||
|
||||
test('a required-versus-actual delta without a tracked owner is rejected', async () => {
|
||||
const root = await fixture('ownerless-delta');
|
||||
await writeGate(root, '#!/bin/sh\nexit 0\n');
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].inertMutation.find = 'exit 0';
|
||||
manifest.gates[0].inertMutation.replace = 'exit 7';
|
||||
manifest.gates[0].cases[0].actual.exitCode = 0;
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*delta.*tracked owner/i);
|
||||
});
|
||||
|
||||
test('moving criterion bindings to unrelated cases is rejected', async () => {
|
||||
const root = await fixture('semantic-misbinding');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.criteria.push({
|
||||
id: 'META-CRIT-2',
|
||||
originalText: 'The fixture reports the second rejection reason.',
|
||||
currentText: 'The fixture reports the second rejection reason.',
|
||||
claimType: 'integrity',
|
||||
source: 'fixture',
|
||||
meaningChanges: [],
|
||||
caseRefs: ['meta-fixture/rejects-second-input'],
|
||||
});
|
||||
manifest.gates[0].cases.push({
|
||||
...manifest.gates[0].cases[0],
|
||||
id: 'rejects-second-input',
|
||||
criterionIds: ['META-CRIT-1'],
|
||||
});
|
||||
manifest.gates[0].cases[0].criterionIds = ['META-CRIT-2'];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /META-CRIT-1.*rejects-bad-input.*not bound/i);
|
||||
assert.match(output(result), /META-CRIT-2.*rejects-second-input.*not bound/i);
|
||||
});
|
||||
|
||||
test('canonical verification preserves binding diagnostics when a case fixture is also stale', async () => {
|
||||
const root = await fixture('misbinding-plus-stale-fixture');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.criteria.push({
|
||||
id: 'META-CRIT-2',
|
||||
originalText: 'The second case rejects its own bad input.',
|
||||
currentText: 'The second case rejects its own bad input.',
|
||||
claimType: 'integrity',
|
||||
source: 'fixture',
|
||||
meaningChanges: [],
|
||||
caseRefs: ['meta-fixture/rejects-second-input'],
|
||||
});
|
||||
manifest.gates[0].cases.push({
|
||||
...manifest.gates[0].cases[0],
|
||||
id: 'rejects-second-input',
|
||||
criterionIds: ['META-CRIT-1'],
|
||||
});
|
||||
manifest.gates[0].cases[0].criterionIds = ['META-CRIT-2'];
|
||||
manifest.gates[0].cases[0].fixture = {
|
||||
replaceFiles: [
|
||||
{
|
||||
path: 'gates/meta-fixture.sh',
|
||||
find: 'text that is not present',
|
||||
replace: 'irrelevant',
|
||||
},
|
||||
],
|
||||
};
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /META-CRIT-1.*rejects-bad-input.*not bound/i);
|
||||
assert.match(output(result), /META-CRIT-2.*rejects-second-input.*not bound/i);
|
||||
assert.match(output(result), /fixture replace.*stale or ambiguous/i);
|
||||
});
|
||||
|
||||
test('moving meaning and prose criteria to an unrelated type error is rejected', async () => {
|
||||
const root = await fixture('real-manifest-misbinding');
|
||||
const manifest = JSON.parse(
|
||||
await readFile(path.join(process.cwd(), 'gates', 'gates.manifest.json'), 'utf8'),
|
||||
);
|
||||
for (const gate of manifest.gates) {
|
||||
for (const gateCase of gate.cases) {
|
||||
gateCase.criterionIds = gateCase.criterionIds.filter(
|
||||
(id) => !['RM02-MEANING-PROVENANCE', 'RM02-PROSE-CONTROL'].includes(id),
|
||||
);
|
||||
}
|
||||
}
|
||||
const typeError = manifest.gates
|
||||
.find((gate) => gate.id === 'quality-typecheck')
|
||||
.cases.find((gateCase) => gateCase.id === 'type-error');
|
||||
typeError.criterionIds.push('RM02-MEANING-PROVENANCE', 'RM02-PROSE-CONTROL');
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root, ['--structure-only']);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /RM02-MEANING-PROVENANCE.*missing-meaning-provenance.*not bound/i);
|
||||
assert.match(output(result), /RM02-PROSE-CONTROL.*prose-claim-misbinding.*not bound/i);
|
||||
assert.match(
|
||||
output(result),
|
||||
/RM02-(?:MEANING-PROVENANCE|PROSE-CONTROL).*undeclared exercising case quality-typecheck\/type-error/i,
|
||||
);
|
||||
});
|
||||
|
||||
test('a criterion with no bound case is rejected', async () => {
|
||||
const root = await fixture('unbound-criterion');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.criteria.push({
|
||||
id: 'ORPHAN',
|
||||
originalText: 'This criterion is not exercised.',
|
||||
currentText: 'This criterion is not exercised.',
|
||||
claimType: 'quality',
|
||||
source: 'fixture',
|
||||
meaningChanges: [],
|
||||
});
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /ORPHAN.*no bound case/i);
|
||||
});
|
||||
|
||||
test('an unbound governing prose marker is rejected', async () => {
|
||||
const root = await fixture('unbound-prose');
|
||||
await writeGate(root);
|
||||
await mkdir(path.join(root, 'docs'), { recursive: true });
|
||||
await writeFile(path.join(root, 'docs', 'governing.md'), 'GATE-CLAIM:UNBOUND\n');
|
||||
const manifest = baseManifest();
|
||||
manifest.governingClaimFiles = ['docs/governing.md'];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /GATE-CLAIM:UNBOUND.*unbound/i);
|
||||
});
|
||||
|
||||
test('directly contradictory modeled scenarios are rejected', async () => {
|
||||
const root = await fixture('conflict');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.compatibilityScenarios = [
|
||||
{
|
||||
id: 'ONE',
|
||||
construction: 'same-input',
|
||||
caseRefs: ['meta-fixture/rejects-bad-input'],
|
||||
invocation: ['sh', '-c', 'exit 0'],
|
||||
expected: { exitCode: 0 },
|
||||
},
|
||||
{
|
||||
id: 'TWO',
|
||||
construction: 'same-input',
|
||||
caseRefs: ['meta-fixture/rejects-bad-input'],
|
||||
invocation: ['sh', '-c', 'exit 1'],
|
||||
expected: { exitCode: 1 },
|
||||
},
|
||||
];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /ONE.*TWO.*conflict/i);
|
||||
});
|
||||
|
||||
test('compatibility scenarios execute referenced conditions as one construction', async () => {
|
||||
const root = await fixture('combined-compatibility');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].cases.push({
|
||||
id: 'second-condition',
|
||||
criterionIds: ['META-CRIT-1'],
|
||||
mustFail: true,
|
||||
invocation: ['sh', '-c', 'echo "$SECOND_REASON" >&2; exit 7'],
|
||||
fixture: { writeFiles: [{ path: 'conditions/second', content: 'present\n' }] },
|
||||
required: { exitCode: 7 },
|
||||
actual: { exitCode: 7 },
|
||||
reasonPattern: 'SECOND_REASON',
|
||||
environment: { SECOND_REASON: 'SECOND_REASON' },
|
||||
});
|
||||
manifest.criteria[0].caseRefs.push('meta-fixture/second-condition');
|
||||
manifest.gates[0].cases[0].fixture = {
|
||||
writeFiles: [{ path: 'conditions/first', content: 'present\n' }],
|
||||
};
|
||||
manifest.compatibilityScenarios = [
|
||||
{
|
||||
id: 'BOTH-CONDITIONS',
|
||||
construction: 'both-fixtures',
|
||||
caseRefs: ['meta-fixture/rejects-bad-input', 'meta-fixture/second-condition'],
|
||||
invocation: ['sh', '-c', 'test -f conditions/first && test -f conditions/second'],
|
||||
expected: { exitCode: 0 },
|
||||
},
|
||||
];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.equal(result.status, 0, output(result));
|
||||
assert.match(output(result), /COMPATIBILITY BOTH-CONDITIONS: observed expected outcome/i);
|
||||
});
|
||||
|
||||
test('a restated criterion without provenance is rejected', async () => {
|
||||
const root = await fixture('provenance');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.criteria[0].currentText = 'The fixture rejects only malformed input.';
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /META-CRIT-1.*meaning-change provenance/i);
|
||||
});
|
||||
|
||||
test('a security criterion bound only to a positive case is unregistered in substance', async () => {
|
||||
const root = await fixture('positive-only-criterion');
|
||||
await writeGate(root);
|
||||
const manifest = baseManifest();
|
||||
manifest.criteria.push({
|
||||
id: 'POSITIVE-ONLY',
|
||||
originalText: 'A security property.',
|
||||
currentText: 'A security property.',
|
||||
claimType: 'security',
|
||||
source: 'fixture',
|
||||
meaningChanges: [],
|
||||
});
|
||||
manifest.gates[0].cases.push({
|
||||
id: 'positive-only',
|
||||
criterionIds: ['POSITIVE-ONLY'],
|
||||
mustFail: false,
|
||||
invocation: ['gates/meta-fixture.sh'],
|
||||
required: { exitCode: 7 },
|
||||
actual: { exitCode: 7 },
|
||||
reasonPattern: '',
|
||||
});
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /POSITIVE-ONLY.*no must-fail case/i);
|
||||
});
|
||||
|
||||
test('a registered prose claim whose marker is absent is rejected', async () => {
|
||||
const root = await fixture('missing-prose-marker');
|
||||
await writeGate(root);
|
||||
await mkdir(path.join(root, 'docs'), { recursive: true });
|
||||
await writeFile(path.join(root, 'docs', 'governing.md'), 'No marker here.\n');
|
||||
const manifest = baseManifest();
|
||||
manifest.governingClaimFiles = ['docs/governing.md'];
|
||||
manifest.proseClaims = [{ id: 'MISSING-MARKER', criterionId: 'META-CRIT-1' }];
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /GATE-CLAIM:MISSING-MARKER.*missing/i);
|
||||
});
|
||||
|
||||
test('source-versus-deployed byte drift is rejected and names the gate', async () => {
|
||||
const root = await fixture('deployment-drift');
|
||||
await writeGate(root);
|
||||
await mkdir(path.join(root, 'deployed'), { recursive: true });
|
||||
await writeFile(path.join(root, 'deployed', 'meta-fixture.sh'), '#!/bin/sh\nexit 0\n');
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].deployment = { kind: 'file', path: 'deployed/meta-fixture.sh' };
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const result = verify(root);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*source.*deployed.*drift/i);
|
||||
});
|
||||
|
||||
test('deployment drift meta-control fails if the shared comparator is made inert', async () => {
|
||||
const root = await fixture('deployment-comparator-inert');
|
||||
await writeGate(root);
|
||||
await mkdir(path.join(root, 'deployed'), { recursive: true });
|
||||
await copyFile(
|
||||
path.join(root, 'gates', 'meta-fixture.sh'),
|
||||
path.join(root, 'deployed', 'meta-fixture.sh'),
|
||||
);
|
||||
const manifest = baseManifest();
|
||||
manifest.gates[0].deployment = { kind: 'file', path: 'deployed/meta-fixture.sh' };
|
||||
await writeManifest(root, manifest);
|
||||
|
||||
const alteredScripts = path.join(root, 'verifier-scripts');
|
||||
await mkdir(alteredScripts, { recursive: true });
|
||||
const verifierSource = await readFile(verifier, 'utf8');
|
||||
const inertSource = verifierSource.replace(
|
||||
'return source.equals(deployed);',
|
||||
'return true; // deliberate test-only inert comparator',
|
||||
);
|
||||
assert.notEqual(inertSource, verifierSource, 'shared deployment comparator mutation went stale');
|
||||
await writeFile(path.join(alteredScripts, 'gate-verify.mjs'), inertSource);
|
||||
await copyFile(
|
||||
path.join(process.cwd(), 'scripts', 'gate-history.mjs'),
|
||||
path.join(alteredScripts, 'gate-history.mjs'),
|
||||
);
|
||||
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[
|
||||
path.join(alteredScripts, 'gate-verify.mjs'),
|
||||
'--root',
|
||||
root,
|
||||
'--manifest',
|
||||
'gates/gates.manifest.json',
|
||||
'--skip-history',
|
||||
],
|
||||
{ cwd: root, encoding: 'utf8', env: { ...process.env, HOME: os.homedir() } },
|
||||
);
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(output(result), /meta-fixture.*drift negative control was ineffective/i);
|
||||
});
|
||||
@@ -0,0 +1,104 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import test from 'node:test';
|
||||
|
||||
const root = process.cwd();
|
||||
const expectedTriggers = `when:
|
||||
# PR + manual CI run on any branch — the pull_request pipeline is the merge gate.
|
||||
# push CI is restricted to protected branches (main) so a feature-branch push no
|
||||
# longer fires a redundant SECOND pipeline alongside its PR pipeline. This ~halves
|
||||
# CI load on the storage-constrained runner with zero loss of gating (branch
|
||||
# protection requires no push/ci status context; main still gets full push CI).
|
||||
- event: [pull_request, manual]
|
||||
- event: push
|
||||
branch: main`;
|
||||
const expectedGateStep = ` image: *node_image
|
||||
# Woodpecker's shallow marker makes merge-base reject even present parents;
|
||||
# full history is required for activation ancestry and manifest provenance.
|
||||
commands:
|
||||
- *enable_pnpm
|
||||
- apk add --no-cache bubblewrap
|
||||
- if [ -f .git/shallow ]; then git fetch --unshallow --no-tags origin; fi
|
||||
- pnpm gate:verify
|
||||
depends_on:
|
||||
- install
|
||||
- sanitization
|
||||
- upgrade-guard`;
|
||||
|
||||
export function assertUnprivilegedGateStep(pipeline) {
|
||||
assert.doesNotMatch(
|
||||
pipeline,
|
||||
/privileged/i,
|
||||
'no pull-request pipeline step may declare privilege',
|
||||
);
|
||||
for (const line of pipeline.split('\n')) {
|
||||
const candidate = line.trimStart().replace(/^-\s+/, '');
|
||||
if (/^(?:["'!<].*|[A-Za-z_][A-Za-z0-9_-]*\s+):(?:\s|$)/.test(candidate)) {
|
||||
assert.fail(`non-canonical or merged YAML key is forbidden: ${candidate}`);
|
||||
}
|
||||
}
|
||||
const triggerMatches = [...pipeline.matchAll(/^when:\n([\s\S]*?)(?=\n\n)/gm)];
|
||||
assert.equal(triggerMatches.length, 1, 'exactly one top-level trigger is required');
|
||||
assert.equal(
|
||||
`when:\n${triggerMatches[0][1].trimEnd()}`,
|
||||
expectedTriggers,
|
||||
'top-level triggers must match closed PR/main construction',
|
||||
);
|
||||
|
||||
const matches = [
|
||||
...pipeline.matchAll(/\n gate-verify:\n([\s\S]*?)(?=\n [a-z][a-z0-9-]+:|\nservices:|$)/g),
|
||||
];
|
||||
assert.equal(matches.length, 1, 'exactly one gate-verify step is required');
|
||||
// Closed textual construction by design: accepting arbitrary YAML syntax here
|
||||
// would require a duplicate-key-preserving parser. Exact equality rejects all
|
||||
// extra keys, quoted/escaped key spellings, aliases, and mapping merges.
|
||||
assert.equal(matches[0][1].trimEnd(), expectedGateStep, 'gate-verify step must match closed unprivileged construction');
|
||||
}
|
||||
|
||||
test('package.json exposes the canonical gate:verify command', async () => {
|
||||
const packageJson = JSON.parse(await readFile(`${root}/package.json`, 'utf8'));
|
||||
assert.equal(packageJson.scripts['gate:verify'], 'node scripts/gate-verify.mjs');
|
||||
});
|
||||
|
||||
test('Woodpecker runs the closed unprivileged gate construction on every pipeline', async () => {
|
||||
const pipeline = await readFile(`${root}/.woodpecker/ci.yml`, 'utf8');
|
||||
assertUnprivilegedGateStep(pipeline);
|
||||
});
|
||||
|
||||
test('gate wiring rejects privilege syntax, merges, duplicate keys, and trigger narrowing', async () => {
|
||||
const pipeline = await readFile(`${root}/.woodpecker/ci.yml`, 'utf8');
|
||||
const additions = [
|
||||
' privileged: *enabled\n',
|
||||
' "privileged": true\n',
|
||||
" 'privileged': true\n",
|
||||
' privileged : true\n',
|
||||
' "priv\\u0069leged": true\n',
|
||||
' <<: *privileged-step\n',
|
||||
' "<<": *privileged-step\n',
|
||||
];
|
||||
for (const addition of additions) {
|
||||
const changed = pipeline.replace(' gate-verify:\n image:', ` gate-verify:\n${addition} image:`);
|
||||
assert.throws(() => assertUnprivilegedGateStep(changed));
|
||||
}
|
||||
const privilegedInstall = pipeline.replace(
|
||||
' install:\n image:',
|
||||
' install:\n privileged: true\n image:',
|
||||
);
|
||||
const duplicate = `${pipeline}\n gate-verify:\n image: *node_image\n`;
|
||||
const noPullRequest = pipeline.replace(
|
||||
' - event: [pull_request, manual]',
|
||||
' - event: manual',
|
||||
);
|
||||
const filteredPullRequest = pipeline.replace(
|
||||
' - event: [pull_request, manual]',
|
||||
' - event: [pull_request, manual]\n path: [scripts/**]',
|
||||
);
|
||||
|
||||
assert.throws(() => assertUnprivilegedGateStep(privilegedInstall), /privilege/i);
|
||||
assert.throws(() => assertUnprivilegedGateStep(duplicate), /exactly one gate-verify/);
|
||||
assert.throws(() => assertUnprivilegedGateStep(noPullRequest), /closed PR\/main construction/);
|
||||
assert.throws(
|
||||
() => assertUnprivilegedGateStep(filteredPullRequest),
|
||||
/closed PR\/main construction/,
|
||||
);
|
||||
});
|
||||
Reference in New Issue
Block a user