91 lines
15 KiB
Markdown
91 lines
15 KiB
Markdown
# RM-02 Gate Registry Scratchpad (#1029)
|
|
|
|
## Objective
|
|
|
|
Deliver the seven-gate registry and RED-first anti-inert verifier on `feat/rm-02-gate-registry`, preserving required-versus-actual defects without laundering them as success.
|
|
|
|
## Constraints and boundaries
|
|
|
|
- Do not modify `ci-queue-wait.sh`; RM-03 owns that fix.
|
|
- Do not modify `docs/remediation/TASKS.md`; workers cannot edit orchestrator tracking.
|
|
- Every declared behavior must be observed, not inferred from an exit code.
|
|
- Repository-local evidence does not establish same-authority tamper resistance; RM-25/RM-59 own the external trust anchor.
|
|
- Coverage is seven logical gates; broader inventory is RM-54.
|
|
- Budget assumption: no explicit token ceiling was supplied. Keep implementation dependency-free (stock Node), avoid repeated full monorepo installs, and keep generated test artifacts under the worktree/main disk.
|
|
|
|
## Plan
|
|
|
|
1. Update PRD and tracking references.
|
|
2. Write black-box meta-negative-control first and observe it fail for the missing verifier behavior.
|
|
3. Implement minimal manifest parser/case runner/mutation detector; observe meta-control succeed.
|
|
4. Add schema, coverage, provenance, prose marker, compatibility, discovery, deployment identity, and defect-delta tests RED-first.
|
|
5. Register seven gates with exact cases and run each case.
|
|
6. Add prospective per-commit replay and bounded provider-evidence reporting.
|
|
7. Wire unconditional Woodpecker CI and update developer/admin documentation.
|
|
8. Run situational and baseline verification, independent Codex review, remediate, commit, queue guard, push, PR, coordinator/reviewer handoff.
|
|
|
|
## Progress
|
|
|
|
- 2026-08-01: Design approved by `mos-remediation`; rulings A-E and source/deployed identity addition incorporated.
|
|
- 2026-08-01: Isolated worktree created from `origin/main` f65e9ea6; RM-01 f58b3699 verified as ancestor.
|
|
- 2026-08-01: Git author set to `f10-coder <[email protected]>`; provider issue #1029 created with `MOSAIC_GIT_IDENTITY=f10-coder`.
|
|
- 2026-08-01: Source/deployed queue-guard SHA-256 observed equal (`19cda2f...`); this observation is not yet an enforced property.
|
|
|
|
## Tests and RED-first evidence
|
|
|
|
- Meta-negative RED first: `node --test scripts/gate-verify.test.mjs` failed because the absent verifier did not name externally inerted `meta-fixture`.
|
|
- Structural RED: nine tests failed before implementation for internal mutation, unregistered executable, missing negative control, ownerless delta, unbound criterion/claim, modeled conflict, missing provenance, and deployment drift.
|
|
- Clause hardening RED: positive-only security criterion and missing registered claim marker both passed incorrectly before validation was added.
|
|
- CI wiring RED: package script and unconditional Woodpecker step tests both failed before wiring.
|
|
- History RED: history test failed with missing module before own-tree manifest selection/provider classification was implemented.
|
|
- `pnpm gate:verify`: exit 0; seven gates each reported `META-NEGATIVE-CONTROL ... observed red`; queue source/deployed drift control observed red; six queue behavior deltas printed as `DEFECT (owner: RM-03)`.
|
|
- Focused Node tests: 54/54 pass after third-round hardening (36 verifier/wiring plus 18 history/provider tests).
|
|
- `pnpm typecheck`: pass (45/45 Turbo tasks).
|
|
- `pnpm lint`: pass (25/25 Turbo tasks).
|
|
- `pnpm format:check`: pass.
|
|
- `pnpm test`: repository suites reached 45/46 Turbo tasks; all application/package tests shown passed, then the known host-specific wake assertion aborted exit 97 (`BASH_LINENO convention violated`, #973/D-16). No test was edited or bypassed. CI remains the authoritative full-suite environment.
|
|
|
|
## Self-surfaced defect
|
|
|
|
The queue guard's `get_state_from_status_json` runs `python3 - <<'PY'` while provider JSON is piped to the shell function. The heredoc owns stdin, so Python never reads provider JSON. Terminal success, no-status, terminal failure, and malformed payloads all classify as `unknown`; the associated fail-open outcomes are recorded under RM-03. No queue-guard source was modified.
|
|
|
|
## Independent review remediation
|
|
|
|
- Final pre-commit Codex code review found three blockers: a tautological deployment drift control, independently observed rather than combined compatibility cases, and unauthorized edits to orchestrator-owned `TASKS.md`.
|
|
- Deployment identity now uses one shared file comparator for both live equality and a temporary drifted deployed copy; a test makes that comparator inert and proves the meta-control fails.
|
|
- Compatibility scenarios now merge referenced fixtures/environments into one isolated construction and execute an exact scenario invocation; a test proves two independently valid fixtures coexist in the combined run.
|
|
- `TASKS.md` changes were reverted. `docs/remediation/GATE-CLAIMS.md` binds source headings and anchored text without editing orchestrator tracking.
|
|
- Codex security review found the Bubblewrap replay shared the runner PID namespace. Replay now unshares PID, IPC, and UTS namespaces, and an abuse-case test proves a sibling runner PID is invisible.
|
|
- Second review found empty reason diagnostics, final-symlink fixture writes, and lifecycle-script mutation of authoritative history files. Must-fail cases now require a reason pattern; writes use no-follow semantics; and replay snapshots every archived file before install and rejects any changed, deleted, or type/mode-shifted source before executing the verifier. Dedicated negative tests cover all three.
|
|
- Third code review found ambiguous duplicate provider steps and order-sensitive JSON outcome comparison. Provider evidence now requires exactly one `gate-verify` step in the authoritative rerun, and structural equality normalizes object keys. Both regressions have RED-first tests. Third security review reported no findings.
|
|
- Initial PR pipeline #2177 exposed Woodpecker's shallow boundary: the activation parent object was present but marked shallow, so `merge-base --is-ancestor` correctly refused to infer ancestry. The unconditional gate step now unshallows before ancestry/provenance checks; its wiring test was observed RED before the CI fix.
|
|
- Pipeline #2178 then proved the unprivileged Docker runner cannot establish Bubblewrap namespaces. A privileged experiment remained uncommitted and was rejected after Codex correctly rated it CRITICAL: PR-controlled code executes before an in-repository sandbox and could directly use the granted capability.
|
|
- `mos-remediation` and `rev-974` independently ruled Option C. RM02-REQ-10 now retains its original text, restatement, and reason: PR CI verifies only the current tree, unprivileged and fail-closed; isolated own-tree replay is deferred to RM-60/#1031's external pre-execution authority, cross-referenced with RM-59. Future protected post-merge replay is detection with quarantine/revert, never pre-merge prevention.
|
|
- RED-first boundary test proved the old path executed an inert intermediate verifier. The revised path states adjacent `DOES`/`DOES NOT` claims, validates historical manifest provenance without executing it, and infers no replay success. Direct sandbox tests remain hard-fail; unprivileged CI asserts terminal refusal instead of treating replay as success. Pipelines #2179/#2180/#2181 exposed two runner refusal forms: namespace denial as `spawnSync bwrap` with `error.code=EPERM`, and a test image without Bubblewrap as `error.code=ENOENT`. The replay diagnostic now preserves spawn errors. Parent-generated launcher/entry metadata distinguishes refusal before sandbox entry from child-controlled output; the detector recognizes exact `spawnSync bwrap` provenance for `EPERM`/`EACCES`/`ENOENT` and known namespace-refusal text only when the entry command provably did not run. Focused negative assertions reject unrelated `spawnSync git EPERM`, verifier output that merely says `bwrap ENOENT`, and exact namespace-denial impersonation without provenance or after sandbox entry.
|
|
- Exact-head independent review at `38f1b249` found one valid diagnostic-masking blocker: canonical verification knew four stable-ID rebinding failures but a thrown stale fixture replacement reached the outer catch first and emitted only the generic error. RED-first reproduction confirmed the canonical path omitted both responsible criterion IDs. Verification now collects labeled failures independently across claims, discovery, deployment, case execution/outcome checks, mutation, and compatibility, preserving structural stable-ID failures alongside the stale-fixture signal. The canonical regression test requires both missing-binding IDs and the generic fixture error.
|
|
- Exact-head independent review at `9b4d4beb` found two valid blockers. RED-first controls reproduced both: denial-looking child stderr was accepted as sandbox unavailability, and moving meaning/prose criterion IDs to an unrelated type-error case left `gate:verify` green. Bubblewrap execution now emits a parent-generated random entry marker and returns parent-owned launcher/entry metadata; unavailability requires Bubblewrap launcher provenance plus proof entry never ran, so exact denial impersonation from plain or entered-child results is rejected. Criterion objects now declare exact `caseRefs`, checked bidirectionally against case-side `criterionIds`; prose claims declare an exact must-fail `caseRef`. Registered must-fail cases move a criterion binding, remove meaning provenance, and redirect a prose claim, each producing its stable reason. The review freeze was deliberately lifted before remediation.
|
|
- Option C security review reported no findings. Code review rejected an initial unrelated typecheck binding for the new security criterion. It was replaced with a dedicated registered `privileged-pr-gate` case: the fixture injects a privilege key into the gate step, the wiring control rejects it for that exact reason, and `gate:verify` observes the boundary negative control. Follow-up hardening uses a closed exact gate-step construction, rejects privilege across the entire pipeline, rejects non-canonical/merged YAML keys, and pins the unrestricted PR/main trigger block; quoted/escaped/alias/merge/duplicate/filter bypass tests pass. Final Codex code review approved with no findings.
|
|
|
|
- Exact-head review at `83d2ecb2` found four silent-defeat paths. Genuine RED-first tests on the pre-fix code proved: author-controlled HEAD/parent/introduction seams produced no boundary failure; cross-commit duplicate pipeline number 7 returned terminal-success; and misspelling `outputPattern` as `outputPatern` in required/actual left canonical verification green. Fixes derive the seam from Git, validate provider identity globally before subject filtering, and recursively close/type-check nested schemas. New registry criteria `RM02-EVIDENCE-SUBJECT-BINDING`, `RM02-TYPE-STRICT-SCHEMA`, and `RM02-HISTORY-BOUNDARY` are bidirectionally bound to eight registered must-fail controls. The broad nested-object typo table and exact derived-boundary test are regression guards added after implementation, not claimed as RED-first. Pre-commit Codex review then found that global evidence failure was only observed—not failed—when HEAD was the sole prospective commit, and that non-object collection entries threw before normalization. Both were reproduced RED-first, then fixed by one collection validator used before iteration and by per-commit assessment. Follow-up review found collection validation still omitted exactly-one-gate-step cardinality for unrelated subjects; a focused test reproduced terminal-success RED-first, and collection validation now rejects that ambiguity globally. Security review then found present-but-empty outcome patterns were truthy-optional assertion bypasses; a reason-specific test reproduced that they lacked the required schema diagnostic, and present pattern fields now require non-whitespace content.
|
|
|
|
- Exact-head review at `32b490a7` found three population-level blockers. Genuine RED-first tests proved a gate change before author-delayed registry introduction fell outside the range, and empty criteria/gates/prose/scenario populations returned zero. History now anchors at the provider target merge-base; delayed introduction is a registered must-fail control. Production verification requires non-empty populations and a hardcoded seven-gate ID/source inventory before quantified checks. D-38/D-40 criteria now quantify over `gateRefs` exactly spanning every registered gate; each gate declares its evidence subject, and population controls mutate evidence subject and comparison type for every gate. The history record states both bootstrap directions: sound against a branch author unable to rewrite main, not sound against compromised/rewritten main, with residual owned by Builds 1-2. Pre-commit review rejected an initial production CLI `--fixture-profile` test relaxation as a vacuity bypass. That flag was removed; synthetic fixtures now use a non-executable test-support runner, while regression tests prove the shipped CLI rejects the flag and production population checks remain mandatory. Follow-up review then proved deleting `gateRefs` skipped population validation; a RED-first loop reproduced all three deletions, and the three general criterion IDs now require the field before exact-span validation.
|
|
|
|
## Documentation checklist
|
|
|
|
- PRD, developer guide, admin guide, governing claim index, sitemap, plan, and scratchpad updated.
|
|
- User/API documentation not applicable: no user workflow or API changed.
|
|
- Independent review documentation check pending rev-974 at the revised exact head.
|
|
- Canonical documentation remains in-repository; no external publication requested.
|
|
|
|
## Rebase onto RM-61
|
|
|
|
- Rebasing `f9746b23` onto main `f4fd5967` completed mechanically with no conflicts.
|
|
- The first post-rebase `pnpm gate:verify` correctly failed because the old activation seam `f65e9ea6` made the newly merged pre-registry RM-61 commit part of prospective history even though that commit predates the registry. An initial manifest update to `f4fd5967` had the right value but retained an author-controlled mechanism. Independent mutation proved HEAD, HEAD's parent, and the introduction commit could each make history coverage vacuous or partial. The manifest field is now forbidden; the verifier derives the boundary as the parent of the first first-parent registry-introduction commit, and registered must-fail controls reject all three unsafe candidates.
|
|
|
|
## Risks/blockers
|
|
|
|
- Current queue guard intentionally has required-versus-actual deltas owned by RM-03.
|
|
- Provider CI cannot report the currently executing pipeline as terminal success; current-commit evidence must be labeled pending and becomes historical current-tree evidence only after provider completion.
|
|
- Isolated per-commit execution requires RM-60/#1031. Until that external authority exists, no replay success is claimed. A future protected post-merge failure requires quarantine/revert.
|
|
- CI containers may not expose the operator-home deployed queue guard. In that layer the verifier checks the pinned observed digest and reports live identity unavailable under RM-04; it does not infer live equality.
|