docs(remediation): bank D-17 — pre-registration confers neither correctness nor coverage

rev-974 blocked PR #1027: AC2 was pre-registered and explicitly required rejecting
symlinked generated state; the implementation accepts it (ln -s into .next, preflight
exits 0 instead of 43) while the acceptance suite ran 21/21 green throughout.

Confirmed independently: preflight.mjs:82-92 rejects symlinks on the SOURCE path,
:28 merely skips symlinked dirs, and the generated-state path :141-163 checks uid but
never isSymbolicLink(). The suite's only symlink cases cover the turbo binary and a
source file.

Sharpens D-8 rather than repeating it. D-8: pre-registration does not confer
CORRECTNESS. D-17: it does not confer COVERAGE — a suite can be green with every
criterion appearing satisfied while a criterion's actual requirement is untested.

RM-02 third clause: the registry must bind each criterion to the specific case that
exercises it and prove that case red before trusting its green. Mutation testing
pointed at the criterion-to-case mapping, not just the gate.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
mos-dt-0
2026-07-31 19:07:58 -05:00
co-authored by Claude Opus 5
parent 4512312b9f
commit ae4baf145d
+41
View File
@@ -93,6 +93,47 @@ and must not be cited as merge evidence. Rely on reviewer clearance + real CI.
Three independent live instances in a single session — format gate, agent context reset, queue guard —
is the class confirmed, not anecdote.
### D-17 — a pre-registered criterion passed a green suite without being satisfied
`rev-974` returned **CHANGES REQUESTED** on PR #1027 with one blocking finding, and it is the sharpest
instance of the session's theme because it occurred **inside our own verification machinery**.
**AC2** was pre-registered before any code was written, and explicitly required that **symlinked
generated state be rejected**. The implementation does not do it:
```sh
ln -s /etc/hosts apps/web/.next/reviewer-symlink
pnpm preflight # → "checkout preflight passed", exit 0
# → required: generated-state exit 43
```
The acceptance suite was **21/21 green** throughout. Confirmed independently rather than relayed:
`scripts/preflight.mjs:82-92` rejects symlinks on the **source** path; `:28` merely _skips_ symlinked
directories rather than rejecting them; and the **generated-state** path at `:141-163` `lstat`s and
checks `uid` (ownership) but **never** calls `isSymbolicLink()`. The suite's only symlink cases
(`preflight.test.mjs:59`, `:115`) cover the turbo binary and a _source_ file. No generated-state case
exists anywhere.
**So: criterion pre-registered, suite green, requirement unmet.** Nobody was careless — the coverage gap
is _invisible from a green_, which is the entire problem.
**This sharpens D-8 rather than repeating it.** D-8 established that pre-registration does not confer
_correctness_ (a check can be wrong when written). D-17 establishes the adjacent failure:
**pre-registration does not confer _coverage_** — a suite can be green, and every registered criterion
can appear satisfied, while a criterion's actual requirement is untested. The two together mean a
registry of checks needs **two** properties, not one: each check must be _right_, and the set must
_actually exercise_ what it claims.
**Requirement on RM-02 (third clause).** The registry must bind each acceptance criterion to the
**specific case that exercises it**, and prove that case red before trusting its green. A criterion with
no case that can fail for _that criterion's stated reason_ is unregistered in substance however it
appears in the manifest. This is mutation testing pointed at the **criterion-to-case mapping**, not
merely at the gate.
**Credit where due:** the reviewer also declined to re-run AC8, stating plainly that the PR carried it
forward with no runnable command rather than silently substituting a different boundary test. That is
the D-8 clause working a second time, in the same review that produced D-17.
### D-16 — the local test gate and the CI test gate disagree by environment
Mos flagged a shape worth chasing: if `pnpm test` exits non-zero on a _pre-existing_ guard, then either