guides: two measurement rules about pinned tool versions #1316

Open
fred wants to merge 2 commits from fred/code-review-pinned-tool-rules into next
@@ -65,6 +65,19 @@ Each of these produced a wrong conclusion before it was written down.
conclusion drawn from it describes the wrong tree. Confirm `git rev-parse --show-toplevel`
is the tree you think it is before trusting any git output.
13. **Run the repository's PINNED tool version.** `npx <tool>` resolves a local `node_modules`
install when one is present and fetches the latest release when one is not, so the same
command answers differently depending on where it ran. A reviewer measuring in a fresh clone
or a detached worktree — which is exactly where reviewers measure — has no `node_modules` and
silently gets the latest release instead of the pinned one. Measured on mosaicstack#1313: the
lockfile pins prettier 3.8.1, under which three guides pass; a version-less `npx` in a
worktree resolved 3.9.6, under which the same three fail; and 3.0.0, the floor of the declared
`^3.0.0` range, fails a different one. Three versions, three verdicts, identical bytes. Use
`node_modules/.bin/<tool>`, or name the version the lockfile pins.
14. **A formatter or linter declared as a range is a dated verdict, not a fact.** If a lockfile
pins it, the gate is reproducible today and will disagree with itself the day the pin moves.
Report a formatting failure with the version that produced it, always.
### Feedback Categories
- **Blocker**: must fix before merge (security, bugs, test failures)