guides: two measurement rules about pinned tool versions #1316

Open
fred wants to merge 2 commits from fred/code-review-pinned-tool-rules into next
Showing only changes of commit 09a1d9f7cd - Show all commits
@@ -65,6 +65,16 @@ Each of these produced a wrong conclusion before it was written down.
conclusion drawn from it describes the wrong tree. Confirm `git rev-parse --show-toplevel`
is the tree you think it is before trusting any git output.
13. **Run the repository's PINNED tool version, not `npx --yes <tool>`.** `npx --yes` ignores the
lockfile and fetches the latest release, so it answers about a version the project does not
use. Measured on mosaicstack#1313: the lockfile pins prettier 3.8.1, under which three guides
pass; `npx --yes prettier` fetched 3.9.6, under which the same three fail; and 3.0.0, the floor
of the declared `^3.0.0` range, fails a different one. Three versions, three verdicts, same
bytes. Use `node_modules/.bin/<tool>`, or name the version the lockfile pins.
14. **A formatter or linter declared as a range is a dated verdict, not a fact.** If a lockfile
pins it, the gate is reproducible today and will disagree with itself the day the pin moves.
Report a formatting failure with the version that produced it, always.
### Feedback Categories
- **Blocker**: must fix before merge (security, bugs, test failures)