guides: two measurement rules about pinned tool versions
ci/woodpecker/pr/ci Pipeline was canceled

Earned on #1313, where I reported three guides as failing the format gate and was
wrong. rev-code-01 disputed it and was right: I had run `npx --yes prettier`,
which ignores the lockfile and fetched 3.9.6.

Measuring it out was more useful than conceding it. Same three files:
  3.0.0  floor of the declared ^3.0.0   CI-CD-PIPELINES fails, other two pass
  3.8.1  pinned by the lockfile         all three pass, and this is CI's answer
  3.9.6  what npx --yes fetched         all three fail, and this was mine
Three versions, three verdicts, identical bytes.

Rule 13 says run the pinned tool. Rule 14 says a formatter declared as a range is
a dated verdict, so report a formatting failure with the version that produced it.

Worth recording separately: those three guides DO fail under 3.9.6, so they become
a real format-gate failure the day the pin moves past 3.8.1. Not fixed here, and
not in scope for a guides change.

Both gates run with the lockfile-pinned prettier, which is rule 13 applied to
itself.
This commit is contained in:
fred
2026-08-19 10:47:50 -05:00
parent fe4fa20309
commit 09a1d9f7cd
@@ -65,6 +65,16 @@ Each of these produced a wrong conclusion before it was written down.
conclusion drawn from it describes the wrong tree. Confirm `git rev-parse --show-toplevel`
is the tree you think it is before trusting any git output.
13. **Run the repository's PINNED tool version, not `npx --yes <tool>`.** `npx --yes` ignores the
lockfile and fetches the latest release, so it answers about a version the project does not
use. Measured on mosaicstack#1313: the lockfile pins prettier 3.8.1, under which three guides
pass; `npx --yes prettier` fetched 3.9.6, under which the same three fail; and 3.0.0, the floor
of the declared `^3.0.0` range, fails a different one. Three versions, three verdicts, same
bytes. Use `node_modules/.bin/<tool>`, or name the version the lockfile pins.
14. **A formatter or linter declared as a range is a dated verdict, not a fact.** If a lockfile
pins it, the gate is reproducible today and will disagree with itself the day the pin moves.
Report a formatting failure with the version that produced it, always.
### Feedback Categories
- **Blocker**: must fix before merge (security, bugs, test failures)