feat(verify-release): wire quality-rails evaluator stage into canonical verification (#1275)
ci/woodpecker/pr/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
- new canonical-only quality-rails stage (after build) invokes the evaluator CLI on the repo root — QC-19 monorepo subject — instead of duplicating presence logic; no ci.yml mirror (same shape as the build stage) - parity spec updated: stage-name list, evaluator-delegation assertions, and negative controls for inline duplication / dropped command - quality-rails README points at the probe inventory (input doc)
This commit is contained in:
@@ -0,0 +1,63 @@
|
|||||||
|
# @mosaicstack/quality-rails
|
||||||
|
|
||||||
|
Quality-rails scaffolding and — since RI-3-002 (RI-N4) — the **typed
|
||||||
|
quality-rails evaluator**: the single authoritative producer of check verdicts
|
||||||
|
for the checks it owns.
|
||||||
|
|
||||||
|
## Evaluator (RI-N4)
|
||||||
|
|
||||||
|
Every verdict is typed and fail-closed:
|
||||||
|
|
||||||
|
```
|
||||||
|
{ status: 'passed' | 'failed' | 'blocked' | 'error' | 'not-applicable',
|
||||||
|
checkId, checkVersion, subject, reason }
|
||||||
|
```
|
||||||
|
|
||||||
|
Missing implementations, missing inputs, unknown check ids, process errors,
|
||||||
|
timeouts, and malformed probe output can never become `passed` or an
|
||||||
|
unqualified skip — they surface as `blocked`/`error` with a reason (vocabulary
|
||||||
|
mirrors MACP's `GateStatus` discipline).
|
||||||
|
|
||||||
|
- Check definitions live as **data** with a version and a sha256 content
|
||||||
|
digest (`definitionDigest`); every recorded verdict names the definition
|
||||||
|
version that produced it.
|
||||||
|
- Check sets are selected **per subject kind** (`node`, `python`, `rust`,
|
||||||
|
`monorepo`, `unknown`) via the versioned, digested check-set policy — this
|
||||||
|
repository (a `monorepo` subject) does not share the node template's file
|
||||||
|
list.
|
||||||
|
- Shell probes stay **thin adapters**: the TS evaluator invokes them and owns
|
||||||
|
the verdict parsing (e.g. QC-20's planted-commit probe).
|
||||||
|
|
||||||
|
### Owned checks
|
||||||
|
|
||||||
|
| check id | canonical check | mechanism |
|
||||||
|
| --------------------------- | --------------- | ---------------------------------------------------------------------- |
|
||||||
|
| `qc-19-rails-files-present` | QC-19 | typed absorption of the former presence-only `check`/`doctor` loop |
|
||||||
|
| `qc-20-enforcement-verify` | QC-20 | thin shell adapter (framework `verify.sh`); verdict parsing owned here |
|
||||||
|
|
||||||
|
The canonical check ids QC-1..QC-21 and their dispositions are defined in
|
||||||
|
`docs/release-integrity/probe-inventory.md` (the RI-3-001 inventory — the
|
||||||
|
evaluator's input, not its output).
|
||||||
|
|
||||||
|
### Usage
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# CLI (same typed report as the programmatic API)
|
||||||
|
node dist/cli.js quality-rails evaluate --project <path> [--probe-path <verify.sh>] [--json]
|
||||||
|
node dist/cli.js quality-rails check --project <path> [--json] # QC-19 only, fail-closed exit
|
||||||
|
```
|
||||||
|
|
||||||
|
```ts
|
||||||
|
import { evaluateSubject } from '@mosaicstack/quality-rails';
|
||||||
|
|
||||||
|
const report = await evaluateSubject({ subjectPath: '/path/to/project' });
|
||||||
|
// report.state: 'passed' | 'failed' | 'blocked' | 'error'
|
||||||
|
```
|
||||||
|
|
||||||
|
`pnpm verify:release` invokes this evaluator as its `quality-rails` stage
|
||||||
|
(canonical-only stage, QC-19 on the monorepo subject).
|
||||||
|
|
||||||
|
## Scaffolding (pre-existing)
|
||||||
|
|
||||||
|
`init` scaffolds rails files per detected kind/profile; `doctor` is advisory
|
||||||
|
and reports typed states.
|
||||||
@@ -22,6 +22,12 @@
|
|||||||
// format | format | pnpm format:check
|
// format | format | pnpm format:check
|
||||||
// test | test | pnpm test
|
// test | test | pnpm test
|
||||||
// build | publish.yml build | pnpm build
|
// build | publish.yml build | pnpm build
|
||||||
|
// quality-rails | (canonical-only) | the TS quality-rails evaluator
|
||||||
|
// | | (RI-N4, QC-19 monorepo subject). Like
|
||||||
|
// | | `build`, this stage has no ci.yml
|
||||||
|
// | | mirror; it is implemented by
|
||||||
|
// | | importing the evaluator CLI rather
|
||||||
|
// | | than duplicating its presence logic.
|
||||||
//
|
//
|
||||||
// Caller-provided prerequisites (kept at the pipeline level — see the comments
|
// Caller-provided prerequisites (kept at the pipeline level — see the comments
|
||||||
// in .woodpecker/ci.yml): `bash` + `rsync` for the guard stages, `openssl` and
|
// in .woodpecker/ci.yml): `bash` + `rsync` for the guard stages, `openssl` and
|
||||||
@@ -90,6 +96,15 @@ export const STAGES = [
|
|||||||
name: 'build',
|
name: 'build',
|
||||||
commands: ['pnpm build'],
|
commands: ['pnpm build'],
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
// RI-N4 (QC-19, card RI-3-002): the typed quality-rails evaluator, invoked
|
||||||
|
// as the implementation of the check it owns instead of a duplicated
|
||||||
|
// presence loop here. Canonical-only stage (no ci.yml mirror — same shape
|
||||||
|
// as `build`); runs AFTER build so the evaluator's dist/ exists. Subject
|
||||||
|
// is this repository (`.` → monorepo subject kind, per-subject check set).
|
||||||
|
name: 'quality-rails',
|
||||||
|
commands: ['node packages/quality-rails/dist/cli.js quality-rails evaluate --project .'],
|
||||||
|
},
|
||||||
];
|
];
|
||||||
|
|
||||||
export function stageByName(name) {
|
export function stageByName(name) {
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ import { createRequire } from 'node:module';
|
|||||||
import path from 'node:path';
|
import path from 'node:path';
|
||||||
import test from 'node:test';
|
import test from 'node:test';
|
||||||
|
|
||||||
import { STAGES } from './verify-release.mjs';
|
import { STAGES, stageByName } from './verify-release.mjs';
|
||||||
|
|
||||||
// SDLC-D-034 checkout invariant: publication in .woodpecker/publish.yml is
|
// SDLC-D-034 checkout invariant: publication in .woodpecker/publish.yml is
|
||||||
// bound to exact-commit terminal verification. This suite parses the real
|
// bound to exact-commit terminal verification. This suite parses the real
|
||||||
@@ -229,10 +229,21 @@ steps:
|
|||||||
function assertStagesMirrorCi(stages, ci) {
|
function assertStagesMirrorCi(stages, ci) {
|
||||||
const canonical = Object.fromEntries(stages.map((stage) => [stage.name, stage.commands]));
|
const canonical = Object.fromEntries(stages.map((stage) => [stage.name, stage.commands]));
|
||||||
|
|
||||||
// The complete mandatory set, in gate order.
|
// The complete mandatory set, in gate order. `quality-rails` is a
|
||||||
|
// canonical-only stage (RI-N4, QC-19): like `build`, it has no ci.yml
|
||||||
|
// mirror to match — its contract is asserted separately below.
|
||||||
assert.deepEqual(
|
assert.deepEqual(
|
||||||
stages.map((stage) => stage.name),
|
stages.map((stage) => stage.name),
|
||||||
['sanitization', 'upgrade-guard', 'typecheck', 'lint', 'format', 'test', 'build'],
|
[
|
||||||
|
'sanitization',
|
||||||
|
'upgrade-guard',
|
||||||
|
'typecheck',
|
||||||
|
'lint',
|
||||||
|
'format',
|
||||||
|
'test',
|
||||||
|
'build',
|
||||||
|
'quality-rails',
|
||||||
|
],
|
||||||
);
|
);
|
||||||
|
|
||||||
// Guard stages: ci.yml commands minus its `apk add` environment prep must be
|
// Guard stages: ci.yml commands minus its `apk add` environment prep must be
|
||||||
@@ -301,3 +312,38 @@ test('the root package.json exposes verify:release as the canonical command', as
|
|||||||
const packageJson = JSON.parse(await readFile(path.join(process.cwd(), 'package.json'), 'utf8'));
|
const packageJson = JSON.parse(await readFile(path.join(process.cwd(), 'package.json'), 'utf8'));
|
||||||
assert.match(packageJson.scripts['verify:release'], /scripts\/verify-release\.mjs/);
|
assert.match(packageJson.scripts['verify:release'], /scripts\/verify-release\.mjs/);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// RI-N4 (card RI-3-002): the `quality-rails` stage must route through the TS
|
||||||
|
// evaluator instead of duplicating its presence logic inline. The evaluator
|
||||||
|
// owns QC-19; this file keeps that delegation honest.
|
||||||
|
function assertEvaluatorStage(stage) {
|
||||||
|
assert.ok(stage, 'canonical stages must include a quality-rails stage');
|
||||||
|
assert.ok(Array.isArray(stage.commands) && stage.commands.length > 0);
|
||||||
|
for (const command of stage.commands) {
|
||||||
|
assert.match(
|
||||||
|
command,
|
||||||
|
/packages\/quality-rails\/dist\/cli\.js.*quality-rails evaluate/,
|
||||||
|
`quality-rails stage command must invoke the evaluator CLI, got: '${command}'`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
test('the quality-rails stage invokes the evaluator rather than duplicating its logic', () => {
|
||||||
|
assertEvaluatorStage(stageByName('quality-rails'));
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a quality-rails stage that re-implements presence logic inline fails the checker', () => {
|
||||||
|
// Negative control: replacing the evaluator invocation with an inline
|
||||||
|
// `test -f` presence loop is exactly the duplication RI-N4 forbids — the
|
||||||
|
// checker must go red on it.
|
||||||
|
const duplicated = {
|
||||||
|
name: 'quality-rails',
|
||||||
|
commands: ['test -f .husky/pre-commit && test -f .husky/pre-push'],
|
||||||
|
};
|
||||||
|
assert.throws(() => assertEvaluatorStage(duplicated), /must invoke the evaluator CLI/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a quality-rails stage that silently drops the evaluator command fails the checker', () => {
|
||||||
|
const empty = { name: 'quality-rails', commands: [] };
|
||||||
|
assert.throws(() => assertEvaluatorStage(empty), /commands/);
|
||||||
|
});
|
||||||
|
|||||||
Reference in New Issue
Block a user