Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 jason.woltje
|
4917df1f07 |
@@ -1,186 +0,0 @@
|
|||||||
# Quality-Rails Probe Inventory — RI-3-001
|
|
||||||
|
|
||||||
- **Task:** RI-3-001 (SDLC-D-037 first half; PRD § Release Integrity Workstream, RI-N4)
|
|
||||||
- **Date:** 2026-08-18
|
|
||||||
- **Base:** `origin/next` @ `8199261c` (branch `docs/ri-050-qr-probe-inventory`)
|
|
||||||
- **Follow-up:** RI-3-002 consumes the dispositions here when building the single TS evaluator.
|
|
||||||
|
|
||||||
## 0. Scope and method
|
|
||||||
|
|
||||||
Every mechanism in this repository that verifies a quality, integrity, safety, or release
|
|
||||||
property — TypeScript checks, shell probes, pipeline steps, git hooks, and installer-side
|
|
||||||
assertions — gets one row. Each row's "what it actually verifies" was written from the
|
|
||||||
probe's **code**, not its name or docs. Framework tool unit/regression suites (git wrappers,
|
|
||||||
wake, tmux, orchestrator, …) are treated as one enforcement surface (`test:framework-shell`)
|
|
||||||
because they test tool behavior rather than repo quality; their wiring integrity is itself
|
|
||||||
guarded by `check-test-enumeration.sh`, and the quality-relevant members are rowed
|
|
||||||
individually.
|
|
||||||
|
|
||||||
**Kinds:** `ts` (TypeScript/Node check), `shell` (bash/python probe), `pipeline-step`
|
|
||||||
(exists only inside a Woodpecker pipeline).
|
|
||||||
|
|
||||||
**Enforcement points:** `local` (operator-invoked), `pre-commit`, `pre-push`,
|
|
||||||
`CI ci.yml#<step>`, `publish.yml#<step>` (CI on push to main/next), `turbo <task>`,
|
|
||||||
`agent-runtime` (framework hooks on an agent host), `installer` (host install path),
|
|
||||||
`unwired`.
|
|
||||||
|
|
||||||
**Dispositions** (recommendations for RI-3-002): `preserve` (keep as-is; already the
|
|
||||||
canonical or a correct guard-of-the-guard), `strengthen` (keep, but a concrete gap must
|
|
||||||
close — usually absorption into the TS evaluator), `strengthen (review)` (viable retirement
|
|
||||||
candidate once the evaluator absorbs it; do not retire yet). Note: RI-N4 requires that
|
|
||||||
effective shell probes be **absorbed before** their independent paths retire — no row here
|
|
||||||
is marked `retire` because no absorption exists yet.
|
|
||||||
|
|
||||||
## 1. Inventory
|
|
||||||
|
|
||||||
### 1.1 Repo-level gate tasks (pnpm / turbo)
|
|
||||||
|
|
||||||
| check | location | kind | what it actually verifies | enforcement point | canonical check | disposition | rationale |
|
|
||||||
| ------------------------------------- | ------------------------------------------------------------------------------------ | ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | ------------------------- | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| `pnpm preflight` (checkout preflight) | `scripts/preflight.mjs` | ts | Six gate binaries (eslint, husky, prettier, tsc, turbo, vitest) exist and are executable in `node_modules/.bin` (exit 42 if not); no stale `.mosaic-test-work/web-build.lock` (exit 43); `apps/web/.next` is a real directory (not a symlink), every entry owned by the current uid, and its `.mosaic-source-hash` fingerprint + `.mosaic-symlink-manifest` hash match the certified build written by `scripts/build-web.mjs` | `pre-push`; inside `pnpm typecheck` (→ `CI ci.yml#typecheck`, verify-release `typecheck` stage) | QC-1 Checkout integrity | preserve | Blocks a poisoned/stale generated `.next` from faking a green typecheck (the five-month-stale-`.next` class); trust chain is self-contained per-checkout. |
|
|
||||||
| `pnpm typecheck` | root `package.json` → `turbo run typecheck` | ts | Per-package `tsc --noEmit` (all 20 packages); turbo `typecheck` depends on `^build`, so package builds must succeed first; prefixed by checkout preflight | `CI ci.yml#typecheck`; `pre-push`; verify-release `typecheck` stage; `turbo typecheck` | QC-2 Workspace typecheck | preserve | The single workspace-wide type gate; CI and hooks invoke the same task, no divergent checklist. |
|
|
||||||
| `pnpm lint` | root `package.json` → `turbo run lint` | ts | Per-package `eslint src` under root `eslint.config.mjs` (ignores `dist`, `.next`, `framework/**`, etc.) | `CI ci.yml#lint`; `pre-push`; verify-release `lint` stage; `turbo lint` | QC-3 Workspace lint | preserve | Same-task invocation from every surface; no second lint definition. |
|
|
||||||
| `pnpm format:check` | root `package.json` → `prettier --check` | ts | Prettier parse/format equality over `**/*.{ts,tsx,js,jsx,json,md}` minus `.prettierignore` (generated trees, `docs/scratchpads/`, venvs, …) | `CI ci.yml#format`; `pre-push`; verify-release `format` stage | QC-4 Format check | preserve | Single formatter, single ignore list, enforced identically everywhere. |
|
|
||||||
| `pnpm test` | root `package.json` `test` = `test:checkout` && `turbo run test` && `test:installer` | ts | (a) `node --test scripts/*.test.mjs` — checkout-tool units; (b) per-package `vitest run` (mosaic appends the 47-command `test:framework-shell` chain); (c) `tools/install-next-lane.test.sh`; turbo `test` declares DB env vars and depends on `^build` | `CI ci.yml#test` (with `DATABASE_URL` + `db:migrate` first); verify-release `test` stage; `turbo test` | QC-5 Test suite execution | preserve | One composed test command; the chain property (any link red ⇒ step red) is the gate. |
|
|
||||||
| `pnpm build` | root `package.json` → `turbo run build` | ts | Per-package build (`tsc`/Next) with `^build` dependency and `dist/**` outputs | `publish.yml#build`; verify-release `build` stage; `turbo build` | QC-6 Workspace build | preserve | Publish artifacts derive from the same build task CI verifies. |
|
|
||||||
|
|
||||||
### 1.2 Framework quality shell probes (`packages/mosaic/framework/tools/quality/`)
|
|
||||||
|
|
||||||
| check | location | kind | what it actually verifies | enforcement point | canonical check | disposition | rationale |
|
|
||||||
| ------------------------------------------- | ----------------------------------------------------------------------- | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| Sanitization gate | `scripts/verify-sanitized.sh` | shell | Built-in self-test first (planted identity/structural/YAML+service fixtures; exit 2 if the regexes or extension coverage break), then: (1) identity denylist grep (`jarvis\|jason\|woltje\|brain.woltje.com\|/home/jwoltje\|\bPDA\b`) over all shipped text files **including** `examples/`; (2) structural grep for private `$HOME/src` defaults in shipped scripts **excluding** `examples/`. Any hit ⇒ exit 1 | `CI ci.yml#sanitization`; verify-release `sanitization` stage | QC-7 Framework sanitization | preserve | Labeled one-time regression guard with a self-test that prevents silent no-op; correctly scoped (identity vs structural) and documented as not a general PII detector. |
|
|
||||||
| Resident-context budget | `scripts/check-resident-budget.sh` (+ `--self-test`) | shell | Self-test of the comparator, then `wc -l` vs per-file ceilings (CONSTITUTION 120, AGENTS 120, each RUNTIME.md 90); missing file ⇒ fail; over ceiling ⇒ exit 1 | `CI ci.yml#sanitization` (both modes); verify-release `sanitization` stage | QC-8 Resident-context budget | preserve | Caps the container (lines), never the wording — the deliberate anti-drift design (DESIGN §7); CI-enforceable half only, by design. |
|
|
||||||
| Test-membership enumeration guard (#1017) | `scripts/check-test-enumeration.sh` + `test-enumeration-exclusions.txt` | shell | Parses surface S1 (`packages/mosaic` `test:framework-shell` via JSON+shlex) and S2 (every `framework/tools/\*.sh | .py`token in`ci.yml`, comment lines stripped); population = `_test_.sh`under`framework/tools`; FAILS on: suite-shaped file on disk neither enumerated nor signed-excluded; surface naming a path missing on disk (both directions); exclusion without reason / stale / outside population / contradicting enumeration. Proves **naming, not reachability** (stated in-file) | `CI ci.yml#sanitization` (direct line); link [0] of `test:framework-shell` (thus `CI ci.yml#test`); verify-release `sanitization` stage | QC-9 Test-membership enumeration | preserve | Makes silent under-run impossible; invoked from both surfaces it audits so severing the chain cannot silence it. |
|
|
||||||
| Enumeration-guard needles | `scripts/test-check-test-enumeration.sh` | shell | Needle/control fixtures driven through `--root`: every promised failure mode must trip the guard **on its own words**, plus controls that must pass (null-case defense); covers commented-out ci.yml lines (F1) and line-range parsing (n2b) | `test:framework-shell` → `CI ci.yml#test`; verify-release `test` stage | QC-9 Test-membership enumeration | preserve | Guard-of-the-guard with both polarities; same canonical check by design. |
|
|
||||||
| Upgrade manifest guard (#791 HARD GATE) | `scripts/test-upgrade-manifest-guard.sh` | shell | Keep-mode `install.sh` upgrade against seeded throwaway `MOSAIC_HOME`: every operator sentinel — including an **unanticipated** one — survives byte-identical with unchanged mtime; framework files still update; retired framework files pruned; matrix run with rsync present AND absent (keep path must be rsync-independent); fail-closed matrix (empty/operator-only/malformed/missing manifest aborts loudly, operator files untouched); operator secret never appears in installer output | `CI ci.yml#upgrade-guard`; verify-release `upgrade-guard` stage | QC-10 Upgrade/install safety | preserve | The operator-data hard gate for the `mosaic update` path; negative controls are load-bearing and documented. |
|
|
||||||
| Upgrade rollback gate (#791 B1) | `scripts/test-upgrade-rollback.sh` | shell | Mid-sync failure (PATH-shadowing `cp` shim) must trigger snapshot restore: restore message fires, corrupted file restored, target byte-identical to pre-upgrade; control installer with `set -E` stripped must NOT roll back (proves errtrace is load-bearing); plus signal/exit-guard controls | `CI ci.yml#upgrade-guard`; verify-release `upgrade-guard` stage | QC-10 Upgrade/install safety | preserve | Proves the rollback trap actually fires; the `-E`-stripped control keeps Part A honest. |
|
|
||||||
| Durable-snapshot gate (#791 PR2) | `scripts/test-upgrade-durable-snapshot.sh` | shell | Pre-update snapshot taken before any mutation (0700/0600 perms, secret never logged, retention-pruned); post-sync verify net restores operator files a manifest bug lets the sync touch; CWE-59 symlink-leaf guard proven with a portable cp shim in both polarities (write-through-link must not happen); v1→v2 migration semantics (intended `bin/` removal not healed) | `CI ci.yml#upgrade-guard`; verify-release `upgrade-guard` stage | QC-10 Upgrade/install safety | preserve | Covers tampering and leak vectors the manifest guard cannot see; the shim rationale (busybox vs GNU cp) is documented in-file. |
|
|
||||||
| Install migration matrix (v2→v3) | `scripts/test-install-migration.sh` | shell | Fixture matrix running the real installer with `MOSAIC_SYNC_ONLY=1`: fresh install seeds + stamps version 3; legacy user-edited AGENTS overwritten with `.pre-constitution.bak` preserved (and idempotent); tuned STANDARDS overwritten; operator files (SOUL, credentials) preserved. Mirrors the TS suite `packages/mosaic/src/config/file-adapter.test.ts` — both installers must behave identically | `CI ci.yml#upgrade-guard`; verify-release `upgrade-guard` stage | QC-10 Upgrade/install safety | preserve | Pins the shell/TS installer parity contract; removal would orphan that parity requirement. |
|
|
||||||
| Enforcement verification probe (bash) | `scripts/verify.sh` | shell | Attempts **real commits** in the target repo: planted type error must produce a commit blocked with `error`; planted `any` must trip `no-explicit-any`; planted lint error must trip `prettier`; gitleaks binary must exist (3a) and detect a planted AWS key via `gitleaks git --pre-commit --staged --redact` (3b). Verdicts are output-grep matches on hook stderr | `local` via installed `mosaic-quality-verify` on scaffolded target projects; **not run in this repo's CI** | QC-20 Downstream enforcement verification | strengthen (review) | Mechanism is genuinely behavioral (stronger than file presence) but verdict logic is grep-on-output and it is unwired here; absorb as the evaluator's enforcement-probe check (the RI-N4 evaluator invokes it or reimplements it) before retiring the shell path. |
|
|
||||||
| Enforcement verification probe (PowerShell) | `scripts/verify.ps1` | shell | Windows port of `verify.sh`: same planted-commit tests with `$output -match` matching; no gitleaks self-test parity beyond the same checks | `local` (Windows operator); no Windows CI runner exists | QC-20 Downstream enforcement verification | strengthen (review) | A hand-maintained twin of `verify.sh` with no CI coverage — exactly the drift shape the single evaluator removes; retire after the TS evaluator owns the probe. |
|
|
||||||
| Quality template installer (bash) | `scripts/install.sh` | shell | Copies template files (`.husky/pre-commit` incl. mandatory gitleaks, `.lintstagedrc.js`, `.eslintrc.js`, `tsconfig.json`, `.woodpecker.yml`, `.gitleaks.toml`) into a target project; **warns** (does not verify) about `package.json` snippet merge; no post-condition check | `local` / via `mosaic-quality-apply` | QC-21 Downstream rails scaffolding | strengthen (review) | Duplicates the TS `quality-rails init` scaffolder for a different template set; converging on one scaffolder (with post-scaffold verification) is prerequisite to retiring this path. |
|
|
||||||
| Quality template installer (PowerShell) | `scripts/install.ps1` | shell | Windows twin of the template copy above | `local` (Windows operator) | QC-21 Downstream rails scaffolding | strengthen (review) | Same twin-drift risk as `verify.ps1`; no runner exercises it. |
|
|
||||||
| `mosaic-quality-verify` adapter | `framework/tools/_scripts/mosaic-quality-verify` | shell | Thin adapter: validates target dir exists, asserts `verify.sh` present+executable, `cd` target, exec it. No verdict logic of its own | `local` (installed framework bin) | QC-20 Downstream enforcement verification | preserve | Already the thin-adapter shape RI-N4 prescribes for shell surfaces. |
|
|
||||||
| `mosaic-quality-apply` adapter | `framework/tools/_scripts/mosaic-quality-apply` | shell | Thin adapter: arg validation then exec of quality `install.sh --template … --target …` | `local` (installed framework bin) | QC-21 Downstream rails scaffolding | preserve | Thin adapter, no separate verdict; disposition follows its target script's convergence. |
|
|
||||||
| Roster schema regression | `scripts/test-roster-schema.py` | shell | jsonschema `Draft202012Validator` over `fleet/roster.schema.json` with valid/invalid connector-kind fixtures (tmux/discord/matrix conditional fields) | **unwired** — not on S1 or S2, not signed-excluded; also outside the enumeration guard's `*.sh` population, so the guard cannot see it | QC-5 Test suite execution | strengthen (review) | A real regression suite that currently runs nowhere; wire it into a CI surface or sign an exclusion — leaving it invisible re-arms the exact gap #1017 closed. |
|
|
||||||
| Framework shell chain (S1) | `packages/mosaic/package.json` `test:framework-shell` | shell | 47-command `&&` chain: enumeration guard + needles, 14 lease-broker/mutator-gate python unitests, `check-runtime-launches.py`, and ~30 framework-tool shell suites (git wrappers, wake, woodpecker, tmux, glpi, orchestrator, `_scripts`). Quality-relevant members rowed separately below | `turbo test` → `CI ci.yml#test`; verify-release `test` stage | QC-5 Test suite execution | preserve | The chain is the execution surface the enumeration guard audits; known residuals: a failing link stops later suites (measured in #1270 — suites after position 44 had not run), and the guard proves naming, not reachability. |
|
|
||||||
|
|
||||||
### 1.3 Framework runtime hooks and their harnesses (agent-host enforcement)
|
|
||||||
|
|
||||||
| check | location | kind | what it actually verifies | enforcement point | canonical check | disposition | rationale |
|
|
||||||
| ------------------------------------- | ----------------------------------------------------------------------------------------- | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------- | --------------------------------------- | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| QA edit hook seam | `framework/tools/qa/qa-hook-stdin.sh` (+ `qa-hook-handler.sh`) | shell | PostToolUse stdin hook: extracts edited file from the tool JSON (jq or grep fallback), skips non-JS/TS, then the deps-preflight gate — exits 1 with the legible sentinel `deps not installed — run pnpm install` when `node_modules/.bin` is missing/empty (the #856 false-red class); the downstream handler only files QA remediation **report templates** (no verification logic) | `agent-runtime` (framework `runtime/claude/settings.json` PostToolUse); never CI | QC-16 Agent-runtime edit-time checks | strengthen (review) | The sentinel gate is real enforcement; the handler's report-filing adds no verdict and its name promises more than the code does — evaluator absorption should keep the sentinel, drop the report theater. |
|
|
||||||
| Typecheck-on-edit hook | `framework/tools/qa/typecheck-hook.sh` | shell | PostToolUse: for edited `.ts/.tsx`, finds nearest `tsconfig.json` and runs `tsc --noEmit`, surfacing errors nonzero to the agent immediately | `agent-runtime` (framework `runtime/claude/settings.json` PostToolUse) | QC-16 Agent-runtime edit-time checks | strengthen (review) | Edit-time duplicate of QC-2 with independent invocation logic; keep behavior, converge invocation through the evaluator adapter. |
|
|
||||||
| Deps-preflight harness | `framework/tools/qa/test-deps-preflight.sh` | shell | Five assertions against the seam incl. a documented RED control (raw `not found`), sentinel behavior for missing and empty `.bin`, and no-false-positive once populated | `test:framework-shell` → `CI ci.yml#test` | QC-16 Agent-runtime edit-time checks | preserve | Guard-of-the-check with a red control; keeps the sentinel from regressing. |
|
|
||||||
| Prompt-helper RCE regression | `framework/tools/_scripts/test-mosaic-init-rce.sh` | shell | Sources the prompt helpers and proves a literal `$(touch /tmp/pwned)` answer round-trips verbatim and never executes (no `/tmp/pwned` created) | `test:framework-shell` → `CI ci.yml#test` | QC-5 Test suite execution | preserve | Cheap, load-bearing security regression on the installer's input path. |
|
|
||||||
| Install-ordering harness (#869 C2) | `framework/tools/_scripts/test-install-ordering-guard.sh` | shell | Drives `mosaic-link-runtime-assets` with a fake `mosaic` on PATH: probe ok ⇒ settings copied + exit 0; probe fail ⇒ exit 1 with degraded outcome but all other runtime files still copied; `--allow-inactive-enforcement` forwarded; no-mosaic-on-PATH ⇒ python3 fallback strips enforcement hooks and exits 1; fallback + flag ⇒ wires as-is, exit 0 | `test:framework-shell` → `CI ci.yml#test` | QC-17 Lease-enforcement wiring safety | preserve | Exercises the shell wiring seam independently of the TS guard's own spec suite (complementary coverage, by design). |
|
|
||||||
| Fleet-transport harness (#1240) | `framework/tools/_scripts/test-fleet-transport-check.sh` | shell | Extracts the shipped `check_fleet_transport`/`fleet_declared_transport` functions **from the shipped scripts** (fails loud if extraction yields nothing) and drives both implementations (mosaic-doctor + `tools/install.sh`) from one case table | `test:framework-shell` → `CI ci.yml#test` | QC-18 Operator-host drift audit | preserve | The anti-drift harness for the one rule shipped twice; extraction-from-source keeps it from testing a stale copy. |
|
|
||||||
| Terminal-green contract (RM-61/#1000) | `framework/tools/woodpecker/test-terminal-green-contract.sh` + `verify-terminal-green.py` | shell | Red-first fixtures: pipeline JSON variants (service failure, step failure, cancelled, etc.) must produce the correct terminal-green verdict; controls must pass | `test:framework-shell` → `CI ci.yml#test` | QC-5 Test suite execution | preserve | Keeps the CI-wait wrapper's green-detection honest; a false green here would poison every merge gate that trusts `pr-ci-wait.sh`. |
|
|
||||||
| Lease-gate launch invariant | `framework/tools/lease-broker/check-runtime-launches.py` | shell | Scans production roots (`packages/`, `apps/`, `plugins/`, `tools/`) across sh/py/ts/yaml suffixes for Claude/Pi process launches **outside** the lease gate; allowlist-based; fails CI on violation | `test:framework-shell` → `CI ci.yml#test` | QC-15 Lease-gate architecture invariant | preserve | The only architectural "no ungated launches" rail; grep+allowlist is the right cost/benefit for this invariant. |
|
|
||||||
|
|
||||||
### 1.4 TypeScript quality logic (`@mosaicstack/quality-rails` + mosaic CLI)
|
|
||||||
|
|
||||||
| check | location | kind | what it actually verifies | enforcement point | canonical check | disposition | rationale |
|
|
||||||
| ---------------------------------------- | ---------------------------------------------------------------------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------- | ------------------------------------- | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| `quality-rails check` | `packages/quality-rails/src/cli.ts` (`mosaic quality-rails check --project`) | ts | **Expected-file presence only**: loops `expectedFilesForKind` (node: `.eslintrc`, `biome.json`, `.githooks/pre-commit`, `PR-CHECKLIST.md`; python: `pyproject.toml`+hooks+checklist; rust: `rustfmt.toml`+…) and exits 1 listing missing paths. Does not execute any linter, formatter, hook, or scanner | `local` (operator CLI); **no CI wiring in this repo** | QC-19 Downstream rails presence check | strengthen | This is the RI-N4 evaluator seed. Today presence ≠ parity (explicitly called out by RI-N4): it must grow typed verdicts (`passed/failed/blocked/error/not-applicable`), check versioning/subject/reason, digested definitions, and absorb the effective shell probes (QC-20 first). |
|
|
||||||
| `quality-rails doctor` | `packages/quality-rails/src/cli.ts` | ts | Same presence data as `check`, printed with ok/missing lines; **cannot fail** (no nonzero exit on missing files) | `local` (operator CLI) | QC-19 Downstream rails presence check | strengthen | A doctor that cannot fail is advisory; fold into `check` (or return typed states) when the evaluator lands. |
|
|
||||||
| `quality-rails init` | `packages/quality-rails/src/cli.ts` + `scaffolder.ts`/`templates.ts` | ts | Scaffolds rails files per detected kind/profile (linters/formatters lists are advisory strings; hooks flag always true); writes files, prints follow-ups — no post-condition verification | `local` (operator CLI) | QC-21 Downstream rails scaffolding | strengthen (review) | Second scaffolding path alongside quality `install.sh` (§1.2); converge on one with post-scaffold verification before retiring either. |
|
|
||||||
| Lease activation probe (#869 C1, hidden) | `packages/mosaic/src/commands/lease-activation-probe.ts` | ts | Real capability probe, not file presence: resolves the installed mosaic CLI and requires it to advertise the exact `{name, version}` activation contract; all deps injectable; registered as hidden CLI command and consumed by C2/C5 | `local` (hidden CLI + consumed by C2/C5); spec-tested via `lease-activation-probe.spec.ts` in `turbo test` | QC-17 Lease-enforcement wiring safety | preserve | The versioned-contract probe is precisely the fail-closed capability check RI-N2 generalizes; already typed and injectable. |
|
|
||||||
| Install-ordering guard (#869 C2, hidden) | `packages/mosaic/src/commands/install-ordering-guard.ts` | ts | Decides whether enforcement hook entries are written into the `~/.claude/settings.json` the framework reseed ships: not activatable ⇒ strip hooks + nonzero loud outcome (default); explicit per-invocation `--allow-inactive-enforcement` opt-out wires-with-warning. Never touches the runtime gate's own fail-closed behavior | `installer` (framework reseed via `mosaic-link-runtime-assets`); spec + shell harness coverage in `turbo test` | QC-17 Lease-enforcement wiring safety | preserve | Correct default-deny with an explicit, non-env opt-out; test-locked from both the TS and shell sides. |
|
|
||||||
| Lease doctor check (#869 C5) | `packages/mosaic/src/commands/lease-doctor-check.ts` | ts | Combines hook-wiring detection in `~/.claude/settings.json` with C1 activatable and C3 broker-supervisor health: wired ∧ (¬activatable ∨ ¬healthy) ⇒ loud `[ERROR]` that forces `mosaic doctor` exit 1 regardless of the bash audit's own exit | `local` (inside `mosaic doctor`); spec coverage in `turbo test` | QC-17 Lease-enforcement wiring safety | preserve | Closes the "bricked host looks green" hole; cannot be masked by the bash script — that composition is the point. |
|
|
||||||
| `mosaic doctor` (framework drift audit) | `packages/mosaic/src/commands/launch.ts` (`doctor`) + `framework/tools/_scripts/mosaic-doctor` | shell+ts | Bash audit of the installed framework home: ~40 expected files/dirs present; runtime files are copies (not symlinks) matching source (`cmp`) or composed runtime-contract markers; hard-gates block present in AGENTS.md; sequential-thinking MCP configured; fleet transport binary present per roster (warn); legacy symlink trees gone; skills synced — **warn-based, exit 1 only with `--fail-on-warn`**, plus C5's forced error | `local` (operator audit) | QC-18 Operator-host drift audit | preserve | Host-state audit CI cannot see (user files by design, DESIGN §7); advisory exit is the documented contract — do not silently change it. |
|
|
||||||
| `mosaic gateway doctor` | `packages/mosaic/src/commands/gateway-doctor.ts` | ts | Probes per-service health (PostgreSQL, Valkey, pgvector) via `@mosaicstack/storage`, reports tier and JSON; exit 1 only when at least one **required** service fails (yellow stays 0) | `local` (operator) | QC-18 Operator-host drift audit | preserve | Service health with correct red/yellow exit semantics; JSON mode exists for scripting. |
|
|
||||||
| `mosaic gateway verify` | `packages/mosaic/src/commands/gateway/verify.ts` | ts | Post-install liveness: daemon meta via HTTP with retries, admin token on file, bootstrap endpoint reachable; aggregated pass/fail | `local`; consumed by `tools/e2e-install-test.sh` | QC-18 Operator-host drift audit | preserve | The first-run proof the installer E2E relies on; retry-aware so startup races don't false-red. |
|
|
||||||
| `mosaic fleet doctor` | `packages/mosaic/src/commands/fleet-reconciler-command.ts` | ts | Classifies local roster-owned drift (no mutation) from the parsed v2 roster | `local` (operator) | QC-18 Operator-host drift audit | preserve | Dry-run classification is the correct non-mutating audit shape. |
|
|
||||||
|
|
||||||
### 1.5 Git hooks (developer machine)
|
|
||||||
|
|
||||||
| check | location | kind | what it actually verifies | enforcement point | canonical check | disposition | rationale |
|
|
||||||
| ------------------------- | --------------------------------------------------------- | ----- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | --------------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| Pre-commit staged hygiene | `.husky/pre-commit` → `npx lint-staged` (`.lintstagedrc`) | shell | On staged files only: `prettier --write` + `eslint --fix` for ts/tsx/js/jsx; `prettier --write` for json/md/yaml/yml. **Mutating** (fixes and re-stages); commit blocks only if a fixer itself fails | `pre-commit` (every local commit; hooks activated by `install-hooks.mjs` via `core.hooksPath .husky/_`) | QC-13 Staged-change hygiene | preserve | Correct scoped fast gate; note it auto-fixes rather than rejects (deliberate). Gap: no secret scan here — see §3. |
|
|
||||||
| Pre-push gate | `.husky/pre-push` | shell | `pnpm preflight && pnpm typecheck && pnpm lint && pnpm format:check` (no test run — documented in AGENTS.md) | `pre-push` | QC-14 Pre-push gate | preserve | Composes QC-1..4 exactly as specified in AGENTS.md; tests intentionally left to CI. |
|
|
||||||
| Hook installer | `scripts/install-hooks.mjs` (`pnpm prepare`) | ts | Stages husky hooks into a scratch repo first, asserts husky produced its `h` shim, quarantines incomplete previous sets, verifies idempotence via full directory snapshot comparison, then sets `core.hooksPath`; skips cleanly with `HUSKY=0` or no git | `installer` (runs on `pnpm install`) | QC-13 Staged-change hygiene | preserve | Self-verifying wiring for the hook gates — a corrupted half-install cannot silently disable them. |
|
|
||||||
|
|
||||||
### 1.6 CI pipeline steps (`.woodpecker/`)
|
|
||||||
|
|
||||||
Step-to-probe mapping for container steps: `ci.yml#sanitization` = QC-7+QC-8+QC-9 (rows §1.2, plus `apk add bash` env prep); `ci.yml#upgrade-guard` = QC-10 (rows §1.2, plus `apk add rsync`); `ci.yml#typecheck`/`#lint`/`#format`/`#test` = QC-2/3/4/5 (rows §1.1). Rows below are mechanisms that exist only in a pipeline.
|
|
||||||
|
|
||||||
| check | location | kind | what it actually verifies | enforcement point | canonical check | disposition | rationale |
|
|
||||||
| -------------------------------------- | -------------------------------------------------------------------------------------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------- | ----------------------------------- | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| Frozen install | `ci.yml#install` | pipeline-step | `pnpm install --frozen-lockfile --prefer-offline` against the baked ci-base store — lockfile supply integrity; a drifted lockfile fails the build before any gate runs | `CI ci.yml#install` | QC-1 Checkout integrity | preserve | Lockfile-pinned dep resolution is the supply-chain floor under every later gate. |
|
|
||||||
| Test-step readiness prelude | `ci.yml#test` prologue | pipeline-step | Installs pinned `@earendil-works/[email protected]` (Invariant R suite requires the real binary) + openssl; waits up to 60×1s on `pg_isready` for the `ci-postgres` service and fails fast if it never comes up; runs `db:migrate` before tests | `CI ci.yml#test` | QC-5 Test suite execution | preserve | Fail-fast environment preconditions — a missing service produces a legible failure, not a wall of red tests. |
|
|
||||||
| Publish verify step (pending RI-1-001) | `publish.yml#verify` (branch `feat/ri-050-publish-gate` @ `46784c8d`, not yet on next) | pipeline-step | (a) Commit identity: fails closed if `CI_COMMIT_SHA` empty, `git rev-parse HEAD` empty, or the two differ; (b) runs the canonical `pnpm verify:release`. **Every publish effect depends on this step; it carries no path filter** | `publish.yml#verify` | QC-11 Terminal release verification | preserve | The RI-N1 exact-commit binding; until it merges, publish steps on next depend on `build` only (see §3 gap 1). |
|
|
||||||
| Publish error classification | `publish.yml#publish-npm` | pipeline-step | Publishes `@mosaicstack/*` (minus web) and classifies outcome: success, or the **only tolerated failure** = already-published (EPUBLISHCONFLICT / "cannot publish over" / "previously published"); explicit fatal on npm `E404/E401/ENEEDAUTH/ECONNREFUSED/ETIMEDOUT/ENOTFOUND` and on any unrecognized failure (replacing the old ` | | echo` that hid a registry 404) | `publish.yml#publish-npm` (main/tags, path-filtered on `packages/**`) | QC-12 Publish-effect integrity | preserve | Converts silent publish fall-on-floor into loud failure; allowlist-of-one error tolerance is the right shape. |
|
|
||||||
| Next-lane publish assertions | `publish.yml#publish-next-npm` | pipeline-step | Guards: branch must be `next`, `CI_PIPELINE_NUMBER` required; registry dist-tags JSON must be usable; walks all manifests, strictly parses stable semver, rewrites `X.Y.(Z+1)-next.<N>`; publishes with `--tag next` (never latest); post-publish asserts `npm view @mosaicstack/mosaic@next` resolves to the exact expected version | `publish.yml#publish-next-npm` (push/manual on next) | QC-12 Publish-effect integrity | preserve | Durable prerelease lane with end-to-end resolution proof — the published artifact is verified, not assumed. |
|
|
||||||
| Image destination policy | `publish.yml#build-gateway` / `#build-appservice` / `#build-web` | pipeline-step | Kaniko builds with destination policy: `next` ⇒ sha-tag only (fatal if a tag event sneaks in); `main` ⇒ sha + `latest`; tag events ⇒ sha + `<tag>`; anything else fatal. Path filters only skip **effects**, never the verify step | `publish.yml#build-*` | QC-12 Publish-effect integrity | preserve | Fail-closed tagging matrix; the exclude-list default-safe design keeps stale images impossible. |
|
|
||||||
|
|
||||||
Adjacent pipeline surface (not a probe): `.woodpecker/ci-image.yml` rebuilds the ci-base image on `pnpm-lock.yaml`/`Dockerfile.ci` change with an immutable `lock-<hash>` tag; pipelines consume `:latest`. Recorded for completeness — no code-quality property is checked.
|
|
||||||
|
|
||||||
### 1.7 Root installer tooling (`tools/`)
|
|
||||||
|
|
||||||
| check | location | kind | what it actually verifies | enforcement point | canonical check | disposition | rationale |
|
|
||||||
| --------------------------- | --------------------------------------------------------- | ----- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------- | ------------------------------- | ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| Next-lane installer test | `tools/install-next-lane.test.sh` (`pnpm test:installer`) | shell | Drives `tools/install.sh --next` with faked `node`/`npm` binaries (no network): Node 20 must be rejected; installs must pin **exact** versions (mutable `@next` forbidden); fast path must not unexpectedly fall back to source; gateway-install failure takes the documented fallback | `turbo`-external tail of `pnpm test` → `CI ci.yml#test` | QC-5 Test suite execution | preserve | Hermetic (shimmed) regression net for the installer lane; runs as part of the standard test command. |
|
|
||||||
| Clean-container install E2E | `tools/e2e-install-test.sh` | shell | Full first-run flow in a node:22-alpine container: `install.sh --yes` → `mosaic wizard` (non-interactive) → `mosaic gateway install` → `mosaic gateway verify` exit check (with EXPECTED-SKIP if the installed CLI predates `gateway verify`); skips gracefully without Docker | `local` (manual; requires Docker); **not wired in CI** | QC-5 Test suite execution | strengthen (review) | The only end-to-end proof of the install→verify path; currently operator-initiated only — wire into a periodic/manual CI lane or sign its exclusion explicitly. |
|
|
||||||
| Host installer advisories | `tools/install.sh` (`--check`; `check_fleet_transport`) | shell | `--check` = version comparison only, no install; `check_fleet_transport` warns (non-blocking, by design — tmux is the fleet's dependency, not mosaic's) when the roster-declared transport binary is absent, naming exactly what it blocks; PATH-persistence warnings | `installer` (operator-run) | QC-18 Operator-host drift audit | preserve | Advisory-by-design warnings; the parallel doctor check is drift-tested by §1.3's harness. |
|
|
||||||
|
|
||||||
### 1.8 Pending workstream additions (branch `feat/ri-050-publish-gate` @ `46784c8d`)
|
|
||||||
|
|
||||||
| check | location | kind | what it actually verifies | enforcement point | canonical check | disposition | rationale |
|
|
||||||
| ------------------------------- | ---------------------------------------------------- | ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | ----------------------------------- | ----------- | ---------------------------------------------------------------------------------------------- |
|
|
||||||
| Canonical terminal verification | `scripts/verify-release.mjs` (`pnpm verify:release`) | ts | One command replaying the full mandatory set as stages — sanitization, upgrade-guard, typecheck (incl. preflight), lint, format, test, build — mirroring `ci.yml` step-for-step; fail-fast on first failing command; requires `bash`+`rsync` on PATH; `--stage <name>` for wiring smoke-tests only | `publish.yml#verify` (pending); `local` (`pnpm verify:release`) | QC-11 Terminal release verification | preserve | The RI-N1 canonical command — CI and publication share one semantic checklist by construction. |
|
|
||||||
| Verify-parity contract test | `scripts/verify-release.test.mjs` | ts | Parses the real `ci.yml`/`publish.yml`: stage table must match ci.yml step-for-step; every publish-effect step (name `publish*` or image-pushing) must transitively depend on `verify`; commit-identity assertion must be present; `verify` must carry no path filter | `test:checkout` → `CI ci.yml#test` (once merged) | QC-11 Terminal release verification | preserve | Guard-of-the-guard at checkout time — the two surfaces cannot drift apart silently. |
|
|
||||||
|
|
||||||
## 2. Canonical check set
|
|
||||||
|
|
||||||
The deduplicated checks every row above maps onto. IDs are stable for RI-3-002 to consume.
|
|
||||||
|
|
||||||
- **QC-1 Checkout integrity.** Owns: the checkout can run its gates — frozen-lockfile dependency resolution, required gate binaries present, no stale build lock, and the `apps/web/.next` generated-state trust chain (real directory, uid ownership, certified source fingerprint, certified symlink manifest). Implemented by `scripts/preflight.mjs` + frozen install steps.
|
|
||||||
- **QC-2 Workspace typecheck.** Owns workspace-wide TypeScript soundness: per-package `tsc --noEmit` over built dependencies (`turbo typecheck`). The single definition invoked by CI, pre-push, and terminal verification.
|
|
||||||
- **QC-3 Workspace lint.** Owns static-analysis policy: per-package ESLint under the root config. One config, one task, every surface.
|
|
||||||
- **QC-4 Format check.** Owns formatting uniformity: Prettier check with the repo ignore list. (The pre-commit variant additionally fixes; the verdict form is this check.)
|
|
||||||
- **QC-5 Test suite execution.** Owns execution of all test surfaces: checkout script units (`node --test`), per-package Vitest suites (including the framework shell chain and its python unitests), the installer-lane shim test, and — once wired — `test-roster-schema.py` and container E2E. Also owns guards-of-the-gate that live inside the chain (terminal-green contract, RCE regression).
|
|
||||||
- **QC-6 Workspace build.** Owns artifact buildability: `turbo build` producing the artifacts publication consumes.
|
|
||||||
- **QC-7 Framework sanitization.** Owns the open-source guarantee for the shipped framework package: no operator-identity tokens anywhere (examples included), no private `$HOME` defaults in shipped scripts, with a self-test that keeps the regexes honest.
|
|
||||||
- **QC-8 Resident-context budget.** Owns the line-count ceilings on framework files injected into every agent's context (Constitution, dispatcher, RUNTIME.md slices) — the CI-enforceable half of the resident-prompt budget.
|
|
||||||
- **QC-9 Test-membership enumeration.** Owns the property that no test suite can silently fall out of CI: disk population vs parsed enumeration surfaces, both-directions staleness, and signed exclusions with reasons. Includes its needle/control harness.
|
|
||||||
- **QC-10 Upgrade/install safety.** Owns the #791 family: operator-path byte-identity across keep-mode upgrades (manifest guard), mid-failure rollback (errtrace-proven), durable pre-update snapshot + verify net + CWE-59 leaf guard, and the v2→v3 migration matrix with shell/TS parity.
|
|
||||||
- **QC-11 Terminal release verification.** Owns the RI-N1 exact-commit binding: commit-identity assertion plus one canonical command (`pnpm verify:release`) replaying the complete mandatory set, with every publish effect depending on it; plus the checkout-time parity/DAG contract test that keeps pipeline and command in sync.
|
|
||||||
- **QC-12 Publish-effect integrity.** Owns publication correctness: npm publish error classification (only already-published tolerated), next-lane versioning and post-publish resolution proof, and image destination/tag policy.
|
|
||||||
- **QC-13 Staged-change hygiene.** Owns commit-time hygiene on staged files (prettier/eslint fix-and-restage) and the self-verifying hook wiring that guarantees the gates are actually installed.
|
|
||||||
- **QC-14 Pre-push gate.** Owns the local push composition: preflight + typecheck + lint + format:check (tests deliberately deferred to CI).
|
|
||||||
- **QC-15 Lease-gate architecture invariant.** Owns "no ungated runtime launches in production code": the scan + allowlist over `packages/`, `apps/`, `plugins/`, `tools/`.
|
|
||||||
- **QC-16 Agent-runtime edit-time checks.** Owns edit-time feedback on agent hosts: the deps-preflight legibility sentinel and typecheck-on-edit, plus their regression harnesses.
|
|
||||||
- **QC-17 Lease-enforcement wiring safety.** Owns the #869 C1/C2/C5 trio: activation capability probe (versioned contract), enforcement-hook wiring gate (default-deny with explicit opt-out), and the doctor check that surfaces a bricked host — with their shell/TS harnesses.
|
|
||||||
- **QC-18 Operator-host drift audit.** Owns host-state health CI cannot see: `mosaic doctor` drift audit (+ fleet transport, both implementations), `fleet doctor` roster classification, `gateway doctor`/`gateway verify` service health, and installer advisories. Advisory exits are part of the contract.
|
|
||||||
- **QC-19 Downstream rails presence check.** Owns "does a scaffolded project still carry its rails files" — today the TS `quality-rails check/doctor` presence loop; per RI-N4 this is the seed that must become the typed evaluator (presence alone is explicitly not parity).
|
|
||||||
- **QC-20 Downstream enforcement verification.** Owns "do the rails actually block" on scaffolded projects: the behavioral planted-commit probe (type error, `any`, lint, gitleaks secret) currently in `verify.sh`/`verify.ps1` behind the `mosaic-quality-verify` adapter.
|
|
||||||
- **QC-21 Downstream rails scaffolding.** Owns putting rails files into a target project: the shell template installer (+ PowerShell twin) and the TS `quality-rails init` scaffolder — currently two paths that must converge.
|
|
||||||
|
|
||||||
## 3. Coverage gaps
|
|
||||||
|
|
||||||
Enforced nowhere but implied, or named in docs/tooling but not wired:
|
|
||||||
|
|
||||||
1. **Publication not yet bound to verification on `next`.** At this base (`8199261c`), `publish.yml` publish steps depend on `build` only; the `verify` step and `scripts/verify-release.mjs` exist on `feat/ri-050-publish-gate` (`46784c8d`) but are not merged. Until RI-1-001 lands, AC-RI-1's negative control cannot hold on the real pipeline.
|
|
||||||
2. **Playwright E2E unwired.** `apps/web` ships `test:e2e` (`playwright test`) with real suites (`admin/auth/chat/navigation.spec.ts`); neither `pnpm test` nor any CI step invokes it. The web UI's user flows are verified only when an operator runs them manually.
|
|
||||||
3. **No secret scanning on this repo.** The framework's own template pre-commit makes gitleaks **required**, and `verify.sh` proves detection with a planted key — but this repository's `.husky/pre-commit` (lint-staged only) and CI run no secret scan. The repo ships the control it does not use.
|
|
||||||
4. **No dependency audit.** The quality `.woodpecker.yml` templates and `docs/CI-SETUP.md` specify `npm audit --audit-level=high` as a pipeline stage; nothing equivalent runs for this repo.
|
|
||||||
5. **No coverage thresholds.** Templates enforce 80% Jest coverage thresholds; this repo's Vitest configs collect coverage with no thresholds — coverage is measured nowhere and enforced nowhere.
|
|
||||||
6. **`test-roster-schema.py` invisible.** A real jsonschema regression suite wired to no surface and invisible to the enumeration guard (its population is `*.sh`; the suite is `.py`). Either enumerate it or sign an exclusion — silence here is the #1017 defect shape.
|
|
||||||
7. **Presence-checker expectations ≠ this repo.** `quality-rails check` expects `.eslintrc`, `biome.json`, `.githooks/pre-commit`, `PR-CHECKLIST.md` for node projects — none describe this monorepo (husky, flat eslint config, no biome, no PR-CHECKLIST.md). The evaluator's check set must be per-subject (versioned, digested), not one global file list.
|
|
||||||
8. **Chain-ordering residual (documented).** `test:framework-shell` is one `&&` chain: a failing link skips every later suite while the step still fails (measured in #1270 — four suites after position 44 had not run since a prior merge). The enumeration guard proves naming, not reachability; both residuals are in-file documented but structurally unfixed.
|
|
||||||
9. **Signed-exclusion burndown open.** 16 signed exclusions remain in `test-enumeration-exclusions.txt`; several are "unmeasured in CI image" or blocked on missing CI tooling (tmux, setsid) — tracked under #1017/#1271. Each is an enforcement promise deferred, not delivered.
|
|
||||||
10. **Windows twins unexercised.** `verify.ps1`, `install.ps1`, `mosaic-doctor.ps1` have no runner anywhere (no Windows CI); behavioral drift from their bash twins is undetectable by construction.
|
|
||||||
11. **QA hook name vs behavior.** `qa-hook-handler.sh` files remediation report templates but performs no verification; the seam's actual gate value is only the deps-preflight sentinel. Anything relying on "QA automation hook" as a check is relying on report-filing.
|
|
||||||
12. **Two test paths, one gated.** CI runs tests against ci-postgres (`DATABASE_URL` set); the local PGlite path is the documented default (AGENTS.md) until KBN-101-02/101-05. Only the CI path is enforced by pipeline.
|
|
||||||
|
|
||||||
## 4. Disposition summary
|
|
||||||
|
|
||||||
| disposition | rows | checks |
|
|
||||||
| ------------------- | ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| preserve | 43 | Every canonical owner (QC-1..QC-18) plus correct guards-of-the-guard and thin adapters: all of §1.1, the CI-invoked framework probes and adapters in §1.2, all of §1.3, the C1/C2/C5 trio and doctors in §1.4, all of §1.5, all pipeline-only steps in §1.6, §1.7 rows 1 and 3, and §1.8. |
|
|
||||||
| strengthen | 2 | `quality-rails check` and `quality-rails doctor` (QC-19) — the RI-N4 evaluator seed: typed verdicts, versioned/digested check definitions, per-subject check sets. |
|
|
||||||
| strengthen (review) | 9 | `verify.sh` + `verify.ps1` (QC-20), quality `install.sh`/`install.ps1` + `quality-rails init` (QC-21 — scaffold-path convergence), `test-roster-schema.py` (QC-5 — wire or sign), `qa-hook-stdin.sh` seam + `typecheck-hook.sh` (QC-16), `tools/e2e-install-test.sh` (QC-5 — CI lane). |
|
|
||||||
| retire | 0 | None meet the bar: RI-N4 requires effective shell probes be **absorbed before** their paths retire, and no absorption exists yet. The `strengthen (review)` rows are the retirement candidates for RI-3-002 once the evaluator owns their behavior. |
|
|
||||||
|
|
||||||
Row total: 54. Canonical checks: 21 (QC-1..QC-21).
|
|
||||||
@@ -34,6 +34,7 @@ export default tseslint.config(
|
|||||||
'packages/storage/vitest.config.ts',
|
'packages/storage/vitest.config.ts',
|
||||||
'packages/mosaic/vitest.config.ts',
|
'packages/mosaic/vitest.config.ts',
|
||||||
'packages/mosaic/__tests__/*.ts',
|
'packages/mosaic/__tests__/*.ts',
|
||||||
|
'packages/forge/__tests__/*.ts',
|
||||||
'tools/federation-harness/*.ts',
|
'tools/federation-harness/*.ts',
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -539,3 +539,43 @@ Not every brief needs full Board of Directors review. The classification system
|
|||||||
### Backward compatibility
|
### Backward compatibility
|
||||||
|
|
||||||
Existing briefs without a `class` field are auto-classified. The default (no matching keywords) is `strategic`, so all existing runs get the full pipeline unless keywords trigger `technical`.
|
Existing briefs without a `class` field are auto-classified. The default (no matching keywords) is `strategic`, so all existing runs get the full pipeline unless keywords trigger `technical`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Fail-Closed Execution & Explicit Simulation (SDLC-D-035)
|
||||||
|
|
||||||
|
**Added:** 2026-08-17
|
||||||
|
|
||||||
|
Forge fails closed when a required capability is missing. It never runs a
|
||||||
|
pipeline with a stub executor and reports success.
|
||||||
|
|
||||||
|
### Normal mode (default)
|
||||||
|
|
||||||
|
- No task executor wired → the CLI exits nonzero with the typed capability
|
||||||
|
error `FORGE_NO_EXECUTOR`. No run is created.
|
||||||
|
- A stage whose gate is approval-based (board approval, planning approvals,
|
||||||
|
remediation re-review, discovery/analysis attestations) records a typed
|
||||||
|
`waiting-for-authority` stage result and raises `FORGE_AUTHORITY_REQUIRED`.
|
||||||
|
It never passes vacuously.
|
||||||
|
- A stage whose gate requires an unwired provider (AI reviewer, CI pipeline)
|
||||||
|
records a typed `blocked` stage result and raises `FORGE_NO_REVIEWER` /
|
||||||
|
`FORGE_NO_CI_PIPELINE`. The synthetic echo-review approval in `06-review`
|
||||||
|
and all vacuous `true` gates were removed.
|
||||||
|
|
||||||
|
### Explicit simulation (`--simulate`)
|
||||||
|
|
||||||
|
Opts into stub/synthetic execution. Every stage result, every gate result, and
|
||||||
|
the run manifest carry the distinct typed status `simulated` (manifest also
|
||||||
|
records `mode: "simulated"`). `simulated` is a non-satisfying outcome:
|
||||||
|
`isSatisfyingOutcome()` and all completion/gate consumers treat only `passed`
|
||||||
|
as satisfying. The CLI exits 0 for a simulated run only because the caller
|
||||||
|
explicitly passed `--simulate`, and prints a loud SIMULATED banner.
|
||||||
|
|
||||||
|
### Typed outcome model
|
||||||
|
|
||||||
|
Every gate/task outcome is one of the closed set
|
||||||
|
`passed | failed | blocked | error | waiting-for-authority | simulated |
|
||||||
|
not-applicable`, with the reason recorded on the stage status and each gate
|
||||||
|
result in `manifest.json`. Missing implementations, missing gate evidence,
|
||||||
|
unknown stages, process errors, and timeouts map to fail-closed members —
|
||||||
|
never to `passed`.
|
||||||
|
|||||||
@@ -0,0 +1,319 @@
|
|||||||
|
import fs from 'node:fs';
|
||||||
|
import os from 'node:os';
|
||||||
|
import path from 'node:path';
|
||||||
|
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
|
||||||
|
|
||||||
|
import { generateBoardTasks } from '../src/board-tasks.js';
|
||||||
|
import { STAGE_SPECS } from '../src/constants.js';
|
||||||
|
import { ForgeCapabilityError } from '../src/errors.js';
|
||||||
|
import {
|
||||||
|
evaluateStageGates,
|
||||||
|
gateLabel,
|
||||||
|
isCommandGate,
|
||||||
|
isSatisfyingOutcome,
|
||||||
|
} from '../src/outcomes.js';
|
||||||
|
import { loadManifest, runPipeline } from '../src/pipeline-runner.js';
|
||||||
|
import type { ForgeTask, ForgeTaskResult, TaskExecutor } from '../src/types.js';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Mock real executor that returns typed results.
|
||||||
|
*
|
||||||
|
* Command gates are "verified" by the mock so normal-mode runs can pass
|
||||||
|
* mechanically gated stages; authority/provider gates are never reported
|
||||||
|
* because they have no mechanical implementation.
|
||||||
|
*/
|
||||||
|
function createTypedExecutor(options?: {
|
||||||
|
failStage?: string;
|
||||||
|
gateOutcomes?: Record<string, 'passed' | 'failed' | 'simulated' | 'error' | 'blocked'>;
|
||||||
|
}): TaskExecutor & { submittedTasks: ForgeTask[] } {
|
||||||
|
const submittedTasks: ForgeTask[] = [];
|
||||||
|
return {
|
||||||
|
submittedTasks,
|
||||||
|
async submitTask(task: ForgeTask) {
|
||||||
|
submittedTasks.push(task);
|
||||||
|
},
|
||||||
|
async waitForCompletion(taskId: string): Promise<ForgeTaskResult> {
|
||||||
|
const task = submittedTasks.find((t) => t.id === taskId);
|
||||||
|
const stageName = task?.metadata?.['stageName'] as string | undefined;
|
||||||
|
|
||||||
|
if (options?.failStage && stageName === options.failStage) {
|
||||||
|
return {
|
||||||
|
task_id: taskId,
|
||||||
|
outcome: 'failed',
|
||||||
|
reason: 'mock task failure',
|
||||||
|
completed_at: new Date().toISOString(),
|
||||||
|
exit_code: 1,
|
||||||
|
gate_results: [],
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const gateResults = (task?.qualityGates ?? [])
|
||||||
|
.filter((gate) => isCommandGate(gate))
|
||||||
|
.map((gate) => {
|
||||||
|
const label = gateLabel(gate);
|
||||||
|
const outcome = options?.gateOutcomes?.[label] ?? 'passed';
|
||||||
|
return {
|
||||||
|
gate: label,
|
||||||
|
outcome,
|
||||||
|
reason: outcome === 'passed' ? 'mock verified' : `mock gate outcome: ${outcome}`,
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
|
return {
|
||||||
|
task_id: taskId,
|
||||||
|
outcome: 'passed',
|
||||||
|
reason: 'mock verified',
|
||||||
|
completed_at: new Date().toISOString(),
|
||||||
|
exit_code: 0,
|
||||||
|
gate_results: gateResults,
|
||||||
|
};
|
||||||
|
},
|
||||||
|
async getTaskStatus() {
|
||||||
|
return 'completed' as const;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('fail-closed: no executor wired', () => {
|
||||||
|
let tmpDir: string;
|
||||||
|
let briefPath: string;
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'forge-failclosed-'));
|
||||||
|
briefPath = path.join(tmpDir, 'brief.md');
|
||||||
|
fs.writeFileSync(briefPath, '# Fix bug\n\nA bugfix for lint cleanup.');
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('throws a typed FORGE_NO_EXECUTOR capability error without --simulate', async () => {
|
||||||
|
await expect(
|
||||||
|
runPipeline(briefPath, tmpDir, {
|
||||||
|
// no executor, no simulate — must fail closed, never run with a stub
|
||||||
|
stages: ['00-intake'],
|
||||||
|
}),
|
||||||
|
).rejects.toMatchObject({
|
||||||
|
name: 'ForgeCapabilityError',
|
||||||
|
code: 'FORGE_NO_EXECUTOR',
|
||||||
|
capability: 'task-executor',
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('does not create a run directory when failing closed on a missing executor', async () => {
|
||||||
|
try {
|
||||||
|
await runPipeline(briefPath, tmpDir, { stages: ['00-intake'] });
|
||||||
|
} catch {
|
||||||
|
// expected
|
||||||
|
}
|
||||||
|
expect(fs.existsSync(path.join(tmpDir, '.forge', 'runs'))).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('completes with every result typed simulated when simulate is set', async () => {
|
||||||
|
const result = await runPipeline(briefPath, tmpDir, {
|
||||||
|
simulate: true,
|
||||||
|
stages: ['00-intake', '00b-discovery', '02-planning-1', '06-review'],
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.manifest.mode).toBe('simulated');
|
||||||
|
expect(result.manifest.status).toBe('simulated');
|
||||||
|
|
||||||
|
for (const stage of result.stages) {
|
||||||
|
const stageStatus = result.manifest.stages[stage];
|
||||||
|
expect(stageStatus?.status, `stage ${stage}`).toBe('simulated');
|
||||||
|
expect(stageStatus?.status, `stage ${stage}`).not.toBe('passed');
|
||||||
|
expect(stageStatus?.reason, `stage ${stage}`).toBeTruthy();
|
||||||
|
for (const gateResult of stageStatus?.gateResults ?? []) {
|
||||||
|
expect(gateResult.outcome, `gate ${gateResult.gate} of ${stage}`).toBe('simulated');
|
||||||
|
expect(gateResult.outcome, `gate ${gateResult.gate} of ${stage}`).not.toBe('passed');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The persisted manifest agrees.
|
||||||
|
const persisted = loadManifest(result.runDir);
|
||||||
|
expect(persisted.mode).toBe('simulated');
|
||||||
|
expect(persisted.status).toBe('simulated');
|
||||||
|
expect(persisted.stages['02-planning-1']?.status).toBe('simulated');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('fail-closed: typed outcome model', () => {
|
||||||
|
it('only passed satisfies the gate/dependency predicate', () => {
|
||||||
|
expect(isSatisfyingOutcome('passed')).toBe(true);
|
||||||
|
expect(isSatisfyingOutcome('failed')).toBe(false);
|
||||||
|
expect(isSatisfyingOutcome('blocked')).toBe(false);
|
||||||
|
expect(isSatisfyingOutcome('error')).toBe(false);
|
||||||
|
expect(isSatisfyingOutcome('waiting-for-authority')).toBe(false);
|
||||||
|
expect(isSatisfyingOutcome('simulated')).toBe(false);
|
||||||
|
expect(isSatisfyingOutcome('not-applicable')).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('a simulated gate result cannot satisfy the stage gate evaluation', () => {
|
||||||
|
const evaluation = evaluateStageGates('05-coding', STAGE_SPECS['05-coding']!.qualityGates, {
|
||||||
|
task_id: 'FORGE-x-05',
|
||||||
|
outcome: 'passed',
|
||||||
|
reason: 'executor claims success',
|
||||||
|
completed_at: new Date().toISOString(),
|
||||||
|
exit_code: 0,
|
||||||
|
gate_results: [{ gate: 'pnpm lint', outcome: 'simulated', reason: 'simulated gate' }],
|
||||||
|
});
|
||||||
|
expect(isSatisfyingOutcome(evaluation.outcome)).toBe(false);
|
||||||
|
expect(evaluation.outcome).toBe('error');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('a simulated task outcome cannot satisfy evaluation in normal mode', () => {
|
||||||
|
const evaluation = evaluateStageGates('00-intake', [], {
|
||||||
|
task_id: 'FORGE-x-00',
|
||||||
|
outcome: 'simulated',
|
||||||
|
reason: 'executor reported simulated',
|
||||||
|
completed_at: new Date().toISOString(),
|
||||||
|
exit_code: 0,
|
||||||
|
gate_results: [],
|
||||||
|
});
|
||||||
|
expect(isSatisfyingOutcome(evaluation.outcome)).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('a missing gate result blocks the stage instead of passing vacuously', () => {
|
||||||
|
const evaluation = evaluateStageGates('05-coding', STAGE_SPECS['05-coding']!.qualityGates, {
|
||||||
|
task_id: 'FORGE-x-05',
|
||||||
|
outcome: 'passed',
|
||||||
|
reason: 'executor claims success',
|
||||||
|
completed_at: new Date().toISOString(),
|
||||||
|
exit_code: 0,
|
||||||
|
gate_results: [],
|
||||||
|
});
|
||||||
|
expect(evaluation.outcome).toBe('blocked');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('fail-closed: authority and provider gates', () => {
|
||||||
|
let tmpDir: string;
|
||||||
|
let briefPath: string;
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'forge-authority-'));
|
||||||
|
briefPath = path.join(tmpDir, 'brief.md');
|
||||||
|
fs.writeFileSync(briefPath, '# Fix bug\n\nA bugfix for lint cleanup.');
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each(['02-planning-1', '03-planning-2', '04-planning-3', '07-remediate'])(
|
||||||
|
'planning/remediation stage %s yields waiting-for-authority (not passed) in normal mode',
|
||||||
|
async (stage) => {
|
||||||
|
const executor = createTypedExecutor();
|
||||||
|
let runDir: string | undefined;
|
||||||
|
|
||||||
|
try {
|
||||||
|
await runPipeline(briefPath, tmpDir, {
|
||||||
|
executor,
|
||||||
|
stages: [stage as string],
|
||||||
|
});
|
||||||
|
expect.unreachable('runPipeline should have failed closed');
|
||||||
|
} catch (err) {
|
||||||
|
expect(err).toBeInstanceOf(ForgeCapabilityError);
|
||||||
|
expect((err as ForgeCapabilityError).code).toBe('FORGE_AUTHORITY_REQUIRED');
|
||||||
|
runDir = path.join(tmpDir, '.forge', 'runs');
|
||||||
|
}
|
||||||
|
|
||||||
|
const runIds = fs.readdirSync(runDir!);
|
||||||
|
expect(runIds).toHaveLength(1);
|
||||||
|
const manifest = loadManifest(path.join(runDir!, runIds[0]!));
|
||||||
|
expect(manifest.stages[stage]?.status).toBe('waiting-for-authority');
|
||||||
|
expect(manifest.stages[stage]?.status).not.toBe('passed');
|
||||||
|
expect(manifest.status).toBe('waiting-for-authority');
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
it('review stage fails closed with a typed FORGE_NO_REVIEWER error in normal mode', async () => {
|
||||||
|
const executor = createTypedExecutor();
|
||||||
|
|
||||||
|
try {
|
||||||
|
await runPipeline(briefPath, tmpDir, {
|
||||||
|
executor,
|
||||||
|
stages: ['06-review'],
|
||||||
|
});
|
||||||
|
expect.unreachable('runPipeline should have failed closed');
|
||||||
|
} catch (err) {
|
||||||
|
expect(err).toBeInstanceOf(ForgeCapabilityError);
|
||||||
|
expect((err as ForgeCapabilityError).code).toBe('FORGE_NO_REVIEWER');
|
||||||
|
expect((err as ForgeCapabilityError).capability).toBe('reviewer');
|
||||||
|
}
|
||||||
|
|
||||||
|
const runsDir = path.join(tmpDir, '.forge', 'runs');
|
||||||
|
const runIds = fs.readdirSync(runsDir);
|
||||||
|
const manifest = loadManifest(path.join(runsDir, runIds[0]!));
|
||||||
|
expect(manifest.stages['06-review']?.status).toBe('blocked');
|
||||||
|
expect(manifest.stages['06-review']?.status).not.toBe('passed');
|
||||||
|
expect(manifest.status).toBe('failed');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('review stage produces simulated results under --simulate', async () => {
|
||||||
|
const result = await runPipeline(briefPath, tmpDir, {
|
||||||
|
simulate: true,
|
||||||
|
stages: ['06-review'],
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.manifest.mode).toBe('simulated');
|
||||||
|
expect(result.manifest.stages['06-review']?.status).toBe('simulated');
|
||||||
|
for (const gateResult of result.manifest.stages['06-review']?.gateResults ?? []) {
|
||||||
|
expect(gateResult.outcome).toBe('simulated');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('deploy stage fails closed without a wired ci-pipeline provider in normal mode', async () => {
|
||||||
|
const executor = createTypedExecutor();
|
||||||
|
|
||||||
|
await expect(
|
||||||
|
runPipeline(briefPath, tmpDir, {
|
||||||
|
executor,
|
||||||
|
stages: ['09-deploy'],
|
||||||
|
}),
|
||||||
|
).rejects.toMatchObject({
|
||||||
|
name: 'ForgeCapabilityError',
|
||||||
|
code: 'FORGE_NO_CI_PIPELINE',
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('fail-closed: no vacuous gate commands remain', () => {
|
||||||
|
it('stage constants contain no echo/synthetic-approval, vacuous true, or empty gate commands', () => {
|
||||||
|
for (const [stageName, spec] of Object.entries(STAGE_SPECS)) {
|
||||||
|
for (const gate of spec.qualityGates) {
|
||||||
|
const serialized = JSON.stringify(gate);
|
||||||
|
// The echo-review synthetic approval must be gone.
|
||||||
|
expect(serialized, `stage ${stageName} gate ${serialized}`).not.toContain('echo');
|
||||||
|
expect(serialized, `stage ${stageName} gate ${serialized}`).not.toMatch(/"verdict"\s*:/);
|
||||||
|
expect(serialized, `stage ${stageName} gate ${serialized}`).not.toMatch(
|
||||||
|
/"summary"\s*:\s*"review-pass"/,
|
||||||
|
);
|
||||||
|
// No vacuous literal `true` gate.
|
||||||
|
expect(gate, `stage ${stageName}`).not.toBe('true');
|
||||||
|
// Command gates must carry a real, non-empty command.
|
||||||
|
if (isCommandGate(gate)) {
|
||||||
|
const command = typeof gate === 'string' ? gate : gate.command;
|
||||||
|
expect(command.trim().length, `stage ${stageName} gate ${serialized}`).toBeGreaterThan(0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('board tasks contain no vacuous true gates', () => {
|
||||||
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'forge-board-gates-'));
|
||||||
|
try {
|
||||||
|
const tasks = generateBoardTasks('# Brief', [], tmpDir, 'BOARD-TEST');
|
||||||
|
for (const task of tasks) {
|
||||||
|
for (const gate of task.qualityGates) {
|
||||||
|
expect(gate, `task ${task.id}`).not.toBe('true');
|
||||||
|
const serialized = JSON.stringify(gate);
|
||||||
|
expect(serialized, `task ${task.id} gate ${serialized}`).not.toContain('echo');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -12,10 +12,10 @@ import {
|
|||||||
resumePipeline,
|
resumePipeline,
|
||||||
getPipelineStatus,
|
getPipelineStatus,
|
||||||
} from '../src/pipeline-runner.js';
|
} from '../src/pipeline-runner.js';
|
||||||
import type { ForgeTask, RunManifest, TaskExecutor } from '../src/types.js';
|
import type { ForgeTask, ForgeTaskResult, RunManifest, TaskExecutor } from '../src/types.js';
|
||||||
import type { TaskResult } from '@mosaicstack/macp';
|
import { gateLabel, isCommandGate } from '../src/outcomes.js';
|
||||||
|
|
||||||
/** Mock TaskExecutor that records submitted tasks and returns success. */
|
/** Mock TaskExecutor that records submitted tasks and returns typed results. */
|
||||||
function createMockExecutor(options?: {
|
function createMockExecutor(options?: {
|
||||||
failStage?: string;
|
failStage?: string;
|
||||||
}): TaskExecutor & { submittedTasks: ForgeTask[] } {
|
}): TaskExecutor & { submittedTasks: ForgeTask[] } {
|
||||||
@@ -25,7 +25,7 @@ function createMockExecutor(options?: {
|
|||||||
async submitTask(task: ForgeTask) {
|
async submitTask(task: ForgeTask) {
|
||||||
submittedTasks.push(task);
|
submittedTasks.push(task);
|
||||||
},
|
},
|
||||||
async waitForCompletion(taskId: string): Promise<TaskResult> {
|
async waitForCompletion(taskId: string): Promise<ForgeTaskResult> {
|
||||||
const failStage = options?.failStage;
|
const failStage = options?.failStage;
|
||||||
const task = submittedTasks.find((t) => t.id === taskId);
|
const task = submittedTasks.find((t) => t.id === taskId);
|
||||||
const stageName = task?.metadata?.['stageName'] as string | undefined;
|
const stageName = task?.metadata?.['stageName'] as string | undefined;
|
||||||
@@ -33,7 +33,8 @@ function createMockExecutor(options?: {
|
|||||||
if (failStage && stageName === failStage) {
|
if (failStage && stageName === failStage) {
|
||||||
return {
|
return {
|
||||||
task_id: taskId,
|
task_id: taskId,
|
||||||
status: 'failed',
|
outcome: 'failed',
|
||||||
|
reason: 'mock task failure',
|
||||||
completed_at: new Date().toISOString(),
|
completed_at: new Date().toISOString(),
|
||||||
exit_code: 1,
|
exit_code: 1,
|
||||||
gate_results: [],
|
gate_results: [],
|
||||||
@@ -41,10 +42,17 @@ function createMockExecutor(options?: {
|
|||||||
}
|
}
|
||||||
return {
|
return {
|
||||||
task_id: taskId,
|
task_id: taskId,
|
||||||
status: 'completed',
|
outcome: 'passed',
|
||||||
|
reason: 'mock verified',
|
||||||
completed_at: new Date().toISOString(),
|
completed_at: new Date().toISOString(),
|
||||||
exit_code: 0,
|
exit_code: 0,
|
||||||
gate_results: [],
|
gate_results: (task?.qualityGates ?? [])
|
||||||
|
.filter((gate) => isCommandGate(gate))
|
||||||
|
.map((gate) => ({
|
||||||
|
gate: gateLabel(gate),
|
||||||
|
outcome: 'passed' as const,
|
||||||
|
reason: 'mock verified',
|
||||||
|
})),
|
||||||
};
|
};
|
||||||
},
|
},
|
||||||
async getTaskStatus() {
|
async getTaskStatus() {
|
||||||
@@ -156,12 +164,13 @@ describe('runPipeline', () => {
|
|||||||
const executor = createMockExecutor();
|
const executor = createMockExecutor();
|
||||||
const result = await runPipeline(briefPath, tmpDir, {
|
const result = await runPipeline(briefPath, tmpDir, {
|
||||||
executor,
|
executor,
|
||||||
stages: ['00-intake', '00b-discovery'],
|
stages: ['00-intake', '05-coding'],
|
||||||
});
|
});
|
||||||
|
|
||||||
expect(result.runId).toMatch(/^\d{8}-\d{6}$/);
|
expect(result.runId).toMatch(/^\d{8}-\d{6}$/);
|
||||||
expect(result.stages).toEqual(['00-intake', '00b-discovery']);
|
expect(result.stages).toEqual(['00-intake', '05-coding']);
|
||||||
expect(result.manifest.status).toBe('completed');
|
expect(result.manifest.status).toBe('completed');
|
||||||
|
expect(result.manifest.mode).toBe('normal');
|
||||||
expect(executor.submittedTasks).toHaveLength(2);
|
expect(executor.submittedTasks).toHaveLength(2);
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -180,12 +189,17 @@ describe('runPipeline', () => {
|
|||||||
const executor = createMockExecutor();
|
const executor = createMockExecutor();
|
||||||
const result = await runPipeline(briefPath, tmpDir, {
|
const result = await runPipeline(briefPath, tmpDir, {
|
||||||
executor,
|
executor,
|
||||||
stages: ['00-intake', '00b-discovery'],
|
stages: ['00-intake', '05-coding'],
|
||||||
});
|
});
|
||||||
|
|
||||||
const manifest = loadManifest(result.runDir);
|
const manifest = loadManifest(result.runDir);
|
||||||
expect(manifest.stages['00-intake']?.status).toBe('passed');
|
expect(manifest.stages['00-intake']?.status).toBe('passed');
|
||||||
expect(manifest.stages['00b-discovery']?.status).toBe('passed');
|
expect(manifest.stages['05-coding']?.status).toBe('passed');
|
||||||
|
expect(manifest.stages['05-coding']?.gateResults?.map((g) => g.outcome)).toEqual([
|
||||||
|
'passed',
|
||||||
|
'passed',
|
||||||
|
'passed',
|
||||||
|
]);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('respects CLI class override', async () => {
|
it('respects CLI class override', async () => {
|
||||||
@@ -215,7 +229,7 @@ describe('runPipeline', () => {
|
|||||||
const executor = createMockExecutor();
|
const executor = createMockExecutor();
|
||||||
await runPipeline(briefPath, tmpDir, {
|
await runPipeline(briefPath, tmpDir, {
|
||||||
executor,
|
executor,
|
||||||
stages: ['00-intake', '00b-discovery', '02-planning-1'],
|
stages: ['00-intake', '05-coding', '08-test'],
|
||||||
});
|
});
|
||||||
|
|
||||||
expect(executor.submittedTasks[0]!.dependsOn).toBeUndefined();
|
expect(executor.submittedTasks[0]!.dependsOn).toBeUndefined();
|
||||||
@@ -224,14 +238,14 @@ describe('runPipeline', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it('handles stage failure', async () => {
|
it('handles stage failure', async () => {
|
||||||
const executor = createMockExecutor({ failStage: '00b-discovery' });
|
const executor = createMockExecutor({ failStage: '05-coding' });
|
||||||
|
|
||||||
await expect(
|
await expect(
|
||||||
runPipeline(briefPath, tmpDir, {
|
runPipeline(briefPath, tmpDir, {
|
||||||
executor,
|
executor,
|
||||||
stages: ['00-intake', '00b-discovery'],
|
stages: ['00-intake', '05-coding'],
|
||||||
}),
|
}),
|
||||||
).rejects.toThrow('Stage 00b-discovery failed');
|
).rejects.toThrow('Stage 05-coding failed');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('marks manifest as failed on stage failure', async () => {
|
it('marks manifest as failed on stage failure', async () => {
|
||||||
@@ -270,30 +284,143 @@ describe('resumePipeline', () => {
|
|||||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||||
});
|
});
|
||||||
|
|
||||||
it('resumes from first incomplete stage', async () => {
|
it('resumes from first incomplete stage and fails closed at the next provider gate', async () => {
|
||||||
// First run fails on discovery
|
// Simulate a run whose authority stages were approved out-of-band
|
||||||
const executor1 = createMockExecutor({ failStage: '00b-discovery' });
|
// (recorded as passed) and whose coding stage failed mechanically.
|
||||||
let runDir: string;
|
const runId = '20260101-000000';
|
||||||
|
const runDir = path.join(tmpDir, '.forge', 'runs', runId);
|
||||||
|
fs.mkdirSync(runDir, { recursive: true });
|
||||||
|
const passed = { status: 'passed' as const, startedAt: '2026-01-01T00:00:00Z' };
|
||||||
|
saveManifest(runDir, {
|
||||||
|
runId,
|
||||||
|
brief: briefPath,
|
||||||
|
codebase: tmpDir,
|
||||||
|
briefClass: 'hotfix',
|
||||||
|
classSource: 'frontmatter',
|
||||||
|
forceBoard: false,
|
||||||
|
mode: 'normal',
|
||||||
|
createdAt: '2026-01-01T00:00:00Z',
|
||||||
|
updatedAt: '2026-01-01T00:00:00Z',
|
||||||
|
currentStage: '05-coding',
|
||||||
|
status: 'failed',
|
||||||
|
stages: {
|
||||||
|
'00-intake': passed,
|
||||||
|
'00b-discovery': passed,
|
||||||
|
'02-planning-1': passed,
|
||||||
|
'03-planning-2': passed,
|
||||||
|
'04-planning-3': passed,
|
||||||
|
'05-coding': { status: 'failed', reason: 'gate failed' },
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
try {
|
// Resume re-runs 05-coding (the first non-passed stage), then fails
|
||||||
await runPipeline(briefPath, tmpDir, {
|
// closed at 06-review because no reviewer provider is wired.
|
||||||
executor: executor1,
|
const executor = createMockExecutor();
|
||||||
stages: ['00-intake', '00b-discovery', '02-planning-1'],
|
await expect(resumePipeline(runDir, executor)).rejects.toMatchObject({
|
||||||
});
|
name: 'ForgeCapabilityError',
|
||||||
} catch {
|
code: 'FORGE_NO_REVIEWER',
|
||||||
// expected
|
});
|
||||||
|
|
||||||
|
const manifest = loadManifest(runDir);
|
||||||
|
expect(manifest.stages['05-coding']?.status).toBe('passed');
|
||||||
|
expect(manifest.stages['06-review']?.status).toBe('blocked');
|
||||||
|
expect(manifest.status).toBe('failed');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('resumes to completion as simulated under explicit simulate', async () => {
|
||||||
|
const runId = '20260101-000003';
|
||||||
|
const runDir = path.join(tmpDir, '.forge', 'runs', runId);
|
||||||
|
fs.mkdirSync(runDir, { recursive: true });
|
||||||
|
const passed = { status: 'passed' as const, startedAt: '2026-01-01T00:00:00Z' };
|
||||||
|
saveManifest(runDir, {
|
||||||
|
runId,
|
||||||
|
brief: briefPath,
|
||||||
|
codebase: tmpDir,
|
||||||
|
briefClass: 'hotfix',
|
||||||
|
classSource: 'frontmatter',
|
||||||
|
forceBoard: false,
|
||||||
|
mode: 'normal',
|
||||||
|
createdAt: '2026-01-01T00:00:00Z',
|
||||||
|
updatedAt: '2026-01-01T00:00:00Z',
|
||||||
|
currentStage: '05-coding',
|
||||||
|
status: 'failed',
|
||||||
|
stages: {
|
||||||
|
'00-intake': passed,
|
||||||
|
'00b-discovery': passed,
|
||||||
|
'02-planning-1': passed,
|
||||||
|
'03-planning-2': passed,
|
||||||
|
'04-planning-3': passed,
|
||||||
|
'05-coding': { status: 'failed', reason: 'gate failed' },
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = await resumePipeline(runDir, undefined, { simulate: true });
|
||||||
|
|
||||||
|
expect(result.manifest.status).toBe('simulated');
|
||||||
|
expect(result.manifest.mode).toBe('simulated');
|
||||||
|
expect(result.stages[0]).toBe('05-coding');
|
||||||
|
for (const stage of result.stages) {
|
||||||
|
expect(result.manifest.stages[stage]?.status).toBe('simulated');
|
||||||
}
|
}
|
||||||
|
});
|
||||||
|
|
||||||
const runsDir = path.join(tmpDir, '.forge', 'runs');
|
it('fails closed on resume when the next stage needs authority sign-off', async () => {
|
||||||
runDir = path.join(runsDir, fs.readdirSync(runsDir)[0]!);
|
const runId = '20260101-000001';
|
||||||
|
const runDir = path.join(tmpDir, '.forge', 'runs', runId);
|
||||||
|
fs.mkdirSync(runDir, { recursive: true });
|
||||||
|
saveManifest(runDir, {
|
||||||
|
runId,
|
||||||
|
brief: briefPath,
|
||||||
|
codebase: tmpDir,
|
||||||
|
briefClass: 'hotfix',
|
||||||
|
classSource: 'frontmatter',
|
||||||
|
forceBoard: false,
|
||||||
|
mode: 'normal',
|
||||||
|
createdAt: '2026-01-01T00:00:00Z',
|
||||||
|
updatedAt: '2026-01-01T00:00:00Z',
|
||||||
|
currentStage: '00-intake',
|
||||||
|
status: 'in_progress',
|
||||||
|
stages: {
|
||||||
|
'00-intake': { status: 'passed' },
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
// Resume should pick up from 00b-discovery
|
const executor = createMockExecutor();
|
||||||
const executor2 = createMockExecutor();
|
await expect(resumePipeline(runDir, executor)).rejects.toMatchObject({
|
||||||
const result = await resumePipeline(runDir, executor2);
|
name: 'ForgeCapabilityError',
|
||||||
|
code: 'FORGE_AUTHORITY_REQUIRED',
|
||||||
|
});
|
||||||
|
|
||||||
expect(result.manifest.status).toBe('completed');
|
const manifest = loadManifest(runDir);
|
||||||
// Should have re-run from 00b-discovery onward
|
expect(manifest.stages['00b-discovery']?.status).toBe('waiting-for-authority');
|
||||||
expect(result.stages[0]).toBe('00b-discovery');
|
expect(manifest.status).toBe('waiting-for-authority');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('fails closed on resume without an executor or --simulate', async () => {
|
||||||
|
const runId = '20260101-000002';
|
||||||
|
const runDir = path.join(tmpDir, '.forge', 'runs', runId);
|
||||||
|
fs.mkdirSync(runDir, { recursive: true });
|
||||||
|
saveManifest(runDir, {
|
||||||
|
runId,
|
||||||
|
brief: briefPath,
|
||||||
|
codebase: tmpDir,
|
||||||
|
briefClass: 'hotfix',
|
||||||
|
classSource: 'frontmatter',
|
||||||
|
forceBoard: false,
|
||||||
|
mode: 'normal',
|
||||||
|
createdAt: '2026-01-01T00:00:00Z',
|
||||||
|
updatedAt: '2026-01-01T00:00:00Z',
|
||||||
|
currentStage: '00-intake',
|
||||||
|
status: 'in_progress',
|
||||||
|
stages: {
|
||||||
|
'00-intake': { status: 'passed' },
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
await expect(resumePipeline(runDir)).rejects.toMatchObject({
|
||||||
|
name: 'ForgeCapabilityError',
|
||||||
|
code: 'FORGE_NO_EXECUTOR',
|
||||||
|
});
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -95,7 +95,14 @@ export function generateBoardTasks(
|
|||||||
briefPath,
|
briefPath,
|
||||||
resultPath: resultRelPath,
|
resultPath: resultRelPath,
|
||||||
timeoutSeconds: 120,
|
timeoutSeconds: 120,
|
||||||
qualityGates: ['true'],
|
qualityGates: [
|
||||||
|
{
|
||||||
|
kind: 'authority',
|
||||||
|
capability: 'board-approval',
|
||||||
|
reason:
|
||||||
|
'persona evaluation is judged by board synthesis (authority review); no mechanical gate exists',
|
||||||
|
},
|
||||||
|
],
|
||||||
metadata: {
|
metadata: {
|
||||||
personaName: persona.name,
|
personaName: persona.name,
|
||||||
personaSlug: persona.slug,
|
personaSlug: persona.slug,
|
||||||
@@ -121,7 +128,13 @@ export function generateBoardTasks(
|
|||||||
timeoutSeconds: 120,
|
timeoutSeconds: 120,
|
||||||
dependsOn: personaTaskIds,
|
dependsOn: personaTaskIds,
|
||||||
dependsOnPolicy: 'all_terminal',
|
dependsOnPolicy: 'all_terminal',
|
||||||
qualityGates: ['true'],
|
qualityGates: [
|
||||||
|
{
|
||||||
|
kind: 'authority',
|
||||||
|
capability: 'board-approval',
|
||||||
|
reason: 'board synthesis is an authority decision; no mechanical gate exists',
|
||||||
|
},
|
||||||
|
],
|
||||||
metadata: {
|
metadata: {
|
||||||
resultOutputPath: synthesisResult,
|
resultOutputPath: synthesisResult,
|
||||||
inputResultPaths: personaResultPaths,
|
inputResultPaths: personaResultPaths,
|
||||||
|
|||||||
@@ -1,7 +1,11 @@
|
|||||||
|
import fs from 'node:fs';
|
||||||
|
import os from 'node:os';
|
||||||
|
import path from 'node:path';
|
||||||
import { Command } from 'commander';
|
import { Command } from 'commander';
|
||||||
import { describe, expect, it } from 'vitest';
|
import { describe, expect, it, vi, beforeEach, afterEach } from 'vitest';
|
||||||
|
|
||||||
import { registerForgeCommand } from './cli.js';
|
import { registerForgeCommand } from './cli.js';
|
||||||
|
import { loadManifest } from './pipeline-runner.js';
|
||||||
|
|
||||||
describe('registerForgeCommand', () => {
|
describe('registerForgeCommand', () => {
|
||||||
it('registers a "forge" command on the parent program', () => {
|
it('registers a "forge" command on the parent program', () => {
|
||||||
@@ -55,3 +59,94 @@ describe('registerForgeCommand', () => {
|
|||||||
}).not.toThrow();
|
}).not.toThrow();
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('forge run fail-closed behavior (SDLC-D-035)', () => {
|
||||||
|
let tmpDir: string;
|
||||||
|
let briefPath: string;
|
||||||
|
let errSpy: ReturnType<typeof vi.spyOn>;
|
||||||
|
let logSpy: ReturnType<typeof vi.spyOn>;
|
||||||
|
let prevExitCode: string | number | null | undefined;
|
||||||
|
|
||||||
|
const parse = (args: string[]) => {
|
||||||
|
const program = new Command();
|
||||||
|
registerForgeCommand(program);
|
||||||
|
return program.parseAsync(['forge', ...args], { from: 'user' });
|
||||||
|
};
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'forge-cli-failclosed-'));
|
||||||
|
briefPath = path.join(tmpDir, 'brief.md');
|
||||||
|
fs.writeFileSync(briefPath, '# Fix bug\n\nA bugfix for lint cleanup.');
|
||||||
|
errSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||||
|
logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||||
|
prevExitCode = process.exitCode;
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
errSpy.mockRestore();
|
||||||
|
logSpy.mockRestore();
|
||||||
|
process.exitCode = prevExitCode;
|
||||||
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('exits nonzero with a typed FORGE_NO_EXECUTOR error when no executor is wired and --simulate is absent', async () => {
|
||||||
|
await parse(['run', '--brief', briefPath, '--codebase', tmpDir]);
|
||||||
|
|
||||||
|
expect(process.exitCode).toBe(1);
|
||||||
|
const errText = errSpy.mock.calls.map((c) => c.join(' ')).join('\n');
|
||||||
|
expect(errText).toContain('FORGE_NO_EXECUTOR');
|
||||||
|
// It must never run the pipeline with a stub and report success.
|
||||||
|
expect(fs.existsSync(path.join(tmpDir, '.forge', 'runs'))).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('completes with typed simulated results and exit 0 under explicit --simulate', async () => {
|
||||||
|
await parse(['run', '--brief', briefPath, '--codebase', tmpDir, '--simulate']);
|
||||||
|
|
||||||
|
expect(process.exitCode).toBeUndefined();
|
||||||
|
|
||||||
|
// Loud simulated-mode summary.
|
||||||
|
const logText = logSpy.mock.calls.map((c) => c.join(' ')).join('\n');
|
||||||
|
expect(logText).toContain('SIMULATED');
|
||||||
|
|
||||||
|
// Manifest records the mode and simulated per-result statuses.
|
||||||
|
const runsDir = path.join(tmpDir, '.forge', 'runs');
|
||||||
|
const runIds = fs.readdirSync(runsDir);
|
||||||
|
expect(runIds).toHaveLength(1);
|
||||||
|
const manifest = loadManifest(path.join(runsDir, runIds[0]!));
|
||||||
|
expect(manifest.mode).toBe('simulated');
|
||||||
|
expect(manifest.status).toBe('simulated');
|
||||||
|
for (const stageStatus of Object.values(manifest.stages)) {
|
||||||
|
expect(stageStatus?.status).toBe('simulated');
|
||||||
|
for (const gateResult of stageStatus?.gateResults ?? []) {
|
||||||
|
expect(gateResult.outcome).toBe('simulated');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('resume exits nonzero with a typed FORGE_NO_EXECUTOR error without --simulate', async () => {
|
||||||
|
const runDir = path.join(tmpDir, '.forge', 'runs', '20260101-000000');
|
||||||
|
fs.mkdirSync(runDir, { recursive: true });
|
||||||
|
fs.writeFileSync(
|
||||||
|
path.join(runDir, 'manifest.json'),
|
||||||
|
JSON.stringify({
|
||||||
|
runId: '20260101-000000',
|
||||||
|
brief: briefPath,
|
||||||
|
codebase: tmpDir,
|
||||||
|
briefClass: 'hotfix',
|
||||||
|
classSource: 'frontmatter',
|
||||||
|
forceBoard: false,
|
||||||
|
createdAt: '2026-01-01T00:00:00Z',
|
||||||
|
updatedAt: '2026-01-01T00:00:00Z',
|
||||||
|
currentStage: '00-intake',
|
||||||
|
status: 'in_progress',
|
||||||
|
stages: { '00-intake': { status: 'passed' } },
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
await parse(['resume', '20260101-000000', '--project', tmpDir]);
|
||||||
|
|
||||||
|
expect(process.exitCode).toBe(1);
|
||||||
|
const errText = errSpy.mock.calls.map((c) => c.join(' ')).join('\n');
|
||||||
|
expect(errText).toContain('FORGE_NO_EXECUTOR');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|||||||
+122
-48
@@ -5,37 +5,47 @@ import type { Command } from 'commander';
|
|||||||
|
|
||||||
import { classifyBrief } from './brief-classifier.js';
|
import { classifyBrief } from './brief-classifier.js';
|
||||||
import { STAGE_LABELS, STAGE_SEQUENCE } from './constants.js';
|
import { STAGE_LABELS, STAGE_SEQUENCE } from './constants.js';
|
||||||
|
import { ForgeCapabilityError } from './errors.js';
|
||||||
import { getEffectivePersonas, loadBoardPersonas } from './persona-loader.js';
|
import { getEffectivePersonas, loadBoardPersonas } from './persona-loader.js';
|
||||||
import { generateRunId, getPipelineStatus, loadManifest, runPipeline } from './pipeline-runner.js';
|
import { generateRunId, getPipelineStatus, loadManifest, runPipeline } from './pipeline-runner.js';
|
||||||
import type { PipelineOptions, RunManifest, TaskExecutor } from './types.js';
|
import { createSimulatedExecutor } from './simulated-executor.js';
|
||||||
|
import type { PipelineOptions, RunManifest, RunMode } from './types.js';
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
// Stub executor — used when no real executor is wired at CLI invocation time.
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
const stubExecutor: TaskExecutor = {
|
|
||||||
async submitTask(task) {
|
|
||||||
console.log(` [forge] stage submitted: ${task.id} (${task.title})`);
|
|
||||||
},
|
|
||||||
async waitForCompletion(taskId, _timeoutMs) {
|
|
||||||
console.log(` [forge] stage complete: ${taskId}`);
|
|
||||||
return {
|
|
||||||
task_id: taskId,
|
|
||||||
status: 'completed' as const,
|
|
||||||
completed_at: new Date().toISOString(),
|
|
||||||
exit_code: 0,
|
|
||||||
gate_results: [],
|
|
||||||
};
|
|
||||||
},
|
|
||||||
async getTaskStatus(_taskId) {
|
|
||||||
return 'completed' as const;
|
|
||||||
},
|
|
||||||
};
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Helpers
|
// Helpers
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/** Resolve a run's effective mode, defaulting legacy manifests to normal. */
|
||||||
|
function runModeOf(manifest: RunManifest): RunMode {
|
||||||
|
return manifest.mode ?? 'normal';
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Print a loud banner so a simulated run can never be misread as verified. */
|
||||||
|
function printSimulatedBanner(): void {
|
||||||
|
console.log('');
|
||||||
|
console.log('[forge] ===============================================================');
|
||||||
|
console.log('[forge] MODE: SIMULATED — no stage or gate was really executed.');
|
||||||
|
console.log('[forge] All results are synthetic and MUST NOT be read as verified');
|
||||||
|
console.log('[forge] success. Wire a real executor/providers and re-run to verify.');
|
||||||
|
console.log('[forge] ===============================================================');
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Print a typed error line for fail-closed capability errors. */
|
||||||
|
function printCapabilityError(err: ForgeCapabilityError): void {
|
||||||
|
console.error(`[forge] error ${err.code}: ${err.message}`);
|
||||||
|
console.error(`[forge] missing capability: ${err.capability}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Handle a pipeline error uniformly: typed capability errors get their code. */
|
||||||
|
function handlePipelineError(err: unknown): void {
|
||||||
|
if (err instanceof ForgeCapabilityError) {
|
||||||
|
printCapabilityError(err);
|
||||||
|
} else {
|
||||||
|
console.error(`[forge] pipeline failed: ${err instanceof Error ? err.message : String(err)}`);
|
||||||
|
}
|
||||||
|
process.exitCode = 1;
|
||||||
|
}
|
||||||
|
|
||||||
function formatDuration(startedAt?: string, completedAt?: string): string {
|
function formatDuration(startedAt?: string, completedAt?: string): string {
|
||||||
if (!startedAt || !completedAt) return '-';
|
if (!startedAt || !completedAt) return '-';
|
||||||
const ms = new Date(completedAt).getTime() - new Date(startedAt).getTime();
|
const ms = new Date(completedAt).getTime() - new Date(startedAt).getTime();
|
||||||
@@ -44,19 +54,24 @@ function formatDuration(startedAt?: string, completedAt?: string): string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
function printManifestTable(manifest: RunManifest): void {
|
function printManifestTable(manifest: RunManifest): void {
|
||||||
|
const mode = runModeOf(manifest);
|
||||||
console.log(`\nRun ID : ${manifest.runId}`);
|
console.log(`\nRun ID : ${manifest.runId}`);
|
||||||
console.log(`Status : ${manifest.status}`);
|
console.log(`Status : ${manifest.status}`);
|
||||||
|
console.log(`Mode : ${mode}`);
|
||||||
|
if (mode === 'simulated') {
|
||||||
|
console.log('WARNING: SIMULATED RUN — results are synthetic, not verified success.');
|
||||||
|
}
|
||||||
console.log(`Brief : ${manifest.brief}`);
|
console.log(`Brief : ${manifest.brief}`);
|
||||||
console.log(`Class : ${manifest.briefClass} (${manifest.classSource})`);
|
console.log(`Class : ${manifest.briefClass} (${manifest.classSource})`);
|
||||||
console.log(`Updated: ${manifest.updatedAt}`);
|
console.log(`Updated: ${manifest.updatedAt}`);
|
||||||
console.log('');
|
console.log('');
|
||||||
console.log('Stage'.padEnd(22) + 'Status'.padEnd(14) + 'Duration');
|
console.log('Stage'.padEnd(22) + 'Status'.padEnd(24) + 'Duration');
|
||||||
console.log('-'.repeat(50));
|
console.log('-'.repeat(60));
|
||||||
for (const stage of STAGE_SEQUENCE) {
|
for (const stage of STAGE_SEQUENCE) {
|
||||||
const s = manifest.stages[stage];
|
const s = manifest.stages[stage];
|
||||||
if (!s) continue;
|
if (!s) continue;
|
||||||
const label = (STAGE_LABELS[stage] ?? stage).padEnd(22);
|
const label = (STAGE_LABELS[stage] ?? stage).padEnd(22);
|
||||||
const status = s.status.padEnd(14);
|
const status = s.status.padEnd(24);
|
||||||
const dur = formatDuration(s.startedAt, s.completedAt);
|
const dur = formatDuration(s.startedAt, s.completedAt);
|
||||||
console.log(`${label}${status}${dur}`);
|
console.log(`${label}${status}${dur}`);
|
||||||
}
|
}
|
||||||
@@ -90,23 +105,58 @@ function listRecentRuns(projectRoot?: string): void {
|
|||||||
}
|
}
|
||||||
|
|
||||||
console.log('\nRecent runs:');
|
console.log('\nRecent runs:');
|
||||||
console.log('Run ID'.padEnd(22) + 'Status'.padEnd(14) + 'Brief');
|
console.log('Run ID'.padEnd(22) + 'Status'.padEnd(24) + 'Mode'.padEnd(12) + 'Brief');
|
||||||
console.log('-'.repeat(70));
|
console.log('-'.repeat(80));
|
||||||
|
|
||||||
for (const runId of entries) {
|
for (const runId of entries) {
|
||||||
const runDir = path.join(runsDir, runId);
|
const runDir = path.join(runsDir, runId);
|
||||||
try {
|
try {
|
||||||
const manifest = loadManifest(runDir);
|
const manifest = loadManifest(runDir);
|
||||||
const status = manifest.status.padEnd(14);
|
const status = manifest.status.padEnd(24);
|
||||||
|
const mode = runModeOf(manifest).padEnd(12);
|
||||||
const brief = path.basename(manifest.brief);
|
const brief = path.basename(manifest.brief);
|
||||||
console.log(`${runId.padEnd(22)}${status}${brief}`);
|
console.log(`${runId.padEnd(22)}${status}${mode}${brief}`);
|
||||||
} catch {
|
} catch {
|
||||||
console.log(`${runId.padEnd(22)}${'(unreadable)'.padEnd(14)}`);
|
console.log(`${runId.padEnd(22)}${'(unreadable)'.padEnd(24)}`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
console.log('');
|
console.log('');
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Apply the exit-code policy for a finished pipeline run (SDLC-D-035):
|
||||||
|
*
|
||||||
|
* - exit 0 only for a verified `completed` normal run, or for an overall
|
||||||
|
* `simulated` run when the caller explicitly passed --simulate;
|
||||||
|
* - anything else exits nonzero so it can never be read as success.
|
||||||
|
*/
|
||||||
|
function applyRunExitPolicy(result: { manifest: RunManifest; runDir: string }, simulate: boolean) {
|
||||||
|
const { manifest } = result;
|
||||||
|
|
||||||
|
if (runModeOf(manifest) === 'simulated') {
|
||||||
|
if (!simulate || manifest.status !== 'simulated') {
|
||||||
|
console.error(
|
||||||
|
'[forge] error FORGE_MODE_MISMATCH: run reports simulated results without an explicit, ' +
|
||||||
|
'consistent --simulate request; refusing to report success.',
|
||||||
|
);
|
||||||
|
process.exitCode = 1;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
printSimulatedBanner();
|
||||||
|
console.log(`[forge] run directory: ${result.runDir}`);
|
||||||
|
return; // exit 0 — the caller explicitly opted into simulation
|
||||||
|
}
|
||||||
|
|
||||||
|
if (manifest.status !== 'completed') {
|
||||||
|
console.error(`[forge] run did not complete: terminal status '${manifest.status}'`);
|
||||||
|
process.exitCode = 1;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`[forge] pipeline complete (mode: normal): ${manifest.runId}`);
|
||||||
|
console.log(`[forge] run directory: ${result.runDir}`);
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Register function
|
// Register function
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -129,6 +179,11 @@ export function registerForgeCommand(parent: Command): void {
|
|||||||
.option('--config <path>', 'Path to forge config file (.forge/config.yaml)')
|
.option('--config <path>', 'Path to forge config file (.forge/config.yaml)')
|
||||||
.option('--codebase <path>', 'Codebase root to pass to the pipeline', process.cwd())
|
.option('--codebase <path>', 'Codebase root to pass to the pipeline', process.cwd())
|
||||||
.option('--dry-run', 'Print planned stages without executing', false)
|
.option('--dry-run', 'Print planned stages without executing', false)
|
||||||
|
.option(
|
||||||
|
'--simulate',
|
||||||
|
'Simulate execution without real providers (every result is typed simulated, never verified)',
|
||||||
|
false,
|
||||||
|
)
|
||||||
.action(
|
.action(
|
||||||
async (opts: {
|
async (opts: {
|
||||||
brief: string;
|
brief: string;
|
||||||
@@ -137,6 +192,7 @@ export function registerForgeCommand(parent: Command): void {
|
|||||||
config?: string;
|
config?: string;
|
||||||
codebase: string;
|
codebase: string;
|
||||||
dryRun: boolean;
|
dryRun: boolean;
|
||||||
|
simulate: boolean;
|
||||||
}) => {
|
}) => {
|
||||||
const briefPath = path.resolve(opts.brief);
|
const briefPath = path.resolve(opts.brief);
|
||||||
|
|
||||||
@@ -149,14 +205,22 @@ export function registerForgeCommand(parent: Command): void {
|
|||||||
const briefContent = fs.readFileSync(briefPath, 'utf-8');
|
const briefContent = fs.readFileSync(briefPath, 'utf-8');
|
||||||
const briefClass = classifyBrief(briefContent);
|
const briefClass = classifyBrief(briefContent);
|
||||||
const projectRoot = opts.codebase;
|
const projectRoot = opts.codebase;
|
||||||
|
// A real executor is never wired at CLI invocation time today, so the
|
||||||
|
// only executor we may construct is the explicitly-requested simulated
|
||||||
|
// one. Normal mode fails closed with FORGE_NO_EXECUTOR.
|
||||||
|
const executor = opts.simulate ? createSimulatedExecutor() : undefined;
|
||||||
|
|
||||||
if (opts.resume) {
|
if (opts.resume) {
|
||||||
const runId = opts.runId ?? generateRunId();
|
const runId = opts.runId ?? generateRunId();
|
||||||
const runDir = resolveRunDir(runId, projectRoot);
|
const runDir = resolveRunDir(runId, projectRoot);
|
||||||
console.log(`[forge] resuming run: ${runId}`);
|
console.log(`[forge] resuming run: ${runId}`);
|
||||||
const { resumePipeline } = await import('./pipeline-runner.js');
|
try {
|
||||||
const result = await resumePipeline(runDir, stubExecutor);
|
const { resumePipeline } = await import('./pipeline-runner.js');
|
||||||
console.log(`[forge] pipeline complete: ${result.runId}`);
|
const result = await resumePipeline(runDir, executor, { simulate: opts.simulate });
|
||||||
|
applyRunExitPolicy(result, opts.simulate);
|
||||||
|
} catch (err) {
|
||||||
|
handlePipelineError(err);
|
||||||
|
}
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -164,7 +228,8 @@ export function registerForgeCommand(parent: Command): void {
|
|||||||
briefClass,
|
briefClass,
|
||||||
codebase: projectRoot,
|
codebase: projectRoot,
|
||||||
dryRun: opts.dryRun,
|
dryRun: opts.dryRun,
|
||||||
executor: stubExecutor,
|
executor,
|
||||||
|
simulate: opts.simulate,
|
||||||
};
|
};
|
||||||
|
|
||||||
if (opts.dryRun) {
|
if (opts.dryRun) {
|
||||||
@@ -180,16 +245,15 @@ export function registerForgeCommand(parent: Command): void {
|
|||||||
|
|
||||||
console.log(`[forge] starting pipeline for brief: ${briefPath}`);
|
console.log(`[forge] starting pipeline for brief: ${briefPath}`);
|
||||||
console.log(`[forge] classified as: ${briefClass}`);
|
console.log(`[forge] classified as: ${briefClass}`);
|
||||||
|
if (opts.simulate) {
|
||||||
|
console.log('[forge] mode: SIMULATED (explicit --simulate)');
|
||||||
|
}
|
||||||
|
|
||||||
try {
|
try {
|
||||||
const result = await runPipeline(briefPath, projectRoot, pipelineOptions);
|
const result = await runPipeline(briefPath, projectRoot, pipelineOptions);
|
||||||
console.log(`[forge] pipeline complete: ${result.runId}`);
|
applyRunExitPolicy(result, opts.simulate);
|
||||||
console.log(`[forge] run directory: ${result.runDir}`);
|
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
console.error(
|
handlePipelineError(err);
|
||||||
`[forge] pipeline failed: ${err instanceof Error ? err.message : String(err)}`,
|
|
||||||
);
|
|
||||||
process.exitCode = 1;
|
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
);
|
);
|
||||||
@@ -224,7 +288,12 @@ export function registerForgeCommand(parent: Command): void {
|
|||||||
.command('resume <runId>')
|
.command('resume <runId>')
|
||||||
.description('Resume a stopped or failed pipeline run')
|
.description('Resume a stopped or failed pipeline run')
|
||||||
.option('--project <path>', 'Project root (defaults to cwd)', process.cwd())
|
.option('--project <path>', 'Project root (defaults to cwd)', process.cwd())
|
||||||
.action(async (runId: string, opts: { project: string }) => {
|
.option(
|
||||||
|
'--simulate',
|
||||||
|
'Simulate execution without real providers (every result is typed simulated, never verified)',
|
||||||
|
false,
|
||||||
|
)
|
||||||
|
.action(async (runId: string, opts: { project: string; simulate: boolean }) => {
|
||||||
const runDir = resolveRunDir(runId, opts.project);
|
const runDir = resolveRunDir(runId, opts.project);
|
||||||
|
|
||||||
if (!fs.existsSync(runDir)) {
|
if (!fs.existsSync(runDir)) {
|
||||||
@@ -234,15 +303,20 @@ export function registerForgeCommand(parent: Command): void {
|
|||||||
}
|
}
|
||||||
|
|
||||||
console.log(`[forge] resuming run: ${runId}`);
|
console.log(`[forge] resuming run: ${runId}`);
|
||||||
|
if (opts.simulate) {
|
||||||
|
console.log('[forge] mode: SIMULATED (explicit --simulate)');
|
||||||
|
}
|
||||||
|
|
||||||
|
// No real executor is wired at CLI invocation time; only the explicitly
|
||||||
|
// requested simulated executor may be constructed (fail closed otherwise).
|
||||||
|
const executor = opts.simulate ? createSimulatedExecutor() : undefined;
|
||||||
|
|
||||||
try {
|
try {
|
||||||
const { resumePipeline } = await import('./pipeline-runner.js');
|
const { resumePipeline } = await import('./pipeline-runner.js');
|
||||||
const result = await resumePipeline(runDir, stubExecutor);
|
const result = await resumePipeline(runDir, executor, { simulate: opts.simulate });
|
||||||
console.log(`[forge] pipeline complete: ${result.runId}`);
|
applyRunExitPolicy(result, opts.simulate);
|
||||||
console.log(`[forge] run directory: ${result.runDir}`);
|
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
console.error(`[forge] resume failed: ${err instanceof Error ? err.message : String(err)}`);
|
handlePipelineError(err);
|
||||||
process.exitCode = 1;
|
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -9,7 +9,16 @@ export const PACKAGE_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.
|
|||||||
/** Pipeline asset directory (stages, agents, rails, gates, templates). */
|
/** Pipeline asset directory (stages, agents, rails, gates, templates). */
|
||||||
export const PIPELINE_DIR = path.join(PACKAGE_ROOT, 'pipeline');
|
export const PIPELINE_DIR = path.join(PACKAGE_ROOT, 'pipeline');
|
||||||
|
|
||||||
/** Stage specifications — defines every pipeline stage. */
|
/** Stage specifications — defines every pipeline stage.
|
||||||
|
*\n * Gate semantics (SDLC-D-035): every gate is one of
|
||||||
|
* - a real command string / GateEntry a mechanical runner can execute,
|
||||||
|
* - an `authority` gate (human/board sign-off; produces waiting-for-authority),
|
||||||
|
* - a `provider` gate (requires a wired provider such as a reviewer or CI pipeline).
|
||||||
|
*
|
||||||
|
* Vacuous gates (`true`, echo'd synthetic approvals, placeholder ci-pipeline
|
||||||
|
* commands) are forbidden: a stage whose gate has no real implementation
|
||||||
|
* fails closed instead of passing.
|
||||||
|
*/
|
||||||
export const STAGE_SPECS: Record<string, StageSpec> = {
|
export const STAGE_SPECS: Record<string, StageSpec> = {
|
||||||
'00-intake': {
|
'00-intake': {
|
||||||
number: '00',
|
number: '00',
|
||||||
@@ -27,7 +36,13 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
|
|||||||
type: 'research',
|
type: 'research',
|
||||||
gate: 'discovery-complete',
|
gate: 'discovery-complete',
|
||||||
promptFile: '00b-discovery.md',
|
promptFile: '00b-discovery.md',
|
||||||
qualityGates: ['true'],
|
qualityGates: [
|
||||||
|
{
|
||||||
|
kind: 'authority',
|
||||||
|
capability: 'discovery-complete',
|
||||||
|
reason: 'discovery completion is attested by an authority; no mechanical check exists',
|
||||||
|
},
|
||||||
|
],
|
||||||
},
|
},
|
||||||
'01-board': {
|
'01-board': {
|
||||||
number: '01',
|
number: '01',
|
||||||
@@ -36,7 +51,13 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
|
|||||||
type: 'review',
|
type: 'review',
|
||||||
gate: 'board-approval',
|
gate: 'board-approval',
|
||||||
promptFile: '01-board.md',
|
promptFile: '01-board.md',
|
||||||
qualityGates: [{ type: 'ci-pipeline', command: 'board-approval (via board-tasks)' }],
|
qualityGates: [
|
||||||
|
{
|
||||||
|
kind: 'authority',
|
||||||
|
capability: 'board-approval',
|
||||||
|
reason: 'board approval is a board/human decision; no mechanical gate exists',
|
||||||
|
},
|
||||||
|
],
|
||||||
},
|
},
|
||||||
'01b-brief-analyzer': {
|
'01b-brief-analyzer': {
|
||||||
number: '01b',
|
number: '01b',
|
||||||
@@ -45,7 +66,13 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
|
|||||||
type: 'research',
|
type: 'research',
|
||||||
gate: 'brief-analysis-complete',
|
gate: 'brief-analysis-complete',
|
||||||
promptFile: '01-board.md',
|
promptFile: '01-board.md',
|
||||||
qualityGates: ['true'],
|
qualityGates: [
|
||||||
|
{
|
||||||
|
kind: 'authority',
|
||||||
|
capability: 'brief-analysis-complete',
|
||||||
|
reason: 'brief analysis completion is attested by an authority; no mechanical check exists',
|
||||||
|
},
|
||||||
|
],
|
||||||
},
|
},
|
||||||
'02-planning-1': {
|
'02-planning-1': {
|
||||||
number: '02',
|
number: '02',
|
||||||
@@ -54,7 +81,13 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
|
|||||||
type: 'research',
|
type: 'research',
|
||||||
gate: 'architecture-approval',
|
gate: 'architecture-approval',
|
||||||
promptFile: '02-planning-1-architecture.md',
|
promptFile: '02-planning-1-architecture.md',
|
||||||
qualityGates: ['true'],
|
qualityGates: [
|
||||||
|
{
|
||||||
|
kind: 'authority',
|
||||||
|
capability: 'architecture-approval',
|
||||||
|
reason: 'ADR approval requires authority sign-off; no mechanical check exists',
|
||||||
|
},
|
||||||
|
],
|
||||||
},
|
},
|
||||||
'03-planning-2': {
|
'03-planning-2': {
|
||||||
number: '03',
|
number: '03',
|
||||||
@@ -63,7 +96,14 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
|
|||||||
type: 'research',
|
type: 'research',
|
||||||
gate: 'implementation-approval',
|
gate: 'implementation-approval',
|
||||||
promptFile: '03-planning-2-implementation.md',
|
promptFile: '03-planning-2-implementation.md',
|
||||||
qualityGates: ['true'],
|
qualityGates: [
|
||||||
|
{
|
||||||
|
kind: 'authority',
|
||||||
|
capability: 'implementation-approval',
|
||||||
|
reason:
|
||||||
|
'implementation spec approval requires authority sign-off; no mechanical check exists',
|
||||||
|
},
|
||||||
|
],
|
||||||
},
|
},
|
||||||
'04-planning-3': {
|
'04-planning-3': {
|
||||||
number: '04',
|
number: '04',
|
||||||
@@ -72,7 +112,14 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
|
|||||||
type: 'research',
|
type: 'research',
|
||||||
gate: 'decomposition-approval',
|
gate: 'decomposition-approval',
|
||||||
promptFile: '04-planning-3-decomposition.md',
|
promptFile: '04-planning-3-decomposition.md',
|
||||||
qualityGates: ['true'],
|
qualityGates: [
|
||||||
|
{
|
||||||
|
kind: 'authority',
|
||||||
|
capability: 'decomposition-approval',
|
||||||
|
reason:
|
||||||
|
'task decomposition approval requires authority sign-off; no mechanical check exists',
|
||||||
|
},
|
||||||
|
],
|
||||||
},
|
},
|
||||||
'05-coding': {
|
'05-coding': {
|
||||||
number: '05',
|
number: '05',
|
||||||
@@ -92,9 +139,10 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
|
|||||||
promptFile: '06-review.md',
|
promptFile: '06-review.md',
|
||||||
qualityGates: [
|
qualityGates: [
|
||||||
{
|
{
|
||||||
type: 'ai-review',
|
kind: 'provider',
|
||||||
command:
|
capability: 'reviewer',
|
||||||
'echo \'{"summary":"review-pass","verdict":"approve","findings":[],"stats":{"blockers":0,"should_fix":0,"suggestions":0}}\'',
|
reason:
|
||||||
|
'review verdicts require a wired reviewer provider; synthetic approvals are not permitted',
|
||||||
},
|
},
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
@@ -105,7 +153,13 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
|
|||||||
type: 'coding',
|
type: 'coding',
|
||||||
gate: 're-review',
|
gate: 're-review',
|
||||||
promptFile: '07-remediate.md',
|
promptFile: '07-remediate.md',
|
||||||
qualityGates: ['true'],
|
qualityGates: [
|
||||||
|
{
|
||||||
|
kind: 'authority',
|
||||||
|
capability: 're-review',
|
||||||
|
reason: 'remediation re-review is an approval-based gate; no mechanical check exists',
|
||||||
|
},
|
||||||
|
],
|
||||||
},
|
},
|
||||||
'08-test': {
|
'08-test': {
|
||||||
number: '08',
|
number: '08',
|
||||||
@@ -123,7 +177,13 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
|
|||||||
type: 'deploy',
|
type: 'deploy',
|
||||||
gate: 'deploy-verification',
|
gate: 'deploy-verification',
|
||||||
promptFile: '09-deploy.md',
|
promptFile: '09-deploy.md',
|
||||||
qualityGates: [{ type: 'ci-pipeline', command: 'deploy-verification' }],
|
qualityGates: [
|
||||||
|
{
|
||||||
|
kind: 'provider',
|
||||||
|
capability: 'ci-pipeline',
|
||||||
|
reason: 'deploy verification requires a wired CI pipeline provider',
|
||||||
|
},
|
||||||
|
],
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,46 @@
|
|||||||
|
/**
|
||||||
|
* Typed fail-closed capability errors (SDLC-D-035).
|
||||||
|
*
|
||||||
|
* A Forge run must fail closed when a required capability (executor, reviewer
|
||||||
|
* provider, CI pipeline, authority sign-off) is missing. These typed errors
|
||||||
|
* name the missing capability so callers can distinguish "not wired" from
|
||||||
|
* ordinary execution failures.
|
||||||
|
*/
|
||||||
|
|
||||||
|
/** Closed set of typed Forge capability error codes. */
|
||||||
|
export const FORGE_ERROR_CODES = [
|
||||||
|
'FORGE_NO_EXECUTOR',
|
||||||
|
'FORGE_NO_REVIEWER',
|
||||||
|
'FORGE_NO_CI_PIPELINE',
|
||||||
|
'FORGE_NO_PROVIDER',
|
||||||
|
'FORGE_AUTHORITY_REQUIRED',
|
||||||
|
] as const;
|
||||||
|
|
||||||
|
export type ForgeErrorCode = (typeof FORGE_ERROR_CODES)[number];
|
||||||
|
|
||||||
|
/** Raised when a required capability is missing and the pipeline must fail closed. */
|
||||||
|
export class ForgeCapabilityError extends Error {
|
||||||
|
/** Typed error code from the closed FORGE_ERROR_CODES set. */
|
||||||
|
readonly code: ForgeErrorCode;
|
||||||
|
/** The missing capability, e.g. `task-executor`, `reviewer`, `board-approval`. */
|
||||||
|
readonly capability: string;
|
||||||
|
|
||||||
|
constructor(code: ForgeErrorCode, capability: string, message: string) {
|
||||||
|
super(message);
|
||||||
|
this.name = 'ForgeCapabilityError';
|
||||||
|
this.code = code;
|
||||||
|
this.capability = capability;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Map a provider gate capability to its typed error code. */
|
||||||
|
export function providerErrorCode(capability: string): ForgeErrorCode {
|
||||||
|
switch (capability) {
|
||||||
|
case 'reviewer':
|
||||||
|
return 'FORGE_NO_REVIEWER';
|
||||||
|
case 'ci-pipeline':
|
||||||
|
return 'FORGE_NO_CI_PIPELINE';
|
||||||
|
default:
|
||||||
|
return 'FORGE_NO_PROVIDER';
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -5,6 +5,13 @@ export type {
|
|||||||
StageSpec,
|
StageSpec,
|
||||||
BriefClass,
|
BriefClass,
|
||||||
ClassSource,
|
ClassSource,
|
||||||
|
ForgeOutcome,
|
||||||
|
AuthorityGate,
|
||||||
|
ProviderGate,
|
||||||
|
ForgeGate,
|
||||||
|
ForgeGateResult,
|
||||||
|
ForgeTaskResult,
|
||||||
|
RunMode,
|
||||||
StageStatus,
|
StageStatus,
|
||||||
RunManifest,
|
RunManifest,
|
||||||
ForgeTaskStatus,
|
ForgeTaskStatus,
|
||||||
@@ -81,5 +88,24 @@ export {
|
|||||||
getPipelineStatus,
|
getPipelineStatus,
|
||||||
} from './pipeline-runner.js';
|
} from './pipeline-runner.js';
|
||||||
|
|
||||||
|
// Fail-closed errors and typed outcome model (SDLC-D-035)
|
||||||
|
export { FORGE_ERROR_CODES, ForgeCapabilityError, providerErrorCode } from './errors.js';
|
||||||
|
export type { ForgeErrorCode } from './errors.js';
|
||||||
|
export {
|
||||||
|
isSatisfyingOutcome,
|
||||||
|
isCapabilityGate,
|
||||||
|
isCommandGate,
|
||||||
|
gateLabel,
|
||||||
|
uniformGateResults,
|
||||||
|
simulatedGateResults,
|
||||||
|
waitingGateResults,
|
||||||
|
blockedGateResults,
|
||||||
|
evaluateStageGates,
|
||||||
|
} from './outcomes.js';
|
||||||
|
export type { StageEvaluation } from './outcomes.js';
|
||||||
|
|
||||||
|
// Simulated executor (explicit --simulate only)
|
||||||
|
export { createSimulatedExecutor } from './simulated-executor.js';
|
||||||
|
|
||||||
// CLI
|
// CLI
|
||||||
export { registerForgeCommand } from './cli.js';
|
export { registerForgeCommand } from './cli.js';
|
||||||
|
|||||||
@@ -0,0 +1,147 @@
|
|||||||
|
import type { GateEntry } from '@mosaicstack/macp';
|
||||||
|
|
||||||
|
import type {
|
||||||
|
AuthorityGate,
|
||||||
|
ForgeGate,
|
||||||
|
ForgeGateResult,
|
||||||
|
ForgeOutcome,
|
||||||
|
ForgeTaskResult,
|
||||||
|
ProviderGate,
|
||||||
|
} from './types.js';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Gate and dependency satisfaction predicate (SDLC-D-035).
|
||||||
|
*
|
||||||
|
* ONLY a verified `passed` outcome satisfies. Every other member of the closed
|
||||||
|
* outcome set — including `simulated` — is non-satisfying, so a simulated or
|
||||||
|
* authority-blocked result can never be read as success-by-verification.
|
||||||
|
*/
|
||||||
|
export function isSatisfyingOutcome(outcome: ForgeOutcome): boolean {
|
||||||
|
return outcome === 'passed';
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Whether a gate is an authority or provider gate (capability-based, command-less). */
|
||||||
|
export function isCapabilityGate(gate: ForgeGate): gate is AuthorityGate | ProviderGate {
|
||||||
|
if (typeof gate !== 'object' || gate === null) return false;
|
||||||
|
const kind = (gate as Record<string, unknown>)['kind'];
|
||||||
|
return kind === 'authority' || kind === 'provider';
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Whether a gate definition carries a real command a mechanical runner can execute. */
|
||||||
|
export function isCommandGate(gate: ForgeGate): gate is string | GateEntry {
|
||||||
|
if (typeof gate === 'string') {
|
||||||
|
return gate.trim().length > 0;
|
||||||
|
}
|
||||||
|
if (isCapabilityGate(gate)) {
|
||||||
|
// Authority and provider gates are satisfied by a capability, not a command.
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return typeof gate.command === 'string' && gate.command.trim().length > 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Typed label identifying a gate in results and logs. */
|
||||||
|
export function gateLabel(gate: ForgeGate): string {
|
||||||
|
if (typeof gate === 'string') return gate;
|
||||||
|
if (isCapabilityGate(gate)) return `${gate.kind}:${gate.capability}`;
|
||||||
|
return gate.command || gate.type || 'unnamed-gate';
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Reason string stamped on every simulated gate result. */
|
||||||
|
export const SIMULATED_GATE_REASON =
|
||||||
|
'simulated execution (--simulate): gate was not evaluated by a real implementation';
|
||||||
|
|
||||||
|
/** Build typed gate results with a uniform outcome for a stage's declared gates. */
|
||||||
|
export function uniformGateResults(
|
||||||
|
gates: ForgeGate[],
|
||||||
|
outcome: ForgeOutcome,
|
||||||
|
reason: string,
|
||||||
|
): ForgeGateResult[] {
|
||||||
|
return gates.map((gate) => ({ gate: gateLabel(gate), outcome, reason }));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Typed simulated gate results — used exclusively in `--simulate` runs. */
|
||||||
|
export function simulatedGateResults(gates: ForgeGate[]): ForgeGateResult[] {
|
||||||
|
return uniformGateResults(gates, 'simulated', SIMULATED_GATE_REASON);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Typed waiting-for-authority gate results for approval-based stages. */
|
||||||
|
export function waitingGateResults(gates: ForgeGate[], reason: string): ForgeGateResult[] {
|
||||||
|
return uniformGateResults(gates, 'waiting-for-authority', reason);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Typed blocked gate results for stages whose provider capability is not wired. */
|
||||||
|
export function blockedGateResults(gates: ForgeGate[], reason: string): ForgeGateResult[] {
|
||||||
|
return uniformGateResults(gates, 'blocked', reason);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Outcome of evaluating a completed stage in normal mode. */
|
||||||
|
export interface StageEvaluation {
|
||||||
|
outcome: ForgeOutcome;
|
||||||
|
reason: string;
|
||||||
|
gateResults: ForgeGateResult[];
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Evaluate a stage's declared gates against the executor's typed result.
|
||||||
|
*
|
||||||
|
* Fail-closed mapping:
|
||||||
|
* - a `simulated` task or gate outcome in normal mode maps to `error`
|
||||||
|
* - a missing gate result for a required command gate maps to `blocked`
|
||||||
|
* - a non-passing task outcome propagates as the stage outcome
|
||||||
|
* - only verified `passed` task and gate outcomes yield a `passed` stage
|
||||||
|
*/
|
||||||
|
export function evaluateStageGates(
|
||||||
|
stageName: string,
|
||||||
|
gates: ForgeGate[],
|
||||||
|
result: ForgeTaskResult,
|
||||||
|
): StageEvaluation {
|
||||||
|
const gateResults = result.gate_results ?? [];
|
||||||
|
|
||||||
|
if (result.outcome === 'simulated') {
|
||||||
|
return {
|
||||||
|
outcome: 'error',
|
||||||
|
reason: `executor reported a simulated outcome for stage '${stageName}' in normal mode — refusing to treat simulated results as verified`,
|
||||||
|
gateResults,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!isSatisfyingOutcome(result.outcome)) {
|
||||||
|
return {
|
||||||
|
outcome: result.outcome,
|
||||||
|
reason: `task outcome is '${result.outcome}': ${result.reason}`,
|
||||||
|
gateResults,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const gate of gates) {
|
||||||
|
// Authority and provider gates are pre-flighted before execution; they have
|
||||||
|
// no mechanical result to verify here.
|
||||||
|
if (!isCommandGate(gate)) continue;
|
||||||
|
|
||||||
|
const label = gateLabel(gate);
|
||||||
|
const gateResult = gateResults.find((r) => r.gate === label);
|
||||||
|
if (!gateResult) {
|
||||||
|
return {
|
||||||
|
outcome: 'blocked',
|
||||||
|
reason: `no gate result was reported for required gate '${label}' (stage '${stageName}')`,
|
||||||
|
gateResults,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
if (!isSatisfyingOutcome(gateResult.outcome)) {
|
||||||
|
return {
|
||||||
|
outcome: gateResult.outcome === 'simulated' ? 'error' : gateResult.outcome,
|
||||||
|
reason: `gate '${label}' outcome is '${gateResult.outcome}': ${gateResult.reason}`,
|
||||||
|
gateResults,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
outcome: 'passed',
|
||||||
|
reason:
|
||||||
|
gates.length === 0
|
||||||
|
? "stage declares no gates; task outcome 'passed' accepted"
|
||||||
|
: 'all declared gates verified passed',
|
||||||
|
gateResults,
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -1,18 +1,33 @@
|
|||||||
import fs from 'node:fs';
|
import fs from 'node:fs';
|
||||||
import path from 'node:path';
|
import path from 'node:path';
|
||||||
|
|
||||||
import { STAGE_SEQUENCE } from './constants.js';
|
import { STAGE_SEQUENCE, STAGE_SPECS } from './constants.js';
|
||||||
import { determineBriefClass, stagesForClass } from './brief-classifier.js';
|
import { determineBriefClass, stagesForClass } from './brief-classifier.js';
|
||||||
|
import { ForgeCapabilityError, providerErrorCode } from './errors.js';
|
||||||
|
import {
|
||||||
|
blockedGateResults,
|
||||||
|
evaluateStageGates,
|
||||||
|
isCapabilityGate,
|
||||||
|
simulatedGateResults,
|
||||||
|
waitingGateResults,
|
||||||
|
} from './outcomes.js';
|
||||||
import { mapStageToTask } from './stage-adapter.js';
|
import { mapStageToTask } from './stage-adapter.js';
|
||||||
|
import { createSimulatedExecutor } from './simulated-executor.js';
|
||||||
import type {
|
import type {
|
||||||
ForgeTask,
|
ForgeTask,
|
||||||
|
ForgeTaskResult,
|
||||||
PipelineOptions,
|
PipelineOptions,
|
||||||
PipelineResult,
|
PipelineResult,
|
||||||
RunManifest,
|
RunManifest,
|
||||||
|
RunMode,
|
||||||
StageStatus,
|
StageStatus,
|
||||||
TaskExecutor,
|
TaskExecutor,
|
||||||
} from './types.js';
|
} from './types.js';
|
||||||
|
|
||||||
|
/** Reason stamped on stages that complete under explicit simulation. */
|
||||||
|
const SIMULATED_STAGE_REASON =
|
||||||
|
'simulated execution (--simulate): stage was not executed by a real executor';
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Generate a timestamp-based run ID.
|
* Generate a timestamp-based run ID.
|
||||||
*/
|
*/
|
||||||
@@ -47,6 +62,7 @@ function createManifest(opts: {
|
|||||||
briefClass: RunManifest['briefClass'];
|
briefClass: RunManifest['briefClass'];
|
||||||
classSource: RunManifest['classSource'];
|
classSource: RunManifest['classSource'];
|
||||||
forceBoard: boolean;
|
forceBoard: boolean;
|
||||||
|
mode: RunMode;
|
||||||
runDir: string;
|
runDir: string;
|
||||||
}): RunManifest {
|
}): RunManifest {
|
||||||
const ts = nowISO();
|
const ts = nowISO();
|
||||||
@@ -57,6 +73,7 @@ function createManifest(opts: {
|
|||||||
briefClass: opts.briefClass,
|
briefClass: opts.briefClass,
|
||||||
classSource: opts.classSource,
|
classSource: opts.classSource,
|
||||||
forceBoard: opts.forceBoard,
|
forceBoard: opts.forceBoard,
|
||||||
|
mode: opts.mode,
|
||||||
createdAt: ts,
|
createdAt: ts,
|
||||||
updatedAt: ts,
|
updatedAt: ts,
|
||||||
currentStage: '',
|
currentStage: '',
|
||||||
@@ -108,20 +125,199 @@ export function selectStages(stages?: string[], skipTo?: string): string[] {
|
|||||||
return selected.slice(skipIndex);
|
return selected.slice(skipIndex);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fail closed when the required executor capability is missing (SDLC-D-035).
|
||||||
|
*/
|
||||||
|
function requireExecutor(executor: TaskExecutor | undefined, simulate: boolean): TaskExecutor {
|
||||||
|
if (executor) return executor;
|
||||||
|
if (simulate) return createSimulatedExecutor({ log: false });
|
||||||
|
throw new ForgeCapabilityError(
|
||||||
|
'FORGE_NO_EXECUTOR',
|
||||||
|
'task-executor',
|
||||||
|
'no task executor is wired; refusing to run the pipeline with a stub executor (fail closed). ' +
|
||||||
|
'Pass --simulate to opt into explicitly simulated execution.',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Pre-flight a stage's gates in normal mode (fail closed, SDLC-D-035).
|
||||||
|
*
|
||||||
|
* - authority gates: record a typed `waiting-for-authority` stage result and
|
||||||
|
* raise FORGE_AUTHORITY_REQUIRED — approval-based gates never pass vacuously.
|
||||||
|
* - provider gates: record a typed `blocked` stage result and raise the typed
|
||||||
|
* capability error for the missing provider.
|
||||||
|
*
|
||||||
|
* Returns the stage status to record when the pre-flight blocks, or undefined
|
||||||
|
* when the stage may proceed.
|
||||||
|
*/
|
||||||
|
function preflightStageGates(
|
||||||
|
stageName: string,
|
||||||
|
manifest: RunManifest,
|
||||||
|
): { status: StageStatus; error: ForgeCapabilityError } | undefined {
|
||||||
|
const spec = STAGE_SPECS[stageName];
|
||||||
|
if (!spec) throw new Error(`Unknown Forge stage: ${stageName}`);
|
||||||
|
|
||||||
|
for (const gate of spec.qualityGates) {
|
||||||
|
if (!isCapabilityGate(gate)) continue;
|
||||||
|
|
||||||
|
const startedAt = manifest.stages[stageName]?.startedAt;
|
||||||
|
const completedAt = nowISO();
|
||||||
|
|
||||||
|
if (gate.kind === 'authority') {
|
||||||
|
const reason = `gate '${gate.capability}' requires authority sign-off; no mechanical implementation exists (${gate.reason})`;
|
||||||
|
return {
|
||||||
|
status: {
|
||||||
|
status: 'waiting-for-authority',
|
||||||
|
reason,
|
||||||
|
startedAt,
|
||||||
|
completedAt,
|
||||||
|
gateResults: waitingGateResults(spec.qualityGates, reason),
|
||||||
|
},
|
||||||
|
error: new ForgeCapabilityError(
|
||||||
|
'FORGE_AUTHORITY_REQUIRED',
|
||||||
|
gate.capability,
|
||||||
|
`stage '${stageName}' is blocked on authority gate '${gate.capability}': ${gate.reason}. ` +
|
||||||
|
'The pipeline fails closed instead of passing vacuously. Record the approval out-of-band ' +
|
||||||
|
'or run with --simulate for explicitly simulated execution.',
|
||||||
|
),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const reason = `gate '${gate.capability}' requires provider '${gate.capability}' and none is wired (${gate.reason})`;
|
||||||
|
return {
|
||||||
|
status: {
|
||||||
|
status: 'blocked',
|
||||||
|
reason,
|
||||||
|
startedAt,
|
||||||
|
completedAt,
|
||||||
|
gateResults: blockedGateResults(spec.qualityGates, reason),
|
||||||
|
},
|
||||||
|
error: new ForgeCapabilityError(
|
||||||
|
providerErrorCode(gate.capability),
|
||||||
|
gate.capability,
|
||||||
|
`stage '${stageName}' requires provider '${gate.capability}' which is not wired: ${gate.reason}. ` +
|
||||||
|
'The pipeline fails closed instead of passing vacuously.',
|
||||||
|
),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Execute the given stage tasks sequentially, updating the manifest.
|
||||||
|
*
|
||||||
|
* Normal mode requires a real executor and evaluates every declared command
|
||||||
|
* gate through the typed outcome model; any non-verified result fails closed.
|
||||||
|
* Simulate mode types every stage and gate result as `simulated`.
|
||||||
|
*/
|
||||||
|
async function executeStages(opts: {
|
||||||
|
manifest: RunManifest;
|
||||||
|
runDir: string;
|
||||||
|
tasks: ForgeTask[];
|
||||||
|
stageNames: string[];
|
||||||
|
executor: TaskExecutor;
|
||||||
|
simulate: boolean;
|
||||||
|
}): Promise<void> {
|
||||||
|
const { manifest, runDir, tasks, stageNames, executor, simulate } = opts;
|
||||||
|
|
||||||
|
for (let i = 0; i < tasks.length; i++) {
|
||||||
|
const task = tasks[i]!;
|
||||||
|
const stageName = stageNames[i]!;
|
||||||
|
const spec = STAGE_SPECS[stageName];
|
||||||
|
if (!spec) throw new Error(`Unknown Forge stage: ${stageName}`);
|
||||||
|
|
||||||
|
// Update manifest: stage in progress
|
||||||
|
manifest.currentStage = stageName;
|
||||||
|
manifest.stages[stageName] = {
|
||||||
|
status: 'in_progress',
|
||||||
|
startedAt: nowISO(),
|
||||||
|
};
|
||||||
|
saveManifest(runDir, manifest);
|
||||||
|
|
||||||
|
// Fail-closed pre-flight (normal mode only): authority/provider gates have
|
||||||
|
// no mechanical implementation and must never pass vacuously.
|
||||||
|
if (!simulate) {
|
||||||
|
const blocked = preflightStageGates(stageName, manifest);
|
||||||
|
if (blocked) {
|
||||||
|
manifest.stages[stageName] = blocked.status;
|
||||||
|
manifest.status =
|
||||||
|
blocked.status.status === 'waiting-for-authority' ? 'waiting-for-authority' : 'failed';
|
||||||
|
saveManifest(runDir, manifest);
|
||||||
|
throw blocked.error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let result: ForgeTaskResult;
|
||||||
|
try {
|
||||||
|
await executor.submitTask(task);
|
||||||
|
result = await executor.waitForCompletion(task.id, task.timeoutSeconds * 1000);
|
||||||
|
} catch (error) {
|
||||||
|
// Process errors (including timeouts) map to the fail-closed `error` outcome.
|
||||||
|
const reason = error instanceof Error ? error.message : String(error);
|
||||||
|
manifest.stages[stageName] = {
|
||||||
|
status: 'error',
|
||||||
|
reason: `executor error: ${reason}`,
|
||||||
|
startedAt: manifest.stages[stageName]?.startedAt,
|
||||||
|
completedAt: nowISO(),
|
||||||
|
gateResults: [],
|
||||||
|
};
|
||||||
|
manifest.status = 'failed';
|
||||||
|
saveManifest(runDir, manifest);
|
||||||
|
throw error instanceof Error ? error : new Error(reason);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (simulate) {
|
||||||
|
manifest.stages[stageName] = {
|
||||||
|
status: 'simulated',
|
||||||
|
reason: SIMULATED_STAGE_REASON,
|
||||||
|
startedAt: manifest.stages[stageName]?.startedAt,
|
||||||
|
completedAt: nowISO(),
|
||||||
|
gateResults: simulatedGateResults(spec.qualityGates),
|
||||||
|
};
|
||||||
|
saveManifest(runDir, manifest);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
const evaluation = evaluateStageGates(stageName, spec.qualityGates, result);
|
||||||
|
manifest.stages[stageName] = {
|
||||||
|
status: evaluation.outcome,
|
||||||
|
reason: evaluation.reason,
|
||||||
|
startedAt: manifest.stages[stageName]?.startedAt,
|
||||||
|
completedAt: nowISO(),
|
||||||
|
gateResults: evaluation.gateResults,
|
||||||
|
};
|
||||||
|
|
||||||
|
if (evaluation.outcome !== 'passed') {
|
||||||
|
manifest.status =
|
||||||
|
evaluation.outcome === 'waiting-for-authority' ? 'waiting-for-authority' : 'failed';
|
||||||
|
saveManifest(runDir, manifest);
|
||||||
|
throw new Error(`Stage ${stageName} ${evaluation.outcome}: ${evaluation.reason}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
saveManifest(runDir, manifest);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Run the Forge pipeline.
|
* Run the Forge pipeline.
|
||||||
*
|
*
|
||||||
* 1. Classify the brief
|
* 1. Fail closed unless a real executor is wired or simulation is explicit
|
||||||
* 2. Generate a run ID and create run directory
|
* 2. Classify the brief
|
||||||
* 3. Map stages to tasks and submit to TaskExecutor
|
* 3. Generate a run ID and create run directory
|
||||||
* 4. Track manifest with stage statuses
|
* 4. Map stages to tasks and submit to TaskExecutor
|
||||||
* 5. Return pipeline result
|
* 5. Track manifest with typed stage outcomes
|
||||||
|
* 6. Return pipeline result
|
||||||
*/
|
*/
|
||||||
export async function runPipeline(
|
export async function runPipeline(
|
||||||
briefPath: string,
|
briefPath: string,
|
||||||
projectRoot: string,
|
projectRoot: string,
|
||||||
options: PipelineOptions,
|
options: PipelineOptions,
|
||||||
): Promise<PipelineResult> {
|
): Promise<PipelineResult> {
|
||||||
|
const simulate = options.simulate ?? false;
|
||||||
|
const executor = requireExecutor(options.executor, simulate);
|
||||||
|
const mode: RunMode = simulate ? 'simulated' : 'normal';
|
||||||
|
|
||||||
const resolvedRoot = path.resolve(projectRoot);
|
const resolvedRoot = path.resolve(projectRoot);
|
||||||
const resolvedBrief = path.resolve(briefPath);
|
const resolvedBrief = path.resolve(briefPath);
|
||||||
const briefContent = fs.readFileSync(resolvedBrief, 'utf-8');
|
const briefContent = fs.readFileSync(resolvedBrief, 'utf-8');
|
||||||
@@ -146,6 +342,7 @@ export async function runPipeline(
|
|||||||
briefClass,
|
briefClass,
|
||||||
classSource,
|
classSource,
|
||||||
forceBoard: options.forceBoard ?? false,
|
forceBoard: options.forceBoard ?? false,
|
||||||
|
mode,
|
||||||
runDir,
|
runDir,
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -172,54 +369,10 @@ export async function runPipeline(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Execute stages
|
// Execute stages
|
||||||
const { executor } = options;
|
await executeStages({ manifest, runDir, tasks, stageNames: selectedStages, executor, simulate });
|
||||||
for (let i = 0; i < tasks.length; i++) {
|
|
||||||
const task = tasks[i]!;
|
|
||||||
const stageName = selectedStages[i]!;
|
|
||||||
|
|
||||||
// Update manifest: stage in progress
|
// All stages reached a terminal state for this mode
|
||||||
manifest.currentStage = stageName;
|
manifest.status = simulate ? 'simulated' : 'completed';
|
||||||
manifest.stages[stageName] = {
|
|
||||||
status: 'in_progress',
|
|
||||||
startedAt: nowISO(),
|
|
||||||
};
|
|
||||||
saveManifest(runDir, manifest);
|
|
||||||
|
|
||||||
try {
|
|
||||||
await executor.submitTask(task);
|
|
||||||
const result = await executor.waitForCompletion(task.id, task.timeoutSeconds * 1000);
|
|
||||||
|
|
||||||
// Update manifest: stage completed or failed
|
|
||||||
const stageStatus: StageStatus = {
|
|
||||||
status: result.status === 'completed' ? 'passed' : 'failed',
|
|
||||||
startedAt: manifest.stages[stageName]!.startedAt,
|
|
||||||
completedAt: nowISO(),
|
|
||||||
};
|
|
||||||
manifest.stages[stageName] = stageStatus;
|
|
||||||
|
|
||||||
if (result.status !== 'completed') {
|
|
||||||
manifest.status = 'failed';
|
|
||||||
saveManifest(runDir, manifest);
|
|
||||||
throw new Error(`Stage ${stageName} failed with status: ${result.status}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
saveManifest(runDir, manifest);
|
|
||||||
} catch (error) {
|
|
||||||
if (!manifest.stages[stageName]?.completedAt) {
|
|
||||||
manifest.stages[stageName] = {
|
|
||||||
status: 'failed',
|
|
||||||
startedAt: manifest.stages[stageName]?.startedAt,
|
|
||||||
completedAt: nowISO(),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
manifest.status = 'failed';
|
|
||||||
saveManifest(runDir, manifest);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// All stages passed
|
|
||||||
manifest.status = 'completed';
|
|
||||||
saveManifest(runDir, manifest);
|
saveManifest(runDir, manifest);
|
||||||
|
|
||||||
return {
|
return {
|
||||||
@@ -234,22 +387,30 @@ export async function runPipeline(
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Resume a pipeline from the last incomplete stage.
|
* Resume a pipeline from the last non-passed stage.
|
||||||
*/
|
*/
|
||||||
export async function resumePipeline(
|
export async function resumePipeline(
|
||||||
runDir: string,
|
runDir: string,
|
||||||
executor: TaskExecutor,
|
executor?: TaskExecutor,
|
||||||
|
options?: { simulate?: boolean },
|
||||||
): Promise<PipelineResult> {
|
): Promise<PipelineResult> {
|
||||||
|
const simulate = options?.simulate ?? false;
|
||||||
|
const wiredExecutor = requireExecutor(executor, simulate);
|
||||||
|
const mode: RunMode = simulate ? 'simulated' : 'normal';
|
||||||
|
|
||||||
const manifest = loadManifest(runDir);
|
const manifest = loadManifest(runDir);
|
||||||
const resolvedRoot = path.dirname(path.dirname(path.dirname(runDir))); // .forge/runs/{id} → project root
|
const resolvedRoot = path.dirname(path.dirname(path.dirname(runDir))); // .forge/runs/{id} → project root
|
||||||
|
|
||||||
const briefContent = fs.readFileSync(manifest.brief, 'utf-8');
|
const briefContent = fs.readFileSync(manifest.brief, 'utf-8');
|
||||||
const allStages = stagesForClass(manifest.briefClass, manifest.forceBoard);
|
const allStages = stagesForClass(manifest.briefClass, manifest.forceBoard);
|
||||||
|
|
||||||
// Find first non-passed stage
|
manifest.mode = mode;
|
||||||
|
|
||||||
|
// Find first non-satisfying stage (only a verified `passed` counts as done;
|
||||||
|
// simulated and waiting-for-authority stages are re-run).
|
||||||
const resumeFrom = allStages.find((s) => manifest.stages[s]?.status !== 'passed');
|
const resumeFrom = allStages.find((s) => manifest.stages[s]?.status !== 'passed');
|
||||||
if (!resumeFrom) {
|
if (!resumeFrom) {
|
||||||
manifest.status = 'completed';
|
manifest.status = mode === 'simulated' ? 'simulated' : 'completed';
|
||||||
saveManifest(runDir, manifest);
|
saveManifest(runDir, manifest);
|
||||||
return {
|
return {
|
||||||
runId: manifest.runId,
|
runId: manifest.runId,
|
||||||
@@ -284,49 +445,16 @@ export async function resumePipeline(
|
|||||||
tasks.push(task);
|
tasks.push(task);
|
||||||
}
|
}
|
||||||
|
|
||||||
for (let i = 0; i < tasks.length; i++) {
|
await executeStages({
|
||||||
const task = tasks[i]!;
|
manifest,
|
||||||
const stageName = remainingStages[i]!;
|
runDir,
|
||||||
|
tasks,
|
||||||
|
stageNames: remainingStages,
|
||||||
|
executor: wiredExecutor,
|
||||||
|
simulate,
|
||||||
|
});
|
||||||
|
|
||||||
manifest.currentStage = stageName;
|
manifest.status = simulate ? 'simulated' : 'completed';
|
||||||
manifest.stages[stageName] = {
|
|
||||||
status: 'in_progress',
|
|
||||||
startedAt: nowISO(),
|
|
||||||
};
|
|
||||||
saveManifest(runDir, manifest);
|
|
||||||
|
|
||||||
try {
|
|
||||||
await executor.submitTask(task);
|
|
||||||
const result = await executor.waitForCompletion(task.id, task.timeoutSeconds * 1000);
|
|
||||||
|
|
||||||
manifest.stages[stageName] = {
|
|
||||||
status: result.status === 'completed' ? 'passed' : 'failed',
|
|
||||||
startedAt: manifest.stages[stageName]!.startedAt,
|
|
||||||
completedAt: nowISO(),
|
|
||||||
};
|
|
||||||
|
|
||||||
if (result.status !== 'completed') {
|
|
||||||
manifest.status = 'failed';
|
|
||||||
saveManifest(runDir, manifest);
|
|
||||||
throw new Error(`Stage ${stageName} failed with status: ${result.status}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
saveManifest(runDir, manifest);
|
|
||||||
} catch (error) {
|
|
||||||
if (!manifest.stages[stageName]?.completedAt) {
|
|
||||||
manifest.stages[stageName] = {
|
|
||||||
status: 'failed',
|
|
||||||
startedAt: manifest.stages[stageName]?.startedAt,
|
|
||||||
completedAt: nowISO(),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
manifest.status = 'failed';
|
|
||||||
saveManifest(runDir, manifest);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
manifest.status = 'completed';
|
|
||||||
saveManifest(runDir, manifest);
|
saveManifest(runDir, manifest);
|
||||||
|
|
||||||
return {
|
return {
|
||||||
|
|||||||
@@ -0,0 +1,32 @@
|
|||||||
|
import type { ForgeTask, ForgeTaskResult, TaskExecutor } from './types.js';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Simulated executor — used ONLY when the caller explicitly passes --simulate.
|
||||||
|
*
|
||||||
|
* It submits no real work and returns typed `simulated` results so a simulated
|
||||||
|
* run can never be confused with a verified one. In normal mode (no --simulate)
|
||||||
|
* the CLI refuses to run at all with FORGE_NO_EXECUTOR instead of wiring this
|
||||||
|
* stub (fail closed, SDLC-D-035).
|
||||||
|
*/
|
||||||
|
export function createSimulatedExecutor(options?: { log?: boolean }): TaskExecutor {
|
||||||
|
const log = options?.log ?? true;
|
||||||
|
return {
|
||||||
|
async submitTask(task: ForgeTask) {
|
||||||
|
if (log) console.log(` [forge:simulated] stage submitted: ${task.id} (${task.title})`);
|
||||||
|
},
|
||||||
|
async waitForCompletion(taskId: string): Promise<ForgeTaskResult> {
|
||||||
|
if (log) console.log(` [forge:simulated] stage complete: ${taskId}`);
|
||||||
|
return {
|
||||||
|
task_id: taskId,
|
||||||
|
outcome: 'simulated',
|
||||||
|
reason: 'no executor wired; simulated execution requested via --simulate',
|
||||||
|
completed_at: new Date().toISOString(),
|
||||||
|
exit_code: 0,
|
||||||
|
gate_results: [],
|
||||||
|
};
|
||||||
|
},
|
||||||
|
async getTaskStatus() {
|
||||||
|
return 'completed' as const;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
import type { GateEntry, TaskResult } from '@mosaicstack/macp';
|
import type { GateEntry } from '@mosaicstack/macp';
|
||||||
|
|
||||||
/** Stage dispatch mode. */
|
/** Stage dispatch mode. */
|
||||||
export type StageDispatch = 'exec' | 'yolo' | 'pi';
|
export type StageDispatch = 'exec' | 'yolo' | 'pi';
|
||||||
@@ -6,6 +6,58 @@ export type StageDispatch = 'exec' | 'yolo' | 'pi';
|
|||||||
/** Stage type — determines agent selection and gate requirements. */
|
/** Stage type — determines agent selection and gate requirements. */
|
||||||
export type StageType = 'research' | 'review' | 'coding' | 'deploy';
|
export type StageType = 'research' | 'review' | 'coding' | 'deploy';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Typed outcome for every gate and stage evaluation — closed set (SDLC-D-035).
|
||||||
|
*
|
||||||
|
* Only `passed` means "verified by a real implementation". `simulated` is
|
||||||
|
* produced exclusively in explicit `--simulate` runs and is never satisfying.
|
||||||
|
*/
|
||||||
|
export type ForgeOutcome =
|
||||||
|
| 'passed'
|
||||||
|
| 'failed'
|
||||||
|
| 'blocked'
|
||||||
|
| 'error'
|
||||||
|
| 'waiting-for-authority'
|
||||||
|
| 'simulated'
|
||||||
|
| 'not-applicable';
|
||||||
|
|
||||||
|
/** A gate that requires authority (human/board) sign-off; no mechanical command can satisfy it. */
|
||||||
|
export interface AuthorityGate {
|
||||||
|
kind: 'authority';
|
||||||
|
capability: string;
|
||||||
|
reason: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A gate that requires a wired provider (e.g. an AI reviewer, CI pipeline) to evaluate. */
|
||||||
|
export interface ProviderGate {
|
||||||
|
kind: 'provider';
|
||||||
|
capability: string;
|
||||||
|
reason: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Forge quality gate: a real command, an authority sign-off, or a provider-backed check. */
|
||||||
|
export type ForgeGate = string | GateEntry | AuthorityGate | ProviderGate;
|
||||||
|
|
||||||
|
/** Typed result of evaluating a single quality gate. */
|
||||||
|
export interface ForgeGateResult {
|
||||||
|
gate: string;
|
||||||
|
outcome: ForgeOutcome;
|
||||||
|
reason: string;
|
||||||
|
exitCode?: number;
|
||||||
|
output?: string;
|
||||||
|
timedOut?: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Typed result of a task/stage execution returned by a TaskExecutor. */
|
||||||
|
export interface ForgeTaskResult {
|
||||||
|
task_id: string;
|
||||||
|
outcome: ForgeOutcome;
|
||||||
|
reason: string;
|
||||||
|
completed_at: string;
|
||||||
|
exit_code: number;
|
||||||
|
gate_results: ForgeGateResult[];
|
||||||
|
}
|
||||||
|
|
||||||
/** Stage specification — defines a single pipeline stage. */
|
/** Stage specification — defines a single pipeline stage. */
|
||||||
export interface StageSpec {
|
export interface StageSpec {
|
||||||
number: string;
|
number: string;
|
||||||
@@ -14,7 +66,7 @@ export interface StageSpec {
|
|||||||
type: StageType;
|
type: StageType;
|
||||||
gate: string;
|
gate: string;
|
||||||
promptFile: string;
|
promptFile: string;
|
||||||
qualityGates: (string | GateEntry)[];
|
qualityGates: ForgeGate[];
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Brief classification. */
|
/** Brief classification. */
|
||||||
@@ -25,11 +77,18 @@ export type ClassSource = 'cli' | 'frontmatter' | 'auto';
|
|||||||
|
|
||||||
/** Per-stage status within a run manifest. */
|
/** Per-stage status within a run manifest. */
|
||||||
export interface StageStatus {
|
export interface StageStatus {
|
||||||
status: 'pending' | 'in_progress' | 'passed' | 'failed';
|
status: 'pending' | 'in_progress' | ForgeOutcome;
|
||||||
|
/** Why the stage reached its current (terminal) outcome, when applicable. */
|
||||||
|
reason?: string;
|
||||||
startedAt?: string;
|
startedAt?: string;
|
||||||
completedAt?: string;
|
completedAt?: string;
|
||||||
|
/** Typed per-gate results recorded alongside the stage outcome. */
|
||||||
|
gateResults?: ForgeGateResult[];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Execution mode of a run. */
|
||||||
|
export type RunMode = 'normal' | 'simulated';
|
||||||
|
|
||||||
/** Run manifest — persisted to disk as manifest.json. */
|
/** Run manifest — persisted to disk as manifest.json. */
|
||||||
export interface RunManifest {
|
export interface RunManifest {
|
||||||
runId: string;
|
runId: string;
|
||||||
@@ -38,10 +97,23 @@ export interface RunManifest {
|
|||||||
briefClass: BriefClass;
|
briefClass: BriefClass;
|
||||||
classSource: ClassSource;
|
classSource: ClassSource;
|
||||||
forceBoard: boolean;
|
forceBoard: boolean;
|
||||||
|
/**
|
||||||
|
* Execution mode. `simulated` runs stub execution; their results are typed
|
||||||
|
* `simulated` and must never be read as verified success. Optional because
|
||||||
|
* manifests written before this field existed default to `normal`.
|
||||||
|
*/
|
||||||
|
mode?: RunMode;
|
||||||
createdAt: string;
|
createdAt: string;
|
||||||
updatedAt: string;
|
updatedAt: string;
|
||||||
currentStage: string;
|
currentStage: string;
|
||||||
status: 'in_progress' | 'completed' | 'failed' | 'interrupted' | 'rejected';
|
status:
|
||||||
|
| 'in_progress'
|
||||||
|
| 'completed'
|
||||||
|
| 'failed'
|
||||||
|
| 'interrupted'
|
||||||
|
| 'rejected'
|
||||||
|
| 'simulated'
|
||||||
|
| 'waiting-for-authority';
|
||||||
stages: Record<string, StageStatus>;
|
stages: Record<string, StageStatus>;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -65,7 +137,7 @@ export interface ForgeTask {
|
|||||||
briefPath: string;
|
briefPath: string;
|
||||||
resultPath: string;
|
resultPath: string;
|
||||||
timeoutSeconds: number;
|
timeoutSeconds: number;
|
||||||
qualityGates: (string | GateEntry)[];
|
qualityGates: ForgeGate[];
|
||||||
worktree?: string;
|
worktree?: string;
|
||||||
command?: string;
|
command?: string;
|
||||||
dependsOn?: string[];
|
dependsOn?: string[];
|
||||||
@@ -76,7 +148,7 @@ export interface ForgeTask {
|
|||||||
/** Abstract task executor — decouples from packages/coord. */
|
/** Abstract task executor — decouples from packages/coord. */
|
||||||
export interface TaskExecutor {
|
export interface TaskExecutor {
|
||||||
submitTask(task: ForgeTask): Promise<void>;
|
submitTask(task: ForgeTask): Promise<void>;
|
||||||
waitForCompletion(taskId: string, timeoutMs: number): Promise<TaskResult>;
|
waitForCompletion(taskId: string, timeoutMs: number): Promise<ForgeTaskResult>;
|
||||||
getTaskStatus(taskId: string): Promise<ForgeTaskStatus>;
|
getTaskStatus(taskId: string): Promise<ForgeTaskStatus>;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -122,7 +194,16 @@ export interface PipelineOptions {
|
|||||||
stages?: string[];
|
stages?: string[];
|
||||||
skipTo?: string;
|
skipTo?: string;
|
||||||
dryRun?: boolean;
|
dryRun?: boolean;
|
||||||
executor: TaskExecutor;
|
/**
|
||||||
|
* Real task executor. Required in normal mode: the pipeline fails closed
|
||||||
|
* with FORGE_NO_EXECUTOR when it is absent.
|
||||||
|
*/
|
||||||
|
executor?: TaskExecutor;
|
||||||
|
/**
|
||||||
|
* Explicit opt-in to simulated execution. Every stage and gate result is
|
||||||
|
* typed `simulated` and is never satisfying.
|
||||||
|
*/
|
||||||
|
simulate?: boolean;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Pipeline run result. */
|
/** Pipeline run result. */
|
||||||
|
|||||||
Reference in New Issue
Block a user