Compare commits

..
Author SHA1 Message Date
fargo 2097379e25 fix(macp): fail-closed typed gate states — no placeholder/empty-command passes (#1275)
ci/woodpecker/pr/ci Pipeline was successful
RI-N2 / SDLC-D-035. Every GateResult now carries a typed status
discriminator (passed|failed|simulated|waiting|capability_failure);
passed:true remains true only for really-executed, really-green gates.

- ci-pipeline without a CI provider → capability_failure (MACP_NO_CI_PIPELINE),
  never the placeholder pass
- empty-command gate → capability_failure (MACP_NO_COMMAND / MACP_NO_REVIEWER),
  and runGates no longer silently skips it
- manual gate type waits (MACP_AUTHORITY_REQUIRED) — neither pass nor fail
- explicit simulation only (simulate option / --simulate): typed simulated
  results can never make the aggregate passed (RunGatesResult.state)
- CLI stubs (tasks list, submit, events tail) exit nonzero with typed
  MACP_NOT_IMPLEMENTED; macp gate is now wired to runGates with --simulate
- legacy __tests__/gate-runner.test.ts consolidated into src/gate-runner.spec.ts
  (root eslint project service does not cover packages/macp/__tests__)
2026-08-17 16:40:16 -05:00
jason.woltje 8199261caa Merge pull request 'fix(ci): unwire test-start-agent-session.sh, restore its signed exclusion — unblocks every PR on next' (#1270) from fix/1269-ci-chain-unblock into next
ci/woodpecker/push/publish Pipeline failed
Reviewed-on: #1270
2026-08-17 20:44:59 +00:00
fred 57a2f2b40e docs(ci): point the exclusion at tracking issue #1271, not the closed first filing
ci/woodpecker/pr/ci Pipeline was successful
The first PR for this change was filed under the retired mos-dt-0 principal
(pr-create.sh has no --login flag and find_tea_login_for_host returns the first
host match) and was closed and refiled as #1270. That left in-tree references
pointing at a closed duplicate PR rather than at the burn-down issue, which is
the wrong target for them anyway: the open design question belongs on #1271.
2026-08-16 18:03:02 -05:00
fred 93c1de51e1 fix(ci): unwire test-start-agent-session.sh, restore its signed exclusion (#1269)
ci/woodpecker/pr/ci Pipeline was canceled
The `test` step has failed on every `next` pipeline since #1017 on exactly one
assertion, and it is the same one on unrelated PRs:

    FAIL: host provides 'pi' in the system path; missing-binary cases are not
    measurable here            (framework/tools/fleet/test-start-agent-session.sh:103)

Measured 2026-08-16 across pipelines 2444 (#1256), 2438 (#1240) and 2441
(#1017-quality): exactly one FAIL line in each full log, identical, this line.
Control `zzz-not-present-zzz` -> 0 on all three.

Cause. #1241 (5c35a250) added the guard: the suite shims fake mosaic/pi/npm into
$FAKE_BIN, but the constructed PANE_PATH always ends in the real system path, so
on a host that installs those binaries the missing-binary cases cannot be
measured and a green run would mean nothing. The guard says so instead of
passing. Its own pipeline 2430 was green only because the suite was CI-excluded
at the time, so the guard had never run in CI. #1017 (c56483eb) then enumerated
it and dropped the exclusion. The CI image installs
@earendil-works/[email protected].1 on purpose, so the precondition is
unsatisfiable there. Both commits are mine.

The guard is correct and is not being softened. A check that cannot measure its
property and reports success is the failure mode this repo has been cataloguing
all week; the error was wiring the suite into an image that violates its
precondition, so the wiring is what gets reverted.

Second effect, which is the reason this cost a day rather than an hour:
test:framework-shell is one && chain and this sat at position 44 of 48, so
glpi/test-list-http-status.sh, orchestrator/test-board-roll.sh,
woodpecker/test-ci-wait-exit-matrix.sh and _scripts/test-fleet-transport-check.sh
have not run at all since the merge. The pipeline reported one failure, never
"one failure plus four unrun". All four are green when run directly on
sb-it-1-dt, so the mask hid nothing broken -- but that is a local result on one
host, not a CI-image result.

Verification, with controls:
- enumeration guard OK (population 52, enumerated 36, signed-excluded 16).
- control A, exclusion line removed while unwired -> FAIL UNENUMERATED.
- control B, exclusion line kept while rewired -> FAIL CONTRADICTORY EXCLUSION.
  The gate discriminates in both directions, so its OK is load-bearing.
- the four formerly-masked suites: rc=0 each, run directly.
- the full chain cannot be run to completion on sb-it-1-dt: it stops earlier, at
  the lease-broker Invariant R test, because this host carries the quarantined
  operator-global pi 0.84.2 against a measured 0.84.1. That is host-specific and
  out of scope here -- CI pins 0.84.1, and the single FAIL line in those three
  pipelines proves positions 1-43 passed there.

Burn-down is to control the tail of PANE_PATH inside the test, not to remove pi
from the image. Recorded in the exclusion reason and in #1269.
2026-08-16 17:58:49 -05:00
24 changed files with 1180 additions and 1695 deletions
-1
View File
@@ -34,7 +34,6 @@ export default tseslint.config(
'packages/storage/vitest.config.ts',
'packages/mosaic/vitest.config.ts',
'packages/mosaic/__tests__/*.ts',
'packages/forge/__tests__/*.ts',
'tools/federation-harness/*.ts',
],
},
-40
View File
@@ -539,43 +539,3 @@ Not every brief needs full Board of Directors review. The classification system
### Backward compatibility
Existing briefs without a `class` field are auto-classified. The default (no matching keywords) is `strategic`, so all existing runs get the full pipeline unless keywords trigger `technical`.
---
## Fail-Closed Execution & Explicit Simulation (SDLC-D-035)
**Added:** 2026-08-17
Forge fails closed when a required capability is missing. It never runs a
pipeline with a stub executor and reports success.
### Normal mode (default)
- No task executor wired → the CLI exits nonzero with the typed capability
error `FORGE_NO_EXECUTOR`. No run is created.
- A stage whose gate is approval-based (board approval, planning approvals,
remediation re-review, discovery/analysis attestations) records a typed
`waiting-for-authority` stage result and raises `FORGE_AUTHORITY_REQUIRED`.
It never passes vacuously.
- A stage whose gate requires an unwired provider (AI reviewer, CI pipeline)
records a typed `blocked` stage result and raises `FORGE_NO_REVIEWER` /
`FORGE_NO_CI_PIPELINE`. The synthetic echo-review approval in `06-review`
and all vacuous `true` gates were removed.
### Explicit simulation (`--simulate`)
Opts into stub/synthetic execution. Every stage result, every gate result, and
the run manifest carry the distinct typed status `simulated` (manifest also
records `mode: "simulated"`). `simulated` is a non-satisfying outcome:
`isSatisfyingOutcome()` and all completion/gate consumers treat only `passed`
as satisfying. The CLI exits 0 for a simulated run only because the caller
explicitly passed `--simulate`, and prints a loud SIMULATED banner.
### Typed outcome model
Every gate/task outcome is one of the closed set
`passed | failed | blocked | error | waiting-for-authority | simulated |
not-applicable`, with the reason recorded on the stage status and each gate
result in `manifest.json`. Missing implementations, missing gate evidence,
unknown stages, process errors, and timeouts map to fail-closed members —
never to `passed`.
@@ -1,319 +0,0 @@
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import { generateBoardTasks } from '../src/board-tasks.js';
import { STAGE_SPECS } from '../src/constants.js';
import { ForgeCapabilityError } from '../src/errors.js';
import {
evaluateStageGates,
gateLabel,
isCommandGate,
isSatisfyingOutcome,
} from '../src/outcomes.js';
import { loadManifest, runPipeline } from '../src/pipeline-runner.js';
import type { ForgeTask, ForgeTaskResult, TaskExecutor } from '../src/types.js';
/**
* Mock real executor that returns typed results.
*
* Command gates are "verified" by the mock so normal-mode runs can pass
* mechanically gated stages; authority/provider gates are never reported
* because they have no mechanical implementation.
*/
function createTypedExecutor(options?: {
failStage?: string;
gateOutcomes?: Record<string, 'passed' | 'failed' | 'simulated' | 'error' | 'blocked'>;
}): TaskExecutor & { submittedTasks: ForgeTask[] } {
const submittedTasks: ForgeTask[] = [];
return {
submittedTasks,
async submitTask(task: ForgeTask) {
submittedTasks.push(task);
},
async waitForCompletion(taskId: string): Promise<ForgeTaskResult> {
const task = submittedTasks.find((t) => t.id === taskId);
const stageName = task?.metadata?.['stageName'] as string | undefined;
if (options?.failStage && stageName === options.failStage) {
return {
task_id: taskId,
outcome: 'failed',
reason: 'mock task failure',
completed_at: new Date().toISOString(),
exit_code: 1,
gate_results: [],
};
}
const gateResults = (task?.qualityGates ?? [])
.filter((gate) => isCommandGate(gate))
.map((gate) => {
const label = gateLabel(gate);
const outcome = options?.gateOutcomes?.[label] ?? 'passed';
return {
gate: label,
outcome,
reason: outcome === 'passed' ? 'mock verified' : `mock gate outcome: ${outcome}`,
};
});
return {
task_id: taskId,
outcome: 'passed',
reason: 'mock verified',
completed_at: new Date().toISOString(),
exit_code: 0,
gate_results: gateResults,
};
},
async getTaskStatus() {
return 'completed' as const;
},
};
}
describe('fail-closed: no executor wired', () => {
let tmpDir: string;
let briefPath: string;
beforeEach(() => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'forge-failclosed-'));
briefPath = path.join(tmpDir, 'brief.md');
fs.writeFileSync(briefPath, '# Fix bug\n\nA bugfix for lint cleanup.');
});
afterEach(() => {
fs.rmSync(tmpDir, { recursive: true, force: true });
});
it('throws a typed FORGE_NO_EXECUTOR capability error without --simulate', async () => {
await expect(
runPipeline(briefPath, tmpDir, {
// no executor, no simulate — must fail closed, never run with a stub
stages: ['00-intake'],
}),
).rejects.toMatchObject({
name: 'ForgeCapabilityError',
code: 'FORGE_NO_EXECUTOR',
capability: 'task-executor',
});
});
it('does not create a run directory when failing closed on a missing executor', async () => {
try {
await runPipeline(briefPath, tmpDir, { stages: ['00-intake'] });
} catch {
// expected
}
expect(fs.existsSync(path.join(tmpDir, '.forge', 'runs'))).toBe(false);
});
it('completes with every result typed simulated when simulate is set', async () => {
const result = await runPipeline(briefPath, tmpDir, {
simulate: true,
stages: ['00-intake', '00b-discovery', '02-planning-1', '06-review'],
});
expect(result.manifest.mode).toBe('simulated');
expect(result.manifest.status).toBe('simulated');
for (const stage of result.stages) {
const stageStatus = result.manifest.stages[stage];
expect(stageStatus?.status, `stage ${stage}`).toBe('simulated');
expect(stageStatus?.status, `stage ${stage}`).not.toBe('passed');
expect(stageStatus?.reason, `stage ${stage}`).toBeTruthy();
for (const gateResult of stageStatus?.gateResults ?? []) {
expect(gateResult.outcome, `gate ${gateResult.gate} of ${stage}`).toBe('simulated');
expect(gateResult.outcome, `gate ${gateResult.gate} of ${stage}`).not.toBe('passed');
}
}
// The persisted manifest agrees.
const persisted = loadManifest(result.runDir);
expect(persisted.mode).toBe('simulated');
expect(persisted.status).toBe('simulated');
expect(persisted.stages['02-planning-1']?.status).toBe('simulated');
});
});
describe('fail-closed: typed outcome model', () => {
it('only passed satisfies the gate/dependency predicate', () => {
expect(isSatisfyingOutcome('passed')).toBe(true);
expect(isSatisfyingOutcome('failed')).toBe(false);
expect(isSatisfyingOutcome('blocked')).toBe(false);
expect(isSatisfyingOutcome('error')).toBe(false);
expect(isSatisfyingOutcome('waiting-for-authority')).toBe(false);
expect(isSatisfyingOutcome('simulated')).toBe(false);
expect(isSatisfyingOutcome('not-applicable')).toBe(false);
});
it('a simulated gate result cannot satisfy the stage gate evaluation', () => {
const evaluation = evaluateStageGates('05-coding', STAGE_SPECS['05-coding']!.qualityGates, {
task_id: 'FORGE-x-05',
outcome: 'passed',
reason: 'executor claims success',
completed_at: new Date().toISOString(),
exit_code: 0,
gate_results: [{ gate: 'pnpm lint', outcome: 'simulated', reason: 'simulated gate' }],
});
expect(isSatisfyingOutcome(evaluation.outcome)).toBe(false);
expect(evaluation.outcome).toBe('error');
});
it('a simulated task outcome cannot satisfy evaluation in normal mode', () => {
const evaluation = evaluateStageGates('00-intake', [], {
task_id: 'FORGE-x-00',
outcome: 'simulated',
reason: 'executor reported simulated',
completed_at: new Date().toISOString(),
exit_code: 0,
gate_results: [],
});
expect(isSatisfyingOutcome(evaluation.outcome)).toBe(false);
});
it('a missing gate result blocks the stage instead of passing vacuously', () => {
const evaluation = evaluateStageGates('05-coding', STAGE_SPECS['05-coding']!.qualityGates, {
task_id: 'FORGE-x-05',
outcome: 'passed',
reason: 'executor claims success',
completed_at: new Date().toISOString(),
exit_code: 0,
gate_results: [],
});
expect(evaluation.outcome).toBe('blocked');
});
});
describe('fail-closed: authority and provider gates', () => {
let tmpDir: string;
let briefPath: string;
beforeEach(() => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'forge-authority-'));
briefPath = path.join(tmpDir, 'brief.md');
fs.writeFileSync(briefPath, '# Fix bug\n\nA bugfix for lint cleanup.');
});
afterEach(() => {
fs.rmSync(tmpDir, { recursive: true, force: true });
});
it.each(['02-planning-1', '03-planning-2', '04-planning-3', '07-remediate'])(
'planning/remediation stage %s yields waiting-for-authority (not passed) in normal mode',
async (stage) => {
const executor = createTypedExecutor();
let runDir: string | undefined;
try {
await runPipeline(briefPath, tmpDir, {
executor,
stages: [stage as string],
});
expect.unreachable('runPipeline should have failed closed');
} catch (err) {
expect(err).toBeInstanceOf(ForgeCapabilityError);
expect((err as ForgeCapabilityError).code).toBe('FORGE_AUTHORITY_REQUIRED');
runDir = path.join(tmpDir, '.forge', 'runs');
}
const runIds = fs.readdirSync(runDir!);
expect(runIds).toHaveLength(1);
const manifest = loadManifest(path.join(runDir!, runIds[0]!));
expect(manifest.stages[stage]?.status).toBe('waiting-for-authority');
expect(manifest.stages[stage]?.status).not.toBe('passed');
expect(manifest.status).toBe('waiting-for-authority');
},
);
it('review stage fails closed with a typed FORGE_NO_REVIEWER error in normal mode', async () => {
const executor = createTypedExecutor();
try {
await runPipeline(briefPath, tmpDir, {
executor,
stages: ['06-review'],
});
expect.unreachable('runPipeline should have failed closed');
} catch (err) {
expect(err).toBeInstanceOf(ForgeCapabilityError);
expect((err as ForgeCapabilityError).code).toBe('FORGE_NO_REVIEWER');
expect((err as ForgeCapabilityError).capability).toBe('reviewer');
}
const runsDir = path.join(tmpDir, '.forge', 'runs');
const runIds = fs.readdirSync(runsDir);
const manifest = loadManifest(path.join(runsDir, runIds[0]!));
expect(manifest.stages['06-review']?.status).toBe('blocked');
expect(manifest.stages['06-review']?.status).not.toBe('passed');
expect(manifest.status).toBe('failed');
});
it('review stage produces simulated results under --simulate', async () => {
const result = await runPipeline(briefPath, tmpDir, {
simulate: true,
stages: ['06-review'],
});
expect(result.manifest.mode).toBe('simulated');
expect(result.manifest.stages['06-review']?.status).toBe('simulated');
for (const gateResult of result.manifest.stages['06-review']?.gateResults ?? []) {
expect(gateResult.outcome).toBe('simulated');
}
});
it('deploy stage fails closed without a wired ci-pipeline provider in normal mode', async () => {
const executor = createTypedExecutor();
await expect(
runPipeline(briefPath, tmpDir, {
executor,
stages: ['09-deploy'],
}),
).rejects.toMatchObject({
name: 'ForgeCapabilityError',
code: 'FORGE_NO_CI_PIPELINE',
});
});
});
describe('fail-closed: no vacuous gate commands remain', () => {
it('stage constants contain no echo/synthetic-approval, vacuous true, or empty gate commands', () => {
for (const [stageName, spec] of Object.entries(STAGE_SPECS)) {
for (const gate of spec.qualityGates) {
const serialized = JSON.stringify(gate);
// The echo-review synthetic approval must be gone.
expect(serialized, `stage ${stageName} gate ${serialized}`).not.toContain('echo');
expect(serialized, `stage ${stageName} gate ${serialized}`).not.toMatch(/"verdict"\s*:/);
expect(serialized, `stage ${stageName} gate ${serialized}`).not.toMatch(
/"summary"\s*:\s*"review-pass"/,
);
// No vacuous literal `true` gate.
expect(gate, `stage ${stageName}`).not.toBe('true');
// Command gates must carry a real, non-empty command.
if (isCommandGate(gate)) {
const command = typeof gate === 'string' ? gate : gate.command;
expect(command.trim().length, `stage ${stageName} gate ${serialized}`).toBeGreaterThan(0);
}
}
}
});
it('board tasks contain no vacuous true gates', () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'forge-board-gates-'));
try {
const tasks = generateBoardTasks('# Brief', [], tmpDir, 'BOARD-TEST');
for (const task of tasks) {
for (const gate of task.qualityGates) {
expect(gate, `task ${task.id}`).not.toBe('true');
const serialized = JSON.stringify(gate);
expect(serialized, `task ${task.id} gate ${serialized}`).not.toContain('echo');
}
}
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true });
}
});
});
+34 -161
View File
@@ -12,10 +12,10 @@ import {
resumePipeline,
getPipelineStatus,
} from '../src/pipeline-runner.js';
import type { ForgeTask, ForgeTaskResult, RunManifest, TaskExecutor } from '../src/types.js';
import { gateLabel, isCommandGate } from '../src/outcomes.js';
import type { ForgeTask, RunManifest, TaskExecutor } from '../src/types.js';
import type { TaskResult } from '@mosaicstack/macp';
/** Mock TaskExecutor that records submitted tasks and returns typed results. */
/** Mock TaskExecutor that records submitted tasks and returns success. */
function createMockExecutor(options?: {
failStage?: string;
}): TaskExecutor & { submittedTasks: ForgeTask[] } {
@@ -25,7 +25,7 @@ function createMockExecutor(options?: {
async submitTask(task: ForgeTask) {
submittedTasks.push(task);
},
async waitForCompletion(taskId: string): Promise<ForgeTaskResult> {
async waitForCompletion(taskId: string): Promise<TaskResult> {
const failStage = options?.failStage;
const task = submittedTasks.find((t) => t.id === taskId);
const stageName = task?.metadata?.['stageName'] as string | undefined;
@@ -33,8 +33,7 @@ function createMockExecutor(options?: {
if (failStage && stageName === failStage) {
return {
task_id: taskId,
outcome: 'failed',
reason: 'mock task failure',
status: 'failed',
completed_at: new Date().toISOString(),
exit_code: 1,
gate_results: [],
@@ -42,17 +41,10 @@ function createMockExecutor(options?: {
}
return {
task_id: taskId,
outcome: 'passed',
reason: 'mock verified',
status: 'completed',
completed_at: new Date().toISOString(),
exit_code: 0,
gate_results: (task?.qualityGates ?? [])
.filter((gate) => isCommandGate(gate))
.map((gate) => ({
gate: gateLabel(gate),
outcome: 'passed' as const,
reason: 'mock verified',
})),
gate_results: [],
};
},
async getTaskStatus() {
@@ -164,13 +156,12 @@ describe('runPipeline', () => {
const executor = createMockExecutor();
const result = await runPipeline(briefPath, tmpDir, {
executor,
stages: ['00-intake', '05-coding'],
stages: ['00-intake', '00b-discovery'],
});
expect(result.runId).toMatch(/^\d{8}-\d{6}$/);
expect(result.stages).toEqual(['00-intake', '05-coding']);
expect(result.stages).toEqual(['00-intake', '00b-discovery']);
expect(result.manifest.status).toBe('completed');
expect(result.manifest.mode).toBe('normal');
expect(executor.submittedTasks).toHaveLength(2);
});
@@ -189,17 +180,12 @@ describe('runPipeline', () => {
const executor = createMockExecutor();
const result = await runPipeline(briefPath, tmpDir, {
executor,
stages: ['00-intake', '05-coding'],
stages: ['00-intake', '00b-discovery'],
});
const manifest = loadManifest(result.runDir);
expect(manifest.stages['00-intake']?.status).toBe('passed');
expect(manifest.stages['05-coding']?.status).toBe('passed');
expect(manifest.stages['05-coding']?.gateResults?.map((g) => g.outcome)).toEqual([
'passed',
'passed',
'passed',
]);
expect(manifest.stages['00b-discovery']?.status).toBe('passed');
});
it('respects CLI class override', async () => {
@@ -229,7 +215,7 @@ describe('runPipeline', () => {
const executor = createMockExecutor();
await runPipeline(briefPath, tmpDir, {
executor,
stages: ['00-intake', '05-coding', '08-test'],
stages: ['00-intake', '00b-discovery', '02-planning-1'],
});
expect(executor.submittedTasks[0]!.dependsOn).toBeUndefined();
@@ -238,14 +224,14 @@ describe('runPipeline', () => {
});
it('handles stage failure', async () => {
const executor = createMockExecutor({ failStage: '05-coding' });
const executor = createMockExecutor({ failStage: '00b-discovery' });
await expect(
runPipeline(briefPath, tmpDir, {
executor,
stages: ['00-intake', '05-coding'],
stages: ['00-intake', '00b-discovery'],
}),
).rejects.toThrow('Stage 05-coding failed');
).rejects.toThrow('Stage 00b-discovery failed');
});
it('marks manifest as failed on stage failure', async () => {
@@ -284,143 +270,30 @@ describe('resumePipeline', () => {
fs.rmSync(tmpDir, { recursive: true, force: true });
});
it('resumes from first incomplete stage and fails closed at the next provider gate', async () => {
// Simulate a run whose authority stages were approved out-of-band
// (recorded as passed) and whose coding stage failed mechanically.
const runId = '20260101-000000';
const runDir = path.join(tmpDir, '.forge', 'runs', runId);
fs.mkdirSync(runDir, { recursive: true });
const passed = { status: 'passed' as const, startedAt: '2026-01-01T00:00:00Z' };
saveManifest(runDir, {
runId,
brief: briefPath,
codebase: tmpDir,
briefClass: 'hotfix',
classSource: 'frontmatter',
forceBoard: false,
mode: 'normal',
createdAt: '2026-01-01T00:00:00Z',
updatedAt: '2026-01-01T00:00:00Z',
currentStage: '05-coding',
status: 'failed',
stages: {
'00-intake': passed,
'00b-discovery': passed,
'02-planning-1': passed,
'03-planning-2': passed,
'04-planning-3': passed,
'05-coding': { status: 'failed', reason: 'gate failed' },
},
});
it('resumes from first incomplete stage', async () => {
// First run fails on discovery
const executor1 = createMockExecutor({ failStage: '00b-discovery' });
let runDir: string;
// Resume re-runs 05-coding (the first non-passed stage), then fails
// closed at 06-review because no reviewer provider is wired.
const executor = createMockExecutor();
await expect(resumePipeline(runDir, executor)).rejects.toMatchObject({
name: 'ForgeCapabilityError',
code: 'FORGE_NO_REVIEWER',
});
const manifest = loadManifest(runDir);
expect(manifest.stages['05-coding']?.status).toBe('passed');
expect(manifest.stages['06-review']?.status).toBe('blocked');
expect(manifest.status).toBe('failed');
});
it('resumes to completion as simulated under explicit simulate', async () => {
const runId = '20260101-000003';
const runDir = path.join(tmpDir, '.forge', 'runs', runId);
fs.mkdirSync(runDir, { recursive: true });
const passed = { status: 'passed' as const, startedAt: '2026-01-01T00:00:00Z' };
saveManifest(runDir, {
runId,
brief: briefPath,
codebase: tmpDir,
briefClass: 'hotfix',
classSource: 'frontmatter',
forceBoard: false,
mode: 'normal',
createdAt: '2026-01-01T00:00:00Z',
updatedAt: '2026-01-01T00:00:00Z',
currentStage: '05-coding',
status: 'failed',
stages: {
'00-intake': passed,
'00b-discovery': passed,
'02-planning-1': passed,
'03-planning-2': passed,
'04-planning-3': passed,
'05-coding': { status: 'failed', reason: 'gate failed' },
},
});
const result = await resumePipeline(runDir, undefined, { simulate: true });
expect(result.manifest.status).toBe('simulated');
expect(result.manifest.mode).toBe('simulated');
expect(result.stages[0]).toBe('05-coding');
for (const stage of result.stages) {
expect(result.manifest.stages[stage]?.status).toBe('simulated');
try {
await runPipeline(briefPath, tmpDir, {
executor: executor1,
stages: ['00-intake', '00b-discovery', '02-planning-1'],
});
} catch {
// expected
}
});
it('fails closed on resume when the next stage needs authority sign-off', async () => {
const runId = '20260101-000001';
const runDir = path.join(tmpDir, '.forge', 'runs', runId);
fs.mkdirSync(runDir, { recursive: true });
saveManifest(runDir, {
runId,
brief: briefPath,
codebase: tmpDir,
briefClass: 'hotfix',
classSource: 'frontmatter',
forceBoard: false,
mode: 'normal',
createdAt: '2026-01-01T00:00:00Z',
updatedAt: '2026-01-01T00:00:00Z',
currentStage: '00-intake',
status: 'in_progress',
stages: {
'00-intake': { status: 'passed' },
},
});
const runsDir = path.join(tmpDir, '.forge', 'runs');
runDir = path.join(runsDir, fs.readdirSync(runsDir)[0]!);
const executor = createMockExecutor();
await expect(resumePipeline(runDir, executor)).rejects.toMatchObject({
name: 'ForgeCapabilityError',
code: 'FORGE_AUTHORITY_REQUIRED',
});
// Resume should pick up from 00b-discovery
const executor2 = createMockExecutor();
const result = await resumePipeline(runDir, executor2);
const manifest = loadManifest(runDir);
expect(manifest.stages['00b-discovery']?.status).toBe('waiting-for-authority');
expect(manifest.status).toBe('waiting-for-authority');
});
it('fails closed on resume without an executor or --simulate', async () => {
const runId = '20260101-000002';
const runDir = path.join(tmpDir, '.forge', 'runs', runId);
fs.mkdirSync(runDir, { recursive: true });
saveManifest(runDir, {
runId,
brief: briefPath,
codebase: tmpDir,
briefClass: 'hotfix',
classSource: 'frontmatter',
forceBoard: false,
mode: 'normal',
createdAt: '2026-01-01T00:00:00Z',
updatedAt: '2026-01-01T00:00:00Z',
currentStage: '00-intake',
status: 'in_progress',
stages: {
'00-intake': { status: 'passed' },
},
});
await expect(resumePipeline(runDir)).rejects.toMatchObject({
name: 'ForgeCapabilityError',
code: 'FORGE_NO_EXECUTOR',
});
expect(result.manifest.status).toBe('completed');
// Should have re-run from 00b-discovery onward
expect(result.stages[0]).toBe('00b-discovery');
});
});
+2 -15
View File
@@ -95,14 +95,7 @@ export function generateBoardTasks(
briefPath,
resultPath: resultRelPath,
timeoutSeconds: 120,
qualityGates: [
{
kind: 'authority',
capability: 'board-approval',
reason:
'persona evaluation is judged by board synthesis (authority review); no mechanical gate exists',
},
],
qualityGates: ['true'],
metadata: {
personaName: persona.name,
personaSlug: persona.slug,
@@ -128,13 +121,7 @@ export function generateBoardTasks(
timeoutSeconds: 120,
dependsOn: personaTaskIds,
dependsOnPolicy: 'all_terminal',
qualityGates: [
{
kind: 'authority',
capability: 'board-approval',
reason: 'board synthesis is an authority decision; no mechanical gate exists',
},
],
qualityGates: ['true'],
metadata: {
resultOutputPath: synthesisResult,
inputResultPaths: personaResultPaths,
+1 -96
View File
@@ -1,11 +1,7 @@
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { Command } from 'commander';
import { describe, expect, it, vi, beforeEach, afterEach } from 'vitest';
import { describe, expect, it } from 'vitest';
import { registerForgeCommand } from './cli.js';
import { loadManifest } from './pipeline-runner.js';
describe('registerForgeCommand', () => {
it('registers a "forge" command on the parent program', () => {
@@ -59,94 +55,3 @@ describe('registerForgeCommand', () => {
}).not.toThrow();
});
});
describe('forge run fail-closed behavior (SDLC-D-035)', () => {
let tmpDir: string;
let briefPath: string;
let errSpy: ReturnType<typeof vi.spyOn>;
let logSpy: ReturnType<typeof vi.spyOn>;
let prevExitCode: string | number | null | undefined;
const parse = (args: string[]) => {
const program = new Command();
registerForgeCommand(program);
return program.parseAsync(['forge', ...args], { from: 'user' });
};
beforeEach(() => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'forge-cli-failclosed-'));
briefPath = path.join(tmpDir, 'brief.md');
fs.writeFileSync(briefPath, '# Fix bug\n\nA bugfix for lint cleanup.');
errSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
prevExitCode = process.exitCode;
});
afterEach(() => {
errSpy.mockRestore();
logSpy.mockRestore();
process.exitCode = prevExitCode;
fs.rmSync(tmpDir, { recursive: true, force: true });
});
it('exits nonzero with a typed FORGE_NO_EXECUTOR error when no executor is wired and --simulate is absent', async () => {
await parse(['run', '--brief', briefPath, '--codebase', tmpDir]);
expect(process.exitCode).toBe(1);
const errText = errSpy.mock.calls.map((c) => c.join(' ')).join('\n');
expect(errText).toContain('FORGE_NO_EXECUTOR');
// It must never run the pipeline with a stub and report success.
expect(fs.existsSync(path.join(tmpDir, '.forge', 'runs'))).toBe(false);
});
it('completes with typed simulated results and exit 0 under explicit --simulate', async () => {
await parse(['run', '--brief', briefPath, '--codebase', tmpDir, '--simulate']);
expect(process.exitCode).toBeUndefined();
// Loud simulated-mode summary.
const logText = logSpy.mock.calls.map((c) => c.join(' ')).join('\n');
expect(logText).toContain('SIMULATED');
// Manifest records the mode and simulated per-result statuses.
const runsDir = path.join(tmpDir, '.forge', 'runs');
const runIds = fs.readdirSync(runsDir);
expect(runIds).toHaveLength(1);
const manifest = loadManifest(path.join(runsDir, runIds[0]!));
expect(manifest.mode).toBe('simulated');
expect(manifest.status).toBe('simulated');
for (const stageStatus of Object.values(manifest.stages)) {
expect(stageStatus?.status).toBe('simulated');
for (const gateResult of stageStatus?.gateResults ?? []) {
expect(gateResult.outcome).toBe('simulated');
}
}
});
it('resume exits nonzero with a typed FORGE_NO_EXECUTOR error without --simulate', async () => {
const runDir = path.join(tmpDir, '.forge', 'runs', '20260101-000000');
fs.mkdirSync(runDir, { recursive: true });
fs.writeFileSync(
path.join(runDir, 'manifest.json'),
JSON.stringify({
runId: '20260101-000000',
brief: briefPath,
codebase: tmpDir,
briefClass: 'hotfix',
classSource: 'frontmatter',
forceBoard: false,
createdAt: '2026-01-01T00:00:00Z',
updatedAt: '2026-01-01T00:00:00Z',
currentStage: '00-intake',
status: 'in_progress',
stages: { '00-intake': { status: 'passed' } },
}),
);
await parse(['resume', '20260101-000000', '--project', tmpDir]);
expect(process.exitCode).toBe(1);
const errText = errSpy.mock.calls.map((c) => c.join(' ')).join('\n');
expect(errText).toContain('FORGE_NO_EXECUTOR');
});
});
+48 -122
View File
@@ -5,47 +5,37 @@ import type { Command } from 'commander';
import { classifyBrief } from './brief-classifier.js';
import { STAGE_LABELS, STAGE_SEQUENCE } from './constants.js';
import { ForgeCapabilityError } from './errors.js';
import { getEffectivePersonas, loadBoardPersonas } from './persona-loader.js';
import { generateRunId, getPipelineStatus, loadManifest, runPipeline } from './pipeline-runner.js';
import { createSimulatedExecutor } from './simulated-executor.js';
import type { PipelineOptions, RunManifest, RunMode } from './types.js';
import type { PipelineOptions, RunManifest, TaskExecutor } from './types.js';
// ---------------------------------------------------------------------------
// Stub executor — used when no real executor is wired at CLI invocation time.
// ---------------------------------------------------------------------------
const stubExecutor: TaskExecutor = {
async submitTask(task) {
console.log(` [forge] stage submitted: ${task.id} (${task.title})`);
},
async waitForCompletion(taskId, _timeoutMs) {
console.log(` [forge] stage complete: ${taskId}`);
return {
task_id: taskId,
status: 'completed' as const,
completed_at: new Date().toISOString(),
exit_code: 0,
gate_results: [],
};
},
async getTaskStatus(_taskId) {
return 'completed' as const;
},
};
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
/** Resolve a run's effective mode, defaulting legacy manifests to normal. */
function runModeOf(manifest: RunManifest): RunMode {
return manifest.mode ?? 'normal';
}
/** Print a loud banner so a simulated run can never be misread as verified. */
function printSimulatedBanner(): void {
console.log('');
console.log('[forge] ===============================================================');
console.log('[forge] MODE: SIMULATED — no stage or gate was really executed.');
console.log('[forge] All results are synthetic and MUST NOT be read as verified');
console.log('[forge] success. Wire a real executor/providers and re-run to verify.');
console.log('[forge] ===============================================================');
}
/** Print a typed error line for fail-closed capability errors. */
function printCapabilityError(err: ForgeCapabilityError): void {
console.error(`[forge] error ${err.code}: ${err.message}`);
console.error(`[forge] missing capability: ${err.capability}`);
}
/** Handle a pipeline error uniformly: typed capability errors get their code. */
function handlePipelineError(err: unknown): void {
if (err instanceof ForgeCapabilityError) {
printCapabilityError(err);
} else {
console.error(`[forge] pipeline failed: ${err instanceof Error ? err.message : String(err)}`);
}
process.exitCode = 1;
}
function formatDuration(startedAt?: string, completedAt?: string): string {
if (!startedAt || !completedAt) return '-';
const ms = new Date(completedAt).getTime() - new Date(startedAt).getTime();
@@ -54,24 +44,19 @@ function formatDuration(startedAt?: string, completedAt?: string): string {
}
function printManifestTable(manifest: RunManifest): void {
const mode = runModeOf(manifest);
console.log(`\nRun ID : ${manifest.runId}`);
console.log(`Status : ${manifest.status}`);
console.log(`Mode : ${mode}`);
if (mode === 'simulated') {
console.log('WARNING: SIMULATED RUN — results are synthetic, not verified success.');
}
console.log(`Brief : ${manifest.brief}`);
console.log(`Class : ${manifest.briefClass} (${manifest.classSource})`);
console.log(`Updated: ${manifest.updatedAt}`);
console.log('');
console.log('Stage'.padEnd(22) + 'Status'.padEnd(24) + 'Duration');
console.log('-'.repeat(60));
console.log('Stage'.padEnd(22) + 'Status'.padEnd(14) + 'Duration');
console.log('-'.repeat(50));
for (const stage of STAGE_SEQUENCE) {
const s = manifest.stages[stage];
if (!s) continue;
const label = (STAGE_LABELS[stage] ?? stage).padEnd(22);
const status = s.status.padEnd(24);
const status = s.status.padEnd(14);
const dur = formatDuration(s.startedAt, s.completedAt);
console.log(`${label}${status}${dur}`);
}
@@ -105,58 +90,23 @@ function listRecentRuns(projectRoot?: string): void {
}
console.log('\nRecent runs:');
console.log('Run ID'.padEnd(22) + 'Status'.padEnd(24) + 'Mode'.padEnd(12) + 'Brief');
console.log('-'.repeat(80));
console.log('Run ID'.padEnd(22) + 'Status'.padEnd(14) + 'Brief');
console.log('-'.repeat(70));
for (const runId of entries) {
const runDir = path.join(runsDir, runId);
try {
const manifest = loadManifest(runDir);
const status = manifest.status.padEnd(24);
const mode = runModeOf(manifest).padEnd(12);
const status = manifest.status.padEnd(14);
const brief = path.basename(manifest.brief);
console.log(`${runId.padEnd(22)}${status}${mode}${brief}`);
console.log(`${runId.padEnd(22)}${status}${brief}`);
} catch {
console.log(`${runId.padEnd(22)}${'(unreadable)'.padEnd(24)}`);
console.log(`${runId.padEnd(22)}${'(unreadable)'.padEnd(14)}`);
}
}
console.log('');
}
/**
* Apply the exit-code policy for a finished pipeline run (SDLC-D-035):
*
* - exit 0 only for a verified `completed` normal run, or for an overall
* `simulated` run when the caller explicitly passed --simulate;
* - anything else exits nonzero so it can never be read as success.
*/
function applyRunExitPolicy(result: { manifest: RunManifest; runDir: string }, simulate: boolean) {
const { manifest } = result;
if (runModeOf(manifest) === 'simulated') {
if (!simulate || manifest.status !== 'simulated') {
console.error(
'[forge] error FORGE_MODE_MISMATCH: run reports simulated results without an explicit, ' +
'consistent --simulate request; refusing to report success.',
);
process.exitCode = 1;
return;
}
printSimulatedBanner();
console.log(`[forge] run directory: ${result.runDir}`);
return; // exit 0 — the caller explicitly opted into simulation
}
if (manifest.status !== 'completed') {
console.error(`[forge] run did not complete: terminal status '${manifest.status}'`);
process.exitCode = 1;
return;
}
console.log(`[forge] pipeline complete (mode: normal): ${manifest.runId}`);
console.log(`[forge] run directory: ${result.runDir}`);
}
// ---------------------------------------------------------------------------
// Register function
// ---------------------------------------------------------------------------
@@ -179,11 +129,6 @@ export function registerForgeCommand(parent: Command): void {
.option('--config <path>', 'Path to forge config file (.forge/config.yaml)')
.option('--codebase <path>', 'Codebase root to pass to the pipeline', process.cwd())
.option('--dry-run', 'Print planned stages without executing', false)
.option(
'--simulate',
'Simulate execution without real providers (every result is typed simulated, never verified)',
false,
)
.action(
async (opts: {
brief: string;
@@ -192,7 +137,6 @@ export function registerForgeCommand(parent: Command): void {
config?: string;
codebase: string;
dryRun: boolean;
simulate: boolean;
}) => {
const briefPath = path.resolve(opts.brief);
@@ -205,22 +149,14 @@ export function registerForgeCommand(parent: Command): void {
const briefContent = fs.readFileSync(briefPath, 'utf-8');
const briefClass = classifyBrief(briefContent);
const projectRoot = opts.codebase;
// A real executor is never wired at CLI invocation time today, so the
// only executor we may construct is the explicitly-requested simulated
// one. Normal mode fails closed with FORGE_NO_EXECUTOR.
const executor = opts.simulate ? createSimulatedExecutor() : undefined;
if (opts.resume) {
const runId = opts.runId ?? generateRunId();
const runDir = resolveRunDir(runId, projectRoot);
console.log(`[forge] resuming run: ${runId}`);
try {
const { resumePipeline } = await import('./pipeline-runner.js');
const result = await resumePipeline(runDir, executor, { simulate: opts.simulate });
applyRunExitPolicy(result, opts.simulate);
} catch (err) {
handlePipelineError(err);
}
const { resumePipeline } = await import('./pipeline-runner.js');
const result = await resumePipeline(runDir, stubExecutor);
console.log(`[forge] pipeline complete: ${result.runId}`);
return;
}
@@ -228,8 +164,7 @@ export function registerForgeCommand(parent: Command): void {
briefClass,
codebase: projectRoot,
dryRun: opts.dryRun,
executor,
simulate: opts.simulate,
executor: stubExecutor,
};
if (opts.dryRun) {
@@ -245,15 +180,16 @@ export function registerForgeCommand(parent: Command): void {
console.log(`[forge] starting pipeline for brief: ${briefPath}`);
console.log(`[forge] classified as: ${briefClass}`);
if (opts.simulate) {
console.log('[forge] mode: SIMULATED (explicit --simulate)');
}
try {
const result = await runPipeline(briefPath, projectRoot, pipelineOptions);
applyRunExitPolicy(result, opts.simulate);
console.log(`[forge] pipeline complete: ${result.runId}`);
console.log(`[forge] run directory: ${result.runDir}`);
} catch (err) {
handlePipelineError(err);
console.error(
`[forge] pipeline failed: ${err instanceof Error ? err.message : String(err)}`,
);
process.exitCode = 1;
}
},
);
@@ -288,12 +224,7 @@ export function registerForgeCommand(parent: Command): void {
.command('resume <runId>')
.description('Resume a stopped or failed pipeline run')
.option('--project <path>', 'Project root (defaults to cwd)', process.cwd())
.option(
'--simulate',
'Simulate execution without real providers (every result is typed simulated, never verified)',
false,
)
.action(async (runId: string, opts: { project: string; simulate: boolean }) => {
.action(async (runId: string, opts: { project: string }) => {
const runDir = resolveRunDir(runId, opts.project);
if (!fs.existsSync(runDir)) {
@@ -303,20 +234,15 @@ export function registerForgeCommand(parent: Command): void {
}
console.log(`[forge] resuming run: ${runId}`);
if (opts.simulate) {
console.log('[forge] mode: SIMULATED (explicit --simulate)');
}
// No real executor is wired at CLI invocation time; only the explicitly
// requested simulated executor may be constructed (fail closed otherwise).
const executor = opts.simulate ? createSimulatedExecutor() : undefined;
try {
const { resumePipeline } = await import('./pipeline-runner.js');
const result = await resumePipeline(runDir, executor, { simulate: opts.simulate });
applyRunExitPolicy(result, opts.simulate);
const result = await resumePipeline(runDir, stubExecutor);
console.log(`[forge] pipeline complete: ${result.runId}`);
console.log(`[forge] run directory: ${result.runDir}`);
} catch (err) {
handlePipelineError(err);
console.error(`[forge] resume failed: ${err instanceof Error ? err.message : String(err)}`);
process.exitCode = 1;
}
});
+12 -72
View File
@@ -9,16 +9,7 @@ export const PACKAGE_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.
/** Pipeline asset directory (stages, agents, rails, gates, templates). */
export const PIPELINE_DIR = path.join(PACKAGE_ROOT, 'pipeline');
/** Stage specifications — defines every pipeline stage.
*\n * Gate semantics (SDLC-D-035): every gate is one of
* - a real command string / GateEntry a mechanical runner can execute,
* - an `authority` gate (human/board sign-off; produces waiting-for-authority),
* - a `provider` gate (requires a wired provider such as a reviewer or CI pipeline).
*
* Vacuous gates (`true`, echo'd synthetic approvals, placeholder ci-pipeline
* commands) are forbidden: a stage whose gate has no real implementation
* fails closed instead of passing.
*/
/** Stage specifications — defines every pipeline stage. */
export const STAGE_SPECS: Record<string, StageSpec> = {
'00-intake': {
number: '00',
@@ -36,13 +27,7 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
type: 'research',
gate: 'discovery-complete',
promptFile: '00b-discovery.md',
qualityGates: [
{
kind: 'authority',
capability: 'discovery-complete',
reason: 'discovery completion is attested by an authority; no mechanical check exists',
},
],
qualityGates: ['true'],
},
'01-board': {
number: '01',
@@ -51,13 +36,7 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
type: 'review',
gate: 'board-approval',
promptFile: '01-board.md',
qualityGates: [
{
kind: 'authority',
capability: 'board-approval',
reason: 'board approval is a board/human decision; no mechanical gate exists',
},
],
qualityGates: [{ type: 'ci-pipeline', command: 'board-approval (via board-tasks)' }],
},
'01b-brief-analyzer': {
number: '01b',
@@ -66,13 +45,7 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
type: 'research',
gate: 'brief-analysis-complete',
promptFile: '01-board.md',
qualityGates: [
{
kind: 'authority',
capability: 'brief-analysis-complete',
reason: 'brief analysis completion is attested by an authority; no mechanical check exists',
},
],
qualityGates: ['true'],
},
'02-planning-1': {
number: '02',
@@ -81,13 +54,7 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
type: 'research',
gate: 'architecture-approval',
promptFile: '02-planning-1-architecture.md',
qualityGates: [
{
kind: 'authority',
capability: 'architecture-approval',
reason: 'ADR approval requires authority sign-off; no mechanical check exists',
},
],
qualityGates: ['true'],
},
'03-planning-2': {
number: '03',
@@ -96,14 +63,7 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
type: 'research',
gate: 'implementation-approval',
promptFile: '03-planning-2-implementation.md',
qualityGates: [
{
kind: 'authority',
capability: 'implementation-approval',
reason:
'implementation spec approval requires authority sign-off; no mechanical check exists',
},
],
qualityGates: ['true'],
},
'04-planning-3': {
number: '04',
@@ -112,14 +72,7 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
type: 'research',
gate: 'decomposition-approval',
promptFile: '04-planning-3-decomposition.md',
qualityGates: [
{
kind: 'authority',
capability: 'decomposition-approval',
reason:
'task decomposition approval requires authority sign-off; no mechanical check exists',
},
],
qualityGates: ['true'],
},
'05-coding': {
number: '05',
@@ -139,10 +92,9 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
promptFile: '06-review.md',
qualityGates: [
{
kind: 'provider',
capability: 'reviewer',
reason:
'review verdicts require a wired reviewer provider; synthetic approvals are not permitted',
type: 'ai-review',
command:
'echo \'{"summary":"review-pass","verdict":"approve","findings":[],"stats":{"blockers":0,"should_fix":0,"suggestions":0}}\'',
},
],
},
@@ -153,13 +105,7 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
type: 'coding',
gate: 're-review',
promptFile: '07-remediate.md',
qualityGates: [
{
kind: 'authority',
capability: 're-review',
reason: 'remediation re-review is an approval-based gate; no mechanical check exists',
},
],
qualityGates: ['true'],
},
'08-test': {
number: '08',
@@ -177,13 +123,7 @@ export const STAGE_SPECS: Record<string, StageSpec> = {
type: 'deploy',
gate: 'deploy-verification',
promptFile: '09-deploy.md',
qualityGates: [
{
kind: 'provider',
capability: 'ci-pipeline',
reason: 'deploy verification requires a wired CI pipeline provider',
},
],
qualityGates: [{ type: 'ci-pipeline', command: 'deploy-verification' }],
},
};
-46
View File
@@ -1,46 +0,0 @@
/**
* Typed fail-closed capability errors (SDLC-D-035).
*
* A Forge run must fail closed when a required capability (executor, reviewer
* provider, CI pipeline, authority sign-off) is missing. These typed errors
* name the missing capability so callers can distinguish "not wired" from
* ordinary execution failures.
*/
/** Closed set of typed Forge capability error codes. */
export const FORGE_ERROR_CODES = [
'FORGE_NO_EXECUTOR',
'FORGE_NO_REVIEWER',
'FORGE_NO_CI_PIPELINE',
'FORGE_NO_PROVIDER',
'FORGE_AUTHORITY_REQUIRED',
] as const;
export type ForgeErrorCode = (typeof FORGE_ERROR_CODES)[number];
/** Raised when a required capability is missing and the pipeline must fail closed. */
export class ForgeCapabilityError extends Error {
/** Typed error code from the closed FORGE_ERROR_CODES set. */
readonly code: ForgeErrorCode;
/** The missing capability, e.g. `task-executor`, `reviewer`, `board-approval`. */
readonly capability: string;
constructor(code: ForgeErrorCode, capability: string, message: string) {
super(message);
this.name = 'ForgeCapabilityError';
this.code = code;
this.capability = capability;
}
}
/** Map a provider gate capability to its typed error code. */
export function providerErrorCode(capability: string): ForgeErrorCode {
switch (capability) {
case 'reviewer':
return 'FORGE_NO_REVIEWER';
case 'ci-pipeline':
return 'FORGE_NO_CI_PIPELINE';
default:
return 'FORGE_NO_PROVIDER';
}
}
-26
View File
@@ -5,13 +5,6 @@ export type {
StageSpec,
BriefClass,
ClassSource,
ForgeOutcome,
AuthorityGate,
ProviderGate,
ForgeGate,
ForgeGateResult,
ForgeTaskResult,
RunMode,
StageStatus,
RunManifest,
ForgeTaskStatus,
@@ -88,24 +81,5 @@ export {
getPipelineStatus,
} from './pipeline-runner.js';
// Fail-closed errors and typed outcome model (SDLC-D-035)
export { FORGE_ERROR_CODES, ForgeCapabilityError, providerErrorCode } from './errors.js';
export type { ForgeErrorCode } from './errors.js';
export {
isSatisfyingOutcome,
isCapabilityGate,
isCommandGate,
gateLabel,
uniformGateResults,
simulatedGateResults,
waitingGateResults,
blockedGateResults,
evaluateStageGates,
} from './outcomes.js';
export type { StageEvaluation } from './outcomes.js';
// Simulated executor (explicit --simulate only)
export { createSimulatedExecutor } from './simulated-executor.js';
// CLI
export { registerForgeCommand } from './cli.js';
-147
View File
@@ -1,147 +0,0 @@
import type { GateEntry } from '@mosaicstack/macp';
import type {
AuthorityGate,
ForgeGate,
ForgeGateResult,
ForgeOutcome,
ForgeTaskResult,
ProviderGate,
} from './types.js';
/**
* Gate and dependency satisfaction predicate (SDLC-D-035).
*
* ONLY a verified `passed` outcome satisfies. Every other member of the closed
* outcome set — including `simulated` — is non-satisfying, so a simulated or
* authority-blocked result can never be read as success-by-verification.
*/
export function isSatisfyingOutcome(outcome: ForgeOutcome): boolean {
return outcome === 'passed';
}
/** Whether a gate is an authority or provider gate (capability-based, command-less). */
export function isCapabilityGate(gate: ForgeGate): gate is AuthorityGate | ProviderGate {
if (typeof gate !== 'object' || gate === null) return false;
const kind = (gate as Record<string, unknown>)['kind'];
return kind === 'authority' || kind === 'provider';
}
/** Whether a gate definition carries a real command a mechanical runner can execute. */
export function isCommandGate(gate: ForgeGate): gate is string | GateEntry {
if (typeof gate === 'string') {
return gate.trim().length > 0;
}
if (isCapabilityGate(gate)) {
// Authority and provider gates are satisfied by a capability, not a command.
return false;
}
return typeof gate.command === 'string' && gate.command.trim().length > 0;
}
/** Typed label identifying a gate in results and logs. */
export function gateLabel(gate: ForgeGate): string {
if (typeof gate === 'string') return gate;
if (isCapabilityGate(gate)) return `${gate.kind}:${gate.capability}`;
return gate.command || gate.type || 'unnamed-gate';
}
/** Reason string stamped on every simulated gate result. */
export const SIMULATED_GATE_REASON =
'simulated execution (--simulate): gate was not evaluated by a real implementation';
/** Build typed gate results with a uniform outcome for a stage's declared gates. */
export function uniformGateResults(
gates: ForgeGate[],
outcome: ForgeOutcome,
reason: string,
): ForgeGateResult[] {
return gates.map((gate) => ({ gate: gateLabel(gate), outcome, reason }));
}
/** Typed simulated gate results — used exclusively in `--simulate` runs. */
export function simulatedGateResults(gates: ForgeGate[]): ForgeGateResult[] {
return uniformGateResults(gates, 'simulated', SIMULATED_GATE_REASON);
}
/** Typed waiting-for-authority gate results for approval-based stages. */
export function waitingGateResults(gates: ForgeGate[], reason: string): ForgeGateResult[] {
return uniformGateResults(gates, 'waiting-for-authority', reason);
}
/** Typed blocked gate results for stages whose provider capability is not wired. */
export function blockedGateResults(gates: ForgeGate[], reason: string): ForgeGateResult[] {
return uniformGateResults(gates, 'blocked', reason);
}
/** Outcome of evaluating a completed stage in normal mode. */
export interface StageEvaluation {
outcome: ForgeOutcome;
reason: string;
gateResults: ForgeGateResult[];
}
/**
* Evaluate a stage's declared gates against the executor's typed result.
*
* Fail-closed mapping:
* - a `simulated` task or gate outcome in normal mode maps to `error`
* - a missing gate result for a required command gate maps to `blocked`
* - a non-passing task outcome propagates as the stage outcome
* - only verified `passed` task and gate outcomes yield a `passed` stage
*/
export function evaluateStageGates(
stageName: string,
gates: ForgeGate[],
result: ForgeTaskResult,
): StageEvaluation {
const gateResults = result.gate_results ?? [];
if (result.outcome === 'simulated') {
return {
outcome: 'error',
reason: `executor reported a simulated outcome for stage '${stageName}' in normal mode — refusing to treat simulated results as verified`,
gateResults,
};
}
if (!isSatisfyingOutcome(result.outcome)) {
return {
outcome: result.outcome,
reason: `task outcome is '${result.outcome}': ${result.reason}`,
gateResults,
};
}
for (const gate of gates) {
// Authority and provider gates are pre-flighted before execution; they have
// no mechanical result to verify here.
if (!isCommandGate(gate)) continue;
const label = gateLabel(gate);
const gateResult = gateResults.find((r) => r.gate === label);
if (!gateResult) {
return {
outcome: 'blocked',
reason: `no gate result was reported for required gate '${label}' (stage '${stageName}')`,
gateResults,
};
}
if (!isSatisfyingOutcome(gateResult.outcome)) {
return {
outcome: gateResult.outcome === 'simulated' ? 'error' : gateResult.outcome,
reason: `gate '${label}' outcome is '${gateResult.outcome}': ${gateResult.reason}`,
gateResults,
};
}
}
return {
outcome: 'passed',
reason:
gates.length === 0
? "stage declares no gates; task outcome 'passed' accepted"
: 'all declared gates verified passed',
gateResults,
};
}
+99 -227
View File
@@ -1,33 +1,18 @@
import fs from 'node:fs';
import path from 'node:path';
import { STAGE_SEQUENCE, STAGE_SPECS } from './constants.js';
import { STAGE_SEQUENCE } from './constants.js';
import { determineBriefClass, stagesForClass } from './brief-classifier.js';
import { ForgeCapabilityError, providerErrorCode } from './errors.js';
import {
blockedGateResults,
evaluateStageGates,
isCapabilityGate,
simulatedGateResults,
waitingGateResults,
} from './outcomes.js';
import { mapStageToTask } from './stage-adapter.js';
import { createSimulatedExecutor } from './simulated-executor.js';
import type {
ForgeTask,
ForgeTaskResult,
PipelineOptions,
PipelineResult,
RunManifest,
RunMode,
StageStatus,
TaskExecutor,
} from './types.js';
/** Reason stamped on stages that complete under explicit simulation. */
const SIMULATED_STAGE_REASON =
'simulated execution (--simulate): stage was not executed by a real executor';
/**
* Generate a timestamp-based run ID.
*/
@@ -62,7 +47,6 @@ function createManifest(opts: {
briefClass: RunManifest['briefClass'];
classSource: RunManifest['classSource'];
forceBoard: boolean;
mode: RunMode;
runDir: string;
}): RunManifest {
const ts = nowISO();
@@ -73,7 +57,6 @@ function createManifest(opts: {
briefClass: opts.briefClass,
classSource: opts.classSource,
forceBoard: opts.forceBoard,
mode: opts.mode,
createdAt: ts,
updatedAt: ts,
currentStage: '',
@@ -125,199 +108,20 @@ export function selectStages(stages?: string[], skipTo?: string): string[] {
return selected.slice(skipIndex);
}
/**
* Fail closed when the required executor capability is missing (SDLC-D-035).
*/
function requireExecutor(executor: TaskExecutor | undefined, simulate: boolean): TaskExecutor {
if (executor) return executor;
if (simulate) return createSimulatedExecutor({ log: false });
throw new ForgeCapabilityError(
'FORGE_NO_EXECUTOR',
'task-executor',
'no task executor is wired; refusing to run the pipeline with a stub executor (fail closed). ' +
'Pass --simulate to opt into explicitly simulated execution.',
);
}
/**
* Pre-flight a stage's gates in normal mode (fail closed, SDLC-D-035).
*
* - authority gates: record a typed `waiting-for-authority` stage result and
* raise FORGE_AUTHORITY_REQUIRED — approval-based gates never pass vacuously.
* - provider gates: record a typed `blocked` stage result and raise the typed
* capability error for the missing provider.
*
* Returns the stage status to record when the pre-flight blocks, or undefined
* when the stage may proceed.
*/
function preflightStageGates(
stageName: string,
manifest: RunManifest,
): { status: StageStatus; error: ForgeCapabilityError } | undefined {
const spec = STAGE_SPECS[stageName];
if (!spec) throw new Error(`Unknown Forge stage: ${stageName}`);
for (const gate of spec.qualityGates) {
if (!isCapabilityGate(gate)) continue;
const startedAt = manifest.stages[stageName]?.startedAt;
const completedAt = nowISO();
if (gate.kind === 'authority') {
const reason = `gate '${gate.capability}' requires authority sign-off; no mechanical implementation exists (${gate.reason})`;
return {
status: {
status: 'waiting-for-authority',
reason,
startedAt,
completedAt,
gateResults: waitingGateResults(spec.qualityGates, reason),
},
error: new ForgeCapabilityError(
'FORGE_AUTHORITY_REQUIRED',
gate.capability,
`stage '${stageName}' is blocked on authority gate '${gate.capability}': ${gate.reason}. ` +
'The pipeline fails closed instead of passing vacuously. Record the approval out-of-band ' +
'or run with --simulate for explicitly simulated execution.',
),
};
}
const reason = `gate '${gate.capability}' requires provider '${gate.capability}' and none is wired (${gate.reason})`;
return {
status: {
status: 'blocked',
reason,
startedAt,
completedAt,
gateResults: blockedGateResults(spec.qualityGates, reason),
},
error: new ForgeCapabilityError(
providerErrorCode(gate.capability),
gate.capability,
`stage '${stageName}' requires provider '${gate.capability}' which is not wired: ${gate.reason}. ` +
'The pipeline fails closed instead of passing vacuously.',
),
};
}
return undefined;
}
/**
* Execute the given stage tasks sequentially, updating the manifest.
*
* Normal mode requires a real executor and evaluates every declared command
* gate through the typed outcome model; any non-verified result fails closed.
* Simulate mode types every stage and gate result as `simulated`.
*/
async function executeStages(opts: {
manifest: RunManifest;
runDir: string;
tasks: ForgeTask[];
stageNames: string[];
executor: TaskExecutor;
simulate: boolean;
}): Promise<void> {
const { manifest, runDir, tasks, stageNames, executor, simulate } = opts;
for (let i = 0; i < tasks.length; i++) {
const task = tasks[i]!;
const stageName = stageNames[i]!;
const spec = STAGE_SPECS[stageName];
if (!spec) throw new Error(`Unknown Forge stage: ${stageName}`);
// Update manifest: stage in progress
manifest.currentStage = stageName;
manifest.stages[stageName] = {
status: 'in_progress',
startedAt: nowISO(),
};
saveManifest(runDir, manifest);
// Fail-closed pre-flight (normal mode only): authority/provider gates have
// no mechanical implementation and must never pass vacuously.
if (!simulate) {
const blocked = preflightStageGates(stageName, manifest);
if (blocked) {
manifest.stages[stageName] = blocked.status;
manifest.status =
blocked.status.status === 'waiting-for-authority' ? 'waiting-for-authority' : 'failed';
saveManifest(runDir, manifest);
throw blocked.error;
}
}
let result: ForgeTaskResult;
try {
await executor.submitTask(task);
result = await executor.waitForCompletion(task.id, task.timeoutSeconds * 1000);
} catch (error) {
// Process errors (including timeouts) map to the fail-closed `error` outcome.
const reason = error instanceof Error ? error.message : String(error);
manifest.stages[stageName] = {
status: 'error',
reason: `executor error: ${reason}`,
startedAt: manifest.stages[stageName]?.startedAt,
completedAt: nowISO(),
gateResults: [],
};
manifest.status = 'failed';
saveManifest(runDir, manifest);
throw error instanceof Error ? error : new Error(reason);
}
if (simulate) {
manifest.stages[stageName] = {
status: 'simulated',
reason: SIMULATED_STAGE_REASON,
startedAt: manifest.stages[stageName]?.startedAt,
completedAt: nowISO(),
gateResults: simulatedGateResults(spec.qualityGates),
};
saveManifest(runDir, manifest);
continue;
}
const evaluation = evaluateStageGates(stageName, spec.qualityGates, result);
manifest.stages[stageName] = {
status: evaluation.outcome,
reason: evaluation.reason,
startedAt: manifest.stages[stageName]?.startedAt,
completedAt: nowISO(),
gateResults: evaluation.gateResults,
};
if (evaluation.outcome !== 'passed') {
manifest.status =
evaluation.outcome === 'waiting-for-authority' ? 'waiting-for-authority' : 'failed';
saveManifest(runDir, manifest);
throw new Error(`Stage ${stageName} ${evaluation.outcome}: ${evaluation.reason}`);
}
saveManifest(runDir, manifest);
}
}
/**
* Run the Forge pipeline.
*
* 1. Fail closed unless a real executor is wired or simulation is explicit
* 2. Classify the brief
* 3. Generate a run ID and create run directory
* 4. Map stages to tasks and submit to TaskExecutor
* 5. Track manifest with typed stage outcomes
* 6. Return pipeline result
* 1. Classify the brief
* 2. Generate a run ID and create run directory
* 3. Map stages to tasks and submit to TaskExecutor
* 4. Track manifest with stage statuses
* 5. Return pipeline result
*/
export async function runPipeline(
briefPath: string,
projectRoot: string,
options: PipelineOptions,
): Promise<PipelineResult> {
const simulate = options.simulate ?? false;
const executor = requireExecutor(options.executor, simulate);
const mode: RunMode = simulate ? 'simulated' : 'normal';
const resolvedRoot = path.resolve(projectRoot);
const resolvedBrief = path.resolve(briefPath);
const briefContent = fs.readFileSync(resolvedBrief, 'utf-8');
@@ -342,7 +146,6 @@ export async function runPipeline(
briefClass,
classSource,
forceBoard: options.forceBoard ?? false,
mode,
runDir,
});
@@ -369,10 +172,54 @@ export async function runPipeline(
}
// Execute stages
await executeStages({ manifest, runDir, tasks, stageNames: selectedStages, executor, simulate });
const { executor } = options;
for (let i = 0; i < tasks.length; i++) {
const task = tasks[i]!;
const stageName = selectedStages[i]!;
// All stages reached a terminal state for this mode
manifest.status = simulate ? 'simulated' : 'completed';
// Update manifest: stage in progress
manifest.currentStage = stageName;
manifest.stages[stageName] = {
status: 'in_progress',
startedAt: nowISO(),
};
saveManifest(runDir, manifest);
try {
await executor.submitTask(task);
const result = await executor.waitForCompletion(task.id, task.timeoutSeconds * 1000);
// Update manifest: stage completed or failed
const stageStatus: StageStatus = {
status: result.status === 'completed' ? 'passed' : 'failed',
startedAt: manifest.stages[stageName]!.startedAt,
completedAt: nowISO(),
};
manifest.stages[stageName] = stageStatus;
if (result.status !== 'completed') {
manifest.status = 'failed';
saveManifest(runDir, manifest);
throw new Error(`Stage ${stageName} failed with status: ${result.status}`);
}
saveManifest(runDir, manifest);
} catch (error) {
if (!manifest.stages[stageName]?.completedAt) {
manifest.stages[stageName] = {
status: 'failed',
startedAt: manifest.stages[stageName]?.startedAt,
completedAt: nowISO(),
};
}
manifest.status = 'failed';
saveManifest(runDir, manifest);
throw error;
}
}
// All stages passed
manifest.status = 'completed';
saveManifest(runDir, manifest);
return {
@@ -387,30 +234,22 @@ export async function runPipeline(
}
/**
* Resume a pipeline from the last non-passed stage.
* Resume a pipeline from the last incomplete stage.
*/
export async function resumePipeline(
runDir: string,
executor?: TaskExecutor,
options?: { simulate?: boolean },
executor: TaskExecutor,
): Promise<PipelineResult> {
const simulate = options?.simulate ?? false;
const wiredExecutor = requireExecutor(executor, simulate);
const mode: RunMode = simulate ? 'simulated' : 'normal';
const manifest = loadManifest(runDir);
const resolvedRoot = path.dirname(path.dirname(path.dirname(runDir))); // .forge/runs/{id} → project root
const briefContent = fs.readFileSync(manifest.brief, 'utf-8');
const allStages = stagesForClass(manifest.briefClass, manifest.forceBoard);
manifest.mode = mode;
// Find first non-satisfying stage (only a verified `passed` counts as done;
// simulated and waiting-for-authority stages are re-run).
// Find first non-passed stage
const resumeFrom = allStages.find((s) => manifest.stages[s]?.status !== 'passed');
if (!resumeFrom) {
manifest.status = mode === 'simulated' ? 'simulated' : 'completed';
manifest.status = 'completed';
saveManifest(runDir, manifest);
return {
runId: manifest.runId,
@@ -445,16 +284,49 @@ export async function resumePipeline(
tasks.push(task);
}
await executeStages({
manifest,
runDir,
tasks,
stageNames: remainingStages,
executor: wiredExecutor,
simulate,
});
for (let i = 0; i < tasks.length; i++) {
const task = tasks[i]!;
const stageName = remainingStages[i]!;
manifest.status = simulate ? 'simulated' : 'completed';
manifest.currentStage = stageName;
manifest.stages[stageName] = {
status: 'in_progress',
startedAt: nowISO(),
};
saveManifest(runDir, manifest);
try {
await executor.submitTask(task);
const result = await executor.waitForCompletion(task.id, task.timeoutSeconds * 1000);
manifest.stages[stageName] = {
status: result.status === 'completed' ? 'passed' : 'failed',
startedAt: manifest.stages[stageName]!.startedAt,
completedAt: nowISO(),
};
if (result.status !== 'completed') {
manifest.status = 'failed';
saveManifest(runDir, manifest);
throw new Error(`Stage ${stageName} failed with status: ${result.status}`);
}
saveManifest(runDir, manifest);
} catch (error) {
if (!manifest.stages[stageName]?.completedAt) {
manifest.stages[stageName] = {
status: 'failed',
startedAt: manifest.stages[stageName]?.startedAt,
completedAt: nowISO(),
};
}
manifest.status = 'failed';
saveManifest(runDir, manifest);
throw error;
}
}
manifest.status = 'completed';
saveManifest(runDir, manifest);
return {
-32
View File
@@ -1,32 +0,0 @@
import type { ForgeTask, ForgeTaskResult, TaskExecutor } from './types.js';
/**
* Simulated executor — used ONLY when the caller explicitly passes --simulate.
*
* It submits no real work and returns typed `simulated` results so a simulated
* run can never be confused with a verified one. In normal mode (no --simulate)
* the CLI refuses to run at all with FORGE_NO_EXECUTOR instead of wiring this
* stub (fail closed, SDLC-D-035).
*/
export function createSimulatedExecutor(options?: { log?: boolean }): TaskExecutor {
const log = options?.log ?? true;
return {
async submitTask(task: ForgeTask) {
if (log) console.log(` [forge:simulated] stage submitted: ${task.id} (${task.title})`);
},
async waitForCompletion(taskId: string): Promise<ForgeTaskResult> {
if (log) console.log(` [forge:simulated] stage complete: ${taskId}`);
return {
task_id: taskId,
outcome: 'simulated',
reason: 'no executor wired; simulated execution requested via --simulate',
completed_at: new Date().toISOString(),
exit_code: 0,
gate_results: [],
};
},
async getTaskStatus() {
return 'completed' as const;
},
};
}
+7 -88
View File
@@ -1,4 +1,4 @@
import type { GateEntry } from '@mosaicstack/macp';
import type { GateEntry, TaskResult } from '@mosaicstack/macp';
/** Stage dispatch mode. */
export type StageDispatch = 'exec' | 'yolo' | 'pi';
@@ -6,58 +6,6 @@ export type StageDispatch = 'exec' | 'yolo' | 'pi';
/** Stage type — determines agent selection and gate requirements. */
export type StageType = 'research' | 'review' | 'coding' | 'deploy';
/**
* Typed outcome for every gate and stage evaluation — closed set (SDLC-D-035).
*
* Only `passed` means "verified by a real implementation". `simulated` is
* produced exclusively in explicit `--simulate` runs and is never satisfying.
*/
export type ForgeOutcome =
| 'passed'
| 'failed'
| 'blocked'
| 'error'
| 'waiting-for-authority'
| 'simulated'
| 'not-applicable';
/** A gate that requires authority (human/board) sign-off; no mechanical command can satisfy it. */
export interface AuthorityGate {
kind: 'authority';
capability: string;
reason: string;
}
/** A gate that requires a wired provider (e.g. an AI reviewer, CI pipeline) to evaluate. */
export interface ProviderGate {
kind: 'provider';
capability: string;
reason: string;
}
/** Forge quality gate: a real command, an authority sign-off, or a provider-backed check. */
export type ForgeGate = string | GateEntry | AuthorityGate | ProviderGate;
/** Typed result of evaluating a single quality gate. */
export interface ForgeGateResult {
gate: string;
outcome: ForgeOutcome;
reason: string;
exitCode?: number;
output?: string;
timedOut?: boolean;
}
/** Typed result of a task/stage execution returned by a TaskExecutor. */
export interface ForgeTaskResult {
task_id: string;
outcome: ForgeOutcome;
reason: string;
completed_at: string;
exit_code: number;
gate_results: ForgeGateResult[];
}
/** Stage specification — defines a single pipeline stage. */
export interface StageSpec {
number: string;
@@ -66,7 +14,7 @@ export interface StageSpec {
type: StageType;
gate: string;
promptFile: string;
qualityGates: ForgeGate[];
qualityGates: (string | GateEntry)[];
}
/** Brief classification. */
@@ -77,18 +25,11 @@ export type ClassSource = 'cli' | 'frontmatter' | 'auto';
/** Per-stage status within a run manifest. */
export interface StageStatus {
status: 'pending' | 'in_progress' | ForgeOutcome;
/** Why the stage reached its current (terminal) outcome, when applicable. */
reason?: string;
status: 'pending' | 'in_progress' | 'passed' | 'failed';
startedAt?: string;
completedAt?: string;
/** Typed per-gate results recorded alongside the stage outcome. */
gateResults?: ForgeGateResult[];
}
/** Execution mode of a run. */
export type RunMode = 'normal' | 'simulated';
/** Run manifest — persisted to disk as manifest.json. */
export interface RunManifest {
runId: string;
@@ -97,23 +38,10 @@ export interface RunManifest {
briefClass: BriefClass;
classSource: ClassSource;
forceBoard: boolean;
/**
* Execution mode. `simulated` runs stub execution; their results are typed
* `simulated` and must never be read as verified success. Optional because
* manifests written before this field existed default to `normal`.
*/
mode?: RunMode;
createdAt: string;
updatedAt: string;
currentStage: string;
status:
| 'in_progress'
| 'completed'
| 'failed'
| 'interrupted'
| 'rejected'
| 'simulated'
| 'waiting-for-authority';
status: 'in_progress' | 'completed' | 'failed' | 'interrupted' | 'rejected';
stages: Record<string, StageStatus>;
}
@@ -137,7 +65,7 @@ export interface ForgeTask {
briefPath: string;
resultPath: string;
timeoutSeconds: number;
qualityGates: ForgeGate[];
qualityGates: (string | GateEntry)[];
worktree?: string;
command?: string;
dependsOn?: string[];
@@ -148,7 +76,7 @@ export interface ForgeTask {
/** Abstract task executor — decouples from packages/coord. */
export interface TaskExecutor {
submitTask(task: ForgeTask): Promise<void>;
waitForCompletion(taskId: string, timeoutMs: number): Promise<ForgeTaskResult>;
waitForCompletion(taskId: string, timeoutMs: number): Promise<TaskResult>;
getTaskStatus(taskId: string): Promise<ForgeTaskStatus>;
}
@@ -194,16 +122,7 @@ export interface PipelineOptions {
stages?: string[];
skipTo?: string;
dryRun?: boolean;
/**
* Real task executor. Required in normal mode: the pipeline fails closed
* with FORGE_NO_EXECUTOR when it is absent.
*/
executor?: TaskExecutor;
/**
* Explicit opt-in to simulated execution. Every stage and gate result is
* typed `simulated` and is never satisfying.
*/
simulate?: boolean;
executor: TaskExecutor;
}
/** Pipeline run result. */
-253
View File
@@ -1,253 +0,0 @@
import { mkdirSync, readFileSync, rmSync } from 'node:fs';
import { join } from 'node:path';
import { tmpdir } from 'node:os';
import { randomUUID } from 'node:crypto';
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import { normalizeGate, countAIFindings, runGate, runGates } from '../src/gate-runner.js';
function makeTmpDir(): string {
const dir = join(tmpdir(), `macp-gate-${randomUUID()}`);
mkdirSync(dir, { recursive: true });
return dir;
}
describe('normalizeGate', () => {
it('normalizes a string to mechanical gate', () => {
expect(normalizeGate('echo test')).toEqual({
command: 'echo test',
type: 'mechanical',
fail_on: 'blocker',
});
});
it('normalizes an object gate with defaults', () => {
expect(normalizeGate({ command: 'lint' })).toEqual({
command: 'lint',
type: 'mechanical',
fail_on: 'blocker',
});
});
it('preserves explicit type and fail_on', () => {
expect(normalizeGate({ command: 'review', type: 'ai-review', fail_on: 'any' })).toEqual({
command: 'review',
type: 'ai-review',
fail_on: 'any',
});
});
it('handles non-string/non-object input', () => {
expect(normalizeGate(42)).toEqual({ command: '', type: 'mechanical', fail_on: 'blocker' });
expect(normalizeGate(null)).toEqual({ command: '', type: 'mechanical', fail_on: 'blocker' });
});
});
describe('countAIFindings', () => {
it('returns zeros for non-object', () => {
expect(countAIFindings(null)).toEqual({ blockers: 0, total: 0 });
expect(countAIFindings('string')).toEqual({ blockers: 0, total: 0 });
expect(countAIFindings([])).toEqual({ blockers: 0, total: 0 });
});
it('counts from stats block', () => {
const output = { stats: { blockers: 2, should_fix: 3, suggestions: 1 } };
expect(countAIFindings(output)).toEqual({ blockers: 2, total: 6 });
});
it('counts from findings array when stats has no blockers', () => {
const output = {
stats: { blockers: 0 },
findings: [{ severity: 'blocker' }, { severity: 'warning' }, { severity: 'blocker' }],
};
expect(countAIFindings(output)).toEqual({ blockers: 2, total: 3 });
});
it('uses stats blockers over findings array when stats has blockers', () => {
const output = {
stats: { blockers: 5 },
findings: [{ severity: 'blocker' }, { severity: 'warning' }],
};
// stats.blockers = 5, total from stats = 5+0+0 = 5, findings not used for total since stats total is non-zero
expect(countAIFindings(output)).toEqual({ blockers: 5, total: 5 });
});
it('counts findings length as total when stats has zero total', () => {
const output = {
findings: [{ severity: 'warning' }, { severity: 'info' }],
};
expect(countAIFindings(output)).toEqual({ blockers: 0, total: 2 });
});
});
describe('runGate', () => {
let tmp: string;
let logPath: string;
beforeEach(() => {
tmp = makeTmpDir();
logPath = join(tmp, 'gate.log');
});
afterEach(() => {
rmSync(tmp, { recursive: true, force: true });
});
it('passes mechanical gate on exit 0', () => {
const result = runGate('echo hello', tmp, logPath, 30);
expect(result.passed).toBe(true);
expect(result.exit_code).toBe(0);
expect(result.type).toBe('mechanical');
expect(result.output).toContain('hello');
});
it('fails mechanical gate on non-zero exit', () => {
const result = runGate('exit 1', tmp, logPath, 30);
expect(result.passed).toBe(false);
expect(result.exit_code).toBe(1);
});
it('ci-pipeline always passes', () => {
const result = runGate({ command: 'anything', type: 'ci-pipeline' }, tmp, logPath, 30);
expect(result.passed).toBe(true);
expect(result.type).toBe('ci-pipeline');
expect(result.output).toBe('CI pipeline gate placeholder');
});
it('empty command passes', () => {
const result = runGate({ command: '' }, tmp, logPath, 30);
expect(result.passed).toBe(true);
});
it('ai-review gate parses JSON output', () => {
const json = JSON.stringify({ stats: { blockers: 0, should_fix: 1 } });
const result = runGate({ command: `echo '${json}'`, type: 'ai-review' }, tmp, logPath, 30);
expect(result.passed).toBe(true);
expect(result.blockers).toBe(0);
expect(result.findings).toBe(1);
});
it('ai-review gate fails on blockers', () => {
const json = JSON.stringify({ stats: { blockers: 2 } });
const result = runGate({ command: `echo '${json}'`, type: 'ai-review' }, tmp, logPath, 30);
expect(result.passed).toBe(false);
expect(result.blockers).toBe(2);
});
it('ai-review gate with fail_on=any fails on any findings', () => {
const json = JSON.stringify({ stats: { blockers: 0, should_fix: 1 } });
const result = runGate(
{ command: `echo '${json}'`, type: 'ai-review', fail_on: 'any' },
tmp,
logPath,
30,
);
expect(result.passed).toBe(false);
expect(result.fail_on).toBe('any');
});
it('ai-review gate fails on invalid JSON output', () => {
const result = runGate({ command: 'echo "not json"', type: 'ai-review' }, tmp, logPath, 30);
expect(result.passed).toBe(false);
expect(result.parse_error).toBeDefined();
});
it('writes to log file', () => {
runGate('echo logged', tmp, logPath, 30);
const log = readFileSync(logPath, 'utf-8');
expect(log).toContain('COMMAND: echo logged');
expect(log).toContain('logged');
expect(log).toContain('EXIT:');
});
});
describe('runGates', () => {
let tmp: string;
let logPath: string;
let eventsPath: string;
beforeEach(() => {
tmp = makeTmpDir();
logPath = join(tmp, 'gates.log');
eventsPath = join(tmp, 'events.ndjson');
});
afterEach(() => {
rmSync(tmp, { recursive: true, force: true });
});
it('runs multiple gates and returns results', () => {
const { allPassed, gateResults } = runGates(
['echo one', 'echo two'],
tmp,
logPath,
30,
eventsPath,
'task-1',
);
expect(allPassed).toBe(true);
expect(gateResults).toHaveLength(2);
});
it('reports failure when any gate fails', () => {
const { allPassed, gateResults } = runGates(
['echo ok', 'exit 1'],
tmp,
logPath,
30,
eventsPath,
'task-2',
);
expect(allPassed).toBe(false);
expect(gateResults[0]!.passed).toBe(true);
expect(gateResults[1]!.passed).toBe(false);
});
it('emits events for each gate', () => {
runGates(['echo test'], tmp, logPath, 30, eventsPath, 'task-3');
const events = readFileSync(eventsPath, 'utf-8')
.trim()
.split('\n')
.map((l) => JSON.parse(l));
expect(events).toHaveLength(2); // started + passed
expect(events[0].event_type).toBe('rail.check.started');
expect(events[1].event_type).toBe('rail.check.passed');
});
it('skips gates with empty command (non ci-pipeline)', () => {
const { gateResults } = runGates(
[{ command: '', type: 'mechanical' }, 'echo real'],
tmp,
logPath,
30,
eventsPath,
'task-4',
);
expect(gateResults).toHaveLength(1);
});
it('does not skip ci-pipeline even with empty command', () => {
const { gateResults } = runGates(
[{ command: '', type: 'ci-pipeline' }],
tmp,
logPath,
30,
eventsPath,
'task-5',
);
expect(gateResults).toHaveLength(1);
expect(gateResults[0]!.passed).toBe(true);
});
it('emits failed event with correct message', () => {
runGates(['exit 42'], tmp, logPath, 30, eventsPath, 'task-6');
const events = readFileSync(eventsPath, 'utf-8')
.trim()
.split('\n')
.map((l) => JSON.parse(l));
const failEvent = events.find(
(e: Record<string, unknown>) => e.event_type === 'rail.check.failed',
);
expect(failEvent).toBeDefined();
expect(failEvent.message).toContain('Gate failed (');
});
});
+163 -1
View File
@@ -1,5 +1,8 @@
import { describe, it, expect } from 'vitest';
import { describe, it, expect, afterEach, beforeEach, vi } from 'vitest';
import { Command } from 'commander';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { registerMacpCommand } from './cli.js';
describe('registerMacpCommand', () => {
@@ -75,3 +78,162 @@ describe('registerMacpCommand', () => {
expect(topLevel).toContain('events');
});
});
/**
* RI-N2 fail-closed CLI behavior: an unimplemented capability is a failure,
* never a success. Every stub exits nonzero with a typed message, and the
* implemented `macp gate` mirrors the typed gate-runner states.
*/
describe('registerMacpCommand fail-closed (RI-N2)', () => {
let tmpDir: string;
function buildProgram(): Command {
const program = new Command();
program.exitOverride();
program.configureOutput({ writeErr: () => {} });
registerMacpCommand(program);
return program;
}
beforeEach(() => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'macp-cli-failclosed-'));
process.exitCode = 0;
});
afterEach(() => {
process.exitCode = 0;
fs.rmSync(tmpDir, { recursive: true, force: true });
});
it('macp tasks list exits nonzero (unimplemented capability)', async () => {
const program = buildProgram();
await program.parseAsync(['macp', 'tasks', 'list'], { from: 'user' });
expect(process.exitCode).not.toBe(0);
});
it('macp submit exits nonzero with a typed MACP_NOT_IMPLEMENTED message', async () => {
const program = buildProgram();
const errSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
try {
await program.parseAsync(['macp', 'submit', 'spec.json'], { from: 'user' });
expect(process.exitCode).not.toBe(0);
const errText = errSpy.mock.calls.map((c) => String(c[0])).join('\n');
expect(errText).toContain('MACP_NOT_IMPLEMENTED');
} finally {
errSpy.mockRestore();
}
});
it('macp events tail exits nonzero (unimplemented capability)', async () => {
const program = buildProgram();
await program.parseAsync(['macp', 'events', 'tail'], { from: 'user' });
expect(process.exitCode).not.toBe(0);
});
it('macp gate runs a green inline command and exits 0', async () => {
const program = buildProgram();
await program.parseAsync(
[
'macp',
'gate',
'exit 0',
'--cwd',
tmpDir,
'--log',
path.join(tmpDir, 'g.log'),
'--timeout',
'10',
],
{ from: 'user' },
);
expect(process.exitCode).toBe(0);
});
it('macp gate exits nonzero on a failing command', async () => {
const program = buildProgram();
await program.parseAsync(
[
'macp',
'gate',
'exit 9',
'--cwd',
tmpDir,
'--log',
path.join(tmpDir, 'g.log'),
'--timeout',
'10',
],
{ from: 'user' },
);
expect(process.exitCode).not.toBe(0);
});
it('macp gate with an unimplemented ci-pipeline capability exits nonzero', async () => {
const program = buildProgram();
const specPath = path.join(tmpDir, 'gates.json');
fs.writeFileSync(specPath, JSON.stringify([{ type: 'ci-pipeline' }]));
await program.parseAsync(
[
'macp',
'gate',
specPath,
'--cwd',
tmpDir,
'--log',
path.join(tmpDir, 'g.log'),
'--timeout',
'10',
],
{ from: 'user' },
);
expect(process.exitCode).not.toBe(0);
});
it('macp gate --simulate completes (exit 0) but reports simulated results', async () => {
const program = buildProgram();
const logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
try {
await program.parseAsync(
[
'macp',
'gate',
'exit 0',
'--simulate',
'--cwd',
tmpDir,
'--log',
path.join(tmpDir, 'g.log'),
'--timeout',
'10',
],
{ from: 'user' },
);
// completes only because the caller explicitly asked to simulate
expect(process.exitCode).toBe(0);
const outText = logSpy.mock.calls.map((c) => String(c[0])).join('\n');
expect(outText).toContain('simulated');
expect(outText).toContain('SIMULATED');
} finally {
logSpy.mockRestore();
}
});
it('macp gate with an empty spec exits nonzero with a typed error', async () => {
const program = buildProgram();
await program.parseAsync(
[
'macp',
'gate',
' ',
'--cwd',
tmpDir,
'--log',
path.join(tmpDir, 'g.log'),
'--timeout',
'10',
],
{ from: 'user' },
);
expect(process.exitCode).not.toBe(0);
});
});
+129 -19
View File
@@ -1,5 +1,73 @@
import { existsSync, readFileSync } from 'node:fs';
import type { Command } from 'commander';
import { runGates } from './gate-runner.js';
import { MACPCapabilityError, type MacpErrorCode } from './errors.js';
/**
* Load gates from a spec: an existing file (JSON gates array, a JSON object
* with `quality_gates`, a JSON gate object, or one command per line) or an
* inline command string. Fails closed with a typed capability error when the
* spec contains no executable gate definition.
*/
function loadGateSpec(spec: string): unknown[] {
if (existsSync(spec)) {
const raw = readFileSync(spec, 'utf-8');
try {
const parsed = JSON.parse(raw) as unknown;
if (Array.isArray(parsed)) {
if (parsed.length === 0) {
throw new MACPCapabilityError(
'MACP_NO_COMMAND',
'gate-spec',
`gate spec file '${spec}' contains an empty gates array`,
);
}
return parsed;
}
if (typeof parsed === 'object' && parsed !== null) {
const obj = parsed as Record<string, unknown>;
if (Array.isArray(obj['quality_gates'])) {
return obj['quality_gates'];
}
return [parsed];
}
throw new MACPCapabilityError(
'MACP_NO_COMMAND',
'gate-spec',
`gate spec file '${spec}' parsed to ${typeof parsed} — expected a gates array, a task with quality_gates, or a gate object`,
);
} catch (exc) {
if (exc instanceof MACPCapabilityError) throw exc;
// Not JSON — treat each non-empty line as a command gate.
const lines = raw
.split('\n')
.map((l) => l.trim())
.filter((l) => l.length > 0);
if (lines.length > 0) return lines;
throw new MACPCapabilityError(
'MACP_NO_COMMAND',
'gate-spec',
`gate spec file '${spec}' contains no gates`,
);
}
}
if (spec.trim().length > 0) return [spec];
throw new MACPCapabilityError('MACP_NO_COMMAND', 'gate-spec', 'gate spec is empty');
}
/** Print a typed not-implemented failure and exit nonzero (RI-N2 fail-closed). */
function notImplemented(subcommand: string, capability: string, hint: string): void {
const err = new MACPCapabilityError(
'MACP_NOT_IMPLEMENTED',
capability,
`${subcommand} is not implemented in @mosaicstack/macp yet (${capability} capability absent) — ${hint}`,
);
console.error(`[macp] ${subcommand}: ${err.message} [${err.code}]`);
process.exitCode = 1;
}
/**
* Register macp subcommands on an existing Commander program.
* This avoids cross-package Commander version mismatches by using the
@@ -24,15 +92,14 @@ export function registerMacpCommand(parent: Command): void {
'Filter by task type (coding|deploy|research|review|documentation|infrastructure)',
)
.action((opts: { status?: string; type?: string }) => {
// not yet wired — task persistence layer is not present in @mosaicstack/macp
console.log('[macp] tasks list: not yet wired — use macp package programmatically');
// unimplemented capability — a failure, never a success (RI-N2)
if (opts.status) {
console.log(` status filter: ${opts.status}`);
}
if (opts.type) {
console.log(` type filter: ${opts.type}`);
}
process.exitCode = 0;
notImplemented('tasks list', 'task-persistence', 'use the macp package programmatically');
});
// ─── submit ──────────────────────────────────────────────────────────────
@@ -41,12 +108,11 @@ export function registerMacpCommand(parent: Command): void {
.command('submit <path>')
.description('Submit a task from a JSON/YAML spec file')
.action((specPath: string) => {
// not yet wired — task submission requires a running MACP server
console.log('[macp] submit: not yet wired — use macp package programmatically');
// unimplemented capability — a failure, never a success (RI-N2)
console.log(` spec path: ${specPath}`);
console.log(' task id: (unavailable — no MACP server connected)');
console.log(' status: (unavailable — no MACP server connected)');
process.exitCode = 0;
notImplemented('submit', 'macp-server', 'use the macp package programmatically');
});
// ─── gate ────────────────────────────────────────────────────────────────
@@ -58,16 +124,58 @@ export function registerMacpCommand(parent: Command): void {
.option('--cwd <path>', 'Working directory for gate execution', process.cwd())
.option('--log <path>', 'Path to write gate log output', '/tmp/macp-gate.log')
.option('--timeout <seconds>', 'Gate timeout in seconds', '60')
.action((spec: string, opts: { failOn: string; cwd: string; log: string; timeout: string }) => {
// not yet wired — gate execution requires a task context and event sink
console.log('[macp] gate: not yet wired — use macp package programmatically');
console.log(` spec: ${spec}`);
console.log(` fail-on: ${opts.failOn}`);
console.log(` cwd: ${opts.cwd}`);
console.log(` log: ${opts.log}`);
console.log(` timeout: ${opts.timeout}s`);
process.exitCode = 0;
});
.option(
'--simulate',
'Simulate gates instead of executing them; results are typed simulated and never satisfy a check',
)
.action(
(
spec: string,
opts: { failOn: string; cwd: string; log: string; timeout: string; simulate?: boolean },
) => {
let gates: unknown[];
try {
gates = loadGateSpec(spec);
} catch (exc) {
if (exc instanceof MACPCapabilityError) {
console.error(`[macp] gate: ${exc.message} [${exc.code}]`);
} else {
console.error(`[macp] gate: ${String(exc)}`);
}
process.exitCode = 1;
return;
}
const timeoutSec = Number.parseInt(opts.timeout, 10) || 60;
const eventsPath = `${opts.log}.events.ndjson`;
const { state, gateResults } = runGates(
gates,
opts.cwd,
opts.log,
timeoutSec,
eventsPath,
'macp-cli-gate',
{
simulate: opts.simulate,
},
);
for (const r of gateResults) {
const label = r.command || r.type;
const reason = r.reason ? `${r.reason}` : '';
console.log(`[macp] gate ${r.status}: ${label}${reason}`);
}
if (opts.simulate) {
console.log(
'[macp] SIMULATED run — every result is typed simulated and can never satisfy a gate, dependency, or release check',
);
}
// Simulated runs may complete (exit 0) only because the caller
// explicitly passed --simulate; the typed state stays 'simulated'.
process.exitCode = state === 'passed' || state === 'simulated' ? 0 : 1;
},
);
// ─── events ──────────────────────────────────────────────────────────────
@@ -79,14 +187,16 @@ export function registerMacpCommand(parent: Command): void {
.option('--file <path>', 'Path to the MACP events NDJSON file')
.option('--follow', 'Follow the file for new events (like tail -f)')
.action((opts: { file?: string; follow?: boolean }) => {
// not yet wired — event streaming requires a live event source
console.log('[macp] events tail: not yet wired — use macp package programmatically');
// unimplemented capability — a failure, never a success (RI-N2)
if (opts.file) {
console.log(` file: ${opts.file}`);
}
if (opts.follow) {
console.log(' mode: follow');
}
process.exitCode = 0;
notImplemented('events tail', 'event-source', 'use the macp package programmatically');
});
}
// Re-export so CLI consumers can surface typed capability codes.
export type { MacpErrorCode };
+35
View File
@@ -0,0 +1,35 @@
/** Typed error code from the closed MACP_ERROR_CODES set. */
export type MacpErrorCode = (typeof MACP_ERROR_CODES)[number];
/**
* Typed fail-closed capability errors (RI-N2, SDLC-D-035).
*
* MACP must fail closed when a required capability (executor, reviewer,
* command, CI provider, human authority) is absent. These typed codes mirror
* the Forge failure vocabulary (FORGE_NO_*) so both packages speak the same
* language: an unimplemented capability is a failure, never a stub success.
*/
/** Closed set of typed MACP capability error codes. */
export const MACP_ERROR_CODES = [
'MACP_NOT_IMPLEMENTED',
'MACP_NO_COMMAND',
'MACP_NO_REVIEWER',
'MACP_NO_CI_PIPELINE',
'MACP_NO_PROVIDER',
'MACP_AUTHORITY_REQUIRED',
] as const;
/** Raised when a required capability is missing and execution must fail closed. */
export class MACPCapabilityError extends Error {
/** Typed error code from the closed MACP_ERROR_CODES set. */
readonly code: MacpErrorCode;
/** The missing capability, e.g. `ci-provider`, `task-persistence`, `command`. */
readonly capability: string;
constructor(code: MacpErrorCode, capability: string, message: string) {
super(message);
this.name = 'MACPCapabilityError';
this.code = code;
this.capability = capability;
}
}
+429
View File
@@ -0,0 +1,429 @@
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { countAIFindings, normalizeGate, runGate, runGates } from './gate-runner.js';
function makeTmpDir(): string {
return fs.mkdtempSync(path.join(os.tmpdir(), 'macp-gate-'));
}
describe('normalizeGate', () => {
it('normalizes a string to mechanical gate', () => {
expect(normalizeGate('echo test')).toEqual({
command: 'echo test',
type: 'mechanical',
fail_on: 'blocker',
});
});
it('normalizes an object gate with defaults', () => {
expect(normalizeGate({ command: 'lint' })).toEqual({
command: 'lint',
type: 'mechanical',
fail_on: 'blocker',
});
});
it('preserves explicit type and fail_on', () => {
expect(normalizeGate({ command: 'review', type: 'ai-review', fail_on: 'any' })).toEqual({
command: 'review',
type: 'ai-review',
fail_on: 'any',
});
});
it('handles non-string/non-object input', () => {
expect(normalizeGate(42)).toEqual({ command: '', type: 'mechanical', fail_on: 'blocker' });
expect(normalizeGate(null)).toEqual({ command: '', type: 'mechanical', fail_on: 'blocker' });
});
});
describe('countAIFindings', () => {
it('returns zeros for non-object', () => {
expect(countAIFindings(null)).toEqual({ blockers: 0, total: 0 });
expect(countAIFindings('string')).toEqual({ blockers: 0, total: 0 });
expect(countAIFindings([])).toEqual({ blockers: 0, total: 0 });
});
it('counts from stats block', () => {
const output = { stats: { blockers: 2, should_fix: 3, suggestions: 1 } };
expect(countAIFindings(output)).toEqual({ blockers: 2, total: 6 });
});
it('counts from findings array when stats has no blockers', () => {
const output = {
stats: { blockers: 0 },
findings: [{ severity: 'blocker' }, { severity: 'warning' }, { severity: 'blocker' }],
};
expect(countAIFindings(output)).toEqual({ blockers: 2, total: 3 });
});
it('uses stats blockers over findings array when stats has blockers', () => {
const output = {
stats: { blockers: 5 },
findings: [{ severity: 'blocker' }, { severity: 'warning' }],
};
// stats.blockers = 5, total from stats = 5+0+0 = 5, findings not used for total since stats total is non-zero
expect(countAIFindings(output)).toEqual({ blockers: 5, total: 5 });
});
it('counts findings length as total when stats has zero total', () => {
const output = {
findings: [{ severity: 'warning' }, { severity: 'info' }],
};
expect(countAIFindings(output)).toEqual({ blockers: 0, total: 2 });
});
});
describe('runGate', () => {
let tmp: string;
let logPath: string;
beforeEach(() => {
tmp = makeTmpDir();
logPath = path.join(tmp, 'gate.log');
});
afterEach(() => {
fs.rmSync(tmp, { recursive: true, force: true });
});
it('passes mechanical gate on exit 0', () => {
const result = runGate('echo hello', tmp, logPath, 30);
expect(result.passed).toBe(true);
expect(result.exit_code).toBe(0);
expect(result.type).toBe('mechanical');
expect(result.output).toContain('hello');
});
it('fails mechanical gate on non-zero exit', () => {
const result = runGate('exit 1', tmp, logPath, 30);
expect(result.passed).toBe(false);
expect(result.exit_code).toBe(1);
});
it('ci-pipeline fails closed without a CI provider (no placeholder pass)', () => {
const result = runGate({ command: 'anything', type: 'ci-pipeline' }, tmp, logPath, 30);
expect(result.passed).toBe(false);
expect(result.status).toBe('capability_failure');
expect(result.capability_code).toBe('MACP_NO_CI_PIPELINE');
expect(result.type).toBe('ci-pipeline');
expect(result.output).not.toBe('CI pipeline gate placeholder');
});
it('empty command is a typed capability failure, never a pass', () => {
const result = runGate({ command: '' }, tmp, logPath, 30);
expect(result.passed).toBe(false);
expect(result.status).toBe('capability_failure');
expect(result.capability_code).toBe('MACP_NO_COMMAND');
});
it('ai-review gate parses JSON output', () => {
const json = JSON.stringify({ stats: { blockers: 0, should_fix: 1 } });
const result = runGate({ command: `echo '${json}'`, type: 'ai-review' }, tmp, logPath, 30);
expect(result.passed).toBe(true);
expect(result.blockers).toBe(0);
expect(result.findings).toBe(1);
});
it('ai-review gate fails on blockers', () => {
const json = JSON.stringify({ stats: { blockers: 2 } });
const result = runGate({ command: `echo '${json}'`, type: 'ai-review' }, tmp, logPath, 30);
expect(result.passed).toBe(false);
expect(result.blockers).toBe(2);
});
it('ai-review gate with fail_on=any fails on any findings', () => {
const json = JSON.stringify({ stats: { blockers: 0, should_fix: 1 } });
const result = runGate(
{ command: `echo '${json}'`, type: 'ai-review', fail_on: 'any' },
tmp,
logPath,
30,
);
expect(result.passed).toBe(false);
expect(result.fail_on).toBe('any');
});
it('ai-review gate fails on invalid JSON output', () => {
const result = runGate({ command: 'echo "not json"', type: 'ai-review' }, tmp, logPath, 30);
expect(result.passed).toBe(false);
expect(result.parse_error).toBeDefined();
});
it('writes to log file', () => {
runGate('echo logged', tmp, logPath, 30);
const log = fs.readFileSync(logPath, 'utf-8');
expect(log).toContain('COMMAND: echo logged');
expect(log).toContain('logged');
expect(log).toContain('EXIT:');
});
});
describe('runGates', () => {
let tmp: string;
let logPath: string;
let eventsPath: string;
beforeEach(() => {
tmp = makeTmpDir();
logPath = path.join(tmp, 'gates.log');
eventsPath = path.join(tmp, 'events.ndjson');
});
afterEach(() => {
fs.rmSync(tmp, { recursive: true, force: true });
});
it('runs multiple gates and returns results', () => {
const { allPassed, gateResults } = runGates(
['echo one', 'echo two'],
tmp,
logPath,
30,
eventsPath,
'task-1',
);
expect(allPassed).toBe(true);
expect(gateResults).toHaveLength(2);
});
it('reports failure when any gate fails', () => {
const { allPassed, gateResults } = runGates(
['echo ok', 'exit 1'],
tmp,
logPath,
30,
eventsPath,
'task-2',
);
expect(allPassed).toBe(false);
expect(gateResults[0]!.passed).toBe(true);
expect(gateResults[1]!.passed).toBe(false);
});
it('emits events for each gate', () => {
runGates(['echo test'], tmp, logPath, 30, eventsPath, 'task-3');
const events = fs
.readFileSync(eventsPath, 'utf-8')
.trim()
.split('\n')
.map((l) => JSON.parse(l));
expect(events).toHaveLength(2); // started + passed
expect(events[0].event_type).toBe('rail.check.started');
expect(events[1].event_type).toBe('rail.check.passed');
});
it('does not silently skip gates with empty command — they become capability failures', () => {
const { gateResults, allPassed, state } = runGates(
[{ command: '', type: 'mechanical' }, 'echo real'],
tmp,
logPath,
30,
eventsPath,
'task-4',
);
expect(gateResults).toHaveLength(2);
expect(gateResults[0]!.status).toBe('capability_failure');
expect(gateResults[1]!.status).toBe('passed');
expect(allPassed).toBe(false);
expect(state).toBe('capability_failure');
});
it('does not skip ci-pipeline even with empty command — typed capability failure', () => {
const { gateResults, allPassed, state } = runGates(
[{ command: '', type: 'ci-pipeline' }],
tmp,
logPath,
30,
eventsPath,
'task-5',
);
expect(gateResults).toHaveLength(1);
expect(gateResults[0]!.passed).toBe(false);
expect(gateResults[0]!.status).toBe('capability_failure');
expect(allPassed).toBe(false);
expect(state).toBe('capability_failure');
});
it('emits failed event with correct message', () => {
runGates(['exit 42'], tmp, logPath, 30, eventsPath, 'task-6');
const events = fs
.readFileSync(eventsPath, 'utf-8')
.trim()
.split('\n')
.map((l) => JSON.parse(l));
const failEvent = events.find(
(e: Record<string, unknown>) => e.event_type === 'rail.check.failed',
);
expect(failEvent).toBeDefined();
expect(failEvent.message).toContain('Gate failed (');
});
});
/**
* RI-N2 / SDLC-D-035 fail-closed controls for the MACP gate runner.
*
* Invariant under test: `passed: true` occurs ONLY when a gate really executed
* and really exited green (`status === 'passed'`). Absent capabilities,
* manual sign-offs, and simulated runs are typed distinctly and can never
* make the aggregate `passed`.
*/
describe('gate-runner fail-closed (RI-N2)', () => {
let tmpDir: string;
let logPath: string;
let eventsPath: string;
beforeEach(() => {
tmpDir = makeTmpDir();
logPath = path.join(tmpDir, 'gate.log');
eventsPath = path.join(tmpDir, 'events.ndjson');
});
afterEach(() => {
fs.rmSync(tmpDir, { recursive: true, force: true });
});
function run(gates: unknown[], options?: { simulate?: boolean }) {
return runGates(gates, tmpDir, logPath, 10, eventsPath, 'spec-task', options);
}
// ─── positive controls ───────────────────────────────────────────────────
it('a really-executed green command gate still passes', () => {
const result = run([{ command: 'exit 0', type: 'mechanical' }]);
expect(result.gateResults[0]!.status).toBe('passed');
expect(result.gateResults[0]!.passed).toBe(true);
expect(result.allPassed).toBe(true);
expect(result.state).toBe('passed');
});
it('explicit simulate completes and types every result simulated', () => {
const result = run([{ command: 'exit 0', type: 'mechanical' }, 'echo hello'], {
simulate: true,
});
expect(result.gateResults).toHaveLength(2);
for (const gate of result.gateResults) {
expect(gate.status).toBe('simulated');
expect(gate.passed).toBe(false);
}
expect(result.state).toBe('simulated');
});
it('a really-executed red command gate fails with typed status failed', () => {
const result = run([{ command: 'exit 3', type: 'mechanical' }]);
expect(result.gateResults[0]!.status).toBe('failed');
expect(result.gateResults[0]!.passed).toBe(false);
expect(result.allPassed).toBe(false);
expect(result.state).toBe('failed');
});
// ─── negative controls — each asserts typed status AND aggregate not passed ──
it('an empty-command gate is a capability_failure, not skipped and not passed', () => {
const result = run([{ command: '', type: 'mechanical' }]);
// runGates must not silently skip it — it produces a typed result
expect(result.gateResults).toHaveLength(1);
const gate = result.gateResults[0]!;
expect(gate.status).toBe('capability_failure');
expect(gate.capability_code).toBe('MACP_NO_COMMAND');
expect(gate.passed).toBe(false);
// aggregate is not passed
expect(result.allPassed).toBe(false);
expect(result.state).toBe('capability_failure');
expect(result.state).not.toBe('passed');
});
it('a commandless ai-review gate is a typed MACP_NO_REVIEWER capability_failure', () => {
const result = run([{ command: '', type: 'ai-review' }]);
expect(result.gateResults[0]!.status).toBe('capability_failure');
expect(result.gateResults[0]!.capability_code).toBe('MACP_NO_REVIEWER');
expect(result.allPassed).toBe(false);
expect(result.state).not.toBe('passed');
});
it('a ci-pipeline gate without a provider implementation is a capability_failure, never a placeholder pass', () => {
const result = run([{ command: '', type: 'ci-pipeline' }]);
const gate = result.gateResults[0]!;
expect(gate.status).toBe('capability_failure');
expect(gate.capability_code).toBe('MACP_NO_CI_PIPELINE');
expect(gate.passed).toBe(false);
// the old false-success placeholder must be gone
expect(gate.output).not.toBe('CI pipeline gate placeholder');
expect(result.allPassed).toBe(false);
expect(result.state).not.toBe('passed');
});
it('a ci-pipeline gate fails closed even alongside an otherwise green run', () => {
const result = run(['exit 0', { type: 'ci-pipeline', command: 'fake-ci' }]);
expect(result.gateResults[1]!.status).toBe('capability_failure');
expect(result.gateResults[0]!.status).toBe('passed');
expect(result.allPassed).toBe(false);
expect(result.state).toBe('capability_failure');
});
it('a manual gate with no automation enters typed waiting — neither pass nor fail', () => {
const result = run([{ type: 'manual' }]);
const gate = result.gateResults[0]!;
expect(gate.status).toBe('waiting');
expect(gate.passed).toBe(false);
expect(gate.exit_code).toBe(0);
// aggregate is not passed while any gate is waiting
expect(result.allPassed).toBe(false);
expect(result.state).toBe('waiting');
expect(result.state).not.toBe('passed');
});
it('a simulated result can never make the aggregate passed', () => {
const result = run(['exit 0', 'exit 0'], { simulate: true });
expect(result.gateResults.every((g) => g.status === 'simulated')).toBe(true);
expect(result.allPassed).toBe(false);
expect(result.state).toBe('simulated');
expect(result.state).not.toBe('passed');
});
it('waiting dominates an otherwise green aggregate', () => {
const result = run(['exit 0', { type: 'manual' }]);
expect(result.allPassed).toBe(false);
expect(result.state).toBe('waiting');
});
});
describe('runGate fail-closed (RI-N2)', () => {
let tmpDir: string;
let logPath: string;
beforeEach(() => {
tmpDir = makeTmpDir();
logPath = path.join(tmpDir, 'gate.log');
});
afterEach(() => {
fs.rmSync(tmpDir, { recursive: true, force: true });
});
it('simulate: true returns a typed simulated result without executing', () => {
const result = runGate('this-command-does-not-exist-xyz', tmpDir, logPath, 10, {
simulate: true,
});
expect(result.status).toBe('simulated');
expect(result.passed).toBe(false);
expect(result.exit_code).toBe(0);
});
it('normal mode executes for real and types a green gate passed', () => {
const result = runGate('echo ok', tmpDir, logPath, 10);
expect(result.status).toBe('passed');
expect(result.passed).toBe(true);
expect(result.output).toContain('ok');
});
it('a bare string gate normalizes to mechanical and executes', () => {
const result = runGate('exit 7', tmpDir, logPath, 10);
expect(result.type).toBe('mechanical');
expect(result.status).toBe('failed');
expect(result.passed).toBe(false);
});
});
+148 -25
View File
@@ -4,7 +4,20 @@ import { dirname } from 'node:path';
import { emitEvent } from './event-emitter.js';
import { nowISO } from './event-emitter.js';
import type { GateResult } from './types.js';
import type { GateResult, GateStatus, RunGatesResult } from './types.js';
/** Typed reason stamped on every simulated gate result. */
export const SIMULATED_GATE_REASON =
'simulated execution (explicit simulate opt-in): gate was not evaluated by a real implementation';
/** Options for gate execution (RI-N2 fail-closed / explicit simulation). */
export interface RunGateOptions {
/**
* Explicit caller opt-in to simulation. Simulated gates are NOT executed;
* every result is typed `simulated` and never satisfies anything.
*/
simulate?: boolean;
}
export interface NormalizedGate {
command: string;
@@ -103,36 +116,91 @@ export function countAIFindings(parsedOutput: unknown): { blockers: number; tota
return { blockers, total };
}
function simulatedResult(gateEntry: NormalizedGate): GateResult {
return {
command: gateEntry.command,
exit_code: 0,
type: gateEntry.type,
output: SIMULATED_GATE_REASON,
timed_out: false,
passed: false,
status: 'simulated',
reason: SIMULATED_GATE_REASON,
};
}
function capabilityFailureResult(
gateEntry: NormalizedGate,
code: GateResult['capability_code'],
reason: string,
): GateResult {
return {
command: gateEntry.command,
exit_code: 1,
type: gateEntry.type,
output: '',
timed_out: false,
passed: false,
status: 'capability_failure',
capability_code: code,
reason,
};
}
function waitingResult(gateEntry: NormalizedGate, reason: string): GateResult {
return {
command: gateEntry.command,
exit_code: 0,
type: gateEntry.type,
output: '',
timed_out: false,
passed: false,
status: 'waiting',
capability_code: 'MACP_AUTHORITY_REQUIRED',
reason,
};
}
export function runGate(
gate: unknown,
cwd: string,
logPath: string,
timeoutSec: number,
options: RunGateOptions = {},
): GateResult {
const gateEntry = normalizeGate(gate);
const gateType = gateEntry.type;
const command = gateEntry.command;
// Explicit simulation only: never executes, typed simulated, never satisfying.
if (options.simulate) {
return simulatedResult(gateEntry);
}
// Fail closed: no CI provider implementation exists in @mosaicstack/macp,
// so a ci-pipeline gate is an absent capability — never a placeholder pass.
if (gateType === 'ci-pipeline') {
return {
command,
exit_code: 0,
type: gateType,
output: 'CI pipeline gate placeholder',
timed_out: false,
passed: true,
};
return capabilityFailureResult(
gateEntry,
'MACP_NO_CI_PIPELINE',
`ci-pipeline gate '${gateEntry.command || gateType}' has no CI provider implementation wired — refusing placeholder pass`,
);
}
if (!command) {
return {
command: '',
exit_code: 0,
type: gateType,
output: '',
timed_out: false,
passed: true,
};
// A manual gate with no automation waits for human sign-off: not pass, not fail.
if (gateType === 'manual') {
return waitingResult(
gateEntry,
`manual gate has no automation — waiting for human sign-off (type: ${gateType})`,
);
}
// Any other commandless gate is an absent capability — never a vacuous pass.
return capabilityFailureResult(
gateEntry,
gateType === 'ai-review' ? 'MACP_NO_REVIEWER' : 'MACP_NO_COMMAND',
`gate of type '${gateType}' has no command to execute — refusing empty-command pass`,
);
}
const { exitCode, output, timedOut } = runShell(command, cwd, logPath, timeoutSec);
@@ -143,10 +211,12 @@ export function runGate(
output,
timed_out: timedOut,
passed: false,
status: 'failed',
};
if (gateType !== 'ai-review') {
result.passed = exitCode === 0;
result.status = result.passed ? 'passed' : 'failed';
return result;
}
@@ -170,6 +240,7 @@ export function runGate(
} else {
result.passed = exitCode === 0 && blockers === 0 && !timedOut && parseError === undefined;
}
result.status = result.passed ? 'passed' : 'failed';
result.fail_on = failOn;
result.blockers = blockers;
@@ -191,16 +262,19 @@ export function runGates(
timeoutSec: number,
eventsPath: string,
taskId: string,
): { allPassed: boolean; gateResults: GateResult[] } {
let allPassed = true;
options: RunGateOptions = {},
): RunGatesResult {
const gateResults: GateResult[] = [];
let hasCapabilityFailure = false;
let hasSimulated = false;
let hasFailed = false;
let hasWaiting = false;
for (const gate of gates) {
const gateEntry = normalizeGate(gate);
const gateCmd = gateEntry.command;
if (!gateCmd && gateEntry.type !== 'ci-pipeline') continue;
const label = gateCmd || gateEntry.type;
// NOTE: no silent skip — every gate produces a typed result (RI-N2).
emitEvent(
eventsPath,
'rail.check.started',
@@ -209,10 +283,10 @@ export function runGates(
'quality-gate',
`Running gate: ${label}`,
);
const result = runGate(gate, cwd, logPath, timeoutSec);
const result = runGate(gate, cwd, logPath, timeoutSec, options);
gateResults.push(result);
if (result.passed) {
if (result.status === 'passed') {
emitEvent(
eventsPath,
'rail.check.passed',
@@ -224,7 +298,46 @@ export function runGates(
continue;
}
allPassed = false;
if (result.status === 'waiting') {
hasWaiting = true;
emitEvent(
eventsPath,
'rail.check.waiting',
taskId,
'gated',
'quality-gate',
`Gate waiting: ${label}${result.reason ?? 'manual gate awaits sign-off'}`,
);
continue;
}
if (result.status === 'simulated') {
hasSimulated = true;
emitEvent(
eventsPath,
'rail.check.simulated',
taskId,
'gated',
'quality-gate',
`Gate simulated (non-satisfying): ${label}`,
);
continue;
}
if (result.status === 'capability_failure') {
hasCapabilityFailure = true;
emitEvent(
eventsPath,
'rail.check.failed',
taskId,
'gated',
'quality-gate',
`Gate capability failure (${result.capability_code ?? 'MACP_NO_PROVIDER'}): ${label}${result.reason ?? 'required capability is absent'}`,
);
continue;
}
hasFailed = true;
let message: string;
if (result.timed_out) {
message = `Gate timed out after ${timeoutSec}s: ${label}`;
@@ -236,5 +349,15 @@ export function runGates(
emitEvent(eventsPath, 'rail.check.failed', taskId, 'gated', 'quality-gate', message);
}
return { allPassed, gateResults };
const state: GateStatus = hasCapabilityFailure
? 'capability_failure'
: hasSimulated
? 'simulated'
: hasFailed
? 'failed'
: hasWaiting
? 'waiting'
: 'passed';
return { allPassed: state === 'passed', gateResults, state };
}
+16 -2
View File
@@ -6,11 +6,13 @@ export type {
DependsOnPolicy,
GateType,
GateFailOn,
GateStatus,
GateEntry,
Task,
EventType,
MACPEvent,
GateResult,
RunGatesResult,
TaskResult,
ProviderMeta,
ProviderRegistry,
@@ -18,6 +20,11 @@ export type {
export { CredentialError } from './types.js';
// Typed fail-closed capability errors (RI-N2, SDLC-D-035)
export { MACP_ERROR_CODES, MACPCapabilityError } from './errors.js';
export type { MacpErrorCode } from './errors.js';
// Credential resolver
export {
DEFAULT_CREDENTIALS_DIR,
@@ -35,9 +42,16 @@ export {
export type { ResolveCredentialsOptions } from './credential-resolver.js';
// Gate runner
export { normalizeGate, runShell, countAIFindings, runGate, runGates } from './gate-runner.js';
export {
normalizeGate,
runShell,
countAIFindings,
runGate,
runGates,
SIMULATED_GATE_REASON,
} from './gate-runner.js';
export type { NormalizedGate } from './gate-runner.js';
export type { NormalizedGate, RunGateOptions } from './gate-runner.js';
// Risk-floor (agent reflection loop — diff review classifier)
export { evaluateRiskFloor, DEFAULT_RISK_THRESHOLD } from './risk-floor.js';
+39 -2
View File
@@ -1,3 +1,5 @@
import type { MacpErrorCode } from './errors.js';
/** Task status values. */
export type TaskStatus = 'pending' | 'running' | 'gated' | 'completed' | 'failed' | 'escalated';
@@ -17,7 +19,17 @@ export type DispatchMode = 'yolo' | 'acp' | 'exec';
export type DependsOnPolicy = 'all' | 'any' | 'all_terminal';
/** Quality gate type. */
export type GateType = 'mechanical' | 'ai-review' | 'ci-pipeline';
export type GateType = 'mechanical' | 'ai-review' | 'ci-pipeline' | 'manual';
/**
* Typed execution state of a gate — closed set (RI-N2, SDLC-D-035).
*
* Only `passed` means "really executed and green". `simulated` is produced
* exclusively under an explicit simulate opt-in and never satisfies anything.
* `capability_failure` means a required executor/provider/command was absent.
* `waiting` means a manual gate awaits human sign-off (neither pass nor fail).
*/
export type GateStatus = 'passed' | 'failed' | 'simulated' | 'waiting' | 'capability_failure';
/** Gate fail_on mode. */
export type GateFailOn = 'blocker' | 'any';
@@ -67,7 +79,9 @@ export type EventType =
| 'task.retry.scheduled'
| 'rail.check.started'
| 'rail.check.passed'
| 'rail.check.failed';
| 'rail.check.failed'
| 'rail.check.waiting'
| 'rail.check.simulated';
/** Structured event record. */
export interface MACPEvent {
@@ -88,7 +102,14 @@ export interface GateResult {
type: string;
output: string;
timed_out: boolean;
/** Back-compat boolean view — true ONLY when `status === 'passed'`. */
passed: boolean;
/** Typed discriminator — the authoritative gate outcome (RI-N2). */
status: GateStatus;
/** Typed capability error code, set when `status === 'capability_failure'`. */
capability_code?: MacpErrorCode;
/** Why a non-executed state (simulated/waiting/capability_failure) was reached. */
reason?: string;
fail_on?: string;
blockers?: number;
findings?: number;
@@ -96,6 +117,22 @@ export interface GateResult {
parse_error?: string;
}
/**
* Aggregate outcome of `runGates` (RI-N2).
*
* `state` is the typed aggregate: it is `passed` only when every gate really
* executed green. A `simulated` result makes the aggregate `simulated` (never
* `passed`); a `waiting` manual gate keeps the aggregate `waiting`; a missing
* capability makes it `capability_failure`. `allPassed` is exactly
* `state === 'passed'`, so a simulated or waiting result can never satisfy a
* dependency, acceptance criterion, gate, merge, or release check.
*/
export interface RunGatesResult {
allPassed: boolean;
gateResults: GateResult[];
state: GateStatus;
}
/** Result from a completed task. */
export interface TaskResult {
task_id: string;
@@ -39,3 +39,20 @@ packages/mosaic/framework/tools/tmux/test-send-message-verdict.sh | requires rea
# recorded judgement. These lines ARE that judgement, signed.)
packages/mosaic/framework/tools/orchestrator/smoke-test.sh | behavior smoke checks for coord continue/run workflows, run manually by orchestrator seats; unmeasured in CI; #1017 burndown
packages/mosaic/framework/tools/wake/validate-973/microtest-wake-assert.sh | #973 instrument self-test, run as a precondition of the validate-973 evidence procedure rather than as a standing CI suite; #1017 burndown candidate
# --- tools/fleet: precondition is unsatisfiable in the CI image (#1271) ---
# Signed by fred (sb-it-1-dt, 2026-08-16) at origin/next 476db12.
# This suite asserts the launcher's behaviour when `mosaic` and `pi` are MISSING.
# It shims fakes into $FAKE_BIN, but the constructed PANE_PATH always ends in the
# real system path, so on a host that installs those binaries the missing-binary
# cases cannot be measured at all. The suite's own guard (line 103) says so and
# fails rather than reporting a pass it cannot back. That guard is correct.
# The error was wiring the suite into CI: #1017 (c56483eb) enumerated it and
# dropped this exclusion, and the CI image provides `pi` in the system path, so
# it has failed on every pipeline since. Measured 2026-08-16 across pipelines
# 2444 (#1256), 2438 (#1240) and 2441 (#1017-quality): exactly one FAIL line in
# each full log, identical, this assertion; control `zzz-not-present-zzz` -> 0.
# Burn-down and the full measurement are tracked in #1271; unwired by PR #1270.
# Because test:framework-shell is one && chain and this sat at position 44 of 48,
# the four suites after it had not run at all since the merge.
packages/mosaic/framework/tools/fleet/test-start-agent-session.sh | precondition unsatisfiable in the CI image: asserts missing-binary behaviour, but PANE_PATH always ends in the system path and the image provides `pi` there; guard at line 103 fails by design rather than passing unmeasured. Burn down by controlling the tail of PANE_PATH inside the test. NOT by removing `pi` from the image: the CI image installs @earendil-works/[email protected] deliberately (measured in pipeline 2444's test-step log), and other suites depend on that pin. Burn-down tracked in #1271
+1 -1
View File
@@ -25,7 +25,7 @@
"lint": "eslint src",
"typecheck": "tsc --noEmit",
"test": "vitest run --passWithNoTests && pnpm run test:framework-shell",
"test:framework-shell": "bash framework/tools/quality/scripts/check-test-enumeration.sh && bash framework/tools/quality/scripts/test-check-test-enumeration.sh && python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/promotion_binding_unittest.py && python3 src/lease-broker/promotion_trigger_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/receipt_observer_client_unittest.py && python3 src/lease-broker/invariant_r_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-ci-queue-wait-tristate.sh && bash framework/tools/git/test-ci-queue-wait-github-checks.sh && bash framework/tools/git/test-pr-merge-queue-branch.sh && bash framework/tools/git/test-pr-merge-head-pin.sh && bash framework/tools/git/test-pr-merge-message-field.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/woodpecker/test-terminal-green-contract.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/_scripts/test-mosaic-init-rce.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-preimage.sh && bash framework/tools/wake/test-wake-install.sh && bash framework/tools/fleet/test-start-agent-session.sh && bash framework/tools/glpi/test-list-http-status.sh && bash framework/tools/orchestrator/test-board-roll.sh && bash framework/tools/woodpecker/test-ci-wait-exit-matrix.sh && bash framework/tools/_scripts/test-fleet-transport-check.sh"
"test:framework-shell": "bash framework/tools/quality/scripts/check-test-enumeration.sh && bash framework/tools/quality/scripts/test-check-test-enumeration.sh && python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/promotion_binding_unittest.py && python3 src/lease-broker/promotion_trigger_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/receipt_observer_client_unittest.py && python3 src/lease-broker/invariant_r_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-ci-queue-wait-tristate.sh && bash framework/tools/git/test-ci-queue-wait-github-checks.sh && bash framework/tools/git/test-pr-merge-queue-branch.sh && bash framework/tools/git/test-pr-merge-head-pin.sh && bash framework/tools/git/test-pr-merge-message-field.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/woodpecker/test-terminal-green-contract.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/_scripts/test-mosaic-init-rce.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-preimage.sh && bash framework/tools/wake/test-wake-install.sh && bash framework/tools/glpi/test-list-http-status.sh && bash framework/tools/orchestrator/test-board-roll.sh && bash framework/tools/woodpecker/test-ci-wait-exit-matrix.sh && bash framework/tools/_scripts/test-fleet-transport-check.sh"
},
"dependencies": {
"@mosaicstack/brain": "workspace:*",