fix(harness): preserve failed Claude result diagnostics (#797)

This commit is contained in:
jason.woltje committed 2026-11-05 21:17:18 -06:00
1 parent 1bd0077391
commit 8094901bce
12 files changed
+144 -23

No files matched your search

+9
View File
@@ -6202,3 +6202,12 @@ Before: isolated branch fix/mvp3-coder-stall-797 starts at 05fdfef9ea1fdddf58c72
After: Claude's private stream-json reader marks real upstream assistant events and emits only the successful final text, matching real CLI newline semantics byte for byte. Pi's existing extension also marks model text/thinking/tool deltas; mock remains a test-only immediate echo. Both reuse the existing exclusive 0600 proxy-first-output marker with fixed output\n content; init/status, synthetic CLI errors, tool results, stderr and workspace writes do not count. Runner supplies only its existing run directory; helper gets a read-only harness mount and bundle source digest. No timeout, schema, role authority, credential, proxy management or live state changed. Silent runs still stop with proxy-stall; interrupted first/resumed turns emit no partial answer or successful new session ID. After: Claude's private stream-json reader marks real upstream assistant events and emits only the successful final text, matching real CLI newline semantics byte for byte. Pi's existing extension also marks model text/thinking/tool deltas; mock remains a test-only immediate echo. Both reuse the existing exclusive 0600 proxy-first-output marker with fixed output\n content; init/status, synthetic CLI errors, tool results, stderr and workspace writes do not count. Runner supplies only its existing run directory; helper gets a read-only harness mount and bundle source digest. No timeout, schema, role authority, credential, proxy management or live state changed. Silent runs still stop with proxy-stall; interrupted first/resumed turns emit no partial answer or successful new session ID.
Verification: Node v24.22.1 with PATH pinned to /mnt/storage/scratch/tmp/z62zsJ/node24/node_modules/node-linux-x64/bin and fresh TMPDIR from mktemp -d /mnt/storage/scratch/tmp/XXXXXX. Final full 28-file command: 376 passed, 0 failed/cancelled/skipped/todo, test-concurrency=4 (coder-stall-final-r2.tap). Includes full launcher/model-stall, harness runner/bundle, seat session/sandbox, real Claude/Pi adapter, confinement, installer/copy, source-digest and import-closure checks. Delayed tool/thinking/header/assistant fixtures use 0.8-second injected watchdog and 1.4-second pre-answer work. Initial tests retained: missing test import and failed-result exit handling, real text-mode trailing-newline difference, then a combined-gate grandchild-reaping assertion race; all corrected and full final gate green. Whitespace and baseline log-prefix preservation checks pass. Report, exact commands, SHA256 manifest, scans and all TAP receipts: /mnt/storage/scratch/mosaic-lead/mvp3/coder-stall-fix-r1-report.md and coder-stall-* siblings. Local commit follows these green suites. Independent exact-commit review, CI/integration and deployment/live GLM acceptance remain with the lead; no push, stash, fleet operation or external message. Verification: Node v24.22.1 with PATH pinned to /mnt/storage/scratch/tmp/z62zsJ/node24/node_modules/node-linux-x64/bin and fresh TMPDIR from mktemp -d /mnt/storage/scratch/tmp/XXXXXX. Final full 28-file command: 376 passed, 0 failed/cancelled/skipped/todo, test-concurrency=4 (coder-stall-final-r2.tap). Includes full launcher/model-stall, harness runner/bundle, seat session/sandbox, real Claude/Pi adapter, confinement, installer/copy, source-digest and import-closure checks. Delayed tool/thinking/header/assistant fixtures use 0.8-second injected watchdog and 1.4-second pre-answer work. Initial tests retained: missing test import and failed-result exit handling, real text-mode trailing-newline difference, then a combined-gate grandchild-reaping assertion race; all corrected and full final gate green. Whitespace and baseline log-prefix preservation checks pass. Report, exact commands, SHA256 manifest, scans and all TAP receipts: /mnt/storage/scratch/mosaic-lead/mvp3/coder-stall-fix-r1-report.md and coder-stall-* siblings. Local commit follows these green suites. Independent exact-commit review, CI/integration and deployment/live GLM acceptance remain with the lead; no push, stash, fleet operation or external message.
## 2026-11-05 — MVP 3 coder-stall review F1 correction (#797), local implementation handoff
Before: candidate 1bd0077391f1925db5e66796b98540e48f13bfc2 discarded complete error-typed result strings, losing text-mode stdout such as API Error: 400 review-local-api-error with exit 1. Restoring that output also required preventing the runner's generic stdout fallback from mistaking synthetic errors for model activity. Explicit correction charter authorizes one local follow-up commit on fix/mvp3-coder-stall-797; the linked-worktree queue refusal was diagnosed and left intact.
After: the private stream reader renders every complete final result string independently of success/failure, retains CLI nonzero exit codes, fails error-typed zero-exit turns, and withholds result text on stream-integrity/helper marker failures. The adapter's existing success-only session persistence remains unchanged. The Claude plugin declares adapter-owned activity in its existing bundle environment; the runner reads module metadata, ignores ambient declarations and keeps stdout fallback for Pi/worker/mock paths. No core harness-name conditional, watchdog timing/default, broker, roles, contracts, policy or live state change. Bundle tests pin the helper SHA-256 to 9c2cc28b7e719c1d49913401fffc952759d8633e50ae39ca36bfc50729b9415b.
Verification: before product edits, whole-file negative controls on 1bd007739 passed 68 and failed 7; the real installed Claude CLI HTTP 400 test failed on empty streaming stdout versus 38 text-mode bytes. It now passes with identical stdout bytes and exit 1, no activity marker and no new session ID, using only localhost, isolated HOME/configuration and a dummy key. Added subtype/is_error/nonzero controls, corrupt-after-result/oversized/truncated/helper marker-failure cases and real runner fallback controls. Final frozen gate: 414 passed, 0 failed/skipped/cancelled/todo across 31 whole files, one node --test invocation per file, at most two concurrently. Node v24.22.1 on the requested PATH; set -o pipefail and fresh scratch TMPDIR. The initial broad gate's sole architecture failure (412/413) is retained honestly; moved the declaration into the plugin and reran all 31 files green. Whitespace, source hashes and append-only prefix checks pass. Report: /mnt/storage/scratch/mosaic-lead/mvp3/coder-stall-fix-r2-report.md; per-file TAPs, exact commands, negative controls, source/pin scans and hashes: coder-stall-fix-r2-evidence/. Local commit follows verification; independent exact-commit review remains with the lead. No push, stash, forge write, real model call, contact with 10.1.1.53, deployment or live recovery.
+7 -3
View File
@@ -77,6 +77,7 @@ session's workspace and its own process group. Same contract, plus:
| `MOSAIC_TOOL_SOCKET` | The runner's tool socket, which the typed tools call. | | `MOSAIC_TOOL_SOCKET` | The runner's tool socket, which the typed tools call. |
| `MOSAIC_TURN_MARKER` | `pi`: the extension writes it at `agent_end`; a pi turn that exits 0 without it failed. | | `MOSAIC_TURN_MARKER` | `pi`: the extension writes it at `agent_end`; a pi turn that exits 0 without it failed. |
| `MOSAIC_MODEL_ACTIVITY_DIR` | Managed runner's existing writable run directory. Trusted model streaming events publish the fixed `output\n` token to `proxy-first-output` (0600, exclusive create). No event contents or credentials enter it. Claude's private stream reader and Pi's extension signal thinking, tool-use and text activity before final stdout; init/status, tool results and workspace writes do not count. | | `MOSAIC_MODEL_ACTIVITY_DIR` | Managed runner's existing writable run directory. Trusted model streaming events publish the fixed `output\n` token to `proxy-first-output` (0600, exclusive create). No event contents or credentials enter it. Claude's private stream reader and Pi's extension signal thinking, tool-use and text activity before final stdout; init/status, tool results and workspace writes do not count. |
| `MOSAIC_MODEL_ACTIVITY_SOURCE` | The harness module may set `adapter` in its bundle environment to declare model events authoritative and disable the runner's stdout activity fallback. Claude declares this because final stdout can be a synthetic error. Pi and mock retain the fallback. The runner reads the declaration from module metadata, ignoring any ambient environment copy. |
| `MOSAIC_EXTENSIONS` | `pi` only: extension files loaded with `-e`; a missing one exits 2. The pi module names two: the gate and typed tools extension, then the unslop check's `tools/unslop-hook/extension.ts` (row 66). | | `MOSAIC_EXTENSIONS` | `pi` only: extension files loaded with `-e`; a missing one exits 2. The pi module names two: the gate and typed tools extension, then the unslop check's `tools/unslop-hook/extension.ts` (row 66). |
| `MOSAIC_UNSLOP_LOG` | The unslop check's JSONL log, `<run>/unslop.jsonl`, which the runner sets for both adapters and reads after each turn (row 66). | | `MOSAIC_UNSLOP_LOG` | The unslop check's JSONL log, `<run>/unslop.jsonl`, which the runner sets for both adapters and reads after each turn (row 66). |
| `MOSAIC_CLAUDE_SETTINGS`, `MOSAIC_CLAUDE_MCP_CONFIG` | `claude` only: the bundle's gate hook and MCP server; required. | | `MOSAIC_CLAUDE_SETTINGS`, `MOSAIC_CLAUDE_MCP_CONFIG` | `claude` only: the bundle's gate hook and MCP server; required. |
@@ -87,9 +88,12 @@ The layers each adapter relies on, and what they don't cover, are in
`packages/harness/README.md`. `packages/harness/README.md`.
Claude uses `--output-format stream-json --verbose --include-partial-messages`. Claude uses `--output-format stream-json --verbose --include-partial-messages`.
The adapter's `stream.mjs` consumes those events and prints only the successful The adapter's `stream.mjs` consumes those events and prints a complete final
final result, with the same newline as the CLI's text format. Malformed or result, including error results, with the same newline as the CLI's text format.
incomplete streams fail the turn; a failed or interrupted first turn still CLI failures retain their nonzero exit status. Claude's model events alone mark
activity; its final stdout may be a synthetic error. The runner retains stdout
activity marking for Pi and mock adapters. Malformed or incomplete streams emit
only the fixed diagnostic and fail the turn; a failed or interrupted first turn
does not persist a reusable session ID. The helper is mounted read-only and does not persist a reusable session ID. The helper is mounted read-only and
its digest is recorded in the launch bundle. Pi retains print mode and signals its digest is recorded in the launch bundle. Pi retains print mode and signals
activity through its existing extension. Mock remains a test-only echo. activity through its existing extension. Mock remains a test-only echo.
+9 -10
View File
@@ -1,5 +1,5 @@
// Consume Claude Code's SDK event stream privately. Only the successful result // Consume Claude Code's SDK event stream privately. A complete final result
// reaches stdout, ensuring a trailing newline like --output-format text. // reaches stdout, including errors, with the same newline as text mode.
import { spawn } from 'node:child_process'; import { spawn } from 'node:child_process';
import { firstOutputMarker } from '../../packages/harness/src/first-output.mjs'; import { firstOutputMarker } from '../../packages/harness/src/first-output.mjs';
@@ -37,10 +37,8 @@ function event(line) {
resultSeen = true; resultSeen = true;
// Let a normal CLI failure exit on its own, retaining its exit code. // Let a normal CLI failure exit on its own, retaining its exit code.
resultFailed = item.subtype !== 'success' || !!item.is_error; resultFailed = item.subtype !== 'success' || !!item.is_error;
if (!resultFailed) { if (typeof item.result === 'string') answer = item.result;
if (typeof item.result !== 'string') throw Error(); else if (!resultFailed) throw Error();
answer = item.result;
}
} }
} }
child.stdout.setEncoding('utf8'); child.stdout.setEncoding('utf8');
@@ -63,10 +61,11 @@ child.on('close', (code, signal) => {
if (!failed && pending) { if (!failed && pending) {
try { event(pending); } catch { fail(); } try { event(pending); } catch { fail(); }
} }
if (failed || resultFailed || code !== 0 || signal || answer === undefined) { // Integrity and rendering are separate from the CLI's success classification.
if (!failed && code === 0 && !signal) fail(); // Buffer until close so corrupt/truncated streams never leak a final result.
process.exitCode = code || 2; if (!failed && (!resultSeen || signal)) fail();
} else { if (!failed && answer !== undefined) {
process.stdout.write(answer.endsWith('\n') ? answer : answer + '\n'); process.stdout.write(answer.endsWith('\n') ? answer : answer + '\n');
} }
process.exitCode = code || (failed || resultFailed ? 2 : 0);
}); });
+1
View File
@@ -2066,3 +2066,4 @@ are never rewritten or removed; corrections are new entries.
2026-11-04 | Mos, skill integrator | #818 / Gitea #1845 | Integrated exact independently approved ms-ux-improvement candidate02 into source;3hashes/validator/whitespace pass and author histories preserved. Separate commit from MVP; its activation target staysbd8120318. No install, activation, publication or live action. 2026-11-04 | Mos, skill integrator | #818 / Gitea #1845 | Integrated exact independently approved ms-ux-improvement candidate02 into source;3hashes/validator/whitespace pass and author histories preserved. Separate commit from MVP; its activation target staysbd8120318. No install, activation, publication or live action.
2026-11-05 | Codex implementation child | MVP 3 #797 D2 coder-stall fix on fix/mvp3-coder-stall-797 from 05fdfef9e | Claude private streaming activity marker + answer-only stdout; Pi model-event coverage; silent/interrupted controls preserved. Final Node 24.22.1 full 28-file gate: 376 pass, 0 fail/skip/cancel. Queue refused linked worktree; no workaround. Local commit only, no push/stash/live actions. Report: /mnt/storage/scratch/mosaic-lead/mvp3/coder-stall-fix-r1-report.md; independent review remains with lead. 2026-11-05 | Codex implementation child | MVP 3 #797 D2 coder-stall fix on fix/mvp3-coder-stall-797 from 05fdfef9e | Claude private streaming activity marker + answer-only stdout; Pi model-event coverage; silent/interrupted controls preserved. Final Node 24.22.1 full 28-file gate: 376 pass, 0 fail/skip/cancel. Queue refused linked worktree; no workaround. Local commit only, no push/stash/live actions. Report: /mnt/storage/scratch/mosaic-lead/mvp3/coder-stall-fix-r1-report.md; independent review remains with lead.
2026-11-05 | Implementation child | MVP 3 #797 coder-stall review F1 on fix/mvp3-coder-stall-797 from 1bd007739 | Preserve complete failed Claude result stdout and exit status; module-declared model activity disables synthetic stdout marking while Pi/worker/mock fallback remains. Real localhost HTTP 400 regression fails baseline and passes fix; final Node 24.22.1 gate 414/414 across 31 whole files, no skips. One local commit only; no push/stash/forge/live actions. Queue refused linked worktree without workaround. Report: /mnt/storage/scratch/mosaic-lead/mvp3/coder-stall-fix-r2-report.md; independent review remains with lead.
+6 -1
View File
@@ -345,7 +345,12 @@ function runTurn({ session, request, n, env, onChild }) {
closeSync(errFd); closeSync(errFd);
onChild(child); onChild(child);
let out = ""; let out = "";
const markOutput = firstOutputMarker(session.runDir); // The harness module may declare its adapter's model events authoritative.
// Such adapters can print synthetic error results, which aren't activity.
// Read the declaration from module metadata, never the ambient environment;
// adapters without it retain the stdout fallback (including worker paths).
const markOutput = session.harnessEnv.MOSAIC_MODEL_ACTIVITY_SOURCE === 'adapter'
? () => {} : firstOutputMarker(session.runDir);
child.stdout.setEncoding("utf8"); child.stdout.setEncoding("utf8");
child.stdout.on("data", (s) => { child.stdout.on("data", (s) => {
markOutput(s); markOutput(s);
+1
View File
@@ -152,6 +152,7 @@ test("a claude-code bundle adds the wrapped gate hook and the MCP config", (t) =
assert.ok(m.typed.every((n) => n.startsWith("mcp__mosaic__"))); assert.ok(m.typed.every((n) => n.startsWith("mcp__mosaic__")));
assert.equal(m.sources.claudeStream.path, join(REPO, "adapters/claude/stream.mjs")); assert.equal(m.sources.claudeStream.path, join(REPO, "adapters/claude/stream.mjs"));
assert.equal(m.sources.claudeStream.sha256, sha(m.sources.claudeStream.path)); assert.equal(m.sources.claudeStream.sha256, sha(m.sources.claudeStream.path));
assert.equal(m.sources.claudeStream.sha256, "9c2cc28b7e719c1d49913401fffc952759d8633e50ae39ca36bfc50729b9415b");
assert.deepEqual(Object.keys(m.sources).sort(), ["claudeGate", "claudeStream", "gate", "mcpServer", "tools", "unslopCheck", "unslopHook", "unslopLists"]); assert.deepEqual(Object.keys(m.sources).sort(), ["claudeGate", "claudeStream", "gate", "mcpServer", "tools", "unslopCheck", "unslopHook", "unslopLists"]);
// Row 66 (#1559): the unslop hooks run after a reply and before the next prompt, and never block. // Row 66 (#1559): the unslop hooks run after a reply and before the next prompt, and never block.
for (const [event, which] of [["Stop", "stop"], ["UserPromptSubmit", "prompt"]]) { for (const [event, which] of [["Stop", "stop"], ["UserPromptSubmit", "prompt"]]) {
+47 -3
View File
@@ -6,6 +6,7 @@ import { fixtureCacheEnv } from '../../../scripts/testing/lane-fixtures.mjs';
// except the argv check, which uses a stand-in claude. // except the argv check, which uses a stand-in claude.
import { test } from "node:test"; import { test } from "node:test";
import { createServer as createHttp } from "node:http";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { spawn, spawnSync } from "node:child_process"; import { spawn, spawnSync } from "node:child_process";
import { chmodSync, existsSync, mkdirSync, readFileSync, realpathSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; import { chmodSync, existsSync, mkdirSync, readFileSync, realpathSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
@@ -66,14 +67,15 @@ async function session(t, script, tools = ["read", "bash"], done = (r) => `ANSWE
function turn(env, request, cwd, adapter = ADAPTER) { function turn(env, request, cwd, adapter = ADAPTER) {
return new Promise((resolve) => { return new Promise((resolve) => {
const child = spawn("/bin/sh", [adapter], { cwd, env: { ...env, ...fixtureCacheEnv(env.CLAUDE_CONFIG_DIR), MOSAIC_REQUEST: request }, stdio: ["ignore", "pipe", "pipe"], detached: true }); const child = spawn("/bin/sh", [adapter], { cwd, env: { ...env, ...fixtureCacheEnv(env.CLAUDE_CONFIG_DIR), MOSAIC_REQUEST: request }, stdio: ["ignore", "pipe", "pipe"], detached: true });
let stdout = ""; const chunks = [];
let stderr = ""; let stderr = "";
child.stdout.on("data", (b) => (stdout += b)); child.stdout.on("data", (b) => chunks.push(b));
child.stderr.on("data", (b) => (stderr += b)); child.stderr.on("data", (b) => (stderr += b));
const clock = setTimeout(() => process.kill(-child.pid, "SIGKILL"), 120_000); const clock = setTimeout(() => process.kill(-child.pid, "SIGKILL"), 120_000);
child.on("close", (code) => { child.on("close", (code) => {
clearTimeout(clock); clearTimeout(clock);
resolve({ code, stdout, stderr }); const stdoutBytes = Buffer.concat(chunks);
resolve({ code, stdout: stdoutBytes.toString("utf8"), stdoutBytes, stderr });
}); });
}); });
} }
@@ -158,6 +160,48 @@ test(`claude: streaming preserves the real CLI text answer byte for byte (${JSON
}); });
} }
test("claude: HTTP 400 keeps real text-mode stdout and exit status without model activity or a new session", { skip }, async t => {
const { dir, workspace, env } = await session(t, [], []);
let requests = 0;
const api = createHttp((req, res) => {
req.resume();
req.on('end', () => {
if (!/\/messages(\?|$)/.test(req.url)) {
res.writeHead(200, { 'content-type': 'application/json' });
return res.end(JSON.stringify({ input_tokens: 1 }));
}
requests++;
res.writeHead(400, { 'content-type': 'application/json' });
res.end(JSON.stringify({ type: 'error', error: { type: 'invalid_request_error', message: 'review-local-api-error' } }));
});
});
await new Promise(resolve => api.listen(0, '127.0.0.1', resolve));
t.after(() => api.close());
const textAdapter = join(dir, 'text-adapter.sh');
writeFileSync(textAdapter, readFileSync(ADAPTER, 'utf8')
.replace('node "$STREAM" "$CLAUDE"', '"$CLAUDE"')
.replace('--output-format stream-json --verbose --include-partial-messages', '--output-format text'));
const results = [];
for (const [name, adapter] of [['text', textAdapter], ['stream', ADAPTER]]) {
const runDir = join(dir, name);
mkdirSync(runDir);
const result = await turn({ ...env,
HOME: join(runDir, 'home'), CLAUDE_CONFIG_DIR: join(runDir, 'config'),
ANTHROPIC_BASE_URL: `http://127.0.0.1:${api.address().port}`,
MOSAIC_SESSION_DIR: join(runDir, 'session'), MOSAIC_MODEL_ACTIVITY_DIR: runDir,
}, 'answer', workspace, adapter);
results.push(result);
assert.equal(result.code, 1, `${name}: ${result.stderr}`);
assert.equal(existsSync(join(runDir, 'session/claude-session-id')), false, name);
assert.equal(existsSync(join(runDir, 'proxy-first-output')), false, name);
}
const [text, stream] = results;
assert.ok(requests >= 2, 'both real CLI invocations reached only the local scripted API');
assert.deepEqual(text.stdoutBytes, Buffer.from('API Error: 400 review-local-api-error\n'));
assert.deepEqual(stream.stdoutBytes, text.stdoutBytes);
assert.equal(stream.code, text.code);
});
// --restricted is what keeps CLAUDE.md files and auto-memory out of the // --restricted is what keeps CLAUDE.md files and auto-memory out of the
// session's prompt (README "Limits"). This one runs without claude: a // session's prompt (README "Limits"). This one runs without claude: a
// stand-in records the adapter's argv. // stand-in records the adapter's argv.
+11 -4
View File
@@ -2,8 +2,9 @@
import { writeFileSync } from 'node:fs'; import { writeFileSync } from 'node:fs';
import { join } from 'node:path'; import { join } from 'node:path';
const args = process.argv.slice(2); const args = process.argv.slice(2);
const answer = ' Answer é 🦆\nsecond line\n';
const mode = process.env.FAKE_CLAUDE_MODE ?? 'tool'; const mode = process.env.FAKE_CLAUDE_MODE ?? 'tool';
const answer = mode === 'failed' || mode === 'failed-success-subtype' || mode === 'failed-zero-exit'
? 'API Error: 400 fixture-error' : ' Answer é 🦆\nsecond line\n';
writeFileSync(join(process.env.MOSAIC_WORKSPACE, 'cli-pid'), String(process.pid)); writeFileSync(join(process.env.MOSAIC_WORKSPACE, 'cli-pid'), String(process.pid));
const emit = item => process.stdout.write(JSON.stringify(item) + '\n'); const emit = item => process.stdout.write(JSON.stringify(item) + '\n');
if (args[args.indexOf('--output-format') + 1] === 'text') { if (args[args.indexOf('--output-format') + 1] === 'text') {
@@ -20,9 +21,15 @@ if (args[args.indexOf('--output-format') + 1] === 'text') {
process.stdout.write('invalid credential-canary\n'); process.stdout.write('invalid credential-canary\n');
} else if (mode === 'missing') { } else if (mode === 'missing') {
// Exit 0 without a result still fails the adapter. // Exit 0 without a result still fails the adapter.
} else if (mode === 'failed') { } else if (['failed', 'failed-success-subtype', 'failed-zero-exit', 'nonzero-success'].includes(mode)) {
emit({ type: 'result', subtype: 'error_during_execution', is_error: true, errors: ['credential-canary'] }); emit({ type: 'result', subtype: mode === 'failed-success-subtype' || mode === 'nonzero-success' ? 'success' : 'error_during_execution',
process.exitCode = 7; is_error: mode === 'failed-success-subtype', result: answer, errors: ['credential-canary'] });
process.exitCode = mode === 'failed-zero-exit' ? 0 : 7;
} else if (['malformed-after-result', 'oversized', 'truncated'].includes(mode)) {
// A complete answer before corrupt trailing data must still be withheld.
emit({ type: 'result', subtype: 'success', result: answer });
process.stdout.write(mode === 'oversized' ? 'x'.repeat(16 * 1024 * 1024 + 1)
: mode === 'truncated' ? '{"type":"result"' : 'invalid credential-canary\n');
} else { } else {
if (mode !== 'silent') { if (mode !== 'silent') {
const event = mode === 'thinking' const event = mode === 'thinking'
+27 -2
View File
@@ -118,17 +118,42 @@ test(`interrupted Claude activity emits no partial answer, preserves session sta
}); });
} }
for (const mode of ['malformed', 'missing', 'failed']) { for (const mode of ['malformed', 'missing', 'malformed-after-result', 'oversized', 'truncated']) {
test(`Claude ${mode} stream fails closed without raw event diagnostics or a session receipt`, async t => { test(`Claude ${mode} stream fails closed without raw event diagnostics or a session receipt`, async t => {
const f = fixture(t, mode), result = await start(t, f).done; const f = fixture(t, mode), result = await start(t, f).done;
assert.notEqual(result.code, 0); assert.notEqual(result.code, 0);
if (mode === 'failed') assert.equal(result.code, 7, 'preserve the CLI exit code');
assert.equal(result.stdout.length, 0); assert.equal(result.stdout.length, 0);
assert.match(result.stderr, /claude adapter: invalid or incomplete model stream/);
assert.doesNotMatch(result.stderr, /credential-canary/); assert.doesNotMatch(result.stderr, /credential-canary/);
assert.equal(existsSync(join(f.runDir, 'session/claude-session-id')), false); assert.equal(existsSync(join(f.runDir, 'session/claude-session-id')), false);
}); });
} }
test('Claude helper marker failure keeps the fixed diagnostic and emits no result', async t => {
const f = fixture(t);
// A file where the marker directory should be makes the private helper fail.
f.env.MOSAIC_MODEL_ACTIVITY_DIR = join(f.runDir, 'prompt');
const result = await start(t, f).done;
assert.notEqual(result.code, 0);
assert.equal(result.stdout.length, 0);
assert.match(result.stderr, /claude adapter: invalid or incomplete model stream/);
assert.doesNotMatch(result.stderr, /credential-canary|ENOTDIR/);
assert.equal(existsSync(join(f.runDir, 'session/claude-session-id')), false);
assert.equal(existsSync(join(f.runDir, 'proxy-first-output')), false);
});
for (const mode of ['failed', 'failed-success-subtype', 'failed-zero-exit', 'nonzero-success']) {
test(`Claude ${mode} renders the final result independently of failure without a session or synthetic activity`, async t => {
const f = fixture(t, mode), result = await start(t, f).done;
assert.equal(result.code, mode === 'failed-zero-exit' ? 2 : 7);
const answer = spawnSync(f.cli, ['--output-format', 'text'], { env: f.env });
assert.deepEqual(result.stdout, answer.stdout);
assert.doesNotMatch(result.stderr, /invalid or incomplete model stream/);
assert.equal(existsSync(join(f.runDir, 'session/claude-session-id')), false);
assert.equal(existsSync(join(f.runDir, 'proxy-first-output')), false);
});
}
test('Pi extension marks real streamed thinking/tools/text, never local activity or user/tool messages', async t => { test('Pi extension marks real streamed thinking/tools/text, never local activity or user/tool messages', async t => {
const runDir = scratch(t); const runDir = scratch(t);
writeFileSync(join(runDir, 'policy'), JSON.stringify({ harness: 'pi', tools: [], typed: [] })); writeFileSync(join(runDir, 'policy'), JSON.stringify({ harness: 'pi', tools: [], typed: [] }));
+22
View File
@@ -228,6 +228,28 @@ test("a message becomes a turn, the answer goes back as a RESULT, SIGTERM releas
assert.ok(!existsSync(join(ctx.runDir, "tools.sock"))); assert.ok(!existsSync(join(ctx.runDir, "tools.sock")));
}); });
// Claude's private stream reader owns model activity; final result stdout alone
// (even a synthetic success) must not cancel the model watchdog. Other adapters
// still need the stdout fallback when no streaming producer is installed.
for (const [harness, exit, activity] of [['claude-code', 1, false], ['claude-code', 0, false], ['pi', 0, true], ['mock', 0, true]]) {
test(`stdout activity fallback: ${harness}, adapter exit ${exit}`, async t => {
const ctx = await setup(t, { session: { harness, turnEnd: 'exit',
harnessEnv: activity ? {} : { MOSAIC_MODEL_ACTIVITY_SOURCE: 'adapter' },
} });
ctx.inbox = [];
writeFileSync(join(ctx.dir, 'adapter.sh'), `printf 'API Error: 400 synthetic-result\\n'\nexit ${exit}\n`);
// An ambient declaration cannot suppress the fallback for other adapters.
const r = startRunner(ctx.runDir, ctx.cap, { MOSAIC_MODEL_ACTIVITY_SOURCE: 'adapter' });
t.after(async () => { r.child.kill('SIGTERM'); await r.exited; });
const reply = await ask(ctx);
if (exit) assert.match(reply.body, /The turn failed: adapter exited 1/);
else assert.equal(reply.body, 'API Error: 400 synthetic-result');
const marker = join(ctx.runDir, 'proxy-first-output');
assert.equal(existsSync(marker), activity);
if (activity) assert.equal(readFileSync(marker, 'utf8'), 'output\n');
});
}
// Row 109 (#1603): each turn reads the state when it starts. The fake adapter echoes the request, // Row 109 (#1603): each turn reads the state when it starts. The fake adapter echoes the request,
// so a reply shows the block the turn was given. // so a reply shows the block the turn was given.
const ask = async (ctx, body = "what now") => { const ask = async (ctx, body = "what now") => {
@@ -124,6 +124,8 @@ export function env({ files, policy, session, proxy = null }) {
MOSAIC_CLAUDE_SETTINGS: files.settings, MOSAIC_CLAUDE_SETTINGS: files.settings,
MOSAIC_CLAUDE_MCP_CONFIG: files.mcp, MOSAIC_CLAUDE_MCP_CONFIG: files.mcp,
MOSAIC_THINKING: session.thinking, MOSAIC_THINKING: session.thinking,
// stream.mjs marks upstream model events; final stdout may be an API error.
MOSAIC_MODEL_ACTIVITY_SOURCE: "adapter",
...(proxy ? { ANTHROPIC_BASE_URL: proxy.baseUrl, ANTHROPIC_AUTH_TOKEN: proxy.key } : {}), ...(proxy ? { ANTHROPIC_BASE_URL: proxy.baseUrl, ANTHROPIC_AUTH_TOKEN: proxy.key } : {}),
}; };
} }
+2
View File
@@ -111,6 +111,7 @@ test("row 65 env passes the thinking level to both adapters, and claude-code's b
const session = { provider: "fixture", model: "m", thinking: "high", toolSocket: "/run/tools.sock" }; const session = { provider: "fixture", model: "m", thinking: "high", toolSocket: "/run/tools.sock" };
const piEnv = pi.env({ policy: { tools: ["read"] }, tools: [{ name: "list_tasks" }], session }); const piEnv = pi.env({ policy: { tools: ["read"] }, tools: [{ name: "list_tasks" }], session });
assert.deepEqual([piEnv.MOSAIC_THINKING, piEnv.PI_MODEL, piEnv.MOSAIC_TOOLS], ["high", "m", "read,list_tasks"]); assert.deepEqual([piEnv.MOSAIC_THINKING, piEnv.PI_MODEL, piEnv.MOSAIC_TOOLS], ["high", "m", "read,list_tasks"]);
assert.equal(piEnv.MOSAIC_MODEL_ACTIVITY_SOURCE, undefined, "Pi retains stdout fallback for paths without streaming activity");
// Row 66: pi loads the unslop check as its second extension. // Row 66: pi loads the unslop check as its second extension.
assert.deepEqual(piEnv.MOSAIC_EXTENSIONS.split(","), [join(REPO, "packages", "harness", "src", "pi-extension.mjs"), join(REPO, "tools", "unslop-hook", "extension.ts")]); assert.deepEqual(piEnv.MOSAIC_EXTENSIONS.split(","), [join(REPO, "packages", "harness", "src", "pi-extension.mjs"), join(REPO, "tools", "unslop-hook", "extension.ts")]);
const dir = scratch(t); const dir = scratch(t);
@@ -127,6 +128,7 @@ test("row 65 env passes the thinking level to both adapters, and claude-code's b
assert.match(settings.hooks.UserPromptSubmit[0].hooks[0].command, /^timeout -k 2 10 .*claude-hook\.js' prompt; exit 0$/); assert.match(settings.hooks.UserPromptSubmit[0].hooks[0].command, /^timeout -k 2 10 .*claude-hook\.js' prompt; exit 0$/);
assert.equal(JSON.parse(readFileSync(join(dir, "mcp.json"), "utf8")).mcpServers.mosaic.args.at(-1), "/run/tools.sock"); assert.equal(JSON.parse(readFileSync(join(dir, "mcp.json"), "utf8")).mcpServers.mosaic.args.at(-1), "/run/tools.sock");
const claudeEnv = claude.env({ files: out.files, policy: { tools: ["read"], typed: [] }, session }); const claudeEnv = claude.env({ files: out.files, policy: { tools: ["read"], typed: [] }, session });
assert.equal(claudeEnv.MOSAIC_MODEL_ACTIVITY_SOURCE, "adapter");
assert.deepEqual([claudeEnv.MOSAIC_THINKING, claudeEnv.MOSAIC_CLAUDE_SETTINGS, claudeEnv.MOSAIC_CLAUDE_MCP_CONFIG], ["high", out.files.settings, out.files.mcp]); assert.deepEqual([claudeEnv.MOSAIC_THINKING, claudeEnv.MOSAIC_CLAUDE_SETTINGS, claudeEnv.MOSAIC_CLAUDE_MCP_CONFIG], ["high", out.files.settings, out.files.mcp]);
}); });