diff --git a/BUILD-LOG.md b/BUILD-LOG.md index 9a79ae8ee..1193403f3 100644 --- a/BUILD-LOG.md +++ b/BUILD-LOG.md @@ -6320,3 +6320,6 @@ R5 quote fallback room (#797 D3, 2026-11-06, MVP3 lead): r4 review P2 found that 2026-11-06 UTC | #797 round A fix r3 (Sol r2 P2) | Before (16384bb12): a mosaic-request-state for an id advanced the thread's epoch only when a remembered miss was deleted, so a 404 still in flight from before the observation could record the miss afterwards and timer refreshes kept fabricating 404s for an open request. After: each observed state advances that id's own generation on every thread that retains it; an in-flight 404 records a miss only if both the thread epoch and the id generation are unchanged. Other ids' slow misses still settle. Generations are pruned to retained rows with the misses. New held-404 test fails on 16384bb12 chat.js at recovery; webui chat-sessions-access, chat and browser files 41/0/0 (Node 26 serial). Lead ruling: no third review round. Evidence: /mnt/storage/scratch/mosaic-lead/mvp3/roundA-fix-r3-evidence/. 2026-11-06 UTC | #797 round A fix r4 (Lead push gate red on 9d3eb9e11) | Before (9d3eb9e11): the chat.js fetch wrapper recorded a request-not-found from /api/requests//workspace as a missing request, then served a synthetic 404 to every later read of that id, including request panel rereads; request-browser.test.mjs failed 11/24 (its stub has no workspace route), base 6b84fe136 24/0. After: only the request's own read records a miss; a remembered miss still answers its workspace read in the page. New regression test fails on 9d3eb9e11 chat.js and passes on the fix. The r3 held-404 test raced Files' cached repaint under concurrency 3 (678/1; alone 5/5 pass); it now waits, bounded, for the refresh's reads to reach the server and still fails on 16384bb12. request-browser 24/0; whole webui suite except project-files 679/0/0/0 (Node 26, concurrency 3). Correction: the r1-r3 author gate ran only chat-sessions-access, chat and browser and omitted request-browser.test.mjs, which also exercises chat.js; that omission let this regression reach the push gate. Evidence: /mnt/storage/scratch/mosaic-lead/mvp3/roundA-fix-push-gate/r4-*. 2026-11-06 UTC | #1847/#1848 fix row (Round B halt, key roundb-2-halt) | Before (1186cf7b1): the runner auto-replied a RESULT to every non-RESULT message, so chief INFO/RESULT replies to assistant RESULTs looped (thread 40df75c5, 54 messages, 1,036,184 assistant tokens); continuation notices keyed on business-wide launch/end/claim seqs, so every unrelated lifecycle event, including kill-switch ends during a halt, re-noticed the chief and drove 13 instance-running refusals (1,198,607 tokens). After: the broker adds in_reply_to_class to agent inbox rows and the runner sends no automatic reply to an agent's non-REQUEST reply to a RESULT (REQUEST, ordinary INFO and human/tester/user messages unchanged); continuation versions use only the target's own lifecycle events and claims, no notice while processing is halted or while the target's latest launch has not ended; the instance-running refusal tells the chief not to relaunch or relay. Builder 4048e0f60 (Codex Terra xhigh); Sol xhigh review r1 CHANGES: a halted broker restart failed on the new retry refusal (P1) and an older unmatched launch hid later ends, worsening #1851 (P2). Lead fix c04529c7b: startup request recovery waits while processing is halted or invalid, as the periodic pass does, and resume reschedules it; only the target's latest lifecycle event gates notices. Each regression test fails on its base. Gate on c04529c7b (Node 26.12.1): touched 123/0, bus except project-files 1648/0, harness 318/0, webui 678/1 then 679/0 on rerun (decide-browser I1 timed out under full-suite load; file alone 9/9 twice; no webui diff vs base). project-files.test.mjs hangs on base and candidate alike. Residual: a latest launch whose end is lost still suppresses notices until host recovery (#1851). Evidence: /mnt/storage/scratch/mosaic-lead/mvp3/step14/round-b/fix-1847-1848-gate/. + +2026-11-06 UTC | #1852 length-stop implementation | Before: an empty Pi length stop could become an empty RESULT, and the GLM registration inherited a 16,384 output cap. Candidate: marker metadata identifies the failed turn, bounded continuations share one usage transcript delta, exhaustion writes a message-trail event and a chief notice, and GLM limits are a separate plugin change. The lead accepted the larger limit's Pi compact-and-retry effect and the later usage limit check. The first bus gate exposed a missing sandbox binding for the shared notice constant; the binding was added, and the three affected launch cases passed in their file. A Claude row-66 assertion failed intermittently in candidate runs, while an untouched base full harness run passed; no base reproduction is claimed. The existing library citation retry's :retry usage calls remain a follow-up. Local work only; no push or deployment. Evidence: /mnt/storage/scratch/tmp/mDnSad/. +2026-11-06 UTC | #1852 verification | Node 26 final harness 332/0, bus excluding project-files 1648/0, launcher and sandbox 69/0, and additional grep-selected tests 252/0. An overlapping bus run had library-sync-busy and a 30-second timeout; both affected files then passed 27/0 and the full bus rerun passed. The candidate's Claude row-66 test failed in an earlier full and isolated run, then the candidate and exact-base full harness suites passed 332/0 and 318/0 respectively; the base failure was not reproduced. Evidence: /mnt/storage/scratch/tmp/mDnSad/. diff --git a/docs/SESSIONS.md b/docs/SESSIONS.md index 71b724f67..6109f4a7a 100644 --- a/docs/SESSIONS.md +++ b/docs/SESSIONS.md @@ -2109,3 +2109,4 @@ are never rewritten or removed; corrections are new entries. 2026-11-06 UTC | mvp3-lead-797 (Claude, #797 lane) | roundA-fix r3 on 16384bb12 for Sol r2 P2 | A request state observed before an in-flight 404 settles now fences that miss via a per-id generation. Negative fails on 16384bb12, webui gate 41/0/0. Evidence: /mnt/storage/scratch/mosaic-lead/mvp3/roundA-fix-r3-evidence/. 2026-11-06 UTC | mvp3-lead-797 (Claude, #797 lane) | roundA-fix r4 on 9d3eb9e11 for the Lead's red webui push gate | Workspace 404s no longer record a request miss; the r3 held-404 test waits for its refresh reads. request-browser 24/0, whole webui 679/0/0/0; r1-r3 gate omission of request-browser recorded in BUILD-LOG. Evidence: /mnt/storage/scratch/mosaic-lead/mvp3/roundA-fix-push-gate/r4-*. 2026-11-06 UTC | mvp3-lead-797 (Claude, #797 lane) | #1847/#1848 fix row: Terra build 4048e0f60, Sol r1 CHANGES, lead fix c04529c7b | Echo auto-replies and duplicate continuation notices stopped; halted-restart and stale-launch review findings fixed with regression tests. Gate touched 123/0, bus 1648/0 (project-files excluded, hangs on base), harness 318/0, webui 679/0 on rerun. Not pushed; Lead pushes. Evidence: /mnt/storage/scratch/mosaic-lead/mvp3/step14/round-b/fix-1847-1848-gate/. +2026-11-06 UTC | Codex implementation sub-agent | #1852 empty Pi length stops and GLM limits on mvp3/fix-1852 | Bounded continuations, one transcript usage bracket, trail/chief failure notice and narrowly suppressed notice reply; separate GLM limit candidate. Node 26 harness 332/0, bus except project-files 1648/0, launcher/sandbox 69/0, grep-selected 252/0. Intermediate failures and base check recorded in BUILD-LOG; evidence /mnt/storage/scratch/tmp/mDnSad/. Local candidate for review; no push or deployment. diff --git a/packages/harness/src/failure-notice.mjs b/packages/harness/src/failure-notice.mjs new file mode 100644 index 000000000..e44cbcd05 --- /dev/null +++ b/packages/harness/src/failure-notice.mjs @@ -0,0 +1,2 @@ +// Shared by the sender and recipient's reply-cycle check. +export const LENGTH_FAILURE_NOTICE_PREFIX = 'Mosaic turn failure: '; diff --git a/packages/harness/src/first-output.mjs b/packages/harness/src/first-output.mjs index f36549793..534c03f49 100644 --- a/packages/harness/src/first-output.mjs +++ b/packages/harness/src/first-output.mjs @@ -51,7 +51,8 @@ export function modelEventStream({ harness, runDir }) { if (event.type === 'agent_end') { const last = event.messages?.at(-1); if (last?.role === 'assistant') { - failed = ['error', 'aborted'].includes(last.stopReason); + failed = ['error', 'aborted'].includes(last.stopReason) || (last.stopReason === 'length' && + !(last.content ?? []).some(part => (part.type === 'text' && typeof part.text === 'string' && part.text.trim()) || part.type === 'toolCall')); answer = (last.content ?? []).filter(part => part.type === 'text').map(part => part.text + '\n').join(''); } } diff --git a/packages/harness/src/pi-extension.mjs b/packages/harness/src/pi-extension.mjs index ef34f8bcf..004f9f8e1 100644 --- a/packages/harness/src/pi-extension.mjs +++ b/packages/harness/src/pi-extension.mjs @@ -40,8 +40,14 @@ export default function (pi) { const verdict = decide(policy, event.toolName, event.input); return verdict.allow ? undefined : { block: true, reason: verdict.reason }; }); - pi.on("agent_end", async () => { - writeFileSync(marker, `${new Date().toISOString()}\n`); + pi.on("agent_end", async (event) => { + const last = event.messages?.at(-1); + const content = last?.role === "assistant" ? last.content ?? [] : []; + writeFileSync(marker, JSON.stringify({ at: new Date().toISOString(), + stopReason: last?.role === "assistant" ? last.stopReason : undefined, + hasText: content.some(part => part.type === "text" && typeof part.text === "string" && part.text.trim().length > 0), + hasToolCall: content.some(part => part.type === "toolCall"), + }) + "\n"); }); for (const t of tools) { diff --git a/packages/harness/src/runner.mjs b/packages/harness/src/runner.mjs index c7b7fecec..d1e4364f1 100644 --- a/packages/harness/src/runner.mjs +++ b/packages/harness/src/runner.mjs @@ -40,10 +40,16 @@ import { createServer, connect } from "node:net"; import { join } from "node:path"; import { fileURLToPath, pathToFileURL } from "node:url"; import { Client } from "../../bus/src/client.mjs"; +import { LENGTH_FAILURE_NOTICE_PREFIX } from "./failure-notice.mjs"; +import { ID_PATTERN } from "../../business/src/vocabulary.mjs"; import { firstOutputMarker, firstTurnMarker } from './first-output.mjs'; import { GUARD_CODES, FIXED_TEXTS } from "../../library/src/reply-protocol.mjs"; import { fetchReview, landWork, pushWorking } from "./git-tools.mjs"; +export const isLengthFailureNotice = m => m.class === "INFO" && typeof m.from_role === "string" + && ID_PATTERN.test(m.from_role) && !["human", "tester", "user"].includes(m.from_role) + && typeof m.body === "string" && m.body.startsWith(LENGTH_FAILURE_NOTICE_PREFIX); + export const EXIT = Object.freeze({ stopped: 0, founder: 20, claim: 21, session: 22, broker: 23, usage: 2 }); // Environment variables that carry a founder (human) credential for a @@ -441,6 +447,14 @@ function runTurn({ session, request, n, env, onChild }) { onChild(null); if (markerFailed) return resolve({ ok: false, markerFailed: true, reason: "first-turn marker publication failed" }); if (timedOut) return resolve({ ok: false, reason: `turn passed its ${session.turnTimeout} s wall clock` }); + // The managed event monitor exits 1 for this same length stop. Never mask + // a signal, timeout, marker publication failure, or other adapter exit. + if (session.turnEnd === "marker" && !signal && (code === 0 || (session.proxyFirstTurn && code === 1))) { + let status; + try { status = JSON.parse(readFileSync(marker, "utf8")); } catch {} + if (status?.stopReason === "length" && status.hasText === false && status.hasToolCall === false) + return resolve({ ok: false, emptyLength: true, reason: "output limit reached while thinking before any answer or tool call" }); + } if (code !== 0) return resolve({ ok: false, reason: signal ? `adapter killed by ${signal}` : `adapter exited ${code}` }); // A "marker" harness (row 65: the module's turnEnd()) writes the marker when its turn ends. if (session.turnEnd === "marker" && !existsSync(marker)) return resolve({ ok: false, reason: `${session.harness} exited before the turn ended (no turn marker)` }); @@ -598,6 +612,34 @@ export async function main(runDir, { stdin = process.stdin, env = process.env, l log("runner: first-turn marker publication failed"); return true; }; + const continuationRequest = "Your last turn hit the output limit while thinking, before any answer or tool call. Now answer or call a tool, and keep your reasoning short."; + // State checks the broker's stop switch before another process can start. + // The surrounding usage bracket captures every attempt in the transcript. + const continueLength = async (initial, counter) => { + let turn = initial; + while (turn.emptyLength && counter.count < 2 && !stopping) { + try { await client.call("state"); } + catch (error) { return { ok: false, brokerError: error, reason: "could not read the current state" }; } + if (stopping) break; + ++counter.count; + ++n; + turn = await runTurn({ session, request: continuationRequest, n, env: turnEnv, onChild: (c) => (child = c) }); + unslop.read(); + } + return turn; + }; + const recordLengthFailure = async m => { + const detail = "turn failed: output limit reached while thinking before any answer or tool call after two continuations"; + log(`runner: message ${m.id} ${detail}`); + // The existing routine event is visible to the chief via trail and to the + // Console on this message's trail; it does not send an automatic reply. + await client.call("event.emit", { kind: "action.allowed", body: { action: "routine", target: detail }, subject: m.id }); + if (m.from_role === "chief") { log(`runner: chief notice for ${m.id} skipped: chief is the sender`); return; } + try { + await client.call("message.send", { to: "chief", class: "INFO", in_reply_to: m.id, + body: `${LENGTH_FAILURE_NOTICE_PREFIX}Turn for message ${m.id} from ${m.from_role} failed: the output limit was reached while thinking, before any answer or tool call, on the turn and 2 continuations. No answer was sent.` }); + } catch (e) { log(`runner: chief notice for ${m.id} failed: ${e.code ?? e.message}`); } + }; for (const e of session.landing ?? []) { if (stopping) break; let block; @@ -655,18 +697,27 @@ export async function main(runDir, { stdin = process.stdin, env = process.env, l try { await client.call("usage.begin", { turn: m.id }); } catch (e) { log(`runner: usage baseline unavailable: ${e.code ?? "failed"}`); } } - const turn = unread + let turn = unread ? { ok: false, reason: unread } : await runTurn({ session, request: `${block}\n${turnRequest(m)}`, n, env: turnEnv, onChild: (c) => (child = c) }); unslop.read(); - // A failed/stopped harness can still have used tokens. Missing data remains unknown. + const continuationCount = { count: 0 }; + if (!turn.markerFailed && !stopping) turn = await continueLength(turn, continuationCount); + // One transcript delta covers the original attempt and all continuations, + // even if the runner is stopped or a state check fails. try { await client.call("usage.end", { turn: m.id }); } catch (e) { log(`runner: usage outcome unavailable: ${e.code ?? "failed"}`); } if (failedMarker(turn)) break loop; if (stopping) break loop; + if (turn.brokerError && ended(turn.brokerError)) break loop; + if (turn.emptyLength) { + try { await recordLengthFailure(m); await client.call("message.read", { id: m.id }); } + catch (e) { log(`runner: failure record for ${m.id} failed: ${e.code ?? e.message}`); if (ended(e, false)) break loop; } + continue; + } if (!turn.ok) log(`runner: turn ${n} for message ${m.id} failed: ${turn.reason}`); try { - if (m.class !== "RESULT" && (m.class === "REQUEST" || ["human", "tester", "user"].includes(m.from_role) || m.in_reply_to_class !== "RESULT")) { + if (!isLengthFailureNotice(m) && m.class !== "RESULT" && (m.class === "REQUEST" || ["human", "tester", "user"].includes(m.from_role) || m.in_reply_to_class !== "RESULT")) { if (!library) { const body = turn.ok ? turn.text || "(the session gave no answer)" : `The turn failed: ${turn.reason}`; await client.call("message.send", { to: m.from_role, body: clip(body) + unslop.note(), class: "RESULT", in_reply_to: m.id }); @@ -707,12 +758,21 @@ export async function main(runDir, { stdin = process.stdin, env = process.env, l ++n; try { await client.call("usage.begin", { turn: `${m.id}:retry` }); } catch (e) { log(`runner: usage baseline unavailable: ${e.code ?? "failed"}`); } - const retry = await runTurn({ session, request: `${block ?? ""}\n${turnRequest(m)}\n\nYour answer was refused with ${error.code}: ${reason}\nReply again in this same session.`, n, env: turnEnv, onChild: (c) => (child = c) }); + let retry = await runTurn({ session, request: `${block ?? ""}\n${turnRequest(m)}\n\nYour answer was refused with ${error.code}: ${reason}\nReply again in this same session.`, n, env: turnEnv, onChild: (c) => (child = c) }); unslop.read(); try { await client.call("usage.end", { turn: `${m.id}:retry` }); } catch (e) { log(`runner: usage outcome unavailable: ${e.code ?? "failed"}`); } if (failedMarker(retry)) break loop; if (stopping) break loop; + retry = await continueLength(retry, continuationCount); + if (failedMarker(retry)) break loop; + if (stopping) break loop; + if (retry.brokerError && ended(retry.brokerError)) break loop; + if (retry.emptyLength) { + await recordLengthFailure(m); + await client.call("message.read", { id: m.id }); + continue; + } if (!retry.ok) log(`runner: retry turn ${n} for message ${m.id} failed: ${retry.reason}`); try { await send(retry); } catch (again) { diff --git a/packages/harness/tests/citation-runner.test.mjs b/packages/harness/tests/citation-runner.test.mjs index 0eecff292..4458b6e7b 100644 --- a/packages/harness/tests/citation-runner.test.mjs +++ b/packages/harness/tests/citation-runner.test.mjs @@ -142,3 +142,40 @@ test('C2-06 the retry turn delivers citation-unavailable reason to the agent', a const requests = readFileSync(join(f.runDir, 'fixture-requests.jsonl'), 'utf8').trim().split('\n').map(JSON.parse); assert.equal(requests[1].includes(`citation-unavailable: ${reason}`), true); }); + +for (const library of [false, true]) { + test(`three empty length stops produce no sender reply and one visible failure, library=${library}`, async t => { + const f = await setup(t, () => Array(3).fill({marker:{stopReason:'length',hasText:false,hasToolCall:false}}), {library}); + const sent = await f.call(f.human, 'message.send', {to:'coder',body:'Fixture question.',class:'REQUEST'}); + await until(() => f.store.get("SELECT 1 FROM deliveries WHERE message=? AND op='read'", sent.id)); + assert.equal(f.count(),3); + assert.equal(f.store.get("SELECT count(*) n FROM messages WHERE from_role='coder' AND to_role='human'").n,0); + const events = f.store.all("SELECT * FROM events WHERE subject=? AND kind='action.allowed'", sent.id); + assert.equal(events.length,1); assert.match(events[0].body,/length|output limit/i); + assert.match(f.log(),/length|output limit/i); + }); +} + + +for (const library of [false, true]) { + test(`normal stop with text and empty text preserves replies, library=${library}`, async t => { + const f = await setup(t, () => [ + {text:'answer',marker:{stopReason:'stop',hasText:true,hasToolCall:false}}, + {text:'',marker:{stopReason:'stop',hasText:false,hasToolCall:false}}, + ], {library,requireCitation:false}); + assert.equal((await f.ask()).body,'answer'); + assert.equal((await f.ask()).body,library ? FIXED_TEXTS.empty : '(the session gave no answer)'); + assert.equal(f.count(),2); + }); +} + +test('a library guard retry that exhausts empty length continuations sends no fixed RESULT', async t => { + const f = await setup(t, () => [ + {text:'uncited'}, ...Array(3).fill({marker:{stopReason:'length',hasText:false,hasToolCall:false}}), + ]); + const sent = await f.call(f.human,'message.send',{to:'coder',body:'Fixture question.',class:'REQUEST'}); + await until(() => f.store.get("SELECT 1 FROM deliveries WHERE message=? AND op='read'",sent.id)); + assert.equal(f.count(),4); + assert.equal(f.store.get("SELECT count(*) n FROM messages WHERE from_role='coder' AND to_role='human'").n,0); + assert.equal(f.store.get("SELECT count(*) n FROM events WHERE subject=? AND kind='action.allowed'",sent.id).n,1); +}); diff --git a/packages/harness/tests/fixtures/citation-adapter.mjs b/packages/harness/tests/fixtures/citation-adapter.mjs index 565aac5d0..400c29267 100644 --- a/packages/harness/tests/fixtures/citation-adapter.mjs +++ b/packages/harness/tests/fixtures/citation-adapter.mjs @@ -16,6 +16,8 @@ for (const args of step.tools ?? []) { catch (e) { result = { ok: false, error: e.message }; } appendFileSync(join(root, 'fixture-tools.jsonl'), JSON.stringify(result) + '\n'); } +if (step.sleep) await new Promise(() => setInterval(() => {}, 1000)); if (step.fail) process.exit(3); process.stdout.write(step.text ?? ''); -writeFileSync(process.env.MOSAIC_TURN_MARKER, 'done\n'); +writeFileSync(process.env.MOSAIC_TURN_MARKER, step.marker ? JSON.stringify(step.marker) : 'done\n'); +if (step.exitCode) process.exit(step.exitCode); diff --git a/packages/harness/tests/model-activity.test.mjs b/packages/harness/tests/model-activity.test.mjs index 53a8c5839..985b427d4 100644 --- a/packages/harness/tests/model-activity.test.mjs +++ b/packages/harness/tests/model-activity.test.mjs @@ -275,3 +275,20 @@ test('Claude adapter preserves text-mode bytes for a tool-only final turn', asyn assert.deepEqual(streamed.stdout,text.stdout); assert.deepEqual(streamed.stdout,Buffer.from('\n')); assert.equal(readFileSync(join(f.runDir,'proxy-first-output'),'utf8'),'output\n'); }); + +test('Pi turn marker records final assistant stop reason and output presence', async t => { + const runDir = scratch(t); + writeFileSync(join(runDir, 'policy'), JSON.stringify({harness:'pi', tools:[], typed:[]})); + writeFileSync(join(runDir, 'tools'), '[]'); + const values = { MOSAIC_POLICY_FILE:join(runDir,'policy'), MOSAIC_TOOLS_FILE:join(runDir,'tools'), MOSAIC_TOOL_SOCKET:join(runDir,'sock'), MOSAIC_TURN_MARKER:join(runDir,'done') }; + const prior = Object.fromEntries(Object.keys(values).map(k => [k, process.env[k]])); + Object.assign(process.env, values); + t.after(() => { for (const [k,v] of Object.entries(prior)) { if (v === undefined) delete process.env[k]; else process.env[k] = v; } }); + const handlers = new Map(); + piExtension({on:(name,fn) => handlers.set(name,fn), registerTool(){}}); + for (const [content, hasText, hasToolCall] of [[[{type:'thinking',thinking:'x'}], false, false], [[{type:'text',text:'answer'}], true, false], [[{type:'toolCall',name:'read'}], false, true]]) { + await handlers.get('agent_end')({messages:[{role:'assistant',stopReason:'length',content}]}); + const marker = JSON.parse(readFileSync(join(runDir,'done'),'utf8')); + assert.equal(marker.stopReason, 'length'); assert.equal(marker.hasText, hasText); assert.equal(marker.hasToolCall, hasToolCall); + } +}); diff --git a/packages/harness/tests/model-stream.test.mjs b/packages/harness/tests/model-stream.test.mjs index 31f13e06c..916a7037b 100644 --- a/packages/harness/tests/model-stream.test.mjs +++ b/packages/harness/tests/model-stream.test.mjs @@ -111,3 +111,18 @@ for (const [name, events, marked] of [ if (marked) { assert.equal(fs.readFileSync(join(root,'proxy-first-output'),'utf8'), 'output\n'); assert.equal(Buffer.concat(chunks).toString(), 'final answer\n'); } }); } + +test('Pi rejects an empty length stop, but accepts text or a tool call', t => { + const root = fs.mkdtempSync(join(tmpdir(), 'length-')); + t.after(() => fs.rmSync(root, { recursive: true, force: true })); + for (const [content, fails] of [ + [[{type:'thinking', thinking:'private'}, {type:'text', text:' '}], true], + [[{type:'text', text:'answer'}], false], + [[{type:'toolCall', id:'t', name:'read'}], false], + ]) { + const stream = output.modelEventStream({ harness:'pi', runDir:root }); + stream.push(JSON.stringify({type:'agent_end',messages:[{role:'assistant',stopReason:'length',content}]}) + '\n'); + if (fails) assert.throws(() => stream.finish(), /model stream did not complete/); + else assert.doesNotThrow(() => stream.finish()); + } +}); diff --git a/packages/harness/tests/runner.test.mjs b/packages/harness/tests/runner.test.mjs index 6f24c6e5c..918a3239e 100644 --- a/packages/harness/tests/runner.test.mjs +++ b/packages/harness/tests/runner.test.mjs @@ -1066,3 +1066,98 @@ test("row 66 a refused send keeps the note: it rides the next RESULT the session assert.equal(next.body, 'env: fake-model\n\n[unslop: 1 reply of this session flagged since the last note: banned word "delve" x1. The session gets a correction notice on its next turn.]'); assert.equal(ctx.inbox.length, 1); }); + +test('empty length continues in the same session, records usage and relays the answer once', async t => { + const calls = []; + let ctx; + ctx = await setup(t, { usage: { collect: async (_session, phase, turn) => { const file = join(ctx.runDir, "fixture-count"); calls.push({phase, turn, attempts: existsSync(file) ? Number(readFileSync(file, "utf8")) : 0}); return {recorded:true}; } } }); + const fixture = new URL('./fixtures/citation-adapter.mjs', import.meta.url).pathname; + writeFileSync(join(ctx.dir, 'adapter.sh'), `exec '${process.execPath}' '${fixture}'\n`); + writeFileSync(join(ctx.runDir, 'fixture-steps.json'), JSON.stringify([ + {marker:{stopReason:'length',hasText:false,hasToolCall:false}}, {text:'recovered',marker:{stopReason:'stop',hasText:true,hasToolCall:false}}, + ])); + const r = startRunner(ctx.runDir, ctx.cap); t.after(() => r.child.kill('SIGKILL')); + await until(() => ctx.store.get("SELECT 1 FROM role_claims WHERE role='coder' AND op='claim'")); + const sent = ctx.call(ctx.pm, 'message.send', {to:'coder',body:'work',class:'REQUEST'}); + await until(() => ctx.store.get("SELECT 1 FROM deliveries WHERE message=? AND op='read'", sent.id)); + const rows = ctx.store.all("SELECT * FROM messages WHERE from_role='coder' AND to_role='pm'"); + assert.equal(rows.length,1); assert.equal(rows[0].class,'RESULT'); assert.equal(rows[0].body,'recovered'); + assert.equal(Number(readFileSync(join(ctx.runDir,'fixture-count'),'utf8')),2); + const requests = readFileSync(join(ctx.runDir,'fixture-requests.jsonl'),'utf8').trim().split('\n').map(JSON.parse); + assert.match(requests[1], /output limit.*thinking/i); assert.match(requests[1], /answer or call a tool/i); + assert.deepEqual(calls, [{phase:'begin',turn:sent.id,attempts:0},{phase:'end',turn:sent.id,attempts:2}]); +}); + + +test('empty length exhaustion records one usage bracket, one chief notice and one trail event', async t => { + const calls = []; let ctx; + ctx = await setup(t, { usage: { collect: async (_s, phase, turn) => { const file = join(ctx.runDir, 'fixture-count'); calls.push({phase,turn,attempts:existsSync(file) ? Number(readFileSync(file,'utf8')) : 0}); return {recorded:true}; } } }); + const fixture = new URL('./fixtures/citation-adapter.mjs', import.meta.url).pathname; + writeFileSync(join(ctx.dir,'adapter.sh'), `exec '${process.execPath}' '${fixture}'\n`); + writeFileSync(join(ctx.runDir,'fixture-steps.json'), JSON.stringify(Array(3).fill({marker:{stopReason:'length',hasText:false,hasToolCall:false}}))); + const r = startRunner(ctx.runDir, ctx.cap); t.after(() => r.child.kill('SIGKILL')); + await until(() => ctx.store.get("SELECT 1 FROM role_claims WHERE role='coder' AND op='claim'")); + const sent = ctx.call(ctx.pm,'message.send',{to:'coder',body:'work',class:'REQUEST'}); + await until(() => ctx.store.get("SELECT 1 FROM deliveries WHERE message=? AND op='read'",sent.id)); + assert.deepEqual(calls,[{phase:'begin',turn:sent.id,attempts:0},{phase:'end',turn:sent.id,attempts:3}]); + assert.equal(ctx.store.get("SELECT count(*) n FROM messages WHERE from_role='coder' AND to_role='pm'").n,0); + const notice = ctx.store.all("SELECT * FROM messages WHERE from_role='coder' AND to_role='chief'"); + assert.equal(notice.length,1); assert.equal(notice[0].class,'INFO'); assert.equal(notice[0].in_reply_to,sent.id); + assert.match(notice[0].body,/output limit.*thinking/i); + assert.equal(ctx.store.get("SELECT count(*) n FROM events WHERE subject=? AND kind='action.allowed'",sent.id).n,1); +}); + +test('chief does not reply to the failure notice, but replies to ordinary agent and prefixed human INFO', async t => { + const { LENGTH_FAILURE_NOTICE_PREFIX } = await import('../src/failure-notice.mjs'); + const ctx = await setup(t,{as:'chief'}); + const coder = ctx.broker.bindLaunch({business:'demo',role:'coder',run:'coder-notifier',harness:'pi'}); + ctx.call(coder,'role.claim'); + const r = startRunner(ctx.runDir,ctx.cap); t.after(() => r.child.kill('SIGKILL')); + await until(() => ctx.store.get("SELECT 1 FROM role_claims WHERE role='chief' AND op='claim'")); + const seed = ctx.call(ctx.pm,'message.send',{to:'coder',body:'work',class:'REQUEST'}); + const notice = ctx.call(coder,'message.send',{to:'chief',body:LENGTH_FAILURE_NOTICE_PREFIX+'fixture',class:'INFO',in_reply_to:seed.id}); + const ordinary = ctx.call(coder,'message.send',{to:'chief',body:'ordinary information',class:'INFO',in_reply_to:seed.id}); + const human = ctx.call(ctx.human,'message.send',{to:'chief',body:LENGTH_FAILURE_NOTICE_PREFIX+'human information',class:'INFO',in_reply_to:seed.id}); + await until(() => ctx.store.get("SELECT 1 FROM deliveries WHERE message=? AND op='read'",human.id)); + assert.equal(ctx.store.get("SELECT count(*) n FROM messages WHERE from_role='chief' AND in_reply_to=?",notice.id).n,0); + for (const id of [ordinary.id,human.id]) assert.equal(ctx.store.get("SELECT count(*) n FROM messages WHERE from_role='chief' AND class='RESULT' AND in_reply_to=?",id).n,1); +}); + + +test('stop interrupts a length continuation and still closes usage', async t => { + const calls = []; let ctx; + ctx = await setup(t,{usage:{collect:async (_s,phase,turn) => { calls.push({phase,turn}); return {recorded:true}; }}}); + const fixture = new URL('./fixtures/citation-adapter.mjs',import.meta.url).pathname; + writeFileSync(join(ctx.dir,'adapter.sh'),`exec '${process.execPath}' '${fixture}'\n`); + writeFileSync(join(ctx.runDir,'fixture-steps.json'),JSON.stringify([{marker:{stopReason:'length',hasText:false,hasToolCall:false}},{sleep:true}])); + const r = startRunner(ctx.runDir,ctx.cap); t.after(() => r.child.kill('SIGKILL')); + await until(() => ctx.store.get("SELECT 1 FROM role_claims WHERE role='coder' AND op='claim'")); + const sent = ctx.call(ctx.pm,'message.send',{to:'coder',body:'work',class:'REQUEST'}); + await until(() => existsSync(join(ctx.runDir,'fixture-count')) && readFileSync(join(ctx.runDir,'fixture-count'),'utf8') === '2'); + r.child.kill('SIGTERM'); assert.equal((await r.exited).code,EXIT.stopped); + assert.deepEqual(calls,[{phase:'begin',turn:sent.id},{phase:'end',turn:sent.id}]); + assert.equal(ctx.store.get("SELECT count(*) n FROM messages WHERE from_role='coder'").n,0); +}); + +test('managed event monitor exit 1 with empty length marker continues', async t => { + const ctx = await setup(t); + const fixture = new URL('./fixtures/citation-adapter.mjs',import.meta.url).pathname; + writeFileSync(join(ctx.dir,'adapter.sh'),`exec '${process.execPath}' '${fixture}'\n`); + writeFileSync(join(ctx.runDir,'proxy-first-turn-enabled'),'enabled\n'); + writeFileSync(join(ctx.runDir,'fixture-steps.json'),JSON.stringify([{marker:{stopReason:'length',hasText:false,hasToolCall:false},exitCode:1},{text:'recovered'}])); + const r = startRunner(ctx.runDir,ctx.cap); t.after(() => r.child.kill('SIGKILL')); + await until(() => ctx.store.get("SELECT 1 FROM role_claims WHERE role='coder' AND op='claim'")); + const sent = ctx.call(ctx.pm,'message.send',{to:'coder',body:'work',class:'REQUEST'}); + const reply = await until(() => ctx.store.get("SELECT * FROM messages WHERE in_reply_to=? AND class='RESULT'",sent.id)); + assert.equal(reply.body,'recovered'); assert.equal(readFileSync(join(ctx.runDir,'fixture-count'),'utf8'),'2'); +}); + +test('failure notice recognition excludes bus notices and non-agent senders', async () => { + const { LENGTH_FAILURE_NOTICE_PREFIX } = await import('../src/failure-notice.mjs'); + const { isLengthFailureNotice } = await import('../src/runner.mjs'); + const notice = {class:'INFO',from_role:'coder',body:LENGTH_FAILURE_NOTICE_PREFIX+'fixture'}; + assert.equal(isLengthFailureNotice(notice),true); + for (const from_role of ['human','tester','user','_bus']) assert.equal(isLengthFailureNotice({...notice,from_role}),false,from_role); + assert.equal(isLengthFailureNotice({...notice,class:'REQUEST'}),false); + assert.equal(isLengthFailureNotice({...notice,body:'ordinary information'}),false); +}); diff --git a/packages/seat/src/sandbox.mjs b/packages/seat/src/sandbox.mjs index 12e5d5878..af6d8e0c8 100644 --- a/packages/seat/src/sandbox.mjs +++ b/packages/seat/src/sandbox.mjs @@ -49,6 +49,7 @@ const KILL_WITH_PARENT = ["/usr/bin/setpriv", "--pdeathsig", "KILL", "--"]; // is missing here. export const RUNNER_FILES = Object.freeze([ "packages/harness/src/runner.mjs", + "packages/harness/src/failure-notice.mjs", // Row 331: fixed output-exists marker; writes only within the existing run directory. "packages/harness/src/first-output.mjs", "packages/harness/src/gate.mjs",