mosaicstack/stack
25
0 Fork 1
Code Issues 406 Pull Requests 13 Packages Projects Releases 15 Wiki Activity

P1: coder ends every turn on thinking at the 16384 output limit; harness relays the empty turn as RESULT #1852

New Issue
Open
opened 2026-11-06 21:29:49 +00:00 by jason.woltje · 0 comments
jason.woltje commented 2026-11-06 21:29:49 +00:00
Owner
Copy link
Copy source

Summary

On ui797 campaign5 (code 3598839e6, pi 0.85.1), the coder (glm-5.3-flash, thinking high) ended four turns in a row with stopReason: "length". Each turn ran out the 16384-token output limit while still thinking. None produced text or a tool call. The harness treated each one as a successful turn with no answer and sent (the session gave no answer) to the reviewer as a RESULT. The reviewer then concluded the coder was dead and raised a role.revoke decision. The coder made no progress after 21:04Z. The rest of the team kept talking, and processing was halted at 21:25:58Z.

Evidence (ui797 on .53)

Run r91287cc01692 (coder). Session file launches/mvp3-ui797/r91287cc01692/session/2026-11-06T21-00-04-471Z_01a1b2a9-55f6-7122-8278-9cc0a9cbf5b2.jsonl:

  • Entry 1: provider mosaic-proxy-glm, model glm-5.3-flash. Entry 2: thinkingLevel high.
  • Real work ran 21:00:04 to 21:04:03. Entry 30 (21:04:03) already had output 10233 and 34275 thinking chars.
  • The stalled turns, all stopReason: "length", output 16384, text 0 chars, no tool call:
entry time (UTC) thinking chars
32 21:08:14 57186
34 21:12:34 55537
36 21:16:45 55460
38 21:21:16 56458
  • Each turn took about 4.3 minutes and about 38k tokens (input plus output plus cache).
  • usage_records for the run: seq 781 (end of turn 1: input 29624, output 30037, cache_read 159168), then 794, 800, 802.
  • Messages: seq 455 (21:12:34.516Z) and 458 (21:21:16.852Z) are coder → reviewer RESULT (the session gave no answer).
  • What followed: seq 456 _bus told the chief the reviewer had no running session, and the chief launched reviewer r0ef841ffad36. Seq 457: the reviewer asked the coder for a proper REVIEW-REQUEST. Seq 459: the reviewer told the chief the coder looks dead and raised decision 814d4579 (role.revoke). Seq 460: the chief relayed this to the assistant.
  • The coder workspace stayed at a6ff951, with no request_events after the resume at 20:59:31Z.

Where the values come from (read at 3598839e6)

Coder thinking level high. It comes from the instance business config, not a role file and not an adapter default.

  • /srv/mosaic/.config/mosaic-ui797/businesses/mvp3-ui797.json, roles.coder.vars {"harness":"pi","model":"glm","thinking":"high"} (about lines 101-103). Every role in that file (pm, cto, coder, reviewer, chief, assistant) is set to high. The run manifest bundle/manifest.json:13 records "thinking": "high".
  • packages/cli/src/launcher.mjs:368: the route's harness, model and thinking come only from the instance's vars (row 65, #1558). The level is checked at :399.
  • packages/plugins/harness/pi/index.mjs:118 passes it on as MOSAIC_THINKING: session.thinking.
  • adapters/pi/adapter.sh:89-93 turns it into --thinking $MOSAIC_THINKING. Without it, pi keeps its own default.

The 16384 output cap is pi's default. Our plugin never sets it.

  • node_modules/@earendil-works/pi-coding-agent/dist/core/provider-composer.js:73: maxTokens: definition.maxTokens ?? 16384 (contextWindow ?? 128000 at :72).
  • packages/plugins/harness/pi/index.mjs:93 registers the proxy model as { id: entry.model, reasoning: true }, with no maxTokens and no contextWindow. The glm route uses api openai-completions with compat: { thinkingFormat: "zai", zaiToolStream: true } (:86-90).
  • node_modules/@earendil-works/pi-ai/dist/api/openai-completions.js:600-605 sends max_completion_tokens from that maxTokens. agents/fethry/work/queue-332/BUILD.md:270 shows the proxy forwarding it as max_tokens=16384.

No thinking budget is sent.

  • openai-completions.js:634-643: the zai format sends thinking: {type: "enabled", clear_thinking: false}, plus reasoning_effort only if the model supports it.
  • :737-742: a token budget field is sent only when thinkingTokenBudgetField / supportsThinkingTokenBudget is set (resolved at :767-771). Our compat sets neither. pi's own comment there describes this failure: reasoning and the answer share max_tokens, so an uncapped reasoning phase can use the whole response and leave no answer and no tool call.
  • pi-ai/dist/api/simple-options.js:39-44: the default high budget is 16384, the same size as the whole output cap.

Harness side

A length turn with no text and no tool call is reported as a success:

  • packages/harness/src/pi-extension.mjs:43-45 writes the turn marker on agent_end whatever the stopReason is.
  • packages/harness/src/runner.mjs:437-447 (runTurn) resolves { ok: true, text: out.trim() } on exit 0 plus the marker. It never sees stopReason.
  • packages/harness/src/runner.mjs:671: turn.ok ? turn.text || "(the session gave no answer)" : \The turn failed: ${turn.reason}`. Under the condition at :669, the reply goes to m.from_roleas a RESULT. The library path does the same withFIXED_TEXTS.empty (packages/library/src/reply-protocol.mjs:8`).
  • packages/harness/src/first-output.mjs:51-55 counts only error and aborted as failed stopReasons. length is not one of them.

The result: the reviewer receives an empty RESULT that looks like a deliberate non-answer, and the dead-coder decision follows. A length turn with no text and no tool call should count as a failed turn. The harness should either continue it (with a bound) or report the failure visibly. It should never relay it as an answer.

Fix options

  1. Raise maxTokens for the proxy's thinking models (plain adapter fix). In packages/plugins/harness/pi/index.mjs:93, set maxTokens (and contextWindow) per proxy model, and set a thinking-budget field in compat if the proxy honors one. Tradeoff: more output and more spend per turn. The limits glm-5.3-flash and the proxy accept are not verified yet. A budget field helps only if GLM or the proxy actually applies it.
  2. Lower the coder's thinking on glm-5.3-flash (policy change). Set thinking to medium or lower in the instance business config vars. This is not a roles/*.json commit, but it changes the instance's model route settings, so it needs Jason's say-so (owner settings). Tradeoff: shallower reasoning on hard steps. It also does not guarantee the next turn fits.
  3. Handle length in the harness (plain harness fix). Carry stopReason from agent_end (pi-extension.mjs:43) into the turn marker. In runTurn / runner.mjs:671, treat length with no text and no tool call as a failed turn. Either send one bounded continuation ("continue; answer now") or send The turn failed: output limit reached while thinking to the sender. Also add length to the failed set in first-output.mjs:54. Tradeoff: a continuation without a hard bound can loop and burn tokens, so the bound has to be one or two attempts and then a visible failure.

Recommendation: 3 plus 1. Option 3 stops an empty turn from ever passing as an answer, whatever the cause. Option 1 removes this cause. Option 2 is a policy change and stays with Jason.

Note against #1849

With the coder making no progress, the chief, pm, reviewer and assistant still used about 2.83M tokens in 25 minutes (2,834,703 of the 15M campaign5 cap at 21:24:54Z; resume was 20:59:31Z). That is about 6-7M an hour. No team-level check noticed that the only producing seat had stopped.

State left

  • Processing halted at 2026-11-06T21:25:58.129Z by ui797-owner through the Console (generation 628caec7-7e0d-4577-95fc-04c9b89bbbaa, quiescent). All five runs ended with reason kill-switch.
  • Decisions 814d4579 and 9770b04b are left open. mosaic-ui797-budget@campaign5 is still active.
  • No fix is built yet.
## Summary On ui797 campaign5 (code 3598839e6, pi 0.85.1), the coder (glm-5.3-flash, thinking `high`) ended four turns in a row with `stopReason: "length"`. Each turn ran out the 16384-token output limit while still thinking. None produced text or a tool call. The harness treated each one as a successful turn with no answer and sent `(the session gave no answer)` to the reviewer as a RESULT. The reviewer then concluded the coder was dead and raised a role.revoke decision. The coder made no progress after 21:04Z. The rest of the team kept talking, and processing was halted at 21:25:58Z. ## Evidence (ui797 on .53) Run `r91287cc01692` (coder). Session file `launches/mvp3-ui797/r91287cc01692/session/2026-11-06T21-00-04-471Z_01a1b2a9-55f6-7122-8278-9cc0a9cbf5b2.jsonl`: - Entry 1: provider `mosaic-proxy-glm`, model `glm-5.3-flash`. Entry 2: thinkingLevel `high`. - Real work ran 21:00:04 to 21:04:03. Entry 30 (21:04:03) already had output 10233 and 34275 thinking chars. - The stalled turns, all `stopReason: "length"`, output 16384, text 0 chars, no tool call: | entry | time (UTC) | thinking chars | |---|---|---| | 32 | 21:08:14 | 57186 | | 34 | 21:12:34 | 55537 | | 36 | 21:16:45 | 55460 | | 38 | 21:21:16 | 56458 | - Each turn took about 4.3 minutes and about 38k tokens (input plus output plus cache). - usage_records for the run: seq 781 (end of turn 1: input 29624, output 30037, cache_read 159168), then 794, 800, 802. - Messages: seq 455 (21:12:34.516Z) and 458 (21:21:16.852Z) are coder → reviewer RESULT `(the session gave no answer)`. - What followed: seq 456 `_bus` told the chief the reviewer had no running session, and the chief launched reviewer `r0ef841ffad36`. Seq 457: the reviewer asked the coder for a proper REVIEW-REQUEST. Seq 459: the reviewer told the chief the coder looks dead and raised decision `814d4579` (role.revoke). Seq 460: the chief relayed this to the assistant. - The coder workspace stayed at `a6ff951`, with no request_events after the resume at 20:59:31Z. ## Where the values come from (read at 3598839e6) **Coder thinking level `high`.** It comes from the instance business config, not a role file and not an adapter default. - `/srv/mosaic/.config/mosaic-ui797/businesses/mvp3-ui797.json`, roles.coder.vars `{"harness":"pi","model":"glm","thinking":"high"}` (about lines 101-103). Every role in that file (pm, cto, coder, reviewer, chief, assistant) is set to `high`. The run manifest `bundle/manifest.json:13` records `"thinking": "high"`. - `packages/cli/src/launcher.mjs:368`: the route's harness, model and thinking come only from the instance's vars (row 65, #1558). The level is checked at `:399`. - `packages/plugins/harness/pi/index.mjs:118` passes it on as `MOSAIC_THINKING: session.thinking`. - `adapters/pi/adapter.sh:89-93` turns it into `--thinking $MOSAIC_THINKING`. Without it, pi keeps its own default. **The 16384 output cap is pi's default. Our plugin never sets it.** - `node_modules/@earendil-works/pi-coding-agent/dist/core/provider-composer.js:73`: `maxTokens: definition.maxTokens ?? 16384` (`contextWindow ?? 128000` at `:72`). - `packages/plugins/harness/pi/index.mjs:93` registers the proxy model as `{ id: entry.model, reasoning: true }`, with no maxTokens and no contextWindow. The glm route uses api `openai-completions` with `compat: { thinkingFormat: "zai", zaiToolStream: true }` (`:86-90`). - `node_modules/@earendil-works/pi-ai/dist/api/openai-completions.js:600-605` sends `max_completion_tokens` from that maxTokens. `agents/fethry/work/queue-332/BUILD.md:270` shows the proxy forwarding it as `max_tokens=16384`. **No thinking budget is sent.** - `openai-completions.js:634-643`: the zai format sends `thinking: {type: "enabled", clear_thinking: false}`, plus `reasoning_effort` only if the model supports it. - `:737-742`: a token budget field is sent only when `thinkingTokenBudgetField` / `supportsThinkingTokenBudget` is set (resolved at `:767-771`). Our compat sets neither. pi's own comment there describes this failure: reasoning and the answer share max_tokens, so an uncapped reasoning phase can use the whole response and leave no answer and no tool call. - `pi-ai/dist/api/simple-options.js:39-44`: the default `high` budget is 16384, the same size as the whole output cap. ## Harness side A `length` turn with no text and no tool call is reported as a success: - `packages/harness/src/pi-extension.mjs:43-45` writes the turn marker on `agent_end` whatever the stopReason is. - `packages/harness/src/runner.mjs:437-447` (`runTurn`) resolves `{ ok: true, text: out.trim() }` on exit 0 plus the marker. It never sees stopReason. - `packages/harness/src/runner.mjs:671`: `turn.ok ? turn.text || "(the session gave no answer)" : \`The turn failed: ${turn.reason}\``. Under the condition at `:669`, the reply goes to `m.from_role` as a RESULT. The library path does the same with `FIXED_TEXTS.empty` (`packages/library/src/reply-protocol.mjs:8`). - `packages/harness/src/first-output.mjs:51-55` counts only `error` and `aborted` as failed stopReasons. `length` is not one of them. The result: the reviewer receives an empty RESULT that looks like a deliberate non-answer, and the dead-coder decision follows. A `length` turn with no text and no tool call should count as a failed turn. The harness should either continue it (with a bound) or report the failure visibly. It should never relay it as an answer. ## Fix options 1. **Raise maxTokens for the proxy's thinking models (plain adapter fix).** In `packages/plugins/harness/pi/index.mjs:93`, set `maxTokens` (and `contextWindow`) per proxy model, and set a thinking-budget field in compat if the proxy honors one. Tradeoff: more output and more spend per turn. The limits glm-5.3-flash and the proxy accept are not verified yet. A budget field helps only if GLM or the proxy actually applies it. 2. **Lower the coder's thinking on glm-5.3-flash (policy change).** Set `thinking` to `medium` or lower in the instance business config vars. This is not a `roles/*.json` commit, but it changes the instance's model route settings, so it needs Jason's say-so (owner settings). Tradeoff: shallower reasoning on hard steps. It also does not guarantee the next turn fits. 3. **Handle `length` in the harness (plain harness fix).** Carry stopReason from `agent_end` (`pi-extension.mjs:43`) into the turn marker. In `runTurn` / `runner.mjs:671`, treat `length` with no text and no tool call as a failed turn. Either send one bounded continuation ("continue; answer now") or send `The turn failed: output limit reached while thinking` to the sender. Also add `length` to the failed set in `first-output.mjs:54`. Tradeoff: a continuation without a hard bound can loop and burn tokens, so the bound has to be one or two attempts and then a visible failure. Recommendation: 3 plus 1. Option 3 stops an empty turn from ever passing as an answer, whatever the cause. Option 1 removes this cause. Option 2 is a policy change and stays with Jason. ## Note against #1849 With the coder making no progress, the chief, pm, reviewer and assistant still used about 2.83M tokens in 25 minutes (2,834,703 of the 15M campaign5 cap at 21:24:54Z; resume was 20:59:31Z). That is about 6-7M an hour. No team-level check noticed that the only producing seat had stopped. ## State left - Processing halted at 2026-11-06T21:25:58.129Z by ui797-owner through the Console (generation `628caec7-7e0d-4577-95fc-04c9b89bbbaa`, quiescent). All five runs ended with reason kill-switch. - Decisions `814d4579` and `9770b04b` are left open. `mosaic-ui797-budget@campaign5` is still active. - No fix is built yet.
jason.woltje added the bug label 2026-11-06 21:29:49 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
bug
Defect in framework tooling or delivered code
 fleet-enhancement
Fleet-wide reliability/speed/cost improvements surfaced by the nightly enhance pass
No labels bug
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
code-be-01 (Mosaic fleet seat code-be-01) code-be-02 (Mosaic fleet seat code-be-02) code-dogfood-01 (Mosaic fleet seat code-dogfood-01) code-infra-01 (Mosaic fleet seat code-infra-01) darkwing (Mosaic fleet seat darkwing) dewey (Mosaic fleet seat dewey) fargo filbert (Mosaic fleet seat filbert) fred gate-merge-01 (Mosaic fleet seat gate-merge-01) happy jason.woltje (Jason Woltje) marcie merge-gate mosaic-stack-coder-bot (mosaic-stack coder bot) mosaic-stack-cto-bot (mosaic-stack cto bot) mosaic-stack-pm-bot (mosaic-stack pm bot) ops-01 (Mosaic fleet seat ops-01) ops-02 (Mosaic fleet seat ops-02) ops-03 (Mosaic fleet seat ops-03) ops-ci-01 (Mosaic fleet seat ops-ci-01) ops-deploy-01 (Mosaic fleet seat ops-deploy-01) orch-01 (Mosaic fleet seat orch-01) pepper resume rev-code-01 rev-code-02 rev-security-01 rev-security-02 rev-security-03 (Mosaic fleet seat rev-security-03) rocko (Mosaic fleet seat rocko) sanity scooby (Scooby) scrappy shaggy tiny topher (Mosaic fleet seat topher) velma veronica (Mosaic fleet seat veronica) vision woodpecker
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mosaicstack/stack#1852