mosaicstack/stack
16
0
Fork 1
Code Issues 330 Pull Requests 12 Actions Packages Projects Releases 15 Wiki Activity

agent message-send wrapper reports delivery from a heuristic that is wrong in both directions #938

New Issue
Open
opened 2026-07-27 14:34:09 +00:00 by Ghost · 1 comment
Ghost commented 2026-07-27 14:34:09 +00:00
Copy Link
Copy Source

The agent message-send wrapper reports delivery status from a heuristic rather than from a confirmed submission, and the heuristic is wrong in both directions. This matters more than an ordinary flaky tool: it is the only control channel between an orchestrator and its workers, so its confirmation is load-bearing for every dispatch.

Both failure modes observed

False negative — reports failure after a successful delivery. Observed four times in a single session. The message arrived, the receiving agent acknowledged it correctly and began the work, and the wrapper reported it could not confirm submission.

False positive — reports success after a swallowed delivery. An interactive prompt can render on top of a working agent pane (a rating/feedback overlay, a confirmation prompt). Such a prompt is input-consuming: keystrokes sent to that pane are eaten by the overlay instead of reaching the agent, while the send path still reports success.

Why the direction matters, separately

  • A false negative invites a resend. A resend of a dispatch that already landed produces double-dispatch — two workers on one task, or one worker with two conflicting charters. The safe-looking response to the error is the harmful one.
  • A false positive silently loses an instruction. Nothing surfaces; the sender believes the work is in flight and the worker never received it. This is only detectable later, by absence.

Both are silent. In practice the only reliable detection is out-of-band corroboration — watching the receiving agent's context/token growth, its own echo of the charter, or subsequent artifacts. That is the tell that the defect is real: the tool's own confirmation is unusable, which defeats the purpose of having one.

Suggested fix

Return a tri-state, not a boolean.

  • delivered — submission positively confirmed
  • draft-unsubmitted — text reached the input but was not submitted
  • unknown — could not determine

Confirm by positive post-submit readback rather than by locating a prompt. The current approach infers state from prompt detection, which is exactly what an overlay defeats. Reading back the submitted content (or a post-submit marker) distinguishes arrived and submitted from arrived and sitting in a draft from never arrived.

Never collapse unknown into either success or failure. The caller can handle a declared unknown safely — corroborate before resending. It cannot handle a confident wrong answer, and both of the observed failures are confident wrong answers.

Optionally, detecting a known input-consuming overlay and refusing to send until it is cleared would remove the false-positive path entirely, rather than reporting around it.

Related but distinct: the issue-create.sh label-drop defect. Both share the shape of a command reporting an outcome its execution did not deliver, but they are separate code paths and separate fixes.

The agent message-send wrapper reports delivery status from a heuristic rather than from a confirmed submission, and **the heuristic is wrong in both directions**. This matters more than an ordinary flaky tool: it is the only control channel between an orchestrator and its workers, so its confirmation is load-bearing for every dispatch. ## Both failure modes observed **False negative — reports failure after a successful delivery.** Observed four times in a single session. The message arrived, the receiving agent acknowledged it correctly and began the work, and the wrapper reported it could not confirm submission. **False positive — reports success after a swallowed delivery.** An interactive prompt can render *on top of* a working agent pane (a rating/feedback overlay, a confirmation prompt). Such a prompt is **input-consuming**: keystrokes sent to that pane are eaten by the overlay instead of reaching the agent, while the send path still reports success. ## Why the direction matters, separately - A **false negative** invites a resend. A resend of a dispatch that already landed produces **double-dispatch** — two workers on one task, or one worker with two conflicting charters. The safe-looking response to the error is the harmful one. - A **false positive** silently loses an instruction. Nothing surfaces; the sender believes the work is in flight and the worker never received it. This is only detectable later, by absence. Both are silent. In practice the only reliable detection is **out-of-band corroboration** — watching the receiving agent's context/token growth, its own echo of the charter, or subsequent artifacts. That is the tell that the defect is real: *the tool's own confirmation is unusable, which defeats the purpose of having one.* ## Suggested fix **Return a tri-state, not a boolean.** - `delivered` — submission positively confirmed - `draft-unsubmitted` — text reached the input but was not submitted - `unknown` — could not determine **Confirm by positive post-submit readback rather than by locating a prompt.** The current approach infers state from prompt detection, which is exactly what an overlay defeats. Reading back the submitted content (or a post-submit marker) distinguishes *arrived and submitted* from *arrived and sitting in a draft* from *never arrived*. **Never collapse `unknown` into either success or failure.** The caller can handle a declared `unknown` safely — corroborate before resending. It cannot handle a confident wrong answer, and both of the observed failures are confident wrong answers. Optionally, detecting a known input-consuming overlay and refusing to send until it is cleared would remove the false-positive path entirely, rather than reporting around it. --- Related but distinct: the `issue-create.sh` label-drop defect. Both share the shape of *a command reporting an outcome its execution did not deliver*, but they are separate code paths and separate fixes.
Ghost commented 2026-07-27 15:59:52 +00:00
Copy Link
Copy Source

Refinement from a subsequent observation — the proposed tri-state is insufficient.

A later occurrence showed a non-zero return with "REPL input prompt not locatable" that was not a failure at all. The target agent was mid-turn, actively working, with no idle prompt to submit into. The send correctly did not deliver, and that was the right outcome — delivering an instruction into an actively-building agent is precisely what the caller's own discipline forbids.

So the same non-zero return currently covers at least three distinct states:

  1. Delivered anyway — the message landed and the agent acted on it; the return code was simply wrong. (Observed repeatedly.)
  2. Not delivered, target busy — the agent is mid-turn and not accepting input. Not an error; arguably the desired behaviour.
  3. Not delivered, unknown cause — genuinely indeterminate.

These require opposite caller responses. State 1 must not be retried — retrying a delivered dispatch causes double-dispatch. State 2 should be retried later, and needs no corroboration at all. State 3 requires out-of-band corroboration before doing anything. Collapsing all three into one non-zero return forces the caller to guess, and the guesses have asymmetric costs.

Revised proposal — four states, not three:

  • delivered — submission positively confirmed
  • draft-unsubmitted — text reached the input but was not submitted
  • target-busy — no input prompt available; target is mid-turn. Safe, expected, retry later.
  • unknown — could not determine

target-busy is the important addition. It is a normal operating condition, not an error, and reporting it distinctly removes the largest source of ambiguity: it tells the caller "your message did not land, and that is fine, and you need not investigate."

There is a secondary benefit worth noting: an accurate target-busy signal is a cheap, reliable liveness check. A target that reports busy is demonstrably alive and working — which is more trustworthy than reading a terminal pane, since a pane shows the last draw rather than current state and can display a stale frame for many minutes during a long turn with no output.

**Refinement from a subsequent observation — the proposed tri-state is insufficient.** A later occurrence showed a non-zero return with "REPL input prompt not locatable" that was **not a failure at all**. The target agent was mid-turn, actively working, with no idle prompt to submit into. The send correctly did not deliver, and *that was the right outcome* — delivering an instruction into an actively-building agent is precisely what the caller's own discipline forbids. So the same non-zero return currently covers at least three distinct states: 1. **Delivered anyway** — the message landed and the agent acted on it; the return code was simply wrong. (Observed repeatedly.) 2. **Not delivered, target busy** — the agent is mid-turn and not accepting input. Not an error; arguably the desired behaviour. 3. **Not delivered, unknown cause** — genuinely indeterminate. These require *opposite* caller responses. State 1 must **not** be retried — retrying a delivered dispatch causes double-dispatch. State 2 **should** be retried later, and needs no corroboration at all. State 3 requires out-of-band corroboration before doing anything. Collapsing all three into one non-zero return forces the caller to guess, and the guesses have asymmetric costs. **Revised proposal — four states, not three:** - `delivered` — submission positively confirmed - `draft-unsubmitted` — text reached the input but was not submitted - `target-busy` — no input prompt available; target is mid-turn. **Safe, expected, retry later.** - `unknown` — could not determine `target-busy` is the important addition. It is a *normal operating condition*, not an error, and reporting it distinctly removes the largest source of ambiguity: it tells the caller "your message did not land, and that is fine, and you need not investigate." There is a secondary benefit worth noting: an accurate `target-busy` signal is a **cheap, reliable liveness check**. A target that reports busy is demonstrably alive and working — which is more trustworthy than reading a terminal pane, since a pane shows the last draw rather than current state and can display a stale frame for many minutes during a long turn with no output.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Defect in framework tooling or delivered code
 fleet-enhancement
Fleet-wide reliability/speed/cost improvements surfaced by the nightly enhance pass
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
code-be-01 (Mosaic fleet seat code-be-01) code-be-02 (Mosaic fleet seat code-be-02) code-infra-01 (Mosaic fleet seat code-infra-01) fargo fred gate-merge-01 (Mosaic fleet seat gate-merge-01) happy jason.woltje (Jason Woltje) merge-gate ops-01 (Mosaic fleet seat ops-01) ops-02 (Mosaic fleet seat ops-02) ops-03 (Mosaic fleet seat ops-03) ops-ci-01 (Mosaic fleet seat ops-ci-01) ops-deploy-01 (Mosaic fleet seat ops-deploy-01) pepper rev-code-01 rev-code-02 rev-security-01 rev-security-02 rev-security-03 (Mosaic fleet seat rev-security-03) sanity scooby (Scooby) scrappy shaggy tiny velma veronica (Mosaic fleet seat veronica) vision woodpecker
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mosaicstack/stack#938