fix(framework): send-message.sh fail-loud submission verdict + regression tests (Patch 6)
Some checks failed
ci/woodpecker/pr/ci Pipeline failed

Invert the delivery verdict from negative to positive evidence. Previously,
success was declared by the ABSENCE of a draft snippet on a prompt line: if
the `❯|^>|│ >` glyph matched nothing (busy-receiver draft on an unmatched
line, or wrong-pane drift), an UNSUBMITTED message read as "✓ delivered"
exit 0 — a silent worker->lead relay stall (filed by UC-LEAD, PR #3046:
three review verdicts reported delivered but never surfaced for ~50 min).

Fix: success now requires POSITIVE evidence — either the queued banner, or
the REPL input box located AND clear of the message tail. If the input box
cannot be located, status is `unconfirmed` -> exit 2 + stderr ("could not
confirm submission ... may be UNDELIVERED"). The near-dead `*)` indeterminate
default also flips from silently returning 0 to exit 2. Bracketed-paste and
double-Enter delivery mechanics are unchanged; queued/draft/delivered
semantics are preserved, only the failure-mode default direction changes.

Adds `test-send-message-verdict.sh`: 3 real tmux-pane fixtures (delivered /
unconfirmed-glyphless / draft) locking the fail-loud verdict logic, and
upgrades `test-send-message-socket.sh` fixtures from a glyphless bash prompt
to `❯`-prompt REPLs so the patched binary can read delivery positively.

Reproduce-first evidence (baseline vs patched, glyphless-pane fixture):
  baseline: rc=0, stdout "✓ delivered to =noglyph" (silent false positive)
  patched:  rc=2, stderr "could not confirm submission ... may be UNDELIVERED"

Test results after port:
  test-send-message-verdict.sh -> PASS=3 FAIL=0
  test-send-message-socket.sh  -> ok

Firewall: grep -niE 'jason|woltje|jarvis' on changed files = 0 hits;
tools/quality/scripts/verify-sanitized.sh -> sanitization gate passed.
shellcheck: 1 pre-existing SC2034 finding (unused loop var `attempt`,
present identically in baseline) carried through; 0 new findings.

Part of #891
This commit is contained in:
mosaic-coder
2026-07-25 16:08:38 -05:00
parent 2698ddb7b5
commit 0f0fa7f1ea
3 changed files with 104 additions and 12 deletions

View File

@@ -31,7 +31,11 @@
# EXIT CODES
# 0 delivered (submitted) or queued (agent busy; will process when free)
# 1 tmux target not found
# 2 message still appears to be an unsubmitted draft after retries
# 2 submission NOT confirmed — either still an unsubmitted draft, or the REPL
# input prompt could not be located to confirm the message actually landed.
# Delivery is NEVER inferred from absence of evidence: if we cannot positively
# see the input box clear of the message (or the queued banner), we fail loud
# so the sender learns immediately instead of a silent worker->lead stall.
# 3 usage error
set -uo pipefail
@@ -93,8 +97,14 @@ printf '%s' "$MSG" | "${tmux_cmd[@]}" load-buffer -b "$BUF" -
# would otherwise accumulate forever.
sleep 0.5
# 2) Submit, then verify; flush with another Enter if it is still a draft.
status="sent"
# 2) Submit, then POSITIVELY confirm submission; flush with another Enter if it is
# still a draft. Success requires positive evidence — the queued banner, OR the
# REPL input box located AND clear of our message tail. The historical bug was
# treating ABSENCE of a draft as delivery: if the prompt glyph was never matched
# (wrong pane / prompt-glyph drift), an unsubmitted message read as "delivered"
# and worker->lead relays stalled silently. We now default to UNCONFIRMED and only
# upgrade to delivered on positive evidence; anything we cannot confirm fails loud.
status="unconfirmed"
for attempt in $(seq 1 $((RETRIES + 1))); do
"${tmux_cmd[@]}" send-keys -t "$EFFECTIVE_TARGET" Enter
sleep 1.2
@@ -103,20 +113,28 @@ for attempt in $(seq 1 $((RETRIES + 1))); do
if printf '%s' "$pane" | grep -qF "$QUEUED_RE"; then
status="queued"; break
fi
# Draft heuristic: the prompt glyph line still carries our message tail.
# (Submitted messages scroll up into history; a draft stays on the line.)
# Locate the REPL input box (prompt glyph). If we cannot see it, we have NO
# evidence of submission state — stay UNCONFIRMED and retry; never infer delivery.
promptline=$(printf '%s' "$pane" | grep -E '|^>|│ >' | tail -1)
if [ -z "$promptline" ]; then
status="unconfirmed"; continue
fi
# Input box located AND still carrying our tail => unsubmitted draft. Flush + retry.
# (Submitted messages scroll up into history; a draft stays on the line.)
if [ -n "$snippet" ] && printf '%s' "$promptline" | grep -qF "$snippet"; then
status="draft"; continue
fi
# Input box located AND clear of our tail => positively submitted. This is the
# only path to success besides the queued banner.
status="delivered"; break
done
[ "$VERBOSE" = 1 ] && { echo "--- pane tail ($TARGET) ---"; printf '%s\n' "$pane" | tail -4; echo "---"; }
case "$status" in
delivered) echo "✓ delivered to $TARGET"; exit 0 ;;
queued) echo "✓ queued to $TARGET (agent busy — will process when it returns to prompt)"; exit 0 ;;
draft) echo "✗ still an unsubmitted draft on $TARGET after $RETRIES flush attempts" >&2; exit 2 ;;
*) echo "✓ sent to $TARGET (submission state indeterminate; verify with -v)"; exit 0 ;;
delivered) echo "✓ delivered to $TARGET"; exit 0 ;;
queued) echo "✓ queued to $TARGET (agent busy — will process when it returns to prompt)"; exit 0 ;;
draft) echo "✗ still an unsubmitted draft on $TARGET after $RETRIES flush attempts" >&2; exit 2 ;;
unconfirmed) echo "✗ could not confirm submission on $TARGET: REPL input prompt not locatable after $((RETRIES + 1)) attempts — message may be UNDELIVERED (check target/pane, retry, or escalate)" >&2; exit 2 ;;
*) echo "✗ could not confirm submission on $TARGET (unexpected state '$status')" >&2; exit 2 ;;
esac

View File

@@ -29,8 +29,8 @@ capture_default() {
require_tmux
tmux -L "$SOCKET" new-session -d -s "$TARGET" -c "$TMPDIR" 'bash --noprofile --norc -i'
tmux new-session -d -s "$DEFAULT_TARGET" -c "$TMPDIR" 'bash --noprofile --norc -i'
tmux -L "$SOCKET" new-session -d -s "$TARGET" -c "$TMPDIR" 'PS1=" " exec bash --noprofile --norc -i'
tmux new-session -d -s "$DEFAULT_TARGET" -c "$TMPDIR" 'PS1=" " exec bash --noprofile --norc -i'
"$SEND_MESSAGE" -L "$SOCKET" -t "=$TARGET" -m "named socket hello" >/tmp/send-message-named.out
sleep 0.2
@@ -52,7 +52,7 @@ fi
# load overwrote load, -d deleted underneath — messages swapped between panes).
CONC_N=5
for i in $(seq 1 "$CONC_N"); do
tmux -L "$SOCKET" new-session -d -s "conc-$i" -c "$TMPDIR" 'bash --noprofile --norc -i'
tmux -L "$SOCKET" new-session -d -s "conc-$i" -c "$TMPDIR" 'PS1=" " exec bash --noprofile --norc -i'
done
pids=()
for i in $(seq 1 "$CONC_N"); do

View File

@@ -0,0 +1,74 @@
#!/usr/bin/env bash
# test-send-message-verdict.sh — locks the fail-loud verdict logic of the patched
# send-message.sh against three real tmux-pane fixtures on a throwaway socket:
#
# 1. DELIVERED — a REPL that renders a ` ` input box and submits on Enter
# (text scrolls to history, box clears) => exit 0 "✓ delivered".
# 2. UNCONFIRMED — a pane with NO locatable prompt glyph. This is the exact
# historical FALSE POSITIVE: pre-patch it printed "✓ delivered"
# exit 0; post-patch it MUST fail loud (exit 2, stderr
# "could not confirm submission").
# 3. DRAFT — a ` `-prompt pane that never submits (message stays on the
# input line) => exit 2, stderr "unsubmitted draft".
set -uo pipefail
HERE=$(cd -- "$(dirname -- "$0")" && pwd)
SEND="$HERE/send-message.sh"
SOCKET="verdict-test-$RANDOM-$$"
TMP=$(mktemp -d)
trap 'tmux -L "$SOCKET" kill-server >/dev/null 2>&1 || true; rm -rf "$TMP"' EXIT
PASS=0; FAIL=0
ok() { PASS=$((PASS+1)); printf ' ok %s\n' "$1"; }
no() { FAIL=$((FAIL+1)); printf ' FAIL %s\n %s\n' "$1" "$2"; }
command -v tmux >/dev/null 2>&1 || { echo "tmux required" >&2; exit 1; }
# --- Fixture 1: a submitting REPL with a prompt box (interactive bash, glyph PS1).
# readline strips bracketed-paste markers just like a real agent REPL; Enter
# executes (text -> scrollback), leaving a fresh empty ` ` box.
tmux -L "$SOCKET" new-session -d -s repl -c "$TMP" \
'PS1=" " exec bash --noprofile --norc -i'
sleep 0.3
out=$("$SEND" -L "$SOCKET" -t "=repl" -m "verdict fixture one delivered ok" 2>"$TMP/e1"); rc=$?
if [ "$rc" -eq 0 ] && printf '%s' "$out" | grep -qF "✓ delivered"; then
ok "delivered: -prompt REPL that submits => exit 0 ✓ delivered"
else
no "delivered: -prompt REPL that submits => exit 0 ✓ delivered" "rc=$rc out=[$out] err=[$(cat "$TMP/e1")]"
fi
# --- Fixture 2: NO prompt glyph (default bash PS1). THE regression: pre-patch this
# was a silent false-positive "delivered"; post-patch it must be unconfirmed→exit 2.
tmux -L "$SOCKET" new-session -d -s noglyph -c "$TMP" \
'PS1="sh-noglyph$ " exec bash --noprofile --norc -i'
sleep 0.3
if out=$("$SEND" -L "$SOCKET" -t "=noglyph" -m "verdict fixture two must fail loud" 2>"$TMP/e2"); then
no "unconfirmed: glyphless pane must NOT report success" "expected exit 2, got 0 (out=[$out])"
else
rc=$?
if [ "$rc" -eq 2 ] && grep -qF "could not confirm submission" "$TMP/e2"; then
ok "unconfirmed: glyphless pane => exit 2 + 'could not confirm submission' (false-positive FIXED)"
else
no "unconfirmed: glyphless pane => exit 2 + stderr" "rc=$rc err=[$(cat "$TMP/e2")]"
fi
fi
# --- Fixture 3: a box that never submits (sleep ignores stdin; TTY echo keeps the
# pasted tail sitting on the line) => draft => exit 2.
tmux -L "$SOCKET" new-session -d -s draft -c "$TMP" \
'printf " "; exec sleep infinity'
sleep 0.3
if out=$("$SEND" -L "$SOCKET" -t "=draft" -r 1 -m "verdict fixture three stuck unsubmitted draft" 2>"$TMP/e3"); then
no "draft: unsubmitted message must NOT report success" "expected exit 2, got 0 (out=[$out])"
else
rc=$?
if [ "$rc" -eq 2 ] && grep -qF "unsubmitted draft" "$TMP/e3"; then
ok "draft: stuck -line message => exit 2 + 'unsubmitted draft'"
else
no "draft: stuck -line message => exit 2 + stderr" "rc=$rc err=[$(cat "$TMP/e3")]"
fi
fi
echo "---"
echo "PASS=$PASS FAIL=$FAIL"
[ "$FAIL" -eq 0 ]