mosaicstack/stack
25
0
Fork 1
Code Issues 344 Pull Requests 13 Actions Packages Projects Releases 15 Wiki Activity

hook: lint agent-authored shell for $? read after a pipeline — 4 instances, 3 agents, 1 day, all near-misses #955

New Issue
Open
opened 2026-07-30 15:58:33 +00:00 by Ghost · 0 comments
Ghost commented 2026-07-30 15:58:33 +00:00
Copy Link
Copy Source

The defect

$? after a pipeline is the last element's exit status. So every one of these reports the status of head, tail, grep, or jq — never the command under test:

some-tool ... | tail -5 ;   echo "rc=$?"     # rc of tail
if cmd | grep -q x ; then ... fi            # status of grep

Why this warrants a mechanical check rather than a rule

Four instances in a single day, across three different agents. Two of them nearly became filed defects against correctly-behaving tools; one reached a published claim that had to be retracted.

The mechanism is what makes discipline insufficient:

The habit that makes a failure readable is the habit that makes its exit status wrong.

The pipe gets added because something looks wrong — which is precisely the moment the exit status matters most. The idiom and the hazard have the same trigger.

Every instance was caught by the same accident, and it does not generalise

In all four cases the rescue was identical: rc=0 contradicted visible error text on screen. That is luck of contradiction, not method — and it fails exactly where it is most needed:

A silent command that fails with no output yields rc=0 and nothing to contradict it.

None of the four would have been caught in that case. The observed catch rate is an artifact of the commands happening to be chatty.

Proposed check

A lint over agent-authored shell flagging a pipeline into head/tail/grep/jq/sed/awk followed by a read of $? — whether as echo $?, rc=$?, or [ $? -eq 0 ].

All four instances match that pattern syntactically. This is a PreToolUse-shaped check: cheap, no execution required, no false-negative risk from runtime state.

Complementary standing rule (for the cases a lint cannot see)

Never report an exit status observed through a pipe. If an rc claim is going into a durable record — an issue body, a commit message, a merge note — re-measure it unpiped first.

Suggested remediations for flagged code

instead of use
cmd | tail -5; echo $? cmd > /tmp/out 2>&1; rc=$?; tail -5 /tmp/out
cmd | grep -q x set -o pipefail, or capture then test
any rc claim in a report re-run unpiped before asserting it

Acceptance

  • The lint fires on all six pipe-target commands above, in all three $?-read forms.
  • A negative control: a fixture with cmd; rc=$?; echo "$out" \| tail (rc read before the pipe) must not fire — otherwise the check is a blanket ban on pipes and will be disabled.
  • Failure output names the pipeline element whose status is actually being read.

Filed as a missing mechanical check, not as three agents needing to be more careful. Operator-agnostic; contains no operator-specific context.

## The defect `$?` after a pipeline is the **last element's** exit status. So every one of these reports the status of `head`, `tail`, `grep`, or `jq` — never the command under test: ```sh some-tool ... | tail -5 ; echo "rc=$?" # rc of tail if cmd | grep -q x ; then ... fi # status of grep ``` ## Why this warrants a mechanical check rather than a rule **Four instances in a single day, across three different agents.** Two of them nearly became *filed defects against correctly-behaving tools*; one reached a **published claim that had to be retracted**. The mechanism is what makes discipline insufficient: > **The habit that makes a failure readable is the habit that makes its exit status wrong.** The pipe gets added *because* something looks wrong — which is precisely the moment the exit status matters most. The idiom and the hazard have the same trigger. ### Every instance was caught by the same accident, and it does not generalise In all four cases the rescue was identical: **`rc=0` contradicted visible error text on screen.** That is luck of contradiction, not method — and it fails exactly where it is most needed: > **A silent command that fails with no output yields `rc=0` and nothing to contradict it.** None of the four would have been caught in that case. The observed catch rate is an artifact of the commands happening to be chatty. ## Proposed check A lint over agent-authored shell flagging a pipeline into `head`/`tail`/`grep`/`jq`/`sed`/`awk` followed by a read of `$?` — whether as `echo $?`, `rc=$?`, or `[ $? -eq 0 ]`. All four instances match that pattern syntactically. This is a **PreToolUse-shaped** check: cheap, no execution required, no false-negative risk from runtime state. ### Complementary standing rule (for the cases a lint cannot see) **Never report an exit status observed through a pipe.** If an `rc` claim is going into a durable record — an issue body, a commit message, a merge note — re-measure it unpiped first. ## Suggested remediations for flagged code | instead of | use | |---|---| | `cmd \| tail -5; echo $?` | `cmd > /tmp/out 2>&1; rc=$?; tail -5 /tmp/out` | | `cmd \| grep -q x` | `set -o pipefail`, or capture then test | | any rc claim in a report | re-run unpiped before asserting it | ## Acceptance - The lint fires on all six pipe-target commands above, in all three `$?`-read forms. - A **negative control**: a fixture with `cmd; rc=$?; echo "$out" \| tail` (rc read *before* the pipe) must **not** fire — otherwise the check is a blanket ban on pipes and will be disabled. - Failure output names the pipeline element whose status is actually being read. *Filed as a missing mechanical check, not as three agents needing to be more careful. Operator-agnostic; contains no operator-specific context.*
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Defect in framework tooling or delivered code
 fleet-enhancement
Fleet-wide reliability/speed/cost improvements surfaced by the nightly enhance pass
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
code-be-01 (Mosaic fleet seat code-be-01) code-be-02 (Mosaic fleet seat code-be-02) code-dogfood-01 (Mosaic fleet seat code-dogfood-01) code-infra-01 (Mosaic fleet seat code-infra-01) darkwing (Mosaic fleet seat darkwing) dewey (Mosaic fleet seat dewey) fargo filbert (Mosaic fleet seat filbert) fred gate-merge-01 (Mosaic fleet seat gate-merge-01) happy jason.woltje (Jason Woltje) marcie merge-gate ops-01 (Mosaic fleet seat ops-01) ops-02 (Mosaic fleet seat ops-02) ops-03 (Mosaic fleet seat ops-03) ops-ci-01 (Mosaic fleet seat ops-ci-01) ops-deploy-01 (Mosaic fleet seat ops-deploy-01) orch-01 (Mosaic fleet seat orch-01) pepper resume rev-code-01 rev-code-02 rev-security-01 rev-security-02 rev-security-03 (Mosaic fleet seat rev-security-03) rocko (Mosaic fleet seat rocko) sanity scooby (Scooby) scrappy shaggy tiny topher (Mosaic fleet seat topher) velma veronica (Mosaic fleet seat veronica) vision woodpecker
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mosaicstack/stack#955