jason.woltjeandClaude Opus 5.5
1abebe5321
Record for #1516 comment 26725: 34/34 cases reproduce, and a wrapped Claude command hook fails closed on crash, missing path, noexec and hang. Co-Authored-By: Claude Opus 5.5 <[email protected]>
1.8 KiB
Row 34, S0 probe matrix, round 1: changes (Darkwing, reviewer under decision 54)
Candidate 771fc3d2700bbadacc76b31534674ca342eb99ff. The full record is agents/darkwing/work/slice1-s0-review/review-r1.md (local commit on refactor, not pushed).
The matrix is accurate. Every cell matches evidence/<case>/ (exit, target, gate calls, tools offered, and the is_error and content the model saw). On my machine all 34 cases reproduce with identical results: 32/32 short cases and 2/2 of the 600 s cases. The scripted mock in a loopback-only namespace is a sound method for this matrix.
Required:
-
a. Rely-on lines 2, 3 and 4 tell S6 to fall back on the tool limit for Claude Code command hooks that crash, sit at a missing path or hang. A wrapped hook fails closed in all three cases. I ran six extra cases on the cc-2 setup:
<gate> crash || exit 2blocks.- A missing gate and a non-executable gate, each with
|| exit 2, block. timeout 2 <gate> hang || exit 2, with hooktimeout10, blocks in 2.4 s.- Allow (
true || exit 2) and the JSON deny still behave as before.
Please add wrapper cases and amend the three lines. The record has suggested text and the caveats: the inner timeout must sit below the hook timeout; use
timeout -k;--barestill disables hooks. -
b.
evidence/cc-1d-block-barehas no case inprobe.shand no row in the table (35 directories for 34 cases). Remove it or add the case.
Minor, can go in the same round:
- c. The cc-5c row omits
--allowedTools Read,Write. - d. Nothing committed writes the
runs.cmp,runs-2.cmpandsaw-*.txtfiles thatcollect.shreads. - e.
SDK_ENTRYcan't be overridden from the environment. - f. "exit 0 in every run that reached a model" doesn't hold for pi-2c (3) and pi-4b (124).