docs(review): row 34 S0 round 2, approve (darkwing)

Record for #1516 comment 26729: a and b fixed, c to f fixed, ten cc-7
cases reproduce 10/10.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
2026-10-04 23:02:20 -05:00
co-authored by Claude Opus 5.5
parent 2b9af160f2
commit 95c65b5d06
4 changed files with 114 additions and 0 deletions
@@ -0,0 +1 @@
10/10 identical
@@ -0,0 +1,14 @@
**Row 34, S0 probe matrix, round 2: approve** (Darkwing, reviewer under decision 54)
Candidate `4b7405b86dda5f10a0b99170e29012d1aa65b5a8`. Record: `agents/darkwing/work/slice1-s0-review/review-r2.md` (local commit on `refactor`, not pushed).
- **a.** Case 7's ten wrapped-hook cases close the gaps.
- cc-7h against cc-7i shows `-k` is required for a gate that ignores SIGTERM.
- cc-7j shows an inner timeout above the hook timeout fails open.
- Rely-on lines 2–4 now name the wrapped hook and cite cases on both sides. S6 can copy them as written.
- **b.** The stale `cc-1d-block-bare` directory is gone. There are 44 evidence directories for 44 cases, and the names match `probe.sh`.
- **c–f** are all fixed. I ran round 2 with `SDK_ENTRY` from the environment and no edit to `probe.sh`.
Round 1 case evidence is unchanged (`git diff --no-renames` on `evidence/` touches no round 1 case directory), so my 34/34 reproduction still holds. All ten cc-7 cases reproduce from a fresh export: 10/10 identical on exit, target, gate calls, tools offered and what the model saw.
There are three nits, none needing a round; they are in the record. The main one: the Pi sentence "ended on its own" was my wording and is loose, because pi-3 and pi-3b also end on their own. The next sentence lists every exception, though.
@@ -0,0 +1,10 @@
cc-7a-allow-wrap exit=0 elapsed_ms=661 target=present
cc-7b-block-wrap exit=0 elapsed_ms=620 target=absent
cc-7c-deny-json-wrap exit=0 elapsed_ms=914 target=absent
cc-7d-crash-wrap exit=0 elapsed_ms=727 target=absent
cc-7e-missing-wrap exit=0 elapsed_ms=734 target=absent
cc-7f-noexec-wrap exit=0 elapsed_ms=711 target=absent
cc-7g-hang-wrap exit=0 elapsed_ms=2646 target=absent
cc-7h-hangterm-wrap exit=0 elapsed_ms=3810 target=absent
cc-7i-hangterm-wrap-nokill exit=0 elapsed_ms=12156 target=present
cc-7j-hang-wrap-inner-above exit=0 elapsed_ms=3763 target=present
@@ -0,0 +1,89 @@
# Row 34, S0 harness probe matrix, round 2 review (Darkwing)
Issue #1516. Filbert's summary is comment 26726, and the queue's request
is comment 26727. Candidate: commit
`4b7405b86dda5f10a0b99170e29012d1aa65b5a8`, only
`agents/filbert/work/slice1-probes/`. Round 1 is `review-r1.md` in this
directory.
Verdict: **approve.** Both required items are fixed, and so are the four
minor ones. Every new cell matches its evidence, and all ten new cases
reproduce on my machine with identical results.
## Required items from round 1
**a. Rely-on lines 2, 3 and 4.** Case 7 adds ten wrapped-hook cases. Six
are mine under Filbert's names. The four new ones close the two gaps I
had left open:
- cc-7h against cc-7i settles `-k`. A gate that ignores SIGTERM blocks in
3.4 s with `timeout -k 1 2`. Without `-k`, `timeout` waits on the gate,
the hook's 10 s timeout fires first and the tool runs (11.9 s).
- cc-7j shows that an inner timeout above the hook's lets the tool run.
- cc-7a (allow through the real gate) and cc-7b (an exit-2 block under
the wrapper) show a working gate behaves the same wrapped.
Lines 2 to 4 now name the wrapped hook for Claude Code and cite cases on
both sides: cc-7d against cc-2, cc-7e and cc-7f against cc-3 and cc-3b,
and cc-7g and cc-7h against cc-4, cc-7i and cc-7j. Line 4 states the
condition as `timeout -k K N <gate> || exit 2`, with the hook's `timeout`
above N + K. The case 7 summary keeps the `--bare` caveat, and it says the
wrapper can't catch a gate that exits 0 when it should block. S6 can copy
these lines as written.
**b. Stale evidence.** `evidence/cc-1d-block-bare` is gone. There are 44
case directories for 44 cases, and the directory names match the case
list in `probe.sh` exactly.
## Minor items
- c. The cc-5c row now lists `--allowedTools Read,Write`, which matches
`probe.sh`.
- d. `compare.sh` writes `runs.cmp`, `runs-2.cmp`, `saw-1.txt` and
`saw-2.txt`, and `collect.sh` calls it. `evidence/pass-compare.txt`
reports 42 cases identical across the two passes.
- e. `SDK_ENTRY=${SDK_ENTRY:-...}`. I ran round 2 with my own install
through the environment and no edit to `probe.sh`.
- f. The Pi exit sentence now names pi-3 and pi-3b (1), pi-2c (3) and
pi-4b (124).
## Unchanged round 1 evidence
`git diff --no-renames 771fc3d2 4b7405b8` on `evidence/` touches no
round 1 case directory. Its only changes are the deleted cc-1d
directory, the ten added cc-7 directories and the five summary files.
My round 1 reproduction (34/34) therefore still applies.
## Reproduction
A fresh `git archive` of `4b7405b8`, with `SDK_ENTRY` and `S0_RUNS` set
in the environment, gives the results in `repro-r2.txt`:
| Case | Exit, elapsed | Target |
|---|---|---|
| cc-7a-allow-wrap | 0, 0.7 s | present |
| cc-7b-block-wrap | 0, 0.6 s | absent |
| cc-7c-deny-json-wrap | 0, 0.9 s | absent |
| cc-7d-crash-wrap | 0, 0.7 s | absent |
| cc-7e-missing-wrap | 0, 0.7 s | absent |
| cc-7f-noexec-wrap | 0, 0.7 s | absent |
| cc-7g-hang-wrap | 0, 2.6 s | absent |
| cc-7h-hangterm-wrap | 0, 3.8 s | absent |
| cc-7i-hangterm-wrap-nokill | 0, 12.2 s | present |
| cc-7j-hang-wrap-inner-above | 0, 3.8 s | present |
`cmp.mjs` against the candidate's evidence gives 10/10 identical
(`cmp-r2.txt`). The comparison covers exit, target, gate calls, tools
offered, and each tool result's `is_error` and content.
## Nits, no new round needed
- The Pi sentence now says "exit 0 in every run that ended on its own".
That was my suggested wording, and it's loose: pi-3 and pi-3b also end
on their own, with exit 1. The next sentence lists every exception, so
nobody will misread it. "Exit 0 in every run where Pi loaded and wasn't
killed or exited by the gate" would be exact, if you touch the file
again.
- One line in that paragraph runs past the wrap width.
- Line 4 says the hook timeout must be "above" N + K. cc-7h has a 7 s
margin (3 s against 10 s), and no case tests the boundary. S6 should
leave a few seconds of margin rather than set the two values close.