agents: Dewey row 138 round 2 review packet (#1632)

Approve: D1-D4 and D5 (N-a) mutants killed on e3f9a400.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
jason.woltjeandClaude Opus 5.5 committed 2026-10-15 20:42:46 -05:00
1 parent c842686d5c
commit de3e93ca62
5 files changed
+144

No files matched your search

@@ -0,0 +1,47 @@
import subprocess, os, signal
# Row 138 r2 review (Dewey): round 1's D1-D4 rerun unchanged, plus D5 for round 2's socket timeout. Each applies once to
# the candidate worktree, runs the owning package suites, and the source is restored after every mutant.
W=os.path.expanduser('~/dewey-scratch/r138/wt')
OUT=os.path.expanduser('~/dewey-scratch/r138/r2/mut')
B='packages/bus/src/broker.mjs'; G='packages/project/src/git.mjs'; S='packages/bus/src/server.mjs'
READ=("const result = await readSpec({ ...context, ...(args.commit === undefined ? {} : { commit: args.commit }) });\n"
" this.#store.transaction(() => this.#agent(s));\n"
" this.#secretCheck(result);\n")
M={
# A commit whose authority receipt can't be recorded is returned as success, not outcome-unknown.
'D1-receipt-failure-reported': (B, ['bus'], " } catch { fail('outcome-unknown'); }", " } catch {}"),
# spec.read doesn't re-check the session after its git await.
'D2-read-recheck-agent': (B, ['bus'], READ, READ.replace(" this.#store.transaction(() => this.#agent(s));\n", '')),
# spec.read returns the file without the secret check.
'D3-read-secret-check': (B, ['bus'], READ, READ.replace(" this.#secretCheck(result);\n", '')),
# gitBytes reads past its byte bound.
'D4-gitbytes-bound': (G, ['project', 'bus'], 'if (size > maxBytes) { failed = true; kill(); }', 'if (false) { failed = true; kill(); }'),
# The async project branch keeps the ordinary idle timeout (round 2's N-a change undone).
'D5-project-socket-timeout': (S, ['bus'], ' socket.setTimeout(120_000);\n', ''),
}
env=dict(os.environ, TMPDIR=os.path.expanduser('~/dewey-scratch/tmp'), DOCKER_HOST='unix:///nonexistent.sock')
origs={p: open(os.path.join(W,p)).read() for p in (B, G, S)}
try:
for name,(path,pkgs,a,b) in M.items():
orig=origs[path]
assert orig.count(a)==1, name
assert a!=b, name
open(os.path.join(W,path),'w').write(orig.replace(a,b))
results=[]
for pkg in pkgs:
p=subprocess.Popen(['bash','-c','node --test --test-reporter=spec tests/*.test.mjs'],cwd=os.path.join(W,'packages',pkg),env=env,
stdout=subprocess.PIPE,stderr=subprocess.STDOUT,text=True,start_new_session=True)
try:
out,_=p.communicate(timeout=600)
except subprocess.TimeoutExpired:
os.killpg(p.pid, signal.SIGKILL); out,_=p.communicate(); out+='\nTIMEOUT\n'
fails=sorted({l.strip()[2:].split(' (')[0] for l in out.splitlines() if l.lstrip().startswith('✖') and 'failing tests' not in l})
open(os.path.join(OUT,f'{name}-{pkg}.log'),'w').write(out)
results.append((pkg,p.returncode,fails))
killed=any(rc!=0 for _,rc,_ in results)
print(name, 'KILLED' if killed else 'SURVIVED', ' '.join(f'{pkg}:rc={rc}' for pkg,rc,_ in results),
'| '+' | '.join(f for _,_,fs in results for f in fs), flush=True)
open(os.path.join(W,path),'w').write(orig)
finally:
for path,orig in origs.items(): open(os.path.join(W,path),'w').write(orig)
print('restored', all(open(os.path.join(W,p)).read()==o for p,o in origs.items()))
@@ -0,0 +1,6 @@
D1-receipt-failure-reported KILLED bus:rc=1 | D1: committed write with a failed receipt refuses outcome-unknown without action.allowed
D2-read-recheck-agent KILLED bus:rc=1 | D2: spec.read refuses without content if its role is released during the read
D3-read-secret-check KILLED bus:rc=1 | D3: spec.read secret check refuses flagged content without content | D3: spec.read secret check refuses flagged subject without content
D4-gitbytes-bound KILLED project:rc=1 bus:rc=0 | D4: spec log refuses a commit subject larger than its 256 KiB read bound
D5-project-socket-timeout KILLED bus:rc=1 | N-a: slow admitted record.write returns its committed result after the idle deadline
restored True
@@ -0,0 +1,16 @@
#!/bin/bash
# Row 138 r2 review: each touched test file run singly on 26a0b0fab + candidate.
set -o pipefail
W=~/dewey-scratch/r138/wt; OUT=~/dewey-scratch/r138/r2/singles
export TMPDIR=~/dewey-scratch/tmp DOCKER_HOST=unix:///nonexistent.sock
rc=0
for f in bus/tests/chief-of-staff bus/tests/comment-migration bus/tests/project-runtime bus/tests/spec-migration \
bus/tests/spec-socket bus/tests/spec bus/tests/thread-migration bus/tests/launch-notice bus/tests/project-await \
business/tests/assistant business/tests/chief-of-staff business/tests/csuite business/tests/record-grants \
harness/tests/bundle harness/tests/spec-tools harness/tests/tools project/tests/read; do
pkg=${f%%/*}; file=${f#*/}.test.mjs; log=$OUT/${f//\//-}.log
(cd $W/packages/$pkg && timeout 600 node --test --test-reporter=spec $file) > $log 2>&1; s=$?
[ $s -ne 0 ] && rc=1
echo "$f exit=$s $(grep -E '^ℹ (pass|fail|skipped)' $log | tr '\n' ' ')"
done
exit $rc
@@ -0,0 +1,23 @@
business-tests-assistant tests=11 pass=11 fail=0 skipped=0
business-tests-chief-of-staff tests=10 pass=10 fail=0 skipped=0
business-tests-csuite tests=6 pass=6 fail=0 skipped=0
business-tests-record-grants tests=1 pass=1 fail=0 skipped=0
bus-tests-chief-of-staff tests=13 pass=13 fail=0 skipped=0
bus-tests-comment-migration tests=5 pass=5 fail=0 skipped=0
bus-tests-launch-notice tests=8 pass=8 fail=0 skipped=0
bus-tests-project-await tests=4 pass=4 fail=0 skipped=0
bus-tests-project-runtime tests=3 pass=3 fail=0 skipped=0
bus-tests-spec tests=4 pass=4 fail=0 skipped=0
bus-tests-spec-migration tests=1 pass=1 fail=0 skipped=0
bus-tests-spec-socket tests=6 pass=6 fail=0 skipped=0
bus-tests-thread-migration tests=3 pass=3 fail=0 skipped=0
harness-tests-bundle tests=21 pass=21 fail=0 skipped=0
harness-tests-spec-tools tests=2 pass=2 fail=0 skipped=0
harness-tests-tools tests=18 pass=18 fail=0 skipped=0
project-tests-read tests=8 pass=8 fail=0 skipped=0
D1-receipt-failure-reported-bus tests=294 pass=293 fail=1 skipped=0
D2-read-recheck-agent-bus tests=294 pass=293 fail=1 skipped=0
D3-read-secret-check-bus tests=294 pass=292 fail=2 skipped=0
D4-gitbytes-bound-bus tests=294 pass=294 fail=0 skipped=0
D4-gitbytes-bound-project tests=30 pass=29 fail=1 skipped=0
D5-project-socket-timeout-bus tests=294 pass=293 fail=1 skipped=0
@@ -0,0 +1,52 @@
**Dewey: approve, round 2, candidate manifest `e3f9a4008c8cd0098f028aa17e9cbde8b15d3b64bc13f3ee306632070a318f70`**
All four round 1 asks are in, and my own D1-D4 mutants, rerun unchanged, are now killed. Each falls to the new test written for it. Sage's three additions from sage-0492 (N-a, the test title, N-f) are in as well. My D5 for the N-a change is killed too.
**What I checked**
- `build.patch` from `agents/rocko/work/queue-138/r4/`, applied with `git apply --index` to a detached worktree of `26a0b0fab`, the same base as round 1. The manifest file hashes to `e3f9a400…`, and `sha256sum -c` checks 43/43. No commit in `26a0b0fab..HEAD` touches a manifest path.
- Round 1 to round 2: I applied the r3 and r4 patches to the same base and diffed the two trees (`fe6ecf98` to `20a0a6c6`). Six files differ, which matches `changes-from-r3.json`:
- packages/bus/src/server.mjs: two lines, the round's only product change;
- packages/bus/tests/project-await.test.mjs: new, D1-D3;
- packages/bus/tests/spec-socket.test.mjs: the N-a test;
- packages/project/tests/read.test.mjs: D4;
- packages/business/tests/chief-of-staff.test.mjs: the title now names `record.write`; the assertion is unchanged;
- packages/bus/README.md: N-f.
The broker, runtime, project sources, schema, migrations, grants, harness and cli are byte-identical to round 1, so round 1's checks on Sage's five focus points still hold.
- **N-a reaches only the async project branch.** `socket.setTimeout(120_000)` sits inside the `spec.read` / `record.write` / cited `decision.raise` branch, before `broker.request`. The task and web branches and the sync path are unchanged.
- **N-f.** The bus README and the r4 MIGRATION.md now say `26a0b0fab`. `schema.sql` is the same at `56639c99b` and `26a0b0fab`, so the fixture and its sha are unaffected.
- **The asks' tests:**
1. D1 makes the injected `writeRecord` release the role, then resolve. It asserts `outcome-unknown`, no returned value and no `record.write` receipt.
2. D2 releases the role during `readSpec` and asserts `not-holder`, with no content in the result or in any event.
3. D3 flags the content in one case and a log subject in the other, through an injected `secretCheck`. Both refuse `secret-refused` and leave no marker in events.
4. D4 commits with a 256 KiB + 1 subject through `git commit -F` and asserts `project-git`.
- **Singles.** Each touched test file, run singly on the candidate, 2026-10-16T01:40:25Z to 01:40:36Z, TMPDIR in my scratch: 17 files, 124 tests, 0 fail, 0 skipped (`summary.txt`, `singles.sh`).
**Mutants**
`mutants.py` and `mutants.txt` are here. D1-D4 are round 1's edits, byte for byte. D5 removes the new `socket.setTimeout(120_000);` line. Each mutant runs the whole owning package suite and the source is restored afterwards. The run took 01:40:47Z to 01:41:49Z, in the slot I took at 01:40:40Z and cleared at 01:41:55Z. Afterwards the worktree checks 43/43.
| Mutant | Suite | Round 1 | Round 2 | Failing tests |
|---|---|---|---|---|
| D1-receipt-failure-reported | bus | survived | killed (293/294) | the D1 test only |
| D2-read-recheck-agent | bus | survived | killed (293/294) | the D2 test only |
| D3-read-secret-check | bus | survived | killed (292/294) | both D3 tests only |
| D4-gitbytes-bound | project, bus | survived | killed (project 29/30; bus 294/294) | the D4 test only |
| D5-project-socket-timeout | bus | (new) | killed (293/294) | the N-a test only |
Across these runs, every bus test other than the target passed, and the D4 run passed the whole bus suite at 294/294. That matches Rocko's bus 294 and project 30.
**Non-blocking notes (follow-ups)**
1. **The 120 s window covers one git process, not a whole write.** `GIT_TIMEOUT_MS` (100 s) applies to each git process. `writeRecord` runs seven or more of them in sequence: `inspectGit`, `requireClean`, `checkProject`, `rev-parse`, `add`, `commit` and `rev-parse`. A write where several calls are each slow can still pass 120 s and get `request-timeout`. That's still not a false success, so it doesn't block. The server comment "Local Git may take 100 seconds" reads as if one limit bounds the verb. A follow-up could give `writeRecord` an overall deadline below the socket's, or derive both from one constant. Today `120_000` is a literal, unconnected to `GIT_TIMEOUT_MS`.
2. **The longer timeout is set before admission.** It's set before the broker checks the session, unlike `requestWeb`'s `onAdmit`. It's harmless here: a refused project call settles at once and ends the socket, so the 120 s only matters while git runs.
3. N-c, N-d, N-e and the `#dispatch` backstop stay with Sage's follow-ups, as sage-0492 says.
**Reproduce**
1. `git worktree add --detach <dir> 26a0b0fab` and link `node_modules`.
2. `git apply --index agents/rocko/work/queue-138/r4/build.patch`.
3. `sha256sum -c` against the candidate manifest.
`singles.sh` and `mutants.py` hold the paths I used; change `W` and `OUT` for your own.