agents: Dewey row 138 round 1 review packet (#1632)

Request changes: tests killing four surviving mutants (D1-D4).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
2026-10-15 20:28:46 -05:00
co-authored by Claude Opus 5.5
parent 79ad018786
commit ff86a4473e
5 changed files with 138 additions and 0 deletions
@@ -0,0 +1,45 @@
import subprocess, os, signal
# Row 138 r1 review (Dewey): one mutant per new check that Rocko's 54 don't cover. Each applies once to
# the candidate worktree, runs the owning package suites, and the source is restored after every mutant.
W=os.path.expanduser('~/dewey-scratch/r138/wt')
OUT=os.path.expanduser('~/dewey-scratch/r138/mut')
B='packages/bus/src/broker.mjs'; G='packages/project/src/git.mjs'
READ=("const result = await readSpec({ ...context, ...(args.commit === undefined ? {} : { commit: args.commit }) });\n"
" this.#store.transaction(() => this.#agent(s));\n"
" this.#secretCheck(result);\n")
M={
# A commit whose authority receipt can't be recorded is returned as success, not outcome-unknown.
'D1-receipt-failure-reported': (B, ['bus'], " } catch { fail('outcome-unknown'); }", " } catch {}"),
# spec.read doesn't re-check the session after its git await.
'D2-read-recheck-agent': (B, ['bus'], READ, READ.replace(" this.#store.transaction(() => this.#agent(s));\n", '')),
# spec.read returns the file without the secret check.
'D3-read-secret-check': (B, ['bus'], READ, READ.replace(" this.#secretCheck(result);\n", '')),
# gitBytes reads past its byte bound.
'D4-gitbytes-bound': (G, ['project', 'bus'], 'if (size > maxBytes) { failed = true; kill(); }', 'if (false) { failed = true; kill(); }'),
}
env=dict(os.environ, TMPDIR=os.path.expanduser('~/dewey-scratch/tmp'), DOCKER_HOST='unix:///nonexistent.sock')
origs={p: open(os.path.join(W,p)).read() for p in (B, G)}
try:
for name,(path,pkgs,a,b) in M.items():
orig=origs[path]
assert orig.count(a)==1, name
assert a!=b, name
open(os.path.join(W,path),'w').write(orig.replace(a,b))
results=[]
for pkg in pkgs:
p=subprocess.Popen(['bash','-c','node --test --test-reporter=spec tests/*.test.mjs'],cwd=os.path.join(W,'packages',pkg),env=env,
stdout=subprocess.PIPE,stderr=subprocess.STDOUT,text=True,start_new_session=True)
try:
out,_=p.communicate(timeout=600)
except subprocess.TimeoutExpired:
os.killpg(p.pid, signal.SIGKILL); out,_=p.communicate(); out+='\nTIMEOUT\n'
fails=sorted({l.strip()[2:].split(' (')[0] for l in out.splitlines() if l.lstrip().startswith('✖') and 'failing tests' not in l})
open(os.path.join(OUT,f'{name}-{pkg}.log'),'w').write(out)
results.append((pkg,p.returncode,fails))
killed=any(rc!=0 for _,rc,_ in results)
print(name, 'KILLED' if killed else 'SURVIVED', ' '.join(f'{pkg}:rc={rc}' for pkg,rc,_ in results),
'| '+' | '.join(f for _,_,fs in results for f in fs), flush=True)
open(os.path.join(W,path),'w').write(orig)
finally:
for path,orig in origs.items(): open(os.path.join(W,path),'w').write(orig)
print('restored', all(open(os.path.join(W,p)).read()==o for p,o in origs.items()))
@@ -0,0 +1,5 @@
D1-receipt-failure-reported SURVIVED bus:rc=0 |
D2-read-recheck-agent SURVIVED bus:rc=0 |
D3-read-secret-check SURVIVED bus:rc=0 |
D4-gitbytes-bound SURVIVED project:rc=0 bus:rc=0 |
restored True
+16
View File
@@ -0,0 +1,16 @@
#!/bin/bash
# Row 138 r1 review: each touched test file run singly on 26a0b0fab + candidate.
set -o pipefail
W=~/dewey-scratch/r138/wt; OUT=~/dewey-scratch/r138/singles
export TMPDIR=~/dewey-scratch/tmp DOCKER_HOST=unix:///nonexistent.sock
rc=0
for f in bus/tests/chief-of-staff bus/tests/comment-migration bus/tests/project-runtime bus/tests/spec-migration \
bus/tests/spec-socket bus/tests/spec bus/tests/thread-migration bus/tests/launch-notice \
business/tests/assistant business/tests/chief-of-staff business/tests/csuite business/tests/record-grants \
harness/tests/bundle harness/tests/spec-tools harness/tests/tools project/tests/read; do
pkg=${f%%/*}; file=${f#*/}.test.mjs; log=$OUT/${f//\//-}.log
(cd $W/packages/$pkg && timeout 600 node --test --test-reporter=spec $file) > $log 2>&1; s=$?
[ $s -ne 0 ] && rc=1
echo "$f exit=$s $(grep -E '^ℹ (pass|fail|skipped)' $log | tr '\n' ' ')"
done
exit $rc
@@ -0,0 +1,21 @@
business-tests-assistant tests=11 pass=11 fail=0 skipped=0
business-tests-chief-of-staff tests=10 pass=10 fail=0 skipped=0
business-tests-csuite tests=6 pass=6 fail=0 skipped=0
business-tests-record-grants tests=1 pass=1 fail=0 skipped=0
bus-tests-chief-of-staff tests=13 pass=13 fail=0 skipped=0
bus-tests-comment-migration tests=5 pass=5 fail=0 skipped=0
bus-tests-launch-notice tests=8 pass=8 fail=0 skipped=0
bus-tests-project-runtime tests=3 pass=3 fail=0 skipped=0
bus-tests-spec tests=4 pass=4 fail=0 skipped=0
bus-tests-spec-migration tests=1 pass=1 fail=0 skipped=0
bus-tests-spec-socket tests=5 pass=5 fail=0 skipped=0
bus-tests-thread-migration tests=3 pass=3 fail=0 skipped=0
harness-tests-bundle tests=21 pass=21 fail=0 skipped=0
harness-tests-spec-tools tests=2 pass=2 fail=0 skipped=0
harness-tests-tools tests=18 pass=18 fail=0 skipped=0
project-tests-read tests=7 pass=7 fail=0 skipped=0
D1-receipt-failure-reported-bus tests=289 pass=289 fail=0 skipped=0
D2-read-recheck-agent-bus tests=289 pass=289 fail=0 skipped=0
D3-read-secret-check-bus tests=289 pass=289 fail=0 skipped=0
D4-gitbytes-bound-bus tests=289 pass=289 fail=0 skipped=0
D4-gitbytes-bound-project tests=29 pass=29 fail=0 skipped=0
@@ -0,0 +1,51 @@
**Dewey: request changes, round 1, candidate manifest `75e07b65b7d1427314b50f81b2b7438aaa708fffd8db01779df18900e0a747d1`**
The design matches the r3 brief, DESIGN-r3 and the rulings, every brief-required test exists and passes, and Rocko's 54 mutants hold up on the logs I checked. Four new checks have no test that fails without them. I mutated each one and ran the whole owning suite: all four survived. Each needs a test that kills it.
**What I checked**
- `build.patch` from `agents/rocko/work/queue-138/r3/`, applied with `git apply --index` to a detached worktree of `26a0b0fab` (row 152's landing). The manifest file hashes to `75e07b65…`, and `sha256sum -c` checks 42/42. No commit in `26a0b0fab..HEAD` touches a manifest path.
- Sage's five focus points (sage-0491):
1. **Grants.** `record.write` is within-role for the eleven v2 roles (sage-0480/0482), and record-grants.test pins the list. `spec.read` is open to any held role in its business, and a human caller refuses `agent-required`. Nothing changes who may resolve a decision.
2. **Schema and migration.** `3e-3f.sql` adds two nullable columns and nothing else. spec-migration.test builds 3e from the pinned fixture (sha `21e6c091…`), seeds every table, keeps an untouched copy, and compares every old row and the sequence counters after migrating. Normal startup refuses 3e. The future-version chain now runs 3f→3g. I read MIGRATION.md as text only; I ran nothing live.
3. **Runtime adapter.** The broker takes `project` in its constructor and defaults it to `null`. With no adapter, `spec.read`, `record.write` and a cited raise each refuse `project-not-configured`. In packages/bus/src, runtime.mjs is the only file that imports packages/project (cli's project.mjs is row 147's), and project-runtime.test pins the broker's exact two imports and the absence of a dynamic `import(`.
4. **Snapshots.** Every changed exact list in assistant.test, csuite.test and bundle.test adds `record.write` or the `read_spec`/`write_record` tools, or bumps 3e to 3f. None is loosened into a subset or pattern check.
5. **Row 152.** `waiting.launchable` and the unlaunchable notice text are intact (broker.mjs:1992, README:387), and launch-notice.test passes 8/8.
- Brief-required tests, each present: a raise records the current spec commit; a stale citation refuses; a settled decision stays settled after a new spec commit; the read tool returns the file and log; an agent's write to `spec/` refuses (packages/project/tests/project.test.mjs:88 and the bus record.write path); the record writer commits with the agent author (`coder`/`coder`).
- Each touched test file run singly on the candidate, 2026-10-16T01:21:55Z to 01:22:07Z, TMPDIR in my scratch: 16 files, 118 tests, 0 fail, 0 skipped (`summary.txt`, `singles.sh`).
- Rocko's mutants: 54/54 killed. I opened the logs for 15, 19, 20, 25, 28, 29, 32, 39, 52 and 54; each fails the test BUILD.md names.
**Mutants**
`mutants.py` and `mutants.txt` here. Each mutant edits one line in the candidate worktree, runs the owning package's whole node suite, and restores the source. Run 01:24:34Z to 01:25:25Z in the slot Sage granted in sage-0491. Afterwards the worktree checks 42/42 again.
| Mutant | Fault | Suite | Result |
|---|---|---|---|
| D1-receipt-failure-reported | in `record.write`, `catch { fail('outcome-unknown'); }` around `#consumeAuthority` becomes `catch {}` | bus 289/289 | survived |
| D2-read-recheck-agent | `spec.read` drops the `#agent(s)` re-check after `await readSpec` | bus 289/289 | survived |
| D3-read-secret-check | `spec.read` drops `#secretCheck(result)` | bus 289/289 | survived |
| D4-gitbytes-bound | git.mjs `if (size > maxBytes) { failed = true; kill(); }` becomes `if (false)` | project 29/29, bus 289/289 | survived |
**Asks (blocking)**
1. **Kill D1.** This is the one that matters most. When `writeRecord` has committed and the receipt can't be recorded, the caller has to hear `outcome-unknown`. With D1, a commit with no `action.allowed` receipt returns as a plain success. The adapter is injected (`new Broker({ project })`), so a fake `writeRecord` can make the consume fail while it awaits. One way is to end the caller's role session before resolving. Another is to let a second write citing the same decision consume it first, which is N-b below. Assert `outcome-unknown` and that no `action.allowed` for that write exists.
2. **Kill D2.** Use a fake `readSpec` that ends the caller's session (or revokes the role) before resolving. Assert `spec.read` refuses and returns no content.
3. **Kill D3.** Use a fake `readSpec` whose `content` (or a log subject) holds a value the broker's secret check refuses. Assert the refusal code and that no content is returned. A real committed spec would work too if the test's `secretCheck` flags a marker string.
4. **Kill D4.** `blob()` pre-checks size with `cat-file -s`, so the bound is redundant for file content. But it is the only bound on the `git log` read in read.mjs:50 (256 KiB). A commit to `spec/north-star.md` whose subject is over 256 KiB (write it with `git commit -F <file>`, since one argv string caps at 128 KiB) should make `readProjectFile` refuse `project-git`. A direct `gitBytes` test with a small `maxBytes` would also do.
**Non-blocking notes**
- **N-a.** The server sends project verbs down the async path but keeps the 5 s idle timeout, while git runs under the 100 s `GIT_TIMEOUT_MS`. The task and web branches extend the socket timeout; the project branch doesn't. A slow `record.write` can commit and record its receipt while the client gets `request-timeout`, not `outcome-unknown`. That's honest about uncertainty, but the agent can't tell it from a refusal. A follow-up can extend the timeout once admitted, like `requestWeb`'s `onAdmit`.
- **N-b.** Two concurrent gated `record.write`s citing one decision, each to a different project (so `project-busy` doesn't serialize them), both pass `#checkAuthority` before either consumes. Both commit; the second ends `outcome-unknown`. It's reported, not silent, and it needs narrowed limits to arise. Ask 1's test can use this scenario.
- **N-c.** `decisions.spec_path` and `spec_commit` have no CHECK constraint. The broker validates both before insert, so only a direct store write could put a bad value there.
- **N-d.** The role `.md` contracts don't name `read_spec` or `write_record`. Not an owned file; a follow-up.
- **N-e.** A cited raise reads git before `#dispatch` validates its other keys, so a malformed raise with a valid citation costs a git read before refusing. Harmless.
- **N-f.** The bus README and MIGRATION.md say the 3e fixture is "pinned from 56639c99b's tree". schema.sql is the same blob at `26a0b0fab`, so it's accurate, only dated. Doc-only (decision 137).
- The business chief-of-staff.test title still lists four held actions; `held` now includes `record.write`. The assertion is right; only the title is stale.
- The guard in `#dispatch` at broker.mjs:1164 can't be reached from `request()`, because routing uses `'spec_path' in args || 'spec_commit' in args`. It's fine as a backstop.
The four asks are tests only; none needs a product change.
**Reproduce**
`git worktree add --detach <dir> 26a0b0fab`, link `node_modules`, `git apply --index agents/rocko/work/queue-138/r3/build.patch`, then `sha256sum -c` against the candidate manifest. `singles.sh` and `mutants.py` hold the paths I used; change `W` and `OUT`.