docs(conversation): row 46 cohort K1/K3/K10 diagnosis, receipts and candidate packet (darkwing)

The ignoreTerm tool child was TERMed before Node installed its handler
when the claim store sits on a fast disk. Candidate fixes the fixture;
the patch itself is not committed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
2026-10-09 08:11:48 -05:00
co-authored by Claude Opus 5.5
parent 76506c7e96
commit 7478ee2c02
29 changed files with 879 additions and 0 deletions
@@ -0,0 +1 @@
fd10b62c10bff4736b9b4e809b283fe4c5b549fa53fe1121a7bb06258147f0e9 packages/conversation/tests/fake-pi.mjs
+152
View File
@@ -0,0 +1,152 @@
# Row 46: K1, K3 and K10 on the scope fixtures
Darkwing, 2026-10-09. Issue #1528, reviewer Dewey. Brief:
`docs/plans/2026-10-09_s4-follow-up-and-cohort.md`, section "Conversation
cohort: K1, K3 and K10 fail on the scope fixtures". Ruling: lead decision 72.
The candidate is not committed, staged or pushed.
## Cause
It's a race in the test fixture. Neither the host's systemd setup nor
`cohort.mjs` is at fault.
`spawnChild` in `packages/conversation/tests/fake-pi.mjs` returns the
child's pid as soon as `spawn()` returns. The child is `node -e`, and it
installs its SIGTERM handler only after Node has booted. That takes 16 to
22 ms on an idle host (`trace-pass.jsonl`) and 43 to 80 ms under 48 CPU burners (`trace-load.jsonl`). A
TERM that arrives before the handler gets the default action, and the child
dies of SIGTERM. Each failing assertion is that death seen from a different
place:
- K1, "the escaped child is a listed member". The child died during the
TERM grace, so the freeze-phase enumeration doesn't list it.
- K3, "the member ignored TERM". `alive(child)` is false at the kill
phase.
- K10, "the member is alive across the crash". Same as K3, before the
restart.
The time between spawn and TERM depends on the disk under TMPDIR. The
fixture puts the claim store there. Before it sends TERM, the controller
publishes claim revisions, each with an fsync on the file and one on the
directory (`packages/conversation/src/claim.mjs:159` and `:176`).
| TMPDIR | Disk | write+fsync median (`fsync.txt`) | spawn to TERM | Unfixed K1/K3/K10 |
|---|---|---|---|---|
| `/mnt/storage/scratch/tmp` | nvme0, ext4 | 0.65 ms | 12 to 13 ms (`trace-fail.jsonl`) | 0/3 (`runs/canon-scratchtmp.txt`) |
| `/tmp` | nvme1, ext4 | 4.63 ms | not traced | 3/3 (`runs/canon-tmp.txt`) |
| `~/darkwing-scratch/tmp` | nvme1, ext4 | 4.69 ms | 53 to 110 ms (`trace-pass.jsonl`) | 3/3 (`runs/canon-hometmp.txt`) |
On the fast disk, TERM lands about 12 ms after spawn, before Node is up.
In `trace-fail.jsonl` the children never log `ready`.
### Why it started failing
This host's seats now get `TMPDIR=/mnt/storage/scratch/tmp`. My shell had
it by default, and so did Sage's gate runs. It comes from Vikunja task 59.
`~/.config/systemd/user/t3code.service.d/tmpdir.conf` was written
2026-10-04 20:33Z. Its own comment says it takes effect only when
`t3code.service` restarts, which hasn't happened (active since 2026-09-23),
and that until then seats get TMPDIR from their harness config. I didn't
establish what TMPDIR the row 44 gate ran with on 2026-10-05, so the
"since when" is likely but unproven. The change that exposed the race is a
seat's TMPDIR. It isn't a systemd or user manager setting, and I changed
nothing on the host.
### The scope is not a factor
Outside any scope, `race.mjs` gives the same split: a TERM 0, 10 or 20 ms
after spawn kills 10/10, and at 30 or 60 ms 10/10 survive (`race.txt`).
Sage's outside-scope probe found the child surviving. I haven't seen that
probe, but it most likely sent TERM after the handler was in place.
## Receipts in a scope
`scope-receipt.mjs` launches a delegated scope per run, with the same
`systemd-run` flags `ScopeLauncher` uses. The scope's main process spawns
the fixture's `ignoreTerm` child and records `systemctl --user show` on the
scope, then `cgroup.procs` before and after TERM, then the child's exit.
Output: `receipts.jsonl`.
| Mode | TERM after spawn | In `cgroup.procs` before | After | Child exit |
|---|---|---|---|---|
| `race 12` | 20 to 25 ms | 5/5 | 1/5 | 4/5 `signal: "SIGTERM"`, 1 alive |
| `race 60` | 69 to 71 ms | 5/5 | 5/5 | 5/5 alive 500 ms after TERM |
| `ready 0` | 28 to 38 ms (ready at 22 to 30) | 5/5 | 5/5 | 5/5 alive 500 ms after TERM |
Each line also carries the scope's `Id`, `LoadState=loaded`,
`ActiveState=active`, `InvocationID`, `ControlGroup` and `Delegate=yes`.
The first line, for example:
`ControlGroup=/user.slice/user-1000.slice/[email protected]/app.slice/dw-r46-race-12-3662746-0.scope`,
`before [3662760, 3662788]`, `after [3662760]`, child exit
`{"code":null,"signal":"SIGTERM","atMs":24}`.
`trace.patch` is the scratch-only instrumentation behind the two traces.
It logs spawn, ready and the shim's `term` per pid to `$DW_TRACE`. It is
not part of the candidate.
## The fix
`build.patch` changes one file, `packages/conversation/tests/fake-pi.mjs`:
- The child writes one byte to stdout as its first action after installing
its TERM handler. A child without `ignoreTerm` writes it once its code
starts.
- `spawnChild` returns a promise. It resolves with the pid on that byte,
then closes its end of the pipe. It rejects if the child exits or errors
first, or if 10 s pass.
- The `child` control op returns that promise. The op dispatcher now
answers `ok: false` when an op's promise rejects. Before, a rejected op
promise went unhandled, and under Node's default that ends fake-pi. The
change covers every op that returns a promise, not only `child`.
`childOf` in `cohort.test.mjs` already asserts `r.ok`.
No assertion changed, and `cohort.mjs` is untouched. The three cases now
test what their names say: TERM reaches a child that is already ignoring
TERM.
`build-manifest.sha256` (sha256
`9228414352a56e0465e896b81643cdce000bcedba70f86adac07fc50cfadad2d`) pins
`fake-pi.mjs` after the patch. `build.patch` sha256 is
`04234ce1d886e36dfd06cbd81153b38cbb86909b6ee933151306b8cdedd629d9`. In a
fresh worktree at `521597bb` the patch applies and the manifest checks 1/1.
## Mutants
Both ran with `TMPDIR=/mnt/storage/scratch/tmp`, the condition that fails.
| Mutant | K1/K3/K10 | File |
|---|---|---|
| M1: resolve at spawn, no wait (the old behavior) | 0/3 | `runs/mutant-m1-nowait.txt` |
| M2: wait for ready, but the child has no TERM handler | 0/3, the same three assertions | `runs/mutant-m2-noignore.txt` |
M1 shows the wait is what fixes it. M2 shows the assertions still catch a
child that dies on TERM.
## Runs
The patch was applied in a scratch worktree at `521597bb`. Node v26.8.1.
Start time and load are in `runs/start.txt`.
| Run | TMPDIR | Result | File |
|---|---|---|---|
| `node --test "packages/conversation/tests/*.test.mjs"` | `/mnt/storage/scratch/tmp` | 152/152 | `runs/node-conversation.txt` |
| `node --test "packages/webui/tests/*.test.mjs"` | `/mnt/storage/scratch/tmp` | 14/14 | `runs/node-webui.txt` |
| K1/K3/K10 isolated, 3 consecutive | `/mnt/storage/scratch/tmp` | 3/3, 3/3, 3/3 | `runs/k-iso-{1,2,3}.txt` |
| K1/K3/K10 isolated | `/tmp` | 3/3 | `runs/k-tmp.txt` |
| K1/K3/K10 isolated | `~/darkwing-scratch/tmp` | 3/3 | `runs/k-home.txt` |
| K1/K3/K10 isolated under 48 CPU burners, 3 runs (load 13.6 to 24.9) | `/mnt/storage/scratch/tmp` | 3/3, 3/3, 3/3 | `runs/k-load48-{1,2,3}.txt` |
For the gate rerun, keep the default `TMPDIR=/mnt/storage/scratch/tmp`.
That's the condition that failed, and a slower TMPDIR would pass with or
without the patch.
## Files
- `build.md`, this file.
- `build.patch`, `build-manifest.sha256`: the candidate.
- `scope-receipt.mjs`, `receipts.jsonl`: the scope receipts.
- `race.mjs`, `race.txt`: TERM timing outside a scope.
- `fsync.mjs`, `fsync.txt`: write+fsync latency per TMPDIR.
- `trace.patch`, `trace-fail.jsonl`, `trace-pass.jsonl`, `trace-load.jsonl`: the traced runs.
- `runs/`: suite, isolated, load and mutant outputs, and the unfixed
canonical tree under three TMPDIRs.
@@ -0,0 +1,67 @@
diff --git a/packages/conversation/tests/fake-pi.mjs b/packages/conversation/tests/fake-pi.mjs
index b3014f86..a78dfebd 100644
--- a/packages/conversation/tests/fake-pi.mjs
+++ b/packages/conversation/tests/fake-pi.mjs
@@ -615,14 +615,34 @@ const children = [];
// K2); `forkLoop` forks every 5 ms (K12); `ignoreTerm` survives SIGTERM, so
// only the kill phase ends it (K3, K10, K11). With `pidLog`, the fork loop
// appends each child's pid and a `term` line when it gets SIGTERM (K12).
+//
+// It resolves only once the child writes its ready byte, which it does after
+// installing its TERM handler. Node takes 15 to 80 ms to get there, and a
+// force stop can send TERM sooner (12 ms when the claim store is on a fast
+// disk). A TERM before the handler kills a child that is meant to ignore it
+// (#1528).
function spawnChild({ setsid = false, forkLoop = false, ignoreTerm = false, pidLog = null } = {}) {
const note = pidLog ? `const note=(s)=>require('node:fs').appendFileSync(${JSON.stringify(pidLog)},s+'\\n');` : "const note=()=>{};";
- const code = note + (ignoreTerm ? "process.on('SIGTERM',()=>note('term'));" : "") + (forkLoop
+ const code = note + (ignoreTerm ? "process.on('SIGTERM',()=>note('term'));" : "") + "process.stdout.write('r');" + (forkLoop
? "const {spawn}=require('node:child_process');setInterval(()=>{try{const c=spawn('sleep',['1000'],{stdio:'ignore'});if(c.pid)note(String(c.pid))}catch{}},5);setInterval(()=>{},1e9)"
: "setInterval(()=>{},1e9)");
- const child = spawn(process.execPath, ["-e", code], { stdio: "ignore", detached: setsid });
+ const child = spawn(process.execPath, ["-e", code], { stdio: ["ignore", "pipe", "ignore"], detached: setsid });
children.push(child.pid);
- return child.pid;
+ return new Promise((resolve, reject) => {
+ const fail = (why) => {
+ clearTimeout(timer);
+ reject(new Error(`tool child ${child.pid} ${why} before it was ready`));
+ };
+ const timer = setTimeout(() => fail("took 10 s"), 10000);
+ child.once("error", (err) => fail(err.code ?? err.message));
+ child.once("exit", (code, signal) => fail(`exited (${signal ?? code})`));
+ child.stdout.once("data", () => {
+ clearTimeout(timer);
+ child.removeAllListeners("exit");
+ child.stdout.destroy();
+ resolve(child.pid);
+ });
+ });
}
// K13: a member writes its own pid to another cgroup's cgroup.procs.
@@ -671,7 +691,7 @@ async function main() {
drop: () => fake.dropResponse(req.type, req.n ?? 1),
extension: () => void fake.extensionPrompt(req.args ?? {}),
state: () => ({ streaming: fake.streaming, runs: fake.runs.length, commands: fake.commands, pid: process.pid, children, appends: fake.appends }),
- child: () => ({ pid: spawnChild(req.args ?? {}) }),
+ child: () => spawnChild(req.args ?? {}).then((pid) => ({ pid })),
escape: () => escape(req.target),
cgroup: () => readFileSync(`/proc/${req.pid ?? process.pid}/cgroup`, "utf8"),
waitPaused: () => fake.waitPaused(req.point),
@@ -679,12 +699,13 @@ async function main() {
stall: () => void process.stdin.pause(),
};
if (!ops[req.op]) return sock.write(encodeLine({ id: req.id, ok: false, error: `unknown op ${req.op}` }));
+ const failed = (err) => sock.write(encodeLine({ id: req.id, ok: false, error: String(err.message) }));
try {
const out = ops[req.op]();
- if (out && typeof out.then === "function") out.then(reply);
+ if (out && typeof out.then === "function") out.then(reply, failed);
else reply(out ?? null);
} catch (err) {
- sock.write(encodeLine({ id: req.id, ok: false, error: String(err.message) }));
+ failed(err);
}
});
sock.on("data", (c) => splitter.push(c));
+17
View File
@@ -0,0 +1,17 @@
import { openSync, writeSync, fsyncSync, closeSync, rmSync, mkdtempSync } from "node:fs";
import { join } from "node:path";
for (const base of process.argv.slice(2)) {
const d = mkdtempSync(join(base, "dw-fsync-"));
const t = [];
for (let i = 0; i < 30; i++) {
const s = performance.now();
const fd = openSync(join(d, `f${i}`), "w");
writeSync(fd, "x".repeat(512));
fsyncSync(fd);
closeSync(fd);
t.push(performance.now() - s);
}
rmSync(d, { recursive: true });
t.sort((a, b) => a - b);
console.log(`${base}: write+fsync median ${t[15].toFixed(2)} ms, p90 ${t[27].toFixed(2)} ms`);
}
+3
View File
@@ -0,0 +1,3 @@
/mnt/storage/scratch/tmp: write+fsync median 0.65 ms, p90 0.79 ms
/tmp: write+fsync median 4.63 ms, p90 4.93 ms
/home/jwoltje/darkwing-scratch/tmp: write+fsync median 4.69 ms, p90 9.30 ms
+17
View File
@@ -0,0 +1,17 @@
// Spawn the K fixtures' ignoreTerm child exactly as fake-pi.mjs spawnChild does, then
// SIGTERM it after a delay. Reports how the child ended within 1 s.
import { spawn } from "node:child_process";
const code = "const note=()=>{};process.on('SIGTERM',()=>note('term'));setInterval(()=>{},1e9)";
const delays = process.argv.slice(2).map(Number);
for (const d of delays) {
const r = { died: 0, survived: 0 };
for (let i = 0; i < 10; i++) {
const c = spawn(process.execPath, ["-e", code], { stdio: "ignore" });
const ended = new Promise((res) => c.on("exit", (code, sig) => res(sig ?? code)));
await new Promise((res) => setTimeout(res, d));
c.kill("SIGTERM");
const out = await Promise.race([ended, new Promise((res) => setTimeout(() => res(null), 1000))]);
if (out === null) { r.survived++; c.kill("SIGKILL"); await ended; } else r.died++;
}
console.log(`TERM ${d} ms after spawn: died ${r.died}/10 survived ${r.survived}/10`);
}
+5
View File
@@ -0,0 +1,5 @@
TERM 0 ms after spawn: died 10/10 survived 0/10
TERM 10 ms after spawn: died 10/10 survived 0/10
TERM 20 ms after spawn: died 10/10 survived 0/10
TERM 30 ms after spawn: died 0/10 survived 10/10
TERM 60 ms after spawn: died 0/10 survived 10/10
@@ -0,0 +1,15 @@
{"mode":"race","delayMs":12,"readyMs":null,"termMs":23,"child":3662788,"self":3662760,"show":["Id=dw-r46-race-12-3662746-0.scope","LoadState=loaded","ActiveState=active","SubState=running","InvocationID=2eac4fa367504b9dbca66a4693cc328a","ControlGroup=/user.slice/user-1000.slice/[email protected]/app.slice/dw-r46-race-12-3662746-0.scope","Delegate=yes"],"before":[3662760,3662788],"after":[3662760],"childInBefore":true,"childInAfter":false,"exit":{"code":null,"signal":"SIGTERM","atMs":24}}
{"mode":"race","delayMs":12,"readyMs":null,"termMs":20,"child":3663479,"self":3663212,"show":["Id=dw-r46-race-12-3662746-1.scope","LoadState=loaded","ActiveState=active","SubState=running","InvocationID=021c6d466fc64961ba3031a0254c3803","ControlGroup=/user.slice/user-1000.slice/[email protected]/app.slice/dw-r46-race-12-3662746-1.scope","Delegate=yes"],"before":[3663212,3663479],"after":[3663212],"childInBefore":true,"childInAfter":false,"exit":{"code":null,"signal":"SIGTERM","atMs":21}}
{"mode":"race","delayMs":12,"readyMs":null,"termMs":25,"child":3663799,"self":3663588,"show":["Id=dw-r46-race-12-3662746-2.scope","LoadState=loaded","ActiveState=active","SubState=running","InvocationID=ce8e2c2dbbe7481885a50152fe3766b5","ControlGroup=/user.slice/user-1000.slice/[email protected]/app.slice/dw-r46-race-12-3662746-2.scope","Delegate=yes"],"before":[3663588,3663799],"after":[3663588],"childInBefore":true,"childInAfter":false,"exit":{"code":null,"signal":"SIGTERM","atMs":27}}
{"mode":"race","delayMs":12,"readyMs":null,"termMs":20,"child":3664563,"self":3664322,"show":["Id=dw-r46-race-12-3662746-3.scope","LoadState=loaded","ActiveState=active","SubState=running","InvocationID=cc912ab9a7fe47dd9e3b97486abf0bc3","ControlGroup=/user.slice/user-1000.slice/[email protected]/app.slice/dw-r46-race-12-3662746-3.scope","Delegate=yes"],"before":[3664322,3664563],"after":[3664322,3664563],"childInBefore":true,"childInAfter":true,"exit":"alive 500 ms after TERM"}
{"mode":"race","delayMs":12,"readyMs":null,"termMs":21,"child":3665177,"self":3665003,"show":["Id=dw-r46-race-12-3662746-4.scope","LoadState=loaded","ActiveState=active","SubState=running","InvocationID=163271c658cc4ee4b84f5aace54bc806","ControlGroup=/user.slice/user-1000.slice/[email protected]/app.slice/dw-r46-race-12-3662746-4.scope","Delegate=yes"],"before":[3665003,3665177],"after":[3665003],"childInBefore":true,"childInAfter":false,"exit":{"code":null,"signal":"SIGTERM","atMs":22}}
{"mode":"race","delayMs":60,"readyMs":null,"termMs":69,"child":3665812,"self":3665626,"show":["Id=dw-r46-race-60-3665599-0.scope","LoadState=loaded","ActiveState=active","SubState=running","InvocationID=14681de5ac844879bdfcbcdd3dae8f10","ControlGroup=/user.slice/user-1000.slice/[email protected]/app.slice/dw-r46-race-60-3665599-0.scope","Delegate=yes"],"before":[3665626,3665812],"after":[3665626,3665812],"childInBefore":true,"childInAfter":true,"exit":"alive 500 ms after TERM"}
{"mode":"race","delayMs":60,"readyMs":null,"termMs":71,"child":3666377,"self":3666179,"show":["Id=dw-r46-race-60-3665599-1.scope","LoadState=loaded","ActiveState=active","SubState=running","InvocationID=3128eba3ed234776ae4a61cb12fe1210","ControlGroup=/user.slice/user-1000.slice/[email protected]/app.slice/dw-r46-race-60-3665599-1.scope","Delegate=yes"],"before":[3666179,3666377],"after":[3666179,3666377],"childInBefore":true,"childInAfter":true,"exit":"alive 500 ms after TERM"}
{"mode":"race","delayMs":60,"readyMs":null,"termMs":70,"child":3666924,"self":3666728,"show":["Id=dw-r46-race-60-3665599-2.scope","LoadState=loaded","ActiveState=active","SubState=running","InvocationID=836de499898d4b6996739e1b2a433fa0","ControlGroup=/user.slice/user-1000.slice/[email protected]/app.slice/dw-r46-race-60-3665599-2.scope","Delegate=yes"],"before":[3666728,3666924],"after":[3666728,3666924],"childInBefore":true,"childInAfter":true,"exit":"alive 500 ms after TERM"}
{"mode":"race","delayMs":60,"readyMs":null,"termMs":69,"child":3667387,"self":3667251,"show":["Id=dw-r46-race-60-3665599-3.scope","LoadState=loaded","ActiveState=active","SubState=running","InvocationID=0ebf8d9d930a43629668e10c08f80e0d","ControlGroup=/user.slice/user-1000.slice/[email protected]/app.slice/dw-r46-race-60-3665599-3.scope","Delegate=yes"],"before":[3667251,3667387],"after":[3667251,3667387],"childInBefore":true,"childInAfter":true,"exit":"alive 500 ms after TERM"}
{"mode":"race","delayMs":60,"readyMs":null,"termMs":70,"child":3667888,"self":3667727,"show":["Id=dw-r46-race-60-3665599-4.scope","LoadState=loaded","ActiveState=active","SubState=running","InvocationID=9c3594ae3018458594bb3ff996dc0d9a","ControlGroup=/user.slice/user-1000.slice/[email protected]/app.slice/dw-r46-race-60-3665599-4.scope","Delegate=yes"],"before":[3667727,3667888],"after":[3667727,3667888],"childInBefore":true,"childInAfter":true,"exit":"alive 500 ms after TERM"}
{"mode":"ready","delayMs":0,"readyMs":30,"termMs":38,"child":3668171,"self":3668085,"show":["Id=dw-r46-ready-0-3668072-0.scope","LoadState=loaded","ActiveState=active","SubState=running","InvocationID=4c2c3b3ad52a477e8f735abedf158cbb","ControlGroup=/user.slice/user-1000.slice/[email protected]/app.slice/dw-r46-ready-0-3668072-0.scope","Delegate=yes"],"before":[3668085,3668171],"after":[3668085,3668171],"childInBefore":true,"childInAfter":true,"exit":"alive 500 ms after TERM"}
{"mode":"ready","delayMs":0,"readyMs":22,"termMs":29,"child":3668354,"self":3668284,"show":["Id=dw-r46-ready-0-3668072-1.scope","LoadState=loaded","ActiveState=active","SubState=running","InvocationID=b9da06280a8a4c6682f05a2d4ff118cf","ControlGroup=/user.slice/user-1000.slice/[email protected]/app.slice/dw-r46-ready-0-3668072-1.scope","Delegate=yes"],"before":[3668284,3668354],"after":[3668284,3668354],"childInBefore":true,"childInAfter":true,"exit":"alive 500 ms after TERM"}
{"mode":"ready","delayMs":0,"readyMs":22,"termMs":28,"child":3668425,"self":3668417,"show":["Id=dw-r46-ready-0-3668072-2.scope","LoadState=loaded","ActiveState=active","SubState=running","InvocationID=aab0b83d085e4bac86fdbe1ce65f23dc","ControlGroup=/user.slice/user-1000.slice/[email protected]/app.slice/dw-r46-ready-0-3668072-2.scope","Delegate=yes"],"before":[3668417,3668425],"after":[3668417,3668425],"childInBefore":true,"childInAfter":true,"exit":"alive 500 ms after TERM"}
{"mode":"ready","delayMs":0,"readyMs":23,"termMs":30,"child":3668509,"self":3668482,"show":["Id=dw-r46-ready-0-3668072-3.scope","LoadState=loaded","ActiveState=active","SubState=running","InvocationID=d076a3b629554b83bded41cb00e667b8","ControlGroup=/user.slice/user-1000.slice/[email protected]/app.slice/dw-r46-ready-0-3668072-3.scope","Delegate=yes"],"before":[3668482,3668509],"after":[3668482,3668509],"childInBefore":true,"childInAfter":true,"exit":"alive 500 ms after TERM"}
{"mode":"ready","delayMs":0,"readyMs":23,"termMs":30,"child":3668635,"self":3668624,"show":["Id=dw-r46-ready-0-3668072-4.scope","LoadState=loaded","ActiveState=active","SubState=running","InvocationID=6b46a0b279ca454eaa034d48a4de7385","ControlGroup=/user.slice/user-1000.slice/[email protected]/app.slice/dw-r46-ready-0-3668072-4.scope","Delegate=yes"],"before":[3668624,3668635],"after":[3668624,3668635],"childInBefore":true,"childInAfter":true,"exit":"alive 500 ms after TERM"}
@@ -0,0 +1,11 @@
✔ K1: force stop kills a tool child that called setsid; stopped with a verified proof (2737.880671ms)
✔ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2557.042479ms)
✔ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (861.841994ms)
ℹ tests 3
ℹ suites 0
ℹ pass 3
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 6282.04986
@@ -0,0 +1,56 @@
✖ K1: force stop kills a tool child that called setsid; stopped with a verified proof (2537.385038ms)
✖ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2331.351365ms)
✖ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (209.932151ms)
ℹ tests 3
ℹ suites 0
ℹ pass 0
ℹ fail 3
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 5448.48373
✖ failing tests:
test at packages/conversation/tests/cohort.test.mjs:139:1
✖ K1: force stop kills a tool child that called setsid; stopped with a verified proof (2537.385038ms)
AssertionError [ERR_ASSERTION]: the escaped child is a listed member
at TestContext.<anonymous> (file:///mnt/storage/src/mosaic-stack/packages/conversation/tests/cohort.test.mjs:151:12)
at async Test.run (node:internal/test_runner/test:1409:7)
at async startSubtestAfterBootstrap (node:internal/test_runner/harness:387:3) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '==',
diff: 'simple'
}
test at packages/conversation/tests/cohort.test.mjs:176:1
✖ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2331.351365ms)
AssertionError [ERR_ASSERTION]: the member ignored TERM
at TestContext.<anonymous> (file:///mnt/storage/src/mosaic-stack/packages/conversation/tests/cohort.test.mjs:190:12)
at async Test.run (node:internal/test_runner/test:1409:7)
at async Test.processPendingSubtests (node:internal/test_runner/test:974:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '==',
diff: 'simple'
}
test at packages/conversation/tests/cohort.test.mjs:426:3
✖ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (209.932151ms)
AssertionError [ERR_ASSERTION]: the member is alive across the crash
at TestContext.<anonymous> (file:///mnt/storage/src/mosaic-stack/packages/conversation/tests/cohort.test.mjs:431:12)
at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
at async Test.run (node:internal/test_runner/test:1409:7)
at async Test.processPendingSubtests (node:internal/test_runner/test:974:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '==',
diff: 'simple'
}
@@ -0,0 +1,11 @@
✔ K1: force stop kills a tool child that called setsid; stopped with a verified proof (2360.366213ms)
✔ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2364.75763ms)
✔ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (628.04725ms)
ℹ tests 3
ℹ suites 0
ℹ pass 3
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 5474.064409
@@ -0,0 +1,11 @@
✔ K1: force stop kills a tool child that called setsid; stopped with a verified proof (2457.36226ms)
✔ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2375.081486ms)
✔ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (636.350971ms)
ℹ tests 3
ℹ suites 0
ℹ pass 3
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 5607.100296
@@ -0,0 +1,11 @@
✔ K1: force stop kills a tool child that called setsid; stopped with a verified proof (2665.443725ms)
✔ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2531.922418ms)
✔ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (706.098022ms)
ℹ tests 3
ℹ suites 0
ℹ pass 3
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 6068.030335
@@ -0,0 +1,11 @@
✔ K1: force stop kills a tool child that called setsid; stopped with a verified proof (2699.60543ms)
✔ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2492.169051ms)
✔ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (675.081525ms)
ℹ tests 3
ℹ suites 0
ℹ pass 3
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 6026.119198
@@ -0,0 +1,11 @@
✔ K1: force stop kills a tool child that called setsid; stopped with a verified proof (2762.539396ms)
✔ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2528.214494ms)
✔ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (644.633095ms)
ℹ tests 3
ℹ suites 0
ℹ pass 3
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 6115.728995
@@ -0,0 +1,11 @@
✔ K1: force stop kills a tool child that called setsid; stopped with a verified proof (2887.135899ms)
✔ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2915.708256ms)
✔ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (2590.181146ms)
ℹ tests 3
ℹ suites 0
ℹ pass 3
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 9662.912642
@@ -0,0 +1,11 @@
✔ K1: force stop kills a tool child that called setsid; stopped with a verified proof (3084.73509ms)
✔ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2750.8295ms)
✔ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (2151.224665ms)
ℹ tests 3
ℹ suites 0
ℹ pass 3
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 8355.450647
@@ -0,0 +1,11 @@
✔ K1: force stop kills a tool child that called setsid; stopped with a verified proof (2746.07054ms)
✔ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2559.848734ms)
✔ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (1985.055632ms)
ℹ tests 3
ℹ suites 0
ℹ pass 3
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 7722.015242
@@ -0,0 +1,11 @@
✔ K1: force stop kills a tool child that called setsid; stopped with a verified proof (2785.421246ms)
✔ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2619.007617ms)
✔ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (889.082424ms)
ℹ tests 3
ℹ suites 0
ℹ pass 3
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 6776.303473
@@ -0,0 +1,56 @@
✖ K1: force stop kills a tool child that called setsid; stopped with a verified proof (2523.134497ms)
✖ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2421.121319ms)
✖ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (211.187938ms)
ℹ tests 3
ℹ suites 0
ℹ pass 0
ℹ fail 3
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 5461.571354
✖ failing tests:
test at cohort.test.mjs:139:1
✖ K1: force stop kills a tool child that called setsid; stopped with a verified proof (2523.134497ms)
AssertionError [ERR_ASSERTION]: the escaped child is a listed member
at TestContext.<anonymous> (file:///home/jwoltje/darkwing-scratch/r46/fix/packages/conversation/tests/cohort.test.mjs:151:12)
at async Test.run (node:internal/test_runner/test:1409:7)
at async startSubtestAfterBootstrap (node:internal/test_runner/harness:387:3) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '==',
diff: 'simple'
}
test at cohort.test.mjs:176:1
✖ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2421.121319ms)
AssertionError [ERR_ASSERTION]: the member ignored TERM
at TestContext.<anonymous> (file:///home/jwoltje/darkwing-scratch/r46/fix/packages/conversation/tests/cohort.test.mjs:190:12)
at async Test.run (node:internal/test_runner/test:1409:7)
at async Test.processPendingSubtests (node:internal/test_runner/test:974:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '==',
diff: 'simple'
}
test at cohort.test.mjs:426:3
✖ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (211.187938ms)
AssertionError [ERR_ASSERTION]: the member is alive across the crash
at TestContext.<anonymous> (file:///home/jwoltje/darkwing-scratch/r46/fix/packages/conversation/tests/cohort.test.mjs:431:12)
at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
at async Test.run (node:internal/test_runner/test:1409:7)
at async Test.processPendingSubtests (node:internal/test_runner/test:974:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '==',
diff: 'simple'
}
@@ -0,0 +1,56 @@
✖ K1: force stop kills a tool child that called setsid; stopped with a verified proof (2545.15719ms)
✖ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2407.231229ms)
✖ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (304.986006ms)
ℹ tests 3
ℹ suites 0
ℹ pass 0
ℹ fail 3
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 5605.846956
✖ failing tests:
test at cohort.test.mjs:139:1
✖ K1: force stop kills a tool child that called setsid; stopped with a verified proof (2545.15719ms)
AssertionError [ERR_ASSERTION]: the escaped child is a listed member
at TestContext.<anonymous> (file:///home/jwoltje/darkwing-scratch/r46/fix/packages/conversation/tests/cohort.test.mjs:151:12)
at async Test.run (node:internal/test_runner/test:1409:7)
at async startSubtestAfterBootstrap (node:internal/test_runner/harness:387:3) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '==',
diff: 'simple'
}
test at cohort.test.mjs:176:1
✖ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2407.231229ms)
AssertionError [ERR_ASSERTION]: the member ignored TERM
at TestContext.<anonymous> (file:///home/jwoltje/darkwing-scratch/r46/fix/packages/conversation/tests/cohort.test.mjs:190:12)
at async Test.run (node:internal/test_runner/test:1409:7)
at async Test.processPendingSubtests (node:internal/test_runner/test:974:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '==',
diff: 'simple'
}
test at cohort.test.mjs:426:3
✖ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (304.986006ms)
AssertionError [ERR_ASSERTION]: the member is alive across the crash
at TestContext.<anonymous> (file:///home/jwoltje/darkwing-scratch/r46/fix/packages/conversation/tests/cohort.test.mjs:431:12)
at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
at async Test.run (node:internal/test_runner/test:1409:7)
at async Test.processPendingSubtests (node:internal/test_runner/test:974:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '==',
diff: 'simple'
}
@@ -0,0 +1,160 @@
✔ W1: two processes acquire the same pair at once; exactly one claim (124.186492ms)
✔ W1: two writers publish the same revision at once: one wins, the other gets null, the winner's record stays (11.494379ms)
✔ W1: a revision name appears only after its bytes are synced; before that, only a temp file exists (5.019078ms)
✔ W2: acquire while a claim is reserved or active refuses already-active (169.129732ms)
✔ W3: acquire while stopping, uncertain, or stopped without proof refuses unsafe-replacement (231.756886ms)
✔ W4: same session with another seat tuple, and the reverse, both refuse; a loser on the seat key closes it no-unit (192.859955ms)
✔ W4: a hard link of one session under another seat is the same session: the second controller refuses already-active and launches nothing (35.003128ms)
✔ W4: a copy of one session under another seat is the same session: the second controller refuses already-active and launches nothing (21.132326ms)
✔ W4: a session header ID that changes after construction refuses target; nothing is claimed or launched (3.242382ms)
✔ W5: SIGKILL between every publication barrier of acquire and transition; restart never finds two holders or a lost claim (5566.910492ms)
✔ W5: SIGKILL between every publication barrier of release; restart finishes or holds the release (22659.492505ms)
✔ W6: controller killed mid-turn while the engine lives; restart is uncertain, no launch, prompts refuse (201.806859ms)
✔ W12: a live owner paused with SIGSTOP; a second controller refuses already-active and changes nothing (116.242029ms)
✔ W13: crash after the engine spawns, before active; restart finds the live unit: uncertain, no second spawn, force stop only (262.226787ms)
✔ W14: crash after reservation, before the spawn marker: stopped with a no-unit observation; the pair is free (220.425841ms)
✔ W20: crash after the spawn marker, scope collected; uncertain in both runs, the marker is copied, no launch until a boot proof (240.127173ms)
✔ W15: crash between the two keys during release; restart finishes it under the same claim ID (35.945246ms)
✔ W7: recorded boot ID differs on the same machine: stopped with a boot proof; open tool calls become uncertain (98.424488ms)
✔ W8: resume after a proven stop with the same pins: new claim ID, generation +1, same conversation, branch and leaf (32.141383ms)
✔ W9: resume with a changed binary, argv digest, branch or leaf is refused and the claim is unchanged (89.672997ms)
✔ W11: the controller writes no session file; only the fake engine's own appends appear (23.121134ms)
✔ W16: a highest revision that won't parse holds the pair uncertain; the older stopped revision is not reused (54.682806ms)
✔ W17: a claim root copied from another host refuses foreign-host and promotes nothing (57.719055ms)
✔ G1: a session path or claim root under .pi/state, ~/.claude, the data root or a registration refuses at construction (4.12463ms)
✔ G2: a symlink inside the fixture root to a live session file is refused by the real-path check (1.12384ms)
✔ G3: a fixture path swapped for a live path after construction is refused at bind (1.84306ms)
✔ K1: force stop kills a tool child that called setsid; stopped with a verified proof (2507.919568ms)
✔ K2: K1 on the process-group fallback ends uncertain, never stopped (153.327228ms)
✔ K3: SIGTERM acknowledged while a member lives: stopping until the kill phase, never stopped from TERM (2414.676654ms)
✔ K4: two engines; force stop one; the other survives by independent observation (4314.515513ms)
✔ K5: a stop during a tool call leaves the effect uncertain, and it is shown (2190.378919ms)
✔ K12: a member forking in a loop: the freeze stops it, enumeration is complete, populated 0 after cgroup.kill (2244.796524ms)
✔ K13: a member writing its pid into another cgroup is refused by the namespace; the kill is complete (2189.63935ms)
✔ K15: the shim gone, engine/cgroup.events unreadable, or the engine cgroup missing: evidence unavailable, not empty; uncertain (4473.315326ms)
✔ K10: controller killed between the TERM and kill phases: restart checks the invocation ID and re-runs from TERM for the same stop (482.430287ms)
✔ K11: controller killed after the confirmation is recorded, before TERM: restart checks the invocation ID and re-runs from TERM for the same stop (408.721794ms)
✔ K14: a unit with the recorded name but another invocation ID: evidence unavailable, no signals, uncertain (307.075836ms)
✔ K6: recover without proof, without confirmation, or with changed pins is refused (71.309069ms)
✔ K7: recover after proof, then launch: new claim and execution, generation +1, same leaf; the cancelled prompt is not replayed (35.403696ms)
✔ K8: an engine that loads another leaf on resume is refused before admission; it stays claimed until a proven stop (45.424183ms)
✔ K9: an interrupt that never settles stays uncertain; force stop stays available; takeover is refused while fenced (3030.941578ms)
✔ K16: a claim from another machine ID refuses foreign-host; no boot proof is issued (5.325961ms)
✔ K17: two launcher calls with one eligibility record: one launch, the other refuses, no second engine (30.688596ms)
✔ K18: the leaf changes after eligibility: launch refused; the reservation stays until released with proof (24.893734ms)
✔ S1: `/goal x`, with leading spaces or a tab, refuses text-policy at admission; zero engine bytes (34.176579ms)
✔ S2: every prefix pinned Pi interprets is refused, from the list the code uses; the rest reach the engine exactly (30.425031ms)
✔ S3: `/goal` on the second line is pinned from the source: Pi checks only index 0, so it is admitted and sent exactly (27.293519ms)
✔ S4: a `/` left in the composer is cleared when control transfers and returns; the next submit sends only the new text (42.046162ms)
✔ S5: an observer terminal gets a paste then Enter, as send-message.sh does: not admitted: controller, nothing sent (21.340651ms)
✔ S6: a mediated-shaped registration (no tmux) passed to the board's replyToRow: 409 no tmux session; exec never runs (1.34914ms)
✔ S7: ESC, bracketed-paste markers and U+2028/U+2029 travel as one JSON string; the engine receives the exact text in one record (29.201111ms)
✔ P3: a Pi confirm, select, input or editor dialog is shown disabled with a reason and never answered (138.365864ms)
✔ E1: send, ack, user, toolCall, toolResult, final answer: shown once, no refresh, draft and reading position kept (39.366805ms)
✔ E2: U+2028, U+2029 inside JSON strings and CRLF line ends each parse as one record, on the splitter and through the controller (21.443107ms)
✔ E3: a multipart final, two blocks, null request correlation and duplicate delivery (31.257304ms)
✔ E4: a page read after message_end but before its entry is persisted: marker at the seam, re-read after run-settled, each message once (26.594189ms)
✔ E4: a gap or a new epoch also reconciles; nothing is concatenated across a gap (11.181347ms)
✔ E5: an unknown native event gives no client event; evidence records its type and bytes; the terminal count goes up (26.220793ms)
✔ E6: a tool result delayed across a pause and a reconnect is reconciled without a manual refresh (42.195378ms)
✔ E7: the terminal renders the same stream as the library client, as observer and then as controller, and submits only as controller (41.055698ms)
✔ terminal: engine control characters are made visible; a lost connection refuses submit (28.737455ms)
✔ terminal: outcome unknown is shown as such, with no resend offer, and nothing is resent (0.53917ms)
✔ terminal: text after Enter in the same input chunk starts the next message; it never joins the one submitted (0.351376ms)
✔ terminal: a paste-start marker split right after its ESC still opens the paste; the Enter inside it never submits (0.356358ms)
✔ terminal: invisible and bidi characters are made visible; head, status and notice lines stay one line (0.168609ms)
✔ every record these fixtures produced is a valid CHAT-01 record (E5: no record fails the schema) (359.398761ms)
✔ H1: two takeovers with the same expected generation: one wins, +1; the other refuses generation (56.47649ms)
✔ H2: the old controller's prompt after a takeover commits is refused with zero engine bytes (79.112044ms)
✔ H3: a takeover while a prompt holds the dispatch lock: written under the old actor, or refused; never both (131.150736ms)
✔ H4: self-takeover is refused (22.440288ms)
✔ H9: Interrupt racing a prompt's dispatch: before the write, dispatch-refused and no-turn; after, §3 rules (87.314186ms)
✔ H10: Interrupt and force stop together: one stop chain, force stop supersedes (104.583327ms)
✔ H10: an overlap during the pause before the abort: no abort, the stop ends uncertain (35.050855ms)
✔ H10: a no-turn Interrupt lifts only its own fence; admission stays closed under force stop, overlap or revocation (70.09079ms)
✔ H11: the controller disconnects mid-turn: work continues, the claim is unchanged, control stays put (129.262166ms)
✔ H12: an exact retry after reconnecting to the same incarnation returns the same receipt; one dispatch (15.147609ms)
✔ H13: a retry with the same request ID and different text is refused (13.200342ms)
✔ H14: late stdout from the old engine after a replacement is dropped by incarnation, counted, never rendered (133.123623ms)
✔ H15: a revoked connection's command is refused; the revocation fence holds (65.378107ms)
✔ H16: a second controller for the same session refuses already-active; the first is untouched (16.417034ms)
✔ H10: a second force stop while the first escalation runs refuses fenced; one escalation, and the claim records only the first stop's phases (64.083554ms)
✔ H17: a confirmation reused, answered from another connection, or used after the stop changed is refused (56.969236ms)
✔ H18: two prompts before any native output: the second refuses busy; one engine write (15.033626ms)
✔ H19: the pipe fails mid-line under a large prompt: delivery-unknown transport-unknown, poisoned, no later write (119.397422ms)
✔ H19: the link itself never writes again after an unknown outcome, whoever calls it (0.606652ms)
✔ H19: the controller dies mid-write of a large line: after restart the outcome is unknown and nothing is resent (464.542066ms)
✔ H20: the line is written but the ack is lost when the controller dies: orphan, outcome unknown, nothing resent (357.812591ms)
✔ H21: a retry of the exact request with the old token after a crash is stale-incarnation; no second write (337.123938ms)
✔ H22: after H21 and a valid recovery, a new request with the new token is admitted (2377.836435ms)
✔ H23: requests pending at a restart are not resent; each shows outcome unknown (482.017907ms)
✔ a plain conversation: catalogue row, one page, CHAT-01 records (10.196509ms)
✔ native entries map to blocks: tools, thinking, bash, notices, ids that do not fit (3.091521ms)
✔ F1: a malformed line is an unavailable part at its position, and reading continues (2.999666ms)
✔ F1: a missing parent stops the history with a notice that names the unreadable lines (3.746417ms)
✔ F1: an unreadable fork is never merged into another branch's history (3.733154ms)
✔ F1: a follow stays on its branch when the next entry's parent is unreadable (3.881773ms)
✔ F1: a file whose entries are all unreadable shows a notice per line (1.563335ms)
✔ F2: a truncated trailing line marks the view incomplete, not an error (2.494119ms)
✔ pagination: 100 parts, then the rest; parts concatenate to the whole branch (5.109673ms)
✔ F3: a replaced file (new inode) refuses old cursors with reconcile (4.939351ms)
✔ F4: a same-inode rewrite of the prefix refuses old cursors with reconcile (5.516645ms)
✔ F5: growth between pages keeps the epoch and the page stops at the pinned length (5.793786ms)
✔ F6: unknown, foreign and expired cursors refuse and leave the cursor usable (10.497498ms)
✔ F7: a symlinked file and a symlinked directory component are refused, never opened (9.313922ms)
✔ F8: a file swapped for a symlink after the catalogue is refused (2.247728ms)
✔ F9: registrations never add or redirect a root (2.060491ms)
✔ F10: a header cwd naming another project is refused (3.630852ms)
✔ F11: parentSession renders with a marker and the parent is never opened (0.936984ms)
✔ F12: two leaves: the default leaf is shown and the other branch reads alone (4.789444ms)
✔ F12: a follow refuses when an appended duplicate id changes the branch's earlier parts (2.415895ms)
✔ F12: a second root (Pi's resetLeaf) starts its own branch (1.349225ms)
✔ F13: compaction is a marker in place, then the retained content (0.756245ms)
✔ F14: long strings split into fragments and parts, reassemble exactly, and pages respect the byte cap (737.947726ms)
✔ fragments never cut a surrogate pair and keep an empty string (9.768434ms)
✔ F15: a Claude seat is an unsupported-harness placeholder whose directory is never read (2.629953ms)
✔ unknown conversations, empty files and non-Pi files refuse (2.988026ms)
✔ an unreadable file or root inside the roots is refused per row, not a failed catalogue (1.388415ms)
✔ a seat directory without search permission refuses that root, not the catalogue (2.248255ms)
✔ every page and cursor is a valid CHAT-01 record (847.042088ms)
✔ the engine pin holds for the installed package (2.933293ms)
✔ pinned Pi, sealed and without credentials, answers the controller's commands with the shapes the fake models (364.994239ms)
✔ pinned Pi appends thinking_level_change at start when the branch lacks one, so the leaf moves (K8 then fails closed) (307.845583ms)
✔ N25: ordinary Interrupt reconciles; a non-empty queue_update in the window is O5 (80.815781ms)
✔ N1: an extension's follow-up queued after the fence is cleared before any abort; O5, Unknown (57.148386ms)
✔ N1: a follow-up queued before the fence is O5 at once; the Interrupt refuses fenced (24.835014ms)
✔ N2: with abort first, the fake runs the external item (the ordering guard has teeth) (21.576912ms)
✔ N3: the fence lands in preflight, preflight errors, no run: failed, No run, uncertain (42.350718ms)
✔ N4: the ack arrives after the first abort and a run starts: clear and abort again; Interrupted (37.456929ms)
✔ N5: an input handler takes the prompt: ack, no run, delivery-unknown handled-without-run (119.71406ms)
✔ N6: an extension queues between clear_queue and abort: O5 and O6, Unknown (46.194593ms)
✔ N7: clear_queue times out: no abort, nativeQueue unknown, force stop still ends it (1530.004862ms)
✔ N7: clear_queue answers an error: no abort, nativeQueue unknown, the link not poisoned (15.860593ms)
✔ N8: an extension prompt starts a run during Mosaic preflight; the losing settle is O3 (71.692215ms)
✔ N9: a run that started before the fence and ends aborted: failed interrupted, Interrupted (16.241237ms)
✔ N9: decision 34: a run that ends aborted with no stop in progress: aborted-without-stop, uncertain, outcome unknown (16.048665ms)
✔ N9: an aborted that lands after the fence but before any abort is written: aborted-without-stop, Unknown (32.548668ms)
✔ N10: fake conformance (30.957332ms)
✔ N11: the run fails before any user message_start: delivery-unknown ack-without-start, never failed (29.834953ms)
✔ N12: input that starts a run after the final empty clear is O1 and not part of the stop's proof (19.905915ms)
✔ N13: agent_start with no slot held is O1; a later prompt refuses with zero engine bytes (66.348845ms)
✔ N14: the run completes while clear_queue is in flight: finished, Completed first, uncertain (26.705932ms)
✔ N14: the run completes after the abort is written, before Pi applies it: finished, never relabelled (25.622164ms)
✔ N15: the fence lands in preflight, then an input handler takes it: handled-without-run, No run (19.804122ms)
✔ N16: Interrupt with no slot and no run refuses no-turn: no stop, no bytes, admission open (13.437887ms)
✔ N17: the run fails on its own during the exchange: failed, Failed on its own (28.095743ms)
✔ N18: no final assistant message_end, or a lost line: working stays working; before working, transport-unknown (113.466843ms)
✔ N19: a losing extension prompt settles inside the Mosaic run before its user message: O3, run-overlap (134.279618ms)
✔ N20: an extension triggerTurn during Mosaic preflight starts first; while streaming it queues with no signal (83.211293ms)
✔ N21: a losing settle after the receipt settled finished is O2; the receipt stays finished (14.999139ms)
✔ N22: an agent-level custom message is dropped by the clear with no signal; evidence names the seal (13.807672ms)
✔ N23: a nextTurn message survives clear and abort and attaches to the next prompt, with no signal (13.596018ms)
✔ N24: the seal is an allow-list: --extension, a missing --no-* flag, a second --mode or --session, a session or output flag, or a stray word refuses unsealed-engine; no engine starts (37.274093ms)
ℹ tests 152
ℹ suites 0
ℹ pass 152
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 32488.500969
@@ -0,0 +1,23 @@
✔ browser edge states: loading, empty, malformed, stale, hostile/long values, in-flight reply and appearance fallback (2364.731167ms)
Rendered contrast: {"failures":[],"count":330,"lowest":4.504658476260286}
✔ served Console browser: real board fixtures, keyboard, drafts, receipts, themes, 320px and failures (2725.60223ms)
✔ conversation view: full history, collapsed tools, hidden thinking, inert hostile content, malformed and reconcile markers (2335.250549ms)
✔ conversation view: a fork keeps the open branch, says so, and opens the new one on request (1375.477504ms)
✔ conversation view: a newer session with no readable history keeps the marker (1002.908949ms)
✔ conversation view: seats without history say so and offer no reply (608.42244ms)
✔ Discord row through real board/WebUI: independent brake/liveness, no Reply, literal content (1957.413637ms)
✔ return flow through the conversation view: send, tool call, delayed result, peer message, exact long answers, relaunch (53771.943793ms)
✔ both presentations replace old activity with relaunch notice, label retained history, then resume after new activity (1987.210959ms)
✔ reported return flow and relative Age: reply sent from the inspector, then the new answer appears there without manual refresh (21912.250845ms)
✔ loopback host and board origin fail closed (7.055591ms)
✔ real board fixture passes through WebUI; assets and isolated seen/reply work (100.024626ms)
✔ proxy preserves exact request bytes, status and receipt, rejects forms and malformed JSON, never follows redirect (227.262245ms)
✔ unreachable board reports URL; CLI rejects unsupported options (1632.267153ms)
ℹ tests 14
ℹ suites 0
ℹ pass 14
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 54170.048795
@@ -0,0 +1,2 @@
2026-10-09T13:05:51Z
08:05:51 up 33 days, 9:40, 5 users, load average: 4.03, 7.84, 5.58
@@ -0,0 +1,61 @@
// Row 46 receipt probe. Runs the fixture's ignoreTerm child inside a
// delegated systemd user scope, sends it SIGTERM, and records the scope's
// `systemctl --user show`, `cgroup.procs` before and after TERM, and the
// child's exit code and signal.
//
// node scope-receipt.mjs <mode> <delayMs> <runs>
//
// mode `race`: TERM goes `delayMs` after spawn, as the fixture does today.
// mode `ready`: TERM goes `delayMs` after the child reports its handler is
// installed, as the fixed fixture does.
// The outer process launches one scope per run; the inner process
// (`--inner`) is the scope's main process and prints one JSON line.
import { spawn, spawnSync } from "node:child_process";
import { readFileSync } from "node:fs";
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
// The fixture's child code for { ignoreTerm: true } with no pidLog
// (packages/conversation/tests/fake-pi.mjs, spawnChild), plus a readiness
// byte on stdout in `ready` mode.
const childCode = (ready) =>
"const note=()=>{};process.on('SIGTERM',()=>note('term'));" +
(ready ? "process.stdout.write('r');" : "") +
"setInterval(()=>{},1e9)";
const procs = (cg) => readFileSync(`/sys/fs/cgroup${cg}/cgroup.procs`, "utf8").split("\n").filter(Boolean).map(Number);
async function inner(mode, delayMs, unit) {
const cg = readFileSync("/proc/self/cgroup", "utf8").trim().split("::")[1];
const ready = mode === "ready";
const t0 = performance.now();
const child = spawn(process.execPath, ["-e", childCode(ready)], { stdio: ["ignore", ready ? "pipe" : "ignore", "ignore"] });
const exited = new Promise((r) => child.on("exit", (code, signal) => r({ code, signal, atMs: Math.round(performance.now() - t0) })));
let readyMs = null;
if (ready) {
await new Promise((r) => child.stdout.once("data", r));
readyMs = Math.round(performance.now() - t0);
}
await sleep(delayMs);
const show = spawnSync("systemctl", ["--user", "show", "-p", "Id,LoadState,ActiveState,SubState,InvocationID,ControlGroup,Delegate", `${unit}.scope`], { encoding: "utf8" }).stdout.trim().split("\n");
const before = procs(cg);
const termMs = Math.round(performance.now() - t0);
child.kill("SIGTERM");
const exit = await Promise.race([exited, sleep(500).then(() => null)]);
const after = procs(cg);
if (!exit) child.kill("SIGKILL");
console.log(JSON.stringify({ mode, delayMs, readyMs, termMs, child: child.pid, self: process.pid, show, before, after, childInBefore: before.includes(child.pid), childInAfter: after.includes(child.pid), exit: exit ?? "alive 500 ms after TERM" }));
}
async function outer(mode, delayMs, runs) {
for (let i = 0; i < runs; i++) {
const unit = `dw-r46-${mode}-${delayMs}-${process.pid}-${i}`;
const r = spawnSync("systemd-run", ["--user", "--scope", "--quiet", "-p", "Delegate=yes", `--unit=${unit}`, "--", process.execPath, import.meta.filename, "--inner", mode, String(delayMs), unit], { encoding: "utf8" });
process.stdout.write(r.stdout || `{"unit":"${unit}","status":${r.status},"stderr":${JSON.stringify(r.stderr)}}\n`);
}
}
const a = process.argv.slice(2);
if (a[0] === "--inner") await inner(a[1], Number(a[2]), a[3]);
else await outer(a[0], Number(a[1]), Number(a[2] ?? 5));
@@ -0,0 +1,9 @@
{"t":1791551027129,"ev":"spawn","pid":3646877,"ignoreTerm":true,"setsid":true}
{"t":1791551027142,"ev":"term","pid":3646862}
{"t":1791551027142,"ev":"term","pid":3646877}
{"t":1791551029516,"ev":"spawn","pid":3647263,"ignoreTerm":true,"setsid":false}
{"t":1791551029528,"ev":"term","pid":3647248}
{"t":1791551029528,"ev":"term","pid":3647263}
{"t":1791551031873,"ev":"spawn","pid":3647581,"ignoreTerm":true,"setsid":false}
{"t":1791551031886,"ev":"term","pid":3647556}
{"t":1791551031887,"ev":"term","pid":3647581}
@@ -0,0 +1,13 @@
{"t":1791550812775,"ev":"spawn","pid":3607494,"ignoreTerm":true,"setsid":true}
{"t":1791550812853,"ev":"ready","pid":3607494}
{"t":1791550812878,"ev":"term","pid":3607446}
{"t":1791550812878,"ev":"term","pid":3607494}
{"t":1791550815693,"ev":"spawn","pid":3607807,"ignoreTerm":true,"setsid":false}
{"t":1791550815773,"ev":"ready","pid":3607807}
{"t":1791550815779,"ev":"term","pid":3607781}
{"t":1791550815780,"ev":"term","pid":3607807}
{"t":1791550819446,"ev":"spawn","pid":3608344,"ignoreTerm":true,"setsid":false}
{"t":1791550819489,"ev":"ready","pid":3608344}
{"t":1791550819556,"ev":"term","pid":3608301}
{"t":1791550819556,"ev":"term","pid":3608344}
{"t":1791550820058,"ev":"term","pid":3608344}
@@ -0,0 +1,13 @@
{"t":1791550802745,"ev":"spawn","pid":3605642,"ignoreTerm":true,"setsid":true}
{"t":1791550802763,"ev":"ready","pid":3605642}
{"t":1791550802798,"ev":"term","pid":3605597}
{"t":1791550802798,"ev":"term","pid":3605642}
{"t":1791550805434,"ev":"spawn","pid":3606162,"ignoreTerm":true,"setsid":false}
{"t":1791550805456,"ev":"ready","pid":3606162}
{"t":1791550805492,"ev":"term","pid":3606153}
{"t":1791550805492,"ev":"term","pid":3606162}
{"t":1791550808090,"ev":"spawn","pid":3606502,"ignoreTerm":true,"setsid":false}
{"t":1791550808106,"ev":"ready","pid":3606502}
{"t":1791550808199,"ev":"term","pid":3606487}
{"t":1791550808200,"ev":"term","pid":3606502}
{"t":1791550808539,"ev":"term","pid":3606502}
@@ -0,0 +1,43 @@
diff --git a/packages/conversation/src/shim.mjs b/packages/conversation/src/shim.mjs
index 2adbe48b..d51c4a1f 100644
--- a/packages/conversation/src/shim.mjs
+++ b/packages/conversation/src/shim.mjs
@@ -22,7 +22,7 @@
// `populated 0`. A missing or unreadable file is unavailable, never empty.
import { spawn } from "node:child_process";
-import { closeSync, mkdirSync, readdirSync, readFileSync, unlinkSync, writeFileSync } from "node:fs";
+import { appendFileSync, closeSync, mkdirSync, readdirSync, readFileSync, unlinkSync, writeFileSync } from "node:fs";
import { createServer } from "node:net";
import { join } from "node:path";
import { LineSplitter, encodeLine, parseLine } from "./framing.mjs";
@@ -136,6 +136,7 @@ async function handle(req) {
if (startOf(pid) !== startTicks) continue;
try {
process.kill(pid, "SIGTERM");
+ if (process.env.DW_TRACE) appendFileSync(process.env.DW_TRACE, JSON.stringify({ t: Date.now(), ev: 'term', pid }) + '\n');
signalled.push(pid);
} catch {
// gone already
diff --git a/packages/conversation/tests/fake-pi.mjs b/packages/conversation/tests/fake-pi.mjs
index b3014f86..f30f5a4c 100644
--- a/packages/conversation/tests/fake-pi.mjs
+++ b/packages/conversation/tests/fake-pi.mjs
@@ -615,13 +615,16 @@ const children = [];
// K2); `forkLoop` forks every 5 ms (K12); `ignoreTerm` survives SIGTERM, so
// only the kill phase ends it (K3, K10, K11). With `pidLog`, the fork loop
// appends each child's pid and a `term` line when it gets SIGTERM (K12).
+import * as __fs from 'node:fs';
+const require0 = () => __fs;
function spawnChild({ setsid = false, forkLoop = false, ignoreTerm = false, pidLog = null } = {}) {
const note = pidLog ? `const note=(s)=>require('node:fs').appendFileSync(${JSON.stringify(pidLog)},s+'\\n');` : "const note=()=>{};";
- const code = note + (ignoreTerm ? "process.on('SIGTERM',()=>note('term'));" : "") + (forkLoop
+ const code = note + (ignoreTerm ? "process.on('SIGTERM',()=>note('term'));" + (process.env.DW_TRACE ? `require('node:fs').appendFileSync(${JSON.stringify(process.env.DW_TRACE)},JSON.stringify({t:Date.now(),ev:'ready',pid:process.pid})+'\\n');` : "") : "") + (forkLoop
? "const {spawn}=require('node:child_process');setInterval(()=>{try{const c=spawn('sleep',['1000'],{stdio:'ignore'});if(c.pid)note(String(c.pid))}catch{}},5);setInterval(()=>{},1e9)"
: "setInterval(()=>{},1e9)");
const child = spawn(process.execPath, ["-e", code], { stdio: "ignore", detached: setsid });
children.push(child.pid);
+ if (process.env.DW_TRACE) { require0().appendFileSync(process.env.DW_TRACE, JSON.stringify({ t: Date.now(), ev: 'spawn', pid: child.pid, ignoreTerm, setsid }) + '\n'); child.on('exit', (code, sig) => require0().appendFileSync(process.env.DW_TRACE, JSON.stringify({ t: Date.now(), ev: 'child-exit', pid: child.pid, code, sig }) + '\n')); }
return child.pid;
}