Compare commits

...
Author SHA1 Message Date
jason.woltje e9cfd92f89 row 250 (Vikunja #450): README lists R13 for cohort.test.mjs (sage-2082) 2026-10-21 14:14:17 -05:00
jason.woltje 156dce5193 row 250 (Vikunja #450): K12 holds the stop's release while it reads the shim; release answers the events read's reason
K12 read engine/cgroup.freeze, events and members after the binding went
stopped. Since row 50 (#1536) the stop then releases the scope, and the
shim exits 10 ms after release, so under load the read found the shim gone
and populated undefined. K12 now holds scope-release until its reads are
done, as E1 does, then asserts the release. The assertions are unchanged.

The shim's release answered an unreadable cgroup.events with "the engine
cgroup is not empty". It now returns the read's own unavailable reason.
R13 covers it: with cgroup.events mode 000 after a proven stop, events is
unavailable with EACCES, and the controller records the release
unavailable with that reason and keeps the scope.
2026-10-21 14:11:31 -05:00
jason.woltje cbba6ff07a row 250 (Vikunja #450): K12 names the cause when the events read fails
The populated assertion prints the shim's answer, with ok and unavailable,
and the controller's release entries. Step 1 of the brief, on its own.
2026-10-21 14:09:57 -05:00
4 changed files with 140 additions and 4 deletions
+89
View File
@@ -0,0 +1,89 @@
# Row 250 build notes (Gizmoduck)
Vikunja #450, Mos ruling mos-1182 item 3. Brief: docs/plans/2026-10-21_k12-cohort-events.md. Reviewer
Filbert, gate owner Sage. Branch gizmoduck/row250 from origin/refactor 3b0aef995. Line numbers below are on
this branch unless a commit is named.
## Step 1
cbba6ff07 changes only the K12 assertion (packages/conversation/tests/cohort.test.mjs:292 on 3b0aef995).
On failure it prints the shim's answer, with `ok` and `unavailable`, and `h.ctrl.evidence.releases`.
## Step 2: the cause
K12 reads the shim after the controller has started to release it.
- The force stop sets the binding `stopped` (packages/conversation/src/controller.mjs:1615). Then it pauses at
`scope-release` (:1623) and calls `#releaseScope` (:1624).
- `#releaseScope` calls `releaseCohort` (:1639). That sends `hello`, then `release`
(packages/conversation/src/cohort.mjs:261-265).
- The shim answers `release` and exits 10 ms later. It unlinks its socket first (shim.mjs:165-176). systemd
then collects the empty scope, `engine` included.
- K12's `forceStop` helper returns once it sees `stopped` (cohort.test.mjs:148, polling every 10 ms). The
test then reads `engine/cgroup.freeze` (:281 on 3b0aef995) and asks the shim for `events` (:291).
Nothing in K12 orders those reads before the release. Unloaded, the test usually sends its `events` request
within the 10 ms before the shim exits, so it passes. Under load the release wins. `shimRequest` then
returns `{ ok: false, unavailable: "the shim socket is gone" }` (cohort.mjs:46), or "unreachable" or
"closed the connection" (:62-63), and `ev.populated` is undefined. Both recorded failures fit this
pattern:
- Row 197 proof 4, pair1-b: K12 failed in 339 ms on `undefined !== 0` at :292. 339 ms is far below the
3 s `shimRequest` timeout (cohort.mjs:56), so the request didn't time out. The `cgroup.freeze` read at
:281 passed, so the scope still existed then.
- BUILD-LOG.md:4092, row 56's gate: "`populated` undefined with load around 12".
Row 50 (a9cc522a0, #1536) added the release after a proven stop. K12 is older. Its comment still says
"the shim holds the scope", and that stopped being true in row 50. E1 (cohort.test.mjs:923-925) already
holds `scope-release` so it can read the shim while the shim still answers. K12 never got the hold. No
other test reads the shim after a proven stop and expects an answer. R1 expects the shim gone, R3 is an
uncertain stop, and K15 reads before its stop.
The cgroup read itself has no race. The kill's `waitFor` (shim.mjs:117-126) had already read `populated 0`
for the stop to be proven. The read is the shim's call to `events()` (shim.mjs:83-96), and it reads one
kernel file. So the brief's allowance for a bounded retry doesn't apply, and there is none.
The fix goes in the test's setup, which the brief allows ("Use the shim, the cohort controller or the
test's setup and teardown, whichever the evidence points to"). The controller's release is correct and
specified (README "Scope release", R1). Delaying it for a reader would change row 50's behavior.
## Step 2: the change
packages/conversation/tests/cohort.test.mjs, K12:
- It runs with `gate()`, holds `scope-release` before the stop, and waits until the hold is reached. Then it
does every read that needs the scope: `cgroup.freeze`, `events` and `members`.
- After the reads it lets the release go and asserts one entry, `["force-stop", "released"]`. This is new
coverage: K12 now also shows the forking cohort's scope released.
- The assertions are unchanged. `populated` must be 0 and the member list must be empty. The step 1 message
stays.
packages/conversation/src/shim.mjs:167-168, `release` (:167 on 3b0aef995). It answered a failed `events()` with "the engine
cgroup is not empty", so it reported unavailable as not empty. The refusal was already safe, but the reason
was false. Now an `events()` answer that isn't `ok` is returned as it is, with the read's own reason. Only
`populated` 1 gives "not empty". Nothing else matched that string (grep of packages/).
## Tests
- K12 as above.
- R13, the brief's new case, placed after R3. After a proven stop, with `scope-release` held, the test makes
`engine/cgroup.events` mode 000. Then:
- `events` returns `ok: false`, has no `populated`, and the reason is "engine/cgroup.events unreadable
(EACCES)".
- On release, the controller records one entry `["force-stop", "unavailable", <that reason>]`.
- The unit stays active, and the shim still answers `hello`.
The `finally` restores mode 444, the kernel's mode for cgroup.events, and releases the scope. On
3b0aef995's shim.mjs, R13 fails on the reason, because it reads "the engine cgroup is not empty".
- K15 already covers a stop, as opposed to a release, with `events` unavailable: the shim gone, the file
mode 000, or `engine` missing, each ending `uncertain` with its reason.
## File added beyond the brief
packages/conversation/README.md:599 listed "R1–R3, R5–R12" for cohort.test.mjs. With R13 it now reads
"R1–R3, R5–R13". The brief's files don't include the README. Sage added it to this row in sage-2082.
## Runs
None yet. They need Sage's slot. Sage's sage-2082 says the run is on the pushed candidate in a clean
worktree, and that it adds the control-board suite whole, so it counts as the gate run.
+1 -1
View File
@@ -596,7 +596,7 @@ no prompt:
| `claim.test.mjs` | W1–W17, W20, G1–G3: the writer claim, crash barriers, the guard |
| `races.test.mjs` | H1–H4, H9–H23: takeover, Interrupt and force stop, retries, incarnations |
| `turns.test.mjs` | N1–N25, N24b: the turn tracker against the fake engine's Pi behaviors; the engine seal and environment |
| `cohort.test.mjs` | K1–K19, R1–R3, R5–R12, E1–E6: scopes, force stop, proofs, recovery, eligibility, the scope's environment, scope release and its retry after a crash, the engine-exit stop (needs a systemd user manager) |
| `cohort.test.mjs` | K1–K19, R1–R3, R5–R13, E1–E6: scopes, force stop, proofs, recovery, eligibility, the scope's environment, scope release and its retry after a crash, the engine-exit stop (needs a systemd user manager) |
| `flows.test.mjs` | S1–S7, P3, E1–E7, the terminal, and a CHAT-01 schema check of every record produced |
| `smoke.test.mjs` | the pinned Pi binary, as above |
+2 -1
View File
@@ -164,7 +164,8 @@ async function handle(req) {
return waitFor((e) => e.populated === 0, Number(req.timeoutMs) || 5000);
case "release": {
const e = events();
if (!e.ok || e.populated !== 0) return { ok: false, unavailable: "the engine cgroup is not empty" };
if (!e.ok) return e;
if (e.populated !== 0) return { ok: false, unavailable: "the engine cgroup is not empty" };
setTimeout(() => {
try {
unlinkSync(socketPath);
+48 -2
View File
@@ -265,14 +265,19 @@ test("K5: a stop during a tool call leaves the effect uncertain, and it is shown
// every child the loop forked after its SIGTERM, which nothing else ends
// before the kill, is in the proof's list. Mutants r2-B5 (no freeze write,
// `frozen 1` answered) and r2-B5b (freeze written, not waited on) fail here.
// The stop's own release ends the shim and the scope (#1536), so the test
// holds it until the reads are done (row 250).
test("K12: a member forking in a loop: the freeze stops it, enumeration is complete, populated 0 after cgroup.kill", NEEDS_SCOPE, async () => {
const h = await live();
const g = gate();
const h = await live({ barrier: g.barrier });
try {
const pidLog = join(h.fx.base, "fork-pids.log");
const forker = await childOf(h, { forkLoop: true, ignoreTerm: true, pidLog });
await until(async () => (await memberPids(h.rec().shim)).length >= 6, 4000, "the fork loop");
const engineDir = join("/sys/fs/cgroup", (await shimRequest(h.rec().shim, "hello")).scope, "engine");
g.hold("scope-release");
await forceStop(h);
await g.waitHeld("scope-release");
assert.equal(h.ctrl.binding.state, "stopped", JSON.stringify(h.ctrl.evidence.stops));
const { members } = h.ctrl.stoppedProof.proof;
assert.ok(members.some((m) => m.pid === forker));
@@ -289,9 +294,14 @@ test("K12: a member forking in a loop: the freeze stops it, enumeration is compl
assert.ok(listed.has(pid), `child ${pid} forked after TERM is in the proof's list`);
}
const ev = await shimRequest(h.rec().shim, "events");
assert.equal(ev.populated, 0);
// On failure, name the cause: the answer with `ok` and `unavailable`, and any release already run (row 250).
assert.equal(ev.populated, 0, JSON.stringify({ ev, releases: h.ctrl.evidence.releases }));
assert.deepEqual(await memberPids(h.rec().shim), []);
g.release("scope-release");
await until(() => h.ctrl.evidence.releases.length > 0, 8000, "the release");
assert.deepEqual(h.ctrl.evidence.releases.map((r) => [r.why, r.outcome]), [["force-stop", "released"]]);
} finally {
g.release("scope-release");
await h.close();
}
});
@@ -852,6 +862,42 @@ test("R3: no release after an unavailable stop or an unverified proof: the scope
}
});
// `release` reads engine/cgroup.events again. An unreadable file answers with
// the read's own reason, never "not empty", and the controller records the
// release unavailable with that reason (row 250).
test("R13: a release with engine/cgroup.events unreadable answers the read's reason; the controller records it unavailable and the scope stays", NEEDS_SCOPE, async () => {
const g = gate();
const h = await live({ barrier: g.barrier });
const unit = h.rec().unitName, shim = h.rec().shim;
let events = null;
try {
g.hold("scope-release");
await forceStop(h);
await g.waitHeld("scope-release");
assert.equal(h.ctrl.binding.state, "stopped", JSON.stringify(h.ctrl.evidence.stops));
events = join("/sys/fs/cgroup", (await shimRequest(shim, "hello")).scope, "engine", "cgroup.events");
chmodSync(events, 0o000);
const ev = await shimRequest(shim, "events");
assert.equal(ev.ok, false, JSON.stringify(ev));
assert.equal("populated" in ev, false, JSON.stringify(ev));
assert.equal(ev.unavailable, "engine/cgroup.events unreadable (EACCES)");
g.release("scope-release");
await until(() => h.ctrl.evidence.releases.length > 0, 8000, "the release");
assert.deepEqual(h.ctrl.evidence.releases.map((r) => [r.why, r.outcome, r.reason]), [["force-stop", "unavailable", ev.unavailable]]);
assert.ok(unitActive(unit), "the scope stays up");
assert.equal((await shimRequest(shim, "hello")).ok, true, "the shim got no release");
} finally {
g.release("scope-release");
if (events) {
chmodSync(events, 0o444);
await releaseCohort({ kind: "scope", unitName: unit, invocationId: h.rec().invocationId, shimSocket: shim });
}
h.c.close();
h.fake.close();
reap(h.fx);
}
});
// A normal engine exit leaves the shim and the scope: the shim exits only on
// `release`. The controller doesn't release then: the binding is uncertain
// and the scope is what a force stop reads. The proven stop releases it.