Compare commits

...
Author SHA1 Message Date
jason.woltje 2a8281a6f0 gizmoduck: row 302 (#1809) gate results: run 1 stopped on launcher-released.test:89 (d6, mos-1430); run 2 on merge 783602e06 (5b63ea323 onto 1ac655005) red as ruled, suites green, M1-M5 M7-M9 killed; RESULTS.md and logs, packet only 2026-10-23 03:39:20 -05:00
jason.woltje 5b63ea3236 gizmoduck: row 302 (#1809) d6: launcher-released.test:89 counts only a-run's delivery rows and pins the one routed row with holder_run null; #route writes that row for every message (gizmoduck-0388, sage-3294, sage-3296, Mos ruling pending sage-3293); local, not pushed 2026-10-23 03:22:31 -05:00
jason.woltje 64f6558ee4 row 302 (#1809, Vikunja 588): a run threadIdle or landRelease released answers launcher-released on every call needing the claim until it ends, role.claim refuses it (Dewey D1), and the runner exits stopped without releasing again 2026-10-23 00:50:20 -05:00
jason.woltje f799e8b155 row 302 (#1809, Vikunja 588): red first, the final forcing tests and the four pins Mos ruled (mos-1414/1415/1416), over unfixed product files; BUILD.md d3-d5 and ASSERTION-PRESERVATION 2026-10-23 00:50:20 -05:00
jason.woltje 4992e49097 gizmoduck: row 302 design d2: threads.test pins are :392/:394/:395 (sage-3204), land-relaunch grep finds land.test:464 as a third old pin under Q1 2026-10-23 00:33:32 -05:00
jason.woltje e86c712f5c gizmoduck: row 302 BUILD.md folds Dewey's d1 approval (dewey-0875): D1 role.claim refuses a released run, landRelease reaches runner.test:742-772 so Q1 moves :771-772, Q2-Q4 and N1-N4 settled 2026-10-23 00:32:50 -05:00
jason.woltje 87e67a765f gizmoduck: row 302 (#1809, Vikunja 588) idle-stop race design round d1 for Dewey: idle-released code from the broker's own record of threadIdle's release, runner maps it to stopped/0, forcing cli test holds the window open until a call lands, threads.test:393 is the one changed pin (sage-3200, mos-1411) 2026-10-23 00:29:08 -05:00
17 changed files with 1512 additions and 24 deletions

No files matched your search

@@ -0,0 +1,111 @@
# Row 302 (#1809): assertion preservation
Base 0e30a85fa. `git diff 0e30a85fa -- 'packages/*/tests/*'` on an existing file changes four lines
and nothing else. Each one is ruled by Mos (mos-1414, amended by mos-1415 and mos-1416, relayed in
sage-3206, sage-3209 and sage-3210). No assertion is removed or loosened. Each one now pins the new
code or exit for the same call.
## 1. packages/bus/tests/threads.test.mjs:394 (mos-1414 a)
Old:
```
assert.throws(() => call('a', 'message.receive'), { code: 'not-holder' });
```
New:
```
assert.throws(() => call('a', 'message.receive'), { code: 'launcher-released' });
```
:392 (threadIdle's own second refusal, not-holder) and :395 (the pending check) are unchanged.
## 2. packages/harness/tests/runner.test.mjs:771 (mos-1415)
Old:
```
assert.equal(out.code, EXIT.session, out.stderr);
```
New:
```
assert.equal(out.code, 0, out.stderr);
```
out.stderr stays as the message.
## 3. packages/harness/tests/runner.test.mjs:772 (mos-1415)
Old:
```
assert.match(out.stderr, new RegExp(`runner: landing turn 1 for decision ${d1.id} \\(vikunja:1/2\\): ok\n[\\s\\S]*runner: session stopped working: not-holder\n`));
```
New:
```
assert.match(out.stderr, new RegExp(`runner: landing turn 1 for decision ${d1.id} \\(vikunja:1/2\\): ok\n[\\s\\S]*runner: session released by the launcher\n`));
```
Only the last line of the pattern changes. The landing-turn part is the same, word for word.
## 4. packages/bus/tests/land.test.mjs:464 (mos-1416)
Old:
```
assert.throws(() => w.call('merge-gate', 'message.receive'), /not-holder/);
```
New:
```
assert.throws(() => w.call('merge-gate', 'message.receive'), /launcher-released/);
```
:457, :463 and :466 are unchanged.
## The not-holder grep, rerun on the build (mos-1414)
`git grep -n "not-holder\|session stopped working\|session-invalid"` over packages/cli, seat and harness
tests and scripts/, outside the new files:
- runner.test.mjs:558, `session stopped working: unauthenticated`. Not a release. Stays.
- runner.test.mjs:628 (a comment) and :647, the limit stop. The limit revoke isn't threadIdle or
landRelease, so it stays not-holder and exit 22 (Dewey Q3).
- session.test.mjs:116, a list of end reasons. No call.
In packages/bus/tests, only end-launch, land and threads call threadIdle or landRelease. Their other
not-holder pins: end-launch:198, threads:383-385 and :392, land:457 and :463 are the ops' own refusals.
threads:493 follows assistant.stop. threads:416 releases a2-run as the test's last line. The fix reaches
none of them, so no hit goes to Mos.
## 5. packages/bus/tests/launcher-released.test.mjs:89, a new test's own line (d6, sage-3294, Mos ruling sage-3293)
This file is new in row 302, so nothing at the base changes. The line changed between the pushed tests
commit f799e8b15 and d6, after the first gate run went red on it (gizmoduck-0388). It's recorded here
because the change narrows what the line counts.
Old:
```
assert.equal(store.get('SELECT count(*) AS n FROM deliveries WHERE message=?', later.id).n, 0, 'nothing delivered to the released run');
```
New:
```
assert.equal(store.get('SELECT count(*) AS n FROM deliveries WHERE message=? AND holder_run=?', later.id, 'a-run').n, 0, 'nothing delivered to the released run');
assert.deepEqual(store.all('SELECT op, holder_run FROM deliveries WHERE message=?', later.id).map((r) => ({ ...r })), [{ op: 'routed', holder_run: null }]);
```
The old line counted every delivery row for the message. `#route` (broker.mjs:6482) writes one routed
row for each message, and `#recipient` (:832-834) gives it holder_run null when no run holds the role.
After threadIdle no run holds thread A, so the count was 1, with no row for a-run. The first new line
keeps the claim it was meant to make: no row names the released run. The second pins the whole set,
one routed row held by nobody. That is tighter than the old line on everything except the null-holder
row, which it requires.
+444
View File
@@ -0,0 +1,444 @@
# Row 302 (#1809, Vikunja 588): the idle-stop race
Owner Gizmoduck. Reviewer Dewey. Gate owner Sage. Brief:
docs/plans/2026-10-23_idle-stop-race-row.md, "Brief: the idle-stop race (Vikunja 588)". Base 0e30a85fa.
This is the design, written static only. Nothing gets built until Dewey approves it (sage-3200).
## The race, as the code has it at 0e30a85fa
- `packages/cli/src/launcher.mjs:703-717`. The idle tick calls the `threadIdle` op. On `released` it adds
the run to `stopping`, records `thread-idle`, and only then sends SIGTERM (:716).
- `packages/bus/src/broker.mjs:1044-1057`. `threadIdle` releases the claim in the same synchronous call
(`#claimEnd(..., 'release', null)`, :1055). The comment at :1040-1043 says the order is deliberate:
row 121 r2, Dewey's F3 in 28591.
- `broker.mjs:1002-1008`. `#agent` is the only place a thread run's own call gets `not-holder`, because
its session's latest claim row isn't a `claim` of its generation. `message.receive` (:5586-5591) and
`state` (:5805-5807) both go through it, and so do `message.send` and `message.read`.
- `packages/harness/src/runner.mjs:55`. `SESSION_GONE` holds `unauthenticated`, `not-holder` and
`run-ended`. Four sites set `code = EXIT.session` (22) on any of them, without checking `stopping`:
the landing `state` (:500-506), `message.receive` (:526-531), the turn's `state` (:545-552) and the
reply (:575-582).
- `packages/seat/src/session.mjs:340` maps 22 to `session-invalid`.
So if a runner call reaches the broker after the release and before SIGTERM sets `stopping`, the run
exits 22 by itself and ends `session-invalid`. A call already in flight when the release lands counts
too. Row 292's diagnostic saw this once in about 100 idle stops. The clear test's run r846b8d083502 logged
`not-holder` and then `exiting 22`.
## Design: a distinct answer for a run the idle stop released (Mos's suggestion, adopted)
### Broker (`packages/bus/src/broker.mjs`)
1. Add a new in-memory set, `#idled`, keyed `business:run`, next to `#ended`, `#received` and `#stops`.
`threadIdle` adds the run's key in the same synchronous call that releases it, and only when it
returns `{released:true}`. Release and mark happen together, so no call can be answered between them.
2. In `#agent`, the refusal becomes
`fail(this.#idled.has(s.business + ':' + s.run) ? 'idle-released' : 'not-holder')`. Nothing else in
`#agent` changes. A run the idle stop released now gets `idle-released` at every call after the
release, whether it's a receive, `state`, a send or a read, and whether or not another run has
claimed the session since. Every other path that takes a claim away still answers `not-holder`:
- `assistant.stop` (:5975), `instance.end` (:6009), the chief lifecycle revoke (:4682) and
`role.revoke` (:5230, :5235)
- the session clear (:1146)
- the limit stop (`#limitStop`, :4842-4855). It releases a thread claim and then refuses
`not-holder` on purpose, so that the run exits 22.
- another run's newer generation, where the run wasn't idle-released
3. `endLaunch` (:1300) drops the key along with `#ended.add`. After the end, the run's caps are gone and
its calls answer `unauthenticated`, as they do today. The set lives only in memory, like `#runs`. A
broker that restarts loses `#runs` too, so an old run's calls answer `unauthenticated` there either
way.
4. `threadIdle`'s own refusals don't change. A second `threadIdle` on a released run is still
`not-holder` (threads.test:392). The answer comes from the `#agent` path only, for the run's own
calls.
Why a new code and not a flag on `not-holder`: the wire carries only `error: <code>`
(server.mjs:96-160), and the client accepts only `^[a-z-]{1,64}$` (client.mjs:42-43). A new code changes
neither of them. Tools and docs that switch on `not-holder` keep their meaning, because `not-holder`
still means "your claim was taken away".
### Runner (`packages/harness/src/runner.mjs`)
5. A new constant, `IDLE_RELEASED = "idle-released"`. It is not added to `SESSION_GONE`. At each of the
four sites, `idle-released` is checked before `SESSION_GONE`. The runner logs
`runner: session released by the launcher's idle stop`, leaves `code` at `EXIT.stopped` (0), sets a
local `released`, and leaves the loop the same way the site does for `SESSION_GONE` today.
6. The `role.release` at exit (:589-595) is skipped when `released`. The claim is already released, so
the call would only be refused and logged.
7. If the runner didn't handle the code, the receive loop would count `idle-released` as an
unreachable broker and retry (:532-537). The run would then end at SIGTERM (0), or after 30 polls
(23). Every site handles it so that neither of those decides how the run ends.
How this meets mos-1411 condition 1: `unauthenticated` (the caps went at the end), `run-ended`, and every
`not-holder` that isn't an idle release still exit 22 and end `session-invalid`. Only the broker's own
record that `threadIdle` released this run maps to `stopped/0`. The launcher's `stopping` set and SIGTERM
play no part in the mapping. That also covers a poll that lands before the signal, and a call that was
in flight when the release landed.
### Launcher and session.mjs
8. No change. The runner exits 0 by itself, and SIGTERM's start-time check (:716) then finds no process,
which the code already handles. `endReason(0)` is `stopped`.
## Tests
### The forcing test (mos-1411 condition 2): new file `packages/cli/tests/idle-stop-race.test.mjs`
This is a thread launch on the same fixture shape as thread-launch.test.mjs (`start(t, prepared, { ops })`
replaces launch ops, :78-101), with `sessionDefaults.pollInterval` set to 100. The test replaces the
`threadIdle` op:
```js
threadIdle: async (m) => {
const r = runtime.threadIdle(m.record);
if (r.released) window.set(m.record.run, await held(m.record.run)); // returns only after the runner's own exit, or 20 s
return r;
}
```
`held(run)` waits for two things, in order. First, an `action.refused` event whose actor is `run` and
whose seq is after the release's `thread_claims` row. That proves a runner call reached the broker inside
the window. Second, the runner's own exit (the run's `runner: exiting N` line in runner.log, or its pid
gone), with no SIGTERM sent. The launcher sends SIGTERM only after the op returns, so the window stays
open until the runner has decided how it ends. The test then asserts all of the following:
- `held` saw a refusal from the run after the release, and the runner exited before the op returned.
- The refusal's code is `idle-released`.
- The run's launch `end` event is `['stopped', 0, null]`.
- runner.log has the idle-released line and no `session stopped working:` line.
Against the code today, the refusal is `not-holder`, the runner exits 22 inside the window, and the end
is `['session-invalid', 22, null]`. The test fails, and fails every time, not at some rate: the
window is held open until a call has landed in it. If the runner fix is dropped, the runner retries and
doesn't exit inside the window, so `held` times out and the "exited before the op returned" assertion
fails. If the broker fix is dropped, the end is 22.
A control test in the same file: after an idle release, a `role.revoke` the human approved on another
thread's run (or `assistant.stop`) still ends that run `['session-invalid', 22, null]`. That shows a
real not-holder still exits 22.
### Bus: new tests in `packages/bus/tests/idle-release.test.mjs` (a new file, so no existing assertion moves)
- After `threadIdle` releases a-run, a-run's `message.receive`, `state`, `message.send` and
`message.read` each refuse `idle-released`. Each writes an `action.refused` event with code
`idle-released`, and none of them delivers a message or writes a receipt.
- `not-holder` still comes back for: a run released by `assistant.stop`, a run released by a session
clear, a limit stop (stop then receive), a run whose session another run claimed without any idle
release, and a second `threadIdle` on a released run. After `endLaunch`, the idle-released run's
calls get `unauthenticated`, as today (its caps go at :1310-1311).
- A role run's `role.release` followed by its own receive still gets `not-holder` (no idle).
### Harness: no new runner.test case
runner.test.mjs's `setup` (:105) binds role runs only, and its claims sit in `role_claims` (:876), so it
can't reach `threadIdle`. The runner's own exit at receive is covered by the cli forcing test above,
which runs the real runner. The forcing test reaches only the site that lands in the window, which in
practice is the receive poll. The landing `state`, the turn's `state` and the reply sites get the same
four lines, but no test forces each one. Q4 asks whether Dewey wants a per-site case. The existing 22
tests (:539-560, :630-647, :772) stay unchanged and pass unchanged.
## The one existing assertion that changes (goes to Mos through Sage, per the brief)
`packages/bus/tests/threads.test.mjs:394`,
`assert.throws(() => call('a', 'message.receive'), { code: 'not-holder' })`. This is a receive by the run
that `threadIdle` released at :388, so under this design it answers `idle-released`. The assertion is
changed, not loosened: the new pin is `{ code: 'idle-released' }`, and the property right below it
(:395, the message stays `pending` for the next holder) stays. I've found no other existing assertion
that pins `not-holder` for an idle-released run's own call. The build reruns
`git grep -n "not-holder" 'packages/*/tests/*'` against every idle path before in-review, and names any
other hit for Mos first.
## Docs
- packages/bus/README.md:320-326 (threadIdle). Add: after a release, the run's own calls answer
`idle-released`, not `not-holder`. Also say why.
- packages/harness/README.md:295-302, the exit table. Row 0 becomes "stopped, by a signal, or because the
launcher's idle stop released the session (`idle-released`)". Row 22 is unchanged.
- packages/bus/README.md:207 (`assistant.stop`): no change. It still gives `not-holder`.
## Open questions for Dewey
- Q1. `landRelease` (broker.mjs:4144-4165, launcher.mjs:876-889) has the same shape for a role run's
land-relaunch: it releases, then the launcher sends SIGTERM, so a poll in between ends the run 22. I
recommend folding it in. That means one line in `landRelease` (add to `#idled` on release), one bus
assertion, and one cli forcing case like the one above. It falls outside this brief's words, so it
needs Sage's yes. If the answer is no, I'll write it up as a follow-up row with the same evidence.
- Q2. The code name `idle-released`. It names the cause, and if Q1 is yes it covers land-relaunch too,
since that is also a release made by the launcher. A more general name like `launcher-released` would
fit both. I prefer `released-by-launcher` only if Q1 is yes. Dewey picks.
- Q3. A limit stop on a run that was already idle-released. `#limitStop` (:4842-4855) finds no claim to
end and refuses `not-holder`, so the run ends 22. I'd leave that alone: a limit hit is a real stop and
ends session-invalid, as it does today. It's possible only if the limit checker answers inside the
window.
- Q4. Per-site coverage in the runner. Only the receive site is forced. I can add a cli case per site
(hold the window until the run's landing `state`, or a turn's `state`, lands in it), at about one
fixture each. I'd add them only if Dewey asks, since all four sites share one helper in the build.
## Gate (brief)
Dewey approves this design, then the manifest. Runs: the new forcing test, then
compactions-launch.test.mjs whole, then the bus, cli and harness suites whole, then every
`scripts/test-*.sh`. The mos-1411 point 4 rerun rule doesn't apply to this row's own gate.
Mutants. Each one must die on the forcing test, with the bus test also named where it applies:
- M1. `threadIdle` doesn't add to `#idled`. Dies on the forcing test (end 22) and the bus test.
- M2. `#agent` ignores `#idled`. Same deaths as M1.
- M3. The runner drops the `idle-released` branch at receive. Dies on the forcing test (no exit inside
the window).
- M4. The runner maps `idle-released` to `EXIT.session`. Dies on the forcing test (end 22).
- M5. `#claimEnd` marks every release, not just threadIdle's. Dies on the bus not-holder controls and
the cli control.
- M6. `endLaunch` doesn't drop the key. This is not a correctness mutant: caps are gone, so the answer
is `unauthenticated` either way. I'll drop it unless Dewey wants an assertion on it, and record it as
an equivalent mutant.
Limits will be set in the run plan after the design round (mos-1275: `timeout -k 10` at three times or
more the longest run).
## After Dewey's d1 review (dewey-0875, 05:32Z): APPROVE on D1, folded here before any build
D1 (required). role.claim's thread branch (broker.mjs:5154-5173) doesn't go through `#agent` and
refuses only `run-revoked`, the session fence and `thread-already-held`. So an idle-released run could
claim its free session again. Its key would still be in `#idled`, and a later real not-holder on that
claim would answer `idle-released` and exit 0. Fix: the thread branch of role.claim refuses
`idle-released` for a run in `#idled`, before the insert. The key stays, because the run's claim is over.
Bus assertion: after threadIdle releases a-run, a-run's role.claim refuses `idle-released`, and no
thread_claims row is added. Mutant M7: drop that check. It dies on that assertion.
Correction to "Harness: no new runner.test case". That's true for threadIdle only. runner.test reaches
landRelease, which is a role-run path: :742-772 releases coder-run with `ctx.broker.landRelease` and no
launcher, and pins `out.code === EXIT.session` (:771) and `session stopped working: not-holder` (:772).
If Q1 is yes, those two lines change (to stopped/0 and the released line). They go to Mos through Sage
with threads.test:394, and runner.test:742-772 then serves as the land-relaunch forcing case. No launcher means no
SIGTERM, so the receive always lands after the release. The cli case for land-relaunch is dropped. If Q1
is no, the follow-up row cites runner.test:771-772 as its evidence.
Q2. The code is `launcher-released` if Q1 is yes, and `idle-released` if Q1 is no. Every name in this
file reads that way.
Q3. Agreed. A limit stop on an idled run stays not-holder and exits 22.
Q4. No per-site cases. Both ops return `released:false` while a delivered message is unread
(threadIdle via `t.unread`, landRelease via its count). The runner reads only after the reply
(runner.mjs:575-580), so a run can't be released mid-turn. landRelease keeps a run that hasn't received
yet (`landing:true`), and thread runs have no landing. So only the receive site can see the code. All
four sites keep the shared helper, and the three unreachable ones get a one-line comment saying why.
N1. The cli control must reach the runner before any launcher signal. It uses a role.revoke on the run
(no launcher stop), or holds the signal the same way the forcing test does. assistant.stop gets checked
first in case it goes through a launcher stop. It also asserts the run's runner.log has
`session stopped working: not-holder` and `exiting 22`.
N2. The control keeps "after an idle release", so `#idled` isn't empty when M5 runs.
N3. M6 is equivalent. The key drop in endLaunch stays, so the set doesn't grow.
N4. The other-pin grep also covers runner.test `session stopped working`. :558 (unauthenticated) and
:647 (revoke) stay, and :772 changes only under Q1.
Docs: if Q1 is yes, landRelease joins the bus README line and the harness exit-table row 0.
## Round d2 (sage-3204, 05:32Z)
Line numbers. Sage's check against 0e30a85fa is right, and the references above are now corrected. The
second threadIdle not-holder pin is threads.test:392 and stays. The receive pin that changes is :394,
and the pending check is :395 and stays. The idle release is at :388.
Sage asked for a grep of the land-relaunch tests for a not-holder pin on the relaunched run's own call.
`git grep -l "landRelease\|land-relaunch" 'packages/*/tests/*'` lists 8 files. In those files,
`not-holder|session stopped working|EXIT.session|session-invalid` hits only land.test.mjs and
runner.test.mjs:
- land.test.mjs:464, `assert.throws(() => w.call('merge-gate', 'message.receive'), /not-holder/)`. This is
the released run's own receive after `landRelease` released it (:458). Under Q1 yes, it answers
`launcher-released`, and the regex fails. That's a third old-test change for Mos through Sage. The new
pin is `/launcher-released/`.
- land.test.mjs:457 (a non-holder run's landRelease) and :463 (a second landRelease) are landRelease's
own refusals. They stay not-holder, the same as threads.test:392.
- runner.test.mjs:771-772 change under Q1, as in the section above. runner.test.mjs:548, :557-558 and
:646-647 are unauthenticated, unauthenticated and a limit stop. They stay.
- The six cli files have no hit.
So under Q1 yes, the Mos package is threads.test:394, land.test:464 and runner.test:771-772. Under Q1 no,
it's threads.test:394 alone. Nothing gets built until Mos rules and Sage gives the go.
## Round d3: as built (after mos-1414/1415/1416 and sage-3220)
Q1 is yes, so the code is `launcher-released` everywhere. The broker set is `#launcherReleased`, not
`#idled`. Every `idle-released` above reads `launcher-released`.
### Correction to d1: message.send doesn't go through `#agent`
d1 said `message.send` and `message.read` both go through `#agent`. `message.read` does. `message.send`
doesn't. So a released run's send isn't refused `launcher-released`. Only `message.receive`, `state`,
`message.read` and `role.claim` answer it, and the bus test asserts exactly those four. That narrows
what Dewey approved in d1, so I'm sending it to Dewey directly (sage-3220).
### Broker (packages/bus/src/broker.mjs)
- `#launcherReleased = new Set()`, keyed `business:run`, next to `#stops`, with a comment.
- threadIdle and landRelease add the key after their release transaction.
- `#agent` refuses `launcher-released` for a run in the set, and `not-holder` otherwise.
- role.claim (D1) refuses `launcher-released` right after the `agent-required` check, before the thread
and role branches. So it covers landRelease's role runs as well as thread runs.
- endLaunch deletes the key after `#ended.add(key)`.
- Q3 needs no code. A limit stop runs `#limitStop` (broker.mjs:4482) before any `#agent` check, and it
fails `not-holder` itself (:4862), so a limit stop after a launcher release still exits 22.
### Runner (packages/harness/src/runner.mjs)
- `const LAUNCHER_RELEASED = "launcher-released"`.
- The shared helper `ended(err, say = true)`. For `launcher-released`, it logs
`runner: session released by the launcher`, sets `released` and leaves the exit code at
`EXIT.stopped`. For the SESSION_GONE codes it keeps the old behaviour (`EXIT.session`, and the log
line unless `say` is false).
- It's used at the landing state check, receive, the turn state check and the reply. The reply site
passes `say = false`, so its logging is the same as before. The three sites that can't see the code
(Q4) carry a comment saying why.
- At exit, the final `role.release` is skipped when `released`, because the launcher already released.
### Tests as built
- `packages/cli/tests/idle-stop-race.test.mjs`. This is the threadIdle forcing case. Inside the first
released run's held threadIdle op, the human's `assistant.stop` revokes the other thread's holder
(N1: no launcher signal can come while the op is held, launcher.mjs:662-663). The op waits for both
runners' verdicts and end events. The idle run logs `session released by the launcher`, ends
`["stopped", 0, null]` before its thread-idle line, and has `exiting 0` and no `session stopped working`.
The control logs `session stopped working: not-holder`, ends `["session-invalid", 22, null]` and
has `exiting 22`. The Store shows `launcher-released` and no `not-holder` refusals for the idle run,
and `not-holder` for the control.
- `packages/bus/tests/launcher-released.test.mjs` (renamed from idle-release.test.mjs). It has three
tests. (1) threadIdle: receive, state, read and role.claim refuse `launcher-released`, role.claim adds
no thread_claims row, the four refusals are logged, nothing is delivered, threadIdle's own second
refusal stays not-holder, and after endLaunch the answer is unauthenticated. (2) The control (N2:
after an idle release): assistant.stop's run answers not-holder and run-revoked, pm's own role.release
answers not-holder, and a new holder for the released thread doesn't change the released run's answer.
(3) landRelease: receive and state refuse `launcher-released`, role.claim refuses and adds no
role_claims row, landRelease's own second refusal stays not-holder, and coder's own role.release
answers not-holder.
- runner.test.mjs:742-772 is the land-relaunch forcing case (mos-1415). There's no cli land-relaunch case.
- The four old lines Mos ruled are in ASSERTION-PRESERVATION.md. The diff on old test files is those four
lines and nothing else.
### Docs
The bus README's threadIdle paragraph names `launcher-released`, the four calls that answer it,
landRelease, and that every other lost claim stays not-holder. The cli README's land-relaunch paragraph
says calls between the release and the signal refuse `launcher-released` and the runner exits 0. The
harness README's exit table row 0 names the launcher release.
### Red first (mos-1415)
There are two commits. The first holds only the tests and the four ruled pin edits. The second holds the
fix (broker.mjs, runner.mjs) and the READMEs. The gate's first stage runs on the tests-only commit, and
these two must fail there:
- idle-stop-race.test.mjs: the idle run ends `["session-invalid", 22, null]`, or its verdict is
`session stopped working: not-holder`, whichever lands first.
- runner.test.mjs, the case at :742: out.code is 22, not 0.
The bus test files and threads.test:394 and land.test:464 fail there too, and the packet records them.
The red run doesn't count as the gate.
### Mutants (each run against the fix commit, restored and checked by sha256 after)
- M1. threadIdle doesn't add the key. Dies: idle-stop-race (end 22), launcher-released.test test 1,
threads.test:394.
- M2. `#agent` ignores the set. Dies: the same as M1, plus launcher-released.test test 3,
land.test:464 and runner.test:771.
- M3. The runner's helper drops the `launcher-released` branch. Dies: idle-stop-race (the run doesn't
exit in the window, so `window.idle` is null) and runner.test:771-772.
- M4. The runner maps `launcher-released` to `EXIT.session`. Dies: idle-stop-race (end 22) and
runner.test:771.
- M5. `#agent` answers `launcher-released` for every lost claim. Dies: the idle-stop-race control
(verdict and end 22), launcher-released.test test 2, and test 3's coder line.
- M6. endLaunch doesn't drop the key. Equivalent (N3), not run.
- M7. role.claim drops the D1 check. Dies: launcher-released.test tests 1 and 3 (the claim succeeds).
- M8. landRelease doesn't add the key. Dies: runner.test:771-772, land.test:464 and
launcher-released.test test 3.
### Run plan (mos-1275 limits, `timeout -k 10`)
1. idle-stop-race.test.mjs alone, 600 s. Then runner.test.mjs whole, 900 s.
2. compactions-launch.test.mjs whole, 600 s.
3. The bus, harness and cli suites whole: bus 1200 s, harness 1200 s, cli 2100 s. 292's cli suite took
630 s.
4. Every scripts/test-*.sh, 900 s each.
5. M1-M5, M7 and M8, each on its named files at the limits above.
TMPDIR comes from `mktemp -d /mnt/storage/scratch/tmp/gXXX`. Every run is slot work and starts only on a
CLEAR from Sage.
## Round d4: Dewey's early static read (dewey-0878, 05:47Z)
C1. The d3 "correction" about message.send was wrong, and d1 was right. `message.send` calls
`#consumeAuthority` (broker.mjs:5506), which calls `#checkAuthority`, which calls `#agent` (:3749).
So a released run's send refuses `launcher-released`, and so does every verb that reaches `#agent`. There
are 29 call sites, including role.release and every authority check. role.claim refuses it before the
claim (D1). I misread the send path in d3, and no code depended on that reading. The runner's reply site
already handles the code. The bus README now says "every call that needs the claim", not a list of four.
Bus test 1 adds a message.send refusal, so the refused list is five `launcher-released`.
N1. Nothing killed a mutant that drops `&& !released` at runner exit (runner.mjs:596). That run calls
role.release, logs `runner: role.release: launcher-released` and still exits 0. idle-stop-race now
asserts the idle run's runner.log has no `runner: role.release:` line.
- M9. Drop `&& !released` at runner.mjs:596. It dies on idle-stop-race (the new doesNotMatch).
N2. idle-stop-race waits for the whole held window (`"revoked" in window`), then asserts
`window.control` is non-null with a message. If the other thread's run isn't holding when A's first
release lands, the test now fails with that message instead of a bare "timed out waiting".
The mutants to run are M1-M5, M7, M8 and M9. These test edits are in a third commit, after the fix.
ad9fa9f16 stays the red-first commit for the two forcing cases mos-1415 names.
## Round d5: red-first reordered (sage-3225, 05:49Z)
Sage pointed out that 8cf2c6d54 changed idle-stop-race.test.mjs and launcher-released.test.mjs after
ad9fa9f16. So a red run on ad9fa9f16 would test older copies than the ones that land. I took the first of
Sage's two options and reordered the commits locally, before any push. The order is now:
1. Tests: the final test files and this packet, over unfixed product files.
2. The fix: broker.mjs, runner.mjs and the three READMEs.
ad9fa9f16, 6e5f782ae and 8cf2c6d54 are superseded and were never pushed. The old tip is kept locally as
gizmoduck/row302-pre-reorder. The fix commit's tree matches 8cf2c6d54's except for this BUILD.md note.
The "third commit" in d4 now lives in these two commits.
The red run uses the tests commit as it stands, with these test blobs (identical in the fix commit):
| Blob | File |
| --- | --- |
| 52343932b231691ba2b4fc117547ea7f6f504704 | packages/bus/tests/land.test.mjs |
| 6c83c9ede41247fd50d40eeac3e8a16a75efacbf | packages/bus/tests/launcher-released.test.mjs |
| 27193a745c4ed8d4936de96c1768ab010a572856 | packages/bus/tests/threads.test.mjs |
| 9dfacfea15a487c4e7056ebae3e311cb514b0467 | packages/cli/tests/idle-stop-race.test.mjs |
| 0792431b60314a17ed74b3df0c5b3748dc3c1286 | packages/harness/tests/runner.test.mjs |
The red run checks these blobs with `git rev-parse HEAD:<path>` before it starts.
## Round d6: the first gate run's red in launcher-released.test.mjs:89 (gizmoduck-0388, sage-3294)
The slot opened at 08:15Z (sage-3289). The gate tree was 649ec67ab, a local merge of 64f6558ee onto
origin/refactor 1ac655005. The red stage on f799e8b15 passed: seven expected reds, with the pins at
threads.test.mjs:394, land.test.mjs:464 and runner.test.mjs:771 (:772 sits behind :771 in one test).
On 649ec67ab, idle-stop-race 1/1, runner.test 40/40 and compactions-launch 5/5 went green. The bus
suite then had 875 of 876 pass. The one red was this row's own test, at launcher-released.test.mjs:89,
1 !== 0. The gate stopped there, and the red went to Sage before any fix or rerun.
The test was wrong. `#route` (broker.mjs:6482) writes one routed delivery row for every message.
After threadIdle no run holds thread A, so `#recipient` (:832-834) returns null and `#delivery`
(:6612) writes holder_run null. The line counted every row for the message. Nothing was delivered to
a-run. ASSERTION-PRESERVATION.md entry 5 has the old and new lines.
The fix narrows the count to rows for a-run and adds the line Sage made mandatory: the message has
exactly one row, routed, holder_run null. bus rows come from node:sqlite with a null prototype, and the
tests use assert/strict, so the second line spreads each row before deepEqual. That is the idiom in
sessions.test.mjs:385 and session-clear.test.mjs:564. Sage's draft compared store.all directly, which
would fail on the prototype alone.
Packaging (sage-3294): this is a third commit on gizmoduck/row302 after the pushed f799e8b15 and
64f6558ee. It touches only that test line, this note and ASSERTION-PRESERVATION.md. It goes out as a
fast-forward push in a gap Sage names, after Mos rules (sage-3293). Then:
1. The red run uses the final test files over f799e8b15's product: f799e8b15 checked out detached, with
the five test files from the new tip checked out on top. The script checks the new blobs first.
2. The whole gate runs again on a fresh merge of the new tip onto origin/refactor.
Dewey reviews the delta. The first run's logs stay in gizmo-302-out/gate-run1/.
+112
View File
@@ -0,0 +1,112 @@
# Row 302 (#1809): gate results
The slot ran from 08:15Z on 2026-10-23 (sage-3289), with Sage as gate owner and Dewey as reviewer.
Logs: run1/gate-run.log and run2/gate-run.log. Script: run2/run-gate.sh and run2/mutants.py. The
full per-file logs stay in /mnt/storage/scratch/gizmo-302-out/gate-run1/ and gate/.
## Commits
| Commit | What |
| --- | --- |
| f799e8b15 | red first: the tests over unfixed product (d5) |
| 64f6558ee | the fix |
| 5b63ea323 | d6: launcher-released.test.mjs:89, Mos ruling mos-1430 (relayed in sage-3297) |
The 5b63ea323 subject still says "Mos ruling pending sage-3293; local, not pushed". I wrote it before
mos-1430 arrived, and Sage said not to amend. The ruling is mos-1430, and the commit was pushed as a
fast-forward at about 08:23Z.
## Run 1 (08:18Z): stopped on a red outside the four pins
- Gate tree: 649ec67ab, a local merge of 64f6558ee onto origin/refactor 1ac655005.
- Red stage: as expected (the same reds as run 2).
- Green on 649ec67ab: idle-stop-race 1/1, runner.test 40/40 and compactions-launch 5/5. Then bus had
875 of 876 pass. The red was this row's own test, at launcher-released.test.mjs:89, 1 !== 0.
- It stopped at 08:19:28Z and went to Sage (gizmoduck-0388), then Mos. BUILD.md d6 has the cause.
## Run 2 (08:23:41Z to 08:38:14Z, 873 s)
The gate tree was 783602e06ed92f99c7c71d1e9c016a0f9e2c9b28, a local merge (not pushed). Parent 1 is
origin/refactor 1ac655005, and parent 2 is 5b63ea323. The 12 row paths equal 5b63ea323's, and none
of them changes in 0e30a85fa..1ac655005.
### Red: f799e8b15's product with the final test files from 5b63ea323
The five final test blobs:
| Blob | File |
| --- | --- |
| 52343932b231691ba2b4fc117547ea7f6f504704 | packages/bus/tests/land.test.mjs |
| 6fb6e15c2fd34f742f9aab718ff654ab2704e198 | packages/bus/tests/launcher-released.test.mjs (new in d6) |
| 27193a745c4ed8d4936de96c1768ab010a572856 | packages/bus/tests/threads.test.mjs |
| 9dfacfea15a487c4e7056ebae3e311cb514b0467 | packages/cli/tests/idle-stop-race.test.mjs |
| 0792431b60314a17ed74b3df0c5b3748dc3c1286 | packages/harness/tests/runner.test.mjs |
The script checked each blob with hash-object on the red tree. Only launcher-released differs from
f799e8b15, and no product file differs from it.
There were seven reds, and nothing else failed:
- idle-stop-race: the released run ended with 'session stopped working: not-holder', exit 22 not 0
- runner.test landing case: pin :771, with 22 vs 0
- threads.test threadIdle case: pin :394
- land.test landRelease case: pin :464
- launcher-released: all three tests
On the bus files, 33 of 38 passed and 5 failed.
Pin :772 sits after :771 in the same test, so :771 stops the test first. :771's message is out.stderr,
and that shows the old "session stopped working: not-holder" line and no "session released by the
launcher" line. :89's two new lines never run on red, because the earlier refusals fail first. Run 2's
green bus stage is their first green.
### Green on 783602e06
| Run | Result | Wall |
| --- | --- | --- |
| idle-stop-race | 1/1 | 2 s |
| runner.test whole | 40/40 | 12 s |
| compactions-launch | 5/5 | 14 s |
| bus | 876/876 | 24 s |
| harness | 151/151 | 13 s |
| cli | 918/918 | 615 s |
| test-auth | 15 | |
| test-conductor | 17 | |
| test-config | 55 | |
| test-discord | 81 | |
| test-extension-package | 18 | |
| test-foundation | 44 | |
| test-land-work | 86 | |
| test-queue | node 156/156, suite 28 | |
| test-release | 14 | |
| test-task | 91 | |
Every scripts/test-*.sh ended with rc 0 and 0 failed. The node suites skipped nothing and left no
todo. The tree was clean after the tests.
The shell scripts skipped two things:
- test-queue: "queue verify and render --check: this checkout is not the queue's canonical root".
The gate runs in a worktree, not /mnt/storage/src/mosaic-stack.
- test-task: "live provider cases (row 244: no provider call in the gate)", printed twice. It printed
no other skip line, so no Docker group was skipped.
### Mutants on 783602e06
Each kill file ran whole, and every listed test name had to fail. After each mutant the file was
checked out again and matched its sha256. At the end, broker.mjs was 0c6dbbc5… and runner.mjs was
72ec39eb…, both equal to 5b63ea323's, and the tree was clean.
| Mutant | Change | Killed by |
| --- | --- | --- |
| M1 | no add after threadIdle's release | launcher-released (threadIdle), idle-stop-race |
| M2 | #agent always not-holder | land, launcher-released (threadIdle, landRelease), idle-stop-race, runner landing case |
| M3 | runner ignores launcher-released | idle-stop-race, runner landing case |
| M4 | a released run exits session | idle-stop-race, runner landing case |
| M5 | every lost claim reads launcher-released | launcher-released (other way), idle-stop-race |
| M7 | role.claim's D1 check removed | launcher-released (threadIdle, landRelease) |
| M8 | no add after landRelease's release | land, launcher-released (landRelease), runner landing case |
| M9 | exit releases a launcher-released run | idle-stop-race |
All eight were killed by rule, and none survived. When the run ended, no node process had its cwd in
the worktree.
@@ -0,0 +1,57 @@
blobs: the five d5 test blobs match in f799e8b15561ea2c0f0735e5328ba149c7a1a7b5 and 649ec67abcd315ac877bbad65742017b998137e4
merge: 649ec67abcd315ac877bbad65742017b998137e4 = 64f6558ee4cc536447c720eb20b1f675f32ffbf0 onto 1ac655005e611157605a0dcc5adf7a37f9bca2b0; 12 row paths equal to 64f6558ee4cc536447c720eb20b1f675f32ffbf0, none touched in 0e30a85fa..1ac655005e611157605a0dcc5adf7a37f9bca2b0
TIP 649ec67abcd315ac877bbad65742017b998137e4 (merge) FIX 64f6558ee4cc536447c720eb20b1f675f32ffbf0 RED f799e8b15561ea2c0f0735e5328ba149c7a1a7b5 base 0e30a85fa onto 1ac655005e611157605a0dcc5adf7a37f9bca2b0 origin/refactor 1ac655005e611157605a0dcc5adf7a37f9bca2b0 start 2026-10-23T08:18:31Z
loadavg: 1.63 1.48 1.89 5/6323 1532949
== 1. red first on f799e8b15561ea2c0f0735e5328ba149c7a1a7b5 (unfixed product, final tests)
TMPDIR /mnt/storage/scratch/tmp/ggiJ
idle-stop-race rc=1 ℹ tests 1 ℹ pass 0 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
TMPDIR /mnt/storage/scratch/tmp/g2U9
runner-land rc=1 ℹ tests 1 ℹ pass 0 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
TMPDIR /mnt/storage/scratch/tmp/g3Lr
bus rc=1 ℹ tests 38 ℹ pass 33 ℹ fail 5 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
✖ failing tests:
✖ landRelease releases the holding run by land-relaunch once it has asked message.receive and nothing delivered to it is unread, and waiting messages go to the next holder (34.180349ms)
✖ REQ-THREAD-2 REQ-LAUNCH-1: threadIdle releases only a thread with nothing waiting, and only for its holder (33.321208ms)
✖ row 302 a run landRelease released answers launcher-released, and claims nothing again (36.389735ms)
✖ row 302 a run threadIdle released answers launcher-released on receive, state, read and send, logs each, and claims nothing again (39.497664ms)
✖ row 302 with a run launcher-released, a run that lost its claim another way still answers not-holder (39.379523ms)
17: AssertionError [ERR_ASSERTION]: runner: acme/assistant claimed by run ra10cf293beb6
21: + actual - expected
32: actual: 'session stopped working: not-holder',
33: expected: 'session released by the launcher',
15: AssertionError [ERR_ASSERTION]: runner: demo/coder claimed by run coder-run
35: actual: 22,
36: expected: 0,
pin red at threads.test.mjs:394:
pin red at land.test.mjs:464:
pin red at runner.test.mjs:771:
:772 pattern absent from stderr (old not-holder line present)
every red on f799e8b15561ea2c0f0735e5328ba149c7a1a7b5:
✖ a run in a landing turn isn't released for a second held decision until its landing turns end and it has read its messages (602.229598ms)
✖ failing tests:
✖ landRelease releases the holding run by land-relaunch once it has asked message.receive and nothing delivered to it is unread, and waiting messages go to the next holder (34.180349ms)
✖ REQ-THREAD-2 REQ-LAUNCH-1: threadIdle releases only a thread with nothing waiting, and only for its holder (33.321208ms)
✖ row 302 a run landRelease released answers launcher-released, and claims nothing again (36.389735ms)
✖ row 302 a run threadIdle released answers launcher-released on receive, state, read and send, logs each, and claims nothing again (39.497664ms)
✖ row 302 REQ-THREAD-2 an idle-released run whose call lands before the SIGTERM ends stopped/0; a run whose claim the human revoked in the same window still ends session-invalid/22 (1672.44448ms)
✖ row 302 with a run launcher-released, a run that lost its claim another way still answers not-holder (39.379523ms)
red stage: both forcing cases and the four pins red
wall 5 s
== 2. green on 649ec67abcd315ac877bbad65742017b998137e4
TMPDIR /mnt/storage/scratch/tmp/gaYE
idle-stop-race rc=0 ℹ tests 1 ℹ pass 1 ℹ fail 0 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
wall 1 s
TMPDIR /mnt/storage/scratch/tmp/g9t4
runner rc=0 ℹ tests 40 ℹ pass 40 ℹ fail 0 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
wall 12 s
TMPDIR /mnt/storage/scratch/tmp/gwAp
compactions-launch rc=0 ℹ tests 5 ℹ pass 5 ℹ fail 0 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
wall 15 s
TMPDIR /mnt/storage/scratch/tmp/gPAB
bus rc=1 ℹ tests 876 ℹ pass 875 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
RED: bus, stop at 2026-10-23T08:19:28Z, wall 57 s
430:✖ row 302 a run threadIdle released answers launcher-released on receive, state, read and send, logs each, and claims nothing again (81.984434ms)
931:✖ failing tests:
934:✖ row 302 a run threadIdle released answers launcher-released on receive, state, read and send, logs each, and claims nothing again (81.984434ms)
935: AssertionError [ERR_ASSERTION]: nothing delivered to the released run
exit 1
@@ -0,0 +1,187 @@
blobs: the five final test blobs (d5, launcher-released from d6) match in 5b63ea3236d82e14956594b41a00cbd7f7dc5f71 and 783602e06ed92f99c7c71d1e9c016a0f9e2c9b28
merge: 783602e06ed92f99c7c71d1e9c016a0f9e2c9b28 = 5b63ea3236d82e14956594b41a00cbd7f7dc5f71 onto 1ac655005e611157605a0dcc5adf7a37f9bca2b0; 12 row paths equal to 5b63ea3236d82e14956594b41a00cbd7f7dc5f71, none touched in 0e30a85fa..1ac655005e611157605a0dcc5adf7a37f9bca2b0
TIP 783602e06ed92f99c7c71d1e9c016a0f9e2c9b28 (merge) FIX 5b63ea3236d82e14956594b41a00cbd7f7dc5f71 RED f799e8b15561ea2c0f0735e5328ba149c7a1a7b5 base 0e30a85fa onto 1ac655005e611157605a0dcc5adf7a37f9bca2b0 origin/refactor 1ac655005e611157605a0dcc5adf7a37f9bca2b0 start 2026-10-23T08:23:41Z
loadavg: 1.49 1.75 1.94 4/6320 1592551
== 1. red first: f799e8b15561ea2c0f0735e5328ba149c7a1a7b5 product with the final test files from 5b63ea3236d82e14956594b41a00cbd7f7dc5f71 checked out on top
red tree: product of f799e8b15561ea2c0f0735e5328ba149c7a1a7b5, tests of 5b63ea3236d82e14956594b41a00cbd7f7dc5f71: packages/bus/tests/launcher-released.test.mjs
TMPDIR /mnt/storage/scratch/tmp/gHpj
idle-stop-race rc=1 ℹ tests 1 ℹ pass 0 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
TMPDIR /mnt/storage/scratch/tmp/gIww
runner-land rc=1 ℹ tests 1 ℹ pass 0 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
TMPDIR /mnt/storage/scratch/tmp/g8I5
bus rc=1 ℹ tests 38 ℹ pass 33 ℹ fail 5 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
✖ failing tests:
✖ landRelease releases the holding run by land-relaunch once it has asked message.receive and nothing delivered to it is unread, and waiting messages go to the next holder (37.255433ms)
✖ REQ-THREAD-2 REQ-LAUNCH-1: threadIdle releases only a thread with nothing waiting, and only for its holder (30.900781ms)
✖ row 302 a run landRelease released answers launcher-released, and claims nothing again (36.454968ms)
✖ row 302 a run threadIdle released answers launcher-released on receive, state, read and send, logs each, and claims nothing again (40.167008ms)
✖ row 302 with a run launcher-released, a run that lost its claim another way still answers not-holder (38.485853ms)
17: AssertionError [ERR_ASSERTION]: runner: acme/assistant claimed by run r3f96e5142838
21: + actual - expected
32: actual: 'session stopped working: not-holder',
33: expected: 'session released by the launcher',
15: AssertionError [ERR_ASSERTION]: runner: demo/coder claimed by run coder-run
35: actual: 22,
36: expected: 0,
pin red at threads.test.mjs:394:
pin red at land.test.mjs:464:
pin red at runner.test.mjs:771:
:772 pattern absent from stderr (old not-holder line present)
every red on f799e8b15561ea2c0f0735e5328ba149c7a1a7b5:
✖ a run in a landing turn isn't released for a second held decision until its landing turns end and it has read its messages (604.606436ms)
✖ failing tests:
✖ landRelease releases the holding run by land-relaunch once it has asked message.receive and nothing delivered to it is unread, and waiting messages go to the next holder (37.255433ms)
✖ REQ-THREAD-2 REQ-LAUNCH-1: threadIdle releases only a thread with nothing waiting, and only for its holder (30.900781ms)
✖ row 302 a run landRelease released answers launcher-released, and claims nothing again (36.454968ms)
✖ row 302 a run threadIdle released answers launcher-released on receive, state, read and send, logs each, and claims nothing again (40.167008ms)
✖ row 302 REQ-THREAD-2 an idle-released run whose call lands before the SIGTERM ends stopped/0; a run whose claim the human revoked in the same window still ends session-invalid/22 (1604.296753ms)
✖ row 302 with a run launcher-released, a run that lost its claim another way still answers not-holder (38.485853ms)
red stage: both forcing cases and the four pins red
wall 4 s
== 2. green on 783602e06ed92f99c7c71d1e9c016a0f9e2c9b28
TMPDIR /mnt/storage/scratch/tmp/gMtc
idle-stop-race rc=0 ℹ tests 1 ℹ pass 1 ℹ fail 0 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
wall 2 s
TMPDIR /mnt/storage/scratch/tmp/gLSw
runner rc=0 ℹ tests 40 ℹ pass 40 ℹ fail 0 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
wall 12 s
TMPDIR /mnt/storage/scratch/tmp/gTQ3
compactions-launch rc=0 ℹ tests 5 ℹ pass 5 ℹ fail 0 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
wall 14 s
TMPDIR /mnt/storage/scratch/tmp/gAbg
bus rc=0 ℹ tests 876 ℹ pass 876 ℹ fail 0 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
wall 24 s
TMPDIR /mnt/storage/scratch/tmp/gzOd
harness rc=0 ℹ tests 151 ℹ pass 151 ℹ fail 0 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
wall 13 s
TMPDIR /mnt/storage/scratch/tmp/gxSF
cli rc=0 ℹ tests 918 ℹ pass 918 ℹ fail 0 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
wall 615 s
TMPDIR /mnt/storage/scratch/tmp/gL7z
test-auth rc=0
selftest: 15 passed, 0 failed
wall 1 s
TMPDIR /mnt/storage/scratch/tmp/ghcv
test-conductor rc=0
selftest: 17 passed, 0 failed
wall 6 s
TMPDIR /mnt/storage/scratch/tmp/gCsn
test-config rc=0
selftest: 55 passed, 0 failed
wall 3 s
TMPDIR /mnt/storage/scratch/tmp/guh1
test-discord rc=0
discord suite: 81 passed, 0 failed
wall 7 s
TMPDIR /mnt/storage/scratch/tmp/gVDP
test-extension-package rc=0
extension package selftest: 18 passed, 0 failed
wall 1 s
TMPDIR /mnt/storage/scratch/tmp/g1V8
test-foundation rc=0
selftest: 44 passed, 0 failed
wall 38 s
TMPDIR /mnt/storage/scratch/tmp/gYsp
test-land-work rc=0
---
land-work selftests: 86 passed, 0 failed
wall 11 s
TMPDIR /mnt/storage/scratch/tmp/gucV
test-queue rc=0 ℹ tests 156 ℹ pass 156 ℹ fail 0
queue suite: 28 passed, 0 failed
wall 21 s
TMPDIR /mnt/storage/scratch/tmp/gGva
test-release rc=0
selftest: 14 passed, 0 failed
wall 3 s
TMPDIR /mnt/storage/scratch/tmp/g4vO
test-task rc=0
selftest: 91 passed, 0 failed
wall 16 s
after tests: clean
== skips
suite/test-queue.log:skip queue verify and render --check: this checkout (/mnt/storage/scratch/gizmo-302) is not the queue's canonical root (/mnt/storage/src/mosaic-stack)
suite/test-task.log:skip live provider cases (row 244: no provider call in the gate)
suite/test-task.log:skip live provider cases (row 244: no provider call in the gate)
== 3. mutants
TMPDIR /mnt/storage/scratch/tmp/gDiY
M1.launcher-released rc=1 ℹ tests 3 ℹ pass 1 ℹ fail 2 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: row 302 a run threadIdle released answers launcher-released
TMPDIR /mnt/storage/scratch/tmp/glVj
M1.idle-stop-race rc=1 ℹ tests 1 ℹ pass 0 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: row 302 REQ-THREAD-2 an idle-released run
M1 KILLED by rule
wall 2 s
TMPDIR /mnt/storage/scratch/tmp/grHO
M2.land rc=1 ℹ tests 18 ℹ pass 17 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: landRelease releases the holding run by land-relaunch
TMPDIR /mnt/storage/scratch/tmp/giID
M2.launcher-released rc=1 ℹ tests 3 ℹ pass 0 ℹ fail 3 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: row 302 a run threadIdle released answers launcher-released
killed: row 302 a run landRelease released answers launcher-released
TMPDIR /mnt/storage/scratch/tmp/gXTo
M2.idle-stop-race rc=1 ℹ tests 1 ℹ pass 0 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: row 302 REQ-THREAD-2 an idle-released run
TMPDIR /mnt/storage/scratch/tmp/gTkJ
M2.runner rc=1 ℹ tests 40 ℹ pass 39 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: a run in a landing turn isn't released for a second held decision
M2 KILLED by rule
wall 15 s
TMPDIR /mnt/storage/scratch/tmp/ge9B
M3.idle-stop-race rc=1 ℹ tests 1 ℹ pass 0 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: row 302 REQ-THREAD-2 an idle-released run
TMPDIR /mnt/storage/scratch/tmp/gYUi
M3.runner rc=1 ℹ tests 40 ℹ pass 39 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: a run in a landing turn isn't released for a second held decision
M3 KILLED by rule
wall 33 s
TMPDIR /mnt/storage/scratch/tmp/gw5X
M4.idle-stop-race rc=1 ℹ tests 1 ℹ pass 0 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: row 302 REQ-THREAD-2 an idle-released run
TMPDIR /mnt/storage/scratch/tmp/g90Y
M4.runner rc=1 ℹ tests 40 ℹ pass 39 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: a run in a landing turn isn't released for a second held decision
M4 KILLED by rule
wall 14 s
TMPDIR /mnt/storage/scratch/tmp/gNlw
M5.launcher-released rc=1 ℹ tests 3 ℹ pass 1 ℹ fail 2 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: row 302 with a run launcher-released, a run that lost its claim
TMPDIR /mnt/storage/scratch/tmp/gCk7
M5.idle-stop-race rc=1 ℹ tests 1 ℹ pass 0 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: row 302 REQ-THREAD-2 an idle-released run
M5 KILLED by rule
wall 2 s
TMPDIR /mnt/storage/scratch/tmp/gaHU
M7.launcher-released rc=1 ℹ tests 3 ℹ pass 1 ℹ fail 2 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: row 302 a run threadIdle released answers launcher-released
killed: row 302 a run landRelease released answers launcher-released
M7 KILLED by rule
wall 1 s
TMPDIR /mnt/storage/scratch/tmp/g3K9
M8.land rc=1 ℹ tests 18 ℹ pass 17 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: landRelease releases the holding run by land-relaunch
TMPDIR /mnt/storage/scratch/tmp/ge6z
M8.launcher-released rc=1 ℹ tests 3 ℹ pass 2 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: row 302 a run landRelease released answers launcher-released
TMPDIR /mnt/storage/scratch/tmp/gWPU
M8.runner rc=1 ℹ tests 40 ℹ pass 39 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: a run in a landing turn isn't released for a second held decision
M8 KILLED by rule
wall 13 s
TMPDIR /mnt/storage/scratch/tmp/gqwU
M9.idle-stop-race rc=1 ℹ tests 1 ℹ pass 0 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0
killed: row 302 REQ-THREAD-2 an idle-released run
M9 KILLED by rule
wall 2 s
end 2026-10-23T08:38:14Z wall 873 s
node with cwd in the worktree: 0
exit 0
@@ -0,0 +1,47 @@
#!/usr/bin/env python3
# Row 302 (#1809) mutants, from BUILD.md d3 and d4. Each replaces one exact string, once, in one file.
# Kill rule: every (test file, failing test name) listed fails in that file's run.
# mutants.py list | apply M | file M | kills M
import sys
BROKER = "packages/bus/src/broker.mjs"
RUNNER = "packages/harness/src/runner.mjs"
ADD = " this.#launcherReleased.add(key);\n"
IDLE = ("packages/cli/tests/idle-stop-race.test.mjs", "row 302 REQ-THREAD-2 an idle-released run")
LR1 = ("packages/bus/tests/launcher-released.test.mjs", "row 302 a run threadIdle released answers launcher-released")
LR2 = ("packages/bus/tests/launcher-released.test.mjs", "row 302 with a run launcher-released, a run that lost its claim")
LR3 = ("packages/bus/tests/launcher-released.test.mjs", "row 302 a run landRelease released answers launcher-released")
LAND = ("packages/bus/tests/land.test.mjs", "landRelease releases the holding run by land-relaunch")
RUN = ("packages/harness/tests/runner.test.mjs", "a run in a landing turn isn't released for a second held decision")
M = {
"M1": (BROKER, "this.#claimEnd(session, this.#own(session), 'release', null));\n" + ADD,
"this.#claimEnd(session, this.#own(session), 'release', null));\n", [IDLE, LR1]),
"M2": (BROKER, "fail(this.#launcherReleased.has(s.business + ':' + s.run) ? 'launcher-released' : 'not-holder');",
"fail('not-holder');", [IDLE, LR1, LR3, LAND, RUN]),
"M3": (RUNNER, " if (err.code === LAUNCHER_RELEASED) {", " if (false) {", [IDLE, RUN]),
"M4": (RUNNER, " released = true;\n return true;", " released = true;\n code = EXIT.session;\n return true;", [IDLE, RUN]),
"M5": (BROKER, "? 'launcher-released' : 'not-holder')", "? 'launcher-released' : 'launcher-released')", [IDLE, LR2]),
"M7": (BROKER, " if (this.#launcherReleased.has(s.business + ':' + s.run)) fail('launcher-released');\n", "", [LR1, LR3]),
"M8": (BROKER, "'land-relaunch'));\n" + ADD, "'land-relaunch'));\n", [LR3, LAND, RUN]),
"M9": (RUNNER, "if (code === EXIT.stopped && !released) {", "if (code === EXIT.stopped) {", [IDLE]),
}
cmd = sys.argv[1]
if cmd == "list":
print(" ".join(M))
elif cmd == "file":
print(M[sys.argv[2]][0])
elif cmd == "kills":
for f, name in M[sys.argv[2]][3]:
print(f + "\t" + name)
elif cmd == "apply":
path, old, new, _ = M[sys.argv[2]]
s = open(path).read()
if s.count(old) != 1:
sys.exit(f"{sys.argv[2]}: anchor found {s.count(old)} times in {path}")
open(path, "w").write(s.replace(old, new))
print(path)
else:
sys.exit("usage: mutants.py list|apply M|file M|kills M")
@@ -0,0 +1,147 @@
#!/bin/bash
# Row 302 (#1809, Vikunja 588) gate. Plan: agents/gizmoduck/work/queue-302/BUILD.md d3 "Run plan", d4, d5.
# 0. Preflight: worktree at TIP and clean, the five test blobs from d5. TIP is the local merge of 64f6558ee
# onto origin/refactor 1ac655005 (sage-3289); the 12 row paths are unchanged in BASE..1ac655005.
# 1. Red (mos-1415): f799e8b15 (final tests, unfixed product). idle-stop-race and the runner.test land case
# must fail. The bus files are run and recorded. A green forcing case here is a STOP.
# 2. On TIP: idle-stop-race alone 600 s, runner.test whole 900 s, compactions-launch whole 600 s,
# bus 1200 s, harness 1200 s, cli 2100 s, then the scripts/test-*.sh at 900 s each.
# 3. M1-M5, M7, M8, M9 (mutants.py). Each kill-listed file runs whole; every listed test name must fail.
# After each: git checkout, sha256 equal to the candidate's, tree clean. A survivor is a STOP to Sage.
# mos-1272: a first-run red in a new test is placed and reported after; any other red stops the run.
set -o pipefail
WT=/mnt/storage/scratch/gizmo-302
# Run 2 (BUILD d6): FIX is the d6 tip; TIP is a fresh merge of it onto ONTO, set when the run is cleared.
TIP=${TIP:?set TIP to the run-2 merge sha}
FIX=5b63ea3236d82e14956594b41a00cbd7f7dc5f71
ONTO=${ONTO:?set ONTO to origin/refactor at the merge}
BR=gizmoduck/row302-gate
RED=f799e8b15561ea2c0f0735e5328ba149c7a1a7b5
BASE=0e30a85fa
OUT=/mnt/storage/scratch/gizmo-302-out/gate
PY=/mnt/storage/scratch/gizmo-302-out/mutants.py
cd "$WT" || exit 9
[ "$(git rev-parse HEAD)" = "$TIP" ] || { echo "not on the gate commit"; exit 9; }
[ -z "$(git status --porcelain --untracked-files=no)" ] || { echo "dirty"; exit 9; }
TESTS=""
while read -r blob f; do
[ "$(git rev-parse "$TIP:$f")" = "$blob" ] && [ "$(git rev-parse "$FIX:$f")" = "$blob" ] || { echo "blob mismatch $f"; exit 9; }
TESTS="$TESTS $f"
done <<'EOF'
52343932b231691ba2b4fc117547ea7f6f504704 packages/bus/tests/land.test.mjs
6fb6e15c2fd34f742f9aab718ff654ab2704e198 packages/bus/tests/launcher-released.test.mjs
27193a745c4ed8d4936de96c1768ab010a572856 packages/bus/tests/threads.test.mjs
9dfacfea15a487c4e7056ebae3e311cb514b0467 packages/cli/tests/idle-stop-race.test.mjs
0792431b60314a17ed74b3df0c5b3748dc3c1286 packages/harness/tests/runner.test.mjs
EOF
echo "blobs: the five final test blobs (d5, launcher-released from d6) match in $FIX and $TIP"
[ -L node_modules ] || ln -s /mnt/storage/src/mosaic-stack/node_modules node_modules
[ -L node_modules ] || { echo "no node_modules link"; exit 8; }
[ "$(git rev-parse $TIP^1) $(git rev-parse $TIP^2)" = "$ONTO $FIX" ] || { echo "merge parents wrong"; exit 9; }
paths=$(git diff --name-only $BASE $FIX)
[ "$(echo "$paths" | wc -l)" -eq 12 ] || { echo "row paths not 12"; exit 9; }
over=$(git diff --name-only $BASE $ONTO -- $paths)
[ -z "$over" ] || { echo "BASE..ONTO touches row paths:"; echo "$over"; exit 9; }
git diff --quiet $FIX $TIP -- $paths || { echo "merge changed a row path"; exit 9; }
echo "merge: $TIP = $FIX onto $ONTO; 12 row paths equal to $FIX, none touched in $BASE..$ONTO"
[ -e "$OUT" ] && { echo "$OUT exists; move it first"; exit 9; }
mkdir -p "$OUT/red" "$OUT/suite" "$OUT/mut"
newtmp() { export TMPDIR=$(mktemp -d /mnt/storage/scratch/tmp/gXXX); echo " TMPDIR $TMPDIR"; }
echo "TIP $TIP (merge) FIX $FIX RED $RED base $BASE onto $ONTO origin/refactor $(git rev-parse origin/refactor) start $(date -u +%FT%TZ)"; echo "loadavg: $(cat /proc/loadavg)"
T0=$SECONDS; t0=$SECONDS; lap() { echo " wall $((SECONDS - t0)) s"; t0=$SECONDS; }
counts() { grep -E '^ℹ (tests|pass|fail|cancelled|skipped|todo) ' "$1" | tr '\n' ' '; echo; }
timed() { # log dir limit args... ; sets rc
local log=$1 dir=$2 lim=$3; shift 3
newtmp
(cd "$dir" && timeout -k 10 "$lim" "$@") > "$log" 2>&1
rc=$?
if [ $rc -eq 124 ] || [ $rc -eq 137 ]; then
echo "TIMEOUT after $lim s: $(basename "$log" .log). To Sage."
case $(basename "$log") in test-*.log) echo "SHELL-PHASE TIMEOUT (mos-1275): to Mos unplaced";; esac
fi
echo "$(basename "$log" .log) rc=$rc $(counts "$log")"
}
green() { # log dir limit args... ; any red stops
timed "$@"
if [ $rc -ne 0 ]; then
echo "RED: $(basename "$1" .log), stop at $(date -u +%FT%TZ), wall $((SECONDS - T0)) s"
grep -nE '^not ok|✖|AssertionError|Error:' "$1" | head -40
exit 1
fi
}
echo "== 1. red first: $RED product with the final test files from $FIX checked out on top"
git checkout -q --detach "$RED" || exit 9
git checkout -q "$FIX" -- $TESTS || exit 9
while read -r f; do [ "$(git hash-object "$f")" = "$(git rev-parse "$FIX:$f")" ] || { echo "red tree blob wrong $f"; exit 9; }; done < <(echo $TESTS | tr ' ' '\n')
[ -z "$(git diff --name-only HEAD -- 'packages/*/src/*')" ] || exit 9
echo " red tree: product of $RED, tests of $FIX: $(git diff --cached --name-only HEAD | tr '\n' ' ')"
[ -z "$(git diff --name-only $BASE HEAD -- 'packages/*/src/*')" ] || { echo "red commit changes product files"; git checkout -q $BR; exit 9; }
timed "$OUT/red/idle-stop-race.log" "$WT/packages/cli" 600 node --test tests/idle-stop-race.test.mjs; r1=$rc
timed "$OUT/red/runner-land.log" "$WT/packages/harness" 600 node --test --test-name-pattern="^a run in a landing turn isn't released for a second held decision" tests/runner.test.mjs; r2=$rc
timed "$OUT/red/bus.log" "$WT/packages/bus" 600 node --test tests/launcher-released.test.mjs tests/threads.test.mjs tests/land.test.mjs
grep -E '✖ ' "$OUT/red/bus.log" | sort -u | sed 's/^/ /' | head -20
git reset -q --hard "$RED" && git checkout -q $BR || exit 9
[ "$(git rev-parse HEAD)" = "$TIP" ] && [ -z "$(git status --porcelain --untracked-files=no)" ] || { echo "not back on TIP clean"; exit 9; }
red_ok=1
[ $r1 -ne 0 ] && grep -qF "✖ row 302 REQ-THREAD-2 an idle-released run" "$OUT/red/idle-stop-race.log" || { echo " idle-stop-race NOT red on $RED"; red_ok=0; }
[ $r2 -ne 0 ] && grep -qF "✖ a run in a landing turn isn't released for a second held decision" "$OUT/red/runner-land.log" || { echo " runner.test land case NOT red on $RED"; red_ok=0; }
grep -nE 'AssertionError|actual|expected' "$OUT/red/idle-stop-race.log" | head -12 | sed 's/^/ /'
grep -nE 'AssertionError|actual|expected' "$OUT/red/runner-land.log" | head -8 | sed 's/^/ /'
# The four ruled pins (mos-1414/1415/1416). :772 sits after :771 in one test, so :771 stops it first;
# :771's message is out.stderr, so the log shows the old "stopped working: not-holder" line and no
# "released by the launcher" line, which is the line :772 now pins.
grep -qF "✖ REQ-THREAD-2 REQ-LAUNCH-1: threadIdle releases only a thread with nothing waiting" "$OUT/red/bus.log" || { echo " threads.test threadIdle case NOT red"; red_ok=0; }
grep -qF "✖ landRelease releases the holding run by land-relaunch" "$OUT/red/bus.log" || { echo " land.test landRelease case NOT red"; red_ok=0; }
for pin in "threads.test.mjs:394:" "land.test.mjs:464:"; do
grep -qF "$pin" "$OUT/red/bus.log" && echo " pin red at $pin" || { echo " pin $pin not in the failure stacks"; red_ok=0; }
done
grep -qF "runner.test.mjs:771:" "$OUT/red/runner-land.log" && echo " pin red at runner.test.mjs:771:" || { echo " pin runner.test.mjs:771 not in the failure stack"; red_ok=0; }
grep -qF "session stopped working: not-holder" "$OUT/red/runner-land.log" && ! grep -qF "session released by the launcher" "$OUT/red/runner-land.log" \
&& echo " :772 pattern absent from stderr (old not-holder line present)" || { echo " :772 evidence missing"; red_ok=0; }
echo " every red on $RED:"; cat "$OUT"/red/*.log | grep -E '^✖ |^ ✖ |✖ ' | sed 's/^ *//' | sort -u | sed 's/^/ /'
[ $red_ok -eq 1 ] && echo " red stage: both forcing cases and the four pins red" || { echo "STOP: red stage failed, to Sage"; exit 3; }
lap
echo "== 2. green on $TIP"
green "$OUT/suite/idle-stop-race.log" "$WT/packages/cli" 600 node --test tests/idle-stop-race.test.mjs; lap
green "$OUT/suite/runner.log" "$WT/packages/harness" 900 node --test tests/runner.test.mjs; lap
green "$OUT/suite/compactions-launch.log" "$WT/packages/cli" 600 node --test tests/compactions-launch.test.mjs; lap
green "$OUT/suite/bus.log" "$WT/packages/bus" 1200 node --test tests/*.test.mjs; lap
green "$OUT/suite/harness.log" "$WT/packages/harness" 1200 node --test tests/*.test.mjs; lap
green "$OUT/suite/cli.log" "$WT/packages/cli" 2100 node --test tests/*.test.mjs; lap
for s in "$WT"/scripts/test-*.sh; do
n=$(basename "$s" .sh)
green "$OUT/suite/$n.log" "$WT" 900 bash "$s"
tail -2 "$OUT/suite/$n.log" | sed 's/^/ /'; lap
done
[ -z "$(git status --porcelain --untracked-files=no)" ] && echo "after tests: clean" || { echo "after tests: DIRTY"; exit 1; }
echo "== skips"
grep -HE '^ℹ skipped [1-9]|^ℹ todo [1-9]' "$OUT"/suite/*.log | sed "s#$OUT/##"
grep -HiE '^(skip|# skip|SKIP)|skipped' "$OUT"/suite/test-*.log | grep -v '^ℹ' | sed "s#$OUT/##" | head -40
echo "== 3. mutants"
limit() { case $1 in *cli/*) echo 600;; *runner.test*) echo 900;; *) echo 600;; esac; }
for m in $(python3 -B "$PY" list); do
src=$(python3 -B "$PY" file "$m")
sum=$(git show "HEAD:$src" | sha256sum | cut -c1-64)
python3 -B "$PY" apply "$m" >/dev/null || { echo "$m apply-failed"; git checkout -- "$src"; exit 8; }
git diff > "$OUT/mut/$m.diff"
ok=1
for f in $(python3 -B "$PY" kills "$m" | cut -f1 | sort -u); do
pkg=${f%%/tests/*}; rel=tests/${f#*/tests/}
log="$OUT/mut/$m.$(basename "$f" .test.mjs).log"
timed "$log" "$WT/$pkg" "$(limit "$f")" node --test "$rel"
[ $rc -ne 0 ] || { echo " $m survived in $f"; ok=0; }
while IFS=$'\t' read -r kf name; do
[ "$kf" = "$f" ] || continue
grep -qF "✖ $name" "$log" && echo " killed: $name" || { echo " NOT failing: $name"; ok=0; }
done < <(python3 -B "$PY" kills "$m")
done
git checkout -- "$src"
[ "$(sha256sum "$WT/$src" | cut -c1-64)" = "$sum" ] && [ -z "$(git status --porcelain --untracked-files=no)" ] || { echo "$m restore-failed"; exit 7; }
if [ $ok -eq 1 ]; then echo "$m KILLED by rule"; else echo "STOP: $m kill rule not met, to Sage"; exit 4; fi
lap
done
echo "end $(date -u +%FT%TZ) wall $((SECONDS - T0)) s"
cd /; echo "node with cwd in the worktree: $(for p in $(pgrep -x node); do readlink /proc/$p/cwd; done 2>/dev/null | grep -c "$WT")"
+8
View File
@@ -324,6 +324,14 @@ verb. It rechecks the thread and releases the run's claim in one call, and retur
launcher's snapshot. The launcher signals the run only after a release, so a message sent
after the snapshot gets a turn from the same run or waits as `pending` for the next one. It
refuses `not-holder` for a run that doesn't hold that thread and `run-ended` for an ended run.
After a release, and until the run ends, every call of the run's that needs the claim
(`message.receive`, `state`, `message.read`, `message.send` and every other verb that checks
the holder) refuses `launcher-released`, not `not-holder` (row 302). `role.claim` refuses it
too, before the claim, so the run can't claim again. A call
the broker answers after the release and before the signal is that stop, not a lost claim, so
the runner exits 0. The launch-landing pass's `landRelease` (`packages/cli/README.md`) answers
the same way for the role run it releases. Any other lost claim (a stop, a revoke, a clear, a
limit, a newer holder) still refuses `not-holder`.
A message delivered to a thread run that then ended without reading it is stranded, as
`stranded` (Agent state, below) describes for a role. It counts as neither `pending` nor
+12 -1
View File
@@ -419,6 +419,10 @@ export class Broker {
#hostRuns = new Map();
// Row 125 (#1619, REQ-LIMIT-4): a run a limit stops, by business:run, until its next receive or state.
#stops = new Map();
// Row 302 (#1809): runs the launcher released, by business:run, until endLaunch. threadIdle and
// landRelease release and then signal, so the run's own calls in between answer launcher-released,
// not not-holder, and the runner exits stopped instead of as a lost session.
#launcherReleased = new Set();
#clock = 0;
#processing;
// credentialStatus is the runtime's metadata-only Credentials.status(); the broker never gets use().
@@ -1003,7 +1007,8 @@ export class Broker {
if (s.reader) fail('read-only');
if (s.human) fail('agent-required');
const h = this.#own(s);
if (h?.op !== 'claim' || (s.thread ? h.generation !== this.#runGeneration(s) : h.holder_run !== s.run)) fail('not-holder');
if (h?.op !== 'claim' || (s.thread ? h.generation !== this.#runGeneration(s) : h.holder_run !== s.run))
fail(this.#launcherReleased.has(s.business + ':' + s.run) ? 'launcher-released' : 'not-holder');
return h;
}
// Row 264a (#1769, T2): the generation a thread run holds, its latest claim's in its session, or
@@ -1053,6 +1058,7 @@ export class Broker {
if (t?.holder !== record.run) fail('not-holder');
if (t.pending || t.unread) return { released: false, pending: t.pending, unread: t.unread };
this.#store.transaction(() => this.#claimEnd(session, this.#own(session), 'release', null));
this.#launcherReleased.add(key);
return { released: true };
}
// Row 271 (264d, #1776, D7 steps 3-5): the business's clears still in the launcher's hands, as of
@@ -1307,6 +1313,7 @@ export class Broker {
const exitCode = record.exitCode ?? null;
if (exitCode !== null && !Number.isSafeInteger(exitCode)) fail('invalid-request');
this.#ended.add(key);
this.#launcherReleased.delete(key);
for (const [cap, s] of this.#sessions)
if (!s.human && !s.reader && s.business === record.business && s.run === record.run) this.#sessions.delete(cap);
return this.#store.transaction(() => {
@@ -4161,6 +4168,7 @@ export class Broker {
);
if (unread) return { released: false, unread };
this.#store.transaction(() => this.#claimEnd(session, h, 'release', null, null, 'land-relaunch'));
this.#launcherReleased.add(key);
return { released: true };
}
// Row 194 (#1692, D1.1): the launcher's lifecycle pass. Trusted launcher API, not socket verbs.
@@ -5149,6 +5157,9 @@ export class Broker {
case 'role.claim': {
keys(a, []);
if (s.human) fail('agent-required');
// Row 302 (#1809, Dewey's D1): a run the launcher released is being stopped and doesn't claim
// again, or a later not-holder on that claim would answer launcher-released.
if (this.#launcherReleased.has(s.business + ':' + s.run)) fail('launcher-released');
// Row 121: a thread run claims its thread, so idle threads hold nothing (REQ-THREAD-2).
// Row 264a (#1769): the claim is the run's session's, with the session's next generation.
if (s.thread) {
+1 -1
View File
@@ -461,7 +461,7 @@ test('landRelease releases the holding run by land-relaunch once it has asked me
{ op: 'release', holder_run: 'merge-gate-run', by: 'land-relaunch', decision: null, reason: null },
);
assert.throws(() => release(), /not-holder/, 'released once');
assert.throws(() => w.call('merge-gate', 'message.receive'), /not-holder/);
assert.throws(() => w.call('merge-gate', 'message.receive'), /launcher-released/);
w.b.endLaunch({ business: 'demo', run: 'merge-gate-run', reason: 'exited' });
assert.throws(() => release(), /run-ended/);
assert.equal(w.store.get("SELECT count(*) AS n FROM deliveries WHERE message=? AND op='delivered'", waiting.id).n, 0, 'still waiting');
@@ -0,0 +1,151 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { mkdtempSync, readFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { Store } from '../src/store.mjs';
import { Broker } from '../src/broker.mjs';
// Row 302 (#1809, Vikunja 588): threadIdle and landRelease release a run's claim and the launcher
// signals it after. The run's own calls in between answer launcher-released, so its runner exits
// stopped. Every other way a run loses its claim still answers not-holder, and a released run
// doesn't claim again (Dewey's D1 in dewey-0875).
const shipped = (name) =>
JSON.parse(readFileSync(fileURLToPath(new URL(`../../../roles/${name}.json`, import.meta.url)), 'utf8'));
function setup(t, roles, definitions) {
const root = mkdtempSync(join(tmpdir(), 'bus-launcher-released-'));
const store = new Store(root);
t.after(() => {
store.close();
rmSync(root, { recursive: true, force: true });
});
const businesses = {
demo: {
id: 'demo',
human: 'jason',
arbiters: { technical: 'pm', delivery: 'pm' },
roles,
...(definitions ? { definitions } : {}),
projects: { stack: { root: '/nonexistent' } },
},
};
const b = new Broker({ store, businesses });
const caps = { human: b.bindHuman({ business: 'demo', human: 'jason', via: 'cli', outsideAgent: true }) };
const call = (who, verb, args = {}) => b.request(caps[who], { verb, args });
const launch = (name, role, extra = {}) => {
caps[name] = b.bindLaunch({ business: 'demo', role, run: `${name}-run`, harness: 'pi', ...extra });
};
const refused = (run) =>
store.all("SELECT body FROM events WHERE kind='action.refused' AND actor_run=? ORDER BY seq", run).map((e) => JSON.parse(e.body).code);
return { b, store, caps, call, launch, refused };
}
// Two threads of the assistant, each with its own run holding it.
function threads(t) {
const w = setup(
t,
{
chief: { definition: 'chief-of-staff', authority: { withinRole: ['message.send'], crossRole: [] } },
assistant: { definition: 'assistant', authority: { withinRole: ['message.send'], crossRole: [] } },
pm: { definition: 'pm', authority: { withinRole: ['message.send'], crossRole: [] } },
},
{ 'chief-of-staff': {}, assistant: { messaging: shipped('assistant').messaging }, pm: {} },
);
const project = w.call('human', 'project.create', { name: 'stack work', repo: 'stack' }).id;
const A = w.call('human', 'thread.create', { project, name: 'alpha' }).id;
const B = w.call('human', 'thread.create', { project, name: 'beta' }).id;
for (const r of ['chief', 'pm']) {
w.launch(r, r);
w.call(r, 'role.claim');
}
w.launch('a', 'assistant', { thread: A });
w.launch('b', 'assistant', { thread: B });
w.call('a', 'role.claim');
w.call('b', 'role.claim');
const idle = (run, thread = A) => w.b.threadIdle({ business: 'demo', thread, run });
return { ...w, A, B, idle };
}
test('row 302 a run threadIdle released answers launcher-released on receive, state, read and send, logs each, and claims nothing again', (t) => {
const { b, store, call, refused, idle, A } = threads(t);
const sent = call('human', 'thread.send', { thread: A, body: 'for A' });
assert.deepEqual(call('a', 'message.receive').map((m) => m.id), [sent.id]);
call('a', 'message.read', { id: sent.id });
assert.deepEqual(idle('a-run'), { released: true });
const later = call('human', 'thread.send', { thread: A, body: 'later for A' });
assert.throws(() => call('a', 'message.receive'), { code: 'launcher-released' });
assert.throws(() => call('a', 'state'), { code: 'launcher-released' });
assert.throws(() => call('a', 'message.read', { id: later.id }), { code: 'launcher-released' });
// message.send needs the claim too, through #consumeAuthority (dewey-0878 C1).
assert.throws(() => call('a', 'message.send', { to: 'chief', body: 'from a released run' }), { code: 'launcher-released' });
// D1: no new claim for the released run, so a later not-holder can't be misreported.
const claims = store.get('SELECT count(*) AS n FROM thread_claims').n;
assert.throws(() => call('a', 'role.claim'), { code: 'launcher-released' });
assert.equal(store.get('SELECT count(*) AS n FROM thread_claims').n, claims);
assert.deepEqual(refused('a-run'), Array(5).fill('launcher-released'));
// #route writes one routed row per message; with no holder its holder_run is null (broker #recipient).
assert.equal(store.get('SELECT count(*) AS n FROM deliveries WHERE message=? AND holder_run=?', later.id, 'a-run').n, 0, 'nothing delivered to the released run');
assert.deepEqual(store.all('SELECT op, holder_run FROM deliveries WHERE message=?', later.id).map((r) => ({ ...r })), [{ op: 'routed', holder_run: null }]);
// threadIdle's own refusal is unchanged.
assert.throws(() => idle('a-run'), { code: 'not-holder' });
// After the end the run's caps are gone.
b.endLaunch({ business: 'demo', run: 'a-run', reason: 'idle' });
assert.throws(() => call('a', 'message.receive'), { code: 'unauthenticated' });
});
test('row 302 with a run launcher-released, a run that lost its claim another way still answers not-holder', (t) => {
const { call, launch, idle, A, B } = threads(t);
assert.deepEqual(idle('a-run'), { released: true });
// The human's stop revokes the other thread's holder.
assert.deepEqual(call('human', 'assistant.stop', { instance: 'assistant' }).stopped, [{ thread: B, run: 'b-run' }]);
assert.throws(() => call('b', 'message.receive'), { code: 'not-holder' });
assert.throws(() => call('b', 'state'), { code: 'not-holder' });
assert.throws(() => call('b', 'role.claim'), { code: 'run-revoked' });
// A role run that released its own claim.
call('pm', 'role.release');
assert.throws(() => call('pm', 'message.receive'), { code: 'not-holder' });
// A new run holds the released thread; the released run's answer stays its own.
launch('a2', 'assistant', { thread: A });
call('a2', 'role.claim');
assert.throws(() => call('a', 'message.receive'), { code: 'launcher-released' });
assert.deepEqual(call('a2', 'message.receive'), []);
});
const MERGE = 'git.merge.protected';
const TASK = 'vikunja:1/2';
test('row 302 a run landRelease released answers launcher-released, and claims nothing again', (t) => {
const w = setup(t, {
pm: { authority: { withinRole: ['message.send'], crossRole: [] } },
coder: { authority: { withinRole: ['git.push.working', 'review.request', 'message.send'], crossRole: [] } },
reviewer: { authority: { withinRole: ['review.verdict', 'message.send'], crossRole: [] } },
'merge-gate': { authority: { withinRole: [], crossRole: [MERGE] } },
});
for (const role of ['pm', 'coder', 'reviewer', 'merge-gate']) {
w.launch(role, role);
w.call(role, 'role.claim');
}
const commit = 'a'.repeat(40);
w.b.checkPush(w.caps.coder, TASK);
w.b.recordPush(w.caps.coder, { task_ref: TASK, project: 'stack', branch: 'feat', ref: 'work/vikunja-1-2', commit, previous: null });
const r = w.call('coder', 'review.request', { task_ref: TASK, commit, to: 'reviewer' });
w.call('reviewer', 'review.verdict', { task_ref: TASK, request: r.request, verdict: 'approve', body: 'good' });
const options = [{ key: 'yes', text: 'Land it' }, { key: 'no', text: 'Not yet' }];
const d = w.call('merge-gate', 'decision.raise', { action: MERGE, target: TASK, question: 'Land?', options, recommendation: 'yes', blocking: false });
w.call('pm', 'decision.resolve', { id: d.id, choice: 'yes' });
const release = () => w.b.landRelease({ business: 'demo', run: 'merge-gate-run', decision: d.id });
for (const m of w.call('merge-gate', 'message.receive')) w.call('merge-gate', 'message.read', { id: m.id });
assert.deepEqual(release(), { released: true });
assert.throws(() => w.call('merge-gate', 'message.receive'), { code: 'launcher-released' });
assert.throws(() => w.call('merge-gate', 'state'), { code: 'launcher-released' });
const claims = w.store.get('SELECT count(*) AS n FROM role_claims').n;
assert.throws(() => w.call('merge-gate', 'role.claim'), { code: 'launcher-released' });
assert.equal(w.store.get('SELECT count(*) AS n FROM role_claims').n, claims);
// landRelease's own refusal is unchanged, and a role run that released itself still answers not-holder.
assert.throws(() => release(), { code: 'not-holder' });
w.call('coder', 'role.release');
assert.throws(() => w.call('coder', 'message.receive'), { code: 'not-holder' });
});
+1 -1
View File
@@ -391,7 +391,7 @@ test('REQ-THREAD-2 REQ-LAUNCH-1: threadIdle releases only a thread with nothing
// Once released, the run can't take the thread's next message; it waits for the next holder.
assert.throws(() => idle('a-run'), { code: 'not-holder' });
const later = call('human', 'thread.send', { thread: A, body: 'later for A' });
assert.throws(() => call('a', 'message.receive'), { code: 'not-holder' });
assert.throws(() => call('a', 'message.receive'), { code: 'launcher-released' });
assert.equal(state()[A].pending, 1);
assert.equal(b.endLaunch({ business: 'demo', run: 'a-run', reason: 'idle' }).released, false);
assert.throws(() => idle('a-run'), { code: 'run-ended' });
+2
View File
@@ -563,6 +563,8 @@ each holder that `launch.instances` lists:
per run as `run <run>: landRelease refused: <code>`. After a release the
launch log gets a `land-relaunch` record (`instance`, `run`,
`decision`) and the runner gets SIGTERM, as `mosaic stop` sends it. The
run's calls between the release and the signal refuse `launcher-released`,
so the runner exits 0 either way (row 302). The
next tick starts the instance with the decision in its `landing`.
The launcher keeps none of this on disk. A host that restarts ends the
+204
View File
@@ -0,0 +1,204 @@
// Row 302 (#1809, Vikunja 588): the idle stop releases a thread run's claim in the threadIdle op
// and sends SIGTERM only after the op returns. A runner call the broker answers in between used to
// get not-holder, and the run ended session-invalid/22 instead of stopped/0. This test holds the op
// open until a call has landed in that window and the runner has exited, so it doesn't depend on
// timing. The launcher skips a tick while one is running (launcher.mjs, the thread pass), so nothing
// else signals a run while the op is held.
//
// The host is in process, as in thread-launch.test.mjs, with the same fixture.
import { test } from "node:test";
import assert from "node:assert/strict";
import { spawnSync } from "node:child_process";
import { existsSync, readFileSync, writeFileSync } from "node:fs";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { launchDoc } from "../../business/tests/helpers.mjs";
import { BusError } from "../../bus/src/broker.mjs";
import { startBroker } from "../../bus/src/runtime.mjs";
import { Store } from "../../bus/src/store.mjs";
import { gitEnv } from "../../seat/src/checkout.mjs";
import { sandboxProblem } from "../../seat/src/sandbox.mjs";
import { launchLogFile, SESSION_CREDENTIALS } from "../../seat/src/session.mjs";
import { bootConfig, loadSystem } from "../src/config.mjs";
import { createLauncher } from "../src/launcher.mjs";
import { fakeHarnesses, fixture, tmp } from "./helpers.mjs";
const ADAPTER = fileURLToPath(new URL("./fixtures/thread-adapter.mjs", import.meta.url));
const skip = sandboxProblem() ?? false;
async function until(fn, ms = 20000) {
const end = Date.now() + ms;
while (Date.now() < end) {
const v = await fn();
if (v) return v;
await new Promise((r) => setTimeout(r, 50));
}
throw new Error("timed out waiting");
}
function gitIn(cwd, ...args) {
const r = spawnSync("git", ["-c", "user.name=test", "-c", "[email protected]", "-C", cwd, ...args], { env: gitEnv(), encoding: "utf8" });
assert.equal(r.status, 0, r.stderr);
}
function prepare(t) {
const root = tmp(t, "mosaic-idle-race-");
const f = fixture(root, "acme", (doc) => {
doc.roles.chief.vars = { harness: "pi", model: "claude-opus-5-5", thinking: "high" };
doc.roles.assistant = { definition: "assistant", vars: { harness: "pi", model: "claude-sonnet-5-5", thinking: "high" } };
doc.launch = launchDoc("chief", ["coder", "assistant"], { opus: 1, sonnet: 2 });
doc.launch.threadIdleSeconds = 1;
return doc;
});
for (const k of SESSION_CREDENTIALS.flat()) delete f.env[k];
const project = f.doc.projects.stack.root;
gitIn(project, "init", "-q", "-b", "main");
writeFileSync(join(project, "file.txt"), "one\n");
gitIn(project, "add", "file.txt");
gitIn(project, "commit", "-q", "-m", "one");
const adapter = join(root, "adapter.sh");
writeFileSync(adapter, `exec '${process.execPath}' '${ADAPTER}'\n`, { mode: 0o644 });
return { root, f, adapter };
}
// thread-launch.test.mjs's start(), with threadIdle replaced by the caller.
async function start(t, { f, adapter }, threadIdle) {
const system = loadSystem({ env: f.env });
const { trackers, ...boot } = bootConfig({ system, businessId: "acme", env: f.env });
assert.equal(trackers, undefined);
const runtime = await startBroker(boot);
const ops = {
identity: (m) => runtime.identity(m.cap),
authorizeLaunch: (m) => runtime.authorizeLaunch(m.cap, m.instance),
refuse: (m) => runtime.refuse(m.cap, m.code, m.subject),
endLaunch: (m) => runtime.endLaunch(m.record),
credentialStatus: (m) => runtime.credentialStatus(m.business, m.role),
threadsState: (m) => runtime.threadsState(m.business),
threadIdle: (m) => threadIdle(runtime, m),
mayLaunch: (m) => runtime.mayLaunch(m),
landHeld: (m) => runtime.landHeld(m.record),
landHolders: (m) => runtime.landHolders(m.business),
landRelease: (m) => runtime.landRelease(m.record),
};
const refusing = (what, fn) => async (m) => {
try {
return await fn(m);
} catch (e) {
const code = e instanceof BusError ? e.code : `${what}-refused`;
throw Object.assign(new Error(`${what} refused: ${code}`), { code });
}
};
const hooks = [];
let closing = null;
const host = {
business: "acme",
path: runtime.path,
dataRoot: f.dataRoot,
op: refusing("launch-op", (m) => {
if (!Object.hasOwn(ops, m.op)) throw new BusError("invalid-host-request");
return ops[m.op](m);
}),
bindLaunch: refusing("launch bind", (r) => runtime.bindLaunch(r)),
beforeClose: (fn) => void hooks.push(fn),
};
const launcher = createLauncher({
host,
system,
env: f.env,
makeLoader: await fakeHarnesses(t, { pi: adapter, "claude-code": adapter }),
sessionDefaults: { pollInterval: 100 },
harnessVersion: "0.85.1",
readOnly: [ADAPTER],
threadTickMs: 100,
log: () => {},
});
const close = () =>
(closing ??= (async () => {
for (const fn of hooks) await fn(0);
await runtime.close();
})());
t.after(close);
await launcher.listen();
const human = runtime.broker.bindHuman({ business: "acme", human: f.doc.human, via: "cli", outsideAgent: true });
const call = (verb, args = {}) => runtime.broker.request(human, { verb, args });
return { runtime, launcher, call, close };
}
const launchLog = (f) =>
existsSync(launchLogFile(f.dataRoot, "acme"))
? readFileSync(launchLogFile(f.dataRoot, "acme"), "utf8").trim().split("\n").map((l) => JSON.parse(l))
: [];
const runnerLog = (f, run) => {
const p = join(f.dataRoot, "launches", "acme", run, "runner.log");
return existsSync(p) ? readFileSync(p, "utf8") : "";
};
const ended = (f, run) => launchLog(f).find((e) => e.event === "end" && e.run === run);
// The runner's own verdict on a refusal: the launcher-release line, or a lost session's.
const verdict = (f, run) => /runner: (session released by the launcher|session stopped working: [a-z-]+)\n/.exec(runnerLog(f, run))?.[1] ?? null;
const settled = async (fn) => {
try {
return await until(fn);
} catch {
return null;
}
};
async function ask(call, thread, body) {
const sent = call("thread.send", { thread, body });
await until(() => call("thread.messages", { thread }).find((m) => m.in_reply_to === sent.id && m.class === "RESULT"));
}
test("row 302 REQ-THREAD-2 an idle-released run whose call lands before the SIGTERM ends stopped/0; a run whose claim the human revoked in the same window still ends session-invalid/22", { skip, timeout: 120000 }, async (t) => {
const prepared = prepare(t);
const { f } = prepared;
// The first run the idle stop releases. Inside its held window, the human stops the other
// thread's run, which no launcher pass can signal while the op is held.
let window = null;
let host = null;
const threadIdle = async (runtime, m) => {
const r = runtime.threadIdle(m.record);
if (!r.released || window) return r;
window = { run: m.record.run, thread: m.record.thread };
const other = runtime.threadsState("acme").find((x) => x.thread !== m.record.thread && x.holder);
window.control = other ? { run: other.holder, stopped: host.call("assistant.stop", { instance: "assistant" }).stopped } : null;
// Both runners decide on their own before the op returns and the launcher can signal.
window.idle = await settled(() => verdict(f, window.run) && ended(f, window.run) && verdict(f, window.run));
window.revoked = other ? await settled(() => verdict(f, other.holder) && ended(f, other.holder) && verdict(f, other.holder)) : null;
return r;
};
host = await start(t, prepared, threadIdle);
const project = host.call("project.create", { name: "stack work", repo: "stack" }).id;
const A = host.call("thread.create", { project, name: "alpha" }).id;
const B = host.call("thread.create", { project, name: "beta" }).id;
await Promise.all([ask(host.call, A, "for A"), ask(host.call, B, "for B")]);
// The op sets window.revoked last, so this waits for the whole window (dewey-0878 N2).
await until(() => window && "revoked" in window, 60000);
assert.ok(window.control, "the other thread's run held B when A's first release landed");
await until(() => ended(f, window.run) && ended(f, window.control.run), 60000);
const { run, control } = window;
assert.deepEqual(control.stopped.map((s) => s.run), [control.run], "the human's stop took only the other thread's run");
// The released run refused in the window and exited by itself before the op returned.
assert.equal(window.idle, "session released by the launcher", runnerLog(f, run));
const log = launchLog(f);
const end = (r) => log.find((e) => e.event === "end" && e.run === r);
assert.deepEqual([end(run).reason, end(run).exitCode, end(run).signal], ["stopped", 0, null]);
assert.ok(log.indexOf(end(run)) < log.findIndex((e) => e.event === "thread-idle" && e.run === run), "the end came before the idle stop's signal");
assert.doesNotMatch(runnerLog(f, run), /session stopped working/);
assert.match(runnerLog(f, run), /runner: exiting 0\n/);
// The launcher already released the claim, so the runner doesn't release it at exit (dewey-0878 N1).
assert.doesNotMatch(runnerLog(f, run), /runner: role\.release:/);
// The control: a real not-holder in the same window still ends 22, with no signal.
assert.equal(window.revoked, "session stopped working: not-holder", runnerLog(f, control.run));
assert.deepEqual([end(control.run).reason, end(control.run).exitCode, end(control.run).signal], ["session-invalid", 22, null]);
assert.match(runnerLog(f, control.run), /runner: exiting 22\n/);
await host.close();
const store = new Store(f.dataRoot);
t.after(() => store.close());
const codes = (r) => store.all("SELECT body FROM events WHERE kind='action.refused' AND actor_run=? ORDER BY seq", r).map((e) => JSON.parse(e.body).code);
assert.ok(codes(run).includes("launcher-released"), JSON.stringify(codes(run)));
assert.ok(!codes(run).includes("not-holder"), JSON.stringify(codes(run)));
assert.ok(codes(control.run).includes("not-holder"), JSON.stringify(codes(control.run)));
});
+1 -1
View File
@@ -293,7 +293,7 @@ it never touches the disk and isn't in the environment or argv.
| Exit | Meaning | Launch end reason |
|---|---|---|
| 0 | stopped | `stopped` |
| 0 | stopped: by a signal, or the launcher's idle stop or land-relaunch released the claim (`launcher-released`) | `stopped` |
| 2 | bad run directory or stdin | `failed` |
| 20 | founder credentials | `founder-credentials` |
| 21 | `role.claim` refused | `claim-refused` |
+25 -18
View File
@@ -53,6 +53,9 @@ export const FOUNDER_ENV = Object.freeze([
const USABLE = new Set(["valid", "expiring", "rotation-due"]);
// Capabilities that stopped working: the run ended, the claim was revoked.
const SESSION_GONE = new Set(["unauthenticated", "not-holder", "run-ended"]);
// Row 302 (#1809): the launcher's idle stop or land-relaunch released the claim and will stop the run.
// That is a stop, not a lost session, so the run exits stopped even when its call lands before the signal.
const LAUNCHER_RELEASED = "launcher-released";
const BODY_MAX = 32000;
const DEFAULTS = Object.freeze({ turnTimeout: 900, pollInterval: 2000, brokerRetries: 30 });
@@ -482,6 +485,20 @@ export async function main(runDir, { stdin = process.stdin, env = process.env, l
let code = EXIT.stopped;
let failures = 0;
// Whether a broker refusal ends the run, and how: a launcher release stops it, a gone session is 22.
// The reply site has logged its failure already, so it passes say=false.
let released = false;
const ended = (err, say = true) => {
if (err.code === LAUNCHER_RELEASED) {
log("runner: session released by the launcher");
released = true;
return true;
}
if (!SESSION_GONE.has(err.code)) return false;
if (say) log(`runner: session stopped working: ${err.code}`);
code = EXIT.session;
return true;
};
let n = 0;
const sleep = (ms) => new Promise((r) => {
const t = setTimeout(r, ms);
@@ -500,9 +517,8 @@ export async function main(runDir, { stdin = process.stdin, env = process.env, l
try {
block = stateBlock(await client.call("state"), new Date().toISOString());
} catch (err) {
if (SESSION_GONE.has(err.code)) {
log(`runner: session stopped working: ${err.code}`);
code = EXIT.session;
// A run in its landing turns hasn't received, so landRelease keeps it; thread runs have no landing.
if (ended(err)) {
gone = true;
break;
}
@@ -524,11 +540,7 @@ export async function main(runDir, { stdin = process.stdin, env = process.env, l
messages = await client.call("message.receive");
failures = 0;
} catch (e) {
if (SESSION_GONE.has(e.code)) {
log(`runner: session stopped working: ${e.code}`);
code = EXIT.session;
break;
}
if (ended(e)) break;
if (++failures >= session.brokerRetries) {
log(`runner: broker unreachable after ${failures} tries: ${e.code ?? e.message}`);
code = EXIT.broker;
@@ -545,11 +557,8 @@ export async function main(runDir, { stdin = process.stdin, env = process.env, l
try {
block = stateBlock(await client.call("state"), new Date().toISOString());
} catch (e) {
if (SESSION_GONE.has(e.code)) {
log(`runner: session stopped working: ${e.code}`);
code = EXIT.session;
break loop;
}
// Neither release happens while a delivered message is unread, so this sees one only by a bug.
if (ended(e)) break loop;
unread = `could not read the current state: ${e.code ?? e.message}`;
}
++n;
@@ -576,17 +585,15 @@ export async function main(runDir, { stdin = process.stdin, env = process.env, l
await client.call("message.read", { id: m.id });
} catch (e) {
log(`runner: reply to ${m.id} failed: ${e.code ?? e.message}`);
if (SESSION_GONE.has(e.code)) {
code = EXIT.session;
break loop;
}
// As for the state above: the message isn't read yet, so no launcher release has happened.
if (ended(e, false)) break loop;
}
}
if (!messages.length) await sleep(session.pollInterval);
}
server.close();
rmSync(session.toolSocket, { force: true });
if (code === EXIT.stopped) {
if (code === EXIT.stopped && !released) {
try {
await client.call("role.release");
} catch (e) {
+2 -2
View File
@@ -768,8 +768,8 @@ test("a run in a landing turn isn't released for a second held decision until it
);
assert.equal(ctx.store.get("SELECT count(*) AS n FROM deliveries WHERE holder_run='coder-run' AND op='read'").n, 2);
const out = await r.exited;
assert.equal(out.code, EXIT.session, out.stderr);
assert.match(out.stderr, new RegExp(`runner: landing turn 1 for decision ${d1.id} \\(vikunja:1/2\\): ok\n[\\s\\S]*runner: session stopped working: not-holder\n`));
assert.equal(out.code, 0, out.stderr);
assert.match(out.stderr, new RegExp(`runner: landing turn 1 for decision ${d1.id} \\(vikunja:1/2\\): ok\n[\\s\\S]*runner: session released by the launcher\n`));
});
test("a malformed landing exits 2 before the claim, with no turn", async (t) => {