Compare commits

..
Author SHA1 Message Date
jason.woltje 2d0d407ba6 row 269 (264b, #1774): the M6 halt test halts after mayLaunch answers
A halt inside the op reaches the bus's own fence first (runtime.mjs:564-566,
processingWork at :352-356), so the launcher refused limit-check-failed
(launcher.mjs:389-391) before its recheck admission() at :397. First-run red of
a new test on 6d9ab3503 early signal, placed under mos-1272.
2026-10-22 10:41:40 -05:00
jason.woltje 6d9ab35039 row 269 (264b, #1774): BUILD.md names the sessioned-survivor residual in Q2 and records N1's mos-1292 allowance (duckworth-0159 N3, N4) 2026-10-22 10:15:32 -05:00
jason.woltje 2c16bc4209 row 269 (264b, #1774): BUILD.md Q2 notes that a survivor's released claim doesn't prove its run has exited, and the per-leftover 5 s wait (duckworth-0158 N1, N2) 2026-10-22 10:14:45 -05:00
jason.woltje 9221786228 row 269 (264b, #1774): BUILD.md answers sage-2750 on the recover() change; the survivor log names its pids
The section "recover() change, for a design read" covers three things, with
file:line against aca16404c. Every leftover is still bound and ended as
host-lost, and a survivor only refuses legacy-run-live while it lives. The
5 s wait applies only in the bwrap kill branch. No old test's expected value
changes. The fallback is to revert to the base recover() and leave the
refusal untested. The survivor log line now names the outer and runner pids
so an operator can clear it, and the new test's log assertion matches.
2026-10-22 10:14:11 -05:00
jason.woltje 3d29876dfb row 269 (264b, #1774): legacy-run-live gets its test (duckworth-0156)
recover() kept a survivor only when a bwrap outer pid outlived SIGKILL, so
the Q2 refusal (BUILD.md:177-179, t1-sessions.md:109) had no test. A leftover
whose runner still ran after its outer pid was gone was ended and dropped.
Now recover() waits on the outer pid and the runner, keeps any leftover where
either still runs, and logs that it still runs. check() reads only the
survivors that still run.

session-launch.test.mjs adds a test where a sessionless leftover's runner is
a sleep child and its outer pid is gone. It asserts the legacy-run-live
refusal with no launch and no session workspace, then one launch once the
child exits. BUILD.md adds mutants M10-M12 and replaces the "no test" note.
2026-10-22 10:11:49 -05:00
jason.woltje a81b3446eb row 269 (264b, #1774): refusedOnce in sessions.test.mjs pins actor_run, closing Duckworth N1 (DEFERRED.md:474-480)
Allowed by Mos in mos-1292 (copy mos-1294) on mos-1285's terms. The SELECT
reads actor_run, the expected row gets the run-name literal, and the three
callers pass `${name}-run`. Nothing else in the test changes. The old
expected row was:

  [['action.refused', { code: 'not-holder' }]]

DEFERRED.md gets a closing line on the entry naming row 269.
2026-10-22 10:09:37 -05:00
jason.woltje d595570a63 row 269 (264b, #1774): per-session store, workspace and single-writer lock
threadsState gives one row per session with `session` and `legacy`; threadIdle
idles a run against its bound session. Each session gets its own store
(threads/<b>/<t>/sessions/<s>/, or the thread's old store for a legacy-3l
session) and its own workspace (session-workspaces/<b>/<i>/<s>/), held by an
O_EXCL <ws>.lock from before the clones to the run's proven exit. A legacy
session's workspace binds in-sandbox at the old instance path. sessions.json
is v2 and reads v1. New tests: cli session-launch.test.mjs, seat
session-workspace.test.mjs. Not yet run: tests wait for Sage's CLEAR.

Two old assertions change as exact updates:
- mos-1285: end-launch.test.mjs threadsState deepEqual. The old row shape was
  { thread: 'th-1', instance: 'coder', pending: 0, unread: 0, lastMessage: null,
  lastOpen: '2026-10-15T00:00:01.000Z', lastClaim: null, holder: null }; the
  row keeps every key and value and adds session: 'th-1', legacy: true.
- mos-1289: thread-launch.test.mjs:182, old line
  `const store = join(f.dataRoot, "threads", "acme", A);`
  now reads A's default session id from threadsState's `session` key; :183-185
  unchanged; one added assert that threads/acme/<A>/session is absent.
2026-10-22 10:06:42 -05:00
jason.woltje e2e51bae1f row 269 (264b, #1774): BUILD.md lists thread-launch.test.mjs:182 as a second old assertion that changes. A thread.create thread's default session isn't legacy-3l, so its store takes D2's layout (t1-sessions.md:100, :294). Proposed as an exact path update, for Mos through Sage 2026-10-22 09:53:09 -05:00
jason.woltje f684317eb0 row 269 (264b, #1774): BUILD.md drops the planned bindLaunch session-running refusal. Two old tests bind a second unended run to one session and expect the claim refusal (threads.test.mjs:88-89, sessions.test.mjs:119-120), so the broker half of the fence is the existing per-session claim, thread-already-held (broker.mjs:4815) 2026-10-22 09:50:49 -05:00
jason.woltje 04676f57b9 row 269 (264b, #1774): design notes for Duckworth's early look (duckworth-0150). C1 adds the Console and R-guard sessions.json readers, the 264a refusals a v2 file causes, the /proc live check and the Console restart; C2 pins M6 to a generation change after check() with all eight start() admission calls named; C3 gives the realpath compare its own code and a symlinked workspaces root as M4's input; C4 lists the old threadsState assertions, end-launch.test:193 the only change, under mos-1285 2026-10-22 09:45:00 -05:00
jason.woltje 304bce2f13 row 269 (264b, #1774): design notes for sage-2717. Rollback names the four steps past the 264a exit-3 refusal and the re-pin plan section that carries them; new M8 binds a non-default session at the legacy path, old M8 becomes M9; Duckworth's legacy-bind check 2026-10-22 09:40:16 -05:00
jason.woltje 875b7eeebf row 269 (264b, #1774): design notes. D9 audit of every sessions.json reader at 84f53540d and of threadsState's consumers; sessions.json goes to v2 because recover() rebinds without session; pi 0.85.1 resumes by cwd, so the legacy session keeps its in-sandbox path; mutants M1-M8 and run limits (sage-2707) 2026-10-22 09:37:39 -05:00
12 changed files with 1335 additions and 93 deletions
+449
View File
@@ -0,0 +1,449 @@
# Row 269 build notes (Gizmoduck)
264b, T1 item 7: per-session native store, workspace and single-writer fence. Gitea #1774. Brief:
docs/plans/2026-10-21_t1-sessions.md (blob ff0be0c79), slices in agents/sage/work/queue-264/RULINGS.md
(5f5b51dd6). Reviewer Duckworth, gate owner Sage. Plan gizmoduck-0228, accepted with rulings in
sage-2707. Branch gizmoduck/row269 from refactor aca16404c. Row 253 lands first, and this branch is
rebased onto its tip, under a new name, before the gate.
Claim: rev 3217, op gizmoduck-0230, 2026-10-22 14:29:42Z. That was 21 s after Gander's slot opened
at 14:29:21Z (sage-2710). Reported in gizmoduck-0231 and not reverted, since a revert would be a
second queue write.
## D9 consumer audit, sessions.json (sage-2707 Q3)
Q3 allows `session` as an optional field with no version bump, if every sessions.json reader at
84f53540d selects by run and not by business, instance or thread. Every reader at 84f53540d:
| Reader | file:line at 84f53540d | Selects by |
| --- | --- | --- |
| `readSessions` (the only parser) | packages/seat/src/session.mjs:232-243 | nothing. Refuses any `sessionsVersion` but 1 (:241) and returns every entry with `live` |
| `stopSession` (`mosaic stop <run>`) | session.mjs:249-250 | run (`s.run === run`) |
| `launches list` | packages/cli/src/cli.mjs:526-530 | nothing. Prints every entry, and `--json` dumps them whole |
| host start, `leftover` | packages/cli/src/launcher.mjs:235 | business, then `recover()` works one entry (one run) at a time (:955-972) |
| runtime launch settings | packages/bus/src/runtime.mjs:149 -> packages/business/src/launch-settings.mjs:166 -> packages/business/src/launch.mjs:116-135 (`usageCounts`) | business. Counts entries per provider and model |
| Console settings (C1, duckworth-0150) | packages/webui/src/reads.mjs:234 -> the same launchSettings `read()` -> readSessions | business, through usageCounts. Any throw becomes `launch: {refused: 'not-configured'}` |
| stack-release R guard (C1) | scripts/stack-release.sh:2091-2096, its own JSON.parse, no version check | nothing. Takes `pid` and `runnerPid` of every entry in `sessions` and fails `sessions-live` if one is alive in /proc |
Not a reader: stack-release.sh:2199-2204 checks only that no sessions.json exists. packages/ledger
readSessions (ledger.mjs:146) is a different function that reads seat directories.
launch-settings `current()` (:129) doesn't call `sessions()`, so the limits the bus enforces don't
read the file; only the `limits` verb's `read()` (:166, runtime.mjs:292) does.
Tests that read the file: packages/cli/tests/launcher.test.mjs (`readSessions` at :164 through
:1097, each by run, by instance or whole) and packages/cli/tests/host.test.mjs:830 (file absent).
Not a file reader: `Registry` (session.mjs:192-230). Its list starts empty (:193), and the constructor
writes it over the file (:198). `running(business, instance, thread)` at :217 is the one key on the
thread, but it reads only the in-memory list this host added (launcher.mjs:529, removed at :553).
Its callers are launcher.mjs:289 (check) and the thread pass. This row changes that key (see Design).
Result: no file reader keys on the thread. Q3's condition holds as worded.
One more finding sends me to v2 anyway. `recover()` rebinds a leftover run with
`{business, role, run, harness, pid, startTime, thread}` and no `session` (launcher.mjs:964).
The broker compares that record's body with the recorded `session.launched` body, and the body
carries `session` whenever the launch record had one (broker.mjs:945). A mismatch refuses
`launch-record-mismatch` (broker.mjs:952). Each new thread run binds with its session (Design), so
after a rollback to 264a, a 264a host that finds a 264b entry logs "not ended, bindLaunch refused:
launch-record-mismatch" (launcher.mjs:968), and the run's claim stays held. The host has
already killed the process (:956-962), but it can't end the run. With v1 and an optional field,
that failure is quiet.
Decision: sessions.json goes to v2 (`SESSIONS_VERSION` 2). New code reads v1 and v2 and writes v2.
A v2 file keeps the `sessions` array and every v1 field of each entry, `pid` and `runnerPid`
included, so the R guard (stack-release.sh:2094-2095) reads it unchanged (C1).
A v1 entry has no `session`, and it is treated as the thread's legacy default session (D9). After a
rollback, a 264a host refuses to start with exit 3, "sessions file is malformed"
(session.mjs:241 via launcher.mjs:237), and the README's remedy applies: check which sessions still
run, then move the file aside. Tradeoff: a rollback while runs are live needs a hand step, but it
fails loudly at host start instead of leaving held claims. The rollback note names this refusal.
## D9 consumer audit, threadsState and threadIdle (sage-2707 audit addition)
These are trusted launcher APIs, not socket verbs (packages/bus/README.md:288, :294), and they're
reached only through the host op table (packages/bus/src/process.mjs:23-24, :54-55;
runtime.mjs:561-562). At aca16404c:
| Consumer | file:line | What it reads |
| --- | --- | --- |
| launcher thread pass | packages/cli/src/launcher.mjs:596, :611 | every row's `thread`, `holder`, `pending`, `unread`; threadIdle `{business, thread, run}` |
| bus end-launch.test | packages/bus/tests/end-launch.test.mjs:193, :198 | whole rows (deepEqual), and `not-holder` |
| bus thread-migration.test | packages/bus/tests/thread-migration.test.mjs:139 | `thread, holder, pending, unread` |
| bus threads.test | packages/bus/tests/threads.test.mjs:352, :375-376, :407, :416 | rows keyed by `thread` |
| cli launcher.test stub | packages/cli/tests/launcher.test.mjs:989 | wraps `host.op`, counts the op name |
| cli lifecycle.test stub | packages/cli/tests/lifecycle.test.mjs:90-91, :749-752 | passes through, wraps `runtime.threadsState` |
| cli thread-launch.test stub | packages/cli/tests/thread-launch.test.mjs:90-91, :163, :264-265, :304, :348, :443 | `.find(x => x.thread === ...)` then `holder` |
Console and webui: no reader. `git grep threadsState` over packages/webui, packages/control-board and
scripts has no hit. The webui reads `thread.claims` (packages/webui/src/serve.mjs, cli.mjs,
public/bus.js, tests shell.test, thread-claims.test, thread-claims-browser.test). This row doesn't change
`thread.claims`. The webui's `session.launched` line reads only `harness` and `address`
(public/bus.js:151), so a `session` field in the body changes nothing it shows.
## D9 consumer audit, workspace and native store
| Consumer | file:line at aca16404c | Change in this row |
| --- | --- | --- |
| `workspaceDir` | packages/seat/src/session.mjs:49 | unchanged, still the non-thread path and the legacy path |
| launcher start, workspace | packages/cli/src/launcher.mjs:397-403 | a thread run uses its session workspace (Q1) |
| launcher instance delete (V5) | launcher.mjs:841-866 | also retires the instance's session workspaces |
| `threadStore` | session.mjs:59-74, launcher.mjs:388-393 | becomes the legacy branch of `sessionStore` |
| sandbox binds | packages/seat/src/sandbox.mjs:102-125 | the workspace gets a separate in-sandbox path |
| harness `transcript()` | packages/plugins/harness/pi/index.mjs:75, claude-code/index.mjs:130 | none: `{session, projects}` keeps its shape |
| in-sandbox readers of `session.workspace` | packages/harness/src/runner.mjs:332, :386, :475-476; bundle.mjs:258, :327; gate.mjs:152 | none: they get the in-sandbox path |
| task runner workspaces | scripts/mosaic-task.mjs:351 | none: `<dataRoot>/workspaces/<task>` is a legacy root the overlap check covers |
| git socket | packages/cli/src/git-socket.mjs:7 | none: it takes bundles and reads no workspace |
## Pi and Claude resume, read offline (sage-2707: no pi run while P19 383 holds)
Pinned `@earendil-works/pi-coding-agent` 0.85.1 (package.json:8), read in node_modules/.../dist:
- adapters/pi/adapter.sh:46-52 passes `--session-dir "$MOSAIC_SESSION_DIR"`, plus `-c` when that
directory isn't empty.
- With `-c` and an explicit session dir, `continueRecent` filters by cwd
(core/session-manager.js:1244-1250). It opens the newest `*.jsonl` whose header cwd resolves to
the process cwd (:397-410, `sessionCwdMatches` :393-395). With no match, it makes a new session in
the same directory (:1251), and nothing fails.
- Both adapters `cd "$MOSAIC_WORKSPACE"` (pi adapter.sh:34-36, claude adapter.sh:46-47), the runner
sets `MOSAIC_WORKSPACE` from `session.workspace` (runner.mjs:386), and the sandbox binds the
workspace at its host path (sandbox.mjs:112, `argsOf` :128-131). So a legacy transcript's header
cwd is `<dataRoot>/workspaces/<business>/<instance>`.
- Opening a file whose header cwd doesn't exist exits 1 in a headless run (main.js:536-548,
core/session-cwd.js:2-15). So `--session <file>` is no way around it either.
Result: a legacy default session run from `session-workspaces/.../<session>` as its cwd would
quietly start a new pi conversation, and its old history wouldn't resume. Claude Code keys its
projects directory on the cwd, and the adapter resumes by id (claude adapter.sh:53-59). I haven't
read the Claude binary, so whether `--resume` finds an id saved under another cwd is unverified. I
treat it as the same break.
## Design
### Legacy mapping (D9, M7)
A session is legacy when its row has `op='legacy-3l'`, which the migration gave every pre-3l thread
with `id` = the thread id (3k-3l.sql:50-51). The rule reads only an immutable row, never the disk.
- A legacy session keeps the thread's existing store, `threads/<b>/<t>/{session,claude-projects}`
(`threadStore`, unchanged).
- Its workspace is new (Q1), `session-workspaces/<b>/<instance>/<session>/`, but inside the sandbox
it's bound at the legacy path `workspaces/<b>/<instance>`. The cwd its transcripts recorded is
unchanged, so pi `-c` and Claude `--resume` find them. The host's old instance workspace isn't
mounted, so it stays as history (Q1, D9). Every in-sandbox reader gets the sandbox path (table
above), and host git (`ensureCheckouts`) gets the host path.
- Every other session uses D2's layout for both, and its sandbox path is its host path.
- A thread made by 264a's `thread.create` has a random default id (broker.mjs:5993), so this rule
doesn't call it legacy. A 264a launcher ran that thread from `threads/<b>/<t>/session`. A start
for a session whose thread has no legacy-3l session, while `threads/<b>/<t>/session` exists,
refuses `legacy-store-unmapped`, so it fails closed and doesn't start a fresh conversation. The
re-pin plan lists such threads. If 264a and 264b go live together, there are none.
### Broker
- `threadsState(business)`: one row per session of every thread, not just the default, ordered by
thread seq, then session seq. Each row adds `session` and `legacy` to the fields it has now.
`#threadRows` joins `sessions` in place of `session_defaults` (broker.mjs:868-896).
- `threadIdle(record)`: same keys. The row is the one for the session the run was bound to
(`#runs` entry, broker.mjs:927), so a run is idled against its own session.
- `bindLaunch`: no new refusal. It already takes `session` and puts it in the body (broker.mjs:945).
The broker half of the single-writer fence is the per-session claim that exists now:
a second claim on a held session refuses `thread-already-held` (broker.mjs:4815). A
`session-running` refusal at bind would break two old tests that bind a second unended run to one
session and then expect the claim refusal (threads.test.mjs:88-89, sessions.test.mjs:119-120).
The launcher's registry check and the workspace lock are the host halves.
### Seat
- `sessionStore(dataRoot, business, thread, session, { legacy })`: the legacy branch is
`threadStore`. Otherwise it's `threads/<b>/<t>/sessions/<s>/{session,claude-projects}`, with
`threadStore`'s checks on every segment and directory (ID_PATTERN, no `..`, owner, 0700, no
symlink), giving the same `{dir, session, projects}` shape.
- `sessionWorkspace(dataRoot, business, instance, session, { live })`: makes
`session-workspaces/<b>/<i>/<s>` 0700 with the same checks, then compares realpaths. It refuses
`session-workspace-overlap` when the path is inside, or contains, `<dataRoot>/workspaces` (legacy,
`.retired` and task workspaces) or any workspace in `live`. A segment that fails the lstat checks
refuses `session-workspace-unsafe` first (C3).
- Lock: `lockSessionWorkspace(path, owner)` makes `<path>.lock` with O_EXCL, next to the workspace
and outside every mount. It holds the host's pid and start time, and the run's outer pid and
start time once spawned. A lock whose host and run are both gone is stale and is taken over.
Anything else refuses `session-workspace-busy`. start() takes it before `ensureCheckouts` and
releases it only after the run's `close` (proven exit) or a failed start.
- `Registry`: entries gain `session` and `workspace`, and the key becomes `running(business,
instance, thread, session)`. `SESSIONS_VERSION` is 2, and `readSessions` reads 1 and 2.
### Launcher
- check(instance, { thread, session }): `instance-running` is per session. Q2 adds
`legacy-run-live`: a start for a thread instance refuses while any registry entry or unrecovered
leftover for that instance has no `session`.
A survivor is still bound and ended host-lost (launcher.mjs:1030-1040), so the broker sees its
claim free while it runs. A released claim doesn't prove the run has exited. Only this launcher
check holds the instance then, for sessionless survivors and thread starts only (duckworth-0158
N1). recover() waits up to 5 s per leftover for the outer pid and the runner (N2, see "recover()
change, for a design read").
Residual (duckworth-0159 N3): a survivor with a `session` counts nowhere, because :309 counts
only sessionless entries. The same session could start again while the old runner lives. The
lock doesn't catch it either: it records the outer pid as runPid (launcher.mjs:574), so with the
outer pid gone it reads stale and is taken over. The base has the same exposure. In practice
the runner dies with its pid namespace when the outer bwrap is killed.
- start(): the store and workspace come from the session. The lock covers checkouts and spawn.
`bindLaunch` gets `session`, and the entry records it.
- threadPass(): one pass per session row. `startedFor` and `refusedThreads` key on the session.
The `thread-idle`, `refused` and `end` records add `session`.
- recover(): passes the entry's `session` when present.
- remove(): moves `session-workspaces/<b>/<instance>` to `session-workspaces/<b>/.retired/<instance>-<transaction>`
after the legacy move, and records `workspace-retired` with that path.
- Admission (RULINGS Q5, D8): a named session starts only through threadPass, then check(), then
start(), so it passes through every `admission()` call start() makes now (launcher.mjs:382-524).
There is no second start path.
### Sandbox
`binds()` takes `workspaceDest` (default `workspace`), the in-sandbox path for the rw workspace
bind. The store binds keep their shape: only the selected session's two directories.
## Tests
T1 run half, with the fake harness adapter from thread-launch.test (no pi or Claude binary):
two sessions of one thread run at once, each with its own sentinel history, store and workspace;
stopping one and restarting it resumes only that one. The legacy session resumes from the old
store at the old in-sandbox cwd. A non-thread instance gets `thread-not-allowed`.
T2: a second start on a live session's workspace refuses `session-workspace-busy` before any
checkout; a hand-made live lock from another host refuses; a symlinked or nested session root
refuses `session-workspace-unsafe`; a workspace root symlinked into the session path refuses
`session-workspace-overlap`; a store segment symlink refuses `thread-store-refused`. Under a halt, a
named session's start refuses `processing-halted` and spawns nothing. A generation change after
check() refuses `processing-generation-changed` and spawns nothing (M6).
N1 (DEFERRED.md:474-480): `refusedOnce` in packages/bus/tests/sessions.test.mjs pins `actor_run`.
## Old assertions that change (C4, duckworth-0150, mos-1285)
Every pre-existing test that reads threadsState at aca16404c, checked against one row per session:
- packages/bus/tests/end-launch.test.mjs:193-195 deepEquals the whole row. This one changes. Mos
allowed it in mos-1285 as an exact update: still a whole deepEqual on threadsState('demo'), still
exactly one th-1 row, every old key and value the same, and two new keys with literal values,
`session: 'th-1'` (the fixture's legacy default, :14-15) and `legacy: true` (op `legacy-3l`). The
commit message names mos-1285 and the old row shape.
- packages/bus/tests/threads.test.mjs:352, :375-376: fromEntries keyed by thread. Neither test, and
neither fixture (`twoThreads`), makes a second session (no `session.create` in the file), so each
thread still has one row. Unchanged.
- packages/bus/tests/thread-migration.test.mjs:139: maps four fields of a one-session thread.
Unchanged.
- packages/cli/tests/thread-launch.test.mjs:163, :304, :348, :443: `find` by thread. One session
per thread in every fixture. Unchanged. The default session also sorts first, by session seq.
- packages/cli/tests/lifecycle.test.mjs:90-91, :749-752 and thread-launch.test.mjs:90-91, :264-265
pass threadsState through. Unchanged.
- packages/cli/tests/thread-launch.test.mjs:182 (found in the code round, after mos-1285). The test
makes threads with `thread.create` (:171-172), so A's default session has a random id and op
`default` (broker.mjs:5992-6009), not `legacy-3l`. The brief puts every new store in D2's layout
(t1-sessions.md:100, :294), so A's store moves to `threads/acme/<A>/sessions/<A's session>`. The
proposed exact update changes only the `store` path on :182, taking the session id from
`threadsState` (the test already reads it at :162). Lines :183-185 stay as they are. One added
line asserts that `threads/acme/<A>/session` doesn't exist. This goes to Mos through Sage, and no
code commit edits the file until Mos rules.
- The same file's :375 (`../evil` refuses `thread-store-refused`) and :381, :384 (no `threads`, no
`workspaces`) stay unchanged: start() checks the store before it makes the session workspace.
Any other old assertion the code changes goes to Mos through Sage before the commit. After the
first run I survey every touched file whole for one.
## Mutants
Each kill counts only as an assertion failure in a test that runs that path, never a throw or
another fence (sage-2707, C3 duckworth-0128).
| Id | Change | Killed by |
| --- | --- | --- |
| M1 | `sessionWorkspace` drops the session segment: a thread's sessions share one checkout | T2 same-thread fence: `session-workspace-busy`, no second spawn |
| M2 | the session segment becomes the session name: two threads' `default` sessions share one checkout | T2 cross-thread fence, same assertion |
| M3 | the lock ignores a live holder | T2 hand-made lock: `session-workspace-busy` |
| M4 | the realpath overlap check removed | T2 nesting: `session-workspace-unsafe` |
| M5 | the launcher passes the thread's default store for every session | T1: the second session's sentinel history is its own |
| M6 | start() calls a no-op in place of every `admission(admissionGeneration)` (launcher.mjs:382, :396, :407, :420, :495, :499, :520, :524) for a non-legacy session | the generation test (below): `assert.rejects` with code `processing-generation-changed`, then `assert.equal` spawn count 0 |
| M7 | the legacy session maps to `sessions/<s>/` | the D9 test: the legacy run resumes the sentinel in `threads/<b>/<t>/session` |
| M8 | a non-default session binds its workspace at the legacy `workspaces/<b>/<instance>` in-sandbox path | T1: `assert.equal` that a named session's in-sandbox workspace is its `session-workspaces/<b>/<instance>/<s>` path, read from the fake adapter's recorded cwd (sage-2717) |
| M9 | `#threadRows` back to the default session only | T1: the named session's message starts its run |
| M10 | the `legacy-run-live` refusal removed from check() | the legacy-run-live test: `outcome` equals `[[S, "legacy-run-live"]]` |
| M11 | recover() keeps a survivor only when the outer pid outlived the kill (the runner ignored) | the legacy-run-live test, same assertion |
| M12 | check() reads every survivor, live or not | the legacy-run-live test: after the runner exits, `settle` gives one launch |
M6 and the halt fences (C2, duckworth-0150). A plain halt never reaches start(). tick() admits first
(:579) and runs no thread pass while halted (:583), and check() admits at :282 and :376. The kill
test lands the change between check() returning and the spawn:
- The fake host's `credentialStatus` op (start() calls it at :395, after check() returned and after
:382) records that it ran, then halts and resumes processing through `transition`. That leaves
`halted` false and a new generation (state.mjs:115-121).
- The tick and check fences see no halt, and start()'s second check() (:419) takes its own
generation, so it passes too. withProcessing at :406 has no parent context (the launcher never
opens one) and checks only `halted`, so it passes. The one fence left is start()'s own
`admission(admissionGeneration)`, which refuses `processing-generation-changed` at :396.
- The test asserts the hook ran (check passed), then the refusal code, then no spawn.
M6 removes all eight calls because removing only :382 leaves :396 to refuse. The T2 halt test stays
as written in Tests, but it doesn't kill M6 and isn't counted for it.
M4 and the segment checks (C3, duckworth-0150). The lstat check on each segment refuses a symlinked
session root before the realpath compare runs, so the compare gets its own code,
`session-workspace-overlap`, and the segment check keeps `session-workspace-unsafe`. M4's kill input
passes every segment check: `<dataRoot>/workspaces/<b>` is a symlink to
`session-workspaces/<b>/<i>/<s>`. Each segment of the session path is a real directory, and only the
realpath compare finds that the session path contains the legacy workspace root. The test asserts
`assert.rejects` with code `session-workspace-overlap` and no spawn. With M4 the start spawns, so
the rejection assertion fails. The segment test keeps asserting `session-workspace-unsafe` for a
symlinked session root, so it tells the two codes apart.
Duckworth's check for the early look (sage-2717): a non-default session never gets the legacy
`workspaces/<b>/<instance>` bind path. Only a session with op `legacy-3l` takes `workspaceDest`
set to the legacy path. M8 is that mutant.
### Code round corrections (local, not yet run)
Written before the first run, which waits for Sage's CLEAR.
- Tests live in two new files: packages/cli/tests/session-launch.test.mjs (T1, D9, T2) with a new
stub adapter, fixtures/session-adapter.mjs (thread-adapter.mjs's turn plus the run's cwd and its
entries; thread-adapter.mjs is unchanged), and packages/seat/tests/session-workspace.test.mjs
(sessionStore, sessionWorkspace, the lock, sessions.json v2). The new launch tests copy
thread-launch.test.mjs's host helpers rather than change them, so no other old test file changes.
- M1 and M2 kill in T1, not in a busy test. A test that expects `session-workspace-busy` still
passes when sessions share a path. T1 starts thread A's default and named sessions and thread B's
default at once and asserts no refusal, three live entries and three distinct cwds. With M1 (A's
two sessions) or M2 (the two `default` sessions) the second start refuses busy, and the
`deepEqual(refused, [])` assertion fails.
- Every wait a mutant could stall returns what it last saw (`settle`, `outcome`), so a kill is the
assertion after it, never a timeout. That covers M3, M4, M6 and M9, where the mutant's run starts
or never starts.
- The M6 test also carries the halt fence. start() admits right after credentialStatus, before it
makes the session workspace, so the first refusal leaves no workspace. On the retry the recheck's
mayLaunch op halts after the lock is taken. The recheck refuses `processing-halted`, and the test
asserts that the lock is gone (startLocked's throw releases it).
- The lock's release() runs once. A second call would unlink a lock another host has written since.
- `legacy-run-live` now has a test (Duckworth's early warning in duckworth-0156). Before, recover()
kept a survivor only when a bwrap pid outlived SIGKILL, which a test can't make. It also ended
a leftover whose runner still ran once the outer pid was gone, so the refusal never saw that run.
Now recover() waits up to 5 s on the outer pid and the runner. It keeps any leftover where
either still runs, and logs `run <run> (<instance>) from an earlier host still runs`. check()
reads only survivors that still run, so a start goes ahead once the run has exited. The test
writes a sessionless leftover whose runner is a sleep child and whose outer pid is gone. It
asserts the refusal, no launch, no session workspace, and then one launch after the child is
killed. M10-M12 cover it. The registry and pre-recover leftover terms stay. A launching host
can't reach them for a thread instance (check() refuses `session-required` first, and listen()
runs recover() first), but they cost nothing.
### recover() change, for a design read (sage-2750)
Sage asked for this because recover() predates the row. Lines are launcher.mjs at 3d29876df plus
the log-line edit after it, against aca16404c:947-976.
- What stays the same: every leftover is still bound and ended as host-lost (:1030-1040, base
:963-973). The survivor push (:1026-1029) comes before the bind, and the loop carries on past it,
so a kept leftover isn't left unended. Its bus run ends and its role is released, as in the base.
The kill is unchanged too: SIGKILL only to a live outer pid whose cmdline is bwrap (:1018-1021,
base :956-959).
- What changes:
1. The wait after the kill runs while the outer pid or the runner still runs (:1023). In the
base it waited on the outer pid only (base :961).
2. A leftover where either still runs after that goes on `survivors` and is logged (:1026-1029).
3. check() counts a survivor only while it still runs (:309, `liveRun` at :255).
- a. Nothing in this row kills a kept leftover's runner. The base didn't either: it ended the run
and let the runner go on, unseen. The run is still ended (:1038), so the runner holds no role.
Its capability died with the old broker (:1004-1008). The only thing a survivor does is refuse
`legacy-run-live` for sessions of its instance while it lives, and each refusal is a `refused`
launch record. For an operator, the log line names the run, the outer pid and the runner pid
(:1028). Killing that pid clears it. check() re-reads liveness on each pass (:309), so the next
tick starts the session with no host restart. If bindLaunch refuses (:1032-1036), the run stays
unended and logged, as in the base (base :965-969). The survivor entry doesn't change that.
- b. The 5 s wait only applies inside the kill branch (:1018-1024), so only to a leftover whose
outer bwrap was still alive. A leftover with nothing running doesn't wait, and nor does one
whose outer pid is gone: neither enters the branch. The wait ends as soon as both processes are
gone. The base already waited up to 5 s in the same branch. It can now last longer only if the
runner outlives its outer pid. With `--unshare-pid` and `--die-with-parent` the runner dies when
the namespace goes.
- c. Old tests that run recover() with a launching host and a leftover:
- launcher.test.mjs:885-943, the "host that died hard" pair. It asserts the outer pid and the
runner are gone right after setup (:917-918), the host-lost log (:919), one session.ended
(:920-922), one end record (:924), an empty sessions.json (:925), and then "already ended" for
a leftover whose processes are dead (:937-941). None of these expected values change.
Waiting on the runner makes :918 hold sooner, not later. The new log line can appear only if
a process outlives 5 s, which would already fail :917-918. The dead-process third host takes
no wait and logs no survivor.
- launcher.test.mjs:950-971 (`--no-launch`) returns before the loop (:1012-1015). It's
unchanged.
- stack-release.test.mjs:1928 writes `{pid: process.pid}` with no start times. `liveRun` is
false for it, and that test reads it through bus status, not recover().
- verbs.test.mjs:213 and the seat tests build a Registry with no launcher, so they aren't
affected.
No expected value in an old test changes, so nothing goes to Mos under mos-1272 for this. If a
run shows otherwise, I stop and send it before any change.
- Fallback, if Duckworth or Sage doesn't accept this: revert recover() and check() to the base
behaviour, keeping only the sessionless check over the registry, the pre-recover leftovers and
bwrap survivors. Drop the legacy-run-live test and M10-M12, and name the untestable path here.
- N1 (refusedOnce pins `actor_run`) changes a pre-existing test. Mos allowed it in mos-1292 (copy
mos-1294) on mos-1285's terms, and it's committed as a81b3446e, with a closing line on the
DEFERRED.md entry.
## Rollback
264b writes sessions.json v2. A 264a host given that file refuses to start, exit 3, "sessions file
is malformed" (session.mjs:241 via launcher.mjs:235-237). Session workspaces and per-session stores
stay on disk, and a 264a host never reads them.
What else refuses on a 264a stack while a v2 file is in place (C1, duckworth-0150). All of them go
through the 264a readSessions, which throws SeatError "sessions file is malformed" (session.mjs:241):
- `mosaic launches list` and `mosaic stop <run>`: exit 2 with that message (cli.mjs:526, :465).
- the bus `limits` verb: `settings.read()` throws (runtime.mjs:149, :292, launch-settings.mjs:166),
so the verb fails with that message. The limits the bus enforces still work, since `current()`
doesn't read the file. I haven't traced the code a bus caller sees for that throw.
- the Console settings view: launch shows `not-configured` (reads.mjs:234).
- the R guard keeps working (above).
All of these read the file on each call, so step 3 below clears them without a restart.
Console restart: the shape of what the Console reads doesn't change. usageCounts still counts
entries per provider and model, and reads.mjs keeps its fields. What changes is the file, and a
Console still on 264a code shows launch `not-configured` from the first v2 write until a human
restarts it on a 264b copy. The re-pin that carries this row needs its Console restart (Jason or
Mos), done right after the host's, as it does for a read-shape change. Step 3 covers the reverse
case after a rollback with no restart.
Seeing live sessions without `launches list` (C1): read the file directly the way the R guard does.
For each entry in `sessions`, a session is live if `/proc/<pid>/stat` or `/proc/<runnerPid>/stat`
exists and its state isn't `Z` (stack-release.sh:2080, :2095). For a stronger check, compare
`runnerStartTime` with field 22 of `/proc/<runnerPid>/stat`, which is what readSessions' `live`
does through startTimeOf.
How a rollback host gets past exit 3 (sage-2717), run on NEW before the rollback activate:
1. Stop the bus host on NEW. Its stop SIGKILLs what is left and ends those launches as `killed`
(packages/cli/README.md:470).
2. Read `sessions.json` with NEW's readSessions (`mosaic launches list`, cli.mjs:526-530). Go on only
if it shows no live entry: every listed runner is gone (pid and start time no longer match). If
NEW's CLI is already gone, use the /proc read above.
3. Move `sessions.json` aside under a dated name, never delete it (packages/cli/README.md:478).
4. Then roll back. The 264a host starts with no leftover file and nothing to recover.
A live entry at step 2 stops the rollback until that run ends or `mosaic stop <run>` ends it.
Which re-pin plan step carries it: no re-pin plan names 264b yet. The 84f53540d re-pin carries 264a
only. The first re-pin whose NEW includes this row puts steps 1-4 in its "Stop conditions and
rollback" section, before `rollback --plan`, the section the 10-24 plan uses
(agents/sage/work/window-plan/repin-1024-draft1.md:144-148). I'll name it to whoever drafts that
plan when the row lands.
## Run limits (sage-2707)
Each node --test file and suite runs under `timeout -k 10 LIMIT`, and each scripts/test-*.sh under at
least 3x its longest recorded run. Records are from agents/gizmoduck/work/queue-264/BUILD.md "Run
limits" unless named.
| Run | Longest recorded | LIMIT |
| --- | --- | --- |
| stack-release.test alone | 575 s (264a r12 survey) | 1800 s |
| each other touched file alone | 15 s (thread-launch.test); the rest have none | 300 s |
| bus suite | 23 s | 900 s |
| cli suite | 591 s (row 281 gate) | 1800 s |
| webui suite | 57 s | 600 s |
| seat suite | none recorded | 600 s |
| harness suite | none recorded | 600 s |
| each scripts/test-*.sh | 42 s (test-foundation) | 900 s |
| each mutant probe | 1 s wall (264a M1-M4) | 300 s |
+2
View File
@@ -478,6 +478,8 @@ at every gate. Started 2026-09-12 during the control board MVP.
not that the event's `actor_run` is the refused run. Binds row 269 (264b,
#1774), or the next row that changes `sessions.test.mjs`: pin `actor_run`
in `refusedOnce`. (2026-10-22)
Closed by row 269 (264b, #1774): `refusedOnce` selects `actor_run` and
expects the refused run's name (mos-1292). (2026-10-22)
- **editor-provenance skips its rebuild check on esbuild 0.28.1.** (Duckworth
N2 on the row 264a manifest round, duckworth-0146.) The webui test
+16 -13
View File
@@ -837,7 +837,7 @@ export class Broker {
return cap;
}
// Row 121 (#1615): what the launcher needs to start, stop and recover thread sessions.
// Trusted launcher API, not a socket verb. Per thread: messages to the instance that no
// Trusted launcher API, not a socket verb. Per session of each thread (row 269, 264b): messages to the instance that no
// run has been delivered yet, messages delivered to the current holder and not yet read
// (a turn in progress), and the latest message, open and claim times. A message delivered
// to an earlier run and never read (it crashed mid-turn) isn't the holder's turn, so it
@@ -857,30 +857,31 @@ export class Broker {
const session = this.#runs.get(key) ?? fail('unknown-run');
if (this.#ended.has(key)) fail('run-ended');
// A run holds at most one session, so this also refuses another thread and a non-thread run.
// Row 264a: the row is the thread's default session's, so a run of another session is refused too.
const [t] = this.#threadRows('t.business=? AND t.id=?', record.business, identifier(record.thread));
// Row 269 (264b): the row is the session the run was bound to, so a run is idled against its own.
const [t] = this.#threadRows('t.business=? AND t.id=? AND se.id=?', record.business, identifier(record.thread), session.session);
if (t?.holder !== record.run) fail('not-holder');
if (t.pending || t.unread) return { released: false, pending: t.pending, unread: t.unread };
this.#store.transaction(() => this.#claimEnd(session, this.#own(session), 'release', null));
return { released: true };
}
// Row 264a (#1769): each row is the thread's default session's. Until 264b the launcher starts
// default sessions only, so another session's messages wait.
// Row 269 (264b, #1774): one row per session of each thread, oldest thread first, then oldest
// session. `legacy` is a session the 3k-3l migration made (op legacy-3l), whose store and
// in-sandbox workspace path are the thread's old ones.
#threadRows(where, ...params) {
return this.#store
.all(
`SELECT t.id, t.instance,
(SELECT count(*) FROM messages m JOIN message_threads mt ON mt.message=m.id WHERE mt.session=ds.session AND m.to_role=t.instance
`SELECT t.id, t.instance, se.id AS session, se.op AS session_op,
(SELECT count(*) FROM messages m JOIN message_threads mt ON mt.message=m.id WHERE mt.session=se.id AND m.to_role=t.instance
AND NOT EXISTS(SELECT 1 FROM deliveries d WHERE d.message=m.id AND d.op IN ('delivered','read'))) AS pending,
(SELECT count(*) FROM messages m JOIN message_threads mt ON mt.message=m.id WHERE mt.session=ds.session AND m.to_role=t.instance
(SELECT count(*) FROM messages m JOIN message_threads mt ON mt.message=m.id WHERE mt.session=se.id AND m.to_role=t.instance
AND c.op='claim' AND EXISTS(SELECT 1 FROM deliveries d WHERE d.message=m.id AND d.op='delivered' AND d.holder_run=c.holder_run)
AND NOT EXISTS(SELECT 1 FROM deliveries d WHERE d.message=m.id AND d.op='read')) AS unread,
(SELECT max(m.at) FROM messages m JOIN message_threads mt ON mt.message=m.id WHERE mt.session=ds.session) AS last_message,
(SELECT max(at) FROM thread_opens WHERE session=ds.session) AS last_open,
(SELECT max(m.at) FROM messages m JOIN message_threads mt ON mt.message=m.id WHERE mt.session=se.id) AS last_message,
(SELECT max(at) FROM thread_opens WHERE session=se.id) AS last_open,
c.at AS claim_at, c.op AS claim_op, c.holder_run AS holder_run
FROM threads t JOIN session_defaults ds ON ds.seq=(SELECT max(seq) FROM session_defaults WHERE thread=t.id)
LEFT JOIN thread_claims c ON c.seq=(SELECT max(seq) FROM thread_claims WHERE session=ds.session)
WHERE ${where} ORDER BY t.seq`,
FROM threads t JOIN sessions se ON se.thread=t.id
LEFT JOIN thread_claims c ON c.seq=(SELECT max(seq) FROM thread_claims WHERE session=se.id)
WHERE ${where} ORDER BY t.seq, se.seq`,
...params,
)
.map((r) => ({
@@ -892,6 +893,8 @@ export class Broker {
lastOpen: r.last_open,
lastClaim: r.claim_at,
holder: r.claim_op === 'claim' ? r.holder_run : null,
session: r.session,
legacy: r.session_op === 'legacy-3l',
}));
}
// Trusted launcher API, deliberately NOT a socket verb. S6 supplies immutable launch records.
+1 -1
View File
@@ -191,7 +191,7 @@ test('broker process: launch ops authorize role.launch, record refusals and end
assert.deepEqual((await ask(child, { op: 'credentialStatus', business: 'demo', role: 'pm' })).result, []);
// The launcher's thread view, through the same child.
assert.deepEqual((await ask(child, { op: 'threadsState', business: 'demo' })).result, [
{ thread: 'th-1', instance: 'coder', pending: 0, unread: 0, lastMessage: null, lastOpen: '2026-10-15T00:00:01.000Z', lastClaim: null, holder: null },
{ thread: 'th-1', instance: 'coder', pending: 0, unread: 0, lastMessage: null, lastOpen: '2026-10-15T00:00:01.000Z', lastClaim: null, holder: null, session: 'th-1', legacy: true },
]);
// And its idle release (Dewey's F3, 28591): the op reaches the broker, which refuses a run
// that doesn't hold the thread.
+7 -7
View File
@@ -199,21 +199,21 @@ test('T2 sage-2257: a released or revoked run\'s retired generation refuses its
// Each refusal leaves its one action.refused event (request()'s catch, #refused); nothing else is
// written: no approval, delivery, read or claim state (mos-1277).
const lastSeq = () => store.get('SELECT max(seq) AS s FROM events').s;
const refusedOnce = (fn, label) => {
const refusedOnce = (fn, label, run) => {
const from = lastSeq();
assert.throws(fn, { code: 'not-holder' }, label);
const added = store.all('SELECT kind, body FROM events WHERE seq > ? ORDER BY seq', from);
const added = store.all('SELECT kind, actor_run, body FROM events WHERE seq > ? ORDER BY seq', from);
assert.deepEqual(
added.map((e) => [e.kind, JSON.parse(e.body)]),
[['action.refused', { code: 'not-holder' }]],
added.map((e) => [e.kind, e.actor_run, JSON.parse(e.body)]),
[['action.refused', run, { code: 'not-holder' }]],
label,
);
};
const refused = (name, { d, taken }) => {
const { events, ...before } = counts();
refusedOnce(() => call(name, 'decision.withdraw', { id: d.id }), 'approval');
refusedOnce(() => call(name, 'message.receive'), 'delivery');
refusedOnce(() => call(name, 'message.read', { id: taken.id }), 'read');
refusedOnce(() => call(name, 'decision.withdraw', { id: d.id }), 'approval', `${name}-run`);
refusedOnce(() => call(name, 'message.receive'), 'delivery', `${name}-run`);
refusedOnce(() => call(name, 'message.read', { id: taken.id }), 'read', `${name}-run`);
const { events: after, ...rest } = counts();
assert.deepEqual(rest, before);
assert.equal(after, events + 3);
+117 -50
View File
@@ -75,6 +75,15 @@
// threadStore()), so the next run for the thread resumes the harness
// session (REQ-THREAD-2).
//
// Row 269 (264b, #1774): threadsState has one row per session of each
// thread, and everything above is per session. Each session has its own
// store (session.mjs sessionStore()) and its own workspace under
// session-workspaces/ (sessionWorkspace()), which a lock beside it keeps to
// one writer from before the clones until the run's proven exit. A legacy
// session (op legacy-3l) keeps the thread's store, and its new workspace
// shows inside the sandbox at the old instance path, the cwd its history
// recorded. The broker's half of the fence is its per-session claim.
//
// Landing runs (row 156, #1651). Every launch of a non-thread instance
// reads the broker's landHolders (launch op): the approved, unspent merge
// decisions each non-thread instance raised. The instance's entries go into
@@ -121,12 +130,14 @@
// processing-halted.
// - delete: instance-running against the registry, then lifecycleDelete
// (the writer's commit), then the rename of workspaces/<b>/I/ to
// workspaces/<b>/.retired/I-<transaction>/. A failed rename closes it
// workspace-rename-failed with the transaction; a later delete retries it.
// workspaces/<b>/.retired/I-<transaction>/, and of
// session-workspaces/<b>/I/ to session-workspaces/<b>/.retired/I-<transaction>/
// (row 269). A failed rename closes it workspace-rename-failed with the
// transaction; a later delete retries it.
import { createServer } from "node:net";
import { chmodSync, existsSync, lstatSync, mkdirSync, readFileSync, renameSync, rmSync } from "node:fs";
import { dirname, join } from "node:path";
import { dirname, join, relative } from "node:path";
import { configDir, loadBusiness, resolveInstance, systemVars } from "../../business/src/index.mjs";
import { limitHit } from "../../business/src/launch.mjs";
import { BundleError, buildBundle, hasFallback, readSkills, sessionModel } from "../../harness/src/bundle.mjs";
@@ -142,8 +153,10 @@ import {
runnerOf,
SESSION_CREDENTIALS,
sessionEnv,
threadStore,
sessionExit,
sessionStore,
sessionWorkspace,
lockSessionWorkspace,
spawnSession,
workspaceDir,
writeSessionFile,
@@ -237,6 +250,9 @@ export function createLauncher({
throw new CliError(`${e.message}; check which sessions still run, then move the file aside`, 3);
}
const registry = new Registry(dataRoot, { readOnly: !launches });
// Row 269: earlier-host runs still live after recover(). check() sees each while it lives.
const survivors = [];
const liveRun = (s) => startTimeOf(s.pid) === s.startTime || startTimeOf(s.runnerPid) === s.runnerStartTime;
const children = new Map();
let chain = Promise.resolve();
let closed = false;
@@ -278,7 +294,7 @@ export function createLauncher({
// nothing else launches for one. Row 194 (#1692, V2): `human` is the
// human's launch request, which may name any declared instance. The listing
// limits the chief, not the human. launch.by still takes the `pm` branch.
async function check(instance, { pm = false, thread = null, human = false } = {}) {
async function check(instance, { pm = false, thread = null, session = null, human = false } = {}) {
const admissionGeneration = admission().generation;
try { b=loadBusiness(host.business,{dir:configDir(env),rolesDir:rolesDir(env)}); } catch(e){throw new Refusal("organization-refused",e.message);}
if (!b.launch) throw new Refusal("launch-not-configured", `business ${b.id} has no launch block`);
@@ -286,7 +302,12 @@ export function createLauncher({
const threaded = b.roles[instance]?.definition === THREAD_DEFINITION;
if (threaded && !thread) throw new Refusal("thread-required", `${instance} runs only for a thread`);
if (!threaded && thread) throw new Refusal("thread-not-allowed", `${instance} isn't a thread instance`);
if (registry.running(b.id, instance, thread)) throw new Refusal("instance-running");
if (threaded && !session) throw new Refusal("session-required", `${instance} runs for one session of a thread`);
if (registry.running(b.id, instance, thread, session)) throw new Refusal("instance-running");
// Row 269 (Q2): a run an earlier host started with no session works in the instance's old
// workspace and the thread's store, so no session of the instance starts while one lives.
if (threaded && [...registry.sessions, ...leftover, ...survivors.filter(liveRun)].some((s) => s.business === b.id && s.instance === instance && !s.session))
throw new Refusal("legacy-run-live", `a run of ${instance} with no session is still live`);
let resolved;
try {
resolved = resolveInstance({ system: vars, business: b, instance });
@@ -378,29 +399,59 @@ export function createLauncher({
}
// Bundle, spawn, bind, register, hand over the capability.
async function start({ instance, launchedBy, resolved, model, account = null, thread = null, human = false, admissionGeneration }) {
async function start({ instance, launchedBy, resolved, model, account = null, thread = null, session = null, legacy = false, legacyThread = legacy, human = false, admissionGeneration }) {
admission(admissionGeneration);
// Every launch path ends here, so a --no-launch host spawns nothing whichever path asked.
if (!launches) throw new Refusal("launches-disabled", "this host was started with --no-launch");
// A thread's store is checked first, so a bad id or an unsafe directory
// launches nothing.
// A thread session's store is checked first, so a bad id or an unsafe
// directory launches nothing.
let store = null;
if (thread) {
try {
store = threadStore(dataRoot, b.id, thread);
store = sessionStore(dataRoot, b.id, thread, session, { legacy, legacyThread });
} catch (e) {
throw new Refusal("thread-store-refused", e.message);
throw new Refusal(e.code === "legacy-store-unmapped" ? e.code : "thread-store-refused", e.message);
}
}
const credentials = await host.op({ op: "credentialStatus", business: b.id, role: instance });
admission(admissionGeneration);
// The clones come before the run, so one that can't be made leaves no
// run behind. The host's git treats each clone as hostile, and its
// checks hold only while no session writes the workspace: a workspace
// is one instance's, check() refused instance-running, and launches are
// serialized, so none does.
const workspace = workspaceDir(dataRoot, b.id, instance);
mkdirSync(workspace, { recursive: true, mode: 0o700 });
// checks hold only while no session writes the workspace. A non-thread
// workspace is one instance's, check() refused instance-running, and
// launches are serialized. A thread session's workspace is its own (row
// 269), and its lock refuses a second writer, whichever host asks.
let workspace;
let workspaceDest = null;
let lock = null;
if (thread) {
try {
const live = registry.sessions.filter((s) => s.business === b.id && s.workspace).map((s) => s.workspace);
workspace = sessionWorkspace(dataRoot, b.id, instance, session, { live });
lock = lockSessionWorkspace(workspace, { hostPid: process.pid, hostStartTime: startTimeOf(process.pid), business: b.id, instance, thread, session });
} catch (e) {
if (["session-workspace-unsafe", "session-workspace-overlap", "session-workspace-busy"].includes(e.code)) throw new Refusal(e.code, e.message);
throw e;
}
if (legacy) workspaceDest = workspaceDir(dataRoot, b.id, instance);
} else {
workspace = workspaceDir(dataRoot, b.id, instance);
mkdirSync(workspace, { recursive: true, mode: 0o700 });
}
try {
return await startLocked({ instance, launchedBy, resolved, model, account, thread, session, human, admissionGeneration, store, credentials, workspace, workspaceDest, lock });
} catch (e) {
// Every failure path in startLocked has proven the run's exit, or never spawned one.
lock?.release();
throw e;
}
}
// The rest of start(): clones, bundle, spawn, bind, register. The lock, when there is one, is
// released at the run's close.
async function startLocked({ instance, launchedBy, resolved, model, account, thread, session, human, admissionGeneration, store, credentials, workspace, workspaceDest, lock }) {
// What the session sees: a legacy session's workspace at the old instance path.
const inSandbox = workspaceDest ?? workspace;
let checkouts;
try {
checkouts = await withProcessing(dataRoot, () => {
@@ -416,7 +467,7 @@ export function createLauncher({
else if (c.outcome === "fetched") record({ event: "checkout-fetched", instance, project: c.project, reason: c.reason });
}
let harness, route, routeReason;
({resolved,model,account,harness,route,routeReason}=await check(instance,{pm:instance===b.launch.by,thread,human}));
({resolved,model,account,harness,route,routeReason}=await check(instance,{pm:instance===b.launch.by,thread,session,human}));
admission(admissionGeneration);
const { run, runDir } = newRun(dataRoot, b.id);
const sessionDir = store ? store.session : join(runDir, "session");
@@ -440,7 +491,7 @@ export function createLauncher({
arbiter: Object.values(b.arbiters).includes(instance),
tracker,
credentials,
session: { run, launchedBy, workspace, toolSocket, turnMarker, checkouts },
session: { run, launchedBy, workspace: inSandbox, toolSocket, turnMarker, checkouts: inSandbox === workspace ? checkouts : checkouts.map((c) => ({ ...c, path: join(inSandbox, relative(workspace, c.path)) })) },
harness,
route,
routeReason,
@@ -471,7 +522,7 @@ export function createLauncher({
brokerSocket: host.path,
launchSocket: launchSocketPath(dataRoot),
gitSocket: git,
workspace,
workspace: inSandbox,
sessionDir,
adapter,
bundle: files,
@@ -484,6 +535,7 @@ export function createLauncher({
const paths = {
workspace,
workspaceDest,
adapter,
harnessMounts,
brokerSocket: host.path,
@@ -519,11 +571,12 @@ export function createLauncher({
updateRuntime(dataRoot, managed.id, { runnerPid: runner.pid, runnerStartTime: runner.startTime });
admission(admissionGeneration);
if (!outerStart) throw new Error("the session process exited at once");
const launch = await host.bindLaunch({ business: b.id, role: instance, run, harness: model.harness, provider: model.provider, model: model.model, account, transcript, pid: child.pid, startTime: outerStart, ...(thread ? { thread } : {}) });
lock?.update({ run, runPid: child.pid, runStartTime: outerStart });
const launch = await host.bindLaunch({ business: b.id, role: instance, run, harness: model.harness, provider: model.provider, model: model.model, account, transcript, pid: child.pid, startTime: outerStart, ...(thread ? { thread, session } : {}) });
bound = true;
admission(admissionGeneration);
entry = {
business: b.id, instance, ...(thread ? { thread } : {}), run, launchedBy, harness: model.harness, model: model.model, provider: model.provider, modelKey: model.modelKey,
business: b.id, instance, ...(thread ? { thread, session, workspace } : {}), run, launchedBy, harness: model.harness, model: model.model, provider: model.provider, modelKey: model.modelKey,
pid: child.pid, startTime: outerStart, runnerPid: runner.pid, runnerStartTime: runner.startTime, runDir, startedAt: new Date().toISOString(),
};
registry.add(entry);
@@ -552,13 +605,15 @@ export function createLauncher({
const ended = await endRun(run, reason, signal ? null : code);
registry.remove(b.id, run);
// Row 125 (#1619, REQ-LIMIT-4): a run a limit check stopped names the limit, from the claim row.
record({ event: "end", instance, ...(thread ? { thread } : {}), run, reason, exitCode: code, signal, ...(haltGeneration ? { haltGeneration } : {}), ...(ended?.limit ? { limit: ended.limit } : {}) });
record({ event: "end", instance, ...(thread ? { thread, session } : {}), run, reason, exitCode: code, signal, ...(haltGeneration ? { haltGeneration } : {}), ...(ended?.limit ? { limit: ended.limit } : {}) });
log(`run ${run} (${instance}) ended: ${reason}`);
} catch (e) {
log(`run ${run}: ${e.message}`);
} finally {
try { unregisterRuntime(dataRoot, managed.id); }
catch { log(`run ${run}: processing receipt cleanup refused`); }
try { lock?.release(); }
catch (e) { log(`run ${run}: workspace lock release: ${e.message}`); }
children.delete(run);
stopping.delete(run);
spawnedAt.delete(run);
@@ -566,7 +621,7 @@ export function createLauncher({
refusedRelease.delete(run);
}
});
record({ event: "launch", instance, ...(thread ? { thread } : {}), run, launchedBy, harness: model.harness, route, routeReason, model: model.model, provider: model.provider, modelKey: model.modelKey, account, pid: child.pid, runnerPid: entry.runnerPid, manifest });
record({ event: "launch", instance, ...(thread ? { thread, session } : {}), run, launchedBy, harness: model.harness, route, routeReason, model: model.model, provider: model.provider, modelKey: model.modelKey, account, pid: child.pid, runnerPid: entry.runnerPid, manifest });
return { run, instance, harness: model.harness, model: model.model };
}
@@ -600,9 +655,12 @@ export function createLauncher({
}
const now = Date.now();
const ms = (at) => (at ? Date.parse(at) : 0);
// Row 269: a thread with a legacy session keeps its old store for that session only.
const legacyThreads = new Set(threads.filter((t) => t.legacy).map((t) => t.thread));
for (const t of threads) {
if (closed) return;
const s = registry.running(b.id, t.instance, t.thread);
const key = `${t.thread}/${t.session}`;
const s = registry.running(b.id, t.instance, t.thread, t.session);
if (s) {
const last = Math.max(ms(s.startedAt), ms(t.lastMessage), ms(t.lastOpen));
if (!stopping.has(s.run) && now - last >= b.launch.threadIdleSeconds * 1000) {
@@ -616,7 +674,7 @@ export function createLauncher({
// A turn in progress, or a message since the snapshot: the run keeps going.
if (!idle.released) continue;
stopping.add(s.run);
record({ event: "thread-idle", instance: t.instance, thread: t.thread, run: s.run });
record({ event: "thread-idle", instance: t.instance, thread: t.thread, session: t.session, run: s.run });
try {
if (startTimeOf(s.runnerPid) === s.runnerStartTime) process.kill(s.runnerPid, "SIGTERM");
} catch {}
@@ -629,20 +687,21 @@ export function createLauncher({
// the work (it died before its claim, say) waits for the next one
// rather than start again every tick. A refused start isn't a start.
const signature = `${t.lastMessage}|${t.lastOpen}`;
if (startedFor.get(t.thread) === signature) continue;
if (startedFor.get(key) === signature) continue;
await serialize(async () => {
if (closed || registry.running(b.id, t.instance, t.thread)) return;
if (closed || registry.running(b.id, t.instance, t.thread, t.session)) return;
try {
await start({ instance: t.instance, launchedBy: b.human, thread: t.thread, ...(await check(t.instance, { thread: t.thread })) });
startedFor.set(t.thread, signature);
refusedThreads.delete(t.thread);
const where = { thread: t.thread, session: t.session };
await start({ instance: t.instance, launchedBy: b.human, ...where, legacy: t.legacy, legacyThread: legacyThreads.has(t.thread), ...(await check(t.instance, where)) });
startedFor.set(key, signature);
refusedThreads.delete(key);
} catch (e) {
const code = e instanceof Refusal ? e.code : "launch-failed";
// Logged once per code, not once per tick.
if (refusedThreads.get(t.thread) === code) return;
refusedThreads.set(t.thread, code);
log(`thread ${t.thread} (${t.instance}): start refused: ${code}`);
record({ event: "refused", by: b.human, instance: t.instance, thread: t.thread, code, why: e.message, ...(e.limit?{limit:e.limit}:{}) });
if (refusedThreads.get(key) === code) return;
refusedThreads.set(key, code);
log(`thread ${t.thread} session ${t.session} (${t.instance}): start refused: ${code}`);
record({ event: "refused", by: b.human, instance: t.instance, thread: t.thread, session: t.session, code, why: e.message, ...(e.limit?{limit:e.limit}:{}) });
}
});
}
@@ -849,20 +908,23 @@ export function createLauncher({
const transaction = out?.transaction ?? null;
if (transaction === null) return { refusal: "workspace-rename-failed" };
const bot = out.bot ? { bot: out.bot } : {};
const from = workspaceDir(dataRoot, b.id, r.instance);
if (!existsSync(from)) return { transaction, ...bot };
const to = join(dirname(from), ".retired", `${r.instance}-${transaction}`);
try {
mkdirSync(dirname(to), { recursive: true, mode: 0o700 });
chmodSync(dirname(to), 0o700);
if (existsSync(to)) throw new Error(`${to} exists`);
renameSync(from, to);
chmodSync(to, 0o700);
} catch (e) {
log(`request ${r.request} (${r.instance}): workspace rename failed: ${e.message}; left at ${from}`);
return { transaction, ...bot, refusal: "workspace-rename-failed", why: `${e.message}; left at ${from}` };
// The old instance workspace, then (row 269) the instance's session workspaces, each moved
// whole. A retry moves whichever is still in place.
for (const from of [workspaceDir(dataRoot, b.id, r.instance), join(dataRoot, "session-workspaces", b.id, r.instance)]) {
if (!existsSync(from)) continue;
const to = join(dirname(from), ".retired", `${r.instance}-${transaction}`);
try {
mkdirSync(dirname(to), { recursive: true, mode: 0o700 });
chmodSync(dirname(to), 0o700);
if (existsSync(to)) throw new Error(`${to} exists`);
renameSync(from, to);
chmodSync(to, 0o700);
} catch (e) {
log(`request ${r.request} (${r.instance}): workspace rename failed: ${e.message}; left at ${from}`);
return { transaction, ...bot, refusal: "workspace-rename-failed", why: `${e.message}; left at ${from}` };
}
record({ event: "workspace-retired", instance: r.instance, request: r.request, transaction, path: to });
}
record({ event: "workspace-retired", instance: r.instance, request: r.request, transaction, path: to });
return { transaction, ...bot };
}
@@ -958,10 +1020,15 @@ export function createLauncher({
process.kill(s.pid, "SIGKILL");
} catch {}
const end = Date.now() + 5000;
while (startTimeOf(s.pid) === s.startTime && Date.now() < end) await new Promise((r) => setTimeout(r, 20));
while (liveRun(s) && Date.now() < end) await new Promise((r) => setTimeout(r, 20));
}
// Row 269 (Q2): the session or its runner outlived the kill, or wasn't a session to kill.
if (liveRun(s)) {
survivors.push(s);
log(`run ${s.run} (${s.instance}) from an earlier host still runs (pid ${s.pid}, runner ${s.runnerPid}): no session of ${s.instance} starts until it exits`);
}
try {
await host.bindLaunch({ business: b.id, role: s.instance, run: s.run, harness: s.harness, pid: s.pid, startTime: s.startTime, ...(s.thread ? { thread: s.thread } : {}) });
await host.bindLaunch({ business: b.id, role: s.instance, run: s.run, harness: s.harness, pid: s.pid, startTime: s.startTime, ...(s.thread ? { thread: s.thread } : {}), ...(s.session ? { session: s.session } : {}) });
} catch (e) {
// A host that died after ending the run but before updating sessions.json.
if (e.code === "run-ended") log(`run ${s.run} (${s.instance}) from an earlier host was already ended`);
@@ -969,7 +1036,7 @@ export function createLauncher({
continue;
}
await endRun(s.run, "host-lost", null);
record({ event: "end", instance: s.instance, ...(s.thread ? { thread: s.thread } : {}), run: s.run, reason: "host-lost", exitCode: null, signal: null });
record({ event: "end", instance: s.instance, ...(s.thread ? { thread: s.thread } : {}), ...(s.session ? { session: s.session } : {}), run: s.run, reason: "host-lost", exitCode: null, signal: null });
log(`run ${s.run} (${s.instance}) from an earlier host ended: host-lost`);
}
leftover = [];
+18
View File
@@ -0,0 +1,18 @@
// A stand-in harness turn for the session launch tests (row 269, #1774).
// thread-adapter.mjs's turn, plus the run's cwd and the names in it, so a
// test reads from the reply which workspace the session saw and at what
// in-sandbox path. It keeps a history in $MOSAIC_SESSION_DIR, the session's
// store.
import { appendFileSync, existsSync, readdirSync, readFileSync, writeFileSync } from "node:fs";
import { join } from "node:path";
const request = process.env.MOSAIC_REQUEST ?? "";
const body = request.match(/^Message [^\n]*:\n\n([\s\S]*)$/m)?.[1] ?? "";
const historyFile = join(process.env.MOSAIC_SESSION_DIR, "history");
const before = existsSync(historyFile) ? readFileSync(historyFile, "utf8").split("\n").filter(Boolean) : [];
const cwd = process.cwd();
const entries = readdirSync(cwd).sort();
appendFileSync(historyFile, `${body}\n`);
process.stdout.write(`THREAD ${JSON.stringify({ body, before, cwd, entries })}\n`);
writeFileSync(process.env.MOSAIC_TURN_MARKER, "done\n");
+414
View File
@@ -0,0 +1,414 @@
// Session runs (row 269, 264b, #1774): every session of a thread runs on its
// own, with its own store under threads/<b>/<t>/sessions/<s>/ and its own
// workspace under session-workspaces/<b>/<instance>/<s>/, and a lock beside
// the workspace keeps it to one writer. A legacy session (op legacy-3l) keeps
// the thread's old store, and its workspace shows inside the sandbox at the
// old instance path. Real runners in the bwrap sandbox; a stub adapter keeps
// a history in the store and answers with what it found, its cwd and what
// the cwd holds.
//
// The host is thread-launch.test.mjs's: startBroker on bootConfig's config,
// with the ops the broker child forwards, and the human capability from
// bindHuman directly.
import { test } from "node:test";
import assert from "node:assert/strict";
import { spawn, spawnSync } from "node:child_process";
import { chmodSync, existsSync, mkdirSync, readdirSync, readFileSync, symlinkSync, writeFileSync } from "node:fs";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { launchDoc } from "../../business/tests/helpers.mjs";
import { BusError } from "../../bus/src/broker.mjs";
import { startBroker } from "../../bus/src/runtime.mjs";
import { Store } from "../../bus/src/store.mjs";
import { transition } from "../../processing-control/src/state.mjs";
import { gitEnv } from "../../seat/src/checkout.mjs";
import { startTimeOf } from "../../seat/src/proc.mjs";
import { sandboxProblem } from "../../seat/src/sandbox.mjs";
import { launchLogFile, readSessions, Registry, SESSION_CREDENTIALS } from "../../seat/src/session.mjs";
import { bootConfig, loadSystem } from "../src/config.mjs";
import { createLauncher } from "../src/launcher.mjs";
import { fakeHarnesses, fixture, tmp } from "./helpers.mjs";
// A thread made by hand gets the default session the 3k-3l migration gives a legacy thread, with
// the thread's id (as in thread-launch.test.mjs).
const DEFAULT_SESSION = [
"INSERT INTO sessions(at,business,thread,id,instance,owner,name,op) SELECT at,business,id,id,instance,owner,'default','legacy-3l' FROM threads WHERE id=?",
"INSERT INTO session_defaults(at,thread,session,by) SELECT at,id,id,'migration-3l' FROM threads WHERE id=?",
];
const ADAPTER = fileURLToPath(new URL("./fixtures/session-adapter.mjs", import.meta.url));
const skip = sandboxProblem() ?? false;
async function until(fn, ms = 20000) {
const end = Date.now() + ms;
while (Date.now() < end) {
const v = await fn();
if (v) return v;
await new Promise((r) => setTimeout(r, 50));
}
throw new Error("timed out waiting");
}
// A mutant's kill is an assertion failure, not a timeout (sage-2707), so the waits a mutant could
// stall return what they last saw and the assertions after them decide.
async function settle(fn, ms = 20000) {
const end = Date.now() + ms;
let v;
while (Date.now() < end) {
v = await fn();
if (v) return v;
await new Promise((r) => setTimeout(r, 50));
}
return v;
}
function gitIn(cwd, ...args) {
const r = spawnSync("git", ["-c", "user.name=test", "-c", "[email protected]", "-C", cwd, ...args], { env: gitEnv(), encoding: "utf8" });
assert.equal(r.status, 0, r.stderr);
}
// Each directory made 0700, as the store and workspace checks want.
function privateDir(...parts) {
for (let i = 1; i <= parts.length; i++) {
const d = join(...parts.slice(0, i));
if (!existsSync(d)) mkdirSync(d, { mode: 0o700 });
if (i > 1) chmodSync(d, 0o700);
}
return join(...parts);
}
// The assistant is the thread instance; sonnet is its model's limit.
function prepare(t, { sonnet = 3, idle = 1 } = {}) {
const root = tmp(t, "mosaic-session-");
const f = fixture(root, "acme", (doc) => {
doc.roles.chief.vars = { harness: "pi", model: "claude-opus-5-5", thinking: "high" };
doc.roles.assistant = { definition: "assistant", vars: { harness: "pi", model: "claude-sonnet-5-5", thinking: "high" } };
doc.launch = launchDoc("chief", ["coder", "assistant"], { opus: 1, sonnet });
doc.launch.threadIdleSeconds = idle;
return doc;
});
for (const k of SESSION_CREDENTIALS.flat()) delete f.env[k];
const project = f.doc.projects.stack.root;
gitIn(project, "init", "-q", "-b", "main");
writeFileSync(join(project, "file.txt"), "one\n");
gitIn(project, "add", "file.txt");
gitIn(project, "commit", "-q", "-m", "one");
const adapter = join(root, "adapter.sh");
writeFileSync(adapter, `exec '${process.execPath}' '${ADAPTER}'\n`, { mode: 0o644 });
return { root, f, adapter };
}
// A host and its launcher on the prepared root. `ops` replaces launch ops, given the runtime.
async function start(t, { f, adapter }, { tickMs = 100, ops: replace } = {}) {
const system = loadSystem({ env: f.env });
const { trackers, ...boot } = bootConfig({ system, businessId: "acme", env: f.env });
assert.equal(trackers, undefined);
const runtime = await startBroker(boot);
const ops = {
identity: (m) => runtime.identity(m.cap),
authorizeLaunch: (m) => runtime.authorizeLaunch(m.cap, m.instance),
refuse: (m) => runtime.refuse(m.cap, m.code, m.subject),
endLaunch: (m) => runtime.endLaunch(m.record),
credentialStatus: (m) => runtime.credentialStatus(m.business, m.role),
threadsState: (m) => runtime.threadsState(m.business),
threadIdle: (m) => runtime.threadIdle(m.record),
mayLaunch: (m) => runtime.mayLaunch(m),
landHeld: (m) => runtime.landHeld(m.record),
landHolders: (m) => runtime.landHolders(m.business),
landRelease: (m) => runtime.landRelease(m.record),
...replace?.(runtime),
};
const refusing = (what, fn) => async (m) => {
try {
return fn(m);
} catch (e) {
const code = e instanceof BusError ? e.code : `${what}-refused`;
throw Object.assign(new Error(`${what} refused: ${code}`), { code });
}
};
const hooks = [];
let closing = null;
const host = {
business: "acme",
path: runtime.path,
dataRoot: f.dataRoot,
op: refusing("launch-op", (m) => {
if (!Object.hasOwn(ops, m.op)) throw new BusError("invalid-host-request");
return ops[m.op](m);
}),
bindLaunch: refusing("launch bind", (r) => runtime.bindLaunch(r)),
beforeClose: (fn) => void hooks.push(fn),
};
const logs = [];
const launcher = createLauncher({
host,
system,
env: f.env,
makeLoader: await fakeHarnesses(t, { pi: adapter, "claude-code": adapter }),
sessionDefaults: { pollInterval: 100 },
harnessVersion: "0.85.1",
readOnly: [ADAPTER],
threadTickMs: tickMs,
log: (l) => logs.push(l),
});
const close = () =>
(closing ??= (async () => {
for (const fn of hooks) await fn(0);
await runtime.close();
})());
t.after(close);
await launcher.listen();
const human = runtime.broker.bindHuman({ business: "acme", human: f.doc.human, via: "cli", outsideAgent: true });
const call = (verb, args = {}) => runtime.broker.request(human, { verb, args });
return { host, runtime, launcher, call, logs, close };
}
const launchLog = (f) =>
existsSync(launchLogFile(f.dataRoot, "acme"))
? readFileSync(launchLogFile(f.dataRoot, "acme"), "utf8").trim().split("\n").map((l) => JSON.parse(l))
: [];
const events = (f, event) => launchLog(f).filter((e) => e.event === event);
const sessionWs = (f, session) => join(f.dataRoot, "session-workspaces", "acme", "assistant", session);
// The refusals once there are n, or once anything launched.
const outcome = async (f, n = 1) => {
await settle(() => events(f, "refused").length >= n || events(f, "launch").length);
return events(f, "refused").map((e) => [e.session, e.code]);
};
const historyOf = (p) => (existsSync(p) ? readFileSync(p, "utf8") : null);
// Send to one session of a thread; reply() waits for the assistant's RESULT reply in that session.
function send(call, thread, session, body) {
const sent = call("thread.send", { thread, ...(session ? { session } : {}), body });
const reply = async () => {
const r = await until(() => call("thread.messages", { thread, ...(session ? { session } : {}) }).find((m) => m.in_reply_to === sent.id && m.class === "RESULT"));
assert.equal(r.from_role, "assistant");
return JSON.parse(r.body.match(/THREAD (\{.*\})/)[1]);
};
return { sent, reply };
}
// A thread made with thread.create, its default session and a second named one.
function twoSessions(call) {
const project = call("project.create", { name: "stack work", repo: "stack" }).id;
const A = call("thread.create", { project, name: "alpha" }).id;
const [{ id: D }] = call("session.list", { thread: A });
const S = call("session.create", { thread: A, name: "second", op: "op-second" }).id;
return { project, A, D, S };
}
test("REQ-THREAD-2 REQ-LAUNCH-1 two sessions of one thread, and another thread's default, run at once, each with its own store and workspace; a restart of one resumes only that one", { skip, timeout: 90000 }, async (t) => {
const prepared = prepare(t, { idle: 5 });
const { f } = prepared;
const host = await start(t, prepared);
const { project, A, D, S } = twoSessions(host.call);
const B = host.call("thread.create", { project, name: "beta" }).id;
const [{ id: BD }] = host.call("session.list", { thread: B });
const toD = send(host.call, A, D, "first for D");
const toS = send(host.call, A, S, "first for S");
const toB = send(host.call, B, null, "first for B");
// All three run together (M1, M2: a shared workspace would refuse the second start busy).
await settle(() => readSessions(f.dataRoot).length === 3 || events(f, "refused").length);
assert.deepEqual(events(f, "refused"), []);
assert.deepEqual(readSessions(f.dataRoot).map((s) => [s.thread, s.session]).sort(), [[A, D], [A, S], [B, BD]].sort());
const [sawD, sawS, sawB] = await Promise.all([toD.reply(), toS.reply(), toB.reply()]);
// Each session's history is its own (M5), and each runs in its own workspace at its own path (M8).
assert.deepEqual([sawD.body, sawD.before, sawD.cwd], ["first for D", [], sessionWs(f, D)]);
assert.deepEqual([sawS.body, sawS.before, sawS.cwd], ["first for S", [], sessionWs(f, S)]);
assert.deepEqual([sawB.body, sawB.before, sawB.cwd], ["first for B", [], sessionWs(f, BD)]);
const store = (thread, session) => join(f.dataRoot, "threads", "acme", thread, "sessions", session, "session", "history");
assert.deepEqual([historyOf(store(A, D)), historyOf(store(A, S)), historyOf(store(B, BD))], ["first for D\n", "first for S\n", "first for B\n"]);
for (const thread of [A, B]) assert.equal(existsSync(join(f.dataRoot, "threads", "acme", thread, "session")), false);
assert.equal(existsSync(join(f.dataRoot, "workspaces", "acme", "assistant")), false, "no thread session uses the old instance workspace");
for (const e of events(f, "launch")) assert.ok([D, S, BD].includes(e.session), JSON.stringify(e));
// All three stop when idle, and each lock goes with its run.
await until(() => events(f, "end").length === 3, 30000);
assert.deepEqual(readSessions(f.dataRoot), []);
for (const s of [D, S, BD]) assert.equal(existsSync(`${sessionWs(f, s)}.lock`), false, s);
// The named session's next message starts only that session (M9), which finds only its history.
const again = await send(host.call, A, S, "second for S").reply();
assert.deepEqual([again.before, again.cwd], [["first for S"], sessionWs(f, S)]);
assert.deepEqual(events(f, "launch").slice(3).map((e) => [e.thread, e.session]), [[A, S]]);
assert.equal(historyOf(store(A, D)), "first for D\n");
});
// D9 (M7): a thread from before 3l. Its legacy session resumes from the thread's old store, and
// its new workspace shows at the old instance path, which the transcripts recorded as the cwd.
// The old host workspace isn't mounted.
test("REQ-THREAD-2 REQ-LAUNCH-1 a legacy session resumes the thread's old store at the old in-sandbox cwd, in a workspace of its own", { skip, timeout: 60000 }, async (t) => {
const prepared = prepare(t);
const { f } = prepared;
const store = new Store(f.dataRoot);
const at = () => new Date().toISOString();
store.run("INSERT INTO projects(id,at,business,name,repo,created_by) VALUES(?,?,?,?,?,?)", "p-hand", at(), "acme", "hand made", "stack", "jason");
store.run("INSERT INTO threads(id,at,business,project,instance,owner,name) VALUES(?,?,?,?,?,?,?)", "th-legacy", at(), "acme", "p-hand", "assistant", "jason", "old");
for (const sql of DEFAULT_SESSION) store.run(sql, "th-legacy");
store.close();
const old = privateDir(f.dataRoot, "threads", "acme", "th-legacy");
writeFileSync(join(privateDir(old, "session"), "history"), "old sentinel\n");
privateDir(old, "claude-projects");
writeFileSync(join(privateDir(f.dataRoot, "workspaces", "acme", "assistant"), "old-file"), "the 264a workspace\n");
const host = await start(t, prepared);
const saw = await send(host.call, "th-legacy", null, "after 3l").reply();
assert.deepEqual(saw.before, ["old sentinel"]);
assert.equal(saw.cwd, join(f.dataRoot, "workspaces", "acme", "assistant"));
assert.equal(saw.entries.includes("old-file"), false, "the old host workspace isn't mounted");
assert.equal(readFileSync(join(old, "session", "history"), "utf8"), "old sentinel\nafter 3l\n");
assert.equal(existsSync(join(old, "sessions")), false);
assert.ok(existsSync(sessionWs(f, "th-legacy")));
assert.deepEqual(readdirSync(join(f.dataRoot, "workspaces", "acme", "assistant")), ["old-file"]);
assert.deepEqual(events(f, "launch").map((e) => [e.thread, e.session]), [["th-legacy", "th-legacy"]]);
});
// T2, M3: another host's live lock on the session's workspace refuses the start before any
// checkout. Once that host is gone the lock is stale, and the next start takes it over.
test("REQ-THREAD-2 REQ-LAUNCH-1 a live lock on a session workspace refuses session-workspace-busy before any checkout; a stale one is taken over", { skip, timeout: 60000 }, async (t) => {
const prepared = prepare(t);
const { f } = prepared;
const host = await start(t, prepared);
const { A, S } = twoSessions(host.call);
const other = spawn("sleep", ["60"], { stdio: "ignore" });
const gone = new Promise((r) => other.once("exit", r));
t.after(() => other.kill("SIGKILL"));
const otherStart = await until(() => startTimeOf(other.pid));
privateDir(f.dataRoot, "session-workspaces", "acme", "assistant");
const lock = `${sessionWs(f, S)}.lock`;
const held = `${JSON.stringify({ hostPid: other.pid, hostStartTime: otherStart, business: "acme", instance: "assistant", thread: A, session: S })}\n`;
writeFileSync(lock, held, { mode: 0o600 });
const toS = send(host.call, A, S, "for S");
assert.deepEqual(await outcome(f), [[S, "session-workspace-busy"]]);
assert.equal(events(f, "refused")[0].thread, A);
assert.deepEqual(events(f, "launch"), []);
assert.deepEqual(readSessions(f.dataRoot), []);
assert.deepEqual(readdirSync(sessionWs(f, S)), [], "no checkout");
assert.equal(readFileSync(lock, "utf8"), held);
other.kill("SIGKILL");
await gone;
const saw = await toS.reply();
assert.deepEqual([saw.before, saw.cwd], [[], sessionWs(f, S)]);
await until(() => events(f, "end").length === 1);
assert.equal(existsSync(lock), false);
});
// Q2 (BUILD.md:177-179, t1-sessions.md:109): an earlier host's sessionless run of the instance is
// still live, so no session of it starts until that run has exited. A sleep child stands in for
// its runner; its outer pid is gone, so recover() has nothing to kill and keeps it as a survivor.
test("REQ-THREAD-2 REQ-LAUNCH-1 a session start refuses legacy-run-live while an earlier host's sessionless run lives, and starts once it has exited", { skip, timeout: 60000 }, async (t) => {
const prepared = prepare(t);
const { f } = prepared;
const runner = spawn("sleep", ["60"], { stdio: "ignore" });
const gone = new Promise((r) => runner.once("exit", r));
t.after(() => runner.kill("SIGKILL"));
const runnerStartTime = await until(() => startTimeOf(runner.pid));
new Registry(f.dataRoot).add({ business: "acme", instance: "assistant", thread: "th-old", run: "run-left", harness: "pi", pid: 2 ** 22 + 7, startTime: "gone", runnerPid: runner.pid, runnerStartTime });
const host = await start(t, prepared);
assert.ok(host.logs.includes(`run run-left (assistant) from an earlier host still runs (pid ${2 ** 22 + 7}, runner ${runner.pid}): no session of assistant starts until it exits`), host.logs.join("\n"));
const { A, S } = twoSessions(host.call);
const toS = send(host.call, A, S, "for S");
assert.deepEqual(await outcome(f), [[S, "legacy-run-live"]]);
assert.equal(events(f, "refused")[0].thread, A);
assert.deepEqual(events(f, "launch"), [], "nothing spawned");
assert.deepEqual(readSessions(f.dataRoot), []);
assert.equal(existsSync(sessionWs(f, S)), false, "refused before the workspace");
runner.kill("SIGKILL");
await gone;
assert.equal(await settle(() => events(f, "launch").length), 1, "it starts once the run has exited");
const saw = await toS.reply();
assert.deepEqual([saw.before, saw.cwd], [[], sessionWs(f, S)]);
await until(() => events(f, "end").length === 1);
});
// T2, C3: a symlinked session root refuses session-workspace-unsafe, at the segment check.
test("REQ-THREAD-2 REQ-LAUNCH-1 a symlinked session workspace root refuses session-workspace-unsafe and launches nothing", { skip, timeout: 60000 }, async (t) => {
const prepared = prepare(t);
const { f } = prepared;
const elsewhere = privateDir(prepared.root, "elsewhere");
symlinkSync(elsewhere, join(f.dataRoot, "session-workspaces"));
const host = await start(t, prepared);
const { A, D } = twoSessions(host.call);
send(host.call, A, D, "for D");
assert.deepEqual(await outcome(f), [[D, "session-workspace-unsafe"]]);
assert.deepEqual(events(f, "launch"), []);
assert.deepEqual(readdirSync(elsewhere), []);
});
// T2, M4: every segment of the session path is a real directory, but workspaces/<b> is a symlink
// to it, so only the realpath compare finds that the session workspace holds the legacy root.
test("REQ-THREAD-2 REQ-LAUNCH-1 a session workspace that holds the legacy workspace root refuses session-workspace-overlap and launches nothing", { skip, timeout: 60000 }, async (t) => {
const prepared = prepare(t);
const { f } = prepared;
const host = await start(t, prepared);
const { A, D } = twoSessions(host.call);
const ws = privateDir(f.dataRoot, "session-workspaces", "acme", "assistant", D);
symlinkSync(ws, join(privateDir(f.dataRoot, "workspaces"), "acme"));
send(host.call, A, D, "for D");
assert.deepEqual(await outcome(f), [[D, "session-workspace-overlap"]]);
assert.deepEqual(events(f, "launch"), []);
assert.deepEqual(readdirSync(ws), []);
});
// T2: a symlink among the store's segments refuses thread-store-refused before the workspace.
test("REQ-THREAD-2 REQ-LAUNCH-1 a symlinked session store refuses thread-store-refused and makes no workspace", { skip, timeout: 60000 }, async (t) => {
const prepared = prepare(t);
const { f } = prepared;
const host = await start(t, prepared);
const { A, D } = twoSessions(host.call);
const elsewhere = privateDir(prepared.root, "elsewhere");
symlinkSync(elsewhere, join(privateDir(f.dataRoot, "threads", "acme", A), "sessions"));
send(host.call, A, D, "for D");
assert.deepEqual(await outcome(f), [[D, "thread-store-refused"]]);
assert.match(events(f, "refused")[0].why, /is unsafe$/);
assert.deepEqual(events(f, "launch"), []);
assert.deepEqual(readdirSync(elsewhere), []);
assert.equal(existsSync(join(f.dataRoot, "session-workspaces")), false);
});
// T2, M6 and the halt fence (C2). The credentialStatus op runs in start() after check() returned.
// Its first call halts and resumes, a new generation with no halt, so only start()'s own
// admission(admissionGeneration) refuses, before the session workspace is made. On the retry,
// the recheck's mayLaunch op (inside check(), after the lock is taken) halts once it has answered,
// and the recheck's admission() refuses processing-halted. Neither attempt spawns, and the second releases its lock.
test("REQ-THREAD-2 REQ-LAUNCH-1 a named session's start refuses processing-generation-changed, then processing-halted, after check() and before any spawn", { skip, timeout: 60000 }, async (t) => {
const prepared = prepare(t);
const { f } = prepared;
let credentials = 0;
let halts = 0;
const halt = () => transition(f.dataRoot, { business: "acme", human: f.doc.human, halted: true });
const host = await start(t, prepared, {
ops: (runtime) => ({
credentialStatus: (m) => {
if (m.role === "assistant" && ++credentials === 1) {
transition(f.dataRoot, { business: "acme", human: f.doc.human, halted: false, generation: halt().generation });
halts++;
}
return runtime.credentialStatus(m.business, m.role);
},
// The halt lands after the op answers: halted inside it, the bus's own fence
// (runtime.mjs processingWork) throws and the launcher refuses limit-check-failed.
mayLaunch: async (m) => {
const may = await runtime.mayLaunch(m);
if (m.instance === "assistant" && credentials === 2 && halts === 1) {
halt();
halts++;
}
return may;
},
}),
});
const { A, S } = twoSessions(host.call);
send(host.call, A, S, "for S");
const refused = await outcome(f, 2);
assert.deepEqual(refused, [[S, "processing-generation-changed"], [S, "processing-halted"]]);
assert.deepEqual([credentials, halts], [2, 2], "check() passed both times");
assert.equal(events(f, "launch").length, 0);
assert.deepEqual(readSessions(f.dataRoot), []);
assert.equal(existsSync(join(f.dataRoot, "launches", "acme")), false, "no run directory");
assert.ok(existsSync(sessionWs(f, S)), "the retry made the workspace");
assert.equal(existsSync(`${sessionWs(f, S)}.lock`), false, "and released its lock");
});
+2 -1
View File
@@ -179,10 +179,11 @@ test("REQ-THREAD-2 REQ-LAUNCH-1 REQ-MSG-1 a thread message starts the thread's s
assert.deepEqual(first.saw, { body: "first for A", before: [], transcripts: [] });
const [run1] = launchLog(f).filter((e) => e.event === "launch" && e.thread === A).map((e) => e.run);
assert.ok(run1);
const store = join(f.dataRoot, "threads", "acme", A);
const store = join(f.dataRoot, "threads", "acme", A, "sessions", host.runtime.threadsState("acme").find((x) => x.thread === A).session);
assert.equal(readFileSync(join(store, "session", "history"), "utf8"), "first for A\n");
assert.deepEqual(readdirSync(join(store, "claude-projects")), ["turn-1.jsonl"]);
assert.equal(existsSync(join(f.dataRoot, "launches", "acme", run1, "session")), false, "a thread run keeps no per-run session directory");
assert.equal(existsSync(join(f.dataRoot, "threads", "acme", A, "session")), false, "a thread.create session keeps no thread-level store");
await idleStop(f, host, A, run1);
// A second run of the same thread finds the first run's history.
+7 -4
View File
@@ -97,9 +97,12 @@ function systemBinds(node = process.execPath) {
// the harness module's mounts (row 65: its mounts(), [{ src, dest, why }],
// read-only), the broker socket, the launch socket (null for a role that may
// not launch), the git socket (null for a role that neither pushes nor
// reviews), the thread's session store (session.mjs threadStore(), or null)
// and node. `readOnly` adds files, for tests only.
export function binds({ runDir, workspace, adapter, harnessMounts = [], brokerSocket, launchSocket = null, gitSocket = null, threadStore = null, node = process.execPath, readOnly = [] }) {
// reviews), the thread's session store (session.mjs threadStore() or
// sessionStore(), or null) and node. workspaceDest (row 269) is where the
// workspace shows inside the sandbox when that isn't its host path: a legacy
// thread session's, so its harness finds the cwd its history recorded.
// `readOnly` adds files, for tests only.
export function binds({ runDir, workspace, workspaceDest = null, adapter, harnessMounts = [], brokerSocket, launchSocket = null, gitSocket = null, threadStore = null, node = process.execPath, readOnly = [] }) {
const out = systemBinds(node);
for (const f of RUNNER_FILES) out.push({ kind: "ro", src: f, why: "a repository file the runner loads" });
out.push({ kind: "ro", src: adapter, why: "the harness adapter the runner starts each turn" });
@@ -109,7 +112,7 @@ export function binds({ runDir, workspace, adapter, harnessMounts = [], brokerSo
if (gitSocket) out.push({ kind: "ro", src: gitSocket, why: "the host's git socket, for push_working and fetch_review, and land_work" });
for (const f of readOnly) out.push({ kind: "ro", src: f, why: "a test fixture" });
out.push(
{ kind: "rw", src: workspace, why: "the instance's workspace, where the harness works" },
{ kind: "rw", src: workspace, ...(workspaceDest && workspaceDest !== workspace ? { dest: workspaceDest } : {}), why: "the instance's workspace, where the harness works" },
{ kind: "rw", src: runDir, why: "the run directory: turns/, session/, tools.sock, turn.done, home/, tmp/" },
{ kind: "ro", src: join(runDir, "bundle"), why: "the run's bundle stays as the host wrote it" },
{ kind: "ro", src: join(runDir, "session.json"), why: "the session file stays as the host wrote it" },
+176 -17
View File
@@ -20,17 +20,22 @@
// every launch, refusal and end
// workspaces/<business>/<instance>/ the instance's workspace (0700), kept
// across launches
// threads/<business>/<thread>/ a thread's session store (0700), kept
// across the thread's runs (row 121):
// threads/<business>/<thread>/ a legacy thread session's store (0700),
// kept across its runs (row 121):
// session/ and claude-projects/
// threads/<business>/<thread>/sessions/<session>/
// any other session's store (row 269)
// session-workspaces/<business>/<instance>/<session>/
// a thread session's workspace (0700),
// with <session>.lock beside it
// bus-host/sessions.json the running sessions (0600), written
// only by the host; `mosaic stop` reads it
import { spawn } from "node:child_process";
import { randomBytes } from "node:crypto";
import { appendFileSync, closeSync, existsSync, lstatSync, mkdirSync, openSync, readFileSync, renameSync, writeFileSync } from "node:fs";
import { appendFileSync, closeSync, existsSync, linkSync, lstatSync, mkdirSync, openSync, readdirSync, readFileSync, realpathSync, renameSync, unlinkSync, writeFileSync } from "node:fs";
import { constants } from "node:os";
import { dirname, join } from "node:path";
import { dirname, join, sep } from "node:path";
import { childrenOf, cmdlineOf, startTimeOf } from "./proc.mjs";
import { binds, prepareRun, REPO, sandboxArgv } from "./sandbox.mjs";
import { SeatError } from "./seat.mjs";
@@ -38,7 +43,10 @@ import { ID_PATTERN } from "../../business/src/vocabulary.mjs";
export { REPO };
export const RUNNER = join(REPO, "packages", "harness", "src", "runner.mjs");
export const SESSIONS_VERSION = 1;
// Row 269 (264b): 2 adds `session` and `workspace` to a thread run's entry.
// readSessions still reads a version 1 file an earlier host left.
export const SESSIONS_VERSION = 2;
const SESSIONS_READ = [1, 2];
export const sessionsFile = (dataRoot) => join(dataRoot, "bus-host", "sessions.json");
export const launchSocketPath = (dataRoot) => join(dataRoot, "bus-host", "launch.sock");
@@ -47,6 +55,7 @@ export const gitSocketPath = (dataRoot) => join(dataRoot, "bus-host", "git.sock"
export const runsDir = (dataRoot, business) => join(dataRoot, "launches", business);
export const launchLogFile = (dataRoot, business) => join(dataRoot, "launches", `${business}.jsonl`);
export const workspaceDir = (dataRoot, business, instance) => join(dataRoot, "workspaces", business, instance);
export const sessionWorkspaceDir = (dataRoot, business, instance, session) => join(dataRoot, "session-workspaces", business, instance, session);
// Row 121 (#1615): a thread's session store, which outlives each of the
// thread's runs so the next run resumes the harness session (REQ-THREAD-2).
@@ -56,24 +65,174 @@ export const workspaceDir = (dataRoot, business, instance) => join(dataRoot, "wo
// is the run's sessionDir; claude-projects/ is bound at the run's
// home/.claude/projects (sandbox.mjs binds()). Throws SeatError.
const segment = (x) => typeof x === "string" && ID_PATTERN.test(x) && !x.includes("..");
export function threadStore(dataRoot, business, thread) {
for (const [what, x] of [["business", business], ["thread", thread]]) {
if (!segment(x)) throw new SeatError(`thread store: ${what} ${JSON.stringify(x)} isn't an id`, 2);
const seatError = (message, code) => Object.assign(new SeatError(message, 2), { code });
function segments(what, pairs, code) {
for (const [name, x] of pairs) {
if (!segment(x)) throw seatError(`${what}: ${name} ${JSON.stringify(x)} isn't an id`, code);
}
const dir = join(dataRoot, "threads", business, thread);
const out = { dir, session: join(dir, "session"), projects: join(dir, "claude-projects") };
for (const d of [join(dataRoot, "threads"), join(dataRoot, "threads", business), dir, out.session, out.projects]) {
}
// Makes each directory in order, then checks it is ours, a real directory and private.
function privateDirs(what, dirs, code) {
for (const d of dirs) {
try {
mkdirSync(d, { mode: 0o700 });
} catch (e) {
if (e.code !== "EEXIST") throw e;
}
const st = lstatSync(d);
if (st.isSymbolicLink() || !st.isDirectory() || st.uid !== process.getuid() || st.mode & 0o077) throw new SeatError(`thread store: ${d} is unsafe`, 2);
if (st.isSymbolicLink() || !st.isDirectory() || st.uid !== process.getuid() || st.mode & 0o077) throw seatError(`${what}: ${d} is unsafe`, code);
}
}
export function threadStore(dataRoot, business, thread) {
segments("thread store", [["business", business], ["thread", thread]], "thread-store-refused");
const dir = join(dataRoot, "threads", business, thread);
const out = { dir, session: join(dir, "session"), projects: join(dir, "claude-projects") };
privateDirs("thread store", [join(dataRoot, "threads"), join(dataRoot, "threads", business), dir, out.session, out.projects], "thread-store-refused");
return out;
}
// Row 269 (264b, #1774): one session's store. A legacy session (op legacy-3l,
// whose id is its thread's) keeps the thread's store, so its harness resumes
// the history it has; any other session gets its own under sessions/, with
// the same checks on every segment and directory. Same shape as threadStore.
// legacyThread: the thread has a legacy session. When it has none, a
// threads/<b>/<t>/session left by a 264a run of the thread's default session
// refuses legacy-store-unmapped, so that history isn't silently left behind.
export function sessionStore(dataRoot, business, thread, session, { legacy = false, legacyThread = legacy } = {}) {
if (legacy) return threadStore(dataRoot, business, thread);
segments("thread store", [["business", business], ["thread", thread], ["session", session]], "thread-store-refused");
const threadDir = join(dataRoot, "threads", business, thread);
if (!legacyThread) {
let old = true;
try {
lstatSync(join(threadDir, "session"));
} catch (e) {
if (e.code !== "ENOENT") throw e;
old = false;
}
if (old) throw seatError(`thread store: ${join(threadDir, "session")} holds history no legacy session maps`, "legacy-store-unmapped");
}
const dir = join(threadDir, "sessions", session);
const out = { dir, session: join(dir, "session"), projects: join(dir, "claude-projects") };
privateDirs("thread store", [join(dataRoot, "threads"), join(dataRoot, "threads", business), threadDir, join(threadDir, "sessions"), dir, out.session, out.projects], "thread-store-refused");
return out;
}
const inside = (a, b) => a === b || a.startsWith(b + sep);
const realOrNull = (p) => {
try {
return realpathSync(p);
} catch (e) {
if (e.code === "ENOENT") return null;
throw e;
}
};
// Row 269: a thread session's workspace, session-workspaces/<b>/<i>/<s>. Each
// segment is an id and each directory ours, real and private
// (session-workspace-unsafe). Then the realpaths: it may not be inside, or
// contain, the legacy workspaces root, its business and instance directories
// or any entry of either (old instance, .retired and task workspaces), nor
// sit above or below a `live` workspace (session-workspace-overlap). The same
// path as a live one is left to the lock, which refuses session-workspace-busy.
export function sessionWorkspace(dataRoot, business, instance, session, { live = [] } = {}) {
segments("session workspace", [["business", business], ["instance", instance], ["session", session]], "session-workspace-unsafe");
const root = join(dataRoot, "session-workspaces");
const path = join(root, business, instance, session);
privateDirs("session workspace", [root, join(root, business), join(root, business, instance), path], "session-workspace-unsafe");
const real = realpathSync(path);
const legacy = join(dataRoot, "workspaces");
const named = [legacy, join(legacy, business), join(legacy, business, instance)];
const entries = (d) => {
try {
return readdirSync(d).map((x) => join(d, x));
} catch (e) {
if (e.code === "ENOENT" || e.code === "ENOTDIR") return [];
throw e;
}
};
for (const p of [...named, ...entries(legacy), ...entries(join(legacy, business))]) {
const other = realOrNull(p);
if (other && (inside(real, other) || inside(other, real))) throw seatError(`session workspace: ${path} overlaps the legacy workspace ${p}`, "session-workspace-overlap");
}
for (const p of live) {
const other = realOrNull(p);
if (other && other !== real && (inside(real, other) || inside(other, real))) throw seatError(`session workspace: ${path} overlaps the live workspace ${p}`, "session-workspace-overlap");
}
return path;
}
// Row 269: the single-writer lock on a session workspace, <workspace>.lock,
// beside it and outside every mount. It names the host (pid, start time) and,
// once spawned, the run's outer process. A lock whose host and run are both
// gone is stale and is taken over; any other refuses session-workspace-busy.
// Returns { file, update(fields), release() }.
const lockAlive = (pid, start) => Number.isSafeInteger(pid) && pid > 0 && start != null && startTimeOf(pid) === start;
export function lockSessionWorkspace(workspace, owner) {
const file = `${workspace}.lock`;
let held = { ...owner };
const write = (flag) => writeFileSync(file, `${JSON.stringify(held)}\n`, { mode: 0o600, flag });
const busy = (why) => seatError(`session workspace ${workspace} is busy: ${why}`, "session-workspace-busy");
for (let attempt = 0; attempt < 2; attempt++) {
try {
write("wx");
// One release per lock: a second unlink could take a lock another host has written since.
let released = false;
return {
file,
update(fields) {
if (released) return;
held = { ...held, ...fields };
write("w");
},
release() {
if (released) return;
released = true;
try {
unlinkSync(file);
} catch (e) {
if (e.code !== "ENOENT") throw e;
}
},
};
} catch (e) {
if (e.code !== "EEXIST") throw e;
}
let raw;
try {
raw = readFileSync(file, "utf8");
} catch (e) {
if (e.code === "ENOENT") continue;
throw e;
}
let other = null;
try {
other = JSON.parse(raw);
} catch {}
// An unreadable lock may be one being written, so it counts as held.
if (!other || typeof other !== "object") throw busy(`${file} is unreadable`);
if (lockAlive(other.hostPid, other.hostStartTime)) throw busy(`host pid ${other.hostPid} holds it`);
if (lockAlive(other.runPid, other.runStartTime)) throw busy(`run pid ${other.runPid} holds it`);
// Stale. Move it aside; if what moved isn't what was judged stale, another
// host took it over in between, so it goes back and this start refuses.
const aside = `${file}.stale-${process.pid}-${randomBytes(4).toString("hex")}`;
try {
renameSync(file, aside);
} catch (e) {
if (e.code === "ENOENT") continue;
throw e;
}
if (readFileSync(aside, "utf8") !== raw) {
try {
linkSync(aside, file);
} catch {}
unlinkSync(aside);
throw busy(`another host took ${file} over`);
}
unlinkSync(aside);
}
throw busy(`${file} could not be taken`);
}
// The session's environment is an allowlist: nothing else from the host's
// environment passes, so a founder credential variable (GITEA_TOKEN,
// GH_TOKEN, SSH_AUTH_SOCK, ...) never reaches a session (REQ-CRED-2).
@@ -212,10 +371,10 @@ export class Registry {
count(business, provider, model = undefined) {
return this.#sessions.filter((s) => s.business === business && s.provider === provider && (model === undefined || s.model === model)).length;
}
// A thread instance runs once per thread (row 121), so `thread` is part of
// the key; null is a session that holds no thread.
running(business, instance, thread = null) {
return this.#sessions.find((s) => s.business === business && s.instance === instance && (s.thread ?? null) === thread) ?? null;
// A thread instance runs once per thread session (row 121, row 269), so
// `thread` and `session` are part of the key; null is an entry without one.
running(business, instance, thread = null, session = null) {
return this.#sessions.find((s) => s.business === business && s.instance === instance && (s.thread ?? null) === thread && (s.session ?? null) === session) ?? null;
}
#write() {
if (this.#readOnly) return;
@@ -238,7 +397,7 @@ export function readSessions(dataRoot) {
} catch {
throw new SeatError(`sessions file is unreadable: ${file}`, 2);
}
if (doc?.sessionsVersion !== SESSIONS_VERSION || !Array.isArray(doc.sessions)) throw new SeatError(`sessions file is malformed: ${file}`, 2);
if (!SESSIONS_READ.includes(doc?.sessionsVersion) || !Array.isArray(doc.sessions)) throw new SeatError(`sessions file is malformed: ${file}`, 2);
return doc.sessions.map((s) => ({ ...s, live: startTimeOf(s.runnerPid) === s.runnerStartTime }));
}
@@ -0,0 +1,126 @@
// Row 269 (264b, #1774): a thread session's store, its workspace and the
// single-writer lock beside the workspace (session.mjs sessionStore(),
// sessionWorkspace(), lockSessionWorkspace()), and sessions.json v2.
import { test } from "node:test";
import assert from "node:assert/strict";
import { spawn } from "node:child_process";
import { existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, realpathSync, rmSync, statSync, symlinkSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";
import { startTimeOf } from "../src/proc.mjs";
import { SeatError } from "../src/seat.mjs";
import { lockSessionWorkspace, readSessions, Registry, sessionsFile, sessionStore, sessionWorkspace, threadStore } from "../src/session.mjs";
function scratch(t) {
const dir = realpathSync(mkdtempSync(join(tmpdir(), "mosaic-session-ws-")));
t.after(() => rmSync(dir, { recursive: true, force: true }));
return dir;
}
const mode = (p) => statSync(p).mode & 0o777;
const refuses = (code) => (e) => e instanceof SeatError && e.code === code;
test("REQ-THREAD-2 sessionStore: a legacy session keeps the thread's store, any other gets sessions/<s>/, each segment checked first", (t) => {
const root = scratch(t);
assert.deepEqual(sessionStore(root, "acme", "th-1", "th-1", { legacy: true }), threadStore(root, "acme", "th-1"));
const s = sessionStore(root, "acme", "th-1", "s-2", { legacyThread: true });
const dir = join(root, "threads", "acme", "th-1", "sessions", "s-2");
assert.deepEqual(s, { dir, session: join(dir, "session"), projects: join(dir, "claude-projects") });
for (const d of [dirname(dir), dir, s.session, s.projects]) assert.equal(mode(d), 0o700, d);
for (const [thread, session] of [["..", "s"], ["th-1", ".."], ["th-1", "a/b"], ["th-1", null]])
assert.throws(() => sessionStore(root, "acme", thread, session, { legacyThread: true }), refuses("thread-store-refused"), `${thread}/${session}`);
assert.deepEqual(readdirSync(join(root, "threads", "acme", "th-1", "sessions")), ["s-2"], "a refused call makes nothing");
});
test("REQ-THREAD-2 sessionStore refuses legacy-store-unmapped for a thread whose old store no legacy session maps", (t) => {
const root = scratch(t);
threadStore(root, "acme", "th-1");
assert.throws(() => sessionStore(root, "acme", "th-1", "s-1"), (e) => refuses("legacy-store-unmapped")(e) && /holds history no legacy session maps$/.test(e.message));
assert.equal(existsSync(join(root, "threads", "acme", "th-1", "sessions")), false);
// A thread with a legacy session: its other sessions take D2's layout.
assert.ok(sessionStore(root, "acme", "th-1", "s-1", { legacyThread: true }).dir.endsWith(join("sessions", "s-1")));
const elsewhere = join(root, "elsewhere");
mkdirSync(elsewhere, { mode: 0o700 });
symlinkSync(elsewhere, join(root, "threads", "acme", "th-2"));
assert.throws(() => sessionStore(root, "acme", "th-2", "s-1"), (e) => refuses("thread-store-refused")(e) && /is unsafe$/.test(e.message));
assert.deepEqual(readdirSync(elsewhere), []);
});
test("REQ-THREAD-2 sessionWorkspace: private directories per session, unsafe segments and overlaps refused", (t) => {
const root = scratch(t);
const ws = sessionWorkspace(root, "acme", "assistant", "s-1");
assert.equal(ws, join(root, "session-workspaces", "acme", "assistant", "s-1"));
for (const d of [join(root, "session-workspaces"), dirname(dirname(ws)), dirname(ws), ws]) assert.equal(mode(d), 0o700, d);
assert.equal(sessionWorkspace(root, "acme", "assistant", "s-1"), ws, "a second call reuses it");
assert.notEqual(sessionWorkspace(root, "acme", "assistant", "s-2"), ws);
for (const session of ["..", "a/b", ""]) assert.throws(() => sessionWorkspace(root, "acme", "assistant", session), refuses("session-workspace-unsafe"), session);
// A live workspace at the same path is the lock's to refuse; one above or below it overlaps.
assert.equal(sessionWorkspace(root, "acme", "assistant", "s-1", { live: [ws] }), ws);
assert.throws(() => sessionWorkspace(root, "acme", "assistant", "s-3", { live: [dirname(ws)] }), refuses("session-workspace-overlap"));
// The legacy root symlinked into a session path: every segment is real, only the realpath finds it.
mkdirSync(join(root, "workspaces"), { mode: 0o700 });
symlinkSync(ws, join(root, "workspaces", "acme"));
assert.throws(() => sessionWorkspace(root, "acme", "assistant", "s-1"), refuses("session-workspace-overlap"));
// A symlinked root fails the segment check first.
const other = scratch(t);
const elsewhere = join(other, "elsewhere");
mkdirSync(elsewhere, { mode: 0o700 });
symlinkSync(elsewhere, join(other, "session-workspaces"));
assert.throws(() => sessionWorkspace(other, "acme", "assistant", "s-1"), refuses("session-workspace-unsafe"));
assert.deepEqual(readdirSync(elsewhere), []);
});
test("REQ-THREAD-2 lockSessionWorkspace: one writer; a live host or run holds it, a stale lock is taken over, an unreadable one holds", async (t) => {
const root = scratch(t);
const ws = sessionWorkspace(root, "acme", "assistant", "s-1");
const me = { hostPid: process.pid, hostStartTime: startTimeOf(process.pid) };
const lock = lockSessionWorkspace(ws, me);
assert.equal(lock.file, `${ws}.lock`);
assert.equal(mode(lock.file), 0o600);
assert.throws(() => lockSessionWorkspace(ws, me), refuses("session-workspace-busy"));
lock.update({ run: "r-1", runPid: 1, runStartTime: "x" });
assert.deepEqual(JSON.parse(readFileSync(lock.file, "utf8")), { ...me, run: "r-1", runPid: 1, runStartTime: "x" });
lock.release();
assert.equal(existsSync(lock.file), false);
// A run that outlives its host still holds it.
const run = spawn("sleep", ["60"], { stdio: "ignore" });
const gone = new Promise((r) => run.once("exit", r));
t.after(() => run.kill("SIGKILL"));
let runStart = null;
for (let i = 0; i < 100 && !runStart; i++) {
runStart = startTimeOf(run.pid);
if (!runStart) await new Promise((r) => setTimeout(r, 20));
}
const held = `${JSON.stringify({ hostPid: 2 ** 22 + 7, hostStartTime: "gone", runPid: run.pid, runStartTime: runStart })}\n`;
writeFileSync(`${ws}.lock`, held, { mode: 0o600 });
assert.throws(() => lockSessionWorkspace(ws, me), refuses("session-workspace-busy"));
assert.equal(readFileSync(`${ws}.lock`, "utf8"), held);
run.kill("SIGKILL");
await gone;
// Now stale: taken over, and nothing is left beside it.
const taken = lockSessionWorkspace(ws, me);
assert.deepEqual(JSON.parse(readFileSync(taken.file, "utf8")), me);
assert.deepEqual(readdirSync(dirname(ws)).sort(), ["s-1", "s-1.lock"]);
// One release per lock: a second doesn't remove a lock written since.
taken.release();
writeFileSync(`${ws}.lock`, "{}\n", { mode: 0o600 });
taken.release();
assert.equal(readFileSync(`${ws}.lock`, "utf8"), "{}\n");
// An unreadable lock may be one being written, so it holds.
writeFileSync(`${ws}.lock`, "{", { mode: 0o600 });
assert.throws(() => lockSessionWorkspace(ws, me), refuses("session-workspace-busy"));
});
test("REQ-THREAD-2 sessions.json v2: entries carry session and workspace, running() keys on the session, a v1 file still reads", (t) => {
const root = scratch(t);
const registry = new Registry(root);
const entry = { business: "acme", instance: "assistant", thread: "th-1", session: "s-1", workspace: "/w", run: "r-1" };
registry.add(entry);
assert.equal(JSON.parse(readFileSync(sessionsFile(root), "utf8")).sessionsVersion, 2);
assert.deepEqual(registry.running("acme", "assistant", "th-1", "s-1"), entry);
assert.equal(registry.running("acme", "assistant", "th-1", "s-2"), null);
assert.equal(registry.running("acme", "assistant", "th-1"), null, "an entry with a session isn't the thread's sessionless run");
writeFileSync(sessionsFile(root), `${JSON.stringify({ sessionsVersion: 1, sessions: [{ business: "acme", instance: "assistant", thread: "th-1", run: "r-0" }] })}\n`);
assert.deepEqual(readSessions(root).map((s) => [s.run, s.session ?? null]), [["r-0", null]]);
});