Compare commits
12
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2d0d407ba6 | ||
|
|
6d9ab35039 | ||
|
|
2c16bc4209 | ||
|
|
9221786228 | ||
|
|
3d29876dfb | ||
|
|
a81b3446eb | ||
|
|
d595570a63 | ||
|
|
e2e51bae1f | ||
|
|
f684317eb0 | ||
|
|
04676f57b9 | ||
|
|
304bce2f13 | ||
|
|
875b7eeebf |
@@ -0,0 +1,449 @@
|
||||
# Row 269 build notes (Gizmoduck)
|
||||
|
||||
264b, T1 item 7: per-session native store, workspace and single-writer fence. Gitea #1774. Brief:
|
||||
docs/plans/2026-10-21_t1-sessions.md (blob ff0be0c79), slices in agents/sage/work/queue-264/RULINGS.md
|
||||
(5f5b51dd6). Reviewer Duckworth, gate owner Sage. Plan gizmoduck-0228, accepted with rulings in
|
||||
sage-2707. Branch gizmoduck/row269 from refactor aca16404c. Row 253 lands first, and this branch is
|
||||
rebased onto its tip, under a new name, before the gate.
|
||||
|
||||
Claim: rev 3217, op gizmoduck-0230, 2026-10-22 14:29:42Z. That was 21 s after Gander's slot opened
|
||||
at 14:29:21Z (sage-2710). Reported in gizmoduck-0231 and not reverted, since a revert would be a
|
||||
second queue write.
|
||||
|
||||
## D9 consumer audit, sessions.json (sage-2707 Q3)
|
||||
|
||||
Q3 allows `session` as an optional field with no version bump, if every sessions.json reader at
|
||||
84f53540d selects by run and not by business, instance or thread. Every reader at 84f53540d:
|
||||
|
||||
| Reader | file:line at 84f53540d | Selects by |
|
||||
| --- | --- | --- |
|
||||
| `readSessions` (the only parser) | packages/seat/src/session.mjs:232-243 | nothing. Refuses any `sessionsVersion` but 1 (:241) and returns every entry with `live` |
|
||||
| `stopSession` (`mosaic stop <run>`) | session.mjs:249-250 | run (`s.run === run`) |
|
||||
| `launches list` | packages/cli/src/cli.mjs:526-530 | nothing. Prints every entry, and `--json` dumps them whole |
|
||||
| host start, `leftover` | packages/cli/src/launcher.mjs:235 | business, then `recover()` works one entry (one run) at a time (:955-972) |
|
||||
| runtime launch settings | packages/bus/src/runtime.mjs:149 -> packages/business/src/launch-settings.mjs:166 -> packages/business/src/launch.mjs:116-135 (`usageCounts`) | business. Counts entries per provider and model |
|
||||
| Console settings (C1, duckworth-0150) | packages/webui/src/reads.mjs:234 -> the same launchSettings `read()` -> readSessions | business, through usageCounts. Any throw becomes `launch: {refused: 'not-configured'}` |
|
||||
| stack-release R guard (C1) | scripts/stack-release.sh:2091-2096, its own JSON.parse, no version check | nothing. Takes `pid` and `runnerPid` of every entry in `sessions` and fails `sessions-live` if one is alive in /proc |
|
||||
|
||||
Not a reader: stack-release.sh:2199-2204 checks only that no sessions.json exists. packages/ledger
|
||||
readSessions (ledger.mjs:146) is a different function that reads seat directories.
|
||||
launch-settings `current()` (:129) doesn't call `sessions()`, so the limits the bus enforces don't
|
||||
read the file; only the `limits` verb's `read()` (:166, runtime.mjs:292) does.
|
||||
|
||||
Tests that read the file: packages/cli/tests/launcher.test.mjs (`readSessions` at :164 through
|
||||
:1097, each by run, by instance or whole) and packages/cli/tests/host.test.mjs:830 (file absent).
|
||||
|
||||
Not a file reader: `Registry` (session.mjs:192-230). Its list starts empty (:193), and the constructor
|
||||
writes it over the file (:198). `running(business, instance, thread)` at :217 is the one key on the
|
||||
thread, but it reads only the in-memory list this host added (launcher.mjs:529, removed at :553).
|
||||
Its callers are launcher.mjs:289 (check) and the thread pass. This row changes that key (see Design).
|
||||
|
||||
Result: no file reader keys on the thread. Q3's condition holds as worded.
|
||||
|
||||
One more finding sends me to v2 anyway. `recover()` rebinds a leftover run with
|
||||
`{business, role, run, harness, pid, startTime, thread}` and no `session` (launcher.mjs:964).
|
||||
The broker compares that record's body with the recorded `session.launched` body, and the body
|
||||
carries `session` whenever the launch record had one (broker.mjs:945). A mismatch refuses
|
||||
`launch-record-mismatch` (broker.mjs:952). Each new thread run binds with its session (Design), so
|
||||
after a rollback to 264a, a 264a host that finds a 264b entry logs "not ended, bindLaunch refused:
|
||||
launch-record-mismatch" (launcher.mjs:968), and the run's claim stays held. The host has
|
||||
already killed the process (:956-962), but it can't end the run. With v1 and an optional field,
|
||||
that failure is quiet.
|
||||
|
||||
Decision: sessions.json goes to v2 (`SESSIONS_VERSION` 2). New code reads v1 and v2 and writes v2.
|
||||
A v2 file keeps the `sessions` array and every v1 field of each entry, `pid` and `runnerPid`
|
||||
included, so the R guard (stack-release.sh:2094-2095) reads it unchanged (C1).
|
||||
A v1 entry has no `session`, and it is treated as the thread's legacy default session (D9). After a
|
||||
rollback, a 264a host refuses to start with exit 3, "sessions file is malformed"
|
||||
(session.mjs:241 via launcher.mjs:237), and the README's remedy applies: check which sessions still
|
||||
run, then move the file aside. Tradeoff: a rollback while runs are live needs a hand step, but it
|
||||
fails loudly at host start instead of leaving held claims. The rollback note names this refusal.
|
||||
|
||||
## D9 consumer audit, threadsState and threadIdle (sage-2707 audit addition)
|
||||
|
||||
These are trusted launcher APIs, not socket verbs (packages/bus/README.md:288, :294), and they're
|
||||
reached only through the host op table (packages/bus/src/process.mjs:23-24, :54-55;
|
||||
runtime.mjs:561-562). At aca16404c:
|
||||
|
||||
| Consumer | file:line | What it reads |
|
||||
| --- | --- | --- |
|
||||
| launcher thread pass | packages/cli/src/launcher.mjs:596, :611 | every row's `thread`, `holder`, `pending`, `unread`; threadIdle `{business, thread, run}` |
|
||||
| bus end-launch.test | packages/bus/tests/end-launch.test.mjs:193, :198 | whole rows (deepEqual), and `not-holder` |
|
||||
| bus thread-migration.test | packages/bus/tests/thread-migration.test.mjs:139 | `thread, holder, pending, unread` |
|
||||
| bus threads.test | packages/bus/tests/threads.test.mjs:352, :375-376, :407, :416 | rows keyed by `thread` |
|
||||
| cli launcher.test stub | packages/cli/tests/launcher.test.mjs:989 | wraps `host.op`, counts the op name |
|
||||
| cli lifecycle.test stub | packages/cli/tests/lifecycle.test.mjs:90-91, :749-752 | passes through, wraps `runtime.threadsState` |
|
||||
| cli thread-launch.test stub | packages/cli/tests/thread-launch.test.mjs:90-91, :163, :264-265, :304, :348, :443 | `.find(x => x.thread === ...)` then `holder` |
|
||||
|
||||
Console and webui: no reader. `git grep threadsState` over packages/webui, packages/control-board and
|
||||
scripts has no hit. The webui reads `thread.claims` (packages/webui/src/serve.mjs, cli.mjs,
|
||||
public/bus.js, tests shell.test, thread-claims.test, thread-claims-browser.test). This row doesn't change
|
||||
`thread.claims`. The webui's `session.launched` line reads only `harness` and `address`
|
||||
(public/bus.js:151), so a `session` field in the body changes nothing it shows.
|
||||
|
||||
## D9 consumer audit, workspace and native store
|
||||
|
||||
| Consumer | file:line at aca16404c | Change in this row |
|
||||
| --- | --- | --- |
|
||||
| `workspaceDir` | packages/seat/src/session.mjs:49 | unchanged, still the non-thread path and the legacy path |
|
||||
| launcher start, workspace | packages/cli/src/launcher.mjs:397-403 | a thread run uses its session workspace (Q1) |
|
||||
| launcher instance delete (V5) | launcher.mjs:841-866 | also retires the instance's session workspaces |
|
||||
| `threadStore` | session.mjs:59-74, launcher.mjs:388-393 | becomes the legacy branch of `sessionStore` |
|
||||
| sandbox binds | packages/seat/src/sandbox.mjs:102-125 | the workspace gets a separate in-sandbox path |
|
||||
| harness `transcript()` | packages/plugins/harness/pi/index.mjs:75, claude-code/index.mjs:130 | none: `{session, projects}` keeps its shape |
|
||||
| in-sandbox readers of `session.workspace` | packages/harness/src/runner.mjs:332, :386, :475-476; bundle.mjs:258, :327; gate.mjs:152 | none: they get the in-sandbox path |
|
||||
| task runner workspaces | scripts/mosaic-task.mjs:351 | none: `<dataRoot>/workspaces/<task>` is a legacy root the overlap check covers |
|
||||
| git socket | packages/cli/src/git-socket.mjs:7 | none: it takes bundles and reads no workspace |
|
||||
|
||||
## Pi and Claude resume, read offline (sage-2707: no pi run while P19 383 holds)
|
||||
|
||||
Pinned `@earendil-works/pi-coding-agent` 0.85.1 (package.json:8), read in node_modules/.../dist:
|
||||
|
||||
- adapters/pi/adapter.sh:46-52 passes `--session-dir "$MOSAIC_SESSION_DIR"`, plus `-c` when that
|
||||
directory isn't empty.
|
||||
- With `-c` and an explicit session dir, `continueRecent` filters by cwd
|
||||
(core/session-manager.js:1244-1250). It opens the newest `*.jsonl` whose header cwd resolves to
|
||||
the process cwd (:397-410, `sessionCwdMatches` :393-395). With no match, it makes a new session in
|
||||
the same directory (:1251), and nothing fails.
|
||||
- Both adapters `cd "$MOSAIC_WORKSPACE"` (pi adapter.sh:34-36, claude adapter.sh:46-47), the runner
|
||||
sets `MOSAIC_WORKSPACE` from `session.workspace` (runner.mjs:386), and the sandbox binds the
|
||||
workspace at its host path (sandbox.mjs:112, `argsOf` :128-131). So a legacy transcript's header
|
||||
cwd is `<dataRoot>/workspaces/<business>/<instance>`.
|
||||
- Opening a file whose header cwd doesn't exist exits 1 in a headless run (main.js:536-548,
|
||||
core/session-cwd.js:2-15). So `--session <file>` is no way around it either.
|
||||
|
||||
Result: a legacy default session run from `session-workspaces/.../<session>` as its cwd would
|
||||
quietly start a new pi conversation, and its old history wouldn't resume. Claude Code keys its
|
||||
projects directory on the cwd, and the adapter resumes by id (claude adapter.sh:53-59). I haven't
|
||||
read the Claude binary, so whether `--resume` finds an id saved under another cwd is unverified. I
|
||||
treat it as the same break.
|
||||
|
||||
## Design
|
||||
|
||||
### Legacy mapping (D9, M7)
|
||||
|
||||
A session is legacy when its row has `op='legacy-3l'`, which the migration gave every pre-3l thread
|
||||
with `id` = the thread id (3k-3l.sql:50-51). The rule reads only an immutable row, never the disk.
|
||||
|
||||
- A legacy session keeps the thread's existing store, `threads/<b>/<t>/{session,claude-projects}`
|
||||
(`threadStore`, unchanged).
|
||||
- Its workspace is new (Q1), `session-workspaces/<b>/<instance>/<session>/`, but inside the sandbox
|
||||
it's bound at the legacy path `workspaces/<b>/<instance>`. The cwd its transcripts recorded is
|
||||
unchanged, so pi `-c` and Claude `--resume` find them. The host's old instance workspace isn't
|
||||
mounted, so it stays as history (Q1, D9). Every in-sandbox reader gets the sandbox path (table
|
||||
above), and host git (`ensureCheckouts`) gets the host path.
|
||||
- Every other session uses D2's layout for both, and its sandbox path is its host path.
|
||||
- A thread made by 264a's `thread.create` has a random default id (broker.mjs:5993), so this rule
|
||||
doesn't call it legacy. A 264a launcher ran that thread from `threads/<b>/<t>/session`. A start
|
||||
for a session whose thread has no legacy-3l session, while `threads/<b>/<t>/session` exists,
|
||||
refuses `legacy-store-unmapped`, so it fails closed and doesn't start a fresh conversation. The
|
||||
re-pin plan lists such threads. If 264a and 264b go live together, there are none.
|
||||
|
||||
### Broker
|
||||
|
||||
- `threadsState(business)`: one row per session of every thread, not just the default, ordered by
|
||||
thread seq, then session seq. Each row adds `session` and `legacy` to the fields it has now.
|
||||
`#threadRows` joins `sessions` in place of `session_defaults` (broker.mjs:868-896).
|
||||
- `threadIdle(record)`: same keys. The row is the one for the session the run was bound to
|
||||
(`#runs` entry, broker.mjs:927), so a run is idled against its own session.
|
||||
- `bindLaunch`: no new refusal. It already takes `session` and puts it in the body (broker.mjs:945).
|
||||
The broker half of the single-writer fence is the per-session claim that exists now:
|
||||
a second claim on a held session refuses `thread-already-held` (broker.mjs:4815). A
|
||||
`session-running` refusal at bind would break two old tests that bind a second unended run to one
|
||||
session and then expect the claim refusal (threads.test.mjs:88-89, sessions.test.mjs:119-120).
|
||||
The launcher's registry check and the workspace lock are the host halves.
|
||||
|
||||
### Seat
|
||||
|
||||
- `sessionStore(dataRoot, business, thread, session, { legacy })`: the legacy branch is
|
||||
`threadStore`. Otherwise it's `threads/<b>/<t>/sessions/<s>/{session,claude-projects}`, with
|
||||
`threadStore`'s checks on every segment and directory (ID_PATTERN, no `..`, owner, 0700, no
|
||||
symlink), giving the same `{dir, session, projects}` shape.
|
||||
- `sessionWorkspace(dataRoot, business, instance, session, { live })`: makes
|
||||
`session-workspaces/<b>/<i>/<s>` 0700 with the same checks, then compares realpaths. It refuses
|
||||
`session-workspace-overlap` when the path is inside, or contains, `<dataRoot>/workspaces` (legacy,
|
||||
`.retired` and task workspaces) or any workspace in `live`. A segment that fails the lstat checks
|
||||
refuses `session-workspace-unsafe` first (C3).
|
||||
- Lock: `lockSessionWorkspace(path, owner)` makes `<path>.lock` with O_EXCL, next to the workspace
|
||||
and outside every mount. It holds the host's pid and start time, and the run's outer pid and
|
||||
start time once spawned. A lock whose host and run are both gone is stale and is taken over.
|
||||
Anything else refuses `session-workspace-busy`. start() takes it before `ensureCheckouts` and
|
||||
releases it only after the run's `close` (proven exit) or a failed start.
|
||||
- `Registry`: entries gain `session` and `workspace`, and the key becomes `running(business,
|
||||
instance, thread, session)`. `SESSIONS_VERSION` is 2, and `readSessions` reads 1 and 2.
|
||||
|
||||
### Launcher
|
||||
|
||||
- check(instance, { thread, session }): `instance-running` is per session. Q2 adds
|
||||
`legacy-run-live`: a start for a thread instance refuses while any registry entry or unrecovered
|
||||
leftover for that instance has no `session`.
|
||||
A survivor is still bound and ended host-lost (launcher.mjs:1030-1040), so the broker sees its
|
||||
claim free while it runs. A released claim doesn't prove the run has exited. Only this launcher
|
||||
check holds the instance then, for sessionless survivors and thread starts only (duckworth-0158
|
||||
N1). recover() waits up to 5 s per leftover for the outer pid and the runner (N2, see "recover()
|
||||
change, for a design read").
|
||||
Residual (duckworth-0159 N3): a survivor with a `session` counts nowhere, because :309 counts
|
||||
only sessionless entries. The same session could start again while the old runner lives. The
|
||||
lock doesn't catch it either: it records the outer pid as runPid (launcher.mjs:574), so with the
|
||||
outer pid gone it reads stale and is taken over. The base has the same exposure. In practice
|
||||
the runner dies with its pid namespace when the outer bwrap is killed.
|
||||
- start(): the store and workspace come from the session. The lock covers checkouts and spawn.
|
||||
`bindLaunch` gets `session`, and the entry records it.
|
||||
- threadPass(): one pass per session row. `startedFor` and `refusedThreads` key on the session.
|
||||
The `thread-idle`, `refused` and `end` records add `session`.
|
||||
- recover(): passes the entry's `session` when present.
|
||||
- remove(): moves `session-workspaces/<b>/<instance>` to `session-workspaces/<b>/.retired/<instance>-<transaction>`
|
||||
after the legacy move, and records `workspace-retired` with that path.
|
||||
- Admission (RULINGS Q5, D8): a named session starts only through threadPass, then check(), then
|
||||
start(), so it passes through every `admission()` call start() makes now (launcher.mjs:382-524).
|
||||
There is no second start path.
|
||||
|
||||
### Sandbox
|
||||
|
||||
`binds()` takes `workspaceDest` (default `workspace`), the in-sandbox path for the rw workspace
|
||||
bind. The store binds keep their shape: only the selected session's two directories.
|
||||
|
||||
## Tests
|
||||
|
||||
T1 run half, with the fake harness adapter from thread-launch.test (no pi or Claude binary):
|
||||
two sessions of one thread run at once, each with its own sentinel history, store and workspace;
|
||||
stopping one and restarting it resumes only that one. The legacy session resumes from the old
|
||||
store at the old in-sandbox cwd. A non-thread instance gets `thread-not-allowed`.
|
||||
|
||||
T2: a second start on a live session's workspace refuses `session-workspace-busy` before any
|
||||
checkout; a hand-made live lock from another host refuses; a symlinked or nested session root
|
||||
refuses `session-workspace-unsafe`; a workspace root symlinked into the session path refuses
|
||||
`session-workspace-overlap`; a store segment symlink refuses `thread-store-refused`. Under a halt, a
|
||||
named session's start refuses `processing-halted` and spawns nothing. A generation change after
|
||||
check() refuses `processing-generation-changed` and spawns nothing (M6).
|
||||
|
||||
N1 (DEFERRED.md:474-480): `refusedOnce` in packages/bus/tests/sessions.test.mjs pins `actor_run`.
|
||||
|
||||
## Old assertions that change (C4, duckworth-0150, mos-1285)
|
||||
|
||||
Every pre-existing test that reads threadsState at aca16404c, checked against one row per session:
|
||||
- packages/bus/tests/end-launch.test.mjs:193-195 deepEquals the whole row. This one changes. Mos
|
||||
allowed it in mos-1285 as an exact update: still a whole deepEqual on threadsState('demo'), still
|
||||
exactly one th-1 row, every old key and value the same, and two new keys with literal values,
|
||||
`session: 'th-1'` (the fixture's legacy default, :14-15) and `legacy: true` (op `legacy-3l`). The
|
||||
commit message names mos-1285 and the old row shape.
|
||||
- packages/bus/tests/threads.test.mjs:352, :375-376: fromEntries keyed by thread. Neither test, and
|
||||
neither fixture (`twoThreads`), makes a second session (no `session.create` in the file), so each
|
||||
thread still has one row. Unchanged.
|
||||
- packages/bus/tests/thread-migration.test.mjs:139: maps four fields of a one-session thread.
|
||||
Unchanged.
|
||||
- packages/cli/tests/thread-launch.test.mjs:163, :304, :348, :443: `find` by thread. One session
|
||||
per thread in every fixture. Unchanged. The default session also sorts first, by session seq.
|
||||
- packages/cli/tests/lifecycle.test.mjs:90-91, :749-752 and thread-launch.test.mjs:90-91, :264-265
|
||||
pass threadsState through. Unchanged.
|
||||
- packages/cli/tests/thread-launch.test.mjs:182 (found in the code round, after mos-1285). The test
|
||||
makes threads with `thread.create` (:171-172), so A's default session has a random id and op
|
||||
`default` (broker.mjs:5992-6009), not `legacy-3l`. The brief puts every new store in D2's layout
|
||||
(t1-sessions.md:100, :294), so A's store moves to `threads/acme/<A>/sessions/<A's session>`. The
|
||||
proposed exact update changes only the `store` path on :182, taking the session id from
|
||||
`threadsState` (the test already reads it at :162). Lines :183-185 stay as they are. One added
|
||||
line asserts that `threads/acme/<A>/session` doesn't exist. This goes to Mos through Sage, and no
|
||||
code commit edits the file until Mos rules.
|
||||
- The same file's :375 (`../evil` refuses `thread-store-refused`) and :381, :384 (no `threads`, no
|
||||
`workspaces`) stay unchanged: start() checks the store before it makes the session workspace.
|
||||
|
||||
Any other old assertion the code changes goes to Mos through Sage before the commit. After the
|
||||
first run I survey every touched file whole for one.
|
||||
|
||||
## Mutants
|
||||
|
||||
Each kill counts only as an assertion failure in a test that runs that path, never a throw or
|
||||
another fence (sage-2707, C3 duckworth-0128).
|
||||
|
||||
| Id | Change | Killed by |
|
||||
| --- | --- | --- |
|
||||
| M1 | `sessionWorkspace` drops the session segment: a thread's sessions share one checkout | T2 same-thread fence: `session-workspace-busy`, no second spawn |
|
||||
| M2 | the session segment becomes the session name: two threads' `default` sessions share one checkout | T2 cross-thread fence, same assertion |
|
||||
| M3 | the lock ignores a live holder | T2 hand-made lock: `session-workspace-busy` |
|
||||
| M4 | the realpath overlap check removed | T2 nesting: `session-workspace-unsafe` |
|
||||
| M5 | the launcher passes the thread's default store for every session | T1: the second session's sentinel history is its own |
|
||||
| M6 | start() calls a no-op in place of every `admission(admissionGeneration)` (launcher.mjs:382, :396, :407, :420, :495, :499, :520, :524) for a non-legacy session | the generation test (below): `assert.rejects` with code `processing-generation-changed`, then `assert.equal` spawn count 0 |
|
||||
| M7 | the legacy session maps to `sessions/<s>/` | the D9 test: the legacy run resumes the sentinel in `threads/<b>/<t>/session` |
|
||||
| M8 | a non-default session binds its workspace at the legacy `workspaces/<b>/<instance>` in-sandbox path | T1: `assert.equal` that a named session's in-sandbox workspace is its `session-workspaces/<b>/<instance>/<s>` path, read from the fake adapter's recorded cwd (sage-2717) |
|
||||
| M9 | `#threadRows` back to the default session only | T1: the named session's message starts its run |
|
||||
| M10 | the `legacy-run-live` refusal removed from check() | the legacy-run-live test: `outcome` equals `[[S, "legacy-run-live"]]` |
|
||||
| M11 | recover() keeps a survivor only when the outer pid outlived the kill (the runner ignored) | the legacy-run-live test, same assertion |
|
||||
| M12 | check() reads every survivor, live or not | the legacy-run-live test: after the runner exits, `settle` gives one launch |
|
||||
|
||||
M6 and the halt fences (C2, duckworth-0150). A plain halt never reaches start(). tick() admits first
|
||||
(:579) and runs no thread pass while halted (:583), and check() admits at :282 and :376. The kill
|
||||
test lands the change between check() returning and the spawn:
|
||||
- The fake host's `credentialStatus` op (start() calls it at :395, after check() returned and after
|
||||
:382) records that it ran, then halts and resumes processing through `transition`. That leaves
|
||||
`halted` false and a new generation (state.mjs:115-121).
|
||||
- The tick and check fences see no halt, and start()'s second check() (:419) takes its own
|
||||
generation, so it passes too. withProcessing at :406 has no parent context (the launcher never
|
||||
opens one) and checks only `halted`, so it passes. The one fence left is start()'s own
|
||||
`admission(admissionGeneration)`, which refuses `processing-generation-changed` at :396.
|
||||
- The test asserts the hook ran (check passed), then the refusal code, then no spawn.
|
||||
M6 removes all eight calls because removing only :382 leaves :396 to refuse. The T2 halt test stays
|
||||
as written in Tests, but it doesn't kill M6 and isn't counted for it.
|
||||
|
||||
M4 and the segment checks (C3, duckworth-0150). The lstat check on each segment refuses a symlinked
|
||||
session root before the realpath compare runs, so the compare gets its own code,
|
||||
`session-workspace-overlap`, and the segment check keeps `session-workspace-unsafe`. M4's kill input
|
||||
passes every segment check: `<dataRoot>/workspaces/<b>` is a symlink to
|
||||
`session-workspaces/<b>/<i>/<s>`. Each segment of the session path is a real directory, and only the
|
||||
realpath compare finds that the session path contains the legacy workspace root. The test asserts
|
||||
`assert.rejects` with code `session-workspace-overlap` and no spawn. With M4 the start spawns, so
|
||||
the rejection assertion fails. The segment test keeps asserting `session-workspace-unsafe` for a
|
||||
symlinked session root, so it tells the two codes apart.
|
||||
|
||||
Duckworth's check for the early look (sage-2717): a non-default session never gets the legacy
|
||||
`workspaces/<b>/<instance>` bind path. Only a session with op `legacy-3l` takes `workspaceDest`
|
||||
set to the legacy path. M8 is that mutant.
|
||||
|
||||
### Code round corrections (local, not yet run)
|
||||
|
||||
Written before the first run, which waits for Sage's CLEAR.
|
||||
|
||||
- Tests live in two new files: packages/cli/tests/session-launch.test.mjs (T1, D9, T2) with a new
|
||||
stub adapter, fixtures/session-adapter.mjs (thread-adapter.mjs's turn plus the run's cwd and its
|
||||
entries; thread-adapter.mjs is unchanged), and packages/seat/tests/session-workspace.test.mjs
|
||||
(sessionStore, sessionWorkspace, the lock, sessions.json v2). The new launch tests copy
|
||||
thread-launch.test.mjs's host helpers rather than change them, so no other old test file changes.
|
||||
- M1 and M2 kill in T1, not in a busy test. A test that expects `session-workspace-busy` still
|
||||
passes when sessions share a path. T1 starts thread A's default and named sessions and thread B's
|
||||
default at once and asserts no refusal, three live entries and three distinct cwds. With M1 (A's
|
||||
two sessions) or M2 (the two `default` sessions) the second start refuses busy, and the
|
||||
`deepEqual(refused, [])` assertion fails.
|
||||
- Every wait a mutant could stall returns what it last saw (`settle`, `outcome`), so a kill is the
|
||||
assertion after it, never a timeout. That covers M3, M4, M6 and M9, where the mutant's run starts
|
||||
or never starts.
|
||||
- The M6 test also carries the halt fence. start() admits right after credentialStatus, before it
|
||||
makes the session workspace, so the first refusal leaves no workspace. On the retry the recheck's
|
||||
mayLaunch op halts after the lock is taken. The recheck refuses `processing-halted`, and the test
|
||||
asserts that the lock is gone (startLocked's throw releases it).
|
||||
- The lock's release() runs once. A second call would unlink a lock another host has written since.
|
||||
- `legacy-run-live` now has a test (Duckworth's early warning in duckworth-0156). Before, recover()
|
||||
kept a survivor only when a bwrap pid outlived SIGKILL, which a test can't make. It also ended
|
||||
a leftover whose runner still ran once the outer pid was gone, so the refusal never saw that run.
|
||||
Now recover() waits up to 5 s on the outer pid and the runner. It keeps any leftover where
|
||||
either still runs, and logs `run <run> (<instance>) from an earlier host still runs`. check()
|
||||
reads only survivors that still run, so a start goes ahead once the run has exited. The test
|
||||
writes a sessionless leftover whose runner is a sleep child and whose outer pid is gone. It
|
||||
asserts the refusal, no launch, no session workspace, and then one launch after the child is
|
||||
killed. M10-M12 cover it. The registry and pre-recover leftover terms stay. A launching host
|
||||
can't reach them for a thread instance (check() refuses `session-required` first, and listen()
|
||||
runs recover() first), but they cost nothing.
|
||||
|
||||
### recover() change, for a design read (sage-2750)
|
||||
|
||||
Sage asked for this because recover() predates the row. Lines are launcher.mjs at 3d29876df plus
|
||||
the log-line edit after it, against aca16404c:947-976.
|
||||
|
||||
- What stays the same: every leftover is still bound and ended as host-lost (:1030-1040, base
|
||||
:963-973). The survivor push (:1026-1029) comes before the bind, and the loop carries on past it,
|
||||
so a kept leftover isn't left unended. Its bus run ends and its role is released, as in the base.
|
||||
The kill is unchanged too: SIGKILL only to a live outer pid whose cmdline is bwrap (:1018-1021,
|
||||
base :956-959).
|
||||
- What changes:
|
||||
1. The wait after the kill runs while the outer pid or the runner still runs (:1023). In the
|
||||
base it waited on the outer pid only (base :961).
|
||||
2. A leftover where either still runs after that goes on `survivors` and is logged (:1026-1029).
|
||||
3. check() counts a survivor only while it still runs (:309, `liveRun` at :255).
|
||||
- a. Nothing in this row kills a kept leftover's runner. The base didn't either: it ended the run
|
||||
and let the runner go on, unseen. The run is still ended (:1038), so the runner holds no role.
|
||||
Its capability died with the old broker (:1004-1008). The only thing a survivor does is refuse
|
||||
`legacy-run-live` for sessions of its instance while it lives, and each refusal is a `refused`
|
||||
launch record. For an operator, the log line names the run, the outer pid and the runner pid
|
||||
(:1028). Killing that pid clears it. check() re-reads liveness on each pass (:309), so the next
|
||||
tick starts the session with no host restart. If bindLaunch refuses (:1032-1036), the run stays
|
||||
unended and logged, as in the base (base :965-969). The survivor entry doesn't change that.
|
||||
- b. The 5 s wait only applies inside the kill branch (:1018-1024), so only to a leftover whose
|
||||
outer bwrap was still alive. A leftover with nothing running doesn't wait, and nor does one
|
||||
whose outer pid is gone: neither enters the branch. The wait ends as soon as both processes are
|
||||
gone. The base already waited up to 5 s in the same branch. It can now last longer only if the
|
||||
runner outlives its outer pid. With `--unshare-pid` and `--die-with-parent` the runner dies when
|
||||
the namespace goes.
|
||||
- c. Old tests that run recover() with a launching host and a leftover:
|
||||
- launcher.test.mjs:885-943, the "host that died hard" pair. It asserts the outer pid and the
|
||||
runner are gone right after setup (:917-918), the host-lost log (:919), one session.ended
|
||||
(:920-922), one end record (:924), an empty sessions.json (:925), and then "already ended" for
|
||||
a leftover whose processes are dead (:937-941). None of these expected values change.
|
||||
Waiting on the runner makes :918 hold sooner, not later. The new log line can appear only if
|
||||
a process outlives 5 s, which would already fail :917-918. The dead-process third host takes
|
||||
no wait and logs no survivor.
|
||||
- launcher.test.mjs:950-971 (`--no-launch`) returns before the loop (:1012-1015). It's
|
||||
unchanged.
|
||||
- stack-release.test.mjs:1928 writes `{pid: process.pid}` with no start times. `liveRun` is
|
||||
false for it, and that test reads it through bus status, not recover().
|
||||
- verbs.test.mjs:213 and the seat tests build a Registry with no launcher, so they aren't
|
||||
affected.
|
||||
|
||||
No expected value in an old test changes, so nothing goes to Mos under mos-1272 for this. If a
|
||||
run shows otherwise, I stop and send it before any change.
|
||||
- Fallback, if Duckworth or Sage doesn't accept this: revert recover() and check() to the base
|
||||
behaviour, keeping only the sessionless check over the registry, the pre-recover leftovers and
|
||||
bwrap survivors. Drop the legacy-run-live test and M10-M12, and name the untestable path here.
|
||||
- N1 (refusedOnce pins `actor_run`) changes a pre-existing test. Mos allowed it in mos-1292 (copy
|
||||
mos-1294) on mos-1285's terms, and it's committed as a81b3446e, with a closing line on the
|
||||
DEFERRED.md entry.
|
||||
|
||||
## Rollback
|
||||
|
||||
264b writes sessions.json v2. A 264a host given that file refuses to start, exit 3, "sessions file
|
||||
is malformed" (session.mjs:241 via launcher.mjs:235-237). Session workspaces and per-session stores
|
||||
stay on disk, and a 264a host never reads them.
|
||||
|
||||
What else refuses on a 264a stack while a v2 file is in place (C1, duckworth-0150). All of them go
|
||||
through the 264a readSessions, which throws SeatError "sessions file is malformed" (session.mjs:241):
|
||||
- `mosaic launches list` and `mosaic stop <run>`: exit 2 with that message (cli.mjs:526, :465).
|
||||
- the bus `limits` verb: `settings.read()` throws (runtime.mjs:149, :292, launch-settings.mjs:166),
|
||||
so the verb fails with that message. The limits the bus enforces still work, since `current()`
|
||||
doesn't read the file. I haven't traced the code a bus caller sees for that throw.
|
||||
- the Console settings view: launch shows `not-configured` (reads.mjs:234).
|
||||
- the R guard keeps working (above).
|
||||
All of these read the file on each call, so step 3 below clears them without a restart.
|
||||
|
||||
Console restart: the shape of what the Console reads doesn't change. usageCounts still counts
|
||||
entries per provider and model, and reads.mjs keeps its fields. What changes is the file, and a
|
||||
Console still on 264a code shows launch `not-configured` from the first v2 write until a human
|
||||
restarts it on a 264b copy. The re-pin that carries this row needs its Console restart (Jason or
|
||||
Mos), done right after the host's, as it does for a read-shape change. Step 3 covers the reverse
|
||||
case after a rollback with no restart.
|
||||
|
||||
Seeing live sessions without `launches list` (C1): read the file directly the way the R guard does.
|
||||
For each entry in `sessions`, a session is live if `/proc/<pid>/stat` or `/proc/<runnerPid>/stat`
|
||||
exists and its state isn't `Z` (stack-release.sh:2080, :2095). For a stronger check, compare
|
||||
`runnerStartTime` with field 22 of `/proc/<runnerPid>/stat`, which is what readSessions' `live`
|
||||
does through startTimeOf.
|
||||
|
||||
How a rollback host gets past exit 3 (sage-2717), run on NEW before the rollback activate:
|
||||
1. Stop the bus host on NEW. Its stop SIGKILLs what is left and ends those launches as `killed`
|
||||
(packages/cli/README.md:470).
|
||||
2. Read `sessions.json` with NEW's readSessions (`mosaic launches list`, cli.mjs:526-530). Go on only
|
||||
if it shows no live entry: every listed runner is gone (pid and start time no longer match). If
|
||||
NEW's CLI is already gone, use the /proc read above.
|
||||
3. Move `sessions.json` aside under a dated name, never delete it (packages/cli/README.md:478).
|
||||
4. Then roll back. The 264a host starts with no leftover file and nothing to recover.
|
||||
A live entry at step 2 stops the rollback until that run ends or `mosaic stop <run>` ends it.
|
||||
|
||||
Which re-pin plan step carries it: no re-pin plan names 264b yet. The 84f53540d re-pin carries 264a
|
||||
only. The first re-pin whose NEW includes this row puts steps 1-4 in its "Stop conditions and
|
||||
rollback" section, before `rollback --plan`, the section the 10-24 plan uses
|
||||
(agents/sage/work/window-plan/repin-1024-draft1.md:144-148). I'll name it to whoever drafts that
|
||||
plan when the row lands.
|
||||
|
||||
## Run limits (sage-2707)
|
||||
|
||||
Each node --test file and suite runs under `timeout -k 10 LIMIT`, and each scripts/test-*.sh under at
|
||||
least 3x its longest recorded run. Records are from agents/gizmoduck/work/queue-264/BUILD.md "Run
|
||||
limits" unless named.
|
||||
|
||||
| Run | Longest recorded | LIMIT |
|
||||
| --- | --- | --- |
|
||||
| stack-release.test alone | 575 s (264a r12 survey) | 1800 s |
|
||||
| each other touched file alone | 15 s (thread-launch.test); the rest have none | 300 s |
|
||||
| bus suite | 23 s | 900 s |
|
||||
| cli suite | 591 s (row 281 gate) | 1800 s |
|
||||
| webui suite | 57 s | 600 s |
|
||||
| seat suite | none recorded | 600 s |
|
||||
| harness suite | none recorded | 600 s |
|
||||
| each scripts/test-*.sh | 42 s (test-foundation) | 900 s |
|
||||
| each mutant probe | 1 s wall (264a M1-M4) | 300 s |
|
||||
@@ -478,6 +478,8 @@ at every gate. Started 2026-09-12 during the control board MVP.
|
||||
not that the event's `actor_run` is the refused run. Binds row 269 (264b,
|
||||
#1774), or the next row that changes `sessions.test.mjs`: pin `actor_run`
|
||||
in `refusedOnce`. (2026-10-22)
|
||||
Closed by row 269 (264b, #1774): `refusedOnce` selects `actor_run` and
|
||||
expects the refused run's name (mos-1292). (2026-10-22)
|
||||
|
||||
- **editor-provenance skips its rebuild check on esbuild 0.28.1.** (Duckworth
|
||||
N2 on the row 264a manifest round, duckworth-0146.) The webui test
|
||||
|
||||
+16
-13
@@ -837,7 +837,7 @@ export class Broker {
|
||||
return cap;
|
||||
}
|
||||
// Row 121 (#1615): what the launcher needs to start, stop and recover thread sessions.
|
||||
// Trusted launcher API, not a socket verb. Per thread: messages to the instance that no
|
||||
// Trusted launcher API, not a socket verb. Per session of each thread (row 269, 264b): messages to the instance that no
|
||||
// run has been delivered yet, messages delivered to the current holder and not yet read
|
||||
// (a turn in progress), and the latest message, open and claim times. A message delivered
|
||||
// to an earlier run and never read (it crashed mid-turn) isn't the holder's turn, so it
|
||||
@@ -857,30 +857,31 @@ export class Broker {
|
||||
const session = this.#runs.get(key) ?? fail('unknown-run');
|
||||
if (this.#ended.has(key)) fail('run-ended');
|
||||
// A run holds at most one session, so this also refuses another thread and a non-thread run.
|
||||
// Row 264a: the row is the thread's default session's, so a run of another session is refused too.
|
||||
const [t] = this.#threadRows('t.business=? AND t.id=?', record.business, identifier(record.thread));
|
||||
// Row 269 (264b): the row is the session the run was bound to, so a run is idled against its own.
|
||||
const [t] = this.#threadRows('t.business=? AND t.id=? AND se.id=?', record.business, identifier(record.thread), session.session);
|
||||
if (t?.holder !== record.run) fail('not-holder');
|
||||
if (t.pending || t.unread) return { released: false, pending: t.pending, unread: t.unread };
|
||||
this.#store.transaction(() => this.#claimEnd(session, this.#own(session), 'release', null));
|
||||
return { released: true };
|
||||
}
|
||||
// Row 264a (#1769): each row is the thread's default session's. Until 264b the launcher starts
|
||||
// default sessions only, so another session's messages wait.
|
||||
// Row 269 (264b, #1774): one row per session of each thread, oldest thread first, then oldest
|
||||
// session. `legacy` is a session the 3k-3l migration made (op legacy-3l), whose store and
|
||||
// in-sandbox workspace path are the thread's old ones.
|
||||
#threadRows(where, ...params) {
|
||||
return this.#store
|
||||
.all(
|
||||
`SELECT t.id, t.instance,
|
||||
(SELECT count(*) FROM messages m JOIN message_threads mt ON mt.message=m.id WHERE mt.session=ds.session AND m.to_role=t.instance
|
||||
`SELECT t.id, t.instance, se.id AS session, se.op AS session_op,
|
||||
(SELECT count(*) FROM messages m JOIN message_threads mt ON mt.message=m.id WHERE mt.session=se.id AND m.to_role=t.instance
|
||||
AND NOT EXISTS(SELECT 1 FROM deliveries d WHERE d.message=m.id AND d.op IN ('delivered','read'))) AS pending,
|
||||
(SELECT count(*) FROM messages m JOIN message_threads mt ON mt.message=m.id WHERE mt.session=ds.session AND m.to_role=t.instance
|
||||
(SELECT count(*) FROM messages m JOIN message_threads mt ON mt.message=m.id WHERE mt.session=se.id AND m.to_role=t.instance
|
||||
AND c.op='claim' AND EXISTS(SELECT 1 FROM deliveries d WHERE d.message=m.id AND d.op='delivered' AND d.holder_run=c.holder_run)
|
||||
AND NOT EXISTS(SELECT 1 FROM deliveries d WHERE d.message=m.id AND d.op='read')) AS unread,
|
||||
(SELECT max(m.at) FROM messages m JOIN message_threads mt ON mt.message=m.id WHERE mt.session=ds.session) AS last_message,
|
||||
(SELECT max(at) FROM thread_opens WHERE session=ds.session) AS last_open,
|
||||
(SELECT max(m.at) FROM messages m JOIN message_threads mt ON mt.message=m.id WHERE mt.session=se.id) AS last_message,
|
||||
(SELECT max(at) FROM thread_opens WHERE session=se.id) AS last_open,
|
||||
c.at AS claim_at, c.op AS claim_op, c.holder_run AS holder_run
|
||||
FROM threads t JOIN session_defaults ds ON ds.seq=(SELECT max(seq) FROM session_defaults WHERE thread=t.id)
|
||||
LEFT JOIN thread_claims c ON c.seq=(SELECT max(seq) FROM thread_claims WHERE session=ds.session)
|
||||
WHERE ${where} ORDER BY t.seq`,
|
||||
FROM threads t JOIN sessions se ON se.thread=t.id
|
||||
LEFT JOIN thread_claims c ON c.seq=(SELECT max(seq) FROM thread_claims WHERE session=se.id)
|
||||
WHERE ${where} ORDER BY t.seq, se.seq`,
|
||||
...params,
|
||||
)
|
||||
.map((r) => ({
|
||||
@@ -892,6 +893,8 @@ export class Broker {
|
||||
lastOpen: r.last_open,
|
||||
lastClaim: r.claim_at,
|
||||
holder: r.claim_op === 'claim' ? r.holder_run : null,
|
||||
session: r.session,
|
||||
legacy: r.session_op === 'legacy-3l',
|
||||
}));
|
||||
}
|
||||
// Trusted launcher API, deliberately NOT a socket verb. S6 supplies immutable launch records.
|
||||
|
||||
@@ -191,7 +191,7 @@ test('broker process: launch ops authorize role.launch, record refusals and end
|
||||
assert.deepEqual((await ask(child, { op: 'credentialStatus', business: 'demo', role: 'pm' })).result, []);
|
||||
// The launcher's thread view, through the same child.
|
||||
assert.deepEqual((await ask(child, { op: 'threadsState', business: 'demo' })).result, [
|
||||
{ thread: 'th-1', instance: 'coder', pending: 0, unread: 0, lastMessage: null, lastOpen: '2026-10-15T00:00:01.000Z', lastClaim: null, holder: null },
|
||||
{ thread: 'th-1', instance: 'coder', pending: 0, unread: 0, lastMessage: null, lastOpen: '2026-10-15T00:00:01.000Z', lastClaim: null, holder: null, session: 'th-1', legacy: true },
|
||||
]);
|
||||
// And its idle release (Dewey's F3, 28591): the op reaches the broker, which refuses a run
|
||||
// that doesn't hold the thread.
|
||||
|
||||
@@ -199,21 +199,21 @@ test('T2 sage-2257: a released or revoked run\'s retired generation refuses its
|
||||
// Each refusal leaves its one action.refused event (request()'s catch, #refused); nothing else is
|
||||
// written: no approval, delivery, read or claim state (mos-1277).
|
||||
const lastSeq = () => store.get('SELECT max(seq) AS s FROM events').s;
|
||||
const refusedOnce = (fn, label) => {
|
||||
const refusedOnce = (fn, label, run) => {
|
||||
const from = lastSeq();
|
||||
assert.throws(fn, { code: 'not-holder' }, label);
|
||||
const added = store.all('SELECT kind, body FROM events WHERE seq > ? ORDER BY seq', from);
|
||||
const added = store.all('SELECT kind, actor_run, body FROM events WHERE seq > ? ORDER BY seq', from);
|
||||
assert.deepEqual(
|
||||
added.map((e) => [e.kind, JSON.parse(e.body)]),
|
||||
[['action.refused', { code: 'not-holder' }]],
|
||||
added.map((e) => [e.kind, e.actor_run, JSON.parse(e.body)]),
|
||||
[['action.refused', run, { code: 'not-holder' }]],
|
||||
label,
|
||||
);
|
||||
};
|
||||
const refused = (name, { d, taken }) => {
|
||||
const { events, ...before } = counts();
|
||||
refusedOnce(() => call(name, 'decision.withdraw', { id: d.id }), 'approval');
|
||||
refusedOnce(() => call(name, 'message.receive'), 'delivery');
|
||||
refusedOnce(() => call(name, 'message.read', { id: taken.id }), 'read');
|
||||
refusedOnce(() => call(name, 'decision.withdraw', { id: d.id }), 'approval', `${name}-run`);
|
||||
refusedOnce(() => call(name, 'message.receive'), 'delivery', `${name}-run`);
|
||||
refusedOnce(() => call(name, 'message.read', { id: taken.id }), 'read', `${name}-run`);
|
||||
const { events: after, ...rest } = counts();
|
||||
assert.deepEqual(rest, before);
|
||||
assert.equal(after, events + 3);
|
||||
|
||||
+117
-50
@@ -75,6 +75,15 @@
|
||||
// threadStore()), so the next run for the thread resumes the harness
|
||||
// session (REQ-THREAD-2).
|
||||
//
|
||||
// Row 269 (264b, #1774): threadsState has one row per session of each
|
||||
// thread, and everything above is per session. Each session has its own
|
||||
// store (session.mjs sessionStore()) and its own workspace under
|
||||
// session-workspaces/ (sessionWorkspace()), which a lock beside it keeps to
|
||||
// one writer from before the clones until the run's proven exit. A legacy
|
||||
// session (op legacy-3l) keeps the thread's store, and its new workspace
|
||||
// shows inside the sandbox at the old instance path, the cwd its history
|
||||
// recorded. The broker's half of the fence is its per-session claim.
|
||||
//
|
||||
// Landing runs (row 156, #1651). Every launch of a non-thread instance
|
||||
// reads the broker's landHolders (launch op): the approved, unspent merge
|
||||
// decisions each non-thread instance raised. The instance's entries go into
|
||||
@@ -121,12 +130,14 @@
|
||||
// processing-halted.
|
||||
// - delete: instance-running against the registry, then lifecycleDelete
|
||||
// (the writer's commit), then the rename of workspaces/<b>/I/ to
|
||||
// workspaces/<b>/.retired/I-<transaction>/. A failed rename closes it
|
||||
// workspace-rename-failed with the transaction; a later delete retries it.
|
||||
// workspaces/<b>/.retired/I-<transaction>/, and of
|
||||
// session-workspaces/<b>/I/ to session-workspaces/<b>/.retired/I-<transaction>/
|
||||
// (row 269). A failed rename closes it workspace-rename-failed with the
|
||||
// transaction; a later delete retries it.
|
||||
|
||||
import { createServer } from "node:net";
|
||||
import { chmodSync, existsSync, lstatSync, mkdirSync, readFileSync, renameSync, rmSync } from "node:fs";
|
||||
import { dirname, join } from "node:path";
|
||||
import { dirname, join, relative } from "node:path";
|
||||
import { configDir, loadBusiness, resolveInstance, systemVars } from "../../business/src/index.mjs";
|
||||
import { limitHit } from "../../business/src/launch.mjs";
|
||||
import { BundleError, buildBundle, hasFallback, readSkills, sessionModel } from "../../harness/src/bundle.mjs";
|
||||
@@ -142,8 +153,10 @@ import {
|
||||
runnerOf,
|
||||
SESSION_CREDENTIALS,
|
||||
sessionEnv,
|
||||
threadStore,
|
||||
sessionExit,
|
||||
sessionStore,
|
||||
sessionWorkspace,
|
||||
lockSessionWorkspace,
|
||||
spawnSession,
|
||||
workspaceDir,
|
||||
writeSessionFile,
|
||||
@@ -237,6 +250,9 @@ export function createLauncher({
|
||||
throw new CliError(`${e.message}; check which sessions still run, then move the file aside`, 3);
|
||||
}
|
||||
const registry = new Registry(dataRoot, { readOnly: !launches });
|
||||
// Row 269: earlier-host runs still live after recover(). check() sees each while it lives.
|
||||
const survivors = [];
|
||||
const liveRun = (s) => startTimeOf(s.pid) === s.startTime || startTimeOf(s.runnerPid) === s.runnerStartTime;
|
||||
const children = new Map();
|
||||
let chain = Promise.resolve();
|
||||
let closed = false;
|
||||
@@ -278,7 +294,7 @@ export function createLauncher({
|
||||
// nothing else launches for one. Row 194 (#1692, V2): `human` is the
|
||||
// human's launch request, which may name any declared instance. The listing
|
||||
// limits the chief, not the human. launch.by still takes the `pm` branch.
|
||||
async function check(instance, { pm = false, thread = null, human = false } = {}) {
|
||||
async function check(instance, { pm = false, thread = null, session = null, human = false } = {}) {
|
||||
const admissionGeneration = admission().generation;
|
||||
try { b=loadBusiness(host.business,{dir:configDir(env),rolesDir:rolesDir(env)}); } catch(e){throw new Refusal("organization-refused",e.message);}
|
||||
if (!b.launch) throw new Refusal("launch-not-configured", `business ${b.id} has no launch block`);
|
||||
@@ -286,7 +302,12 @@ export function createLauncher({
|
||||
const threaded = b.roles[instance]?.definition === THREAD_DEFINITION;
|
||||
if (threaded && !thread) throw new Refusal("thread-required", `${instance} runs only for a thread`);
|
||||
if (!threaded && thread) throw new Refusal("thread-not-allowed", `${instance} isn't a thread instance`);
|
||||
if (registry.running(b.id, instance, thread)) throw new Refusal("instance-running");
|
||||
if (threaded && !session) throw new Refusal("session-required", `${instance} runs for one session of a thread`);
|
||||
if (registry.running(b.id, instance, thread, session)) throw new Refusal("instance-running");
|
||||
// Row 269 (Q2): a run an earlier host started with no session works in the instance's old
|
||||
// workspace and the thread's store, so no session of the instance starts while one lives.
|
||||
if (threaded && [...registry.sessions, ...leftover, ...survivors.filter(liveRun)].some((s) => s.business === b.id && s.instance === instance && !s.session))
|
||||
throw new Refusal("legacy-run-live", `a run of ${instance} with no session is still live`);
|
||||
let resolved;
|
||||
try {
|
||||
resolved = resolveInstance({ system: vars, business: b, instance });
|
||||
@@ -378,29 +399,59 @@ export function createLauncher({
|
||||
}
|
||||
|
||||
// Bundle, spawn, bind, register, hand over the capability.
|
||||
async function start({ instance, launchedBy, resolved, model, account = null, thread = null, human = false, admissionGeneration }) {
|
||||
async function start({ instance, launchedBy, resolved, model, account = null, thread = null, session = null, legacy = false, legacyThread = legacy, human = false, admissionGeneration }) {
|
||||
admission(admissionGeneration);
|
||||
// Every launch path ends here, so a --no-launch host spawns nothing whichever path asked.
|
||||
if (!launches) throw new Refusal("launches-disabled", "this host was started with --no-launch");
|
||||
// A thread's store is checked first, so a bad id or an unsafe directory
|
||||
// launches nothing.
|
||||
// A thread session's store is checked first, so a bad id or an unsafe
|
||||
// directory launches nothing.
|
||||
let store = null;
|
||||
if (thread) {
|
||||
try {
|
||||
store = threadStore(dataRoot, b.id, thread);
|
||||
store = sessionStore(dataRoot, b.id, thread, session, { legacy, legacyThread });
|
||||
} catch (e) {
|
||||
throw new Refusal("thread-store-refused", e.message);
|
||||
throw new Refusal(e.code === "legacy-store-unmapped" ? e.code : "thread-store-refused", e.message);
|
||||
}
|
||||
}
|
||||
const credentials = await host.op({ op: "credentialStatus", business: b.id, role: instance });
|
||||
admission(admissionGeneration);
|
||||
// The clones come before the run, so one that can't be made leaves no
|
||||
// run behind. The host's git treats each clone as hostile, and its
|
||||
// checks hold only while no session writes the workspace: a workspace
|
||||
// is one instance's, check() refused instance-running, and launches are
|
||||
// serialized, so none does.
|
||||
const workspace = workspaceDir(dataRoot, b.id, instance);
|
||||
mkdirSync(workspace, { recursive: true, mode: 0o700 });
|
||||
// checks hold only while no session writes the workspace. A non-thread
|
||||
// workspace is one instance's, check() refused instance-running, and
|
||||
// launches are serialized. A thread session's workspace is its own (row
|
||||
// 269), and its lock refuses a second writer, whichever host asks.
|
||||
let workspace;
|
||||
let workspaceDest = null;
|
||||
let lock = null;
|
||||
if (thread) {
|
||||
try {
|
||||
const live = registry.sessions.filter((s) => s.business === b.id && s.workspace).map((s) => s.workspace);
|
||||
workspace = sessionWorkspace(dataRoot, b.id, instance, session, { live });
|
||||
lock = lockSessionWorkspace(workspace, { hostPid: process.pid, hostStartTime: startTimeOf(process.pid), business: b.id, instance, thread, session });
|
||||
} catch (e) {
|
||||
if (["session-workspace-unsafe", "session-workspace-overlap", "session-workspace-busy"].includes(e.code)) throw new Refusal(e.code, e.message);
|
||||
throw e;
|
||||
}
|
||||
if (legacy) workspaceDest = workspaceDir(dataRoot, b.id, instance);
|
||||
} else {
|
||||
workspace = workspaceDir(dataRoot, b.id, instance);
|
||||
mkdirSync(workspace, { recursive: true, mode: 0o700 });
|
||||
}
|
||||
try {
|
||||
return await startLocked({ instance, launchedBy, resolved, model, account, thread, session, human, admissionGeneration, store, credentials, workspace, workspaceDest, lock });
|
||||
} catch (e) {
|
||||
// Every failure path in startLocked has proven the run's exit, or never spawned one.
|
||||
lock?.release();
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
// The rest of start(): clones, bundle, spawn, bind, register. The lock, when there is one, is
|
||||
// released at the run's close.
|
||||
async function startLocked({ instance, launchedBy, resolved, model, account, thread, session, human, admissionGeneration, store, credentials, workspace, workspaceDest, lock }) {
|
||||
// What the session sees: a legacy session's workspace at the old instance path.
|
||||
const inSandbox = workspaceDest ?? workspace;
|
||||
let checkouts;
|
||||
try {
|
||||
checkouts = await withProcessing(dataRoot, () => {
|
||||
@@ -416,7 +467,7 @@ export function createLauncher({
|
||||
else if (c.outcome === "fetched") record({ event: "checkout-fetched", instance, project: c.project, reason: c.reason });
|
||||
}
|
||||
let harness, route, routeReason;
|
||||
({resolved,model,account,harness,route,routeReason}=await check(instance,{pm:instance===b.launch.by,thread,human}));
|
||||
({resolved,model,account,harness,route,routeReason}=await check(instance,{pm:instance===b.launch.by,thread,session,human}));
|
||||
admission(admissionGeneration);
|
||||
const { run, runDir } = newRun(dataRoot, b.id);
|
||||
const sessionDir = store ? store.session : join(runDir, "session");
|
||||
@@ -440,7 +491,7 @@ export function createLauncher({
|
||||
arbiter: Object.values(b.arbiters).includes(instance),
|
||||
tracker,
|
||||
credentials,
|
||||
session: { run, launchedBy, workspace, toolSocket, turnMarker, checkouts },
|
||||
session: { run, launchedBy, workspace: inSandbox, toolSocket, turnMarker, checkouts: inSandbox === workspace ? checkouts : checkouts.map((c) => ({ ...c, path: join(inSandbox, relative(workspace, c.path)) })) },
|
||||
harness,
|
||||
route,
|
||||
routeReason,
|
||||
@@ -471,7 +522,7 @@ export function createLauncher({
|
||||
brokerSocket: host.path,
|
||||
launchSocket: launchSocketPath(dataRoot),
|
||||
gitSocket: git,
|
||||
workspace,
|
||||
workspace: inSandbox,
|
||||
sessionDir,
|
||||
adapter,
|
||||
bundle: files,
|
||||
@@ -484,6 +535,7 @@ export function createLauncher({
|
||||
|
||||
const paths = {
|
||||
workspace,
|
||||
workspaceDest,
|
||||
adapter,
|
||||
harnessMounts,
|
||||
brokerSocket: host.path,
|
||||
@@ -519,11 +571,12 @@ export function createLauncher({
|
||||
updateRuntime(dataRoot, managed.id, { runnerPid: runner.pid, runnerStartTime: runner.startTime });
|
||||
admission(admissionGeneration);
|
||||
if (!outerStart) throw new Error("the session process exited at once");
|
||||
const launch = await host.bindLaunch({ business: b.id, role: instance, run, harness: model.harness, provider: model.provider, model: model.model, account, transcript, pid: child.pid, startTime: outerStart, ...(thread ? { thread } : {}) });
|
||||
lock?.update({ run, runPid: child.pid, runStartTime: outerStart });
|
||||
const launch = await host.bindLaunch({ business: b.id, role: instance, run, harness: model.harness, provider: model.provider, model: model.model, account, transcript, pid: child.pid, startTime: outerStart, ...(thread ? { thread, session } : {}) });
|
||||
bound = true;
|
||||
admission(admissionGeneration);
|
||||
entry = {
|
||||
business: b.id, instance, ...(thread ? { thread } : {}), run, launchedBy, harness: model.harness, model: model.model, provider: model.provider, modelKey: model.modelKey,
|
||||
business: b.id, instance, ...(thread ? { thread, session, workspace } : {}), run, launchedBy, harness: model.harness, model: model.model, provider: model.provider, modelKey: model.modelKey,
|
||||
pid: child.pid, startTime: outerStart, runnerPid: runner.pid, runnerStartTime: runner.startTime, runDir, startedAt: new Date().toISOString(),
|
||||
};
|
||||
registry.add(entry);
|
||||
@@ -552,13 +605,15 @@ export function createLauncher({
|
||||
const ended = await endRun(run, reason, signal ? null : code);
|
||||
registry.remove(b.id, run);
|
||||
// Row 125 (#1619, REQ-LIMIT-4): a run a limit check stopped names the limit, from the claim row.
|
||||
record({ event: "end", instance, ...(thread ? { thread } : {}), run, reason, exitCode: code, signal, ...(haltGeneration ? { haltGeneration } : {}), ...(ended?.limit ? { limit: ended.limit } : {}) });
|
||||
record({ event: "end", instance, ...(thread ? { thread, session } : {}), run, reason, exitCode: code, signal, ...(haltGeneration ? { haltGeneration } : {}), ...(ended?.limit ? { limit: ended.limit } : {}) });
|
||||
log(`run ${run} (${instance}) ended: ${reason}`);
|
||||
} catch (e) {
|
||||
log(`run ${run}: ${e.message}`);
|
||||
} finally {
|
||||
try { unregisterRuntime(dataRoot, managed.id); }
|
||||
catch { log(`run ${run}: processing receipt cleanup refused`); }
|
||||
try { lock?.release(); }
|
||||
catch (e) { log(`run ${run}: workspace lock release: ${e.message}`); }
|
||||
children.delete(run);
|
||||
stopping.delete(run);
|
||||
spawnedAt.delete(run);
|
||||
@@ -566,7 +621,7 @@ export function createLauncher({
|
||||
refusedRelease.delete(run);
|
||||
}
|
||||
});
|
||||
record({ event: "launch", instance, ...(thread ? { thread } : {}), run, launchedBy, harness: model.harness, route, routeReason, model: model.model, provider: model.provider, modelKey: model.modelKey, account, pid: child.pid, runnerPid: entry.runnerPid, manifest });
|
||||
record({ event: "launch", instance, ...(thread ? { thread, session } : {}), run, launchedBy, harness: model.harness, route, routeReason, model: model.model, provider: model.provider, modelKey: model.modelKey, account, pid: child.pid, runnerPid: entry.runnerPid, manifest });
|
||||
return { run, instance, harness: model.harness, model: model.model };
|
||||
}
|
||||
|
||||
@@ -600,9 +655,12 @@ export function createLauncher({
|
||||
}
|
||||
const now = Date.now();
|
||||
const ms = (at) => (at ? Date.parse(at) : 0);
|
||||
// Row 269: a thread with a legacy session keeps its old store for that session only.
|
||||
const legacyThreads = new Set(threads.filter((t) => t.legacy).map((t) => t.thread));
|
||||
for (const t of threads) {
|
||||
if (closed) return;
|
||||
const s = registry.running(b.id, t.instance, t.thread);
|
||||
const key = `${t.thread}/${t.session}`;
|
||||
const s = registry.running(b.id, t.instance, t.thread, t.session);
|
||||
if (s) {
|
||||
const last = Math.max(ms(s.startedAt), ms(t.lastMessage), ms(t.lastOpen));
|
||||
if (!stopping.has(s.run) && now - last >= b.launch.threadIdleSeconds * 1000) {
|
||||
@@ -616,7 +674,7 @@ export function createLauncher({
|
||||
// A turn in progress, or a message since the snapshot: the run keeps going.
|
||||
if (!idle.released) continue;
|
||||
stopping.add(s.run);
|
||||
record({ event: "thread-idle", instance: t.instance, thread: t.thread, run: s.run });
|
||||
record({ event: "thread-idle", instance: t.instance, thread: t.thread, session: t.session, run: s.run });
|
||||
try {
|
||||
if (startTimeOf(s.runnerPid) === s.runnerStartTime) process.kill(s.runnerPid, "SIGTERM");
|
||||
} catch {}
|
||||
@@ -629,20 +687,21 @@ export function createLauncher({
|
||||
// the work (it died before its claim, say) waits for the next one
|
||||
// rather than start again every tick. A refused start isn't a start.
|
||||
const signature = `${t.lastMessage}|${t.lastOpen}`;
|
||||
if (startedFor.get(t.thread) === signature) continue;
|
||||
if (startedFor.get(key) === signature) continue;
|
||||
await serialize(async () => {
|
||||
if (closed || registry.running(b.id, t.instance, t.thread)) return;
|
||||
if (closed || registry.running(b.id, t.instance, t.thread, t.session)) return;
|
||||
try {
|
||||
await start({ instance: t.instance, launchedBy: b.human, thread: t.thread, ...(await check(t.instance, { thread: t.thread })) });
|
||||
startedFor.set(t.thread, signature);
|
||||
refusedThreads.delete(t.thread);
|
||||
const where = { thread: t.thread, session: t.session };
|
||||
await start({ instance: t.instance, launchedBy: b.human, ...where, legacy: t.legacy, legacyThread: legacyThreads.has(t.thread), ...(await check(t.instance, where)) });
|
||||
startedFor.set(key, signature);
|
||||
refusedThreads.delete(key);
|
||||
} catch (e) {
|
||||
const code = e instanceof Refusal ? e.code : "launch-failed";
|
||||
// Logged once per code, not once per tick.
|
||||
if (refusedThreads.get(t.thread) === code) return;
|
||||
refusedThreads.set(t.thread, code);
|
||||
log(`thread ${t.thread} (${t.instance}): start refused: ${code}`);
|
||||
record({ event: "refused", by: b.human, instance: t.instance, thread: t.thread, code, why: e.message, ...(e.limit?{limit:e.limit}:{}) });
|
||||
if (refusedThreads.get(key) === code) return;
|
||||
refusedThreads.set(key, code);
|
||||
log(`thread ${t.thread} session ${t.session} (${t.instance}): start refused: ${code}`);
|
||||
record({ event: "refused", by: b.human, instance: t.instance, thread: t.thread, session: t.session, code, why: e.message, ...(e.limit?{limit:e.limit}:{}) });
|
||||
}
|
||||
});
|
||||
}
|
||||
@@ -849,20 +908,23 @@ export function createLauncher({
|
||||
const transaction = out?.transaction ?? null;
|
||||
if (transaction === null) return { refusal: "workspace-rename-failed" };
|
||||
const bot = out.bot ? { bot: out.bot } : {};
|
||||
const from = workspaceDir(dataRoot, b.id, r.instance);
|
||||
if (!existsSync(from)) return { transaction, ...bot };
|
||||
const to = join(dirname(from), ".retired", `${r.instance}-${transaction}`);
|
||||
try {
|
||||
mkdirSync(dirname(to), { recursive: true, mode: 0o700 });
|
||||
chmodSync(dirname(to), 0o700);
|
||||
if (existsSync(to)) throw new Error(`${to} exists`);
|
||||
renameSync(from, to);
|
||||
chmodSync(to, 0o700);
|
||||
} catch (e) {
|
||||
log(`request ${r.request} (${r.instance}): workspace rename failed: ${e.message}; left at ${from}`);
|
||||
return { transaction, ...bot, refusal: "workspace-rename-failed", why: `${e.message}; left at ${from}` };
|
||||
// The old instance workspace, then (row 269) the instance's session workspaces, each moved
|
||||
// whole. A retry moves whichever is still in place.
|
||||
for (const from of [workspaceDir(dataRoot, b.id, r.instance), join(dataRoot, "session-workspaces", b.id, r.instance)]) {
|
||||
if (!existsSync(from)) continue;
|
||||
const to = join(dirname(from), ".retired", `${r.instance}-${transaction}`);
|
||||
try {
|
||||
mkdirSync(dirname(to), { recursive: true, mode: 0o700 });
|
||||
chmodSync(dirname(to), 0o700);
|
||||
if (existsSync(to)) throw new Error(`${to} exists`);
|
||||
renameSync(from, to);
|
||||
chmodSync(to, 0o700);
|
||||
} catch (e) {
|
||||
log(`request ${r.request} (${r.instance}): workspace rename failed: ${e.message}; left at ${from}`);
|
||||
return { transaction, ...bot, refusal: "workspace-rename-failed", why: `${e.message}; left at ${from}` };
|
||||
}
|
||||
record({ event: "workspace-retired", instance: r.instance, request: r.request, transaction, path: to });
|
||||
}
|
||||
record({ event: "workspace-retired", instance: r.instance, request: r.request, transaction, path: to });
|
||||
return { transaction, ...bot };
|
||||
}
|
||||
|
||||
@@ -958,10 +1020,15 @@ export function createLauncher({
|
||||
process.kill(s.pid, "SIGKILL");
|
||||
} catch {}
|
||||
const end = Date.now() + 5000;
|
||||
while (startTimeOf(s.pid) === s.startTime && Date.now() < end) await new Promise((r) => setTimeout(r, 20));
|
||||
while (liveRun(s) && Date.now() < end) await new Promise((r) => setTimeout(r, 20));
|
||||
}
|
||||
// Row 269 (Q2): the session or its runner outlived the kill, or wasn't a session to kill.
|
||||
if (liveRun(s)) {
|
||||
survivors.push(s);
|
||||
log(`run ${s.run} (${s.instance}) from an earlier host still runs (pid ${s.pid}, runner ${s.runnerPid}): no session of ${s.instance} starts until it exits`);
|
||||
}
|
||||
try {
|
||||
await host.bindLaunch({ business: b.id, role: s.instance, run: s.run, harness: s.harness, pid: s.pid, startTime: s.startTime, ...(s.thread ? { thread: s.thread } : {}) });
|
||||
await host.bindLaunch({ business: b.id, role: s.instance, run: s.run, harness: s.harness, pid: s.pid, startTime: s.startTime, ...(s.thread ? { thread: s.thread } : {}), ...(s.session ? { session: s.session } : {}) });
|
||||
} catch (e) {
|
||||
// A host that died after ending the run but before updating sessions.json.
|
||||
if (e.code === "run-ended") log(`run ${s.run} (${s.instance}) from an earlier host was already ended`);
|
||||
@@ -969,7 +1036,7 @@ export function createLauncher({
|
||||
continue;
|
||||
}
|
||||
await endRun(s.run, "host-lost", null);
|
||||
record({ event: "end", instance: s.instance, ...(s.thread ? { thread: s.thread } : {}), run: s.run, reason: "host-lost", exitCode: null, signal: null });
|
||||
record({ event: "end", instance: s.instance, ...(s.thread ? { thread: s.thread } : {}), ...(s.session ? { session: s.session } : {}), run: s.run, reason: "host-lost", exitCode: null, signal: null });
|
||||
log(`run ${s.run} (${s.instance}) from an earlier host ended: host-lost`);
|
||||
}
|
||||
leftover = [];
|
||||
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
// A stand-in harness turn for the session launch tests (row 269, #1774).
|
||||
// thread-adapter.mjs's turn, plus the run's cwd and the names in it, so a
|
||||
// test reads from the reply which workspace the session saw and at what
|
||||
// in-sandbox path. It keeps a history in $MOSAIC_SESSION_DIR, the session's
|
||||
// store.
|
||||
|
||||
import { appendFileSync, existsSync, readdirSync, readFileSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
|
||||
const request = process.env.MOSAIC_REQUEST ?? "";
|
||||
const body = request.match(/^Message [^\n]*:\n\n([\s\S]*)$/m)?.[1] ?? "";
|
||||
const historyFile = join(process.env.MOSAIC_SESSION_DIR, "history");
|
||||
const before = existsSync(historyFile) ? readFileSync(historyFile, "utf8").split("\n").filter(Boolean) : [];
|
||||
const cwd = process.cwd();
|
||||
const entries = readdirSync(cwd).sort();
|
||||
appendFileSync(historyFile, `${body}\n`);
|
||||
process.stdout.write(`THREAD ${JSON.stringify({ body, before, cwd, entries })}\n`);
|
||||
writeFileSync(process.env.MOSAIC_TURN_MARKER, "done\n");
|
||||
@@ -0,0 +1,414 @@
|
||||
// Session runs (row 269, 264b, #1774): every session of a thread runs on its
|
||||
// own, with its own store under threads/<b>/<t>/sessions/<s>/ and its own
|
||||
// workspace under session-workspaces/<b>/<instance>/<s>/, and a lock beside
|
||||
// the workspace keeps it to one writer. A legacy session (op legacy-3l) keeps
|
||||
// the thread's old store, and its workspace shows inside the sandbox at the
|
||||
// old instance path. Real runners in the bwrap sandbox; a stub adapter keeps
|
||||
// a history in the store and answers with what it found, its cwd and what
|
||||
// the cwd holds.
|
||||
//
|
||||
// The host is thread-launch.test.mjs's: startBroker on bootConfig's config,
|
||||
// with the ops the broker child forwards, and the human capability from
|
||||
// bindHuman directly.
|
||||
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { spawn, spawnSync } from "node:child_process";
|
||||
import { chmodSync, existsSync, mkdirSync, readdirSync, readFileSync, symlinkSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { launchDoc } from "../../business/tests/helpers.mjs";
|
||||
import { BusError } from "../../bus/src/broker.mjs";
|
||||
import { startBroker } from "../../bus/src/runtime.mjs";
|
||||
import { Store } from "../../bus/src/store.mjs";
|
||||
import { transition } from "../../processing-control/src/state.mjs";
|
||||
import { gitEnv } from "../../seat/src/checkout.mjs";
|
||||
import { startTimeOf } from "../../seat/src/proc.mjs";
|
||||
import { sandboxProblem } from "../../seat/src/sandbox.mjs";
|
||||
import { launchLogFile, readSessions, Registry, SESSION_CREDENTIALS } from "../../seat/src/session.mjs";
|
||||
import { bootConfig, loadSystem } from "../src/config.mjs";
|
||||
import { createLauncher } from "../src/launcher.mjs";
|
||||
import { fakeHarnesses, fixture, tmp } from "./helpers.mjs";
|
||||
// A thread made by hand gets the default session the 3k-3l migration gives a legacy thread, with
|
||||
// the thread's id (as in thread-launch.test.mjs).
|
||||
const DEFAULT_SESSION = [
|
||||
"INSERT INTO sessions(at,business,thread,id,instance,owner,name,op) SELECT at,business,id,id,instance,owner,'default','legacy-3l' FROM threads WHERE id=?",
|
||||
"INSERT INTO session_defaults(at,thread,session,by) SELECT at,id,id,'migration-3l' FROM threads WHERE id=?",
|
||||
];
|
||||
|
||||
const ADAPTER = fileURLToPath(new URL("./fixtures/session-adapter.mjs", import.meta.url));
|
||||
const skip = sandboxProblem() ?? false;
|
||||
|
||||
async function until(fn, ms = 20000) {
|
||||
const end = Date.now() + ms;
|
||||
while (Date.now() < end) {
|
||||
const v = await fn();
|
||||
if (v) return v;
|
||||
await new Promise((r) => setTimeout(r, 50));
|
||||
}
|
||||
throw new Error("timed out waiting");
|
||||
}
|
||||
// A mutant's kill is an assertion failure, not a timeout (sage-2707), so the waits a mutant could
|
||||
// stall return what they last saw and the assertions after them decide.
|
||||
async function settle(fn, ms = 20000) {
|
||||
const end = Date.now() + ms;
|
||||
let v;
|
||||
while (Date.now() < end) {
|
||||
v = await fn();
|
||||
if (v) return v;
|
||||
await new Promise((r) => setTimeout(r, 50));
|
||||
}
|
||||
return v;
|
||||
}
|
||||
|
||||
function gitIn(cwd, ...args) {
|
||||
const r = spawnSync("git", ["-c", "user.name=test", "-c", "[email protected]", "-C", cwd, ...args], { env: gitEnv(), encoding: "utf8" });
|
||||
assert.equal(r.status, 0, r.stderr);
|
||||
}
|
||||
|
||||
// Each directory made 0700, as the store and workspace checks want.
|
||||
function privateDir(...parts) {
|
||||
for (let i = 1; i <= parts.length; i++) {
|
||||
const d = join(...parts.slice(0, i));
|
||||
if (!existsSync(d)) mkdirSync(d, { mode: 0o700 });
|
||||
if (i > 1) chmodSync(d, 0o700);
|
||||
}
|
||||
return join(...parts);
|
||||
}
|
||||
|
||||
// The assistant is the thread instance; sonnet is its model's limit.
|
||||
function prepare(t, { sonnet = 3, idle = 1 } = {}) {
|
||||
const root = tmp(t, "mosaic-session-");
|
||||
const f = fixture(root, "acme", (doc) => {
|
||||
doc.roles.chief.vars = { harness: "pi", model: "claude-opus-5-5", thinking: "high" };
|
||||
doc.roles.assistant = { definition: "assistant", vars: { harness: "pi", model: "claude-sonnet-5-5", thinking: "high" } };
|
||||
doc.launch = launchDoc("chief", ["coder", "assistant"], { opus: 1, sonnet });
|
||||
doc.launch.threadIdleSeconds = idle;
|
||||
return doc;
|
||||
});
|
||||
for (const k of SESSION_CREDENTIALS.flat()) delete f.env[k];
|
||||
const project = f.doc.projects.stack.root;
|
||||
gitIn(project, "init", "-q", "-b", "main");
|
||||
writeFileSync(join(project, "file.txt"), "one\n");
|
||||
gitIn(project, "add", "file.txt");
|
||||
gitIn(project, "commit", "-q", "-m", "one");
|
||||
const adapter = join(root, "adapter.sh");
|
||||
writeFileSync(adapter, `exec '${process.execPath}' '${ADAPTER}'\n`, { mode: 0o644 });
|
||||
return { root, f, adapter };
|
||||
}
|
||||
|
||||
// A host and its launcher on the prepared root. `ops` replaces launch ops, given the runtime.
|
||||
async function start(t, { f, adapter }, { tickMs = 100, ops: replace } = {}) {
|
||||
const system = loadSystem({ env: f.env });
|
||||
const { trackers, ...boot } = bootConfig({ system, businessId: "acme", env: f.env });
|
||||
assert.equal(trackers, undefined);
|
||||
const runtime = await startBroker(boot);
|
||||
const ops = {
|
||||
identity: (m) => runtime.identity(m.cap),
|
||||
authorizeLaunch: (m) => runtime.authorizeLaunch(m.cap, m.instance),
|
||||
refuse: (m) => runtime.refuse(m.cap, m.code, m.subject),
|
||||
endLaunch: (m) => runtime.endLaunch(m.record),
|
||||
credentialStatus: (m) => runtime.credentialStatus(m.business, m.role),
|
||||
threadsState: (m) => runtime.threadsState(m.business),
|
||||
threadIdle: (m) => runtime.threadIdle(m.record),
|
||||
mayLaunch: (m) => runtime.mayLaunch(m),
|
||||
landHeld: (m) => runtime.landHeld(m.record),
|
||||
landHolders: (m) => runtime.landHolders(m.business),
|
||||
landRelease: (m) => runtime.landRelease(m.record),
|
||||
...replace?.(runtime),
|
||||
};
|
||||
const refusing = (what, fn) => async (m) => {
|
||||
try {
|
||||
return fn(m);
|
||||
} catch (e) {
|
||||
const code = e instanceof BusError ? e.code : `${what}-refused`;
|
||||
throw Object.assign(new Error(`${what} refused: ${code}`), { code });
|
||||
}
|
||||
};
|
||||
const hooks = [];
|
||||
let closing = null;
|
||||
const host = {
|
||||
business: "acme",
|
||||
path: runtime.path,
|
||||
dataRoot: f.dataRoot,
|
||||
op: refusing("launch-op", (m) => {
|
||||
if (!Object.hasOwn(ops, m.op)) throw new BusError("invalid-host-request");
|
||||
return ops[m.op](m);
|
||||
}),
|
||||
bindLaunch: refusing("launch bind", (r) => runtime.bindLaunch(r)),
|
||||
beforeClose: (fn) => void hooks.push(fn),
|
||||
};
|
||||
const logs = [];
|
||||
const launcher = createLauncher({
|
||||
host,
|
||||
system,
|
||||
env: f.env,
|
||||
makeLoader: await fakeHarnesses(t, { pi: adapter, "claude-code": adapter }),
|
||||
sessionDefaults: { pollInterval: 100 },
|
||||
harnessVersion: "0.85.1",
|
||||
readOnly: [ADAPTER],
|
||||
threadTickMs: tickMs,
|
||||
log: (l) => logs.push(l),
|
||||
});
|
||||
const close = () =>
|
||||
(closing ??= (async () => {
|
||||
for (const fn of hooks) await fn(0);
|
||||
await runtime.close();
|
||||
})());
|
||||
t.after(close);
|
||||
await launcher.listen();
|
||||
const human = runtime.broker.bindHuman({ business: "acme", human: f.doc.human, via: "cli", outsideAgent: true });
|
||||
const call = (verb, args = {}) => runtime.broker.request(human, { verb, args });
|
||||
return { host, runtime, launcher, call, logs, close };
|
||||
}
|
||||
|
||||
const launchLog = (f) =>
|
||||
existsSync(launchLogFile(f.dataRoot, "acme"))
|
||||
? readFileSync(launchLogFile(f.dataRoot, "acme"), "utf8").trim().split("\n").map((l) => JSON.parse(l))
|
||||
: [];
|
||||
const events = (f, event) => launchLog(f).filter((e) => e.event === event);
|
||||
const sessionWs = (f, session) => join(f.dataRoot, "session-workspaces", "acme", "assistant", session);
|
||||
// The refusals once there are n, or once anything launched.
|
||||
const outcome = async (f, n = 1) => {
|
||||
await settle(() => events(f, "refused").length >= n || events(f, "launch").length);
|
||||
return events(f, "refused").map((e) => [e.session, e.code]);
|
||||
};
|
||||
const historyOf = (p) => (existsSync(p) ? readFileSync(p, "utf8") : null);
|
||||
|
||||
// Send to one session of a thread; reply() waits for the assistant's RESULT reply in that session.
|
||||
function send(call, thread, session, body) {
|
||||
const sent = call("thread.send", { thread, ...(session ? { session } : {}), body });
|
||||
const reply = async () => {
|
||||
const r = await until(() => call("thread.messages", { thread, ...(session ? { session } : {}) }).find((m) => m.in_reply_to === sent.id && m.class === "RESULT"));
|
||||
assert.equal(r.from_role, "assistant");
|
||||
return JSON.parse(r.body.match(/THREAD (\{.*\})/)[1]);
|
||||
};
|
||||
return { sent, reply };
|
||||
}
|
||||
|
||||
// A thread made with thread.create, its default session and a second named one.
|
||||
function twoSessions(call) {
|
||||
const project = call("project.create", { name: "stack work", repo: "stack" }).id;
|
||||
const A = call("thread.create", { project, name: "alpha" }).id;
|
||||
const [{ id: D }] = call("session.list", { thread: A });
|
||||
const S = call("session.create", { thread: A, name: "second", op: "op-second" }).id;
|
||||
return { project, A, D, S };
|
||||
}
|
||||
|
||||
test("REQ-THREAD-2 REQ-LAUNCH-1 two sessions of one thread, and another thread's default, run at once, each with its own store and workspace; a restart of one resumes only that one", { skip, timeout: 90000 }, async (t) => {
|
||||
const prepared = prepare(t, { idle: 5 });
|
||||
const { f } = prepared;
|
||||
const host = await start(t, prepared);
|
||||
const { project, A, D, S } = twoSessions(host.call);
|
||||
const B = host.call("thread.create", { project, name: "beta" }).id;
|
||||
const [{ id: BD }] = host.call("session.list", { thread: B });
|
||||
|
||||
const toD = send(host.call, A, D, "first for D");
|
||||
const toS = send(host.call, A, S, "first for S");
|
||||
const toB = send(host.call, B, null, "first for B");
|
||||
// All three run together (M1, M2: a shared workspace would refuse the second start busy).
|
||||
await settle(() => readSessions(f.dataRoot).length === 3 || events(f, "refused").length);
|
||||
assert.deepEqual(events(f, "refused"), []);
|
||||
assert.deepEqual(readSessions(f.dataRoot).map((s) => [s.thread, s.session]).sort(), [[A, D], [A, S], [B, BD]].sort());
|
||||
const [sawD, sawS, sawB] = await Promise.all([toD.reply(), toS.reply(), toB.reply()]);
|
||||
// Each session's history is its own (M5), and each runs in its own workspace at its own path (M8).
|
||||
assert.deepEqual([sawD.body, sawD.before, sawD.cwd], ["first for D", [], sessionWs(f, D)]);
|
||||
assert.deepEqual([sawS.body, sawS.before, sawS.cwd], ["first for S", [], sessionWs(f, S)]);
|
||||
assert.deepEqual([sawB.body, sawB.before, sawB.cwd], ["first for B", [], sessionWs(f, BD)]);
|
||||
const store = (thread, session) => join(f.dataRoot, "threads", "acme", thread, "sessions", session, "session", "history");
|
||||
assert.deepEqual([historyOf(store(A, D)), historyOf(store(A, S)), historyOf(store(B, BD))], ["first for D\n", "first for S\n", "first for B\n"]);
|
||||
for (const thread of [A, B]) assert.equal(existsSync(join(f.dataRoot, "threads", "acme", thread, "session")), false);
|
||||
assert.equal(existsSync(join(f.dataRoot, "workspaces", "acme", "assistant")), false, "no thread session uses the old instance workspace");
|
||||
for (const e of events(f, "launch")) assert.ok([D, S, BD].includes(e.session), JSON.stringify(e));
|
||||
|
||||
// All three stop when idle, and each lock goes with its run.
|
||||
await until(() => events(f, "end").length === 3, 30000);
|
||||
assert.deepEqual(readSessions(f.dataRoot), []);
|
||||
for (const s of [D, S, BD]) assert.equal(existsSync(`${sessionWs(f, s)}.lock`), false, s);
|
||||
|
||||
// The named session's next message starts only that session (M9), which finds only its history.
|
||||
const again = await send(host.call, A, S, "second for S").reply();
|
||||
assert.deepEqual([again.before, again.cwd], [["first for S"], sessionWs(f, S)]);
|
||||
assert.deepEqual(events(f, "launch").slice(3).map((e) => [e.thread, e.session]), [[A, S]]);
|
||||
assert.equal(historyOf(store(A, D)), "first for D\n");
|
||||
});
|
||||
|
||||
// D9 (M7): a thread from before 3l. Its legacy session resumes from the thread's old store, and
|
||||
// its new workspace shows at the old instance path, which the transcripts recorded as the cwd.
|
||||
// The old host workspace isn't mounted.
|
||||
test("REQ-THREAD-2 REQ-LAUNCH-1 a legacy session resumes the thread's old store at the old in-sandbox cwd, in a workspace of its own", { skip, timeout: 60000 }, async (t) => {
|
||||
const prepared = prepare(t);
|
||||
const { f } = prepared;
|
||||
const store = new Store(f.dataRoot);
|
||||
const at = () => new Date().toISOString();
|
||||
store.run("INSERT INTO projects(id,at,business,name,repo,created_by) VALUES(?,?,?,?,?,?)", "p-hand", at(), "acme", "hand made", "stack", "jason");
|
||||
store.run("INSERT INTO threads(id,at,business,project,instance,owner,name) VALUES(?,?,?,?,?,?,?)", "th-legacy", at(), "acme", "p-hand", "assistant", "jason", "old");
|
||||
for (const sql of DEFAULT_SESSION) store.run(sql, "th-legacy");
|
||||
store.close();
|
||||
const old = privateDir(f.dataRoot, "threads", "acme", "th-legacy");
|
||||
writeFileSync(join(privateDir(old, "session"), "history"), "old sentinel\n");
|
||||
privateDir(old, "claude-projects");
|
||||
writeFileSync(join(privateDir(f.dataRoot, "workspaces", "acme", "assistant"), "old-file"), "the 264a workspace\n");
|
||||
|
||||
const host = await start(t, prepared);
|
||||
const saw = await send(host.call, "th-legacy", null, "after 3l").reply();
|
||||
assert.deepEqual(saw.before, ["old sentinel"]);
|
||||
assert.equal(saw.cwd, join(f.dataRoot, "workspaces", "acme", "assistant"));
|
||||
assert.equal(saw.entries.includes("old-file"), false, "the old host workspace isn't mounted");
|
||||
assert.equal(readFileSync(join(old, "session", "history"), "utf8"), "old sentinel\nafter 3l\n");
|
||||
assert.equal(existsSync(join(old, "sessions")), false);
|
||||
assert.ok(existsSync(sessionWs(f, "th-legacy")));
|
||||
assert.deepEqual(readdirSync(join(f.dataRoot, "workspaces", "acme", "assistant")), ["old-file"]);
|
||||
assert.deepEqual(events(f, "launch").map((e) => [e.thread, e.session]), [["th-legacy", "th-legacy"]]);
|
||||
});
|
||||
|
||||
// T2, M3: another host's live lock on the session's workspace refuses the start before any
|
||||
// checkout. Once that host is gone the lock is stale, and the next start takes it over.
|
||||
test("REQ-THREAD-2 REQ-LAUNCH-1 a live lock on a session workspace refuses session-workspace-busy before any checkout; a stale one is taken over", { skip, timeout: 60000 }, async (t) => {
|
||||
const prepared = prepare(t);
|
||||
const { f } = prepared;
|
||||
const host = await start(t, prepared);
|
||||
const { A, S } = twoSessions(host.call);
|
||||
const other = spawn("sleep", ["60"], { stdio: "ignore" });
|
||||
const gone = new Promise((r) => other.once("exit", r));
|
||||
t.after(() => other.kill("SIGKILL"));
|
||||
const otherStart = await until(() => startTimeOf(other.pid));
|
||||
privateDir(f.dataRoot, "session-workspaces", "acme", "assistant");
|
||||
const lock = `${sessionWs(f, S)}.lock`;
|
||||
const held = `${JSON.stringify({ hostPid: other.pid, hostStartTime: otherStart, business: "acme", instance: "assistant", thread: A, session: S })}\n`;
|
||||
writeFileSync(lock, held, { mode: 0o600 });
|
||||
|
||||
const toS = send(host.call, A, S, "for S");
|
||||
assert.deepEqual(await outcome(f), [[S, "session-workspace-busy"]]);
|
||||
assert.equal(events(f, "refused")[0].thread, A);
|
||||
assert.deepEqual(events(f, "launch"), []);
|
||||
assert.deepEqual(readSessions(f.dataRoot), []);
|
||||
assert.deepEqual(readdirSync(sessionWs(f, S)), [], "no checkout");
|
||||
assert.equal(readFileSync(lock, "utf8"), held);
|
||||
|
||||
other.kill("SIGKILL");
|
||||
await gone;
|
||||
const saw = await toS.reply();
|
||||
assert.deepEqual([saw.before, saw.cwd], [[], sessionWs(f, S)]);
|
||||
await until(() => events(f, "end").length === 1);
|
||||
assert.equal(existsSync(lock), false);
|
||||
});
|
||||
|
||||
// Q2 (BUILD.md:177-179, t1-sessions.md:109): an earlier host's sessionless run of the instance is
|
||||
// still live, so no session of it starts until that run has exited. A sleep child stands in for
|
||||
// its runner; its outer pid is gone, so recover() has nothing to kill and keeps it as a survivor.
|
||||
test("REQ-THREAD-2 REQ-LAUNCH-1 a session start refuses legacy-run-live while an earlier host's sessionless run lives, and starts once it has exited", { skip, timeout: 60000 }, async (t) => {
|
||||
const prepared = prepare(t);
|
||||
const { f } = prepared;
|
||||
const runner = spawn("sleep", ["60"], { stdio: "ignore" });
|
||||
const gone = new Promise((r) => runner.once("exit", r));
|
||||
t.after(() => runner.kill("SIGKILL"));
|
||||
const runnerStartTime = await until(() => startTimeOf(runner.pid));
|
||||
new Registry(f.dataRoot).add({ business: "acme", instance: "assistant", thread: "th-old", run: "run-left", harness: "pi", pid: 2 ** 22 + 7, startTime: "gone", runnerPid: runner.pid, runnerStartTime });
|
||||
const host = await start(t, prepared);
|
||||
assert.ok(host.logs.includes(`run run-left (assistant) from an earlier host still runs (pid ${2 ** 22 + 7}, runner ${runner.pid}): no session of assistant starts until it exits`), host.logs.join("\n"));
|
||||
const { A, S } = twoSessions(host.call);
|
||||
|
||||
const toS = send(host.call, A, S, "for S");
|
||||
assert.deepEqual(await outcome(f), [[S, "legacy-run-live"]]);
|
||||
assert.equal(events(f, "refused")[0].thread, A);
|
||||
assert.deepEqual(events(f, "launch"), [], "nothing spawned");
|
||||
assert.deepEqual(readSessions(f.dataRoot), []);
|
||||
assert.equal(existsSync(sessionWs(f, S)), false, "refused before the workspace");
|
||||
|
||||
runner.kill("SIGKILL");
|
||||
await gone;
|
||||
assert.equal(await settle(() => events(f, "launch").length), 1, "it starts once the run has exited");
|
||||
const saw = await toS.reply();
|
||||
assert.deepEqual([saw.before, saw.cwd], [[], sessionWs(f, S)]);
|
||||
await until(() => events(f, "end").length === 1);
|
||||
});
|
||||
|
||||
// T2, C3: a symlinked session root refuses session-workspace-unsafe, at the segment check.
|
||||
test("REQ-THREAD-2 REQ-LAUNCH-1 a symlinked session workspace root refuses session-workspace-unsafe and launches nothing", { skip, timeout: 60000 }, async (t) => {
|
||||
const prepared = prepare(t);
|
||||
const { f } = prepared;
|
||||
const elsewhere = privateDir(prepared.root, "elsewhere");
|
||||
symlinkSync(elsewhere, join(f.dataRoot, "session-workspaces"));
|
||||
const host = await start(t, prepared);
|
||||
const { A, D } = twoSessions(host.call);
|
||||
send(host.call, A, D, "for D");
|
||||
assert.deepEqual(await outcome(f), [[D, "session-workspace-unsafe"]]);
|
||||
assert.deepEqual(events(f, "launch"), []);
|
||||
assert.deepEqual(readdirSync(elsewhere), []);
|
||||
});
|
||||
|
||||
// T2, M4: every segment of the session path is a real directory, but workspaces/<b> is a symlink
|
||||
// to it, so only the realpath compare finds that the session workspace holds the legacy root.
|
||||
test("REQ-THREAD-2 REQ-LAUNCH-1 a session workspace that holds the legacy workspace root refuses session-workspace-overlap and launches nothing", { skip, timeout: 60000 }, async (t) => {
|
||||
const prepared = prepare(t);
|
||||
const { f } = prepared;
|
||||
const host = await start(t, prepared);
|
||||
const { A, D } = twoSessions(host.call);
|
||||
const ws = privateDir(f.dataRoot, "session-workspaces", "acme", "assistant", D);
|
||||
symlinkSync(ws, join(privateDir(f.dataRoot, "workspaces"), "acme"));
|
||||
send(host.call, A, D, "for D");
|
||||
assert.deepEqual(await outcome(f), [[D, "session-workspace-overlap"]]);
|
||||
assert.deepEqual(events(f, "launch"), []);
|
||||
assert.deepEqual(readdirSync(ws), []);
|
||||
});
|
||||
|
||||
// T2: a symlink among the store's segments refuses thread-store-refused before the workspace.
|
||||
test("REQ-THREAD-2 REQ-LAUNCH-1 a symlinked session store refuses thread-store-refused and makes no workspace", { skip, timeout: 60000 }, async (t) => {
|
||||
const prepared = prepare(t);
|
||||
const { f } = prepared;
|
||||
const host = await start(t, prepared);
|
||||
const { A, D } = twoSessions(host.call);
|
||||
const elsewhere = privateDir(prepared.root, "elsewhere");
|
||||
symlinkSync(elsewhere, join(privateDir(f.dataRoot, "threads", "acme", A), "sessions"));
|
||||
send(host.call, A, D, "for D");
|
||||
assert.deepEqual(await outcome(f), [[D, "thread-store-refused"]]);
|
||||
assert.match(events(f, "refused")[0].why, /is unsafe$/);
|
||||
assert.deepEqual(events(f, "launch"), []);
|
||||
assert.deepEqual(readdirSync(elsewhere), []);
|
||||
assert.equal(existsSync(join(f.dataRoot, "session-workspaces")), false);
|
||||
});
|
||||
|
||||
// T2, M6 and the halt fence (C2). The credentialStatus op runs in start() after check() returned.
|
||||
// Its first call halts and resumes, a new generation with no halt, so only start()'s own
|
||||
// admission(admissionGeneration) refuses, before the session workspace is made. On the retry,
|
||||
// the recheck's mayLaunch op (inside check(), after the lock is taken) halts once it has answered,
|
||||
// and the recheck's admission() refuses processing-halted. Neither attempt spawns, and the second releases its lock.
|
||||
test("REQ-THREAD-2 REQ-LAUNCH-1 a named session's start refuses processing-generation-changed, then processing-halted, after check() and before any spawn", { skip, timeout: 60000 }, async (t) => {
|
||||
const prepared = prepare(t);
|
||||
const { f } = prepared;
|
||||
let credentials = 0;
|
||||
let halts = 0;
|
||||
const halt = () => transition(f.dataRoot, { business: "acme", human: f.doc.human, halted: true });
|
||||
const host = await start(t, prepared, {
|
||||
ops: (runtime) => ({
|
||||
credentialStatus: (m) => {
|
||||
if (m.role === "assistant" && ++credentials === 1) {
|
||||
transition(f.dataRoot, { business: "acme", human: f.doc.human, halted: false, generation: halt().generation });
|
||||
halts++;
|
||||
}
|
||||
return runtime.credentialStatus(m.business, m.role);
|
||||
},
|
||||
// The halt lands after the op answers: halted inside it, the bus's own fence
|
||||
// (runtime.mjs processingWork) throws and the launcher refuses limit-check-failed.
|
||||
mayLaunch: async (m) => {
|
||||
const may = await runtime.mayLaunch(m);
|
||||
if (m.instance === "assistant" && credentials === 2 && halts === 1) {
|
||||
halt();
|
||||
halts++;
|
||||
}
|
||||
return may;
|
||||
},
|
||||
}),
|
||||
});
|
||||
const { A, S } = twoSessions(host.call);
|
||||
send(host.call, A, S, "for S");
|
||||
const refused = await outcome(f, 2);
|
||||
assert.deepEqual(refused, [[S, "processing-generation-changed"], [S, "processing-halted"]]);
|
||||
assert.deepEqual([credentials, halts], [2, 2], "check() passed both times");
|
||||
assert.equal(events(f, "launch").length, 0);
|
||||
assert.deepEqual(readSessions(f.dataRoot), []);
|
||||
assert.equal(existsSync(join(f.dataRoot, "launches", "acme")), false, "no run directory");
|
||||
assert.ok(existsSync(sessionWs(f, S)), "the retry made the workspace");
|
||||
assert.equal(existsSync(`${sessionWs(f, S)}.lock`), false, "and released its lock");
|
||||
});
|
||||
@@ -179,10 +179,11 @@ test("REQ-THREAD-2 REQ-LAUNCH-1 REQ-MSG-1 a thread message starts the thread's s
|
||||
assert.deepEqual(first.saw, { body: "first for A", before: [], transcripts: [] });
|
||||
const [run1] = launchLog(f).filter((e) => e.event === "launch" && e.thread === A).map((e) => e.run);
|
||||
assert.ok(run1);
|
||||
const store = join(f.dataRoot, "threads", "acme", A);
|
||||
const store = join(f.dataRoot, "threads", "acme", A, "sessions", host.runtime.threadsState("acme").find((x) => x.thread === A).session);
|
||||
assert.equal(readFileSync(join(store, "session", "history"), "utf8"), "first for A\n");
|
||||
assert.deepEqual(readdirSync(join(store, "claude-projects")), ["turn-1.jsonl"]);
|
||||
assert.equal(existsSync(join(f.dataRoot, "launches", "acme", run1, "session")), false, "a thread run keeps no per-run session directory");
|
||||
assert.equal(existsSync(join(f.dataRoot, "threads", "acme", A, "session")), false, "a thread.create session keeps no thread-level store");
|
||||
await idleStop(f, host, A, run1);
|
||||
|
||||
// A second run of the same thread finds the first run's history.
|
||||
|
||||
@@ -97,9 +97,12 @@ function systemBinds(node = process.execPath) {
|
||||
// the harness module's mounts (row 65: its mounts(), [{ src, dest, why }],
|
||||
// read-only), the broker socket, the launch socket (null for a role that may
|
||||
// not launch), the git socket (null for a role that neither pushes nor
|
||||
// reviews), the thread's session store (session.mjs threadStore(), or null)
|
||||
// and node. `readOnly` adds files, for tests only.
|
||||
export function binds({ runDir, workspace, adapter, harnessMounts = [], brokerSocket, launchSocket = null, gitSocket = null, threadStore = null, node = process.execPath, readOnly = [] }) {
|
||||
// reviews), the thread's session store (session.mjs threadStore() or
|
||||
// sessionStore(), or null) and node. workspaceDest (row 269) is where the
|
||||
// workspace shows inside the sandbox when that isn't its host path: a legacy
|
||||
// thread session's, so its harness finds the cwd its history recorded.
|
||||
// `readOnly` adds files, for tests only.
|
||||
export function binds({ runDir, workspace, workspaceDest = null, adapter, harnessMounts = [], brokerSocket, launchSocket = null, gitSocket = null, threadStore = null, node = process.execPath, readOnly = [] }) {
|
||||
const out = systemBinds(node);
|
||||
for (const f of RUNNER_FILES) out.push({ kind: "ro", src: f, why: "a repository file the runner loads" });
|
||||
out.push({ kind: "ro", src: adapter, why: "the harness adapter the runner starts each turn" });
|
||||
@@ -109,7 +112,7 @@ export function binds({ runDir, workspace, adapter, harnessMounts = [], brokerSo
|
||||
if (gitSocket) out.push({ kind: "ro", src: gitSocket, why: "the host's git socket, for push_working and fetch_review, and land_work" });
|
||||
for (const f of readOnly) out.push({ kind: "ro", src: f, why: "a test fixture" });
|
||||
out.push(
|
||||
{ kind: "rw", src: workspace, why: "the instance's workspace, where the harness works" },
|
||||
{ kind: "rw", src: workspace, ...(workspaceDest && workspaceDest !== workspace ? { dest: workspaceDest } : {}), why: "the instance's workspace, where the harness works" },
|
||||
{ kind: "rw", src: runDir, why: "the run directory: turns/, session/, tools.sock, turn.done, home/, tmp/" },
|
||||
{ kind: "ro", src: join(runDir, "bundle"), why: "the run's bundle stays as the host wrote it" },
|
||||
{ kind: "ro", src: join(runDir, "session.json"), why: "the session file stays as the host wrote it" },
|
||||
|
||||
+176
-17
@@ -20,17 +20,22 @@
|
||||
// every launch, refusal and end
|
||||
// workspaces/<business>/<instance>/ the instance's workspace (0700), kept
|
||||
// across launches
|
||||
// threads/<business>/<thread>/ a thread's session store (0700), kept
|
||||
// across the thread's runs (row 121):
|
||||
// threads/<business>/<thread>/ a legacy thread session's store (0700),
|
||||
// kept across its runs (row 121):
|
||||
// session/ and claude-projects/
|
||||
// threads/<business>/<thread>/sessions/<session>/
|
||||
// any other session's store (row 269)
|
||||
// session-workspaces/<business>/<instance>/<session>/
|
||||
// a thread session's workspace (0700),
|
||||
// with <session>.lock beside it
|
||||
// bus-host/sessions.json the running sessions (0600), written
|
||||
// only by the host; `mosaic stop` reads it
|
||||
|
||||
import { spawn } from "node:child_process";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { appendFileSync, closeSync, existsSync, lstatSync, mkdirSync, openSync, readFileSync, renameSync, writeFileSync } from "node:fs";
|
||||
import { appendFileSync, closeSync, existsSync, linkSync, lstatSync, mkdirSync, openSync, readdirSync, readFileSync, realpathSync, renameSync, unlinkSync, writeFileSync } from "node:fs";
|
||||
import { constants } from "node:os";
|
||||
import { dirname, join } from "node:path";
|
||||
import { dirname, join, sep } from "node:path";
|
||||
import { childrenOf, cmdlineOf, startTimeOf } from "./proc.mjs";
|
||||
import { binds, prepareRun, REPO, sandboxArgv } from "./sandbox.mjs";
|
||||
import { SeatError } from "./seat.mjs";
|
||||
@@ -38,7 +43,10 @@ import { ID_PATTERN } from "../../business/src/vocabulary.mjs";
|
||||
|
||||
export { REPO };
|
||||
export const RUNNER = join(REPO, "packages", "harness", "src", "runner.mjs");
|
||||
export const SESSIONS_VERSION = 1;
|
||||
// Row 269 (264b): 2 adds `session` and `workspace` to a thread run's entry.
|
||||
// readSessions still reads a version 1 file an earlier host left.
|
||||
export const SESSIONS_VERSION = 2;
|
||||
const SESSIONS_READ = [1, 2];
|
||||
|
||||
export const sessionsFile = (dataRoot) => join(dataRoot, "bus-host", "sessions.json");
|
||||
export const launchSocketPath = (dataRoot) => join(dataRoot, "bus-host", "launch.sock");
|
||||
@@ -47,6 +55,7 @@ export const gitSocketPath = (dataRoot) => join(dataRoot, "bus-host", "git.sock"
|
||||
export const runsDir = (dataRoot, business) => join(dataRoot, "launches", business);
|
||||
export const launchLogFile = (dataRoot, business) => join(dataRoot, "launches", `${business}.jsonl`);
|
||||
export const workspaceDir = (dataRoot, business, instance) => join(dataRoot, "workspaces", business, instance);
|
||||
export const sessionWorkspaceDir = (dataRoot, business, instance, session) => join(dataRoot, "session-workspaces", business, instance, session);
|
||||
|
||||
// Row 121 (#1615): a thread's session store, which outlives each of the
|
||||
// thread's runs so the next run resumes the harness session (REQ-THREAD-2).
|
||||
@@ -56,24 +65,174 @@ export const workspaceDir = (dataRoot, business, instance) => join(dataRoot, "wo
|
||||
// is the run's sessionDir; claude-projects/ is bound at the run's
|
||||
// home/.claude/projects (sandbox.mjs binds()). Throws SeatError.
|
||||
const segment = (x) => typeof x === "string" && ID_PATTERN.test(x) && !x.includes("..");
|
||||
export function threadStore(dataRoot, business, thread) {
|
||||
for (const [what, x] of [["business", business], ["thread", thread]]) {
|
||||
if (!segment(x)) throw new SeatError(`thread store: ${what} ${JSON.stringify(x)} isn't an id`, 2);
|
||||
const seatError = (message, code) => Object.assign(new SeatError(message, 2), { code });
|
||||
function segments(what, pairs, code) {
|
||||
for (const [name, x] of pairs) {
|
||||
if (!segment(x)) throw seatError(`${what}: ${name} ${JSON.stringify(x)} isn't an id`, code);
|
||||
}
|
||||
const dir = join(dataRoot, "threads", business, thread);
|
||||
const out = { dir, session: join(dir, "session"), projects: join(dir, "claude-projects") };
|
||||
for (const d of [join(dataRoot, "threads"), join(dataRoot, "threads", business), dir, out.session, out.projects]) {
|
||||
}
|
||||
// Makes each directory in order, then checks it is ours, a real directory and private.
|
||||
function privateDirs(what, dirs, code) {
|
||||
for (const d of dirs) {
|
||||
try {
|
||||
mkdirSync(d, { mode: 0o700 });
|
||||
} catch (e) {
|
||||
if (e.code !== "EEXIST") throw e;
|
||||
}
|
||||
const st = lstatSync(d);
|
||||
if (st.isSymbolicLink() || !st.isDirectory() || st.uid !== process.getuid() || st.mode & 0o077) throw new SeatError(`thread store: ${d} is unsafe`, 2);
|
||||
if (st.isSymbolicLink() || !st.isDirectory() || st.uid !== process.getuid() || st.mode & 0o077) throw seatError(`${what}: ${d} is unsafe`, code);
|
||||
}
|
||||
}
|
||||
export function threadStore(dataRoot, business, thread) {
|
||||
segments("thread store", [["business", business], ["thread", thread]], "thread-store-refused");
|
||||
const dir = join(dataRoot, "threads", business, thread);
|
||||
const out = { dir, session: join(dir, "session"), projects: join(dir, "claude-projects") };
|
||||
privateDirs("thread store", [join(dataRoot, "threads"), join(dataRoot, "threads", business), dir, out.session, out.projects], "thread-store-refused");
|
||||
return out;
|
||||
}
|
||||
|
||||
// Row 269 (264b, #1774): one session's store. A legacy session (op legacy-3l,
|
||||
// whose id is its thread's) keeps the thread's store, so its harness resumes
|
||||
// the history it has; any other session gets its own under sessions/, with
|
||||
// the same checks on every segment and directory. Same shape as threadStore.
|
||||
// legacyThread: the thread has a legacy session. When it has none, a
|
||||
// threads/<b>/<t>/session left by a 264a run of the thread's default session
|
||||
// refuses legacy-store-unmapped, so that history isn't silently left behind.
|
||||
export function sessionStore(dataRoot, business, thread, session, { legacy = false, legacyThread = legacy } = {}) {
|
||||
if (legacy) return threadStore(dataRoot, business, thread);
|
||||
segments("thread store", [["business", business], ["thread", thread], ["session", session]], "thread-store-refused");
|
||||
const threadDir = join(dataRoot, "threads", business, thread);
|
||||
if (!legacyThread) {
|
||||
let old = true;
|
||||
try {
|
||||
lstatSync(join(threadDir, "session"));
|
||||
} catch (e) {
|
||||
if (e.code !== "ENOENT") throw e;
|
||||
old = false;
|
||||
}
|
||||
if (old) throw seatError(`thread store: ${join(threadDir, "session")} holds history no legacy session maps`, "legacy-store-unmapped");
|
||||
}
|
||||
const dir = join(threadDir, "sessions", session);
|
||||
const out = { dir, session: join(dir, "session"), projects: join(dir, "claude-projects") };
|
||||
privateDirs("thread store", [join(dataRoot, "threads"), join(dataRoot, "threads", business), threadDir, join(threadDir, "sessions"), dir, out.session, out.projects], "thread-store-refused");
|
||||
return out;
|
||||
}
|
||||
|
||||
const inside = (a, b) => a === b || a.startsWith(b + sep);
|
||||
const realOrNull = (p) => {
|
||||
try {
|
||||
return realpathSync(p);
|
||||
} catch (e) {
|
||||
if (e.code === "ENOENT") return null;
|
||||
throw e;
|
||||
}
|
||||
};
|
||||
// Row 269: a thread session's workspace, session-workspaces/<b>/<i>/<s>. Each
|
||||
// segment is an id and each directory ours, real and private
|
||||
// (session-workspace-unsafe). Then the realpaths: it may not be inside, or
|
||||
// contain, the legacy workspaces root, its business and instance directories
|
||||
// or any entry of either (old instance, .retired and task workspaces), nor
|
||||
// sit above or below a `live` workspace (session-workspace-overlap). The same
|
||||
// path as a live one is left to the lock, which refuses session-workspace-busy.
|
||||
export function sessionWorkspace(dataRoot, business, instance, session, { live = [] } = {}) {
|
||||
segments("session workspace", [["business", business], ["instance", instance], ["session", session]], "session-workspace-unsafe");
|
||||
const root = join(dataRoot, "session-workspaces");
|
||||
const path = join(root, business, instance, session);
|
||||
privateDirs("session workspace", [root, join(root, business), join(root, business, instance), path], "session-workspace-unsafe");
|
||||
const real = realpathSync(path);
|
||||
const legacy = join(dataRoot, "workspaces");
|
||||
const named = [legacy, join(legacy, business), join(legacy, business, instance)];
|
||||
const entries = (d) => {
|
||||
try {
|
||||
return readdirSync(d).map((x) => join(d, x));
|
||||
} catch (e) {
|
||||
if (e.code === "ENOENT" || e.code === "ENOTDIR") return [];
|
||||
throw e;
|
||||
}
|
||||
};
|
||||
for (const p of [...named, ...entries(legacy), ...entries(join(legacy, business))]) {
|
||||
const other = realOrNull(p);
|
||||
if (other && (inside(real, other) || inside(other, real))) throw seatError(`session workspace: ${path} overlaps the legacy workspace ${p}`, "session-workspace-overlap");
|
||||
}
|
||||
for (const p of live) {
|
||||
const other = realOrNull(p);
|
||||
if (other && other !== real && (inside(real, other) || inside(other, real))) throw seatError(`session workspace: ${path} overlaps the live workspace ${p}`, "session-workspace-overlap");
|
||||
}
|
||||
return path;
|
||||
}
|
||||
|
||||
// Row 269: the single-writer lock on a session workspace, <workspace>.lock,
|
||||
// beside it and outside every mount. It names the host (pid, start time) and,
|
||||
// once spawned, the run's outer process. A lock whose host and run are both
|
||||
// gone is stale and is taken over; any other refuses session-workspace-busy.
|
||||
// Returns { file, update(fields), release() }.
|
||||
const lockAlive = (pid, start) => Number.isSafeInteger(pid) && pid > 0 && start != null && startTimeOf(pid) === start;
|
||||
export function lockSessionWorkspace(workspace, owner) {
|
||||
const file = `${workspace}.lock`;
|
||||
let held = { ...owner };
|
||||
const write = (flag) => writeFileSync(file, `${JSON.stringify(held)}\n`, { mode: 0o600, flag });
|
||||
const busy = (why) => seatError(`session workspace ${workspace} is busy: ${why}`, "session-workspace-busy");
|
||||
for (let attempt = 0; attempt < 2; attempt++) {
|
||||
try {
|
||||
write("wx");
|
||||
// One release per lock: a second unlink could take a lock another host has written since.
|
||||
let released = false;
|
||||
return {
|
||||
file,
|
||||
update(fields) {
|
||||
if (released) return;
|
||||
held = { ...held, ...fields };
|
||||
write("w");
|
||||
},
|
||||
release() {
|
||||
if (released) return;
|
||||
released = true;
|
||||
try {
|
||||
unlinkSync(file);
|
||||
} catch (e) {
|
||||
if (e.code !== "ENOENT") throw e;
|
||||
}
|
||||
},
|
||||
};
|
||||
} catch (e) {
|
||||
if (e.code !== "EEXIST") throw e;
|
||||
}
|
||||
let raw;
|
||||
try {
|
||||
raw = readFileSync(file, "utf8");
|
||||
} catch (e) {
|
||||
if (e.code === "ENOENT") continue;
|
||||
throw e;
|
||||
}
|
||||
let other = null;
|
||||
try {
|
||||
other = JSON.parse(raw);
|
||||
} catch {}
|
||||
// An unreadable lock may be one being written, so it counts as held.
|
||||
if (!other || typeof other !== "object") throw busy(`${file} is unreadable`);
|
||||
if (lockAlive(other.hostPid, other.hostStartTime)) throw busy(`host pid ${other.hostPid} holds it`);
|
||||
if (lockAlive(other.runPid, other.runStartTime)) throw busy(`run pid ${other.runPid} holds it`);
|
||||
// Stale. Move it aside; if what moved isn't what was judged stale, another
|
||||
// host took it over in between, so it goes back and this start refuses.
|
||||
const aside = `${file}.stale-${process.pid}-${randomBytes(4).toString("hex")}`;
|
||||
try {
|
||||
renameSync(file, aside);
|
||||
} catch (e) {
|
||||
if (e.code === "ENOENT") continue;
|
||||
throw e;
|
||||
}
|
||||
if (readFileSync(aside, "utf8") !== raw) {
|
||||
try {
|
||||
linkSync(aside, file);
|
||||
} catch {}
|
||||
unlinkSync(aside);
|
||||
throw busy(`another host took ${file} over`);
|
||||
}
|
||||
unlinkSync(aside);
|
||||
}
|
||||
throw busy(`${file} could not be taken`);
|
||||
}
|
||||
|
||||
// The session's environment is an allowlist: nothing else from the host's
|
||||
// environment passes, so a founder credential variable (GITEA_TOKEN,
|
||||
// GH_TOKEN, SSH_AUTH_SOCK, ...) never reaches a session (REQ-CRED-2).
|
||||
@@ -212,10 +371,10 @@ export class Registry {
|
||||
count(business, provider, model = undefined) {
|
||||
return this.#sessions.filter((s) => s.business === business && s.provider === provider && (model === undefined || s.model === model)).length;
|
||||
}
|
||||
// A thread instance runs once per thread (row 121), so `thread` is part of
|
||||
// the key; null is a session that holds no thread.
|
||||
running(business, instance, thread = null) {
|
||||
return this.#sessions.find((s) => s.business === business && s.instance === instance && (s.thread ?? null) === thread) ?? null;
|
||||
// A thread instance runs once per thread session (row 121, row 269), so
|
||||
// `thread` and `session` are part of the key; null is an entry without one.
|
||||
running(business, instance, thread = null, session = null) {
|
||||
return this.#sessions.find((s) => s.business === business && s.instance === instance && (s.thread ?? null) === thread && (s.session ?? null) === session) ?? null;
|
||||
}
|
||||
#write() {
|
||||
if (this.#readOnly) return;
|
||||
@@ -238,7 +397,7 @@ export function readSessions(dataRoot) {
|
||||
} catch {
|
||||
throw new SeatError(`sessions file is unreadable: ${file}`, 2);
|
||||
}
|
||||
if (doc?.sessionsVersion !== SESSIONS_VERSION || !Array.isArray(doc.sessions)) throw new SeatError(`sessions file is malformed: ${file}`, 2);
|
||||
if (!SESSIONS_READ.includes(doc?.sessionsVersion) || !Array.isArray(doc.sessions)) throw new SeatError(`sessions file is malformed: ${file}`, 2);
|
||||
return doc.sessions.map((s) => ({ ...s, live: startTimeOf(s.runnerPid) === s.runnerStartTime }));
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
// Row 269 (264b, #1774): a thread session's store, its workspace and the
|
||||
// single-writer lock beside the workspace (session.mjs sessionStore(),
|
||||
// sessionWorkspace(), lockSessionWorkspace()), and sessions.json v2.
|
||||
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { spawn } from "node:child_process";
|
||||
import { existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, realpathSync, rmSync, statSync, symlinkSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join } from "node:path";
|
||||
import { startTimeOf } from "../src/proc.mjs";
|
||||
import { SeatError } from "../src/seat.mjs";
|
||||
import { lockSessionWorkspace, readSessions, Registry, sessionsFile, sessionStore, sessionWorkspace, threadStore } from "../src/session.mjs";
|
||||
|
||||
function scratch(t) {
|
||||
const dir = realpathSync(mkdtempSync(join(tmpdir(), "mosaic-session-ws-")));
|
||||
t.after(() => rmSync(dir, { recursive: true, force: true }));
|
||||
return dir;
|
||||
}
|
||||
const mode = (p) => statSync(p).mode & 0o777;
|
||||
const refuses = (code) => (e) => e instanceof SeatError && e.code === code;
|
||||
|
||||
test("REQ-THREAD-2 sessionStore: a legacy session keeps the thread's store, any other gets sessions/<s>/, each segment checked first", (t) => {
|
||||
const root = scratch(t);
|
||||
assert.deepEqual(sessionStore(root, "acme", "th-1", "th-1", { legacy: true }), threadStore(root, "acme", "th-1"));
|
||||
const s = sessionStore(root, "acme", "th-1", "s-2", { legacyThread: true });
|
||||
const dir = join(root, "threads", "acme", "th-1", "sessions", "s-2");
|
||||
assert.deepEqual(s, { dir, session: join(dir, "session"), projects: join(dir, "claude-projects") });
|
||||
for (const d of [dirname(dir), dir, s.session, s.projects]) assert.equal(mode(d), 0o700, d);
|
||||
for (const [thread, session] of [["..", "s"], ["th-1", ".."], ["th-1", "a/b"], ["th-1", null]])
|
||||
assert.throws(() => sessionStore(root, "acme", thread, session, { legacyThread: true }), refuses("thread-store-refused"), `${thread}/${session}`);
|
||||
assert.deepEqual(readdirSync(join(root, "threads", "acme", "th-1", "sessions")), ["s-2"], "a refused call makes nothing");
|
||||
});
|
||||
|
||||
test("REQ-THREAD-2 sessionStore refuses legacy-store-unmapped for a thread whose old store no legacy session maps", (t) => {
|
||||
const root = scratch(t);
|
||||
threadStore(root, "acme", "th-1");
|
||||
assert.throws(() => sessionStore(root, "acme", "th-1", "s-1"), (e) => refuses("legacy-store-unmapped")(e) && /holds history no legacy session maps$/.test(e.message));
|
||||
assert.equal(existsSync(join(root, "threads", "acme", "th-1", "sessions")), false);
|
||||
// A thread with a legacy session: its other sessions take D2's layout.
|
||||
assert.ok(sessionStore(root, "acme", "th-1", "s-1", { legacyThread: true }).dir.endsWith(join("sessions", "s-1")));
|
||||
const elsewhere = join(root, "elsewhere");
|
||||
mkdirSync(elsewhere, { mode: 0o700 });
|
||||
symlinkSync(elsewhere, join(root, "threads", "acme", "th-2"));
|
||||
assert.throws(() => sessionStore(root, "acme", "th-2", "s-1"), (e) => refuses("thread-store-refused")(e) && /is unsafe$/.test(e.message));
|
||||
assert.deepEqual(readdirSync(elsewhere), []);
|
||||
});
|
||||
|
||||
test("REQ-THREAD-2 sessionWorkspace: private directories per session, unsafe segments and overlaps refused", (t) => {
|
||||
const root = scratch(t);
|
||||
const ws = sessionWorkspace(root, "acme", "assistant", "s-1");
|
||||
assert.equal(ws, join(root, "session-workspaces", "acme", "assistant", "s-1"));
|
||||
for (const d of [join(root, "session-workspaces"), dirname(dirname(ws)), dirname(ws), ws]) assert.equal(mode(d), 0o700, d);
|
||||
assert.equal(sessionWorkspace(root, "acme", "assistant", "s-1"), ws, "a second call reuses it");
|
||||
assert.notEqual(sessionWorkspace(root, "acme", "assistant", "s-2"), ws);
|
||||
for (const session of ["..", "a/b", ""]) assert.throws(() => sessionWorkspace(root, "acme", "assistant", session), refuses("session-workspace-unsafe"), session);
|
||||
// A live workspace at the same path is the lock's to refuse; one above or below it overlaps.
|
||||
assert.equal(sessionWorkspace(root, "acme", "assistant", "s-1", { live: [ws] }), ws);
|
||||
assert.throws(() => sessionWorkspace(root, "acme", "assistant", "s-3", { live: [dirname(ws)] }), refuses("session-workspace-overlap"));
|
||||
// The legacy root symlinked into a session path: every segment is real, only the realpath finds it.
|
||||
mkdirSync(join(root, "workspaces"), { mode: 0o700 });
|
||||
symlinkSync(ws, join(root, "workspaces", "acme"));
|
||||
assert.throws(() => sessionWorkspace(root, "acme", "assistant", "s-1"), refuses("session-workspace-overlap"));
|
||||
// A symlinked root fails the segment check first.
|
||||
const other = scratch(t);
|
||||
const elsewhere = join(other, "elsewhere");
|
||||
mkdirSync(elsewhere, { mode: 0o700 });
|
||||
symlinkSync(elsewhere, join(other, "session-workspaces"));
|
||||
assert.throws(() => sessionWorkspace(other, "acme", "assistant", "s-1"), refuses("session-workspace-unsafe"));
|
||||
assert.deepEqual(readdirSync(elsewhere), []);
|
||||
});
|
||||
|
||||
test("REQ-THREAD-2 lockSessionWorkspace: one writer; a live host or run holds it, a stale lock is taken over, an unreadable one holds", async (t) => {
|
||||
const root = scratch(t);
|
||||
const ws = sessionWorkspace(root, "acme", "assistant", "s-1");
|
||||
const me = { hostPid: process.pid, hostStartTime: startTimeOf(process.pid) };
|
||||
const lock = lockSessionWorkspace(ws, me);
|
||||
assert.equal(lock.file, `${ws}.lock`);
|
||||
assert.equal(mode(lock.file), 0o600);
|
||||
assert.throws(() => lockSessionWorkspace(ws, me), refuses("session-workspace-busy"));
|
||||
lock.update({ run: "r-1", runPid: 1, runStartTime: "x" });
|
||||
assert.deepEqual(JSON.parse(readFileSync(lock.file, "utf8")), { ...me, run: "r-1", runPid: 1, runStartTime: "x" });
|
||||
lock.release();
|
||||
assert.equal(existsSync(lock.file), false);
|
||||
|
||||
// A run that outlives its host still holds it.
|
||||
const run = spawn("sleep", ["60"], { stdio: "ignore" });
|
||||
const gone = new Promise((r) => run.once("exit", r));
|
||||
t.after(() => run.kill("SIGKILL"));
|
||||
let runStart = null;
|
||||
for (let i = 0; i < 100 && !runStart; i++) {
|
||||
runStart = startTimeOf(run.pid);
|
||||
if (!runStart) await new Promise((r) => setTimeout(r, 20));
|
||||
}
|
||||
const held = `${JSON.stringify({ hostPid: 2 ** 22 + 7, hostStartTime: "gone", runPid: run.pid, runStartTime: runStart })}\n`;
|
||||
writeFileSync(`${ws}.lock`, held, { mode: 0o600 });
|
||||
assert.throws(() => lockSessionWorkspace(ws, me), refuses("session-workspace-busy"));
|
||||
assert.equal(readFileSync(`${ws}.lock`, "utf8"), held);
|
||||
run.kill("SIGKILL");
|
||||
await gone;
|
||||
// Now stale: taken over, and nothing is left beside it.
|
||||
const taken = lockSessionWorkspace(ws, me);
|
||||
assert.deepEqual(JSON.parse(readFileSync(taken.file, "utf8")), me);
|
||||
assert.deepEqual(readdirSync(dirname(ws)).sort(), ["s-1", "s-1.lock"]);
|
||||
// One release per lock: a second doesn't remove a lock written since.
|
||||
taken.release();
|
||||
writeFileSync(`${ws}.lock`, "{}\n", { mode: 0o600 });
|
||||
taken.release();
|
||||
assert.equal(readFileSync(`${ws}.lock`, "utf8"), "{}\n");
|
||||
// An unreadable lock may be one being written, so it holds.
|
||||
writeFileSync(`${ws}.lock`, "{", { mode: 0o600 });
|
||||
assert.throws(() => lockSessionWorkspace(ws, me), refuses("session-workspace-busy"));
|
||||
});
|
||||
|
||||
test("REQ-THREAD-2 sessions.json v2: entries carry session and workspace, running() keys on the session, a v1 file still reads", (t) => {
|
||||
const root = scratch(t);
|
||||
const registry = new Registry(root);
|
||||
const entry = { business: "acme", instance: "assistant", thread: "th-1", session: "s-1", workspace: "/w", run: "r-1" };
|
||||
registry.add(entry);
|
||||
assert.equal(JSON.parse(readFileSync(sessionsFile(root), "utf8")).sessionsVersion, 2);
|
||||
assert.deepEqual(registry.running("acme", "assistant", "th-1", "s-1"), entry);
|
||||
assert.equal(registry.running("acme", "assistant", "th-1", "s-2"), null);
|
||||
assert.equal(registry.running("acme", "assistant", "th-1"), null, "an entry with a session isn't the thread's sessionless run");
|
||||
writeFileSync(sessionsFile(root), `${JSON.stringify({ sessionsVersion: 1, sessions: [{ business: "acme", instance: "assistant", thread: "th-1", run: "r-0" }] })}\n`);
|
||||
assert.deepEqual(readSessions(root).map((s) => [s.run, s.session ?? null]), [["r-0", null]]);
|
||||
});
|
||||
Reference in New Issue
Block a user