Compare commits
33
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
49cee846e3 | ||
|
|
53406f3e46 | ||
|
|
5f697db995 | ||
|
|
0f5e5d62d6 | ||
|
|
ac44d0de93 | ||
|
|
e0b905c28d | ||
|
|
6364d88690 | ||
|
|
cb98ac7a57 | ||
|
|
2b964a0827 | ||
|
|
c2b2697bc8 | ||
|
|
e9a0307700 | ||
|
|
c346f055ef | ||
|
|
076690452e | ||
|
|
e5641769a7 | ||
|
|
aabf3273a5 | ||
|
|
0e0a82051a | ||
|
|
08e33e1db4 | ||
|
|
0a74c02163 | ||
|
|
6a9a769209 | ||
|
|
87438afe05 | ||
|
|
4dfea109d6 | ||
|
|
810ec319fd | ||
|
|
ac1ea1fc02 | ||
|
|
ac9455229f | ||
|
|
d0cb42d7b9 | ||
|
|
65318f8289 | ||
|
|
092b11899c | ||
|
|
2163f8a8aa | ||
|
|
b5457a7bb7 | ||
|
|
b02db594bf | ||
|
|
a7cf7d32f0 | ||
|
|
72e797f4d1 | ||
|
|
a8da47d64a |
No files matched your search
@@ -0,0 +1,669 @@
|
||||
# Row 270 build notes (Gizmoduck)
|
||||
|
||||
264c, T1 item 7: the compaction count, exact for Pi. Gitea #1775. Brief:
|
||||
docs/plans/2026-10-21_t1-sessions.md section 4 (D6), with RULINGS Q6 (Pi exact, Claude `unknown` with
|
||||
the Vikunja 466 reason). Reviewer Duckworth, gate owner Sage. Branch gizmoduck/row270 from
|
||||
gizmoduck/row269 2d0d407ba (sage-2797, Q220). Row 269 lands first, and this branch rebases with it.
|
||||
|
||||
## Contract for 264e (pinned for Huey, row 272)
|
||||
|
||||
The surface reads the count from `session.list`. Each row gains one key, `compactions`:
|
||||
|
||||
```json
|
||||
{ "id": "...", "at": "...", "name": "...", "predecessor": null, "default": true,
|
||||
"compactions": { "count": 3, "known": true, "reason": null, "at": "2026-10-22T16:40:00.000Z" } }
|
||||
```
|
||||
|
||||
- `known: true`: `count` is the exact number of completed compactions over the session's lifetime,
|
||||
an integer of 0 or more. `reason` is null.
|
||||
- `known: false`: `count` is a lower bound (an integer) or null when there is no bound. Show
|
||||
`unknown` with the reason, never a bare 0. `reason` is one of:
|
||||
- `harness-unsupported`: Claude Code, until Vikunja 466 pins its format.
|
||||
- `not-collected`: no collection has run for this session yet.
|
||||
- `store-unreadable`: a store file couldn't be read, or isn't a Pi session.
|
||||
- `store-incomplete`: a stream file got shorter, was replaced, or has unreadable lines, so
|
||||
entries may be missing.
|
||||
- `at`: ISO time of the last collection, or null if none has run.
|
||||
- A clear successor (264d) starts at `known: true, count: 0` once its first collection runs. The
|
||||
retired session keeps its row and its count.
|
||||
- On a base without 264c, the key is absent. The surface shows the count as unavailable, which is
|
||||
what Huey's plan does (Sage Q1).
|
||||
|
||||
No new verb and no new input key. Access is `session.list`'s: the thread owner, or the tester.
|
||||
|
||||
## Pi source contract (pinned offline at 0.85.1)
|
||||
|
||||
`@earendil-works/pi-coding-agent` 0.85.1 (package.json pin), read in node_modules:
|
||||
|
||||
- One line per completed compaction: `{"type":"compaction","id":...,"parentId":...,"timestamp":...,
|
||||
"summary":...,"firstKeptEntryId":...,"tokensBefore":...}` from `appendCompaction`
|
||||
(dist/core/session-manager.js:818-833). The two callers write it after the summary exists
|
||||
(dist/core/agent-session.js:1539, :1826). A start or an aborted compaction writes no entry.
|
||||
- What doesn't count: a `message` entry with role `compactionSummary` (a rendered summary,
|
||||
docs/session-format.md:154), any message inside a compaction's `retainedTail`, and
|
||||
`branch_summary` entries.
|
||||
- Identity: a compaction is its (entry `id`, entry `timestamp`) across the whole store, not per
|
||||
stream. Pi's ids are unique only within a file (`generateId(this.byId)`, session-manager.js:21-29),
|
||||
and `createBranchedSession` (:1093-1140) and `forkFrom` (:1264-1300) copy entries into a new
|
||||
`*.jsonl` in the same directory with their id and timestamp unchanged. A (stream, id) key would
|
||||
count a compaction made before a branch or fork once per copy. Reported to Sage in
|
||||
gizmoduck-0262; accepted in sage-2814. The ledger line records the stream it was first seen in.
|
||||
- Residual (sage-2814): two distinct compactions that share an id and a timestamp count once. That
|
||||
needs an 8-hex collision between two files at the same millisecond. It is the one undercount
|
||||
this key allows, and it never overcounts.
|
||||
- Partial trailing line: wait. Only lines that end with a newline are read, as
|
||||
packages/conversation/src/pi.mjs:35-38 does.
|
||||
|
||||
## Storage: A, a host ledger (sage-2806)
|
||||
|
||||
Sage ruled A over B. A keeps the count in `<store dir>/compactions.jsonl`, a file beside the
|
||||
mounted `session/` and `claude-projects/`, in the store directory no bind mounts
|
||||
(sandbox.mjs:122-127). B would have been a bus table. The tradeoff: A needs no schema step, so 271's
|
||||
clear journal stays the only new schema step in the 264 rows, and the count lives with the native
|
||||
store it is collected from. The cost is that the bus reads a file on each `session.list`, through a
|
||||
reader runtime.mjs injects, because broker.mjs is a RUNNER_FILES file and can't import seat code.
|
||||
|
||||
### Ledger
|
||||
|
||||
`packages/seat/src/compactions.mjs`. One JSON object per line, `v: 1`:
|
||||
|
||||
- `{kind: "compaction", entry, completedAt, stream, at}`: once per (entry, completedAt) key.
|
||||
- `{kind: "cursor", stream, bytes, tail, at}`: the stream was read through `bytes`. `tail` is the
|
||||
sha256 of the up to 4096 bytes ending there. Written only when it moved.
|
||||
- `{kind: "collected", at, known, reason, harness, version}`: one per collection.
|
||||
|
||||
Conditions (sage-2806):
|
||||
|
||||
1. Append-only, the launcher its only writer, a key never written twice, nothing rewrites or
|
||||
truncates. The key is the one above, not (stream, id).
|
||||
2. Mode 0600, opened `O_WRONLY|O_APPEND|O_CREAT|O_NOFOLLOW`. **Lock used: the session's workspace
|
||||
lock** (`lockSessionWorkspace`, taken in launcher.mjs `start()` and released in the run's exit
|
||||
`finally` or on a start failure). Collection runs at run start, after `check()` gives the
|
||||
harness and before the spawn, and at run end, before the lock is released. Each collection is
|
||||
one `writeSync` loop on one O_APPEND descriptor. A collection error is logged and recorded as a
|
||||
`compactions` launch-log event and never stops or fails the run.
|
||||
3. `compactionCount(dir)`:
|
||||
|
||||
| Ledger | Answer |
|
||||
| --- | --- |
|
||||
| missing | `not-collected`, count null |
|
||||
| unreadable, not a regular file, a symlink | `store-unreadable`, count null |
|
||||
| any bad line, a torn last line included | `store-incomplete`, count a lower bound or null |
|
||||
| any collection under a harness other than pi | `harness-unsupported`, lower bound or null |
|
||||
| any collection that found a stream shrunk, changed under its cursor or gone, or a bad stream line | `store-incomplete`, for good |
|
||||
| the last collection hit an unreadable stream or a non-session first line | `store-unreadable` until a clean collection |
|
||||
| otherwise | `known: true`, the exact count |
|
||||
|
||||
Never count 0 when unknown. After a torn line, the next write starts on a fresh line, so the torn
|
||||
line stays one bad line and the new lines stay whole.
|
||||
4. `packages/seat/tests/compactions.test.mjs` "the ledger is under no sandbox mount": for a legacy
|
||||
and a non-legacy store, no `binds()` source contains the ledger path.
|
||||
5. This section.
|
||||
|
||||
### Wiring
|
||||
|
||||
- launcher.mjs: `compactions(store, model.harness, ...)` at start (after `newRun`) and in the exit
|
||||
`finally` before `lock?.release()`. Thread sessions only.
|
||||
- broker.mjs: a `compactions` constructor option, TypeError-checked like `access`. `session.list`
|
||||
selects `op` to tell a legacy session (`legacy-3l`) and adds `compactions` to each row; `op` is
|
||||
not returned. No reader gives `not-collected`; a reader that throws gives `store-unreadable`.
|
||||
No import added.
|
||||
- runtime.mjs: passes `({business, thread, session, legacy}) => compactionCount(storeDir(...))`.
|
||||
- packages/bus/README.md: the `session.list` row.
|
||||
|
||||
Huey's 272 broker hunk (reply citation predicate, near broker.mjs:5167 on huey/row272) doesn't
|
||||
touch `session.list`.
|
||||
|
||||
## Tests
|
||||
|
||||
New files only, so no pre-existing test changes:
|
||||
|
||||
- packages/seat/tests/compactions.test.mjs: zero, a failed start and non-compaction entries; one and
|
||||
three; a compaction copied into a branched stream counts once (sage-2814); one copy in two
|
||||
streams in one collection; the same id at another time counts twice; resume with cursors and
|
||||
keys written once; ledger mode 0600; a partial trailing stream line waits; a missing, a directory
|
||||
and a symlinked ledger; a torn and a bad ledger line; shrink, rewrite and remove sticky; a
|
||||
symlinked stream and a non-header first line; Claude harness-unsupported and sticky; `storeDir`
|
||||
against `sessionStore`; the mount test.
|
||||
- packages/bus/tests/session-compactions.test.mjs: the key per row with the store identity
|
||||
asked for, legacy true for `legacy-3l`, no reader and a throwing reader, the TypeError.
|
||||
|
||||
Run limits (mos-1275): each file whole with `timeout -k 10 120`, TMPDIR from
|
||||
`mktemp -d /mnt/storage/scratch/tmp/gXXX`. Then the touched files' suites: seat session.test,
|
||||
sandbox.test; cli thread-launch, session-launch; bus sessions, project-runtime, tester-writes.
|
||||
|
||||
## Mutants
|
||||
|
||||
Each must turn a new test red:
|
||||
|
||||
- M1 key per stream at write: `keyOf(f.entry, f.completedAt)` → `keyOf(f.stream + f.entry, ...)` in
|
||||
`collectCompactions`. The reader still dedupes, so the count holds; the kill is the
|
||||
branched-copy test's ledger lines (`[["c1","a.jsonl"],["c2","b.jsonl"]]`) and the two-stream
|
||||
test's one line.
|
||||
- M2 key without the timestamp: `keyOf(entry, null)` everywhere. Kill: same-id-another-time test.
|
||||
- M3 count `compactionSummary` messages: count `e.message?.role === "compactionSummary"` too. Kill:
|
||||
the zero test.
|
||||
- M4 ignore the cursor tail: `digest(tail) === c.tail` → `true`. Kill: rewrite subtest.
|
||||
- M5 torn line skipped: `bad = torn = true` → `torn = true`. Kill: torn-line test.
|
||||
- M6 no newline after a torn line: drop the `ledger.torn ? "\n"` prefix. Kill: torn-line test's bad
|
||||
lines.
|
||||
- M7 incomplete not sticky: drop `if (ledger.incomplete)`. Kill: the "later clean collection"
|
||||
assertion.
|
||||
- M8 harness not sticky: drop `if (ledger.unsupported)`. Kill: Claude then Pi test.
|
||||
- M9 0 when unknown: `count: count || null` → `count`. Kill: torn-line test's null.
|
||||
- M10 removed stream ignored: drop the missing-cursor loop. Kill: remove subtest.
|
||||
- M11 broker legacy flag: `op === 'legacy-3l'` → `false`. Kill: bus legacy test.
|
||||
- M12 broker default 0: no reader → `{count: 0, known: true, ...}`. Kill: bus no-reader test.
|
||||
|
||||
## Round 1 for review (sage-3024)
|
||||
|
||||
270's code reached the 271 branch in 09b20784f and 48c34539c and has no verdict and no mutant run.
|
||||
This round submits it on its own: the manifest is the paths 270's commits change, hashed at
|
||||
7d199f033 (the 271 r3 tip). queue-271/BUILD.md, which 8365e4e5b also touched, stays with 271.
|
||||
broker.mjs, runtime.mjs and launcher.mjs carry 271's changes too, so their blobs match the 271 round.
|
||||
|
||||
Mutants: `mutants.py` beside this file, anchors at 7d199f033, each matching once (`check`, python
|
||||
only). Each kill below is read from the test, not run yet.
|
||||
|
||||
| Mutant | Edit | Kill: first failing assertion |
|
||||
| --- | --- | --- |
|
||||
| M1 | collect key `keyOf(f.stream + f.entry, ...)` | seat compactions.test.mjs:64, ledger lines gain `[c1,b.jsonl]` (counts hold, readLedger dedupes) |
|
||||
| M2 | `keyOf` drops the timestamp | :81, count 1 not 2 |
|
||||
| M3 | count `compactionSummary` messages | :40, m2 counts 1 |
|
||||
| M4 | tail digest check `true` | :170 in `rewrite`, known true |
|
||||
| M5 | `bad = torn = true` to `torn = true` | :131, the torn line is popped unparsed, count reads known 0 |
|
||||
| M6 | no newline after a torn line | :133, c1 joins the torn line, count null not 1 |
|
||||
| M7 | drop `if (ledger.incomplete)` | :174 in `shrink` (`remove` stays incomplete by its missing cursor) |
|
||||
| M8 | drop `if (ledger.unsupported)` | :201, Pi run reads known |
|
||||
| M9 | `count \|\| null` to `count` | :131, count 0 not null |
|
||||
| M10 | drop the missing-cursor loop | :170 in `remove` |
|
||||
| M11 | broker `op === 'legacy-3l'` to `false` | bus session-compactions.test.mjs:76 |
|
||||
| M12 | broker no reader gives `{count: 0, known: true}` | bus session-compactions.test.mjs:81 |
|
||||
|
||||
Run plan, on a CLEAR only: each mutant runs its test file whole under `timeout -k 10 120` (M1-M10
|
||||
seat compactions.test.mjs, M11-M12 bus session-compactions.test.mjs), TMPDIR from
|
||||
`mktemp -d /mnt/storage/scratch/tmp/gXXX`, restore with `git checkout` and a clean-tree check after
|
||||
each. A kill is an AssertionError at the named line. M11 and M12 edit broker.mjs, which 271 also
|
||||
changed, so after their restore bus session-clear.test.mjs and sessions.test.mjs run whole, each
|
||||
under `timeout -k 10 300`. compactions.mjs has no 271 change.
|
||||
|
||||
Not covered: launcher.mjs's two `compactions(store, ...)` calls (run start, run end) and runtime.mjs's
|
||||
reader closure have no test in either file, and no test elsewhere names compactions. A mutant
|
||||
dropping either launcher call or making the closure answer a constant would survive. This is the
|
||||
T3 Pi half as built; the Tests section above never claimed the wiring. Adding a cli and a runtime
|
||||
test is a scope call for Sage.
|
||||
|
||||
Prior coverage: none. 271's M1-M14 edit no 270 line, and no past verdict names 270.
|
||||
|
||||
## Round 2 (duckworth-0223 B1-B3, sage-3030, sage-3031, mos-1352)
|
||||
|
||||
New commits on 8a5eb3c8b (round 1, on 7d199f033). Nothing here has run: no test, no mutant, no node.
|
||||
Each kill below is read from the test.
|
||||
|
||||
### B1: the read budget, a human-only setting
|
||||
|
||||
`scanPi` reads a stream a CHUNK (1 MiB) at a time from its cursor, carries a line split across
|
||||
reads, and stops when the collection's budget runs out, over all streams together. The whole-range
|
||||
`Buffer.alloc(to - from)` that compactions.mjs:57 and :101 had is gone from the stream read;
|
||||
`readRange` now serves only the tail digest reads, at most 4096 bytes. A collection the budget stops
|
||||
short records `known: false`, reason `not-collected`, and cursors at the last whole line it read. The
|
||||
reason isn't sticky: `compactionCount` takes `last.reason`, so a later collection that reads to the
|
||||
end reads known again. When a collection has more than one problem the order is store-incomplete,
|
||||
store-unreadable, then not-collected.
|
||||
|
||||
The setting (mos-1352, DECISIONS Q222). It's `compactionReadBytes`, a top-level key of the instance's
|
||||
system config, the file `MOSAIC_CONFIG` names (`~/.config/mosaic-NAME/config.json`). Under invariant 2
|
||||
a human writes that file and nothing auto-writes it. The launcher reads it once, at host start,
|
||||
through `loadSystem`; nothing reads it from the session mount, the workspace or an environment
|
||||
variable an agent's process sets. `readBudget` (compactions.mjs) takes a whole number of bytes from
|
||||
1 MiB to 1 GiB and gives the default for anything else, missing included. The launcher logs a bad
|
||||
value once at start, naming the default. A change takes effect at the next host start.
|
||||
|
||||
The schema change, for review. `scripts/mosaic-config.mjs` refused any top-level key outside
|
||||
`configVersion, environment, dataRoot, execution, console` (`unsupported configuration key`). It now
|
||||
also takes `compactionReadBytes`, and passes it on as written, a bad value included, because the
|
||||
fallback is the launcher's and a bad value must not refuse the config. The output carries the key
|
||||
only when the file has it, so a config without it prints what it always did (test-config.sh's
|
||||
"config without console" case still pins that). The old list literal stays as it was and the key is
|
||||
added by `.concat`, because packages/cli/tests/stack-release-fixture.mjs:300 edits that literal to
|
||||
make an older loader.
|
||||
|
||||
A consequence for Duckworth to rule on. A loader from before this round refuses a config that sets
|
||||
the key. scripts/stack-release.sh `config_checks` (:1079-1097) runs the target copy's loader only when
|
||||
the config has a `console` key. So a human who sets `compactionReadBytes` and then rolls back to a
|
||||
copy from before 270 passes that check, and the bus fails at its next config read. Nobody has to set
|
||||
the key, since the default needs none. I've left stack-release.sh alone: it's outside this row's
|
||||
manifest. The bounded follow-up is to make `config_checks` run the target loader whenever the config
|
||||
has any key the target might refuse, not just `console`.
|
||||
|
||||
The default, 64 MiB, from sizes measured 2026-10-22 by stat and line length only (no content printed,
|
||||
nothing under the live stack read):
|
||||
|
||||
| Store | Files | p50 | p90 | p99 | max |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| Pi sessions, `~/.pi/agent/sessions` | 224 | 298 KB | 1.1 MB | 4.7 MB | 11.6 MB |
|
||||
| Claude Code transcripts, `~/.claude/projects` (not read by the scanner) | 549 | 607 KB | 6.6 MB | 120 MB | 603 MB |
|
||||
|
||||
The longest Pi line is 0.84 MB, and no Pi line is over 1 MiB. 64 MiB is more than five times the
|
||||
largest Pi stream, so a session's first collection reads it whole, and later ones read only what
|
||||
was appended. Memory per collection is one 1 MiB chunk plus the carry of one unfinished line, and
|
||||
the budget bounds both. The Claude numbers show how far a long agent session can grow. A Pi session
|
||||
that size would take several collections to read through, each one reading unknown until the last.
|
||||
I haven't measured how long a 64 MiB collection takes; the sparse test reads 64 MiB of a hole, and its
|
||||
wall time in the slot run will be the first number.
|
||||
|
||||
Known limits:
|
||||
- A single line longer than the budget is never read past. The count stays not-collected until a
|
||||
human raises the setting. The longest Pi line measured is 0.84 MB.
|
||||
- `readLedger` still reads the ledger whole; B1 named the stream reads. Ledger lines are 120 bytes
|
||||
(collected), 226 (cursor) and 199 (compaction), so a run with two collections, one changed stream
|
||||
and one compaction adds about 0.9 KB. One session reaches 64 MiB of ledger after about 75,000 runs.
|
||||
|
||||
### B3: Claude history from before the ledger
|
||||
|
||||
When a Pi collection finds no ledger and the store's `claude-projects/` (`store.projects`) isn't
|
||||
empty, it skips the scan and writes one collected record: `known: false`, reason
|
||||
`harness-unsupported`, harness `pi`. `readLedger` now sets `unsupported` for a record whose harness
|
||||
isn't `pi` or whose reason is `harness-unsupported`, so the count stays unknown for good, as after a
|
||||
Claude run. A `claude-projects/` that can't be read throws, so nothing is recorded and the next
|
||||
collection checks again. An empty one is no history. Once a ledger exists the check doesn't run: a
|
||||
later Claude run collects for itself.
|
||||
|
||||
A Pi run mounts `claude-projects/` read-write (sandbox.mjs:125). The run-start collection comes before
|
||||
the agent's first turn, though, so an agent can't put a file there before a fresh session's first
|
||||
collection. The exception is a run whose start collection failed. Even then, the agent can only
|
||||
make the count unknown, never wrong.
|
||||
|
||||
The Vikunja 466 reason. For a Claude run, seat compactions.test.mjs:199 asserts `harness-unsupported`,
|
||||
and :201 asserts that it stays after a Pi run. For history from before the ledger,
|
||||
compactions-budget.test.mjs:88 and :89 assert it (sage-3030 item 2).
|
||||
|
||||
### B2: the run wiring, the reader and a clear
|
||||
|
||||
The new tests live in files 271 doesn't touch:
|
||||
- `packages/seat/tests/compactions-budget.test.mjs`, B1 and B3.
|
||||
- `packages/cli/tests/compactions-launch.test.mjs`, with its stub adapter
|
||||
`packages/cli/tests/fixtures/compaction-adapter.mjs`, real runners in bwrap.
|
||||
|
||||
compactions-launch tests:
|
||||
1. A run's start and end collections, across a resume and two host restarts. session.list reads the
|
||||
ledger through the runtime's reader (:166-198).
|
||||
2. A collection that fails, with the ledger's path made a directory. Each failure is logged as
|
||||
`run X: compactions: ...` and recorded as `store-unreadable`. The run answers, and it ends with the
|
||||
same reason, exit code and signal as the next run, whose collections count (:200-224).
|
||||
3. The setting. At 1 MiB, a 2 MiB hole is not-collected at both collections. With `"x"`, the host
|
||||
logs the fallback once, and the default reads the hole through (:226-261).
|
||||
4. A clear. The retired session keeps its count, 1, and the successor reads `{count: 0, known: true}`
|
||||
after its first collection. A legacy session (op legacy-3l, seeded with the SQL from
|
||||
bus session-compactions.test.mjs:72-74) reads the thread store's ledger through the runtime's reader
|
||||
after a restart (:263-308).
|
||||
|
||||
### Mutants M13-M21
|
||||
|
||||
`mutants.py`, with all 21 anchors matching once at this round's tip (`check`, python only).
|
||||
|
||||
| Mutant | Edit | Kill: first failing assertion |
|
||||
| --- | --- | --- |
|
||||
| M13 | drop the run-start `compactions(...)` call (launcher.mjs:501) | cli compactions-launch.test.mjs:182, run 1 logs only `[D,2,true,null]` |
|
||||
| M14 | drop the run-end call (launcher.mjs:642) | :182, run 1 logs only `[D,1,true,null]` |
|
||||
| M15 | the runtime's reader returns `null` (runtime.mjs:268) | :197, compactions null; :286 and :306 fail the same way |
|
||||
| M16 | no budget, `stop = size` | seat compactions-budget.test.mjs:48, the hole is read whole: known true |
|
||||
| M17 | `readBudget` returns `value ?? READ_BYTES` | :37, `"8388608"` comes back as the string |
|
||||
| M18 | the launcher ignores the setting, `budget = READ_BYTES` | cli :248, the 2 MiB hole reads known twice |
|
||||
| M19 | no carry, `take(view.subarray(i, nl))` | seat :79, c1's halves fail to parse: store-incomplete |
|
||||
| M20 | no Claude history check | :88, c1 counts known |
|
||||
| M21 | `harness-unsupported` not sticky on a `pi` record | :89, the second collection reads known |
|
||||
|
||||
The files they touch, sha256 at this round's tip, to check each restore against (sage-3030 item 3):
|
||||
|
||||
```
|
||||
45a3744598cfb8aa1cfcffd8a0ee2dd2bb631061efd192d8461f1901141993ee packages/seat/src/compactions.mjs
|
||||
d7cdc8a17e337e00dfe2d1bf6249cc2c2e696b16c5b426b2fe616089e5559509 packages/bus/src/broker.mjs
|
||||
5d36cc0fa19ab8e478ecf9a807e3afab8181cb650778e80b13efe785e6c44995 packages/cli/src/launcher.mjs
|
||||
5ae423438a178b1c8eac3b8ce0e0c22f5bb7caebb9011ce7098a36d95b53f1ca packages/bus/src/runtime.mjs
|
||||
```
|
||||
|
||||
broker.mjs and runtime.mjs are byte-identical to 7d199f033.
|
||||
|
||||
### Run plan, on a CLEAR only
|
||||
|
||||
Every run uses TMPDIR from `mktemp -d /mnt/storage/scratch/tmp/gXXX` and `timeout -k 10`. A first-run
|
||||
red in a new test is placed and reported after the run (mos-1272). A red in an old test, a product
|
||||
fix or a second red goes to Sage and Mos first.
|
||||
|
||||
1. Each mutant runs its kill file whole: M1-M10 seat compactions.test.mjs, M16, M17 and M19-M21 seat
|
||||
compactions-budget.test.mjs (120 s each), M11-M12 bus session-compactions.test.mjs (120 s), and
|
||||
M13-M15 and M18 cli compactions-launch.test.mjs (600 s). After each one: `git checkout` the file,
|
||||
check its sha256 against the table, and confirm the tree is clean. A byte-identical restore needs
|
||||
no other rerun.
|
||||
2. The new test files, whole: seat compactions-budget.test.mjs and cli compactions-launch.test.mjs.
|
||||
3. Every file that reads a path this round touches, whole:
|
||||
- compactions.mjs: seat compactions.test.mjs, bus session-compactions.test.mjs.
|
||||
- mosaic-config.mjs: scripts/test-config.sh, cli stack-instances.test.mjs, cli stack-release.test.mjs
|
||||
(through stack-release-fixture.mjs), webui pinned-copy.test.mjs, and the business suite (it
|
||||
writes configs through helpers.mjs:79).
|
||||
- launcher.mjs: the cli files that import it, all in the cli suite.
|
||||
- README.md: no test reads it.
|
||||
4. The gate (sage-3031) runs on the combined 270+271 tip in one slot after Gander's 276 run5. It
|
||||
covers bus, seat and cli whole, the shell suites, M1-M21 and 271's mutants. compactions.mjs is in
|
||||
271's manifest, so 271's earlier gate evidence is stale. Webui pinned-copy and the business suite
|
||||
join it because the loader changed.
|
||||
|
||||
### Known residuals (duckworth-0223, sage-3031)
|
||||
|
||||
- N1. A session.list read can briefly see a half-written line. The bus reads without the workspace
|
||||
lock, so a read during an append sees a torn last line. That call alone reports store-incomplete,
|
||||
and nothing is persisted. Accepted.
|
||||
- N2 (README). A launcher killed mid-write leaves a torn ledger line, and the count stays unknown
|
||||
(store-incomplete) for good, by design (sage-2806 condition 3). This is now a line in the bus
|
||||
README's `session.list` row.
|
||||
- N3. Pi 0.85.1 rewrites a session file only when it's empty (session-manager.js:625-632), on a
|
||||
version migration (:675-677), or for a new branched file (:1173). None of these moves bytes under a
|
||||
0.85.1 cursor, so the 4096-byte tail check is enough. A non-Pi writer that rewrites earlier bytes and
|
||||
keeps the tail goes unseen. That residual is acceptable.
|
||||
|
||||
## Round 3 (duckworth-0226 B4, sage-3038)
|
||||
|
||||
Round 2 at 62a279cf3 got CHANGES on B4. B1-B3 were met, and Duckworth accepted the schema change.
|
||||
|
||||
### B4: a cap on the streams one collection opens
|
||||
|
||||
Before this round, one collection opened every `*.jsonl` in the session directory. It also checked
|
||||
each ledger cursor with `names.includes`, which is O(cursors × names), and wrote a cursor line for
|
||||
every stream, empty ones included. The agent can write its rw session directory (sandbox.mjs:124),
|
||||
and nothing limits its file count. So 100,000 empty files meant about 22 MB of cursor lines at one
|
||||
run end, then about 10^10 string compares on the launcher's loop at the next.
|
||||
|
||||
Now:
|
||||
|
||||
- scanPi (compactions.mjs:123-125) sorts the `*.jsonl` names and opens at most `streams` of them.
|
||||
When there are more, the collection adds not-collected. The streams it didn't open get no cursor
|
||||
line. A stream that already has a cursor keeps its old one.
|
||||
- The missing-stream check uses a Set (compactions.mjs:121-122).
|
||||
- The cap is a setting, per sage-3038 (the same reason as mos-1352 and Q222): top-level
|
||||
`compactionStreams` in the instance system config. It follows compactionReadBytes. It is
|
||||
human-only. `readStreams()` accepts a safe integer from 32 to 65536 and returns the default, 1024,
|
||||
for anything else. The launcher reads it once and logs one `system config:` line for a bad value.
|
||||
mosaic-config.mjs adds the key to the same `.concat` and passes it on verbatim. The fixture literal
|
||||
at stack-release-fixture.mjs:300 still matches.
|
||||
- Name order stays sorted, so a collection under a larger cap, or with fewer files, catches up.
|
||||
Under the same cap and the same files, each collection opens the same streams and adds no cursor
|
||||
lines for streams that haven't moved.
|
||||
|
||||
Default, from stat only (`find ~/.pi/agent/sessions -name '*.jsonl'`, counted per directory, no
|
||||
content read). Each of these directories holds every session for one cwd, so a single session's
|
||||
count is at most these:
|
||||
|
||||
| directories | p50 | p90 | p99 | max |
|
||||
|---|---|---|---|---|
|
||||
| 40 | 3 | 19 | 27 | 27 |
|
||||
|
||||
1024 is about 38 times the max. The cursor lines one collection writes are bounded by 1024 × 226 B,
|
||||
about 231 KB. At the setting's 65536 maximum the bound is about 14.8 MB.
|
||||
|
||||
### N4: docs/TOOLS.md
|
||||
|
||||
docs/TOOLS.md has a paragraph on both keys, after the `config-console-key` paragraph. It gives the
|
||||
range, the default, that only a human writes them, the fallback with its log line, not-collected,
|
||||
and C1. No test reads TOOLS.md. stack-release.test.mjs:78 only names it inside a message string.
|
||||
|
||||
### C1 (duckworth-0226, sage-3038)
|
||||
|
||||
Any re-pin or rollback plan whose OLD copy is before row 270 removes both `compactionReadBytes` and
|
||||
`compactionStreams` from config.json first. A pre-270 loader refuses either key. stack-release.sh
|
||||
config_checks (:1079-1097) runs the target loader only when there is a `console` key, so the release
|
||||
doesn't catch this. Sage is drafting the config_checks follow-up row.
|
||||
|
||||
### Tests
|
||||
|
||||
- seat compactions-budget.test.mjs:114-120: readStreams over good and bad values.
|
||||
- :122-134: four streams under a cap of 3. Both collections are not-collected, and the cursors are
|
||||
only a, b and c, with no new cursor lines on the second. A cap of 4 then reads count 4, known.
|
||||
- :136-143: 1025 empty streams under the default cap. The result is not-collected with 1024 cursor
|
||||
lines, the last one e1023.
|
||||
- cli compactions-launch.test.mjs:264-298: compactionStreams 32 with 34 streams. pi.jsonl sorts
|
||||
last, so its c0 isn't opened and both collections are `[D,null,false,not-collected]`. Then `"x"`
|
||||
logs the fallback, and the next run's collections read `[D,1,true,null]`.
|
||||
- scripts/test-config.sh: 2048 and `"x"` load and pass through, as for compactionReadBytes.
|
||||
|
||||
### Mutants
|
||||
|
||||
M10's anchor moved to the Set line. The mutant is the same, removing the missing-stream check. Every
|
||||
anchor in mutants.py matches once (`check`). Round 3 shifted round 2's kill lines: seat budget
|
||||
:37/:48/:79/:88/:89 are now :39/:50/:81/:90/:91, and cli launch :182/:197/:248 are now
|
||||
:183/:198/:249. mutants.py K has the current lines. The launcher call sites are now
|
||||
launcher.mjs:507 and :648.
|
||||
|
||||
| id | change | kill |
|
||||
|---|---|---|
|
||||
| M22 | no cap, `for (const name of visit)` | seat compactions-budget.test.mjs:127, all four are opened: count 4 known; :141 fails the same way |
|
||||
| M23 | `readStreams` returns `value ?? STREAMS` | :118, `"2048"` comes back as the string |
|
||||
| M24 | the launcher ignores the setting, `streams = STREAMS` | cli compactions-launch.test.mjs:285, all 34 open: `[D,1,true,null]` twice |
|
||||
|
||||
sha256 at this round's tip, to check each restore against. These replace round 2's table:
|
||||
|
||||
```
|
||||
b5a078a096e447c4afe15812696d1f2f8c3f2b7665868c02e61a4e1edc8cd874 packages/seat/src/compactions.mjs
|
||||
d7cdc8a17e337e00dfe2d1bf6249cc2c2e696b16c5b426b2fe616089e5559509 packages/bus/src/broker.mjs
|
||||
21a7e910d81848aeb99036028d5b87354a2a515f3642779105c881b43a8b6e05 packages/cli/src/launcher.mjs
|
||||
5ae423438a178b1c8eac3b8ce0e0c22f5bb7caebb9011ce7098a36d95b53f1ca packages/bus/src/runtime.mjs
|
||||
```
|
||||
|
||||
### Run plan
|
||||
|
||||
This is round 2's plan with M1-M24. M22 and M23 run seat compactions-budget.test.mjs (120 s). M24
|
||||
runs cli compactions-launch.test.mjs (600 s). Per sage-3038, the slot goes to Megavolt's 207, then
|
||||
203, then this row's combined 270+271 gate after Duckworth approves r3.
|
||||
|
||||
### Residuals
|
||||
|
||||
- N5. readdirSync still lists the whole session directory, and the names are sorted before the
|
||||
cap. Both are O(files) in memory and time on the launcher. The cap bounds opens, reads and ledger
|
||||
lines, not the listing. Bounding the listing needs opendirSync with a stop. That would lose name
|
||||
order, and name order is what lets a later collection catch up. Accepted for this row
|
||||
(duckworth-0226).
|
||||
- The ledger's cursor map still grows when the agent deletes and recreates streams, by at most the
|
||||
cap per collection. Each deletion of a stream with a cursor is store-incomplete for good. This is
|
||||
the known limit that the bus reads the ledger whole (round 2).
|
||||
|
||||
## Round 4: rebase onto row 203 (sage-3073)
|
||||
|
||||
Rows 270 and 271 are rebased onto origin/refactor 25c09f19f (sage-3078): row 203 (FED R2) at
|
||||
a4da4e6df, plus three docs-only commits for the O2 design (BUILD-LOG.md, docs/SESSIONS.md and
|
||||
docs/plans/2026-10-21_q195-writing-editor.md, none of them in either manifest). The new branch is
|
||||
gizmoduck/row270-271-r4. gizmoduck/row271-r2 stays at b9da7697f. The old
|
||||
merge-base was 0cbe2e9b3, and the rebase moved 28 commits.
|
||||
|
||||
`git range-diff 0cbe2e9b3..b9da7697f 25c09f19f..HEAD~1` shows 26 commits
|
||||
unchanged. Round 2 and round 3 differ, and only in scripts/mosaic-config.mjs:
|
||||
|
||||
- The allowlist keeps 203's literal and adds both keys after it:
|
||||
`["configVersion", "environment", "dataRoot", "execution", "console", "federation"].concat(["compactionReadBytes", "compactionStreams"])`.
|
||||
203 changed packages/cli/tests/stack-release-fixture.mjs, and its LOADER_OLD replace is now at
|
||||
:301 (rounds 2 and 3 say :300). It matches the literal with `"federation"`, which is still
|
||||
intact before the `.concat`.
|
||||
- The output has 203's `federation` line, then the two `Object.hasOwn` lines.
|
||||
|
||||
Rows 203, 270 and 271 also all touch scripts/test-config.sh, scripts/stack-release.sh and
|
||||
packages/cli/tests/stack-release.test.mjs. Those merged without conflict, and the hunks are far
|
||||
apart:
|
||||
|
||||
- test-config.sh: 203's federation cases start at :194. Mine are at :266-:281, after the
|
||||
no-console case.
|
||||
- stack-release.sh: 203 changed config_checks at :1078-:1097. Row 271's change is COUNTED at
|
||||
:420, still the line 271's BUILD.md cites.
|
||||
- stack-release.test.mjs: 203 changed :1468 and :1582. Row 271's LISTED is still at :861 and T46
|
||||
at :913.
|
||||
|
||||
203 widens config_checks to run the target loader for a `federation` key as well as `console`.
|
||||
It still does not run the loader for compactionReadBytes or compactionStreams, so C1 stands as
|
||||
written.
|
||||
|
||||
`mutants.py check` passes: every anchor matches once. The restore hashes in round 3's table are
|
||||
unchanged, because 203 touches none of those four files. No test pins a sha256 of mosaic-config.mjs,
|
||||
test-config.sh, stack-release.sh or stack-release.test.mjs.
|
||||
|
||||
Nothing has run on this base yet. The combined gate is sage-3073 item 4, on Sage's CLEAR.
|
||||
|
||||
## Combined 270+271 gate (C2), written before the first run (sage-3073 item 4, sage-3086, mos-1351)
|
||||
|
||||
The gate runs on the commit that adds this section. Its code is ac44d0de9's: `git diff --name-only
|
||||
ac44d0de9 HEAD` is this file only. The review keys stay manifest-270-r4 4f5e7041… and manifest-271-r4
|
||||
3ab1f5c8…, Duckworth approves at revs 3379 and 3380. Worktree /mnt/storage/scratch/gizmo-270, with
|
||||
node_modules linked to the canonical checkout. Base 25c09f19f. Script
|
||||
/mnt/storage/scratch/gizmo-271-out/run-r4.sh, run with pipefail and tee, logs in
|
||||
/mnt/storage/scratch/gizmo-271-out/r4/. TMPDIR comes from `mktemp -d /mnt/storage/scratch/tmp/gXXX`.
|
||||
|
||||
### Blob table
|
||||
|
||||
sha256 of every source file a mutant edits, at the gate commit. Each restore is `git checkout --` the
|
||||
file, then this hash, then a clean tree.
|
||||
|
||||
```
|
||||
b5a078a096e447c4afe15812696d1f2f8c3f2b7665868c02e61a4e1edc8cd874 packages/seat/src/compactions.mjs (270)
|
||||
d7cdc8a17e337e00dfe2d1bf6249cc2c2e696b16c5b426b2fe616089e5559509 packages/bus/src/broker.mjs (270, 271)
|
||||
21a7e910d81848aeb99036028d5b87354a2a515f3642779105c881b43a8b6e05 packages/cli/src/launcher.mjs (270, 271)
|
||||
5ae423438a178b1c8eac3b8ce0e0c22f5bb7caebb9011ce7098a36d95b53f1ca packages/bus/src/runtime.mjs (270)
|
||||
```
|
||||
|
||||
### Timeout table (mos-1275)
|
||||
|
||||
Every run is `timeout -k 10 LIMIT`. The longest records are 271's gate on e8af389e6 (BUILD.md queue-271
|
||||
"Gate on e8af389e6") and 271's "Run limits" table.
|
||||
|
||||
| Run | Longest recorded | LIMIT |
|
||||
| --- | --- | --- |
|
||||
| stack-release.test alone | 575 s | 1800 s |
|
||||
| each other touched test file alone, stack-instances.test, webui pinned-copy.test | 15 s | 300 s |
|
||||
| business suite | none recorded | 900 s |
|
||||
| bus suite | 24 s | 900 s |
|
||||
| seat suite | 6 s | 600 s |
|
||||
| cli suite | 591 s | 1800 s |
|
||||
| each scripts/test-*.sh | 42 s (test-foundation; 39 s at e8af389e6) | 900 s |
|
||||
| 270 mutant, seat or bus kill file | none recorded | 120 s |
|
||||
| 270 mutant, cli compactions-launch.test | none recorded | 600 s |
|
||||
| 271 mutant, each session-clear.test file | 57 s for both files (e8af389e6) | 300 s |
|
||||
|
||||
A shell-phase timeout goes to Mos unplaced. A timeout with no failed test is a hang and goes to Sage.
|
||||
|
||||
### GREEN and RED
|
||||
|
||||
- GREEN for a test run: rc 0, `fail 0`, and at least one pass for a single file. A skip is named in the
|
||||
report.
|
||||
- RED: any other rc, a failed test, or zero passes.
|
||||
- A mutant is killed when its kill file is RED with a `not ok` on the test K names. 270's K is in
|
||||
queue-270/mutants.py. 271's kills are the bus or cli session-clear.test failures in queue-271/BUILD.md.
|
||||
A mutant whose kill file stays GREEN survived.
|
||||
- A first-run red in a new test (seat compactions-budget.test, cli compactions-launch.test, 271's new
|
||||
tests) is placed and reported after (mos-1272). A red in an old test, a product change, a looser
|
||||
assertion or a second red on a row stops the run and goes to Sage before any change (sage-3086).
|
||||
- Mutants run only on a GREEN base.
|
||||
|
||||
### Order
|
||||
|
||||
1. The 20 touched test files whole, then cli stack-instances.test, webui pinned-copy.test and the
|
||||
business suite (the config loader's readers, round 2 run plan item 3).
|
||||
2. bus, seat and cli suites whole.
|
||||
3. The ten scripts/test-*.sh, test-config.sh among them, with skips named.
|
||||
4. 270 M1-M24, each on its K file whole. Then 271 M1-M14, each on bus and cli session-clear.test whole.
|
||||
|
||||
### Run 1: a first-run red, placed (mos-1272)
|
||||
|
||||
Run 1 started 00:06:57Z on 0f5e5d62d. In phase 1, the 15 bus files and session-compactions passed.
|
||||
Then cli compactions-launch.test.mjs, new in this row and on its first run, went RED: 4 of 5 pass. The
|
||||
failure was in the clear test, "REQ-THREAD-2 T3 T4 a clear leaves the retired session's count...", at
|
||||
:319 (now :324):
|
||||
`[done?.state, done?.start]` was `[undefined, undefined]`, not `["retired", "started"]`, after the
|
||||
30 s settle (31433 ms).
|
||||
|
||||
The cause is in the test, not the product. The test's host `ops` table had no entries for row 271's
|
||||
five clear ops: sessionClears, sessionClearExited, sessionClearUncertain, sessionClearRetire and
|
||||
sessionClearStart. The launcher's clear calls through host.op were refused as invalid-host-request,
|
||||
so the successor never started. cli session-clear.test.mjs:96-100 wires the same five, and its clear
|
||||
tests passed in this run (5 of 5).
|
||||
|
||||
The placement adds those five lines, copied from session-clear.test.mjs, after landRelease. No assertion
|
||||
changed, and no product file changed. The file alone then passed 5 of 5
|
||||
(/mnt/storage/scratch/gizmo-271-out/place-compactions-launch.log). The five lines move 270's
|
||||
compactions-launch kill lines by +5 in queue-270/mutants.py K: :188, :203, :254 and :290. Round 2
|
||||
and round 3 above cite the old lines.
|
||||
|
||||
Run 1 stopped at 00:07:59Z (the log's last write), after phase 1's cli session-clear.test, so that the gate runs on one
|
||||
tree. No node process was left, and the tree was clean. Its log is
|
||||
/mnt/storage/scratch/gizmo-271-out/r4-run1/. This is the row's first red. Run 2 is the whole gate
|
||||
again on the commit that adds the placement. That commit changes the 270 manifest (compactions-launch.test.mjs,
|
||||
mutants.py and this file), so Sage and Duckworth see it before landing.
|
||||
|
||||
### Run 2: a second red, two test defects (mos-1364, sage-3093)
|
||||
|
||||
Run 2 started 00:09:50Z on 5f697db99. Phase 1 passed except one file: the 15 bus files, session-compactions
|
||||
4/4, compactions-launch 5/5, cli session-clear 5/5, stack-release 219/219, seat compactions 20/20,
|
||||
stack-instances 24/24, webui pinned-copy 10/10 and business 231/231. seat compactions-budget.test.mjs
|
||||
went RED: 8 of 10 pass. The file is new in round 2, and this was its first run. Rounds 2 and 3 were not
|
||||
run. This is the row's second red, so I stopped the run (r4.out's last write 00:20:12Z) and changed
|
||||
nothing until Mos ruled yes on both fixes (mos-1364, passed on in sage-3093). Phase 2 did not start. Its
|
||||
log is /mnt/storage/scratch/gizmo-271-out/r4-run2/.
|
||||
|
||||
Both failures are defects in the test, not in compactions.mjs.
|
||||
|
||||
F1, :39 in "the budget is the setting's whole number of bytes...". `readBudget(8.5 * MiB)` returned
|
||||
8912896, not the default 67108864. 8.5 MiB is 8912896 bytes: a whole number, in range. readBudget
|
||||
(compactions.mjs:80-82, Number.isSafeInteger and the 1 MiB to 1 GiB bounds) is right to take it. The case
|
||||
meant a value that is not a whole number of bytes. The fix changes only that input, to `MiB + 0.5`. The
|
||||
other twelve bad values stay. M17's kill line is this :39, so M17 had no valid kill before the fix.
|
||||
|
||||
F2, :107 in "a claude-projects/ that can't be read...". The hook `t.after(() => chmodSync(projects,
|
||||
0o700))` threw ENOENT. A test's after hooks run in the order they were registered. setup()'s rmSync of the
|
||||
root (:20) was registered first, so the tree was gone when this hook ran. The test's assertions had passed.
|
||||
The fix puts the 0o000 window in try/finally, which restores 0o700 before the rm even when an assertion
|
||||
fails, and drops the hook and the inline chmod. It keeps the same assertions in the same order, and none
|
||||
is removed. Net +2 lines, so in mutants.py K, M23 moves :118 to :120 and M22 :127 to :129. M22's note
|
||||
cites the default-cap test's :141, now :143. M15's note cites the clear test's :323 and the legacy :343,
|
||||
which are :328 and :348 since run 1. That change is to note text only. `mutants.py check` shows all 24
|
||||
anchors matching once.
|
||||
|
||||
The file alone then passed 10 of 10 (/mnt/storage/scratch/gizmo-271-out/place-compactions-budget.log).
|
||||
Run 3 is the whole combined gate again, on one tree, on the commit with F1 and F2. A third red on 270
|
||||
stops it and goes to Sage and Mos before any change.
|
||||
|
||||
### Runs 3 and 4: two old-test races, not 270 or 271 (mos-1367, mos-1369)
|
||||
|
||||
Run 3 started 00:23:35Z on 53406f3e4. Phase 1 passed whole (business 231/231), bus 873/873 and seat
|
||||
202/202. cli went RED, 904 of 905: case1-accept.test.mjs:368, the P10 inner hop probe took 1.08 s, under
|
||||
its 1.25 s floor. The file is old and unchanged in 25c09f19f..53406f3e4. I stopped the run during phase 3
|
||||
(r4.out's last write 00:46:24Z) and changed nothing. Mos ruled (mos-1367, passed on in sage-3097): A1 run
|
||||
case1-accept alone, which passed 50/50 (00:50:01Z to 00:54:17Z, load 3.19 3.58 3.71); A2 run 4 on the
|
||||
same tip. Vikunja 573 (owner gander) owns the P10 band fix. Log: /mnt/storage/scratch/gizmo-271-out/r4-run3/.
|
||||
|
||||
Run 4 started 00:54:25Z on 53406f3e4 (load 4.45 4.20 3.94). Phase 1 passed 23/23 files and bus 873/873.
|
||||
seat went RED, 201 of 202: checkout.test.mjs:239 "a dirty clone whose fetch fails is kept, fetch-failed,
|
||||
exactly as it was" failed :91 "the root is unchanged" at its first once(w) (:241). The file is old and
|
||||
unchanged in 25c09f19f..53406f3e4, and it passed in run 3. Load was 8.19 at 01:06Z. I stopped the run
|
||||
(r4.out's last write 01:06:31Z) and changed nothing. The cause is the fixture, not 270 or 271: its git
|
||||
helper (checkout.test.mjs:35) runs with gitEnv() (checkout.mjs:80-90), which does not set
|
||||
maintenance.auto=false, so its commit can leave a detached `git maintenance run --auto` writing in the
|
||||
root that once() fingerprints. Mos watched objects/maintenance.lock vanish after commit returned in 1 of
|
||||
30 fresh repos (mos-1369, passed on in sage-3106). Vikunja 574 (owner gander) owns the fixture fix; no test
|
||||
changed in this slot. Log: /mnt/storage/scratch/gizmo-271-out/r4-run4/.
|
||||
|
||||
### Run 5: GREEN (mos-1369, sage-3106)
|
||||
|
||||
Run 5 waited until the one-minute load was under 4, then started 01:17:11Z on 53406f3e4, unchanged since
|
||||
run 3 (loadavg 3.86 4.91 4.69, up 9 days 11:37; origin/refactor 062646487, no drift from 25c09f19f in
|
||||
packages or scripts). It ended 01:49:41Z, wall 1950 s, exit 0.
|
||||
|
||||
| Phase | Result | Wall |
|
||||
| --- | --- | --- |
|
||||
| 1, touched test files whole and the loader's readers | 23/23 files pass; stack-release 219/219, compactions-budget 10/10, business 231/231 | 673 s |
|
||||
| 2, bus suite | 873/873 | 25 s |
|
||||
| 2, seat suite | 202/202 | 7 s |
|
||||
| 2, cli suite | 905/905; P10 inner hop probe 2.31 s, outer 4.14 s | 682 s |
|
||||
| 3, scripts/test-*.sh | all ten pass; tree clean after | |
|
||||
| 4, 270 M1-M24 | 24/24 killed, each on its K file, each restore matched | |
|
||||
| 4, 271 M1-M14 | 14/14 killed in the file the 271 table names (271 BUILD.md:713), each restore matched | |
|
||||
|
||||
No zero-pass file. Three skips, the same as before: test-queue's queue verify and render --check (this
|
||||
checkout is not the queue's canonical root) and test-task's live provider cases, twice (row 244). No node
|
||||
process was left with its cwd in the worktree. Sage ran one `git fetch` from the canonical checkout at
|
||||
about 01:38:10Z, inside the cli suite (sage-3122); the suite had no red. Log:
|
||||
/mnt/storage/scratch/gizmo-271-out/r4/ and r4.out.
|
||||
@@ -0,0 +1,122 @@
|
||||
#!/usr/bin/env python3
|
||||
# Row 270 mutants. M1-M12 are round 1's (anchors at 7d199f033); M13-M21 are round 2's
|
||||
# (duckworth-0223 B1-B3, sage-3030, sage-3031, mos-1352); M22-M24 are round 3's (duckworth-0226
|
||||
# B4, sage-3038). Round 3's kill lines are at round 3's tests. `apply <id>` edits the worktree and
|
||||
# prints the touched path; each edit asserts its anchor appears exactly once. `check` prints each
|
||||
# anchor's count and kill. K names the test and the first assertion each mutant should fail
|
||||
# (checked by reading, not run). The base blob of every touched file is in BUILD.md; a restore is
|
||||
# `git checkout -- <path>`, then sha256 against it (sage-3030 item 3).
|
||||
import sys
|
||||
WT = "/mnt/storage/scratch/gizmo-270/"
|
||||
C = "packages/seat/src/compactions.mjs"
|
||||
B = "packages/bus/src/broker.mjs"
|
||||
L = "packages/cli/src/launcher.mjs"
|
||||
R = "packages/bus/src/runtime.mjs"
|
||||
ST = "packages/seat/tests/compactions.test.mjs"
|
||||
SB = "packages/seat/tests/compactions-budget.test.mjs"
|
||||
BT = "packages/bus/tests/session-compactions.test.mjs"
|
||||
CT = "packages/cli/tests/compactions-launch.test.mjs"
|
||||
M = {
|
||||
"M1": (C, " const key = keyOf(f.entry, f.completedAt);\n if (ledger.events.has(key)) continue;",
|
||||
" const key = keyOf(f.stream + f.entry, f.completedAt);\n if (ledger.events.has(key)) continue;"),
|
||||
"M2": (C, "const keyOf = (entry, completedAt) => JSON.stringify([entry, completedAt]);",
|
||||
"const keyOf = (entry, completedAt) => JSON.stringify([entry, null]);"),
|
||||
"M3": (C, 'if (e.type === "compaction" && typeof e.id === "string" && e.id) {',
|
||||
'if ((e.type === "compaction" || e.message?.role === "compactionSummary") && typeof e.id === "string" && e.id) {'),
|
||||
"M4": (C, "digest(tail) === c.tail) from = c.bytes;", "true) from = c.bytes;"),
|
||||
"M5": (C, 'if (lines.pop() !== "") bad = torn = true;', 'if (lines.pop() !== "") torn = true;'),
|
||||
"M6": (C, '(ledger.torn ? "\\n" : "") + out.map', 'out.map'),
|
||||
"M7": (C, ' if (ledger.incomplete) return unknown("store-incomplete", count, at);\n', ""),
|
||||
"M8": (C, ' if (ledger.unsupported) return unknown("harness-unsupported", count, at);\n', ""),
|
||||
"M9": (C, "({ count: count || null, known: false, reason, at })", "({ count, known: false, reason, at })"),
|
||||
# Round 3 moved M10's anchor to the Set (duckworth-0226 B4); the mutant is the same.
|
||||
"M10": (C, ' for (const stream of Object.keys(cursors)) if (!present.has(stream)) problems.add("store-incomplete");\n', ""),
|
||||
"M11": (B, "op === 'legacy-3l')", "false)"),
|
||||
"M12": (B, "if (!this.#compactions) return unknown('not-collected');",
|
||||
"if (!this.#compactions) return { count: 0, known: true, reason: null, at: null };"),
|
||||
# B2: the launcher's two collections and the runtime's reader.
|
||||
"M13": (L, " if (store) compactions(store, model.harness, { instance, thread, session, run });\n const toolSocket",
|
||||
" const toolSocket"),
|
||||
"M14": (L, " if (store) compactions(store, model.harness, { instance, thread, session, run });\n try { lock?.release(); }",
|
||||
" try { lock?.release(); }"),
|
||||
"M15": (R, " return dir ? compactionCount(dir) : { count: null, known: false, reason: 'store-unreadable', at: null };",
|
||||
" return null;"),
|
||||
# B1: the budget, its setting and the carry across reads.
|
||||
"M16": (C, " const stop = Math.min(size, from + left);", " const stop = size;"),
|
||||
"M17": (C, " return Number.isSafeInteger(value) && value >= READ_MIN && value <= READ_MAX ? value : READ_BYTES;",
|
||||
" return value ?? READ_BYTES;"),
|
||||
"M18": (L, " const budget = readBudget(system?.compactionReadBytes);", " const budget = READ_BYTES;"),
|
||||
"M19": (C, " take(carry.length ? Buffer.concat([...carry, view.subarray(i, nl)]) : view.subarray(i, nl));",
|
||||
" take(view.subarray(i, nl));"),
|
||||
# B3: Claude history from before the ledger, and its stickiness.
|
||||
"M20": (C, ' if (harness === "pi" && !prior && claudeHistory(store)) {', ' if (harness === "pi" && !prior && false) {'),
|
||||
"M21": (C, 'if (r.harness !== "pi" || r.reason === "harness-unsupported") unsupported = true;',
|
||||
'if (r.harness !== "pi") unsupported = true;'),
|
||||
# B4: the stream cap and its setting.
|
||||
"M22": (C, ' if (visit.length > streams) problems.add("not-collected");\n for (const name of visit.slice(0, streams)) {',
|
||||
' for (const name of visit) {'),
|
||||
"M23": (C, " return Number.isSafeInteger(value) && value >= STREAMS_MIN && value <= STREAMS_MAX ? value : STREAMS;",
|
||||
" return value ?? STREAMS;"),
|
||||
"M24": (L, " const streams = readStreams(system?.compactionStreams);", " const streams = STREAMS;"),
|
||||
}
|
||||
K = {
|
||||
"M1": (ST, "a compaction copied into a branched stream counts once, under the stream it was first seen in", 64,
|
||||
"c1 is written again under b.jsonl: [[c1,a],[c1,b],[c2,b]]; the count holds at 1 and 2 because readLedger dedupes"),
|
||||
"M2": (ST, "the same entry id at another time in another stream is another compaction", 81, "count 1, not 2"),
|
||||
"M3": (ST, "an empty store, a store with no compaction and a failed start count 0, known", 40,
|
||||
"the compactionSummary message m2 counts: count 1, not 0"),
|
||||
"M4": (ST, "a stream that shrank, changed under its cursor or went away is store-incomplete for good > rewrite", 170,
|
||||
"same length, other bytes, read from the old cursor: known true"),
|
||||
"M5": (ST, "a torn last ledger line makes the count unknown, a lower bound, and stays so", 131,
|
||||
"the torn line is popped unparsed, so nothing sets bad: {count 0, known true}"),
|
||||
"M6": (ST, "a torn last ledger line makes the count unknown, a lower bound, and stays so", 133,
|
||||
"the c1 record joins the torn line and is lost: count null, not 1"),
|
||||
"M7": (ST, "a stream that shrank, changed under its cursor or went away is store-incomplete for good > shrink", 174,
|
||||
"the later clean collection reads known; remove would pass, its cursor stays missing"),
|
||||
"M8": (ST, "a Claude run is harness-unsupported, and stays so after a Pi run", 201, "the Pi run reads known true"),
|
||||
"M9": (ST, "a torn last ledger line makes the count unknown, a lower bound, and stays so", 131,
|
||||
"count 0, not null; the symlinked-stream test's :184 fails the same way"),
|
||||
"M10": (ST, "a stream that shrank, changed under its cursor or went away is store-incomplete for good > remove", 170,
|
||||
"a removed stream raises no problem: known true"),
|
||||
"M11": (BT, "a legacy session is asked for with legacy true", 76, "asked with legacy false"),
|
||||
"M12": (BT, "without a reader the count is unknown, never 0; a reader that throws is store-unreadable", 81,
|
||||
"{count 0, known true} for no reader"),
|
||||
"M13": (CT, "a run collects compactions at its start and its end, and the count holds across a resume and a host restart", 188,
|
||||
"no start collection: run 1 logs only [D,2,true,null]; settle returns after 5 s with one event"),
|
||||
"M14": (CT, "a run collects compactions at its start and its end, and the count holds across a resume and a host restart", 188,
|
||||
"no end collection: run 1 logs only [D,1,true,null]"),
|
||||
"M15": (CT, "a run collects compactions at its start and its end, and the count holds across a resume and a host restart", 203,
|
||||
"session.list's compactions is null: {at: 'undefined'} and no count; the clear test's :328 and the legacy :348 fail the same way"),
|
||||
"M16": (SB, "a stream past the default budget is not-collected with a lower bound, and a later collection catches up", 50,
|
||||
"the whole 66 MiB is read: {count 1, known true, reason null}"),
|
||||
"M17": (SB, "the budget is the setting's whole number of bytes from 1 MiB to 1 GiB, and the default for anything else", 39,
|
||||
"readBudget(null) is the default, but readBudget('8388608') is the string"),
|
||||
"M18": (CT, "the read budget is the system config's compactionReadBytes, and a bad value takes the default", 254,
|
||||
"the 1 MiB setting is ignored, the 2 MiB hole is read whole: [D,1,true,null] twice"),
|
||||
"M19": (SB, "a line split across reads counts once", 81,
|
||||
"c1's two halves parse alone and fail: {count null, known false, reason store-incomplete}"),
|
||||
"M20": (SB, "Claude history from before the ledger makes the first Pi collection harness-unsupported, for good", 90,
|
||||
"the first collection counts c1: {count 1, known true}"),
|
||||
"M21": (SB, "Claude history from before the ledger makes the first Pi collection harness-unsupported, for good", 91,
|
||||
"a Pi collected record isn't sticky: the second reads {count 1, known true}"),
|
||||
"M22": (SB, "more streams than the cap is not-collected, with cursors only for the streams opened, and a larger cap catches up", 129,
|
||||
"all four streams are opened: {count 4, known true, reason null}; the default-cap test's :143 fails the same way (1025 cursors)"),
|
||||
"M23": (SB, "the stream cap is the setting's whole number from 32 to 65536, and the default for anything else", 120,
|
||||
"readStreams(null) is the default, but readStreams('2048') is the string"),
|
||||
"M24": (CT, "the stream cap is the system config's compactionStreams, and a bad value takes the default", 290,
|
||||
"the cap of 32 is ignored, all 34 streams are opened: [D,1,true,null] twice"),
|
||||
}
|
||||
cmd = sys.argv[1]
|
||||
if cmd == "list":
|
||||
print(" ".join(M)); sys.exit(0)
|
||||
if cmd == "check":
|
||||
for k, (f, a, b) in M.items():
|
||||
n = open(WT + f).read().count(a)
|
||||
tf, name, line, why = K[k]
|
||||
print(k, f, n, "kill", f"{tf}:{line}", name)
|
||||
sys.exit(0)
|
||||
f, a, b = M[sys.argv[2]]
|
||||
s = open(WT + f).read()
|
||||
assert s.count(a) == 1, (sys.argv[2], s.count(a))
|
||||
open(WT + f, "w").write(s.replace(a, b))
|
||||
print(f)
|
||||
@@ -0,0 +1,790 @@
|
||||
# Row 271 build notes (Gizmoduck)
|
||||
|
||||
264d, T1 item 7: clear context through row 194's lifecycle stop. Gitea #1776. Brief:
|
||||
docs/plans/2026-10-21_t1-sessions.md section 5 (D7, D8), with RULINGS Q4 (rebase onto 194's
|
||||
landing, no 194 code copied). Reviewer Duckworth, gate owner Sage. Built after row 270 (sage-2797).
|
||||
|
||||
## Contract for 264e (pinned for Huey, row 272)
|
||||
|
||||
Proposed, pending Duckworth's review. Any change goes to Huey before it's pushed.
|
||||
|
||||
### session.clear
|
||||
|
||||
Input, every key required, with no other key (`keys()` as `session.create` does):
|
||||
|
||||
```json
|
||||
{ "thread": "<thread id>", "session": "<session id>", "revision": 0, "confirm": true, "op": "<op key>" }
|
||||
```
|
||||
|
||||
- `revision`: the integer from the session's `session.list` row (below). It names the state the
|
||||
human confirmed.
|
||||
- `confirm`: the literal `true`. Anything else refuses `confirm-required`.
|
||||
- `op`: the caller's operation key, an identifier, unique per thread. It is also the idempotency
|
||||
key.
|
||||
|
||||
Answer, the operation envelope:
|
||||
|
||||
```json
|
||||
{ "op": "...", "thread": "...", "session": "<old id>", "successor": "<new id>",
|
||||
"state": "clearing", "start": null, "refusal": null, "at": "..." }
|
||||
```
|
||||
|
||||
- `successor` is bound when the request is accepted, before any asynchronous work. It is a
|
||||
lowercase UUID, `xxxxxxxx-xxxx-8xxx-Vxxx-xxxxxxxxxxxx` with V one of `8`, `9`, `a`, `b` (RFC 9562
|
||||
version 8, variant 10). It's the first 32 hex digits of sha256(business, thread, old session, op)
|
||||
with those two nibbles set. It matches the UUID test every session selector uses
|
||||
(cli/src/thread.mjs:41), so 264a's selectors and deep links take it unchanged (Huey's question,
|
||||
gizmoduck-0274).
|
||||
- `state`:
|
||||
- `clearing`: accepted. Sends and launches to the old session are fenced, and the drain or stop
|
||||
is in progress.
|
||||
- `uncertain`: stop proof isn't available. Capacity stays reserved and nothing is forced. It
|
||||
resumes when proof arrives.
|
||||
- `retired`: the old session is retired and the successor exists, in one bus transaction.
|
||||
- `failed`: no retirement happened, and `refusal` names why. The old session stays as it was.
|
||||
- `cancelled`: a human cancelled the clear (session.cancel below, sage-2828). `retired` is
|
||||
null if it was cancelled before retirement and true if after. `start` stays as it was.
|
||||
- `start` is null until `state` is `retired`. After that it is `held` (waiting for capacity),
|
||||
`started`, or `refused` with `refusal` set.
|
||||
- `at`: the time of the latest state change.
|
||||
|
||||
Repeats and conflicts:
|
||||
|
||||
- The same `op` with the same payload returns the current envelope and starts nothing new.
|
||||
- The same `op` with any other payload refuses `session-op-conflict`.
|
||||
- A new `op` on a session with an open clear refuses `session-clearing`.
|
||||
- A `revision` that isn't current refuses `session-revision-stale`.
|
||||
- Other refusals: `session-not-found`, `session-retired`, `processing-halted` (D8: a halt refuses
|
||||
and changes nothing), and the owner refusals `session.create` already gives.
|
||||
|
||||
### session.cancel (sage-2828)
|
||||
|
||||
Input, every key required, with no other key:
|
||||
|
||||
```json
|
||||
{ "thread": "<thread id>", "op": "<the clear's op>", "confirm": true }
|
||||
```
|
||||
|
||||
This is a human write with the same owner or write-grant check as session.clear, rechecked under
|
||||
the session fence. An agent caller refuses as session.clear does. `confirm` must be the literal
|
||||
`true`, or it refuses `confirm-required`. The answer is the operation envelope with `state`
|
||||
`cancelled`.
|
||||
|
||||
- Before retirement, cancel stops the replacement:
|
||||
- The successor id is never created or started.
|
||||
- Messages already cancelled stay cancelled and are never replayed.
|
||||
- The old session is `active` again.
|
||||
- After retirement, cancel stops the startup only:
|
||||
- No further start attempts are made.
|
||||
- Both sessions stay as they are. Nothing un-retires.
|
||||
- Repeats and refusals:
|
||||
- A repeat with the same payload returns the original cancelled envelope.
|
||||
- The payload digest covers `{thread, op}` and the calling principal. A cancel from a
|
||||
different principal, or a payload that otherwise differs, refuses `session-op-conflict`.
|
||||
- An op that isn't a clear refuses `operation-not-found`.
|
||||
- A cancel of a clear whose successor has started refuses `session-clear-complete`.
|
||||
|
||||
### session.operation
|
||||
|
||||
Input `{ "thread": "...", "op": "..." }`. Answer: the same envelope. Unknown op refuses
|
||||
`operation-not-found`. A CLI that waits 30 s and then reports pending uses this to poll. A retry
|
||||
queries the op here and never resends the clear.
|
||||
|
||||
### session.list additions
|
||||
|
||||
Each row gains:
|
||||
|
||||
- `state`: `active`, `clearing`, `uncertain` or `retired`. A cancelled clear leaves the session
|
||||
`active` if cancelled before retirement, and `retired` if after.
|
||||
- `revision`: an integer. 0 at creation, plus 1 for each lifecycle event on that session.
|
||||
- `successor`: the successor's id once retired, otherwise null.
|
||||
|
||||
A successor row has `predecessor` set (264a's column). When the cleared session was the thread's
|
||||
default, `default` moves to the successor at retirement. Clearing any other session leaves the
|
||||
default where it is.
|
||||
|
||||
On a base without 264d, `session.clear`, `session.cancel` and `session.operation` refuse
|
||||
`unknown-verb`, and the rows have no `state`,
|
||||
`revision` or `successor`.
|
||||
|
||||
## Design (draft 2: Sage's sage-2826 rulings and Mos's mos-1315 folded in)
|
||||
|
||||
Measured on 2d0d407ba, which already holds row 194 (ab7377bcf). Nothing has been run. Draft 1
|
||||
was 3105de889. Draft 2 changes where the drained acknowledgement comes from (stage 2): threadIdle
|
||||
alone can't show that no child is running.
|
||||
|
||||
### What exists and what doesn't
|
||||
|
||||
- No per-session lifecycle state. `sessions` is append-only (schema.sql:370-372). The nearest
|
||||
per-session state is the latest `thread_claims` row (op and generation). Row 194's lifecycle
|
||||
is per instance, as `action.allowed` events keyed by a random request id (broker.mjs:622-666).
|
||||
- No cancelled state for an undelivered message. `deliveries.op` is a CHECK list on an
|
||||
append-only table.
|
||||
- The runner calls `message.receive` only when no harness child exists:
|
||||
- `runTurn`'s close handler SIGKILLs the turn's whole process group and clears `child`
|
||||
(harness/src/runner.mjs:363-366).
|
||||
- The loop asks receive only after the landing turns (:493-517) and after each message's
|
||||
RESULT and `message.read` (:524-580).
|
||||
- So a receive from the run is a completed turn boundary, made by the runner itself.
|
||||
- `threadIdle` (broker.mjs:865-877) is not enough. A landing turn reads no message, so `unread`
|
||||
is 0 while its child runs. That is why landRelease keeps a run until it has asked receive
|
||||
(`#received`, broker.mjs:363-365, :3815-3829).
|
||||
- There is precedent for acting at receive. `#limitStop` (broker.mjs:4147-4148, :4499-4520)
|
||||
holds a stop until the run's next `message.receive` or `state`, so the turn's RESULT and read
|
||||
land first.
|
||||
- The runner's SIGTERM handler escalates to KILL after 5 s only when `child` is set
|
||||
(runner.mjs:405-423). With no child it sets `stopping`, the loop ends, and it exits 0.
|
||||
- The launcher's idle stop (launcher.mjs:683-701) sends a SIGTERM after a start-time check. The
|
||||
exit proof is the run's `close` handler, which runs endRun, removes the registry entry and
|
||||
releases the workspace lock (:600-641).
|
||||
|
||||
### Clear, stage by stage
|
||||
|
||||
1. **Accept** (`session.clear`, the human's write). One transaction:
|
||||
- Check owner and grant (the `session.create` path), the revision, and that processing isn't
|
||||
halted.
|
||||
- Bind the successor before any async work:
|
||||
- The id is deterministic, from a sha256 of (business, thread, old session, op), shaped as a
|
||||
lowercase version 8 UUID (the contract above). 264a's ids are UUIDs, and every selector
|
||||
checks the UUID shape, so draft 2's `ID_PATTERN` shape is replaced.
|
||||
- The name is `<old name>-N`, with N the next free number in that thread. If a user already
|
||||
made a session called `<old>-N`, the next free N is taken.
|
||||
- Both go on the `requested` row, so a repeat of the op returns the same id and name.
|
||||
- Append journal stage `requested`, with a payload digest for the repeat and conflict checks.
|
||||
- The answer is `clearing`. From here:
|
||||
- `thread.send` and `thread.open` naming the session refuse `session-clearing`. So does a
|
||||
thread-only send whose default is that session.
|
||||
- `threadsState` reports the session as `clearing`, so `threadPass` never starts it.
|
||||
- Undelivered messages to it are cancelled with a receipt each (`message_cancels`), in the
|
||||
same transaction. Old messages, results and run evidence don't change.
|
||||
2. **Drain, acknowledged by the runner.** Once the session is clearing, the run's
|
||||
`message.receive` (the runner's call, at a turn boundary with no child) takes one transaction:
|
||||
- It appends `drained` with the run and its claim generation, once.
|
||||
- It delivers nothing and answers `[]`.
|
||||
- The claim stays held. That holds the dequeue fence, so no turn and no child can start
|
||||
before the stop, and the old capacity stays reserved.
|
||||
Later receives keep answering `[]`. `state` needs no fence, because the runner asks it only
|
||||
for a message it received. A clear on a session with no holder appends `drained` with run
|
||||
null at accept.
|
||||
3. **Stop and proof** (launcher). For a `drained` op whose run and generation are still the
|
||||
holder's, the launcher sends the idle stop's start-time-checked SIGTERM, the same reviewed
|
||||
path. The runner has no child, so its handler never reaches KILL.
|
||||
- The exit proof is the run's own `close` handler (endRun, registry remove, lock release),
|
||||
recorded with `sessionClearExited {business, op, run}`. A signal, a revoked claim or a dead
|
||||
socket is not proof.
|
||||
- Under the host, the runner is pid 1 of its own pid namespace (`--unshare-pid --as-pid-1`),
|
||||
so its exit ends anything a turn left outside its process group. **This is a build check**,
|
||||
tested against bwrap, not assumed.
|
||||
- With no run, the proof is that the launcher takes `lockSessionWorkspace` for the session and
|
||||
the registry has no run for it.
|
||||
- If no `drained` arrives (a turn still running, or a stuck child), the session stays
|
||||
`clearing`. Capacity stays reserved, the successor stays unstarted, the status stays
|
||||
actionable, and no signal is sent.
|
||||
- Thirty seconds after the SIGTERM with no proof, `uncertain` is appended once. The launcher
|
||||
keeps waiting and sends no second signal. A launcher restart finds the run among its row 269
|
||||
survivors and waits on its identity.
|
||||
4. **Retire** (trusted op `sessionClearRetire {business, op}`), one transaction:
|
||||
- Check the exit proof (the run is in `#ended`), or that there was no run.
|
||||
- Append `retired`, and end the old claim.
|
||||
- Insert the successor `sessions` row with the bound id and name, `predecessor` set to the old
|
||||
id and `op` set to the clear op.
|
||||
- Add a `session_defaults` row only if the old session was the default.
|
||||
- Add a `thread_opens` row for the successor.
|
||||
The old workspace stays at its path with no new writer mount. The successor gets its own store,
|
||||
HOME and workspace from row 269's per-session layout, and a fresh checkout of the approved
|
||||
revision. Nothing is copied.
|
||||
5. **Start** (D8). `threadPass` starts the successor from its `thread_opens` row through the same
|
||||
`check()`/`start()` as any session. The launcher reports the outcome with `sessionClearStart
|
||||
{business, op, start, refusal}`, which appends `start-held`, `start-started` or
|
||||
`start-refused`, each with the next report counter. A held start keeps its op and spends no
|
||||
work attempt.
|
||||
|
||||
`failed`: a refusal at retire (for example, the owner lost the grant) appends `failed` with the
|
||||
code. Nothing is retired, and the old session goes back to `active` with its messages still
|
||||
cancelled.
|
||||
|
||||
**Cancel** (sage-2828, re-ruling sage-2826 Q3, D7:263-264). `session.cancel` checks the owner or
|
||||
write grant under the fence and appends `cancelled`, attempt 0, in one transaction:
|
||||
- Before `retired` it releases the fence:
|
||||
- If no `drained` was recorded, the drain is still waiting. Nothing is signalled, the clear
|
||||
ends, and the run's next receive delivers again.
|
||||
- If `drained` was recorded with the claim still held and no SIGTERM sent, the launcher skips
|
||||
the stop because the op is no longer open. The run keeps its claim, and its next receive
|
||||
delivers again.
|
||||
- If the SIGTERM was already sent or the run has exited, the old session is `active` with no
|
||||
runner. The next send launches it through D8 like any idle session.
|
||||
- The successor row is never inserted, and the cancelled messages stay cancelled.
|
||||
- The launcher reads the op state right before its SIGTERM, in the same tick as its start-time
|
||||
check. A cancel after that point still finds a stopping runner, which takes the "active with
|
||||
no runner" path above. This is a residual: a cancel can't take back a SIGTERM already sent.
|
||||
- A pre-retire cancel doesn't touch the claim (duckworth-0171 C2):
|
||||
- A run that still holds it keeps it, generation and all. Releasing the fence only stops its
|
||||
receives answering `[]`.
|
||||
- A run already stopped ended its own claim through endRun, so there's nothing to change.
|
||||
- Messages already cancelled keep their `message_cancels` rows and are never delivered.
|
||||
- After `retired` and before `start-started`, no further start attempts are made.
|
||||
- The successor row and the default move stay. Nothing un-retires the old session.
|
||||
- threadPass doesn't start the successor from the clear's `thread_opens` row: a cancelled op
|
||||
suppresses that open.
|
||||
- **My reading, stated for Sage:** the successor is then an ordinary idle session. A later new
|
||||
send to it, including a thread-only send while it's the default, launches it through D8 like
|
||||
any idle session. That's new input, not the clear's startup. The other reading leaves the
|
||||
thread's default pointing at a session that can never start, because there is no verb to
|
||||
move a default (the session verbs are list and create).
|
||||
- The race with the drained receive: `drained` and `cancelled` are each one synchronous
|
||||
transaction on the open op. Whichever commits second sees the first. A receive after the
|
||||
cancel finds no open op and delivers normally. A cancel after `drained` takes the "drained, no
|
||||
SIGTERM yet" path above.
|
||||
- After `start-started` it refuses `session-clear-complete`.
|
||||
|
||||
### Early look d2 folds (duckworth-0170, 98dc69f0c)
|
||||
|
||||
- **B1, messages routed by `#sessionFor` (broker.mjs:722-748).** A role reply chain or a spec
|
||||
notice can resolve to a clearing or retired session with no state check. Rule: **cancel with a
|
||||
receipt at arrival.** The message is inserted (old results stay as evidence, D7 step 4) and its
|
||||
`message_cancels` row goes in the same transaction, so it is never delivered and never pending.
|
||||
The send's answer gains `cancelled: "session-clearing"` or `"session-retired"`, an additive key.
|
||||
- Refusing was the other option. It would lose a teammate's RESULT outright: the runner only
|
||||
logs a failed reply (harness/src/runner.mjs:575-580).
|
||||
- Routing to the successor would replay old input, which D7 forbids.
|
||||
- A spec notice resolves through `#defaultSession`. During a clear of the default session that
|
||||
is still the old session, so the notice is cancelled. After retirement the default is the
|
||||
successor, and a new notice is new input that goes there.
|
||||
- `threadsState` reports `retired`, and `threadPass` starts neither a `clearing` nor a
|
||||
`retired` session.
|
||||
- **B2, a claim racing a no-holder clear.** role.claim's thread path (broker.mjs:4827-4845)
|
||||
refuses `session-clearing` for a session with an open op and `session-retired` for a retired
|
||||
one. The runner gets refused and exits 21. Retire rechecks in its own transaction that no
|
||||
`claim` is current for the session, unless it's the proven-ended run's. The no-run proof also
|
||||
takes `lockSessionWorkspace`, which a spawned run holds before it claims.
|
||||
- **B3, op namespace.** `sessions` has `UNIQUE(thread, op)` (schema.sql:263), and the successor
|
||||
takes the clear op. Both directions refuse `session-op-conflict` at accept, before any stop:
|
||||
- `session.clear` refuses an op already in `sessions` or `session_ops` for the thread.
|
||||
- `session.create` (broker.mjs:5859-5872) refuses an op already in `session_ops`, instead of
|
||||
answering the successor row.
|
||||
- **B4** is in the schema step below (the `attempt` key).
|
||||
- **N2, name cap.** `name()` caps names at 200 characters (broker.mjs:323). When `<old>-N` would
|
||||
be longer than that, session.clear refuses `session-name-too-long` at accept and writes nothing
|
||||
(sage-2828). Test: a 199-character name clears to a 201-character `<old>-1` and is refused, and
|
||||
a 198-character name clears.
|
||||
- **N3, batch.** Drain waits for the runner's whole receive batch. Messages already delivered
|
||||
are a turn in progress, and the runner runs each of them before its next receive (:543-590).
|
||||
Only undelivered messages are cancelled. This is a residual: a batch of k delivered messages
|
||||
delays the drain by k turns.
|
||||
- **N4, D7 step 1 rechecks.** Retire rechecks owner, write grant and halt in its transaction, and
|
||||
refuses to `failed` with the code. External effects: the old run's endRun record is the stop
|
||||
evidence the op joins (its run and generation are on `drained`). No other effect is reconciled
|
||||
in 271.
|
||||
- **N5, claim end.** Retire ends the old claim only if that run's `claim` is still the latest
|
||||
`thread_claims` row for the session. If endRun already ended it, nothing is written twice. Test:
|
||||
retire after endRun, with exactly one end row.
|
||||
- **N6, pid 1.** `--as-pid-1` is in `NAMESPACES` (seat/src/sandbox.mjs:68), and `sandboxArgv`
|
||||
uses it (:168) for the runner command that session.mjs:302 builds. At build I'll confirm that
|
||||
thread runs reach session.mjs:302. Teardown on exit is kernel behaviour, and the test stays.
|
||||
- **N7**: Duckworth checks the mos-1315 list line by line at the code round.
|
||||
- **N8**: row 206 is approved at 3l→3m, so if it lands first, 271 is 3m→3n (as mos-1315 item 4
|
||||
says).
|
||||
|
||||
Cancel tests (sage-2828, duckworth-0171 C2):
|
||||
- before retirement (no drained yet; and drained with the claim held)
|
||||
- a cancel racing the drained receive, each order
|
||||
- a cancel after `start-held`: no further report, and threadPass doesn't start the successor
|
||||
- held, held, then started on one op gives three start rows with reports 1-3 and one work
|
||||
attempt (C1)
|
||||
- after retirement, with no further start
|
||||
- an idempotent repeat
|
||||
- a changed payload refused
|
||||
- after completion: `session-clear-complete`
|
||||
- an agent caller refused
|
||||
|
||||
Mutant: drop the fence release in cancel's pre-retire path. The kill is the pre-retire test's next
|
||||
send/receive.
|
||||
|
||||
Tests gained: B1 a cancel at arrival (reply chain and spec notice) for clearing and retired; B2
|
||||
a claim refused while clearing, and retire refusing a live claim; B3 both op directions; B4 a
|
||||
second `start-held` appended; N2; N5.
|
||||
|
||||
duckworth-0171 N1: the `cancelled` key on message.send is additive. At build I'll grep the
|
||||
whole-answer `deepEqual` pins on message.send in `packages/*/tests/*`. A pin breaks only where a
|
||||
test's routed message lands on a clearing or retired session, which no test does before 271.
|
||||
Any hit goes into the mos-1315 list first.
|
||||
|
||||
### Schema step (label by landing order, mos-1315 item 4)
|
||||
|
||||
- `session_ops(seq, at, business, thread, session, op, stage, attempt, successor, successor_name,
|
||||
run, generation, digest, refusal)`:
|
||||
- append-only, with `UNIQUE(thread, op, stage, attempt)` and a CHECK on `stage`
|
||||
- the stages are `requested`, `drained`, `uncertain`, `exited`, `retired`, `failed`,
|
||||
`cancelled`, `start-held`, `start-started` and `start-refused`
|
||||
- `attempt` is 0 for every stage up to `retired`, and for `failed` and `cancelled`. For the
|
||||
three start stages it is a report counter: 1 plus the number of start rows the op already
|
||||
has. Every `sessionClearStart` report bumps it, a held report included. This keys D7's
|
||||
retryable start (duckworth-0170 B4, duckworth-0171 C1).
|
||||
- The launcher's work attempt goes in its own `work_attempt` column, null outside the start
|
||||
stages. A held start spends none (D8, t1-sessions.md:274), so held, held, started share one
|
||||
work attempt and take reports 1, 2 and 3.
|
||||
- The stage CHECK pins `attempt = 0` for every non-start stage, `cancelled` included.
|
||||
- `session_ops_one_open` treats an op as open while its latest stage is `requested`, `drained`,
|
||||
`uncertain` or `exited`. `retired`, `failed` and `cancelled` close it, so a new clear on the
|
||||
same session can follow a cancelled or failed one. A cancelled op stays in the namespace (B3).
|
||||
- a trigger refuses a second open op per session
|
||||
- the envelope comes from the latest stage, and `revision` is the count of this session's rows
|
||||
- `message_cancels(seq, at, message UNIQUE, session, op)`, append-only.
|
||||
- `threadRows`'s `pending` gains `AND NOT EXISTS (cancel)`, and each row gains `state`.
|
||||
- Label: 3m if 271 lands before row 206. If 206 lands first (3l to 3m at Launchpad's 631a256ba),
|
||||
271 rebases on it and is 3m to 3n. Sage sets the order.
|
||||
|
||||
### Pre-existing test changes (mos-1315)
|
||||
|
||||
These are allowed:
|
||||
- the current schema label
|
||||
- the fabricated future step in comment-migration.test.mjs, moved up one label
|
||||
- the migration file lists, which gain the new step's file
|
||||
|
||||
No assertion is removed or loosened, and no other expected value changes. This list was measured
|
||||
on 2d0d407ba with `git grep` for `'3l'`, `3k-3l` and `'3m'` in `packages/*/tests/*`.
|
||||
|
||||
- Current label (`'3l'` becomes the new label):
|
||||
- account-migration.test.mjs:94, :144, :163, plus the rewrite source :156
|
||||
- chief-of-staff.test.mjs:641, plus the rewrite source :617
|
||||
- comment-migration.test.mjs:36
|
||||
- gitea-migration.test.mjs:98, :124, :142, plus the rewrite source :135
|
||||
- grant-migration.test.mjs:103, :128, :146, plus the rewrite source :139
|
||||
- name-migration.test.mjs:68
|
||||
- sessions.test.mjs:377, :450, plus the rewrite source :433
|
||||
- spec-migration.test.mjs:66
|
||||
- thread-migration.test.mjs:104, plus the rewrite source :52
|
||||
- usage-migration.test.mjs:71
|
||||
- A rewrite source is the `.replace("export const SCHEMA_VERSION = '3l';", ...)` string. It has
|
||||
to match store.mjs:17's new text, or the rewrite silently does nothing.
|
||||
- Fabricated future step: comment-migration.test.mjs:78-79, :86 (`3l`→`3m` becomes one label up).
|
||||
- Migration file lists:
|
||||
- comment-migration.test.mjs:82
|
||||
- the `copyFileSync` lists that copy `3k-3l.sql`: account-migration.test.mjs:86,
|
||||
gitea-migration.test.mjs:76, grant-migration.test.mjs:79
|
||||
- plus each one's sibling copy loop, read whole at the edit
|
||||
- sessions.test.mjs:435 asserts which step names the edited source holds. If its expected list
|
||||
needs the new step's file, that falls under the migration-file-list allowance. I'll read it
|
||||
whole at the edit.
|
||||
- **Restore lists (mos-1318 item 1).** Both new tables have an `at` column, so both join
|
||||
`COUNTED` in scripts/stack-release.sh:420. That is product scope for 271: a rollback with
|
||||
`--restore` counts their rows written after the backup and refuses on them, as it does for
|
||||
`sessions` and `thread_opens`. T46 (stack-release.test.mjs:913) then passes unchanged. Neither
|
||||
table goes in `untimed`, and nothing is removed from any list.
|
||||
- `session_ops`: COUNTED. Every write is a clear stage, and a restore that dropped one uncounted
|
||||
could leave a retired session's successor without its journal.
|
||||
- `message_cancels`: COUNTED. A dropped cancel would put a cancelled message back in `pending`.
|
||||
- T45's `LISTED` (stack-release.test.mjs:858) is a hand copy of COUNTED, so that a table dropped
|
||||
from the script fails T45. **Both names are added** (mos-1319). Mos counts this as the
|
||||
stricter change mos-1285 allows, because T45's loop then checks each new table on its own.
|
||||
The reviewer checks that LISTED matches COUNTED name for name: 15 names after 271, where
|
||||
there were 13. On this base COUNTED is at stack-release.sh:420. Mos measured it at :418 on
|
||||
aca16404c.
|
||||
- **Count pins (mos-1318 item 2).** schema.test.mjs:525-527 (table, trigger and view counts):
|
||||
- tables 22 to 24: `session_ops`, `message_cancels`
|
||||
- triggers 86 to 93:
|
||||
- `session_ops_no_update`, `session_ops_no_delete`, `session_ops_no_replace`
|
||||
- `session_ops_one_open` (refuses a second open op on a session)
|
||||
- `message_cancels_no_update`, `message_cancels_no_delete`, `message_cancels_no_replace`
|
||||
- views: 5, unchanged
|
||||
- The comment block above the pins gains a row 271 line, as rows 124-264a have.
|
||||
- If the build adds or drops an object, this list changes first and the reviewer checks it.
|
||||
- If row 206 (631a256ba) lands first, 271 re-adds its own objects on top of 206's counts.
|
||||
- Any other count pin found at build goes here before it's edited.
|
||||
- After the move, grep `packages/*/tests` for the old label. Only history fixtures and migration
|
||||
chains may still name it. The result is recorded here (mos-1315 item 3).
|
||||
- **TESTER_VERBS pin (mos-1325 item 1).** tester-writes.test.mjs:28-29 gains one comment line and
|
||||
the three names after 264a's `session.list`, `session.create`. The deepEqual stays exact.
|
||||
- **Engines that load the whole current store** need the new step's file. That's the
|
||||
migration-file-list allowance. account-, gitea- and grant-migration.test.mjs each gain a
|
||||
`copyFileSync` of `3l-3m.sql` after their `3k-3l.sql` copy, for their `engine-failing`, which
|
||||
keeps every step. The engines that truncate the step list (engine-3h, -3j, -3i, -3c, -3d, -3k)
|
||||
need nothing.
|
||||
- Applied on gizmoduck/row271 (edit script gizmo-271-edits/mos1315.py). The grep for `'3l'` in
|
||||
`packages/*/tests/*` afterwards has no hit. `3k-3l` stays in the migration copy lists and in
|
||||
sessions.test.mjs:435's "no 3k→3l step" assertion, both history.
|
||||
- Comment-only mentions (`3k-3l migration` in end-launch, message-filter, thread-claims, threads,
|
||||
session-launch, thread-cli, thread-launch, chat, session-compactions) are history and stay.
|
||||
|
||||
### Tests (T4, to write)
|
||||
|
||||
- clear on a held session:
|
||||
- accept fences the session
|
||||
- the run's next receive appends `drained` and answers `[]`
|
||||
- SIGTERM, then exit proof, `retired`, the successor row and the default move
|
||||
- a landing-turn run (no receive yet) is never drained or signalled
|
||||
- a run whose turn is running (delivered, unread) stays `clearing` with no signal, past 30 s
|
||||
- a non-default session clears with the default unchanged
|
||||
- `<old>-N` collision: a user session already named `<old>-1` gives `<old>-2`
|
||||
- repeat with the same payload gives the same envelope, id and name
|
||||
- conflict: a changed payload refuses `session-op-conflict`
|
||||
- a second op on a clearing session refuses `session-clearing`
|
||||
- stale revision, halted processing, retired session
|
||||
- undelivered messages are cancelled with receipts, not delivered to the successor
|
||||
- sends to a clearing session refuse
|
||||
- `uncertain` after 30 s with no proof, and no second signal
|
||||
- the successor start is held, started or refused, through check()/start()
|
||||
- the pid-namespace build check
|
||||
|
||||
## Build round 1 (code on gizmoduck/row271, not run: the slot was held)
|
||||
|
||||
Where the code departs from the design above. Each one is for Duckworth at the code round.
|
||||
|
||||
- **No `state` key in `threadsState`.** end-launch.test.mjs:193 pins a whole threadsState row, and
|
||||
a new key would be an old-test change. Instead:
|
||||
- `threadRows` drops retired sessions (broker.mjs `#threadRows`), and a cancelled message isn't
|
||||
`pending`.
|
||||
- A new trusted op, `sessionClears {business}` (broker.mjs:988), lists each open or starting
|
||||
clear: thread, instance, session, op, stage, run, generation, successor, refusal. The
|
||||
launcher reads it once per tick in `clearPass` (launcher.mjs:742), before its threads loop.
|
||||
Any session a listed clear owns (the old session while stopping, the successor while
|
||||
starting) is skipped by the normal start and idle code, so threadPass never starts or idles
|
||||
it.
|
||||
- **No `thread_opens` row at retire.** The successor's start comes from `sessionClears` (stages
|
||||
`retired`, `start-held`, `start-refused`), not from an open. A cancelled op isn't listed, so
|
||||
nothing starts its successor. A later new send gives the successor a `pending` message, and
|
||||
threadPass starts it like any idle session (sage-2830's reading).
|
||||
- **The run's end is a drain too.** `endLaunch` appends `drained` (run, generation) when the
|
||||
ending run's session has a `requested` op. A run that exits before its next receive needs no
|
||||
signal.
|
||||
- **Exit proof.** `sessionClearExited {business, thread, op, run}` refuses `run-not-ended` unless
|
||||
the run has ended. With run null, the launcher first takes and releases the session's workspace
|
||||
lock. A busy lock (a spawning run) throws and the clear waits a tick.
|
||||
- Side effect: `sessionWorkspace` creates the session's workspace directory, so a session that
|
||||
never ran gets an empty one at its clear.
|
||||
- **Restart.** `recover()` SIGKILLs an earlier host's runs and ends them `host-lost`. A run that
|
||||
outlived the kill is a row 269 survivor and is ended too. `clearStop` waits while a survivor
|
||||
with the clear's run is live (`liveRun`), and sends it nothing. When the old run is gone, the
|
||||
ended run is the proof.
|
||||
- `clearStops` is in memory, so a restarted launcher has no stop times. That's safe because
|
||||
recover() has already killed or listed every run from the earlier host, so none is in the new
|
||||
registry to signal. An `uncertain` op it finds is never signalled.
|
||||
- **Halt at retire waits.** `sessionClearRetire` throws `processing-halted` while processing is
|
||||
halted, and the op stays `exited` until it lifts. Only an owner or grant refusal appends
|
||||
`failed`.
|
||||
- **Held codes.** A successor start refused `capacity-full` or `limit-reached` reports `held`.
|
||||
Every other refusal reports `refused`, and any non-Refusal throw reports `launch-failed`. Each
|
||||
code is logged and reported once per op, not once per tick.
|
||||
- **Names stay bound.** `#nameTaken` counts any op's `successor_name`, a cancelled or failed one
|
||||
included. So a session.create of `<old>-1` after a cancelled clear refuses `session-exists`, and
|
||||
the next clear takes `<old>-2`. That's stricter than needed. Freeing a name would let a repeat
|
||||
of the cancelled op's envelope name a session that isn't its successor.
|
||||
- **Cancel after `failed`** refuses `session-clear-complete`, like cancel after `start-started`.
|
||||
- **message.send** answers `cancelled: "session-clearing"` or `"session-retired"` only when its
|
||||
route lands on such a session (B1). N1 grep of whole-answer pins is still to do.
|
||||
|
||||
### Tests added to T4 (sage-2830)
|
||||
|
||||
- After a post-retire cancel, a thread-only send to the thread (the successor is the default)
|
||||
launches the successor through D8 as new input. The clear's latest row stays `cancelled`, and
|
||||
no `start-*` row is appended.
|
||||
- After a restart, a live survivor of the cleared run holds the clear at `drained` with no
|
||||
signal. Its exit lets the clear retire.
|
||||
|
||||
### Tester grant (mos-1325)
|
||||
|
||||
The brief's section 3 says tester writes aren't granted by default and use fixtures only. 271
|
||||
grants the tester `session.clear`, `session.cancel` and `session.operation` because the Q173 live
|
||||
check has to clear a tester thread through the tester socket. Without the grant, D7 can't be
|
||||
proved live by an agent tester.
|
||||
- The three verbs act only on the tester's own threads. Each resolves its thread with `#thread`
|
||||
(broker.mjs:871-877), which refuses `thread-not-owned` unless the thread's owner is the caller's
|
||||
`#owner`, and a tester's owner is `tester` (:879-880). So a tester can't reach a human's or a
|
||||
user's thread.
|
||||
- Test: a tester's `session.clear`, `session.cancel` and `session.operation` on a thread a human
|
||||
owns each refuse `thread-not-owned` and write nothing (no `session_ops` or `message_cancels`
|
||||
row).
|
||||
- The R3 user gate refuses all three for a user (user-principal.test.mjs:133 spreads
|
||||
TESTER_VERBS). Those refusals are this row's own checks (mos-1325 item 3).
|
||||
|
||||
### N1 and N6 at build
|
||||
|
||||
- **N1.** message.send adds `cancelled` only when `#route` returns a code (broker.mjs:5505,
|
||||
`if (cancelled) return { id: mid, request, cancelled }`). Every other send answers exactly what
|
||||
it did before. `#route` returns a code only for a session with an open or retired clear, and
|
||||
before 271 no test can make one. So no whole-answer pin on message.send can change, and nothing
|
||||
joins the mos-1315 list.
|
||||
- **N6.** The launcher's only spawn is `spawnSession` at launcher.mjs:574, inside `start()`
|
||||
(:422), which runs every session, thread runs included. spawnSession (seat/src/session.mjs:300-302)
|
||||
builds the runner argv through `sandboxArgv`, whose argv starts with `NAMESPACES` (sandbox.mjs:168),
|
||||
and `NAMESPACES` holds `--unshare-pid` and `--as-pid-1` (:68). The bwrap test stays in T4.
|
||||
|
||||
### Tests written (round 1, not run: the slot was held)
|
||||
|
||||
New files only. No old test changes beyond the mos-1315 list above.
|
||||
|
||||
- `packages/bus/tests/session-clear.test.mjs` (15 tests), the bus half:
|
||||
- held default: accept fences send, open and claim; the receive drains once; no proof before the
|
||||
run's end; retire, successor row, default move; threadsState drops the retired session
|
||||
- unheld session: drained at accept, proof with run null; a non-default clear keeps the default
|
||||
- a run that ends before its next receive drains at its end, with its generation
|
||||
- delivered and unread: stays `clearing`, `not-drained` for uncertain and proof; uncertain once;
|
||||
later proof resumes
|
||||
- B1: undelivered inputs cancelled at accept with receipts; a routed reply answers `cancelled`
|
||||
`session-clearing`, later `session-retired`; the successor gets none
|
||||
- repeat, conflict, second op, stale revision, confirm, halt (writes nothing), B3 both ways
|
||||
- Q4 `<old>-N` with a taken `default-1`, and a cancelled op's name staying bound
|
||||
- D8 start counters: held, refused, held, started, with attempt and work_attempt; complete after
|
||||
- halt at retirement waits and writes nothing
|
||||
- cancel before retirement (sage-2828); cancel after retirement and the thread-only send to the
|
||||
successor (sage-2830); cancel after started
|
||||
- B2 claim refusal and the claim after a cancel
|
||||
- mos-1325 tester: own thread clears; clear, cancel and operation on a human thread refuse
|
||||
`thread-not-owned` and write nothing; agents refuse `human-required`
|
||||
- append-only triggers, `session_ops_one_open`, CHECKs and UNIQUEs below the broker
|
||||
- `packages/cli/tests/session-clear.test.mjs` (3 tests, bwrap), the launcher half, with
|
||||
`fixtures/clear-adapter.mjs` (session-adapter's turn plus /proc pids and pid 1's argv):
|
||||
- a running session: one `session-clear-stop`, end, retired, successor launch in that order; the
|
||||
successor starts fresh in its own workspace; N6 pid namespace check on the first turn
|
||||
- held: capacity-full reported `held` and logged once, started when the side session idles out
|
||||
- restart (sage-2830, Q2): a live survivor holds the clear at `drained` with no signal; its exit
|
||||
lets the clear retire and start
|
||||
|
||||
Open, still to write before the code round:
|
||||
|
||||
- A `failed` clear (N4: a user's thread whose grant is revoked before retirement) and cancel after
|
||||
`failed` refusing `session-clear-complete`.
|
||||
- The launcher's `uncertain` path (a run that ignores SIGTERM past `stopTimeoutMs`). It needs an
|
||||
adapter that traps TERM, and a short stopTimeoutMs.
|
||||
- Mutants, at the run: the fence (`#sessionFence` in role.claim, thread.send, thread.open), the
|
||||
drain at receive, the survivor wait in clearStop, CLEAR_HELD, and the `cancelled` filter in
|
||||
`#threadRows`.
|
||||
|
||||
### N4 failed clear (round 1, not run)
|
||||
|
||||
Test "T4 N4: a user's thread whose grant is gone by retirement..." in packages/bus/tests/session-clear.test.mjs.
|
||||
Finding for Duckworth: the `failed` path in sessionClearRetire (broker.mjs:1137, #ownerRefusal at broker.mjs:1191)
|
||||
has no verb route in 271. A user can't call session.clear (USER_VERBS, broker.mjs:138, gate at broker.mjs:4452
|
||||
gives access-refused), and the human or tester gets thread-not-owned on a user's thread. Tester and human threads
|
||||
always pass #ownerRefusal while the business names them. So the test writes the op's requested and drained rows
|
||||
directly, as the accept would, then runs exited and retire with the grant removed. The path becomes reachable
|
||||
when a user verb for clear lands (264e or later). Question for review: keep the path as a guard, or drop it until then.
|
||||
|
||||
### Launcher uncertain path (round 1, not run)
|
||||
|
||||
Test "T4 D7: a drained run that outlives the stop timeout..." in packages/cli/tests/session-clear.test.mjs. The test
|
||||
drives the ticks (tickMs one hour, stopTimeoutMs 1000) and SIGSTOPs the runner after the drain and before the tick
|
||||
that sends the TERM, so the TERM stays pending. It checks one stop, one uncertain, no force, then SIGCONT and
|
||||
retired plus started. Open: mutants (fence in claim/send/open, drain at receive, survivor wait, CLEAR_HELD,
|
||||
the cancelled filter in #threadRows, the uncertain once-only branch at launcher.mjs:788).
|
||||
|
||||
### Mutants (round 1, defined, not run)
|
||||
|
||||
Each mutant is one edit on the 271 tree, and each kill is an assertion failure in a named test, not a timeout
|
||||
(sage-2707). Lines are on gizmoduck/row271-r2 at e567f949c, after the rebase onto 0cbe2e9b3 and the B1
|
||||
fix, which put one comment line above broker.mjs:1194 (launcher.mjs didn't move).
|
||||
|
||||
| # | File:line | Edit | Expected kill |
|
||||
| --- | --- | --- | --- |
|
||||
| M1 | broker.mjs:5158 | drop `#sessionFence(s.session)` in role.claim | bus T4 B2 (claim on a clearing session gives session-clearing) |
|
||||
| M2 | broker.mjs:6146 | drop the fence in thread.send | bus T4 D7 held default (send during the clear refuses session-clearing) |
|
||||
| M3 | broker.mjs:6134 | drop the fence in thread.open | bus T4 D7 held default (open refuses) |
|
||||
| M4 | broker.mjs:6287 | drop the fence in session.clear | bus T4 D7 repeat/conflict: a second op refuses session-clearing. The session_ops trigger (one open clear) is a second fence and would refuse with its own text, so the kill is the code check, not "refused at all" (mutant-masked-by-second-fence) |
|
||||
| M5 | broker.mjs:5599 | receive returns [] without writing drained | bus T4 D7 held default (stages after receive) |
|
||||
| M6 | broker.mjs:1319 | drop the drain at endLaunch | bus T4 D7 run ends before its next receive |
|
||||
| M7 | broker.mjs:6298 | always skip the drain at accept | bus T4 D7 unheld session |
|
||||
| M8 | launcher.mjs:798 | drop the survivor wait | cli T4 D7 Q2 survivor (stage stays drained, no stop) |
|
||||
| M9 | launcher.mjs:815 | CLEAR_HELD empty | cli T4 D8 held (start held, one log line) |
|
||||
| M10 | broker.mjs:1209 | drop the message_cancels filter in #threadRows | cli T4 D7 running session (cancelled input would launch D again) or bus B1; the run decides which, both are named |
|
||||
| M11 | launcher.mjs:788 | `stage === "drained"` becomes `true` | cli T4 D7 uncertain (one uncertain event); bus T4 D7 delivered-unread keeps its broker-side once |
|
||||
| M12 | broker.mjs:1137 | `#ownerRefusal` returns null | bus T4 N4 failed clear |
|
||||
| M13 | broker.mjs:1195 | drop the `'session.clear'` verb arg from `#ownerRefusal`'s `#granted` call (the r2 defect, duckworth-0201 B1) | bus T4 N4 live grant (refusal access-refused, state failed) |
|
||||
|
||||
duckworth-0201 B1 (early look on 911b90903): `#ownerRefusal` called `#granted(s, project, instance)`
|
||||
against the signature `#granted(s, verb, project, instance)` at broker.mjs:919, so the project sat in
|
||||
the verb slot and the instance in the project slot, and a user's thread with a live grant failed
|
||||
retirement with access-refused. The N4 test revoked the grant first, so it could not tell the two
|
||||
apart. The call now passes `'session.clear'`, which is not in VIEWER_VERBS, so the recheck is at
|
||||
collaborator. The N4 setup is a shared helper, the revoked case is unchanged, and a second test keeps
|
||||
the grant and expects retired plus the successor. M13 restores the defect. Both N4 tests are new 271
|
||||
tests, so mos-1292 doesn't apply.
|
||||
|
||||
M10's kill is a guess until it runs; if neither test fails, I add a pending-count assertion and report it as a
|
||||
first-run change under mos-1272.
|
||||
|
||||
## Rebase onto row 206: schema label 3m to 3n (sage-2968)
|
||||
|
||||
Row 206 landed 0cbe2e9b3 on refactor at 19:47Z with schema step 3l to 3m (thread_shares), and
|
||||
origin/gizmoduck/row271 at 74b5e3638 also set 3m (Mos found the collision in mos-1335). The branch is
|
||||
rebased on 0cbe2e9b3 as gizmoduck/row271-r2, a new name, so nothing is force-pushed. The rebase took
|
||||
`git rebase --onto 0cbe2e9b3 2d0d407ba`, since the 269 commits under 271 landed as 269 r3. Row 271
|
||||
becomes step 3m to 3n:
|
||||
|
||||
- `packages/bus/src/migrations/3m-3n.sql` is the old 271 `3l-3m.sql`, byte for byte. `3l-3m.sql` is
|
||||
206's.
|
||||
- store.mjs: `SCHEMA_VERSION = '3n'`, the 3m to 3n step after 206's, `session_ops` and
|
||||
`message_cancels` stay last in `timedTables`, and `SCHEMA_SHA256` is the sha256 of the new
|
||||
schema.sql (a2ba3a46...).
|
||||
- schema.sql is refactor's with the 271 objects appended after 206's, unchanged.
|
||||
- schema.test.mjs:526-529: 25 tables and 96 triggers, 206's 23 and 89 plus 2 and 7. `grep -c` in
|
||||
schema.sql gives 25 `CREATE TABLE` and 97 `CREATE TRIGGER`; at 0cbe2e9b3 it gives 23 and 90 against
|
||||
the pinned 89, so the extra hit is in the base file and the step adds 7.
|
||||
- `packages/bus/tests/fixtures/schema-3m.sql` equals refactor's schema.sql at 0cbe2e9b3 (sha256
|
||||
d5ca9bb9...).
|
||||
- stack-release.sh:420 COUNTED and stack-release.test.mjs:861 LISTED: both tables after 206's
|
||||
`thread_shares`, at the end of the list as before.
|
||||
|
||||
New tests in packages/bus/tests/session-clear.test.mjs, written like 206's in thread-shares.test.mjs:
|
||||
- "T4 schema: a 3m store migrates to 3n": a 3m store from the fixture, with a row in each thread table
|
||||
and a share, migrates. Both tables are new and empty, every other row stays, the migrated
|
||||
sqlite_master equals a fresh store's, both tables refuse update, delete and replace, the one-open
|
||||
trigger refuses a second open clear, and a cancel naming no message refuses on its foreign key.
|
||||
- "T4 schema: the 3m engine refuses a 3n store": the 3m engine refuses with and without
|
||||
migrateSchema and leaves the store as it was, and it opens a fresh 3m store.
|
||||
|
||||
### Pre-existing test changes on the rebase
|
||||
|
||||
The mos-1315 kinds, one label up:
|
||||
- Current label `'3m'` becomes `'3n'`: account-migration.test.mjs:98, :148, :167 and rewrite source
|
||||
:160; gitea-migration.test.mjs:102, :128, :146 and :139; grant-migration.test.mjs:107, :132, :150
|
||||
and :143; chief-of-staff.test.mjs:641 and :617; comment-migration.test.mjs:36; name-migration :68;
|
||||
spec-migration :66; usage-migration :71; thread-migration :104 and :52; sessions.test.mjs:377,
|
||||
:450 and :433; thread-shares.test.mjs:406, :459 and the engine3l rewrite source :442.
|
||||
- The offline command's ready line, `bus schema 3m ready` becomes `3n`: comment-migration.test.mjs:112,
|
||||
thread-migration.test.mjs:164. My first build missed these two (`3l ready` then), so the first run
|
||||
would have shown them red.
|
||||
- The future step in comment-migration.test.mjs:78-79 moves to 3n to 3o, :86 expects `'3o'`, and the
|
||||
:82 file list gains `3m-3n.sql`.
|
||||
- Engine copies: account- :90, gitea- :80 and grant-migration.test.mjs :83 copy `3m-3n.sql` after
|
||||
206's `3l-3m.sql`.
|
||||
|
||||
Changes outside the mos-1315 list, the same kind 206 made for thread_shares. Mos approved all three
|
||||
under mos-1292 in mos-1336 (19:58Z). Conditions: neither name list grows past these two names without
|
||||
Mos, and a gate red in these files outside the listed lines goes to Mos:
|
||||
- `later3l` in account- :21-22, gitea- :38-39 and grant-migration.test.mjs :24-25 gains
|
||||
`session_ops` and `message_cancels`, with a comment line. These tests compare sqlite_master before
|
||||
and after older steps, and the 3m to 3n objects are new there just as thread_shares is. My first
|
||||
build missed this too.
|
||||
- `rows()` in gitea- :43 and grant-migration.test.mjs :45 leaves out both tables, as it leaves out
|
||||
thread_shares.
|
||||
- thread-shares.test.mjs:410-414: 206's 3l migration test now ends at 3n, so after it asserts
|
||||
thread_shares empty it asserts session_ops and message_cancels empty and drops them before the
|
||||
whole-table compare. Nothing it checked is removed.
|
||||
|
||||
A grep for `'3m'` in these files afterwards has no hit. session-clear.test.mjs's 3m engine is new. This is a rebase,
|
||||
so the next run reruns every touched file whole.
|
||||
|
||||
## Run limits (mos-1275, sage-2991)
|
||||
|
||||
The gate on e567f949c (sage-2991). Each node --test file and suite runs under `timeout -k 10 LIMIT`, and
|
||||
each scripts/test-*.sh under at least 3x its longest recorded run. The records are from
|
||||
agents/gizmoduck/work/queue-269/BUILD.md "Run limits", plus row 269's r3 gate.
|
||||
|
||||
| Run | Longest recorded | LIMIT |
|
||||
| --- | --- | --- |
|
||||
| stack-release.test alone | 575 s (264a r12 survey) | 1800 s |
|
||||
| each other touched file alone | 15 s (thread-launch.test) | 300 s |
|
||||
| bus suite | 23 s | 900 s |
|
||||
| seat suite | none recorded | 600 s |
|
||||
| cli suite | 591 s (row 281 gate) | 1800 s |
|
||||
| each scripts/test-*.sh | 42 s (test-foundation) | 900 s |
|
||||
| each mutant run (one session-clear file whole) | none recorded | 300 s |
|
||||
|
||||
A timeout with no failed test is a hang and goes to Sage. A timeout in the shell phase goes to Mos
|
||||
unplaced (mos-1275). A first red in a new 271 test is placed and reported after (mos-1272). A second
|
||||
red, a red in an old test outside the mos-1336 lines, or a fix that needs a product change outside the
|
||||
manifest or a looser assertion stops the run and goes to Mos and Sage (sage-2991). Mutants run only on
|
||||
a green base.
|
||||
|
||||
## Gate on e8af389e6 (sage-2991)
|
||||
|
||||
The code on e8af389e6 is the same as on e567f949c. The only difference is the BUILD.md "Run limits"
|
||||
section. The run went from 20:31:19Z to 20:56:32Z (1513 s wall), on base 0cbe2e9b3 with origin/refactor
|
||||
at f2735bb6a, whose drift touches only docs/plans/QUEUE.md and queue.json. The logs are in
|
||||
/mnt/storage/scratch/gizmo-271-out/gate/.
|
||||
|
||||
- The 18 touched test files, each run whole, all pass with 0 fail and 0 skip. Among them, bus
|
||||
session-clear 19/19, cli session-clear 4/4, stack-release.test 218/218, thread-shares 16/16 and seat
|
||||
compactions 20/20. Phase wall 596 s.
|
||||
- Suites: bus 873/873 (24 s), seat 192/192 (6 s), cli 873/873 (588 s).
|
||||
- scripts/test-*.sh: all 10 pass. auth 15, conductor 17, config 31, discord 81, extension-package 18,
|
||||
foundation 44, land-work 86, queue 156 node tests plus 28, release 14, task 91. The skips are
|
||||
test-queue's verify/render --check (this worktree isn't the canonical root) and test-task's live
|
||||
provider cases (row 244). test-task has no Docker skip.
|
||||
- The tree was clean after the tests and after each mutant restore. No node process was left with its
|
||||
cwd in the worktree.
|
||||
- First-run reds: none, so nothing was placed under mos-1272.
|
||||
|
||||
Each mutant ran with bus and cli session-clear.test whole. Every kill is an AssertionError in the
|
||||
test this section named, never a timeout:
|
||||
|
||||
| # | Killed by |
|
||||
| --- | --- |
|
||||
| M1 | bus T4 B2 and T4 D7 held default |
|
||||
| M2 | bus T4 D7 held default and T4 D7 delivered-unread |
|
||||
| M3 | bus T4 D7 held default |
|
||||
| M4 | bus T4 D7 repeat/conflict: `storage-refused` where `session-clearing` was expected, so the code check failed and the trigger was the only fence left. Also T4 D7 held default (`session-retired` expected, got null) |
|
||||
| M5 | bus T4 D7 held default and delivered-unread; cli T4 D7 uncertain, D7 D8 N6 running session, and D7 Q2 survivor |
|
||||
| M6 | bus T4 D7 run ends before its next receive |
|
||||
| M7 | bus T4 D7 unheld session (stages `['requested']` where `['requested','drained']` was expected), and 7 later bus tests, 5 of them on `not-drained` or `session-revision-stale` refusals that follow from the missing drain; cli T4 D8 capacity-full |
|
||||
| M8 | cli T4 D7 Q2 survivor |
|
||||
| M9 | cli T4 D8 capacity-full held |
|
||||
| M10 | bus T4 D7 B1 (cancelled inputs never reach the successor). The cli running-session test passes, so no pending-count assertion was added |
|
||||
| M11 | cli T4 D7 uncertain |
|
||||
| M12 | bus T4 N4 grant gone |
|
||||
| M13 | bus T4 N4 grant live |
|
||||
|
||||
Open: the row's gate names tests T4 and T6 (queue row 271). This build maps T4. Of T6, these are
|
||||
covered: clear under a kill or restart (cli D7 Q2 survivor, D7 uncertain), grant revocation (bus N4),
|
||||
limits (cli D8 capacity-full held) and a halt at retirement (bus D7 N4 halt). These are not: a Q198
|
||||
hold, a halt between reservation, spawn and bind, and a manager configured but unavailable. That goes
|
||||
to Sage before the in-review move.
|
||||
|
||||
## Round 3: T6 (sage-3001, duckworth-0209, duckworth-0210)
|
||||
|
||||
Sage ruled the three T6 items left open above. Written with no runs; the r3 gate runs on a CLEAR.
|
||||
|
||||
- Halt between reservation, spawn and bind: covered by citation. session-launch.test.mjs:372 (the
|
||||
halt fence, C2) halts once after check() and once inside its recheck, and neither attempt spawns.
|
||||
A successor goes through the same path: clearStart at launcher.mjs:824 calls `check()` and
|
||||
`start()` unchanged, so the fence that test pins is the successor's fence too.
|
||||
- Q198 hold: binds row 257 (Morgana). Row 271 has no Q198 code to test: 257 hasn't landed, and
|
||||
packages/*/src holds no pressure or Q198 code on this branch or on origin/refactor f2735bb6a.
|
||||
Carry for 257 (duckworth-0210): 257 either puts its hold code in CLEAR_HELD (launcher.mjs:815) or
|
||||
rules that a Q198 hold counts as a refusal, and either way it adds a clear-path test.
|
||||
- Manager configured but unavailable: a new cli test, "T4 T6: a bus unavailable at the exit proof
|
||||
and then at retirement" (packages/cli/tests/session-clear.test.mjs). The test's host op rejects
|
||||
with `broker-exited`, the code host.mjs:303 gives when the broker exits before it replies. It is
|
||||
down first at `sessionClearExited`, then only at `sessionClearRetire`. While it is down the test
|
||||
expects no retired or failed event, no launch but D's, nothing in the session registry (the
|
||||
successor takes no slot) and D still the thread's only and default session. The stage holds at
|
||||
drained, then at exited. The failure is logged once in all, and no `thread tick:` line appears.
|
||||
Once the bus answers, the clear retires and the successor starts, in the same event order as the
|
||||
running-session test.
|
||||
|
||||
Reading of "capacity release": the launcher counts capacity from its registry of running sessions
|
||||
(`limitHit`, launcher.mjs:403). D's run really ends, so its slot is free on the host either way;
|
||||
what an unproven stop must not do is hand that slot to the successor. The test pins that.
|
||||
|
||||
| # | Where | Mutant | Killed by |
|
||||
| --- | --- | --- | --- |
|
||||
| M14 | launcher.mjs:755-759 | drop clearPass's per-clear try/catch, so a failed op leaves the pass and tick() | cli T4 T6 bus unavailable: the first `lines()` assertion. The clear's line is never logged; the ticker's catch logs `thread tick:` instead |
|
||||
|
||||
The harness the gate applies, M1-M14 with their exact anchors, is committed beside this file as
|
||||
mutants.py (duckworth-0212 C1). `check` counts each anchor; `apply <id>` edits the worktree.
|
||||
|
||||
r3 gate, on a CLEAR (sage-3001): the touched files run whole, the cli suite runs whole, and M14.
|
||||
No src file changed in r3, so the full gate isn't due again. A first-run red in the new test is
|
||||
placed under mos-1272 and reported after; a fix that needs a product change goes to Sage and Mos
|
||||
before any edit.
|
||||
|
||||
## r3 gate on 9e18a9aaf (sage-3017)
|
||||
|
||||
Run 21:16:37Z to 21:27:04Z (627 s wall), TMPDIR /mnt/storage/scratch/tmp/gJmI, limits as in "Run
|
||||
limits" above (300 s per file and per mutant run, 1800 s for the cli suite). origin/refactor was
|
||||
f2735bb6a. No src file differs from e567f949c, so the full gate wasn't due (sage-3001).
|
||||
|
||||
| Run | Result | Wall |
|
||||
| --- | --- | --- |
|
||||
| cli session-clear.test whole | 5/5 pass | 12 s for both files |
|
||||
| bus session-clear.test whole (r3 touched no other test; the bus half of the clear) | 19/19 pass | |
|
||||
| cli suite whole | 874/874 pass, 0 skipped or todo | 585 s |
|
||||
| M14, bus session-clear.test | 19/19 pass (M14 is launcher-only) | 30 s for both |
|
||||
| M14, cli session-clear.test | 1 fail: the T6 test | |
|
||||
|
||||
- First-run reds: none. The new T6 test passed on its first run, so nothing was placed under mos-1272.
|
||||
- M14 kill: an AssertionError at session-clear.test.mjs:365, the T6 test's first `lines()`
|
||||
assertion, as named above. The ticker's catch logged `thread tick: broker exited before it
|
||||
replied` once per tick and the clear's own line never appeared.
|
||||
- The tree was clean after the tests and after the M14 restore; launcher.mjs matched HEAD. No node
|
||||
process was left with its cwd in the worktree.
|
||||
@@ -0,0 +1,44 @@
|
||||
#!/usr/bin/env python3
|
||||
# Row 271 mutants M1-M14 (BUILD.md "Mutants", lines at e567f949c; M14 is r3's). `apply <id>` edits the
|
||||
# worktree and prints the touched path; each edit asserts its anchor appears exactly once.
|
||||
import sys
|
||||
WT = "/mnt/storage/scratch/gizmo-270/"
|
||||
B = "packages/bus/src/broker.mjs"
|
||||
L = "packages/cli/src/launcher.mjs"
|
||||
M = {
|
||||
"M1": (B, " this.#sessionFence(s.session);\n if (this.#sessionHolder(s.session)?.op === 'claim') fail('thread-already-held');",
|
||||
" if (this.#sessionHolder(s.session)?.op === 'claim') fail('thread-already-held');"),
|
||||
"M2": (B, " // Row 271 (264d, D7 step 1): nor sent to, by name or as the default.\n this.#sessionFence(session);\n",
|
||||
" // Row 271 (264d, D7 step 1): nor sent to, by name or as the default.\n"),
|
||||
"M3": (B, " // Row 271 (264d, D7 step 1): a session being cleared, or a retired one, isn't opened.\n this.#sessionFence(session);\n",
|
||||
" // Row 271 (264d, D7 step 1): a session being cleared, or a retired one, isn't opened.\n"),
|
||||
"M4": (B, " this.#processingWork();\n this.#sessionFence(session);\n if (this.#revision(session) !== a.revision)",
|
||||
" this.#processingWork();\n if (this.#revision(session) !== a.revision)"),
|
||||
"M5": (B, " if (!this.#opRow(open.thread, open.op, 'drained')) this.#appendOp(open, 'drained', { run: s.run, generation: this.#runGeneration(s) });\n return [];",
|
||||
" return [];"),
|
||||
"M6": (B, " if (open?.stage === 'requested') this.#appendOp(open, 'drained', { run: session.run, generation: h.generation });\n", ""),
|
||||
"M7": (B, " if (this.#sessionHolder(session)?.op !== 'claim') this.#appendOp(op, 'drained');",
|
||||
" if (false && this.#sessionHolder(session)?.op !== 'claim') this.#appendOp(op, 'drained');"),
|
||||
"M8": (L, " if (survivors.some((x) => x.business === b.id && x.run === c.run && liveRun(x))) return;\n", ""),
|
||||
"M9": (L, 'const CLEAR_HELD = new Set(["capacity-full", "limit-reached"]);', 'const CLEAR_HELD = new Set([]);'),
|
||||
"M10": (B, "\n AND NOT EXISTS(SELECT 1 FROM message_cancels x WHERE x.message=m.id)) AS pending,", ") AS pending,"),
|
||||
"M11": (L, '} else if (stage === "drained" && now - clearStops', '} else if (true && now - clearStops'),
|
||||
"M12": (B, " const refusal = this.#ownerRefusal(t);", " const refusal = null;"),
|
||||
"M13": (B, "this.#granted({ ...s, grants: this.#userGrants(s) }, 'session.clear', t.project, t.instance)",
|
||||
"this.#granted({ ...s, grants: this.#userGrants(s) }, t.project, t.instance)"),
|
||||
"M14": (L, " try {\n await (start ? clearStart(c, legacyThreads) : clearStop(c, now));\n } catch (e) {\n clearLog(c.op, e.code ?? e.message, `clear ${c.op} (session ${c.session}): ${e.code ?? e.message}`);\n }\n",
|
||||
" await (start ? clearStart(c, legacyThreads) : clearStop(c, now));\n"),
|
||||
}
|
||||
cmd = sys.argv[1]
|
||||
if cmd == "list":
|
||||
print(" ".join(M)); sys.exit(0)
|
||||
if cmd == "check":
|
||||
for k, (f, a, b) in M.items():
|
||||
n = open(WT + f).read().count(a)
|
||||
print(k, f, n)
|
||||
sys.exit(0)
|
||||
f, a, b = M[sys.argv[2]]
|
||||
s = open(WT + f).read()
|
||||
assert s.count(a) == 1, (sys.argv[2], s.count(a))
|
||||
open(WT + f, "w").write(s.replace(a, b))
|
||||
print(f)
|
||||
@@ -638,6 +638,18 @@ before row 192 refuses the key). The message says to remove the key first. So a
|
||||
window re-pins to row 192 or later before it writes the key, and a rollback past
|
||||
row 192 removes the key first.
|
||||
|
||||
Two more optional system config keys bound the host's compaction collections
|
||||
(row 270, #1775, DECISIONS Q222). Only a human writes them, like `console`.
|
||||
`compactionReadBytes` is the stream bytes one collection reads, a whole number
|
||||
from 1048576 to 1073741824, default 67108864 (64 MiB). `compactionStreams` is the
|
||||
stream files one collection opens, a whole number from 32 to 65536, default 1024.
|
||||
The loader passes either on as written. The launcher takes the default for a
|
||||
missing or bad value and logs one `system config:` line. A collection that hits
|
||||
either bound counts as `not-collected`, and the next one reads on. A copy before
|
||||
row 270 refuses both keys, and the release's config check doesn't catch that
|
||||
(it runs the target's loader only for a `console` key). So any re-pin or
|
||||
rollback plan whose OLD copy is before row 270 removes both keys first.
|
||||
|
||||
#### A target past c216f47db: the PM after the harness kind
|
||||
|
||||
From c216f47db on, the bus launches the PM only with a `harness` kind in the
|
||||
|
||||
@@ -246,7 +246,7 @@ a receipt with empty `changes`. A refusal is `action.refused` with its code alon
|
||||
| `thread.messages` | `{thread,session?,after?,limit?}`; human sessions only. The session's messages oldest first, after the message `after` names (`message-not-found` when it isn't in the session), at most `limit` (default 50, 1 to 200). A thread shared with the caller reads too, any session of it (row 206). Writes nothing. |
|
||||
| `thread.share`, `thread.unshare` | `{thread,to}`; the thread's owner only (row 206, #1704). `to` is `user:<id>`, the business human's name or `*`; yourself refuses `invalid-request`. A share refuses `share-exists` when one is in force, and `share-not-covered` when a `user:` target has no live grant covering the thread. An unshare refuses `share-not-found` when none is. Each writes one `thread_shares` row. Returns `{thread,to,op}`. |
|
||||
| `thread.shares` | `{thread}`; the thread's owner only. The shares in force, `[{to,at,by}]`, oldest first. Writes nothing. |
|
||||
| `session.list` | `{thread}`; human and tester sessions (row 264a, #1769). The thread's sessions oldest first, `[{id,at,name,predecessor,default}]`. Writes nothing. |
|
||||
| `session.list` | `{thread}`; human and tester sessions (row 264a, #1769). The thread's sessions oldest first, `[{id,at,name,predecessor,default,compactions}]`. `compactions` is `{count,known,reason,at}` from the host's ledger (row 270, #1775): `known` true gives the exact count of completed Pi compactions; otherwise `count` is a lower bound or null and `reason` is `harness-unsupported`, `not-collected`, `store-unreadable` or `store-incomplete`. `not-collected` passes: the next collection reads on. A launcher killed mid-write leaves a torn ledger line, and the count stays unknown (`store-incomplete`) for good, by design (sage-2806 condition 3). Writes nothing. |
|
||||
| `session.create` | `{thread,name,op}`; human and tester sessions. A new session of the thread, metadata only. A retry with the same `op` and name returns the first row; another name under that `op` refuses `session-op-conflict`, and a taken name `session-exists`. Returns `{id,thread,name}`. |
|
||||
|
||||
### Projects and threads (row 121, #1615)
|
||||
|
||||
@@ -493,3 +493,40 @@ CREATE INDEX thread_shares_thread ON thread_shares (thread, target, seq);
|
||||
CREATE TRIGGER thread_shares_no_update BEFORE UPDATE ON thread_shares BEGIN SELECT RAISE(ABORT, 'thread_shares is append-only'); END;
|
||||
CREATE TRIGGER thread_shares_no_delete BEFORE DELETE ON thread_shares BEGIN SELECT RAISE(ABORT, 'thread_shares is append-only'); END;
|
||||
CREATE TRIGGER thread_shares_no_replace BEFORE INSERT ON thread_shares WHEN EXISTS (SELECT 1 FROM thread_shares WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'thread_shares is append-only'); END;
|
||||
CREATE TABLE session_ops (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
thread TEXT NOT NULL REFERENCES threads(id),
|
||||
session TEXT NOT NULL REFERENCES sessions(id),
|
||||
op TEXT NOT NULL,
|
||||
stage TEXT NOT NULL CHECK (stage IN ('requested','drained','uncertain','exited','retired','failed','cancelled','start-held','start-started','start-refused')),
|
||||
attempt INTEGER NOT NULL CHECK (CASE WHEN stage GLOB 'start-*' THEN attempt >= 1 ELSE attempt = 0 END),
|
||||
work_attempt INTEGER CHECK (work_attempt IS NULL OR (stage GLOB 'start-*' AND work_attempt >= 1)),
|
||||
successor TEXT NOT NULL,
|
||||
successor_name TEXT NOT NULL,
|
||||
run TEXT,
|
||||
generation INTEGER CHECK (generation IS NULL OR generation >= 1),
|
||||
digest TEXT,
|
||||
refusal TEXT,
|
||||
UNIQUE (thread, op, stage, attempt)
|
||||
) STRICT;
|
||||
CREATE TABLE message_cancels (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
message TEXT NOT NULL UNIQUE REFERENCES messages(id),
|
||||
session TEXT NOT NULL REFERENCES sessions(id),
|
||||
op TEXT NOT NULL
|
||||
) STRICT;
|
||||
CREATE INDEX session_ops_session ON session_ops (session, seq);
|
||||
CREATE TRIGGER session_ops_no_update BEFORE UPDATE ON session_ops BEGIN SELECT RAISE(ABORT, 'session_ops is append-only'); END;
|
||||
CREATE TRIGGER session_ops_no_delete BEFORE DELETE ON session_ops BEGIN SELECT RAISE(ABORT, 'session_ops is append-only'); END;
|
||||
CREATE TRIGGER session_ops_no_replace BEFORE INSERT ON session_ops WHEN EXISTS (SELECT 1 FROM session_ops WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'session_ops is append-only'); END;
|
||||
CREATE TRIGGER session_ops_one_open BEFORE INSERT ON session_ops
|
||||
WHEN NEW.stage = 'requested' AND EXISTS (SELECT 1 FROM session_ops o WHERE o.session = NEW.session
|
||||
AND o.seq = (SELECT max(seq) FROM session_ops WHERE thread = o.thread AND op = o.op)
|
||||
AND o.stage IN ('requested','drained','uncertain','exited'))
|
||||
BEGIN SELECT RAISE(ABORT, 'session has an open clear'); END;
|
||||
CREATE TRIGGER message_cancels_no_update BEFORE UPDATE ON message_cancels BEGIN SELECT RAISE(ABORT, 'message_cancels is append-only'); END;
|
||||
CREATE TRIGGER message_cancels_no_delete BEFORE DELETE ON message_cancels BEGIN SELECT RAISE(ABORT, 'message_cancels is append-only'); END;
|
||||
CREATE TRIGGER message_cancels_no_replace BEFORE INSERT ON message_cancels WHEN EXISTS (SELECT 1 FROM message_cancels WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'message_cancels is append-only'); END;
|
||||
+382
-11
@@ -127,7 +127,9 @@ export const TESTER_VERBS = Object.freeze(['message.send', 'message.receive', 'm
|
||||
// Row 194 (#1692, D4.1): launch and terminate, and the read of the requests they make.
|
||||
'instance.launch', 'instance.end', 'instance.request',
|
||||
// Row 264a (#1769): the sessions of the tester's own threads.
|
||||
'session.list', 'session.create']);
|
||||
'session.list', 'session.create',
|
||||
// Row 271 (264d, #1776): clear context in the tester's own threads.
|
||||
'session.clear', 'session.cancel', 'session.operation']);
|
||||
// FED R3 (#414, federation design 3.5): the collaborator level is this exact list. Every other verb
|
||||
// refuses a user access-refused before its own checks: owner verbs, board reads, message.*,
|
||||
// decision.*, project files, spec.read, assistant.stop, launch.*, name.set and usage.*.
|
||||
@@ -394,6 +396,7 @@ export class Broker {
|
||||
// FED R3: () => the checked access document (R1's loadAccess), read once per user request. Null,
|
||||
// a throw or no document leaves every user with no grant (design 3.5, fail closed).
|
||||
#access;
|
||||
#compactions;
|
||||
// r6 X3: interrupted host grants whose remote undo waits for reconcileHosts, each with its claim.
|
||||
#hostUndo = [];
|
||||
// r6 section 6 (N4): revokes a restart cut short, whose step 3 waits for reconcileHosts.
|
||||
@@ -419,13 +422,14 @@ export class Broker {
|
||||
#clock = 0;
|
||||
#processing;
|
||||
// credentialStatus is the runtime's metadata-only Credentials.status(); the broker never gets use().
|
||||
constructor({ store, businesses, secretCheck = () => {}, credentialStatus = () => [], organization = null, project = null, library = null, usage = null, limits = null, harnessOn = null, credentialVerbs = null, hostGrant = null, hostKeys = null, processing = null, testerOn = null, roleFile = null, giteaGrant = null, giteaKeys = null, access = null }) {
|
||||
constructor({ store, businesses, secretCheck = () => {}, credentialStatus = () => [], organization = null, project = null, library = null, usage = null, limits = null, harnessOn = null, credentialVerbs = null, hostGrant = null, hostKeys = null, processing = null, testerOn = null, roleFile = null, giteaGrant = null, giteaKeys = null, access = null, compactions = null }) {
|
||||
if (!object(businesses) || !Object.keys(businesses).length) fail('invalid-business');
|
||||
// Row 194 b15 (sage-2501): the function-argument checks come before anything reads the store.
|
||||
if (harnessOn !== null && typeof harnessOn !== 'function') throw new TypeError('harnessOn must be a function');
|
||||
if (testerOn !== null && typeof testerOn !== 'function') throw new TypeError('testerOn must be a function');
|
||||
if (roleFile !== null && typeof roleFile !== 'function') throw new TypeError('roleFile must be a function');
|
||||
if (access !== null && typeof access !== 'function') throw new TypeError('access must be a function');
|
||||
if (compactions !== null && typeof compactions !== 'function') throw new TypeError('compactions must be a function');
|
||||
this.#businesses = structuredClone(businesses);
|
||||
this.#store = store;
|
||||
// A Broker is a trusted host startup, including in-process hosts. Never repair an existing state as off.
|
||||
@@ -445,6 +449,7 @@ export class Broker {
|
||||
this.#giteaGrant = giteaGrant;
|
||||
this.#giteaKeys = giteaKeys;
|
||||
this.#access = access;
|
||||
this.#compactions = compactions;
|
||||
this.#project = project;
|
||||
this.#library = library;
|
||||
this.#usage = usage;
|
||||
@@ -610,6 +615,108 @@ export class Broker {
|
||||
if (!this.#store.get('SELECT 1 FROM sessions WHERE id=? AND thread=?', identifier(session), t.id)) fail('session-not-found');
|
||||
return session;
|
||||
}
|
||||
// Row 271 (264d, #1776, D7): a clear of a session is one op, its rows in session_ops. The op is
|
||||
// open while its latest row is requested, drained, uncertain or exited (session_ops_one_open).
|
||||
#openOp(session) {
|
||||
return this.#store.get(
|
||||
`SELECT o.* FROM session_ops o WHERE o.session=? AND o.seq=(SELECT max(seq) FROM session_ops WHERE thread=o.thread AND op=o.op)
|
||||
AND o.stage IN ('requested','drained','uncertain','exited')`,
|
||||
session,
|
||||
);
|
||||
}
|
||||
// active, clearing, uncertain or retired, with the op that made it so.
|
||||
#sessionState(session) {
|
||||
const retired = this.#store.get("SELECT op, successor FROM session_ops WHERE session=? AND stage='retired'", session);
|
||||
if (retired) return { state: 'retired', op: retired.op, successor: retired.successor };
|
||||
const open = this.#openOp(session);
|
||||
if (!open) return { state: 'active', op: null, successor: null };
|
||||
return { state: open.stage === 'uncertain' ? 'uncertain' : 'clearing', op: open.op, successor: null, open };
|
||||
}
|
||||
// The fence: no send, open or claim reaches a session being cleared, or a retired one.
|
||||
#sessionFence(session) {
|
||||
const { state } = this.#sessionState(session);
|
||||
if (state === 'retired') fail('session-retired');
|
||||
if (state !== 'active') fail('session-clearing');
|
||||
}
|
||||
// A session's revision: 0 at creation, plus one for each row of its clears.
|
||||
#revision(session) {
|
||||
return this.#store.get('SELECT count(*) AS n FROM session_ops WHERE session=?', session).n;
|
||||
}
|
||||
// A name a session has, or one a clear has bound for its successor.
|
||||
#nameTaken(thread, sessionName) {
|
||||
return Boolean(
|
||||
this.#store.get('SELECT 1 FROM sessions WHERE thread=? AND name=?', thread, sessionName) ??
|
||||
this.#store.get('SELECT 1 FROM session_ops WHERE thread=? AND successor_name=?', thread, sessionName),
|
||||
);
|
||||
}
|
||||
// The successor's id, bound at accept so a repeat finds it: a lowercase RFC 9562 version 8 UUID
|
||||
// from a sha256 of the op, which every session selector takes (cli/src/thread.mjs:41, gizmoduck-0274).
|
||||
#successorId(business, thread, session, op) {
|
||||
const h = createHash('sha256').update(JSON.stringify([business, thread, session, op])).digest('hex').split('');
|
||||
h[12] = '8';
|
||||
h[16] = '89ab'[parseInt(h[16], 16) & 3];
|
||||
const x = h.join('');
|
||||
return `${x.slice(0, 8)}-${x.slice(8, 12)}-${x.slice(12, 16)}-${x.slice(16, 20)}-${x.slice(20, 32)}`;
|
||||
}
|
||||
// A clear verb's payload with its caller, for the repeat and conflict checks.
|
||||
#opDigest(s, ...payload) {
|
||||
return createHash('sha256').update(JSON.stringify([this.#owner(s), ...payload])).digest('hex');
|
||||
}
|
||||
// One session_ops row. `op` is the op's requested row, or the fields it is made from.
|
||||
#appendOp(op, stage, { attempt = 0, workAttempt = null, run = null, generation = null, digest = null, refusal = null } = {}) {
|
||||
this.#store.run(
|
||||
'INSERT INTO session_ops(at,business,thread,session,op,stage,attempt,work_attempt,successor,successor_name,run,generation,digest,refusal) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?)',
|
||||
this.#now(),
|
||||
op.business,
|
||||
op.thread,
|
||||
op.session,
|
||||
op.op,
|
||||
stage,
|
||||
attempt,
|
||||
workAttempt,
|
||||
op.successor,
|
||||
op.successor_name,
|
||||
run,
|
||||
generation,
|
||||
digest,
|
||||
refusal,
|
||||
);
|
||||
}
|
||||
#opRow(thread, op, stage) {
|
||||
return this.#store.get('SELECT * FROM session_ops WHERE thread=? AND op=? AND stage=? ORDER BY seq LIMIT 1', thread, op, stage);
|
||||
}
|
||||
// The answer to every clear verb, from the op's rows (BUILD.md's contract for 264e).
|
||||
#clearEnvelope(thread, op) {
|
||||
const rows = this.#store.all('SELECT * FROM session_ops WHERE thread=? AND op=? ORDER BY seq', thread, op);
|
||||
if (!rows.length) fail('operation-not-found');
|
||||
const has = (stage) => rows.some((r) => r.stage === stage);
|
||||
const last = rows.at(-1);
|
||||
const start = rows.filter((r) => r.stage.startsWith('start-')).at(-1) ?? null;
|
||||
const state = has('cancelled') ? 'cancelled' : has('failed') ? 'failed' : has('retired') ? 'retired' : last.stage === 'uncertain' ? 'uncertain' : 'clearing';
|
||||
const refusal = state === 'failed' ? rows.find((r) => r.stage === 'failed').refusal : start?.stage === 'start-refused' ? start.refusal : null;
|
||||
return {
|
||||
op,
|
||||
thread,
|
||||
session: rows[0].session,
|
||||
successor: rows[0].successor,
|
||||
state,
|
||||
start: start ? start.stage.slice('start-'.length) : null,
|
||||
refusal,
|
||||
...(state === 'cancelled' ? { retired: has('retired') ? true : null } : {}),
|
||||
at: last.at,
|
||||
};
|
||||
}
|
||||
// D7 step 1: the session's undelivered inputs are cancelled, each with a receipt, never replayed.
|
||||
#cancelInputs(op, instance) {
|
||||
const rows = this.#store.all(
|
||||
`SELECT m.id FROM messages m JOIN message_threads mt ON mt.message=m.id WHERE mt.session=? AND m.to_role=?
|
||||
AND NOT EXISTS(SELECT 1 FROM deliveries d WHERE d.message=m.id AND d.op IN ('delivered','read'))
|
||||
AND NOT EXISTS(SELECT 1 FROM message_cancels c WHERE c.message=m.id) ORDER BY m.seq`,
|
||||
op.session,
|
||||
instance,
|
||||
);
|
||||
for (const m of rows) this.#store.run('INSERT INTO message_cancels(at,message,session,op) VALUES(?,?,?,?)', this.#now(), m.id, op.session, op.op);
|
||||
}
|
||||
// Row 155 (#1649): the held claims of every session of a thread instance, oldest thread first, or
|
||||
// only the threads `owner` made. An end of the instance ends each of them.
|
||||
#threadHolders(business, instance, owner = null) {
|
||||
@@ -763,6 +870,15 @@ export class Broker {
|
||||
if (s.tester && this.#store.get('SELECT owner FROM threads WHERE id=?', thread).owner !== this.#owner(s)) fail('thread-not-owned');
|
||||
return thread;
|
||||
}
|
||||
#compactionCount(t, session, legacy) {
|
||||
const unknown = (reason) => ({ count: null, known: false, reason, at: null });
|
||||
if (!this.#compactions) return unknown('not-collected');
|
||||
try {
|
||||
return this.#compactions({ business: t.business, thread: t.id, session, legacy });
|
||||
} catch {
|
||||
return unknown('store-unreadable');
|
||||
}
|
||||
}
|
||||
#thread(s, id) {
|
||||
const t = this.#threadRow(s, id);
|
||||
// REQ-USER-2: a user reaches only their own threads.
|
||||
@@ -939,15 +1055,158 @@ export class Broker {
|
||||
this.#store.transaction(() => this.#claimEnd(session, this.#own(session), 'release', null));
|
||||
return { released: true };
|
||||
}
|
||||
// Row 271 (264d, #1776, D7 steps 3-5): the business's clears still in the launcher's hands, as of
|
||||
// each op's latest row. An open op waits for its drained run's stop and exit proof, then its
|
||||
// retirement. A retired op whose successor hasn't started, and wasn't cancelled, waits for its
|
||||
// start through D8. `run` and `generation` are the drained row's. Trusted launcher API, not a
|
||||
// socket verb.
|
||||
sessionClears(business) {
|
||||
this.#business(identifier(business));
|
||||
return this.#store
|
||||
.all(
|
||||
`SELECT o.*, t.instance FROM session_ops o JOIN threads t ON t.id=o.thread
|
||||
WHERE o.business=? AND o.seq=(SELECT max(seq) FROM session_ops WHERE thread=o.thread AND op=o.op)
|
||||
AND o.stage IN ('requested','drained','uncertain','exited','retired','start-held','start-refused')
|
||||
ORDER BY o.seq`,
|
||||
business,
|
||||
)
|
||||
.map((o) => {
|
||||
const drained = this.#opRow(o.thread, o.op, 'drained');
|
||||
return {
|
||||
thread: o.thread,
|
||||
instance: o.instance,
|
||||
session: o.session,
|
||||
op: o.op,
|
||||
stage: o.stage,
|
||||
run: drained?.run ?? null,
|
||||
generation: drained?.generation ?? null,
|
||||
successor: o.successor,
|
||||
refusal: o.refusal,
|
||||
};
|
||||
});
|
||||
}
|
||||
// The requested row of a clear the launcher names, in the business it names.
|
||||
#launcherOp(record, extra = []) {
|
||||
keys(record, ['business', 'thread', 'op', ...extra], ['business', 'thread', 'op', ...extra]);
|
||||
this.#business(identifier(record.business));
|
||||
const first = this.#opRow(identifier(record.thread), identifier(record.op), 'requested');
|
||||
if (!first || first.business !== record.business) fail('operation-not-found');
|
||||
return first;
|
||||
}
|
||||
// D7 step 3: the exit proof, from the launcher. The drained run's own close handler wrote its
|
||||
// session.ended (endLaunch), which is checked here; a signal, a revoked claim or a dead socket
|
||||
// is not proof. With no run, the proof is the launcher's: it took the session's workspace lock,
|
||||
// and its registry holds no run of the session. Trusted launcher API.
|
||||
sessionClearExited(record) {
|
||||
const first = this.#launcherOp(record, ['run']);
|
||||
if (record.run !== null) identifier(record.run);
|
||||
return this.#store.transaction(() => {
|
||||
const open = this.#openOp(first.session);
|
||||
if (open?.op !== first.op) fail('operation-not-open');
|
||||
if (open.stage === 'exited') return this.#clearEnvelope(first.thread, first.op);
|
||||
const drained = this.#opRow(first.thread, first.op, 'drained') ?? fail('not-drained');
|
||||
if (record.run !== drained.run) fail('not-drained-run');
|
||||
if (drained.run !== null && !this.#store.get("SELECT 1 FROM events WHERE business=? AND kind='session.ended' AND subject=?", first.business, drained.run))
|
||||
fail('run-not-ended');
|
||||
this.#appendOp(first, 'exited', { run: drained.run, generation: drained.generation });
|
||||
return this.#clearEnvelope(first.thread, first.op);
|
||||
});
|
||||
}
|
||||
// D7 step 3: 30 s after the stop with no proof, once. Nothing else is signalled or forced.
|
||||
sessionClearUncertain(record) {
|
||||
const first = this.#launcherOp(record);
|
||||
return this.#store.transaction(() => {
|
||||
const open = this.#openOp(first.session);
|
||||
if (open?.op !== first.op) fail('operation-not-open');
|
||||
if (open.stage === 'drained') this.#appendOp(first, 'uncertain');
|
||||
else if (open.stage !== 'uncertain') fail('not-drained');
|
||||
return this.#clearEnvelope(first.thread, first.op);
|
||||
});
|
||||
}
|
||||
// D7 step 4: retirement and the successor, in one transaction, after the exit proof. Owner, grant
|
||||
// and halt are rechecked here (N4). A halt refuses and writes nothing, so the launcher asks again
|
||||
// once processing resumes; a lost grant ends the clear `failed`. Only the proven run may still hold
|
||||
// the claim, and its claim ends here only if endRun hasn't ended it (N5).
|
||||
sessionClearRetire(record) {
|
||||
const first = this.#launcherOp(record);
|
||||
return this.#store.transaction(() => {
|
||||
const open = this.#openOp(first.session);
|
||||
if (open?.op !== first.op || open.stage !== 'exited') fail('not-exited');
|
||||
this.#processingWork();
|
||||
const t = this.#store.get('SELECT * FROM threads WHERE id=?', first.thread);
|
||||
const refusal = this.#ownerRefusal(t);
|
||||
if (refusal) {
|
||||
this.#appendOp(first, 'failed', { refusal });
|
||||
return this.#clearEnvelope(first.thread, first.op);
|
||||
}
|
||||
const drained = this.#opRow(first.thread, first.op, 'drained');
|
||||
const h = this.#sessionHolder(first.session);
|
||||
if (h?.op === 'claim') {
|
||||
if (h.holder_run !== drained.run) fail('thread-already-held');
|
||||
this.#claimEnd({ business: t.business }, h, 'release', null, null, 'session-clear');
|
||||
}
|
||||
const wasDefault = this.#defaultSession(t.id) === first.session;
|
||||
this.#appendOp(first, 'retired');
|
||||
const at = this.#now();
|
||||
this.#store.run(
|
||||
'INSERT INTO sessions(id,at,business,thread,instance,owner,name,op,predecessor) VALUES(?,?,?,?,?,?,?,?,?)',
|
||||
first.successor,
|
||||
at,
|
||||
t.business,
|
||||
t.id,
|
||||
t.instance,
|
||||
t.owner,
|
||||
first.successor_name,
|
||||
first.op,
|
||||
first.session,
|
||||
);
|
||||
// D7 step 5: the default moves only off the session that was the default.
|
||||
if (wasDefault) this.#store.run('INSERT INTO session_defaults(at,thread,session,by) VALUES(?,?,?,?)', at, t.id, first.successor, t.owner);
|
||||
return this.#clearEnvelope(first.thread, first.op);
|
||||
});
|
||||
}
|
||||
// D7 step 5, D8: the successor's start, as the launcher's check()/start() decided it. Each report
|
||||
// takes the next counter (C1). A held start spends no work attempt; a refused one spends one.
|
||||
sessionClearStart(record) {
|
||||
keys(record, ['business', 'thread', 'op', 'start', 'refusal'], ['business', 'thread', 'op', 'start']);
|
||||
const { start, refusal = null } = record;
|
||||
const first = this.#launcherOp({ business: record.business, thread: record.thread, op: record.op });
|
||||
if (!['held', 'started', 'refused'].includes(start)) fail('invalid-request');
|
||||
if (start === 'started' ? refusal !== null : !id(refusal)) fail('invalid-request');
|
||||
return this.#store.transaction(() => {
|
||||
if (!this.#opRow(first.thread, first.op, 'retired')) fail('not-retired');
|
||||
if (this.#opRow(first.thread, first.op, 'cancelled')) fail('session-clear-cancelled');
|
||||
if (this.#opRow(first.thread, first.op, 'start-started')) fail('session-clear-complete');
|
||||
const { reports, refused } = this.#store.get(
|
||||
"SELECT count(*) AS reports, count(*) FILTER (WHERE stage='start-refused') AS refused FROM session_ops WHERE thread=? AND op=? AND stage GLOB 'start-*'",
|
||||
first.thread,
|
||||
first.op,
|
||||
);
|
||||
this.#appendOp(first, 'start-' + start, { attempt: reports + 1, workAttempt: refused + 1, refusal });
|
||||
return this.#clearEnvelope(first.thread, first.op);
|
||||
});
|
||||
}
|
||||
// N4: why the thread's owner may no longer clear it, or null. A user's grants are read again at
|
||||
// session.clear's level (collaborator, not a viewer verb); the business human and the tester own
|
||||
// their threads for as long as the business names them.
|
||||
#ownerRefusal(t) {
|
||||
if (t.owner.startsWith('user:')) {
|
||||
const s = { business: t.business, user: t.owner.slice('user:'.length) };
|
||||
return this.#granted({ ...s, grants: this.#userGrants(s) }, 'session.clear', t.project, t.instance) ? null : 'access-refused';
|
||||
}
|
||||
return t.owner === 'tester' || t.owner === this.#business(t.business).human ? null : 'thread-not-owned';
|
||||
}
|
||||
// Row 269 (264b, #1774): one row per session of each thread, oldest thread first, then oldest
|
||||
// session. `legacy` is a session the 3k-3l migration made (op legacy-3l), whose store and
|
||||
// in-sandbox workspace path are the thread's old ones.
|
||||
// in-sandbox workspace path are the thread's old ones. Row 271 (264d): a retired session has
|
||||
// none, so nothing starts or idles it, and a cancelled message isn't pending.
|
||||
#threadRows(where, ...params) {
|
||||
return this.#store
|
||||
.all(
|
||||
`SELECT t.id, t.instance, se.id AS session, se.op AS session_op,
|
||||
(SELECT count(*) FROM messages m JOIN message_threads mt ON mt.message=m.id WHERE mt.session=se.id AND m.to_role=t.instance
|
||||
AND NOT EXISTS(SELECT 1 FROM deliveries d WHERE d.message=m.id AND d.op IN ('delivered','read'))) AS pending,
|
||||
AND NOT EXISTS(SELECT 1 FROM deliveries d WHERE d.message=m.id AND d.op IN ('delivered','read'))
|
||||
AND NOT EXISTS(SELECT 1 FROM message_cancels x WHERE x.message=m.id)) AS pending,
|
||||
(SELECT count(*) FROM messages m JOIN message_threads mt ON mt.message=m.id WHERE mt.session=se.id AND m.to_role=t.instance
|
||||
AND c.op='claim' AND EXISTS(SELECT 1 FROM deliveries d WHERE d.message=m.id AND d.op='delivered' AND d.holder_run=c.holder_run)
|
||||
AND NOT EXISTS(SELECT 1 FROM deliveries d WHERE d.message=m.id AND d.op='read')) AS unread,
|
||||
@@ -956,7 +1215,7 @@ export class Broker {
|
||||
c.at AS claim_at, c.op AS claim_op, c.holder_run AS holder_run
|
||||
FROM threads t JOIN sessions se ON se.thread=t.id
|
||||
LEFT JOIN thread_claims c ON c.seq=(SELECT max(seq) FROM thread_claims WHERE session=se.id)
|
||||
WHERE ${where} ORDER BY t.seq, se.seq`,
|
||||
WHERE ${where} AND NOT EXISTS(SELECT 1 FROM session_ops o WHERE o.session=se.id AND o.stage='retired') ORDER BY t.seq, se.seq`,
|
||||
...params,
|
||||
)
|
||||
.map((r) => ({
|
||||
@@ -1054,6 +1313,10 @@ export class Broker {
|
||||
const h = this.#own(session);
|
||||
const released = h?.op === 'claim' && h.holder_run === session.run;
|
||||
if (released) this.#claimEnd(session, h, 'release', null);
|
||||
// Row 271 (264d, D7 step 2): the holder of a session being cleared that ends before its next
|
||||
// receive leaves no turn running either. Its exit is the drain, recorded with its generation.
|
||||
const open = released && session.thread ? this.#openOp(session.session) : null;
|
||||
if (open?.stage === 'requested') this.#appendOp(open, 'drained', { run: session.run, generation: h.generation });
|
||||
// Row 125 (#1619, REQ-LIMIT-4): a run a limit stopped carries the claim row's reason.
|
||||
const limit = this.#store.get(
|
||||
`SELECT reason FROM ${session.thread ? 'thread_claims' : 'role_claims'} WHERE business=? AND holder_run=? AND by='limit-check' ORDER BY seq DESC LIMIT 1`,
|
||||
@@ -4891,6 +5154,8 @@ export class Broker {
|
||||
if (s.thread) {
|
||||
if (this.#store.get("SELECT 1 FROM thread_claims WHERE session=? AND holder_run=? AND op='revoke' LIMIT 1", s.session, s.run))
|
||||
fail('run-revoked');
|
||||
// Row 271 (264d, B2): no run takes a session being cleared, or a retired one.
|
||||
this.#sessionFence(s.session);
|
||||
if (this.#sessionHolder(s.session)?.op === 'claim') fail('thread-already-held');
|
||||
this.#store.run(
|
||||
"INSERT INTO thread_claims(at,business,thread,session,generation,op,holder_run,harness,address,by) VALUES(?,?,?,?,1+(SELECT count(*) FROM thread_claims WHERE session=? AND op='claim'),?,?,?,?,?)",
|
||||
@@ -5309,10 +5574,12 @@ export class Broker {
|
||||
a.corrects ?? null,
|
||||
a.body,
|
||||
);
|
||||
this.#route(s.business, mid, to, thread);
|
||||
const cancelled = this.#route(s.business, mid, to, thread);
|
||||
const request = s.human
|
||||
? this.#event(s, 'human.input', { kind: 'instruction', message: mid }, mid)
|
||||
: null;
|
||||
// Row 271 (264d, B1): kept as evidence and cancelled at arrival, so it waits for nobody.
|
||||
if (cancelled) return { id: mid, request, cancelled };
|
||||
const waiting = this.#waiting(s, mid, to, thread);
|
||||
return waiting ? { id: mid, request, waiting } : { id: mid, request };
|
||||
}
|
||||
@@ -5323,10 +5590,21 @@ export class Broker {
|
||||
this.#agent(s);
|
||||
this.#received.add(s.business + ':' + s.run);
|
||||
}
|
||||
// Row 271 (264d, D7 step 2): a thread run's receive comes at a completed turn with no child
|
||||
// (the runner asks it only then). At a clear of its session it records the drain once and
|
||||
// delivers nothing. The run keeps its claim, so no turn starts before the launcher's stop.
|
||||
if (s.thread) {
|
||||
const open = this.#openOp(s.session);
|
||||
if (open) {
|
||||
if (!this.#opRow(open.thread, open.op, 'drained')) this.#appendOp(open, 'drained', { run: s.run, generation: this.#runGeneration(s) });
|
||||
return [];
|
||||
}
|
||||
}
|
||||
const role = s.human ? 'human' : s.role;
|
||||
// A thread run receives its own session's messages only (REQ-THREAD-1, row 264a).
|
||||
const rows = this.#store.all(
|
||||
"SELECT * FROM messages m WHERE business=? AND to_role=? AND NOT EXISTS(SELECT 1 FROM deliveries d WHERE d.message=m.id AND d.op IN ('delivered','read'))" +
|
||||
' AND NOT EXISTS(SELECT 1 FROM message_cancels c WHERE c.message=m.id)' +
|
||||
(s.thread ? ' AND EXISTS(SELECT 1 FROM message_threads mt WHERE mt.message=m.id AND mt.session=?)' : '') +
|
||||
' ORDER BY seq LIMIT 100',
|
||||
s.business,
|
||||
@@ -5852,6 +6130,8 @@ export class Broker {
|
||||
keys(a, ['thread', 'session'], ['thread']);
|
||||
const t = this.#grantedThread(s, 'thread.open', this.#thread(s, a.thread));
|
||||
const session = this.#threadSession(t, a.session);
|
||||
// Row 271 (264d, D7 step 1): a session being cleared, or a retired one, isn't opened.
|
||||
this.#sessionFence(session);
|
||||
this.#store.run('INSERT INTO thread_opens(at,thread,session,by) VALUES(?,?,?,?)', this.#now(), t.id, session, this.#owner(s));
|
||||
return { opened: t.id };
|
||||
}
|
||||
@@ -5862,6 +6142,8 @@ export class Broker {
|
||||
keys(a, ['thread', 'session', 'body', 'class'], ['thread', 'body']);
|
||||
const t = this.#grantedThread(s, 'thread.send', this.#thread(s, a.thread));
|
||||
const session = this.#threadSession(t, a.session);
|
||||
// Row 271 (264d, D7 step 1): nor sent to, by name or as the default.
|
||||
this.#sessionFence(session);
|
||||
// Row 120's lists hold here as on message.send: the instance must hear from the human.
|
||||
this.#checkMessaging(s.business, 'human', t.instance);
|
||||
string(a.body, 32768);
|
||||
@@ -5939,14 +6221,28 @@ export class Broker {
|
||||
}
|
||||
// Row 264a (#1769, T1 item 7): a thread's sessions, oldest first, and which one is the default.
|
||||
// Each user reaches only their own threads' sessions (REQ-USER-2); there are no shares (Q7).
|
||||
// Row 270 (264c, D6): each row's compaction count, from the host's ledger reader (the bus
|
||||
// host's seat compactions.mjs; this file is mounted in the sandbox and can't import it).
|
||||
// Without a reader the count is unknown, never 0.
|
||||
case 'session.list': {
|
||||
if (!s.tester) this.#human(s);
|
||||
keys(a, ['thread'], ['thread']);
|
||||
const t = this.#thread(s, a.thread);
|
||||
const fallback = this.#defaultSession(t.id);
|
||||
// Row 271 (264d): each row's clear state, revision and, once retired, successor.
|
||||
return this.#store
|
||||
.all('SELECT id,at,name,predecessor FROM sessions WHERE thread=? ORDER BY seq', t.id)
|
||||
.map((r) => ({ ...r, default: r.id === fallback }));
|
||||
.all('SELECT id,at,name,predecessor,op FROM sessions WHERE thread=? ORDER BY seq', t.id)
|
||||
.map(({ op, ...r }) => {
|
||||
const { state, successor } = this.#sessionState(r.id);
|
||||
return {
|
||||
...r,
|
||||
default: r.id === fallback,
|
||||
compactions: this.#compactionCount(t, r.id, op === 'legacy-3l'),
|
||||
state,
|
||||
revision: this.#revision(r.id),
|
||||
successor,
|
||||
};
|
||||
});
|
||||
}
|
||||
// D8: a new session is metadata only. Its run starts when the launcher starts it (264b).
|
||||
// The op key makes a retry find the first row: the same op and name answer it, another name
|
||||
@@ -5957,14 +6253,80 @@ export class Broker {
|
||||
if (!name(a.name)) fail('invalid-request');
|
||||
identifier(a.op);
|
||||
const t = this.#thread(s, a.thread);
|
||||
// Row 271 (264d, B3): a clear's op is the clear's, and its successor takes it.
|
||||
if (this.#store.get('SELECT 1 FROM session_ops WHERE thread=? AND op=?', t.id, a.op)) fail('session-op-conflict');
|
||||
const old = this.#store.get('SELECT id,name FROM sessions WHERE thread=? AND op=?', t.id, a.op);
|
||||
if (old) {
|
||||
if (old.name !== a.name) fail('session-op-conflict');
|
||||
return { id: old.id, thread: t.id, name: old.name };
|
||||
}
|
||||
if (this.#store.get('SELECT 1 FROM sessions WHERE thread=? AND name=?', t.id, a.name)) fail('session-exists');
|
||||
if (this.#nameTaken(t.id, a.name)) fail('session-exists');
|
||||
return { id: this.#newSession(s, t, a.name, a.op, false), thread: t.id, name: a.name };
|
||||
}
|
||||
// Row 271 (264d, #1776, D7): clear context. This accept binds the successor, fences the session
|
||||
// and cancels its undelivered inputs in one transaction. The drain (message.receive or
|
||||
// endLaunch), the stop and proof (the launcher), retirement and the start follow. The answer is
|
||||
// the op's envelope, and a repeat of the same payload answers it again.
|
||||
case 'session.clear': {
|
||||
if (!s.tester) this.#human(s);
|
||||
keys(a, ['thread', 'session', 'revision', 'confirm', 'op'], ['thread', 'session', 'revision', 'confirm', 'op']);
|
||||
identifier(a.op);
|
||||
if (!Number.isSafeInteger(a.revision) || a.revision < 0) fail('invalid-request');
|
||||
const t = this.#thread(s, a.thread);
|
||||
const session = this.#threadSession(t, a.session);
|
||||
if (a.confirm !== true) fail('confirm-required');
|
||||
const digest = this.#opDigest(s, 'session.clear', t.id, session, a.revision, a.op);
|
||||
const first = this.#opRow(t.id, a.op, 'requested');
|
||||
if (first) {
|
||||
if (first.digest !== digest) fail('session-op-conflict');
|
||||
return this.#clearEnvelope(t.id, a.op);
|
||||
}
|
||||
// B3: an op names one thing in its thread, a session or a clear.
|
||||
if (this.#store.get('SELECT 1 FROM sessions WHERE thread=? AND op=?', t.id, a.op)) fail('session-op-conflict');
|
||||
this.#processingWork();
|
||||
this.#sessionFence(session);
|
||||
if (this.#revision(session) !== a.revision) fail('session-revision-stale');
|
||||
// Q4 (sage-2826): `<old>-N`, the next N free in the thread.
|
||||
const base = this.#store.get('SELECT name FROM sessions WHERE id=?', session).name;
|
||||
let n = 1;
|
||||
while (this.#nameTaken(t.id, `${base}-${n}`)) n++;
|
||||
if (!name(`${base}-${n}`)) fail('session-name-too-long');
|
||||
const op = { business: s.business, thread: t.id, session, op: a.op, successor: this.#successorId(s.business, t.id, session, a.op), successor_name: `${base}-${n}` };
|
||||
this.#appendOp(op, 'requested', { digest });
|
||||
this.#cancelInputs(op, t.instance);
|
||||
// With no holder there is no turn to wait for, and B2 refuses a claim from here on.
|
||||
if (this.#sessionHolder(session)?.op !== 'claim') this.#appendOp(op, 'drained');
|
||||
return this.#clearEnvelope(t.id, a.op);
|
||||
}
|
||||
// Row 271 (sage-2828, D7:263-264): a human's cancel of a clear. Before retirement it closes the
|
||||
// op, which releases the fence: the claim and the cancelled messages stay as they are, and no
|
||||
// successor is made. After retirement it ends the startup only, and both sessions stay. After
|
||||
// the successor started, or a failed clear, there is nothing to cancel.
|
||||
case 'session.cancel': {
|
||||
if (!s.tester) this.#human(s);
|
||||
keys(a, ['thread', 'op', 'confirm'], ['thread', 'op', 'confirm']);
|
||||
identifier(a.op);
|
||||
const t = this.#thread(s, a.thread);
|
||||
if (a.confirm !== true) fail('confirm-required');
|
||||
const first = this.#opRow(t.id, a.op, 'requested') ?? fail('operation-not-found');
|
||||
const digest = this.#opDigest(s, 'session.cancel', t.id, a.op);
|
||||
const done = this.#opRow(t.id, a.op, 'cancelled');
|
||||
if (done) {
|
||||
if (done.digest !== digest) fail('session-op-conflict');
|
||||
return this.#clearEnvelope(t.id, a.op);
|
||||
}
|
||||
if (this.#opRow(t.id, a.op, 'start-started') || this.#opRow(t.id, a.op, 'failed')) fail('session-clear-complete');
|
||||
this.#appendOp(first, 'cancelled', { digest });
|
||||
return this.#clearEnvelope(t.id, a.op);
|
||||
}
|
||||
// Row 271: a clear's envelope, for a caller that waited and reports pending. Reading writes
|
||||
// nothing, and a retry asks here rather than sending the clear again.
|
||||
case 'session.operation': {
|
||||
if (!s.tester) this.#human(s);
|
||||
keys(a, ['thread', 'op'], ['thread', 'op']);
|
||||
identifier(a.op);
|
||||
return this.#clearEnvelope(this.#thread(s, a.thread).id, a.op);
|
||||
}
|
||||
case 'replies': {
|
||||
if (!s.human && !s.reader) fail('human-required');
|
||||
keys(a, ['id'], ['id']);
|
||||
@@ -6093,12 +6455,21 @@ export class Broker {
|
||||
}
|
||||
// Row 121: a message names its thread (REQ-MSG-1) and goes to the holder that may have it.
|
||||
// Row 264a (#1769): it names one session of the thread, and the session's holder may have it.
|
||||
// Row 271 (264d, B1): a message to the instance in a session being cleared, or a retired one, is
|
||||
// kept and cancelled at arrival with a receipt. It is never delivered there or to the successor,
|
||||
// and the answer is why; otherwise null.
|
||||
#route(business, mid, to, session) {
|
||||
if (session) {
|
||||
const { thread } = this.#store.get('SELECT thread FROM sessions WHERE id=?', session) ?? fail('session-not-found');
|
||||
const { thread, instance } = this.#store.get('SELECT thread, instance FROM sessions WHERE id=?', session) ?? fail('session-not-found');
|
||||
this.#store.run('INSERT INTO message_threads(message,thread,session) VALUES(?,?,?)', mid, thread, session);
|
||||
const { state, op } = this.#sessionState(session);
|
||||
if (to === instance && state !== 'active') {
|
||||
this.#store.run('INSERT INTO message_cancels(at,message,session,op) VALUES(?,?,?,?)', this.#now(), mid, session, op);
|
||||
return state === 'retired' ? 'session-retired' : 'session-clearing';
|
||||
}
|
||||
}
|
||||
this.#delivery(mid, 'routed', this.#recipient(business, to, session));
|
||||
return null;
|
||||
}
|
||||
// Row 264a (#1769): a session row and, for a new thread, its default. The 3k-3l migration made the
|
||||
// legacy defaults, whose id is the thread id; a new session's id is random, and a trigger refuses
|
||||
@@ -6251,7 +6622,7 @@ export class Broker {
|
||||
#routeSession(session) {
|
||||
const h = this.#sessionHolder(session);
|
||||
const rows = this.#store.all(
|
||||
"SELECT m.id FROM messages m JOIN message_threads mt ON mt.message=m.id JOIN threads t ON t.id=mt.thread WHERE mt.session=? AND m.to_role=t.instance AND NOT EXISTS(SELECT 1 FROM deliveries WHERE message=m.id AND op IN ('delivered','read'))",
|
||||
"SELECT m.id FROM messages m JOIN message_threads mt ON mt.message=m.id JOIN threads t ON t.id=mt.thread WHERE mt.session=? AND m.to_role=t.instance AND NOT EXISTS(SELECT 1 FROM deliveries WHERE message=m.id AND op IN ('delivered','read')) AND NOT EXISTS(SELECT 1 FROM message_cancels WHERE message=m.id)",
|
||||
session,
|
||||
);
|
||||
for (const m of rows) this.#delivery(m.id, 'routed', h);
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
-- Row 271 (264d, #1776, T1 item 7 D7/D8): clear context. session_ops is the clear journal, one row
|
||||
-- per stage, and message_cancels holds a receipt for each message a clear cancelled. Only new
|
||||
-- tables, an index and their triggers.
|
||||
CREATE TABLE session_ops (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
thread TEXT NOT NULL REFERENCES threads(id),
|
||||
session TEXT NOT NULL REFERENCES sessions(id),
|
||||
op TEXT NOT NULL,
|
||||
stage TEXT NOT NULL CHECK (stage IN ('requested','drained','uncertain','exited','retired','failed','cancelled','start-held','start-started','start-refused')),
|
||||
attempt INTEGER NOT NULL CHECK (CASE WHEN stage GLOB 'start-*' THEN attempt >= 1 ELSE attempt = 0 END),
|
||||
work_attempt INTEGER CHECK (work_attempt IS NULL OR (stage GLOB 'start-*' AND work_attempt >= 1)),
|
||||
successor TEXT NOT NULL,
|
||||
successor_name TEXT NOT NULL,
|
||||
run TEXT,
|
||||
generation INTEGER CHECK (generation IS NULL OR generation >= 1),
|
||||
digest TEXT,
|
||||
refusal TEXT,
|
||||
UNIQUE (thread, op, stage, attempt)
|
||||
) STRICT;
|
||||
CREATE TABLE message_cancels (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
message TEXT NOT NULL UNIQUE REFERENCES messages(id),
|
||||
session TEXT NOT NULL REFERENCES sessions(id),
|
||||
op TEXT NOT NULL
|
||||
) STRICT;
|
||||
CREATE INDEX session_ops_session ON session_ops (session, seq);
|
||||
CREATE TRIGGER session_ops_no_update BEFORE UPDATE ON session_ops BEGIN SELECT RAISE(ABORT, 'session_ops is append-only'); END;
|
||||
CREATE TRIGGER session_ops_no_delete BEFORE DELETE ON session_ops BEGIN SELECT RAISE(ABORT, 'session_ops is append-only'); END;
|
||||
CREATE TRIGGER session_ops_no_replace BEFORE INSERT ON session_ops WHEN EXISTS (SELECT 1 FROM session_ops WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'session_ops is append-only'); END;
|
||||
CREATE TRIGGER session_ops_one_open BEFORE INSERT ON session_ops
|
||||
WHEN NEW.stage = 'requested' AND EXISTS (SELECT 1 FROM session_ops o WHERE o.session = NEW.session
|
||||
AND o.seq = (SELECT max(seq) FROM session_ops WHERE thread = o.thread AND op = o.op)
|
||||
AND o.stage IN ('requested','drained','uncertain','exited'))
|
||||
BEGIN SELECT RAISE(ABORT, 'session has an open clear'); END;
|
||||
CREATE TRIGGER message_cancels_no_update BEFORE UPDATE ON message_cancels BEGIN SELECT RAISE(ABORT, 'message_cancels is append-only'); END;
|
||||
CREATE TRIGGER message_cancels_no_delete BEFORE DELETE ON message_cancels BEGIN SELECT RAISE(ABORT, 'message_cancels is append-only'); END;
|
||||
CREATE TRIGGER message_cancels_no_replace BEFORE INSERT ON message_cancels WHEN EXISTS (SELECT 1 FROM message_cancels WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'message_cancels is append-only'); END;
|
||||
@@ -29,6 +29,11 @@ const LAUNCH_OPS = new Set([
|
||||
'lifecycleTake',
|
||||
'lifecycleDelete',
|
||||
'lifecycleDone',
|
||||
'sessionClears',
|
||||
'sessionClearExited',
|
||||
'sessionClearUncertain',
|
||||
'sessionClearRetire',
|
||||
'sessionClearStart',
|
||||
]);
|
||||
function launchOp(m) {
|
||||
if (m.op === 'identity') return runtime.identity(m.cap);
|
||||
@@ -64,6 +69,12 @@ function launchOp(m) {
|
||||
if (m.op === 'lifecycleTake') return runtime.lifecycleTake(m.record);
|
||||
if (m.op === 'lifecycleDelete') return runtime.lifecycleDelete(m.record);
|
||||
if (m.op === 'lifecycleDone') return runtime.lifecycleDone(m.record);
|
||||
// Row 271 (264d, #1776, D7 steps 3-5): the launcher's clear pass.
|
||||
if (m.op === 'sessionClears') return runtime.sessionClears(m.business);
|
||||
if (m.op === 'sessionClearExited') return runtime.sessionClearExited(m.record);
|
||||
if (m.op === 'sessionClearUncertain') return runtime.sessionClearUncertain(m.record);
|
||||
if (m.op === 'sessionClearRetire') return runtime.sessionClearRetire(m.record);
|
||||
if (m.op === 'sessionClearStart') return runtime.sessionClearStart(m.record);
|
||||
if (m.op === 'credentialStatus') return runtime.credentialStatus(m.business, m.role);
|
||||
// Every LAUNCH_OPS name has its line above. A missing one used to answer as credentialStatus,
|
||||
// whose [] for an unknown role read as "no holders" for landHolders (row 156 mutant P02).
|
||||
|
||||
@@ -9,6 +9,7 @@ import { Credentials } from './credentials.mjs';
|
||||
import { serve } from './server.mjs';
|
||||
import { launchSettings } from '../../business/src/launch-settings.mjs';
|
||||
import { readSessions } from '../../seat/src/session.mjs';
|
||||
import { compactionCount, storeDir } from '../../seat/src/compactions.mjs';
|
||||
import { publishRemote, workingRepo, WorkingError } from '../../seat/src/working.mjs';
|
||||
import { createLoader, hasModule } from '../../plugins/src/loader.mjs';
|
||||
import { pluginSettings } from '../../business/src/plugin-settings.mjs';
|
||||
@@ -261,6 +262,11 @@ export async function startBroker({
|
||||
broker = new Broker({
|
||||
// FED R3: the access file is read per user request; a refusal leaves the user no grant.
|
||||
access: access ? () => loadAccess(access.file, access.context) : null,
|
||||
// Row 270 (264c): session.list's compaction count, read from the session store's ledger.
|
||||
compactions: ({ business, thread, session, legacy }) => {
|
||||
const dir = storeDir(dataRoot, business, thread, session, { legacy });
|
||||
return dir ? compactionCount(dir) : { count: null, known: false, reason: 'store-unreadable', at: null };
|
||||
},
|
||||
hostGrant: runRoot ? hostGrant({ configDir: organization.dir, runRoot }) : null,
|
||||
// Row 235 (#1741, r6 X4, X5): the grant executor's claim, register and issue, never use().
|
||||
hostKeys: storeRoot ? grantKeys({ credentials, storeRoot }) : null,
|
||||
@@ -588,6 +594,12 @@ export async function startBroker({
|
||||
lifecycleTake: (r) => broker.lifecycleTake(r),
|
||||
lifecycleDelete: (r) => broker.lifecycleDelete(r),
|
||||
lifecycleDone: (r) => broker.lifecycleDone(r),
|
||||
// Row 271 (264d, #1776): the launcher's clear pass.
|
||||
sessionClears: (business) => broker.sessionClears(business),
|
||||
sessionClearExited: (r) => broker.sessionClearExited(r),
|
||||
sessionClearUncertain: (r) => broker.sessionClearUncertain(r),
|
||||
sessionClearRetire: (r) => broker.sessionClearRetire(r),
|
||||
sessionClearStart: (r) => broker.sessionClearStart(r),
|
||||
refuse: (cap, code, subject) => broker.refuse(cap, code, subject),
|
||||
credentialStatus,
|
||||
launches: bound,
|
||||
|
||||
@@ -13,8 +13,8 @@ import { join, isAbsolute } from 'node:path';
|
||||
import { createHash } from 'node:crypto';
|
||||
|
||||
const ddl = readFileSync(new URL('../schema.sql', import.meta.url), 'utf8');
|
||||
export const SCHEMA_SHA256 = 'd5ca9bb9268994b175ac79d38939be4bb7f2e80bc0d9ea273eedaae50c29b8a9';
|
||||
export const SCHEMA_VERSION = '3m';
|
||||
export const SCHEMA_SHA256 = 'a2ba3a46f12e56139ecae9e55227d00847842aedec50c1d92aaae28844ca48e7';
|
||||
export const SCHEMA_VERSION = '3n';
|
||||
// Ordered, named offline steps. New schema changes append a step and keep prior DDL pinned.
|
||||
const legacyDDL = readFileSync(new URL('./migrations/3b.sql', import.meta.url), 'utf8');
|
||||
const LEGACY_SHA256 = '179ffe356d4ff19a49b5ebad39b6c6bfd7771deb8e1b7c55b5e746f039d69e65';
|
||||
@@ -77,6 +77,11 @@ const migrations = [{
|
||||
from: '3l',
|
||||
to: '3m',
|
||||
sql: readFileSync(new URL('./migrations/3l-3m.sql', import.meta.url), 'utf8'),
|
||||
}, {
|
||||
// Row 271 (#1776): clear context. The clear journal and the cancelled-message receipts. Only new tables.
|
||||
from: '3m',
|
||||
to: '3n',
|
||||
sql: readFileSync(new URL('./migrations/3m-3n.sql', import.meta.url), 'utf8'),
|
||||
}];
|
||||
const timedTables = [
|
||||
'events',
|
||||
@@ -95,6 +100,8 @@ const timedTables = [
|
||||
'thread_shares',
|
||||
'instance_names',
|
||||
'usage_records',
|
||||
'session_ops',
|
||||
'message_cancels',
|
||||
];
|
||||
const hex = (value) => createHash('sha256').update(value).digest('hex');
|
||||
if (hex(ddl) !== SCHEMA_SHA256) throw Error('schema-source-mismatch');
|
||||
|
||||
@@ -18,7 +18,8 @@ const here = (p) => new URL(p, import.meta.url);
|
||||
// Row 264a (#1769): what the 3k→3l step rebuilds or adds. sessions.test.mjs tests those objects.
|
||||
const SESSIONS_3L = ['sessions', 'session_defaults', 'thread_claims', 'message_threads', 'thread_opens'];
|
||||
// Row 206 (#1704): and the 3l→3m step's thread_shares. thread-shares.test.mjs tests it.
|
||||
const later3l = (r) => [...SESSIONS_3L, 'thread_shares'].includes(r.tbl_name) || ['delivery_same_thread', 'delivery_same_session', 'threads_id_not_session'].includes(r.name);
|
||||
// Row 271 (#1776): and the 3m→3n step's session_ops and message_cancels. session-clear.test.mjs tests them.
|
||||
const later3l = (r) => [...SESSIONS_3L, 'thread_shares', 'session_ops', 'message_cancels'].includes(r.tbl_name) || ['delivery_same_thread', 'delivery_same_session', 'threads_id_not_session'].includes(r.name);
|
||||
function source3h(t) {
|
||||
const root = mkdtempSync(join(tmpdir(), 'account-migrate-'));
|
||||
t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
@@ -86,6 +87,7 @@ async function engine(root, name, { store, schema, step = (s) => s }) {
|
||||
copyFileSync(here('../src/migrations/3j-3k.sql'), join(moduleRoot, 'src/migrations/3j-3k.sql'));
|
||||
copyFileSync(here('../src/migrations/3k-3l.sql'), join(moduleRoot, 'src/migrations/3k-3l.sql'));
|
||||
copyFileSync(here('../src/migrations/3l-3m.sql'), join(moduleRoot, 'src/migrations/3l-3m.sql'));
|
||||
copyFileSync(here('../src/migrations/3m-3n.sql'), join(moduleRoot, 'src/migrations/3m-3n.sql'));
|
||||
return (await import(pathToFileURL(join(moduleRoot, 'src/store.mjs')))).Store;
|
||||
}
|
||||
|
||||
@@ -93,7 +95,7 @@ test('REQ-LIMIT-4 3h copy migrates to 3i: every row keeps its seq, the sequence
|
||||
const { root, path } = source3h(t), before = snapshot(path), was = master(path);
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
assert.equal(version(path), '3m');
|
||||
assert.equal(version(path), '3n');
|
||||
// Every table's rows as they were, usage_records with its seq values and sqlite_sequence with 50.
|
||||
assert.deepEqual(migrated3l(snapshot(path, before.columns).rows), before.rows);
|
||||
assert.deepEqual(before.rows.usage_records.map(r => r.seq), [1, 2, 3]);
|
||||
@@ -143,7 +145,7 @@ test('REQ-LIMIT-4 a 3h→3i step that fails partway leaves the store at 3h with
|
||||
assert.deepEqual(snapshot(path, before.columns).rows, before.rows);
|
||||
// Still a whole 3h store: the real step migrates it.
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
assert.equal(version(path), '3m');
|
||||
assert.equal(version(path), '3n');
|
||||
assert.deepEqual(migrated3l(snapshot(path, before.columns).rows), before.rows);
|
||||
});
|
||||
|
||||
@@ -155,14 +157,14 @@ test('REQ-LIMIT-4 the 3h engine refuses a migrated store, migrating or not, and
|
||||
schema: ddl3h,
|
||||
store: (source) => source
|
||||
.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3h).digest('hex')}';`)
|
||||
.replace("export const SCHEMA_VERSION = '3m';", "export const SCHEMA_VERSION = '3h';")
|
||||
.replace("export const SCHEMA_VERSION = '3n';", "export const SCHEMA_VERSION = '3h';")
|
||||
.replace(/\}, \{\n {2}\/\/ Row 126 \(#1620\)[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables'),
|
||||
});
|
||||
const pinned = readFileSync(join(root, 'engine-3h/src/store.mjs'), 'utf8');
|
||||
assert.ok(pinned.includes("export const SCHEMA_VERSION = '3h';") && !pinned.includes('3h-3i'), 'the 3h engine has no 3h→3i step');
|
||||
assert.throws(() => new Store3h(root), /schema-mismatch/);
|
||||
assert.throws(() => new Store3h(root, { migrateSchema: true }), /schema-mismatch/);
|
||||
assert.equal(version(path), '3m');
|
||||
assert.equal(version(path), '3n');
|
||||
assert.deepEqual(master(path), was);
|
||||
assert.deepEqual(snapshot(path, after.columns).rows, after.rows);
|
||||
});
|
||||
@@ -614,7 +614,7 @@ async function database3c(t) {
|
||||
let source = readFileSync(here('../src/store.mjs'), 'utf8');
|
||||
const pinned = source
|
||||
.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3c).digest('hex')}';`)
|
||||
.replace("export const SCHEMA_VERSION = '3m';", "export const SCHEMA_VERSION = '3c';")
|
||||
.replace("export const SCHEMA_VERSION = '3n';", "export const SCHEMA_VERSION = '3c';")
|
||||
.replace(/\}, \{\n {2}from: '3c',[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables');
|
||||
assert.notEqual(pinned, source);
|
||||
assert.ok(!pinned.includes('3c-3d'), 'the 3c engine has no 3c→3d step');
|
||||
@@ -638,7 +638,7 @@ test('REQ-ROLE-2, REQ-COS-1: the 3c→3d migration lets the chief of staff end a
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
const store = new Store(root, { migrateSchema: true });
|
||||
t.after(() => store.close());
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3m');
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3n');
|
||||
assert.equal(store.get("SELECT body FROM events WHERE id='preserved'").body, '{"fixture":true}');
|
||||
assert.equal(store.get("SELECT holder_run FROM role_claims WHERE role='coder'").holder_run, 'coder-run');
|
||||
const b = new Broker({ store, businesses: demo(), organization: null });
|
||||
|
||||
@@ -33,7 +33,7 @@ test('explicit 3b copy migration preserves evidence and opens as 3g; normal star
|
||||
const original = readFileSync(source);
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
const store = new Store(root, { migrateSchema: true });
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3m');
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3n');
|
||||
assert.equal(store.get("SELECT body FROM events WHERE id='preserved'").body, '{"fixture":true}');
|
||||
assert.equal(store.get('SELECT count(*) n FROM task_comment_cursors').n, 0);
|
||||
const epoch = store.get("SELECT value FROM meta WHERE key='comment_cursor_epoch'").value;
|
||||
@@ -75,15 +75,15 @@ test('named migration registry chains 3b through 3g to a fixture-only future ver
|
||||
const sha = createHash('sha256').update(futureDDL).digest('hex');
|
||||
let source = readFileSync(new URL('../src/store.mjs', import.meta.url), 'utf8');
|
||||
source = source.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${sha}';`)
|
||||
.replace("export const SCHEMA_VERSION = '3m';", "export const SCHEMA_VERSION = '3n';")
|
||||
.replace('}];\nconst timedTables', `}, { from: '3m', to: '3n', sql: ${JSON.stringify(futureSQL)} }];\nconst timedTables`);
|
||||
.replace("export const SCHEMA_VERSION = '3n';", "export const SCHEMA_VERSION = '3o';")
|
||||
.replace('}];\nconst timedTables', `}, { from: '3n', to: '3o', sql: ${JSON.stringify(futureSQL)} }];\nconst timedTables`);
|
||||
writeFileSync(join(moduleRoot, 'src/store.mjs'), source);
|
||||
writeFileSync(join(moduleRoot, 'schema.sql'), futureDDL);
|
||||
for (const name of ['3b.sql', '3b-3c.sql', '3c-3d.sql', '3d-3e.sql', '3e-3f.sql', '3f-3g.sql', '3g-3h.sql', '3h-3i.sql', '3i-3j.sql', '3j-3k.sql', '3k-3l.sql', '3l-3m.sql'])
|
||||
for (const name of ['3b.sql', '3b-3c.sql', '3c-3d.sql', '3d-3e.sql', '3e-3f.sql', '3f-3g.sql', '3g-3h.sql', '3h-3i.sql', '3i-3j.sql', '3j-3k.sql', '3k-3l.sql', '3l-3m.sql', '3m-3n.sql'])
|
||||
copyFileSync(new URL('../src/migrations/' + name, import.meta.url), join(moduleRoot, 'src/migrations', name));
|
||||
const { Store: FutureStore } = await import(pathToFileURL(join(moduleRoot, 'src/store.mjs')));
|
||||
const future = new FutureStore(root, { migrateSchema: true });
|
||||
assert.equal(future.get("SELECT value FROM meta WHERE key='schema_version'").value, '3n');
|
||||
assert.equal(future.get("SELECT value FROM meta WHERE key='schema_version'").value, '3o');
|
||||
assert.equal(future.get("SELECT name FROM sqlite_master WHERE name='future_fixture'").name, 'future_fixture');
|
||||
assert.equal(future.get("SELECT count(*) n FROM task_comment_cursors").n, 0);
|
||||
assert.equal(future.get("SELECT id FROM events").id, 'preserved');
|
||||
@@ -109,7 +109,7 @@ test('offline command migrates only the supplied fixture root and reports usage
|
||||
const command = new URL('../src/migrate-schema.mjs', import.meta.url).pathname;
|
||||
const migrated = spawnSync(process.execPath, [command, root], { encoding: 'utf8' });
|
||||
assert.equal(migrated.status, 0, migrated.stderr);
|
||||
assert.match(migrated.stdout, /bus schema 3m ready/);
|
||||
assert.match(migrated.stdout, /bus schema 3n ready/);
|
||||
new Store(root).close();
|
||||
const usage = spawnSync(process.execPath, [command], { encoding: 'utf8' });
|
||||
assert.equal(usage.status, 2);
|
||||
|
||||
+495
@@ -0,0 +1,495 @@
|
||||
PRAGMA journal_mode = WAL;
|
||||
PRAGMA foreign_keys = ON;
|
||||
CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL) STRICT;
|
||||
CREATE TABLE events (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
id TEXT NOT NULL UNIQUE,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
kind TEXT NOT NULL CHECK (kind IN (
|
||||
'session.launched',
|
||||
'session.ended',
|
||||
'action.allowed',
|
||||
'action.refused',
|
||||
'task.created',
|
||||
'task.assigned',
|
||||
'task.state',
|
||||
'task.closed',
|
||||
'task.changed.external',
|
||||
'task.conflict',
|
||||
'task.missing',
|
||||
'review.requested',
|
||||
'review.verdict',
|
||||
'human.input',
|
||||
'config.refused',
|
||||
'credential.expiring',
|
||||
'credential.expired',
|
||||
'credential.changed',
|
||||
'launch.revoked',
|
||||
'launch.restored',
|
||||
'digest.sent')),
|
||||
actor_role TEXT, actor_run TEXT,
|
||||
subject TEXT,
|
||||
corrects TEXT REFERENCES events(id),
|
||||
body TEXT NOT NULL CHECK (json_valid(body))
|
||||
) STRICT;
|
||||
CREATE TABLE role_claims (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL, role TEXT NOT NULL,
|
||||
op TEXT NOT NULL CHECK (op IN ('claim','release','revoke')),
|
||||
holder_run TEXT NOT NULL,
|
||||
harness TEXT NOT NULL, address TEXT,
|
||||
by TEXT NOT NULL, reason TEXT,
|
||||
decision TEXT
|
||||
) STRICT;
|
||||
CREATE TABLE decisions (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
id TEXT NOT NULL UNIQUE,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL, project TEXT,
|
||||
raised_by_role TEXT NOT NULL, raised_by_run TEXT NOT NULL,
|
||||
class TEXT NOT NULL CHECK (class IN ('routine','within-role','cross-role','gated')),
|
||||
action TEXT NOT NULL,
|
||||
route_to TEXT NOT NULL,
|
||||
question TEXT NOT NULL,
|
||||
options TEXT NOT NULL CHECK (json_valid(options) AND json_array_length(options) BETWEEN 2 AND 9),
|
||||
recommendation TEXT NOT NULL,
|
||||
task_ref TEXT CHECK (task_ref IS NULL OR (task_ref GLOB 'vikunja:[1-9]*/[1-9]*' AND NOT substr(task_ref, 9) GLOB '*[^0-9/]*' AND NOT substr(task_ref, 9) GLOB '*/*/*')), requirement_ref TEXT,
|
||||
blocking INTEGER NOT NULL CHECK (blocking IN (0,1)),
|
||||
supersedes TEXT REFERENCES decisions(id)
|
||||
, spec_path TEXT, spec_commit TEXT) STRICT;
|
||||
CREATE TABLE decision_events (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
decision TEXT NOT NULL REFERENCES decisions(id),
|
||||
at TEXT NOT NULL,
|
||||
op TEXT NOT NULL CHECK (op IN ('seen','resolved','withdrawn','expired')),
|
||||
by TEXT NOT NULL,
|
||||
choice TEXT, note TEXT, via TEXT
|
||||
) STRICT;
|
||||
CREATE TABLE messages (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
id TEXT NOT NULL UNIQUE,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
from_role TEXT NOT NULL, from_run TEXT NOT NULL,
|
||||
to_role TEXT NOT NULL,
|
||||
class TEXT NOT NULL,
|
||||
in_reply_to TEXT REFERENCES messages(id),
|
||||
decision TEXT REFERENCES decisions(id),
|
||||
corrects TEXT REFERENCES messages(id),
|
||||
body TEXT NOT NULL
|
||||
, to_name TEXT) STRICT;
|
||||
CREATE TABLE deliveries (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
message TEXT NOT NULL REFERENCES messages(id),
|
||||
at TEXT NOT NULL,
|
||||
op TEXT NOT NULL CHECK (op IN ('routed','delivered','failed','read')),
|
||||
holder_run TEXT, transport TEXT, address TEXT, detail TEXT
|
||||
) STRICT;
|
||||
CREATE TABLE task_snapshots (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
task_ref TEXT NOT NULL CHECK (task_ref GLOB 'vikunja:[1-9]*/[1-9]*' AND NOT substr(task_ref, 9) GLOB '*[^0-9/]*' AND NOT substr(task_ref, 9) GLOB '*/*/*'),
|
||||
updated TEXT NOT NULL,
|
||||
etag TEXT,
|
||||
digest TEXT NOT NULL CHECK (length(digest) = 64 AND NOT digest GLOB '*[^0-9a-f]*'),
|
||||
fields TEXT NOT NULL CHECK (json_valid(fields)),
|
||||
source TEXT NOT NULL CHECK (source IN ('self','poll')),
|
||||
via TEXT CHECK (via IN ('board','cursor','task','reconcile')),
|
||||
read_at TEXT,
|
||||
role TEXT, run TEXT,
|
||||
CHECK ((source = 'self') = (role IS NOT NULL AND run IS NOT NULL)),
|
||||
CHECK ((source = 'poll') = (via IS NOT NULL AND read_at IS NOT NULL)),
|
||||
CHECK (json_type(fields, '$.bucket') IS 'integer'
|
||||
OR (source IS 'poll' AND via IS 'task' AND json_type(fields, '$.gone') IS 'text'))
|
||||
) STRICT;
|
||||
CREATE INDEX task_snapshots_ref ON task_snapshots (business, task_ref, seq);
|
||||
CREATE TRIGGER decisions_resolve_once BEFORE INSERT ON decision_events
|
||||
WHEN NEW.op IN ('resolved','withdrawn','expired') AND EXISTS (
|
||||
SELECT 1 FROM decision_events WHERE decision = NEW.decision AND op IN ('resolved','withdrawn','expired'))
|
||||
BEGIN SELECT RAISE(ABORT, 'decision already closed'); END;
|
||||
CREATE TRIGGER decisions_resolved_choice BEFORE INSERT ON decision_events
|
||||
WHEN NEW.op = 'resolved' AND (NEW.choice IS NULL OR NOT EXISTS (
|
||||
SELECT 1 FROM decisions d, json_each(d.options) o WHERE d.id = NEW.decision AND json_extract(o.value,'$.key') = NEW.choice))
|
||||
BEGIN SELECT RAISE(ABORT, 'resolution must name one of the options'); END;
|
||||
CREATE TRIGGER role_one_holder BEFORE INSERT ON role_claims
|
||||
WHEN NEW.op = 'claim' AND (SELECT op FROM role_claims WHERE business = NEW.business AND role = NEW.role ORDER BY seq DESC LIMIT 1) = 'claim'
|
||||
BEGIN SELECT RAISE(ABORT, 'role already held'); END;
|
||||
CREATE TRIGGER role_release_by_holder BEFORE INSERT ON role_claims
|
||||
WHEN NEW.op IN ('release','revoke') AND COALESCE((SELECT op FROM role_claims WHERE business = NEW.business AND role = NEW.role ORDER BY seq DESC LIMIT 1),'') <> 'claim'
|
||||
BEGIN SELECT RAISE(ABORT, 'role is not held'); END;
|
||||
CREATE TRIGGER role_release_same_run BEFORE INSERT ON role_claims
|
||||
WHEN NEW.op = 'release' AND (SELECT holder_run FROM role_claims WHERE business = NEW.business AND role = NEW.role ORDER BY seq DESC LIMIT 1) <> NEW.holder_run
|
||||
BEGIN SELECT RAISE(ABORT, 'only the holder releases; others revoke'); END;
|
||||
CREATE TRIGGER role_revoke_needs_decision BEFORE INSERT ON role_claims
|
||||
WHEN NEW.op = 'revoke' AND NEW.decision IS NULL AND NEW.reason IS NULL
|
||||
BEGIN SELECT RAISE(ABORT, 'revoke needs a resolved decision or a reason'); END;
|
||||
CREATE TRIGGER meta_no_update BEFORE UPDATE ON meta BEGIN SELECT RAISE(ABORT, 'meta is append-only'); END;
|
||||
CREATE TRIGGER meta_no_delete BEFORE DELETE ON meta BEGIN SELECT RAISE(ABORT, 'meta is append-only'); END;
|
||||
CREATE TRIGGER events_no_update BEFORE UPDATE ON events BEGIN SELECT RAISE(ABORT, 'events is append-only'); END;
|
||||
CREATE TRIGGER events_no_delete BEFORE DELETE ON events BEGIN SELECT RAISE(ABORT, 'events is append-only'); END;
|
||||
CREATE TRIGGER role_claims_no_update BEFORE UPDATE ON role_claims BEGIN SELECT RAISE(ABORT, 'role_claims is append-only'); END;
|
||||
CREATE TRIGGER role_claims_no_delete BEFORE DELETE ON role_claims BEGIN SELECT RAISE(ABORT, 'role_claims is append-only'); END;
|
||||
CREATE TRIGGER decisions_no_update BEFORE UPDATE ON decisions BEGIN SELECT RAISE(ABORT, 'decisions is append-only'); END;
|
||||
CREATE TRIGGER decisions_no_delete BEFORE DELETE ON decisions BEGIN SELECT RAISE(ABORT, 'decisions is append-only'); END;
|
||||
CREATE TRIGGER decision_events_no_update BEFORE UPDATE ON decision_events BEGIN SELECT RAISE(ABORT, 'decision_events is append-only'); END;
|
||||
CREATE TRIGGER decision_events_no_delete BEFORE DELETE ON decision_events BEGIN SELECT RAISE(ABORT, 'decision_events is append-only'); END;
|
||||
CREATE TRIGGER messages_no_update BEFORE UPDATE ON messages BEGIN SELECT RAISE(ABORT, 'messages is append-only'); END;
|
||||
CREATE TRIGGER messages_no_delete BEFORE DELETE ON messages BEGIN SELECT RAISE(ABORT, 'messages is append-only'); END;
|
||||
CREATE TRIGGER deliveries_no_update BEFORE UPDATE ON deliveries BEGIN SELECT RAISE(ABORT, 'deliveries is append-only'); END;
|
||||
CREATE TRIGGER deliveries_no_delete BEFORE DELETE ON deliveries BEGIN SELECT RAISE(ABORT, 'deliveries is append-only'); END;
|
||||
CREATE TRIGGER meta_no_replace BEFORE INSERT ON meta WHEN EXISTS (SELECT 1 FROM meta WHERE key = NEW.key) BEGIN SELECT RAISE(ABORT, 'meta is append-only'); END;
|
||||
CREATE TRIGGER events_no_replace BEFORE INSERT ON events WHEN EXISTS (SELECT 1 FROM events WHERE seq = NEW.seq OR id = NEW.id) BEGIN SELECT RAISE(ABORT, 'events is append-only'); END;
|
||||
CREATE TRIGGER role_claims_no_replace BEFORE INSERT ON role_claims WHEN EXISTS (SELECT 1 FROM role_claims WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'role_claims is append-only'); END;
|
||||
CREATE TRIGGER decisions_no_replace BEFORE INSERT ON decisions WHEN EXISTS (SELECT 1 FROM decisions WHERE seq = NEW.seq OR id = NEW.id) BEGIN SELECT RAISE(ABORT, 'decisions is append-only'); END;
|
||||
CREATE TRIGGER decision_events_no_replace BEFORE INSERT ON decision_events WHEN EXISTS (SELECT 1 FROM decision_events WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'decision_events is append-only'); END;
|
||||
CREATE TRIGGER messages_no_replace BEFORE INSERT ON messages WHEN EXISTS (SELECT 1 FROM messages WHERE seq = NEW.seq OR id = NEW.id) BEGIN SELECT RAISE(ABORT, 'messages is append-only'); END;
|
||||
CREATE TRIGGER deliveries_no_replace BEFORE INSERT ON deliveries WHEN EXISTS (SELECT 1 FROM deliveries WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'deliveries is append-only'); END;
|
||||
CREATE TRIGGER task_snapshots_no_update BEFORE UPDATE ON task_snapshots BEGIN SELECT RAISE(ABORT, 'task_snapshots is append-only'); END;
|
||||
CREATE TRIGGER task_snapshots_no_delete BEFORE DELETE ON task_snapshots BEGIN SELECT RAISE(ABORT, 'task_snapshots is append-only'); END;
|
||||
CREATE TRIGGER task_snapshots_no_replace BEFORE INSERT ON task_snapshots WHEN EXISTS (SELECT 1 FROM task_snapshots WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'task_snapshots is append-only'); END;
|
||||
CREATE TRIGGER decisions_blocking_needs_task BEFORE INSERT ON decisions
|
||||
WHEN NEW.blocking = 1 AND NEW.task_ref IS NULL
|
||||
BEGIN SELECT RAISE(ABORT, 'a blocking decision cites the task it blocks'); END;
|
||||
CREATE TRIGGER events_launch_by_human BEFORE INSERT ON events
|
||||
WHEN NEW.kind IN ('launch.revoked','launch.restored') AND (NEW.actor_role IS NOT NULL OR NEW.actor_run IS NOT NULL)
|
||||
BEGIN SELECT RAISE(ABORT, 'only the human revokes or restores launching'); END;
|
||||
CREATE TRIGGER events_credential_body BEFORE INSERT ON events
|
||||
WHEN NEW.kind GLOB 'credential.*' AND (
|
||||
json_extract(NEW.body, '$.service') IS NULL OR json_extract(NEW.body, '$.service') NOT IN ('gitea','vikunja')
|
||||
OR json_extract(NEW.body, '$.instance') IS NULL)
|
||||
BEGIN SELECT RAISE(ABORT, 'credential events name a service and a role instance'); END;
|
||||
CREATE TRIGGER events_task_missing_body BEFORE INSERT ON events
|
||||
WHEN NEW.kind = 'task.missing' AND (
|
||||
json_extract(NEW.body, '$.reason') IS NULL OR json_extract(NEW.body, '$.reason') NOT IN ('moved','not-found','no-access')
|
||||
OR (json_extract(NEW.body, '$.reason') = 'moved' AND json_type(NEW.body, '$.project') IS NOT 'integer'))
|
||||
BEGIN SELECT RAISE(ABORT, 'task.missing carries a reason, and the new project when moved'); END;
|
||||
CREATE TRIGGER events_task_subject BEFORE INSERT ON events
|
||||
WHEN NEW.kind GLOB 'task.*' AND (NEW.subject IS NULL OR NOT (NEW.subject GLOB 'vikunja:[1-9]*/[1-9]*' AND NOT substr(NEW.subject, 9) GLOB '*[^0-9/]*' AND NOT substr(NEW.subject, 9) GLOB '*/*/*'))
|
||||
BEGIN SELECT RAISE(ABORT, 'a task event names its task in subject'); END;
|
||||
CREATE TRIGGER events_task_created_body BEFORE INSERT ON events
|
||||
WHEN NEW.kind = 'task.created' AND (
|
||||
json_type(NEW.body, '$.request') IS NOT 'text'
|
||||
OR NOT EXISTS (SELECT 1 FROM events WHERE id = json_extract(NEW.body, '$.request')
|
||||
AND kind = 'human.input' AND business = NEW.business)
|
||||
OR json_type(NEW.body, '$.requirement') IS NOT 'text'
|
||||
OR NOT json_extract(NEW.body, '$.requirement') GLOB 'REQ-[A-Z]*-[1-9]*'
|
||||
OR json_extract(NEW.body, '$.requirement') GLOB 'REQ-*[^A-Z0-9-]*')
|
||||
BEGIN SELECT RAISE(ABORT, 'task.created cites the human.input that asked for it and a requirement id'); END;
|
||||
CREATE VIEW launch_state AS
|
||||
SELECT business, CASE kind WHEN 'launch.revoked' THEN 'revoked' ELSE 'allowed' END AS state, at
|
||||
FROM events e WHERE kind IN ('launch.revoked','launch.restored')
|
||||
AND seq = (SELECT max(seq) FROM events WHERE business = e.business AND kind IN ('launch.revoked','launch.restored'));
|
||||
CREATE VIEW task_external_changes AS
|
||||
SELECT p.business, p.task_ref, p.seq, p.via, p.updated, p.digest, ls.digest AS self_digest
|
||||
FROM task_snapshots p
|
||||
LEFT JOIN task_snapshots ls ON ls.seq = (SELECT max(seq) FROM task_snapshots
|
||||
WHERE business = p.business AND task_ref = p.task_ref AND source = 'self')
|
||||
WHERE p.source = 'poll'
|
||||
AND p.seq = (SELECT max(seq) FROM task_snapshots WHERE business = p.business AND task_ref = p.task_ref)
|
||||
AND (ls.seq IS NULL OR (p.updated >= ls.updated AND p.read_at > ls.at AND p.digest <> ls.digest));
|
||||
CREATE VIEW task_current AS
|
||||
SELECT s.seq, s.at, s.business, s.task_ref, s.updated, s.etag, s.digest, s.fields, s.source, s.via, s.read_at, s.role, s.run
|
||||
FROM task_snapshots s
|
||||
WHERE s.seq = (SELECT max(c.seq) FROM task_snapshots c
|
||||
WHERE c.business = s.business AND c.task_ref = s.task_ref
|
||||
AND (c.source = 'self'
|
||||
OR c.read_at > coalesce((SELECT ls.at FROM task_snapshots ls
|
||||
WHERE ls.business = c.business AND ls.task_ref = c.task_ref AND ls.source = 'self'
|
||||
ORDER BY ls.seq DESC LIMIT 1), '')));
|
||||
CREATE VIEW tasks_open AS
|
||||
SELECT s.business, s.task_ref, json_extract(s.fields, '$.bucket') AS bucket, s.seq
|
||||
FROM task_current s
|
||||
WHERE json_type(s.fields, '$.gone') IS NULL
|
||||
AND json_extract(s.fields, '$.done') = 0;
|
||||
CREATE VIEW urgent_inbox AS
|
||||
SELECT d.id, d.business, d.task_ref, d.question, d.at
|
||||
FROM decisions d
|
||||
WHERE d.class = 'gated' AND d.blocking = 1
|
||||
AND NOT EXISTS (SELECT 1 FROM decision_events x WHERE x.decision = d.id AND x.op IN ('resolved','withdrawn','expired'));
|
||||
CREATE TABLE task_comment_cursors (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
business TEXT NOT NULL,
|
||||
task_id INTEGER NOT NULL CHECK (task_id > 0),
|
||||
last_comment INTEGER NOT NULL CHECK (last_comment > 0)
|
||||
) STRICT;
|
||||
CREATE INDEX task_comment_cursors_latest ON task_comment_cursors (business, task_id, seq);
|
||||
CREATE TRIGGER task_comment_cursors_no_update BEFORE UPDATE ON task_comment_cursors
|
||||
BEGIN SELECT RAISE(ABORT, 'task_comment_cursors is append-only'); END;
|
||||
CREATE TRIGGER task_comment_cursors_no_delete BEFORE DELETE ON task_comment_cursors
|
||||
BEGIN SELECT RAISE(ABORT, 'task_comment_cursors is append-only'); END;
|
||||
CREATE TRIGGER task_comment_cursors_no_replace BEFORE INSERT ON task_comment_cursors
|
||||
WHEN EXISTS (SELECT 1 FROM task_comment_cursors WHERE seq = NEW.seq)
|
||||
BEGIN SELECT RAISE(ABORT, 'task_comment_cursors is append-only'); END;
|
||||
CREATE TRIGGER task_comment_cursors_forward BEFORE INSERT ON task_comment_cursors
|
||||
WHEN NEW.last_comment <= COALESCE((SELECT last_comment FROM task_comment_cursors
|
||||
WHERE business = NEW.business AND task_id = NEW.task_id ORDER BY seq DESC LIMIT 1), 0)
|
||||
BEGIN SELECT RAISE(ABORT, 'comment cursor must advance'); END;
|
||||
CREATE TABLE projects (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
id TEXT NOT NULL UNIQUE,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
repo TEXT,
|
||||
created_by TEXT NOT NULL,
|
||||
UNIQUE (business, name)
|
||||
) STRICT;
|
||||
CREATE TABLE threads (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
id TEXT NOT NULL UNIQUE,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
project TEXT NOT NULL REFERENCES projects(id),
|
||||
instance TEXT NOT NULL,
|
||||
owner TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
UNIQUE (project, name)
|
||||
) STRICT;
|
||||
CREATE TABLE sessions (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
id TEXT NOT NULL UNIQUE,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
thread TEXT NOT NULL REFERENCES threads(id),
|
||||
instance TEXT NOT NULL,
|
||||
owner TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
op TEXT NOT NULL,
|
||||
predecessor TEXT REFERENCES sessions(id),
|
||||
UNIQUE (thread, name),
|
||||
UNIQUE (thread, op)
|
||||
) STRICT;
|
||||
CREATE TABLE session_defaults (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
thread TEXT NOT NULL REFERENCES threads(id),
|
||||
session TEXT NOT NULL REFERENCES sessions(id),
|
||||
by TEXT NOT NULL
|
||||
) STRICT;
|
||||
CREATE TABLE thread_claims (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
thread TEXT NOT NULL REFERENCES threads(id),
|
||||
session TEXT NOT NULL REFERENCES sessions(id),
|
||||
generation INTEGER NOT NULL CHECK (generation >= 1),
|
||||
op TEXT NOT NULL CHECK (op IN ('claim','release','revoke')),
|
||||
holder_run TEXT NOT NULL,
|
||||
harness TEXT NOT NULL, address TEXT,
|
||||
by TEXT NOT NULL, reason TEXT,
|
||||
decision TEXT
|
||||
) STRICT;
|
||||
CREATE TABLE message_threads (
|
||||
message TEXT PRIMARY KEY REFERENCES messages(id),
|
||||
thread TEXT NOT NULL REFERENCES threads(id),
|
||||
session TEXT NOT NULL REFERENCES sessions(id)
|
||||
) STRICT;
|
||||
CREATE TABLE thread_opens (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
thread TEXT NOT NULL REFERENCES threads(id),
|
||||
session TEXT NOT NULL REFERENCES sessions(id),
|
||||
by TEXT NOT NULL
|
||||
) STRICT;
|
||||
CREATE INDEX thread_claims_thread ON thread_claims (thread, seq);
|
||||
CREATE INDEX thread_claims_session ON thread_claims (session, seq);
|
||||
CREATE INDEX message_threads_thread ON message_threads (thread);
|
||||
CREATE INDEX message_threads_session ON message_threads (session);
|
||||
CREATE INDEX session_defaults_thread ON session_defaults (thread, seq);
|
||||
CREATE TRIGGER threads_project_business BEFORE INSERT ON threads
|
||||
WHEN (SELECT business FROM projects WHERE id = NEW.project) IS NOT NEW.business
|
||||
BEGIN SELECT RAISE(ABORT, 'a thread belongs to its project''s business'); END;
|
||||
CREATE TRIGGER sessions_thread_match BEFORE INSERT ON sessions
|
||||
WHEN NOT EXISTS (SELECT 1 FROM threads t WHERE t.id = NEW.thread AND t.business = NEW.business AND t.instance = NEW.instance AND t.owner = NEW.owner)
|
||||
BEGIN SELECT RAISE(ABORT, 'a session has its thread''s business, instance and owner'); END;
|
||||
CREATE TRIGGER sessions_predecessor BEFORE INSERT ON sessions
|
||||
WHEN NEW.predecessor IS NOT NULL AND (SELECT thread FROM sessions WHERE id = NEW.predecessor) IS NOT NEW.thread
|
||||
BEGIN SELECT RAISE(ABORT, 'a session''s predecessor is in its thread'); END;
|
||||
CREATE TRIGGER sessions_id_not_thread BEFORE INSERT ON sessions
|
||||
WHEN NEW.id IS NOT NEW.thread AND EXISTS (SELECT 1 FROM threads WHERE id = NEW.id)
|
||||
BEGIN SELECT RAISE(ABORT, 'a session id names another thread'); END;
|
||||
CREATE TRIGGER threads_id_not_session BEFORE INSERT ON threads
|
||||
WHEN EXISTS (SELECT 1 FROM sessions WHERE id = NEW.id)
|
||||
BEGIN SELECT RAISE(ABORT, 'a thread id names a session'); END;
|
||||
CREATE TRIGGER session_defaults_match BEFORE INSERT ON session_defaults
|
||||
WHEN (SELECT thread FROM sessions WHERE id = NEW.session) IS NOT NEW.thread
|
||||
BEGIN SELECT RAISE(ABORT, 'a default session is in its thread'); END;
|
||||
CREATE TRIGGER thread_one_holder BEFORE INSERT ON thread_claims
|
||||
WHEN NEW.op = 'claim' AND (SELECT op FROM thread_claims WHERE session = NEW.session ORDER BY seq DESC LIMIT 1) = 'claim'
|
||||
BEGIN SELECT RAISE(ABORT, 'session already held'); END;
|
||||
CREATE TRIGGER thread_one_per_run BEFORE INSERT ON thread_claims
|
||||
WHEN NEW.op = 'claim' AND EXISTS (SELECT 1 FROM thread_claims c WHERE c.holder_run = NEW.holder_run AND c.business = NEW.business
|
||||
AND c.seq = (SELECT max(seq) FROM thread_claims WHERE session = c.session) AND c.op = 'claim')
|
||||
BEGIN SELECT RAISE(ABORT, 'run already holds a session'); END;
|
||||
CREATE TRIGGER thread_release_by_holder BEFORE INSERT ON thread_claims
|
||||
WHEN NEW.op IN ('release','revoke') AND COALESCE((SELECT op FROM thread_claims WHERE session = NEW.session ORDER BY seq DESC LIMIT 1),'') <> 'claim'
|
||||
BEGIN SELECT RAISE(ABORT, 'session is not held'); END;
|
||||
CREATE TRIGGER thread_release_same_run BEFORE INSERT ON thread_claims
|
||||
WHEN NEW.op = 'release' AND (SELECT holder_run FROM thread_claims WHERE session = NEW.session ORDER BY seq DESC LIMIT 1) <> NEW.holder_run
|
||||
BEGIN SELECT RAISE(ABORT, 'only the holder releases; others revoke'); END;
|
||||
CREATE TRIGGER thread_claim_generation BEFORE INSERT ON thread_claims
|
||||
WHEN NEW.generation IS NOT (CASE WHEN NEW.op = 'claim'
|
||||
THEN 1 + (SELECT count(*) FROM thread_claims WHERE session = NEW.session AND op = 'claim')
|
||||
ELSE (SELECT generation FROM thread_claims WHERE session = NEW.session ORDER BY seq DESC LIMIT 1) END)
|
||||
BEGIN SELECT RAISE(ABORT, 'a claim row names its session''s generation'); END;
|
||||
CREATE TRIGGER thread_revoke_needs_decision BEFORE INSERT ON thread_claims
|
||||
WHEN NEW.op = 'revoke' AND NEW.decision IS NULL AND NEW.reason IS NULL
|
||||
BEGIN SELECT RAISE(ABORT, 'revoke needs a resolved decision or a reason'); END;
|
||||
CREATE TRIGGER thread_claims_business BEFORE INSERT ON thread_claims
|
||||
WHEN (SELECT business FROM threads WHERE id = NEW.thread) IS NOT NEW.business
|
||||
BEGIN SELECT RAISE(ABORT, 'a thread claim names the thread''s business'); END;
|
||||
CREATE TRIGGER thread_claims_session BEFORE INSERT ON thread_claims
|
||||
WHEN (SELECT thread FROM sessions WHERE id = NEW.session) IS NOT NEW.thread
|
||||
BEGIN SELECT RAISE(ABORT, 'a claim''s session is in its thread'); END;
|
||||
CREATE TRIGGER message_threads_match BEFORE INSERT ON message_threads
|
||||
WHEN NOT EXISTS (SELECT 1 FROM messages m JOIN threads t ON t.id = NEW.thread
|
||||
WHERE m.id = NEW.message AND m.business = t.business AND t.instance IN (m.to_role, m.from_role))
|
||||
BEGIN SELECT RAISE(ABORT, 'a threaded message is to or from the thread''s instance'); END;
|
||||
CREATE TRIGGER message_threads_session BEFORE INSERT ON message_threads
|
||||
WHEN (SELECT thread FROM sessions WHERE id = NEW.session) IS NOT NEW.thread
|
||||
BEGIN SELECT RAISE(ABORT, 'a threaded message''s session is in its thread'); END;
|
||||
CREATE TRIGGER thread_opens_session BEFORE INSERT ON thread_opens
|
||||
WHEN (SELECT thread FROM sessions WHERE id = NEW.session) IS NOT NEW.thread
|
||||
BEGIN SELECT RAISE(ABORT, 'an open''s session is in its thread'); END;
|
||||
CREATE TRIGGER delivery_same_session BEFORE INSERT ON deliveries
|
||||
WHEN NEW.op IN ('routed','delivered') AND (NEW.op = 'delivered' OR NEW.holder_run IS NOT NULL)
|
||||
AND EXISTS (SELECT 1 FROM message_threads mt JOIN messages m ON m.id = mt.message JOIN threads t ON t.id = mt.thread
|
||||
WHERE mt.message = NEW.message AND m.to_role = t.instance
|
||||
AND NOT EXISTS (SELECT 1 FROM thread_claims c WHERE c.seq = (SELECT max(seq) FROM thread_claims WHERE session = mt.session)
|
||||
AND c.op = 'claim' AND c.holder_run IS NEW.holder_run))
|
||||
BEGIN SELECT RAISE(ABORT, 'a threaded message goes only to its session''s holder'); END;
|
||||
CREATE TRIGGER projects_no_update BEFORE UPDATE ON projects BEGIN SELECT RAISE(ABORT, 'projects is append-only'); END;
|
||||
CREATE TRIGGER projects_no_delete BEFORE DELETE ON projects BEGIN SELECT RAISE(ABORT, 'projects is append-only'); END;
|
||||
CREATE TRIGGER projects_no_replace BEFORE INSERT ON projects WHEN EXISTS (SELECT 1 FROM projects WHERE seq = NEW.seq OR id = NEW.id) BEGIN SELECT RAISE(ABORT, 'projects is append-only'); END;
|
||||
CREATE TRIGGER threads_no_update BEFORE UPDATE ON threads BEGIN SELECT RAISE(ABORT, 'threads is append-only'); END;
|
||||
CREATE TRIGGER threads_no_delete BEFORE DELETE ON threads BEGIN SELECT RAISE(ABORT, 'threads is append-only'); END;
|
||||
CREATE TRIGGER threads_no_replace BEFORE INSERT ON threads WHEN EXISTS (SELECT 1 FROM threads WHERE seq = NEW.seq OR id = NEW.id) BEGIN SELECT RAISE(ABORT, 'threads is append-only'); END;
|
||||
CREATE TRIGGER sessions_no_update BEFORE UPDATE ON sessions BEGIN SELECT RAISE(ABORT, 'sessions is append-only'); END;
|
||||
CREATE TRIGGER sessions_no_delete BEFORE DELETE ON sessions BEGIN SELECT RAISE(ABORT, 'sessions is append-only'); END;
|
||||
CREATE TRIGGER sessions_no_replace BEFORE INSERT ON sessions WHEN EXISTS (SELECT 1 FROM sessions WHERE seq = NEW.seq OR id = NEW.id) BEGIN SELECT RAISE(ABORT, 'sessions is append-only'); END;
|
||||
CREATE TRIGGER session_defaults_no_update BEFORE UPDATE ON session_defaults BEGIN SELECT RAISE(ABORT, 'session_defaults is append-only'); END;
|
||||
CREATE TRIGGER session_defaults_no_delete BEFORE DELETE ON session_defaults BEGIN SELECT RAISE(ABORT, 'session_defaults is append-only'); END;
|
||||
CREATE TRIGGER session_defaults_no_replace BEFORE INSERT ON session_defaults WHEN EXISTS (SELECT 1 FROM session_defaults WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'session_defaults is append-only'); END;
|
||||
CREATE TRIGGER thread_claims_no_update BEFORE UPDATE ON thread_claims BEGIN SELECT RAISE(ABORT, 'thread_claims is append-only'); END;
|
||||
CREATE TRIGGER thread_claims_no_delete BEFORE DELETE ON thread_claims BEGIN SELECT RAISE(ABORT, 'thread_claims is append-only'); END;
|
||||
CREATE TRIGGER thread_claims_no_replace BEFORE INSERT ON thread_claims WHEN EXISTS (SELECT 1 FROM thread_claims WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'thread_claims is append-only'); END;
|
||||
CREATE TRIGGER message_threads_no_update BEFORE UPDATE ON message_threads BEGIN SELECT RAISE(ABORT, 'message_threads is append-only'); END;
|
||||
CREATE TRIGGER message_threads_no_delete BEFORE DELETE ON message_threads BEGIN SELECT RAISE(ABORT, 'message_threads is append-only'); END;
|
||||
CREATE TRIGGER message_threads_no_replace BEFORE INSERT ON message_threads WHEN EXISTS (SELECT 1 FROM message_threads WHERE message = NEW.message) BEGIN SELECT RAISE(ABORT, 'message_threads is append-only'); END;
|
||||
CREATE TRIGGER thread_opens_no_update BEFORE UPDATE ON thread_opens BEGIN SELECT RAISE(ABORT, 'thread_opens is append-only'); END;
|
||||
CREATE TRIGGER thread_opens_no_delete BEFORE DELETE ON thread_opens BEGIN SELECT RAISE(ABORT, 'thread_opens is append-only'); END;
|
||||
CREATE TRIGGER thread_opens_no_replace BEFORE INSERT ON thread_opens WHEN EXISTS (SELECT 1 FROM thread_opens WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'thread_opens is append-only'); END;
|
||||
CREATE TABLE instance_names (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
instance TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
by TEXT NOT NULL
|
||||
) STRICT;
|
||||
CREATE TRIGGER instance_names_unique BEFORE INSERT ON instance_names
|
||||
WHEN EXISTS (SELECT 1 FROM instance_names n WHERE n.business = NEW.business AND n.name = NEW.name
|
||||
AND n.instance <> NEW.instance
|
||||
AND n.seq = (SELECT max(seq) FROM instance_names WHERE business = n.business AND instance = n.instance))
|
||||
BEGIN SELECT RAISE(ABORT, 'an instance name is unique in its business'); END;
|
||||
CREATE TRIGGER instance_names_no_update BEFORE UPDATE ON instance_names BEGIN SELECT RAISE(ABORT, 'instance_names is append-only'); END;
|
||||
CREATE TRIGGER instance_names_no_delete BEFORE DELETE ON instance_names BEGIN SELECT RAISE(ABORT, 'instance_names is append-only'); END;
|
||||
CREATE TRIGGER instance_names_no_replace BEFORE INSERT ON instance_names WHEN EXISTS (SELECT 1 FROM instance_names WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'instance_names is append-only'); END;
|
||||
-- Row 124 (#1618): immutable source snapshots and turn outcomes, never transcript text.
|
||||
CREATE TABLE usage_records (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
run TEXT NOT NULL,
|
||||
instance TEXT NOT NULL,
|
||||
provider TEXT,
|
||||
model TEXT,
|
||||
account TEXT,
|
||||
turn TEXT NOT NULL,
|
||||
source TEXT NOT NULL,
|
||||
phase TEXT NOT NULL CHECK (phase IN ('begin','end')),
|
||||
status TEXT NOT NULL CHECK (status IN ('known','unknown')),
|
||||
code TEXT,
|
||||
cursor TEXT CHECK (cursor IS NULL OR json_valid(cursor)),
|
||||
input INTEGER,
|
||||
output INTEGER,
|
||||
cache_write INTEGER,
|
||||
cache_read INTEGER,
|
||||
UNIQUE (business,run,turn,source,phase),
|
||||
CHECK ((status='unknown' AND input IS NULL AND output IS NULL AND cache_write IS NULL AND cache_read IS NULL)
|
||||
OR (status='known' AND input >= 0 AND output >= 0 AND cache_write >= 0 AND cache_read >= 0
|
||||
AND input IS NOT NULL AND output IS NOT NULL AND cache_write IS NOT NULL AND cache_read IS NOT NULL))
|
||||
) STRICT;
|
||||
CREATE TRIGGER usage_records_no_update BEFORE UPDATE ON usage_records BEGIN SELECT RAISE(ABORT, 'usage_records is append-only'); END;
|
||||
CREATE TRIGGER usage_records_no_delete BEFORE DELETE ON usage_records BEGIN SELECT RAISE(ABORT, 'usage_records is append-only'); END;
|
||||
CREATE TRIGGER usage_records_no_replace BEFORE INSERT ON usage_records WHEN EXISTS (SELECT 1 FROM usage_records WHERE seq=NEW.seq OR (business=NEW.business AND run=NEW.run AND turn=NEW.turn AND source=NEW.source AND phase=NEW.phase)) BEGIN SELECT RAISE(ABORT, 'usage_records is append-only'); END;
|
||||
CREATE TABLE hosts (
|
||||
business TEXT NOT NULL,
|
||||
host TEXT NOT NULL,
|
||||
address TEXT NOT NULL,
|
||||
port INTEGER NOT NULL CHECK (port BETWEEN 1 AND 65535),
|
||||
user TEXT NOT NULL,
|
||||
sudo INTEGER NOT NULL CHECK (sudo IN (0,1)),
|
||||
host_keys TEXT NOT NULL CHECK (json_valid(host_keys) AND json_type(host_keys) = 'array' AND json_array_length(host_keys) > 0),
|
||||
installer_fingerprint TEXT,
|
||||
state TEXT NOT NULL CHECK (state IN ('pending-install','ready')),
|
||||
decision TEXT NOT NULL REFERENCES decisions(id),
|
||||
checked_at TEXT,
|
||||
PRIMARY KEY (business, host),
|
||||
CHECK (state = 'pending-install' OR (installer_fingerprint IS NOT NULL AND checked_at IS NOT NULL))
|
||||
) STRICT;
|
||||
CREATE TABLE host_grants (
|
||||
business TEXT NOT NULL,
|
||||
agent TEXT NOT NULL,
|
||||
host TEXT NOT NULL,
|
||||
commands TEXT NOT NULL CHECK (json_valid(commands) AND json_type(commands) = 'array' AND json_array_length(commands) > 0),
|
||||
rotate_by TEXT NOT NULL,
|
||||
tag TEXT NOT NULL CHECK (length(tag) = 39 AND tag GLOB 'mosaic-*' AND NOT substr(tag, 8) GLOB '*[^0-9a-f]*'),
|
||||
decision TEXT NOT NULL REFERENCES decisions(id),
|
||||
PRIMARY KEY (business, agent, host),
|
||||
FOREIGN KEY (business, host) REFERENCES hosts(business, host)
|
||||
) STRICT;
|
||||
CREATE TRIGGER events_grant_actor BEFORE INSERT ON events
|
||||
WHEN NEW.kind = 'action.allowed'
|
||||
AND (json_extract(NEW.body, '$.action') GLOB 'grant.*' OR json_extract(NEW.body, '$.operation') GLOB 'grant.*')
|
||||
AND NOT (NEW.actor_role IS NULL AND NEW.actor_run IS 'grant')
|
||||
BEGIN SELECT RAISE(ABORT, 'only the grant writer records grant events'); END;
|
||||
CREATE TABLE gitea_users (
|
||||
business TEXT NOT NULL,
|
||||
agent TEXT NOT NULL,
|
||||
user TEXT NOT NULL,
|
||||
id INTEGER NOT NULL CHECK (id > 0),
|
||||
nonce TEXT NOT NULL CHECK (length(nonce) = 16 AND NOT nonce GLOB '*[^0-9a-f]*'),
|
||||
decision TEXT NOT NULL REFERENCES decisions(id),
|
||||
PRIMARY KEY (business, agent)
|
||||
) STRICT;
|
||||
CREATE TABLE gitea_grants (
|
||||
business TEXT NOT NULL,
|
||||
agent TEXT NOT NULL,
|
||||
user TEXT NOT NULL,
|
||||
token_id INTEGER NOT NULL CHECK (token_id > 0),
|
||||
token_name TEXT NOT NULL,
|
||||
projects TEXT NOT NULL CHECK (json_valid(projects) AND json_type(projects) = 'array' AND json_array_length(projects) > 0),
|
||||
rotate_by TEXT NOT NULL,
|
||||
decision TEXT NOT NULL REFERENCES decisions(id),
|
||||
PRIMARY KEY (business, agent),
|
||||
FOREIGN KEY (business, agent) REFERENCES gitea_users(business, agent)
|
||||
) STRICT;
|
||||
-- Row 206 (193b2, #1704, row 193 design 5.2): a thread owner's read-only shares. The latest row per
|
||||
-- (thread, target) is the share's state. Append-only.
|
||||
CREATE TABLE thread_shares (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
thread TEXT NOT NULL REFERENCES threads(id),
|
||||
op TEXT NOT NULL CHECK (op IN ('share','unshare')),
|
||||
target TEXT NOT NULL,
|
||||
by TEXT NOT NULL
|
||||
) STRICT;
|
||||
CREATE INDEX thread_shares_thread ON thread_shares (thread, target, seq);
|
||||
CREATE TRIGGER thread_shares_no_update BEFORE UPDATE ON thread_shares BEGIN SELECT RAISE(ABORT, 'thread_shares is append-only'); END;
|
||||
CREATE TRIGGER thread_shares_no_delete BEFORE DELETE ON thread_shares BEGIN SELECT RAISE(ABORT, 'thread_shares is append-only'); END;
|
||||
CREATE TRIGGER thread_shares_no_replace BEFORE INSERT ON thread_shares WHEN EXISTS (SELECT 1 FROM thread_shares WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'thread_shares is append-only'); END;
|
||||
@@ -35,11 +35,12 @@ function source3j(t) {
|
||||
// Row 264a (#1769): what the 3k→3l step rebuilds or adds. sessions.test.mjs tests those objects.
|
||||
const SESSIONS_3L = ['sessions', 'session_defaults', 'thread_claims', 'message_threads', 'thread_opens'];
|
||||
// Row 206 (#1704): and the 3l→3m step's thread_shares. thread-shares.test.mjs tests it.
|
||||
const later3l = (r) => [...SESSIONS_3L, 'thread_shares'].includes(r.tbl_name) || ['delivery_same_thread', 'delivery_same_session', 'threads_id_not_session'].includes(r.name);
|
||||
// Row 271 (#1776): and the 3m→3n step's session_ops and message_cancels. session-clear.test.mjs tests them.
|
||||
const later3l = (r) => [...SESSIONS_3L, 'thread_shares', 'session_ops', 'message_cancels'].includes(r.tbl_name) || ['delivery_same_thread', 'delivery_same_session', 'threads_id_not_session'].includes(r.name);
|
||||
function rows(path) {
|
||||
const db = new DatabaseSync(path, { readOnly: true });
|
||||
try {
|
||||
return Object.fromEntries(db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name NOT IN ('gitea_users','gitea_grants','sessions','session_defaults','thread_shares')").all()
|
||||
return Object.fromEntries(db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name NOT IN ('gitea_users','gitea_grants','sessions','session_defaults','thread_shares','session_ops','message_cancels')").all()
|
||||
.map(({ name }) => [name, db.prepare(`SELECT * FROM ${name} ORDER BY rowid`).all()
|
||||
.filter(r => name !== 'meta' || !['schema_version', 'schema_digest'].includes(r.key)).map(r => ({ ...r }))]));
|
||||
} finally { db.close(); }
|
||||
@@ -76,6 +77,7 @@ async function engine(root, name, { store, schema, step = (s) => s }) {
|
||||
writeFileSync(join(moduleRoot, 'src/migrations/3j-3k.sql'), step(readFileSync(here('../src/migrations/3j-3k.sql'), 'utf8')));
|
||||
copyFileSync(here('../src/migrations/3k-3l.sql'), join(moduleRoot, 'src/migrations/3k-3l.sql'));
|
||||
copyFileSync(here('../src/migrations/3l-3m.sql'), join(moduleRoot, 'src/migrations/3l-3m.sql'));
|
||||
copyFileSync(here('../src/migrations/3m-3n.sql'), join(moduleRoot, 'src/migrations/3m-3n.sql'));
|
||||
return (await import(pathToFileURL(join(moduleRoot, 'src/store.mjs')))).Store;
|
||||
}
|
||||
function fresh(t) {
|
||||
@@ -97,7 +99,7 @@ test('3j copy migrates to 3k: every row kept, and sqlite_master gains only the t
|
||||
const { root, path } = source3j(t), before = rows(path), was = master(path);
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
assert.equal(version(path), '3m');
|
||||
assert.equal(version(path), '3n');
|
||||
assert.deepEqual(migrated3l(rows(path)), before);
|
||||
const now = master(path);
|
||||
assert.deepEqual(now.filter(r => !NEW.includes(r.name) && !later3l(r)), was.filter(r => !later3l(r)));
|
||||
@@ -123,7 +125,7 @@ test('a 3j→3k step that fails partway leaves the store at 3j with every row an
|
||||
assert.deepEqual(master(path), was);
|
||||
assert.deepEqual(rows(path), before);
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
assert.equal(version(path), '3m');
|
||||
assert.equal(version(path), '3n');
|
||||
});
|
||||
|
||||
test('the 3j engine refuses a migrated store, migrating or not, and leaves it as it was', async t => {
|
||||
@@ -134,14 +136,14 @@ test('the 3j engine refuses a migrated store, migrating or not, and leaves it as
|
||||
schema: ddl3j,
|
||||
store: (source) => source
|
||||
.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3j).digest('hex')}';`)
|
||||
.replace("export const SCHEMA_VERSION = '3m';", "export const SCHEMA_VERSION = '3j';")
|
||||
.replace("export const SCHEMA_VERSION = '3n';", "export const SCHEMA_VERSION = '3j';")
|
||||
.replace(/\}, \{\n {2}\/\/ Row 236 \(#1742[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables'),
|
||||
});
|
||||
const pinned = readFileSync(join(root, 'engine-3j/src/store.mjs'), 'utf8');
|
||||
assert.ok(pinned.includes("export const SCHEMA_VERSION = '3j';") && !pinned.includes('3j-3k'), 'the 3j engine has no 3j→3k step');
|
||||
assert.throws(() => new Store3j(root), /schema-mismatch/);
|
||||
assert.throws(() => new Store3j(root, { migrateSchema: true }), /schema-mismatch/);
|
||||
assert.equal(version(path), '3m');
|
||||
assert.equal(version(path), '3n');
|
||||
assert.deepEqual(master(path), was);
|
||||
assert.deepEqual(rows(path), after);
|
||||
});
|
||||
|
||||
@@ -21,7 +21,8 @@ const LATER = ['gitea_users', 'gitea_grants', 'sqlite_autoindex_gitea_users_1',
|
||||
// Row 264a (#1769): what the 3k→3l step rebuilds or adds. sessions.test.mjs tests those objects.
|
||||
const SESSIONS_3L = ['sessions', 'session_defaults', 'thread_claims', 'message_threads', 'thread_opens'];
|
||||
// Row 206 (#1704): and the 3l→3m step's thread_shares. thread-shares.test.mjs tests it.
|
||||
const later3l = (r) => [...SESSIONS_3L, 'thread_shares'].includes(r.tbl_name) || ['delivery_same_thread', 'delivery_same_session', 'threads_id_not_session'].includes(r.name);
|
||||
// Row 271 (#1776): and the 3m→3n step's session_ops and message_cancels. session-clear.test.mjs tests them.
|
||||
const later3l = (r) => [...SESSIONS_3L, 'thread_shares', 'session_ops', 'message_cancels'].includes(r.tbl_name) || ['delivery_same_thread', 'delivery_same_session', 'threads_id_not_session'].includes(r.name);
|
||||
function source3i(t) {
|
||||
const root = mkdtempSync(join(tmpdir(), 'grant-migrate-'));
|
||||
t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
@@ -41,7 +42,7 @@ function source3i(t) {
|
||||
function rows(path) {
|
||||
const db = new DatabaseSync(path, { readOnly: true });
|
||||
try {
|
||||
return Object.fromEntries(db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name NOT IN ('hosts','host_grants','gitea_users','gitea_grants','sessions','session_defaults','thread_shares')").all()
|
||||
return Object.fromEntries(db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name NOT IN ('hosts','host_grants','gitea_users','gitea_grants','sessions','session_defaults','thread_shares','session_ops','message_cancels')").all()
|
||||
.map(({ name }) => [name, db.prepare(`SELECT * FROM ${name} ORDER BY rowid`).all()
|
||||
.filter(r => name !== 'meta' || !['schema_version', 'schema_digest'].includes(r.key)).map(r => ({ ...r }))]));
|
||||
} finally { db.close(); }
|
||||
@@ -79,6 +80,7 @@ async function engine(root, name, { store, schema, step = (s) => s }) {
|
||||
copyFileSync(here('../src/migrations/3j-3k.sql'), join(moduleRoot, 'src/migrations/3j-3k.sql'));
|
||||
copyFileSync(here('../src/migrations/3k-3l.sql'), join(moduleRoot, 'src/migrations/3k-3l.sql'));
|
||||
copyFileSync(here('../src/migrations/3l-3m.sql'), join(moduleRoot, 'src/migrations/3l-3m.sql'));
|
||||
copyFileSync(here('../src/migrations/3m-3n.sql'), join(moduleRoot, 'src/migrations/3m-3n.sql'));
|
||||
return (await import(pathToFileURL(join(moduleRoot, 'src/store.mjs')))).Store;
|
||||
}
|
||||
function fresh(t) {
|
||||
@@ -102,7 +104,7 @@ test('3i copy migrates to 3j: every row kept, and sqlite_master gains only the t
|
||||
const { root, path } = source3i(t), before = rows(path), was = master(path);
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
assert.equal(version(path), '3m');
|
||||
assert.equal(version(path), '3n');
|
||||
assert.deepEqual(migrated3l(rows(path)), before);
|
||||
const now = master(path).filter(r => !LATER.includes(r.name));
|
||||
assert.deepEqual(now.filter(r => !NEW.includes(r.name) && !later3l(r)), was.filter(r => !later3l(r)));
|
||||
@@ -127,7 +129,7 @@ test('a 3i→3j step that fails partway leaves the store at 3i with every row an
|
||||
assert.deepEqual(master(path), was);
|
||||
assert.deepEqual(rows(path), before);
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
assert.equal(version(path), '3m');
|
||||
assert.equal(version(path), '3n');
|
||||
});
|
||||
|
||||
test('the 3i engine refuses a migrated store, migrating or not, and leaves it as it was', async t => {
|
||||
@@ -138,14 +140,14 @@ test('the 3i engine refuses a migrated store, migrating or not, and leaves it as
|
||||
schema: ddl3i,
|
||||
store: (source) => source
|
||||
.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3i).digest('hex')}';`)
|
||||
.replace("export const SCHEMA_VERSION = '3m';", "export const SCHEMA_VERSION = '3i';")
|
||||
.replace("export const SCHEMA_VERSION = '3n';", "export const SCHEMA_VERSION = '3i';")
|
||||
.replace(/\}, \{\n {2}\/\/ Row 234 \(#1740\)[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables'),
|
||||
});
|
||||
const pinned = readFileSync(join(root, 'engine-3i/src/store.mjs'), 'utf8');
|
||||
assert.ok(pinned.includes("export const SCHEMA_VERSION = '3i';") && !pinned.includes('3i-3j'), 'the 3i engine has no 3i→3j step');
|
||||
assert.throws(() => new Store3i(root), /schema-mismatch/);
|
||||
assert.throws(() => new Store3i(root, { migrateSchema: true }), /schema-mismatch/);
|
||||
assert.equal(version(path), '3m');
|
||||
assert.equal(version(path), '3n');
|
||||
assert.deepEqual(master(path), was);
|
||||
assert.deepEqual(rows(path), after);
|
||||
});
|
||||
|
||||
@@ -65,7 +65,7 @@ test('3f copy migrates to 3g preserving every old column/row, with an empty inst
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
const store = new Store(root, { migrateSchema: true });
|
||||
try {
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3m');
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3n');
|
||||
assert.deepEqual({ ...store.get('SELECT to_name FROM messages') }, { to_name: null });
|
||||
assert.equal(store.get('SELECT count(*) AS n FROM instance_names').n, 0);
|
||||
assert.deepEqual({ ...store.get('SELECT spec_path,spec_commit FROM decisions') }, { spec_path: 'docs/spec.md', spec_commit: 'c'.repeat(40) });
|
||||
|
||||
@@ -523,7 +523,8 @@ test('v3b prototype refusals, views and append-only mutations', (t) => {
|
||||
// Row 236 (#1742) adds gitea_users and gitea_grants; gitea-migration.test.mjs tests them.
|
||||
// Row 264a (#1769) adds sessions, session_defaults and 15 triggers; sessions.test.mjs tests them.
|
||||
// Row 206 (#1704) adds thread_shares and its 3 append-only triggers; thread-shares.test.mjs tests them.
|
||||
assert.equal(count('table') - 1, 23);
|
||||
assert.equal(count('trigger'), 89);
|
||||
// Row 271 (#1776) adds session_ops, message_cancels and 7 triggers; session-clear.test.mjs tests them.
|
||||
assert.equal(count('table') - 1, 25);
|
||||
assert.equal(count('trigger'), 96);
|
||||
assert.equal(count('view'), 5);
|
||||
});
|
||||
@@ -0,0 +1,597 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { DatabaseSync } from 'node:sqlite';
|
||||
import { chmodSync, copyFileSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { fileURLToPath, pathToFileURL } from 'node:url';
|
||||
import { Store, schemaDigest } from '../src/store.mjs';
|
||||
import { Broker } from '../src/broker.mjs';
|
||||
|
||||
// Row 271 (264d, #1776, T4): clear context, the bus half. D7 and D8 of
|
||||
// docs/plans/2026-10-21_t1-sessions.md, the contract in agents/gizmoduck/work/queue-271/BUILD.md.
|
||||
// The launcher's stop, proof and start are in packages/cli/tests/session-clear.test.mjs.
|
||||
const shipped = (name) =>
|
||||
JSON.parse(readFileSync(fileURLToPath(new URL(`../../../roles/${name}.json`, import.meta.url)), 'utf8'));
|
||||
const roles = {
|
||||
chief: { definition: 'chief-of-staff', authority: { withinRole: ['message.send'], crossRole: [] } },
|
||||
assistant: { definition: 'assistant', authority: { withinRole: ['message.send'], crossRole: [] } },
|
||||
};
|
||||
const GENERATION = '0b6e7f6e-4c1d-4a8e-9a51-6d2f3c4b5a69';
|
||||
const UUID8 = /^[0-9a-f]{8}-[0-9a-f]{4}-8[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/;
|
||||
|
||||
function setup(t, { access = null } = {}) {
|
||||
const root = mkdtempSync(join(tmpdir(), 'bus-session-clear-'));
|
||||
const store = new Store(root);
|
||||
t.after(() => {
|
||||
store.close();
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
const sw = { state: { version: 1, halted: false, generation: GENERATION } };
|
||||
const b = new Broker({
|
||||
store,
|
||||
businesses: {
|
||||
demo: {
|
||||
id: 'demo',
|
||||
human: 'jason',
|
||||
arbiters: { technical: 'chief', delivery: 'chief' },
|
||||
vars: { 'bus.tester': true },
|
||||
roles,
|
||||
definitions: { 'chief-of-staff': {}, assistant: { messaging: shipped('assistant').messaging } },
|
||||
projects: { stack: { root: '/nonexistent' } },
|
||||
},
|
||||
},
|
||||
processing: { read: () => sw.state },
|
||||
...(access ? { access: () => access.doc } : {}),
|
||||
});
|
||||
const caps = {
|
||||
human: b.bindHuman({ business: 'demo', human: 'jason', via: 'cli', outsideAgent: true }),
|
||||
tester: b.bindTester({ business: 'demo' }),
|
||||
};
|
||||
const call = (who, verb, args = {}) => b.request(caps[who], { verb, args });
|
||||
const code = (fn) => {
|
||||
try {
|
||||
fn();
|
||||
} catch (e) {
|
||||
return e.code ?? e.message;
|
||||
}
|
||||
return null;
|
||||
};
|
||||
const launch = (name, thread, session) => {
|
||||
caps[name] = b.bindLaunch({ business: 'demo', role: 'assistant', run: `${name}-run`, harness: 'pi', thread, ...(session ? { session } : {}) });
|
||||
return call(name, 'role.claim');
|
||||
};
|
||||
caps.chief = b.bindLaunch({ business: 'demo', role: 'chief', run: 'chief-run', harness: 'pi' });
|
||||
call('chief', 'role.claim');
|
||||
const project = call('human', 'project.create', { name: 'stack work', repo: 'stack' }).id;
|
||||
const A = call('human', 'thread.create', { project, name: 'alpha' }).id;
|
||||
const list = (thread = A) => call('human', 'session.list', { thread });
|
||||
const D = list()[0].id;
|
||||
const stages = (op, thread = A) => store.all('SELECT stage FROM session_ops WHERE thread=? AND op=? ORDER BY seq', thread, op).map((r) => r.stage);
|
||||
const cancels = (session) => store.all('SELECT message FROM message_cancels WHERE session=? ORDER BY seq', session).map((r) => r.message);
|
||||
const count = (table) => store.get(`SELECT count(*) AS n FROM ${table}`).n;
|
||||
const clear = (who, args) => call(who, 'session.clear', { thread: A, confirm: true, ...args });
|
||||
const where = (op, thread = A) => ({ business: 'demo', thread, op });
|
||||
return { b, store, sw, caps, call, code, launch, project, A, D, list, stages, cancels, count, clear, where };
|
||||
}
|
||||
|
||||
const successorOf = (thread, session, op) => {
|
||||
const h = createHash('sha256').update(JSON.stringify(['demo', thread, session, op])).digest('hex').split('');
|
||||
h[12] = '8';
|
||||
h[16] = '89ab'[parseInt(h[16], 16) & 3];
|
||||
const x = h.join('');
|
||||
return `${x.slice(0, 8)}-${x.slice(8, 12)}-${x.slice(12, 16)}-${x.slice(16, 20)}-${x.slice(20, 32)}`;
|
||||
};
|
||||
|
||||
test('T4 D7: a clear of a held default session fences it, drains at the next receive, retires at the exit proof and moves the default', (t) => {
|
||||
const { b, call, code, launch, A, D, list, stages, clear, where, store } = setup(t);
|
||||
launch('a1', A);
|
||||
call('human', 'thread.send', { thread: A, body: 'first turn' });
|
||||
assert.deepEqual(call('a1', 'message.receive').map((m) => m.body), ['first turn']);
|
||||
assert.deepEqual(list().map((r) => [r.id, r.state, r.revision, r.successor, r.default]), [[D, 'active', 0, null, true]]);
|
||||
const accepted = clear('human', { session: D, revision: 0, op: 'clear-1' });
|
||||
const S = successorOf(A, D, 'clear-1');
|
||||
assert.match(S, UUID8);
|
||||
assert.deepEqual({ ...accepted, at: null }, { op: 'clear-1', thread: A, session: D, successor: S, state: 'clearing', start: null, refusal: null, at: null });
|
||||
// Accept fences the session: no send, open or new claim reaches it. The holder keeps its claim.
|
||||
assert.equal(code(() => call('human', 'thread.send', { thread: A, body: 'late' })), 'session-clearing');
|
||||
assert.equal(code(() => call('human', 'thread.open', { thread: A })), 'session-clearing');
|
||||
assert.equal(code(() => launch('a2', A)), 'session-clearing');
|
||||
assert.deepEqual(stages('clear-1'), ['requested']);
|
||||
assert.deepEqual(list().map((r) => [r.state, r.revision]), [['clearing', 1]]);
|
||||
// The run's next receive, at its completed turn, records the drain once and delivers nothing.
|
||||
assert.deepEqual(call('a1', 'message.receive'), []);
|
||||
assert.deepEqual(call('a1', 'message.receive'), []);
|
||||
assert.deepEqual(stages('clear-1'), ['requested', 'drained']);
|
||||
assert.deepEqual(
|
||||
{ ...store.get("SELECT run, generation FROM session_ops WHERE op='clear-1' AND stage='drained'") },
|
||||
{ run: 'a1-run', generation: 1 },
|
||||
);
|
||||
assert.deepEqual(b.sessionClears('demo'), [
|
||||
{ thread: A, instance: 'assistant', session: D, op: 'clear-1', stage: 'drained', run: 'a1-run', generation: 1, successor: S, refusal: null },
|
||||
]);
|
||||
// No proof until the run itself has ended: a signal or a dead socket isn't one.
|
||||
assert.equal(code(() => b.sessionClearExited({ ...where('clear-1'), run: 'a1-run' })), 'run-not-ended');
|
||||
assert.equal(code(() => b.sessionClearExited({ ...where('clear-1'), run: 'other-run' })), 'not-drained-run');
|
||||
assert.equal(code(() => b.sessionClearRetire(where('clear-1'))), 'not-exited');
|
||||
b.endLaunch({ business: 'demo', run: 'a1-run', reason: 'stopped', exitCode: 0 });
|
||||
assert.equal(b.sessionClearExited({ ...where('clear-1'), run: 'a1-run' }).state, 'clearing');
|
||||
// A repeat of the proof answers the envelope and appends nothing.
|
||||
b.sessionClearExited({ ...where('clear-1'), run: 'a1-run' });
|
||||
assert.deepEqual(stages('clear-1'), ['requested', 'drained', 'exited']);
|
||||
const retired = b.sessionClearRetire(where('clear-1'));
|
||||
assert.deepEqual([retired.state, retired.start, retired.successor], ['retired', null, S]);
|
||||
assert.deepEqual(list().map((r) => [r.id, r.name, r.predecessor, r.state, r.successor, r.default]), [
|
||||
[D, 'default', null, 'retired', S, false],
|
||||
[S, 'default-1', D, 'active', null, true],
|
||||
]);
|
||||
// A retired session has no threadsState row; the successor has one and no message.
|
||||
assert.deepEqual(b.threadsState('demo').map((r) => [r.session, r.pending, r.holder]), [[S, 0, null]]);
|
||||
assert.equal(code(() => call('human', 'thread.send', { thread: A, session: D, body: 'to old' })), 'session-retired');
|
||||
assert.equal(code(() => clear('human', { session: D, revision: 4, op: 'clear-2' })), 'session-retired');
|
||||
// A thread-only send now goes to the successor, the default.
|
||||
const m = call('human', 'thread.send', { thread: A, body: 'fresh' });
|
||||
assert.equal(store.get('SELECT session FROM message_threads WHERE message=?', m.id).session, S);
|
||||
});
|
||||
|
||||
test('T4 D7: an unheld session drains at accept, its proof is run null, and a non-default clear leaves the default', (t) => {
|
||||
const { b, call, code, A, D, list, stages, clear, where } = setup(t);
|
||||
const N = call('human', 'session.create', { thread: A, name: 'side', op: 'op-side' }).id;
|
||||
clear('human', { session: N, revision: 0, op: 'clear-side' });
|
||||
assert.deepEqual(stages('clear-side'), ['requested', 'drained']);
|
||||
assert.deepEqual(b.sessionClears('demo').map((c) => [c.session, c.stage, c.run, c.generation]), [[N, 'drained', null, null]]);
|
||||
assert.equal(code(() => b.sessionClearExited({ ...where('clear-side'), run: 'x-run' })), 'not-drained-run');
|
||||
b.sessionClearExited({ ...where('clear-side'), run: null });
|
||||
b.sessionClearRetire(where('clear-side'));
|
||||
const S = successorOf(A, N, 'clear-side');
|
||||
assert.deepEqual(list().map((r) => [r.id, r.name, r.default, r.state]), [
|
||||
[D, 'default', true, 'active'],
|
||||
[N, 'side', false, 'retired'],
|
||||
[S, 'side-1', false, 'active'],
|
||||
]);
|
||||
});
|
||||
|
||||
test('T4 D7: a run that ends before its next receive drains at its end, with its generation', (t) => {
|
||||
const { b, call, launch, A, D, stages, clear, store } = setup(t);
|
||||
launch('a1', A);
|
||||
clear('human', { session: D, revision: 0, op: 'clear-end' });
|
||||
assert.deepEqual(stages('clear-end'), ['requested']);
|
||||
b.endLaunch({ business: 'demo', run: 'a1-run', reason: 'exited', exitCode: 0 });
|
||||
assert.deepEqual(stages('clear-end'), ['requested', 'drained']);
|
||||
assert.deepEqual({ ...store.get("SELECT run, generation FROM session_ops WHERE op='clear-end' AND stage='drained'") }, { run: 'a1-run', generation: 1 });
|
||||
});
|
||||
|
||||
test('T4 D7: a run whose turn is running (delivered, unread) stays clearing; uncertain is recorded once and nothing more', (t) => {
|
||||
const { b, call, code, launch, A, D, stages, clear, where, list } = setup(t);
|
||||
launch('a1', A);
|
||||
call('human', 'thread.send', { thread: A, body: 'long turn' });
|
||||
call('a1', 'message.receive');
|
||||
clear('human', { session: D, revision: 0, op: 'clear-busy' });
|
||||
// Without a receive or an end, it isn't drained, so the launcher has nothing to stop.
|
||||
assert.deepEqual(b.sessionClears('demo').map((c) => c.stage), ['requested']);
|
||||
assert.equal(code(() => b.sessionClearUncertain(where('clear-busy'))), 'not-drained');
|
||||
assert.equal(code(() => b.sessionClearExited({ ...where('clear-busy'), run: 'a1-run' })), 'not-drained');
|
||||
assert.deepEqual(call('a1', 'message.receive'), []);
|
||||
assert.equal(b.sessionClearUncertain(where('clear-busy')).state, 'uncertain');
|
||||
b.sessionClearUncertain(where('clear-busy'));
|
||||
assert.deepEqual(stages('clear-busy'), ['requested', 'drained', 'uncertain']);
|
||||
assert.deepEqual(list().map((r) => r.state), ['uncertain']);
|
||||
assert.equal(code(() => call('human', 'thread.send', { thread: A, body: 'x' })), 'session-clearing');
|
||||
// Proof arriving later resumes the clear.
|
||||
b.endLaunch({ business: 'demo', run: 'a1-run', reason: 'stopped', exitCode: 0 });
|
||||
b.sessionClearExited({ ...where('clear-busy'), run: 'a1-run' });
|
||||
assert.equal(b.sessionClearRetire(where('clear-busy')).state, 'retired');
|
||||
});
|
||||
|
||||
test('T4 D7 B1: undelivered inputs are cancelled with receipts at accept, never delivered to the successor, and a routed reply is cancelled at arrival', (t) => {
|
||||
const { b, call, launch, A, D, stages, cancels, clear, where, store } = setup(t);
|
||||
launch('a1', A);
|
||||
const ask = call('a1', 'message.send', { to: 'chief', body: 'need help', class: 'REQUEST' });
|
||||
const m1 = call('human', 'thread.send', { thread: A, body: 'waiting one' });
|
||||
const m2 = call('human', 'thread.send', { thread: A, body: 'waiting two' });
|
||||
clear('human', { session: D, revision: 0, op: 'clear-msgs' });
|
||||
assert.deepEqual(cancels(D), [m1.id, m2.id]);
|
||||
assert.deepEqual(b.threadsState('demo').map((r) => r.pending), [0]);
|
||||
// The chief's reply routes to the session being cleared: kept, cancelled, and the answer says why.
|
||||
const back = call('chief', 'message.send', { to: 'assistant', body: 'here', in_reply_to: ask.id });
|
||||
assert.equal(back.cancelled, 'session-clearing');
|
||||
assert.equal(back.waiting, undefined);
|
||||
assert.deepEqual(cancels(D), [m1.id, m2.id, back.id]);
|
||||
assert.deepEqual({ ...store.get('SELECT op FROM message_cancels WHERE message=?', back.id) }, { op: 'clear-msgs' });
|
||||
assert.deepEqual(call('a1', 'message.receive'), []);
|
||||
b.endLaunch({ business: 'demo', run: 'a1-run', reason: 'stopped', exitCode: 0 });
|
||||
b.sessionClearExited({ ...where('clear-msgs'), run: 'a1-run' });
|
||||
b.sessionClearRetire(where('clear-msgs'));
|
||||
const S = successorOf(A, D, 'clear-msgs');
|
||||
launch('s1', A, S);
|
||||
assert.deepEqual(call('s1', 'message.receive'), []);
|
||||
// A reply to the retired session's message is cancelled as retired.
|
||||
const late = call('chief', 'message.send', { to: 'assistant', body: 'later', in_reply_to: ask.id });
|
||||
assert.equal(late.cancelled, 'session-retired');
|
||||
assert.deepEqual(call('s1', 'message.receive'), []);
|
||||
assert.deepEqual(stages('clear-msgs'), ['requested', 'drained', 'exited', 'retired']);
|
||||
});
|
||||
|
||||
test('T4 D7: repeat, conflict, a second op, a stale revision, confirm and a halt', (t) => {
|
||||
const { sw, call, code, A, D, stages, clear, count } = setup(t);
|
||||
// A halt refuses and writes nothing.
|
||||
sw.state = { version: 1, halted: true, generation: GENERATION };
|
||||
assert.equal(code(() => clear('human', { session: D, revision: 0, op: 'clear-1' })), 'processing-halted');
|
||||
assert.equal(count('session_ops'), 0);
|
||||
sw.state = { version: 1, halted: false, generation: GENERATION };
|
||||
assert.equal(code(() => clear('human', { session: D, revision: 1, op: 'clear-1' })), 'session-revision-stale');
|
||||
assert.equal(code(() => call('human', 'session.clear', { thread: A, session: D, revision: 0, op: 'clear-1' })), 'invalid-request');
|
||||
assert.equal(code(() => call('human', 'session.clear', { thread: A, session: D, revision: 0, confirm: 'yes', op: 'clear-1' })), 'confirm-required');
|
||||
assert.equal(code(() => clear('human', { session: D, revision: -1, op: 'clear-1' })), 'invalid-request');
|
||||
assert.equal(code(() => clear('human', { session: 'no-such-session', revision: 0, op: 'clear-1' })), 'session-not-found');
|
||||
assert.equal(count('session_ops'), 0);
|
||||
const first = clear('human', { session: D, revision: 0, op: 'clear-1' });
|
||||
// The same op and payload answers the same envelope, id and name, and appends nothing.
|
||||
assert.deepEqual(clear('human', { session: D, revision: 0, op: 'clear-1' }), first);
|
||||
assert.deepEqual(stages('clear-1'), ['requested', 'drained']);
|
||||
assert.equal(code(() => clear('human', { session: D, revision: 1, op: 'clear-1' })), 'session-op-conflict');
|
||||
assert.equal(code(() => clear('human', { session: D, revision: 2, op: 'clear-2' })), 'session-clearing');
|
||||
// B3: an op names a session or a clear in its thread, never both.
|
||||
assert.equal(code(() => call('human', 'session.create', { thread: A, name: 'other', op: 'clear-1' })), 'session-op-conflict');
|
||||
call('human', 'session.create', { thread: A, name: 'side', op: 'op-side' });
|
||||
const side = call('human', 'session.list', { thread: A }).find((r) => r.name === 'side').id;
|
||||
assert.equal(code(() => clear('human', { session: side, revision: 0, op: 'op-side' })), 'session-op-conflict');
|
||||
assert.equal(count('session_ops'), 2);
|
||||
});
|
||||
|
||||
test('T4 Q4: the successor takes <old>-N, the next N free, and a bound name stays taken', (t) => {
|
||||
const { b, call, code, A, D, list, clear, where } = setup(t);
|
||||
call('human', 'session.create', { thread: A, name: 'default-1', op: 'op-d1' });
|
||||
clear('human', { session: D, revision: 0, op: 'clear-1' });
|
||||
assert.equal(code(() => call('human', 'session.create', { thread: A, name: 'default-2', op: 'op-d2' })), 'session-exists');
|
||||
// Cancelled before retirement: the name stays bound to the op.
|
||||
call('human', 'session.cancel', { thread: A, op: 'clear-1', confirm: true });
|
||||
assert.equal(code(() => call('human', 'session.create', { thread: A, name: 'default-2', op: 'op-d2' })), 'session-exists');
|
||||
clear('human', { session: D, revision: 3, op: 'clear-2' });
|
||||
b.sessionClearExited({ ...where('clear-2'), run: null });
|
||||
b.sessionClearRetire(where('clear-2'));
|
||||
assert.deepEqual(list().map((r) => [r.name, r.state]), [
|
||||
['default', 'retired'],
|
||||
['default-1', 'active'],
|
||||
['default-3', 'active'],
|
||||
]);
|
||||
});
|
||||
|
||||
test('T4 D8: the successor start is held, refused or started, each report on its own counter', (t) => {
|
||||
const { b, code, A, D, stages, clear, where, store } = setup(t);
|
||||
clear('human', { session: D, revision: 0, op: 'clear-1' });
|
||||
assert.equal(code(() => b.sessionClearStart({ ...where('clear-1'), start: 'held', refusal: 'capacity-full' })), 'not-retired');
|
||||
b.sessionClearExited({ ...where('clear-1'), run: null });
|
||||
b.sessionClearRetire(where('clear-1'));
|
||||
assert.deepEqual(b.sessionClears('demo').map((c) => c.stage), ['retired']);
|
||||
assert.equal(code(() => b.sessionClearStart({ ...where('clear-1'), start: 'held' })), 'invalid-request');
|
||||
assert.equal(code(() => b.sessionClearStart({ ...where('clear-1'), start: 'started', refusal: 'x' })), 'invalid-request');
|
||||
assert.equal(code(() => b.sessionClearStart({ ...where('clear-1'), start: 'begun', refusal: null })), 'invalid-request');
|
||||
const held = b.sessionClearStart({ ...where('clear-1'), start: 'held', refusal: 'capacity-full' });
|
||||
assert.deepEqual([held.state, held.start, held.refusal], ['retired', 'held', null]);
|
||||
const refused = b.sessionClearStart({ ...where('clear-1'), start: 'refused', refusal: 'launch-failed' });
|
||||
assert.deepEqual([refused.start, refused.refusal], ['refused', 'launch-failed']);
|
||||
assert.deepEqual(b.sessionClears('demo').map((c) => [c.stage, c.refusal]), [['start-refused', 'launch-failed']]);
|
||||
b.sessionClearStart({ ...where('clear-1'), start: 'held', refusal: 'limit-reached' });
|
||||
const started = b.sessionClearStart({ ...where('clear-1'), start: 'started' });
|
||||
assert.deepEqual([started.state, started.start, started.refusal], ['retired', 'started', null]);
|
||||
assert.deepEqual(
|
||||
store.all("SELECT stage, attempt, work_attempt FROM session_ops WHERE op='clear-1' AND stage GLOB 'start-*' ORDER BY seq").map((r) => ({ ...r })),
|
||||
[
|
||||
{ stage: 'start-held', attempt: 1, work_attempt: 1 },
|
||||
{ stage: 'start-refused', attempt: 2, work_attempt: 1 },
|
||||
{ stage: 'start-held', attempt: 3, work_attempt: 2 },
|
||||
{ stage: 'start-started', attempt: 4, work_attempt: 2 },
|
||||
],
|
||||
);
|
||||
assert.deepEqual(b.sessionClears('demo'), []);
|
||||
assert.equal(code(() => b.sessionClearStart({ ...where('clear-1'), start: 'started' })), 'session-clear-complete');
|
||||
assert.equal(stages('clear-1').length, 8);
|
||||
});
|
||||
|
||||
test('T4 D7 N4: a halt at retirement waits and writes nothing; retirement goes once processing resumes', (t) => {
|
||||
const { b, sw, code, D, stages, clear, where } = setup(t);
|
||||
clear('human', { session: D, revision: 0, op: 'clear-1' });
|
||||
b.sessionClearExited({ ...where('clear-1'), run: null });
|
||||
sw.state = { version: 1, halted: true, generation: GENERATION };
|
||||
assert.equal(code(() => b.sessionClearRetire(where('clear-1'))), 'processing-halted');
|
||||
assert.deepEqual(stages('clear-1'), ['requested', 'drained', 'exited']);
|
||||
sw.state = { version: 1, halted: false, generation: GENERATION };
|
||||
assert.equal(b.sessionClearRetire(where('clear-1')).state, 'retired');
|
||||
});
|
||||
|
||||
test('T4 sage-2828: a cancel before retirement makes the session active again, keeps the claim and the cancelled messages, and never makes the successor', (t) => {
|
||||
const { b, call, code, launch, A, D, list, stages, cancels, clear, where, store } = setup(t);
|
||||
launch('a1', A);
|
||||
// Routed to a1's claim but not delivered: the accept cancels it.
|
||||
const m = call('human', 'thread.send', { thread: A, body: 'waiting' });
|
||||
clear('human', { session: D, revision: 0, op: 'clear-1' });
|
||||
const S = successorOf(A, D, 'clear-1');
|
||||
assert.equal(code(() => call('human', 'session.cancel', { thread: A, op: 'clear-1' })), 'invalid-request');
|
||||
assert.equal(code(() => call('human', 'session.cancel', { thread: A, op: 'clear-1', confirm: false })), 'confirm-required');
|
||||
assert.equal(code(() => call('human', 'session.cancel', { thread: A, op: 'no-such-op', confirm: true })), 'operation-not-found');
|
||||
const cancelled = call('human', 'session.cancel', { thread: A, op: 'clear-1', confirm: true });
|
||||
assert.deepEqual([cancelled.state, cancelled.retired, cancelled.start, cancelled.successor], ['cancelled', null, null, S]);
|
||||
assert.deepEqual(call('human', 'session.cancel', { thread: A, op: 'clear-1', confirm: true }), cancelled);
|
||||
assert.deepEqual(call('human', 'session.operation', { thread: A, op: 'clear-1' }), cancelled);
|
||||
assert.deepEqual(stages('clear-1'), ['requested', 'cancelled']);
|
||||
assert.deepEqual(list().map((r) => [r.id, r.state, r.successor]), [[D, 'active', null]]);
|
||||
assert.equal(store.get('SELECT 1 AS x FROM sessions WHERE id=?', S), undefined);
|
||||
assert.deepEqual(b.sessionClears('demo'), []);
|
||||
assert.deepEqual(cancels(D), [m.id]);
|
||||
// The holder kept its claim: a new send reaches it, and the cancelled message never does.
|
||||
call('human', 'thread.send', { thread: A, body: 'after cancel' });
|
||||
assert.deepEqual(call('a1', 'message.receive').map((r) => r.body), ['after cancel']);
|
||||
// The launcher's late reports refuse.
|
||||
assert.equal(code(() => b.sessionClearExited({ ...where('clear-1'), run: null })), 'operation-not-open');
|
||||
assert.equal(code(() => b.sessionClearUncertain(where('clear-1'))), 'operation-not-open');
|
||||
assert.equal(code(() => b.sessionClearRetire(where('clear-1'))), 'not-exited');
|
||||
// A new clear may follow; its revision counts the cancelled op's rows.
|
||||
assert.deepEqual(list().map((r) => r.revision), [2]);
|
||||
assert.equal(clear('human', { session: D, revision: 2, op: 'clear-2' }).state, 'clearing');
|
||||
});
|
||||
|
||||
test('T4 sage-2828 sage-2830: a cancel after retirement ends the startup only, and a later thread-only send makes the successor pending as new input', (t) => {
|
||||
const { b, call, code, A, D, list, stages, clear, where } = setup(t);
|
||||
clear('human', { session: D, revision: 0, op: 'clear-1' });
|
||||
b.sessionClearExited({ ...where('clear-1'), run: null });
|
||||
b.sessionClearRetire(where('clear-1'));
|
||||
b.sessionClearStart({ ...where('clear-1'), start: 'held', refusal: 'capacity-full' });
|
||||
const S = successorOf(A, D, 'clear-1');
|
||||
const cancelled = call('human', 'session.cancel', { thread: A, op: 'clear-1', confirm: true });
|
||||
assert.deepEqual([cancelled.state, cancelled.retired, cancelled.start], ['cancelled', true, 'held']);
|
||||
assert.deepEqual(b.sessionClears('demo'), []);
|
||||
assert.equal(code(() => b.sessionClearStart({ ...where('clear-1'), start: 'started' })), 'session-clear-cancelled');
|
||||
assert.deepEqual(list().map((r) => [r.id, r.state, r.default]), [
|
||||
[D, 'retired', false],
|
||||
[S, 'active', true],
|
||||
]);
|
||||
call('human', 'thread.send', { thread: A, body: 'new input' });
|
||||
assert.deepEqual(b.threadsState('demo').map((r) => [r.session, r.pending]), [[S, 1]]);
|
||||
assert.deepEqual(stages('clear-1'), ['requested', 'drained', 'exited', 'retired', 'start-held', 'cancelled']);
|
||||
});
|
||||
|
||||
// N4: retirement rechecks the owner. Only a user's thread can fail it (a lost grant), and no verb
|
||||
// clears a user's thread in 271: the R3 gate refuses all three clear verbs to a user, and #thread
|
||||
// refuses them to anyone else. So the op's first two rows are written here as the accept writes
|
||||
// them, and the launcher's ops run as they would. Row 264e carry (duckworth-0201): once a user
|
||||
// clear verb lands, these tests take the verb path instead of the raw session_ops inserts.
|
||||
const userClearExited = (t) => {
|
||||
const access = { doc: { accessVersion: 1, revision: 1, users: [], grants: [] } };
|
||||
const ctx = setup(t, { access });
|
||||
const { b, call, code, project, store, where } = ctx;
|
||||
access.doc.grants = [{ id: 'g-1', user: 'alice', business: 'demo', projects: [project], instances: ['assistant'], level: 'collaborator' }];
|
||||
const alice = b.bindUser({ business: 'demo', user: 'alice' });
|
||||
const U = b.request(alice, { verb: 'thread.create', args: { project, name: 'alice work' } }).id;
|
||||
const UD = store.get('SELECT id FROM sessions WHERE thread=?', U).id;
|
||||
assert.equal(code(() => b.request(alice, { verb: 'session.clear', args: { thread: U, session: UD, revision: 0, confirm: true, op: 'u1' } })), 'access-refused');
|
||||
assert.equal(code(() => call('human', 'session.clear', { thread: U, session: UD, revision: 0, confirm: true, op: 'u1' })), 'thread-not-owned');
|
||||
const S = successorOf(U, UD, 'u1');
|
||||
for (const stage of ['requested', 'drained'])
|
||||
store.run(
|
||||
"INSERT INTO session_ops(at,business,thread,session,op,stage,attempt,successor,successor_name) VALUES(?,'demo',?,?,'u1',?,0,?,'default-1')",
|
||||
new Date().toISOString(),
|
||||
U,
|
||||
UD,
|
||||
stage,
|
||||
S,
|
||||
);
|
||||
b.sessionClearExited({ ...where('u1', U), run: null });
|
||||
return { ...ctx, access, U, UD, S };
|
||||
};
|
||||
|
||||
test('T4 N4: a user\'s thread whose grant is gone by retirement ends the clear failed, writes no successor, and leaves the session as it was', (t) => {
|
||||
const { b, code, stages, store, where, access, U, UD, S } = userClearExited(t);
|
||||
access.doc.grants = [];
|
||||
const failed = b.sessionClearRetire(where('u1', U));
|
||||
assert.deepEqual([failed.state, failed.refusal, failed.start], ['failed', 'access-refused', null]);
|
||||
assert.deepEqual(stages('u1', U), ['requested', 'drained', 'exited', 'failed']);
|
||||
assert.equal(store.get('SELECT 1 AS x FROM sessions WHERE id=?', S), undefined);
|
||||
assert.deepEqual(b.sessionClears('demo'), []);
|
||||
// The session is active again, with no successor; the default stays.
|
||||
assert.deepEqual(
|
||||
store.all('SELECT session FROM session_defaults WHERE thread=? ORDER BY seq', U).map((r) => r.session),
|
||||
[UD],
|
||||
);
|
||||
assert.equal(code(() => b.sessionClearStart({ ...where('u1', U), start: 'started' })), 'not-retired');
|
||||
});
|
||||
|
||||
// duckworth-0201 B1: the recheck passes while the user's collaborator grant is live.
|
||||
test('T4 N4: a user\'s thread whose grant is live at retirement retires the session and makes its successor', (t) => {
|
||||
const { b, stages, store, where, U, UD, S } = userClearExited(t);
|
||||
const retired = b.sessionClearRetire(where('u1', U));
|
||||
assert.deepEqual([retired.state, retired.refusal ?? null, retired.start, retired.successor], ['retired', null, null, S]);
|
||||
assert.deepEqual(stages('u1', U), ['requested', 'drained', 'exited', 'retired']);
|
||||
assert.equal(store.get('SELECT predecessor FROM sessions WHERE id=?', S).predecessor, UD);
|
||||
assert.deepEqual(
|
||||
store.all('SELECT session FROM session_defaults WHERE thread=? ORDER BY seq', U).map((r) => r.session),
|
||||
[UD, S],
|
||||
);
|
||||
});
|
||||
|
||||
// This one covers the started successor.
|
||||
test('T4 sage-2828: a cancel after the successor started refuses session-clear-complete', (t) => {
|
||||
const { b, call, code, A, D, clear, where } = setup(t);
|
||||
clear('human', { session: D, revision: 0, op: 'clear-1' });
|
||||
b.sessionClearExited({ ...where('clear-1'), run: null });
|
||||
b.sessionClearRetire(where('clear-1'));
|
||||
b.sessionClearStart({ ...where('clear-1'), start: 'started' });
|
||||
assert.equal(code(() => call('human', 'session.cancel', { thread: A, op: 'clear-1', confirm: true })), 'session-clear-complete');
|
||||
});
|
||||
|
||||
test('T4 B2: a run bound to a clearing session refuses its claim, and claims again once the clear is cancelled', (t) => {
|
||||
const { call, code, launch, A, D, clear } = setup(t);
|
||||
clear('human', { session: D, revision: 0, op: 'clear-1' });
|
||||
assert.equal(code(() => launch('a1', A)), 'session-clearing');
|
||||
assert.equal(code(() => launch('a2', A, D)), 'session-clearing');
|
||||
call('human', 'session.cancel', { thread: A, op: 'clear-1', confirm: true });
|
||||
assert.deepEqual(launch('a3', A), { role: 'assistant', run: 'a3-run', thread: A });
|
||||
});
|
||||
|
||||
test('T4 mos-1325: the tester clears its own thread; its clear, cancel and operation on a human-owned thread refuse thread-not-owned and write nothing', (t) => {
|
||||
const { b, call, code, project, A, D, count } = setup(t);
|
||||
const T = call('tester', 'thread.create', { project, name: 'tester work' }).id;
|
||||
const [{ id: TD }] = call('tester', 'session.list', { thread: T });
|
||||
const own = call('tester', 'session.clear', { thread: T, session: TD, revision: 0, confirm: true, op: 'tclear' });
|
||||
assert.equal(own.state, 'clearing');
|
||||
assert.deepEqual(call('tester', 'session.operation', { thread: T, op: 'tclear' }), own);
|
||||
call('human', 'thread.send', { thread: A, body: 'human waiting' });
|
||||
call('human', 'session.clear', { thread: A, session: D, revision: 0, confirm: true, op: 'hclear' });
|
||||
const before = [count('session_ops'), count('message_cancels')];
|
||||
assert.equal(code(() => call('tester', 'session.clear', { thread: A, session: D, revision: 2, confirm: true, op: 'tx' })), 'thread-not-owned');
|
||||
assert.equal(code(() => call('tester', 'session.cancel', { thread: A, op: 'hclear', confirm: true })), 'thread-not-owned');
|
||||
assert.equal(code(() => call('tester', 'session.operation', { thread: A, op: 'hclear' })), 'thread-not-owned');
|
||||
assert.deepEqual([count('session_ops'), count('message_cancels')], before);
|
||||
// And the human can't reach the tester's.
|
||||
assert.equal(code(() => call('human', 'session.cancel', { thread: T, op: 'tclear', confirm: true })), 'thread-not-owned');
|
||||
// An agent refuses before any check.
|
||||
assert.equal(code(() => call('chief', 'session.clear', { thread: A, session: D, revision: 2, confirm: true, op: 'ax' })), 'human-required');
|
||||
assert.equal(code(() => call('chief', 'session.operation', { thread: A, op: 'hclear' })), 'human-required');
|
||||
assert.deepEqual(b.sessionClears('demo').map((c) => [c.thread, c.op]), [
|
||||
[T, 'tclear'],
|
||||
[A, 'hclear'],
|
||||
]);
|
||||
});
|
||||
|
||||
test('T4 schema: session_ops and message_cancels are append-only, and a second open clear of a session is refused below the broker', (t) => {
|
||||
const { call, A, D, clear, store } = setup(t);
|
||||
const m = call('human', 'thread.send', { thread: A, body: 'w' });
|
||||
clear('human', { session: D, revision: 0, op: 'clear-1' });
|
||||
assert.throws(() => store.run("UPDATE session_ops SET stage='retired'"), /session_ops is append-only/);
|
||||
assert.throws(() => store.run('DELETE FROM session_ops'), /session_ops is append-only/);
|
||||
assert.throws(() => store.run("UPDATE message_cancels SET op='x'"), /message_cancels is append-only/);
|
||||
assert.throws(() => store.run('DELETE FROM message_cancels'), /message_cancels is append-only/);
|
||||
const raw = (op, stage, attempt = 0) =>
|
||||
store.run(
|
||||
"INSERT INTO session_ops(at,business,thread,session,op,stage,attempt,successor,successor_name) VALUES(?,'demo',?,?,?,?,?,'s','n')",
|
||||
new Date().toISOString(),
|
||||
A,
|
||||
D,
|
||||
op,
|
||||
stage,
|
||||
attempt,
|
||||
);
|
||||
assert.throws(() => raw('clear-2', 'requested'), /session has an open clear/);
|
||||
assert.throws(() => raw('clear-1', 'start-held', 0), /CHECK constraint failed/);
|
||||
assert.throws(() => raw('clear-1', 'retired', 1), /CHECK constraint failed/);
|
||||
assert.throws(() => raw('clear-1', 'bogus'), /CHECK constraint failed/);
|
||||
assert.throws(() => raw('clear-1', 'drained'), /UNIQUE constraint failed/);
|
||||
assert.throws(
|
||||
() => store.run("INSERT INTO message_cancels(at,message,session,op) VALUES(?,?,?,'again')", new Date().toISOString(), m.id, D),
|
||||
/UNIQUE constraint failed/,
|
||||
);
|
||||
});
|
||||
|
||||
// The 3m→3n step (sage-2968: row 206 holds 3l→3m). A 3m store from fixtures/schema-3m.sql, which is
|
||||
// refactor's schema.sql at 0cbe2e9b3, with one row in each thread table and a share.
|
||||
const AT = '2026-10-22T00:00:00.000Z';
|
||||
const here = (p) => new URL(p, import.meta.url);
|
||||
const ddl3m = readFileSync(here('./fixtures/schema-3m.sql'), 'utf8');
|
||||
function source3m(t) {
|
||||
const root = mkdtempSync(join(tmpdir(), 'session-clear-migrate-'));
|
||||
t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
mkdirSync(join(root, 'bus'), { mode: 0o700 });
|
||||
const path = join(root, 'bus/bus.sqlite'), db = new DatabaseSync(path);
|
||||
assert.equal(createHash('sha256').update(ddl3m).digest('hex'), 'd5ca9bb9268994b175ac79d38939be4bb7f2e80bc0d9ea273eedaae50c29b8a9');
|
||||
db.exec(ddl3m);
|
||||
const run = (sql, ...args) => db.prepare(sql).run(...args);
|
||||
run('INSERT INTO meta VALUES (?,?)', 'schema_version', '3m');
|
||||
run('INSERT INTO meta VALUES (?,?)', 'schema_digest', schemaDigest(db));
|
||||
run('INSERT INTO meta VALUES (?,?)', 'comment_cursor_epoch', AT);
|
||||
run('INSERT INTO projects(id,at,business,name,repo,created_by) VALUES(?,?,?,?,?,?)', 'p-1', AT, 'demo', 'stack work', 'stack', 'jason');
|
||||
for (const [id, owner] of [['t1', 'jason'], ['t2', 'user:alice']]) {
|
||||
run('INSERT INTO threads(id,at,business,project,instance,owner,name) VALUES(?,?,?,?,?,?,?)', id, AT, 'demo', 'p-1', 'assistant', owner, id);
|
||||
run("INSERT INTO sessions(at,business,thread,id,instance,owner,name,op) VALUES(?,?,?,?,?,?,'default','legacy-3l')", AT, 'demo', id, id, 'assistant', owner);
|
||||
run("INSERT INTO session_defaults(at,thread,session,by) VALUES(?,?,?,'migration-3l')", AT, id, id);
|
||||
}
|
||||
run('INSERT INTO messages(id,at,business,from_role,from_run,to_role,class,body) VALUES(?,?,?,?,?,?,?,?)', 'm-1', AT, 'demo', 'user', 'user:alice', 'assistant', 'INFO', 'm-1');
|
||||
run('INSERT INTO message_threads VALUES(?,?,?)', 'm-1', 't2', 't2');
|
||||
run('INSERT INTO thread_opens(seq,at,thread,session,by) VALUES(?,?,?,?,?)', 7, AT, 't2', 't2', 'user:alice');
|
||||
run('INSERT INTO thread_shares(at,business,thread,op,target,by) VALUES(?,?,?,?,?,?)', AT, 'demo', 't2', 'share', 'jason', 'user:alice');
|
||||
db.close();
|
||||
chmodSync(path, 0o600);
|
||||
return { root, path };
|
||||
}
|
||||
const open = (path, fn) => {
|
||||
const db = new DatabaseSync(path, { readOnly: true });
|
||||
try { return fn(db); } finally { db.close(); }
|
||||
};
|
||||
const master = (path) => open(path, (db) => db.prepare('SELECT type,name,tbl_name,sql FROM sqlite_master ORDER BY type,name').all().map((r) => ({ ...r })));
|
||||
// Every table's rows but meta's, whose version and digest the step changes.
|
||||
const tables = (path) => open(path, (db) => Object.fromEntries(
|
||||
db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name NOT IN ('meta') ORDER BY name").all()
|
||||
.map(({ name }) => [name, db.prepare(`SELECT * FROM "${name}" ORDER BY rowid`).all().map((r) => ({ ...r }))]),
|
||||
));
|
||||
const version = (path) => open(path, (db) => db.prepare("SELECT value FROM meta WHERE key='schema_version'").get().value);
|
||||
|
||||
test('T4 schema: a 3m store migrates to 3n, session_ops and message_cancels are new and empty, and every other row stays as it was', (t) => {
|
||||
const { root, path } = source3m(t);
|
||||
const before = tables(path);
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
assert.equal(version(path), '3n');
|
||||
const after = tables(path);
|
||||
for (const name of ['session_ops', 'message_cancels']) {
|
||||
assert.deepEqual(after[name], [], name);
|
||||
delete after[name];
|
||||
}
|
||||
assert.deepEqual(after, before);
|
||||
// The migrated schema is the fresh one, object for object.
|
||||
const freshRoot = mkdtempSync(join(tmpdir(), 'session-clear-fresh-'));
|
||||
t.after(() => rmSync(freshRoot, { recursive: true, force: true }));
|
||||
new Store(freshRoot).close();
|
||||
assert.deepEqual(master(path), master(join(freshRoot, 'bus/bus.sqlite')));
|
||||
// The new tables are append-only on the migrated store, and a session holds one open clear.
|
||||
const store = new Store(root);
|
||||
t.after(() => store.close());
|
||||
const op = (stage, over = {}) => ['INSERT INTO session_ops(at,business,thread,session,op,stage,attempt,successor,successor_name) VALUES(?,?,?,?,?,?,?,?,?)',
|
||||
AT, 'demo', 't2', 't2', over.op ?? 'c-1', stage, 0, over.successor ?? 's-1', 'default-1'];
|
||||
const [insert, ...args] = op('requested');
|
||||
store.run(insert, ...args);
|
||||
const [again, ...againArgs] = op('requested', { op: 'c-2', successor: 's-2' });
|
||||
assert.throws(() => store.run(again, ...againArgs), /session has an open clear/);
|
||||
store.run('INSERT INTO message_cancels(at,message,session,op) VALUES(?,?,?,?)', AT, 'm-1', 't2', 'c-1');
|
||||
for (const [table, sql, sqlArgs] of [
|
||||
['session_ops', "UPDATE session_ops SET stage='retired'", []],
|
||||
['session_ops', 'DELETE FROM session_ops', []],
|
||||
['session_ops', 'INSERT OR REPLACE INTO session_ops(seq,at,business,thread,session,op,stage,attempt,successor,successor_name) VALUES(?,?,?,?,?,?,?,?,?,?)', [1, AT, 'demo', 't2', 't2', 'c-1', 'drained', 0, 's-1', 'default-1']],
|
||||
['message_cancels', "UPDATE message_cancels SET op='c-9'", []],
|
||||
['message_cancels', 'DELETE FROM message_cancels', []],
|
||||
['message_cancels', 'INSERT OR REPLACE INTO message_cancels(seq,at,message,session,op) VALUES(?,?,?,?,?)', [1, AT, 'm-1', 't2', 'c-9']],
|
||||
]) assert.throws(() => store.run(sql, ...sqlArgs), new RegExp(`${table} is append-only`), sql);
|
||||
assert.throws(() => store.run('INSERT INTO message_cancels(at,message,session,op) VALUES(?,?,?,?)', AT, 'no-such', 't2', 'c-1'), /FOREIGN KEY/);
|
||||
assert.deepEqual(store.all('SELECT seq,thread,session,op,stage FROM session_ops').map((r) => ({ ...r })), [{ seq: 1, thread: 't2', session: 't2', op: 'c-1', stage: 'requested' }]);
|
||||
});
|
||||
|
||||
// The 3m engine is store.mjs at this change with the 3m schema and version and no 3m→3n step.
|
||||
async function engine3m(root) {
|
||||
const moduleRoot = join(root, 'engine-3m');
|
||||
mkdirSync(join(moduleRoot, 'src/migrations'), { recursive: true });
|
||||
const source = readFileSync(here('../src/store.mjs'), 'utf8');
|
||||
const edited = source
|
||||
.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3m).digest('hex')}';`)
|
||||
.replace("export const SCHEMA_VERSION = '3n';", "export const SCHEMA_VERSION = '3m';")
|
||||
.replace(/\}, \{\n {2}\/\/ Row 271 \(#1776\)[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables');
|
||||
assert.ok(edited.includes("export const SCHEMA_VERSION = '3m';") && edited.includes('3l-3m') && !edited.includes('3m-3n'), 'the 3m engine has no 3m→3n step');
|
||||
writeFileSync(join(moduleRoot, 'src/store.mjs'), edited);
|
||||
writeFileSync(join(moduleRoot, 'schema.sql'), ddl3m);
|
||||
for (const file of ['3b.sql', '3b-3c.sql', '3c-3d.sql', '3d-3e.sql', '3e-3f.sql', '3f-3g.sql', '3g-3h.sql', '3h-3i.sql', '3i-3j.sql', '3j-3k.sql', '3k-3l.sql', '3l-3m.sql'])
|
||||
copyFileSync(here('../src/migrations/' + file), join(moduleRoot, 'src/migrations', file));
|
||||
return (await import(pathToFileURL(join(moduleRoot, 'src/store.mjs')))).Store;
|
||||
}
|
||||
|
||||
test('T4 schema: the 3m engine refuses a 3n store, migrating or not, and leaves it as it was', async (t) => {
|
||||
const { root, path } = source3m(t);
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
const was = master(path), before = tables(path);
|
||||
const Store3m = await engine3m(root);
|
||||
assert.throws(() => new Store3m(root), /schema-mismatch/);
|
||||
assert.throws(() => new Store3m(root, { migrateSchema: true }), /schema-mismatch/);
|
||||
assert.equal(version(path), '3n');
|
||||
assert.deepEqual(master(path), was);
|
||||
assert.deepEqual(tables(path), before);
|
||||
// The same engine opens a fresh 3m store, so the refusal is the version's and not a broken copy.
|
||||
const { root: old } = source3m(t);
|
||||
new Store3m(old).close();
|
||||
});
|
||||
@@ -0,0 +1,98 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { mkdtempSync, readFileSync, rmSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { Store } from '../src/store.mjs';
|
||||
import { Broker } from '../src/broker.mjs';
|
||||
|
||||
// Row 270 (264c, #1775, T1 item 7 D6): session.list's `compactions` key, from the host's ledger
|
||||
// reader (runtime.mjs gives the seat's compactionCount). The reader itself is
|
||||
// packages/seat/tests/compactions.test.mjs's.
|
||||
const shipped = (name) =>
|
||||
JSON.parse(readFileSync(fileURLToPath(new URL(`../../../roles/${name}.json`, import.meta.url)), 'utf8'));
|
||||
const businesses = {
|
||||
demo: {
|
||||
id: 'demo',
|
||||
human: 'jason',
|
||||
arbiters: { technical: 'pm', delivery: 'chief' },
|
||||
roles: {
|
||||
chief: { definition: 'chief-of-staff', authority: { withinRole: ['message.send'], crossRole: [] } },
|
||||
assistant: { definition: 'assistant', authority: { withinRole: ['message.send'], crossRole: [] } },
|
||||
pm: { definition: 'pm', authority: { withinRole: ['message.send'], crossRole: [] } },
|
||||
},
|
||||
definitions: { 'chief-of-staff': {}, assistant: { messaging: shipped('assistant').messaging }, pm: {} },
|
||||
projects: { stack: { root: '/nonexistent' } },
|
||||
},
|
||||
};
|
||||
|
||||
function setup(t, options = {}) {
|
||||
const root = mkdtempSync(join(tmpdir(), 'bus-compactions-'));
|
||||
const store = new Store(root);
|
||||
t.after(() => {
|
||||
store.close();
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
const b = new Broker({ store, businesses, ...options });
|
||||
const human = b.bindHuman({ business: 'demo', human: 'jason', via: 'cli', outsideAgent: true });
|
||||
const call = (verb, args = {}) => b.request(human, { verb, args });
|
||||
const project = call('project.create', { name: 'stack work', repo: 'stack' }).id;
|
||||
const A = call('thread.create', { project, name: 'alpha' }).id;
|
||||
return { store, call, project, A };
|
||||
}
|
||||
|
||||
test('session.list rows carry the reader count for each session, with its store identity', (t) => {
|
||||
const asked = [];
|
||||
const f = setup(t, {
|
||||
compactions: (q) => {
|
||||
asked.push(q);
|
||||
return { count: asked.length, known: true, reason: null, at: '2026-10-22T12:00:00.000Z' };
|
||||
},
|
||||
});
|
||||
const S = f.call('session.create', { thread: f.A, name: 'second', op: 'op1' }).id;
|
||||
const rows = f.call('session.list', { thread: f.A });
|
||||
assert.deepEqual(rows.map((r) => [r.id, r.compactions]), [
|
||||
[rows[0].id, { count: 1, known: true, reason: null, at: '2026-10-22T12:00:00.000Z' }],
|
||||
[S, { count: 2, known: true, reason: null, at: '2026-10-22T12:00:00.000Z' }],
|
||||
]);
|
||||
assert.deepEqual(asked, rows.map((r) => ({ business: 'demo', thread: f.A, session: r.id, legacy: false })));
|
||||
assert.ok(rows.every((r) => !('op' in r)));
|
||||
});
|
||||
|
||||
// A legacy session (op legacy-3l, from the 3k-3l migration, its id the thread's) keeps the
|
||||
// thread's store. The thread is made by hand with the session the migration gives it, as
|
||||
// message-filter.test.mjs does; sessions is append-only.
|
||||
test('a legacy session is asked for with legacy true', (t) => {
|
||||
const asked = [];
|
||||
const f = setup(t, { compactions: (q) => (asked.push(q), { count: 0, known: true, reason: null, at: null }) });
|
||||
const L = randomUUID();
|
||||
const at = '2026-10-21T00:00:00.000Z';
|
||||
f.store.run('INSERT INTO threads(id,at,business,project,instance,owner,name) VALUES(?,?,?,?,?,?,?)', L, at, 'demo', f.project, 'assistant', 'jason', 'legacy');
|
||||
f.store.run("INSERT INTO sessions(at,business,thread,id,instance,owner,name,op) SELECT at,business,id,id,instance,owner,'default','legacy-3l' FROM threads WHERE id=?", L);
|
||||
f.store.run("INSERT INTO session_defaults(at,thread,session,by) SELECT at,id,id,'migration-3l' FROM threads WHERE id=?", L);
|
||||
f.call('session.list', { thread: L });
|
||||
assert.deepEqual(asked, [{ business: 'demo', thread: L, session: L, legacy: true }]);
|
||||
});
|
||||
|
||||
test('without a reader the count is unknown, never 0; a reader that throws is store-unreadable', (t) => {
|
||||
const f = setup(t);
|
||||
assert.deepEqual(f.call('session.list', { thread: f.A }).map((r) => r.compactions), [{ count: null, known: false, reason: 'not-collected', at: null }]);
|
||||
const g = setup(t, {
|
||||
compactions: () => {
|
||||
throw new Error('EACCES');
|
||||
},
|
||||
});
|
||||
assert.deepEqual(g.call('session.list', { thread: g.A }).map((r) => r.compactions), [{ count: null, known: false, reason: 'store-unreadable', at: null }]);
|
||||
});
|
||||
|
||||
test('compactions must be a function', (t) => {
|
||||
const root = mkdtempSync(join(tmpdir(), 'bus-compactions-'));
|
||||
const store = new Store(root);
|
||||
t.after(() => {
|
||||
store.close();
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
assert.throws(() => new Broker({ store, businesses, compactions: {} }), { name: 'TypeError', message: 'compactions must be a function' });
|
||||
});
|
||||
@@ -374,7 +374,7 @@ test('T7: a 3k store migrates to 3l, each thread\'s default session has the thre
|
||||
const { root, path } = source3k(t);
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
assert.equal(version(path), '3m');
|
||||
assert.equal(version(path), '3n');
|
||||
// The migrated schema is the fresh one, object for object.
|
||||
const freshRoot = mkdtempSync(join(tmpdir(), 'sessions-fresh-'));
|
||||
t.after(() => rmSync(freshRoot, { recursive: true, force: true }));
|
||||
@@ -430,7 +430,7 @@ async function engine3k(root) {
|
||||
const source = readFileSync(here('../src/store.mjs'), 'utf8');
|
||||
const edited = source
|
||||
.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3k).digest('hex')}';`)
|
||||
.replace("export const SCHEMA_VERSION = '3m';", "export const SCHEMA_VERSION = '3k';")
|
||||
.replace("export const SCHEMA_VERSION = '3n';", "export const SCHEMA_VERSION = '3k';")
|
||||
.replace(/\}, \{\n {2}\/\/ Row 264a \(#1769\)[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables');
|
||||
assert.ok(edited.includes("export const SCHEMA_VERSION = '3k';") && edited.includes('3j-3k') && !edited.includes('3k-3l'), 'the 3k engine has no 3k→3l step');
|
||||
writeFileSync(join(moduleRoot, 'src/store.mjs'), edited);
|
||||
@@ -447,7 +447,7 @@ test('T7 D9: the 3k engine refuses a 3l store, migrating or not, and leaves it a
|
||||
const Store3k = await engine3k(root);
|
||||
assert.throws(() => new Store3k(root), /schema-mismatch/);
|
||||
assert.throws(() => new Store3k(root, { migrateSchema: true }), /schema-mismatch/);
|
||||
assert.equal(version(path), '3m');
|
||||
assert.equal(version(path), '3n');
|
||||
assert.deepEqual(master(path), was);
|
||||
assert.deepEqual(rows(path), before);
|
||||
// The same engine opens a fresh 3k store, so the refusal is the version's and not a broken copy.
|
||||
|
||||
@@ -63,7 +63,7 @@ test('3e copy migrates through 3f to 3g preserving every old column/row and addi
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
const store = new Store(root, { migrateSchema: true });
|
||||
try {
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3m');
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3n');
|
||||
assert.deepEqual({ ...store.get('SELECT spec_path,spec_commit FROM decisions') }, { spec_path: null, spec_commit: null });
|
||||
assert.equal(store.get('PRAGMA quick_check').quick_check, 'ok');
|
||||
assert.deepEqual(store.all('PRAGMA foreign_key_check'), []);
|
||||
|
||||
@@ -25,6 +25,8 @@ test('row 175: TESTER_VERBS and the organization branch tester set are pinned',
|
||||
'instance.launch', 'instance.end', 'instance.request',
|
||||
// Row 264a (#1769): the sessions of the tester's own threads.
|
||||
'session.list', 'session.create',
|
||||
// Row 271 (264d, #1776): clear context in the tester's own threads (mos-1325).
|
||||
'session.clear', 'session.cancel', 'session.operation',
|
||||
]);
|
||||
assert.deepEqual([...TESTER_ORGANIZATION_VERBS], ['limits', 'plugins', 'instances', 'plugins.set', 'instances.set', 'routes.set', 'routes']);
|
||||
assert.ok(Object.isFrozen(TESTER_VERBS) && Object.isFrozen(TESTER_ORGANIZATION_VERBS));
|
||||
|
||||
@@ -49,7 +49,7 @@ async function database3d(t) {
|
||||
const source = readFileSync(here('../src/store.mjs'), 'utf8');
|
||||
const pinned = source
|
||||
.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3d).digest('hex')}';`)
|
||||
.replace("export const SCHEMA_VERSION = '3m';", "export const SCHEMA_VERSION = '3d';")
|
||||
.replace("export const SCHEMA_VERSION = '3n';", "export const SCHEMA_VERSION = '3d';")
|
||||
.replace(/\}, \{\n {2}from: '3d',[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables');
|
||||
assert.ok(!pinned.includes('3d-3e'), 'the 3d engine has no 3d→3e step');
|
||||
writeFileSync(join(moduleRoot, 'src/store.mjs'), pinned);
|
||||
@@ -101,7 +101,7 @@ test('REQ-THREAD-1 REQ-ORG-2: the 3d→3e migration adds the thread tables and l
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
const store = new Store(root, { migrateSchema: true });
|
||||
t.after(() => store.close());
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3m');
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3n');
|
||||
for (const table of ['projects', 'threads', 'thread_claims', 'message_threads', 'thread_opens'])
|
||||
assert.equal(store.get(`SELECT count(*) n FROM ${table}`).n, 0, table);
|
||||
// Parked: the undelivered ones are counted per business and role, and none is changed or given a thread.
|
||||
@@ -161,5 +161,5 @@ test('REQ-THREAD-1: the offline command prints the kept-message counts before th
|
||||
const command = new URL('../src/migrate-schema.mjs', import.meta.url).pathname;
|
||||
const migrated = spawnSync(process.execPath, [command, root], { encoding: 'utf8' });
|
||||
assert.equal(migrated.status, 0, migrated.stderr);
|
||||
assert.equal(migrated.stdout, [...REPORT, 'bus schema 3m ready', ''].join('\n'));
|
||||
assert.equal(migrated.stdout, [...REPORT, 'bus schema 3n ready', ''].join('\n'));
|
||||
});
|
||||
@@ -403,10 +403,15 @@ test('a 3l store migrates to 3m: thread_shares is new and empty, and every other
|
||||
const before = tables(path);
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
assert.equal(version(path), '3m');
|
||||
assert.equal(version(path), '3n');
|
||||
const after = tables(path);
|
||||
assert.deepEqual(after.thread_shares, []);
|
||||
delete after.thread_shares;
|
||||
// Row 271 (#1776): the 3m→3n step's session_ops and message_cancels, new and empty too.
|
||||
for (const name of ['session_ops', 'message_cancels']) {
|
||||
assert.deepEqual(after[name], [], name);
|
||||
delete after[name];
|
||||
}
|
||||
assert.deepEqual(after, before);
|
||||
// The migrated schema is the fresh one, object for object.
|
||||
const freshRoot = mkdtempSync(join(tmpdir(), 'thread-shares-fresh-'));
|
||||
@@ -434,7 +439,7 @@ async function engine3l(root) {
|
||||
const source = readFileSync(here('../src/store.mjs'), 'utf8');
|
||||
const edited = source
|
||||
.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3l).digest('hex')}';`)
|
||||
.replace("export const SCHEMA_VERSION = '3m';", "export const SCHEMA_VERSION = '3l';")
|
||||
.replace("export const SCHEMA_VERSION = '3n';", "export const SCHEMA_VERSION = '3l';")
|
||||
.replace(/\}, \{\n {2}\/\/ Row 206 \(193b2[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables');
|
||||
assert.ok(edited.includes("export const SCHEMA_VERSION = '3l';") && edited.includes('3k-3l') && !edited.includes('3l-3m'), 'the 3l engine has no 3l→3m step');
|
||||
writeFileSync(join(moduleRoot, 'src/store.mjs'), edited);
|
||||
@@ -451,7 +456,7 @@ test('the 3l engine refuses a 3m store, migrating or not, and leaves it as it wa
|
||||
const Store3l = await engine3l(root);
|
||||
assert.throws(() => new Store3l(root), /schema-mismatch/);
|
||||
assert.throws(() => new Store3l(root, { migrateSchema: true }), /schema-mismatch/);
|
||||
assert.equal(version(path), '3m');
|
||||
assert.equal(version(path), '3n');
|
||||
assert.deepEqual(master(path), was);
|
||||
assert.deepEqual(tables(path), before);
|
||||
// The same engine opens a fresh 3l store, so the refusal is the version's and not a broken copy.
|
||||
|
||||
@@ -68,7 +68,7 @@ test('REQ-LIMIT-4 3g copy migrates through 3h, 3i, 3j and 3k to 3l preserving ev
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
const store = new Store(root, { migrateSchema: true });
|
||||
try {
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3m');
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3n');
|
||||
assert.deepEqual({ ...store.get('SELECT to_name FROM messages') }, { to_name: null });
|
||||
assert.equal(store.get('SELECT count(*) AS n FROM usage_records').n, 0);
|
||||
assert.equal(store.get('SELECT name FROM instance_names').name, 'Builder');
|
||||
|
||||
@@ -162,6 +162,7 @@ import {
|
||||
writeSessionFile,
|
||||
} from "../../seat/src/session.mjs";
|
||||
import { BWRAP, sandboxProblem } from "../../seat/src/sandbox.mjs";
|
||||
import { collectCompactions, READ_BYTES, readBudget, readStreams, STREAMS } from "../../seat/src/compactions.mjs";
|
||||
import { CheckoutError, ensureCheckouts } from "../../seat/src/checkout.mjs";
|
||||
import { createGitSocket } from "./git-socket.mjs";
|
||||
import { cmdlineOf, startTimeOf } from "../../seat/src/proc.mjs";
|
||||
@@ -262,6 +263,9 @@ export function createLauncher({
|
||||
const stopping = new Set();
|
||||
const refusedThreads = new Map();
|
||||
const startedFor = new Map();
|
||||
// Row 271 (264d): each clear's stop time (by op), and each clear refusal logged once (by op and code).
|
||||
const clearStops = new Map();
|
||||
const clearLogged = new Map();
|
||||
// Row 194 N6: run -> when its process was spawned, so an end signals only the runs spawned by its verb.
|
||||
const spawnedAt = new Map();
|
||||
// Row 156: run -> the decisions its landing names; decision -> started; decision -> last
|
||||
@@ -283,6 +287,34 @@ export function createLauncher({
|
||||
}
|
||||
};
|
||||
|
||||
// Row 270 (264c, D6): count a thread session's completed compactions into its
|
||||
// store's ledger, at run start (what earlier runs and a resume left) and at
|
||||
// run end. The caller holds the session's workspace lock. A failure is
|
||||
// logged and recorded, and never stops or fails the run: the count then
|
||||
// reads unknown with its reason. Row 270 r2 (B1, mos-1352): each collection
|
||||
// reads at most the system config's compactionReadBytes, a human-only
|
||||
// setting; a missing or bad value takes the default.
|
||||
const budget = readBudget(system?.compactionReadBytes);
|
||||
if (system?.compactionReadBytes !== undefined && budget !== system.compactionReadBytes) {
|
||||
log(`system config: compactionReadBytes ${JSON.stringify(system.compactionReadBytes)} isn't a whole number of bytes from 1048576 to 1073741824; compaction collections read up to the default ${READ_BYTES}`);
|
||||
}
|
||||
// Row 270 r3 (B4, sage-3038): and opens at most compactionStreams stream
|
||||
// files, the same kind of setting.
|
||||
const streams = readStreams(system?.compactionStreams);
|
||||
if (system?.compactionStreams !== undefined && streams !== system.compactionStreams) {
|
||||
log(`system config: compactionStreams ${JSON.stringify(system.compactionStreams)} isn't a whole number from 32 to 65536; compaction collections open up to the default ${STREAMS} streams`);
|
||||
}
|
||||
const compactions = (store, harness, { instance, thread, session, run }) => {
|
||||
let c;
|
||||
try {
|
||||
c = collectCompactions(store, harness, { budget, streams });
|
||||
} catch (e) {
|
||||
log(`run ${run}: compactions: ${e.message}`);
|
||||
c = { count: null, known: false, reason: "store-unreadable" };
|
||||
}
|
||||
record({ event: "compactions", instance, thread, session, run, count: c.count, known: c.known, reason: c.reason });
|
||||
};
|
||||
|
||||
function serialize(fn) {
|
||||
const run = chain.then(fn);
|
||||
chain = run.catch(() => {});
|
||||
@@ -472,6 +504,7 @@ export function createLauncher({
|
||||
const { run, runDir } = newRun(dataRoot, b.id);
|
||||
const sessionDir = store ? store.session : join(runDir, "session");
|
||||
if (!store) mkdirSync(sessionDir, { mode: 0o700 });
|
||||
if (store) compactions(store, model.harness, { instance, thread, session, run });
|
||||
const toolSocket = join(runDir, "tools.sock");
|
||||
const turnMarker = join(runDir, "turn.done");
|
||||
const version = harnessVersion !== undefined ? harnessVersion : call(model.harness, "version", () => harness.version({ env }));
|
||||
@@ -612,6 +645,7 @@ export function createLauncher({
|
||||
} finally {
|
||||
try { unregisterRuntime(dataRoot, managed.id); }
|
||||
catch { log(`run ${run}: processing receipt cleanup refused`); }
|
||||
if (store) compactions(store, model.harness, { instance, thread, session, run });
|
||||
try { lock?.release(); }
|
||||
catch (e) { log(`run ${run}: workspace lock release: ${e.message}`); }
|
||||
children.delete(run);
|
||||
@@ -657,9 +691,12 @@ export function createLauncher({
|
||||
const ms = (at) => (at ? Date.parse(at) : 0);
|
||||
// Row 269: a thread with a legacy session keeps its old store for that session only.
|
||||
const legacyThreads = new Set(threads.filter((t) => t.legacy).map((t) => t.thread));
|
||||
// Row 271 (264d): a session in a clear is the clear pass's, and nothing below starts or idles it.
|
||||
const clearing = await clearPass(now, legacyThreads);
|
||||
for (const t of threads) {
|
||||
if (closed) return;
|
||||
const key = `${t.thread}/${t.session}`;
|
||||
if (clearing.has(t.session)) continue;
|
||||
const s = registry.running(b.id, t.instance, t.thread, t.session);
|
||||
if (s) {
|
||||
const last = Math.max(ms(s.startedAt), ms(t.lastMessage), ms(t.lastOpen));
|
||||
@@ -707,6 +744,112 @@ export function createLauncher({
|
||||
}
|
||||
}
|
||||
|
||||
// Row 271 (264d, #1776, D7 steps 3-5): one step of each clear the broker lists. The drain comes
|
||||
// from the run itself (its receive, or its end), so a requested clear waits here. A drained run
|
||||
// gets the idle stop's start-time-checked SIGTERM once, and 30 s without its exit proof records
|
||||
// uncertain once; nothing is signalled again or forced. The proof is the run's own close handler,
|
||||
// whose endRun the broker checks, or for no run, the session's workspace lock. Then retirement,
|
||||
// and the successor's start through the same check()/start() as any session. Returns the
|
||||
// sessions those steps own this tick.
|
||||
async function clearPass(now, legacyThreads) {
|
||||
const owned = new Set();
|
||||
let clears;
|
||||
try {
|
||||
clears = await host.op({ op: "sessionClears", business: b.id });
|
||||
} catch (e) {
|
||||
clearLog("sessionClears", e.code ?? e.message, `sessionClears refused: ${e.code ?? e.message}`);
|
||||
return owned;
|
||||
}
|
||||
for (const c of clears) {
|
||||
if (closed) break;
|
||||
const start = ["retired", "start-held", "start-refused"].includes(c.stage);
|
||||
owned.add(start ? c.successor : c.session);
|
||||
try {
|
||||
await (start ? clearStart(c, legacyThreads) : clearStop(c, now));
|
||||
} catch (e) {
|
||||
clearLog(c.op, e.code ?? e.message, `clear ${c.op} (session ${c.session}): ${e.code ?? e.message}`);
|
||||
}
|
||||
}
|
||||
return owned;
|
||||
}
|
||||
|
||||
function clearLog(op, code, line) {
|
||||
if (clearLogged.get(op) === code) return;
|
||||
clearLogged.set(op, code);
|
||||
log(line);
|
||||
}
|
||||
|
||||
async function clearStop(c, now) {
|
||||
const where = { business: b.id, thread: c.thread, op: c.op };
|
||||
let stage = c.stage;
|
||||
if (stage === "drained" || stage === "uncertain") {
|
||||
const s = registry.running(b.id, c.instance, c.thread, c.session);
|
||||
if (s && s.run === c.run) {
|
||||
if (!clearStops.has(c.op)) {
|
||||
// An uncertain clear was signalled before this launcher started, and isn't signalled again.
|
||||
if (stage === "uncertain") return void clearStops.set(c.op, now);
|
||||
// A cancel since the snapshot leaves the run alone; one after this read is a residual.
|
||||
const fresh = (await host.op({ op: "sessionClears", business: b.id })).find((x) => x.thread === c.thread && x.op === c.op);
|
||||
if (fresh?.stage !== "drained") return;
|
||||
clearStops.set(c.op, now);
|
||||
stopping.add(s.run);
|
||||
record({ event: "session-clear-stop", instance: c.instance, thread: c.thread, session: c.session, run: s.run, op: c.op });
|
||||
try {
|
||||
if (startTimeOf(s.runnerPid) === s.runnerStartTime) process.kill(s.runnerPid, "SIGTERM");
|
||||
} catch {}
|
||||
} else if (stage === "drained" && now - clearStops.get(c.op) >= stopTimeoutMs) {
|
||||
await host.op({ op: "sessionClearUncertain", record: where });
|
||||
record({ event: "session-clear-uncertain", instance: c.instance, thread: c.thread, session: c.session, run: s.run, op: c.op });
|
||||
}
|
||||
return;
|
||||
}
|
||||
// Some other run of the session (one that never claimed) exits on its refused claim first.
|
||||
if (s) return;
|
||||
// An earlier host's run that outlived recover()'s kill was ended host-lost, but it still runs:
|
||||
// its exit, not that end, is the proof (row 269 Q2). It isn't signalled.
|
||||
if (survivors.some((x) => x.business === b.id && x.run === c.run && liveRun(x))) return;
|
||||
if (c.run === null) {
|
||||
// No run: the proof is the session's workspace lock, which a spawned run holds until its close.
|
||||
const live = registry.sessions.filter((x) => x.business === b.id && x.workspace).map((x) => x.workspace);
|
||||
const workspace = sessionWorkspace(dataRoot, b.id, c.instance, c.session, { live });
|
||||
lockSessionWorkspace(workspace, { hostPid: process.pid, hostStartTime: startTimeOf(process.pid), business: b.id, instance: c.instance, thread: c.thread, session: c.session }).release();
|
||||
}
|
||||
await host.op({ op: "sessionClearExited", record: { ...where, run: c.run } });
|
||||
stage = "exited";
|
||||
}
|
||||
if (stage !== "exited") return;
|
||||
const out = await host.op({ op: "sessionClearRetire", record: where });
|
||||
clearStops.delete(c.op);
|
||||
record({ event: out.state === "retired" ? "session-clear-retired" : "session-clear-failed", instance: c.instance, thread: c.thread, session: c.session, successor: c.successor, op: c.op, ...(out.refusal ? { code: out.refusal } : {}) });
|
||||
}
|
||||
|
||||
// D8: capacity and limits hold a start, which spends no work attempt. Any other refusal is one.
|
||||
const CLEAR_HELD = new Set(["capacity-full", "limit-reached"]);
|
||||
|
||||
async function clearStart(c, legacyThreads) {
|
||||
const where = { business: b.id, thread: c.thread, op: c.op };
|
||||
await serialize(async () => {
|
||||
if (closed) return;
|
||||
if (!registry.running(b.id, c.instance, c.thread, c.successor)) {
|
||||
const session = { thread: c.thread, session: c.successor };
|
||||
try {
|
||||
await start({ instance: c.instance, launchedBy: b.human, ...session, legacy: false, legacyThread: legacyThreads.has(c.thread), ...(await check(c.instance, session)) });
|
||||
} catch (e) {
|
||||
const code = e instanceof Refusal ? e.code : "launch-failed";
|
||||
// Reported and logged once per code, not once per tick.
|
||||
if (clearLogged.get(c.op) === code) return;
|
||||
clearLogged.set(c.op, code);
|
||||
log(`clear ${c.op}: successor ${c.successor} (${c.instance}): start ${CLEAR_HELD.has(code) ? "held" : "refused"}: ${code}`);
|
||||
record({ event: "refused", by: b.human, instance: c.instance, thread: c.thread, session: c.successor, code, why: e.message, ...(e.limit ? { limit: e.limit } : {}) });
|
||||
await host.op({ op: "sessionClearStart", record: { ...where, start: CLEAR_HELD.has(code) ? "held" : "refused", refusal: code } });
|
||||
return;
|
||||
}
|
||||
}
|
||||
clearLogged.delete(c.op);
|
||||
await host.op({ op: "sessionClearStart", record: { ...where, start: "started" } });
|
||||
});
|
||||
}
|
||||
|
||||
// Row 156 (#1651): start each listed merge decision holder once, and stop a
|
||||
// running holder that started before its decision's resolution.
|
||||
async function landPass() {
|
||||
|
||||
@@ -0,0 +1,350 @@
|
||||
// Compaction counts on a real run (row 270, 264c, #1775, T3's Pi half): the
|
||||
// launcher collects a thread session's completed compactions into its
|
||||
// store's ledger at run start and at run end, and session.list reads the
|
||||
// ledger through the runtime's reader. The count holds across a resume and a
|
||||
// host restart. Real runners in the bwrap sandbox; a stub Pi adapter writes
|
||||
// the session stream (fixtures/compaction-adapter.mjs).
|
||||
//
|
||||
// Round 2 (duckworth-0223 B2, sage-3031, mos-1352): a failed collection, the
|
||||
// read budget from the system config, a clear's retired session and its
|
||||
// successor, and a legacy session read through the runtime's reader. Round 3
|
||||
// (duckworth-0226 B4, sage-3038): the stream cap from the system config.
|
||||
//
|
||||
// The host is session-launch.test.mjs's: startBroker on bootConfig's config,
|
||||
// with the ops the broker child forwards, and the human capability from
|
||||
// bindHuman directly.
|
||||
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { existsSync, mkdirSync, readFileSync, rmSync, truncateSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { launchDoc } from "../../business/tests/helpers.mjs";
|
||||
import { BusError } from "../../bus/src/broker.mjs";
|
||||
import { startBroker } from "../../bus/src/runtime.mjs";
|
||||
import { Store } from "../../bus/src/store.mjs";
|
||||
import { collectCompactions, LEDGER } from "../../seat/src/compactions.mjs";
|
||||
import { gitEnv } from "../../seat/src/checkout.mjs";
|
||||
import { sandboxProblem } from "../../seat/src/sandbox.mjs";
|
||||
import { launchLogFile, SESSION_CREDENTIALS, sessionStore } from "../../seat/src/session.mjs";
|
||||
import { bootConfig, loadSystem } from "../src/config.mjs";
|
||||
import { createLauncher } from "../src/launcher.mjs";
|
||||
import { fakeHarnesses, fixture, tmp } from "./helpers.mjs";
|
||||
|
||||
const ADAPTER = fileURLToPath(new URL("./fixtures/compaction-adapter.mjs", import.meta.url));
|
||||
const skip = sandboxProblem() ?? false;
|
||||
|
||||
async function until(fn, ms = 20000) {
|
||||
const end = Date.now() + ms;
|
||||
while (Date.now() < end) {
|
||||
const v = await fn();
|
||||
if (v) return v;
|
||||
await new Promise((r) => setTimeout(r, 50));
|
||||
}
|
||||
throw new Error("timed out waiting");
|
||||
}
|
||||
// A mutant's kill is an assertion failure, not a timeout (sage-2707), so a wait a mutant could
|
||||
// stall returns what it last saw and the assertions after it decide.
|
||||
async function settle(fn, ms = 5000) {
|
||||
const end = Date.now() + ms;
|
||||
let v;
|
||||
while (Date.now() < end) {
|
||||
v = await fn();
|
||||
if (v) return v;
|
||||
await new Promise((r) => setTimeout(r, 50));
|
||||
}
|
||||
return v;
|
||||
}
|
||||
|
||||
function gitIn(cwd, ...args) {
|
||||
const r = spawnSync("git", ["-c", "user.name=test", "-c", "[email protected]", "-C", cwd, ...args], { env: gitEnv(), encoding: "utf8" });
|
||||
assert.equal(r.status, 0, r.stderr);
|
||||
}
|
||||
|
||||
// The assistant is the thread instance, on Pi.
|
||||
function prepare(t) {
|
||||
const root = tmp(t, "mosaic-compactions-");
|
||||
const f = fixture(root, "acme", (doc) => {
|
||||
doc.roles.chief.vars = { harness: "pi", model: "claude-opus-5-5", thinking: "high" };
|
||||
doc.roles.assistant = { definition: "assistant", vars: { harness: "pi", model: "claude-sonnet-5-5", thinking: "high" } };
|
||||
doc.launch = launchDoc("chief", ["coder", "assistant"], { opus: 1, sonnet: 3 });
|
||||
doc.launch.threadIdleSeconds = 1;
|
||||
return doc;
|
||||
});
|
||||
for (const k of SESSION_CREDENTIALS.flat()) delete f.env[k];
|
||||
const project = f.doc.projects.stack.root;
|
||||
gitIn(project, "init", "-q", "-b", "main");
|
||||
writeFileSync(join(project, "file.txt"), "one\n");
|
||||
gitIn(project, "add", "file.txt");
|
||||
gitIn(project, "commit", "-q", "-m", "one");
|
||||
const adapter = join(root, "adapter.sh");
|
||||
writeFileSync(adapter, `exec '${process.execPath}' '${ADAPTER}'\n`, { mode: 0o644 });
|
||||
return { root, f, adapter };
|
||||
}
|
||||
|
||||
// A host and its launcher on the prepared root.
|
||||
async function start(t, { f, adapter }) {
|
||||
const system = loadSystem({ env: f.env });
|
||||
const { trackers, ...boot } = bootConfig({ system, businessId: "acme", env: f.env });
|
||||
assert.equal(trackers, undefined);
|
||||
const runtime = await startBroker(boot);
|
||||
const ops = {
|
||||
identity: (m) => runtime.identity(m.cap),
|
||||
authorizeLaunch: (m) => runtime.authorizeLaunch(m.cap, m.instance),
|
||||
refuse: (m) => runtime.refuse(m.cap, m.code, m.subject),
|
||||
endLaunch: (m) => runtime.endLaunch(m.record),
|
||||
credentialStatus: (m) => runtime.credentialStatus(m.business, m.role),
|
||||
threadsState: (m) => runtime.threadsState(m.business),
|
||||
threadIdle: (m) => runtime.threadIdle(m.record),
|
||||
mayLaunch: (m) => runtime.mayLaunch(m),
|
||||
landHeld: (m) => runtime.landHeld(m.record),
|
||||
landHolders: (m) => runtime.landHolders(m.business),
|
||||
landRelease: (m) => runtime.landRelease(m.record),
|
||||
sessionClears: (m) => runtime.sessionClears(m.business),
|
||||
sessionClearExited: (m) => runtime.sessionClearExited(m.record),
|
||||
sessionClearUncertain: (m) => runtime.sessionClearUncertain(m.record),
|
||||
sessionClearRetire: (m) => runtime.sessionClearRetire(m.record),
|
||||
sessionClearStart: (m) => runtime.sessionClearStart(m.record),
|
||||
};
|
||||
const refusing = (what, fn) => async (m) => {
|
||||
try {
|
||||
return fn(m);
|
||||
} catch (e) {
|
||||
const code = e instanceof BusError ? e.code : `${what}-refused`;
|
||||
throw Object.assign(new Error(`${what} refused: ${code}`), { code });
|
||||
}
|
||||
};
|
||||
const hooks = [];
|
||||
let closing = null;
|
||||
const host = {
|
||||
business: "acme",
|
||||
path: runtime.path,
|
||||
dataRoot: f.dataRoot,
|
||||
op: refusing("launch-op", (m) => {
|
||||
if (!Object.hasOwn(ops, m.op)) throw new BusError("invalid-host-request");
|
||||
return ops[m.op](m);
|
||||
}),
|
||||
bindLaunch: refusing("launch bind", (r) => runtime.bindLaunch(r)),
|
||||
beforeClose: (fn) => void hooks.push(fn),
|
||||
};
|
||||
const logs = [];
|
||||
const launcher = createLauncher({
|
||||
host,
|
||||
system,
|
||||
env: f.env,
|
||||
makeLoader: await fakeHarnesses(t, { pi: adapter, "claude-code": adapter }),
|
||||
sessionDefaults: { pollInterval: 100 },
|
||||
harnessVersion: "0.85.1",
|
||||
readOnly: [ADAPTER],
|
||||
threadTickMs: 100,
|
||||
log: (l) => logs.push(l),
|
||||
});
|
||||
const close = () =>
|
||||
(closing ??= (async () => {
|
||||
for (const fn of hooks) await fn(0);
|
||||
await runtime.close();
|
||||
})());
|
||||
t.after(close);
|
||||
await launcher.listen();
|
||||
const human = runtime.broker.bindHuman({ business: "acme", human: f.doc.human, via: "cli", outsideAgent: true });
|
||||
const call = (verb, args = {}) => runtime.broker.request(human, { verb, args });
|
||||
return { runtime, launcher, call, logs, close };
|
||||
}
|
||||
|
||||
const launchLog = (f) =>
|
||||
existsSync(launchLogFile(f.dataRoot, "acme"))
|
||||
? readFileSync(launchLogFile(f.dataRoot, "acme"), "utf8").trim().split("\n").map((l) => JSON.parse(l))
|
||||
: [];
|
||||
const events = (f, event) => launchLog(f).filter((e) => e.event === event);
|
||||
const counted = (f) => events(f, "compactions").map((e) => [e.session, e.count, e.known, e.reason]);
|
||||
|
||||
// Send to a thread's default session and wait for the assistant's RESULT reply.
|
||||
async function ask(call, thread, body) {
|
||||
const sent = call("thread.send", { thread, body });
|
||||
const r = await until(() => call("thread.messages", { thread }).find((m) => m.in_reply_to === sent.id && m.class === "RESULT"));
|
||||
assert.equal(r.from_role, "assistant");
|
||||
return JSON.parse(r.body.match(/THREAD (\{.*\})/)[1]);
|
||||
}
|
||||
|
||||
const line = (e) => `${JSON.stringify(e)}\n`;
|
||||
|
||||
test("REQ-THREAD-2 T3 a run collects compactions at its start and its end, and the count holds across a resume and a host restart", { skip, timeout: 90000 }, async (t) => {
|
||||
const prepared = prepare(t);
|
||||
const { f } = prepared;
|
||||
let host = await start(t, prepared);
|
||||
const project = host.call("project.create", { name: "stack work", repo: "stack" }).id;
|
||||
const A = host.call("thread.create", { project, name: "alpha" }).id;
|
||||
const [{ id: D }] = host.call("session.list", { thread: A });
|
||||
// An earlier run left a compaction no run-end collection saw (a host that died mid-run).
|
||||
// Only the run-start collection counts it before the agent's first turn.
|
||||
const store = sessionStore(f.dataRoot, "acme", A, D);
|
||||
writeFileSync(join(store.session, "pi.jsonl"), line({ type: "session", version: 3, id: "pi", timestamp: "2026-10-22T11:00:00.000Z", cwd: "/w" }) + line({ type: "compaction", id: "c0", parentId: null, timestamp: "2026-10-22T10:59:00.000Z", summary: "s", firstKeptEntryId: "k", tokensBefore: 100 }), { mode: 0o600 });
|
||||
|
||||
// Run 1: start counts c0 (M13), the turn compacts c1, the end counts it (M14).
|
||||
assert.deepEqual(await ask(host.call, A, "compact c1"), { body: "compact c1", before: [] });
|
||||
await until(() => events(f, "end").length === 1, 30000);
|
||||
await settle(() => counted(f).length >= 2);
|
||||
assert.deepEqual(counted(f), [[D, 1, true, null], [D, 2, true, null]]);
|
||||
|
||||
// A restart: a new host and launcher on the same data root. Run 2 resumes the session.
|
||||
await host.close();
|
||||
host = await start(t, prepared);
|
||||
assert.deepEqual(await ask(host.call, A, "compact c2"), { body: "compact c2", before: ["compact c1"] });
|
||||
await until(() => events(f, "end").length === 2, 30000);
|
||||
await settle(() => counted(f).length >= 4);
|
||||
assert.deepEqual(counted(f), [[D, 1, true, null], [D, 2, true, null], [D, 2, true, null], [D, 3, true, null]]);
|
||||
|
||||
// session.list reads the ledger through the runtime's reader (M15), after another restart.
|
||||
await host.close();
|
||||
host = await start(t, prepared);
|
||||
const [row] = host.call("session.list", { thread: A });
|
||||
assert.equal(row.id, D);
|
||||
assert.deepEqual({ ...row.compactions, at: typeof row.compactions?.at }, { count: 3, known: true, reason: null, at: "string" });
|
||||
});
|
||||
|
||||
test("REQ-THREAD-2 T3 a collection that fails is logged and recorded store-unreadable, and the run ends as one whose collections count", { skip, timeout: 90000 }, async (t) => {
|
||||
const prepared = prepare(t);
|
||||
const { f } = prepared;
|
||||
const host = await start(t, prepared);
|
||||
const project = host.call("project.create", { name: "stack work", repo: "stack" }).id;
|
||||
const A = host.call("thread.create", { project, name: "alpha" }).id;
|
||||
const [{ id: D }] = host.call("session.list", { thread: A });
|
||||
// The ledger's path is a directory, so readLedger refuses it (ledger-unsafe).
|
||||
const store = sessionStore(f.dataRoot, "acme", A, D);
|
||||
mkdirSync(join(store.dir, LEDGER));
|
||||
assert.deepEqual(await ask(host.call, A, "compact c1"), { body: "compact c1", before: [] });
|
||||
await until(() => events(f, "end").length === 1, 30000);
|
||||
await settle(() => counted(f).length >= 2);
|
||||
assert.deepEqual(counted(f), [[D, null, false, "store-unreadable"], [D, null, false, "store-unreadable"]]);
|
||||
const [{ run: first }] = events(f, "launch");
|
||||
assert.equal(host.logs.filter((l) => l.startsWith(`run ${first}: compactions: `)).length, 2);
|
||||
// With the ledger's path free again, the next run's collections count what the stream holds.
|
||||
rmSync(join(store.dir, LEDGER), { recursive: true });
|
||||
assert.deepEqual(await ask(host.call, A, "compact c2"), { body: "compact c2", before: ["compact c1"] });
|
||||
await until(() => events(f, "end").length === 2, 30000);
|
||||
await settle(() => counted(f).length >= 4);
|
||||
assert.deepEqual(counted(f).slice(2), [[D, 1, true, null], [D, 2, true, null]]);
|
||||
const ends = events(f, "end").map((e) => [e.reason, e.exitCode, e.signal]);
|
||||
assert.deepEqual(ends[0], ends[1]);
|
||||
});
|
||||
|
||||
test("REQ-THREAD-2 T3 the read budget is the system config's compactionReadBytes, and a bad value takes the default", { skip, timeout: 90000 }, async (t) => {
|
||||
const prepared = prepare(t);
|
||||
const { f } = prepared;
|
||||
const setting = (v) => {
|
||||
const doc = JSON.parse(readFileSync(f.config, "utf8"));
|
||||
doc.compactionReadBytes = v;
|
||||
writeFileSync(f.config, JSON.stringify(doc));
|
||||
};
|
||||
setting(1 << 20);
|
||||
let host = await start(t, prepared);
|
||||
const project = host.call("project.create", { name: "stack work", repo: "stack" }).id;
|
||||
const A = host.call("thread.create", { project, name: "alpha" }).id;
|
||||
const [{ id: D }] = host.call("session.list", { thread: A });
|
||||
// c0, then a 2 MiB hole with no newline: a line past 1 MiB, inside the 64 MiB default.
|
||||
const store = sessionStore(f.dataRoot, "acme", A, D);
|
||||
const path = join(store.session, "pi.jsonl");
|
||||
const whole = line({ type: "session", version: 3, id: "pi", timestamp: "2026-10-22T11:00:00.000Z", cwd: "/w" }) + line({ type: "compaction", id: "c0", parentId: null, timestamp: "2026-10-22T10:59:00.000Z", summary: "s", firstKeptEntryId: "k", tokensBefore: 100 });
|
||||
writeFileSync(path, whole, { mode: 0o600 });
|
||||
truncateSync(path, Buffer.byteLength(whole) + 2 * (1 << 20));
|
||||
assert.deepEqual(await ask(host.call, A, "one"), { body: "one", before: [] });
|
||||
await until(() => events(f, "end").length === 1, 30000);
|
||||
await settle(() => counted(f).length >= 2);
|
||||
assert.deepEqual(counted(f), [[D, 1, false, "not-collected"], [D, 1, false, "not-collected"]]);
|
||||
assert.deepEqual(host.logs.filter((l) => l.startsWith("system config: ")), []);
|
||||
|
||||
setting("x");
|
||||
await host.close();
|
||||
host = await start(t, prepared);
|
||||
assert.deepEqual(host.logs.filter((l) => l.startsWith("system config: ")), [
|
||||
'system config: compactionReadBytes "x" isn\'t a whole number of bytes from 1048576 to 1073741824; compaction collections read up to the default 67108864',
|
||||
]);
|
||||
assert.deepEqual(await ask(host.call, A, "two"), { body: "two", before: ["one"] });
|
||||
await until(() => events(f, "end").length === 2, 30000);
|
||||
await settle(() => counted(f).length >= 4);
|
||||
assert.deepEqual(counted(f).slice(2), [[D, 1, true, null], [D, 1, true, null]]);
|
||||
});
|
||||
|
||||
test("REQ-THREAD-2 T3 the stream cap is the system config's compactionStreams, and a bad value takes the default", { skip, timeout: 90000 }, async (t) => {
|
||||
const prepared = prepare(t);
|
||||
const { f } = prepared;
|
||||
const setting = (v) => {
|
||||
const doc = JSON.parse(readFileSync(f.config, "utf8"));
|
||||
doc.compactionStreams = v;
|
||||
writeFileSync(f.config, JSON.stringify(doc));
|
||||
};
|
||||
setting(32);
|
||||
let host = await start(t, prepared);
|
||||
const project = host.call("project.create", { name: "stack work", repo: "stack" }).id;
|
||||
const A = host.call("thread.create", { project, name: "alpha" }).id;
|
||||
const [{ id: D }] = host.call("session.list", { thread: A });
|
||||
// 33 empty streams that sort before pi.jsonl: 34 in all, past a cap of 32, so pi.jsonl and its
|
||||
// c0 aren't opened. The default cap opens all 34.
|
||||
const store = sessionStore(f.dataRoot, "acme", A, D);
|
||||
for (let i = 0; i < 33; i++) writeFileSync(join(store.session, `e${String(i).padStart(2, "0")}.jsonl`), "", { mode: 0o600 });
|
||||
writeFileSync(join(store.session, "pi.jsonl"), line({ type: "session", version: 3, id: "pi", timestamp: "2026-10-22T11:00:00.000Z", cwd: "/w" }) + line({ type: "compaction", id: "c0", parentId: null, timestamp: "2026-10-22T10:59:00.000Z", summary: "s", firstKeptEntryId: "k", tokensBefore: 100 }), { mode: 0o600 });
|
||||
assert.deepEqual(await ask(host.call, A, "one"), { body: "one", before: [] });
|
||||
await until(() => events(f, "end").length === 1, 30000);
|
||||
await settle(() => counted(f).length >= 2);
|
||||
assert.deepEqual(counted(f), [[D, null, false, "not-collected"], [D, null, false, "not-collected"]]);
|
||||
assert.deepEqual(host.logs.filter((l) => l.startsWith("system config: ")), []);
|
||||
|
||||
setting("x");
|
||||
await host.close();
|
||||
host = await start(t, prepared);
|
||||
assert.deepEqual(host.logs.filter((l) => l.startsWith("system config: ")), [
|
||||
'system config: compactionStreams "x" isn\'t a whole number from 32 to 65536; compaction collections open up to the default 1024 streams',
|
||||
]);
|
||||
assert.deepEqual(await ask(host.call, A, "two"), { body: "two", before: ["one"] });
|
||||
await until(() => events(f, "end").length === 2, 30000);
|
||||
await settle(() => counted(f).length >= 4);
|
||||
assert.deepEqual(counted(f).slice(2), [[D, 1, true, null], [D, 1, true, null]]);
|
||||
});
|
||||
|
||||
test("REQ-THREAD-2 T3 T4 a clear leaves the retired session's count as it was, its successor counts from 0, and a legacy session reads the thread's ledger", { skip, timeout: 90000 }, async (t) => {
|
||||
const prepared = prepare(t);
|
||||
const { f } = prepared;
|
||||
let host = await start(t, prepared);
|
||||
const project = host.call("project.create", { name: "stack work", repo: "stack" }).id;
|
||||
const A = host.call("thread.create", { project, name: "alpha" }).id;
|
||||
const [{ id: D }] = host.call("session.list", { thread: A });
|
||||
assert.deepEqual(await ask(host.call, A, "compact c1"), { body: "compact c1", before: [] });
|
||||
await until(() => events(f, "end").length === 1, 30000);
|
||||
await settle(() => counted(f).length >= 2);
|
||||
assert.deepEqual(counted(f), [[D, 0, true, null], [D, 1, true, null]]);
|
||||
|
||||
// The run has ended, so the clear retires D at once and starts the successor, whose run-start
|
||||
// collection is its first.
|
||||
const S = host.call("session.clear", { thread: A, session: D, revision: 0, confirm: true, op: "clear-1" }).successor;
|
||||
const done = await settle(() => {
|
||||
const e = host.call("session.operation", { thread: A, op: "clear-1" });
|
||||
return e.start === "started" && e;
|
||||
}, 30000);
|
||||
assert.deepEqual([done?.state, done?.start], ["retired", "started"]);
|
||||
await settle(() => counted(f).some(([s]) => s === S));
|
||||
assert.deepEqual(counted(f).filter(([s]) => s === S)[0], [S, 0, true, null]);
|
||||
const list = () => Object.fromEntries(host.call("session.list", { thread: A }).map((r) => [r.id, r.compactions && { count: r.compactions.count, known: r.compactions.known, reason: r.compactions.reason }]));
|
||||
assert.deepEqual(list(), { [D]: { count: 1, known: true, reason: null }, [S]: { count: 0, known: true, reason: null } });
|
||||
|
||||
// A legacy session (op legacy-3l, from the 3k-3l migration): its store is the thread's, made by
|
||||
// hand as packages/bus/tests/session-compactions.test.mjs does, with a ledger the launcher's
|
||||
// collect wrote.
|
||||
await until(() => events(f, "end").length === 2, 30000);
|
||||
await host.close();
|
||||
const L = "4b1c3a52-9a51-4d7e-8f2e-6c0a1b2d3e4f";
|
||||
const seeded = new Store(f.dataRoot);
|
||||
const at = "2026-10-21T00:00:00.000Z";
|
||||
seeded.run("INSERT INTO threads(id,at,business,project,instance,owner,name) VALUES(?,?,?,?,?,?,?)", L, at, "acme", project, "assistant", f.doc.human, "legacy");
|
||||
seeded.run("INSERT INTO sessions(at,business,thread,id,instance,owner,name,op) SELECT at,business,id,id,instance,owner,'default','legacy-3l' FROM threads WHERE id=?", L);
|
||||
seeded.run("INSERT INTO session_defaults(at,thread,session,by) SELECT at,id,id,'migration-3l' FROM threads WHERE id=?", L);
|
||||
seeded.close();
|
||||
const legacy = sessionStore(f.dataRoot, "acme", L, L, { legacy: true });
|
||||
writeFileSync(join(legacy.session, "pi.jsonl"), line({ type: "session", version: 3, id: "pi", timestamp: "2026-10-22T11:00:00.000Z", cwd: "/w" }) + line({ type: "compaction", id: "c7", parentId: null, timestamp: "2026-10-22T11:07:00.000Z", summary: "s", firstKeptEntryId: "k", tokensBefore: 100 }), { mode: 0o600 });
|
||||
assert.equal(collectCompactions(legacy, "pi").count, 1);
|
||||
host = await start(t, prepared);
|
||||
const [row] = host.call("session.list", { thread: L });
|
||||
assert.equal(row.id, L);
|
||||
assert.deepEqual({ ...row.compactions, at: typeof row.compactions?.at }, { count: 1, known: true, reason: null, at: "string" });
|
||||
assert.deepEqual(list(), { [D]: { count: 1, known: true, reason: null }, [S]: { count: 0, known: true, reason: null } });
|
||||
});
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
// A stand-in harness turn for the clear context tests (row 271, #1776).
|
||||
// session-adapter.mjs's turn, plus what the run sees of the pid namespace:
|
||||
// the numeric entries of /proc and pid 1's argv, so a test can tell that the
|
||||
// session runs in its own namespace (N6).
|
||||
|
||||
import { appendFileSync, existsSync, readdirSync, readFileSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
|
||||
const request = process.env.MOSAIC_REQUEST ?? "";
|
||||
const body = request.match(/^Message [^\n]*:\n\n([\s\S]*)$/m)?.[1] ?? "";
|
||||
const historyFile = join(process.env.MOSAIC_SESSION_DIR, "history");
|
||||
const before = existsSync(historyFile) ? readFileSync(historyFile, "utf8").split("\n").filter(Boolean) : [];
|
||||
const cwd = process.cwd();
|
||||
const pids = readdirSync("/proc").filter((n) => /^\d+$/.test(n)).map(Number).sort((a, b) => a - b);
|
||||
const pid1 = readFileSync("/proc/1/cmdline", "utf8").split("\0").filter(Boolean);
|
||||
appendFileSync(historyFile, `${body}\n`);
|
||||
process.stdout.write(`THREAD ${JSON.stringify({ body, before, cwd, pids, pid1 })}\n`);
|
||||
writeFileSync(process.env.MOSAIC_TURN_MARKER, "done\n");
|
||||
@@ -0,0 +1,24 @@
|
||||
// A stand-in Pi turn for the compaction launch tests (row 270, #1775).
|
||||
// session-adapter.mjs's turn, plus a Pi session stream in
|
||||
// $MOSAIC_SESSION_DIR: a session header when the stream is new, and for a
|
||||
// message "compact <id>" one completed compaction entry with that id, in
|
||||
// Pi 0.85.1's shape.
|
||||
|
||||
import { appendFileSync, existsSync, readFileSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
|
||||
const request = process.env.MOSAIC_REQUEST ?? "";
|
||||
const body = request.match(/^Message [^\n]*:\n\n([\s\S]*)$/m)?.[1] ?? "";
|
||||
const historyFile = join(process.env.MOSAIC_SESSION_DIR, "history");
|
||||
const before = existsSync(historyFile) ? readFileSync(historyFile, "utf8").split("\n").filter(Boolean) : [];
|
||||
const stream = join(process.env.MOSAIC_SESSION_DIR, "pi.jsonl");
|
||||
const line = (e) => `${JSON.stringify(e)}\n`;
|
||||
if (!existsSync(stream)) appendFileSync(stream, line({ type: "session", version: 3, id: "pi", timestamp: "2026-10-22T11:00:00.000Z", cwd: process.cwd() }));
|
||||
const id = body.match(/^compact (c\d+)$/)?.[1];
|
||||
if (id) {
|
||||
const timestamp = new Date(Date.UTC(2026, 9, 22, 11, Number(id.slice(1)))).toISOString();
|
||||
appendFileSync(stream, line({ type: "compaction", id, parentId: null, timestamp, summary: "s", firstKeptEntryId: "k", tokensBefore: 100 }));
|
||||
}
|
||||
appendFileSync(historyFile, `${body}\n`);
|
||||
process.stdout.write(`THREAD ${JSON.stringify({ body, before })}\n`);
|
||||
writeFileSync(process.env.MOSAIC_TURN_MARKER, "done\n");
|
||||
@@ -0,0 +1,397 @@
|
||||
// Clear context (row 271, 264d, #1776, T4): the launcher half of D7 and D8. A clear of a running
|
||||
// session stops its drained run once, retires it on the run's own end, and starts the successor
|
||||
// through the same check()/start() as any session; a held start waits for capacity; an earlier
|
||||
// host's live run of the cleared session holds the clear until it exits; and a session runs in its
|
||||
// own pid namespace (N6); a bus that can't answer holds a clear where it is (T6). Real runners in
|
||||
// the bwrap sandbox, with clear-adapter.mjs as the turn.
|
||||
// The bus half is packages/bus/tests/session-clear.test.mjs.
|
||||
//
|
||||
// The host is session-launch.test.mjs's, with the five clear ops the broker child forwards.
|
||||
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { spawn, spawnSync } from "node:child_process";
|
||||
import { existsSync, readFileSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { launchDoc } from "../../business/tests/helpers.mjs";
|
||||
import { BusError } from "../../bus/src/broker.mjs";
|
||||
import { startBroker } from "../../bus/src/runtime.mjs";
|
||||
import { gitEnv } from "../../seat/src/checkout.mjs";
|
||||
import { startTimeOf } from "../../seat/src/proc.mjs";
|
||||
import { BWRAP, sandboxProblem } from "../../seat/src/sandbox.mjs";
|
||||
import { launchLogFile, readSessions, Registry, SESSION_CREDENTIALS } from "../../seat/src/session.mjs";
|
||||
import { bootConfig, loadSystem } from "../src/config.mjs";
|
||||
import { createLauncher } from "../src/launcher.mjs";
|
||||
import { fakeHarnesses, fixture, tmp } from "./helpers.mjs";
|
||||
|
||||
const ADAPTER = fileURLToPath(new URL("./fixtures/clear-adapter.mjs", import.meta.url));
|
||||
const skip = sandboxProblem() ?? false;
|
||||
|
||||
async function until(fn, ms = 20000) {
|
||||
const end = Date.now() + ms;
|
||||
while (Date.now() < end) {
|
||||
const v = await fn();
|
||||
if (v) return v;
|
||||
await new Promise((r) => setTimeout(r, 50));
|
||||
}
|
||||
throw new Error("timed out waiting");
|
||||
}
|
||||
// A mutant's kill is an assertion failure, not a timeout (sage-2707): the waits return what they
|
||||
// last saw and the assertions after them decide.
|
||||
async function settle(fn, ms = 20000) {
|
||||
const end = Date.now() + ms;
|
||||
let v;
|
||||
while (Date.now() < end) {
|
||||
v = await fn();
|
||||
if (v) return v;
|
||||
await new Promise((r) => setTimeout(r, 50));
|
||||
}
|
||||
return v;
|
||||
}
|
||||
|
||||
function gitIn(cwd, ...args) {
|
||||
const r = spawnSync("git", ["-c", "user.name=test", "-c", "[email protected]", "-C", cwd, ...args], { env: gitEnv(), encoding: "utf8" });
|
||||
assert.equal(r.status, 0, r.stderr);
|
||||
}
|
||||
|
||||
// The assistant is the thread instance; sonnet is its model's limit.
|
||||
function prepare(t, { sonnet = 3, idle = 30 } = {}) {
|
||||
const root = tmp(t, "mosaic-clear-");
|
||||
const f = fixture(root, "acme", (doc) => {
|
||||
doc.roles.chief.vars = { harness: "pi", model: "claude-opus-5-5", thinking: "high" };
|
||||
doc.roles.assistant = { definition: "assistant", vars: { harness: "pi", model: "claude-sonnet-5-5", thinking: "high" } };
|
||||
doc.launch = launchDoc("chief", ["coder", "assistant"], { opus: 1, sonnet });
|
||||
doc.launch.threadIdleSeconds = idle;
|
||||
return doc;
|
||||
});
|
||||
for (const k of SESSION_CREDENTIALS.flat()) delete f.env[k];
|
||||
const project = f.doc.projects.stack.root;
|
||||
gitIn(project, "init", "-q", "-b", "main");
|
||||
writeFileSync(join(project, "file.txt"), "one\n");
|
||||
gitIn(project, "add", "file.txt");
|
||||
gitIn(project, "commit", "-q", "-m", "one");
|
||||
const adapter = join(root, "adapter.sh");
|
||||
writeFileSync(adapter, `exec '${process.execPath}' '${ADAPTER}'\n`, { mode: 0o644 });
|
||||
return { root, f, adapter };
|
||||
}
|
||||
|
||||
async function start(t, { f, adapter }, { tickMs = 100, stopTimeoutMs } = {}) {
|
||||
const system = loadSystem({ env: f.env });
|
||||
const { trackers, ...boot } = bootConfig({ system, businessId: "acme", env: f.env });
|
||||
assert.equal(trackers, undefined);
|
||||
const runtime = await startBroker(boot);
|
||||
const ops = {
|
||||
identity: (m) => runtime.identity(m.cap),
|
||||
authorizeLaunch: (m) => runtime.authorizeLaunch(m.cap, m.instance),
|
||||
refuse: (m) => runtime.refuse(m.cap, m.code, m.subject),
|
||||
endLaunch: (m) => runtime.endLaunch(m.record),
|
||||
credentialStatus: (m) => runtime.credentialStatus(m.business, m.role),
|
||||
threadsState: (m) => runtime.threadsState(m.business),
|
||||
threadIdle: (m) => runtime.threadIdle(m.record),
|
||||
mayLaunch: (m) => runtime.mayLaunch(m),
|
||||
landHeld: (m) => runtime.landHeld(m.record),
|
||||
landHolders: (m) => runtime.landHolders(m.business),
|
||||
landRelease: (m) => runtime.landRelease(m.record),
|
||||
sessionClears: (m) => runtime.sessionClears(m.business),
|
||||
sessionClearExited: (m) => runtime.sessionClearExited(m.record),
|
||||
sessionClearUncertain: (m) => runtime.sessionClearUncertain(m.record),
|
||||
sessionClearRetire: (m) => runtime.sessionClearRetire(m.record),
|
||||
sessionClearStart: (m) => runtime.sessionClearStart(m.record),
|
||||
};
|
||||
const refusing = (what, fn) => async (m) => {
|
||||
try {
|
||||
return fn(m);
|
||||
} catch (e) {
|
||||
const code = e instanceof BusError ? e.code : `${what}-refused`;
|
||||
throw Object.assign(new Error(`${what} refused: ${code}`), { code });
|
||||
}
|
||||
};
|
||||
const hooks = [];
|
||||
let closing = null;
|
||||
const host = {
|
||||
business: "acme",
|
||||
path: runtime.path,
|
||||
dataRoot: f.dataRoot,
|
||||
op: refusing("launch-op", (m) => {
|
||||
if (!Object.hasOwn(ops, m.op)) throw new BusError("invalid-host-request");
|
||||
return ops[m.op](m);
|
||||
}),
|
||||
bindLaunch: refusing("launch bind", (r) => runtime.bindLaunch(r)),
|
||||
beforeClose: (fn) => void hooks.push(fn),
|
||||
};
|
||||
const logs = [];
|
||||
const launcher = createLauncher({
|
||||
host,
|
||||
system,
|
||||
env: f.env,
|
||||
makeLoader: await fakeHarnesses(t, { pi: adapter, "claude-code": adapter }),
|
||||
sessionDefaults: { pollInterval: 100 },
|
||||
harnessVersion: "0.85.1",
|
||||
readOnly: [ADAPTER],
|
||||
threadTickMs: tickMs,
|
||||
...(stopTimeoutMs ? { stopTimeoutMs } : {}),
|
||||
log: (l) => logs.push(l),
|
||||
});
|
||||
const close = () =>
|
||||
(closing ??= (async () => {
|
||||
for (const fn of hooks) await fn(0);
|
||||
await runtime.close();
|
||||
})());
|
||||
t.after(close);
|
||||
await launcher.listen();
|
||||
const human = runtime.broker.bindHuman({ business: "acme", human: f.doc.human, via: "cli", outsideAgent: true });
|
||||
const call = (verb, args = {}) => runtime.broker.request(human, { verb, args });
|
||||
return { host, runtime, launcher, call, logs, close };
|
||||
}
|
||||
|
||||
const launchLog = (f) =>
|
||||
existsSync(launchLogFile(f.dataRoot, "acme"))
|
||||
? readFileSync(launchLogFile(f.dataRoot, "acme"), "utf8").trim().split("\n").map((l) => JSON.parse(l))
|
||||
: [];
|
||||
const events = (f, event) => launchLog(f).filter((e) => e.event === event);
|
||||
const sessionWs = (f, session) => join(f.dataRoot, "session-workspaces", "acme", "assistant", session);
|
||||
const historyOf = (f, thread, session) => {
|
||||
const p = join(f.dataRoot, "threads", "acme", thread, "sessions", session, "session", "history");
|
||||
return existsSync(p) ? readFileSync(p, "utf8") : null;
|
||||
};
|
||||
|
||||
function send(call, thread, session, body) {
|
||||
const sent = call("thread.send", { thread, ...(session ? { session } : {}), body });
|
||||
const reply = async () => {
|
||||
const r = await until(() => call("thread.messages", { thread, ...(session ? { session } : {}) }).find((m) => m.in_reply_to === sent.id && m.class === "RESULT"));
|
||||
assert.equal(r.from_role, "assistant");
|
||||
return JSON.parse(r.body.match(/THREAD (\{.*\})/)[1]);
|
||||
};
|
||||
return { sent, reply };
|
||||
}
|
||||
|
||||
function thread(call) {
|
||||
const project = call("project.create", { name: "stack work", repo: "stack" }).id;
|
||||
const A = call("thread.create", { project, name: "alpha" }).id;
|
||||
const [{ id: D }] = call("session.list", { thread: A });
|
||||
return { project, A, D };
|
||||
}
|
||||
|
||||
test("REQ-THREAD-2 T4 D7 D8 N6: a clear of a running session stops its drained run once, retires it on the run's end, and starts the successor fresh in its own namespace", { skip, timeout: 90000 }, async (t) => {
|
||||
const prepared = prepare(t);
|
||||
const { f } = prepared;
|
||||
const host = await start(t, prepared);
|
||||
const { A, D } = thread(host.call);
|
||||
const first = await send(host.call, A, D, "before the clear").reply();
|
||||
assert.deepEqual([first.before, first.cwd], [[], sessionWs(f, D)]);
|
||||
// N6: the run sees its own pid namespace. Pid 1 there is the sandboxed command, not bwrap, and
|
||||
// the host's processes aren't listed.
|
||||
assert.notEqual(first.pid1[0], BWRAP);
|
||||
assert.ok(first.pids.length < 20, JSON.stringify(first.pids));
|
||||
assert.equal(first.pids.includes(1), true);
|
||||
const [{ run }] = events(f, "launch");
|
||||
|
||||
const op = host.call("session.clear", { thread: A, session: D, revision: 0, confirm: true, op: "clear-1" });
|
||||
const S = op.successor;
|
||||
const done = await settle(() => {
|
||||
const e = host.call("session.operation", { thread: A, op: "clear-1" });
|
||||
return e.start === "started" && e;
|
||||
}, 30000);
|
||||
assert.deepEqual([done.state, done.start, done.refusal], ["retired", "started", null]);
|
||||
// The drained run got one stop, and ended stopped; the clear retired it after that end.
|
||||
const stops = events(f, "session-clear-stop");
|
||||
assert.deepEqual(stops.map((e) => [e.session, e.run, e.op]), [[D, run, "clear-1"]]);
|
||||
assert.deepEqual(events(f, "session-clear-uncertain"), []);
|
||||
const kinds = launchLog(f).map((e) => [e.event, e.session ?? null]).filter(([e]) => ["session-clear-stop", "end", "session-clear-retired", "launch"].includes(e));
|
||||
assert.deepEqual(kinds, [
|
||||
["launch", D],
|
||||
["session-clear-stop", D],
|
||||
["end", D],
|
||||
["session-clear-retired", D],
|
||||
["launch", S],
|
||||
]);
|
||||
assert.deepEqual(events(f, "session-clear-retired").map((e) => [e.successor, e.op]), [[S, "clear-1"]]);
|
||||
// The successor is the default now; a thread-only send reaches it, with no history and its own
|
||||
// workspace. The old session's store stays as it was.
|
||||
const after = await send(host.call, A, null, "after the clear").reply();
|
||||
assert.deepEqual([after.body, after.before, after.cwd], ["after the clear", [], sessionWs(f, S)]);
|
||||
assert.equal(historyOf(f, A, D), "before the clear\n");
|
||||
assert.equal(historyOf(f, A, S), "after the clear\n");
|
||||
assert.deepEqual(host.call("session.list", { thread: A }).map((r) => [r.id, r.state, r.default]), [
|
||||
[D, "retired", false],
|
||||
[S, "active", true],
|
||||
]);
|
||||
assert.deepEqual(readSessions(f.dataRoot).map((s) => s.session), [S]);
|
||||
});
|
||||
|
||||
test("REQ-THREAD-2 T4 D8: a successor start refused capacity-full is held, reported and logged once, and starts when capacity frees", { skip, timeout: 90000 }, async (t) => {
|
||||
const prepared = prepare(t, { sonnet: 1, idle: 4 });
|
||||
const { f } = prepared;
|
||||
const host = await start(t, prepared);
|
||||
const { A, D } = thread(host.call);
|
||||
const N = host.call("session.create", { thread: A, name: "side", op: "op-side" }).id;
|
||||
// The side session takes the one sonnet slot; the default never ran, so its clear has no run.
|
||||
await send(host.call, A, N, "fill the slot").reply();
|
||||
const S = host.call("session.clear", { thread: A, session: D, revision: 0, confirm: true, op: "clear-1" }).successor;
|
||||
const held = await settle(() => {
|
||||
const e = host.call("session.operation", { thread: A, op: "clear-1" });
|
||||
return e.start === "held" && e;
|
||||
});
|
||||
assert.deepEqual([held.state, held.start, held.refusal], ["retired", "held", null]);
|
||||
assert.deepEqual(events(f, "session-clear-stop"), [], "no run to stop");
|
||||
// The side session idles out, and the held start goes on the next tick after.
|
||||
const started = await settle(() => {
|
||||
const e = host.call("session.operation", { thread: A, op: "clear-1" });
|
||||
return e.start === "started" && e;
|
||||
}, 30000);
|
||||
assert.equal(started.start, "started");
|
||||
assert.deepEqual(events(f, "launch").map((e) => e.session), [N, S]);
|
||||
const lines = host.logs.filter((l) => l.startsWith("clear clear-1:"));
|
||||
assert.deepEqual(lines, [`clear clear-1: successor ${S} (assistant): start held: capacity-full`]);
|
||||
assert.deepEqual(events(f, "refused").map((e) => [e.session, e.code]), [[S, "capacity-full"]]);
|
||||
});
|
||||
|
||||
// sage-2830, Q2: after a restart, an earlier host's run of the cleared session that outlived
|
||||
// recover() was ended host-lost, but it still runs. Its exit, not that end, is the proof, and it
|
||||
// isn't signalled. A sleep child stands in for its runner, as in session-launch.test.mjs's Q2 test.
|
||||
test("REQ-THREAD-2 T4 D7 Q2: after a restart, a live survivor of the cleared run holds the clear at drained with no signal, and its exit lets it retire", { skip, timeout: 90000 }, async (t) => {
|
||||
const prepared = prepare(t);
|
||||
const { f } = prepared;
|
||||
const one = await start(t, prepared);
|
||||
const { A, D } = thread(one.call);
|
||||
const pid = 2 ** 22 + 7;
|
||||
const record = { business: "acme", role: "assistant", run: "run-left", harness: "pi", pid, startTime: "1", thread: A, session: D };
|
||||
const left = one.runtime.bindLaunch(record);
|
||||
one.runtime.broker.request(left.cap, { verb: "role.claim" });
|
||||
one.call("session.clear", { thread: A, session: D, revision: 0, confirm: true, op: "clear-1" });
|
||||
assert.deepEqual(one.runtime.broker.request(left.cap, { verb: "message.receive" }), []);
|
||||
assert.deepEqual(one.runtime.sessionClears("acme").map((c) => [c.stage, c.run]), [["drained", "run-left"]]);
|
||||
await one.close();
|
||||
|
||||
const runner = spawn("sleep", ["60"], { stdio: "ignore" });
|
||||
const gone = new Promise((r) => runner.once("exit", r));
|
||||
t.after(() => runner.kill("SIGKILL"));
|
||||
const runnerStartTime = await until(() => startTimeOf(runner.pid));
|
||||
new Registry(f.dataRoot).add({ business: "acme", instance: "assistant", thread: A, session: D, run: "run-left", harness: "pi", pid, startTime: "1", runnerPid: runner.pid, runnerStartTime });
|
||||
const two = await start(t, prepared);
|
||||
assert.ok(two.logs.includes(`run run-left (assistant) from an earlier host ended: host-lost`), two.logs.join("\n"));
|
||||
// Ended in the bus, live on the host: the clear waits, and nothing is signalled.
|
||||
for (let i = 0; i < 5; i++) await two.launcher.tick();
|
||||
assert.deepEqual(two.runtime.sessionClears("acme").map((c) => c.stage), ["drained"]);
|
||||
assert.deepEqual(events(f, "session-clear-stop"), []);
|
||||
assert.equal(startTimeOf(runner.pid), runnerStartTime, "the survivor wasn't signalled");
|
||||
|
||||
runner.kill("SIGKILL");
|
||||
await gone;
|
||||
const done = await settle(() => {
|
||||
const e = two.call("session.operation", { thread: A, op: "clear-1" });
|
||||
return e.start === "started" && e;
|
||||
}, 30000);
|
||||
assert.deepEqual([done.state, done.start], ["retired", "started"]);
|
||||
assert.deepEqual(events(f, "session-clear-stop"), []);
|
||||
});
|
||||
|
||||
// D7 step 3: a drained run that doesn't exit within the stop timeout. The ticks are the test's, so
|
||||
// its runner is stopped from outside its namespace before the clear's one TERM, as
|
||||
// launcher.test.mjs's SIGTERM test does, and the TERM waits. The clear records uncertain once,
|
||||
// sends nothing more, and retires when the run's end comes.
|
||||
test("REQ-THREAD-2 T4 D7: a drained run that outlives the stop timeout makes the clear uncertain once, with one signal, and it retires on the run's end", { skip, timeout: 90000 }, async (t) => {
|
||||
const prepared = prepare(t);
|
||||
const { f } = prepared;
|
||||
const host = await start(t, prepared, { tickMs: 3600000, stopTimeoutMs: 1000 });
|
||||
const tick = () => host.launcher.tick();
|
||||
const { A, D } = thread(host.call);
|
||||
const first = send(host.call, A, D, "before the clear");
|
||||
await tick();
|
||||
await first.reply();
|
||||
const [s] = readSessions(f.dataRoot);
|
||||
const kill = () => {
|
||||
if (startTimeOf(s.runnerPid) === s.runnerStartTime) process.kill(s.runnerPid, "SIGKILL");
|
||||
};
|
||||
t.after(kill);
|
||||
host.call("session.clear", { thread: A, session: D, revision: 0, confirm: true, op: "clear-1" });
|
||||
// The run's own receive drains it; no tick is needed for that.
|
||||
const drained = await settle(() => host.runtime.sessionClears("acme").find((c) => c.stage === "drained"));
|
||||
assert.equal(drained?.run, s.run);
|
||||
process.kill(s.runnerPid, "SIGSTOP");
|
||||
await tick();
|
||||
assert.equal(events(f, "session-clear-stop").length, 1);
|
||||
const uncertain = await settle(async () => {
|
||||
await tick();
|
||||
return host.runtime.sessionClears("acme").find((c) => c.stage === "uncertain");
|
||||
}, 15000);
|
||||
assert.equal(uncertain?.run, s.run);
|
||||
for (let i = 0; i < 5; i++) await tick();
|
||||
assert.equal(events(f, "session-clear-stop").length, 1);
|
||||
assert.equal(events(f, "session-clear-uncertain").length, 1);
|
||||
assert.equal(startTimeOf(s.runnerPid), s.runnerStartTime, "nothing forced it");
|
||||
process.kill(s.runnerPid, "SIGCONT");
|
||||
const done = await settle(async () => {
|
||||
await tick();
|
||||
const e = host.call("session.operation", { thread: A, op: "clear-1" });
|
||||
return e.start === "started" && e;
|
||||
}, 30000);
|
||||
assert.deepEqual([done.state, done.start], ["retired", "started"]);
|
||||
assert.equal(events(f, "session-clear-stop").length, 1);
|
||||
assert.equal(events(f, "session-clear-uncertain").length, 1);
|
||||
});
|
||||
|
||||
// T6 (sage-3001): the bus that proves the stop is configured but unavailable. Its host's op
|
||||
// rejects as host.mjs does when the broker exits before it replies (broker-exited), first at the
|
||||
// exit proof, then at retirement. While it is down nothing retires, no successor launches and the
|
||||
// cleared session's slot isn't handed on; each failure is logged once, and the clear resumes where
|
||||
// it stopped once the bus answers again.
|
||||
test("REQ-THREAD-2 T4 T6: a bus unavailable at the exit proof and then at retirement holds the clear with no retirement, successor or launch, and it resumes when the bus is back", { skip, timeout: 90000 }, async (t) => {
|
||||
const prepared = prepare(t);
|
||||
const { f } = prepared;
|
||||
const host = await start(t, prepared);
|
||||
const down = new Set();
|
||||
const op = host.host.op;
|
||||
host.host.op = (m) => (down.has(m.op) ? Promise.reject(Object.assign(new Error("broker exited before it replied"), { code: "broker-exited" })) : op(m));
|
||||
const { A, D } = thread(host.call);
|
||||
await send(host.call, A, D, "before the clear").reply();
|
||||
const stage = () => host.runtime.sessionClears("acme").map((c) => c.stage);
|
||||
const lines = () => host.logs.filter((l) => l.startsWith("clear clear-1") || l.startsWith("thread tick:"));
|
||||
const line = `clear clear-1 (session ${D}): broker-exited`;
|
||||
const held = () => {
|
||||
assert.deepEqual(events(f, "session-clear-retired"), []);
|
||||
assert.deepEqual(events(f, "session-clear-failed"), []);
|
||||
assert.deepEqual(events(f, "launch").map((e) => e.session), [D]);
|
||||
assert.deepEqual(readSessions(f.dataRoot), [], "nothing runs in the cleared session's slot");
|
||||
assert.deepEqual(host.call("session.list", { thread: A }).map((r) => [r.id, r.default]), [[D, true]]);
|
||||
};
|
||||
|
||||
down.add("sessionClearExited");
|
||||
const S = host.call("session.clear", { thread: A, session: D, revision: 0, confirm: true, op: "clear-1" }).successor;
|
||||
// The drained run is stopped and ends; the exit proof is what fails.
|
||||
await settle(() => events(f, "end").length === 1 && host.logs.includes(line));
|
||||
await new Promise((r) => setTimeout(r, 1000));
|
||||
assert.deepEqual(lines(), [line]);
|
||||
assert.deepEqual(stage(), ["drained"]);
|
||||
assert.equal(events(f, "session-clear-stop").length, 1);
|
||||
held();
|
||||
|
||||
down.clear();
|
||||
down.add("sessionClearRetire");
|
||||
await settle(() => stage()[0] === "exited");
|
||||
await new Promise((r) => setTimeout(r, 1000));
|
||||
assert.deepEqual(stage(), ["exited"]);
|
||||
assert.deepEqual(lines(), [line], "the same code isn't logged again");
|
||||
held();
|
||||
|
||||
down.clear();
|
||||
const done = await settle(() => {
|
||||
const e = host.call("session.operation", { thread: A, op: "clear-1" });
|
||||
return e.start === "started" && e;
|
||||
}, 30000);
|
||||
assert.deepEqual([done.state, done.start, done.refusal], ["retired", "started", null]);
|
||||
const kinds = launchLog(f).map((e) => [e.event, e.session ?? null]).filter(([e]) => ["session-clear-stop", "end", "session-clear-retired", "launch"].includes(e));
|
||||
assert.deepEqual(kinds, [
|
||||
["launch", D],
|
||||
["session-clear-stop", D],
|
||||
["end", D],
|
||||
["session-clear-retired", D],
|
||||
["launch", S],
|
||||
]);
|
||||
assert.deepEqual(lines(), [line]);
|
||||
assert.deepEqual(host.call("session.list", { thread: A }).map((r) => [r.id, r.state, r.default]), [
|
||||
[D, "retired", false],
|
||||
[S, "active", true],
|
||||
]);
|
||||
});
|
||||
@@ -858,7 +858,7 @@ const COUNTED=/^COUNTED="([^"]*)"$/m.exec(srText)[1].split(' ');
|
||||
const STORE_JS=/^STORE_JS='([\s\S]*?)'$/m.exec(srText)[1];
|
||||
// sage-1163's list 2, written out so that a table dropped from the script's COUNTED fails T45 as
|
||||
// well as T46.
|
||||
const LISTED=['role_claims','thread_claims','decision_events','messages','deliveries','task_snapshots','projects','threads','sessions','session_defaults','thread_opens','thread_shares','instance_names','usage_records'];
|
||||
const LISTED=['role_claims','thread_claims','decision_events','messages','deliveries','task_snapshots','projects','threads','sessions','session_defaults','thread_opens','thread_shares','instance_names','usage_records','session_ops','message_cancels'];
|
||||
const LATE='2999-01-01T00:00:00.000Z';
|
||||
const addRow=(file,table,at=LATE)=>{const db=new DatabaseSync(file);db.exec(`CREATE TABLE IF NOT EXISTS ${table} (seq INTEGER PRIMARY KEY, at TEXT NOT NULL)`);db.prepare(`INSERT INTO ${table} (at) VALUES (?)`).run(at);db.close()};
|
||||
|
||||
|
||||
@@ -0,0 +1,318 @@
|
||||
// Row 270 (264c, #1775, T1 item 7 D6): the count of completed compactions over
|
||||
// one thread session's lifetime, across its runs and native branches.
|
||||
//
|
||||
// The host collects it from the session's native store and appends what it
|
||||
// found to a ledger, <store dir>/compactions.jsonl (0600, append-only). The
|
||||
// store dir holds the mounted session/ and claude-projects/ but is not mounted
|
||||
// itself (sandbox.mjs binds()), so the agent can't reach the ledger. The
|
||||
// launcher is its only writer, and collects while it holds the session's
|
||||
// workspace lock (session.mjs lockSessionWorkspace); each collection is one
|
||||
// O_APPEND write. The count is rebuilt from the ledger, never from a page.
|
||||
//
|
||||
// Pi (pinned 0.85.1): a completed compaction is one `type: "compaction"` entry,
|
||||
// written by SessionManager.appendCompaction after the summary exists. A start
|
||||
// or an aborted compaction writes none. A `compactionSummary` message, a
|
||||
// message inside a compaction's retainedTail and a branch_summary are not
|
||||
// compactions. Pi's createBranchedSession and forkFrom copy entries into a new
|
||||
// file of the same directory with their id and timestamp unchanged, so a
|
||||
// compaction's key is (entry id, entry timestamp) over the whole store, and the
|
||||
// stream it was first seen in is recorded beside it.
|
||||
//
|
||||
// Claude Code has no pinned compaction format yet (Vikunja 466): a session
|
||||
// any Claude run collected for is unknown with harness-unsupported, never 0.
|
||||
// So is one whose first collection finds Claude history in claude-projects/
|
||||
// from before this ledger existed (row 270 r2, B3).
|
||||
//
|
||||
// One collection reads at most a budget of stream bytes (row 270 r2, B1).
|
||||
// The budget is the system config's compactionReadBytes, which only a human
|
||||
// writes (mos-1352, DECISIONS Q222); readBudget() takes the default for a
|
||||
// missing or bad value. A collection the budget stops short is unknown with
|
||||
// not-collected, and its cursors stop at the last whole line read, so a later
|
||||
// collection reads on from there.
|
||||
//
|
||||
// One collection opens at most a cap of stream files, in name order (row 270
|
||||
// r3, B4). The cap is the system config's compactionStreams, human-only too
|
||||
// (sage-3038); readStreams() takes the default for a missing or bad value.
|
||||
// Past the cap a collection is not-collected, and the streams it didn't open
|
||||
// get no cursor line, so neither a collection's work nor the ledger lines it
|
||||
// writes grow with the number of files the agent makes.
|
||||
//
|
||||
// Ledger lines, each one JSON object with v 1:
|
||||
// { kind: "compaction", entry, completedAt, stream, at } once per key
|
||||
// { kind: "cursor", stream, bytes, tail, at } read through `bytes`; `tail`
|
||||
// is the sha256 of the up to 4096 bytes that end there
|
||||
// { kind: "collected", at, known, reason, harness, version }
|
||||
// A cursor lets the next collection read only what was appended. A stream
|
||||
// that shrank, changed under its cursor or went away is store-incomplete from
|
||||
// then on: entries may be lost, so the count stays a lower bound for good.
|
||||
|
||||
import { createHash } from "node:crypto";
|
||||
import { closeSync, constants as fsc, fstatSync, lstatSync, openSync, readdirSync, readFileSync, readSync, writeSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { ID_PATTERN } from "../../business/src/vocabulary.mjs";
|
||||
|
||||
export const PI_VERSION = "0.85.1";
|
||||
export const LEDGER = "compactions.jsonl";
|
||||
const TAIL = 4096;
|
||||
const CHUNK = 1 << 20;
|
||||
// BUILD.md (queue-270, round 2) has the measured stream sizes behind the default.
|
||||
export const READ_BYTES = 64 * CHUNK;
|
||||
const READ_MIN = CHUNK, READ_MAX = 1024 * CHUNK;
|
||||
// BUILD.md (queue-270, round 3) has the measured stream counts behind the default.
|
||||
export const STREAMS = 1024;
|
||||
const STREAMS_MIN = 32, STREAMS_MAX = 65536;
|
||||
const STREAM = /^[A-Za-z0-9._-]{1,200}\.jsonl$/;
|
||||
|
||||
const digest = (buf) => createHash("sha256").update(buf).digest("hex");
|
||||
const keyOf = (entry, completedAt) => JSON.stringify([entry, completedAt]);
|
||||
const segment = (x) => typeof x === "string" && ID_PATTERN.test(x) && !x.includes("..");
|
||||
|
||||
// The store directory session.mjs sessionStore() makes, as a path only: the
|
||||
// reader never creates it. Null for an id that isn't one.
|
||||
export function storeDir(dataRoot, business, thread, session, { legacy = false } = {}) {
|
||||
if (![business, thread, session].every(segment)) return null;
|
||||
const dir = join(dataRoot, "threads", business, thread);
|
||||
return legacy ? dir : join(dir, "sessions", session);
|
||||
}
|
||||
|
||||
// The budget from the system config's compactionReadBytes: a whole number of
|
||||
// bytes from 1 MiB to 1 GiB, or the default for anything else, missing included.
|
||||
export function readBudget(value) {
|
||||
return Number.isSafeInteger(value) && value >= READ_MIN && value <= READ_MAX ? value : READ_BYTES;
|
||||
}
|
||||
|
||||
// The stream cap from the system config's compactionStreams: a whole number
|
||||
// from 32 to 65536, or the default for anything else, missing included.
|
||||
export function readStreams(value) {
|
||||
return Number.isSafeInteger(value) && value >= STREAMS_MIN && value <= STREAMS_MAX ? value : STREAMS;
|
||||
}
|
||||
|
||||
// Only the short tail reads (at most TAIL bytes) use this.
|
||||
function readRange(fd, from, to) {
|
||||
const out = Buffer.alloc(to - from);
|
||||
let off = 0;
|
||||
while (off < out.length) {
|
||||
const n = readSync(fd, out, off, Math.min(CHUNK, out.length - off), from + off);
|
||||
if (n === 0) break;
|
||||
off += n;
|
||||
}
|
||||
return out.subarray(0, off);
|
||||
}
|
||||
|
||||
// One collection over a Pi session directory. `cursors` are the ledger's,
|
||||
// by stream. Opens at most `streams` streams, in name order, and reads at
|
||||
// most `budget` bytes past their cursors, over all of them.
|
||||
// Returns the compactions found past each cursor, the new cursors, and the
|
||||
// problems: store-unreadable (a file that can't be read or isn't a Pi
|
||||
// session), store-incomplete (a stream that shrank, changed or went away, or
|
||||
// a line that isn't JSON) and not-collected (the budget or the stream cap
|
||||
// ran out first). A
|
||||
// missing directory is empty.
|
||||
export function scanPi(dir, cursors = {}, budget = READ_BYTES, streams = STREAMS) {
|
||||
const found = [], next = {}, problems = new Set();
|
||||
let left = budget;
|
||||
let names;
|
||||
try {
|
||||
names = readdirSync(dir);
|
||||
} catch (e) {
|
||||
if (e.code !== "ENOENT") problems.add("store-unreadable");
|
||||
names = [];
|
||||
}
|
||||
const present = new Set(names);
|
||||
for (const stream of Object.keys(cursors)) if (!present.has(stream)) problems.add("store-incomplete");
|
||||
const visit = names.filter((n) => n.endsWith(".jsonl")).sort();
|
||||
if (visit.length > streams) problems.add("not-collected");
|
||||
for (const name of visit.slice(0, streams)) {
|
||||
const path = join(dir, name);
|
||||
let fd;
|
||||
try {
|
||||
if (!STREAM.test(name) || !lstatSync(path).isFile()) throw new Error("not a regular stream file");
|
||||
fd = openSync(path, fsc.O_RDONLY | fsc.O_NOFOLLOW);
|
||||
} catch {
|
||||
problems.add("store-unreadable");
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
const size = fstatSync(fd).size;
|
||||
const c = cursors[name];
|
||||
let from = 0;
|
||||
if (c) {
|
||||
const tail = c.bytes > size ? null : readRange(fd, Math.max(0, c.bytes - TAIL), c.bytes);
|
||||
if (tail && tail.length === Math.min(TAIL, c.bytes) && digest(tail) === c.tail) from = c.bytes;
|
||||
else problems.add("store-incomplete");
|
||||
}
|
||||
let header = from > 0;
|
||||
const take = (raw) => {
|
||||
const line = raw.toString("utf8");
|
||||
if (!line.trim()) return;
|
||||
let e;
|
||||
try {
|
||||
e = JSON.parse(line);
|
||||
} catch {
|
||||
e = null;
|
||||
}
|
||||
if (!e || typeof e !== "object" || Array.isArray(e)) {
|
||||
problems.add("store-incomplete");
|
||||
return;
|
||||
}
|
||||
if (!header) {
|
||||
header = true;
|
||||
if (e.type !== "session") problems.add("store-unreadable");
|
||||
return;
|
||||
}
|
||||
if (e.type === "compaction" && typeof e.id === "string" && e.id) {
|
||||
found.push({ entry: e.id, completedAt: typeof e.timestamp === "string" ? e.timestamp : null, stream: name });
|
||||
}
|
||||
};
|
||||
// A CHUNK at a time, up to the budget left. Complete lines only: a line
|
||||
// split across chunks waits in `carry`, and a partial last line waits
|
||||
// for its newline. `end` is past the last whole line.
|
||||
const stop = Math.min(size, from + left);
|
||||
const chunk = Buffer.alloc(Math.min(CHUNK, stop - from));
|
||||
let pos = from, end = from, carry = [];
|
||||
while (pos < stop) {
|
||||
const n = readSync(fd, chunk, 0, Math.min(chunk.length, stop - pos), pos);
|
||||
if (n === 0) break;
|
||||
const view = chunk.subarray(0, n);
|
||||
let i = 0;
|
||||
for (let nl = view.indexOf(0x0a); nl !== -1; nl = view.indexOf(0x0a, i)) {
|
||||
take(carry.length ? Buffer.concat([...carry, view.subarray(i, nl)]) : view.subarray(i, nl));
|
||||
carry = [];
|
||||
i = nl + 1;
|
||||
end = pos + i;
|
||||
}
|
||||
if (i < n) carry.push(Buffer.from(view.subarray(i)));
|
||||
pos += n;
|
||||
}
|
||||
left -= pos - from;
|
||||
if (stop < size) problems.add("not-collected");
|
||||
next[name] = { bytes: end, tail: digest(readRange(fd, Math.max(0, end - TAIL), end)) };
|
||||
} catch {
|
||||
problems.add("store-unreadable");
|
||||
} finally {
|
||||
closeSync(fd);
|
||||
}
|
||||
}
|
||||
return { found, cursors: next, problems: [...problems].sort() };
|
||||
}
|
||||
|
||||
// The ledger as recorded. Throws on a ledger that can't be read. `bad` is set
|
||||
// by any line that isn't a v 1 record, a torn last line included.
|
||||
export function readLedger(dir) {
|
||||
const events = new Map(), cursors = {};
|
||||
let last = null, incomplete = false, unsupported = false, bad = false, torn = false, text;
|
||||
const file = join(dir, LEDGER);
|
||||
try {
|
||||
if (!lstatSync(file).isFile()) throw Object.assign(new Error(`${file} isn't a regular file`), { code: "ledger-unsafe" });
|
||||
text = readFileSync(file, { encoding: "utf8", flag: fsc.O_RDONLY | fsc.O_NOFOLLOW });
|
||||
} catch (e) {
|
||||
if (e.code === "ENOENT") return null;
|
||||
throw e;
|
||||
}
|
||||
const lines = text.split("\n");
|
||||
if (lines.pop() !== "") bad = torn = true;
|
||||
for (const line of lines) {
|
||||
let r;
|
||||
try {
|
||||
r = JSON.parse(line);
|
||||
} catch {
|
||||
r = null;
|
||||
}
|
||||
if (!r || typeof r !== "object" || r.v !== 1) {
|
||||
bad = true;
|
||||
continue;
|
||||
}
|
||||
if (r.kind === "compaction") {
|
||||
const key = keyOf(r.entry, r.completedAt);
|
||||
if (!events.has(key)) events.set(key, r);
|
||||
} else if (r.kind === "cursor") cursors[r.stream] = { bytes: r.bytes, tail: r.tail };
|
||||
else if (r.kind === "collected") {
|
||||
last = r;
|
||||
if (r.reason === "store-incomplete") incomplete = true;
|
||||
if (r.harness !== "pi" || r.reason === "harness-unsupported") unsupported = true;
|
||||
} else bad = true;
|
||||
}
|
||||
return { events, cursors, last, incomplete, unsupported, bad, torn };
|
||||
}
|
||||
|
||||
// Row 270 r2 (B3): whether the store's claude-projects/ holds anything. A
|
||||
// directory that can't be read throws, so the collection is not recorded and
|
||||
// the next one checks again.
|
||||
function claudeHistory(store) {
|
||||
if (!store.projects) return false;
|
||||
try {
|
||||
return readdirSync(store.projects).length > 0;
|
||||
} catch (e) {
|
||||
if (e.code === "ENOENT") return false;
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
// What session.list shows: { count, known, reason, at }. Known only when the
|
||||
// ledger is whole, every collection ran on Pi and none lost entries. Any other
|
||||
// answer is unknown with a reason and a lower bound, or null for none: never 0.
|
||||
export function compactionCount(dir) {
|
||||
const unknown = (reason, count = null, at = null) => ({ count: count || null, known: false, reason, at });
|
||||
let ledger;
|
||||
try {
|
||||
ledger = readLedger(dir);
|
||||
} catch {
|
||||
return unknown("store-unreadable");
|
||||
}
|
||||
if (!ledger) return unknown("not-collected");
|
||||
const count = ledger.events.size;
|
||||
const at = ledger.last?.at ?? null;
|
||||
if (ledger.bad) return unknown("store-incomplete", count, at);
|
||||
if (!ledger.last) return unknown("not-collected", count);
|
||||
if (ledger.unsupported) return unknown("harness-unsupported", count, at);
|
||||
if (ledger.incomplete) return unknown("store-incomplete", count, at);
|
||||
if (!ledger.last.known) return unknown(ledger.last.reason, count, at);
|
||||
return { count, known: true, reason: null, at };
|
||||
}
|
||||
|
||||
// One collection for a session's store ({ dir, session } from sessionStore),
|
||||
// at run start (resume) and run end, under the session's workspace lock.
|
||||
// Appends the new compactions, the moved cursors and one collected line in a
|
||||
// single O_APPEND write, then answers the count. A key already in the ledger
|
||||
// is never written again, and nothing rewrites the file.
|
||||
export function collectCompactions(store, harness, { now = () => new Date().toISOString(), budget = READ_BYTES, streams = STREAMS } = {}) {
|
||||
const prior = readLedger(store.dir);
|
||||
const ledger = prior ?? { events: new Map(), cursors: {}, torn: false };
|
||||
const at = now();
|
||||
const out = [];
|
||||
if (harness === "pi" && !prior && claudeHistory(store)) {
|
||||
// Claude runs before this ledger: their compactions can't be counted
|
||||
// (Vikunja 466), so the count is unknown for good, as after a Claude run.
|
||||
out.push({ v: 1, kind: "collected", at, known: false, reason: "harness-unsupported", harness, version: PI_VERSION });
|
||||
} else if (harness === "pi") {
|
||||
const scan = scanPi(store.session, ledger.cursors, budget, streams);
|
||||
for (const f of scan.found) {
|
||||
const key = keyOf(f.entry, f.completedAt);
|
||||
if (ledger.events.has(key)) continue;
|
||||
ledger.events.set(key, f);
|
||||
out.push({ v: 1, kind: "compaction", entry: f.entry, completedAt: f.completedAt, stream: f.stream, at });
|
||||
}
|
||||
for (const [stream, c] of Object.entries(scan.cursors)) {
|
||||
const old = ledger.cursors[stream];
|
||||
if (!old || old.bytes !== c.bytes || old.tail !== c.tail) out.push({ v: 1, kind: "cursor", stream, bytes: c.bytes, tail: c.tail, at });
|
||||
}
|
||||
// store-incomplete outranks store-unreadable: lost entries don't come back.
|
||||
// not-collected is last: the next collection reads on.
|
||||
const reason = ["store-incomplete", "store-unreadable", "not-collected"].find((p) => scan.problems.includes(p)) ?? null;
|
||||
out.push({ v: 1, kind: "collected", at, known: reason === null, reason, harness, version: PI_VERSION });
|
||||
} else {
|
||||
out.push({ v: 1, kind: "collected", at, known: false, reason: "harness-unsupported", harness, version: null });
|
||||
}
|
||||
const fd = openSync(join(store.dir, LEDGER), fsc.O_WRONLY | fsc.O_APPEND | fsc.O_CREAT | fsc.O_NOFOLLOW, 0o600);
|
||||
try {
|
||||
if (!fstatSync(fd).isFile()) throw Object.assign(new Error(`${join(store.dir, LEDGER)} isn't a regular file`), { code: "ledger-unsafe" });
|
||||
// After a torn line, start on a fresh one: the torn line stays bad and
|
||||
// keeps the count unknown, and the new lines stay whole.
|
||||
const buf = Buffer.from((ledger.torn ? "\n" : "") + out.map((r) => JSON.stringify(r)).join("\n") + "\n");
|
||||
let off = 0;
|
||||
while (off < buf.length) off += writeSync(fd, buf, off, buf.length - off);
|
||||
} finally {
|
||||
closeSync(fd);
|
||||
}
|
||||
return compactionCount(store.dir);
|
||||
}
|
||||
@@ -0,0 +1,145 @@
|
||||
// Row 270 round 2 (264c, #1775): the read budget (B1, duckworth-0223, sage-3031,
|
||||
// mos-1352) and Claude history from before the ledger (B3). Round 3: the cap on
|
||||
// streams one collection opens (B4, duckworth-0226). The streams are Pi
|
||||
// 0.85.1's shapes, written by hand as in compactions.test.mjs. A stream over
|
||||
// the budget is a sparse file: truncate() makes the hole, so no test writes
|
||||
// the bytes it reads past.
|
||||
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { appendFileSync, chmodSync, mkdtempSync, readFileSync, rmSync, truncateSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { collectCompactions, LEDGER, READ_BYTES, readBudget, readStreams, STREAMS } from "../src/compactions.mjs";
|
||||
import { sessionStore } from "../src/session.mjs";
|
||||
|
||||
const MiB = 1 << 20;
|
||||
|
||||
function setup(t) {
|
||||
const root = mkdtempSync(join(tmpdir(), "seat-compactions-budget-"));
|
||||
t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
const store = sessionStore(root, "acme", "th1", "s1");
|
||||
let tick = 0;
|
||||
const now = () => new Date(Date.UTC(2026, 9, 22, 12, 0, tick++)).toISOString();
|
||||
const collect = (harness = "pi", budget = undefined, streams = undefined) =>
|
||||
collectCompactions(store, harness, { now, ...(budget === undefined ? {} : { budget }), ...(streams === undefined ? {} : { streams }) });
|
||||
const records = () => readFileSync(join(store.dir, LEDGER), "utf8").trim().split("\n").map((l) => JSON.parse(l));
|
||||
return { root, store, collect, records };
|
||||
}
|
||||
|
||||
const header = (id) => ({ type: "session", version: 3, id, timestamp: "2026-10-22T11:00:00.000Z", cwd: "/w" });
|
||||
const compaction = (id, timestamp) => ({ type: "compaction", id, parentId: null, timestamp, summary: "s", firstKeptEntryId: "k", tokensBefore: 100 });
|
||||
const line = (e) => `${JSON.stringify(e)}\n`;
|
||||
const at = (n) => `2026-10-22T12:00:0${n}.000Z`;
|
||||
|
||||
test("the budget is the setting's whole number of bytes from 1 MiB to 1 GiB, and the default for anything else", () => {
|
||||
assert.equal(READ_BYTES, 64 * MiB);
|
||||
for (const v of [MiB, 8 * MiB, 1024 * MiB]) assert.equal(readBudget(v), v);
|
||||
for (const v of [undefined, null, "8388608", MiB + 0.5, -1, 0, MiB - 1, 1024 * MiB + 1, NaN, Infinity, {}, [MiB], true]) {
|
||||
assert.equal(readBudget(v), READ_BYTES, JSON.stringify(v) ?? String(v));
|
||||
}
|
||||
});
|
||||
|
||||
test("a stream past the default budget is not-collected with a lower bound, and a later collection catches up", (t) => {
|
||||
const f = setup(t);
|
||||
const path = join(f.store.session, "a.jsonl");
|
||||
const whole = line(header("a")) + line(compaction("c1", "2026-10-22T11:01:00.000Z"));
|
||||
writeFileSync(path, whole);
|
||||
// A hole past the default budget, with no newline in it: a line still being written.
|
||||
truncateSync(path, READ_BYTES + 2 * MiB);
|
||||
assert.deepEqual(f.collect(), { count: 1, known: false, reason: "not-collected", at: at(0) });
|
||||
// The cursor stops at the last whole line read.
|
||||
assert.deepEqual(f.records().filter((r) => r.kind === "cursor").map((r) => r.bytes), [Buffer.byteLength(whole)]);
|
||||
// The long line ends up as c2: the bytes past the cursor are read again, from the cursor.
|
||||
truncateSync(path, Buffer.byteLength(whole));
|
||||
appendFileSync(path, line(compaction("c2", "2026-10-22T11:02:00.000Z")));
|
||||
assert.deepEqual(f.collect(), { count: 2, known: true, reason: null, at: at(1) });
|
||||
});
|
||||
|
||||
test("a budget smaller than the streams counts over several collections, once each, and is known when it catches up", (t) => {
|
||||
const f = setup(t);
|
||||
const h = Buffer.byteLength(line(header("a"))), c = Buffer.byteLength(line(compaction("c1", "2026-10-22T11:01:00.000Z")));
|
||||
writeFileSync(join(f.store.session, "a.jsonl"), line(header("a")) + line(compaction("c1", "2026-10-22T11:01:00.000Z")) + line(compaction("c2", "2026-10-22T11:02:00.000Z")));
|
||||
writeFileSync(join(f.store.session, "b.jsonl"), line(header("b")) + line(compaction("c3", "2026-10-22T11:03:00.000Z")));
|
||||
// One byte past a's first compaction: a stops there and b isn't reached; then all of c2 and b's
|
||||
// header; then c3.
|
||||
const budget = h + c + 1;
|
||||
assert.deepEqual(f.collect("pi", budget), { count: 1, known: false, reason: "not-collected", at: at(0) });
|
||||
assert.deepEqual(f.collect("pi", budget), { count: 2, known: false, reason: "not-collected", at: at(1) });
|
||||
assert.deepEqual(f.collect("pi", budget), { count: 3, known: true, reason: null, at: at(2) });
|
||||
assert.deepEqual(f.records().filter((r) => r.kind === "compaction").map((r) => [r.entry, r.stream]), [["c1", "a.jsonl"], ["c2", "a.jsonl"], ["c3", "b.jsonl"]]);
|
||||
});
|
||||
|
||||
test("a line split across reads counts once", (t) => {
|
||||
const f = setup(t);
|
||||
// A message that ends just short of the first read's end, so c1 crosses it.
|
||||
const first = line(header("a"));
|
||||
const pad = { type: "message", id: "m1", parentId: null, timestamp: "2026-10-22T11:00:01.000Z", message: { role: "user", content: "" } };
|
||||
pad.message.content = "x".repeat(MiB - 100 - Buffer.byteLength(first) - Buffer.byteLength(line(pad)));
|
||||
writeFileSync(join(f.store.session, "a.jsonl"), first + line(pad) + line(compaction("c1", "2026-10-22T11:01:00.000Z")));
|
||||
assert.equal(Buffer.byteLength(first + line(pad)), MiB - 100);
|
||||
assert.deepEqual(f.collect(), { count: 1, known: true, reason: null, at: at(0) });
|
||||
});
|
||||
|
||||
// B3 (Vikunja 466): Claude Code runs before this ledger left history in claude-projects/ that
|
||||
// can't be counted. The first collection is harness-unsupported, and it stays so.
|
||||
test("Claude history from before the ledger makes the first Pi collection harness-unsupported, for good", (t) => {
|
||||
const f = setup(t);
|
||||
writeFileSync(join(f.store.projects, "old.jsonl"), "{}\n");
|
||||
writeFileSync(join(f.store.session, "a.jsonl"), line(header("a")) + line(compaction("c1", "2026-10-22T11:01:00.000Z")));
|
||||
assert.deepEqual(f.collect(), { count: null, known: false, reason: "harness-unsupported", at: at(0) });
|
||||
assert.deepEqual(f.collect(), { count: 1, known: false, reason: "harness-unsupported", at: at(1) });
|
||||
});
|
||||
|
||||
test("an empty claude-projects/ is no Claude history, and one filled after the ledger exists isn't checked", (t) => {
|
||||
const f = setup(t);
|
||||
writeFileSync(join(f.store.session, "a.jsonl"), line(header("a")) + line(compaction("c1", "2026-10-22T11:01:00.000Z")));
|
||||
assert.deepEqual(f.collect(), { count: 1, known: true, reason: null, at: at(0) });
|
||||
// A Claude run after the ledger exists collects for itself (compactions.test.mjs).
|
||||
writeFileSync(join(f.store.projects, "later.jsonl"), "{}\n");
|
||||
assert.deepEqual(f.collect(), { count: 1, known: true, reason: null, at: at(1) });
|
||||
});
|
||||
|
||||
test("a claude-projects/ that can't be read records nothing, and the next collection checks again", (t) => {
|
||||
const f = setup(t);
|
||||
writeFileSync(join(f.store.projects, "old.jsonl"), "{}\n");
|
||||
chmodSync(f.store.projects, 0o000);
|
||||
try {
|
||||
assert.throws(() => f.collect(), { code: "EACCES" });
|
||||
assert.throws(() => readFileSync(join(f.store.dir, LEDGER)), { code: "ENOENT" });
|
||||
} finally {
|
||||
chmodSync(f.store.projects, 0o700);
|
||||
}
|
||||
assert.deepEqual(f.collect(), { count: null, known: false, reason: "harness-unsupported", at: at(1) });
|
||||
});
|
||||
|
||||
test("the stream cap is the setting's whole number from 32 to 65536, and the default for anything else", () => {
|
||||
assert.equal(STREAMS, 1024);
|
||||
for (const v of [32, 2048, 65536]) assert.equal(readStreams(v), v);
|
||||
for (const v of [undefined, null, "2048", 40.5, -1, 0, 31, 65537, NaN, Infinity, {}, [64], true]) {
|
||||
assert.equal(readStreams(v), STREAMS, JSON.stringify(v) ?? String(v));
|
||||
}
|
||||
});
|
||||
|
||||
test("more streams than the cap is not-collected, with cursors only for the streams opened, and a larger cap catches up", (t) => {
|
||||
const f = setup(t);
|
||||
const names = ["a", "b", "c", "d"];
|
||||
names.forEach((n, i) => writeFileSync(join(f.store.session, `${n}.jsonl`), line(header(n)) + line(compaction(`c${n}`, `2026-10-22T11:0${i}:00.000Z`))));
|
||||
const cursors = () => f.records().filter((r) => r.kind === "cursor").map((r) => r.stream);
|
||||
assert.deepEqual(f.collect("pi", undefined, 3), { count: 3, known: false, reason: "not-collected", at: at(0) });
|
||||
assert.deepEqual(cursors(), ["a.jsonl", "b.jsonl", "c.jsonl"]);
|
||||
// The same cap again opens the same three, which haven't moved: no new lines but the collected one.
|
||||
assert.deepEqual(f.collect("pi", undefined, 3), { count: 3, known: false, reason: "not-collected", at: at(1) });
|
||||
assert.deepEqual(cursors(), ["a.jsonl", "b.jsonl", "c.jsonl"]);
|
||||
assert.deepEqual(f.collect("pi", undefined, 4), { count: 4, known: true, reason: null, at: at(2) });
|
||||
assert.deepEqual(cursors(), ["a.jsonl", "b.jsonl", "c.jsonl", "d.jsonl"]);
|
||||
});
|
||||
|
||||
test("the default cap opens STREAMS streams and writes no more cursor lines than that", (t) => {
|
||||
const f = setup(t);
|
||||
for (let i = 0; i <= STREAMS; i++) writeFileSync(join(f.store.session, `e${String(i).padStart(4, "0")}.jsonl`), "");
|
||||
assert.deepEqual(f.collect(), { count: null, known: false, reason: "not-collected", at: at(0) });
|
||||
const cursors = f.records().filter((r) => r.kind === "cursor");
|
||||
assert.equal(cursors.length, STREAMS);
|
||||
assert.equal(cursors.at(-1).stream, "e1023.jsonl");
|
||||
});
|
||||
@@ -0,0 +1,230 @@
|
||||
// Row 270 (264c, #1775, T1 item 7 D6, T3's Pi half): the compaction count
|
||||
// from a session's native Pi store, through the host ledger
|
||||
// (src/compactions.mjs). The streams are Pi 0.85.1's shapes, written by hand:
|
||||
// a session header, then entries, one JSON object per line.
|
||||
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { appendFileSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, rmSync, statSync, symlinkSync, unlinkSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join, relative, isAbsolute } from "node:path";
|
||||
import { collectCompactions, compactionCount, LEDGER, readLedger, storeDir } from "../src/compactions.mjs";
|
||||
import { binds } from "../src/sandbox.mjs";
|
||||
import { sessionStore } from "../src/session.mjs";
|
||||
|
||||
function setup(t, { legacy = false } = {}) {
|
||||
const root = mkdtempSync(join(tmpdir(), "seat-compactions-"));
|
||||
t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
const store = sessionStore(root, "acme", "th1", legacy ? "th1" : "s1", { legacy });
|
||||
let tick = 0;
|
||||
const now = () => new Date(Date.UTC(2026, 9, 22, 12, 0, tick++)).toISOString();
|
||||
const collect = (harness = "pi") => collectCompactions(store, harness, { now });
|
||||
const ledger = () => readFileSync(join(store.dir, LEDGER), "utf8");
|
||||
const records = () => ledger().trim().split("\n").map((l) => JSON.parse(l));
|
||||
return { root, store, collect, ledger, records };
|
||||
}
|
||||
|
||||
const header = (id) => ({ type: "session", version: 3, id, timestamp: "2026-10-22T11:00:00.000Z", cwd: "/w" });
|
||||
const message = (id, role = "user") => ({ type: "message", id, parentId: null, timestamp: "2026-10-22T11:00:01.000Z", message: { role, content: "x" } });
|
||||
const compaction = (id, timestamp) => ({ type: "compaction", id, parentId: null, timestamp, summary: "s", firstKeptEntryId: "k", tokensBefore: 100 });
|
||||
const line = (e) => `${JSON.stringify(e)}\n`;
|
||||
const stream = (store, name, entries) => writeFileSync(join(store.session, name), entries.map(line).join(""));
|
||||
const append = (store, name, ...entries) => appendFileSync(join(store.session, name), entries.map(line).join(""));
|
||||
|
||||
test("an empty store, a store with no compaction and a failed start count 0, known", (t) => {
|
||||
const f = setup(t);
|
||||
assert.deepEqual(f.collect(), { count: 0, known: true, reason: null, at: "2026-10-22T12:00:00.000Z" });
|
||||
// A started and aborted compaction writes no entry; a compactionSummary message, a
|
||||
// branch_summary and a message are not compactions.
|
||||
stream(f.store, "a.jsonl", [header("a"), message("m1"), message("m2", "compactionSummary"), { type: "branch_summary", id: "b1", parentId: null, timestamp: "2026-10-22T11:00:02.000Z", summary: "b" }]);
|
||||
assert.deepEqual(f.collect(), { count: 0, known: true, reason: null, at: "2026-10-22T12:00:01.000Z" });
|
||||
});
|
||||
|
||||
test("one and three completed compactions", (t) => {
|
||||
const f = setup(t);
|
||||
stream(f.store, "a.jsonl", [header("a"), message("m1"), compaction("c1", "2026-10-22T11:01:00.000Z")]);
|
||||
assert.equal(f.collect().count, 1);
|
||||
append(f.store, "a.jsonl", message("m2"), compaction("c2", "2026-10-22T11:02:00.000Z"), compaction("c3", "2026-10-22T11:03:00.000Z"));
|
||||
assert.deepEqual(f.collect(), { count: 3, known: true, reason: null, at: "2026-10-22T12:00:01.000Z" });
|
||||
assert.equal(compactionCount(f.store.dir).count, 3);
|
||||
});
|
||||
|
||||
// sage-2814: Pi's createBranchedSession and forkFrom copy entries into a new stream with
|
||||
// their id and timestamp, so a copied compaction counts once.
|
||||
test("a compaction copied into a branched stream counts once, under the stream it was first seen in", (t) => {
|
||||
const f = setup(t);
|
||||
const c1 = compaction("c1", "2026-10-22T11:01:00.000Z");
|
||||
stream(f.store, "a.jsonl", [header("a"), message("m1"), c1]);
|
||||
assert.equal(f.collect().count, 1);
|
||||
stream(f.store, "b.jsonl", [header("b"), message("m1"), c1, message("m3")]);
|
||||
assert.deepEqual(f.collect(), { count: 1, known: true, reason: null, at: "2026-10-22T12:00:01.000Z" });
|
||||
append(f.store, "b.jsonl", compaction("c2", "2026-10-22T11:05:00.000Z"));
|
||||
assert.equal(f.collect().count, 2);
|
||||
const lines = f.records().filter((r) => r.kind === "compaction");
|
||||
assert.deepEqual(lines.map((r) => [r.entry, r.stream]), [["c1", "a.jsonl"], ["c2", "b.jsonl"]]);
|
||||
});
|
||||
|
||||
test("one collection over two streams holding the same copy counts it once and writes it once", (t) => {
|
||||
const f = setup(t);
|
||||
const c1 = compaction("c1", "2026-10-22T11:01:00.000Z");
|
||||
stream(f.store, "a.jsonl", [header("a"), c1]);
|
||||
stream(f.store, "b.jsonl", [header("b"), c1]);
|
||||
assert.equal(f.collect().count, 1);
|
||||
assert.equal(f.records().filter((r) => r.kind === "compaction").length, 1);
|
||||
});
|
||||
|
||||
// Pi's ids are unique only within a file: the same id at another time is another compaction.
|
||||
test("the same entry id at another time in another stream is another compaction", (t) => {
|
||||
const f = setup(t);
|
||||
stream(f.store, "a.jsonl", [header("a"), compaction("c1", "2026-10-22T11:01:00.000Z")]);
|
||||
stream(f.store, "b.jsonl", [header("b"), compaction("c1", "2026-10-22T11:09:00.000Z")]);
|
||||
assert.equal(f.collect().count, 2);
|
||||
});
|
||||
|
||||
test("resume: each collection reads past its cursor and writes a key once", (t) => {
|
||||
const f = setup(t);
|
||||
stream(f.store, "a.jsonl", [header("a"), compaction("c1", "2026-10-22T11:01:00.000Z")]);
|
||||
f.collect();
|
||||
f.collect();
|
||||
append(f.store, "a.jsonl", compaction("c2", "2026-10-22T11:02:00.000Z"));
|
||||
assert.equal(f.collect().count, 2);
|
||||
const r = f.records();
|
||||
assert.deepEqual(r.filter((x) => x.kind === "compaction").map((x) => x.entry), ["c1", "c2"]);
|
||||
assert.deepEqual(r.filter((x) => x.kind === "cursor").map((x) => x.bytes), [statSync(join(f.store.session, "a.jsonl")).size - line(compaction("c2", "2026-10-22T11:02:00.000Z")).length, statSync(join(f.store.session, "a.jsonl")).size]);
|
||||
assert.equal(r.filter((x) => x.kind === "collected").length, 3);
|
||||
assert.equal(lstatSync(join(f.store.dir, LEDGER)).mode & 0o777, 0o600);
|
||||
});
|
||||
|
||||
test("a partial trailing line waits for its newline", (t) => {
|
||||
const f = setup(t);
|
||||
stream(f.store, "a.jsonl", [header("a")]);
|
||||
const whole = line(compaction("c1", "2026-10-22T11:01:00.000Z"));
|
||||
appendFileSync(join(f.store.session, "a.jsonl"), whole.slice(0, 30));
|
||||
assert.deepEqual(f.collect(), { count: 0, known: true, reason: null, at: "2026-10-22T12:00:00.000Z" });
|
||||
appendFileSync(join(f.store.session, "a.jsonl"), whole.slice(30));
|
||||
assert.deepEqual(f.collect(), { count: 1, known: true, reason: null, at: "2026-10-22T12:00:01.000Z" });
|
||||
});
|
||||
|
||||
test("a missing ledger is not-collected and an unreadable one store-unreadable, never 0", (t) => {
|
||||
const f = setup(t);
|
||||
assert.deepEqual(compactionCount(f.store.dir), { count: null, known: false, reason: "not-collected", at: null });
|
||||
mkdirSync(join(f.store.dir, LEDGER));
|
||||
assert.deepEqual(compactionCount(f.store.dir), { count: null, known: false, reason: "store-unreadable", at: null });
|
||||
assert.throws(() => f.collect());
|
||||
});
|
||||
|
||||
test("a symlinked ledger is store-unreadable and is not written through", (t) => {
|
||||
const f = setup(t);
|
||||
const target = join(f.root, "elsewhere.jsonl");
|
||||
writeFileSync(target, "");
|
||||
symlinkSync(target, join(f.store.dir, LEDGER));
|
||||
assert.equal(compactionCount(f.store.dir).reason, "store-unreadable");
|
||||
assert.throws(() => f.collect());
|
||||
assert.equal(readFileSync(target, "utf8"), "");
|
||||
});
|
||||
|
||||
// sage-2806 condition 3: a torn last line is a bad line, not a skipped one.
|
||||
test("a torn last ledger line makes the count unknown, a lower bound, and stays so", (t) => {
|
||||
const f = setup(t);
|
||||
assert.equal(f.collect().count, 0);
|
||||
appendFileSync(join(f.store.dir, LEDGER), '{"v":1,"kind":"compac');
|
||||
assert.deepEqual(compactionCount(f.store.dir), { count: null, known: false, reason: "store-incomplete", at: "2026-10-22T12:00:00.000Z" });
|
||||
stream(f.store, "a.jsonl", [header("a"), compaction("c1", "2026-10-22T11:01:00.000Z")]);
|
||||
assert.deepEqual(f.collect(), { count: 1, known: false, reason: "store-incomplete", at: "2026-10-22T12:00:01.000Z" });
|
||||
// The next write started on a fresh line: every line after the torn one is whole.
|
||||
const lines = f.ledger().split("\n");
|
||||
assert.equal(lines.pop(), "");
|
||||
const bad = lines.filter((l) => {
|
||||
try {
|
||||
JSON.parse(l);
|
||||
return false;
|
||||
} catch {
|
||||
return true;
|
||||
}
|
||||
});
|
||||
assert.deepEqual(bad, ['{"v":1,"kind":"compac']);
|
||||
assert.equal(readLedger(f.store.dir).events.size, 1);
|
||||
});
|
||||
|
||||
test("a bad ledger line in the middle makes the count unknown", (t) => {
|
||||
const f = setup(t);
|
||||
stream(f.store, "a.jsonl", [header("a"), compaction("c1", "2026-10-22T11:01:00.000Z")]);
|
||||
f.collect();
|
||||
appendFileSync(join(f.store.dir, LEDGER), "not json\n");
|
||||
append(f.store, "a.jsonl", compaction("c2", "2026-10-22T11:02:00.000Z"));
|
||||
assert.deepEqual(f.collect(), { count: 2, known: false, reason: "store-incomplete", at: "2026-10-22T12:00:01.000Z" });
|
||||
});
|
||||
|
||||
test("a stream that shrank, changed under its cursor or went away is store-incomplete for good", async (t) => {
|
||||
for (const change of ["shrink", "rewrite", "remove"]) {
|
||||
await t.test(change, (t) => {
|
||||
const f = setup(t);
|
||||
stream(f.store, "a.jsonl", [header("a"), compaction("c1", "2026-10-22T11:01:00.000Z")]);
|
||||
assert.equal(f.collect().known, true);
|
||||
const path = join(f.store.session, "a.jsonl");
|
||||
if (change === "shrink") stream(f.store, "a.jsonl", [header("a")]);
|
||||
// Same length, other bytes: only the cursor's tail digest sees it.
|
||||
if (change === "rewrite") stream(f.store, "a.jsonl", [header("a"), compaction("c9", "2026-10-22T11:01:00.000Z")]);
|
||||
if (change === "remove") unlinkSync(path);
|
||||
const c = f.collect();
|
||||
assert.equal(c.known, false);
|
||||
assert.equal(c.reason, "store-incomplete");
|
||||
assert.equal(c.count, change === "rewrite" ? 2 : 1);
|
||||
// A later clean collection doesn't make it known again.
|
||||
assert.equal(f.collect().reason, "store-incomplete");
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
test("a stream that isn't a regular file or a Pi session is store-unreadable, and that passes", (t) => {
|
||||
const f = setup(t);
|
||||
const target = join(f.root, "outside.jsonl");
|
||||
writeFileSync(target, line(header("x")) + line(compaction("c1", "2026-10-22T11:01:00.000Z")));
|
||||
symlinkSync(target, join(f.store.session, "link.jsonl"));
|
||||
assert.deepEqual(f.collect(), { count: null, known: false, reason: "store-unreadable", at: "2026-10-22T12:00:00.000Z" });
|
||||
unlinkSync(join(f.store.session, "link.jsonl"));
|
||||
assert.deepEqual(f.collect(), { count: 0, known: true, reason: null, at: "2026-10-22T12:00:01.000Z" });
|
||||
});
|
||||
|
||||
test("a first line that isn't a session header is store-unreadable", (t) => {
|
||||
const f = setup(t);
|
||||
stream(f.store, "a.jsonl", [compaction("c1", "2026-10-22T11:01:00.000Z")]);
|
||||
assert.equal(f.collect().reason, "store-unreadable");
|
||||
});
|
||||
|
||||
// RULINGS Q6: Claude Code is unknown with the Vikunja 466 reason, never 0, and a session any
|
||||
// Claude run collected for stays unknown.
|
||||
test("a Claude run is harness-unsupported, and stays so after a Pi run", (t) => {
|
||||
const f = setup(t);
|
||||
assert.deepEqual(f.collect("claude"), { count: null, known: false, reason: "harness-unsupported", at: "2026-10-22T12:00:00.000Z" });
|
||||
stream(f.store, "a.jsonl", [header("a"), compaction("c1", "2026-10-22T11:01:00.000Z")]);
|
||||
assert.deepEqual(f.collect("pi"), { count: 1, known: false, reason: "harness-unsupported", at: "2026-10-22T12:00:01.000Z" });
|
||||
});
|
||||
|
||||
test("storeDir is the store sessionStore makes, without making it", (t) => {
|
||||
const f = setup(t);
|
||||
assert.equal(storeDir(f.root, "acme", "th1", "s1"), f.store.dir);
|
||||
assert.equal(storeDir(f.root, "acme", "th1", "th1", { legacy: true }), join(f.root, "threads", "acme", "th1"));
|
||||
assert.equal(storeDir(f.root, "acme", "th1", "s2"), join(f.store.dir, "..", "s2"));
|
||||
assert.throws(() => lstatSync(join(f.store.dir, "..", "s2")), { code: "ENOENT" });
|
||||
for (const bad of ["..", "a/b", "", "A", null]) assert.equal(storeDir(f.root, "acme", "th1", bad), null);
|
||||
assert.equal(storeDir(f.root, "../x", "th1", "s1"), null);
|
||||
});
|
||||
|
||||
// sage-2806 condition 4: the agent can't reach the ledger. It sits in the store dir, which no
|
||||
// bind mounts; only the store's session/ and claude-projects/ are.
|
||||
test("the ledger is under no sandbox mount", (t) => {
|
||||
for (const legacy of [false, true]) {
|
||||
const f = setup(t, { legacy });
|
||||
f.collect();
|
||||
const file = join(f.store.dir, LEDGER);
|
||||
const runDir = join(f.root, "runs", "r1");
|
||||
const list = binds({ runDir, workspace: join(f.root, "ws"), adapter: join(f.root, "a.sh"), brokerSocket: join(f.root, "bus.sock"), threadStore: f.store });
|
||||
assert.ok(list.some((b) => b.src === f.store.session));
|
||||
for (const b of list) {
|
||||
if (!b.src || !isAbsolute(b.src)) continue;
|
||||
const rel = relative(b.src, file);
|
||||
assert.ok(rel.startsWith("..") || isAbsolute(rel), `${b.src} mounts ${file}`);
|
||||
}
|
||||
}
|
||||
});
|
||||
@@ -294,9 +294,14 @@ async function validate(document, file) {
|
||||
if (!isPlainObject(document)) {
|
||||
fail(2, "configuration must be a JSON object");
|
||||
}
|
||||
// Row 270 r2 and r3 (mos-1352, DECISIONS Q222, sage-3038): compactionReadBytes and
|
||||
// compactionStreams, optional and human-only, pass on as written; the launcher takes the
|
||||
// default for a missing or bad value (packages/seat/src/compactions.mjs readBudget and
|
||||
// readStreams). The first list stays as it was:
|
||||
// packages/cli/tests/stack-release-fixture.mjs edits it to make an older loader.
|
||||
rejectUnknownKeys(
|
||||
document,
|
||||
["configVersion", "environment", "dataRoot", "execution", "console", "federation"],
|
||||
["configVersion", "environment", "dataRoot", "execution", "console", "federation"].concat(["compactionReadBytes", "compactionStreams"]),
|
||||
"configuration",
|
||||
);
|
||||
|
||||
@@ -349,6 +354,8 @@ async function validate(document, file) {
|
||||
},
|
||||
...(consoleKey === undefined ? {} : { console: consoleKey }),
|
||||
...(federationKey === undefined ? {} : { federation: federationKey }),
|
||||
...(Object.hasOwn(document, "compactionReadBytes") ? { compactionReadBytes: document.compactionReadBytes } : {}),
|
||||
...(Object.hasOwn(document, "compactionStreams") ? { compactionStreams: document.compactionStreams } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -417,7 +417,7 @@ plugin_checks() {
|
||||
# Row 181: the tables a bus writer fills without an event in the same transaction, in schema
|
||||
# order. A restore drops their rows after STAMP as it drops events, so the guard counts them too.
|
||||
# BUILD.md for row 181 names each writer; the T46 guard classifies every table in the schema.
|
||||
COUNTED="role_claims thread_claims decision_events messages deliveries task_snapshots projects threads sessions session_defaults thread_opens thread_shares instance_names usage_records"
|
||||
COUNTED="role_claims thread_claims decision_events messages deliveries task_snapshots projects threads sessions session_defaults thread_opens thread_shares instance_names usage_records session_ops message_cancels"
|
||||
|
||||
STORE_JS='
|
||||
const {DatabaseSync}=require("node:sqlite");const {pathToFileURL}=require("node:url");
|
||||
|
||||
@@ -263,6 +263,23 @@ EXPECTED="$(printf '{\n "configVersion": 1,\n "environment": "development",\n
|
||||
[ "$(MOSAIC_CONFIG="$SANDBOX/no-console.json" timeout 20 $CONFIG_OP validate)" = "$EXPECTED" ]
|
||||
check "config without console loads with unchanged output" $?
|
||||
|
||||
# --- Row 270 r2 (mos-1352, DECISIONS Q222): the optional compactionReadBytes key ---
|
||||
# The loader passes it on as written, a bad value too: the launcher takes the default for a
|
||||
# missing or bad one (packages/seat/src/compactions.mjs readBudget), so it never refuses a config.
|
||||
for v in 8388608 '"x"'; do
|
||||
cfg budget.json '{"configVersion":1,"environment":"development","dataRoot":"'$DATA_ROOT'","execution":{"backend":"docker","provider":"zai","model":"m"},"compactionReadBytes":'"$v"'}'
|
||||
OUT="$(MOSAIC_CONFIG="$SANDBOX/budget.json" timeout 20 $CONFIG_OP validate 2>&1)"
|
||||
[ $? -eq 0 ] && printf '%s' "$OUT" | grep -qxF " \"compactionReadBytes\": $v"
|
||||
check "compactionReadBytes $v loads and is in the output as written" $?
|
||||
done
|
||||
# Row 270 r3 (sage-3038): compactionStreams the same way (readStreams).
|
||||
for v in 2048 '"x"'; do
|
||||
cfg streams.json '{"configVersion":1,"environment":"development","dataRoot":"'$DATA_ROOT'","execution":{"backend":"docker","provider":"zai","model":"m"},"compactionStreams":'"$v"'}'
|
||||
OUT="$(MOSAIC_CONFIG="$SANDBOX/streams.json" timeout 20 $CONFIG_OP validate 2>&1)"
|
||||
[ $? -eq 0 ] && printf '%s' "$OUT" | grep -qxF " \"compactionStreams\": $v"
|
||||
check "compactionStreams $v loads and is in the output as written" $?
|
||||
done
|
||||
|
||||
# --- validation must not modify the file ---
|
||||
SUM_INVALID_BEFORE=$(sha256sum "$SANDBOX/invalid.json" | cut -d' ' -f1)
|
||||
MOSAIC_CONFIG="$SANDBOX/invalid.json" $CONFIG_OP validate >/dev/null 2>&1
|
||||
|
||||
Reference in new issue
Block a user