gizmoduck: row 271 (264d) design draft 2: drained acknowledged by the run's own receive at a turn boundary (a landing turn has unread 0 with a child running), SIGTERM only after it; successor <old>-N bound at request with a deterministic id; no cancel (sage-2826); pre-existing test change list per mos-1315
This commit is contained in:
@@ -71,96 +71,170 @@ default where it is.
|
||||
On a base without 264d, `session.clear` refuses `unknown-verb`, and the rows have no `state`,
|
||||
`revision` or `successor`.
|
||||
|
||||
## Design (draft 1, for Duckworth's early look before code)
|
||||
## Design (draft 2: Sage's sage-2826 rulings and Mos's mos-1315 folded in)
|
||||
|
||||
Measured on 2d0d407ba, which already holds row 194 (ab7377bcf). Nothing has been run.
|
||||
Measured on 2d0d407ba, which already holds row 194 (ab7377bcf). Nothing has been run. Draft 1
|
||||
was 3105de889. Draft 2 changes where the drained acknowledgement comes from (stage 2): threadIdle
|
||||
alone can't show that no child is running.
|
||||
|
||||
### What exists and what doesn't
|
||||
|
||||
- No per-session lifecycle state. `sessions` is append-only
|
||||
(schema.sql:370-372). The nearest per-session state is the latest `thread_claims` row (op and
|
||||
generation). Row 194's lifecycle is per instance, as `action.allowed` events keyed by a random
|
||||
request id (broker.mjs:622-666).
|
||||
- No drain request in the runner. Its only stop is SIGTERM, which aborts a turn in progress
|
||||
(runner.mjs:407-423).
|
||||
- There is a fence. `threadIdle` (broker.mjs:865-877) rechecks `pending` and `unread`
|
||||
for the run's own session and releases the claim in one synchronous call, only when both are 0.
|
||||
After that, the runner's next `message.receive` gets `not-holder` and it exits 22 by itself
|
||||
(runner.mjs:55, :524-530). `unread` 0 means every message delivered to that holder was read, and
|
||||
the runner reads a message only after its turn's RESULT (runner.mjs:570-577). So a release
|
||||
happens only at a completed turn boundary.
|
||||
- The launcher's idle stop (launcher.mjs:683-701) is threadIdle, then a start-time-checked
|
||||
SIGTERM to an idle runner. Exit proof is the run's `close` handler: `endRun`, registry remove,
|
||||
and the workspace lock release (:600-641).
|
||||
- No cancelled state for an undelivered message. `deliveries.op` is a CHECK list on an append-only
|
||||
table.
|
||||
- No per-session lifecycle state. `sessions` is append-only (schema.sql:370-372). The nearest
|
||||
per-session state is the latest `thread_claims` row (op and generation). Row 194's lifecycle
|
||||
is per instance, as `action.allowed` events keyed by a random request id (broker.mjs:622-666).
|
||||
- No cancelled state for an undelivered message. `deliveries.op` is a CHECK list on an
|
||||
append-only table.
|
||||
- The runner calls `message.receive` only when no harness child exists:
|
||||
- `runTurn`'s close handler SIGKILLs the turn's whole process group and clears `child`
|
||||
(harness/src/runner.mjs:363-366).
|
||||
- The loop asks receive only after the landing turns (:493-517) and after each message's
|
||||
RESULT and `message.read` (:524-580).
|
||||
- So a receive from the run is a completed turn boundary, made by the runner itself.
|
||||
- `threadIdle` (broker.mjs:865-877) is not enough. A landing turn reads no message, so `unread`
|
||||
is 0 while its child runs. That is why landRelease keeps a run until it has asked receive
|
||||
(`#received`, broker.mjs:363-365, :3815-3829).
|
||||
- There is precedent for acting at receive. `#limitStop` (broker.mjs:4147-4148, :4499-4520)
|
||||
holds a stop until the run's next `message.receive` or `state`, so the turn's RESULT and read
|
||||
land first.
|
||||
- The runner's SIGTERM handler escalates to KILL after 5 s only when `child` is set
|
||||
(runner.mjs:405-423). With no child it sets `stopping`, the loop ends, and it exits 0.
|
||||
- The launcher's idle stop (launcher.mjs:683-701) sends a SIGTERM after a start-time check. The
|
||||
exit proof is the run's `close` handler, which runs endRun, removes the registry entry and
|
||||
releases the workspace lock (:600-641).
|
||||
|
||||
### Clear, stage by stage
|
||||
|
||||
1. **Accept** (`session.clear`, the human's write). One transaction:
|
||||
- check owner and grant (the session.create path), the revision, and processing not halted
|
||||
- append journal stage `requested`, with a successor id bound now (`randomUUID`) and a payload
|
||||
digest for the repeat/conflict check
|
||||
Answer `clearing`. From here:
|
||||
- `thread.send` / `thread.open` naming the session refuse `session-clearing`. A thread-only
|
||||
send whose default is that session refuses the same way.
|
||||
- `threadsState` reports the session as `clearing`, so `threadPass` never starts it.
|
||||
- `message.receive` for a run of that session returns `[]`, which fences the next-turn dequeue.
|
||||
2. **Drain** (the launcher, each tick, for each open clear). A trusted op `sessionClearDrain
|
||||
{business, op}` runs in one transaction:
|
||||
- Cancel every undelivered message to the session, with a receipt row per message (new table
|
||||
`message_cancels`). The cancelled messages stop counting as pending.
|
||||
- If the session has a holder: run threadIdle's check. If `unread > 0`, the turn is still
|
||||
running. Answer `{drained: false}` and change nothing else. Otherwise release the claim,
|
||||
generation-bound (`#claimEnd`), and append `drained` with the released run and generation.
|
||||
- If the session has no holder: append `drained` with run null.
|
||||
After `drained` with a run, the launcher sends the idle stop's start-time-checked SIGTERM, the
|
||||
same reviewed path, to an idle runner whose dequeue is already fenced. Even without the signal,
|
||||
the runner exits on its next receive. No KILL, no stopRun, and no signal before `drained`.
|
||||
3. **Proof.** The released run's `close` handler (endRun, registry remove, lock release) is the
|
||||
proof.
|
||||
- The runner is pid 1 of its own pid namespace (sandbox `--unshare-pid --as-pid-1`), so its exit
|
||||
ends every process in it. **To verify in the build** (bwrap's pid-1 behaviour), not assumed.
|
||||
- With no run, the proof is that the launcher takes `lockSessionWorkspace` for the session (no
|
||||
writer alive) and the registry has no run for it.
|
||||
- After 30 s with no proof, append `uncertain` once. Keep waiting, with capacity still held by
|
||||
the live run and no second signal. A launcher restart finds the run among its row 269
|
||||
survivors and keeps waiting on its identity.
|
||||
4. **Retire** (trusted op `sessionClearRetire {business, op, run}`), one transaction:
|
||||
- check the run ended (`#ended`) or that there was none
|
||||
- append `retired`
|
||||
- insert the successor `sessions` row: `predecessor` the old id, `op` the clear op, `name` the
|
||||
old name plus a suffix (`UNIQUE(thread,name)`)
|
||||
- move the default with a `session_defaults` row, only if the old session was the default
|
||||
- insert a `thread_opens` row for the successor
|
||||
The old workspace stays at its path, untouched. The successor gets its own store and workspace
|
||||
from row 269's per-session layout and a fresh approved checkout. Nothing is copied.
|
||||
- Check owner and grant (the `session.create` path), the revision, and that processing isn't
|
||||
halted.
|
||||
- Bind the successor before any async work:
|
||||
- The id is deterministic, from a sha256 of (business, thread, old session, op), shaped to
|
||||
`ID_PATTERN` (vocabulary.mjs:199).
|
||||
- The name is `<old name>-N`, with N the next free number in that thread. If a user already
|
||||
made a session called `<old>-N`, the next free N is taken.
|
||||
- Both go on the `requested` row, so a repeat of the op returns the same id and name.
|
||||
- Append journal stage `requested`, with a payload digest for the repeat and conflict checks.
|
||||
- The answer is `clearing`. From here:
|
||||
- `thread.send` and `thread.open` naming the session refuse `session-clearing`. So does a
|
||||
thread-only send whose default is that session.
|
||||
- `threadsState` reports the session as `clearing`, so `threadPass` never starts it.
|
||||
- Undelivered messages to it are cancelled with a receipt each (`message_cancels`), in the
|
||||
same transaction. Old messages, results and run evidence don't change.
|
||||
2. **Drain, acknowledged by the runner.** Once the session is clearing, the run's
|
||||
`message.receive` (the runner's call, at a turn boundary with no child) takes one transaction:
|
||||
- It appends `drained` with the run and its claim generation, once.
|
||||
- It delivers nothing and answers `[]`.
|
||||
- The claim stays held. That holds the dequeue fence, so no turn and no child can start
|
||||
before the stop, and the old capacity stays reserved.
|
||||
Later receives keep answering `[]`. `state` needs no fence, because the runner asks it only
|
||||
for a message it received. A clear on a session with no holder appends `drained` with run
|
||||
null at accept.
|
||||
3. **Stop and proof** (launcher). For a `drained` op whose run and generation are still the
|
||||
holder's, the launcher sends the idle stop's start-time-checked SIGTERM, the same reviewed
|
||||
path. The runner has no child, so its handler never reaches KILL.
|
||||
- The exit proof is the run's own `close` handler (endRun, registry remove, lock release),
|
||||
recorded with `sessionClearExited {business, op, run}`. A signal, a revoked claim or a dead
|
||||
socket is not proof.
|
||||
- Under the host, the runner is pid 1 of its own pid namespace (`--unshare-pid --as-pid-1`),
|
||||
so its exit ends anything a turn left outside its process group. **This is a build check**,
|
||||
tested against bwrap, not assumed.
|
||||
- With no run, the proof is that the launcher takes `lockSessionWorkspace` for the session and
|
||||
the registry has no run for it.
|
||||
- If no `drained` arrives (a turn still running, or a stuck child), the session stays
|
||||
`clearing`. Capacity stays reserved, the successor stays unstarted, the status stays
|
||||
actionable, and no signal is sent.
|
||||
- Thirty seconds after the SIGTERM with no proof, `uncertain` is appended once. The launcher
|
||||
keeps waiting and sends no second signal. A launcher restart finds the run among its row 269
|
||||
survivors and waits on its identity.
|
||||
4. **Retire** (trusted op `sessionClearRetire {business, op}`), one transaction:
|
||||
- Check the exit proof (the run is in `#ended`), or that there was no run.
|
||||
- Append `retired`, and end the old claim.
|
||||
- Insert the successor `sessions` row with the bound id and name, `predecessor` set to the old
|
||||
id and `op` set to the clear op.
|
||||
- Add a `session_defaults` row only if the old session was the default.
|
||||
- Add a `thread_opens` row for the successor.
|
||||
The old workspace stays at its path with no new writer mount. The successor gets its own store,
|
||||
HOME and workspace from row 269's per-session layout, and a fresh checkout of the approved
|
||||
revision. Nothing is copied.
|
||||
5. **Start** (D8). `threadPass` starts the successor from its `thread_opens` row through the same
|
||||
`check()`/`start()` as any session. The launcher reports the outcome with `sessionClearStart
|
||||
{business, op, start, refusal}`, appending `start-held`, `start-started` or `start-refused`.
|
||||
A held start keeps its op and spends no attempt.
|
||||
{business, op, start, refusal}`, which appends `start-held`, `start-started` or
|
||||
`start-refused`. A held start keeps its op and spends no attempt.
|
||||
|
||||
`failed`: a refusal at retire (for example the thread's owner lost the grant) appends `failed`
|
||||
with the code. No retirement happens and the old session goes back to `active`. Human cancel is a
|
||||
later row unless Sage wants it in 271.
|
||||
`failed`: a refusal at retire (for example, the owner lost the grant) appends `failed` with the
|
||||
code. Nothing is retired, and the old session goes back to `active` with its messages still
|
||||
cancelled.
|
||||
|
||||
### Schema step (new, label by landing order)
|
||||
Human cancel of an open clear is out of 271 (sage-2826 Q3). D7 defines no cancel, and recovery
|
||||
resumes the same operation. Sage is opening a DEFERRED entry for it.
|
||||
|
||||
- `session_ops(seq, at, business, thread, session, op, stage, successor, run, generation,
|
||||
digest, refusal)`. Append-only, with `UNIQUE(thread, op, stage)` and a CHECK on `stage`. A
|
||||
trigger refuses a second open op per session. The envelope is derived from the latest stage.
|
||||
`revision` is the count of this session's rows.
|
||||
- `message_cancels(seq, at, message UNIQUE, session, op)`. Append-only.
|
||||
- `threadRows`'s `pending` gains `AND NOT EXISTS (cancel)`. Each row gains `state`.
|
||||
### Schema step (label by landing order, mos-1315 item 4)
|
||||
|
||||
### Questions for Sage (and Mos where marked)
|
||||
- `session_ops(seq, at, business, thread, session, op, stage, successor, successor_name, run,
|
||||
generation, digest, refusal)`:
|
||||
- append-only, with `UNIQUE(thread, op, stage)` and a CHECK on `stage`
|
||||
- a trigger refuses a second open op per session
|
||||
- the envelope comes from the latest stage, and `revision` is the count of this session's rows
|
||||
- `message_cancels(seq, at, message UNIQUE, session, op)`, append-only.
|
||||
- `threadRows`'s `pending` gains `AND NOT EXISTS (cancel)`, and each row gains `state`.
|
||||
- Label: 3m if 271 lands before row 206. If 206 lands first (3l to 3m at Launchpad's 631a256ba),
|
||||
271 rebases on it and is 3m to 3n. Sage sets the order.
|
||||
|
||||
- Q1 (Mos, mos-1292). A real `3l→3m` step collides with comment-migration.test.mjs:76-82, which
|
||||
fabricates a future `3l→3m`. That pre-existing test has to move to `3m→3n`, and the `'3l'` pins
|
||||
in the account, gitea, grant and name migration tests and sessions.test.mjs:435 move too. Does
|
||||
that go to Mos as one change list? And does 271 take `3m`, or whatever label landing order
|
||||
gives it after row 206?
|
||||
- Q2. Stop: fence plus idle-stop SIGTERM, as above, rather than a new `stopRun` reason. That adds
|
||||
no KILL path. OK?
|
||||
- Q3. Human cancel of a clear: out of 271?
|
||||
- Q4. Successor name: `<old name>-2`, `-3` and so on. Or should clear take a name?
|
||||
### Pre-existing test changes (mos-1315)
|
||||
|
||||
These are allowed:
|
||||
- the current schema label
|
||||
- the fabricated future step in comment-migration.test.mjs, moved up one label
|
||||
- the migration file lists, which gain the new step's file
|
||||
|
||||
No assertion is removed or loosened, and no other expected value changes. This list was measured
|
||||
on 2d0d407ba with `git grep` for `'3l'`, `3k-3l` and `'3m'` in `packages/*/tests/*`.
|
||||
|
||||
- Current label (`'3l'` becomes the new label):
|
||||
- account-migration.test.mjs:94, :144, :163, plus the rewrite source :156
|
||||
- chief-of-staff.test.mjs:641, plus the rewrite source :617
|
||||
- comment-migration.test.mjs:36
|
||||
- gitea-migration.test.mjs:98, :124, :142, plus the rewrite source :135
|
||||
- grant-migration.test.mjs:103, :128, :146, plus the rewrite source :139
|
||||
- name-migration.test.mjs:68
|
||||
- sessions.test.mjs:377, :450, plus the rewrite source :433
|
||||
- spec-migration.test.mjs:66
|
||||
- thread-migration.test.mjs:104, plus the rewrite source :52
|
||||
- usage-migration.test.mjs:71
|
||||
- A rewrite source is the `.replace("export const SCHEMA_VERSION = '3l';", ...)` string. It has
|
||||
to match store.mjs:17's new text, or the rewrite silently does nothing.
|
||||
- Fabricated future step: comment-migration.test.mjs:78-79, :86 (`3l`→`3m` becomes one label up).
|
||||
- Migration file lists:
|
||||
- comment-migration.test.mjs:82
|
||||
- the `copyFileSync` lists that copy `3k-3l.sql`: account-migration.test.mjs:86,
|
||||
gitea-migration.test.mjs:76, grant-migration.test.mjs:79
|
||||
- plus each one's sibling copy loop, read whole at the edit
|
||||
- To check at build: sessions.test.mjs:435 asserts which step names the edited source holds. Its
|
||||
expected step list may need the new file. Schema step-count pins: schema.test.mjs and
|
||||
stack-release.test.mjs T46 (:913). T46's four table lists gain the two new tables. That is a
|
||||
list gaining a new entry, which falls under item 1's file-list allowance only if Mos reads it
|
||||
that way, so I'll ask before touching it.
|
||||
- After the move, grep `packages/*/tests` for the old label. Only history fixtures and migration
|
||||
chains may still name it. The result is recorded here (mos-1315 item 3).
|
||||
- Comment-only mentions (`3k-3l migration` in end-launch, message-filter, thread-claims, threads,
|
||||
session-launch, thread-cli, thread-launch, chat, session-compactions) are history and stay.
|
||||
|
||||
### Tests (T4, to write)
|
||||
|
||||
- clear on a held session:
|
||||
- accept fences the session
|
||||
- the run's next receive appends `drained` and answers `[]`
|
||||
- SIGTERM, then exit proof, `retired`, the successor row and the default move
|
||||
- a landing-turn run (no receive yet) is never drained or signalled
|
||||
- a run whose turn is running (delivered, unread) stays `clearing` with no signal, past 30 s
|
||||
- a non-default session clears with the default unchanged
|
||||
- `<old>-N` collision: a user session already named `<old>-1` gives `<old>-2`
|
||||
- repeat with the same payload gives the same envelope, id and name
|
||||
- conflict: a changed payload refuses `session-op-conflict`
|
||||
- a second op on a clearing session refuses `session-clearing`
|
||||
- stale revision, halted processing, retired session
|
||||
- undelivered messages are cancelled with receipts, not delivered to the successor
|
||||
- sends to a clearing session refuse
|
||||
- `uncertain` after 30 s with no proof, and no second signal
|
||||
- the successor start is held, started or refused, through check()/start()
|
||||
- the pid-namespace build check
|
||||
|
||||
Reference in New Issue
Block a user