stack/packages/conversation/README.md

conversation

Two layers over Pi conversations, issue #1507 (row 5). Plain ESM, no dependencies, Node 24 or newer.

• The reader (CHAT-02). Read-only histories for the Console: a catalogue of approved session files, full branch history in CHAT-01 pages, and cursors. Opening a conversation never resumes, forks, launches or controls anything, and the reader writes no file. Brief: agents/dewey/work/chat-02/BRIEF.md R4. A library with no server: the control board serves it on two GET routes (D3).
• Mediated control (CHAT-03, increment I1). One controller process per execution owns a sealed headless Pi's stdin and serves a local socket; clients observe, prompt, take control, interrupt, force stop and recover through it. Fixture sessions only. Brief: agents/dewey/work/chat-03/BRIEF.md (pinned 1ef15ac0) with lead decisions 30–34 and 36. See Mediated control.

API

import { rootsFromSpecs, createReader } from "@mosaic/conversation";

const reader = createReader({ roots: () => rootsFromSpecs(specs, registrations) });
reader.catalogue();                                  // { ok, conversations, refusedRoots, generatedAt }
reader.open({ conversation, branch? });              // first page
reader.next({ cursor, conversation, branch });       // next page, or a follow

open and next return { ok: true, page, cursor, follow, view } or { ok: false, refusal: { code, reconcile, message } }.

• page is a CHAT-01 page, and cursor and follow are CHAT-01 cursor records. page.nextCursor is cursor.id when more parts remain in the snapshot.
• On the last page, follow replaces cursor. Calling next with it takes a fresh snapshot and returns only what was appended since (see Follow).
• view holds what the Console needs beyond CHAT-01: defaultBranch, branches, incomplete (a truncated trailing line), forkedFromEarlierSession and unreadableLines.
• actor defaults to local-operator and purpose to history. Any other value is refused.

Sources

Roots come from the board's repository specs only: <projectRoot>/.pi/state/<seat>/sessions, with the project named after the root directory. Fleet and connector specs are not roots, and there is no global scan. A conversation id is pi- plus a hash of project root, seat and file name. It is resolved by listing the roots again, so no path comes from the caller.

A seat registration is seat-written, so it is a hint, not authority:

• It counts for a root only when seat, layout repo, project and sessionsDir all match (samePath). Anything else is ignored.
• It supplies engineStartedAt for conversations created at or after its startedAt.
• A non-Pi harness makes the root unsupported. The root appears as one catalogue row with unsupportedReason: "unsupported-harness" (D2), and its directory is never read.
• A seat on another harness has no Pi sessions directory, so the board has no spec for it. Its registration adds that placeholder row when it names the standard directory under a project root that is already approved. This is Rocko's case today: claude-code, no sessions.

Deviation from brief §2.1: the brief speaks of a registration sessionFile. Registrations have no such field, only sessionsDir, so the rule above applies to the directory. A registration never adds a root or names a file.

Opening a file

src/safe-fs.mjs:

• Every component from the project root down to the sessions directory must be a real directory (lstat).
• A session file must be a regular *.jsonl directly in the root.
• It is opened O_RDONLY | O_NOFOLLOW | O_NONBLOCK, and the descriptor's (dev, ino) must equal the lstat taken before the open.
• The components are checked again after the open. Node has no openat, so a directory swapped between the checks and the open is detected afterwards, not prevented.

The project root itself may be a symlink (the compatibility path to this checkout). The header cwd must be the project or inside it. Real paths are compared, and a cwd that no longer exists is compared as written. That comparison calls realpath on the recorded cwd, which resolves the path but opens nothing.

SessionManager.open is never used.

Parser

src/pi.mjs, pinned against @earendil-works/pi-coding-agent 0.85.1 (docs/session-format.md, dist/core/session-manager.js):

• Line 1 must be the session header.
• Entries form an id/parentId tree. Where an id repeats, the later entry wins, as in Pi's index.
• The default leaf is the last valid entry in file order, as Pi loads it. Each leaf ends one branch; other branches are read-only and opened by branch.
• Branch names do not change while the file grows. The first root's line is main, even before the file has entries. Each later root (Pi's resetLeaf, or an entry whose parent is missing) and each later child at a fork starts a branch named b.<entry id>; the earliest child in file order continues its parent's branch. An inner entry is not a branch.
• A malformed line becomes a notice at its file position on every branch, after the leaf too, and reading continues. The same placement on every branch keeps a branch's earlier parts unchanged while the file grows.
• A missing parent stops the history with a notice. When unreadable lines sit just before the entry, the notice names them as the likely place of the parent. The history is never joined across the gap: the entries before it may belong to another branch, and they read as their own branch. A loop stops with a notice.
• parentSession gives a "forked from an earlier session" notice and is never opened.
• Redacted reasoning (redacted: true) is unavailable with empty text. thinkingSignature is never read.
• A header cwd must be absolute and inside the project; a relative one is refused as foreign-project.
• A compaction is a marker in place, and the full history stays on the path. retainedTail entries are already on the path, so they are not rendered twice.
• Model and thinking-level changes, labels, session names and extension state are not shown. custom_message shows only with display: true. An unknown entry type or role becomes a notice.
• Ids that do not fit the CHAT-01 id pattern are hashed (h- plus 40 hex). OpenAI tool call ids such as call_x|fc_y are the real case. Entry ids are namespaced (n. for native, x. for notices), so no native id can collide with a notice.
• get_entries order is not used.

Pages

src/parts.mjs applies the CHAT-01 limits:

• at most 100 parts and 8 MiB of serialized UTF-8 per page;
• at most 64 blocks per part;
• at most 262144 characters per string.

Strings split into fragments. A fragment is also cut at 1 MiB of JSON, and blocks group into parts of at most 4 MiB, so one part always fits a page. Content is never clipped, and a surrogate pair is never cut.

Snapshots, epochs, cursors

• A snapshot is the file up to its last newline when the descriptor was read. A trailing partial line is left out and sets view.incomplete. snapshotDigest is the SHA-256 of those bytes.
• sourceEpoch is a hash of (dev, ino) and the digest at open. Follows carry it forward while the prefix verifies, so growth keeps the epoch. A new open after growth hashes the longer prefix and gets a different id for the same epoch, so an epoch id is comparable only within one cursor chain.
• Every next re-reads the pinned prefix and refuses source-replaced (with reconcile) when dev or ino changed, the file is shorter, or the prefix digest differs (an in-place rewrite with the same inode).
• Cursors live in memory, with an LRU cap (1000) and a 10-minute TTL. They are bound to actor, purpose, conversation, branch, snapshot, epoch and expiry.
• Refusals: cursor-unknown (never issued or evicted), cursor-expired, cursor-foreign (any binding differs) and source-replaced. All carry reconcile: true. A foreign attempt does not consume the cursor, so the old view keeps working.
• local-operator is the only actor on this unauthenticated loopback route. That is not multi-actor safety. Authenticated actors come with CHAT-04R.

Follow

A follow cursor verifies the old prefix, then takes a fresh snapshot and reads the same branch in it. page.branch is always the cursor's branch.

• The parts before the cursor must be unchanged (compared as a digest of entry ids). If they are, the page holds only the parts appended to this branch, which is empty when the conversation continued on another branch. view.defaultBranch shows where Pi's default leaf is now.
• If the branch is gone or its earlier parts changed (a duplicate id that replaces an entry, or a missing parent that turns up later), it refuses source-replaced.

Refusal codes

Code	Meaning	Reconcile
unknown-conversation	not in the approved roots now	yes
unknown-branch	not a branch of this conversation	yes
unavailable	the session root no longer exists	no
unsupported-harness	non-Pi seat (D2)	no
unsafe-path	symlink, bad name, not a regular file, swapped	no
foreign-project	header cwd outside the project	no
unreadable	permission denied on a file, a root or a directory above it inside the approved roots	no
not-a-pi-session, incomplete-header	first line is not a complete Pi header	incomplete: yes
too-large	over 256 MiB	no
cursor-unknown, cursor-expired, cursor-foreign, source-replaced	see above	yes
unknown-actor, unsupported-purpose	not local-operator / history	no

Costs

Each page reads and hashes the whole pinned prefix. A parse cache and a branch-entries cache (two snapshots each) avoid re-parsing. On this checkout's 18 roots (largest file 18.6 MB), reading every page of every conversation took at most 680 ms per conversation.

Tests

node --test packages/conversation/tests/ runs both layers. For the reader, reader.test.mjs covers fixtures F1–F15 and F17 of the CHAT-02 brief (F16 is in the control-board suite, which serves the routes). The CHAT-03 suites are listed under Mediated control tests.

• Every reader call runs inside a fingerprint of the fixture tree: size, SHA-256, mtime, (dev, ino), mode and every directory listing, before and after.
• Files that no read may touch are mode 000, so an attempted open would throw EACCES rather than refuse.
• Every page and cursor is validated against docs/plans/chat-01/contracts.schema.json with Python jsonschema.

Mediated control (CHAT-03)

Increment I1 of CHAT-03. The approval races H5–H8 and the Claude adapter are I4; recorded runs against a real model (I3) wait on Jason's go. Everything here runs on fixture sessions in temporary directories. No code path opens a live session, and the controller never writes a session file (W11).

Pieces

File	Role
src/controller.mjs	Controller: claim, launch, socket, admission, dispatch, stop chain, recovery
src/client.mjs	ConversationClient: the library every client uses, the terminal included
src/transcript.mjs	Transcript: page plus live stream, with the seam rules below
src/terminal.mjs	the mediated terminal, node packages/conversation/src/terminal.mjs --socket <path> [--grant <id>]
src/claim.mjs	ClaimStore: the writer claim per seat key and session key, revisions r<10 digits>.json
src/cohort.mjs	ScopeLauncher (systemd user scope plus shim.mjs), PgroupLauncher, force stop, cohort and boot proofs
src/shim.mjs	the scope's first process, outside the engine cgroup; ops hello, events, members, term, freeze, kill, release; ignores SIGTERM
src/guard.mjs	LiveSessionGuard: refuses live sessions and paths under live roots
src/pi-pin.mjs	the Pi pin (0.85.1 and its integrity) and the seal
src/engine.mjs, src/framing.mjs	the engine link and LF framing
src/turns.mjs	Tracker: runs, the dispatch slot, overlap signals O1–O6
src/events.mjs	native event to CHAT-01 event mapping
src/text-policy.mjs	slash refusal and the prefix table
src/records.mjs	CHAT-01 v2 records, hashes, FixtureVerifier

import { Controller } from "./src/controller.mjs";
import { ConversationClient } from "./src/client.mjs";
import { Transcript } from "./src/transcript.mjs";

const ctrl = new Controller({ fixtureRoot, claimRoot, socketDir, sessionFile, seat, verifier });
await ctrl.start();                                   // claim, launch, K8 load check, listen
const client = new ConversationClient({ socketPath: ctrl.socketPath });
const view = new Transcript({ client });              // reads the page on every welcome
await client.connect();                               // starts as an observer
await client.takeover();                              // becomes the controller
const r = await client.prompt("hello");               // { outcome: "admitted", receipt } or { outcome: "refused:<code>", refusal }
await client.interrupt();
await client.confirmed("force-stop");                 // issue, answer, then use a confirmation

The four required paths have no defaults: a missing one refuses configuration. The session file must sit at <project>/.pi/state/<seat>/sessions/<name>.jsonl inside fixtureRoot.

Choices

Each is a reading of the brief or a lead decision, recorded so a reviewer can disagree with it.

• Seal. The argv is --mode rpc --no-extensions --no-prompt-templates --no-themes --session <absolute file>, then optional engine.extraArgs. The seal is an allow-list: extraArgs may carry only --model, --provider and --thinking, each at most once with one plain value (not starting with - or @). Anything else refuses unsealed-engine at construction and again at bind, before spawn: an -e/--extension argument, a missing --no-* flag, a second --mode or --session (Pi keeps the last of each), a session or output flag (--no-session, --fork, --export, --print, --continue, ...) or a bare word, which Pi reads as a prompt (lead decision 31; N24, including the missing flag through checkSeal).
• Engine command. engine.command and engine.preArgs default to node <pinRoot>/node_modules/@earendil-works/pi-coding-agent/dist/bundle/cli.js. A non-default value is a test hook for the fake engine. The pin check reads only the lock files under pinRoot and the seal checks only Pi's arguments, so neither says what runs under an overridden command. The binding's argvDigest records the full command line. preArgs carrying --extension still refuses.
• Session key. The claim's session key is the Pi header ID (D1), read at construction. A hard link or a copy of a session under another seat has a different conversation ID but the same header ID, so its controller refuses already-active (W4). Every later read of the session refuses target if the header ID changed.
• Live roots. The guard protects ~/.pi, ~/.claude and ~/.mosaic-dev through os.homedir(), which follows $HOME. Under a scratch HOME the real directories are protected only by the fixture-root containment, or by passing guardOptions.homes.
• Seal and attribution. Under the seal the Mosaic prompt is the only input path, so working is attributed through the seal: the slot's run is the one whose first user message follows the ack with no overlap signal.
• Overlap. aborted with no stop in progress is an overlap signal, never a stop link (lead decision 34, N9). An aborted links to the stop in progress only once the controller wrote an abort in that stop's chain (the stop or one it superseded). One that lands after the fence but before any abort is the same overlap (N9). An overlap signal (O1–O6) closes admission, makes the binding uncertain and settles the slot's receipt with reason run-overlap.
• Interrupt. It fences admission, clears the queue, then aborts. With no slot and no run it refuses no-turn and lifts only the fence it set. It never reopens admission that a concurrent force stop, overlap signal or revocation closed (Rocko's build note, brief line 288; H10). It checks for an overlap again right before the abort, so an overlap read during the pause (an O5 in the same chunk as the clear's response) means no abort, which would run what was queued (H10). A stop that ends uncertain while its queue state was still pending records nativeQueue: "unknown".
• Force stop supersedes a running Interrupt: one stop chain (H10). One escalation runs at a time: a second force stop while one runs refuses fenced, and only the running stop's phases reach the claim. After an escalation ends uncertain, a fresh confirmation can retry it (H10, H17). The scope launcher runs a TERM phase first: the shim sends SIGTERM to each member and waits a bounded grace for populated 0. The kill phase then freezes engine, waits for frozen 1, enumerates the members for the proof, writes cgroup.kill and waits for populated 0 (K3, K12). The shim sits in a supervisor cgroup outside engine and ignores SIGTERM only so that a stray TERM never drops the scope's anchor. A missing or unreadable engine cgroup is an absent observation, never an empty one: the stop ends uncertain with no proof (K15). The process-group fallback can't enumerate a member that left the group, so its force stop ends uncertain, never stopped (K2). The fake launcher's forceStop is fixture-only.
• Confirmations bind the target, operation and stop, and are consumed on use (K6). One issued before the stop changed is refused (H17).
• Recovery. A confirmed recover after a proven stop returns a single-use eligibility record, and the launcher calls launch with it. A second call, a record from another incarnation, or a reserved claim that changed refuses eligibility (K17); a session leaf or branch that moved since eligibility refuses target (K18). In K7, "incarnation" means the execution: a recovery mints a new execution and controller incarnation, generation +1, on the same leaf. An orphan (H20) has no controller: its controllerConnection is null.
• Approval. A stop's approvalDisposition is resolved unless a Pi dialog was seen during the execution, then uncertain. Pi dialogs (select, confirm, input, editor) are pushed to connected clients disabled with a reason and never answered (lead decision 30, P3). A dialog pushed before a client connected is not replayed to it; the controller's evidence keeps the list.
• Run order. N8 pins the wire order with the fake's run-start hold point: the Mosaic ack, another run's agent_start and user message, then the losing Mosaic settle, which is O3.
• Startup. G2 refuses a symlinked live path at construction; G3 refuses a swap at bind. The K8 load check reads get_state (sessionFile, sessionId) and get_tree (leafId) after launch. If the engine loaded another file or leaf, the binding goes uncertain, admission never opens, the claim stays held until a proven stop, and a prompt refuses preflight (K8).
• Observe. limit (1–100) is advisory and marked limitAdvisory: true; the reader's page size governs. Drafts are client-local: the library sends a draft ID and revision 1 with each prompt and keeps no draft state.
• Escape hatches. K13 (a member writing its pid into another cgroup) is refused by the cgroup namespace the shim gives the engine (unshare --cgroup), on a host whose cgroup2 is mounted nsdelegate. Nothing checks nsdelegate at runtime; on a host without it the refusal isn't shown, and that is a portability question for the cutover increment.

The seam

Replay is unavailable in CHAT-03, and page entries and live events carry different IDs, so overlap at the seam can't be deduplicated (CHAT-01 lines 104–110). The controller's observe reply carries seam: { replay: "unavailable", streamEpoch, fromSequence, reconcile: true, quiet }.

• quiet is true when no run was visible and no prompt held the slot as the page was read. Pi persists each message on message_end, before the run settles (agent-session.js 386–398), so a quiet cut misses nothing.
• A cut that isn't quiet puts a reconcile marker at the seam. Near it a message may repeat or be missing. Transcript re-reads the page after the next run-settled and clears the marker once a read is quiet (E4).
• A sequence gap, a new stream epoch, or a repeated event ID with different bytes marks the seam and re-reads at once. Events past a gap are held, never concatenated. An identical repeat is dropped (E3).
• A tool call shows while it runs. Once a finished message carries its result, from the stream or the page, the progress item is hidden.
• thinking_level_change entries are not messages and never appear in pages.

Slash text

textPolicy refuses any prompt whose text, after leading whitespace, starts with /: pinned Pi runs extension commands, skills (/skill:) and prompt templates from index 0 (S1, S2). !, !! and @ are interpreted only by Pi's interactive mode and CLI, not on the RPC prompt path, so they are admitted and sent as text. A / on a later line is not interpreted (LATER_LINE_SLASH_INTERPRETED = false, S3). The table with its source lines is PREFIXES in src/text-policy.mjs; S2 iterates the same table.

The terminal

A thin view over the client library; it renders the same Transcript (E7).

• Enter submits, Ctrl-J or Alt-Enter adds a newline, Ctrl-T takes control, Ctrl-G interrupts, Ctrl-O reconnects if needed and re-reads the page, PageUp and PageDown scroll, Ctrl-C or Ctrl-D quits.
• The composer is local. It clears after each submit and whenever the controller changes (S4). An observer's Enter shows not admitted: controller and sends nothing; the buffer is kept (S5).
• Enter takes the composer at that key: text after it in the same input chunk starts the next message.
• A bracketed paste is inserted literally, newlines included, and never submits by itself. A paste marker split across input chunks, even right after its ESC, is still a paste marker; a lone trailing ESC waits for the next chunk.
• Engine text is shown with C0 and C1 controls, DEL, U+2028, U+2029, bidi controls (U+061C, U+200E, U+200F, U+202A–U+202E, U+2066–U+2069) and invisible characters (U+200B, U+2060–U+2064, U+FEFF, tag characters U+E0000–U+E007F) made visible (^[, <U+202E>), so transcript content can't drive or spoof the operator's terminal. ZWJ and ZWNJ pass, for emoji sequences and joining scripts. The header, status, notices and dialogs also show LF as ^J, so each stays one line.
• A request whose outcome is lost shows outcome unknown, check the transcript. Nothing is resent and no resend is offered.

Refusals

Replies are { outcome: "refused:<code>", refusal: "<code>" }. Admission runs in CHAT-01 check.mjs order, then the CHAT-03 narrowings.

Code	When
malformed	a line that isn't JSON, a message other than hello first, or a request that fails the envelope check
grant	the hello names no active grant
channel	the connection isn't connected or its channel isn't authenticated; also the library's reply when its socket is closed
unsupported-capability	a private-host transport, or an operation I1 doesn't verify
scope, mapping, audit	grant scope, source mapping or audit sink doesn't hold
capability	the grant lacks the operation's capability
target	the target names another conversation, execution or branch; the session header ID changed since construction (W4); the leaf moved since proof (recover) or since eligibility (launch, K18)
generation	the request's controller generation is stale (H1, H2)
controller	the connection isn't the controller
conflicting-request	a request ID reused with other content (H13)
stale-incarnation	the request carries an old controller incarnation (H21)
fenced	admission is closed (Interrupt, force stop, overlap, revocation, uncertain), or a force stop while one escalation runs (H10)
busy	a prompt already holds the dispatch slot; CHAT-03 has no broker queue (H18)
text-policy	slash text (S1, S2)
draft	prompt text missing or longer than 262,144 characters
no-turn	Interrupt with no slot and no run (N16)
already-controller, controller-present	self-takeover (H4); recovery control while a controller is connected
confirmation	missing, reused, foreign, expired or stop-changed confirmation (H17)
stop-proof	recovery without a verified stop
preflight	the K8 load check didn't pass; admission never opened
cursor	an observe cursor the reader refuses (detail names the reader code)
eligibility	launch with a used or unknown eligibility record, one from another incarnation, or a changed reservation (K17)
already-active, unsafe-replacement, foreign-host	the writer claim (W2, W3, W17)
live-session-refused	the live-session guard, at construction, bind and launch (G1–G3)
unsealed-engine, engine-pin-mismatch	the seal or the Pi pin (N24)
configuration	a required constructor path is missing or misplaced, the session file is unreadable, or the session header ID isn't a CHAT-01 ID

malformed, eligibility and preflight are names this build adds; the brief describes those refusals without naming them.

The controller, claim store, live-session guard and engine pin throw ControlRefusal (safe-fs.mjs), a subclass of the reader's Refusal. These codes reach a socket client, never the reader's HTTP routes, so the control board's status map (REFUSAL_STATUS, which its tests check against every new Refusal("…") in src/) covers only the reader table above. If the board ever serves the controller, these codes need statuses there first.

Receipt reason codes, not refusals: transport-unknown, handled-without-run, ack-without-start, interrupted, run-overlap.

Findings against pinned Pi

From tests/smoke.test.mjs, which starts the pinned binary sealed, in a scratch home with an empty agent dir and no inherited environment, and sends no prompt:

• It starts with no credentials and answers get_state, get_commands, clear_queue, abort and get_tree. Every field the fake engine sends for those commands, pinned Pi sends with the same type. clear_queue emits queue_update with empty queues before its response, as the fake does. Pi writes auth.json ({}) and models-store.json into the agent dir.
• Startup append. Pi appends to the session at startup in two cases (sdk.js 82–83 and 240–252). A session with messages but no thinking_level_change on its branch gains one. A session with no messages takes Pi's new-session branch and gains a thinking_level_change at every start, plus a model_change when a model is set, whether or not it already has a thinking entry. That covers a Pi-created session that was opened and never prompted. Either way the leaf moves after launch, the K8 load check fails, the binding goes uncertain and prompts refuse preflight. A pre-spawn check would refuse both kinds before launch; that is a later increment, not built here.
• One inline extension command. Under the seal, get_commands still lists /llama (source extension, inline: Pi's bundled llama.cpp router). Slash text is refused at admission, so it can't be invoked. The smoke test pins the list so a change shows.

Mediated control tests

Suite	Covers
claim.test.mjs	W1–W17, W20, G1–G3: the writer claim, crash barriers, the guard
races.test.mjs	H1–H4, H9–H23: takeover, Interrupt and force stop, retries, incarnations
turns.test.mjs	N1–N25: the turn tracker against the fake engine's Pi behaviors
cohort.test.mjs	K1–K18: scopes, force stop, proofs, recovery, eligibility (needs a systemd user manager)
flows.test.mjs	S1–S7, P3, E1–E7, the terminal, and a CHAT-01 schema check of every record produced
smoke.test.mjs	the pinned Pi binary, as above

fake-pi.mjs models pinned Pi's RPC mode, including the startup append, and ctrl-child.mjs runs a controller in a child process for the crash tests. Fixtures live in temporary directories and are removed after each file.