stack/packages/conversation/README.md

# conversation

Read-only Pi conversation histories for the Console: a catalogue of approved
session files, full branch history in CHAT-01 pages, and cursors. Opening a
conversation never resumes, forks, launches or controls anything, and nothing
here writes a file.

Issue #1507 (CHAT-02, row 5). Brief: `agents/dewey/work/chat-02/BRIEF.md` R4.
Plain ESM, no dependencies, Node 24 or newer. A library with no server: the
control board serves it on two GET routes (D3).

## API

```js
import { rootsFromSpecs, createReader } from "@mosaic/conversation";

const reader = createReader({ roots: () => rootsFromSpecs(specs, registrations) });
reader.catalogue();                                  // { ok, conversations, refusedRoots, generatedAt }
reader.open({ conversation, branch? });              // first page
reader.next({ cursor, conversation, branch });       // next page, or a follow
```

`open` and `next` return `{ ok: true, page, cursor, follow, view }` or
`{ ok: false, refusal: { code, reconcile, message } }`.

- `page` is a CHAT-01 `page`, and `cursor` and `follow` are CHAT-01 `cursor`
  records. `page.nextCursor` is `cursor.id` when more parts remain in the
  snapshot.
- On the last page, `follow` replaces `cursor`. Calling `next` with it takes
  a fresh snapshot and returns only what was appended since (see Follow).
- `view` holds what the Console needs beyond CHAT-01: `defaultBranch`,
  `branches`, `incomplete` (a truncated trailing line),
  `forkedFromEarlierSession` and `unreadableLines`.
- `actor` defaults to `local-operator` and `purpose` to `history`. Any other
  value is refused.

## Sources

Roots come from the board's repository specs only:
`<projectRoot>/.pi/state/<seat>/sessions`, with the project named after the
root directory. Fleet and connector specs are not roots, and there is no
global scan. A conversation id is `pi-` plus a hash of project root, seat and
file name. It is resolved by listing the roots again, so no path comes from
the caller.

A seat registration is seat-written, so it is a hint, not authority:

- It counts for a root only when seat, layout `repo`, project and
  `sessionsDir` all match (`samePath`). Anything else is ignored.
- It supplies `engineStartedAt` for conversations created at or after its
  `startedAt`.
- A non-Pi `harness` makes the root unsupported. The root appears as one
  catalogue row with `unsupportedReason: "unsupported-harness"` (D2), and its
  directory is never read.
- A seat on another harness has no Pi sessions directory, so the board has no
  spec for it. Its registration adds that placeholder row when it names the
  standard directory under a project root that is already approved. This is
  Rocko's case today: `claude-code`, no `sessions`.

Deviation from brief §2.1: the brief speaks of a registration `sessionFile`.
Registrations have no such field, only `sessionsDir`, so the rule above
applies to the directory. A registration never adds a root or names a file.

## Opening a file

`src/safe-fs.mjs`:

- Every component from the project root down to the sessions directory must
  be a real directory (lstat).
- A session file must be a regular `*.jsonl` directly in the root.
- It is opened `O_RDONLY | O_NOFOLLOW | O_NONBLOCK`, and the descriptor's
  (dev, ino) must equal the lstat taken before the open.
- The components are checked again after the open. Node has no `openat`, so
  a directory swapped between the checks and the open is detected afterwards,
  not prevented.

The project root itself may be a symlink (the compatibility path to this
checkout). The header `cwd` must be the project or inside it. Real paths are
compared, and a `cwd` that no longer exists is compared as written. That
comparison calls `realpath` on the recorded `cwd`, which resolves the path but
opens nothing.

`SessionManager.open` is never used.

## Parser

`src/pi.mjs`, pinned against `@earendil-works/pi-coding-agent` 0.85.1
(`docs/session-format.md`, `dist/core/session-manager.js`):

- Line 1 must be the session header.
- Entries form an `id`/`parentId` tree. Where an id repeats, the later entry
  wins, as in Pi's index.
- The default leaf is the last valid entry in file order, as Pi loads it.
  Each leaf ends one branch; other branches are read-only and opened by
  `branch`.
- Branch names do not change while the file grows. The first root's line is
  `main`, even before the file has entries. Each later root (Pi's
  `resetLeaf`, or an entry whose parent is missing) and each later child at a
  fork starts a branch named `b.<entry id>`; the earliest child in file order
  continues its parent's branch. An inner entry is not a branch.
- A malformed line becomes a notice at its file position on every branch,
  after the leaf too, and reading continues. The same placement on every
  branch keeps a branch's earlier parts unchanged while the file grows.
- A missing parent stops the history with a notice. When unreadable lines sit
  just before the entry, the notice names them as the likely place of the
  parent. The history is never joined across the gap: the entries before it
  may belong to another branch, and they read as their own branch. A loop
  stops with a notice.
- `parentSession` gives a "forked from an earlier session" notice and is
  never opened.
- Redacted reasoning (`redacted: true`) is `unavailable` with empty text.
  `thinkingSignature` is never read.
- A header `cwd` must be absolute and inside the project; a relative one is
  refused as `foreign-project`.
- A compaction is a marker in place, and the full history stays on the path.
  `retainedTail` entries are already on the path, so they are not rendered
  twice.
- Model and thinking-level changes, labels, session names and extension state
  are not shown. `custom_message` shows only with `display: true`. An unknown
  entry type or role becomes a notice.
- Ids that do not fit the CHAT-01 id pattern are hashed (`h-` plus 40 hex).
  OpenAI tool call ids such as `call_x|fc_y` are the real case. Entry ids are
  namespaced (`n.` for native, `x.` for notices), so no native id can collide
  with a notice.
- `get_entries` order is not used.

## Pages

`src/parts.mjs` applies the CHAT-01 limits:

- at most 100 parts and 8 MiB of serialized UTF-8 per page;
- at most 64 blocks per part;
- at most 262144 characters per string.

Strings split into fragments. A fragment is also cut at 1 MiB of JSON, and
blocks group into parts of at most 4 MiB, so one part always fits a page.
Content is never clipped, and a surrogate pair is never cut.

## Snapshots, epochs, cursors

- A snapshot is the file up to its last newline when the descriptor was read.
  A trailing partial line is left out and sets `view.incomplete`.
  `snapshotDigest` is the SHA-256 of those bytes.
- `sourceEpoch` is a hash of (dev, ino) and the digest at open. Follows carry
  it forward while the prefix verifies, so growth keeps the epoch. A new open
  after growth hashes the longer prefix and gets a different id for the same
  epoch, so an epoch id is comparable only within one cursor chain.
- Every `next` re-reads the pinned prefix and refuses `source-replaced` (with
  reconcile) when dev or ino changed, the file is shorter, or the prefix
  digest differs (an in-place rewrite with the same inode).
- Cursors live in memory, with an LRU cap (1000) and a 10-minute TTL. They are
  bound to actor, purpose, conversation, branch, snapshot, epoch and expiry.
- Refusals: `cursor-unknown` (never issued or evicted), `cursor-expired`,
  `cursor-foreign` (any binding differs) and `source-replaced`. All carry
  `reconcile: true`. A foreign attempt does not consume the cursor, so the old
  view keeps working.
- `local-operator` is the only actor on this unauthenticated loopback route.
  That is not multi-actor safety. Authenticated actors come with CHAT-04R.

### Follow

A follow cursor verifies the old prefix, then takes a fresh snapshot and
reads the same branch in it. `page.branch` is always the cursor's branch.

- The parts before the cursor must be unchanged (compared as a digest of
  entry ids). If they are, the page holds only the parts appended to this
  branch, which is empty when the conversation continued on another branch.
  `view.defaultBranch` shows where Pi's default leaf is now.
- If the branch is gone or its earlier parts changed (a duplicate id that
  replaces an entry, or a missing parent that turns up later), it refuses
  `source-replaced`.

## Refusal codes

| Code | Meaning | Reconcile |
|---|---|---|
| `unknown-conversation` | not in the approved roots now | yes |
| `unknown-branch` | not a branch of this conversation | yes |
| `unavailable` | the session root no longer exists | no |
| `unsupported-harness` | non-Pi seat (D2) | no |
| `unsafe-path` | symlink, bad name, not a regular file, swapped | no |
| `foreign-project` | header `cwd` outside the project | no |
| `unreadable` | permission denied on a file, a root or a directory above it inside the approved roots | no |
| `not-a-pi-session`, `incomplete-header` | first line is not a complete Pi header | incomplete: yes |
| `too-large` | over 256 MiB | no |
| `cursor-unknown`, `cursor-expired`, `cursor-foreign`, `source-replaced` | see above | yes |
| `unknown-actor`, `unsupported-purpose` | not `local-operator` / `history` | no |

## Costs

Each page reads and hashes the whole pinned prefix. A parse cache and a
branch-entries cache (two snapshots each) avoid re-parsing. On this checkout's
18 roots (largest file 18.6 MB), reading every page of every conversation took
at most 680 ms per conversation.

## Tests

`node --test packages/conversation/tests/` covers fixtures F1–F15 and F17 of
the brief (F16 is in the control-board suite, which serves the routes).

- Every reader call runs inside a fingerprint of the fixture tree: size,
  SHA-256, mtime, (dev, ino), mode and every directory listing, before and
  after.
- Files that no read may touch are mode 000, so an attempted open would throw
  `EACCES` rather than refuse.
- Every page and cursor is validated against
  `docs/plans/chat-01/contracts.schema.json` with Python `jsonschema`.