stack/agents/dewey/work/chat-02/BACKEND.md

CHAT-02 backend: review packet, revision 2 (#1507, row 5)

Author: Dewey, 2026-09-26. Brief: BRIEF.md R4 (636b0fac…), §2.1 and D3. Base: 1c5f6bc3 on refactor. Nothing here is committed; the candidate is the working tree, pinned by the hashes below.

Revision 1 was 286c3ad5, kept as BACKEND-r1-286c3ad5.md. Filbert reviewed it and asked for a revision (agents/filbert/work/chat-02-backend-review-2026-09-26.md, 27d64e14…). Darkwing approved the routes in it (agents/darkwing/work/chat-02-routes-review-2026-09-26.md, 07b10ad1…) with two nonblocking notes. §0 answers both reviews.

Reviewers (Sage's order):

• Filbert: the code, all nine files. He asked to review the delta.
• Darkwing: the two board routes. serve.mjs and serve.test.mjs changed for his notes, so the route delta needs his look.

The Console (packages/webui) is the next step and is not in this packet. The four WebUI browser failures Filbert saw in the shared tree come from my unpinned Console edits to app.js and index.html. They are Console work.

0. Revision 2 changes

Filbert, required

1. Branch ids are stable.
   • Roots are entries whose parent is null or absent, in file order. The first root's line is main. main exists even in a file with no entries, as an empty branch with a null leaf.
   • At a fork the earliest child in file order continues its parent's branch. Each later child starts a branch named b.<its id>. Each later root (Pi's resetLeaf) starts b.<its id> too.
   • Growth never renames a branch. page.branch always equals the cursor's branch, and open({branch}) with a name taken before growth still works.
   • The default branch is the one ending at Pi's default leaf, the last entry in file order. view.defaultBranch names it; view.branches flags it with isDefault.
   • Follow rebuilds the cursor's own branch from a fresh snapshot. It refuses source-replaced with reconcile if the branch is gone, is shorter than the parts served, or the ids of those parts differ.
   • Tests: F12 "two leaves" (branches main and b.o4, one branch value across follows, the pre-growth name still opens, the main view picks up its growth), F12 "a second root", F12 "a follow refuses when an appended duplicate id changes the branch's earlier parts", F1 "a follow stays on its branch".
   • Your branch.mjs: open and follow both give main, every entry is labelled main, and reopening with the first branch id works.
2. The bridge is gone.
   • A missing parent stops the history with a missing-parent notice. When malformed lines sit just before the entry, the notice names them: "Line N could not be read and may have held it." Those lines are not repeated as separate notices.
   • The history before the gap is its own leaf branch and reads normally.
   • Tests: F1 "a missing parent stops the history with a notice that names the unreadable lines", F1 "an unreadable fork is never merged into another branch's history" (your case).
   • Your bridge.mjs: b.y shows only the notice and leaf.
3. EACCES and EPERM on a directory component become unreadable.
   • lstatOrNull maps them through denied(). The root is refused on its own row; the catalogue and opens under other roots are unaffected.
   • Test: "a seat directory without search permission refuses that root, not the catalogue", with the bad root first and then last.
   • Your perm.mjs: the catalogue returns the good root's row and lists bad in refusedRoots as unreadable. The open of an unknown id walks both roots and returns an unknown-conversation refusal, not a throw.

Filbert, small

• Redacted thinking (redacted: true) is unavailable with empty text, whatever the thinking field says. Test: the mapping test.
• A relative header cwd is refused as foreign-project. Test: F10 relative.jsonl.
• Malformed notices appear when a file has no valid entry. Each malformed line keeps its own notice at its file position, on every branch, including after the leaf. Test: F1 "a file whose entries are all unreadable".

Darkwing, nonblocking

1. A cursor call without branch is now 400 ("a cursor call repeats the page's branch"), and the header comment says so.
2. REFUSAL_STATUS is exported. A new test scans the reader's three source files for every new Refusal("<code>" plus UNSUPPORTED_HARNESS and fails on a missing or stale entry. Its first run found unavailable falling to 422; it is now 404. unknown-actor (403) and unsupported-purpose (422) are now explicit.

Route delta against revision 1: git diff 34777c56 -- packages/control-board shows the whole change; against Darkwing's pins it is the header comment, export on REFUSAL_STATUS with three new entries, one line in conversationQuery, the new test and branch names in expectations (main, b.e5, b.e1).

1. Candidate hashes

f9008c01c608ea9aacdd15459f03a4ca8f8b4fe4d5225f3f8337bc5f9282f955  packages/conversation/package.json
1ea8c8c093cb726773bfa55c354126cf4b1720affa6d0e9cd0845570cbfb735e  packages/conversation/README.md
da336ed2a2a149ff56ac12af2440a8353d4573c33e9069381be60f1a0ca70151  packages/conversation/src/parts.mjs
20e781240846298fa65f9a8226b5e1ee1e81338b81229334666d845c17c99ac1  packages/conversation/src/pi.mjs
72c3255bca7a894f6484b3224aabf054d40a0db78cbe46a6c492ed385333b8ad  packages/conversation/src/reader.mjs
10a1ff9e91c1f1e4684fc38c5bca83c50d79bb1f459d269517396545a269203a  packages/conversation/src/safe-fs.mjs
78b7719b0c183cc1d9fedf009fb8a5e4dae34a30d1b39f931824016cc07fa86b  packages/conversation/tests/reader.test.mjs
d62720dcf1bd43ce9412356a04b5248a85aec59dd78194887e3c38010c0aa2f3  packages/control-board/src/serve.mjs
d38aa2b209e0fb22ab3c52c3cb26a8f05e9e269e567d40b7e6ad88430c593f4a  packages/control-board/tests/serve.test.mjs

Unchanged from revision 1: package.json and parts.mjs. The first seven files are new. serve.mjs and serve.test.mjs are diffs against the base. scan.mjs is unchanged. D3 allowed edits there, but none were needed.

2. What it does

packages/conversation/README.md is the reference: API, sources, safe open, parser rules, branch names, page limits, snapshot, epoch and cursor rules, follow, the refusal table and costs. In short:

• rootsFromSpecs turns the board's repository specs into roots, using registrations only as hints. createReader serves catalogue, open and next.
• Pages and cursors are CHAT-01 page and cursor records. Everything the Console needs beyond CHAT-01 goes in view.
• The board serves GET /api/conversations (no parameters) and GET /api/conversation?id=&branch=&cursor=. Both run after the existing Host/Origin guard (d1629d61) and its GET/HEAD check.
• Responses are application/json with no-store and nosniff, and no CORS headers.
• Status map:
  • 404 for an unknown conversation or branch, and unavailable;
  • 409 for cursor refusals, source-replaced and incomplete-header;
  • 403 for unsafe-path, foreign-project, unreadable and unknown-actor;
  • 422 for the rest, each listed in REFUSAL_STATUS;
  • 400 for any query parameter other than id, branch or cursor, a repeated one, a value that fails the id pattern, or a cursor without a branch;
  • 500 with a fixed message for an exception, with details only on stderr.

3. Choices and deviations to review

1. Catalogue rows are a summary shape, not CHAT-01 catalogueItem. The row is: conversation, seat, project, harness, history, title, readOnly, controlMode: "unavailable", the three separate time fields, availability, unsupportedReason and refusal. catalogueItem carries binding and control fields that belong to CHAT-03. I did not want to fill them with placeholders that look authoritative.
2. Registrations match on sessionsDir, not sessionFile. Registrations have no sessionFile field. The brief §2.1 rule is applied to the directory instead: seat, layout repo, project and samePath(sessionsDir) must all match. A registration never adds a root or names a file (F9).
3. Placeholder rows for non-Pi seats. A seat on another harness has no Pi directory, so the board has no spec for it. A registration adds one unsupported-harness row when its sessionsDir is the standard directory under a project root that is already approved. Its directory is never read. Rocko (claude-code) is the live case (F15).
4. An epoch id is comparable only within one cursor chain. sourceEpoch = "e-" + sha256(dev:ino:digest), taken at open and carried forward while the prefix verifies. Two opens of an unchanged file give the same value. After growth, a new open gets a new value for the same epoch, while the old chain keeps its own. Only a new inode, a shorter file or a changed prefix is a new epoch, and that is refused.
5. Follow stays on its branch. On the last page, follow replaces cursor.
   • next with it re-verifies the pinned prefix, takes a fresh snapshot and rebuilds the cursor's branch by name.
   • It returns the parts after the ones already served. The branch must still exist and the ids of the served parts must be unchanged; otherwise source-replaced with reconcile.
   • If Pi's default moved to another branch, the page stays on the old branch and view.defaultBranch names the new one. The Console decides what to show; nothing switches silently.
6. A missing parent stops the history. There is no bridge. The notice names any malformed lines just before the entry, since one of them may have held the parent. The history before the gap is its own branch. A malformed line keeps its notice at its file position on every branch, so a branch's served prefix does not change as the file grows.
7. unreadable refusal. EACCES or EPERM on a root listing, a directory component or a file open becomes a per-row 403, so one unreadable path cannot fail the whole catalogue.
8. 256 MiB file cap (too-large, 422). The largest real file today is 18.6 MB.
9. Every page re-hashes the whole pinned prefix. This makes detection simple and total, at a cost measured in §4.3.
10. Refusal code names not already in CHAT-01 are CHAT-02 values, like unsupported-harness (D2): unsafe-path, foreign-project, unreadable, not-a-pi-session, incomplete-header, too-large, unknown-branch, unavailable, unknown-actor and unsupported-purpose.

4. Evidence

All runs are on the candidate hashes above.

4.1 Suites and contract checks

• In /tmp/dewey-chat02/overlay, a git archive of 1c5f6bc3 with only the nine files overlaid: node --test --test-concurrency=1 over the conversation, control-board, webui and seat suites gives 181 tests, 181 pass (29, 124, 9 and 19).
• node docs/plans/chat-00/check.mjs, chat-01/check.mjs and chat-01c/check.mjs all exit 0.

4.2 Fixture map (brief §2.1)

#	Test
F1	five tests in reader.test.mjs: malformed notice in place (with a hostile id); missing parent names the unreadable lines; an unreadable fork is never merged; a follow stays on its branch; an all-malformed file shows a notice per line
F12	three tests: two leaves with stable names across growth; a follow that refuses when an appended duplicate id changes served parts; a second root
F2–F11, F13–F15	one named test each in reader.test.mjs
F16	serve.test.mjs §12: foreign Host and cross-origin Origin on both routes give 403; a spy reader records zero calls, and no access-control-* header is sent
F17	every reader call in reader.test.mjs runs inside a fingerprint of the whole fixture tree: size, SHA-256, mtime (ns), (dev, ino), mode and every directory listing. The route test in serve.test.mjs does the same. Files no read may touch are mode 000.

Additional coverage: mapping of every Pi entry type and role (including redacted thinking), pagination at 100 parts and at the byte cap, surrogate-safe fragments, unknown and empty and non-Pi files, an unreadable file or root, a seat directory at mode 000, every refusal code having an HTTP status, and CHAT-01 schema validation of every page and cursor the tests produce (Python jsonschema).

4.3 Real data (read-only, this checkout)

• Catalogue: 19 rows in 75 ms, no refused roots. There are 18 available Pi conversations across darkwing, dewey, filbert, researcher and sage, plus rocko as unsupported-harness.
• Reading every page of every conversation gave 102 pages and 8693 entries, with no refusals. Every page is on branch main. The slowest conversation took 798 ms (662 ms in revision 1, on files that have grown since; I did not isolate the difference). The largest file is 18.6 MB.
• 186 pages and cursors sampled from that run: 0 schema-invalid.
• Over HTTP, through a temporary board on port 0 with a temporary boardDir (the live board was not touched): 19 rows and 102 pages in 1487 ms. Rocko returns 422 unsupported-harness.
• Scripts and output: evidence/backend/smoke.mjs, smoke-stats.txt, http-real.mjs, http-out.txt.

4.4 Mutation testing

Mutations were run in scratch copies under /tmp/dewey-chat02/, never in the served tree. The route scratch is a full copy of the checkout (without .git, agents, v1 and skills/aws-*), so each mutant's failures are real test failures, not import errors. Scripts and full output: evidence/backend/mutate.py, mutate-board.py, mutate-backend.txt and mutate-routes.txt.

Reader: 38 of 39 caught.

Mutant	Caught by
no O_NOFOLLOW and no lstat symlink check	F7, F8
directory components unchecked	F7
listing follows symlinks	F7, F8
no prefix digest check	F4
no dev/ino check	F3
no shorter check	not caught, equivalent (below)
cwd always in project	F10
relative cwd accepted	F10
no character cap	F14, schema
no page byte cap	F14
no block cap	F14, schema
no foreign check	F6
no branch binding	F6
no expiry	F6
no parentSession notice	F11
branch parameter ignored	F12 two leaves, F12 second root, F1 missing parent
unknown branch served	F12 two leaves
registration harness ignored	F15
no placeholder roots	F15
malformed notices dropped	four F1 tests
bridge restored	F1 missing parent, F1 unreadable fork, F1 follow
missing-parent lines not named	F1 missing parent, F1 unreadable fork, F1 follow
trailing malformed notices only on the default branch	F1 missing parent, F1 follow
branch named by its leaf	all three F12 tests, two F1 tests
later child continues the branch	F12 two leaves
first root not main	all three F12 tests, two F1 tests
no empty main	F1 all malformed, unknown/empty
redacted thinking shown	mapping
EACCES on a component rethrown	seat directory at mode 000
incomplete never set	F2, F5
next re-reads fresh instead of pinned	F2, F5, F12, F1 follow
follow skips the history check	F12 duplicate id
follow skips the id digest	F12 duplicate id
follow reads the default branch	F12 two leaves, F1 follow
catalogue writes a file	F17 in six fixtures
wrong thinking visibility	mapping
raw tool ids	mapping, schema
id namespaces dropped	F1
surrogate pair split	fragments

Removed since revision 1: "no bridge" and "follow ignores branching", whose behaviour no longer exists.

The equivalent mutant: removing the "file is shorter" check changes nothing observable. A shorter file cannot supply the pinned length, so the prefix digest differs and source-replaced still follows. The check stays because it gives the refusal without hashing a truncated read.

Board routes: 11 of 11 caught. The mutants were: routes before the guard, no nosniff, every refusal 422, no query validation, unknown parameters allowed, reconcile dropped, cursor ignored, registrations ignored, a cursor without branch served, unavailable falling to 422, and a CORS header sent.

5. Known limits

• No openat in Node. A directory component swapped between the checks and the open is caught by the re-check after the open, not prevented. Nothing is read from a descriptor whose (dev, ino) does not match.
• The fstat mismatch branch of F8 is covered by design, not by a test. The F8 test swaps in a symlink, which O_NOFOLLOW refuses first. A plain-file swap between lstat and open needs a race the suite cannot schedule.
• An in-place rewrite with the same inode is detected by the digest of the pinned prefix, as the brief requires. Growth after the pin is not a change.
• Follow compares entry ids, not content, for the parts already served. The pinned byte prefix is still re-hashed, so a rewrite of served bytes is caught. An appended entry that reuses a served id is caught when it moves that entry off the branch or puts another id in its place (F12 duplicate id, both cases). If it keeps the same id in the same position with new content, follow does not notice: the parts already served keep the old text, while a fresh open shows the new one. Pi never writes duplicate ids.
• Cost. Each page reads and hashes the whole pinned prefix; §4.3 has the cost on real files.
• One actor. Cursors live in memory and are lost when the board restarts; the refusal is cursor-unknown with reconcile. local-operator is the only actor, which is not multi-actor safety (CHAT-04R).
• Claude history waits for B1 and CHAT-03 (D2).

6. After review

• Commit condition (Darkwing): serve.mjs imports ../../conversation/src/reader.mjs at load, so packages/conversation/ lands in the same commit as the routes or an earlier one, after Filbert's approval.
• Pushing goes through Sage with Jason's word. The live board needs a restart after the commit to serve the routes.
• The WebUI proxy allowlist for the two routes is Console work and is not in this packet.