jason.woltjeandClaude Opus 5.5
a5beb6d97d
packages/conversation is a library with no server: safe-fs, the Pi session parser, CHAT-01 pages, pinned snapshots, cursors and follow. The control board adds GET /api/conversations and /api/conversation behind the Host and Origin guard. Both are read-only, their queries are validated, and each refusal code maps to a status. Dewey authored it (packet 0cf177b1, revision 2). Filbert reviewed the code: R1 revise (branch ids moving on append, the assumed-link bridge merging branches, one unreadable seat directory turning the catalogue into a 500), then R2 approve (3b14d66c). Darkwing reviewed the routes: R1 approve (07b10ad1), R2 approve (b9d92003). The package lands with the routes, because serve.mjs imports the reader at load. On an index export: the eight suites 24/90/43/17/14/15/63/18, conversation and control-board 153/153, webui 9/9. Co-Authored-By: Claude Opus 5.5 <[email protected]>
219 lines
11 KiB
Markdown
219 lines
11 KiB
Markdown
# CHAT-02 backend: review packet (#1507, row 5)
|
||
|
||
Author: Dewey, 2026-09-26. Brief: `BRIEF.md` R4 (`636b0fac…`), §2.1 and D3.
|
||
Base: `34777c56` on `refactor`. Nothing here is committed; the candidate is
|
||
the working tree, pinned by the hashes below.
|
||
|
||
Reviewers (Sage's order):
|
||
- Filbert: the code, all nine files.
|
||
- Darkwing: the two board routes, `serve.mjs` and `serve.test.mjs`.
|
||
|
||
The Console (`packages/webui`) is the next step and is not in this packet.
|
||
|
||
## 1. Candidate hashes
|
||
|
||
```
|
||
f9008c01c608ea9aacdd15459f03a4ca8f8b4fe4d5225f3f8337bc5f9282f955 packages/conversation/package.json
|
||
6f2cab5a31a26f71cf1cc00d6aed2a683e011b6cc2b52ebd4bb50b404af96d5c packages/conversation/README.md
|
||
da336ed2a2a149ff56ac12af2440a8353d4573c33e9069381be60f1a0ca70151 packages/conversation/src/parts.mjs
|
||
28e22501ab133af7dee1a38f0dc209d005ead4192b64dae6ef999ec4191db8e0 packages/conversation/src/pi.mjs
|
||
1e1db046c65292dbcfa2d59b6f0d5009dc48062cbd6a7ec123a18b6707ec8142 packages/conversation/src/reader.mjs
|
||
98835b171c1473de2ba242347b191773dfac1e50f383e43ec43b2ef4a5122a3a packages/conversation/src/safe-fs.mjs
|
||
3f89f5e7a3cdd5bb31623c799105c538fd5b5eacae368af0170e43f1130267e6 packages/conversation/tests/reader.test.mjs
|
||
afc95bdb850e4b533516984d2dd7f3ff852c90b46b384f42bc550adf9f2c540d packages/control-board/src/serve.mjs
|
||
e60aa14b77350a2807e9e9b887acb9daec1c0468d94baaf9f77548dc54fcecbc packages/control-board/tests/serve.test.mjs
|
||
```
|
||
|
||
The first seven files are new. `serve.mjs` and `serve.test.mjs` are diffs
|
||
against `34777c56`: `git diff 34777c56 -- packages/control-board`.
|
||
`scan.mjs` is unchanged. D3 allowed edits there, but none were needed.
|
||
|
||
## 2. What it does
|
||
|
||
`packages/conversation/README.md` is the reference: API, sources, safe open,
|
||
parser rules, page limits, snapshot, epoch and cursor rules, follow, the
|
||
refusal table and costs. In short:
|
||
|
||
- `rootsFromSpecs` turns the board's repository specs into roots, using
|
||
registrations only as hints. `createReader` serves `catalogue`, `open` and
|
||
`next`.
|
||
- Pages and cursors are CHAT-01 `page` and `cursor` records. Everything the
|
||
Console needs beyond CHAT-01 goes in `view`.
|
||
- The board serves `GET /api/conversations` (no parameters) and
|
||
`GET /api/conversation?id=&branch=&cursor=`. Both run after the existing
|
||
Host/Origin guard (`d1629d61`) and its GET/HEAD check.
|
||
- Responses are `application/json` with `no-store` and `nosniff`, and no CORS
|
||
headers.
|
||
- Status map:
|
||
- 404 for an unknown conversation or branch;
|
||
- 409 for cursor refusals, `source-replaced` and `incomplete-header`;
|
||
- 403 for `unsafe-path`, `foreign-project` and `unreadable`;
|
||
- 422 for the rest;
|
||
- 400 for any query parameter other than `id`, `branch` or `cursor`, a
|
||
repeated one, or a value that fails the id pattern;
|
||
- 500 with a fixed message for an exception, with details only on stderr.
|
||
|
||
## 3. Choices and deviations to review
|
||
|
||
1. **Catalogue rows are a summary shape, not CHAT-01 `catalogueItem`.** The
|
||
row is: conversation, seat, project, harness, history, title, readOnly,
|
||
`controlMode: "unavailable"`, the three separate time fields,
|
||
availability, `unsupportedReason` and refusal. `catalogueItem` carries
|
||
binding and control fields that belong to CHAT-03. I did not want to fill
|
||
them with placeholders that look authoritative.
|
||
2. **Registrations match on `sessionsDir`, not `sessionFile`.** Registrations
|
||
have no `sessionFile` field. The brief §2.1 rule is applied to the
|
||
directory instead: seat, layout `repo`, project and `samePath(sessionsDir)`
|
||
must all match. A registration never adds a root or names a file (F9).
|
||
3. **Placeholder rows for non-Pi seats.** A seat on another harness has no Pi
|
||
directory, so the board has no spec for it. A registration adds one
|
||
`unsupported-harness` row when its `sessionsDir` is the standard
|
||
directory under a project root that is already approved. Its directory is
|
||
never read. Rocko (`claude-code`) is the live case (F15).
|
||
4. **An epoch id is comparable only within one cursor chain.**
|
||
`sourceEpoch = "e-" + sha256(dev:ino:digest)`, taken at open and carried
|
||
forward while the prefix verifies. Two opens of an unchanged file give the
|
||
same value. After growth, a new open gets a new value for the same epoch,
|
||
while the old chain keeps its own. Only a new inode, a shorter file or a
|
||
changed prefix is a new epoch, and that is refused.
|
||
5. **Follow semantics.** On the last page, `follow` replaces `cursor`.
|
||
- `next` with it re-verifies the pinned prefix and takes a fresh snapshot.
|
||
- If the view's leaf is still on the new default branch, the ids of the
|
||
parts already served must be unchanged. It then returns only the new
|
||
parts.
|
||
- If the conversation went on down another branch, it returns an empty
|
||
page on the old branch and reports the new default in `view`.
|
||
- Silently switching branches would break "nothing switches silently".
|
||
6. **Missing parent next to malformed lines is bridged.** The history
|
||
continues from the previous valid entry, with an "assumed link" notice.
|
||
Pi itself would drop the history before that point. Without adjacent
|
||
malformed lines, the history stops with a notice. A follow refuses if a
|
||
line that was malformed now reads differently, which changes meaning
|
||
(F1 follow test).
|
||
7. **`unreadable` refusal.** `EACCES` or `EPERM` on a root listing or file
|
||
open becomes a per-row 403, so one unreadable file cannot fail the whole
|
||
catalogue.
|
||
8. **256 MiB file cap** (`too-large`, 422). The largest real file today is
|
||
18.6 MB.
|
||
9. **Every page re-hashes the whole pinned prefix.** This makes detection
|
||
simple and total, at a cost measured in §4.3.
|
||
10. **Refusal code names** not already in CHAT-01 are CHAT-02 values, like
|
||
`unsupported-harness` (D2): `unsafe-path`, `foreign-project`,
|
||
`unreadable`, `not-a-pi-session`, `incomplete-header`, `too-large`,
|
||
`unknown-branch`, `unknown-actor` and `unsupported-purpose`.
|
||
|
||
## 4. Evidence
|
||
|
||
All runs are on the candidate hashes above.
|
||
|
||
### 4.1 Suites and contract checks
|
||
|
||
- `node --test --test-concurrency=1 packages/conversation/tests/
|
||
packages/control-board/tests/ packages/webui/tests/ packages/seat/tests/`:
|
||
175 tests, 175 pass.
|
||
- `node docs/plans/chat-00/check.mjs`, `chat-01/check.mjs` and
|
||
`chat-01c/check.mjs` all exit 0.
|
||
|
||
### 4.2 Fixture map (brief §2.1)
|
||
|
||
| # | Test |
|
||
|---|---|
|
||
| F1 | three tests in `reader.test.mjs`: malformed notice in place (with a hostile id), bridge and missing parent, and the follow that refuses when history changes meaning |
|
||
| F2–F15 | one named test each in `reader.test.mjs` |
|
||
| F16 | `serve.test.mjs` §12: foreign Host and cross-origin Origin on both routes give 403; a spy reader records zero calls, and no `access-control-*` header is sent |
|
||
| F17 | every reader call in `reader.test.mjs` runs inside a fingerprint of the whole fixture tree: size, SHA-256, mtime (ns), (dev, ino), mode and every directory listing. The route test in `serve.test.mjs` does the same. Files no read may touch are mode 000. |
|
||
|
||
Additional coverage: mapping of every Pi entry type and role, pagination
|
||
at 100 parts and at the byte cap, surrogate-safe fragments, unknown and
|
||
empty and non-Pi files, `unreadable`, and CHAT-01 schema validation of
|
||
every page and cursor the tests produce (Python `jsonschema`).
|
||
|
||
### 4.3 Real data (read-only, this checkout)
|
||
|
||
- Catalogue: 19 rows. There are 18 available Pi conversations across darkwing,
|
||
dewey, filbert, researcher and sage, plus rocko as `unsupported-harness`.
|
||
- Reading every page of every conversation gave 102 pages and 8693 entries,
|
||
with no refusals. The slowest conversation took 662 ms; the largest file is
|
||
18.6 MB.
|
||
- 186 pages and cursors sampled from that run: 0 schema-invalid.
|
||
- Over HTTP, through a temporary board on port 0 with a temporary `boardDir`
|
||
(the live board was not touched): 19 rows and 102 pages in 1517 ms. Rocko
|
||
returns 422 `unsupported-harness`.
|
||
- Scripts and output: `evidence/backend/smoke.mjs`, `smoke-stats.txt`,
|
||
`http-real.mjs`, `http-out.txt`.
|
||
|
||
### 4.4 Mutation testing
|
||
|
||
Mutations were run in scratch copies under `/tmp/dewey-chat02/`, never in
|
||
the served tree. Scripts and full output: `evidence/backend/mutate.py`,
|
||
`mutate-board.py`, `mutate-backend.txt` and `mutate-routes.txt`.
|
||
|
||
Reader: 27 of 28 caught.
|
||
|
||
| Mutant | Caught by |
|
||
|---|---|
|
||
| no O_NOFOLLOW and no lstat symlink check | F7, F8 |
|
||
| directory components unchecked | F7 |
|
||
| listing follows symlinks | F7, F8 |
|
||
| no prefix digest check | F4 |
|
||
| no dev/ino check | F3 |
|
||
| no shorter check | **not caught, equivalent** (below) |
|
||
| cwd always in project | F10 |
|
||
| no character cap | F14, schema |
|
||
| no page byte cap | F14 |
|
||
| no block cap | F14, schema |
|
||
| no foreign check | F6 |
|
||
| no branch binding | F6 |
|
||
| no expiry | F6 |
|
||
| no parentSession notice | F11 |
|
||
| branch parameter ignored | F12 |
|
||
| registration harness ignored | F15 |
|
||
| no placeholder roots | F15 |
|
||
| malformed notices dropped | F1 |
|
||
| no bridge | F1 |
|
||
| `incomplete` never set | F2, F5 |
|
||
| `next` re-reads fresh instead of pinned | F2, F5, F12, unknown/empty |
|
||
| follow skips history check | F1 follow |
|
||
| follow ignores branching | F12 |
|
||
| catalogue writes a file | F17 in six fixtures |
|
||
| wrong thinking visibility | mapping |
|
||
| raw tool ids | mapping, schema |
|
||
| id namespaces dropped | F1 |
|
||
| surrogate pair split | fragments |
|
||
|
||
The equivalent mutant: removing the "file is shorter" check changes nothing
|
||
observable. A shorter file cannot supply the pinned length, so the prefix
|
||
digest differs and `source-replaced` still follows. The check stays because
|
||
it gives the refusal without hashing a truncated read.
|
||
|
||
Board routes: 9 of 9 caught. The mutants were: routes before the guard, no
|
||
`nosniff`, every refusal 422, no query validation, unknown parameters
|
||
allowed, `reconcile` dropped, cursor ignored, registrations ignored, and a
|
||
CORS header sent.
|
||
|
||
## 5. Known limits
|
||
|
||
- **No `openat` in Node.** A directory component swapped between the checks
|
||
and the open is caught by the re-check after the open, not prevented.
|
||
Nothing is read from a descriptor whose (dev, ino) does not match.
|
||
- **The `fstat` mismatch branch of F8** is covered by design, not by a
|
||
test. The F8 test swaps in a symlink, which `O_NOFOLLOW` refuses first. A
|
||
plain-file swap between `lstat` and `open` needs a race the suite cannot
|
||
schedule.
|
||
- **An in-place rewrite with the same inode** is detected by the digest of
|
||
the pinned prefix, as the brief requires. Growth after the pin is not a
|
||
change.
|
||
- **Cost.** Each page reads and hashes the whole pinned prefix; §4.3 has the
|
||
cost on real files.
|
||
- **One actor.** Cursors live in memory and are lost when the board restarts;
|
||
the refusal is `cursor-unknown` with reconcile. `local-operator` is the
|
||
only actor, which is not multi-actor safety (CHAT-04R).
|
||
- **Claude history** waits for B1 and CHAT-03 (D2).
|
||
|
||
## 6. After review
|
||
|
||
- Pushing goes through Sage with Jason's word. The live board needs a restart
|
||
after the commit to serve the routes.
|
||
- The WebUI proxy allowlist for the two routes is Console work and is not in
|
||
this packet.
|