feat(conversation): CHAT-02 read-only Pi history reader and two board routes (#1507)

packages/conversation is a library with no server: safe-fs, the Pi session
parser, CHAT-01 pages, pinned snapshots, cursors and follow. The control
board adds GET /api/conversations and /api/conversation behind the Host
and Origin guard. Both are read-only, their queries are validated, and
each refusal code maps to a status.

Dewey authored it (packet 0cf177b1, revision 2). Filbert reviewed the code:
R1 revise (branch ids moving on append, the assumed-link bridge merging
branches, one unreadable seat directory turning the catalogue into a 500),
then R2 approve (3b14d66c). Darkwing reviewed the routes: R1 approve
(07b10ad1), R2 approve (b9d92003). The package lands with the routes,
because serve.mjs imports the reader at load.

On an index export: the eight suites 24/90/43/17/14/15/63/18,
conversation and control-board 153/153, webui 9/9.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
2026-09-26 16:36:20 -05:00
co-authored by Claude Opus 5.5
parent c5db8c8819
commit a5beb6d97d
22 changed files with 3247 additions and 3 deletions
@@ -0,0 +1,3 @@
rows 19 researcher:available dewey:available sage:available darkwing:available filbert:available darkwing:available filbert:available filbert:available filbert:available filbert:available filbert:available darkwing:available filbert:available dewey:available darkwing:available darkwing:available darkwing:available darkwing:available rocko:unsupported
pages 102 ms 1487
unsupported 422 {"error":"this harness has no history reader yet","refusal":{"code":"unsupported-harness","reconcile":false}}
@@ -0,0 +1,20 @@
import { startServer } from "/mnt/storage/src/mosaic-stack/packages/control-board/src/serve.mjs";
import { discoverRepoAgents } from "/mnt/storage/src/mosaic-stack/packages/control-board/src/scan.mjs";
import { homedir } from "node:os";
import { mkdtempSync } from "node:fs";
const specs = discoverRepoAgents("/mnt/storage/src/mosaic-stack");
const server = await startServer({ host: "127.0.0.1", port: 0, specs, boardDir: mkdtempSync("/tmp/dewey-chat02/board-"), seatsDir: homedir() + "/.mosaic-dev/seats", isAlive: () => true, page: "x" });
const base = `http://127.0.0.1:${server.address().port}`;
const cat = await (await fetch(base + "/api/conversations")).json();
console.log("rows", cat.conversations.length, cat.conversations.map(c => `${c.seat}:${c.availability}`).join(" "));
const big = cat.conversations.filter(c => c.availability === "available");
let pages = 0, t = Date.now();
for (const c of big) {
let r = await (await fetch(`${base}/api/conversation?id=${c.conversation}`)).json();
pages++;
while (r.page?.hasMore) { r = await (await fetch(`${base}/api/conversation?id=${c.conversation}&branch=${r.page.branch}&cursor=${r.page.nextCursor}`)).json(); pages++; if (!r.ok) { console.log("refused", r); break; } }
}
console.log("pages", pages, "ms", Date.now() - t);
const u = cat.conversations.find(c => c.availability === "unsupported");
const ur = await fetch(`${base}/api/conversation?id=${u.conversation}`); console.log("unsupported", ur.status, JSON.stringify(await ur.json()));
server.close();
@@ -0,0 +1,39 @@
CAUGHT no O_NOFOLLOW + no lstat symlink -> F7: a symlinked file and a symlinked directory component are refused, ; F8: a file swapped for a symlink after the catalogue is refused
CAUGHT dir components unchecked -> F7: a symlinked file and a symlinked directory component are refused,
CAUGHT listing follows symlinks -> F7: a symlinked file and a symlinked directory component are refused, ; F8: a file swapped for a symlink after the catalogue is refused
CAUGHT no prefix digest check -> F4: a same-inode rewrite of the prefix refuses old cursors with reconc
CAUGHT no dev/ino check -> F3: a replaced file
MISSED no shorter check ->
CAUGHT cwd always in project -> F10: a header cwd naming another project is refused
CAUGHT relative cwd accepted -> F10: a header cwd naming another project is refused
CAUGHT no char cap -> F14: long strings split into fragments and parts, reassemble exactly, ; every page and cursor is a valid CHAT-01 record
CAUGHT no page byte cap -> F14: long strings split into fragments and parts, reassemble exactly,
CAUGHT no block cap -> F14: long strings split into fragments and parts, reassemble exactly, ; every page and cursor is a valid CHAT-01 record
CAUGHT no foreign check -> F6: unknown, foreign and expired cursors refuse and leave the cursor u
CAUGHT no branch binding -> F6: unknown, foreign and expired cursors refuse and leave the cursor u
CAUGHT no expiry -> F6: unknown, foreign and expired cursors refuse and leave the cursor u
CAUGHT no parentSession notice -> F11: parentSession renders with a marker and the parent is never opene
CAUGHT branch param ignored -> F12: a second root; F12: two leaves: the default leaf is shown and the other branch reads ; F1: a missing parent stops the history with a notice that names the un
CAUGHT unknown branch served -> F12: two leaves: the default leaf is shown and the other branch reads
CAUGHT registration harness ignored -> F15: a Claude seat is an unsupported-harness placeholder whose directo
CAUGHT no placeholder roots -> F15: a Claude seat is an unsupported-harness placeholder whose directo
CAUGHT malformed notices dropped -> F1: a file whose entries are all unreadable shows a notice per line; F1: a follow stays on its branch when the next entry's parent is unrea; F1: a malformed line is an unavailable part at its position, and readi; F1: a missing parent stops the history with a notice that names the un
CAUGHT bridge restored -> F1: a follow stays on its branch when the next entry's parent is unrea; F1: a missing parent stops the history with a notice that names the un; F1: an unreadable fork is never merged into another branch's history
CAUGHT missing-parent lines not named -> F1: a follow stays on its branch when the next entry's parent is unrea; F1: a missing parent stops the history with a notice that names the un; F1: an unreadable fork is never merged into another branch's history
CAUGHT trailing malformed only on default -> F1: a follow stays on its branch when the next entry's parent is unrea; F1: a missing parent stops the history with a notice that names the un
CAUGHT branch named by leaf -> F12: a follow refuses when an appended duplicate id changes the branch; F12: a second root; F12: two leaves: the default leaf is shown and the other branch reads ; F1: a follow stays on its branch when the next entry's parent is unrea; F1: a missing parent stops the history with a notice that names
CAUGHT later child continues branch -> F12: two leaves: the default leaf is shown and the other branch reads
CAUGHT first root not main -> F12: a follow refuses when an appended duplicate id changes the branch; F12: a second root; F12: two leaves: the default leaf is shown and the other branch reads ; F1: a follow stays on its branch when the next entry's parent is unrea; F1: a missing parent stops the history with a notice that names
CAUGHT no empty main -> F1: a file whose entries are all unreadable shows a notice per line; unknown conversations, empty files and non-Pi files refuse
CAUGHT redacted thinking shown -> native entries map to blocks: tools, thinking, bash, notices, ids that
CAUGHT EACCES on a component rethrown -> a seat directory without search permission refuses that root, not the
CAUGHT incomplete never set -> F2: a truncated trailing line marks the view incomplete, not an error; F5: growth between pages keeps the epoch and the page stops at the pin
CAUGHT next re-reads fresh -> F12: a second root; F12: two leaves: the default leaf is shown and the other branch reads ; F1: a follow stays on its branch when the next entry's parent is unrea; F2: a truncated trailing line marks the view incomplete, not an error; F5: growth between pages keeps the epoch and the page stops at th
CAUGHT follow skips history check -> F12: a follow refuses when an appended duplicate id changes the branch
CAUGHT follow skips the id digest -> F12: a follow refuses when an appended duplicate id changes the branch
CAUGHT follow reads the default branch -> F12: two leaves: the default leaf is shown and the other branch reads ; F1: a follow stays on its branch when the next entry's parent is unrea
CAUGHT catalogue writes a file -> F10: a header cwd naming another project is refused; F15: a Claude seat is an unsupported-harness placeholder whose directo; F1: a malformed line is an unavailable part at its position, and readi; F7: a symlinked file and a symlinked directory component are refused, ; F8: a file swapped for a symlin
CAUGHT thinking visibility wrong -> native entries map to blocks: tools, thinking, bash, notices, ids that
CAUGHT raw tool ids -> every page and cursor is a valid CHAT-01 record; native entries map to blocks: tools, thinking, bash, notices, ids that
CAUGHT id namespaces dropped -> F1: a malformed line is an unavailable part at its position, and readi
CAUGHT surrogate split -> fragments never cut a surrogate pair and keep an empty string
@@ -0,0 +1,30 @@
import subprocess, os
S="/tmp/dewey-chat02/scratch-board"
f=os.path.join(S,"packages/control-board/src/serve.mjs")
orig=open(f).read()
M=[
("conversation routes before the guard",[(""" const refused = foreignRequest(req);""",""" { const u0 = new URL(req.url, "http://localhost"); if (u0.pathname.startsWith("/api/conversation")) { let o; try { o = conversationResponse(reader, u0); } catch { o = { status: 500, body: {} }; } return sendConversationJson(res, o.status, o.body); } }
const refused = foreignRequest(req);""")]),
("no nosniff",[(', "x-content-type-options": "nosniff"','')]),
("all refusals 422",[("status: REFUSAL_STATUS[out.refusal.code] ?? 422","status: 422")]),
("no query validation",[(" if (values.length !== 1 || !QUERY_VALUE.test(values[0])) return { error: `invalid ${key}` };\n","")]),
("unknown params allowed",[(""" if (!["id", "branch", "cursor"].includes(key)) return { error: `unknown parameter: ${key}` };\n""","")]),
("reconcile dropped",[("refusal: { code: out.refusal.code, reconcile: out.refusal.reconcile }","refusal: { code: out.refusal.code }")]),
("cursor ignored",[(" const out = query.cursor\n"," const out = false\n")]),
("registrations ignored",[("rootsFromSpecs(specs, loadRegistrations(seatsDir).registrations)","rootsFromSpecs(specs, [])")]),
("cursor without branch served",[(" if (out.cursor && !out.branch) return"," if (false) return")]),
("unavailable falls to 422",[(" unavailable: 404,\n","")]),
("CORS header sent",[('"x-content-type-options": "nosniff" });','"x-content-type-options": "nosniff", "access-control-allow-origin": "*" });')]),
]
for name,reps in M:
src=orig
ok=True
for a,b in reps:
if a not in src: print("NOT APPLIED",name); ok=False; break
src=src.replace(a,b,1)
if not ok: continue
open(f,"w").write(src)
r=subprocess.run(["node","--test","--test-concurrency=1","tests/serve.test.mjs"],cwd=os.path.join(S,"packages/control-board"),capture_output=True,text=True,timeout=600)
fails=sorted(set(l.strip()[2:].split(" (")[0][:60] for l in r.stdout.splitlines() if l.lstrip().startswith("✖") and "failing tests" not in l))
print(("CAUGHT " if r.returncode else "MISSED ")+name+" -> "+"; ".join(fails))
open(f,"w").write(orig)
@@ -0,0 +1,11 @@
CAUGHT conversation routes before the guard -> conversation routes
CAUGHT no nosniff -> conversation routes: catalogue, first page, next page and fo
CAUGHT all refusals 422 -> conversation routes: catalogue, first page, next page and fo
CAUGHT no query validation -> conversation routes: catalogue, first page, next page and fo
CAUGHT unknown params allowed -> conversation routes: catalogue, first page, next page and fo
CAUGHT reconcile dropped -> conversation routes: catalogue, first page, next page and fo
CAUGHT cursor ignored -> conversation routes: catalogue, first page, next page and fo
CAUGHT registrations ignored -> conversation routes: catalogue, first page, next page and fo
CAUGHT cursor without branch served -> conversation routes: catalogue, first page, next page and fo
CAUGHT unavailable falls to 422 -> every refusal code the reader can raise has an HTTP status
CAUGHT CORS header sent -> conversation routes: catalogue, first page, next page and fo
@@ -0,0 +1,59 @@
import subprocess, sys, shutil, os
S="/tmp/dewey-chat02/scratch/packages/conversation"
M=[
("no O_NOFOLLOW + no lstat symlink","src/safe-fs.mjs",[("constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK","constants.O_RDONLY"),(' if (before.isSymbolicLink()) throw new Refusal("unsafe-path", "session file is a symlink");\n',''),(' if (!before.isFile()) throw',' if (false) throw'),("if (!st.isFile() || st.dev !== before.dev","if (st.dev !== before.dev")]),
("dir components unchecked","src/safe-fs.mjs",[(' if (st.isSymbolicLink()) throw new Refusal("unsafe-path", "session root contains a symlink");\n',''),(' if (!st.isDirectory()) throw',' if (false) throw')]),
("listing follows symlinks","src/safe-fs.mjs",[(' else if (dirent.isSymbolicLink()) refused.push({ name: dirent.name, code: "unsafe-path" });\n','')]),
("no prefix digest check","src/reader.mjs",[('if (pinned.digest !== state.digest) throw','if (false) throw')]),
("no dev/ino check","src/reader.mjs",[('if (pinned.file.dev !== state.dev || pinned.file.ino !== state.ino) throw','if (false) throw'),('if (fresh.file.dev !== state.dev || fresh.file.ino !== state.ino || fresh.length < state.length) throw','if (fresh.length < state.length) throw')]),
("no shorter check","src/reader.mjs",[('if (pinned.shorter) throw','if (false) throw')]),
("cwd always in project","src/reader.mjs",[(' if (typeof cwd !== "string" || !isAbsolute(cwd)) return false;',' return true;')]),
("relative cwd accepted","src/reader.mjs",[(' if (typeof cwd !== "string" || !isAbsolute(cwd)) return false;',' if (typeof cwd !== "string" || !cwd) return false;')]),
("no char cap","src/parts.mjs",[("chars + 1 > LIMITS.chars ||","false ||")]),
("no page byte cap","src/parts.mjs",[("if (bytes + add > LIMITS.pageBytes) break;","")]),
("no block cap","src/parts.mjs",[("current.length === LIMITS.blocks ||","false ||")]),
("no foreign check","src/reader.mjs",[("if (actor !== record.actor || purpose !== record.purpose || conversation !== record.conversation || branch !== record.branch) {","if (false) {")]),
("no branch binding","src/reader.mjs",[("|| branch !== record.branch) {",") {")]),
("no expiry","src/reader.mjs",[("if (Date.parse(record.expiresAt) <= now()) {","if (false) {")]),
("no parentSession notice","src/pi.mjs",[(" if (parsed.header.parentSession !== undefined && parsed.header.parentSession !== null) {\n units.push"," if (false) {\n units.push")]),
("branch param ignored","src/reader.mjs",[("const built = build(snap, found.root, branch ?? null, conversation);","const built = build(snap, found.root, null, conversation);")]),
("unknown branch served","src/reader.mjs",[("if (!parsed.branches.has(branch)) return null;","if (false) return null;")]),
("registration harness ignored","src/reader.mjs",[("unsupportedReason: harness !== null && harness !== HISTORY ? UNSUPPORTED_HARNESS : null,","unsupportedReason: null,")]),
("no placeholder roots","src/reader.mjs",[(" if (!reg || reg.layout !== \"repo\""," if (true || reg.layout !== \"repo\"")]),
("malformed notices dropped","src/pi.mjs",[('out.push({ notice: "malformed", lines: [pending[mi++].line] });','mi++;')]),
("bridge restored","src/pi.mjs",[(' path.push({ notice: "missing-parent", lines: lost });\n break;',' const prev = r.index > 0 ? parsed.byId.get(parsed.entries[r.index - 1].entry.id) : null;\n if (lost.length && prev) { path.push({ notice: "missing-parent", lines: lost }); r = prev; continue; }\n path.push({ notice: "missing-parent", lines: lost });\n break;')]),
("missing-parent lines not named","src/pi.mjs",[("const lost = parsed.malformed.filter((m) => m.after === r.index - 1).map((m) => m.line);","const lost = [];")]),
("trailing malformed only on default","src/pi.mjs",[(" flushBefore(Infinity);\n return out;"," if (leaf === parsed.defaultLeaf) flushBefore(Infinity);\n return out;")]),
("branch named by leaf","src/pi.mjs",[(" const branchOf = (leaf) => {"," const branchOf = (leaf) => branchName(leaf);\n const unused = (leaf) => {")]),
("later child continues branch","src/pi.mjs",[("if (children.get(p.entry.id)[0] !== r) return branchName(r);","if (children.get(p.entry.id).at(-1) !== r) return branchName(r);")]),
("first root not main","src/pi.mjs",[("return r === firstRoot ? MAIN : branchName(r);","return branchName(r);")]),
("no empty main","src/pi.mjs",[(" if (!branches.size) branches.set(MAIN, null);\n","")]),
("redacted thinking shown","src/pi.mjs",[('const t = b.redacted === true ? "" : typeof b.thinking','const t = typeof b.thinking')]),
("EACCES on a component rethrown","src/safe-fs.mjs",[('throw denied(err, "a session path component");','throw err;')]),
("incomplete never set","src/reader.mjs",[("incomplete: buf.length > length };","incomplete: false };")]),
("next re-reads fresh","src/reader.mjs",[("const pinned = snapshot(state.root, state.name, state.length);","const pinned = { ...snapshot(state.root, state.name), length: state.length };")]),
("follow skips history check","src/reader.mjs",[("if (!built || built.entries.length < state.offset || idsDigest(built.entries, state.offset) !== state.prefix) {","if (!built) {")]),
("follow skips the id digest","src/reader.mjs",[("|| idsDigest(built.entries, state.offset) !== state.prefix) {",") {")]),
("follow reads the default branch","src/reader.mjs",[("const built = build(fresh, state.root, state.branch, conversation);","const built = build(fresh, state.root, null, conversation);")]),
("catalogue writes a file","src/reader.mjs",[(" function catalogue() {"," function catalogue() {\n for (const r of listRoots()) { try { require_(r); } catch {} }")]),
("thinking visibility wrong","src/pi.mjs",[('visibility: t ? "permitted-visible" : "unavailable"','visibility: "permitted-visible"')]),
("raw tool ids","src/pi.mjs",[("call: safeId(b.id), name: safeId(b.name)","call: String(b.id), name: String(b.name)")]),
("id namespaces dropped","src/pi.mjs",[("id: `n.${e.id}`","id: e.id")]),
("surrogate split","src/parts.mjs",[("const width = cp > 0xffff ? 2 : 1;","const width = 1;")]),
]
orig={}
for f in ["src/safe-fs.mjs","src/reader.mjs","src/parts.mjs","src/pi.mjs"]:
orig[f]=open(os.path.join(S,f)).read()
for name,f,reps in M:
src=orig[f]
for a,b in reps:
if a not in src: print("NOT APPLIED",name,repr(a[:60])); break
src=src.replace(a,b)
else:
if name=="catalogue writes a file":
src=src.replace("require_(r);",'(await_import => 0)(); (require_fs => 0)(); import_fs.writeFileSync(r.dir + "/x.tmp", "")').replace('import { createHash, randomBytes } from "node:crypto";','import { createHash, randomBytes } from "node:crypto";\nimport * as import_fs from "node:fs";')
open(os.path.join(S,f),"w").write(src)
r=subprocess.run(["node","--test","tests/"],cwd=S,capture_output=True,text=True,timeout=600)
fails=sorted(set(l.strip()[2:].split(" (")[0] for l in r.stdout.splitlines() if l.lstrip().startswith("✖") and "failing tests" not in l))
print(("CAUGHT " if r.returncode else "MISSED ")+name+" -> "+"; ".join(x[:70] for x in fails)[:300])
open(os.path.join(S,f),"w").write(orig[f])
@@ -0,0 +1,2 @@
{ convs: 18, pages: 102, entries: 8693, refusals: {}, maxMs: 798 }
catalogue 19 rows in 75 ms, refusedRoots []; 186 sampled pages and cursors: 0 schema-invalid; every page branch: main (102)
@@ -0,0 +1,37 @@
import { discoverRepoAgents } from "/mnt/storage/src/mosaic-stack/packages/control-board/src/scan.mjs";
import { rootsFromSpecs, createReader } from "/mnt/storage/src/mosaic-stack/packages/conversation/src/reader.mjs";
import { readRegistration, LAYOUTS } from "/mnt/storage/src/mosaic-stack/packages/seat/src/seat.mjs";
import { readdirSync, writeFileSync } from "node:fs";
import { homedir } from "node:os";
const seats = homedir() + "/.mosaic-dev/seats";
const regs = [];
import { loadRegistrations } from "/mnt/storage/src/mosaic-stack/packages/control-board/src/scan.mjs"; regs.push(...loadRegistrations(seats).registrations);
const roots = rootsFromSpecs(discoverRepoAgents("/mnt/storage/src/mosaic-stack"), regs);
console.log(roots.map(r => `${r.seat} ${r.harness} ${r.unsupportedReason} ${r.engineStartedAt}`).join("\n"));
const reader = createReader({ roots });
let t = Date.now();
const cat = reader.catalogue();
console.log("catalogue", cat.conversations.length, "ms", Date.now() - t, "refusedRoots", JSON.stringify(cat.refusedRoots));
const byAvail = {}; for (const c of cat.conversations) byAvail[c.availability + ":" + c.refusal] = (byAvail[c.availability + ":" + c.refusal] ?? 0) + 1;
console.log(byAvail);
console.log(cat.conversations.slice(0, 3));
const pages = [];
let stats = { convs: 0, pages: 0, entries: 0, refusals: {} , maxMs: 0};
for (const c of cat.conversations.filter(c => c.availability === "available")) {
t = Date.now();
let r = reader.open({ conversation: c.conversation });
if (!r.ok) { stats.refusals[r.refusal.code] = (stats.refusals[r.refusal.code] ?? 0) + 1; continue; }
stats.convs++;
let n = 0;
while (true) {
stats.pages++; stats.entries += r.page.entries.length; n++;
if (pages.length < 400) pages.push(r.page);
if (r.cursor) pages.length < 400 && pages.push(r.cursor);
if (!r.page.hasMore) break;
r = reader.next({ cursor: r.page.nextCursor, conversation: c.conversation, branch: r.page.branch });
if (!r.ok) { console.log("next refused", r.refusal); break; }
}
stats.maxMs = Math.max(stats.maxMs, Date.now() - t);
}
console.log(stats);
writeFileSync("/tmp/dewey-chat02/pages.json", JSON.stringify(pages));