diff --git a/BUILD-LOG.md b/BUILD-LOG.md index ddf2da58..12afe89d 100644 --- a/BUILD-LOG.md +++ b/BUILD-LOG.md @@ -3120,3 +3120,40 @@ names jason and carmen. Receipt (0600): `restart-row25-approvers-20260926T213021Z.txt`. Jason reruns the approval check. DEC-009 needs a `record_update` to names, or a new decision, and then a new approval request. + +## 2026-09-26: Gate F build, the ledger's T3 source (#1506, Darkwing, for Filbert's review) + +Before: Table 2 read only `.pi/state//sessions/*.jsonl`. Seats that run +in T3 never reach a Pi log, so their Human column showed "no source" as zero. +The class pattern in both preambles accepted lowercase only. + +After (uncommitted candidate on 1c5f6bc3, per brief R3 f3c05c1b): new +`packages/ledger/src/t3.mjs` reads `~/.t3/userdata/state.sqlite` read-only in +one transaction. It maps threads to seats by title, cross-checks headers +addressed to their own thread, and adds a `t3:unmapped` row. It refuses on a +missing or unreadable database, symlinks, schema drift, no single project for +the root, bad rows or a header conflict. `--no-t3` and `--t3-db PATH` are new. +Classes now match in either case. Pins: +`agents/darkwing/work/ledger-t3-source/build-manifest.sha256`; record `build.md` +in the same directory. + +Beyond the brief: the live read refused on HEAD too, at a Filbert Pi log line +holding a raw U+2028. Node 26.8.1's `readline` ends lines there. The Pi reader +now splits on `\n` only, with a test. + +Evidence: no suite runs the ledger tests. `node --test +packages/ledger/tests/ledger.test.mjs` passes 44/44 (47/47 with the gitea +helper file). The eight suites passed on a clone of 1c5f6bc3 with the five +files (24/90/43/17/14/15/63/18); the first task run showed one failure I didn't +capture, and three reruns passed. Union 434/434 three times. Live read exit 0, +no conflict, diagnostic 14. Two mutations don't bite (`mode=ro` removed; +snapshot isolation untested), recorded in `build.md`. Not committed or pushed. + +## 2026-09-26: Gate F build review (#1506, Filbert) + +Filbert approved manifest ba73a163 and, as its own item, the Pi reader's +newline-only splitter. Review: `agents/filbert/work/ledger-t3-build-review-2026-09-26.md` +(e47ec6da). Correction to the entry above: line 611 holds both a U+2028 and a +U+2029, and `readline` splits at each. The splitter covers both. His four +nonblocking notes are listed in `agents/darkwing/work/ledger-t3-source/build.md` +for a follow-up after the commit. diff --git a/agents/darkwing/work/ledger-t3-source/build-manifest.sha256 b/agents/darkwing/work/ledger-t3-source/build-manifest.sha256 new file mode 100644 index 00000000..5fbd88ca --- /dev/null +++ b/agents/darkwing/work/ledger-t3-source/build-manifest.sha256 @@ -0,0 +1,5 @@ +0afb0320e9a1833f133426169c389ffb71be3d490e0e402070162aa22e820296 packages/ledger/src/ledger.mjs +d6092a538a059e8869544df903e89a8b9a4c3b6b492645db762b2f03d5630a44 packages/ledger/src/cli.mjs +dfb092aabee5cf5197029c6e8978df570ee50f08d84babde931c6a763befafe9 packages/ledger/src/t3.mjs +7444abd1dbd8e637705def0fd98105e6e69970f1bd397a11f8277996521579d6 packages/ledger/tests/ledger.test.mjs +27f7366dd5edc30a93a8c54bfb46b3fed87e1a44f11e1b22bd159a0718625273 packages/ledger/README.md diff --git a/agents/darkwing/work/ledger-t3-source/build.md b/agents/darkwing/work/ledger-t3-source/build.md new file mode 100644 index 00000000..e840ff70 --- /dev/null +++ b/agents/darkwing/work/ledger-t3-source/build.md @@ -0,0 +1,161 @@ +# Gate F build: the ledger's T3 source (#1506), candidate for review + +Darkwing built this on 2026-09-26 from the approved brief R3, +`docs/plans/2026-09-26_ledger-t3-source.md` (sha256 f3c05c1b, committed in +ffc22c04). Sage gave the go once Filbert confirmed R3. Filbert reviews the +code; Sage commits after the suites. Base is HEAD 1c5f6bc3. Nothing is +committed or pushed. + +## Files + +`build-manifest.sha256` pins the five files, and `build.patch` is the diff +against 1c5f6bc3 with `t3.mjs` included as a new file. + +- `packages/ledger/src/t3.mjs` (new). `readT3(root, range, {dbPath, isDefault})`: + path checks, one read transaction, schema check, project, title mapping, + header cross-check, counts, mentions, diagnostic. +- `packages/ledger/src/ledger.mjs`. The class fix, `t3Header()`, + `readSeats()`, `mergeSources()`, the `pi` and `t3` keys in the report, the + text line for `--no-t3` or a non-default path, and the U+2028 fix below. +- `packages/ledger/src/cli.mjs`. `--no-t3` and `--t3-db PATH`, which refuse + each other; the usage line. +- `packages/ledger/tests/ledger.test.mjs`. HOME at both spawn sites, the + empty default database, 25 new tests. +- `packages/ledger/README.md`. A new "T3 source" section. + +The commit should also carry Filbert's updated review, +`agents/filbert/work/ledger-t3-source-review-2026-09-26.md` (be1aa414), and +this directory's new files. + +## Beyond the brief: the Pi reader split valid lines + +The brief's live read has to exit 0. It didn't, and T3 wasn't the cause. HEAD +refuses the live checkout the same way: +`Malformed session JSON: filbert/2026-09-12T16-38-58-597Z_01a0967c-….jsonl:611`. +That line parses. It holds a raw U+2028 inside a JSON string, which JSON +allows and `JSON.stringify` writes unescaped. Node 26.8.1's `readline` ends a +line at U+2028 too, so it cut the record in two (733 lines by `readline`, 732 +by `\n`). The reader parses every line before it checks the range, so on +Node 26.8.1 every live run refuses, whatever the dates. The file was last +written 2026-09-14. I haven't checked which Node version first split there. + +The fix replaces `readline` with a small splitter that ends lines at `\n` +only. It sits in `ledger.mjs`, which this build already changes, and it +blocked acceptance, so I made it here instead of filing it. A new test writes a +Pi log with a raw U+2028 and CRLF endings; it fails with `readline` and passes +with the splitter. Please review it as its own item. + +## Choices the brief left open + +- Imported threads are excluded by the `import:` prefix alone. Live, all + 1678 `historyImport` events sit in `import:` streams, so the two rules agree + today. The events table stays optional, so the exclusion doesn't depend on it. +- The header cross-check runs over every user message in a counted thread, in + range or not. The title mapping is current state, so a conflict in old + history still misassigns counts for any range that includes it. +- Validation (role, text, `created_at`) also covers every message in a + counted thread, assistant rows included, and not only rows in range. +- The diagnostic is in range: `humanSentThroughApi` and `humanWithoutEvent`. + One unparseable event, or an event with no string `messageId`, makes both + `unknown`, the same as a missing table (F5). +- Project and thread matching compare `workspace_root` in JavaScript, so a + declared collation on the column can't loosen byte-for-byte equality. +- JSON adds top-level `pi` (Pi rows) and `t3` (read flag, database, seats with + threads, unmapped, excluded, diagnostic). `seats` and `totals` keep their + shape, so existing consumers and tests are unchanged. `t3.seats` lists a + seat whenever it has a mapped thread, even with zero counts in range. +- The unmapped row comes last in `seats`, and only when it has counts. + +## Evidence + +- Ledger tests: `node --test packages/ledger/tests/`, 47/47 (ledger 44, + of which 25 are new, and gitea helper 3). The busy-timeout test takes about 5.4 s. +- Class fix against HEAD. HEAD's `messageKind` (from `git show + 1c5f6bc3:packages/ledger/src/ledger.mjs`) calls a T3 header with + `class=REVIEW-REQUEST`, a tmux preamble with `class=DECISION` and a T3 header + with `class=Actionable` all human. The build calls them agent. The existing + test asserting `class=Actionable` is human now asserts agent. +- Mutations, each on a scratch copy of the package. Three `gitea-helper` + tests fail in every scratch copy because they need the repository's + `scripts/`, so the counts below leave them out. + - Classes back to `[a-z-]+`: 6 fail. + - No header cross-check: 2 fail. + - No symlink refusal: 4 fail. + - Busy timeout 0: 1 fails. + - Two projects allowed: 1 fails. + - Deleted threads kept, imported threads kept, or range filter removed: + 4 fail each. + - No role check: 1 fails. + - No diagnostic table check: 1 fails. + - `readline` restored: 1 fails. +- Two mutations pass, and I'm naming them rather than hiding them: + - Removing `mode=ro` changes nothing, because `readOnly: true` already + opens read-only. Both stay, as the brief says. + - Removing `BEGIN` fails 19 tests, but only because `COMMIT` then has no + transaction. No test proves that the queries share one snapshot. +- Eight suites on a local clone of 1c5f6bc3 with the five files: config 24, + task 90, foundation 43, conductor 17, release 14, auth 15, discord 63, + extension-package 18. The first task run showed 89/1, and I didn't capture + the failing line. Three more task runs passed 90/90. I count it as a flake + I can't name, not as green on the first try. +- Union (control-board, webui, seat, mosaic, ledger, discord) on the same + clone: 434/434 three times, 23 to 24 s each. No fake pi left running. +- No test opens the real `~/.t3`. Every CLI spawn sets `HOME` to a temp + directory, and no test calls `readT3` in process. After the runs, no + `ledger-*` temp directories remained. + +## Live read + +`node packages/ledger/src/cli.mjs --since 2026-09-01 --until 2026-09-26 +--no-issues`, exit 0 three times, no header conflict. The table is the run at +2026-09-26T21:31:02Z. + +| Seat | T3 threads | T3 board / agent / human | Pi board / agent / human | +|---|---|---|---| +| darkwing | Darkwing; Darkwing in Claude (archived) | 0 / 19 / 28 | 14 / 109 / 141 | +| dewey | Dewey; Dewey in Claude | 0 / 17 / 7 | 7 / 57 / 16 | +| filbert | Filbert | 0 / 25 / 1 | 5 / 92 / 9 | +| rocko | Rocko | 0 / 20 / 1 | none | +| sage | Sage | 0 / 52 / 10 | 0 / 193 / 72 | +| researcher | none | none | 3 / 1 / 1 | +| t3:unmapped | Discord Bot | 0 / 0 / 68 | none | + +This matches the brief, allowing for messages sent since 20:54Z. It maps the +same seven threads. Discord Bot has 68 human: 54 without a header and the 14 +free-text headers. The diagnostic reads exactly those 14 +(`humanSentThroughApi: 14`, `humanWithoutEvent: 0`). T3 agent messages total +133, against the brief's 96 API headers (80 plus the 16 uppercase ones) at +20:54Z. Two imported threads are excluded, and this project has no deleted +threads. + +## Not covered + +- Snapshot isolation across the queries (see the `BEGIN` mutation above). +- A seat directory named `t3:unmapped` would share the unmapped row. Directory + names that contain a colon aren't used in `agents/`. +- The live read's effect on the main database file can't be checked while T3 + writes to it. The stopped and writer-attached WAL tests check it on + fixtures. + +## Review and correction + +Filbert approved manifest ba73a163 and the U+2028 fix as its own item: +`agents/filbert/work/ledger-t3-build-review-2026-09-26.md`, sha256 e47ec6da. + +Correction to "Beyond the brief" above. Line 611 holds a raw U+2028 and a raw +U+2029, and `readline` ends a line at each. The file has 731 lines by `\n` +(`wc -l` agrees), and `readline` makes 733. I wrote 732 because I counted the +empty string after the final newline. The splitter already ends lines at `\n` +only, so the fix covers both characters. The test and the README name only +U+2028. + +Filbert's nonblocking notes, for a follow-up after the Gate F commit, since +changing the pinned files now would void the approval: +1. Add a U+2029 to the splitter test and the README line. +2. Two diagnostic mutations survive: `humanWithoutEvent` hardcoded to 0, and + an unparseable event skipped instead of making the diagnostic `unknown`. + Each needs one fixture message. +3. `readT3`'s catch reports any error that isn't a `SourceError` as a SQLite + read failure. It still exits 1, but a bug would read as a database + problem. Rethrow errors that carry no `errcode`. +4. Snapshot isolation stays untested, as recorded above. diff --git a/agents/darkwing/work/ledger-t3-source/build.patch b/agents/darkwing/work/ledger-t3-source/build.patch new file mode 100644 index 00000000..4ff57d99 --- /dev/null +++ b/agents/darkwing/work/ledger-t3-source/build.patch @@ -0,0 +1,828 @@ +diff --git a/packages/ledger/README.md b/packages/ledger/README.md +index da0ab1c5..393e9c37 100644 +--- a/packages/ledger/README.md ++++ b/packages/ledger/README.md +@@ -1,13 +1,16 @@ + # Ledger + + Read-only counts from local `refactor` commit subjects, one Gitea issue-list +-request through `scripts/gitea-api.sh`, and repo seats' Pi session logs. ++request through `scripts/gitea-api.sh`, repo seats' Pi session logs, and T3's ++thread messages in `~/.t3/userdata/state.sqlite`. + No board changes, data-root writes, fleet reads, transcript output, or scheduler. + + ```sh + node packages/ledger/src/cli.mjs --since 2026-09-06 --until 2026-09-12 + node packages/ledger/src/cli.mjs --since 2026-09-06 --until 2026-09-12 --json + node packages/ledger/src/cli.mjs --since 2026-09-06 --no-issues ++node packages/ledger/src/cli.mjs --since 2026-09-06 --no-t3 ++node packages/ledger/src/cli.mjs --since 2026-09-06 --t3-db /tmp/fixture.sqlite + node --test packages/ledger/tests/ + ``` + +@@ -36,12 +39,17 @@ No install, build, service restart, or configuration change is needed. + duplicated entries in copied logs are not deduplicated. No transcript content + leaves the parser. Assistant messages and logs outside repo seats do not count. + Symlink source directories are refused and symlink files are not followed. ++ A line ends at `\n` only. A U+2028 inside a JSON string does not split a record. ++- Table 2 also counts T3 thread messages with role `user`. The T3 source ++ follows. A seat's row sums its Pi and T3 counts; the JSON keeps the split in ++ `pi` (Pi rows) and `t3.seats` (T3 rows). + - The first text line alone classifies a message. Two addressing forms count: + the tmux preamble `[host:session -> host:session]` that `agent-send.sh` + writes, and the T3 header `[from: role (thread-id) -> to: role (thread-id)]` + from `docs/guides/T3-AGENT-COMMS.md`. Either may carry ` class=` before + the closing bracket. A preamble whose sender is `control-board` (tmux session + or T3 role) is board; any other valid preamble is agent; otherwise human. ++ The class may be in either case: seats send `class=DECISION`. + This is a format count, not proof of who typed the message. Text blocks are + joined with newlines. The entry timestamp is used, falling back to the + message timestamp. +@@ -53,6 +61,69 @@ No install, build, service restart, or configuration change is needed. + ratios and durations with one decimal. Titles truncate to 48 characters in + text only. Missing evidence is the literal string `unknown`. + ++## T3 source ++ ++The rules come from `docs/plans/2026-09-26_ledger-t3-source.md` (Gate F). ++The source is on by default. `--no-t3` skips it, and the report then says ++`T3: not read (--no-t3)`. `--t3-db ` reads another database file with ++the same checks. The JSON records the database path and whether it was the ++default. When it wasn't, the text report prints the path, so a fixture result ++can't pass for a live one. The two flags can't be combined. ++ ++The reader opens `state.sqlite` read-only through `node:sqlite` and reads no ++other file in `~/.t3`. It runs every query in one read transaction with a 5 s ++busy timeout. It never writes the main database file. Like any SQLite ++connection it may create `-wal` and `-shm` beside it, so a directory that ++isn't writable refuses when SQLite needs them. ++ ++- **Project.** Only threads in the one non-deleted T3 project whose ++ `workspace_root` equals this checkout's root byte for byte. The root is the ++ realpath of the package, so a project opened through the compatibility ++ symlink `~/src/mosaic-stack-dev-test` does not match, and the report refuses ++ with no project. ++- **Thread to seat.** A thread belongs to seat `` when `` is a real ++ directory in `agents/` and the lower-cased title equals `` or starts ++ with `` and a space. "Dewey in Claude" maps to `dewey`; "Sagebrush" maps ++ to nothing. Several threads can map to one seat. Threads that map to no ++ seat share one row, `t3:unmapped`, so their human messages still reach the ++ totals. `t3.seats` and `t3.unmapped` in the JSON list the thread ids and ++ titles behind each row. ++- **Titles are current state.** T3 titles an unnamed thread from its first ++ prompt, and a rename moves a thread's whole history to another row. This ++ moves counts between rows, never out of the totals. ++- **Header check.** A user message whose T3 header is addressed to its own ++ thread id must name that thread's seat as the `to:` role (compared lower ++ case). In an unmapped thread the `to:` role must not be a seat. A conflict ++ exits 1 and names the thread, its title and both roles. A header addressed ++ to another thread isn't checked. The check misses a renamed thread that no ++ agent writes to. Such a thread can only add human counts to a row. ++- **Excluded.** Imported threads (id prefix `import:`) are partial copies of ++ Claude Code sessions, not T3 traffic; every T3 event marked `historyImport` ++ sits in one today. Deleted threads don't count; archived threads do. ++ `t3.excluded` gives both thread counts. ++- **Blind spot.** Threads in other T3 projects are not counted, even if they ++ worked on this repository. Live, there is a project at `/home/jwoltje` and ++ a deleted one at `/mnt/storage/src`. ++- **Diagnostic.** `t3.diagnostic.humanSentThroughApi` counts in-range user ++ messages the header rule calls human that T3 recorded as sent through its ++ API (no `appVersion` in the event's origin). Those are seat messages whose ++ header the rule doesn't accept, such as the older free-text Discord Bot ++ headers, and would show the next format drift. `humanWithoutEvent` counts ++ human messages with no `thread.message-sent` event. This is T3's internal ++ metadata, so it feeds no table or total. If `orchestration_events` or a ++ column it needs is missing, or an event doesn't parse, both read `unknown`. ++ ++These refuse the report with exit 1, and the ones about the database name ++`--no-t3`: a missing, unreadable or unopenable database (including a busy ++lock past the timeout); a symlink at `~/.t3`, `~/.t3/userdata` or ++`state.sqlite` (with `--t3-db`, the file or its directory); a missing table or ++column the counts need; no project or more than one for this root; a message ++in a counted thread with a role other than `user` or `assistant`, non-text ++content, or a `created_at` that doesn't parse; a header conflict. A missing Pi ++directory means no Pi seats ran here; a missing T3 database means the path or ++T3 changed, so it refuses instead of counting zero. Error messages name ids ++and paths, never message text. ++ + ## One Gitea call and missing evidence + + The client requests issues updated since the start date, all states, first page, +@@ -62,8 +133,8 @@ Use a narrower range or `--no-issues`, not hidden pagination. A commit-linked + issue not returned by the updated-since query still has a row, with unknown + metadata. This is the cost of the brief's one-call boundary. + +-Exit 0 means a report was computed. Exit 1 means bad arguments or unreadable git +-or session evidence. Malformed JSONL, including a partially written last line, ++Exit 0 means a report was computed. Exit 1 means bad arguments or unreadable git, ++session or T3 evidence. Malformed JSONL, including a partially written last line, + refuses the report; rerun after the seat finishes writing. Exit 2 means issue + credentials, API, payload, or completeness failure. The CLI never prints API + error bodies or reads authentication files itself. `--no-issues` makes no API +@@ -72,8 +143,9 @@ median duration, and human-per-closed ratio. It cannot invent close-only rows. + + For fixtures, a fake `gitea-api.sh` can be placed first on PATH. Otherwise the + repository scripts directory is appended to PATH for the issue request. +-Tests use only temporary repositories, logs, and fake API tools, with no real +-credentials or network. The helper regression stubs Node before any credential ++Tests use only temporary repositories, logs, T3 databases and fake API tools, ++with no real credentials or network. Every CLI run in the tests sets `HOME` to ++a temporary directory, so no test opens the real `~/.t3`. The helper regression stubs Node before any credential + read and checks successful GET, successful POST, and failed HTTP status. + + ## Acceptance +diff --git a/packages/ledger/src/cli.mjs b/packages/ledger/src/cli.mjs +index 66cd1417..5cf6705c 100644 +--- a/packages/ledger/src/cli.mjs ++++ b/packages/ledger/src/cli.mjs +@@ -1,11 +1,12 @@ + #!/usr/bin/env node + import path from 'node:path'; + import { fileURLToPath } from 'node:url'; +-import { dateRange, readCommits, readIssues, readSessions, summarize, formatTable, SourceError } from './ledger.mjs'; ++import { dateRange, readCommits, readIssues, readSessions, mergeSources, summarize, formatTable, SourceError } from './ledger.mjs'; ++import { readT3, defaultT3Path } from './t3.mjs'; + +-const usage = 'Usage: node packages/ledger/src/cli.mjs --since YYYY-MM-DD [--until YYYY-MM-DD] [--json] [--no-issues]'; ++const usage = 'Usage: node packages/ledger/src/cli.mjs --since YYYY-MM-DD [--until YYYY-MM-DD] [--json] [--no-issues] [--no-t3 | --t3-db PATH]'; + export async function main(args, root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../..')) { +- let since, until, json = false, noIssues = false; ++ let since, until, t3Db, json = false, noIssues = false, noT3 = false; + const seen = new Set(); + for (let i = 0; i < args.length; i++) { + const flag = args[i]; +@@ -14,13 +15,18 @@ export async function main(args, root = path.resolve(path.dirname(fileURLToPath( + if (flag === '--help') { console.log(usage); return; } + if (flag === '--json') json = true; + else if (flag === '--no-issues') noIssues = true; +- else if (flag === '--since' || flag === '--until') { ++ else if (flag === '--no-t3') noT3 = true; ++ else if (flag === '--t3-db') { ++ t3Db = args[++i]; ++ if (!t3Db || t3Db.startsWith('--')) throw new SourceError('--t3-db requires a path'); ++ } else if (flag === '--since' || flag === '--until') { + const value = args[++i]; + if (!value || value.startsWith('--')) throw new SourceError(`${flag} requires a date`); + if (flag === '--since') since = value; else until = value; + } else throw new SourceError('Unknown option; ' + usage); + } + if (!since) throw new SourceError(usage); ++ if (noT3 && t3Db !== undefined) throw new SourceError('--no-t3 and --t3-db cannot be combined'); + const range = dateRange(since, until); + const commits = readCommits(root, range); + // Fixture tools may be placed first on PATH. The repository client is the +@@ -30,7 +36,9 @@ export async function main(args, root = path.resolve(path.dirname(fileURLToPath( + let issues; + try { issues = noIssues ? null : readIssues(root, range); } + finally { if (priorPath === undefined) delete process.env.PATH; else process.env.PATH = priorPath; } +- const sessions = await readSessions(root, range); ++ // T3 is on by default. A missing or unreadable database refuses the report. ++ const t3 = noT3 ? null : await readT3(root, range, t3Db === undefined ? { dbPath: defaultT3Path(), isDefault: true } : { dbPath: t3Db, isDefault: false }); ++ const sessions = mergeSources(await readSessions(root, range), t3); + const report = summarize(range, commits, issues, sessions); + console.log(json ? JSON.stringify(report, null, 2) : formatTable(report)); + return report; +diff --git a/packages/ledger/src/ledger.mjs b/packages/ledger/src/ledger.mjs +index b09e95f5..3669d1cc 100644 +--- a/packages/ledger/src/ledger.mjs ++++ b/packages/ledger/src/ledger.mjs +@@ -1,7 +1,6 @@ + import { execFileSync } from 'node:child_process'; + import { createReadStream } from 'node:fs'; + import { readdir, lstat } from 'node:fs/promises'; +-import { createInterface } from 'node:readline'; + import path from 'node:path'; + + const DAY = 86400000; +@@ -23,7 +22,7 @@ export function dateRange(since, until = new Date().toISOString().slice(0, 10)) + if (end <= start) throw new SourceError('--until must not precede --since'); + return { since, until, start, end }; + } +-const inRange = (value, range) => { ++export const inRange = (value, range) => { + const ms = typeof value === 'number' ? value : Date.parse(value); + return Number.isFinite(ms) && ms >= range.start && ms < range.end; + }; +@@ -75,15 +74,32 @@ export function messageText(content) { + if (Array.isArray(content)) return content.filter(c => c?.type === 'text' && typeof c.text === 'string').map(c => c.text).join('\n'); + return ''; + } ++// Classes are matched in either case: seats send DECISION and REVIEW-REQUEST. ++// tmux preamble from agent-send.sh: [host:session -> host:session class=x] ++const TMUX = /^\[([^\s:\[\]]+):([^\s\[\]]+) -> ([^\s:\[\]]+):([^\s\[\]]+)(?: class=[A-Za-z-]+)?\](?:\s|$)/; ++// T3 header (docs/guides/T3-AGENT-COMMS.md): [from: role (id) -> to: role (id) class=x] ++const T3 = /^\[from: ([^\s()\[\]]+) \(([^()\[\]]+)\) -> to: ([^\s()\[\]]+) \(([^()\[\]]+)\)(?: class=[A-Za-z-]+)?\](?:\s|$)/; ++const firstLine = text => text.split(/\r?\n/, 1)[0]; ++export function t3Header(text) { ++ const m = firstLine(text).match(T3); ++ return m ? { from: m[1], fromId: m[2], to: m[3], toId: m[4] } : null; ++} + export function messageKind(text) { +- const firstLine = text.split(/\r?\n/, 1)[0]; +- // tmux preamble from agent-send.sh: [host:session -> host:session class=x] +- const tmux = firstLine.match(/^\[([^\s:\[\]]+):([^\s\[\]]+) -> ([^\s:\[\]]+):([^\s\[\]]+)(?: class=[a-z-]+)?\](?:\s|$)/); +- // T3 header (docs/guides/T3-AGENT-COMMS.md): [from: role (id) -> to: role (id) class=x] +- const t3 = firstLine.match(/^\[from: ([^\s()\[\]]+) \(([^()\[\]]+)\) -> to: ([^\s()\[\]]+) \(([^()\[\]]+)\)(?: class=[a-z-]+)?\](?:\s|$)/); +- const sender = tmux ? tmux[2] : t3 ? t3[1] : null; ++ const tmux = firstLine(text).match(TMUX), t3 = t3Header(text); ++ const sender = tmux ? tmux[2] : t3 ? t3.from : null; + return sender === null ? 'human' : sender === 'control-board' ? 'board' : 'agent'; + } ++// JSONL lines end at \n only. readline also ends a line at U+2028, which JSON ++// allows raw inside a string, so it split valid records (Node 26.8.1). ++async function* jsonLines(input) { ++ let rest = ''; ++ for await (const chunk of input) { ++ const parts = (rest + chunk).split('\n'); ++ rest = parts.pop(); ++ yield* parts; ++ } ++ if (rest) yield rest; ++} + async function directories(dir, optional = false) { + try { + if (!(await lstat(dir)).isDirectory()) throw new SourceError('Session source must be a real directory'); +@@ -93,11 +109,15 @@ async function directories(dir, optional = false) { + throw new SourceError(`Cannot read ledger directory: ${dir}`); + } + } ++// Seats are the real directories in agents/, sorted. ++export async function readSeats(root) { ++ return (await directories(path.join(root, 'agents'))).filter(e => e.isDirectory()).map(e => e.name).sort((a, b) => a.localeCompare(b)); ++} + export async function readSessions(root, range) { + const rows = []; + const mentions = new Map(); + // No symlink traversal, no fleet paths, no transcript content in the report. +- const agents = (await directories(path.join(root, 'agents'))).filter(e => e.isDirectory()).sort((a, b) => a.name.localeCompare(b.name)); ++ const agents = (await readSeats(root)).map(name => ({ name })); + const state = path.join(root, '.pi', 'state'); + // Check every source ancestor, not only the leaf directory. + if (!(await directories(path.join(root, '.pi'), true)).length) return { rows, mentions }; +@@ -109,11 +129,10 @@ export async function readSessions(root, range) { + const files = (await directories(dir, true)).filter(e => e.isFile() && e.name.endsWith('.jsonl')); + const row = { seat: agent.name, board: 0, agent: 0, human: 0 }; + for (const file of files) { +- const input = createReadStream(path.join(dir, file.name)); +- const lines = createInterface({ input, crlfDelay: Infinity }); ++ const input = createReadStream(path.join(dir, file.name), { encoding: 'utf8' }); + let lineNumber = 0; + try { +- for await (const line of lines) { ++ for await (const line of jsonLines(input)) { + lineNumber++; + if (!line.trim()) continue; + let entry; +@@ -132,12 +151,33 @@ export async function readSessions(root, range) { + mentions.get(number).add(agent.name); + } + } +- } finally { lines.close(); input.destroy(); } ++ } finally { input.destroy(); } + } + if (row.board + row.agent + row.human) rows.push(row); + } + return { rows, mentions }; + } ++// Adds T3 counts to the Pi rows per seat. Unmapped T3 threads get one row, ++// last. The report keeps the Pi rows and the T3 section so the split shows. ++export function mergeSources(pi, t3, unmapped = 't3:unmapped') { ++ if (!t3) return { rows: pi.rows, mentions: pi.mentions, pi: pi.rows, t3: { read: false } }; ++ const bySeat = new Map(pi.rows.map(r => [r.seat, { ...r }])); ++ for (const [seat, counts] of t3.rows) { ++ if (seat === unmapped || !(counts.board + counts.agent + counts.human)) continue; ++ const row = bySeat.get(seat) ?? { seat, board: 0, agent: 0, human: 0 }; ++ for (const kind of ['board', 'agent', 'human']) row[kind] += counts[kind]; ++ bySeat.set(seat, row); ++ } ++ const rows = [...bySeat.values()].sort((a, b) => a.seat.localeCompare(b.seat)); ++ const extra = t3.rows.get(unmapped); ++ if (extra.board + extra.agent + extra.human) rows.push({ seat: unmapped, ...extra }); ++ const mentions = new Map([...pi.mentions].map(([n, seats]) => [n, new Set(seats)])); ++ for (const [n, seats] of t3.mentions) { ++ if (!mentions.has(n)) mentions.set(n, new Set()); ++ for (const seat of seats) mentions.get(n).add(seat); ++ } ++ return { rows, mentions, pi: pi.rows, t3: t3.section }; ++} + const round = value => Math.round(value * 10) / 10; + function duration(issue) { + if (!issue) return UNKNOWN; +@@ -163,13 +203,14 @@ export function summarize(range, commits, issues, sessions) { + const median = hours.includes(UNKNOWN) ? UNKNOWN : hours.length ? + round(hours.length % 2 ? hours[middle] : (hours[middle - 1] + hours[middle]) / 2) : 0; + const human = sessions.rows.reduce((sum, r) => sum + r.human, 0); +- return { since: range.since, until: range.until, timezone: 'UTC', issues: rows, seats: sessions.rows, ++ const sources = sessions.t3 ? { pi: sessions.pi, t3: sessions.t3 } : {}; ++ return { since: range.since, until: range.until, timezone: 'UTC', issues: rows, seats: sessions.rows, ...sources, + totals: { issuesClosed: issues === null ? UNKNOWN : closed.length, + medianHoursOpen: issues === null ? UNKNOWN : median, commits: commits.length, + followUpsPerIssue: rows.length ? round(rows.reduce((sum, r) => sum + r.followUps, 0) / rows.length) : 0, + humanMessagesPerClosedIssue: issues === null ? UNKNOWN : closed.length ? round(human / closed.length) : human ? UNKNOWN : 0 } }; + } +-const clean = value => String(value).replace(/[\x00-\x1f\x7f-\x9f]/g, ' '); ++export const clean = value => String(value).replace(/[\x00-\x1f\x7f-\x9f]/g, ' '); + const decimal = value => typeof value === 'number' ? value.toFixed(1) : value; + export function totalsLine(t) { + return `Totals: issues closed ${t.issuesClosed} | median hours open ${decimal(t.medianHoursOpen)} | commits ${t.commits} | follow-ups per issue ${decimal(t.followUpsPerIssue)} | human messages per closed issue ${decimal(t.humanMessagesPerClosedIssue)}`; +@@ -181,5 +222,12 @@ export function formatTable(report) { + decimal(r.hoursOpen), r.commits, r.followUps, r.seats.join(', ')].join(' | ')), + '', 'Seat | Board | Agent | Human', + ...report.seats.map(r => [clean(r.seat), r.board, r.agent, r.human].join(' | ')), ++ ...t3Line(report.t3), + '', totalsLine(report.totals)].join('\n'); + } ++// One line when T3 was skipped or read from somewhere other than the default. ++function t3Line(t3) { ++ if (!t3) return []; ++ if (!t3.read) return ['T3: not read (--no-t3)']; ++ return t3.database.default ? [] : [`T3: read from ${clean(t3.database.path)}, not the default`]; ++} +diff --git a/packages/ledger/tests/ledger.test.mjs b/packages/ledger/tests/ledger.test.mjs +index 175f5d5e..8e60b96b 100644 +--- a/packages/ledger/tests/ledger.test.mjs ++++ b/packages/ledger/tests/ledger.test.mjs +@@ -1,6 +1,7 @@ + import test from 'node:test'; + import assert from 'node:assert/strict'; +-import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync, cpSync, symlinkSync } from 'node:fs'; ++import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync, cpSync, symlinkSync, realpathSync } from 'node:fs'; ++import { DatabaseSync } from 'node:sqlite'; + import os from 'node:os'; + import path from 'node:path'; + import { fileURLToPath } from 'node:url'; +@@ -9,13 +10,50 @@ import { dateRange, messageKind, issueNumbers, totalsLine, summarize } from '../ + + const source = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../src'); + const range = dateRange('2026-09-06', '2026-09-12'); ++// T3 fixture schema: the live tables, cut to the columns the reader uses plus ++// one it doesn't. `text` allows NULL so a non-text row can be tested. ++const T3_SCHEMA = ` ++ create table projection_projects (project_id text primary key, title text not null, workspace_root text not null, deleted_at text); ++ create table projection_threads (thread_id text primary key, project_id text not null, title text not null, archived_at text, deleted_at text); ++ create table projection_thread_messages (message_id text primary key, thread_id text not null, role text not null, text, created_at text not null); ++ create table orchestration_events (sequence integer primary key autoincrement, stream_id text not null, event_type text not null, payload_json text not null, metadata_json text not null);`; ++// Writes a T3 database in WAL mode. Threads default to project p1, which is ++// the fixture root. Returns the open writer when keepOpen is set. ++function t3db(file, { root, projects, threads = [], messages = [], after = [], keepOpen = false }) { ++ mkdirSync(path.dirname(file), { recursive: true }); ++ for (const old of [file, `${file}-wal`, `${file}-shm`]) rmSync(old, { force: true }); ++ const db = new DatabaseSync(file); ++ db.exec('pragma journal_mode=wal'); db.exec(T3_SCHEMA); ++ for (const [id, workspace, deleted = null] of projects ?? [['p1', root]]) { ++ db.prepare('insert into projection_projects values (?, ?, ?, ?)').run(id, 'project', workspace, deleted); ++ } ++ for (const t of threads) { ++ db.prepare('insert into projection_threads values (?, ?, ?, ?, ?)').run(t.id, t.project ?? 'p1', t.title, t.archived ?? null, t.deleted ?? null); ++ } ++ for (const m of messages) addMessage(db, m); ++ for (const sql of after) db.exec(sql); ++ if (keepOpen) return db; ++ db.close(); ++} ++let messageId = 0; ++function addMessage(db, { thread, text, role = 'user', at = '2026-09-08T12:00:00Z', origin = 'app' }) { ++ const id = `m${++messageId}`; ++ db.prepare('insert into projection_thread_messages values (?, ?, ?, ?, ?)').run(id, thread, role, text, at); ++ if (origin !== 'none') db.prepare('insert into orchestration_events (stream_id, event_type, payload_json, metadata_json) values (?, ?, ?, ?)') ++ .run(thread, 'thread.message-sent', JSON.stringify({ messageId: id, threadId: thread, role, text }), JSON.stringify({ origin: origin === 'app' ? { appVersion: '0.0.0' } : {} })); ++} + const fixtureIssues = [ + { number: 1, title: 'First issue', created_at: '2026-09-06T00:00:00Z', closed_at: '2026-09-07T12:00:00Z' }, + { number: 2, title: 'Second issue', created_at: '2026-09-06T00:00:00Z', closed_at: null }, + ]; + function fixture(t) { + const root = mkdtempSync(path.join(os.tmpdir(), 'ledger-test-')); +- t.after(() => rmSync(root, { recursive: true, force: true })); ++ // No test opens the real ~/.t3: every CLI run gets this HOME, with an empty ++ // T3 database at the default path. The CLI's root is a realpath. ++ const home = mkdtempSync(path.join(os.tmpdir(), 'ledger-home-')); ++ t.after(() => { rmSync(root, { recursive: true, force: true }); rmSync(home, { recursive: true, force: true }); }); ++ const defaultDb = path.join(home, '.t3/userdata/state.sqlite'); ++ t3db(defaultDb, { root: realpathSync(root) }); + const put = (name, data) => { const p = path.join(root, name); mkdirSync(path.dirname(p), { recursive: true }); writeFileSync(p, data); return p; }; + const git = (args, date = '2026-09-07T00:00:00Z') => execFileSync('git', args, { cwd: root, env: { ...process.env, GIT_AUTHOR_DATE: date, GIT_COMMITTER_DATE: date, GIT_CONFIG_NOSYSTEM: '1', GIT_CONFIG_GLOBAL: '/dev/null' }, stdio: 'pipe' }); + git(['init', '-b', 'refactor']); git(['config', 'user.email', 'fixture@example.invalid']); git(['config', 'user.name', 'Fixture']); +@@ -30,8 +68,8 @@ function fixture(t) { + const entry = (text, timestamp = '2026-09-08T12:00:00Z') => ({ type: 'message', timestamp, message: { role: 'user', content: [{ type: 'text', text }] } }); + const logs = [entry('[host:control-board -> host:alice] do #1'), entry('[host:bob -> host:alice] review #2'), entry('build #2'), entry('old #1', '2026-09-05T23:59:59Z'), { type: 'message', timestamp: '2026-09-08T00:00:00Z', message: { role: 'assistant', content: 'not a user #1' } }]; + put('.pi/state/alice/sessions/one.jsonl', logs.map(x => JSON.stringify(x)).join('\n') + '\n'); +- const run = (args = [], env = {}) => spawnSync(process.execPath, [path.join(root, 'packages/ledger/src/cli.mjs'), '--since', '2026-09-06', '--until', '2026-09-12', ...args], { cwd: root, encoding: 'utf8', env: { ...process.env, PATH: `${path.join(root, 'bin')}:${process.env.PATH}`, ISSUES: path.join(root, 'issues.json'), CALLS: path.join(root, 'calls.jsonl'), ...env } }); +- return { root, put, commit, run, entry, logs }; ++ const run = (args = [], env = {}) => spawnSync(process.execPath, [path.join(root, 'packages/ledger/src/cli.mjs'), '--since', '2026-09-06', '--until', '2026-09-12', ...args], { cwd: root, encoding: 'utf8', env: { ...process.env, HOME: home, PATH: `${path.join(root, 'bin')}:${process.env.PATH}`, ISSUES: path.join(root, 'issues.json'), CALLS: path.join(root, 'calls.jsonl'), ...env } }); ++ return { root, real: realpathSync(root), home, defaultDb, put, commit, run, entry, logs }; + } + test('fixture git subjects only, follow-ups and three session kinds', t => { + const f = fixture(t), result = f.run(['--json']); +@@ -63,7 +101,7 @@ test('missing credentials exit 2, no-issues never calls API and shows unknown', + test('empty range gives no rows and zero totals', t => { + const f = fixture(t); + f.put('issues.json', '[]'); +- const result = spawnSync(process.execPath, [path.join(f.root, 'packages/ledger/src/cli.mjs'), '--since', '2027-01-01', '--until', '2027-01-02', '--json'], { encoding: 'utf8', env: { ...process.env, PATH: `${f.root}/bin:${process.env.PATH}`, ISSUES: `${f.root}/issues.json`, CALLS: `${f.root}/calls.jsonl` } }); ++ const result = spawnSync(process.execPath, [path.join(f.root, 'packages/ledger/src/cli.mjs'), '--since', '2027-01-01', '--until', '2027-01-02', '--json'], { encoding: 'utf8', env: { ...process.env, HOME: f.home, PATH: `${f.root}/bin:${process.env.PATH}`, ISSUES: `${f.root}/issues.json`, CALLS: `${f.root}/calls.jsonl` } }); + assert.equal(result.status, 0, result.stderr); const r = JSON.parse(result.stdout); + assert.deepEqual(r.issues, []); assert.deepEqual(r.seats, []); assert.ok(Object.values(r.totals).every(n => n === 0)); + }); +@@ -96,6 +134,13 @@ test('partial or malformed session log refuses with location, not content', t => + const f = fixture(t); f.put('.pi/state/alice/sessions/bad.jsonl', '{sensitive'); const r = f.run(); + assert.equal(r.status, 1); assert.match(r.stderr, /Malformed session JSON: alice\/bad.jsonl:1/); assert.doesNotMatch(r.stderr, /sensitive/); + }); ++test('a U+2028 inside a session string is one line, not a malformed record', t => { ++ const f = fixture(t); ++ f.put('.pi/state/bob/sessions/sep.jsonl', [f.entry('Jason: one\u2028two #2'), f.entry('[h:alice -> h:bob] ok')].map(x => JSON.stringify(x)).join('\r\n') + '\r\n'); ++ assert.ok(readFileSync(path.join(f.root, '.pi/state/bob/sessions/sep.jsonl'), 'utf8').includes('\u2028')); ++ const r = f.run(['--json']); assert.equal(r.status, 0, r.stderr); ++ assert.deepEqual(JSON.parse(r.stdout).seats[1], { seat: 'bob', board: 0, agent: 1, human: 1 }); ++}); + test('no sessions is an empty table; symlink source refuses', t => { + const f = fixture(t); rmSync(path.join(f.root, '.pi'), { recursive: true }); + assert.deepEqual(JSON.parse(f.run(['--json']).stdout).seats, []); +@@ -140,7 +185,11 @@ test('T3 header: agent, or board from control-board; anything short of the full + assert.equal(messageKind(`Jason here\n[from: ${sage} -> to: ${filbert}]\nquoted`), 'human'); + assert.equal(messageKind(` [from: ${sage} -> to: ${filbert}]`), 'human'); + assert.equal(messageKind(`[from: sage -> to: filbert]\nno thread ids`), 'human'); +- assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=Actionable]`), 'human'); ++ // Classes match in either case (Gate F). HEAD before the fix called these human. ++ assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=Actionable]`), 'agent'); ++ assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=REVIEW-REQUEST]\nreview`), 'agent'); ++ assert.equal(messageKind('[h:sage -> h:bob class=DECISION] go'), 'agent'); ++ assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=review_request]`), 'human'); + assert.equal(messageKind(`[from: ${sage} -> to: ${filbert}]trailing`), 'human'); + assert.equal(messageKind(`[From: ${sage} -> to: ${filbert}]`), 'human'); + }); +@@ -148,3 +197,218 @@ test('no closed issues with human messages means undefined ratio, not invented z + const r = summarize(range, [], [], { rows: [{ seat: 'a', human: 1, board: 0, agent: 0 }], mentions: new Map() }); + assert.equal(r.totals.humanMessagesPerClosedIssue, 'unknown'); + }); ++ ++// T3 thread source (Gate F, docs/plans/2026-09-26_ledger-t3-source.md). ++const T1 = 't-alice', T2 = 't-bob', T3 = 't-sagebrush', T4 = 't-researcher', T5 = 't-discord'; ++const header = (from, to, toId, cls = '') => `[from: ${from} (x1) -> to: ${to} (${toId})${cls}]`; ++function t3Fixture(t) { ++ const f = fixture(t); ++ for (const seat of ['sage', 'researcher']) mkdirSync(path.join(f.root, 'agents', seat), { recursive: true }); ++ const db = path.join(f.home, 'fixture/t3.sqlite'); ++ const threads = [ ++ { id: T1, title: 'Alice' }, { id: T2, title: 'Bob in Claude', archived: '2026-09-09T00:00:00Z' }, ++ { id: T3, title: 'Sagebrush' }, { id: T4, title: 'Researcher' }, { id: T5, title: 'Discord Bot' }, ++ { id: 'import:claudeAgent:1', title: 'alice' }, { id: 't-deleted', title: 'Alice', deleted: '2026-09-09T00:00:00Z' }, ++ { id: 't-other', project: 'p2', title: 'Alice' }, ++ ]; ++ const messages = [ ++ { thread: T1, text: 'Jason: go #1' }, ++ { thread: T1, text: `${header('sage', 'alice', T1, ' class=REVIEW-REQUEST')}\nreview #2`, origin: 'api' }, ++ { thread: T1, text: '[h:sage -> h:alice class=DECISION] go', origin: 'api' }, ++ { thread: T1, text: `${header('control-board', 'alice', T1)}\nbuzz`, origin: 'api' }, ++ { thread: T1, text: 'outside', at: '2026-09-13T00:00:00Z' }, ++ { thread: T1, text: 'an answer #9', role: 'assistant', origin: 'none' }, ++ { thread: T2, text: 'archived still counts #2' }, ++ { thread: T3, text: 'Sagebrush is not sage' }, ++ { thread: T3, text: `${header('sage', 'discord', T3)}\nnot a seat role`, origin: 'api' }, ++ { thread: T4, text: 'research this' }, ++ { thread: T5, text: '[from: SetSpark coordinator (x1) -> to: Discord Bot (x2)]\nfree text', origin: 'api' }, ++ { thread: 'import:claudeAgent:1', text: 'imported' }, ++ { thread: 't-deleted', text: 'deleted' }, ++ { thread: 't-other', text: 'other project' }, ++ ]; ++ const write = (overrides = {}) => t3db(db, { root: f.real, projects: [['p1', f.real], ['p2', '/elsewhere']], threads, messages, ...overrides }); ++ return { ...f, db, threads, messages, write }; ++} ++test('T3: seat, archived, unmapped and Researcher threads count; imported, deleted and other-project threads do not', t => { ++ const f = t3Fixture(t); f.write(); ++ const result = f.run(['--json', '--t3-db', f.db]); ++ assert.equal(result.status, 0, result.stderr); ++ const r = JSON.parse(result.stdout); ++ assert.deepEqual(r.seats, [ ++ { seat: 'alice', board: 2, agent: 3, human: 2 }, { seat: 'bob', board: 0, agent: 0, human: 1 }, ++ { seat: 'researcher', board: 0, agent: 0, human: 1 }, { seat: 't3:unmapped', board: 0, agent: 1, human: 2 }, ++ ]); ++ assert.deepEqual(r.pi, [{ seat: 'alice', board: 1, agent: 1, human: 1 }]); ++ assert.deepEqual(r.t3.database, { path: f.db, default: false }); ++ assert.deepEqual(r.t3.seats, [ ++ { seat: 'alice', board: 1, agent: 2, human: 1, threads: [{ id: T1, title: 'Alice', archived: false }] }, ++ { seat: 'bob', board: 0, agent: 0, human: 1, threads: [{ id: T2, title: 'Bob in Claude', archived: true }] }, ++ { seat: 'researcher', board: 0, agent: 0, human: 1, threads: [{ id: T4, title: 'Researcher', archived: false }] }, ++ ]); ++ assert.deepEqual(r.t3.unmapped, { board: 0, agent: 1, human: 2, threads: [ ++ { id: T5, title: 'Discord Bot', archived: false }, { id: T3, title: 'Sagebrush', archived: false }] }); ++ assert.deepEqual(r.t3.excluded, { importedThreads: 1, deletedThreads: 1 }); ++ // The free-text header counts as human; only the diagnostic shows it was sent through the API. ++ assert.deepEqual(r.t3.diagnostic, { humanSentThroughApi: 1, humanWithoutEvent: 0 }); ++ assert.equal(r.totals.humanMessagesPerClosedIssue, 6); ++ assert.deepEqual(r.issues.map(x => [x.issue, x.seats]), [[1, ['alice']], [2, ['alice', 'bob']]]); ++ const text = f.run(['--t3-db', f.db]); ++ assert.equal(text.status, 0, text.stderr); ++ assert.ok(text.stdout.includes(`T3: read from ${f.db}, not the default`)); ++ assert.match(text.stdout, /t3:unmapped \| 0 \| 1 \| 2/); ++}); ++test('T3: the default path is read from HOME and prints no path line; --no-t3 says so', t => { ++ const f = t3Fixture(t); f.write(); ++ rmSync(f.defaultDb); cpSync(f.db, f.defaultDb); ++ const json = JSON.parse(f.run(['--json']).stdout); ++ assert.deepEqual(json.t3.database, { path: f.defaultDb, default: true }); ++ assert.equal(json.seats.at(-1).seat, 't3:unmapped'); ++ const text = f.run(); assert.equal(text.status, 0, text.stderr); assert.doesNotMatch(text.stdout, /^T3:/m); ++ rmSync(path.join(f.home, '.t3'), { recursive: true }); ++ const off = f.run(['--no-t3']); assert.equal(off.status, 0, off.stderr); ++ assert.match(off.stdout, /^T3: not read \(--no-t3\)$/m); ++ const offJson = JSON.parse(f.run(['--no-t3', '--json']).stdout); ++ assert.deepEqual(offJson.t3, { read: false }); assert.deepEqual(offJson.seats, [{ seat: 'alice', board: 1, agent: 1, human: 1 }]); ++ const both = f.run(['--no-t3', '--t3-db', f.db]); assert.equal(both.status, 1); assert.match(both.stderr, /cannot be combined/); ++ assert.equal(f.run(['--t3-db']).status, 1); ++}); ++test('T3: a HOME with no database exits 1 and names --no-t3', t => { ++ const f = fixture(t); rmSync(path.join(f.home, '.t3'), { recursive: true }); ++ const r = f.run(); assert.equal(r.status, 1); assert.equal(r.stdout, ''); ++ assert.match(r.stderr, /T3 database unavailable: .*\.t3 is missing or unreadable; use --no-t3/); ++}); ++test('T3: a file that is not a database exits 1 and names --no-t3', t => { ++ const f = fixture(t); writeFileSync(f.defaultDb, 'not sqlite'.repeat(100)); ++ const r = f.run(); assert.equal(r.status, 1); assert.match(r.stderr, /T3 database cannot be read: .*\(SQLite \d+\); use --no-t3/); ++}); ++test('T3: a seat thread renamed to another seat exits 1 naming thread, title and roles', t => { ++ const f = t3Fixture(t); ++ f.write({ messages: [...f.messages, { thread: T2, text: `${header('sage', 'alice', T2, ' class=INFO')}\nfor alice`, origin: 'api' }] }); ++ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); ++ assert.equal(r.stderr.trim(), `T3 header conflict: thread ${T2} "Bob in Claude" maps to bob, but a header addresses alice`); ++}); ++test('T3: an unmapped thread addressed as a seat exits 1', t => { ++ const f = t3Fixture(t); ++ f.write({ messages: [...f.messages, { thread: T3, text: `${header('bob', 'Sage', T3)}\nhi`, origin: 'api' }] }); ++ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); ++ assert.match(r.stderr, /thread t-sagebrush "Sagebrush" maps to no seat, but a header addresses Sage/); ++}); ++test('T3: a header to another thread id is not cross-checked', t => { ++ const f = t3Fixture(t); ++ f.write({ messages: [...f.messages, { thread: T2, text: `${header('sage', 'alice', T1)}\ncopied`, origin: 'api' }] }); ++ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr); ++ assert.equal(JSON.parse(r.stdout).t3.seats[1].agent, 1); ++}); ++test('T3: no project, or two, for this root exits 1', t => { ++ const f = t3Fixture(t); ++ f.write({ projects: [['p1', `${f.real}-link`], ['p2', '/elsewhere']] }); ++ let r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, /T3 has no project for .*symlink does not match/); ++ f.write({ projects: [['p1', f.real], ['p2', f.real]] }); ++ r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, /T3 has more than one project for/); ++ f.write({ projects: [['p1', f.real], ['p2', f.real, '2026-09-01T00:00:00Z']] }); ++ assert.equal(f.run(['--t3-db', f.db]).status, 0); ++}); ++for (const [name, after, pattern] of [ ++ ['a removed column', ['alter table projection_threads drop column title'], /T3 schema changed: missing projection_threads.title/], ++ ['a missing table', ['drop table projection_thread_messages'], /T3 schema changed: missing projection_thread_messages$/m], ++]) test(`T3: ${name} exits 1 and names it`, t => { ++ const f = t3Fixture(t); f.write({ after, messages: [] }); ++ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, pattern); ++}); ++for (const [name, message, pattern] of [ ++ ['an unknown role', { role: 'system' }, /T3 message m\d+ in thread t-alice has an unknown role/], ++ ['non-text content', { text: null }, /has non-text content/], ++ ['an unparseable created_at', { at: 'yesterday' }, /has an invalid created_at/], ++]) test(`T3: a counted row with ${name} exits 1 without its text`, t => { ++ const f = t3Fixture(t); ++ f.write({ messages: [...f.messages, { thread: T1, text: 'secret words', ...message }] }); ++ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, pattern); assert.doesNotMatch(r.stderr, /secret/); ++}); ++test('T3: a missing orchestration_events makes the diagnostic unknown and keeps the counts', t => { ++ const f = t3Fixture(t); f.write(); ++ const before = JSON.parse(f.run(['--json', '--t3-db', f.db]).stdout); ++ f.write({ after: ['drop table orchestration_events'] }); ++ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr); ++ const after = JSON.parse(r.stdout); ++ assert.deepEqual(after.t3.diagnostic, { humanSentThroughApi: 'unknown', humanWithoutEvent: 'unknown' }); ++ assert.deepEqual(after.seats, before.seats); assert.deepEqual(after.totals, before.totals); ++}); ++for (const link of ['.t3', '.t3/userdata', '.t3/userdata/state.sqlite']) test(`T3: a symlink at ~/${link} exits 1`, t => { ++ const f = fixture(t), target = path.join(f.home, 'real', link); ++ mkdirSync(path.dirname(target), { recursive: true }); ++ cpSync(path.join(f.home, link), target, { recursive: true }); ++ rmSync(path.join(f.home, link), { recursive: true }); symlinkSync(target, path.join(f.home, link)); ++ const r = f.run(); assert.equal(r.status, 1); assert.match(r.stderr, new RegExp(`${link.replaceAll('.', '\\.')} is a symlink; use --no-t3`)); ++}); ++test('T3: with --t3-db, a symlinked file or directory exits 1', t => { ++ const f = t3Fixture(t); f.write(); ++ const file = path.join(f.home, 'file-link.sqlite'); symlinkSync(f.db, file); ++ let r = f.run(['--t3-db', file]); assert.equal(r.status, 1); assert.match(r.stderr, /file-link.sqlite is a symlink/); ++ const dir = path.join(f.home, 'dir-link'); symlinkSync(path.dirname(f.db), dir); ++ r = f.run(['--t3-db', path.join(dir, 't3.sqlite')]); assert.equal(r.status, 1); assert.match(r.stderr, /dir-link is a symlink/); ++}); ++ ++// WAL states. The CLI reads with mode=ro; it may create -wal and -shm but must ++// never change the main file. ++const sha = file => execFileSync('sha256sum', [file], { encoding: 'utf8' }).split(' ')[0]; ++const humans = r => JSON.parse(r.stdout).t3.seats.find(s => s.seat === 'alice').human; ++const asRoot = process.getuid?.() === 0; ++function killedWriter(db) { ++ // A writer that commits into the WAL and dies without a checkpoint. ++ const code = `const { DatabaseSync } = require('node:sqlite'); const db = new DatabaseSync(${JSON.stringify(db)}); ++ db.exec('pragma wal_autocheckpoint=0'); ++ db.prepare("insert into projection_thread_messages values ('late', 't-alice', 'user', 'late human', '2026-09-08T13:00:00Z')").run(); ++ process.kill(process.pid, 'SIGKILL');`; ++ const r = spawnSync(process.execPath, ['-e', code]); ++ assert.equal(r.signal, 'SIGKILL'); ++ rmSync(`${db}-shm`); ++} ++function inReadOnlyDir(dir, check) { ++ execFileSync('chmod', ['0555', dir]); ++ try { check(); } finally { execFileSync('chmod', ['0755', dir]); } ++} ++test('T3 WAL: the newest message only in -wal, writer attached, is counted', t => { ++ const f = t3Fixture(t), writer = f.write({ keepOpen: true }); ++ t.after(() => writer.close()); ++ writer.exec('pragma wal_autocheckpoint=0'); ++ addMessage(writer, { thread: T1, text: 'newest', at: '2026-09-08T13:00:00Z' }); ++ const main = sha(f.db); ++ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr); ++ assert.equal(humans(r), 2); assert.equal(sha(f.db), main); ++}); ++test('T3 WAL: stopped cleanly, counts are correct and the main file is unchanged', t => { ++ const f = t3Fixture(t); f.write(); ++ assert.throws(() => readFileSync(`${f.db}-wal`)); ++ const main = sha(f.db); ++ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr); ++ assert.equal(humans(r), 1); assert.equal(sha(f.db), main); ++}); ++test('T3 WAL: -wal without -shm in a writable directory is read', t => { ++ const f = t3Fixture(t); f.write(); killedWriter(f.db); ++ const main = sha(f.db); ++ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr); ++ assert.equal(humans(r), 2); assert.equal(sha(f.db), main); ++}); ++test('T3 WAL: -wal without -shm in a read-only directory exits 1', { skip: asRoot && 'mode bits do not bind root' }, t => { ++ const f = t3Fixture(t); f.write(); killedWriter(f.db); ++ inReadOnlyDir(path.dirname(f.db), () => { ++ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, /cannot be read: .*\(SQLite 14\); use --no-t3/); ++ }); ++}); ++test('T3 WAL: stopped cleanly in a read-only directory exits 1', { skip: asRoot && 'mode bits do not bind root' }, t => { ++ const f = t3Fixture(t); f.write(); ++ inReadOnlyDir(path.dirname(f.db), () => { ++ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, /cannot be read: .*\(SQLite 1544\); use --no-t3/); ++ }); ++}); ++test('T3: a lock held past the 5 s busy timeout exits 1 and names --no-t3', t => { ++ const f = t3Fixture(t), writer = f.write({ keepOpen: true }); ++ t.after(() => writer.close()); ++ writer.exec('pragma locking_mode=exclusive'); writer.exec('begin exclusive'); ++ addMessage(writer, { thread: T1, text: 'held' }); ++ const started = Date.now(), r = f.run(['--t3-db', f.db]); ++ writer.exec('commit'); ++ assert.equal(r.status, 1); assert.match(r.stderr, /cannot be read: .*\(SQLite 5\); use --no-t3/); ++ assert.ok(Date.now() - started >= 4500, 'the reader waited for the busy timeout'); ++}); +diff --git a/packages/ledger/src/t3.mjs b/packages/ledger/src/t3.mjs +new file mode 100644 +index 00000000..9672fcc9 +--- /dev/null ++++ b/packages/ledger/src/t3.mjs +@@ -0,0 +1,151 @@ ++import { lstat } from 'node:fs/promises'; ++import { DatabaseSync } from 'node:sqlite'; ++import { pathToFileURL } from 'node:url'; ++import os from 'node:os'; ++import path from 'node:path'; ++import { SourceError, UNKNOWN, clean, inRange, issueNumbers, messageKind, readSeats, t3Header } from './ledger.mjs'; ++ ++// T3 keeps every thread message in one SQLite database. This reader opens that ++// file read-only and nothing else in ~/.t3. See ++// docs/plans/2026-09-26_ledger-t3-source.md for the rules below. ++export const UNMAPPED = 't3:unmapped'; ++const SKIP = 'use --no-t3 to skip T3'; ++const REQUIRED = { ++ projection_projects: ['project_id', 'workspace_root', 'deleted_at'], ++ projection_threads: ['thread_id', 'project_id', 'title', 'archived_at', 'deleted_at'], ++ projection_thread_messages: ['message_id', 'thread_id', 'role', 'text', 'created_at'], ++}; ++const DIAGNOSTIC = { orchestration_events: ['stream_id', 'event_type', 'payload_json', 'metadata_json'] }; ++ ++export const defaultT3Path = () => path.join(os.homedir(), '.t3', 'userdata', 'state.sqlite'); ++ ++// Every named path must exist and must not be a symlink. Skipping one would be ++// a silent zero, so each problem refuses the report. ++async function checkPaths(dbPath, isDefault) { ++ const dirs = isDefault ? [path.dirname(path.dirname(dbPath)), path.dirname(dbPath)] : [path.dirname(dbPath)]; ++ for (const [target, wantDir] of [...dirs.map(d => [d, true]), [dbPath, false]]) { ++ let stat; ++ try { stat = await lstat(target); } ++ catch { throw new SourceError(`T3 database unavailable: ${target} is missing or unreadable; ${SKIP}`); } ++ if (stat.isSymbolicLink()) throw new SourceError(`T3 database refused: ${target} is a symlink; ${SKIP}`); ++ if (wantDir ? !stat.isDirectory() : !stat.isFile()) { ++ throw new SourceError(`T3 database refused: ${target} is not a ${wantDir ? 'directory' : 'regular file'}; ${SKIP}`); ++ } ++ } ++} ++ ++function missingColumns(db, tables) { ++ const missing = []; ++ for (const [table, columns] of Object.entries(tables)) { ++ const have = new Set(db.prepare('select name from pragma_table_info(?)').all(table).map(r => r.name)); ++ if (!have.size) missing.push(table); ++ else for (const column of columns) if (!have.has(column)) missing.push(`${table}.${column}`); ++ } ++ return missing; ++} ++ ++// Seat for a thread title: the lower-cased title equals the seat or starts ++// with the seat and a space. Longest seat first, so the most specific wins. ++export function seatForTitle(title, seats) { ++ const lower = title.toLowerCase(); ++ return [...seats].sort((a, b) => b.length - a.length).find(s => lower === s || lower.startsWith(`${s} `)) ?? null; ++} ++ ++// Origin per message id from thread.message-sent events. Any missing table, ++// column or unparseable event makes the diagnostic unknown; it decides nothing. ++function origins(db, projectId) { ++ if (missingColumns(db, DIAGNOSTIC).length) return null; ++ const byMessage = new Map(); ++ const events = db.prepare(`select e.payload_json, e.metadata_json from orchestration_events e ++ join projection_threads t on t.thread_id = e.stream_id ++ where e.event_type = 'thread.message-sent' and t.project_id = ?`).all(projectId); ++ for (const event of events) { ++ let payload, metadata; ++ try { payload = JSON.parse(event.payload_json); metadata = JSON.parse(event.metadata_json); } ++ catch { return null; } ++ if (typeof payload?.messageId !== 'string') return null; ++ byMessage.set(payload.messageId, typeof metadata?.origin?.appVersion === 'string'); ++ } ++ return byMessage; ++} ++ ++function query(db, root, range, seats) { ++ const missing = missingColumns(db, REQUIRED); ++ if (missing.length) throw new SourceError(`T3 schema changed: missing ${missing.join(', ')}`); ++ // Compared in JavaScript so a declared collation can't loosen the match. ++ const projects = db.prepare('select project_id, workspace_root from projection_projects where deleted_at is null').all() ++ .filter(p => p.workspace_root === root); ++ if (projects.length !== 1) { ++ throw new SourceError(`T3 has ${projects.length ? 'more than one project' : 'no project'} for ${root}; a project opened through a symlink does not match; ${SKIP}`); ++ } ++ const projectId = projects[0].project_id; ++ const threads = new Map(), excluded = { importedThreads: 0, deletedThreads: 0 }; ++ for (const t of db.prepare('select thread_id, title, archived_at, deleted_at from projection_threads where project_id = ?').all(projectId)) { ++ if (typeof t.thread_id !== 'string' || typeof t.title !== 'string') throw new SourceError('T3 thread with a non-text id or title'); ++ if (t.thread_id.startsWith('import:')) { excluded.importedThreads++; continue; } ++ if (t.deleted_at !== null) { excluded.deletedThreads++; continue; } ++ threads.set(t.thread_id, { id: t.thread_id, title: t.title, archived: t.archived_at !== null, seat: seatForTitle(t.title, seats) }); ++ } ++ const rows = new Map([...seats, UNMAPPED].map(s => [s, { board: 0, agent: 0, human: 0 }])); ++ const mentions = new Map(), human = []; ++ const messages = db.prepare(`select m.message_id, m.thread_id, m.role, m.text, m.created_at from projection_thread_messages m ++ join projection_threads t on t.thread_id = m.thread_id where t.project_id = ?`).all(projectId); ++ for (const m of messages) { ++ const thread = threads.get(m.thread_id); ++ if (!thread) continue; ++ const where = `T3 message ${clean(m.message_id)} in thread ${clean(m.thread_id)}`; ++ if (m.role !== 'user' && m.role !== 'assistant') throw new SourceError(`${where} has an unknown role`); ++ if (typeof m.text !== 'string') throw new SourceError(`${where} has non-text content`); ++ if (typeof m.created_at !== 'string' || !Number.isFinite(Date.parse(m.created_at))) throw new SourceError(`${where} has an invalid created_at`); ++ if (m.role !== 'user') continue; ++ // A header addressed to its own thread must agree with the title mapping. ++ const header = t3Header(m.text); ++ if (header && header.toId === thread.id) { ++ const to = header.to.toLowerCase(); ++ if (thread.seat ? to !== thread.seat : seats.includes(to)) { ++ throw new SourceError(`T3 header conflict: thread ${clean(thread.id)} "${clean(thread.title)}" maps to ${thread.seat ?? 'no seat'}, but a header addresses ${clean(header.to)}`); ++ } ++ } ++ if (!inRange(m.created_at, range)) continue; ++ const kind = messageKind(m.text), seat = thread.seat ?? UNMAPPED; ++ rows.get(seat)[kind]++; ++ if (kind === 'human') human.push(m.message_id); ++ for (const number of issueNumbers(m.text)) { ++ if (!mentions.has(number)) mentions.set(number, new Set()); ++ mentions.get(number).add(seat); ++ } ++ } ++ const byMessage = origins(db, projectId); ++ const sentThroughApi = byMessage === null ? UNKNOWN : human.filter(id => byMessage.get(id) === false).length; ++ const noEvent = byMessage === null ? UNKNOWN : human.filter(id => !byMessage.has(id)).length; ++ const listed = seat => [...threads.values()].filter(t => (t.seat ?? UNMAPPED) === seat) ++ .sort((a, b) => a.id.localeCompare(b.id)).map(({ id, title, archived }) => ({ id, title, archived })); ++ const seatRows = seats.map(seat => ({ seat, ...rows.get(seat), threads: listed(seat) })).filter(r => r.threads.length); ++ return { rows, mentions, excluded, seats: seatRows, unmapped: { ...rows.get(UNMAPPED), threads: listed(UNMAPPED) }, ++ diagnostic: { humanSentThroughApi: sentThroughApi, humanWithoutEvent: noEvent } }; ++} ++ ++// Reads one snapshot of T3's database. Returns the per-seat rows and issue ++// mentions the ledger merges with Pi, and the report's `t3` section. ++export async function readT3(root, range, { dbPath = defaultT3Path(), isDefault = true } = {}) { ++ dbPath = path.resolve(dbPath); ++ await checkPaths(dbPath, isDefault); ++ const seats = await readSeats(root); ++ const url = pathToFileURL(dbPath); ++ url.searchParams.set('mode', 'ro'); ++ let db, result; ++ try { ++ db = new DatabaseSync(url, { readOnly: true, timeout: 5000 }); ++ db.exec('BEGIN'); ++ result = query(db, root, range, seats); ++ db.exec('COMMIT'); ++ } catch (error) { ++ if (error instanceof SourceError) throw error; ++ throw new SourceError(`T3 database cannot be read: ${dbPath} (SQLite ${error.errcode ?? 'error'}); ${SKIP}`); ++ } finally { ++ try { if (db?.isTransaction) db.exec('ROLLBACK'); } catch { /* the close below still runs */ } ++ try { db?.close(); } catch { /* nothing was written */ } ++ } ++ const { rows, mentions, ...section } = result; ++ return { rows, mentions, section: { read: true, database: { path: dbPath, default: isDefault }, ...section } }; ++} diff --git a/agents/filbert/work/ledger-t3-build-review-2026-09-26.md b/agents/filbert/work/ledger-t3-build-review-2026-09-26.md new file mode 100644 index 00000000..91956560 --- /dev/null +++ b/agents/filbert/work/ledger-t3-build-review-2026-09-26.md @@ -0,0 +1,146 @@ +# Gate F build: Filbert's code review + +Reviewer: Filbert, 2026-09-26. Requested by Darkwing, assigned by Sage (#1506). + +Candidate: `agents/darkwing/work/ledger-t3-source/`, measured against brief R3 +(`docs/plans/2026-09-26_ledger-t3-source.md`, sha256 `f3c05c1b`, verified). + +| File | sha256 | +|---|---| +| `build-manifest.sha256` | `ba73a163…97b0` | +| `build.patch` | `dc4cf73e…d4d4` | +| `build.md` | `f0ea61db…d8bc` | + +All five manifest pins verify in the shared tree. `build.patch` applies +cleanly to a detached worktree at `1c5f6bc3` and reproduces the same five pins. + +**Verdict: approve** the manifest `ba73a163`, including the U+2028 fix as a +separate item. The notes below don't block. Carry them into the commit or a +follow-up; either way the verdict stands. + +## 1. What I ran + +- **Ledger tests, clean worktree:** `node --test --test-concurrency=1 + packages/ledger/tests/` gives 47/47 (18.5 s, 5.5 s of it the busy-timeout + test). +- **Suites.** Nothing outside `packages/ledger` imports the ledger, and the + patch touches only `packages/ledger`. I didn't rerun Darkwing's eight + suites or the union; they can't reach this code. +- **Live read**, shared tree (pins verified), read-only: `cli.mjs --since + 2026-09-01 --until 2026-09-26 --no-issues`. It exits 0 in 1.6 s with no + header conflict. + - The rows match `build.md`, plus messages sent since 21:31Z (T3 agent: +1 + darkwing, +1 dewey, +1 filbert, +3 sage). Human counts are unchanged. + - The same seven threads map. Discord Bot is 68 human in `t3:unmapped`. + - The diagnostic is `humanSentThroughApi: 14`, `humanWithoutEvent: 0`. Two + imported threads are excluded and none deleted. `database` is the default + path. +- **No-project refusal, live:** run from the worktree root + (`/tmp/fb-gatef-wt`, which has no T3 project), the CLI exits 1 with "T3 has + no project for /tmp/fb-gatef-wt; … use --no-t3". +- **The class fix against HEAD:** HEAD's `messageKind`, taken from `git show + 1c5f6bc3`, calls `class=REVIEW-REQUEST` (T3), `class=DECISION` (tmux) and + `class=Actionable` human. The build calls all three agent. +- **Two mutations of my own** (below, note 2). Both survive. + +## 2. Against the brief + +- **§1 reading.** + - The URI comes from `pathToFileURL` with `mode=ro` set through + `searchParams`, plus `readOnly: true`, and the timeout is 5000. + - Every query, the schema checks and the diagnostic included, runs between + `BEGIN` and `COMMIT`. A deferred read transaction pins its WAL snapshot at + the first read, so the queries share one snapshot. That holds by + construction, and no test proves it. `build.md` names this gap. + - The code has no `immutable` and makes no copy. + - Each of the five WAL states has its own test. The two read-only-directory + tests assert SQLite 14 and 1544 and skip as root. The stopped-clean test + asserts that no `-wal` exists beforehand, which is the case my R1 test got + wrong. +- **§2 classes.** Both regexes take `[A-Za-z-]+`. A class with `_` or with no + `class=` form is still human (tested). +- **§3 mapping.** + - `workspace_root` is compared byte for byte in JavaScript, and exactly one + non-deleted project must match. + - Titles map by longest seat first. + - The cross-check covers T3 headers whose `to:` id is the message's own + thread, in both directions. + - `t3:unmapped` comes last. + - The JSON lists the thread ids and titles per seat and for the unmapped + row. + - The fixture root is stored as `realpathSync(root)`. +- **§4 fail closed.** + - `~/.t3`, `userdata` and the file are each checked with `lstat`, and a + symlink or the wrong type refuses. With `--t3-db`, the file and its + directory are checked. Each case has a test. + - Missing tables and columns are named. There are tests for 0 and 2 + projects, and for a deleted second project, which passes. + - A bad role, text or date refuses without printing the text (tested). + - `--no-t3` and `--t3-db` refuse each other. + - No path falls back to Pi alone. +- **§6 acceptance.** + - `HOME` is set at both spawn sites (lines 71 and 104). + - Every fixture has an empty default database with a realpath project row. + - A `HOME` with no database exits 1 and names `--no-t3`. + - The fixture covers every listed case: seat, archived, unmapped, + Sagebrush, Researcher, imported, deleted, another project, out of range, + all three header forms with uppercase classes, and a rename conflict. + - The README covers the source, both flags, the mapping, the header check, + the exclusions, the symlinked checkout and the other-project blind spot. + - The BUILD-LOG entry naming the test file is part of Sage's commit. +- **Choices the brief left open.** I accept all of them: + - exclusion by the `import:` prefix alone, which the README states; + - the cross-check and row validation over every message in a counted + thread; + - an in-range diagnostic; + - `pi` and `t3` added without changing `seats` or `totals`; + - a `--t3-db` pointed at the default path still printing the path line, + which errs toward visible. + +## 3. The U+2028 fix, as its own item + +It is correct and minimal, and it blocked the brief's live acceptance. +`createReadStream` with `encoding: 'utf8'` decodes through a StringDecoder, +so a multibyte character split across chunks survives. A CRLF line keeps its +`\r`, which `JSON.parse` accepts as whitespace, and `trim()` still skips a +blank CRLF line. Line numbers match `readline`'s for `\n` files. The new test +fails with `readline` and passes with the splitter. The longest live Pi line +is 1.2 MB, so repeated `rest + chunk` concatenation costs nothing that +matters. + +What I confirmed on Node 26.8.1: +- `readline` ends a line at U+2028 **and** U+2029. Two records containing + one each came out as four lines. +- On the named log, line 611 holds one U+2028 and one U+2029. The file has + 731 lines by `\n` (`wc -l` agrees) and 733 by `readline`, and every + `\n`-line parses. + +## 4. Nonblocking notes + +1. **U+2029.** + - Record: `build.md` says the line holds "a raw U+2028" and gives 732 lines + by `\n`. The record should say U+2028 and U+2029, and 731. + - Test and README: add a U+2029 to the test string and to the README line. + The splitter already handles it; this just pins it down. +2. **Two diagnostic paths have no test.** Each mutation leaves the T3 tests + at 26/26: + - `humanWithoutEvent` hardcoded to 0: no fixture has a human message + without an event (`origin: 'none'` is used only on an assistant row); + - an unparseable event skipped instead of making the diagnostic + `unknown`: no fixture has a bad `payload_json`. + + The diagnostic decides nothing, so these don't block. Each is one fixture + message. +3. **The catch-all in `readT3`.** Any error that isn't a `SourceError` + becomes "T3 database cannot be read … (SQLite error)", including a + `TypeError` from a future bug in `query`. It still exits 1, so the source + stays fail-closed. The message would blame the database, though. + Rethrowing errors that carry no `errcode` would keep bugs visible. +4. **Snapshot isolation is untested,** as `build.md` says. The code is right, + and a test would need a hook between statements. Leave it recorded. + +## Scratch + +The worktree was `/tmp/fb-gatef-wt`, removed after the run. The mutations ran +there and were reverted, and the pins were re-verified before removal. No +commit or push. diff --git a/agents/filbert/work/ledger-t3-source-review-2026-09-26.md b/agents/filbert/work/ledger-t3-source-review-2026-09-26.md index 94dea091..706e5115 100644 --- a/agents/filbert/work/ledger-t3-source-review-2026-09-26.md +++ b/agents/filbert/work/ledger-t3-source-review-2026-09-26.md @@ -315,3 +315,16 @@ and I'll confirm the hash: any file. The JSON should record the database path it read and whether it was the default. The text report should add one line when it wasn't. A Gate F result then can't come from a fixture without saying so. + +**R3 confirmed** (Filbert, 2026-09-26): `docs/plans/2026-09-26_ledger-t3-source.md` +sha256 `f3c05c1b4d28a419ab817621e15708b147f71dff588664980e22696eaafbc342`. +`manifest.sha256` verifies all five files. The diff against the frozen R2 +(`r2.md`, `e8300cb6`) has six hunks: +- the header; +- the settled measurement; +- the database path in section 4; +- the realpath project row; +- the stopped test as a plain exit 1; +- the path in the acceptance list. + +Nothing else changed. Approved. diff --git a/packages/ledger/README.md b/packages/ledger/README.md index da0ab1c5..393e9c37 100644 --- a/packages/ledger/README.md +++ b/packages/ledger/README.md @@ -1,13 +1,16 @@ # Ledger Read-only counts from local `refactor` commit subjects, one Gitea issue-list -request through `scripts/gitea-api.sh`, and repo seats' Pi session logs. +request through `scripts/gitea-api.sh`, repo seats' Pi session logs, and T3's +thread messages in `~/.t3/userdata/state.sqlite`. No board changes, data-root writes, fleet reads, transcript output, or scheduler. ```sh node packages/ledger/src/cli.mjs --since 2026-09-06 --until 2026-09-12 node packages/ledger/src/cli.mjs --since 2026-09-06 --until 2026-09-12 --json node packages/ledger/src/cli.mjs --since 2026-09-06 --no-issues +node packages/ledger/src/cli.mjs --since 2026-09-06 --no-t3 +node packages/ledger/src/cli.mjs --since 2026-09-06 --t3-db /tmp/fixture.sqlite node --test packages/ledger/tests/ ``` @@ -36,12 +39,17 @@ No install, build, service restart, or configuration change is needed. duplicated entries in copied logs are not deduplicated. No transcript content leaves the parser. Assistant messages and logs outside repo seats do not count. Symlink source directories are refused and symlink files are not followed. + A line ends at `\n` only. A U+2028 inside a JSON string does not split a record. +- Table 2 also counts T3 thread messages with role `user`. The T3 source + follows. A seat's row sums its Pi and T3 counts; the JSON keeps the split in + `pi` (Pi rows) and `t3.seats` (T3 rows). - The first text line alone classifies a message. Two addressing forms count: the tmux preamble `[host:session -> host:session]` that `agent-send.sh` writes, and the T3 header `[from: role (thread-id) -> to: role (thread-id)]` from `docs/guides/T3-AGENT-COMMS.md`. Either may carry ` class=` before the closing bracket. A preamble whose sender is `control-board` (tmux session or T3 role) is board; any other valid preamble is agent; otherwise human. + The class may be in either case: seats send `class=DECISION`. This is a format count, not proof of who typed the message. Text blocks are joined with newlines. The entry timestamp is used, falling back to the message timestamp. @@ -53,6 +61,69 @@ No install, build, service restart, or configuration change is needed. ratios and durations with one decimal. Titles truncate to 48 characters in text only. Missing evidence is the literal string `unknown`. +## T3 source + +The rules come from `docs/plans/2026-09-26_ledger-t3-source.md` (Gate F). +The source is on by default. `--no-t3` skips it, and the report then says +`T3: not read (--no-t3)`. `--t3-db ` reads another database file with +the same checks. The JSON records the database path and whether it was the +default. When it wasn't, the text report prints the path, so a fixture result +can't pass for a live one. The two flags can't be combined. + +The reader opens `state.sqlite` read-only through `node:sqlite` and reads no +other file in `~/.t3`. It runs every query in one read transaction with a 5 s +busy timeout. It never writes the main database file. Like any SQLite +connection it may create `-wal` and `-shm` beside it, so a directory that +isn't writable refuses when SQLite needs them. + +- **Project.** Only threads in the one non-deleted T3 project whose + `workspace_root` equals this checkout's root byte for byte. The root is the + realpath of the package, so a project opened through the compatibility + symlink `~/src/mosaic-stack-dev-test` does not match, and the report refuses + with no project. +- **Thread to seat.** A thread belongs to seat `` when `` is a real + directory in `agents/` and the lower-cased title equals `` or starts + with `` and a space. "Dewey in Claude" maps to `dewey`; "Sagebrush" maps + to nothing. Several threads can map to one seat. Threads that map to no + seat share one row, `t3:unmapped`, so their human messages still reach the + totals. `t3.seats` and `t3.unmapped` in the JSON list the thread ids and + titles behind each row. +- **Titles are current state.** T3 titles an unnamed thread from its first + prompt, and a rename moves a thread's whole history to another row. This + moves counts between rows, never out of the totals. +- **Header check.** A user message whose T3 header is addressed to its own + thread id must name that thread's seat as the `to:` role (compared lower + case). In an unmapped thread the `to:` role must not be a seat. A conflict + exits 1 and names the thread, its title and both roles. A header addressed + to another thread isn't checked. The check misses a renamed thread that no + agent writes to. Such a thread can only add human counts to a row. +- **Excluded.** Imported threads (id prefix `import:`) are partial copies of + Claude Code sessions, not T3 traffic; every T3 event marked `historyImport` + sits in one today. Deleted threads don't count; archived threads do. + `t3.excluded` gives both thread counts. +- **Blind spot.** Threads in other T3 projects are not counted, even if they + worked on this repository. Live, there is a project at `/home/jwoltje` and + a deleted one at `/mnt/storage/src`. +- **Diagnostic.** `t3.diagnostic.humanSentThroughApi` counts in-range user + messages the header rule calls human that T3 recorded as sent through its + API (no `appVersion` in the event's origin). Those are seat messages whose + header the rule doesn't accept, such as the older free-text Discord Bot + headers, and would show the next format drift. `humanWithoutEvent` counts + human messages with no `thread.message-sent` event. This is T3's internal + metadata, so it feeds no table or total. If `orchestration_events` or a + column it needs is missing, or an event doesn't parse, both read `unknown`. + +These refuse the report with exit 1, and the ones about the database name +`--no-t3`: a missing, unreadable or unopenable database (including a busy +lock past the timeout); a symlink at `~/.t3`, `~/.t3/userdata` or +`state.sqlite` (with `--t3-db`, the file or its directory); a missing table or +column the counts need; no project or more than one for this root; a message +in a counted thread with a role other than `user` or `assistant`, non-text +content, or a `created_at` that doesn't parse; a header conflict. A missing Pi +directory means no Pi seats ran here; a missing T3 database means the path or +T3 changed, so it refuses instead of counting zero. Error messages name ids +and paths, never message text. + ## One Gitea call and missing evidence The client requests issues updated since the start date, all states, first page, @@ -62,8 +133,8 @@ Use a narrower range or `--no-issues`, not hidden pagination. A commit-linked issue not returned by the updated-since query still has a row, with unknown metadata. This is the cost of the brief's one-call boundary. -Exit 0 means a report was computed. Exit 1 means bad arguments or unreadable git -or session evidence. Malformed JSONL, including a partially written last line, +Exit 0 means a report was computed. Exit 1 means bad arguments or unreadable git, +session or T3 evidence. Malformed JSONL, including a partially written last line, refuses the report; rerun after the seat finishes writing. Exit 2 means issue credentials, API, payload, or completeness failure. The CLI never prints API error bodies or reads authentication files itself. `--no-issues` makes no API @@ -72,8 +143,9 @@ median duration, and human-per-closed ratio. It cannot invent close-only rows. For fixtures, a fake `gitea-api.sh` can be placed first on PATH. Otherwise the repository scripts directory is appended to PATH for the issue request. -Tests use only temporary repositories, logs, and fake API tools, with no real -credentials or network. The helper regression stubs Node before any credential +Tests use only temporary repositories, logs, T3 databases and fake API tools, +with no real credentials or network. Every CLI run in the tests sets `HOME` to +a temporary directory, so no test opens the real `~/.t3`. The helper regression stubs Node before any credential read and checks successful GET, successful POST, and failed HTTP status. ## Acceptance diff --git a/packages/ledger/src/cli.mjs b/packages/ledger/src/cli.mjs index 66cd1417..5cf6705c 100644 --- a/packages/ledger/src/cli.mjs +++ b/packages/ledger/src/cli.mjs @@ -1,11 +1,12 @@ #!/usr/bin/env node import path from 'node:path'; import { fileURLToPath } from 'node:url'; -import { dateRange, readCommits, readIssues, readSessions, summarize, formatTable, SourceError } from './ledger.mjs'; +import { dateRange, readCommits, readIssues, readSessions, mergeSources, summarize, formatTable, SourceError } from './ledger.mjs'; +import { readT3, defaultT3Path } from './t3.mjs'; -const usage = 'Usage: node packages/ledger/src/cli.mjs --since YYYY-MM-DD [--until YYYY-MM-DD] [--json] [--no-issues]'; +const usage = 'Usage: node packages/ledger/src/cli.mjs --since YYYY-MM-DD [--until YYYY-MM-DD] [--json] [--no-issues] [--no-t3 | --t3-db PATH]'; export async function main(args, root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../..')) { - let since, until, json = false, noIssues = false; + let since, until, t3Db, json = false, noIssues = false, noT3 = false; const seen = new Set(); for (let i = 0; i < args.length; i++) { const flag = args[i]; @@ -14,13 +15,18 @@ export async function main(args, root = path.resolve(path.dirname(fileURLToPath( if (flag === '--help') { console.log(usage); return; } if (flag === '--json') json = true; else if (flag === '--no-issues') noIssues = true; - else if (flag === '--since' || flag === '--until') { + else if (flag === '--no-t3') noT3 = true; + else if (flag === '--t3-db') { + t3Db = args[++i]; + if (!t3Db || t3Db.startsWith('--')) throw new SourceError('--t3-db requires a path'); + } else if (flag === '--since' || flag === '--until') { const value = args[++i]; if (!value || value.startsWith('--')) throw new SourceError(`${flag} requires a date`); if (flag === '--since') since = value; else until = value; } else throw new SourceError('Unknown option; ' + usage); } if (!since) throw new SourceError(usage); + if (noT3 && t3Db !== undefined) throw new SourceError('--no-t3 and --t3-db cannot be combined'); const range = dateRange(since, until); const commits = readCommits(root, range); // Fixture tools may be placed first on PATH. The repository client is the @@ -30,7 +36,9 @@ export async function main(args, root = path.resolve(path.dirname(fileURLToPath( let issues; try { issues = noIssues ? null : readIssues(root, range); } finally { if (priorPath === undefined) delete process.env.PATH; else process.env.PATH = priorPath; } - const sessions = await readSessions(root, range); + // T3 is on by default. A missing or unreadable database refuses the report. + const t3 = noT3 ? null : await readT3(root, range, t3Db === undefined ? { dbPath: defaultT3Path(), isDefault: true } : { dbPath: t3Db, isDefault: false }); + const sessions = mergeSources(await readSessions(root, range), t3); const report = summarize(range, commits, issues, sessions); console.log(json ? JSON.stringify(report, null, 2) : formatTable(report)); return report; diff --git a/packages/ledger/src/ledger.mjs b/packages/ledger/src/ledger.mjs index b09e95f5..3669d1cc 100644 --- a/packages/ledger/src/ledger.mjs +++ b/packages/ledger/src/ledger.mjs @@ -1,7 +1,6 @@ import { execFileSync } from 'node:child_process'; import { createReadStream } from 'node:fs'; import { readdir, lstat } from 'node:fs/promises'; -import { createInterface } from 'node:readline'; import path from 'node:path'; const DAY = 86400000; @@ -23,7 +22,7 @@ export function dateRange(since, until = new Date().toISOString().slice(0, 10)) if (end <= start) throw new SourceError('--until must not precede --since'); return { since, until, start, end }; } -const inRange = (value, range) => { +export const inRange = (value, range) => { const ms = typeof value === 'number' ? value : Date.parse(value); return Number.isFinite(ms) && ms >= range.start && ms < range.end; }; @@ -75,15 +74,32 @@ export function messageText(content) { if (Array.isArray(content)) return content.filter(c => c?.type === 'text' && typeof c.text === 'string').map(c => c.text).join('\n'); return ''; } +// Classes are matched in either case: seats send DECISION and REVIEW-REQUEST. +// tmux preamble from agent-send.sh: [host:session -> host:session class=x] +const TMUX = /^\[([^\s:\[\]]+):([^\s\[\]]+) -> ([^\s:\[\]]+):([^\s\[\]]+)(?: class=[A-Za-z-]+)?\](?:\s|$)/; +// T3 header (docs/guides/T3-AGENT-COMMS.md): [from: role (id) -> to: role (id) class=x] +const T3 = /^\[from: ([^\s()\[\]]+) \(([^()\[\]]+)\) -> to: ([^\s()\[\]]+) \(([^()\[\]]+)\)(?: class=[A-Za-z-]+)?\](?:\s|$)/; +const firstLine = text => text.split(/\r?\n/, 1)[0]; +export function t3Header(text) { + const m = firstLine(text).match(T3); + return m ? { from: m[1], fromId: m[2], to: m[3], toId: m[4] } : null; +} export function messageKind(text) { - const firstLine = text.split(/\r?\n/, 1)[0]; - // tmux preamble from agent-send.sh: [host:session -> host:session class=x] - const tmux = firstLine.match(/^\[([^\s:\[\]]+):([^\s\[\]]+) -> ([^\s:\[\]]+):([^\s\[\]]+)(?: class=[a-z-]+)?\](?:\s|$)/); - // T3 header (docs/guides/T3-AGENT-COMMS.md): [from: role (id) -> to: role (id) class=x] - const t3 = firstLine.match(/^\[from: ([^\s()\[\]]+) \(([^()\[\]]+)\) -> to: ([^\s()\[\]]+) \(([^()\[\]]+)\)(?: class=[a-z-]+)?\](?:\s|$)/); - const sender = tmux ? tmux[2] : t3 ? t3[1] : null; + const tmux = firstLine(text).match(TMUX), t3 = t3Header(text); + const sender = tmux ? tmux[2] : t3 ? t3.from : null; return sender === null ? 'human' : sender === 'control-board' ? 'board' : 'agent'; } +// JSONL lines end at \n only. readline also ends a line at U+2028, which JSON +// allows raw inside a string, so it split valid records (Node 26.8.1). +async function* jsonLines(input) { + let rest = ''; + for await (const chunk of input) { + const parts = (rest + chunk).split('\n'); + rest = parts.pop(); + yield* parts; + } + if (rest) yield rest; +} async function directories(dir, optional = false) { try { if (!(await lstat(dir)).isDirectory()) throw new SourceError('Session source must be a real directory'); @@ -93,11 +109,15 @@ async function directories(dir, optional = false) { throw new SourceError(`Cannot read ledger directory: ${dir}`); } } +// Seats are the real directories in agents/, sorted. +export async function readSeats(root) { + return (await directories(path.join(root, 'agents'))).filter(e => e.isDirectory()).map(e => e.name).sort((a, b) => a.localeCompare(b)); +} export async function readSessions(root, range) { const rows = []; const mentions = new Map(); // No symlink traversal, no fleet paths, no transcript content in the report. - const agents = (await directories(path.join(root, 'agents'))).filter(e => e.isDirectory()).sort((a, b) => a.name.localeCompare(b.name)); + const agents = (await readSeats(root)).map(name => ({ name })); const state = path.join(root, '.pi', 'state'); // Check every source ancestor, not only the leaf directory. if (!(await directories(path.join(root, '.pi'), true)).length) return { rows, mentions }; @@ -109,11 +129,10 @@ export async function readSessions(root, range) { const files = (await directories(dir, true)).filter(e => e.isFile() && e.name.endsWith('.jsonl')); const row = { seat: agent.name, board: 0, agent: 0, human: 0 }; for (const file of files) { - const input = createReadStream(path.join(dir, file.name)); - const lines = createInterface({ input, crlfDelay: Infinity }); + const input = createReadStream(path.join(dir, file.name), { encoding: 'utf8' }); let lineNumber = 0; try { - for await (const line of lines) { + for await (const line of jsonLines(input)) { lineNumber++; if (!line.trim()) continue; let entry; @@ -132,12 +151,33 @@ export async function readSessions(root, range) { mentions.get(number).add(agent.name); } } - } finally { lines.close(); input.destroy(); } + } finally { input.destroy(); } } if (row.board + row.agent + row.human) rows.push(row); } return { rows, mentions }; } +// Adds T3 counts to the Pi rows per seat. Unmapped T3 threads get one row, +// last. The report keeps the Pi rows and the T3 section so the split shows. +export function mergeSources(pi, t3, unmapped = 't3:unmapped') { + if (!t3) return { rows: pi.rows, mentions: pi.mentions, pi: pi.rows, t3: { read: false } }; + const bySeat = new Map(pi.rows.map(r => [r.seat, { ...r }])); + for (const [seat, counts] of t3.rows) { + if (seat === unmapped || !(counts.board + counts.agent + counts.human)) continue; + const row = bySeat.get(seat) ?? { seat, board: 0, agent: 0, human: 0 }; + for (const kind of ['board', 'agent', 'human']) row[kind] += counts[kind]; + bySeat.set(seat, row); + } + const rows = [...bySeat.values()].sort((a, b) => a.seat.localeCompare(b.seat)); + const extra = t3.rows.get(unmapped); + if (extra.board + extra.agent + extra.human) rows.push({ seat: unmapped, ...extra }); + const mentions = new Map([...pi.mentions].map(([n, seats]) => [n, new Set(seats)])); + for (const [n, seats] of t3.mentions) { + if (!mentions.has(n)) mentions.set(n, new Set()); + for (const seat of seats) mentions.get(n).add(seat); + } + return { rows, mentions, pi: pi.rows, t3: t3.section }; +} const round = value => Math.round(value * 10) / 10; function duration(issue) { if (!issue) return UNKNOWN; @@ -163,13 +203,14 @@ export function summarize(range, commits, issues, sessions) { const median = hours.includes(UNKNOWN) ? UNKNOWN : hours.length ? round(hours.length % 2 ? hours[middle] : (hours[middle - 1] + hours[middle]) / 2) : 0; const human = sessions.rows.reduce((sum, r) => sum + r.human, 0); - return { since: range.since, until: range.until, timezone: 'UTC', issues: rows, seats: sessions.rows, + const sources = sessions.t3 ? { pi: sessions.pi, t3: sessions.t3 } : {}; + return { since: range.since, until: range.until, timezone: 'UTC', issues: rows, seats: sessions.rows, ...sources, totals: { issuesClosed: issues === null ? UNKNOWN : closed.length, medianHoursOpen: issues === null ? UNKNOWN : median, commits: commits.length, followUpsPerIssue: rows.length ? round(rows.reduce((sum, r) => sum + r.followUps, 0) / rows.length) : 0, humanMessagesPerClosedIssue: issues === null ? UNKNOWN : closed.length ? round(human / closed.length) : human ? UNKNOWN : 0 } }; } -const clean = value => String(value).replace(/[\x00-\x1f\x7f-\x9f]/g, ' '); +export const clean = value => String(value).replace(/[\x00-\x1f\x7f-\x9f]/g, ' '); const decimal = value => typeof value === 'number' ? value.toFixed(1) : value; export function totalsLine(t) { return `Totals: issues closed ${t.issuesClosed} | median hours open ${decimal(t.medianHoursOpen)} | commits ${t.commits} | follow-ups per issue ${decimal(t.followUpsPerIssue)} | human messages per closed issue ${decimal(t.humanMessagesPerClosedIssue)}`; @@ -181,5 +222,12 @@ export function formatTable(report) { decimal(r.hoursOpen), r.commits, r.followUps, r.seats.join(', ')].join(' | ')), '', 'Seat | Board | Agent | Human', ...report.seats.map(r => [clean(r.seat), r.board, r.agent, r.human].join(' | ')), + ...t3Line(report.t3), '', totalsLine(report.totals)].join('\n'); } +// One line when T3 was skipped or read from somewhere other than the default. +function t3Line(t3) { + if (!t3) return []; + if (!t3.read) return ['T3: not read (--no-t3)']; + return t3.database.default ? [] : [`T3: read from ${clean(t3.database.path)}, not the default`]; +} diff --git a/packages/ledger/src/t3.mjs b/packages/ledger/src/t3.mjs new file mode 100644 index 00000000..9672fcc9 --- /dev/null +++ b/packages/ledger/src/t3.mjs @@ -0,0 +1,151 @@ +import { lstat } from 'node:fs/promises'; +import { DatabaseSync } from 'node:sqlite'; +import { pathToFileURL } from 'node:url'; +import os from 'node:os'; +import path from 'node:path'; +import { SourceError, UNKNOWN, clean, inRange, issueNumbers, messageKind, readSeats, t3Header } from './ledger.mjs'; + +// T3 keeps every thread message in one SQLite database. This reader opens that +// file read-only and nothing else in ~/.t3. See +// docs/plans/2026-09-26_ledger-t3-source.md for the rules below. +export const UNMAPPED = 't3:unmapped'; +const SKIP = 'use --no-t3 to skip T3'; +const REQUIRED = { + projection_projects: ['project_id', 'workspace_root', 'deleted_at'], + projection_threads: ['thread_id', 'project_id', 'title', 'archived_at', 'deleted_at'], + projection_thread_messages: ['message_id', 'thread_id', 'role', 'text', 'created_at'], +}; +const DIAGNOSTIC = { orchestration_events: ['stream_id', 'event_type', 'payload_json', 'metadata_json'] }; + +export const defaultT3Path = () => path.join(os.homedir(), '.t3', 'userdata', 'state.sqlite'); + +// Every named path must exist and must not be a symlink. Skipping one would be +// a silent zero, so each problem refuses the report. +async function checkPaths(dbPath, isDefault) { + const dirs = isDefault ? [path.dirname(path.dirname(dbPath)), path.dirname(dbPath)] : [path.dirname(dbPath)]; + for (const [target, wantDir] of [...dirs.map(d => [d, true]), [dbPath, false]]) { + let stat; + try { stat = await lstat(target); } + catch { throw new SourceError(`T3 database unavailable: ${target} is missing or unreadable; ${SKIP}`); } + if (stat.isSymbolicLink()) throw new SourceError(`T3 database refused: ${target} is a symlink; ${SKIP}`); + if (wantDir ? !stat.isDirectory() : !stat.isFile()) { + throw new SourceError(`T3 database refused: ${target} is not a ${wantDir ? 'directory' : 'regular file'}; ${SKIP}`); + } + } +} + +function missingColumns(db, tables) { + const missing = []; + for (const [table, columns] of Object.entries(tables)) { + const have = new Set(db.prepare('select name from pragma_table_info(?)').all(table).map(r => r.name)); + if (!have.size) missing.push(table); + else for (const column of columns) if (!have.has(column)) missing.push(`${table}.${column}`); + } + return missing; +} + +// Seat for a thread title: the lower-cased title equals the seat or starts +// with the seat and a space. Longest seat first, so the most specific wins. +export function seatForTitle(title, seats) { + const lower = title.toLowerCase(); + return [...seats].sort((a, b) => b.length - a.length).find(s => lower === s || lower.startsWith(`${s} `)) ?? null; +} + +// Origin per message id from thread.message-sent events. Any missing table, +// column or unparseable event makes the diagnostic unknown; it decides nothing. +function origins(db, projectId) { + if (missingColumns(db, DIAGNOSTIC).length) return null; + const byMessage = new Map(); + const events = db.prepare(`select e.payload_json, e.metadata_json from orchestration_events e + join projection_threads t on t.thread_id = e.stream_id + where e.event_type = 'thread.message-sent' and t.project_id = ?`).all(projectId); + for (const event of events) { + let payload, metadata; + try { payload = JSON.parse(event.payload_json); metadata = JSON.parse(event.metadata_json); } + catch { return null; } + if (typeof payload?.messageId !== 'string') return null; + byMessage.set(payload.messageId, typeof metadata?.origin?.appVersion === 'string'); + } + return byMessage; +} + +function query(db, root, range, seats) { + const missing = missingColumns(db, REQUIRED); + if (missing.length) throw new SourceError(`T3 schema changed: missing ${missing.join(', ')}`); + // Compared in JavaScript so a declared collation can't loosen the match. + const projects = db.prepare('select project_id, workspace_root from projection_projects where deleted_at is null').all() + .filter(p => p.workspace_root === root); + if (projects.length !== 1) { + throw new SourceError(`T3 has ${projects.length ? 'more than one project' : 'no project'} for ${root}; a project opened through a symlink does not match; ${SKIP}`); + } + const projectId = projects[0].project_id; + const threads = new Map(), excluded = { importedThreads: 0, deletedThreads: 0 }; + for (const t of db.prepare('select thread_id, title, archived_at, deleted_at from projection_threads where project_id = ?').all(projectId)) { + if (typeof t.thread_id !== 'string' || typeof t.title !== 'string') throw new SourceError('T3 thread with a non-text id or title'); + if (t.thread_id.startsWith('import:')) { excluded.importedThreads++; continue; } + if (t.deleted_at !== null) { excluded.deletedThreads++; continue; } + threads.set(t.thread_id, { id: t.thread_id, title: t.title, archived: t.archived_at !== null, seat: seatForTitle(t.title, seats) }); + } + const rows = new Map([...seats, UNMAPPED].map(s => [s, { board: 0, agent: 0, human: 0 }])); + const mentions = new Map(), human = []; + const messages = db.prepare(`select m.message_id, m.thread_id, m.role, m.text, m.created_at from projection_thread_messages m + join projection_threads t on t.thread_id = m.thread_id where t.project_id = ?`).all(projectId); + for (const m of messages) { + const thread = threads.get(m.thread_id); + if (!thread) continue; + const where = `T3 message ${clean(m.message_id)} in thread ${clean(m.thread_id)}`; + if (m.role !== 'user' && m.role !== 'assistant') throw new SourceError(`${where} has an unknown role`); + if (typeof m.text !== 'string') throw new SourceError(`${where} has non-text content`); + if (typeof m.created_at !== 'string' || !Number.isFinite(Date.parse(m.created_at))) throw new SourceError(`${where} has an invalid created_at`); + if (m.role !== 'user') continue; + // A header addressed to its own thread must agree with the title mapping. + const header = t3Header(m.text); + if (header && header.toId === thread.id) { + const to = header.to.toLowerCase(); + if (thread.seat ? to !== thread.seat : seats.includes(to)) { + throw new SourceError(`T3 header conflict: thread ${clean(thread.id)} "${clean(thread.title)}" maps to ${thread.seat ?? 'no seat'}, but a header addresses ${clean(header.to)}`); + } + } + if (!inRange(m.created_at, range)) continue; + const kind = messageKind(m.text), seat = thread.seat ?? UNMAPPED; + rows.get(seat)[kind]++; + if (kind === 'human') human.push(m.message_id); + for (const number of issueNumbers(m.text)) { + if (!mentions.has(number)) mentions.set(number, new Set()); + mentions.get(number).add(seat); + } + } + const byMessage = origins(db, projectId); + const sentThroughApi = byMessage === null ? UNKNOWN : human.filter(id => byMessage.get(id) === false).length; + const noEvent = byMessage === null ? UNKNOWN : human.filter(id => !byMessage.has(id)).length; + const listed = seat => [...threads.values()].filter(t => (t.seat ?? UNMAPPED) === seat) + .sort((a, b) => a.id.localeCompare(b.id)).map(({ id, title, archived }) => ({ id, title, archived })); + const seatRows = seats.map(seat => ({ seat, ...rows.get(seat), threads: listed(seat) })).filter(r => r.threads.length); + return { rows, mentions, excluded, seats: seatRows, unmapped: { ...rows.get(UNMAPPED), threads: listed(UNMAPPED) }, + diagnostic: { humanSentThroughApi: sentThroughApi, humanWithoutEvent: noEvent } }; +} + +// Reads one snapshot of T3's database. Returns the per-seat rows and issue +// mentions the ledger merges with Pi, and the report's `t3` section. +export async function readT3(root, range, { dbPath = defaultT3Path(), isDefault = true } = {}) { + dbPath = path.resolve(dbPath); + await checkPaths(dbPath, isDefault); + const seats = await readSeats(root); + const url = pathToFileURL(dbPath); + url.searchParams.set('mode', 'ro'); + let db, result; + try { + db = new DatabaseSync(url, { readOnly: true, timeout: 5000 }); + db.exec('BEGIN'); + result = query(db, root, range, seats); + db.exec('COMMIT'); + } catch (error) { + if (error instanceof SourceError) throw error; + throw new SourceError(`T3 database cannot be read: ${dbPath} (SQLite ${error.errcode ?? 'error'}); ${SKIP}`); + } finally { + try { if (db?.isTransaction) db.exec('ROLLBACK'); } catch { /* the close below still runs */ } + try { db?.close(); } catch { /* nothing was written */ } + } + const { rows, mentions, ...section } = result; + return { rows, mentions, section: { read: true, database: { path: dbPath, default: isDefault }, ...section } }; +} diff --git a/packages/ledger/tests/ledger.test.mjs b/packages/ledger/tests/ledger.test.mjs index 175f5d5e..8e60b96b 100644 --- a/packages/ledger/tests/ledger.test.mjs +++ b/packages/ledger/tests/ledger.test.mjs @@ -1,6 +1,7 @@ import test from 'node:test'; import assert from 'node:assert/strict'; -import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync, cpSync, symlinkSync } from 'node:fs'; +import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync, cpSync, symlinkSync, realpathSync } from 'node:fs'; +import { DatabaseSync } from 'node:sqlite'; import os from 'node:os'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; @@ -9,13 +10,50 @@ import { dateRange, messageKind, issueNumbers, totalsLine, summarize } from '../ const source = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../src'); const range = dateRange('2026-09-06', '2026-09-12'); +// T3 fixture schema: the live tables, cut to the columns the reader uses plus +// one it doesn't. `text` allows NULL so a non-text row can be tested. +const T3_SCHEMA = ` + create table projection_projects (project_id text primary key, title text not null, workspace_root text not null, deleted_at text); + create table projection_threads (thread_id text primary key, project_id text not null, title text not null, archived_at text, deleted_at text); + create table projection_thread_messages (message_id text primary key, thread_id text not null, role text not null, text, created_at text not null); + create table orchestration_events (sequence integer primary key autoincrement, stream_id text not null, event_type text not null, payload_json text not null, metadata_json text not null);`; +// Writes a T3 database in WAL mode. Threads default to project p1, which is +// the fixture root. Returns the open writer when keepOpen is set. +function t3db(file, { root, projects, threads = [], messages = [], after = [], keepOpen = false }) { + mkdirSync(path.dirname(file), { recursive: true }); + for (const old of [file, `${file}-wal`, `${file}-shm`]) rmSync(old, { force: true }); + const db = new DatabaseSync(file); + db.exec('pragma journal_mode=wal'); db.exec(T3_SCHEMA); + for (const [id, workspace, deleted = null] of projects ?? [['p1', root]]) { + db.prepare('insert into projection_projects values (?, ?, ?, ?)').run(id, 'project', workspace, deleted); + } + for (const t of threads) { + db.prepare('insert into projection_threads values (?, ?, ?, ?, ?)').run(t.id, t.project ?? 'p1', t.title, t.archived ?? null, t.deleted ?? null); + } + for (const m of messages) addMessage(db, m); + for (const sql of after) db.exec(sql); + if (keepOpen) return db; + db.close(); +} +let messageId = 0; +function addMessage(db, { thread, text, role = 'user', at = '2026-09-08T12:00:00Z', origin = 'app' }) { + const id = `m${++messageId}`; + db.prepare('insert into projection_thread_messages values (?, ?, ?, ?, ?)').run(id, thread, role, text, at); + if (origin !== 'none') db.prepare('insert into orchestration_events (stream_id, event_type, payload_json, metadata_json) values (?, ?, ?, ?)') + .run(thread, 'thread.message-sent', JSON.stringify({ messageId: id, threadId: thread, role, text }), JSON.stringify({ origin: origin === 'app' ? { appVersion: '0.0.0' } : {} })); +} const fixtureIssues = [ { number: 1, title: 'First issue', created_at: '2026-09-06T00:00:00Z', closed_at: '2026-09-07T12:00:00Z' }, { number: 2, title: 'Second issue', created_at: '2026-09-06T00:00:00Z', closed_at: null }, ]; function fixture(t) { const root = mkdtempSync(path.join(os.tmpdir(), 'ledger-test-')); - t.after(() => rmSync(root, { recursive: true, force: true })); + // No test opens the real ~/.t3: every CLI run gets this HOME, with an empty + // T3 database at the default path. The CLI's root is a realpath. + const home = mkdtempSync(path.join(os.tmpdir(), 'ledger-home-')); + t.after(() => { rmSync(root, { recursive: true, force: true }); rmSync(home, { recursive: true, force: true }); }); + const defaultDb = path.join(home, '.t3/userdata/state.sqlite'); + t3db(defaultDb, { root: realpathSync(root) }); const put = (name, data) => { const p = path.join(root, name); mkdirSync(path.dirname(p), { recursive: true }); writeFileSync(p, data); return p; }; const git = (args, date = '2026-09-07T00:00:00Z') => execFileSync('git', args, { cwd: root, env: { ...process.env, GIT_AUTHOR_DATE: date, GIT_COMMITTER_DATE: date, GIT_CONFIG_NOSYSTEM: '1', GIT_CONFIG_GLOBAL: '/dev/null' }, stdio: 'pipe' }); git(['init', '-b', 'refactor']); git(['config', 'user.email', 'fixture@example.invalid']); git(['config', 'user.name', 'Fixture']); @@ -30,8 +68,8 @@ function fixture(t) { const entry = (text, timestamp = '2026-09-08T12:00:00Z') => ({ type: 'message', timestamp, message: { role: 'user', content: [{ type: 'text', text }] } }); const logs = [entry('[host:control-board -> host:alice] do #1'), entry('[host:bob -> host:alice] review #2'), entry('build #2'), entry('old #1', '2026-09-05T23:59:59Z'), { type: 'message', timestamp: '2026-09-08T00:00:00Z', message: { role: 'assistant', content: 'not a user #1' } }]; put('.pi/state/alice/sessions/one.jsonl', logs.map(x => JSON.stringify(x)).join('\n') + '\n'); - const run = (args = [], env = {}) => spawnSync(process.execPath, [path.join(root, 'packages/ledger/src/cli.mjs'), '--since', '2026-09-06', '--until', '2026-09-12', ...args], { cwd: root, encoding: 'utf8', env: { ...process.env, PATH: `${path.join(root, 'bin')}:${process.env.PATH}`, ISSUES: path.join(root, 'issues.json'), CALLS: path.join(root, 'calls.jsonl'), ...env } }); - return { root, put, commit, run, entry, logs }; + const run = (args = [], env = {}) => spawnSync(process.execPath, [path.join(root, 'packages/ledger/src/cli.mjs'), '--since', '2026-09-06', '--until', '2026-09-12', ...args], { cwd: root, encoding: 'utf8', env: { ...process.env, HOME: home, PATH: `${path.join(root, 'bin')}:${process.env.PATH}`, ISSUES: path.join(root, 'issues.json'), CALLS: path.join(root, 'calls.jsonl'), ...env } }); + return { root, real: realpathSync(root), home, defaultDb, put, commit, run, entry, logs }; } test('fixture git subjects only, follow-ups and three session kinds', t => { const f = fixture(t), result = f.run(['--json']); @@ -63,7 +101,7 @@ test('missing credentials exit 2, no-issues never calls API and shows unknown', test('empty range gives no rows and zero totals', t => { const f = fixture(t); f.put('issues.json', '[]'); - const result = spawnSync(process.execPath, [path.join(f.root, 'packages/ledger/src/cli.mjs'), '--since', '2027-01-01', '--until', '2027-01-02', '--json'], { encoding: 'utf8', env: { ...process.env, PATH: `${f.root}/bin:${process.env.PATH}`, ISSUES: `${f.root}/issues.json`, CALLS: `${f.root}/calls.jsonl` } }); + const result = spawnSync(process.execPath, [path.join(f.root, 'packages/ledger/src/cli.mjs'), '--since', '2027-01-01', '--until', '2027-01-02', '--json'], { encoding: 'utf8', env: { ...process.env, HOME: f.home, PATH: `${f.root}/bin:${process.env.PATH}`, ISSUES: `${f.root}/issues.json`, CALLS: `${f.root}/calls.jsonl` } }); assert.equal(result.status, 0, result.stderr); const r = JSON.parse(result.stdout); assert.deepEqual(r.issues, []); assert.deepEqual(r.seats, []); assert.ok(Object.values(r.totals).every(n => n === 0)); }); @@ -96,6 +134,13 @@ test('partial or malformed session log refuses with location, not content', t => const f = fixture(t); f.put('.pi/state/alice/sessions/bad.jsonl', '{sensitive'); const r = f.run(); assert.equal(r.status, 1); assert.match(r.stderr, /Malformed session JSON: alice\/bad.jsonl:1/); assert.doesNotMatch(r.stderr, /sensitive/); }); +test('a U+2028 inside a session string is one line, not a malformed record', t => { + const f = fixture(t); + f.put('.pi/state/bob/sessions/sep.jsonl', [f.entry('Jason: one\u2028two #2'), f.entry('[h:alice -> h:bob] ok')].map(x => JSON.stringify(x)).join('\r\n') + '\r\n'); + assert.ok(readFileSync(path.join(f.root, '.pi/state/bob/sessions/sep.jsonl'), 'utf8').includes('\u2028')); + const r = f.run(['--json']); assert.equal(r.status, 0, r.stderr); + assert.deepEqual(JSON.parse(r.stdout).seats[1], { seat: 'bob', board: 0, agent: 1, human: 1 }); +}); test('no sessions is an empty table; symlink source refuses', t => { const f = fixture(t); rmSync(path.join(f.root, '.pi'), { recursive: true }); assert.deepEqual(JSON.parse(f.run(['--json']).stdout).seats, []); @@ -140,7 +185,11 @@ test('T3 header: agent, or board from control-board; anything short of the full assert.equal(messageKind(`Jason here\n[from: ${sage} -> to: ${filbert}]\nquoted`), 'human'); assert.equal(messageKind(` [from: ${sage} -> to: ${filbert}]`), 'human'); assert.equal(messageKind(`[from: sage -> to: filbert]\nno thread ids`), 'human'); - assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=Actionable]`), 'human'); + // Classes match in either case (Gate F). HEAD before the fix called these human. + assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=Actionable]`), 'agent'); + assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=REVIEW-REQUEST]\nreview`), 'agent'); + assert.equal(messageKind('[h:sage -> h:bob class=DECISION] go'), 'agent'); + assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=review_request]`), 'human'); assert.equal(messageKind(`[from: ${sage} -> to: ${filbert}]trailing`), 'human'); assert.equal(messageKind(`[From: ${sage} -> to: ${filbert}]`), 'human'); }); @@ -148,3 +197,218 @@ test('no closed issues with human messages means undefined ratio, not invented z const r = summarize(range, [], [], { rows: [{ seat: 'a', human: 1, board: 0, agent: 0 }], mentions: new Map() }); assert.equal(r.totals.humanMessagesPerClosedIssue, 'unknown'); }); + +// T3 thread source (Gate F, docs/plans/2026-09-26_ledger-t3-source.md). +const T1 = 't-alice', T2 = 't-bob', T3 = 't-sagebrush', T4 = 't-researcher', T5 = 't-discord'; +const header = (from, to, toId, cls = '') => `[from: ${from} (x1) -> to: ${to} (${toId})${cls}]`; +function t3Fixture(t) { + const f = fixture(t); + for (const seat of ['sage', 'researcher']) mkdirSync(path.join(f.root, 'agents', seat), { recursive: true }); + const db = path.join(f.home, 'fixture/t3.sqlite'); + const threads = [ + { id: T1, title: 'Alice' }, { id: T2, title: 'Bob in Claude', archived: '2026-09-09T00:00:00Z' }, + { id: T3, title: 'Sagebrush' }, { id: T4, title: 'Researcher' }, { id: T5, title: 'Discord Bot' }, + { id: 'import:claudeAgent:1', title: 'alice' }, { id: 't-deleted', title: 'Alice', deleted: '2026-09-09T00:00:00Z' }, + { id: 't-other', project: 'p2', title: 'Alice' }, + ]; + const messages = [ + { thread: T1, text: 'Jason: go #1' }, + { thread: T1, text: `${header('sage', 'alice', T1, ' class=REVIEW-REQUEST')}\nreview #2`, origin: 'api' }, + { thread: T1, text: '[h:sage -> h:alice class=DECISION] go', origin: 'api' }, + { thread: T1, text: `${header('control-board', 'alice', T1)}\nbuzz`, origin: 'api' }, + { thread: T1, text: 'outside', at: '2026-09-13T00:00:00Z' }, + { thread: T1, text: 'an answer #9', role: 'assistant', origin: 'none' }, + { thread: T2, text: 'archived still counts #2' }, + { thread: T3, text: 'Sagebrush is not sage' }, + { thread: T3, text: `${header('sage', 'discord', T3)}\nnot a seat role`, origin: 'api' }, + { thread: T4, text: 'research this' }, + { thread: T5, text: '[from: SetSpark coordinator (x1) -> to: Discord Bot (x2)]\nfree text', origin: 'api' }, + { thread: 'import:claudeAgent:1', text: 'imported' }, + { thread: 't-deleted', text: 'deleted' }, + { thread: 't-other', text: 'other project' }, + ]; + const write = (overrides = {}) => t3db(db, { root: f.real, projects: [['p1', f.real], ['p2', '/elsewhere']], threads, messages, ...overrides }); + return { ...f, db, threads, messages, write }; +} +test('T3: seat, archived, unmapped and Researcher threads count; imported, deleted and other-project threads do not', t => { + const f = t3Fixture(t); f.write(); + const result = f.run(['--json', '--t3-db', f.db]); + assert.equal(result.status, 0, result.stderr); + const r = JSON.parse(result.stdout); + assert.deepEqual(r.seats, [ + { seat: 'alice', board: 2, agent: 3, human: 2 }, { seat: 'bob', board: 0, agent: 0, human: 1 }, + { seat: 'researcher', board: 0, agent: 0, human: 1 }, { seat: 't3:unmapped', board: 0, agent: 1, human: 2 }, + ]); + assert.deepEqual(r.pi, [{ seat: 'alice', board: 1, agent: 1, human: 1 }]); + assert.deepEqual(r.t3.database, { path: f.db, default: false }); + assert.deepEqual(r.t3.seats, [ + { seat: 'alice', board: 1, agent: 2, human: 1, threads: [{ id: T1, title: 'Alice', archived: false }] }, + { seat: 'bob', board: 0, agent: 0, human: 1, threads: [{ id: T2, title: 'Bob in Claude', archived: true }] }, + { seat: 'researcher', board: 0, agent: 0, human: 1, threads: [{ id: T4, title: 'Researcher', archived: false }] }, + ]); + assert.deepEqual(r.t3.unmapped, { board: 0, agent: 1, human: 2, threads: [ + { id: T5, title: 'Discord Bot', archived: false }, { id: T3, title: 'Sagebrush', archived: false }] }); + assert.deepEqual(r.t3.excluded, { importedThreads: 1, deletedThreads: 1 }); + // The free-text header counts as human; only the diagnostic shows it was sent through the API. + assert.deepEqual(r.t3.diagnostic, { humanSentThroughApi: 1, humanWithoutEvent: 0 }); + assert.equal(r.totals.humanMessagesPerClosedIssue, 6); + assert.deepEqual(r.issues.map(x => [x.issue, x.seats]), [[1, ['alice']], [2, ['alice', 'bob']]]); + const text = f.run(['--t3-db', f.db]); + assert.equal(text.status, 0, text.stderr); + assert.ok(text.stdout.includes(`T3: read from ${f.db}, not the default`)); + assert.match(text.stdout, /t3:unmapped \| 0 \| 1 \| 2/); +}); +test('T3: the default path is read from HOME and prints no path line; --no-t3 says so', t => { + const f = t3Fixture(t); f.write(); + rmSync(f.defaultDb); cpSync(f.db, f.defaultDb); + const json = JSON.parse(f.run(['--json']).stdout); + assert.deepEqual(json.t3.database, { path: f.defaultDb, default: true }); + assert.equal(json.seats.at(-1).seat, 't3:unmapped'); + const text = f.run(); assert.equal(text.status, 0, text.stderr); assert.doesNotMatch(text.stdout, /^T3:/m); + rmSync(path.join(f.home, '.t3'), { recursive: true }); + const off = f.run(['--no-t3']); assert.equal(off.status, 0, off.stderr); + assert.match(off.stdout, /^T3: not read \(--no-t3\)$/m); + const offJson = JSON.parse(f.run(['--no-t3', '--json']).stdout); + assert.deepEqual(offJson.t3, { read: false }); assert.deepEqual(offJson.seats, [{ seat: 'alice', board: 1, agent: 1, human: 1 }]); + const both = f.run(['--no-t3', '--t3-db', f.db]); assert.equal(both.status, 1); assert.match(both.stderr, /cannot be combined/); + assert.equal(f.run(['--t3-db']).status, 1); +}); +test('T3: a HOME with no database exits 1 and names --no-t3', t => { + const f = fixture(t); rmSync(path.join(f.home, '.t3'), { recursive: true }); + const r = f.run(); assert.equal(r.status, 1); assert.equal(r.stdout, ''); + assert.match(r.stderr, /T3 database unavailable: .*\.t3 is missing or unreadable; use --no-t3/); +}); +test('T3: a file that is not a database exits 1 and names --no-t3', t => { + const f = fixture(t); writeFileSync(f.defaultDb, 'not sqlite'.repeat(100)); + const r = f.run(); assert.equal(r.status, 1); assert.match(r.stderr, /T3 database cannot be read: .*\(SQLite \d+\); use --no-t3/); +}); +test('T3: a seat thread renamed to another seat exits 1 naming thread, title and roles', t => { + const f = t3Fixture(t); + f.write({ messages: [...f.messages, { thread: T2, text: `${header('sage', 'alice', T2, ' class=INFO')}\nfor alice`, origin: 'api' }] }); + const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); + assert.equal(r.stderr.trim(), `T3 header conflict: thread ${T2} "Bob in Claude" maps to bob, but a header addresses alice`); +}); +test('T3: an unmapped thread addressed as a seat exits 1', t => { + const f = t3Fixture(t); + f.write({ messages: [...f.messages, { thread: T3, text: `${header('bob', 'Sage', T3)}\nhi`, origin: 'api' }] }); + const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); + assert.match(r.stderr, /thread t-sagebrush "Sagebrush" maps to no seat, but a header addresses Sage/); +}); +test('T3: a header to another thread id is not cross-checked', t => { + const f = t3Fixture(t); + f.write({ messages: [...f.messages, { thread: T2, text: `${header('sage', 'alice', T1)}\ncopied`, origin: 'api' }] }); + const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr); + assert.equal(JSON.parse(r.stdout).t3.seats[1].agent, 1); +}); +test('T3: no project, or two, for this root exits 1', t => { + const f = t3Fixture(t); + f.write({ projects: [['p1', `${f.real}-link`], ['p2', '/elsewhere']] }); + let r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, /T3 has no project for .*symlink does not match/); + f.write({ projects: [['p1', f.real], ['p2', f.real]] }); + r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, /T3 has more than one project for/); + f.write({ projects: [['p1', f.real], ['p2', f.real, '2026-09-01T00:00:00Z']] }); + assert.equal(f.run(['--t3-db', f.db]).status, 0); +}); +for (const [name, after, pattern] of [ + ['a removed column', ['alter table projection_threads drop column title'], /T3 schema changed: missing projection_threads.title/], + ['a missing table', ['drop table projection_thread_messages'], /T3 schema changed: missing projection_thread_messages$/m], +]) test(`T3: ${name} exits 1 and names it`, t => { + const f = t3Fixture(t); f.write({ after, messages: [] }); + const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, pattern); +}); +for (const [name, message, pattern] of [ + ['an unknown role', { role: 'system' }, /T3 message m\d+ in thread t-alice has an unknown role/], + ['non-text content', { text: null }, /has non-text content/], + ['an unparseable created_at', { at: 'yesterday' }, /has an invalid created_at/], +]) test(`T3: a counted row with ${name} exits 1 without its text`, t => { + const f = t3Fixture(t); + f.write({ messages: [...f.messages, { thread: T1, text: 'secret words', ...message }] }); + const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, pattern); assert.doesNotMatch(r.stderr, /secret/); +}); +test('T3: a missing orchestration_events makes the diagnostic unknown and keeps the counts', t => { + const f = t3Fixture(t); f.write(); + const before = JSON.parse(f.run(['--json', '--t3-db', f.db]).stdout); + f.write({ after: ['drop table orchestration_events'] }); + const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr); + const after = JSON.parse(r.stdout); + assert.deepEqual(after.t3.diagnostic, { humanSentThroughApi: 'unknown', humanWithoutEvent: 'unknown' }); + assert.deepEqual(after.seats, before.seats); assert.deepEqual(after.totals, before.totals); +}); +for (const link of ['.t3', '.t3/userdata', '.t3/userdata/state.sqlite']) test(`T3: a symlink at ~/${link} exits 1`, t => { + const f = fixture(t), target = path.join(f.home, 'real', link); + mkdirSync(path.dirname(target), { recursive: true }); + cpSync(path.join(f.home, link), target, { recursive: true }); + rmSync(path.join(f.home, link), { recursive: true }); symlinkSync(target, path.join(f.home, link)); + const r = f.run(); assert.equal(r.status, 1); assert.match(r.stderr, new RegExp(`${link.replaceAll('.', '\\.')} is a symlink; use --no-t3`)); +}); +test('T3: with --t3-db, a symlinked file or directory exits 1', t => { + const f = t3Fixture(t); f.write(); + const file = path.join(f.home, 'file-link.sqlite'); symlinkSync(f.db, file); + let r = f.run(['--t3-db', file]); assert.equal(r.status, 1); assert.match(r.stderr, /file-link.sqlite is a symlink/); + const dir = path.join(f.home, 'dir-link'); symlinkSync(path.dirname(f.db), dir); + r = f.run(['--t3-db', path.join(dir, 't3.sqlite')]); assert.equal(r.status, 1); assert.match(r.stderr, /dir-link is a symlink/); +}); + +// WAL states. The CLI reads with mode=ro; it may create -wal and -shm but must +// never change the main file. +const sha = file => execFileSync('sha256sum', [file], { encoding: 'utf8' }).split(' ')[0]; +const humans = r => JSON.parse(r.stdout).t3.seats.find(s => s.seat === 'alice').human; +const asRoot = process.getuid?.() === 0; +function killedWriter(db) { + // A writer that commits into the WAL and dies without a checkpoint. + const code = `const { DatabaseSync } = require('node:sqlite'); const db = new DatabaseSync(${JSON.stringify(db)}); + db.exec('pragma wal_autocheckpoint=0'); + db.prepare("insert into projection_thread_messages values ('late', 't-alice', 'user', 'late human', '2026-09-08T13:00:00Z')").run(); + process.kill(process.pid, 'SIGKILL');`; + const r = spawnSync(process.execPath, ['-e', code]); + assert.equal(r.signal, 'SIGKILL'); + rmSync(`${db}-shm`); +} +function inReadOnlyDir(dir, check) { + execFileSync('chmod', ['0555', dir]); + try { check(); } finally { execFileSync('chmod', ['0755', dir]); } +} +test('T3 WAL: the newest message only in -wal, writer attached, is counted', t => { + const f = t3Fixture(t), writer = f.write({ keepOpen: true }); + t.after(() => writer.close()); + writer.exec('pragma wal_autocheckpoint=0'); + addMessage(writer, { thread: T1, text: 'newest', at: '2026-09-08T13:00:00Z' }); + const main = sha(f.db); + const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr); + assert.equal(humans(r), 2); assert.equal(sha(f.db), main); +}); +test('T3 WAL: stopped cleanly, counts are correct and the main file is unchanged', t => { + const f = t3Fixture(t); f.write(); + assert.throws(() => readFileSync(`${f.db}-wal`)); + const main = sha(f.db); + const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr); + assert.equal(humans(r), 1); assert.equal(sha(f.db), main); +}); +test('T3 WAL: -wal without -shm in a writable directory is read', t => { + const f = t3Fixture(t); f.write(); killedWriter(f.db); + const main = sha(f.db); + const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr); + assert.equal(humans(r), 2); assert.equal(sha(f.db), main); +}); +test('T3 WAL: -wal without -shm in a read-only directory exits 1', { skip: asRoot && 'mode bits do not bind root' }, t => { + const f = t3Fixture(t); f.write(); killedWriter(f.db); + inReadOnlyDir(path.dirname(f.db), () => { + const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, /cannot be read: .*\(SQLite 14\); use --no-t3/); + }); +}); +test('T3 WAL: stopped cleanly in a read-only directory exits 1', { skip: asRoot && 'mode bits do not bind root' }, t => { + const f = t3Fixture(t); f.write(); + inReadOnlyDir(path.dirname(f.db), () => { + const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, /cannot be read: .*\(SQLite 1544\); use --no-t3/); + }); +}); +test('T3: a lock held past the 5 s busy timeout exits 1 and names --no-t3', t => { + const f = t3Fixture(t), writer = f.write({ keepOpen: true }); + t.after(() => writer.close()); + writer.exec('pragma locking_mode=exclusive'); writer.exec('begin exclusive'); + addMessage(writer, { thread: T1, text: 'held' }); + const started = Date.now(), r = f.run(['--t3-db', f.db]); + writer.exec('commit'); + assert.equal(r.status, 1); assert.match(r.stderr, /cannot be read: .*\(SQLite 5\); use --no-t3/); + assert.ok(Date.now() - started >= 4500, 'the reader waited for the busy timeout'); +});