From ffc22c04c67b54cb1b4e23a3556d8231de23df73 Mon Sep 17 00:00:00 2001 From: Jason Woltje Date: Sat, 26 Sep 2026 16:04:56 -0500 Subject: [PATCH] docs(ledger): Gate F T3 thread source brief R2, Filbert approved (#1506) Brief e8300cb6 (Darkwing), review bb02d8d3 (Filbert, approve with three nits), R1 record and R1-to-R2 diff. Lead rulings in item 14. Co-Authored-By: Claude Opus 5.5 --- .../work/ledger-t3-source/manifest.sha256 | 5 + .../work/ledger-t3-source/r1-to-r2.diff | 419 ++++++++++++++++++ agents/darkwing/work/ledger-t3-source/r1.md | 198 +++++++++ .../work/ledger-t3-source/r2-to-r3.diff | 74 ++++ agents/darkwing/work/ledger-t3-source/r2.md | 306 +++++++++++++ .../ledger-t3-source-review-2026-09-26.md | 317 +++++++++++++ docs/plans/2026-09-26_lead-decisions.md | 6 + docs/plans/2026-09-26_ledger-t3-source.md | 310 +++++++++++++ 8 files changed, 1635 insertions(+) create mode 100644 agents/darkwing/work/ledger-t3-source/manifest.sha256 create mode 100644 agents/darkwing/work/ledger-t3-source/r1-to-r2.diff create mode 100644 agents/darkwing/work/ledger-t3-source/r1.md create mode 100644 agents/darkwing/work/ledger-t3-source/r2-to-r3.diff create mode 100644 agents/darkwing/work/ledger-t3-source/r2.md create mode 100644 agents/filbert/work/ledger-t3-source-review-2026-09-26.md create mode 100644 docs/plans/2026-09-26_ledger-t3-source.md diff --git a/agents/darkwing/work/ledger-t3-source/manifest.sha256 b/agents/darkwing/work/ledger-t3-source/manifest.sha256 new file mode 100644 index 00000000..c172ee41 --- /dev/null +++ b/agents/darkwing/work/ledger-t3-source/manifest.sha256 @@ -0,0 +1,5 @@ +08959a05574264e4f8243a90af94746e73a2fde3706f22e38f4ff1105b7a45a8 agents/darkwing/work/ledger-t3-source/r1.md +e8300cb6abea70819aba7cf10040d19b4d6019b5663c37203209537a5f10ee62 agents/darkwing/work/ledger-t3-source/r2.md +f3c05c1b4d28a419ab817621e15708b147f71dff588664980e22696eaafbc342 docs/plans/2026-09-26_ledger-t3-source.md +aa4740ae5d045aa12971af5de36a5107805bd31da839aae4e4d398a818d471fc agents/darkwing/work/ledger-t3-source/r1-to-r2.diff +4128375121673e0a49ef6789390503ff7395ba59f87bfc450764ea56b536c5b2 agents/darkwing/work/ledger-t3-source/r2-to-r3.diff diff --git a/agents/darkwing/work/ledger-t3-source/r1-to-r2.diff b/agents/darkwing/work/ledger-t3-source/r1-to-r2.diff new file mode 100644 index 00000000..2b6822c3 --- /dev/null +++ b/agents/darkwing/work/ledger-t3-source/r1-to-r2.diff @@ -0,0 +1,419 @@ +--- r1.md ++++ docs/plans/2026-09-26_ledger-t3-source.md +@@ -1,8 +1,11 @@ + # Ledger: a read-only T3 thread source for Table 2 (Gate F brief) + +-Brief only, no code. Darkwing wrote it on 2026-09-26 at Sage's request. Filbert +-reviews it, and Jason sees it on the decision sheet before anyone builds it. +-Issue #1506. ++Brief only, no code. Darkwing wrote it on 2026-09-26 at Sage's request, issue ++#1506. R1 (sha256 08959a05) went to Filbert, whose review asked for ++revisions: `agents/filbert/work/ledger-t3-source-review-2026-09-26.md`, sha256 ++19dda29a. This is R2. It takes every finding, and it records Sage's rulings ++on the three open questions. Section 1 has one measurement that differs from ++the review. + + ## Why + +@@ -18,8 +21,8 @@ + ## Where T3 keeps messages + + T3 keeps its state in one SQLite database, `~/.t3/userdata/state.sqlite`, in +-WAL mode (`state.sqlite-wal` and `state.sqlite-shm` sit beside it). Three +-projection tables are enough: ++WAL mode (`state.sqlite-wal` and `state.sqlite-shm` sit beside it). The counts ++need three projection tables: + + - `projection_projects`: `project_id`, `workspace_root`, `deleted_at`. + - `projection_threads`: `thread_id`, `project_id`, `title`, `archived_at`, +@@ -27,51 +30,72 @@ + - `projection_thread_messages`: `message_id` (primary key), `thread_id`, + `role` (`user` or `assistant`), `text`, `created_at` (ISO UTC). + +-One more table is optional. In `orchestration_events`, each ++The JSON diagnostic reads one more. In `orchestration_events`, each + `thread.message-sent` event carries `metadata_json.origin`. Messages typed in + the T3 app carry an `appVersion` there. Messages sent through T3's API or MCP +-tools, which is how seats talk to each other, don't. See the cross-check below. ++tools, which is how seats talk to each other, don't. + + The same directory also holds `secrets/`, `clerk-tokens.json` and other + settings files. The reader opens `state.sqlite` and nothing else, and it + selects named columns only, never `*`. + +-## Reading it, with T3 running or not ++## 1. Reading it, with T3 running or not + +-The file stays on disk whether T3 runs or not. The reader opens it with Node's +-built-in `node:sqlite` (`DatabaseSync`, `file:?mode=ro`, `readOnly: +-true`). That needs no dependency, and Node 26.8.1 prints no warning for it. I +-read the live database this way today, while T3 was running, with no errors +-and no locks. A WAL reader sees every committed message, including those still +-in the `-wal` file. +- +-Two rules: +-- Never open with `immutable=1` and never copy the file. Both skip the WAL +- and silently lose the newest messages. A copy of the three files is also +- not atomic. +-- If T3 stopped uncleanly and left a `-wal` without its `-shm`, a read-only +- connection may be unable to rebuild the index. If the open fails, the +- ledger reports it and refuses. I have not tested this case or the fully +- stopped case. Both are acceptance checks below. ++The reader uses Node's built-in `node:sqlite` (`DatabaseSync`). That needs no ++dependency, and Node 26.8.1 (SQLite 3.53.4) prints no warning for it. + +-## Jason or agent ++- **URI.** Build it with `pathToFileURL(dbPath)` and set `mode=ro` through ++ `searchParams`, then pass `readOnly: true`. A `?`, `#` or `%` in the home ++ path would break a string-built URI. ++- **One snapshot.** Run every query, from the schema checks through the ++ diagnostic, inside one `BEGIN` … `COMMIT`. In autocommit mode each ++ statement sees its own snapshot while T3 writes between them. ++- **Busy timeout.** Set `DatabaseSync`'s `timeout` to 5 s. A transient ++ `SQLITE_BUSY` during a T3 checkpoint then waits instead of failing. A busy ++ error after the timeout exits 1 like any open failure. ++- **No `immutable=1` and no copy.** Both lose the WAL. Filbert found worse ++ than lost messages: with a table created inside the WAL, `immutable=1` ++ fails with `no such table`. ++ ++What happens on disk. Filbert and I both tested these in scratch ++directories: ++ ++| State | Directory writable | Result | ++|---|---|---| ++| T3 running, writer attached, newest rows only in `-wal` | yes | reads them | ++| `-wal` without `-shm` (writer killed, `-shm` removed) | yes | reads the WAL rows and creates `-shm` | ++| `-wal` without `-shm` | no | open fails, SQLite 14 | ++| T3 stopped cleanly, no `-wal` or `-shm` | yes | reads, then leaves an empty `-wal` and a 32 KiB `-shm` | ++| T3 stopped cleanly | no | fails, SQLite 1544 "attempt to write a readonly database" | ++ ++In every case the main file's bytes stayed the same. The last row is where ++Filbert and I differ. His review says the stopped-case read works with the ++directory read-only. In my run it failed with and without the read ++transaction. The build's test settles it. Either way a failed open is exit 1. ++ ++So the accurate claim: the reader never writes the main database file. Like ++any SQLite connection, it may create or update `-wal` and `-shm` beside it ++and takes read locks in `-shm`. T3 opens normally afterwards. ++ ++## 2. Jason or agent + + Reuse the 6a rule. The first line of `text` decides: a T3 header or the tmux + preamble counts as agent, `control-board` as the sender counts as board, and + anything else counts as human. Messages with role `user` count; assistant + messages don't. + +-6a has a defect this source would expose. Its regex allows only a lowercase +-class (`class=[a-z-]+`). Seats send uppercase classes: Sage's DECISION, INFO, +-REVIEW-REQUEST and REVIEW-NOTE, and my own REVIEW-REQUEST. In this project's +-threads, 16 real agent headers fail on that alone and would count as human. +-The fix is to make the class match case-insensitive. It belongs in this build +-or just before it, reviewed with it. The ms-communications table lists +-lowercase names, so the fix follows what seats send, not the table. ++The class fix rides in this build (Sage's ruling). HEAD's ++`packages/ledger/src/ledger.mjs:81` (tmux) and `:83` (T3) both allow only ++`class=[a-z-]+`. Both become case-insensitive. Seats send uppercase classes: ++Sage's DECISION, INFO, REVIEW-REQUEST and REVIEW-NOTE, and my own ++REVIEW-REQUEST. In this project's threads 16 real agent headers failed on ++that alone at 20:54Z. The ms-communications table lists lowercase names, so ++the fix follows what seats send, not the table. + + Cross-check, read at 2026-09-26T20:54Z for the mosaic-stack project (209 + user messages outside imported and deleted threads, every one with its +-`thread.message-sent` event): ++`thread.message-sent` event). Filbert's later read agreed, plus messages sent ++since. + + | T3 origin | Header matches 6a | Count | + |---|---|---| +@@ -83,110 +107,193 @@ + No message typed in the app carries a header, and every API message in this + project carries one of the three forms. The 14 free-text ones are older + Discord Bot thread headers such as `[from: SetSpark coordinator (…) -> to: +-Discord Bot (…)]`, written before the guide fixed the format. With the class +-fix they still count as human. That's 14 wrong human counts, all dated +-2026-09-17 to 2026-09-22. +- +-Recommendation: the header rule decides, as Sage asked. The reader also +-reports one diagnostic number, not used in any table: user messages the rule +-calls human that T3 recorded as sent through the API. That count is how the +-uppercase-class bug showed up, and it would catch the next format drift. The +-origin field is T3's internal metadata, not a documented contract, so it +-shouldn't decide anything. I'd make it JSON only, so Table 2's layout stays +-the same. +- +-## Thread to seat +- +-A thread counts for this checkout only if its project's `workspace_root` is +-the ledger's repository root. That is `/mnt/storage/src/mosaic-stack`, project +-`34050c07`. +- +-Thread IDs change whenever Jason starts a new thread for a seat, so there's no +-fixed map. T3-AGENT-COMMS.md already names threads after the seat ("Darkwing", +-"Sage", "Dewey in Claude"). Proposed rule: a thread belongs to seat `` when +-`` is a real directory under `agents/` and the lower-cased title equals +-`` or starts with `` followed by a space. Several threads can map to one +-seat. Their counts add up, as several Pi session files already do. ++Discord Bot (…)]`, written before the guide fixed the format. Sage ruled they ++stay as recorded: they count as human, dated 2026-09-17 to 2026-09-22. ++ ++The header rule decides. The JSON also carries one diagnostic that feeds no ++table or total: user messages the rule calls human that T3 recorded as sent ++through the API. That number exposed the class bug and would catch the next ++format drift. `origin` is T3's internal metadata, not a documented contract, ++so it decides nothing. If `orchestration_events` or a column it needs is ++missing, the diagnostic reads `unknown` and the report goes on (Sage's ++ruling on F5). Missing tables the counts depend on still exit 1. ++ ++## 3. Thread to seat ++ ++**Project.** A thread counts for this checkout only if its project's ++`workspace_root` equals the ledger's repository root, byte for byte. The CLI ++already takes that root from the realpath of its own URL, today ++`/mnt/storage/src/mosaic-stack`, project `34050c07`. So a T3 project opened ++through the compatibility symlink `~/src/mosaic-stack-dev-test` doesn't ++match, and "no project row" is the right refusal. The README says so. ++ ++**Title rule.** Thread IDs change whenever Jason starts a new thread for a ++seat, so there's no fixed map. T3-AGENT-COMMS.md already names threads after ++the seat ("Darkwing", "Sage", "Dewey in Claude"). A thread belongs to seat ++`` when `` is a real directory under `agents/` and the lower-cased ++title equals `` or starts with `` followed by a space. So "Sagebrush" ++stays unmapped. Several threads can map to one seat, and their counts add ++up, as several Pi session files already do. + + Today that maps Sage, Darkwing, Filbert, Dewey and Rocko (one thread each, +-created 2026-09-26), plus "Darkwing in Claude" (archived) and "Dewey in +-Claude". Three threads map to no seat. Two are imported and excluded anyway +-("FINDINGS.md review" and "[dragon-lin:darkwing -> …"). The third is +-"Discord Bot" with 68 user messages: 54 without a header, and the 14 +-free-text headers above. The guide's own advice, titles like `review: +-`, will produce more unmapped threads. +- +-Unmapped threads go in one Table 2 row, `t3:unmapped`, so Jason's messages +-there still count toward the Human column and the human-per-closed ratio. The +-other choice is to drop them, which would hide those 54 headerless prompts. +-That is Jason's decision. I recommend the row. +- +-A seat's row sums its Pi and T3 counts. JSON splits them by source. Nothing is +-counted twice: every T3 session today runs on `claudeAgent` or `codex`, which +-don't write `.pi/state`, and Filbert found no T3 header in any Pi log. ++created 2026-09-26, titles set by hand), plus "Darkwing in Claude" (archived) ++and "Dewey in Claude". Researcher has a directory and no thread. Three ++threads map to no seat. Two are imported and excluded anyway ("FINDINGS.md ++review" and "[dragon-lin:darkwing -> …"). The third is "Discord Bot" with 68 ++user messages: 54 without a header, and the 14 free-text headers. ++ ++Titles are current state, and T3 can write them itself. They go wrong three ++ways. T3 auto-titles an unnamed thread from Jason's first prompt, so "Rocko ++review of the plan" maps to rocko. A rename moves the whole history to ++another row. A seat thread titled for a topic drops into `t3:unmapped`. ++None of this changes the Human total or the human-per-closed ratio. It only ++moves counts between rows, but Gate F reads one seat's row. ++ ++**Header cross-check.** The headers already say which seat a thread belongs ++to. For every user message whose header matches the fixed 6a rule and whose ++`to:` id equals the message's own `thread_id`: ++- in a mapped thread, the `to:` role, lower-cased, must equal that thread's ++ seat; ++- in an unmapped thread, the `to:` role must not be a seat name. ++ ++A conflict exits 1 and names the thread id, its title and both roles. A ++header whose `to:` id is some other thread is not checked. The check reads ++message text only, not T3 metadata. In a live read at 21:02Z every header ++agreed: all 104 addressed to their own thread carried the full thread id and ++named that thread's seat (Sage 40, Darkwing 15, Filbert 18, Dewey 15, Rocko ++16). ++ ++It catches a seat thread renamed to another seat or to a topic, once any ++agent writes to it. It also catches an auto-titled thread that agents ++address by a different seat. It misses a thread no agent ever writes to. ++Such a thread can only add human counts to a seat's row, never hide them, so ++for Gate F it errs toward a visible failure. The README says so. ++ ++**Unmapped row.** Unmapped threads go in one Table 2 row, `t3:unmapped` ++(Sage's ruling), so their human messages still reach the Human column and ++the human-per-closed ratio. ++ ++**Mapping in the JSON.** For each seat, the T3 thread ids and titles that ++made its row, and the unmapped thread ids and titles. Anyone checking a Gate ++F result can then see which threads the row came from. ++ ++A seat's row sums its Pi and T3 counts, and the JSON splits them by source. ++Nothing is counted twice. Every T3 session today runs on `claudeAgent` or ++`codex`, which don't write `.pi/state`, and Filbert found no T3 header in any ++Pi log (6a record). + +-Excluded, with the reason stated in the README: ++**Excluded,** with the reason stated in the README: + - Imported threads (`thread_id` starting `import:`, events marked + `historyImport`). They are partial copies of Claude Code sessions, not T3 + traffic: 55 user messages in two threads here. + - Deleted threads (`deleted_at` set). Across all projects there are 3, with + 3 messages. Archived threads count. +- +-## What fails closed +- +-With the T3 source on, each of these refuses the report with exit 1, the +-code the ledger already uses for unreadable session evidence. The report +-never falls back to Pi logs alone. As with `--no-issues`, `--no-t3` turns the +-source off, and the report then says T3 was not read. +-- The database is missing, unreadable, or won't open read-only (including +- the `-wal` without `-shm` case). This differs from the Pi reader, which +- treats a missing `.pi` as no messages. A missing Pi directory means no Pi +- seats ran here. A missing T3 database on this host means the path or T3 +- changed, and a silent zero is the failure Gate F exists to prevent. +-- A required table or column is missing. The reader checks `PRAGMA ++- Threads in other T3 projects. Live, there is a project at `/home/jwoltje` ++ and a deleted one at `/mnt/storage/src`. A thread in either could work on ++ this repository and would not be counted. The workspace-root rule is still ++ the right one, but the README names this blind spot. ++ ++## 4. What fails closed ++ ++The source is on by default (Sage's ruling). `--no-t3` turns it off, and the ++report then says T3 was not read. `--t3-db ` reads another database ++file instead of `~/.t3/userdata/state.sqlite`. It exists for fixtures and ++gets the same checks. ++ ++Each of these refuses the report with exit 1, the code the ledger already ++uses for unreadable session evidence. The report never falls back to Pi logs ++alone. Where the database is missing or won't open, the message names ++`--no-t3`. ++- The database is missing or unreadable, or won't open read-only. That ++ includes a directory that isn't writable when SQLite needs to create ++ `-shm`, and a busy error after the timeout. The Pi reader treats a missing ++ `.pi` as no messages, and this departs from it on purpose. A missing Pi ++ directory means no Pi seats ran here. A missing T3 database on this host ++ means the path or T3 changed, and a silent zero is the failure Gate F ++ exists to prevent. ++- `~/.t3`, `~/.t3/userdata` or `state.sqlite` is a symlink. With `--t3-db`, ++ the file and its directory are checked. The Pi reader checks every ++ ancestor too, but it skips symlinked entries. Skipping one named file ++ would be another silent zero, so this reader refuses. ++- A table or column the counts need is missing. The reader checks `PRAGMA + table_info` and names what's missing. This catches a T3 upgrade that + changes the schema. + - No project row, or more than one non-deleted row, for this repository root. + - A counted row has a bad `role`, non-string `text`, or a `created_at` that + doesn't parse. The Pi reader already refuses malformed JSONL and bad + timestamps the same way. +-- `state.sqlite` or `~/.t3/userdata` is a symlink. The Pi reader skips +- symlinked entries instead. For one named file, skipping would be another +- silent zero, so this reader refuses. +- +-The source never writes to the database. It never reads other files in +-`~/.t3`, and it passes no message text beyond `messageKind` and +-`issueNumbers`, the same rule as for Pi logs. The one outside effect is +-SQLite's own: a WAL reader takes read locks in the `-shm` file, as T3's own +-connections do. +- +-## Decisions for Jason +- +-1. The source is on by default, with `--no-t3` to turn it off. The other +- choice is off by default with `--t3` to turn it on. I recommend on by +- default, because Gate F exists to count these messages. +-2. Unmapped threads get a `t3:unmapped` row. The other choice is to drop +- them. I recommend the row. +-3. The 14 free-text headers from 09-17 to 09-22 stay counted as human. Fixing +- them would mean loosening the header grammar for history only, and I don't +- recommend it. +- +-## Acceptance for the build +- +-- Fixture databases built with `node:sqlite` in a temp dir, in WAL mode: +- seat threads and an unmapped thread; imported, deleted and archived +- threads; all three header forms, uppercase classes included; a message +- outside the date range; another project with the same seat titles. +-- Each fail-closed case above has its own test, including a schema column +- removed and `-wal` without `-shm`. One test opens a database whose newest +- message is still in the WAL and counts it. Another reads a database closed +- cleanly with no writer attached, which is the T3-stopped case. +-- The class fix is proven against HEAD's `messageKind`: an uppercase class +- counts as agent after the fix and as human before it. ++- A header conflicts with the title mapping (section 3). ++ ++The reader never reads other files in `~/.t3`. It passes no message text ++beyond `messageKind`, `issueNumbers` and the header's `to:` role and id, the ++same rule as for Pi logs. ++ ++## 5. Rulings ++ ++Sage ruled on the three questions R1 put to Jason, as lead calls: ++1. The source is on by default. A missing or unreadable database exits 1, ++ and the message names `--no-t3`. ++2. Unmapped threads get the `t3:unmapped` row. ++3. The 14 free-text headers stay as recorded. They show only in the JSON ++ diagnostic. ++ ++Sage also ruled that the class fix rides in this build, and that a missing ++diagnostic table reads `unknown` (F5). ++ ++## 6. Acceptance for the build ++ ++**No test opens the real `~/.t3`.** Both places in ++`packages/ledger/tests/ledger.test.mjs` that spawn `cli.mjs` (the shared ++`run()` helper and the direct `spawnSync` at line 66) set `HOME` to the ++fixture's temp directory. A test that forgets `--t3-db` or `--no-t3` then ++finds no database and fails closed. The existing tests aren't about T3. Each ++gets an empty fixture database at the fixture `HOME`'s default path, with ++one project row for the fixture root. So they run with the source on, and ++their expected rows don't change. One test asserts that a `HOME` with no ++database exits 1 and names `--no-t3`. ++ ++Fixture databases are built with `node:sqlite` in a temp directory, in WAL ++mode: ++- seat threads and an unmapped thread; imported, deleted and archived ++ threads; a message outside the date range; ++- all three header forms, with uppercase classes in both the tmux preamble ++ and the T3 header; ++- a thread with the same seat title in another project; ++- a seat thread renamed to another seat, with an agent header to its own ++ id, which exits 1; ++- a "Sagebrush" title, which stays unmapped; ++- a thread titled "Researcher", which maps to the seat that has no thread ++ live. ++ ++WAL states, each with its own test: ++- The newest message is only in `-wal`, with the writer still attached (the ++ live-T3 case). It is counted. ++- T3 stopped: the database closed cleanly with no writer. Counts are ++ correct, and the main file's bytes are unchanged afterwards. ++- `-wal` without `-shm` in a writable directory: made by a child writer with ++ `wal_autocheckpoint=0` that is SIGKILLed, then `-shm` deleted. The WAL ++ rows are counted. ++- `-wal` without `-shm` in a directory that isn't writable: exit 1, naming ++ `--no-t3`. Skipped when the tests run as root, where the mode bits don't ++ bind. ++- The stopped case in a directory that isn't writable: exit 1, naming ++ `--no-t3`, as I measured it. If the build reads there instead, the builder ++ changes this test to assert correct counts and records the correction in ++ the BUILD-LOG entry. Skipped as root too. ++ ++Also: ++- Each other fail-closed case in section 4 has its own test, including a ++ removed schema column and each symlink. ++- A missing `orchestration_events` gives `unknown` for the diagnostic and ++ the same counts. ++- The class fix is proven against HEAD's `messageKind`. An uppercase class ++ in either preamble counts as agent after the fix and as human before it. ++- The JSON lists each seat's threads and the unmapped threads. + - A read against the live database gives the counts in this brief, allowing +- for messages sent since. +-- The ledger README's counting rules name the new source, the mapping rule +- and the exclusions. ++ for messages sent since. It exits 0 with no header conflict. ++- The ledger README's counting rules name the new source, both flags, the ++ mapping rule and the header check, and the exclusions. That includes the ++ symlinked-checkout case and the other-project blind spot. + - No suite runs the ledger tests, so the BUILD-LOG entry names the test file. + + ## Not in scope +@@ -194,5 +301,6 @@ + - Claude Code transcripts (`~/.claude/projects`) and Codex sessions + (`~/.codex/sessions`). T3's database already holds every message T3 + delivered, so those files would only duplicate it. +-- Any write to T3, any T3 API call, or anything that needs T3 running. ++- Any T3 API call, anything that needs T3 running, and any write beyond ++ SQLite's own `-wal` and `-shm` handling. + - Fixing the two tmux misclassifications Filbert found in 6a. diff --git a/agents/darkwing/work/ledger-t3-source/r1.md b/agents/darkwing/work/ledger-t3-source/r1.md new file mode 100644 index 00000000..d2593fb3 --- /dev/null +++ b/agents/darkwing/work/ledger-t3-source/r1.md @@ -0,0 +1,198 @@ +# Ledger: a read-only T3 thread source for Table 2 (Gate F brief) + +Brief only, no code. Darkwing wrote it on 2026-09-26 at Sage's request. Filbert +reviews it, and Jason sees it on the decision sheet before anyone builds it. +Issue #1506. + +## Why + +Table 2 counts user messages per seat from `.pi/state//sessions/*.jsonl` +only. Development seats now run in T3 on the Claude and Codex harnesses, so +their prompts, Jason's included, never reach a Pi log. Today the Human column +can't see T3 at all, and the zero it shows for T3 seats means "no source", not +"no human prompts". 6a (ef0020ad) taught `messageKind` the T3 header, but no +source the ledger reads contains one. Gate F (QUEUE row 6) passes when +Filbert's item closes with zero human messages from Jason. While the ledger +can't see T3, a zero there proves nothing. + +## Where T3 keeps messages + +T3 keeps its state in one SQLite database, `~/.t3/userdata/state.sqlite`, in +WAL mode (`state.sqlite-wal` and `state.sqlite-shm` sit beside it). Three +projection tables are enough: + +- `projection_projects`: `project_id`, `workspace_root`, `deleted_at`. +- `projection_threads`: `thread_id`, `project_id`, `title`, `archived_at`, + `deleted_at`. +- `projection_thread_messages`: `message_id` (primary key), `thread_id`, + `role` (`user` or `assistant`), `text`, `created_at` (ISO UTC). + +One more table is optional. In `orchestration_events`, each +`thread.message-sent` event carries `metadata_json.origin`. Messages typed in +the T3 app carry an `appVersion` there. Messages sent through T3's API or MCP +tools, which is how seats talk to each other, don't. See the cross-check below. + +The same directory also holds `secrets/`, `clerk-tokens.json` and other +settings files. The reader opens `state.sqlite` and nothing else, and it +selects named columns only, never `*`. + +## Reading it, with T3 running or not + +The file stays on disk whether T3 runs or not. The reader opens it with Node's +built-in `node:sqlite` (`DatabaseSync`, `file:?mode=ro`, `readOnly: +true`). That needs no dependency, and Node 26.8.1 prints no warning for it. I +read the live database this way today, while T3 was running, with no errors +and no locks. A WAL reader sees every committed message, including those still +in the `-wal` file. + +Two rules: +- Never open with `immutable=1` and never copy the file. Both skip the WAL + and silently lose the newest messages. A copy of the three files is also + not atomic. +- If T3 stopped uncleanly and left a `-wal` without its `-shm`, a read-only + connection may be unable to rebuild the index. If the open fails, the + ledger reports it and refuses. I have not tested this case or the fully + stopped case. Both are acceptance checks below. + +## Jason or agent + +Reuse the 6a rule. The first line of `text` decides: a T3 header or the tmux +preamble counts as agent, `control-board` as the sender counts as board, and +anything else counts as human. Messages with role `user` count; assistant +messages don't. + +6a has a defect this source would expose. Its regex allows only a lowercase +class (`class=[a-z-]+`). Seats send uppercase classes: Sage's DECISION, INFO, +REVIEW-REQUEST and REVIEW-NOTE, and my own REVIEW-REQUEST. In this project's +threads, 16 real agent headers fail on that alone and would count as human. +The fix is to make the class match case-insensitive. It belongs in this build +or just before it, reviewed with it. The ms-communications table lists +lowercase names, so the fix follows what seats send, not the table. + +Cross-check, read at 2026-09-26T20:54Z for the mosaic-stack project (209 +user messages outside imported and deleted threads, every one with its +`thread.message-sent` event): + +| T3 origin | Header matches 6a | Count | +|---|---|---| +| typed in the app (has `appVersion`) | no | 99 | +| sent through the API (no `appVersion`) | yes | 80 | +| sent through the API | no, uppercase class | 16 | +| sent through the API | no, free-text roles | 14 | + +No message typed in the app carries a header, and every API message in this +project carries one of the three forms. The 14 free-text ones are older +Discord Bot thread headers such as `[from: SetSpark coordinator (…) -> to: +Discord Bot (…)]`, written before the guide fixed the format. With the class +fix they still count as human. That's 14 wrong human counts, all dated +2026-09-17 to 2026-09-22. + +Recommendation: the header rule decides, as Sage asked. The reader also +reports one diagnostic number, not used in any table: user messages the rule +calls human that T3 recorded as sent through the API. That count is how the +uppercase-class bug showed up, and it would catch the next format drift. The +origin field is T3's internal metadata, not a documented contract, so it +shouldn't decide anything. I'd make it JSON only, so Table 2's layout stays +the same. + +## Thread to seat + +A thread counts for this checkout only if its project's `workspace_root` is +the ledger's repository root. That is `/mnt/storage/src/mosaic-stack`, project +`34050c07`. + +Thread IDs change whenever Jason starts a new thread for a seat, so there's no +fixed map. T3-AGENT-COMMS.md already names threads after the seat ("Darkwing", +"Sage", "Dewey in Claude"). Proposed rule: a thread belongs to seat `` when +`` is a real directory under `agents/` and the lower-cased title equals +`` or starts with `` followed by a space. Several threads can map to one +seat. Their counts add up, as several Pi session files already do. + +Today that maps Sage, Darkwing, Filbert, Dewey and Rocko (one thread each, +created 2026-09-26), plus "Darkwing in Claude" (archived) and "Dewey in +Claude". Three threads map to no seat. Two are imported and excluded anyway +("FINDINGS.md review" and "[dragon-lin:darkwing -> …"). The third is +"Discord Bot" with 68 user messages: 54 without a header, and the 14 +free-text headers above. The guide's own advice, titles like `review: +`, will produce more unmapped threads. + +Unmapped threads go in one Table 2 row, `t3:unmapped`, so Jason's messages +there still count toward the Human column and the human-per-closed ratio. The +other choice is to drop them, which would hide those 54 headerless prompts. +That is Jason's decision. I recommend the row. + +A seat's row sums its Pi and T3 counts. JSON splits them by source. Nothing is +counted twice: every T3 session today runs on `claudeAgent` or `codex`, which +don't write `.pi/state`, and Filbert found no T3 header in any Pi log. + +Excluded, with the reason stated in the README: +- Imported threads (`thread_id` starting `import:`, events marked + `historyImport`). They are partial copies of Claude Code sessions, not T3 + traffic: 55 user messages in two threads here. +- Deleted threads (`deleted_at` set). Across all projects there are 3, with + 3 messages. Archived threads count. + +## What fails closed + +With the T3 source on, each of these refuses the report with exit 1, the +code the ledger already uses for unreadable session evidence. The report +never falls back to Pi logs alone. As with `--no-issues`, `--no-t3` turns the +source off, and the report then says T3 was not read. +- The database is missing, unreadable, or won't open read-only (including + the `-wal` without `-shm` case). This differs from the Pi reader, which + treats a missing `.pi` as no messages. A missing Pi directory means no Pi + seats ran here. A missing T3 database on this host means the path or T3 + changed, and a silent zero is the failure Gate F exists to prevent. +- A required table or column is missing. The reader checks `PRAGMA + table_info` and names what's missing. This catches a T3 upgrade that + changes the schema. +- No project row, or more than one non-deleted row, for this repository root. +- A counted row has a bad `role`, non-string `text`, or a `created_at` that + doesn't parse. The Pi reader already refuses malformed JSONL and bad + timestamps the same way. +- `state.sqlite` or `~/.t3/userdata` is a symlink. The Pi reader skips + symlinked entries instead. For one named file, skipping would be another + silent zero, so this reader refuses. + +The source never writes to the database. It never reads other files in +`~/.t3`, and it passes no message text beyond `messageKind` and +`issueNumbers`, the same rule as for Pi logs. The one outside effect is +SQLite's own: a WAL reader takes read locks in the `-shm` file, as T3's own +connections do. + +## Decisions for Jason + +1. The source is on by default, with `--no-t3` to turn it off. The other + choice is off by default with `--t3` to turn it on. I recommend on by + default, because Gate F exists to count these messages. +2. Unmapped threads get a `t3:unmapped` row. The other choice is to drop + them. I recommend the row. +3. The 14 free-text headers from 09-17 to 09-22 stay counted as human. Fixing + them would mean loosening the header grammar for history only, and I don't + recommend it. + +## Acceptance for the build + +- Fixture databases built with `node:sqlite` in a temp dir, in WAL mode: + seat threads and an unmapped thread; imported, deleted and archived + threads; all three header forms, uppercase classes included; a message + outside the date range; another project with the same seat titles. +- Each fail-closed case above has its own test, including a schema column + removed and `-wal` without `-shm`. One test opens a database whose newest + message is still in the WAL and counts it. Another reads a database closed + cleanly with no writer attached, which is the T3-stopped case. +- The class fix is proven against HEAD's `messageKind`: an uppercase class + counts as agent after the fix and as human before it. +- A read against the live database gives the counts in this brief, allowing + for messages sent since. +- The ledger README's counting rules name the new source, the mapping rule + and the exclusions. +- No suite runs the ledger tests, so the BUILD-LOG entry names the test file. + +## Not in scope + +- Claude Code transcripts (`~/.claude/projects`) and Codex sessions + (`~/.codex/sessions`). T3's database already holds every message T3 + delivered, so those files would only duplicate it. +- Any write to T3, any T3 API call, or anything that needs T3 running. +- Fixing the two tmux misclassifications Filbert found in 6a. diff --git a/agents/darkwing/work/ledger-t3-source/r2-to-r3.diff b/agents/darkwing/work/ledger-t3-source/r2-to-r3.diff new file mode 100644 index 00000000..83e75815 --- /dev/null +++ b/agents/darkwing/work/ledger-t3-source/r2-to-r3.diff @@ -0,0 +1,74 @@ +--- r2.md ++++ docs/plans/2026-09-26_ledger-t3-source.md +@@ -3,9 +3,9 @@ + Brief only, no code. Darkwing wrote it on 2026-09-26 at Sage's request, issue + #1506. R1 (sha256 08959a05) went to Filbert, whose review asked for + revisions: `agents/filbert/work/ledger-t3-source-review-2026-09-26.md`, sha256 +-19dda29a. This is R2. It takes every finding, and it records Sage's rulings +-on the three open questions. Section 1 has one measurement that differs from +-the review. ++19dda29a. R2 (sha256 e8300cb6) took every finding and recorded Sage's ++rulings on the three open questions. Filbert approved R2 with three nits, ++review sha256 bb02d8d3. This is R3, which takes the nits. + + ## Why + +@@ -68,10 +68,10 @@ + | T3 stopped cleanly, no `-wal` or `-shm` | yes | reads, then leaves an empty `-wal` and a 32 KiB `-shm` | + | T3 stopped cleanly | no | fails, SQLite 1544 "attempt to write a readonly database" | + +-In every case the main file's bytes stayed the same. The last row is where +-Filbert and I differ. His review says the stopped-case read works with the +-directory read-only. In my run it failed with and without the read +-transaction. The build's test settles it. Either way a failed open is exit 1. ++In every case the main file's bytes stayed the same. Filbert's first ++review said the last case reads. His test had reused a database whose empty ++`-wal` and `-shm` were still present. On a true clean stop he also got 1544, ++and his review records the correction. A failed open is exit 1. + + So the accurate claim: the reader never writes the main database file. Like + any SQLite connection, it may create or update `-wal` and `-shm` beside it +@@ -198,7 +198,9 @@ + The source is on by default (Sage's ruling). `--no-t3` turns it off, and the + report then says T3 was not read. `--t3-db ` reads another database + file instead of `~/.t3/userdata/state.sqlite`. It exists for fixtures and +-gets the same checks. ++gets the same checks. The JSON records the database path read and whether ++it was the default. When it wasn't, the text report adds one line naming the ++path, so a Gate F result can't come from a fixture unnoticed. + + Each of these refuses the report with exit 1, the code the ledger already + uses for unreadable session evidence. The report never falls back to Pi logs +@@ -248,7 +250,9 @@ + fixture's temp directory. A test that forgets `--t3-db` or `--no-t3` then + finds no database and fails closed. The existing tests aren't about T3. Each + gets an empty fixture database at the fixture `HOME`'s default path, with +-one project row for the fixture root. So they run with the source on, and ++one project row for the fixture root. That row stores ++`fs.realpathSync(root)`, because the CLI resolves its root through realpath ++and a symlinked temp directory would otherwise not match. So they run with the source on, and + their expected rows don't change. One test asserts that a `HOME` with no + database exits 1 and names `--no-t3`. + +@@ -277,9 +281,7 @@ + `--no-t3`. Skipped when the tests run as root, where the mode bits don't + bind. + - The stopped case in a directory that isn't writable: exit 1, naming +- `--no-t3`, as I measured it. If the build reads there instead, the builder +- changes this test to assert correct counts and records the correction in +- the BUILD-LOG entry. Skipped as root too. ++ `--no-t3`. Skipped as root too. + + Also: + - Each other fail-closed case in section 4 has its own test, including a +@@ -288,7 +290,9 @@ + the same counts. + - The class fix is proven against HEAD's `messageKind`. An uppercase class + in either preamble counts as agent after the fix and as human before it. +-- The JSON lists each seat's threads and the unmapped threads. ++- The JSON lists each seat's threads and the unmapped threads, and the ++ database path with whether it was the default. A `--t3-db` run prints the ++ path line in the text report, and a default run doesn't. + - A read against the live database gives the counts in this brief, allowing + for messages sent since. It exits 0 with no header conflict. + - The ledger README's counting rules name the new source, both flags, the diff --git a/agents/darkwing/work/ledger-t3-source/r2.md b/agents/darkwing/work/ledger-t3-source/r2.md new file mode 100644 index 00000000..506c2297 --- /dev/null +++ b/agents/darkwing/work/ledger-t3-source/r2.md @@ -0,0 +1,306 @@ +# Ledger: a read-only T3 thread source for Table 2 (Gate F brief) + +Brief only, no code. Darkwing wrote it on 2026-09-26 at Sage's request, issue +#1506. R1 (sha256 08959a05) went to Filbert, whose review asked for +revisions: `agents/filbert/work/ledger-t3-source-review-2026-09-26.md`, sha256 +19dda29a. This is R2. It takes every finding, and it records Sage's rulings +on the three open questions. Section 1 has one measurement that differs from +the review. + +## Why + +Table 2 counts user messages per seat from `.pi/state//sessions/*.jsonl` +only. Development seats now run in T3 on the Claude and Codex harnesses, so +their prompts, Jason's included, never reach a Pi log. Today the Human column +can't see T3 at all, and the zero it shows for T3 seats means "no source", not +"no human prompts". 6a (ef0020ad) taught `messageKind` the T3 header, but no +source the ledger reads contains one. Gate F (QUEUE row 6) passes when +Filbert's item closes with zero human messages from Jason. While the ledger +can't see T3, a zero there proves nothing. + +## Where T3 keeps messages + +T3 keeps its state in one SQLite database, `~/.t3/userdata/state.sqlite`, in +WAL mode (`state.sqlite-wal` and `state.sqlite-shm` sit beside it). The counts +need three projection tables: + +- `projection_projects`: `project_id`, `workspace_root`, `deleted_at`. +- `projection_threads`: `thread_id`, `project_id`, `title`, `archived_at`, + `deleted_at`. +- `projection_thread_messages`: `message_id` (primary key), `thread_id`, + `role` (`user` or `assistant`), `text`, `created_at` (ISO UTC). + +The JSON diagnostic reads one more. In `orchestration_events`, each +`thread.message-sent` event carries `metadata_json.origin`. Messages typed in +the T3 app carry an `appVersion` there. Messages sent through T3's API or MCP +tools, which is how seats talk to each other, don't. + +The same directory also holds `secrets/`, `clerk-tokens.json` and other +settings files. The reader opens `state.sqlite` and nothing else, and it +selects named columns only, never `*`. + +## 1. Reading it, with T3 running or not + +The reader uses Node's built-in `node:sqlite` (`DatabaseSync`). That needs no +dependency, and Node 26.8.1 (SQLite 3.53.4) prints no warning for it. + +- **URI.** Build it with `pathToFileURL(dbPath)` and set `mode=ro` through + `searchParams`, then pass `readOnly: true`. A `?`, `#` or `%` in the home + path would break a string-built URI. +- **One snapshot.** Run every query, from the schema checks through the + diagnostic, inside one `BEGIN` … `COMMIT`. In autocommit mode each + statement sees its own snapshot while T3 writes between them. +- **Busy timeout.** Set `DatabaseSync`'s `timeout` to 5 s. A transient + `SQLITE_BUSY` during a T3 checkpoint then waits instead of failing. A busy + error after the timeout exits 1 like any open failure. +- **No `immutable=1` and no copy.** Both lose the WAL. Filbert found worse + than lost messages: with a table created inside the WAL, `immutable=1` + fails with `no such table`. + +What happens on disk. Filbert and I both tested these in scratch +directories: + +| State | Directory writable | Result | +|---|---|---| +| T3 running, writer attached, newest rows only in `-wal` | yes | reads them | +| `-wal` without `-shm` (writer killed, `-shm` removed) | yes | reads the WAL rows and creates `-shm` | +| `-wal` without `-shm` | no | open fails, SQLite 14 | +| T3 stopped cleanly, no `-wal` or `-shm` | yes | reads, then leaves an empty `-wal` and a 32 KiB `-shm` | +| T3 stopped cleanly | no | fails, SQLite 1544 "attempt to write a readonly database" | + +In every case the main file's bytes stayed the same. The last row is where +Filbert and I differ. His review says the stopped-case read works with the +directory read-only. In my run it failed with and without the read +transaction. The build's test settles it. Either way a failed open is exit 1. + +So the accurate claim: the reader never writes the main database file. Like +any SQLite connection, it may create or update `-wal` and `-shm` beside it +and takes read locks in `-shm`. T3 opens normally afterwards. + +## 2. Jason or agent + +Reuse the 6a rule. The first line of `text` decides: a T3 header or the tmux +preamble counts as agent, `control-board` as the sender counts as board, and +anything else counts as human. Messages with role `user` count; assistant +messages don't. + +The class fix rides in this build (Sage's ruling). HEAD's +`packages/ledger/src/ledger.mjs:81` (tmux) and `:83` (T3) both allow only +`class=[a-z-]+`. Both become case-insensitive. Seats send uppercase classes: +Sage's DECISION, INFO, REVIEW-REQUEST and REVIEW-NOTE, and my own +REVIEW-REQUEST. In this project's threads 16 real agent headers failed on +that alone at 20:54Z. The ms-communications table lists lowercase names, so +the fix follows what seats send, not the table. + +Cross-check, read at 2026-09-26T20:54Z for the mosaic-stack project (209 +user messages outside imported and deleted threads, every one with its +`thread.message-sent` event). Filbert's later read agreed, plus messages sent +since. + +| T3 origin | Header matches 6a | Count | +|---|---|---| +| typed in the app (has `appVersion`) | no | 99 | +| sent through the API (no `appVersion`) | yes | 80 | +| sent through the API | no, uppercase class | 16 | +| sent through the API | no, free-text roles | 14 | + +No message typed in the app carries a header, and every API message in this +project carries one of the three forms. The 14 free-text ones are older +Discord Bot thread headers such as `[from: SetSpark coordinator (…) -> to: +Discord Bot (…)]`, written before the guide fixed the format. Sage ruled they +stay as recorded: they count as human, dated 2026-09-17 to 2026-09-22. + +The header rule decides. The JSON also carries one diagnostic that feeds no +table or total: user messages the rule calls human that T3 recorded as sent +through the API. That number exposed the class bug and would catch the next +format drift. `origin` is T3's internal metadata, not a documented contract, +so it decides nothing. If `orchestration_events` or a column it needs is +missing, the diagnostic reads `unknown` and the report goes on (Sage's +ruling on F5). Missing tables the counts depend on still exit 1. + +## 3. Thread to seat + +**Project.** A thread counts for this checkout only if its project's +`workspace_root` equals the ledger's repository root, byte for byte. The CLI +already takes that root from the realpath of its own URL, today +`/mnt/storage/src/mosaic-stack`, project `34050c07`. So a T3 project opened +through the compatibility symlink `~/src/mosaic-stack-dev-test` doesn't +match, and "no project row" is the right refusal. The README says so. + +**Title rule.** Thread IDs change whenever Jason starts a new thread for a +seat, so there's no fixed map. T3-AGENT-COMMS.md already names threads after +the seat ("Darkwing", "Sage", "Dewey in Claude"). A thread belongs to seat +`` when `` is a real directory under `agents/` and the lower-cased +title equals `` or starts with `` followed by a space. So "Sagebrush" +stays unmapped. Several threads can map to one seat, and their counts add +up, as several Pi session files already do. + +Today that maps Sage, Darkwing, Filbert, Dewey and Rocko (one thread each, +created 2026-09-26, titles set by hand), plus "Darkwing in Claude" (archived) +and "Dewey in Claude". Researcher has a directory and no thread. Three +threads map to no seat. Two are imported and excluded anyway ("FINDINGS.md +review" and "[dragon-lin:darkwing -> …"). The third is "Discord Bot" with 68 +user messages: 54 without a header, and the 14 free-text headers. + +Titles are current state, and T3 can write them itself. They go wrong three +ways. T3 auto-titles an unnamed thread from Jason's first prompt, so "Rocko +review of the plan" maps to rocko. A rename moves the whole history to +another row. A seat thread titled for a topic drops into `t3:unmapped`. +None of this changes the Human total or the human-per-closed ratio. It only +moves counts between rows, but Gate F reads one seat's row. + +**Header cross-check.** The headers already say which seat a thread belongs +to. For every user message whose header matches the fixed 6a rule and whose +`to:` id equals the message's own `thread_id`: +- in a mapped thread, the `to:` role, lower-cased, must equal that thread's + seat; +- in an unmapped thread, the `to:` role must not be a seat name. + +A conflict exits 1 and names the thread id, its title and both roles. A +header whose `to:` id is some other thread is not checked. The check reads +message text only, not T3 metadata. In a live read at 21:02Z every header +agreed: all 104 addressed to their own thread carried the full thread id and +named that thread's seat (Sage 40, Darkwing 15, Filbert 18, Dewey 15, Rocko +16). + +It catches a seat thread renamed to another seat or to a topic, once any +agent writes to it. It also catches an auto-titled thread that agents +address by a different seat. It misses a thread no agent ever writes to. +Such a thread can only add human counts to a seat's row, never hide them, so +for Gate F it errs toward a visible failure. The README says so. + +**Unmapped row.** Unmapped threads go in one Table 2 row, `t3:unmapped` +(Sage's ruling), so their human messages still reach the Human column and +the human-per-closed ratio. + +**Mapping in the JSON.** For each seat, the T3 thread ids and titles that +made its row, and the unmapped thread ids and titles. Anyone checking a Gate +F result can then see which threads the row came from. + +A seat's row sums its Pi and T3 counts, and the JSON splits them by source. +Nothing is counted twice. Every T3 session today runs on `claudeAgent` or +`codex`, which don't write `.pi/state`, and Filbert found no T3 header in any +Pi log (6a record). + +**Excluded,** with the reason stated in the README: +- Imported threads (`thread_id` starting `import:`, events marked + `historyImport`). They are partial copies of Claude Code sessions, not T3 + traffic: 55 user messages in two threads here. +- Deleted threads (`deleted_at` set). Across all projects there are 3, with + 3 messages. Archived threads count. +- Threads in other T3 projects. Live, there is a project at `/home/jwoltje` + and a deleted one at `/mnt/storage/src`. A thread in either could work on + this repository and would not be counted. The workspace-root rule is still + the right one, but the README names this blind spot. + +## 4. What fails closed + +The source is on by default (Sage's ruling). `--no-t3` turns it off, and the +report then says T3 was not read. `--t3-db ` reads another database +file instead of `~/.t3/userdata/state.sqlite`. It exists for fixtures and +gets the same checks. + +Each of these refuses the report with exit 1, the code the ledger already +uses for unreadable session evidence. The report never falls back to Pi logs +alone. Where the database is missing or won't open, the message names +`--no-t3`. +- The database is missing or unreadable, or won't open read-only. That + includes a directory that isn't writable when SQLite needs to create + `-shm`, and a busy error after the timeout. The Pi reader treats a missing + `.pi` as no messages, and this departs from it on purpose. A missing Pi + directory means no Pi seats ran here. A missing T3 database on this host + means the path or T3 changed, and a silent zero is the failure Gate F + exists to prevent. +- `~/.t3`, `~/.t3/userdata` or `state.sqlite` is a symlink. With `--t3-db`, + the file and its directory are checked. The Pi reader checks every + ancestor too, but it skips symlinked entries. Skipping one named file + would be another silent zero, so this reader refuses. +- A table or column the counts need is missing. The reader checks `PRAGMA + table_info` and names what's missing. This catches a T3 upgrade that + changes the schema. +- No project row, or more than one non-deleted row, for this repository root. +- A counted row has a bad `role`, non-string `text`, or a `created_at` that + doesn't parse. The Pi reader already refuses malformed JSONL and bad + timestamps the same way. +- A header conflicts with the title mapping (section 3). + +The reader never reads other files in `~/.t3`. It passes no message text +beyond `messageKind`, `issueNumbers` and the header's `to:` role and id, the +same rule as for Pi logs. + +## 5. Rulings + +Sage ruled on the three questions R1 put to Jason, as lead calls: +1. The source is on by default. A missing or unreadable database exits 1, + and the message names `--no-t3`. +2. Unmapped threads get the `t3:unmapped` row. +3. The 14 free-text headers stay as recorded. They show only in the JSON + diagnostic. + +Sage also ruled that the class fix rides in this build, and that a missing +diagnostic table reads `unknown` (F5). + +## 6. Acceptance for the build + +**No test opens the real `~/.t3`.** Both places in +`packages/ledger/tests/ledger.test.mjs` that spawn `cli.mjs` (the shared +`run()` helper and the direct `spawnSync` at line 66) set `HOME` to the +fixture's temp directory. A test that forgets `--t3-db` or `--no-t3` then +finds no database and fails closed. The existing tests aren't about T3. Each +gets an empty fixture database at the fixture `HOME`'s default path, with +one project row for the fixture root. So they run with the source on, and +their expected rows don't change. One test asserts that a `HOME` with no +database exits 1 and names `--no-t3`. + +Fixture databases are built with `node:sqlite` in a temp directory, in WAL +mode: +- seat threads and an unmapped thread; imported, deleted and archived + threads; a message outside the date range; +- all three header forms, with uppercase classes in both the tmux preamble + and the T3 header; +- a thread with the same seat title in another project; +- a seat thread renamed to another seat, with an agent header to its own + id, which exits 1; +- a "Sagebrush" title, which stays unmapped; +- a thread titled "Researcher", which maps to the seat that has no thread + live. + +WAL states, each with its own test: +- The newest message is only in `-wal`, with the writer still attached (the + live-T3 case). It is counted. +- T3 stopped: the database closed cleanly with no writer. Counts are + correct, and the main file's bytes are unchanged afterwards. +- `-wal` without `-shm` in a writable directory: made by a child writer with + `wal_autocheckpoint=0` that is SIGKILLed, then `-shm` deleted. The WAL + rows are counted. +- `-wal` without `-shm` in a directory that isn't writable: exit 1, naming + `--no-t3`. Skipped when the tests run as root, where the mode bits don't + bind. +- The stopped case in a directory that isn't writable: exit 1, naming + `--no-t3`, as I measured it. If the build reads there instead, the builder + changes this test to assert correct counts and records the correction in + the BUILD-LOG entry. Skipped as root too. + +Also: +- Each other fail-closed case in section 4 has its own test, including a + removed schema column and each symlink. +- A missing `orchestration_events` gives `unknown` for the diagnostic and + the same counts. +- The class fix is proven against HEAD's `messageKind`. An uppercase class + in either preamble counts as agent after the fix and as human before it. +- The JSON lists each seat's threads and the unmapped threads. +- A read against the live database gives the counts in this brief, allowing + for messages sent since. It exits 0 with no header conflict. +- The ledger README's counting rules name the new source, both flags, the + mapping rule and the header check, and the exclusions. That includes the + symlinked-checkout case and the other-project blind spot. +- No suite runs the ledger tests, so the BUILD-LOG entry names the test file. + +## Not in scope + +- Claude Code transcripts (`~/.claude/projects`) and Codex sessions + (`~/.codex/sessions`). T3's database already holds every message T3 + delivered, so those files would only duplicate it. +- Any T3 API call, anything that needs T3 running, and any write beyond + SQLite's own `-wal` and `-shm` handling. +- Fixing the two tmux misclassifications Filbert found in 6a. diff --git a/agents/filbert/work/ledger-t3-source-review-2026-09-26.md b/agents/filbert/work/ledger-t3-source-review-2026-09-26.md new file mode 100644 index 00000000..94dea091 --- /dev/null +++ b/agents/filbert/work/ledger-t3-source-review-2026-09-26.md @@ -0,0 +1,317 @@ +# Gate F brief: Filbert's review of the T3 ledger source + +Reviewer: Filbert, 2026-09-26. Requested by Sage. + +Candidate: `docs/plans/2026-09-26_ledger-t3-source.md`, sha256 +`08959a05574264e4f8243a90af94746e73a2fde3706f22e38f4ff1105b7a45a8`. I +verified the hash. The file is uncommitted and has no code. + +I reviewed it against Sage's rulings, not as open choices: +1. on by default, where a missing or unreadable database exits 1 and the + message names `--no-t3`; +2. the `t3:unmapped` row stays; +3. the 14 free-text Discord Bot headers stay as recorded; +4. case-insensitive classes (the 6a fix) are folded in. + +**Verdict: revise.** The design is sound, and the counts check out against +the live database. Three findings would make the build's tests fail or +depend on the host, and each needs a text change: +- one acceptance check expects the wrong result (W1); +- the "never writes" claim is inaccurate (W2); +- existing tests would read the real T3 database (F1). + +The rest are smaller additions. None needs a new ruling from Sage or Jason. + +## What I checked + +- **Scratch WAL tests** with `node:sqlite` on Node 26.8.1 (SQLite 3.53.4), + in `/tmp`. +- **The live database**, opened read-only with `mode=ro`. I read the schema, + project roots, thread titles, and message counts per thread. I read no + message text except the first line of each user message, which the + classifier runs over. I printed text only for my own thread's one human + message. +- **`packages/ledger/src/ledger.mjs` and `tests/ledger.test.mjs`** at HEAD. + +The brief's numbers hold: + +| Check | Live result | +|---|---| +| Discord Bot user messages | 68 | +| Typed-in-app (human) messages, outside imported and deleted threads | 113 = the brief's 99 + 14 | +| Agent messages with the class fix | 101 = the brief's 80 + 16, plus 5 sent since 20:54Z | +| Journal and locking mode | `wal` and `normal` | +| The three required tables | the columns the brief names, all present | + +The `orchestration_events` diagnostic query costs about 30 ms on the 394 MB +database. + +## 1. WAL and the read-only open (the part Sage asked about first) + +The core claim holds. In my test, a `mode=ro` reader saw all three rows +while they were still only in the `-wal` file, with the writer attached. +`immutable=1` did worse than "lose the newest messages": with the table +created inside the WAL, it failed with `no such table`. So the brief's ban +on `immutable=1` and on copies is right. + +Two of the brief's statements are wrong, though, and one acceptance check +would fail as written. + +**W1. `-wal` without `-shm` does not fail. (Blocking.)** I made that case by +copying a SIGKILLed writer's `-wal` without its `-shm`. +- A `mode=ro` open succeeded, read all three rows, and created the missing + `-shm` itself. +- It failed, with SQLite error 14 ("unable to open database file"), only + when the directory was not writable. +- Adding `readonly_shm=1` didn't help. It made both this case and the + clean-stop case fail, and it still created a `-wal`. + +So the acceptance line "Each fail-closed case above has its own test, +including … `-wal` without `-shm`" expects a refusal that doesn't happen. +Change it to two tests: +- **`-wal` without `-shm`, directory writable:** the read counts the WAL + rows. +- **`-wal` without `-shm`, directory not writable:** exit 1, naming + `--no-t3`. + +Also fix line 142, which lists this case as a refusal. + +**W2. The reader can create files in `~/.t3/userdata`. (Blocking as a +claim.)** Line 157 says "The source never writes to the database", and line +160 says the one outside effect is read locks in `-shm`. Line 44 says "no +locks", which contradicts line 160. What actually happens: +- **T3 stopped cleanly:** `-wal` and `-shm` are gone. A `mode=ro` open + creates an empty `-wal` and a 32 KiB `-shm`, and leaves both after it + closes. The main file is untouched, and the read works even with the + directory read-only. +- **`-wal` without `-shm`:** the reader rebuilds the WAL index into a new + `-shm`. + +This is standard SQLite behaviour, and T3 opens normally afterwards. +Replace the claim with the accurate one: +- the reader never writes the main database file; +- it may create or update `-wal` and `-shm` beside it, as any SQLite + connection does. + +The T3-stopped test should then assert: +- the counts are correct; +- the main file's bytes are unchanged. + +**W3. One read transaction, and a busy timeout.** The brief runs several +queries (schema, project, threads, messages, the diagnostic's events) while +T3 writes. +- In autocommit mode, each statement gets its own snapshot. Wrap the whole + read in `BEGIN` … `COMMIT` (a deferred read transaction), so every query + sees one committed state. +- Set `DatabaseSync`'s `timeout` (a few seconds). Then a transient + `SQLITE_BUSY`, for example while T3 runs a checkpoint or WAL recovery, + waits instead of failing Gate F. A `BUSY` after the timeout is exit 1, + like any other open failure. + +**W4. Build the URI with `pathToFileURL`, not string concatenation.** In my +test, a string `file:?mode=ro` worked on Node 26.8.1. But a `?`, `#` +or `%` in the home path would break it. `new URL` plus `searchParams` avoids +that. + +**The two untested cases, as acceptance checks.** With W1 and W2 applied, +both are proper checks: a fixture can create each state deterministically. +- **Stopped:** close a WAL database cleanly. +- **`-wal` without `-shm`:** SIGKILL a child writer that set + `wal_autocheckpoint=0`, then delete `-shm`. Copying the `-wal` before any + checkpoint also works. + +Add one more: the newest message is in the WAL while the writer holds the +database open. The brief has this ("One test opens a database whose newest +message is still in the WAL"). Say that the writer is still attached, +because that is the live-T3 case. + +## 2. Thread-to-seat title rule + +Checked live: +- The five 2026-09-26 seat threads have `title_state_json.source` set to + `manual`. +- "Darkwing in Claude" and "Dewey in Claude" predate that field (null). +- T3 also generates titles itself: the table has `title_regeneration_*` + columns and `needsRefinement`. + +**How the rule can misassign:** +1. **An auto-generated title.** If Jason opens a thread without naming it, + T3 titles it from his first prompt. A title like "Rocko review of the + plan" maps to rocko. +2. **A rename.** Titles are current state. Renaming a thread moves its + whole history to another row. +3. **A seat's thread titled for a topic.** For example, "review: queue" + drops that seat's messages into `t3:unmapped`. + +Misassignment never changes the Human total or the human-per-closed ratio. +It only moves counts between rows. It does matter for Gate F, which reads +one seat's row. + +**T1. Add a header cross-check. (Required.)** The headers already carry the +answer. Every agent header addressed to a seat thread names the recipient +and that thread's own id. Live, every one agrees with the title rule: + +| Thread | Headers addressed to it | +|---|---| +| Sage | 39, all `to: sage` | +| Darkwing | 14, all `to: darkwing` | +| Filbert | 18, all `to: filbert` | +| Dewey | 15, all `to: dewey` | +| Rocko | 15, all `to: rocko` | + +Rule: a user message whose header matches 6a, and whose `to:` id is the +message's own thread, must name the seat the title rule gave that thread. +The same goes for an unmapped thread receiving a seat-addressed header. On +a mismatch, the report refuses with exit 1 and names the thread id and both +roles. This uses only message text, not T3 metadata, so it fits the brief's +principle that `origin` decides nothing. + +It catches: +- a seat thread renamed to another seat; +- a seat thread renamed to a topic, once any agent writes to it; +- a Jason-started thread auto-titled with a seat name that agents then + address by a different seat. + +It doesn't catch a thread that no agent ever writes to. That case can only +add human counts to a seat row, never hide them, so for Gate F it errs +toward a visible failure. Say so in the README. + +**T2. List the mapping in the JSON.** Give each seat's T3 thread ids and +titles, and the unmapped thread ids. A reader of a Gate F result can then +see which threads made up the row. + +**T3. Test the rule** with these fixtures: +- a same-title thread in another project (the brief has this); +- a seat thread renamed to another seat, which must refuse under T1; +- a title that starts with a seat name but not followed by a space ("Sagebrush"), + which stays unmapped; +- "Researcher", a real directory with no thread today. + +## 3. The fail-closed departure + +The reasoning is right, and it follows Sage's ruling 1. A missing `.pi` +directory means no Pi seat ran here. A missing or changed T3 database on a +host that uses T3 means the evidence moved, and a zero there would be the +false pass that Gate F exists to catch. Refusing on a missing table or +column, a missing project or more than one project, a bad row, or a symlink +is consistent with how the Pi reader refuses malformed JSONL. + +**F1. The existing tests would read the real T3 database. (Blocking.)** +- `tests/ledger.test.mjs` spawns `cli.mjs` with `env: { ...process.env, … }`, + so `HOME` is the developer's. +- With the source on by default, every existing test would open the real + `~/.t3/userdata/state.sqlite`, find no project for the temp fixture root, + and exit 1. +- On a host without T3, they would exit 1 too. + +The brief needs one explicit way to point the reader at a fixture: +- either a fixture `HOME` in the spawned environment, or a documented + override such as `--t3-db PATH`; +- an acceptance line saying no test opens the real `~/.t3`. For example, + the tests set `HOME` to a temp dir for every run. + +Existing tests that aren't about T3 should pass `--no-t3` or get an empty +fixture database. Say which. + +**F2. Symlink check.** The brief checks `state.sqlite` and `~/.t3/userdata`. +Check `~/.t3` as well, because the Pi reader checks every ancestor from the +root down ("Check every source ancestor"). Today all three are real +directories or files. + +**F3. Match the repository root exactly.** Compare `workspace_root` byte for +byte with the ledger's root. The CLI gets that root from the realpath of its +own URL. `~/src/mosaic-stack-dev-test` is a compatibility symlink to this +checkout. A T3 project opened through it would not match, and then "no +project row" is the correct refusal. The README should say this. + +**F4. Name the blind spot.** Live, there is a T3 project at `/home/jwoltje` +and a deleted one at `/mnt/storage/src`. A thread in either could work on +this repository and would not be counted. The workspace-root rule is still +right, but the README's exclusions should name this case. + +**F5. When the diagnostic's table is missing.** The diagnostic (ruling 3) +reads `orchestration_events`. Say what happens if that table or its columns +are missing: +- The report prints the diagnostic as `unknown`, and the counts are + unaffected, because the diagnostic decides nothing. +- Or add the table to the required schema and refuse. + +I'd take the first, because a T3 change to internal metadata shouldn't stop +the counts. Either way, the brief should say which. + +## 4. The class fix and the counts + +The fix is correct and scoped as Sage ruled. HEAD's +`packages/ledger/src/ledger.mjs:81` (tmux) and `:83` (T3) both have +`class=[a-z-]+`. Make both case-insensitive, not only the T3 one. The +acceptance line "an uppercase class counts as agent after the fix and as +human before it" should cover both preambles. One thing the count doesn't +show: the only human-classified message in my thread is Jason's opening +assignment (19:31:07Z). With the fix, all 18 agent headers to Filbert +classify as agent. + +## To reach approve + +- **W1:** correct the `-wal`-without-`-shm` expectation and the line-142 + refusal; split it into writable and non-writable tests. +- **W2:** replace the "never writes" and "no locks" statements with the + accurate effect; the stopped test asserts the main file's bytes are + unchanged. +- **W3:** one read transaction and a busy timeout. **W4:** build the URI + from a URL. +- **T1:** the header cross-check, refusing on a conflict. **T2:** the + mapping in the JSON. **T3:** the extra fixtures. +- **F1:** a fixture path and no test opening the real `~/.t3`. **F2–F5:** + the ancestor symlink check, an exact root match, the blind spot in the + README, and the diagnostic when its table is missing. +- **§4:** the class fix covers both preambles. + +Send the revision with its hash and I'll review the delta. + +## Correction (Filbert, 2026-09-26, after R2) + +W2 said the clean-stop read "works even with the directory read-only". That +was wrong, and Darkwing's measurement is right. My test for that case reused +a database that the previous test had already opened, so its leftover `-wal` +(0 bytes) and `-shm` were still present. It was not a clean stop. I re-ran it +with a truly clean stop (no `-wal`, no `-shm`) in a non-writable directory +on Node 26.8.1 / SQLite 3.53.4. The open fails with errcode 1544, "attempt +to write a readonly database". So a stopped T3 in a non-writable directory +is exit 1, as R2 expects. The rest of W2 stands: the reader never writes the +main file, and it may create `-wal` and `-shm`. + +## R2 delta review + +Candidate: `docs/plans/2026-09-26_ledger-t3-source.md`, sha256 +`e8300cb6abea70819aba7cf10040d19b4d6019b5663c37203209537a5f10ee62`. I +verified Darkwing's delta files against `manifest.sha256`: +- `r1.md` is `08959a05…45a8`, the R1 I reviewed; +- `r1-to-r2.diff` is `aa4740ae…71fc`. + +I read R2 in full. + +**Verdict: approve** `e8300cb6`. Every finding is answered: +- W1 and W2: the section 1 table, the accurate claim about side files, and + split tests. +- W3 and W4: one transaction, a 5 s timeout, and `pathToFileURL`. +- T1: as asked, plus Darkwing's addition that an unmapped thread can't + receive an own-id header naming a seat. +- T2 and T3: the mapping in the JSON, and the fixtures. +- F1: `--t3-db`, `HOME` set at both spawn sites, and a no-database test. +- F2–F4: done. F5 follows Sage's ruling. +- §4: both preambles. + +Three nits, none blocking. Carry them into the build, or fix them in an R3 +and I'll confirm the hash: +1. **The measurement is settled.** Lines 71–74 and 279–282 treat the + non-writable clean stop as disputed. I've corrected my review above: it + fails with 1544. State it as agreed. Make the test a plain exit-1 + assertion, and drop the "if the build reads there instead" branch. +2. **Real path in the fixture project row.** The fixture's project row + should store `fs.realpathSync(root)`. The CLI takes its root from the + realpath of `cli.mjs`, and a temp directory under a symlinked `/tmp` + would otherwise fail with "no project row". +3. **Say which database was read.** `--t3-db` can point a real Gate F run at + any file. The JSON should record the database path it read and whether it + was the default. The text report should add one line when it wasn't. A + Gate F result then can't come from a fixture without saying so. diff --git a/docs/plans/2026-09-26_lead-decisions.md b/docs/plans/2026-09-26_lead-decisions.md index 8724db14..12ce721a 100644 --- a/docs/plans/2026-09-26_lead-decisions.md +++ b/docs/plans/2026-09-26_lead-decisions.md @@ -166,3 +166,9 @@ which stay with him. Each item names who decided it and what happened. committed and pushed as 6c06a6f3. The restart ran at 20:58:03Z: PID 890894 was replaced by 499064, the gateway was READY at 20:58:04Z, and pi has the SetSpark verbs. Jason's live check comes next. +14. **Gate F brief approved.** Filbert approved R2 (e8300cb6…); his review is + at bb02d8d3…. He corrected one of his own facts: a cleanly stopped T3 + database in a read-only directory fails with 1544, as Darkwing measured. + Either way it exits 1. The three nits ride in the build. The JSON records + which database file it read, so a fixture can't pass for Gate F evidence. + Darkwing builds. Filbert reviews the code, and Sage commits. diff --git a/docs/plans/2026-09-26_ledger-t3-source.md b/docs/plans/2026-09-26_ledger-t3-source.md new file mode 100644 index 00000000..d158cab6 --- /dev/null +++ b/docs/plans/2026-09-26_ledger-t3-source.md @@ -0,0 +1,310 @@ +# Ledger: a read-only T3 thread source for Table 2 (Gate F brief) + +Brief only, no code. Darkwing wrote it on 2026-09-26 at Sage's request, issue +#1506. R1 (sha256 08959a05) went to Filbert, whose review asked for +revisions: `agents/filbert/work/ledger-t3-source-review-2026-09-26.md`, sha256 +19dda29a. R2 (sha256 e8300cb6) took every finding and recorded Sage's +rulings on the three open questions. Filbert approved R2 with three nits, +review sha256 bb02d8d3. This is R3, which takes the nits. + +## Why + +Table 2 counts user messages per seat from `.pi/state//sessions/*.jsonl` +only. Development seats now run in T3 on the Claude and Codex harnesses, so +their prompts, Jason's included, never reach a Pi log. Today the Human column +can't see T3 at all, and the zero it shows for T3 seats means "no source", not +"no human prompts". 6a (ef0020ad) taught `messageKind` the T3 header, but no +source the ledger reads contains one. Gate F (QUEUE row 6) passes when +Filbert's item closes with zero human messages from Jason. While the ledger +can't see T3, a zero there proves nothing. + +## Where T3 keeps messages + +T3 keeps its state in one SQLite database, `~/.t3/userdata/state.sqlite`, in +WAL mode (`state.sqlite-wal` and `state.sqlite-shm` sit beside it). The counts +need three projection tables: + +- `projection_projects`: `project_id`, `workspace_root`, `deleted_at`. +- `projection_threads`: `thread_id`, `project_id`, `title`, `archived_at`, + `deleted_at`. +- `projection_thread_messages`: `message_id` (primary key), `thread_id`, + `role` (`user` or `assistant`), `text`, `created_at` (ISO UTC). + +The JSON diagnostic reads one more. In `orchestration_events`, each +`thread.message-sent` event carries `metadata_json.origin`. Messages typed in +the T3 app carry an `appVersion` there. Messages sent through T3's API or MCP +tools, which is how seats talk to each other, don't. + +The same directory also holds `secrets/`, `clerk-tokens.json` and other +settings files. The reader opens `state.sqlite` and nothing else, and it +selects named columns only, never `*`. + +## 1. Reading it, with T3 running or not + +The reader uses Node's built-in `node:sqlite` (`DatabaseSync`). That needs no +dependency, and Node 26.8.1 (SQLite 3.53.4) prints no warning for it. + +- **URI.** Build it with `pathToFileURL(dbPath)` and set `mode=ro` through + `searchParams`, then pass `readOnly: true`. A `?`, `#` or `%` in the home + path would break a string-built URI. +- **One snapshot.** Run every query, from the schema checks through the + diagnostic, inside one `BEGIN` … `COMMIT`. In autocommit mode each + statement sees its own snapshot while T3 writes between them. +- **Busy timeout.** Set `DatabaseSync`'s `timeout` to 5 s. A transient + `SQLITE_BUSY` during a T3 checkpoint then waits instead of failing. A busy + error after the timeout exits 1 like any open failure. +- **No `immutable=1` and no copy.** Both lose the WAL. Filbert found worse + than lost messages: with a table created inside the WAL, `immutable=1` + fails with `no such table`. + +What happens on disk. Filbert and I both tested these in scratch +directories: + +| State | Directory writable | Result | +|---|---|---| +| T3 running, writer attached, newest rows only in `-wal` | yes | reads them | +| `-wal` without `-shm` (writer killed, `-shm` removed) | yes | reads the WAL rows and creates `-shm` | +| `-wal` without `-shm` | no | open fails, SQLite 14 | +| T3 stopped cleanly, no `-wal` or `-shm` | yes | reads, then leaves an empty `-wal` and a 32 KiB `-shm` | +| T3 stopped cleanly | no | fails, SQLite 1544 "attempt to write a readonly database" | + +In every case the main file's bytes stayed the same. Filbert's first +review said the last case reads. His test had reused a database whose empty +`-wal` and `-shm` were still present. On a true clean stop he also got 1544, +and his review records the correction. A failed open is exit 1. + +So the accurate claim: the reader never writes the main database file. Like +any SQLite connection, it may create or update `-wal` and `-shm` beside it +and takes read locks in `-shm`. T3 opens normally afterwards. + +## 2. Jason or agent + +Reuse the 6a rule. The first line of `text` decides: a T3 header or the tmux +preamble counts as agent, `control-board` as the sender counts as board, and +anything else counts as human. Messages with role `user` count; assistant +messages don't. + +The class fix rides in this build (Sage's ruling). HEAD's +`packages/ledger/src/ledger.mjs:81` (tmux) and `:83` (T3) both allow only +`class=[a-z-]+`. Both become case-insensitive. Seats send uppercase classes: +Sage's DECISION, INFO, REVIEW-REQUEST and REVIEW-NOTE, and my own +REVIEW-REQUEST. In this project's threads 16 real agent headers failed on +that alone at 20:54Z. The ms-communications table lists lowercase names, so +the fix follows what seats send, not the table. + +Cross-check, read at 2026-09-26T20:54Z for the mosaic-stack project (209 +user messages outside imported and deleted threads, every one with its +`thread.message-sent` event). Filbert's later read agreed, plus messages sent +since. + +| T3 origin | Header matches 6a | Count | +|---|---|---| +| typed in the app (has `appVersion`) | no | 99 | +| sent through the API (no `appVersion`) | yes | 80 | +| sent through the API | no, uppercase class | 16 | +| sent through the API | no, free-text roles | 14 | + +No message typed in the app carries a header, and every API message in this +project carries one of the three forms. The 14 free-text ones are older +Discord Bot thread headers such as `[from: SetSpark coordinator (…) -> to: +Discord Bot (…)]`, written before the guide fixed the format. Sage ruled they +stay as recorded: they count as human, dated 2026-09-17 to 2026-09-22. + +The header rule decides. The JSON also carries one diagnostic that feeds no +table or total: user messages the rule calls human that T3 recorded as sent +through the API. That number exposed the class bug and would catch the next +format drift. `origin` is T3's internal metadata, not a documented contract, +so it decides nothing. If `orchestration_events` or a column it needs is +missing, the diagnostic reads `unknown` and the report goes on (Sage's +ruling on F5). Missing tables the counts depend on still exit 1. + +## 3. Thread to seat + +**Project.** A thread counts for this checkout only if its project's +`workspace_root` equals the ledger's repository root, byte for byte. The CLI +already takes that root from the realpath of its own URL, today +`/mnt/storage/src/mosaic-stack`, project `34050c07`. So a T3 project opened +through the compatibility symlink `~/src/mosaic-stack-dev-test` doesn't +match, and "no project row" is the right refusal. The README says so. + +**Title rule.** Thread IDs change whenever Jason starts a new thread for a +seat, so there's no fixed map. T3-AGENT-COMMS.md already names threads after +the seat ("Darkwing", "Sage", "Dewey in Claude"). A thread belongs to seat +`` when `` is a real directory under `agents/` and the lower-cased +title equals `` or starts with `` followed by a space. So "Sagebrush" +stays unmapped. Several threads can map to one seat, and their counts add +up, as several Pi session files already do. + +Today that maps Sage, Darkwing, Filbert, Dewey and Rocko (one thread each, +created 2026-09-26, titles set by hand), plus "Darkwing in Claude" (archived) +and "Dewey in Claude". Researcher has a directory and no thread. Three +threads map to no seat. Two are imported and excluded anyway ("FINDINGS.md +review" and "[dragon-lin:darkwing -> …"). The third is "Discord Bot" with 68 +user messages: 54 without a header, and the 14 free-text headers. + +Titles are current state, and T3 can write them itself. They go wrong three +ways. T3 auto-titles an unnamed thread from Jason's first prompt, so "Rocko +review of the plan" maps to rocko. A rename moves the whole history to +another row. A seat thread titled for a topic drops into `t3:unmapped`. +None of this changes the Human total or the human-per-closed ratio. It only +moves counts between rows, but Gate F reads one seat's row. + +**Header cross-check.** The headers already say which seat a thread belongs +to. For every user message whose header matches the fixed 6a rule and whose +`to:` id equals the message's own `thread_id`: +- in a mapped thread, the `to:` role, lower-cased, must equal that thread's + seat; +- in an unmapped thread, the `to:` role must not be a seat name. + +A conflict exits 1 and names the thread id, its title and both roles. A +header whose `to:` id is some other thread is not checked. The check reads +message text only, not T3 metadata. In a live read at 21:02Z every header +agreed: all 104 addressed to their own thread carried the full thread id and +named that thread's seat (Sage 40, Darkwing 15, Filbert 18, Dewey 15, Rocko +16). + +It catches a seat thread renamed to another seat or to a topic, once any +agent writes to it. It also catches an auto-titled thread that agents +address by a different seat. It misses a thread no agent ever writes to. +Such a thread can only add human counts to a seat's row, never hide them, so +for Gate F it errs toward a visible failure. The README says so. + +**Unmapped row.** Unmapped threads go in one Table 2 row, `t3:unmapped` +(Sage's ruling), so their human messages still reach the Human column and +the human-per-closed ratio. + +**Mapping in the JSON.** For each seat, the T3 thread ids and titles that +made its row, and the unmapped thread ids and titles. Anyone checking a Gate +F result can then see which threads the row came from. + +A seat's row sums its Pi and T3 counts, and the JSON splits them by source. +Nothing is counted twice. Every T3 session today runs on `claudeAgent` or +`codex`, which don't write `.pi/state`, and Filbert found no T3 header in any +Pi log (6a record). + +**Excluded,** with the reason stated in the README: +- Imported threads (`thread_id` starting `import:`, events marked + `historyImport`). They are partial copies of Claude Code sessions, not T3 + traffic: 55 user messages in two threads here. +- Deleted threads (`deleted_at` set). Across all projects there are 3, with + 3 messages. Archived threads count. +- Threads in other T3 projects. Live, there is a project at `/home/jwoltje` + and a deleted one at `/mnt/storage/src`. A thread in either could work on + this repository and would not be counted. The workspace-root rule is still + the right one, but the README names this blind spot. + +## 4. What fails closed + +The source is on by default (Sage's ruling). `--no-t3` turns it off, and the +report then says T3 was not read. `--t3-db ` reads another database +file instead of `~/.t3/userdata/state.sqlite`. It exists for fixtures and +gets the same checks. The JSON records the database path read and whether +it was the default. When it wasn't, the text report adds one line naming the +path, so a Gate F result can't come from a fixture unnoticed. + +Each of these refuses the report with exit 1, the code the ledger already +uses for unreadable session evidence. The report never falls back to Pi logs +alone. Where the database is missing or won't open, the message names +`--no-t3`. +- The database is missing or unreadable, or won't open read-only. That + includes a directory that isn't writable when SQLite needs to create + `-shm`, and a busy error after the timeout. The Pi reader treats a missing + `.pi` as no messages, and this departs from it on purpose. A missing Pi + directory means no Pi seats ran here. A missing T3 database on this host + means the path or T3 changed, and a silent zero is the failure Gate F + exists to prevent. +- `~/.t3`, `~/.t3/userdata` or `state.sqlite` is a symlink. With `--t3-db`, + the file and its directory are checked. The Pi reader checks every + ancestor too, but it skips symlinked entries. Skipping one named file + would be another silent zero, so this reader refuses. +- A table or column the counts need is missing. The reader checks `PRAGMA + table_info` and names what's missing. This catches a T3 upgrade that + changes the schema. +- No project row, or more than one non-deleted row, for this repository root. +- A counted row has a bad `role`, non-string `text`, or a `created_at` that + doesn't parse. The Pi reader already refuses malformed JSONL and bad + timestamps the same way. +- A header conflicts with the title mapping (section 3). + +The reader never reads other files in `~/.t3`. It passes no message text +beyond `messageKind`, `issueNumbers` and the header's `to:` role and id, the +same rule as for Pi logs. + +## 5. Rulings + +Sage ruled on the three questions R1 put to Jason, as lead calls: +1. The source is on by default. A missing or unreadable database exits 1, + and the message names `--no-t3`. +2. Unmapped threads get the `t3:unmapped` row. +3. The 14 free-text headers stay as recorded. They show only in the JSON + diagnostic. + +Sage also ruled that the class fix rides in this build, and that a missing +diagnostic table reads `unknown` (F5). + +## 6. Acceptance for the build + +**No test opens the real `~/.t3`.** Both places in +`packages/ledger/tests/ledger.test.mjs` that spawn `cli.mjs` (the shared +`run()` helper and the direct `spawnSync` at line 66) set `HOME` to the +fixture's temp directory. A test that forgets `--t3-db` or `--no-t3` then +finds no database and fails closed. The existing tests aren't about T3. Each +gets an empty fixture database at the fixture `HOME`'s default path, with +one project row for the fixture root. That row stores +`fs.realpathSync(root)`, because the CLI resolves its root through realpath +and a symlinked temp directory would otherwise not match. So they run with the source on, and +their expected rows don't change. One test asserts that a `HOME` with no +database exits 1 and names `--no-t3`. + +Fixture databases are built with `node:sqlite` in a temp directory, in WAL +mode: +- seat threads and an unmapped thread; imported, deleted and archived + threads; a message outside the date range; +- all three header forms, with uppercase classes in both the tmux preamble + and the T3 header; +- a thread with the same seat title in another project; +- a seat thread renamed to another seat, with an agent header to its own + id, which exits 1; +- a "Sagebrush" title, which stays unmapped; +- a thread titled "Researcher", which maps to the seat that has no thread + live. + +WAL states, each with its own test: +- The newest message is only in `-wal`, with the writer still attached (the + live-T3 case). It is counted. +- T3 stopped: the database closed cleanly with no writer. Counts are + correct, and the main file's bytes are unchanged afterwards. +- `-wal` without `-shm` in a writable directory: made by a child writer with + `wal_autocheckpoint=0` that is SIGKILLed, then `-shm` deleted. The WAL + rows are counted. +- `-wal` without `-shm` in a directory that isn't writable: exit 1, naming + `--no-t3`. Skipped when the tests run as root, where the mode bits don't + bind. +- The stopped case in a directory that isn't writable: exit 1, naming + `--no-t3`. Skipped as root too. + +Also: +- Each other fail-closed case in section 4 has its own test, including a + removed schema column and each symlink. +- A missing `orchestration_events` gives `unknown` for the diagnostic and + the same counts. +- The class fix is proven against HEAD's `messageKind`. An uppercase class + in either preamble counts as agent after the fix and as human before it. +- The JSON lists each seat's threads and the unmapped threads, and the + database path with whether it was the default. A `--t3-db` run prints the + path line in the text report, and a default run doesn't. +- A read against the live database gives the counts in this brief, allowing + for messages sent since. It exits 0 with no header conflict. +- The ledger README's counting rules name the new source, both flags, the + mapping rule and the header check, and the exclusions. That includes the + symlinked-checkout case and the other-project blind spot. +- No suite runs the ledger tests, so the BUILD-LOG entry names the test file. + +## Not in scope + +- Claude Code transcripts (`~/.claude/projects`) and Codex sessions + (`~/.codex/sessions`). T3's database already holds every message T3 + delivered, so those files would only duplicate it. +- Any T3 API call, anything that needs T3 running, and any write beyond + SQLite's own `-wal` and `-shm` handling. +- Fixing the two tmux misclassifications Filbert found in 6a.