packages/ledger/src/t3.mjs reads ~/.t3/userdata/state.sqlite read-only, in one transaction. It maps each thread to a seat by title and checks self-addressed headers. Unmatched threads go in a t3:unmapped row. A missing or locked database exits 1 and names --no-t3. Gate F is on by default (lead decision 12). The 6a uppercase-class fix rides here. Separate item: the Pi session reader splits lines only on \n, so a raw U+2028 or U+2029 in a string no longer splits a record. Node 26.8.1's readline split there, and the live ledger refused on HEAD. Darkwing built to brief R3 (f3c05c1b); manifest ba73a163. Filbert approved the build (e47ec6da) and the U+2028 fix as its own item; brief review be1aa414. On an index export: the eight suites 24/90/43/17/14/15/63/18, ledger 47/47. Four nonblocking notes go to a small follow-up. Co-Authored-By: Claude Opus 5.5 <[email protected]>
829 lines
52 KiB
Diff
829 lines
52 KiB
Diff
diff --git a/packages/ledger/README.md b/packages/ledger/README.md
|
|
index da0ab1c5..393e9c37 100644
|
|
--- a/packages/ledger/README.md
|
|
+++ b/packages/ledger/README.md
|
|
@@ -1,13 +1,16 @@
|
|
# Ledger
|
|
|
|
Read-only counts from local `refactor` commit subjects, one Gitea issue-list
|
|
-request through `scripts/gitea-api.sh`, and repo seats' Pi session logs.
|
|
+request through `scripts/gitea-api.sh`, repo seats' Pi session logs, and T3's
|
|
+thread messages in `~/.t3/userdata/state.sqlite`.
|
|
No board changes, data-root writes, fleet reads, transcript output, or scheduler.
|
|
|
|
```sh
|
|
node packages/ledger/src/cli.mjs --since 2026-09-06 --until 2026-09-12
|
|
node packages/ledger/src/cli.mjs --since 2026-09-06 --until 2026-09-12 --json
|
|
node packages/ledger/src/cli.mjs --since 2026-09-06 --no-issues
|
|
+node packages/ledger/src/cli.mjs --since 2026-09-06 --no-t3
|
|
+node packages/ledger/src/cli.mjs --since 2026-09-06 --t3-db /tmp/fixture.sqlite
|
|
node --test packages/ledger/tests/
|
|
```
|
|
|
|
@@ -36,12 +39,17 @@ No install, build, service restart, or configuration change is needed.
|
|
duplicated entries in copied logs are not deduplicated. No transcript content
|
|
leaves the parser. Assistant messages and logs outside repo seats do not count.
|
|
Symlink source directories are refused and symlink files are not followed.
|
|
+ A line ends at `\n` only. A U+2028 inside a JSON string does not split a record.
|
|
+- Table 2 also counts T3 thread messages with role `user`. The T3 source
|
|
+ follows. A seat's row sums its Pi and T3 counts; the JSON keeps the split in
|
|
+ `pi` (Pi rows) and `t3.seats` (T3 rows).
|
|
- The first text line alone classifies a message. Two addressing forms count:
|
|
the tmux preamble `[host:session -> host:session]` that `agent-send.sh`
|
|
writes, and the T3 header `[from: role (thread-id) -> to: role (thread-id)]`
|
|
from `docs/guides/T3-AGENT-COMMS.md`. Either may carry ` class=<class>` before
|
|
the closing bracket. A preamble whose sender is `control-board` (tmux session
|
|
or T3 role) is board; any other valid preamble is agent; otherwise human.
|
|
+ The class may be in either case: seats send `class=DECISION`.
|
|
This is a format count, not proof of who typed the message. Text blocks are
|
|
joined with newlines. The entry timestamp is used, falling back to the
|
|
message timestamp.
|
|
@@ -53,6 +61,69 @@ No install, build, service restart, or configuration change is needed.
|
|
ratios and durations with one decimal. Titles truncate to 48 characters in
|
|
text only. Missing evidence is the literal string `unknown`.
|
|
|
|
+## T3 source
|
|
+
|
|
+The rules come from `docs/plans/2026-09-26_ledger-t3-source.md` (Gate F).
|
|
+The source is on by default. `--no-t3` skips it, and the report then says
|
|
+`T3: not read (--no-t3)`. `--t3-db <path>` reads another database file with
|
|
+the same checks. The JSON records the database path and whether it was the
|
|
+default. When it wasn't, the text report prints the path, so a fixture result
|
|
+can't pass for a live one. The two flags can't be combined.
|
|
+
|
|
+The reader opens `state.sqlite` read-only through `node:sqlite` and reads no
|
|
+other file in `~/.t3`. It runs every query in one read transaction with a 5 s
|
|
+busy timeout. It never writes the main database file. Like any SQLite
|
|
+connection it may create `-wal` and `-shm` beside it, so a directory that
|
|
+isn't writable refuses when SQLite needs them.
|
|
+
|
|
+- **Project.** Only threads in the one non-deleted T3 project whose
|
|
+ `workspace_root` equals this checkout's root byte for byte. The root is the
|
|
+ realpath of the package, so a project opened through the compatibility
|
|
+ symlink `~/src/mosaic-stack-dev-test` does not match, and the report refuses
|
|
+ with no project.
|
|
+- **Thread to seat.** A thread belongs to seat `<s>` when `<s>` is a real
|
|
+ directory in `agents/` and the lower-cased title equals `<s>` or starts
|
|
+ with `<s>` and a space. "Dewey in Claude" maps to `dewey`; "Sagebrush" maps
|
|
+ to nothing. Several threads can map to one seat. Threads that map to no
|
|
+ seat share one row, `t3:unmapped`, so their human messages still reach the
|
|
+ totals. `t3.seats` and `t3.unmapped` in the JSON list the thread ids and
|
|
+ titles behind each row.
|
|
+- **Titles are current state.** T3 titles an unnamed thread from its first
|
|
+ prompt, and a rename moves a thread's whole history to another row. This
|
|
+ moves counts between rows, never out of the totals.
|
|
+- **Header check.** A user message whose T3 header is addressed to its own
|
|
+ thread id must name that thread's seat as the `to:` role (compared lower
|
|
+ case). In an unmapped thread the `to:` role must not be a seat. A conflict
|
|
+ exits 1 and names the thread, its title and both roles. A header addressed
|
|
+ to another thread isn't checked. The check misses a renamed thread that no
|
|
+ agent writes to. Such a thread can only add human counts to a row.
|
|
+- **Excluded.** Imported threads (id prefix `import:`) are partial copies of
|
|
+ Claude Code sessions, not T3 traffic; every T3 event marked `historyImport`
|
|
+ sits in one today. Deleted threads don't count; archived threads do.
|
|
+ `t3.excluded` gives both thread counts.
|
|
+- **Blind spot.** Threads in other T3 projects are not counted, even if they
|
|
+ worked on this repository. Live, there is a project at `/home/jwoltje` and
|
|
+ a deleted one at `/mnt/storage/src`.
|
|
+- **Diagnostic.** `t3.diagnostic.humanSentThroughApi` counts in-range user
|
|
+ messages the header rule calls human that T3 recorded as sent through its
|
|
+ API (no `appVersion` in the event's origin). Those are seat messages whose
|
|
+ header the rule doesn't accept, such as the older free-text Discord Bot
|
|
+ headers, and would show the next format drift. `humanWithoutEvent` counts
|
|
+ human messages with no `thread.message-sent` event. This is T3's internal
|
|
+ metadata, so it feeds no table or total. If `orchestration_events` or a
|
|
+ column it needs is missing, or an event doesn't parse, both read `unknown`.
|
|
+
|
|
+These refuse the report with exit 1, and the ones about the database name
|
|
+`--no-t3`: a missing, unreadable or unopenable database (including a busy
|
|
+lock past the timeout); a symlink at `~/.t3`, `~/.t3/userdata` or
|
|
+`state.sqlite` (with `--t3-db`, the file or its directory); a missing table or
|
|
+column the counts need; no project or more than one for this root; a message
|
|
+in a counted thread with a role other than `user` or `assistant`, non-text
|
|
+content, or a `created_at` that doesn't parse; a header conflict. A missing Pi
|
|
+directory means no Pi seats ran here; a missing T3 database means the path or
|
|
+T3 changed, so it refuses instead of counting zero. Error messages name ids
|
|
+and paths, never message text.
|
|
+
|
|
## One Gitea call and missing evidence
|
|
|
|
The client requests issues updated since the start date, all states, first page,
|
|
@@ -62,8 +133,8 @@ Use a narrower range or `--no-issues`, not hidden pagination. A commit-linked
|
|
issue not returned by the updated-since query still has a row, with unknown
|
|
metadata. This is the cost of the brief's one-call boundary.
|
|
|
|
-Exit 0 means a report was computed. Exit 1 means bad arguments or unreadable git
|
|
-or session evidence. Malformed JSONL, including a partially written last line,
|
|
+Exit 0 means a report was computed. Exit 1 means bad arguments or unreadable git,
|
|
+session or T3 evidence. Malformed JSONL, including a partially written last line,
|
|
refuses the report; rerun after the seat finishes writing. Exit 2 means issue
|
|
credentials, API, payload, or completeness failure. The CLI never prints API
|
|
error bodies or reads authentication files itself. `--no-issues` makes no API
|
|
@@ -72,8 +143,9 @@ median duration, and human-per-closed ratio. It cannot invent close-only rows.
|
|
|
|
For fixtures, a fake `gitea-api.sh` can be placed first on PATH. Otherwise the
|
|
repository scripts directory is appended to PATH for the issue request.
|
|
-Tests use only temporary repositories, logs, and fake API tools, with no real
|
|
-credentials or network. The helper regression stubs Node before any credential
|
|
+Tests use only temporary repositories, logs, T3 databases and fake API tools,
|
|
+with no real credentials or network. Every CLI run in the tests sets `HOME` to
|
|
+a temporary directory, so no test opens the real `~/.t3`. The helper regression stubs Node before any credential
|
|
read and checks successful GET, successful POST, and failed HTTP status.
|
|
|
|
## Acceptance
|
|
diff --git a/packages/ledger/src/cli.mjs b/packages/ledger/src/cli.mjs
|
|
index 66cd1417..5cf6705c 100644
|
|
--- a/packages/ledger/src/cli.mjs
|
|
+++ b/packages/ledger/src/cli.mjs
|
|
@@ -1,11 +1,12 @@
|
|
#!/usr/bin/env node
|
|
import path from 'node:path';
|
|
import { fileURLToPath } from 'node:url';
|
|
-import { dateRange, readCommits, readIssues, readSessions, summarize, formatTable, SourceError } from './ledger.mjs';
|
|
+import { dateRange, readCommits, readIssues, readSessions, mergeSources, summarize, formatTable, SourceError } from './ledger.mjs';
|
|
+import { readT3, defaultT3Path } from './t3.mjs';
|
|
|
|
-const usage = 'Usage: node packages/ledger/src/cli.mjs --since YYYY-MM-DD [--until YYYY-MM-DD] [--json] [--no-issues]';
|
|
+const usage = 'Usage: node packages/ledger/src/cli.mjs --since YYYY-MM-DD [--until YYYY-MM-DD] [--json] [--no-issues] [--no-t3 | --t3-db PATH]';
|
|
export async function main(args, root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../..')) {
|
|
- let since, until, json = false, noIssues = false;
|
|
+ let since, until, t3Db, json = false, noIssues = false, noT3 = false;
|
|
const seen = new Set();
|
|
for (let i = 0; i < args.length; i++) {
|
|
const flag = args[i];
|
|
@@ -14,13 +15,18 @@ export async function main(args, root = path.resolve(path.dirname(fileURLToPath(
|
|
if (flag === '--help') { console.log(usage); return; }
|
|
if (flag === '--json') json = true;
|
|
else if (flag === '--no-issues') noIssues = true;
|
|
- else if (flag === '--since' || flag === '--until') {
|
|
+ else if (flag === '--no-t3') noT3 = true;
|
|
+ else if (flag === '--t3-db') {
|
|
+ t3Db = args[++i];
|
|
+ if (!t3Db || t3Db.startsWith('--')) throw new SourceError('--t3-db requires a path');
|
|
+ } else if (flag === '--since' || flag === '--until') {
|
|
const value = args[++i];
|
|
if (!value || value.startsWith('--')) throw new SourceError(`${flag} requires a date`);
|
|
if (flag === '--since') since = value; else until = value;
|
|
} else throw new SourceError('Unknown option; ' + usage);
|
|
}
|
|
if (!since) throw new SourceError(usage);
|
|
+ if (noT3 && t3Db !== undefined) throw new SourceError('--no-t3 and --t3-db cannot be combined');
|
|
const range = dateRange(since, until);
|
|
const commits = readCommits(root, range);
|
|
// Fixture tools may be placed first on PATH. The repository client is the
|
|
@@ -30,7 +36,9 @@ export async function main(args, root = path.resolve(path.dirname(fileURLToPath(
|
|
let issues;
|
|
try { issues = noIssues ? null : readIssues(root, range); }
|
|
finally { if (priorPath === undefined) delete process.env.PATH; else process.env.PATH = priorPath; }
|
|
- const sessions = await readSessions(root, range);
|
|
+ // T3 is on by default. A missing or unreadable database refuses the report.
|
|
+ const t3 = noT3 ? null : await readT3(root, range, t3Db === undefined ? { dbPath: defaultT3Path(), isDefault: true } : { dbPath: t3Db, isDefault: false });
|
|
+ const sessions = mergeSources(await readSessions(root, range), t3);
|
|
const report = summarize(range, commits, issues, sessions);
|
|
console.log(json ? JSON.stringify(report, null, 2) : formatTable(report));
|
|
return report;
|
|
diff --git a/packages/ledger/src/ledger.mjs b/packages/ledger/src/ledger.mjs
|
|
index b09e95f5..3669d1cc 100644
|
|
--- a/packages/ledger/src/ledger.mjs
|
|
+++ b/packages/ledger/src/ledger.mjs
|
|
@@ -1,7 +1,6 @@
|
|
import { execFileSync } from 'node:child_process';
|
|
import { createReadStream } from 'node:fs';
|
|
import { readdir, lstat } from 'node:fs/promises';
|
|
-import { createInterface } from 'node:readline';
|
|
import path from 'node:path';
|
|
|
|
const DAY = 86400000;
|
|
@@ -23,7 +22,7 @@ export function dateRange(since, until = new Date().toISOString().slice(0, 10))
|
|
if (end <= start) throw new SourceError('--until must not precede --since');
|
|
return { since, until, start, end };
|
|
}
|
|
-const inRange = (value, range) => {
|
|
+export const inRange = (value, range) => {
|
|
const ms = typeof value === 'number' ? value : Date.parse(value);
|
|
return Number.isFinite(ms) && ms >= range.start && ms < range.end;
|
|
};
|
|
@@ -75,15 +74,32 @@ export function messageText(content) {
|
|
if (Array.isArray(content)) return content.filter(c => c?.type === 'text' && typeof c.text === 'string').map(c => c.text).join('\n');
|
|
return '';
|
|
}
|
|
+// Classes are matched in either case: seats send DECISION and REVIEW-REQUEST.
|
|
+// tmux preamble from agent-send.sh: [host:session -> host:session class=x]
|
|
+const TMUX = /^\[([^\s:\[\]]+):([^\s\[\]]+) -> ([^\s:\[\]]+):([^\s\[\]]+)(?: class=[A-Za-z-]+)?\](?:\s|$)/;
|
|
+// T3 header (docs/guides/T3-AGENT-COMMS.md): [from: role (id) -> to: role (id) class=x]
|
|
+const T3 = /^\[from: ([^\s()\[\]]+) \(([^()\[\]]+)\) -> to: ([^\s()\[\]]+) \(([^()\[\]]+)\)(?: class=[A-Za-z-]+)?\](?:\s|$)/;
|
|
+const firstLine = text => text.split(/\r?\n/, 1)[0];
|
|
+export function t3Header(text) {
|
|
+ const m = firstLine(text).match(T3);
|
|
+ return m ? { from: m[1], fromId: m[2], to: m[3], toId: m[4] } : null;
|
|
+}
|
|
export function messageKind(text) {
|
|
- const firstLine = text.split(/\r?\n/, 1)[0];
|
|
- // tmux preamble from agent-send.sh: [host:session -> host:session class=x]
|
|
- const tmux = firstLine.match(/^\[([^\s:\[\]]+):([^\s\[\]]+) -> ([^\s:\[\]]+):([^\s\[\]]+)(?: class=[a-z-]+)?\](?:\s|$)/);
|
|
- // T3 header (docs/guides/T3-AGENT-COMMS.md): [from: role (id) -> to: role (id) class=x]
|
|
- const t3 = firstLine.match(/^\[from: ([^\s()\[\]]+) \(([^()\[\]]+)\) -> to: ([^\s()\[\]]+) \(([^()\[\]]+)\)(?: class=[a-z-]+)?\](?:\s|$)/);
|
|
- const sender = tmux ? tmux[2] : t3 ? t3[1] : null;
|
|
+ const tmux = firstLine(text).match(TMUX), t3 = t3Header(text);
|
|
+ const sender = tmux ? tmux[2] : t3 ? t3.from : null;
|
|
return sender === null ? 'human' : sender === 'control-board' ? 'board' : 'agent';
|
|
}
|
|
+// JSONL lines end at \n only. readline also ends a line at U+2028, which JSON
|
|
+// allows raw inside a string, so it split valid records (Node 26.8.1).
|
|
+async function* jsonLines(input) {
|
|
+ let rest = '';
|
|
+ for await (const chunk of input) {
|
|
+ const parts = (rest + chunk).split('\n');
|
|
+ rest = parts.pop();
|
|
+ yield* parts;
|
|
+ }
|
|
+ if (rest) yield rest;
|
|
+}
|
|
async function directories(dir, optional = false) {
|
|
try {
|
|
if (!(await lstat(dir)).isDirectory()) throw new SourceError('Session source must be a real directory');
|
|
@@ -93,11 +109,15 @@ async function directories(dir, optional = false) {
|
|
throw new SourceError(`Cannot read ledger directory: ${dir}`);
|
|
}
|
|
}
|
|
+// Seats are the real directories in agents/, sorted.
|
|
+export async function readSeats(root) {
|
|
+ return (await directories(path.join(root, 'agents'))).filter(e => e.isDirectory()).map(e => e.name).sort((a, b) => a.localeCompare(b));
|
|
+}
|
|
export async function readSessions(root, range) {
|
|
const rows = [];
|
|
const mentions = new Map();
|
|
// No symlink traversal, no fleet paths, no transcript content in the report.
|
|
- const agents = (await directories(path.join(root, 'agents'))).filter(e => e.isDirectory()).sort((a, b) => a.name.localeCompare(b.name));
|
|
+ const agents = (await readSeats(root)).map(name => ({ name }));
|
|
const state = path.join(root, '.pi', 'state');
|
|
// Check every source ancestor, not only the leaf directory.
|
|
if (!(await directories(path.join(root, '.pi'), true)).length) return { rows, mentions };
|
|
@@ -109,11 +129,10 @@ export async function readSessions(root, range) {
|
|
const files = (await directories(dir, true)).filter(e => e.isFile() && e.name.endsWith('.jsonl'));
|
|
const row = { seat: agent.name, board: 0, agent: 0, human: 0 };
|
|
for (const file of files) {
|
|
- const input = createReadStream(path.join(dir, file.name));
|
|
- const lines = createInterface({ input, crlfDelay: Infinity });
|
|
+ const input = createReadStream(path.join(dir, file.name), { encoding: 'utf8' });
|
|
let lineNumber = 0;
|
|
try {
|
|
- for await (const line of lines) {
|
|
+ for await (const line of jsonLines(input)) {
|
|
lineNumber++;
|
|
if (!line.trim()) continue;
|
|
let entry;
|
|
@@ -132,12 +151,33 @@ export async function readSessions(root, range) {
|
|
mentions.get(number).add(agent.name);
|
|
}
|
|
}
|
|
- } finally { lines.close(); input.destroy(); }
|
|
+ } finally { input.destroy(); }
|
|
}
|
|
if (row.board + row.agent + row.human) rows.push(row);
|
|
}
|
|
return { rows, mentions };
|
|
}
|
|
+// Adds T3 counts to the Pi rows per seat. Unmapped T3 threads get one row,
|
|
+// last. The report keeps the Pi rows and the T3 section so the split shows.
|
|
+export function mergeSources(pi, t3, unmapped = 't3:unmapped') {
|
|
+ if (!t3) return { rows: pi.rows, mentions: pi.mentions, pi: pi.rows, t3: { read: false } };
|
|
+ const bySeat = new Map(pi.rows.map(r => [r.seat, { ...r }]));
|
|
+ for (const [seat, counts] of t3.rows) {
|
|
+ if (seat === unmapped || !(counts.board + counts.agent + counts.human)) continue;
|
|
+ const row = bySeat.get(seat) ?? { seat, board: 0, agent: 0, human: 0 };
|
|
+ for (const kind of ['board', 'agent', 'human']) row[kind] += counts[kind];
|
|
+ bySeat.set(seat, row);
|
|
+ }
|
|
+ const rows = [...bySeat.values()].sort((a, b) => a.seat.localeCompare(b.seat));
|
|
+ const extra = t3.rows.get(unmapped);
|
|
+ if (extra.board + extra.agent + extra.human) rows.push({ seat: unmapped, ...extra });
|
|
+ const mentions = new Map([...pi.mentions].map(([n, seats]) => [n, new Set(seats)]));
|
|
+ for (const [n, seats] of t3.mentions) {
|
|
+ if (!mentions.has(n)) mentions.set(n, new Set());
|
|
+ for (const seat of seats) mentions.get(n).add(seat);
|
|
+ }
|
|
+ return { rows, mentions, pi: pi.rows, t3: t3.section };
|
|
+}
|
|
const round = value => Math.round(value * 10) / 10;
|
|
function duration(issue) {
|
|
if (!issue) return UNKNOWN;
|
|
@@ -163,13 +203,14 @@ export function summarize(range, commits, issues, sessions) {
|
|
const median = hours.includes(UNKNOWN) ? UNKNOWN : hours.length ?
|
|
round(hours.length % 2 ? hours[middle] : (hours[middle - 1] + hours[middle]) / 2) : 0;
|
|
const human = sessions.rows.reduce((sum, r) => sum + r.human, 0);
|
|
- return { since: range.since, until: range.until, timezone: 'UTC', issues: rows, seats: sessions.rows,
|
|
+ const sources = sessions.t3 ? { pi: sessions.pi, t3: sessions.t3 } : {};
|
|
+ return { since: range.since, until: range.until, timezone: 'UTC', issues: rows, seats: sessions.rows, ...sources,
|
|
totals: { issuesClosed: issues === null ? UNKNOWN : closed.length,
|
|
medianHoursOpen: issues === null ? UNKNOWN : median, commits: commits.length,
|
|
followUpsPerIssue: rows.length ? round(rows.reduce((sum, r) => sum + r.followUps, 0) / rows.length) : 0,
|
|
humanMessagesPerClosedIssue: issues === null ? UNKNOWN : closed.length ? round(human / closed.length) : human ? UNKNOWN : 0 } };
|
|
}
|
|
-const clean = value => String(value).replace(/[\x00-\x1f\x7f-\x9f]/g, ' ');
|
|
+export const clean = value => String(value).replace(/[\x00-\x1f\x7f-\x9f]/g, ' ');
|
|
const decimal = value => typeof value === 'number' ? value.toFixed(1) : value;
|
|
export function totalsLine(t) {
|
|
return `Totals: issues closed ${t.issuesClosed} | median hours open ${decimal(t.medianHoursOpen)} | commits ${t.commits} | follow-ups per issue ${decimal(t.followUpsPerIssue)} | human messages per closed issue ${decimal(t.humanMessagesPerClosedIssue)}`;
|
|
@@ -181,5 +222,12 @@ export function formatTable(report) {
|
|
decimal(r.hoursOpen), r.commits, r.followUps, r.seats.join(', ')].join(' | ')),
|
|
'', 'Seat | Board | Agent | Human',
|
|
...report.seats.map(r => [clean(r.seat), r.board, r.agent, r.human].join(' | ')),
|
|
+ ...t3Line(report.t3),
|
|
'', totalsLine(report.totals)].join('\n');
|
|
}
|
|
+// One line when T3 was skipped or read from somewhere other than the default.
|
|
+function t3Line(t3) {
|
|
+ if (!t3) return [];
|
|
+ if (!t3.read) return ['T3: not read (--no-t3)'];
|
|
+ return t3.database.default ? [] : [`T3: read from ${clean(t3.database.path)}, not the default`];
|
|
+}
|
|
diff --git a/packages/ledger/tests/ledger.test.mjs b/packages/ledger/tests/ledger.test.mjs
|
|
index 175f5d5e..8e60b96b 100644
|
|
--- a/packages/ledger/tests/ledger.test.mjs
|
|
+++ b/packages/ledger/tests/ledger.test.mjs
|
|
@@ -1,6 +1,7 @@
|
|
import test from 'node:test';
|
|
import assert from 'node:assert/strict';
|
|
-import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync, cpSync, symlinkSync } from 'node:fs';
|
|
+import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync, cpSync, symlinkSync, realpathSync } from 'node:fs';
|
|
+import { DatabaseSync } from 'node:sqlite';
|
|
import os from 'node:os';
|
|
import path from 'node:path';
|
|
import { fileURLToPath } from 'node:url';
|
|
@@ -9,13 +10,50 @@ import { dateRange, messageKind, issueNumbers, totalsLine, summarize } from '../
|
|
|
|
const source = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../src');
|
|
const range = dateRange('2026-09-06', '2026-09-12');
|
|
+// T3 fixture schema: the live tables, cut to the columns the reader uses plus
|
|
+// one it doesn't. `text` allows NULL so a non-text row can be tested.
|
|
+const T3_SCHEMA = `
|
|
+ create table projection_projects (project_id text primary key, title text not null, workspace_root text not null, deleted_at text);
|
|
+ create table projection_threads (thread_id text primary key, project_id text not null, title text not null, archived_at text, deleted_at text);
|
|
+ create table projection_thread_messages (message_id text primary key, thread_id text not null, role text not null, text, created_at text not null);
|
|
+ create table orchestration_events (sequence integer primary key autoincrement, stream_id text not null, event_type text not null, payload_json text not null, metadata_json text not null);`;
|
|
+// Writes a T3 database in WAL mode. Threads default to project p1, which is
|
|
+// the fixture root. Returns the open writer when keepOpen is set.
|
|
+function t3db(file, { root, projects, threads = [], messages = [], after = [], keepOpen = false }) {
|
|
+ mkdirSync(path.dirname(file), { recursive: true });
|
|
+ for (const old of [file, `${file}-wal`, `${file}-shm`]) rmSync(old, { force: true });
|
|
+ const db = new DatabaseSync(file);
|
|
+ db.exec('pragma journal_mode=wal'); db.exec(T3_SCHEMA);
|
|
+ for (const [id, workspace, deleted = null] of projects ?? [['p1', root]]) {
|
|
+ db.prepare('insert into projection_projects values (?, ?, ?, ?)').run(id, 'project', workspace, deleted);
|
|
+ }
|
|
+ for (const t of threads) {
|
|
+ db.prepare('insert into projection_threads values (?, ?, ?, ?, ?)').run(t.id, t.project ?? 'p1', t.title, t.archived ?? null, t.deleted ?? null);
|
|
+ }
|
|
+ for (const m of messages) addMessage(db, m);
|
|
+ for (const sql of after) db.exec(sql);
|
|
+ if (keepOpen) return db;
|
|
+ db.close();
|
|
+}
|
|
+let messageId = 0;
|
|
+function addMessage(db, { thread, text, role = 'user', at = '2026-09-08T12:00:00Z', origin = 'app' }) {
|
|
+ const id = `m${++messageId}`;
|
|
+ db.prepare('insert into projection_thread_messages values (?, ?, ?, ?, ?)').run(id, thread, role, text, at);
|
|
+ if (origin !== 'none') db.prepare('insert into orchestration_events (stream_id, event_type, payload_json, metadata_json) values (?, ?, ?, ?)')
|
|
+ .run(thread, 'thread.message-sent', JSON.stringify({ messageId: id, threadId: thread, role, text }), JSON.stringify({ origin: origin === 'app' ? { appVersion: '0.0.0' } : {} }));
|
|
+}
|
|
const fixtureIssues = [
|
|
{ number: 1, title: 'First issue', created_at: '2026-09-06T00:00:00Z', closed_at: '2026-09-07T12:00:00Z' },
|
|
{ number: 2, title: 'Second issue', created_at: '2026-09-06T00:00:00Z', closed_at: null },
|
|
];
|
|
function fixture(t) {
|
|
const root = mkdtempSync(path.join(os.tmpdir(), 'ledger-test-'));
|
|
- t.after(() => rmSync(root, { recursive: true, force: true }));
|
|
+ // No test opens the real ~/.t3: every CLI run gets this HOME, with an empty
|
|
+ // T3 database at the default path. The CLI's root is a realpath.
|
|
+ const home = mkdtempSync(path.join(os.tmpdir(), 'ledger-home-'));
|
|
+ t.after(() => { rmSync(root, { recursive: true, force: true }); rmSync(home, { recursive: true, force: true }); });
|
|
+ const defaultDb = path.join(home, '.t3/userdata/state.sqlite');
|
|
+ t3db(defaultDb, { root: realpathSync(root) });
|
|
const put = (name, data) => { const p = path.join(root, name); mkdirSync(path.dirname(p), { recursive: true }); writeFileSync(p, data); return p; };
|
|
const git = (args, date = '2026-09-07T00:00:00Z') => execFileSync('git', args, { cwd: root, env: { ...process.env, GIT_AUTHOR_DATE: date, GIT_COMMITTER_DATE: date, GIT_CONFIG_NOSYSTEM: '1', GIT_CONFIG_GLOBAL: '/dev/null' }, stdio: 'pipe' });
|
|
git(['init', '-b', 'refactor']); git(['config', 'user.email', '[email protected]']); git(['config', 'user.name', 'Fixture']);
|
|
@@ -30,8 +68,8 @@ function fixture(t) {
|
|
const entry = (text, timestamp = '2026-09-08T12:00:00Z') => ({ type: 'message', timestamp, message: { role: 'user', content: [{ type: 'text', text }] } });
|
|
const logs = [entry('[host:control-board -> host:alice] do #1'), entry('[host:bob -> host:alice] review #2'), entry('build #2'), entry('old #1', '2026-09-05T23:59:59Z'), { type: 'message', timestamp: '2026-09-08T00:00:00Z', message: { role: 'assistant', content: 'not a user #1' } }];
|
|
put('.pi/state/alice/sessions/one.jsonl', logs.map(x => JSON.stringify(x)).join('\n') + '\n');
|
|
- const run = (args = [], env = {}) => spawnSync(process.execPath, [path.join(root, 'packages/ledger/src/cli.mjs'), '--since', '2026-09-06', '--until', '2026-09-12', ...args], { cwd: root, encoding: 'utf8', env: { ...process.env, PATH: `${path.join(root, 'bin')}:${process.env.PATH}`, ISSUES: path.join(root, 'issues.json'), CALLS: path.join(root, 'calls.jsonl'), ...env } });
|
|
- return { root, put, commit, run, entry, logs };
|
|
+ const run = (args = [], env = {}) => spawnSync(process.execPath, [path.join(root, 'packages/ledger/src/cli.mjs'), '--since', '2026-09-06', '--until', '2026-09-12', ...args], { cwd: root, encoding: 'utf8', env: { ...process.env, HOME: home, PATH: `${path.join(root, 'bin')}:${process.env.PATH}`, ISSUES: path.join(root, 'issues.json'), CALLS: path.join(root, 'calls.jsonl'), ...env } });
|
|
+ return { root, real: realpathSync(root), home, defaultDb, put, commit, run, entry, logs };
|
|
}
|
|
test('fixture git subjects only, follow-ups and three session kinds', t => {
|
|
const f = fixture(t), result = f.run(['--json']);
|
|
@@ -63,7 +101,7 @@ test('missing credentials exit 2, no-issues never calls API and shows unknown',
|
|
test('empty range gives no rows and zero totals', t => {
|
|
const f = fixture(t);
|
|
f.put('issues.json', '[]');
|
|
- const result = spawnSync(process.execPath, [path.join(f.root, 'packages/ledger/src/cli.mjs'), '--since', '2027-01-01', '--until', '2027-01-02', '--json'], { encoding: 'utf8', env: { ...process.env, PATH: `${f.root}/bin:${process.env.PATH}`, ISSUES: `${f.root}/issues.json`, CALLS: `${f.root}/calls.jsonl` } });
|
|
+ const result = spawnSync(process.execPath, [path.join(f.root, 'packages/ledger/src/cli.mjs'), '--since', '2027-01-01', '--until', '2027-01-02', '--json'], { encoding: 'utf8', env: { ...process.env, HOME: f.home, PATH: `${f.root}/bin:${process.env.PATH}`, ISSUES: `${f.root}/issues.json`, CALLS: `${f.root}/calls.jsonl` } });
|
|
assert.equal(result.status, 0, result.stderr); const r = JSON.parse(result.stdout);
|
|
assert.deepEqual(r.issues, []); assert.deepEqual(r.seats, []); assert.ok(Object.values(r.totals).every(n => n === 0));
|
|
});
|
|
@@ -96,6 +134,13 @@ test('partial or malformed session log refuses with location, not content', t =>
|
|
const f = fixture(t); f.put('.pi/state/alice/sessions/bad.jsonl', '{sensitive'); const r = f.run();
|
|
assert.equal(r.status, 1); assert.match(r.stderr, /Malformed session JSON: alice\/bad.jsonl:1/); assert.doesNotMatch(r.stderr, /sensitive/);
|
|
});
|
|
+test('a U+2028 inside a session string is one line, not a malformed record', t => {
|
|
+ const f = fixture(t);
|
|
+ f.put('.pi/state/bob/sessions/sep.jsonl', [f.entry('Jason: one\u2028two #2'), f.entry('[h:alice -> h:bob] ok')].map(x => JSON.stringify(x)).join('\r\n') + '\r\n');
|
|
+ assert.ok(readFileSync(path.join(f.root, '.pi/state/bob/sessions/sep.jsonl'), 'utf8').includes('\u2028'));
|
|
+ const r = f.run(['--json']); assert.equal(r.status, 0, r.stderr);
|
|
+ assert.deepEqual(JSON.parse(r.stdout).seats[1], { seat: 'bob', board: 0, agent: 1, human: 1 });
|
|
+});
|
|
test('no sessions is an empty table; symlink source refuses', t => {
|
|
const f = fixture(t); rmSync(path.join(f.root, '.pi'), { recursive: true });
|
|
assert.deepEqual(JSON.parse(f.run(['--json']).stdout).seats, []);
|
|
@@ -140,7 +185,11 @@ test('T3 header: agent, or board from control-board; anything short of the full
|
|
assert.equal(messageKind(`Jason here\n[from: ${sage} -> to: ${filbert}]\nquoted`), 'human');
|
|
assert.equal(messageKind(` [from: ${sage} -> to: ${filbert}]`), 'human');
|
|
assert.equal(messageKind(`[from: sage -> to: filbert]\nno thread ids`), 'human');
|
|
- assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=Actionable]`), 'human');
|
|
+ // Classes match in either case (Gate F). HEAD before the fix called these human.
|
|
+ assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=Actionable]`), 'agent');
|
|
+ assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=REVIEW-REQUEST]\nreview`), 'agent');
|
|
+ assert.equal(messageKind('[h:sage -> h:bob class=DECISION] go'), 'agent');
|
|
+ assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=review_request]`), 'human');
|
|
assert.equal(messageKind(`[from: ${sage} -> to: ${filbert}]trailing`), 'human');
|
|
assert.equal(messageKind(`[From: ${sage} -> to: ${filbert}]`), 'human');
|
|
});
|
|
@@ -148,3 +197,218 @@ test('no closed issues with human messages means undefined ratio, not invented z
|
|
const r = summarize(range, [], [], { rows: [{ seat: 'a', human: 1, board: 0, agent: 0 }], mentions: new Map() });
|
|
assert.equal(r.totals.humanMessagesPerClosedIssue, 'unknown');
|
|
});
|
|
+
|
|
+// T3 thread source (Gate F, docs/plans/2026-09-26_ledger-t3-source.md).
|
|
+const T1 = 't-alice', T2 = 't-bob', T3 = 't-sagebrush', T4 = 't-researcher', T5 = 't-discord';
|
|
+const header = (from, to, toId, cls = '') => `[from: ${from} (x1) -> to: ${to} (${toId})${cls}]`;
|
|
+function t3Fixture(t) {
|
|
+ const f = fixture(t);
|
|
+ for (const seat of ['sage', 'researcher']) mkdirSync(path.join(f.root, 'agents', seat), { recursive: true });
|
|
+ const db = path.join(f.home, 'fixture/t3.sqlite');
|
|
+ const threads = [
|
|
+ { id: T1, title: 'Alice' }, { id: T2, title: 'Bob in Claude', archived: '2026-09-09T00:00:00Z' },
|
|
+ { id: T3, title: 'Sagebrush' }, { id: T4, title: 'Researcher' }, { id: T5, title: 'Discord Bot' },
|
|
+ { id: 'import:claudeAgent:1', title: 'alice' }, { id: 't-deleted', title: 'Alice', deleted: '2026-09-09T00:00:00Z' },
|
|
+ { id: 't-other', project: 'p2', title: 'Alice' },
|
|
+ ];
|
|
+ const messages = [
|
|
+ { thread: T1, text: 'Jason: go #1' },
|
|
+ { thread: T1, text: `${header('sage', 'alice', T1, ' class=REVIEW-REQUEST')}\nreview #2`, origin: 'api' },
|
|
+ { thread: T1, text: '[h:sage -> h:alice class=DECISION] go', origin: 'api' },
|
|
+ { thread: T1, text: `${header('control-board', 'alice', T1)}\nbuzz`, origin: 'api' },
|
|
+ { thread: T1, text: 'outside', at: '2026-09-13T00:00:00Z' },
|
|
+ { thread: T1, text: 'an answer #9', role: 'assistant', origin: 'none' },
|
|
+ { thread: T2, text: 'archived still counts #2' },
|
|
+ { thread: T3, text: 'Sagebrush is not sage' },
|
|
+ { thread: T3, text: `${header('sage', 'discord', T3)}\nnot a seat role`, origin: 'api' },
|
|
+ { thread: T4, text: 'research this' },
|
|
+ { thread: T5, text: '[from: SetSpark coordinator (x1) -> to: Discord Bot (x2)]\nfree text', origin: 'api' },
|
|
+ { thread: 'import:claudeAgent:1', text: 'imported' },
|
|
+ { thread: 't-deleted', text: 'deleted' },
|
|
+ { thread: 't-other', text: 'other project' },
|
|
+ ];
|
|
+ const write = (overrides = {}) => t3db(db, { root: f.real, projects: [['p1', f.real], ['p2', '/elsewhere']], threads, messages, ...overrides });
|
|
+ return { ...f, db, threads, messages, write };
|
|
+}
|
|
+test('T3: seat, archived, unmapped and Researcher threads count; imported, deleted and other-project threads do not', t => {
|
|
+ const f = t3Fixture(t); f.write();
|
|
+ const result = f.run(['--json', '--t3-db', f.db]);
|
|
+ assert.equal(result.status, 0, result.stderr);
|
|
+ const r = JSON.parse(result.stdout);
|
|
+ assert.deepEqual(r.seats, [
|
|
+ { seat: 'alice', board: 2, agent: 3, human: 2 }, { seat: 'bob', board: 0, agent: 0, human: 1 },
|
|
+ { seat: 'researcher', board: 0, agent: 0, human: 1 }, { seat: 't3:unmapped', board: 0, agent: 1, human: 2 },
|
|
+ ]);
|
|
+ assert.deepEqual(r.pi, [{ seat: 'alice', board: 1, agent: 1, human: 1 }]);
|
|
+ assert.deepEqual(r.t3.database, { path: f.db, default: false });
|
|
+ assert.deepEqual(r.t3.seats, [
|
|
+ { seat: 'alice', board: 1, agent: 2, human: 1, threads: [{ id: T1, title: 'Alice', archived: false }] },
|
|
+ { seat: 'bob', board: 0, agent: 0, human: 1, threads: [{ id: T2, title: 'Bob in Claude', archived: true }] },
|
|
+ { seat: 'researcher', board: 0, agent: 0, human: 1, threads: [{ id: T4, title: 'Researcher', archived: false }] },
|
|
+ ]);
|
|
+ assert.deepEqual(r.t3.unmapped, { board: 0, agent: 1, human: 2, threads: [
|
|
+ { id: T5, title: 'Discord Bot', archived: false }, { id: T3, title: 'Sagebrush', archived: false }] });
|
|
+ assert.deepEqual(r.t3.excluded, { importedThreads: 1, deletedThreads: 1 });
|
|
+ // The free-text header counts as human; only the diagnostic shows it was sent through the API.
|
|
+ assert.deepEqual(r.t3.diagnostic, { humanSentThroughApi: 1, humanWithoutEvent: 0 });
|
|
+ assert.equal(r.totals.humanMessagesPerClosedIssue, 6);
|
|
+ assert.deepEqual(r.issues.map(x => [x.issue, x.seats]), [[1, ['alice']], [2, ['alice', 'bob']]]);
|
|
+ const text = f.run(['--t3-db', f.db]);
|
|
+ assert.equal(text.status, 0, text.stderr);
|
|
+ assert.ok(text.stdout.includes(`T3: read from ${f.db}, not the default`));
|
|
+ assert.match(text.stdout, /t3:unmapped \| 0 \| 1 \| 2/);
|
|
+});
|
|
+test('T3: the default path is read from HOME and prints no path line; --no-t3 says so', t => {
|
|
+ const f = t3Fixture(t); f.write();
|
|
+ rmSync(f.defaultDb); cpSync(f.db, f.defaultDb);
|
|
+ const json = JSON.parse(f.run(['--json']).stdout);
|
|
+ assert.deepEqual(json.t3.database, { path: f.defaultDb, default: true });
|
|
+ assert.equal(json.seats.at(-1).seat, 't3:unmapped');
|
|
+ const text = f.run(); assert.equal(text.status, 0, text.stderr); assert.doesNotMatch(text.stdout, /^T3:/m);
|
|
+ rmSync(path.join(f.home, '.t3'), { recursive: true });
|
|
+ const off = f.run(['--no-t3']); assert.equal(off.status, 0, off.stderr);
|
|
+ assert.match(off.stdout, /^T3: not read \(--no-t3\)$/m);
|
|
+ const offJson = JSON.parse(f.run(['--no-t3', '--json']).stdout);
|
|
+ assert.deepEqual(offJson.t3, { read: false }); assert.deepEqual(offJson.seats, [{ seat: 'alice', board: 1, agent: 1, human: 1 }]);
|
|
+ const both = f.run(['--no-t3', '--t3-db', f.db]); assert.equal(both.status, 1); assert.match(both.stderr, /cannot be combined/);
|
|
+ assert.equal(f.run(['--t3-db']).status, 1);
|
|
+});
|
|
+test('T3: a HOME with no database exits 1 and names --no-t3', t => {
|
|
+ const f = fixture(t); rmSync(path.join(f.home, '.t3'), { recursive: true });
|
|
+ const r = f.run(); assert.equal(r.status, 1); assert.equal(r.stdout, '');
|
|
+ assert.match(r.stderr, /T3 database unavailable: .*\.t3 is missing or unreadable; use --no-t3/);
|
|
+});
|
|
+test('T3: a file that is not a database exits 1 and names --no-t3', t => {
|
|
+ const f = fixture(t); writeFileSync(f.defaultDb, 'not sqlite'.repeat(100));
|
|
+ const r = f.run(); assert.equal(r.status, 1); assert.match(r.stderr, /T3 database cannot be read: .*\(SQLite \d+\); use --no-t3/);
|
|
+});
|
|
+test('T3: a seat thread renamed to another seat exits 1 naming thread, title and roles', t => {
|
|
+ const f = t3Fixture(t);
|
|
+ f.write({ messages: [...f.messages, { thread: T2, text: `${header('sage', 'alice', T2, ' class=INFO')}\nfor alice`, origin: 'api' }] });
|
|
+ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1);
|
|
+ assert.equal(r.stderr.trim(), `T3 header conflict: thread ${T2} "Bob in Claude" maps to bob, but a header addresses alice`);
|
|
+});
|
|
+test('T3: an unmapped thread addressed as a seat exits 1', t => {
|
|
+ const f = t3Fixture(t);
|
|
+ f.write({ messages: [...f.messages, { thread: T3, text: `${header('bob', 'Sage', T3)}\nhi`, origin: 'api' }] });
|
|
+ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1);
|
|
+ assert.match(r.stderr, /thread t-sagebrush "Sagebrush" maps to no seat, but a header addresses Sage/);
|
|
+});
|
|
+test('T3: a header to another thread id is not cross-checked', t => {
|
|
+ const f = t3Fixture(t);
|
|
+ f.write({ messages: [...f.messages, { thread: T2, text: `${header('sage', 'alice', T1)}\ncopied`, origin: 'api' }] });
|
|
+ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
|
|
+ assert.equal(JSON.parse(r.stdout).t3.seats[1].agent, 1);
|
|
+});
|
|
+test('T3: no project, or two, for this root exits 1', t => {
|
|
+ const f = t3Fixture(t);
|
|
+ f.write({ projects: [['p1', `${f.real}-link`], ['p2', '/elsewhere']] });
|
|
+ let r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, /T3 has no project for .*symlink does not match/);
|
|
+ f.write({ projects: [['p1', f.real], ['p2', f.real]] });
|
|
+ r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, /T3 has more than one project for/);
|
|
+ f.write({ projects: [['p1', f.real], ['p2', f.real, '2026-09-01T00:00:00Z']] });
|
|
+ assert.equal(f.run(['--t3-db', f.db]).status, 0);
|
|
+});
|
|
+for (const [name, after, pattern] of [
|
|
+ ['a removed column', ['alter table projection_threads drop column title'], /T3 schema changed: missing projection_threads.title/],
|
|
+ ['a missing table', ['drop table projection_thread_messages'], /T3 schema changed: missing projection_thread_messages$/m],
|
|
+]) test(`T3: ${name} exits 1 and names it`, t => {
|
|
+ const f = t3Fixture(t); f.write({ after, messages: [] });
|
|
+ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, pattern);
|
|
+});
|
|
+for (const [name, message, pattern] of [
|
|
+ ['an unknown role', { role: 'system' }, /T3 message m\d+ in thread t-alice has an unknown role/],
|
|
+ ['non-text content', { text: null }, /has non-text content/],
|
|
+ ['an unparseable created_at', { at: 'yesterday' }, /has an invalid created_at/],
|
|
+]) test(`T3: a counted row with ${name} exits 1 without its text`, t => {
|
|
+ const f = t3Fixture(t);
|
|
+ f.write({ messages: [...f.messages, { thread: T1, text: 'secret words', ...message }] });
|
|
+ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, pattern); assert.doesNotMatch(r.stderr, /secret/);
|
|
+});
|
|
+test('T3: a missing orchestration_events makes the diagnostic unknown and keeps the counts', t => {
|
|
+ const f = t3Fixture(t); f.write();
|
|
+ const before = JSON.parse(f.run(['--json', '--t3-db', f.db]).stdout);
|
|
+ f.write({ after: ['drop table orchestration_events'] });
|
|
+ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
|
|
+ const after = JSON.parse(r.stdout);
|
|
+ assert.deepEqual(after.t3.diagnostic, { humanSentThroughApi: 'unknown', humanWithoutEvent: 'unknown' });
|
|
+ assert.deepEqual(after.seats, before.seats); assert.deepEqual(after.totals, before.totals);
|
|
+});
|
|
+for (const link of ['.t3', '.t3/userdata', '.t3/userdata/state.sqlite']) test(`T3: a symlink at ~/${link} exits 1`, t => {
|
|
+ const f = fixture(t), target = path.join(f.home, 'real', link);
|
|
+ mkdirSync(path.dirname(target), { recursive: true });
|
|
+ cpSync(path.join(f.home, link), target, { recursive: true });
|
|
+ rmSync(path.join(f.home, link), { recursive: true }); symlinkSync(target, path.join(f.home, link));
|
|
+ const r = f.run(); assert.equal(r.status, 1); assert.match(r.stderr, new RegExp(`${link.replaceAll('.', '\\.')} is a symlink; use --no-t3`));
|
|
+});
|
|
+test('T3: with --t3-db, a symlinked file or directory exits 1', t => {
|
|
+ const f = t3Fixture(t); f.write();
|
|
+ const file = path.join(f.home, 'file-link.sqlite'); symlinkSync(f.db, file);
|
|
+ let r = f.run(['--t3-db', file]); assert.equal(r.status, 1); assert.match(r.stderr, /file-link.sqlite is a symlink/);
|
|
+ const dir = path.join(f.home, 'dir-link'); symlinkSync(path.dirname(f.db), dir);
|
|
+ r = f.run(['--t3-db', path.join(dir, 't3.sqlite')]); assert.equal(r.status, 1); assert.match(r.stderr, /dir-link is a symlink/);
|
|
+});
|
|
+
|
|
+// WAL states. The CLI reads with mode=ro; it may create -wal and -shm but must
|
|
+// never change the main file.
|
|
+const sha = file => execFileSync('sha256sum', [file], { encoding: 'utf8' }).split(' ')[0];
|
|
+const humans = r => JSON.parse(r.stdout).t3.seats.find(s => s.seat === 'alice').human;
|
|
+const asRoot = process.getuid?.() === 0;
|
|
+function killedWriter(db) {
|
|
+ // A writer that commits into the WAL and dies without a checkpoint.
|
|
+ const code = `const { DatabaseSync } = require('node:sqlite'); const db = new DatabaseSync(${JSON.stringify(db)});
|
|
+ db.exec('pragma wal_autocheckpoint=0');
|
|
+ db.prepare("insert into projection_thread_messages values ('late', 't-alice', 'user', 'late human', '2026-09-08T13:00:00Z')").run();
|
|
+ process.kill(process.pid, 'SIGKILL');`;
|
|
+ const r = spawnSync(process.execPath, ['-e', code]);
|
|
+ assert.equal(r.signal, 'SIGKILL');
|
|
+ rmSync(`${db}-shm`);
|
|
+}
|
|
+function inReadOnlyDir(dir, check) {
|
|
+ execFileSync('chmod', ['0555', dir]);
|
|
+ try { check(); } finally { execFileSync('chmod', ['0755', dir]); }
|
|
+}
|
|
+test('T3 WAL: the newest message only in -wal, writer attached, is counted', t => {
|
|
+ const f = t3Fixture(t), writer = f.write({ keepOpen: true });
|
|
+ t.after(() => writer.close());
|
|
+ writer.exec('pragma wal_autocheckpoint=0');
|
|
+ addMessage(writer, { thread: T1, text: 'newest', at: '2026-09-08T13:00:00Z' });
|
|
+ const main = sha(f.db);
|
|
+ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
|
|
+ assert.equal(humans(r), 2); assert.equal(sha(f.db), main);
|
|
+});
|
|
+test('T3 WAL: stopped cleanly, counts are correct and the main file is unchanged', t => {
|
|
+ const f = t3Fixture(t); f.write();
|
|
+ assert.throws(() => readFileSync(`${f.db}-wal`));
|
|
+ const main = sha(f.db);
|
|
+ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
|
|
+ assert.equal(humans(r), 1); assert.equal(sha(f.db), main);
|
|
+});
|
|
+test('T3 WAL: -wal without -shm in a writable directory is read', t => {
|
|
+ const f = t3Fixture(t); f.write(); killedWriter(f.db);
|
|
+ const main = sha(f.db);
|
|
+ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
|
|
+ assert.equal(humans(r), 2); assert.equal(sha(f.db), main);
|
|
+});
|
|
+test('T3 WAL: -wal without -shm in a read-only directory exits 1', { skip: asRoot && 'mode bits do not bind root' }, t => {
|
|
+ const f = t3Fixture(t); f.write(); killedWriter(f.db);
|
|
+ inReadOnlyDir(path.dirname(f.db), () => {
|
|
+ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, /cannot be read: .*\(SQLite 14\); use --no-t3/);
|
|
+ });
|
|
+});
|
|
+test('T3 WAL: stopped cleanly in a read-only directory exits 1', { skip: asRoot && 'mode bits do not bind root' }, t => {
|
|
+ const f = t3Fixture(t); f.write();
|
|
+ inReadOnlyDir(path.dirname(f.db), () => {
|
|
+ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, /cannot be read: .*\(SQLite 1544\); use --no-t3/);
|
|
+ });
|
|
+});
|
|
+test('T3: a lock held past the 5 s busy timeout exits 1 and names --no-t3', t => {
|
|
+ const f = t3Fixture(t), writer = f.write({ keepOpen: true });
|
|
+ t.after(() => writer.close());
|
|
+ writer.exec('pragma locking_mode=exclusive'); writer.exec('begin exclusive');
|
|
+ addMessage(writer, { thread: T1, text: 'held' });
|
|
+ const started = Date.now(), r = f.run(['--t3-db', f.db]);
|
|
+ writer.exec('commit');
|
|
+ assert.equal(r.status, 1); assert.match(r.stderr, /cannot be read: .*\(SQLite 5\); use --no-t3/);
|
|
+ assert.ok(Date.now() - started >= 4500, 'the reader waited for the busy timeout');
|
|
+});
|
|
diff --git a/packages/ledger/src/t3.mjs b/packages/ledger/src/t3.mjs
|
|
new file mode 100644
|
|
index 00000000..9672fcc9
|
|
--- /dev/null
|
|
+++ b/packages/ledger/src/t3.mjs
|
|
@@ -0,0 +1,151 @@
|
|
+import { lstat } from 'node:fs/promises';
|
|
+import { DatabaseSync } from 'node:sqlite';
|
|
+import { pathToFileURL } from 'node:url';
|
|
+import os from 'node:os';
|
|
+import path from 'node:path';
|
|
+import { SourceError, UNKNOWN, clean, inRange, issueNumbers, messageKind, readSeats, t3Header } from './ledger.mjs';
|
|
+
|
|
+// T3 keeps every thread message in one SQLite database. This reader opens that
|
|
+// file read-only and nothing else in ~/.t3. See
|
|
+// docs/plans/2026-09-26_ledger-t3-source.md for the rules below.
|
|
+export const UNMAPPED = 't3:unmapped';
|
|
+const SKIP = 'use --no-t3 to skip T3';
|
|
+const REQUIRED = {
|
|
+ projection_projects: ['project_id', 'workspace_root', 'deleted_at'],
|
|
+ projection_threads: ['thread_id', 'project_id', 'title', 'archived_at', 'deleted_at'],
|
|
+ projection_thread_messages: ['message_id', 'thread_id', 'role', 'text', 'created_at'],
|
|
+};
|
|
+const DIAGNOSTIC = { orchestration_events: ['stream_id', 'event_type', 'payload_json', 'metadata_json'] };
|
|
+
|
|
+export const defaultT3Path = () => path.join(os.homedir(), '.t3', 'userdata', 'state.sqlite');
|
|
+
|
|
+// Every named path must exist and must not be a symlink. Skipping one would be
|
|
+// a silent zero, so each problem refuses the report.
|
|
+async function checkPaths(dbPath, isDefault) {
|
|
+ const dirs = isDefault ? [path.dirname(path.dirname(dbPath)), path.dirname(dbPath)] : [path.dirname(dbPath)];
|
|
+ for (const [target, wantDir] of [...dirs.map(d => [d, true]), [dbPath, false]]) {
|
|
+ let stat;
|
|
+ try { stat = await lstat(target); }
|
|
+ catch { throw new SourceError(`T3 database unavailable: ${target} is missing or unreadable; ${SKIP}`); }
|
|
+ if (stat.isSymbolicLink()) throw new SourceError(`T3 database refused: ${target} is a symlink; ${SKIP}`);
|
|
+ if (wantDir ? !stat.isDirectory() : !stat.isFile()) {
|
|
+ throw new SourceError(`T3 database refused: ${target} is not a ${wantDir ? 'directory' : 'regular file'}; ${SKIP}`);
|
|
+ }
|
|
+ }
|
|
+}
|
|
+
|
|
+function missingColumns(db, tables) {
|
|
+ const missing = [];
|
|
+ for (const [table, columns] of Object.entries(tables)) {
|
|
+ const have = new Set(db.prepare('select name from pragma_table_info(?)').all(table).map(r => r.name));
|
|
+ if (!have.size) missing.push(table);
|
|
+ else for (const column of columns) if (!have.has(column)) missing.push(`${table}.${column}`);
|
|
+ }
|
|
+ return missing;
|
|
+}
|
|
+
|
|
+// Seat for a thread title: the lower-cased title equals the seat or starts
|
|
+// with the seat and a space. Longest seat first, so the most specific wins.
|
|
+export function seatForTitle(title, seats) {
|
|
+ const lower = title.toLowerCase();
|
|
+ return [...seats].sort((a, b) => b.length - a.length).find(s => lower === s || lower.startsWith(`${s} `)) ?? null;
|
|
+}
|
|
+
|
|
+// Origin per message id from thread.message-sent events. Any missing table,
|
|
+// column or unparseable event makes the diagnostic unknown; it decides nothing.
|
|
+function origins(db, projectId) {
|
|
+ if (missingColumns(db, DIAGNOSTIC).length) return null;
|
|
+ const byMessage = new Map();
|
|
+ const events = db.prepare(`select e.payload_json, e.metadata_json from orchestration_events e
|
|
+ join projection_threads t on t.thread_id = e.stream_id
|
|
+ where e.event_type = 'thread.message-sent' and t.project_id = ?`).all(projectId);
|
|
+ for (const event of events) {
|
|
+ let payload, metadata;
|
|
+ try { payload = JSON.parse(event.payload_json); metadata = JSON.parse(event.metadata_json); }
|
|
+ catch { return null; }
|
|
+ if (typeof payload?.messageId !== 'string') return null;
|
|
+ byMessage.set(payload.messageId, typeof metadata?.origin?.appVersion === 'string');
|
|
+ }
|
|
+ return byMessage;
|
|
+}
|
|
+
|
|
+function query(db, root, range, seats) {
|
|
+ const missing = missingColumns(db, REQUIRED);
|
|
+ if (missing.length) throw new SourceError(`T3 schema changed: missing ${missing.join(', ')}`);
|
|
+ // Compared in JavaScript so a declared collation can't loosen the match.
|
|
+ const projects = db.prepare('select project_id, workspace_root from projection_projects where deleted_at is null').all()
|
|
+ .filter(p => p.workspace_root === root);
|
|
+ if (projects.length !== 1) {
|
|
+ throw new SourceError(`T3 has ${projects.length ? 'more than one project' : 'no project'} for ${root}; a project opened through a symlink does not match; ${SKIP}`);
|
|
+ }
|
|
+ const projectId = projects[0].project_id;
|
|
+ const threads = new Map(), excluded = { importedThreads: 0, deletedThreads: 0 };
|
|
+ for (const t of db.prepare('select thread_id, title, archived_at, deleted_at from projection_threads where project_id = ?').all(projectId)) {
|
|
+ if (typeof t.thread_id !== 'string' || typeof t.title !== 'string') throw new SourceError('T3 thread with a non-text id or title');
|
|
+ if (t.thread_id.startsWith('import:')) { excluded.importedThreads++; continue; }
|
|
+ if (t.deleted_at !== null) { excluded.deletedThreads++; continue; }
|
|
+ threads.set(t.thread_id, { id: t.thread_id, title: t.title, archived: t.archived_at !== null, seat: seatForTitle(t.title, seats) });
|
|
+ }
|
|
+ const rows = new Map([...seats, UNMAPPED].map(s => [s, { board: 0, agent: 0, human: 0 }]));
|
|
+ const mentions = new Map(), human = [];
|
|
+ const messages = db.prepare(`select m.message_id, m.thread_id, m.role, m.text, m.created_at from projection_thread_messages m
|
|
+ join projection_threads t on t.thread_id = m.thread_id where t.project_id = ?`).all(projectId);
|
|
+ for (const m of messages) {
|
|
+ const thread = threads.get(m.thread_id);
|
|
+ if (!thread) continue;
|
|
+ const where = `T3 message ${clean(m.message_id)} in thread ${clean(m.thread_id)}`;
|
|
+ if (m.role !== 'user' && m.role !== 'assistant') throw new SourceError(`${where} has an unknown role`);
|
|
+ if (typeof m.text !== 'string') throw new SourceError(`${where} has non-text content`);
|
|
+ if (typeof m.created_at !== 'string' || !Number.isFinite(Date.parse(m.created_at))) throw new SourceError(`${where} has an invalid created_at`);
|
|
+ if (m.role !== 'user') continue;
|
|
+ // A header addressed to its own thread must agree with the title mapping.
|
|
+ const header = t3Header(m.text);
|
|
+ if (header && header.toId === thread.id) {
|
|
+ const to = header.to.toLowerCase();
|
|
+ if (thread.seat ? to !== thread.seat : seats.includes(to)) {
|
|
+ throw new SourceError(`T3 header conflict: thread ${clean(thread.id)} "${clean(thread.title)}" maps to ${thread.seat ?? 'no seat'}, but a header addresses ${clean(header.to)}`);
|
|
+ }
|
|
+ }
|
|
+ if (!inRange(m.created_at, range)) continue;
|
|
+ const kind = messageKind(m.text), seat = thread.seat ?? UNMAPPED;
|
|
+ rows.get(seat)[kind]++;
|
|
+ if (kind === 'human') human.push(m.message_id);
|
|
+ for (const number of issueNumbers(m.text)) {
|
|
+ if (!mentions.has(number)) mentions.set(number, new Set());
|
|
+ mentions.get(number).add(seat);
|
|
+ }
|
|
+ }
|
|
+ const byMessage = origins(db, projectId);
|
|
+ const sentThroughApi = byMessage === null ? UNKNOWN : human.filter(id => byMessage.get(id) === false).length;
|
|
+ const noEvent = byMessage === null ? UNKNOWN : human.filter(id => !byMessage.has(id)).length;
|
|
+ const listed = seat => [...threads.values()].filter(t => (t.seat ?? UNMAPPED) === seat)
|
|
+ .sort((a, b) => a.id.localeCompare(b.id)).map(({ id, title, archived }) => ({ id, title, archived }));
|
|
+ const seatRows = seats.map(seat => ({ seat, ...rows.get(seat), threads: listed(seat) })).filter(r => r.threads.length);
|
|
+ return { rows, mentions, excluded, seats: seatRows, unmapped: { ...rows.get(UNMAPPED), threads: listed(UNMAPPED) },
|
|
+ diagnostic: { humanSentThroughApi: sentThroughApi, humanWithoutEvent: noEvent } };
|
|
+}
|
|
+
|
|
+// Reads one snapshot of T3's database. Returns the per-seat rows and issue
|
|
+// mentions the ledger merges with Pi, and the report's `t3` section.
|
|
+export async function readT3(root, range, { dbPath = defaultT3Path(), isDefault = true } = {}) {
|
|
+ dbPath = path.resolve(dbPath);
|
|
+ await checkPaths(dbPath, isDefault);
|
|
+ const seats = await readSeats(root);
|
|
+ const url = pathToFileURL(dbPath);
|
|
+ url.searchParams.set('mode', 'ro');
|
|
+ let db, result;
|
|
+ try {
|
|
+ db = new DatabaseSync(url, { readOnly: true, timeout: 5000 });
|
|
+ db.exec('BEGIN');
|
|
+ result = query(db, root, range, seats);
|
|
+ db.exec('COMMIT');
|
|
+ } catch (error) {
|
|
+ if (error instanceof SourceError) throw error;
|
|
+ throw new SourceError(`T3 database cannot be read: ${dbPath} (SQLite ${error.errcode ?? 'error'}); ${SKIP}`);
|
|
+ } finally {
|
|
+ try { if (db?.isTransaction) db.exec('ROLLBACK'); } catch { /* the close below still runs */ }
|
|
+ try { db?.close(); } catch { /* nothing was written */ }
|
|
+ }
|
|
+ const { rows, mentions, ...section } = result;
|
|
+ return { rows, mentions, section: { read: true, database: { path: dbPath, default: isDefault }, ...section } };
|
|
+}
|