fix(ledger): Gate F follow-up, Filbert's notes 1 to 3 (#1506)

Darkwing's follow-up to the T3 thread source: manifest 382f5bb0 pins
t3.mjs, ledger.test.mjs and README.md. Filbert approved it (review
6fd693b6). Ledger 51/51; the eight suites pass on the index.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
2026-09-26 16:57:51 -05:00
co-authored by Claude Opus 5.5
parent a4d38a3d93
commit 3a209eeafe
9 changed files with 268 additions and 7 deletions
@@ -0,0 +1,3 @@
5acbc1075a5d0ad709faf14698235c8c2332c408cb4fccc580bd4a75e2c314fb packages/ledger/src/t3.mjs
6546dbaf59c046d599a9d378c1a1f2d9afc9487189db06f50fa3201c9cadc63b packages/ledger/tests/ledger.test.mjs
101013def3b168ae1b7e291ff86200b49ed6b27927787585d5ca32a283bf38bd packages/ledger/README.md
@@ -0,0 +1,67 @@
# Gate F follow-up: Filbert's notes 1 to 3 (#1506), candidate for review
Darkwing, 2026-09-26. Filbert's build review
(`agents/filbert/work/ledger-t3-build-review-2026-09-26.md`, e47ec6da) left
four nonblocking notes on Gate F (136958c9). Sage asked for 1 to 3 as one small
change that Filbert reviews and Sage commits. Note 4, snapshot isolation, went
to DEFERRED (a68dc174). Base is HEAD a4d38a3d, which changes nothing under
`packages/ledger` since 136958c9. Nothing is committed or pushed.
`followup-manifest.sha256` pins the three files. `followup.patch` is the diff
against a4d38a3d.
## Changes
1. **U+2029.** The splitter test now writes a Pi entry holding a raw U+2028
and a raw U+2029, with CRLF endings, and asserts that the file contains
both. The README line names both characters. `ledger.mjs` is unchanged,
because the splitter already ends lines at `\n` only.
2. **Diagnostic.** Three new tests:
- A human message with no `thread.message-sent` event gives
`humanWithoutEvent: 1`.
- A `thread.message-sent` event whose payload doesn't parse makes both
diagnostic fields `unknown` and leaves `seats` unchanged.
- The same for an event whose `messageId` isn't a string. Filbert didn't
list this one, but it's the third `return null` in `origins()` and had
no test either.
3. **Rethrow.** `readT3`'s catch now rethrows anything that is not a
`SourceError` and carries no numeric `errcode`. The CLI prints such an
error as `Ledger failed: cannot read source evidence`, exit 1. That's the
CLI's existing message for a non-source error, and it no longer points
at SQLite or `--no-t3`. With only numeric errcodes left, the message's
`?? 'error'` fallback could no longer fire, so I removed it. The new test
calls `readT3` in process with an explicit fixture path and a `null`
range, so `inRange` throws a `TypeError` inside the read transaction. It
asserts the `TypeError` comes out. It never touches the real `~/.t3`, and
the fixture comment says so.
## Evidence
- Ledger tests: 51/51, the Gate F 47 plus 4 new.
- Mutations on a scratch copy of the package. The three `gitea-helper` tests
fail in every scratch copy, as before, so the counts leave them out:
| Mutation | Result |
|---|---|
| `humanWithoutEvent` hardcoded to 0 | 1 fails (no-event test) |
| unparseable event skipped (`continue`) | 1 fails (unparseable test) |
| non-string `messageId` skipped | 1 fails (messageId test) |
| rethrow removed (Gate F catch) | 1 fails (rethrow test) |
| splitter also splits at U+2028 | 1 fails (splitter test) |
| splitter also splits at U+2029 | 1 fails (splitter test) |
My first try at the last two put a raw U+2028 or U+2029 in the regex
source. That ends a JS regex literal, so the whole test file failed to
load, which doesn't count as a kill. I reran with the escape written out
literally, and the rows above come from that rerun.
- Eight suites on a local clone of a4d38a3d with the three files: config 24,
task 90, foundation 43, conductor 17, release 14, auth 15, discord 63,
extension-package 18. I ran them twice, and the second run was on the final
files after the errcode edit.
- Union on the same clone. Control-board, webui, seat, mosaic, ledger and
discord, plus conversation, which CHAT-02 committed: 474/474 twice before
the errcode edit and once after. No `ledger-*` temp directories remained.
- Live read, `--since 2026-09-01 --until 2026-09-26 --no-issues --json`, at
2026-09-26T21:47Z: exit 0, no header conflict, diagnostic
`{humanSentThroughApi: 15, humanWithoutEvent: 0}`, two imported threads
excluded. The Gate F build read 14; messages have been sent since then.
@@ -0,0 +1,99 @@
diff --git a/packages/ledger/README.md b/packages/ledger/README.md
index 393e9c37..3a2ce27c 100644
--- a/packages/ledger/README.md
+++ b/packages/ledger/README.md
@@ -39,7 +39,8 @@ No install, build, service restart, or configuration change is needed.
duplicated entries in copied logs are not deduplicated. No transcript content
leaves the parser. Assistant messages and logs outside repo seats do not count.
Symlink source directories are refused and symlink files are not followed.
- A line ends at `\n` only. A U+2028 inside a JSON string does not split a record.
+ A line ends at `\n` only. A U+2028 or U+2029 inside a JSON string does not
+ split a record.
- Table 2 also counts T3 thread messages with role `user`. The T3 source
follows. A seat's row sums its Pi and T3 counts; the JSON keeps the split in
`pi` (Pi rows) and `t3.seats` (T3 rows).
diff --git a/packages/ledger/src/t3.mjs b/packages/ledger/src/t3.mjs
index 9672fcc9..fc9da14e 100644
--- a/packages/ledger/src/t3.mjs
+++ b/packages/ledger/src/t3.mjs
@@ -140,8 +140,10 @@ export async function readT3(root, range, { dbPath = defaultT3Path(), isDefault
result = query(db, root, range, seats);
db.exec('COMMIT');
} catch (error) {
- if (error instanceof SourceError) throw error;
- throw new SourceError(`T3 database cannot be read: ${dbPath} (SQLite ${error.errcode ?? 'error'}); ${SKIP}`);
+ // Only a SQLite failure carries an errcode. Anything else is a bug and
+ // surfaces as itself, not as a database problem.
+ if (error instanceof SourceError || typeof error?.errcode !== 'number') throw error;
+ throw new SourceError(`T3 database cannot be read: ${dbPath} (SQLite ${error.errcode}); ${SKIP}`);
} finally {
try { if (db?.isTransaction) db.exec('ROLLBACK'); } catch { /* the close below still runs */ }
try { db?.close(); } catch { /* nothing was written */ }
diff --git a/packages/ledger/tests/ledger.test.mjs b/packages/ledger/tests/ledger.test.mjs
index 8e60b96b..834dbd55 100644
--- a/packages/ledger/tests/ledger.test.mjs
+++ b/packages/ledger/tests/ledger.test.mjs
@@ -7,6 +7,7 @@ import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { execFileSync, spawnSync } from 'node:child_process';
import { dateRange, messageKind, issueNumbers, totalsLine, summarize } from '../src/ledger.mjs';
+import { readT3 } from '../src/t3.mjs';
const source = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../src');
const range = dateRange('2026-09-06', '2026-09-12');
@@ -49,7 +50,8 @@ const fixtureIssues = [
function fixture(t) {
const root = mkdtempSync(path.join(os.tmpdir(), 'ledger-test-'));
// No test opens the real ~/.t3: every CLI run gets this HOME, with an empty
- // T3 database at the default path. The CLI's root is a realpath.
+ // T3 database at the default path. The one in-process readT3 call passes an
+ // explicit fixture path. The CLI's root is a realpath.
const home = mkdtempSync(path.join(os.tmpdir(), 'ledger-home-'));
t.after(() => { rmSync(root, { recursive: true, force: true }); rmSync(home, { recursive: true, force: true }); });
const defaultDb = path.join(home, '.t3/userdata/state.sqlite');
@@ -134,10 +136,11 @@ test('partial or malformed session log refuses with location, not content', t =>
const f = fixture(t); f.put('.pi/state/alice/sessions/bad.jsonl', '{sensitive'); const r = f.run();
assert.equal(r.status, 1); assert.match(r.stderr, /Malformed session JSON: alice\/bad.jsonl:1/); assert.doesNotMatch(r.stderr, /sensitive/);
});
-test('a U+2028 inside a session string is one line, not a malformed record', t => {
+test('a U+2028 or U+2029 inside a session string is one line, not a malformed record', t => {
const f = fixture(t);
- f.put('.pi/state/bob/sessions/sep.jsonl', [f.entry('Jason: one\u2028two #2'), f.entry('[h:alice -> h:bob] ok')].map(x => JSON.stringify(x)).join('\r\n') + '\r\n');
- assert.ok(readFileSync(path.join(f.root, '.pi/state/bob/sessions/sep.jsonl'), 'utf8').includes('\u2028'));
+ f.put('.pi/state/bob/sessions/sep.jsonl', [f.entry('Jason: one\u2028two\u2029three #2'), f.entry('[h:alice -> h:bob] ok')].map(x => JSON.stringify(x)).join('\r\n') + '\r\n');
+ const written = readFileSync(path.join(f.root, '.pi/state/bob/sessions/sep.jsonl'), 'utf8');
+ assert.ok(written.includes('\u2028') && written.includes('\u2029'));
const r = f.run(['--json']); assert.equal(r.status, 0, r.stderr);
assert.deepEqual(JSON.parse(r.stdout).seats[1], { seat: 'bob', board: 0, agent: 1, human: 1 });
});
@@ -334,6 +337,30 @@ test('T3: a missing orchestration_events makes the diagnostic unknown and keeps
assert.deepEqual(after.t3.diagnostic, { humanSentThroughApi: 'unknown', humanWithoutEvent: 'unknown' });
assert.deepEqual(after.seats, before.seats); assert.deepEqual(after.totals, before.totals);
});
+test('T3: a human message with no event counts in humanWithoutEvent', t => {
+ const f = t3Fixture(t);
+ f.write({ messages: [...f.messages, { thread: T1, text: 'typed, no event', origin: 'none' }] });
+ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
+ assert.deepEqual(JSON.parse(r.stdout).t3.diagnostic, { humanSentThroughApi: 1, humanWithoutEvent: 1 });
+});
+const badEvent = payload => `insert into orchestration_events (stream_id, event_type, payload_json, metadata_json) values ('${T1}', 'thread.message-sent', '${payload}', '{}')`;
+for (const [name, payload] of [['an unparseable event', '{bad'], ['an event with no string messageId', '{"messageId":7}']]) {
+ test(`T3: ${name} makes the diagnostic unknown and keeps the counts`, t => {
+ const f = t3Fixture(t); f.write();
+ const before = JSON.parse(f.run(['--json', '--t3-db', f.db]).stdout);
+ f.write({ after: [badEvent(payload)] });
+ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
+ const after = JSON.parse(r.stdout);
+ assert.deepEqual(after.t3.diagnostic, { humanSentThroughApi: 'unknown', humanWithoutEvent: 'unknown' });
+ assert.deepEqual(after.seats, before.seats);
+ });
+}
+test('T3: an error that is not from SQLite is rethrown, not reported as a database failure', async t => {
+ const f = t3Fixture(t); f.write();
+ // In process with an explicit path, so the real ~/.t3 stays closed. A null
+ // range makes inRange throw a TypeError inside the read transaction.
+ await assert.rejects(readT3(f.real, null, { dbPath: f.db, isDefault: false }), TypeError);
+});
for (const link of ['.t3', '.t3/userdata', '.t3/userdata/state.sqlite']) test(`T3: a symlink at ~/${link} exits 1`, t => {
const f = fixture(t), target = path.join(f.home, 'real', link);
mkdirSync(path.dirname(target), { recursive: true });
@@ -144,3 +144,56 @@ What I confirmed on Node 26.8.1:
The worktree was `/tmp/fb-gatef-wt`, removed after the run. The mutations ran
there and were reverted, and the pins were re-verified before removal. No
commit or push.
## Follow-up: notes 1–3
Candidate: `followup.md` `4bae617f…28e8`, `followup.patch` `856feee4…f44d`,
and `followup-manifest.sha256` `382f5bb0…c971`, which pins `t3.mjs`
`5acbc107…14fb`, `ledger.test.mjs` `6546dbaf…c63b` and `README.md`
`101013de…38bd`. All hashes verify. Base `a4d38a3d`.
- 136958c9 committed the pins I approved: `t3.mjs` `dfb092aa`, tests
`7444abd1`, README `27f7366d`, `ledger.mjs` `0afb0320`, `cli.mjs`
`d6092a53`.
- Nothing under `packages/ledger` changes between 136958c9 and a4d38a3d.
- The patch applies cleanly to a detached worktree at a4d38a3d and
reproduces the three pins.
Checks:
- **Ledger tests, clean worktree:** 51/51.
- **Note 1.**
- The test file now holds a raw U+2028 and a raw U+2029 with CRLF, and it
asserts both are present.
- The README names both.
- `ledger.mjs` is unchanged.
- **Note 2.** The new tests cover:
- `humanWithoutEvent` at 1;
- an unparseable event;
- a non-string `messageId`.
The two bad-event tests assert `unknown` for both fields and unchanged
seats. Those are the three `return null` paths in `origins()` plus the
count, and together they cover my two surviving mutations.
- **Note 3.**
- The rethrow keys on `typeof errcode === 'number'`. The existing tests for
SQLite 14, 1544, 5 and the not-a-database case still pass, so real SQLite
failures keep the `--no-t3` message.
- With the old `if` restored, the new test fails (0/1).
- I also checked the CLI path. I put a `null.x` into `query()` and ran the
CLI against a HOME fixture. It prints "Ledger failed: cannot read source
evidence" and exits 1. With the pin restored, the same fixture exits 0.
That fixture had no `orchestration_events`, and it also showed the
diagnostic's unknown path end to end.
- The in-process test passes an explicit fixture path, so the real `~/.t3`
stays closed.
**Verdict: approve** manifest `382f5bb0`.
**Observation for Sage, not about this patch.** Darkwing's live diagnostic is
now 15 where the build had 14. I traced the extra message read-only. I printed
the header shape only, never the body. It is a message in the Sage thread at
2026-09-26T21:41:24Z, sent through T3's API, whose header reads `[from: main
content-engine agent (<id>) -> to: sage (<id>) class=REPLY]`. The sender role
has spaces, so the 6a rule calls it human, and it counts in Sage's human
column. The diagnostic caught exactly the drift it exists for. It doesn't
affect Gate F, which reads Filbert's row, but Sage's row now holds an agent
message as human.
+2
View File
@@ -401,3 +401,5 @@ are never rewritten or removed; corrections are new entries.
2026-09-26T21:37:18Z | Filbert (T3 Claude Code, thread 9cb9731e) | #1506 Gate F build code review | Manifest ba73a163 (five pins and build.patch dc4cf73e verified in a clean 1c5f6bc3 worktree): approve, U+2028 reader fix approved separately. Ledger 47/47; live read exit 0, 1.6 s, no conflict. Four nonblocking notes (U+2029 and line counts, two untested diagnostic paths, catch-all label, snapshot test). Review e47ec6da. Reported to Darkwing and Sage. No commit or push.
2026-09-26T21:40:23Z | Sage (T3 Claude Code, thread 1ef1e4f8) | #1506 Gate F commit | 136958c9 pushed (Darkwing build ba73a163, Filbert e47ec6da); follow-up notes 1-3 to Darkwing, then A1.
2026-09-26T21:42:17Z | Sage (T3 Claude Code, thread 1ef1e4f8) | #1509 row 25 live check | Jason's button approval passed (DEC-010 request 2, 21:38:58Z). Approver validation gap sent to the SetSpark lead (ac03938d), who does not own shared-signals; owner and DEC-009 go to Jason. Lead decision 19.
2026-09-26T21:54:38Z | Filbert (T3 Claude Code, thread 9cb9731e) | #1506 Gate F follow-up (notes 1-3) code review | Manifest 382f5bb0 (three pins, clean a4d38a3d worktree): approve. Ledger 51/51; rethrow mutation killed; CLI non-SQLite path prints generic message, exit 1. Live diagnostic 15th traced (shape only): content-engine agent header with spaces in Sage thread, counted human. Review 6fd693b6. Reported to Darkwing and Sage. No commit or push.
2026-09-26T21:57:51Z | Sage (T3 Claude Code, thread 1ef1e4f8) | #1506 Gate F follow-up commit | Darkwing notes 1-3 (manifest 382f5bb0, pins and patch verified), Filbert approve 6fd693b6. Ledger 51/51, eight suites green on the index. Header-role gap noted in DEFERRED.
+7
View File
@@ -146,6 +146,13 @@ at every gate. Started 2026-09-12 during the control board MVP.
query in one read transaction, but no test proves a write landing during
the read stays invisible. Filbert's note 4 on the Gate F build (e47ec6da).
Notes 1 to 3 go to Darkwing's follow-up. (2026-09-26, #1506)
- **A T3 sender role with spaces counts as human.** The SetSpark lead's
reply to Sage (21:41:24Z) used the role "main content-engine agent". The
6a rule requires a one-word role, so the ledger counts that message as
human in Sage's row. Gate F reads only Filbert's row, so it is not
affected. Filbert found it through the live diagnostic (14 went to 15).
Sage asked the SetSpark lead to use a one-word role. The rule stays as
it is. (2026-09-26, #1506)
## Queue
+2 -1
View File
@@ -39,7 +39,8 @@ No install, build, service restart, or configuration change is needed.
duplicated entries in copied logs are not deduplicated. No transcript content
leaves the parser. Assistant messages and logs outside repo seats do not count.
Symlink source directories are refused and symlink files are not followed.
A line ends at `\n` only. A U+2028 inside a JSON string does not split a record.
A line ends at `\n` only. A U+2028 or U+2029 inside a JSON string does not
split a record.
- Table 2 also counts T3 thread messages with role `user`. The T3 source
follows. A seat's row sums its Pi and T3 counts; the JSON keeps the split in
`pi` (Pi rows) and `t3.seats` (T3 rows).
+4 -2
View File
@@ -140,8 +140,10 @@ export async function readT3(root, range, { dbPath = defaultT3Path(), isDefault
result = query(db, root, range, seats);
db.exec('COMMIT');
} catch (error) {
if (error instanceof SourceError) throw error;
throw new SourceError(`T3 database cannot be read: ${dbPath} (SQLite ${error.errcode ?? 'error'}); ${SKIP}`);
// Only a SQLite failure carries an errcode. Anything else is a bug and
// surfaces as itself, not as a database problem.
if (error instanceof SourceError || typeof error?.errcode !== 'number') throw error;
throw new SourceError(`T3 database cannot be read: ${dbPath} (SQLite ${error.errcode}); ${SKIP}`);
} finally {
try { if (db?.isTransaction) db.exec('ROLLBACK'); } catch { /* the close below still runs */ }
try { db?.close(); } catch { /* nothing was written */ }
+31 -4
View File
@@ -7,6 +7,7 @@ import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { execFileSync, spawnSync } from 'node:child_process';
import { dateRange, messageKind, issueNumbers, totalsLine, summarize } from '../src/ledger.mjs';
import { readT3 } from '../src/t3.mjs';
const source = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../src');
const range = dateRange('2026-09-06', '2026-09-12');
@@ -49,7 +50,8 @@ const fixtureIssues = [
function fixture(t) {
const root = mkdtempSync(path.join(os.tmpdir(), 'ledger-test-'));
// No test opens the real ~/.t3: every CLI run gets this HOME, with an empty
// T3 database at the default path. The CLI's root is a realpath.
// T3 database at the default path. The one in-process readT3 call passes an
// explicit fixture path. The CLI's root is a realpath.
const home = mkdtempSync(path.join(os.tmpdir(), 'ledger-home-'));
t.after(() => { rmSync(root, { recursive: true, force: true }); rmSync(home, { recursive: true, force: true }); });
const defaultDb = path.join(home, '.t3/userdata/state.sqlite');
@@ -134,10 +136,11 @@ test('partial or malformed session log refuses with location, not content', t =>
const f = fixture(t); f.put('.pi/state/alice/sessions/bad.jsonl', '{sensitive'); const r = f.run();
assert.equal(r.status, 1); assert.match(r.stderr, /Malformed session JSON: alice\/bad.jsonl:1/); assert.doesNotMatch(r.stderr, /sensitive/);
});
test('a U+2028 inside a session string is one line, not a malformed record', t => {
test('a U+2028 or U+2029 inside a session string is one line, not a malformed record', t => {
const f = fixture(t);
f.put('.pi/state/bob/sessions/sep.jsonl', [f.entry('Jason: one\u2028two #2'), f.entry('[h:alice -> h:bob] ok')].map(x => JSON.stringify(x)).join('\r\n') + '\r\n');
assert.ok(readFileSync(path.join(f.root, '.pi/state/bob/sessions/sep.jsonl'), 'utf8').includes('\u2028'));
f.put('.pi/state/bob/sessions/sep.jsonl', [f.entry('Jason: one\u2028two\u2029three #2'), f.entry('[h:alice -> h:bob] ok')].map(x => JSON.stringify(x)).join('\r\n') + '\r\n');
const written = readFileSync(path.join(f.root, '.pi/state/bob/sessions/sep.jsonl'), 'utf8');
assert.ok(written.includes('\u2028') && written.includes('\u2029'));
const r = f.run(['--json']); assert.equal(r.status, 0, r.stderr);
assert.deepEqual(JSON.parse(r.stdout).seats[1], { seat: 'bob', board: 0, agent: 1, human: 1 });
});
@@ -334,6 +337,30 @@ test('T3: a missing orchestration_events makes the diagnostic unknown and keeps
assert.deepEqual(after.t3.diagnostic, { humanSentThroughApi: 'unknown', humanWithoutEvent: 'unknown' });
assert.deepEqual(after.seats, before.seats); assert.deepEqual(after.totals, before.totals);
});
test('T3: a human message with no event counts in humanWithoutEvent', t => {
const f = t3Fixture(t);
f.write({ messages: [...f.messages, { thread: T1, text: 'typed, no event', origin: 'none' }] });
const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
assert.deepEqual(JSON.parse(r.stdout).t3.diagnostic, { humanSentThroughApi: 1, humanWithoutEvent: 1 });
});
const badEvent = payload => `insert into orchestration_events (stream_id, event_type, payload_json, metadata_json) values ('${T1}', 'thread.message-sent', '${payload}', '{}')`;
for (const [name, payload] of [['an unparseable event', '{bad'], ['an event with no string messageId', '{"messageId":7}']]) {
test(`T3: ${name} makes the diagnostic unknown and keeps the counts`, t => {
const f = t3Fixture(t); f.write();
const before = JSON.parse(f.run(['--json', '--t3-db', f.db]).stdout);
f.write({ after: [badEvent(payload)] });
const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
const after = JSON.parse(r.stdout);
assert.deepEqual(after.t3.diagnostic, { humanSentThroughApi: 'unknown', humanWithoutEvent: 'unknown' });
assert.deepEqual(after.seats, before.seats);
});
}
test('T3: an error that is not from SQLite is rethrown, not reported as a database failure', async t => {
const f = t3Fixture(t); f.write();
// In process with an explicit path, so the real ~/.t3 stays closed. A null
// range makes inRange throw a TypeError inside the read transaction.
await assert.rejects(readT3(f.real, null, { dbPath: f.db, isDefault: false }), TypeError);
});
for (const link of ['.t3', '.t3/userdata', '.t3/userdata/state.sqlite']) test(`T3: a symlink at ~/${link} exits 1`, t => {
const f = fixture(t), target = path.join(f.home, 'real', link);
mkdirSync(path.dirname(target), { recursive: true });