fix(ledger): Gate F follow-up, Filbert's notes 1 to 3 (#1506)
Darkwing's follow-up to the T3 thread source: manifest 382f5bb0 pins t3.mjs, ledger.test.mjs and README.md. Filbert approved it (review 6fd693b6). Ledger 51/51; the eight suites pass on the index. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
@@ -0,0 +1,3 @@
|
||||
5acbc1075a5d0ad709faf14698235c8c2332c408cb4fccc580bd4a75e2c314fb packages/ledger/src/t3.mjs
|
||||
6546dbaf59c046d599a9d378c1a1f2d9afc9487189db06f50fa3201c9cadc63b packages/ledger/tests/ledger.test.mjs
|
||||
101013def3b168ae1b7e291ff86200b49ed6b27927787585d5ca32a283bf38bd packages/ledger/README.md
|
||||
@@ -0,0 +1,67 @@
|
||||
# Gate F follow-up: Filbert's notes 1 to 3 (#1506), candidate for review
|
||||
|
||||
Darkwing, 2026-09-26. Filbert's build review
|
||||
(`agents/filbert/work/ledger-t3-build-review-2026-09-26.md`, e47ec6da) left
|
||||
four nonblocking notes on Gate F (136958c9). Sage asked for 1 to 3 as one small
|
||||
change that Filbert reviews and Sage commits. Note 4, snapshot isolation, went
|
||||
to DEFERRED (a68dc174). Base is HEAD a4d38a3d, which changes nothing under
|
||||
`packages/ledger` since 136958c9. Nothing is committed or pushed.
|
||||
|
||||
`followup-manifest.sha256` pins the three files. `followup.patch` is the diff
|
||||
against a4d38a3d.
|
||||
|
||||
## Changes
|
||||
|
||||
1. **U+2029.** The splitter test now writes a Pi entry holding a raw U+2028
|
||||
and a raw U+2029, with CRLF endings, and asserts that the file contains
|
||||
both. The README line names both characters. `ledger.mjs` is unchanged,
|
||||
because the splitter already ends lines at `\n` only.
|
||||
2. **Diagnostic.** Three new tests:
|
||||
- A human message with no `thread.message-sent` event gives
|
||||
`humanWithoutEvent: 1`.
|
||||
- A `thread.message-sent` event whose payload doesn't parse makes both
|
||||
diagnostic fields `unknown` and leaves `seats` unchanged.
|
||||
- The same for an event whose `messageId` isn't a string. Filbert didn't
|
||||
list this one, but it's the third `return null` in `origins()` and had
|
||||
no test either.
|
||||
3. **Rethrow.** `readT3`'s catch now rethrows anything that is not a
|
||||
`SourceError` and carries no numeric `errcode`. The CLI prints such an
|
||||
error as `Ledger failed: cannot read source evidence`, exit 1. That's the
|
||||
CLI's existing message for a non-source error, and it no longer points
|
||||
at SQLite or `--no-t3`. With only numeric errcodes left, the message's
|
||||
`?? 'error'` fallback could no longer fire, so I removed it. The new test
|
||||
calls `readT3` in process with an explicit fixture path and a `null`
|
||||
range, so `inRange` throws a `TypeError` inside the read transaction. It
|
||||
asserts the `TypeError` comes out. It never touches the real `~/.t3`, and
|
||||
the fixture comment says so.
|
||||
|
||||
## Evidence
|
||||
|
||||
- Ledger tests: 51/51, the Gate F 47 plus 4 new.
|
||||
- Mutations on a scratch copy of the package. The three `gitea-helper` tests
|
||||
fail in every scratch copy, as before, so the counts leave them out:
|
||||
|
||||
| Mutation | Result |
|
||||
|---|---|
|
||||
| `humanWithoutEvent` hardcoded to 0 | 1 fails (no-event test) |
|
||||
| unparseable event skipped (`continue`) | 1 fails (unparseable test) |
|
||||
| non-string `messageId` skipped | 1 fails (messageId test) |
|
||||
| rethrow removed (Gate F catch) | 1 fails (rethrow test) |
|
||||
| splitter also splits at U+2028 | 1 fails (splitter test) |
|
||||
| splitter also splits at U+2029 | 1 fails (splitter test) |
|
||||
|
||||
My first try at the last two put a raw U+2028 or U+2029 in the regex
|
||||
source. That ends a JS regex literal, so the whole test file failed to
|
||||
load, which doesn't count as a kill. I reran with the escape written out
|
||||
literally, and the rows above come from that rerun.
|
||||
- Eight suites on a local clone of a4d38a3d with the three files: config 24,
|
||||
task 90, foundation 43, conductor 17, release 14, auth 15, discord 63,
|
||||
extension-package 18. I ran them twice, and the second run was on the final
|
||||
files after the errcode edit.
|
||||
- Union on the same clone. Control-board, webui, seat, mosaic, ledger and
|
||||
discord, plus conversation, which CHAT-02 committed: 474/474 twice before
|
||||
the errcode edit and once after. No `ledger-*` temp directories remained.
|
||||
- Live read, `--since 2026-09-01 --until 2026-09-26 --no-issues --json`, at
|
||||
2026-09-26T21:47Z: exit 0, no header conflict, diagnostic
|
||||
`{humanSentThroughApi: 15, humanWithoutEvent: 0}`, two imported threads
|
||||
excluded. The Gate F build read 14; messages have been sent since then.
|
||||
@@ -0,0 +1,99 @@
|
||||
diff --git a/packages/ledger/README.md b/packages/ledger/README.md
|
||||
index 393e9c37..3a2ce27c 100644
|
||||
--- a/packages/ledger/README.md
|
||||
+++ b/packages/ledger/README.md
|
||||
@@ -39,7 +39,8 @@ No install, build, service restart, or configuration change is needed.
|
||||
duplicated entries in copied logs are not deduplicated. No transcript content
|
||||
leaves the parser. Assistant messages and logs outside repo seats do not count.
|
||||
Symlink source directories are refused and symlink files are not followed.
|
||||
- A line ends at `\n` only. A U+2028 inside a JSON string does not split a record.
|
||||
+ A line ends at `\n` only. A U+2028 or U+2029 inside a JSON string does not
|
||||
+ split a record.
|
||||
- Table 2 also counts T3 thread messages with role `user`. The T3 source
|
||||
follows. A seat's row sums its Pi and T3 counts; the JSON keeps the split in
|
||||
`pi` (Pi rows) and `t3.seats` (T3 rows).
|
||||
diff --git a/packages/ledger/src/t3.mjs b/packages/ledger/src/t3.mjs
|
||||
index 9672fcc9..fc9da14e 100644
|
||||
--- a/packages/ledger/src/t3.mjs
|
||||
+++ b/packages/ledger/src/t3.mjs
|
||||
@@ -140,8 +140,10 @@ export async function readT3(root, range, { dbPath = defaultT3Path(), isDefault
|
||||
result = query(db, root, range, seats);
|
||||
db.exec('COMMIT');
|
||||
} catch (error) {
|
||||
- if (error instanceof SourceError) throw error;
|
||||
- throw new SourceError(`T3 database cannot be read: ${dbPath} (SQLite ${error.errcode ?? 'error'}); ${SKIP}`);
|
||||
+ // Only a SQLite failure carries an errcode. Anything else is a bug and
|
||||
+ // surfaces as itself, not as a database problem.
|
||||
+ if (error instanceof SourceError || typeof error?.errcode !== 'number') throw error;
|
||||
+ throw new SourceError(`T3 database cannot be read: ${dbPath} (SQLite ${error.errcode}); ${SKIP}`);
|
||||
} finally {
|
||||
try { if (db?.isTransaction) db.exec('ROLLBACK'); } catch { /* the close below still runs */ }
|
||||
try { db?.close(); } catch { /* nothing was written */ }
|
||||
diff --git a/packages/ledger/tests/ledger.test.mjs b/packages/ledger/tests/ledger.test.mjs
|
||||
index 8e60b96b..834dbd55 100644
|
||||
--- a/packages/ledger/tests/ledger.test.mjs
|
||||
+++ b/packages/ledger/tests/ledger.test.mjs
|
||||
@@ -7,6 +7,7 @@ import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { execFileSync, spawnSync } from 'node:child_process';
|
||||
import { dateRange, messageKind, issueNumbers, totalsLine, summarize } from '../src/ledger.mjs';
|
||||
+import { readT3 } from '../src/t3.mjs';
|
||||
|
||||
const source = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../src');
|
||||
const range = dateRange('2026-09-06', '2026-09-12');
|
||||
@@ -49,7 +50,8 @@ const fixtureIssues = [
|
||||
function fixture(t) {
|
||||
const root = mkdtempSync(path.join(os.tmpdir(), 'ledger-test-'));
|
||||
// No test opens the real ~/.t3: every CLI run gets this HOME, with an empty
|
||||
- // T3 database at the default path. The CLI's root is a realpath.
|
||||
+ // T3 database at the default path. The one in-process readT3 call passes an
|
||||
+ // explicit fixture path. The CLI's root is a realpath.
|
||||
const home = mkdtempSync(path.join(os.tmpdir(), 'ledger-home-'));
|
||||
t.after(() => { rmSync(root, { recursive: true, force: true }); rmSync(home, { recursive: true, force: true }); });
|
||||
const defaultDb = path.join(home, '.t3/userdata/state.sqlite');
|
||||
@@ -134,10 +136,11 @@ test('partial or malformed session log refuses with location, not content', t =>
|
||||
const f = fixture(t); f.put('.pi/state/alice/sessions/bad.jsonl', '{sensitive'); const r = f.run();
|
||||
assert.equal(r.status, 1); assert.match(r.stderr, /Malformed session JSON: alice\/bad.jsonl:1/); assert.doesNotMatch(r.stderr, /sensitive/);
|
||||
});
|
||||
-test('a U+2028 inside a session string is one line, not a malformed record', t => {
|
||||
+test('a U+2028 or U+2029 inside a session string is one line, not a malformed record', t => {
|
||||
const f = fixture(t);
|
||||
- f.put('.pi/state/bob/sessions/sep.jsonl', [f.entry('Jason: one\u2028two #2'), f.entry('[h:alice -> h:bob] ok')].map(x => JSON.stringify(x)).join('\r\n') + '\r\n');
|
||||
- assert.ok(readFileSync(path.join(f.root, '.pi/state/bob/sessions/sep.jsonl'), 'utf8').includes('\u2028'));
|
||||
+ f.put('.pi/state/bob/sessions/sep.jsonl', [f.entry('Jason: one\u2028two\u2029three #2'), f.entry('[h:alice -> h:bob] ok')].map(x => JSON.stringify(x)).join('\r\n') + '\r\n');
|
||||
+ const written = readFileSync(path.join(f.root, '.pi/state/bob/sessions/sep.jsonl'), 'utf8');
|
||||
+ assert.ok(written.includes('\u2028') && written.includes('\u2029'));
|
||||
const r = f.run(['--json']); assert.equal(r.status, 0, r.stderr);
|
||||
assert.deepEqual(JSON.parse(r.stdout).seats[1], { seat: 'bob', board: 0, agent: 1, human: 1 });
|
||||
});
|
||||
@@ -334,6 +337,30 @@ test('T3: a missing orchestration_events makes the diagnostic unknown and keeps
|
||||
assert.deepEqual(after.t3.diagnostic, { humanSentThroughApi: 'unknown', humanWithoutEvent: 'unknown' });
|
||||
assert.deepEqual(after.seats, before.seats); assert.deepEqual(after.totals, before.totals);
|
||||
});
|
||||
+test('T3: a human message with no event counts in humanWithoutEvent', t => {
|
||||
+ const f = t3Fixture(t);
|
||||
+ f.write({ messages: [...f.messages, { thread: T1, text: 'typed, no event', origin: 'none' }] });
|
||||
+ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
|
||||
+ assert.deepEqual(JSON.parse(r.stdout).t3.diagnostic, { humanSentThroughApi: 1, humanWithoutEvent: 1 });
|
||||
+});
|
||||
+const badEvent = payload => `insert into orchestration_events (stream_id, event_type, payload_json, metadata_json) values ('${T1}', 'thread.message-sent', '${payload}', '{}')`;
|
||||
+for (const [name, payload] of [['an unparseable event', '{bad'], ['an event with no string messageId', '{"messageId":7}']]) {
|
||||
+ test(`T3: ${name} makes the diagnostic unknown and keeps the counts`, t => {
|
||||
+ const f = t3Fixture(t); f.write();
|
||||
+ const before = JSON.parse(f.run(['--json', '--t3-db', f.db]).stdout);
|
||||
+ f.write({ after: [badEvent(payload)] });
|
||||
+ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
|
||||
+ const after = JSON.parse(r.stdout);
|
||||
+ assert.deepEqual(after.t3.diagnostic, { humanSentThroughApi: 'unknown', humanWithoutEvent: 'unknown' });
|
||||
+ assert.deepEqual(after.seats, before.seats);
|
||||
+ });
|
||||
+}
|
||||
+test('T3: an error that is not from SQLite is rethrown, not reported as a database failure', async t => {
|
||||
+ const f = t3Fixture(t); f.write();
|
||||
+ // In process with an explicit path, so the real ~/.t3 stays closed. A null
|
||||
+ // range makes inRange throw a TypeError inside the read transaction.
|
||||
+ await assert.rejects(readT3(f.real, null, { dbPath: f.db, isDefault: false }), TypeError);
|
||||
+});
|
||||
for (const link of ['.t3', '.t3/userdata', '.t3/userdata/state.sqlite']) test(`T3: a symlink at ~/${link} exits 1`, t => {
|
||||
const f = fixture(t), target = path.join(f.home, 'real', link);
|
||||
mkdirSync(path.dirname(target), { recursive: true });
|
||||
Reference in New Issue
Block a user