jason.woltje
158205c5cc
Unclaimed files left in the canonical checkout. Inventory: canonical-cleanup-inventory.md (commit-as-evidence groups). detect-secrets scan clean on 2026-11-02: hits are sha256 pins, git object ids, test sentinels already in tracked tests and unit-file lines.
168 lines
10 KiB
Markdown
168 lines
10 KiB
Markdown
# Row 181: the restore guard counts every row a restore drops
|
|
|
|
Brief: "# Brief: the restore guard counts every row a restore drops (row B)" in
|
|
`docs/plans/2026-10-20_stack-release-after-window.md` (blob aafa471ae). Gitea #1679, Vikunja P19 390. Owner
|
|
Darkwing, reviewer Filbert, gate owner Sage. Sage's rulings in sage-1163 (restated in sage-1180) settle the
|
|
lists and the deviations below.
|
|
|
|
## Files
|
|
|
|
| File | What |
|
|
|---|---|
|
|
| `build.patch` | the change, `git diff a757f8fe6` over the three paths below |
|
|
| `candidate-manifest.sha256` | one line per changed path |
|
|
|
|
- Base: a757f8fe6 (row 180's landing, sage-1180).
|
|
- Changed: `scripts/stack-release.sh`, `packages/cli/tests/stack-release.test.mjs`, `docs/TOOLS.md`. These are
|
|
the brief's touched files. Nothing else changed.
|
|
- Row 178 (Rocko) also touches `scripts/stack-release.sh`. Whichever lands second rebases (sage-1180).
|
|
- Line numbers in the script and test are at the candidate. Bus line numbers are at a757f8fe6.
|
|
|
|
## Item 1: the tables a bus writer fills without an event
|
|
|
|
`COUNTED` (stack-release.sh:296) sits next to `STORE_JS`, in schema order:
|
|
|
|
```
|
|
role_claims thread_claims decision_events messages deliveries task_snapshots projects threads thread_opens instance_names usage_records
|
|
```
|
|
|
|
The only event writer is `#event` (broker.mjs:551, the INSERT at :561). The table below covers every INSERT
|
|
in `packages/bus/src` (`grep -nE "INSERT (OR [A-Z]+ )?INTO|UPDATE [a-z_]+ SET"`). Outside the bus package, no
|
|
code under `packages/` or `scripts/` writes these tables. The one other hit is a slice 1 prototype under
|
|
`agents/darkwing/work/slice1-proto/`, which never opens a live store.
|
|
|
|
| Table | Writer (broker.mjs unless named) | Why no event covers it |
|
|
|---|---|---|
|
|
| role_claims | :1575 role.claim (case :1542); :2483 `#claimEnd` (fn :2467), reached from role.release :1591, threadIdle :401 and :506, land-relaunch :1062, limit-check :1365, revoke :1614 and :1619, assistant.stop :2278 | none of these write an event in the same transaction |
|
|
| thread_claims | :1551; :2470 `#claimEnd` | same |
|
|
| decision_events | :1687 (withdraw), :1734, :1769 | the event at :1778 is written only `if (s.human)` |
|
|
| messages | :999, :1806, :1894 (tester message.send), :2377 (thread.send), :2518, :2565, :2596 (`#specNotice` :2588, from :1450) | no event |
|
|
| deliveries | :2645 `#delivery` (fn :2643), from message.receive :1915 and :1933, read :1953, routeWaiting :2661, routeThread :2670 | no event (deviation 1) |
|
|
| task_snapshots | :1159 `#snapshot` (:1145) | recordTask's events are optional (:1116-1139) |
|
|
| projects | :2301 (case :2292) | no event |
|
|
| threads | :2331 (case :2318) | no event |
|
|
| thread_opens | :2362 thread.open, the tester path included (row 177) | no event |
|
|
| instance_names | :2257 name.set (:2242) | no event; consumeAuthority (:2255) is agents only |
|
|
| usage_records | usage.mjs:92, its own transaction (:88-98), `at` from `new Date()` | no event |
|
|
|
|
Tables that are not counted:
|
|
|
|
- `events` is list 1, counted as before.
|
|
- `decisions` (list 3) is written only at :1696, and the `action.allowed` event at :1716 is in the same
|
|
transaction.
|
|
- List 4 has no `at` column: `meta`, `task_comment_cursors` (:1129), `message_threads` (:2535) and
|
|
`sqlite_sequence`. `#route` (:2535) writes a `message_threads` row only right after a `messages` row, which
|
|
is counted (for example :2518 then :2531). A `task_comment_cursors` advance can be written with no event and no counted row, so a restore can
|
|
drop it uncounted. The cost is that the PM sees comments a second time. Follow-up note F1 below.
|
|
|
|
### The UPDATE list is empty
|
|
|
|
Every one of the 16 schema tables has a `BEFORE UPDATE` abort trigger, and a `BEFORE DELETE` one
|
|
(`packages/bus/schema.sql`, for example meta :129-130). The only UPDATE in the bus source is the migration
|
|
step at store.mjs:213-214, which drops `meta_no_update` inside the step's transaction (:212) and puts it back
|
|
before the step commits. The guard counts rows by `at`, and no write path changes a row once written.
|
|
|
|
### Deviations from the brief's examples (sage-1163; the brief is not re-pinned)
|
|
|
|
1. The brief names `deliveries` for list 3. message.receive (:1915, :1933) writes a `deliveries` row and no
|
|
event, so it is in list 2.
|
|
2. The brief says `projects` has `updated`. It has `at` (`packages/bus/src/migrations/3d-3e.sql:5`), and the
|
|
count uses it like every other table.
|
|
3. The brief says 14 tables have `at`. The head schema has 13: events, decisions and the 11 in `COUNTED`.
|
|
|
|
## Item 2: one count over every listed table
|
|
|
|
`STORE_JS` count and heldcount take `COUNTED` as a fourth argument and count in the same open as before
|
|
(readOnly for heldcount, `?immutable=1` for count):
|
|
|
|
- events after STAMP, then each listed table after STAMP;
|
|
- a listed table the store lacks counts 0 (no `instance_names` before 3g, no `usage_records` before 3h);
|
|
- a listed table that exists but cannot be read throws, as a failed events count does, and rollback refuses
|
|
`restore-mismatch` ("cannot count the rows after STAMP"; the old text said "events", and no test pinned it).
|
|
|
|
The output is one line, `TOTAL<tab>TEXT` (:308). TEXT is `N event(s)` and then `, M <table> row(s)` for each
|
|
listed table with rows, in `COUNTED` order. An events-only store prints the old text, so T13, T14 and T43
|
|
pass unchanged.
|
|
|
|
The brief's example puts usage_records before instance_names. I used schema order, which is `COUNTED` order.
|
|
|
|
## Item 3: the refusal and the in-run recount
|
|
|
|
- `post-backup-events` (:917) refuses unless the total is 0 or `--events` equals it. The message is the full
|
|
TEXT, so it names each table with rows. The flag and the code keep their names.
|
|
- The usage text (:39) and TOOLS.md say N is the total of the printed counts, read off the refusal.
|
|
- The in-run recount (:845-846) passes `COUNTED` too and compares the total with the pre-stop total. Rows
|
|
are append-only (item 1), so an equal total means equal counts per table. Comparing the whole TEXT would add
|
|
nothing.
|
|
|
|
## Item 4: every table is in exactly one list
|
|
|
|
T46 runs `packages/bus/schema.sql` in memory, reads every table from `sqlite_master`, and checks that each one
|
|
is in exactly one of:
|
|
|
|
1. `events`;
|
|
2. `COUNTED`, read from the script;
|
|
3. `decisions`;
|
|
4. `meta`, `task_comment_cursors`, `message_threads`, `sqlite_sequence`.
|
|
|
|
It also checks that every table in lists 1-3 has an `at` column and no table in list 4 does.
|
|
|
|
## Tests
|
|
|
|
- T45 (stack-release.test.mjs:1220) has four subtests.
|
|
- (a) Both modes, by unit, on scratch stores. Each of the 11 tables from a literal list (`LISTED`, so that a
|
|
table dropped from the script's `COUNTED` fails here as well as in T46) counts `2\t1 event(s), 1 <t>
|
|
row(s)`. One event, one usage_records row and one instance_names row count 3 in both modes. A 3f store
|
|
with no usage_records counts 0 for it and doesn't throw. Neither mode changes the file's sha or leaves a
|
|
`-wal` or `-shm`.
|
|
- (b) End to end, for instance_names and usage_records. After `activate --migrate`, one row and no event
|
|
refuses `post-backup-events` naming the table, with `--events 1` in the message. With no `--events` and
|
|
with `--events 2`, the run exits 3 before any systemctl stop and the fingerprint is unchanged. Then
|
|
`--events 1` restores to B/3g.
|
|
- (c) A `usage_records` with no `at` refuses `restore-mismatch` before any stop.
|
|
- (d) The in-run recount. A new one-shot fault in the systemctl stand-in, `stop-writes-row` (:139), writes a
|
|
`thread_opens` row during the stop. The pre-stop total is 0, the recount finds 1, and the run rolls back
|
|
`post-backup-events` to C.
|
|
- T46 (:1268) is item 4.
|
|
- T14 is unchanged and passes.
|
|
|
|
## Runs
|
|
|
|
Sage gave the slot in sage-1182. The busy check passed, and the run (`run181.sh`, TMPDIR and output in my
|
|
scratch) went from 2026-10-20T04:17:23.094Z to 04:23:54.822Z. Each mutant edits one file after a checked
|
|
backup, runs the named test with `--test-name-pattern`, and puts the file back by the backup's sha.
|
|
|
|
| Run | Test | Result | Failing subtests |
|
|
|---|---|---|---|
|
|
| `stack-release.test.mjs`, whole file | all | 116 tests, 116 pass, 0 fail (row 180 had 109; T45 adds 6, T46 adds 1) | none |
|
|
| drop each of the 11 tables from `COUNTED` | T45 | 11 killed | (a) in every case; also (b) for instance_names and usage_records, (c) for usage_records, (d) for thread_opens |
|
|
| in-run recount on events only (`"$COUNTED"` removed at :845) | T45 | killed | (b) both tables, since `--events 1` no longer matches a recount of 0, and (d) |
|
|
| `CREATE TABLE extra (seq INTEGER PRIMARY KEY, at TEXT NOT NULL)` appended to `schema.sql` | T46 | killed | "tables in no list" |
|
|
| `decisions` added to `COUNTED` | T46 | killed | "a table is in two lists: decisions" |
|
|
|
|
No test reads or pins a hash of the three changed paths except `stack-release.test.mjs` itself
|
|
(`git grep` over `packages/*/tests` and `scripts/test-*.sh`).
|
|
|
|
## Follow-up notes
|
|
|
|
- F1. A restore can drop `task_comment_cursors` advances without counting them (item 1). The table has no time
|
|
column, so the guard can't count rows after STAMP. A fix needs either an `at` column (a schema change) or an
|
|
event at :1129.
|
|
- F2 (Filbert's N1, #1679 comment 29372; optional). stack-release.sh:909 still says "the events after STAMP
|
|
were counted" in the held-count recheck refusal. T29 pins that text (stack-release.test.mjs:815), so
|
|
rewording it changes an existing test. I left it for a later row.
|
|
|
|
## Candidate
|
|
|
|
- Base a757f8fe6. `git log a757f8fe6..origin/refactor` over the three paths is empty at 04:25Z (origin/refactor
|
|
is a757f8fe6).
|
|
- `build.patch`: sha256 30a9b9e36afcf8f8ceaacce3fd3dddbde3733ba1a76fc87944766c0876093dda, patch-id (stable)
|
|
1fef78100554af5f4f1aa6a3e84663c23406e081, 3 files, 129 insertions, 16 deletions.
|
|
- `candidate-manifest.sha256`: sha256 46a4433ca4108682af2027dc27ad72359b45709006f809fbf45c4b45dfacc283.
|
|
|
|
| Path | sha256 |
|
|
|---|---|
|
|
| `scripts/stack-release.sh` | 64f947a040d95b90b006dad43d6140eb4533082925e4e2a82ac4dc91c773f139 |
|
|
| `packages/cli/tests/stack-release.test.mjs` | 5ea483c114901de585a500e9cd87a02f5c06a63bb62f106290578784db6ca7d0 |
|
|
| `docs/TOOLS.md` | 4756c0ccf43be061ce9d2361882ec2ab78a0d2b622cda5224669d6bfd9bdda3b |
|