Files
stack/agents/darkwing/work/queue-a1/build.md
T
jason.woltjeandClaude Opus 5.5 34a72af912 feat(queue): queue as data A1, journal, lock, CLI and verify (#1508)
packages/queue, scripts/queue-commit.sh, scripts/git-hooks and
scripts/test-queue.sh, plus docs/plans/BRIEF-TEMPLATE.md. There is no
queue.json yet, so verify skips until the genesis commit after A2.

Darkwing built it, and Filbert reviewed R0 (6933b885, changes requested)
and r1 (e464be6c, approved). The 20 files match manifest 85a8a453. The
nine suites passed on an index export, including the new queue suite.
test-queue.sh joins the suite list in AGENTS.md. Lead decisions 20, 23
and 26.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-26 19:07:48 -05:00

187 lines
8.8 KiB
Markdown

# Queue A1 build (#1508), candidate for review
Darkwing built this on 2026-09-26 from section 8 of
`agents/filbert/work/queue-as-data-plan-2026-09-26.md` (sha256 282fabbb,
the only spec), split as Sage approved: render is in A1, `move in-review`
needs `--candidate` until piece D, and a round's issue is the row's first
issue. Filbert reviews the code; Sage commits after the suites. Base is HEAD
3a209eea. Nothing is committed, staged or pushed.
A stray pkill at 22:02:32Z stopped my first turn with only
`src/errors.mjs` and `src/io.mjs` on disk. I reread both against what I had
meant them to be. They match: the exit-code class, and the file layer with
`realIo` as the only layer the CLI uses. Everything else was written after
the restart.
## Files
`build-manifest.sha256` pins the 20 files. `build.patch` (sha256 419804f2)
adds all 20 as new files with their modes. It applies cleanly to 3a209eea,
and the applied tree matches the manifest and passes `scripts/test-queue.sh`.
- `packages/queue/src/`: `errors.mjs`, `io.mjs` (the fault-injectable file
layer), `lock.mjs` (lock and unlock gate, 8.4), `queue.mjs` (serialization,
replay, the transition matrix, `next`, render), `store.mjs` (canonical
checks, the write path, witness, views, snapshot, verify), `cli.mjs`.
- `packages/queue/tests/`: data 19, lock 17, store 18, write 20, commit 21
tests, plus `helpers.mjs` and two child fixtures.
- `packages/queue/package.json` and `README.md`. The package has no
dependencies.
- `scripts/queue-commit.sh` (0755): the 8.12 procedure and
`--install-hook`.
- `scripts/git-hooks/pre-commit` (0755, POSIX sh): the queue guard.
- `scripts/test-queue.sh` (0755): the suite, in the style of
`test-discord.sh`.
- `docs/plans/BRIEF-TEMPLATE.md`: the 8.13 template.
## Which path runs `verify` once genesis is in
`scripts/test-queue.sh` runs `node packages/queue/src/cli.mjs verify` when
`git cat-file -e HEAD:docs/plans/queue.json` succeeds. At HEAD today there is
no `queue.json`, so it prints `skip queue verify: HEAD has no
docs/plans/queue.json (before the genesis commit)` and stays green. A2 moves
that call to `scripts/mosaic queue verify` when it adds the dispatch.
`queue-commit.sh` also calls `node packages/queue/src/cli.mjs` directly
(`snapshot`, then `verify --snapshot` from HEAD's archive) until A2.
## Sage's five conditions
1. Nothing ran against the canonical `.git`. After all runs, `.git/hooks`
holds only the samples, `.git` has no `mosaic-queue*` file, and
`git config --show-scope --get-all core.hooksPath` returns nothing in any
scope (rc 1). Every hook install, genesis, lock and gate test runs in a
scratch repo under the system temp directory. Suite runs used a
`--shared` clone at `/tmp/qa1-verify`.
2. `docs/plans/QUEUE.md`, `AGENTS.md` and `docs/TOOLS.md` are unedited.
`docs/SESSIONS.md` shows as modified in the working tree, but that was
someone else's edit before my session began; I didn't touch it.
3. `scripts/test-queue.sh` is green at HEAD with no `queue.json`: 19 checks
passed, `verify` skipped as above.
4. H is recorded before the canary. `commit.test.mjs` has "H recorded before
the canary": a shim commits on the first `git hook run`, and
`queue-commit.sh` exits 1 with `refs/heads/<branch> moved since <H>;
nothing published`. A second test moves HEAD after `commit-tree` with the
same result. Mutation M1 (read H after the canary) fails the first test.
5. The fault file layer is reachable only from tests. Faults enter through
the options the API takes (`io`, `proc`, `hook`, `now`, `readOrder`,
`lockWaitMs`); `cli.mjs` passes none. The only `process.env` read in
`src/` is the default `env` in `store.mjs`'s context.
## Bugs found while building
- A nested `node --test` inherits `NODE_TEST_CONTEXT` and exits 0 whatever
its tests do. Step 4's run of HEAD's archived tests therefore passed with a
failing test in the archive. `queue-commit.sh` and `test-queue.sh` now run
it under `env -u NODE_TEST_CONTEXT`, and a test commits an archive with a
failing test and expects a refusal (mutation M4). Other suites in this repo
that nest `node --test` may have the same blind spot. I haven't checked
them.
- git 2.55 does not hold `index.lock` while the commit editor is open. The
plan expected a paused `git commit -e` to block step 8. It doesn't: the
paused commit loses later at its own HEAD update with `cannot lock ref
'HEAD': is at C but expected H`. The test now asserts that outcome.
Nothing is lost, but the reason differs from the plan's.
- ext4 hands a freed inode number straight back. My first test for release's
inode check wrote a byte-identical lock after unlinking the original and
got the same inode back, so it proved nothing. It now writes a copy and
renames it over the lock, which guarantees a new inode.
## Choices the spec left open
- Verb names `release` and `set`. `add` requires `--gate`. A null brief is
allowed only on rows that genesis creates as done. `sync --op` is
optional.
- Reads need no actor. `next` with no seat and no `$MOSAIC_AGENT_NAME`
refuses.
- `accept-history` accepts a stale table but not an unknown one. When a
write succeeds but the table write is skipped, the CLI warns and exits 0.
- An invalid witness is treated as absent.
- `GIT_DIR`, `GIT_WORK_TREE` and `GIT_COMMON_DIR` refuse in the CLI.
`queue-commit.sh` also refuses `GIT_INDEX_FILE`, `GIT_OBJECT_DIRECTORY`
and `GIT_ALTERNATE_OBJECT_DIRECTORIES`.
- Leftover temp files are unlinked. Genesis uses `link`, so it cannot
replace an existing file.
- `unlock` works on a missing or invalid lock file.
- `note` on a blocked row edits `blockedReason`.
- Messages already name `scripts/mosaic queue`. The lost-history refusal
names `sync` when the file holds genesis alone.
- `--candidate` auto-detects: an existing file is a manifest, anything else
is a commit reachable from `refs/heads` or `refs/tags`.
- Only `--install-hook` is privileged in `queue-commit.sh` (jason or sage).
The commit itself relies on the protocol that the lead runs it.
- After the snapshot, `queue-commit.sh` checks the genesis entry's branch
and root against the current branch and root. For `--genesis` it also
checks that `H:<map>` is the map blob genesis read.
- Step 8 compares the index's two queue entries with H's before it looks
for `index.lock`, so "someone staged a queue path" is reported ahead of a
lock.
## Deferred
- 8.12's test of `verify-commit` on a prospective tree belongs to piece D,
which adds `verify-commit`. It is not in A1.
- A2 holds the real migration map, the QUEUE.md markers and header, the
row-7 pointer, the golden render, `scripts/mosaic` dispatch and
`docs/TOOLS.md`.
- Sage adds `queue` to the suite list when A1 lands (lead decision 20).
- Bootstrap happens after A2's map and markers land:
`scripts/queue-commit.sh --install-hook --by sage`, then `queue genesis`,
then `scripts/queue-commit.sh --genesis -m MSG`.
## Verification
In `/tmp/qa1-verify` (HEAD 3a209eea plus the 20 files):
| Suite | Result |
|---|---|
| config | 24/24 |
| task | 90/90 |
| foundation | 43/43 |
| conductor | 17/17 |
| release | 14/14 |
| auth | 15/15 |
| discord | 63/63 |
| extension-package | 18/18 |
| queue | 19 checks; `node --test` 95/95 |
The queue tests take about 18 s and were stable over two runs. A combined
`node --test` run over every package came to 474/474.
I also ran the post-genesis path in a scratch repo: install the hook,
genesis, `--genesis` commit, then `test-queue.sh`. `verify` printed `ok
verify rev 1: file valid, witness matches, view current`. After a hand edit
to one table cell it failed with `view unknown`.
### Mutations
Each mutation went into the verify clone, the queue tests ran, and the
original was restored. Every one was caught; the number is how many tests
failed.
| Id | Mutation | Failing tests |
|---|---|---|
| M1 | read H after the canary | 1 |
| M2 | drop the step-7 guard recheck | 2 |
| M3 | drop step 8's entry comparison | 1 |
| M4 | keep `NODE_TEST_CONTEXT` | 1 |
| M5 | drop step 1's staged-path check | 1 |
| M6 | `update-ref` without the old value | 2 |
| M7 | skip the canary | 2 |
| M8 | guard hook always passes | 21 |
| M9 | drop step 8's `index.lock` check | 1 |
| M10 | drop the exec-bit check | 2 |
| M11 | drop the `core.hooksPath` check | 1 |
| M12 | drop the map-blob check | 1 |
| S1 | drop the "unchanged since read" check | 1 |
| S2 | swallow the directory fsync error | 2 |
| S3 | drop the recheck under the lock for unlocked reads | 2 |
| S4 | drop the unlock-gate check | 2 |
| S5a | release ignores the inode | 1 |
| S5b | release ignores the record bytes | 1 |
| S6 | write the witness before the rename | 10 |
| S7 | drop genesis's fsync | 1 |
| S8 | treat a reused pid as dead | 11 |
S5 survived at first: the delayed-release test was caught by the byte
comparison alone. The inode test in `lock.test.mjs` closes that gap.