stack/packages/queue/README.md

Queue

The work queue as data (#1508). docs/plans/queue.json holds the rows and an append-only log of every operation. The table in docs/plans/QUEUE.md between the two mosaic-queue markers is generated from it. The specification is section 8 of agents/filbert/work/queue-as-data-plan-2026-09-26.md. This README states what the code does and the limits it accepts.

Until A2 adds scripts/mosaic queue, run the CLI directly from the canonical checkout. Messages already name scripts/mosaic queue.

node packages/queue/src/cli.mjs list
node packages/queue/src/cli.mjs next darkwing
node packages/queue/src/cli.mjs move 9 in-progress --op darkwing-9-start-1 --by darkwing
node packages/queue/src/cli.mjs verify --current
node --test packages/queue/tests/
scripts/test-queue.sh

Verbs

Verb	Lock	Writes
list, show ID, next [SEAT]	no (see "Unlocked reads")	nothing
add, move, release, assign, note, set	yes	one log entry, the witness, the table
genesis, accept-history	yes, privileged	one log entry, the witness, the table
render	yes	the table, only when it is stale
render --check, verify [--current]	yes	nothing
verify --snapshot DIR	no	nothing; reads only the files it is given
sync [--op ID]	yes	fsyncs and the witness; logs nothing
snapshot --out DIR	yes	copies of both files into an empty DIR outside the repository
unlock [--check-gate]	no; takes the unlock gate	removes a dead or mismatched lock

Every change takes --op ID (8 to 72 characters of [a-z0-9._-], starting with a letter or digit, not ending in .outcome) and an actor from --by NAME or $MOSAIC_AGENT_NAME. Reads need no actor. next with neither a SEAT nor $MOSAIC_AGENT_NAME refuses. The privileged actors are jason and sage.

Exit codes: 0 ok; 1 the operation failed; 2 invalid data or refused; 3 uncertain (visible or durable, not acknowledged); 4 usage.

Until piece D, move ID in-review needs --candidate: an existing file is read as a manifest (one <sha256> <path> line per file), anything else as a commit reachable from refs/heads or refs/tags. The candidate is frozen for the round.

The review's issue follows lead decision 23. A row with no issues can't request review. A row with one issue uses it. A row with several needs --issue N, one of its issues. Later rounds keep the previous round's issue unless --issue names another; if the row no longer lists the kept issue, the request refuses until --issue names one.

move ID done from in-review needs --evidence comment=<id>,round=<n>,candidate=<digest>. The round must be the current one and the digest its candidate's, so a comment from an earlier round can't close a later one, even when the candidate is the same.

Where the files live

• docs/plans/queue.json: {version, canonicalRoot, revision, rows, log}, serialized deterministically. A file that doesn't re-serialize byte for byte, or doesn't replay from genesis to its rows, is refused by every verb.
• docs/plans/QUEUE.md: hand-written header, generated body between the markers. Each log entry records the SHA-256 of the body it rendered, so a body is current, stale (an earlier render) or unknown (no render).
• .git/mosaic-queue.head: the witness, {revision, logDigest, fileSha, at} for the last write whose directory fsync succeeded. It holds no rows.
• .git/mosaic-queue.lock and .git/mosaic-queue.unlock: the lock and the unlock gate.

Canonical checkout only

Every verb except verify --snapshot refuses unless the working directory's toplevel is the canonicalRoot recorded at genesis, its git directory is <root>/.git (a linked worktree fails), HEAD names the genesis branch, and the CLI's own file lies under the root. GIT_DIR, GIT_WORK_TREE and GIT_COMMON_DIR refuse. A symlink to the root works. Only one host is supported.

Trust boundary

The queue is cooperative (J2). An actor is whoever --by or $MOSAIC_AGENT_NAME claims. The CLI checks each claimed actor against the transition matrix and records it, and replay rechecks every entry, but nothing stops a seat from claiming another name or writing the file with other tools. Detection is what the queue offers: replay catches a hand edit, the witness catches a rollback, and piece E lists changes for review. Enforcement needs a boundary outside #1508.

The git side of the protocol

The lock only binds the queue CLI. Git doesn't honour it. So these rules are protocol, not enforcement:

• While the working queue.json has ops not yet committed, no one runs checkout, stash, restore, reset or a branch switch that touches docs/plans/queue.json or docs/plans/QUEUE.md. Only the lead restores, and only by the manual recovery below.
• Seats commit with plain git commit, never --no-verify, so the queue guard runs.
• Merge, rebase, cherry-pick, revert and am skip the guard. On the queue branch they are the lead's.
• Nobody disables, replaces or overrides the guard: no core.hooksPath in any scope, no -c core.hooksPath=, no GIT_CONFIG_* environment, no edit to .git/hooks/pre-commit and no change to its mode.

The witness detects lost history afterwards. It can't prevent a same-user git write. A passing replay shows the file agrees with itself, not that no history was lost.

Committing the queue

The CLI never commits. The lead runs scripts/queue-commit.sh -m MSG. It commits exactly the snapshot bytes of the two queue files on top of HEAD through a temporary index and commit-tree, so nothing else that is staged is swept in:

1. Record H, then check the guard is active: .git/hooks/pre-commit is a regular executable file owned by the user with HEAD's bytes, no core.hooksPath is set in any scope, and a canary run of git hook run pre-commit on a temporary index passes clean and refuses a changed queue entry. Refuse if a queue path is staged.
2. queue snapshot under the lock.
3. The base is H:docs/plans/queue.json from the object database.
4. HEAD's archived code, unpacked outside any repository, runs its queue tests and verify --snapshot against the base.
5. to 7. Blobs, a tree from a temporary index, commit-tree, the guard checks again, then update-ref with H as the expected old value. If the branch moved at any point after step 1, this fails and nothing is published.
6. Reconcile the shared index's two queue entries with git reset -q -- docs/plans/queue.json docs/plans/QUEUE.md, only if HEAD is the new commit and those entries are still H's. Otherwise exit 3 and print the command. Until it runs, the guard refuses ordinary commits.

scripts/git-hooks/pre-commit is the guard. scripts/queue-commit.sh --install-hook installs it (jason or sage). It refuses any git commit whose index entries for the two queue files differ from HEAD's.

Bootstrap: the implementation lands with no queue.json; the lead installs the guard, runs queue genesis from the reviewed migration map, then scripts/queue-commit.sh --genesis -m MSG. Every mutating verb refuses with genesis not committed until that commit exists. After that, scripts/test-queue.sh also runs verify on the live queue.

Committing still needs its own authorization. The script never pushes.

Durability

Supported: Linux with docs/plans/ and .git/ on local ext4, xfs or btrfs (tmpfs for tests). The CLI checks the filesystem type and refuses others. The host-crash claims assume that rename replaces its target atomically and that fsync of a directory persists the renamed entry. The tests kill processes with SIGKILL at each step. Power loss is not tested.

A write has three separate points:

Point	When	Meaning
visible	the rename returns	readers see the new revision
durable	the directory fsync succeeds and the witness is written	it survives a host crash, within the assumptions above
acknowledged	the receipt ok <op> rev N ... prints	the caller may treat the op as done

The unacknowledged-op rule. Only an op whose receipt printed is done. If a call exits 3, is killed, or prints nothing, retry the same op id with the same arguments. A recorded op answers with its original receipt and changes nothing. A different verb or arguments under a used op id refuses. queue sync confirms a visible tail and prints each op that became durable without being acknowledged. Tell the seat named there.

Unlocked reads

list, show and next read the witness, then queue.json, without the lock. A file ahead of the witness prints rev N visible, not confirmed durable. Any other disagreement takes the lock and rechecks before reporting, so a write in progress is never reported as lost history.

Known windows and limits

• Check to rename. A write checks that queue.json is unchanged since it was read, then renames its temp file over it. A git write between the check and the rename is not caught by the check. The witness catches the result afterwards. The table write has the same window: a hand edit to QUEUE.md between its check and its rename can be overwritten.
• Same-seat claims. A claim is the seat name only. Two sessions of the same seat are one claimant; the queue can't tell them apart and doesn't try.
• Snapshot verify checks that the given pair is valid, that the table is its render, and that its log extends the base. It does not certify witness continuity. Canonical verify, under the lock, does that.
• Commit candidates stay retrievable only while some ref keeps the commit. The queue keeps the manifest text, not the source bytes.
• Locks are never removed for their age. unknown and invalid locks wait for a person. A host rename makes old locks unknown.
• queue unlock --check-gate classifies a stale unlock gate. Remove the gate by hand only after it says dead or mismatch and no queue command is running.

Manual recovery

For an invalid queue.json, lost history (the file doesn't extend the witness) or a missing witness. No verb repairs anything; accept-history is the only verb that runs in these states, and a file holding genesis alone with no witness can also be confirmed by sync.

1. Copy the current queue.json bytes to agents/<lead>/work/queue-recovery-<date>/ before touching anything. Restoring HEAD would discard every op written since the last queue commit.
2. Find the lost ops, the entries after the last committed revision, in the preserved bytes, git stash list, the reflog, the receipts seats hold, and issue comments carrying mosaic-queue-op markers.
3. Account for external effects. A lost op id is no longer deduplicated, and reusing it could act again.
4. Put a valid file in place: the restored version with the lost ops re-applied under new op ids, or a reviewed hand-built file.
5. Run queue accept-history --op ID --reason TEXT --yes as a privileged actor. It records the old witness (or its absence), the new revision and the reason, then rewrites the witness. The at-most-once guarantee does not hold for the lost range, and the entry says so.

A stale table needs a person to look, then queue render. An unknown table needs the table restored with git, or the edit re-applied as queue ops, then queue render.

Tests

node --test packages/queue/tests/ runs everything in scratch repositories under the system temp directory; nothing touches this checkout's .git. Faults reach the code only through options the tests pass in (io, proc, hook, now, readOrder, lockWaitMs); the CLI passes none.

• data.test.mjs: serialization, replay, the transition matrix, next ordering, render.
• lock.test.mjs: every lock and unlock schedule in 8.4.
• store.test.mjs: genesis, canonical checks, op ids, claims, briefs, views, snapshot.
• write.test.mjs: every write fault, SIGKILL at each step, git interference, unlocked-read races.
• commit.test.mjs: queue-commit.sh and the guard, through PATH shims that run an action at an exact point in the procedure.