# Queue The work queue as data (#1508). `docs/plans/queue.json` holds the rows and an append-only log of every operation. The table in `docs/plans/QUEUE.md` between the two `mosaic-queue` markers is generated from it. The specification is section 8 of `agents/filbert/work/queue-as-data-plan-2026-09-26.md`. This README states what the code does and the limits it accepts. Run it from the canonical checkout as `scripts/mosaic queue `, which execs `node packages/queue/src/cli.mjs` with the same arguments and exit code. ```sh scripts/mosaic queue list scripts/mosaic queue next darkwing scripts/mosaic queue move 9 in-progress --op darkwing-9-start-1 --by darkwing scripts/mosaic queue verify --current node --test packages/queue/tests/ scripts/test-queue.sh ``` ## Verbs | Verb | Lock | Writes | |---|---|---| | `list`, `show ID`, `next [SEAT]` | no (see "Unlocked reads") | nothing | | `add`, `move`, `release`, `assign`, `note`, `set` | yes | one log entry, the witness, the table | | `genesis`, `accept-history` | yes, privileged | one log entry, the witness, the table | | `render` | yes | the table, only when it is stale | | `render --check`, `verify [--current]` | yes | nothing | | `verify --snapshot DIR` | no | nothing; reads only the files it is given | | `sync [--op ID]` | yes | fsyncs and the witness; logs nothing | | `snapshot --out DIR` | yes | copies of both files into an empty DIR outside the repository | | `unlock [--check-gate]` | no; takes the unlock gate | removes a dead or mismatched lock | | `review request`, `review resolve`, `review abandon`, `review record` | yes | one log entry, the witness, the table; a request also posts one comment and logs its outcome | | `review verify-commit ID REF` | no | nothing; reads the queue and git objects | Every change takes `--op ID` (8 to 72 characters of `[a-z0-9._-]`, starting with a letter or digit, not ending in `.outcome`) and an actor from `--by NAME` or `$MOSAIC_AGENT_NAME`. `--by` wins when both are set; if they differ, the command prints a warning on stderr and goes on. Reads need no actor. `next` with neither a SEAT nor `$MOSAIC_AGENT_NAME` refuses. The privileged actors are `jason` and `sage`. Text the table shows (piece, gate, note, the move reason, the brief anchor, a blocked reason) refuses `\` and `<`. The table escapes `|` itself; a backslash could undo that escape and `<` starts raw HTML, so the CLI refuses both rather than escape them. `set ID issues` also moves `closes`. If `closes` equals the old issues, it becomes the new list. If it was narrowed (`set ID closes` with `--reason`, or genesis from the map), it keeps the part still among the new issues, and the receipt ends in `(kept narrowed)`. Exit codes: 0 ok; 1 the operation failed; 2 invalid data or refused; 3 uncertain (visible or durable, not acknowledged; for a review request, not known to be posted); 4 usage. `move ID in-review` needs `--candidate`. An existing file is read as a manifest (one ` ` line per file), anything else as a commit reachable from `refs/heads` or `refs/tags`. The candidate is frozen for the round. The review's issue follows lead decision 23. A row with no issues can't request review. A row with one issue uses it. A row with several needs `--issue N`, one of its issues. Later rounds keep the previous round's issue unless `--issue` names another; if the row no longer lists the kept issue, the request refuses until `--issue` names one. Each round records the issue it used (`review.rounds[].issue`). A row with reviewers opens a request round and posts the review request comment; see "Review requests". A row with no reviewers opens a round that posts nothing. For that kind of round, `move ID done` from in-review needs `--evidence comment=,round=,candidate=`. The round must be the current one and the digest its candidate's, so a comment from an earlier round can't close a later one, even when the candidate is the same. ## Review requests Piece D, section 8.9 of the plan. `move ID in-review` from in-progress, on a row with reviewers, logs the new round and a request attempt in the `requesting` state, then posts one comment on the round's issue as the acting seat and logs what the transport said as a second entry, `.outcome`. `review request ID` makes a new attempt in the current round after a failed or abandoned one. - **Credentials.** The request posts with the acting seat's own token file, named by `MOSAIC_GITEA_CREDENTIAL_FILE`. It must end in `/agents//secrets/gitea-mosaicstack-.token`, be a regular file (no symlink, and its real path ends the same way), belong to this user and be mode 0600. The lead's login is `jarvis`. The shared `~/secrets/mosaic.gitea.json` refuses. The queue checks the file with `lstat` and never opens it; `scripts/gitea-api.sh` reads it. Before the POST, `GET user` must name the seat's login. - **Outcome.** HTTP 201 with a comment id is `posted` (exit 0). HTTP 400, 401, 403, 404 or 422, or a failed pre-send check, is `failed` (exit 1). Anything else is `uncertain` (exit 3): another status, 201 without an id, a transport error, or no answer before the 30-second deadline, which kills the helper and curl with it. The log records the status and a fixed detail, never the response body. - **Retries never post.** Running the same `--op` again prints what the log holds and sends nothing. A request left `requesting` (the process died after the log entry) or `uncertain` needs a person to look on the issue for a comment carrying ``. If it is there, the owner or a privileged actor runs `review resolve ID OP --comment N`. The queue fetches that comment and refuses unless it is on the round's issue, written by the requester's login, and carries both markers for this attempt, round and candidate. If it is not there, a privileged actor runs `review abandon ID OP --reason TEXT --yes`. An abandon sets the round's `duplicateRisk`, because the comment may still land. - **Late outcomes.** An outcome that arrives after a resolve with the same comment id keeps `posted`. One that disagrees, or a posted outcome after an abandon, makes the attempt `conflict`, which a resolve settles. The outcome entry is the one entry a done row still accepts, so a late answer is recorded even after the row closed. - **Unresolved requests** (`requesting`, `uncertain`, `conflict`) block a new request, a new round, a move to `waiting-on-jason` and `done`. - **Verdicts.** Each listed reviewer other than the owner posts a comment on the issue, then runs `review record ID --verdict approve|changes --comment N --candidate DIGEST`, once per round. DIGEST must be the round's candidate. On a request round, `move ID done` and `move ID waiting-on-jason` each need an approval recorded by every reviewer in the current round, so a Jason-gated row reaches Jason only after its reviewers approved. `done` also refuses `--evidence`. The queue doesn't fetch the verdict comment; `record` is cooperative like every other actor claim. - **`review verify-commit ID REF`** checks a prospective commit or tree against the current round's candidate. For a manifest, every listed path must hash to its digest. For a commit candidate, every path the candidate changed from its first parent must match in REF by blob and mode, and every path it deleted must be absent. Each log entry records the `semantics` it was written under. Entries from semantics 1, before piece D, replay under the old rules: a move to in-review opens a round that posts nothing. Review entries need semantics 2. ## Where the files live - `docs/plans/queue.json`: `{version, canonicalRoot, revision, rows, log}`, serialized deterministically. A file that doesn't re-serialize byte for byte, or doesn't replay from genesis to its `rows`, is refused by every verb. - `docs/plans/QUEUE.md`: hand-written header, generated body between the markers. Each log entry records the SHA-256 of the body it rendered, so a body is `current`, `stale` (an earlier render) or `unknown` (no render). - `.git/mosaic-queue.head`: the witness, `{revision, logDigest, fileSha, at}` for the last write whose directory fsync succeeded. It holds no rows. - `.git/mosaic-queue.lock` and `.git/mosaic-queue.unlock`: the lock and the unlock gate. ## Canonical checkout only Every verb except `verify --snapshot` refuses unless the working directory's toplevel is the `canonicalRoot` recorded at genesis, its git directory is `/.git` (a linked worktree fails), HEAD names the genesis branch, and the CLI's own file lies under the root. `GIT_DIR`, `GIT_WORK_TREE` and `GIT_COMMON_DIR` refuse. A symlink to the root works. Only one host is supported. ## Trust boundary The queue is cooperative (J2). An actor is whoever `--by` or `$MOSAIC_AGENT_NAME` claims. The CLI checks each claimed actor against the transition matrix and records it, and replay rechecks every entry, but nothing stops a seat from claiming another name or writing the file with other tools. Detection is what the queue offers: replay catches a hand edit, the witness catches a rollback, and piece E lists changes for review. Enforcement needs a boundary outside #1508. ## The git side of the protocol The lock only binds the queue CLI. Git doesn't honour it. So these rules are protocol, not enforcement: - While the working `queue.json` has ops not yet committed, no one runs `checkout`, `stash`, `restore`, `reset` or a branch switch that touches `docs/plans/queue.json` or `docs/plans/QUEUE.md`. Only the lead restores, and only by the manual recovery below. - Seats commit with plain `git commit`, never `--no-verify`, so the queue guard runs. - Merge, rebase, cherry-pick, revert and `am` skip the guard. On the queue branch they are the lead's. - Nobody disables, replaces or overrides the guard: no `core.hooksPath` in any scope, no `-c core.hooksPath=`, no `GIT_CONFIG_*` environment, no edit to `.git/hooks/pre-commit` and no change to its mode. The witness detects lost history afterwards. It can't prevent a same-user git write. A passing replay shows the file agrees with itself, not that no history was lost. ## Committing the queue The CLI never commits. The lead runs `scripts/queue-commit.sh -m MSG`. It commits exactly the snapshot bytes of the two queue files on top of HEAD through a temporary index and `commit-tree`, so nothing else that is staged is swept in: 1. Record H, then check the guard is active: `.git/hooks/pre-commit` is a regular executable file owned by the user with HEAD's bytes, no `core.hooksPath` is set in any scope, and a canary run of `git hook run pre-commit` on a temporary index passes clean and refuses a changed queue entry. Refuse if a queue path is staged. 2. `queue snapshot` under the lock. 3. The base is `H:docs/plans/queue.json` from the object database. 4. HEAD's archived code, unpacked outside any repository, runs its queue tests and `verify --snapshot` against the base. 5. to 7. Blobs, a tree from a temporary index, `commit-tree`, the guard checks again, then `update-ref` with H as the expected old value. If the branch moved at any point after step 1, this fails and nothing is published. 8. Reconcile the shared index's two queue entries with `git reset -q -- docs/plans/queue.json docs/plans/QUEUE.md`, only if HEAD is the new commit and those entries are still H's. Otherwise exit 3 and print the command. Until it runs, the guard refuses ordinary commits. `queue-commit.sh` calls `node packages/queue/src/cli.mjs` directly, not `scripts/mosaic queue`. That stays after A2 on purpose: step 4 runs HEAD's archived copy, which then depends only on the package, not on the dispatcher. `scripts/git-hooks/pre-commit` is the guard. `scripts/queue-commit.sh --install-hook` installs it (jason or sage). It refuses any `git commit` whose index entries for the two queue files differ from HEAD's. Bootstrap: the implementation lands with no `queue.json`; the lead installs the guard, runs `queue genesis` from the reviewed migration map, then `scripts/queue-commit.sh --genesis -m MSG`. Every mutating verb refuses with `genesis not committed` until that commit exists. After that, `scripts/test-queue.sh` also runs `verify` on the live queue. The map is `agents/darkwing/work/queue-migration-map.md`. It names the QUEUE.md blob it was built from. Before genesis, `node agents/darkwing/work/queue-a2/map-check.mjs` lists every table line changed since that blob and every row whose piece, owner or issues no longer match the map; exit 0 means no drift. QUEUE.md needs its two markers around the table and the parked list before genesis, since genesis keeps the text between them as `legacyView`. Committing still needs its own authorization. The script never pushes. ## Durability Supported: Linux with `docs/plans/` and `.git/` on local ext4, xfs or btrfs. The CLI checks the filesystem type and refuses others. ext2 and ext3 report ext4's magic number, so the check can't tell them apart. tmpfs passes only for a test that hands in an io layer with `allowTmpfs: true`; the CLI's own layer never has it. The host-crash claims assume that rename replaces its target atomically and that fsync of a directory persists the renamed entry. The tests kill processes with SIGKILL at each step. **Power loss is not tested.** A write has three separate points: | Point | When | Meaning | |---|---|---| | visible | the rename returns | readers see the new revision | | durable | the directory fsync succeeds and the witness is written | it survives a host crash, within the assumptions above | | acknowledged | the receipt `ok rev N ...` prints | the caller may treat the op as done | **The unacknowledged-op rule.** Only an op whose receipt printed is done. If a call exits 3, is killed, or prints nothing, retry the same op id with the same arguments. A recorded op answers with its original receipt and changes nothing. A different verb or arguments under a used op id refuses. `queue sync` confirms a visible tail and prints each op that became durable without being acknowledged. Tell the seat named there. ## Unlocked reads `list`, `show` and `next` read the witness, then `queue.json`, without the lock. A file ahead of the witness prints `rev N visible, not confirmed durable`. Any other disagreement takes the lock and rechecks before reporting, so a write in progress is never reported as lost history. ## Known windows and limits - **Check to rename.** A write checks that `queue.json` is unchanged since it was read, then renames its temp file over it. A git write between the check and the rename is not caught by the check. The witness catches the result afterwards. The table write has the same window: a hand edit to QUEUE.md between its check and its rename can be overwritten. - **Same-seat claims.** A claim is the seat name only. Two sessions of the same seat are one claimant; the queue can't tell them apart and doesn't try. - **Snapshot verify** checks that the given pair is valid, that the table is its render, and that its log extends the base. It does not certify witness continuity. Canonical `verify`, under the lock, does that. - **Commit candidates** stay retrievable only while some ref keeps the commit. The queue keeps the manifest text, not the source bytes. `review verify-commit` on a commit candidate needs that commit. - **Post to outcome.** A kill after the POST and before the outcome entry leaves the attempt `requesting` with a comment on the issue. The queue never posts on its own again; `review resolve` records the comment. - **Verdict comments** are not fetched. `review record` trusts the reviewer's comment id the way the queue trusts `--by`. - **Locks** are never removed for their age. `unknown` and `invalid` locks wait for a person. A host rename makes old locks `unknown`. - `queue unlock --check-gate` classifies a stale unlock gate. Remove the gate by hand only after it says `dead` or `mismatch` and no queue command is running. - A process killed while taking the lock can leave `.git/mosaic-queue.lock...tmp`. Nothing reads it or removes it. Delete it by hand once that pid is gone. ## Manual recovery For an invalid `queue.json`, lost history (the file doesn't extend the witness) or a missing witness. No verb repairs anything; `accept-history` is the only verb that runs in these states, and a file holding genesis alone with no witness can also be confirmed by `sync`. 1. Copy the current `queue.json` bytes to `agents//work/queue-recovery-/` before touching anything. Restoring HEAD would discard every op written since the last queue commit. 2. Find the lost ops, the entries after the last committed revision, in the preserved bytes, `git stash list`, the reflog, the receipts seats hold, and issue comments carrying `mosaic-queue-op` markers. 3. Account for external effects. A lost op id is no longer deduplicated, and reusing it could act again. 4. Put a valid file in place: the restored version with the lost ops re-applied under new op ids, or a reviewed hand-built file. 5. Run `queue accept-history --op ID --reason TEXT --yes` as a privileged actor. It records the old witness (or its absence), the new revision and the reason, then rewrites the witness. The at-most-once guarantee does not hold for the lost range, and the entry says so. A stale table needs a person to look, then `queue render`. An unknown table needs the table restored with git, or the edit re-applied as queue ops, then `queue render`. ## Tests `node --test packages/queue/tests/` runs everything in scratch repositories under the system temp directory; nothing touches this checkout's `.git`. Faults reach the code only through options the tests pass in (`io`, `proc`, `hook`, `now`, `readOrder`, `lockWaitMs`, `deadlineMs`); the CLI passes none. - `data.test.mjs`: serialization, replay, the transition matrix, `next` ordering, render. - `lock.test.mjs`: every lock and unlock schedule in 8.4. - `store.test.mjs`: genesis, canonical checks, op ids, claims, briefs, views, snapshot. - `write.test.mjs`: every write fault, SIGKILL at each step, git interference, unlocked-read races. - `dispatch.test.mjs`: `scripts/mosaic queue` passes arguments and exit codes through, and the seat commands still reach the seat CLI. - `migration.test.mjs`: the real map renders the golden genesis table, the marked QUEUE.md fixture holds every row between its markers, and `map-check.mjs` reports each kind of drift. - `commit.test.mjs`: `queue-commit.sh` and the guard, through PATH shims that run an action at an exact point in the procedure. - `review.test.mjs`: review requests against `fixtures/fake-gitea.mjs`, installed as the scratch repository's `scripts/gitea-api.sh`, with dummy token files. Every transport answer, the deadline, kills at each step, a held lock at the outcome, late outcomes, resolve checks, credential checks, verdicts across rounds, `verify-commit` and semantics 1 replay.