mosaicstack/stack
25
0
Fork 1
Code Issues 401 Pull Requests 13 Actions Packages Projects Releases 15 Wiki Activity

darkwing: row 162 packet, rounds 1-2, scratch plans and follow-up draft (#1659)

Browse Source 
Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
jason.woltje
2026-10-18 15:39:38 -05:00
co-authored by Claude Opus 5.5
parent 7e7f8e208e
commit d132b369ea
17 changed files with 5012 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
agents/darkwing/work/queue-162/BUILD.md
+864
View File
@@ -0,0 +1,864 @@
# Row 162 (D1, #1659): stack activate and rollback, health-gated
Owner: Darkwing. Reviewer: Filbert. Gate owner: Sage.
Brief: `docs/plans/2026-10-18_stack-release.md`, section "## Stack activate
and rollback, health-gated (D1)", commit cd9c071ae, blob cfc468c9df88.
Base: df4d5702f for round 2 (origin/refactor after row 125 landed; see
"Candidate round 2"). Round 1 was on cfde62f9f (origin/refactor after row 164 landed). The draft
started on cd9c071ae and moved up with `git checkout --detach`, through
70504d23a (row 124) to cfde62f9f. None of the 162 paths is among row
164's 21; the only base change on them is row 129's appended "Console
Chat" section in `docs/TOOLS.md`, and the TOOLS.md patch applied cleanly.
The store schema on refactor is 3h. The live copy 52f35e7a0 is at 3f,
not 3g as this packet said before 2026-10-18T19:40Z: 3g came with row
141 (de352dfaf), which isn't live yet.
Worktree: `~/darkwing-scratch/r162/wt`.
Status: built and tested on Sage's slot (19:25:10Z to 19:46:14Z on
2026-10-18). `stack-release.test.mjs` 58/58 and `stack-copy.test.mjs`
12/12, each run singly. 47 mutants killed before the steps change, then
M41, M45b, M46 and M47 to M49 killed after it. The scratch-root plan for
Mos is in `scratch-plan.txt` next to this file.
## Files
| File | Change |
|---|---|
| `scripts/stack-release.sh` | new: `adopt`, `activate`, `rollback`, `status` |
| `scripts/stack-copy.sh` | a `reachable` function split out of `copy_commit`, and a guard so the file can be sourced for its helpers |
| `packages/cli/tests/stack-release.test.mjs` | new |
| `docs/TOOLS.md` | the "Pinned stack update" section only |
| `packages/cli/README.md` | the pinned-stack paragraph only |
| this packet | |
Nothing else changes. `bus-service.sh` and `webui-service.sh` are called
as they are.
## Shape of the script
```
scripts/stack-release.sh adopt BUSINESS
scripts/stack-release.sh activate ID BUSINESS [--pm] [--migrate] [--timeout S] [--plan]
scripts/stack-release.sh rollback BUSINESS [--restore ACTIVATION [--events N]] [--timeout S] [--plan]
scripts/stack-release.sh status BUSINESS
common options: --copies DIR, --bin DIR (passed to stack-copy.sh and the install scripts)
```
Exit codes: 0 ok; 1 the gate failed and the rollback inside the run passed;
2 `rollback-failed`, the stack is left stopped; 3 a refusal, nothing
changed; 4 usage. Usage errors write no log line. Every other outcome
writes exactly one final line.
`--pm` on `activate` must match the pointer's `pm`. The brief has
`activate` install "exactly the pointer's `--pm`", so the flag is a check
the operator states, not a way to change it. A mismatch refuses
`unit-flags-drift`.
The header comment states decision 96's four conditions (item 10) and
points at decisions 94 and 102 for who starts the Console.
### Where things come from
- Data root: `load_config` from `scripts/common.sh`, run from the canonical
checkout (`MOSAIC_CONFIG` is honoured, as everywhere). The bus resolves
its root the same way, so both see one root.
- The canonical checkout: the `REPO` of this script. `bus-service.sh`,
`webui-service.sh` and `stack-copy.sh` are this checkout's, as in the
window plans. `webui-service.sh install` already refuses outside the
canonical checkout.
- Unit names: `[email protected]` and `mosaic-webui.service`.
Unit directory: `${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user`, the
default both install scripts write to.
- `claude`: `MOSAIC_CLAUDE_BIN=` from `systemctl --user show
mosaic-bus@B.service -p Environment`.
- `pm`: a `--pm` token after `bus start B` in the bus unit's `ExecStart`
`argv[]`.
- `console`: the `argv[]` tokens after `serve` in the Console unit's
`ExecStart`, minus `--checkout X`. The install script adds `--checkout`
itself.
- The copy's `SCHEMA_VERSION` and chain: text read from the copy's
`packages/bus/src/store.mjs`. `export const SCHEMA_VERSION = 'X';` must
match exactly once, and the chain is the `from: 'X'` lines in file
order. The copy's code is never imported or run for this. A copy where
either read fails refuses `copy-schema-unknown`. The migration file
names would also work, but `3b.sql` is a base, not a step, so the
store.mjs lines are the cleaner source.
### Sharing with stack-copy.sh
`copy_commit`'s reachability loop moves into `reachable FULL`, and
`copy_commit` calls it. The dispatch at the bottom gets
`[ "${BASH_SOURCE[0]}" = "$0" ] || return 0`, so sourcing defines the
functions and runs nothing. `stack-release.sh` sources it inside a
subshell function, so stack-copy's `die`, `parse` and globals never mix
with its own:
```
copy_ok() ( source "$REPO/scripts/stack-copy.sh"; ID="$1"; COPIES="$COPIES_DIR"; copy_root >/dev/null && reachable "$ID" )
```
`check` and `wrapper` run as commands, as an operator would run them.
## Pointer and log (item 1)
`<dataRoot>/state/stack-active.json`, written to a temp file in `state/`
and renamed:
```json
{"copy":"<40 hex>","business":"mosaic-stack","pm":true,
"claude":"/home/u/.local/bin/claude","console":["--port","7332","--business","mosaic-stack"],
"schema":"3g","activation":"20261018T170000Z-52f35e7a0abc","at":"2026-10-18T17:00:00Z"}
```
`schema` is the copy's `SCHEMA_VERSION` at the time it went live. On
`adopt` that is the adopted copy's, since a running host proves the store
is at its copy's schema.
`<dataRoot>/state/stack-activation-log.jsonl`, appended with `>>`, one
JSON object per line:
```json
{"at":"...","op":"activate","from":"<copy>","to":"<copy>","backup":null,"result":"ok"}
{"at":"...","op":"activate","from":"...","to":"...","backup":null,"result":"refused","code":"unit-flags-drift"}
{"at":"...","op":"activate","from":"...","to":"...","backup":"<activation>","result":"gate-failed","code":"healthz","rollback":"ok"}
{"at":"...","op":"activate","from":"...","to":"...","backup":"<activation>","result":"rollback-failed","code":"bus-status","rollback":"failed","rollbackCode":"healthz"}
```
Brief item 4 says "each step is logged". Ruling 3: a run that gets past
the refusals writes `result: "started"` before step 1 and one final line
at the end. The per-step detail goes to stdout and to `steps.log` inside
the run's rollback set. `rollback` reads final lines only.
A `started` line with no final line after it for the same activation is
an interrupted run. `status` shows it, and every later `adopt`,
`activate` or `rollback` refuses `interrupted-run`, naming the
activation. The script has no verb to clear it. Clearing it is Jason's or
Mos's call (ruling 3).
Every `adopt`, `activate` and `rollback` takes `flock -n` on
`state/stack-release.lock` first and refuses `run-locked` when another
run holds it. `status` and `--plan` take no lock (ruling 6).
Activation id: `<UTC %Y%m%dT%H%M%SZ>-<first 12 of the target copy>`. The
rollback set directory is made with `mkdir -m 0700` and no `-p`, so a
clash fails before anything changes.
## adopt (item 2)
1. The run lock, else `run-locked`. No `interrupted-run` in the log.
2. No pointer, else `already-adopted`.
3. `stack-copy.sh check --business B` passes, else `check-failed` with
its output.
4. Read `claude`, `pm` and `console` from `systemctl show`. No
`MOSAIC_CLAUDE_BIN` refuses `no-claude-pin` (ruling 5).
5. Write the pointer, then log `ok`.
It stops, starts and installs nothing.
## activate (items 3 to 6)
### Refusals before anything changes (item 3)
The checks run in this order. Each one gives exit 3, one log line with its
code, and no other write:
1. `run-locked`, then `interrupted-run`. `run-locked` writes no log
line, because the lock holder may be writing the log.
2. `no-pointer`, then `business-mismatch`: the pointer names another
business. The brief names no code for it; the pointer holds one
business, so a run for another one would install units the pointer
doesn't describe.
3. `copy-missing` (`copy_root` fails) or `copy-unreachable` (`reachable`
fails). The brief names no codes for these two, so these are mine.
4. `copy-schema-unknown`, from the store.mjs read above.
5. `already-active`: ID is the pointer's copy.
6. `instance-busy` (ruling 1): another business's `mosaic-bus@` instance
is in any state but inactive or failed, since the copy and wrapper
switch reach it too; or this business's instance is activating,
deactivating or reloading. The message names the instance and its
state. The read is `install_guard`'s `list-units` call. A literal
reading (refuse any running instance) would refuse every activate,
because the bus is running when activate starts.
7. `drop-in-override`: the `install_guard` scan of
`systemd-analyze --user unit-paths`, naming the file.
8. `no-claude-pin` (ruling 5): the effective bus unit has no
`MOSAIC_CLAUDE_BIN`.
9. `unit-flags-drift`: effective `pm`, `claude` or `console` differ from
the pointer's, or `--pm` differs from the pointer's `pm`. Prints
`pointer:` and `unit:` lines for each field that differs.
### The run (item 4)
1. Capture to `state/stack-rollback/<activation>/` (0700): the two unit
files from the unit directory, the drop-in directories
`[email protected]`, `mosaic-bus@B.service.d` and
`mosaic-webui.service.d` under that directory when present, the
wrapper `$BIN/mosaic` with `stat -c %a` in `WRAPPER_MODE`, the
`systemctl show` output for both units (it includes `DropInPaths`), and
`SHA256SUMS` over the captured files. `cp -p` throughout. This is
`capture-rollback.sh`. The `started` line is already written, so a
failure here writes the final line `refused` with code
`capture-failed` and exits 3. Nothing has stopped, and only the
partial rollback set exists.
2. `systemctl --user stop mosaic-webui.service`, then the bus unit. Poll
`is-active` for both to read inactive or failed, then the stopped
check: no `<dataRoot>/bus/writer.lock`, and no `/proc/*/fd` link
resolving under `<dataRoot>/bus/`. Bounded by `--timeout`. A timeout
goes to item 6 with code `stop-timeout`.
3. Schema read: see "The store read" below. A `-wal` or `-shm` beside
the store goes to item 6 with `store-unclean`.
- The same as the copy's `SCHEMA_VERSION`: no backup.
- In the chain and older, no `--migrate`: `migration-needs-flag`,
item 6.
- Older with `--migrate`: back up, then `node
<copy>/packages/bus/src/migrate-schema.mjs <dataRoot>`. A non-zero
exit goes to item 6 with code `migrate-failed` and the store
restored.
- Newer, or not in the chain: `store-newer`, item 6.
- No `bus.sqlite` at all (a fresh root): no backup, the new host
creates the store.
4. `bus-service.sh install --copy ID --copies DIR [--pm] --claude PATH`,
then `webui-service.sh install --copy ID --copies DIR <console
flags>`, then `stack-copy.sh wrapper ID --copies DIR --bin DIR`. Any
non-zero exit goes to item 6 with code `install-bus`, `install-console`
or `wrapper`.
5. Start the bus, then the Console.
From step 2 on, `trap` on INT, TERM and HUP sends the run to item 6, once.
A Ctrl-C mid-window leaves the old stack started, not a stopped one. The
one exception is after a `rollback --restore` has swapped the store; see
ruling 4 under rollback.
Most of the script's functions run as `f || handler`. Bash turns `set -e`
off inside such a call, so every step in them that can fail carries its
own `|| return 1` or sits in an `&&` chain.
### The backup
`<dataRoot>/backups/stack/<activation>/`, as `backup.sh` does it: umask
077, `mkdir -m 700` for the directory and `bus/`, `cp -p` of `bus.sqlite`
(only that file: a `-wal` or `-shm` was already refused as
`store-unclean`), `SOURCE.sha256` taken before the copy and
checked against `SHA256SUMS` after it, `MODES.txt`, `LISTING.txt`,
`DATA_ROOT`, `STAMP` (UTC ISO with milliseconds, the same form as
`events.at`), `SCHEMA` (the version read in step 3) and `ROLLBACK_SET`.
`backup.sh`, `restore.sh` and `stopped-check.sh` here mean Sage's
committed window scripts in `agents/sage/work/window-plan/repin-de352dfaf/`.
`stack-release.sh` carries its own copies of their steps and doesn't call
them. The stopped check runs again after the copy. Any failure goes to item 6
before the migration runs.
### The store read
`node -e` with `node:sqlite`:
`` new DatabaseSync(`${pathToFileURL(file).href}?immutable=1`, { readOnly: true }) ``,
then `SELECT value FROM meta WHERE key='schema_version'`, then close.
The probe settled the open mode. It ran on Sage's slot (sage-0738),
16:02:53Z to 16:03:08Z, node v26.8.1, on a WAL-mode fixture store in
`~/darkwing-scratch/r162/probe/store/`, never the live store or a copy of
it. The fixture was rebuilt before each trial and closed cleanly, so only
`bus.sqlite` was present at the start of each one.
| Open | Read | Files after | Main file mtime, sha256 |
|---|---|---|---|
| `{ readOnly: true }`, store stopped | version and count right | `bus.sqlite-shm` and `bus.sqlite-wal` created and left behind | unchanged |
| `file:...?immutable=1` with `readOnly`, store stopped | version and count right | none created | unchanged |
| `{ readOnly: true }` while a WAL connection holds the store | sees the holder's row still in the WAL | the holder's `-wal` and `-shm`, nothing new | not compared |
A plain read-only open of a stopped store writes two files next to it, so
it fails Sage's condition, and immutable is the mode. Immutable tells
SQLite the file can't change and makes it skip the WAL entirely. That is
sound only when there is no WAL to skip. So the rule (Sage approved it in
sage-0739):
- Before every stopped-store read, `bus.sqlite-wal` and `bus.sqlite-shm`
must both be absent. A clean close checkpoints and removes them. One
left behind means the bus did not close the store cleanly, and an
immutable read could return a stale version or count. The run refuses
`store-unclean`, naming the file, and goes to item 6. `stack-release.sh`
never opens, copies or moves that store. Whatever happens to it next is
Jason's or Mos's call. There is no copy fallback.
- With both absent, the read uses immutable.
- An unreadable version goes to item 6 with code `schema-unreadable`.
The `--restore` count in item 7 step 5 is taken before the stop, from a
store the running bus holds. There the `-wal` and `-shm` belong to the
bus, the bus unit must be active, and the read is `{ readOnly: true }`,
which the probe shows adds no file. If the bus unit isn't active, the
count is a stopped-store read under the rule above.
The no-write test below catches a read that creates a file: it compares
the file set and the sums under `bus/` before and after.
### The health gate (item 5)
Polled every second until all pass or `--timeout` (default 60) runs out.
The failing check named at the end is the first one still failing:
| Check | Code |
|---|---|
| `systemctl --user is-active` for both units | `units-inactive` |
| `<copy>/scripts/mosaic bus status --json` exits 0, `host.business` is B, `host.live` is true, `host.pid` is the bus unit's `MainPID`, `boot.ok` is true, `boot.business` is B, `boot.at` is at or after the moment step 5 began | `bus-status` |
| `GET http://127.0.0.1:<port>/healthz` returns 200, port from the `--port` flag in `console`, else 7332 (the `webui-service.sh` default) | `healthz` |
| `stack-copy.sh check --business B` exits 0 | `copy-check` |
| with `pm`: exactly one `launched` line in the bus unit's journal since step 5, and its run is live in `<copy>/scripts/mosaic launches list --json`, for B, with `startedAt` at or after the moment step 5 began | `pm-launch-unconfirmed` |
Ruling 2 (sage-0736, refined in sage-0738): the check must match the PM
role without reading the business file (Jason's org file) and without
naming `launch.by`. The session entries carry `instance`, but nothing in
them or in `launches/<business>.jsonl` marks a launch as the PM's. The
`bus start --pm` path is the only code that knows. It prints
`launched <instance>: run <run>, <harness> <model>` to stdout
(`packages/cli/src/cli.mjs:503`, the only place that prints such a line),
and under the unit that stdout is the journal.
So the gate reads `journalctl --user -u mosaic-bus@B.service --since
@<epoch seconds of step 5> -o cat`. The `@` form is unambiguous. This
host's journalctl reads a bare time as local CDT, and in Q14 an empty
window passed for a clean start that way. Of the lines it returns, it
keeps those starting with `launched `. It passes only when:
- there is exactly one such line;
- it matches `^launched (\S+): run (\S+), `;
- that run is live in `launches list --json` for B, with `startedAt` at
or after step 5.
Zero lines, two or more, a line that doesn't match, or a run that isn't
live all give the one code `pm-launch-unconfirmed`, and the message says
which. A human `mosaic launch` in the same window writes no line to the
unit's journal, so it can't pass for the PM.
The review should know that this check depends on the wording of that
stdout line. A change to it in `cli.mjs` makes the gate fail closed, not
pass. A test pins the dependency: T10b's unparseable-line subtest feeds a
line that still starts with `launched ` but has a later word changed
(`job` for `run`) and expects the refusal.
Follow-up, not in D1 (Sage files it if wanted): a bus-side PM marker, for
example a `pm: true` field on the launch event the `--pm` path writes,
would let the gate drop the journal read and the stdout-format
dependency.
`MainPID` equals `host.pid` because `scripts/mosaic` ends in `exec node
...cli.mjs`. Every `bus start` that loads the business calls
`recordBootOk` (`packages/cli/src/cli.mjs:494`), which writes `boot.at` as
the current time, so a `boot.at` from before step 5 means the status
describes an earlier start.
The healthz fetch is `node -e fetch(...)` with a 5 s abort, as
`webui-service.sh status` does it.
On a pass: rename the pointer into place, log `ok`, exit 0.
### Rollback inside the run (item 6)
1. Stop the Console, then the bus, and wait as in step 2.
2. Put back the two unit files and the drop-in directories from the
rollback set (temp file and rename each), and the wrapper bytes and
mode. Check them against `SHA256SUMS`. `daemon-reload`.
3. If step 3 of this run took a backup, restore it as `restore.sh` does
(a `rollback --restore` backup is never restored here): check
`DATA_ROOT`, the stopped check and `sha256sum -c`; rename `bus` to
`bus.post-attempt-<UTC stamp>`; `mkdir -m 700 bus`; copy in; check the
sums and that the file set matches exactly. Nothing is deleted.
4. Start the bus, then the Console. Run the gate against the pointer's
copy.
5. Log `gate-failed` (or the step's code) with `rollback: ok`, exit 1. If
the rollback's gate fails: print both results, stop both units, log
`rollback-failed` with both codes, exit 2. No retry.
## rollback (item 7)
1. `no-pointer`.
2. Target: scan the log newest first for `result: "ok"` lines with
`op` adopt, activate or rollback, and take the first `to` that isn't
the pointer's copy. None found refuses `no-previous`. This is
`release.sh`'s rule, and it shares that rule's property: two rollbacks
in a row go back and forth between two copies.
3. The item 3 refusals apply to the target (copy, schema, busy,
drop-ins, flag drift).
4. `schema-differs`: the pointer's `schema` is not the target's
`SCHEMA_VERSION` and there's no `--restore`. Decided from the pointer
before anything stops.
5. With `--restore ACTIVATION`: `backups/stack/ACTIVATION/` must exist
with `SHA256SUMS`, a `DATA_ROOT` equal to this root, and a `SCHEMA`
equal to the target's `SCHEMA_VERSION`, else `restore-mismatch`. Print
the post-attempt path the run will keep, and the count of events with
`at` after the backup's `STAMP`, from a read-only open of the store
before anything stops. Count above 0 and `--events N` not equal to it
refuses `post-backup-events`. No change.
6. Steps 4.1 to 4.5 with the target. In step 3, the stopped store's
version must equal the pointer's `schema`, else `store-differs`, item
6. With `--restore`, the count is read again after the stop. A
different count goes to item 6 with `post-backup-events`, since an
event landed between the check and the stop. Then the restore runs in
place of the migration.
7. The gate, as for activate. A pass renames the pointer (with the
target's schema) and logs `ok` with `backup: ACTIVATION`.
Ruling 4 (sage-0736, confirmed in sage-0738) has two cases:
- A `rollback --restore` that fails after the restore has moved the old
`bus/` aside: the run does not go through item 6. That covers the gate
and also an install, start or signal failure after the swap; the script
treats every one the same and logs `rollbackCode: "not-attempted"`. A
restore that fails before the move goes through item 6 like any other
step, since the store hasn't changed. Putting the newer store back would
be a second swap, which is a retry under another name. The run prints
both results, stops both units, logs `rollback-failed` and exits 2. The
post-attempt directory and the restored `bus` stay where they are for
Jason. The unit files and wrapper also stay as the run installed them,
since they match the restored store. Nothing is swapped back. Tested in
T14b.
- A `rollback` without `--restore` whose gate fails: the store was never
swapped, so the run goes through item 6 as `activate` does. It puts back
the unit files and wrapper from its rollback set, restarts the pointer's
copy and runs the gate on it. Exit 1 with `rollback: ok`, or exit 2 with
`rollback-failed` if that gate fails too. Tested in T14c.
## --plan (item 8)
It reads the pointer, the log, the target copy's store.mjs and the
store's schema version. It writes nothing: no lock, no log line, no file
under the data root.
Mos takes the `--plan` output to Jason before the combined re-pin
(mos-0741, sage-0745), so a migration must be stated plainly. For a
store at 3f and refactor at 3h, `activate --plan` prints:
```
store schema: 3f (read from the stopped store)
target schema: 3h
migration: 3f to 3h, after a backup of the stopped store
migration steps: 3f to 3g, 3g to 3h
needs: --migrate (not given; the run would roll back with migration-needs-flag)
needs: Jason present for this run (decision 96 condition 2)
```
I added the `migration steps:` line in the slot on 2026-10-18, before the
scratch-root run (see "The scratch-root plan for Mos"). It walks the target store.mjs's `from:` and
`to:` pairs from the store's version, as text, the same way
`copy_schema` reads the version, so nothing in the copy runs. When the
lines don't pair up, or the walk doesn't reach the target's version, it
prints `migration steps: cannot be read from <file>` and the rest of the
plan is unchanged. The run itself still calls `migrate-schema.mjs`, which
applies the copy's own chain; the steps line only names it.
The store version comes from the store itself. While the bus unit is
active the read is `{ readOnly: true }`, which adds no file next to a
store the bus holds (the probe). With the bus stopped and no `-wal` or
`-shm`, it is the immutable read. With no store file, a `-wal` or `-shm`
present, or a failed read, it falls back to the pointer's `schema` and
says which case applied. When the store and the pointer disagree, a
`note:` line says so; the run itself goes by the stopped store.
Other cases print `migration: none` (same version), a `store-newer`
warning (not in the target's chain) or, for rollback, `restore: ... needs
--restore ACTIVATION` with the activation the log's backup field names
for the pointer's copy. Then the exact `bus-service.sh`,
`webui-service.sh` and `stack-copy.sh wrapper` command lines. Exit 0.
The refusals it can see before a run (no-pointer, business-mismatch,
copy, already-active, no-previous) still print, with exit 3 and no log
line.
After the migration or restore lines comes the `commits:` block
(sage-0778): `commits: the run adds N` when the pointer's copy is an
ancestor of the target, `commits: the run takes away N` when the target
is an ancestor of the pointer's copy (a rollback), then the
`git log --oneline --no-merges` lines, newest first, at most 200. When
neither copy is an ancestor of the other, `--plan` refuses
`copies-unrelated`. My 16:53Z message to Sage said "refuse if FROM isn't
an ancestor of TO". The code is wider than that on purpose, so a
rollback still lists what it takes away. T25 and T26 test it.
### The scratch-root plan for Mos (sage-0837)
Mos asked (mos-0765) for the combined re-pin's `--plan` output with the
3g to 3h migration and rows 148, 141, 143, 128, 124 and 149. Before any
live adopt there is no pointer, so I run it against a scratch root under
`~/darkwing-scratch/r162/`: a data root, a config naming it, a
hand-written pointer and a copies directory, all scratch. Nothing under
`~/.mosaic-dev`, the live copies directory or the units is written.
- The live copy id: `systemctl --user show mosaic-bus@mosaic-stack.service
-p WorkingDirectory` at 2026-10-18T19:32:16Z gave
`.../copies/52f35e7a0933e9fa845f59860a5de1709f9e5756`. It reads the
unit's property and writes nothing. sage-0859 also allowed a read-only
`systemctl --user is-active` on the bus unit name; the run never made
that call, because with no store file `plan_store` doesn't reach it.
- The pointer's `schema` is that copy's `SCHEMA_VERSION`, read with
`git show 52f35e7a0:packages/bus/src/store.mjs`: `3f`. I had expected
3g. That is why the steps line exists: `migration: 3f to 3h` alone
doesn't show the 3g to 3h step Mos asked for, and hand-writing 3g would
have been a false pointer.
- The target copy, from `scratch-plan-steps.log`:
`(cd ~/darkwing-scratch/r162/wt && scripts/stack-copy.sh copy
606fafd43154ded993a503933681198e737987cd --copies
~/darkwing-scratch/r162/plan/copies --allow-worktree)` at 19:45:54Z,
copied by 19:45:56Z. Removed at 19:45:56Z with `chmod -R u+w` and
`rm -rf` of that one copy directory; the copies dir was left with 0
entries. 606fafd43 was origin/refactor at the run; it moved past
cfde62f9f after sage-0859.
- A header I put above the script's output names the scratch root and
says the pointer is hand-written. The script prints no such header
itself.
- The scratch root has no `bus/bus.sqlite`, so the store schema line
reads `from the pointer; there is no store file`. `--plan` runs only
`check_target` and `plan`, not the unit checks, so the only other reads
are git in the checkout and the copies directory. The data root held
only `state/stack-active.json` and an empty `bus/` after the run.
The result, `scratch-plan.txt` (134 lines, exit 0): `migration: 3f to
3h`, `migration steps: 3f to 3g, 3g to 3h`, both `needs:` lines, then
`commits: the run adds 112`. All six rows are in the list (grep hits per
row: 148 7, 141 5, 143 8, 128 9, 124 9, 149 18). The range also carries
row 164, the row 165 brief and everything else since 52f35e7a0.
Mos then narrowed the window target to `8e7ea89a1`, the six rows only
(mos-0810 D1-D2, sage-0871). I reran the same plan in a new slot with
`r1/plan-8e7.sh`, a copy of plan.sh with the target, the scratch root
(`plan-8e7`) and the output file changed. It ran from this worktree
while it still matched the r1 manifest. The result is
`scratch-plan-8e7ea89a1.txt` (114 lines, exit 0). The `migration` and
`migration steps` lines are unchanged. The plan lists 92 commits, the
same count as `git rev-list --count 52f35e7a0..8e7ea89a1`. Each of the
six landing commits appears once. Row 164's `008746b6e` and `850484a7d`
are not in it. Of the 92, only the six touch code paths. The step log
is `scratch-plan-8e7ea89a1-steps.log`. Its "plan: written to" line names
the wrong file, and `live-reads.md` explains why. Both 606fafd43 files
stay as they were, since this review uses them.
Sage forwards the output to Mos as a scratch-root plan. The real
`--plan` against the live pointer runs again in Mos's window before any
adopt.
## status (item 9)
The pointer, the last five log lines, any interrupted run,
`stack-copy.sh check` output and result, `<pointer copy>/scripts/mosaic
bus status`, whether the unit flags match the pointer, and the drift:
whether the copy is an ancestor of `refactor` and
`git rev-list --count COPY..refactor`. It only reads.
## Tests (`packages/cli/tests/stack-release.test.mjs`)
Fixture, built once per file:
- A temp canonical repo, made the way `stack-copy.test.mjs` `setup` does
it: `git ls-files -co` of scripts and packages copied in, `git init -b
refactor`, the light package.json. I copy the setup code rather than
import it, because importing a test file runs its tests too. The base
commit is F.
- On `refactor`, empty commits A, B, C1 and C (subjects `row 901: A` to
`row 904: C`). A side branch from A holds S and merges into `refactor`
as M, with `--no-ff`. A branch named `unreached` from A holds U.
`refactor` and `origin/refactor` both point at M.
- Read-only stub copies under a temp `copies/<id>/` for F, A, B, C1, C, S
and U. M has no copy, so it is the `copy-missing` case. Each copy has
`scripts/mosaic`, `packages/webui/src/cli.mjs` (present only, never
run), `packages/bus/src/store.mjs` and
`packages/bus/src/migrate-schema.mjs`. The store file in A, B, C1, S
and U says `3g` with a `from: '3f'` step. C says `3h` with `3f` and
`3g` steps. F has no `SCHEMA_VERSION`, which gives
`copy-schema-unknown`.
- The stub migrate records the `backups/stack/` entries it saw, and
whether each has `SHA256SUMS`, in `migrate.json`, then sets the store
to `3h` with `node:sqlite`. The `migrate-fails` fault sets
`3h-partial` and exits 1.
- The stand-ins `systemctl`, `journalctl` and `systemd-analyze` are node
scripts with a `#!<node>` shebang in a `sbin/` directory that comes
first on `PATH`. All three exist, so the real ones never run. A
`package.json` with `"type":"commonjs"` in the fixture's base
directory covers them, the holder and the copies' `scripts/mosaic`.
Per test: a temp `HOME`, `XDG_CONFIG_HOME`, `XDG_DATA_HOME`, data root,
`MOSAIC_CONFIG` (a valid config naming the temp data root), `bin/` with
the Q81 wrapper, `mosaic-old` and a fake `claude`, and a stand-in state
directory with a `faults/` directory. A fault is a file named for the
copy it hits, such as `healthz-<id>`, so one fault can fail the new copy
and leave the rollback target alone.
`seed` builds the stack the way an operator would: a real WAL-mode
`bus.sqlite` with a `meta` row and three events, then the real
`bus-service.sh install`, `webui-service.sh install` and
`stack-copy.sh wrapper` against the fixture copy, then a start of both
units. The pointer comes from a real `adopt`, or is written by hand with
no log (T23, T25), or is left out.
The stand-in `systemctl`:
- `show UNIT -p ...` reads the installed unit file and its drop-ins
(`[email protected]` and `<unit>.d`), fills in `%i` and `%h`, and
answers `LoadState`, `WorkingDirectory`, `ExecStart={ path=... ;
argv[]=... ; }`, `Environment`, `MainPID` and `DropInPaths`. The
`check-<id>` fault moves the Console's `WorkingDirectory`, so
`stack-copy.sh check` fails.
- `list-units`, `is-active`, `is-enabled`, `start`, `stop` and
`daemon-reload` keep unit state in `units.json` (written through a temp
file and a rename) and append every call to `calls.log`, so the order
is testable.
- `start` of the bus runs a holder process. The holder opens
`bus.sqlite` in WAL mode and keeps it open, writes `bus/writer.lock`
and a `host.json` with the business, its pid and a boot status, and
closes the store and removes the lock on SIGTERM. That gives the real
`-wal` and `-shm` files of a held store, and a clean close on stop. The
holder starts through `bash -c` with every fd above 2 closed, so it
never inherits the script's fd 9 lock. `SD_CLOCK_OFFSET` backdates the
seeded host and its PM run by two minutes, so they predate any run's
start.
- `start` of the bus with `--pm` in the unit's ExecStart appends the
`launched` line, as `cli.mjs:503` writes it, to the stand-in journal,
and adds the run to `launches.json`. A `pm-<id>` fault picks another
case: no line, two lines, a changed word, a run missing from the list,
an early `startedAt`, a human launch with no line, a run for another
business, a dead run.
- `stop` sends SIGTERM and waits for the holder to exit. The one-shot
`stop-leaves-wal` and `stop-leaves-shm` faults leave an empty `-wal` or
`-shm` beside the store, as an unclean close would. Each bus start and
stop records the name, size, mtime and sha256 of every file under
`bus/` in `snap.jsonl`, so T22 can compare the store before and after
the script's read.
- `start-dies-<id>` sets the bus to `failed`.
The stub `scripts/mosaic` prints `bus status --json` from `host.json`,
with `live` recomputed from `/proc`, and `launches list --json` from
`launches.json`. The `status-<id>` and `launches-<id>` faults make it
fail. The Console is an HTTP server inside the test process on a free
port. It answers 200 when the stand-in says the Console is active, and
500 when a `healthz-<id>` fault names the Console unit's current copy.
That is why the test runs the script with async `spawn`: `spawnSync`
would block the server.
`systemd-analyze --user unit-paths` prints the temp unit directory.
The stand-in `journalctl` logs the `--since` argument it got and prints
the stand-in journal lines for the unit at or after it. It accepts
`--since` only as `@<epoch seconds>` or `YYYY-MM-DD HH:MM:SS UTC`, and
exits 1 on any other form, so a bare local time gives no lines.
Every `run` call also checks T17: the listing of `backups/stack/`,
`state/stack-rollback/`, `bus.post-attempt-*` and `copies/` taken before
the run is a subset of the one after.
Clean-up kills only the holder pids the stand-in logged, after checking
each one's `/proc/<pid>/cmdline` names this file's holder.
Gate failure cases run with `--timeout 3`.
| # | Case | Brief |
|---|---|---|
| T1 | `adopt` writes the pointer (claude and console from `show`) and one `ok` line; a second adopt refuses `already-adopted`; a failing check refuses `check-failed` with no pointer | item 2 |
| T2 | `activate` installs exactly the pointer's `--pm`, `--claude` and Console flags (unit file text) | items 3, 4 |
| T3 | a hand-edited unit (each of pm, claude, a console flag) refuses `unit-flags-drift`, prints the difference, changes nothing | item 3 |
| T4 | each item 3 refusal: units, wrapper, pointer and store unchanged by sums, exactly one new log line with the code, no `start` or `stop` in `calls.log`; `instance-busy` has two subtests (another business's instance active, this one's activating) and both name the instance and state | item 3, ruling 1 |
| T5 | no-schema activate: `calls.log` shows stop Console, stop bus, daemon-reloads, start bus, start Console in order; pointer names C1; no `backups/stack/` | item 4 |
| T6 | B to C without `--migrate`: `migration-needs-flag`, old stack started, pointer unchanged | item 4.3 |
| T7 | B to C with `--migrate`: backup exists before the migrate stub ran (the stub records whether it saw the backup's sums), pointer schema `3h` | item 4.3 |
| T8 | B to C with `--migrate` and a failing migration: post-attempt dir exists, restored files match the backup's sums, old stack started | item 6 |
| T9 | `store-newer` (store at a version outside C's chain) | item 4.3 |
| T10 | each gate check failing alone (five subtests): rollback, pointer unchanged, unit files and wrapper back byte for byte, log names the check | items 5, 6 |
| T10b | the PM check, one subtest per case, each refusing `pm-launch-unconfirmed` with the case named: no `launched` line; two lines; a line that starts with `launched ` but has a later word changed (`launched mosaic-pm: job R, ...`); one line whose run isn't in `launches list`; one line whose run has `startedAt` before step 5; a live human `mosaic launch` session after step 5 with no journal line; a run for another business; a run that isn't live. A last subtest passes with one well-formed line and a live run, and checks the `--since` argument the stand-in received starts with `@` | item 5, ruling 2 |
| T11 | rollback gate fails too: `rollback-failed`, exit 2, both units stopped, no `start` after the failing one | item 6 |
| T12 | `rollback` finds the previous copy from the log | item 7 |
| T13 | `rollback` across a migration refuses `schema-differs`; with `--restore` it restores and goes live on the old copy | item 7 |
| T14 | `--restore` with post-backup events and no or a wrong `--events N` refuses `post-backup-events`; the right N proceeds | item 7 |
| T14b | a `rollback --restore` whose gate fails: `rollback-failed`, exit 2, both units stopped, the post-attempt dir and the restored `bus` in place, no rename after the restore | item 7, ruling 4 |
| T14c | a `rollback` without `--restore` whose gate fails: item 6 runs, the unit files and wrapper are back byte for byte from the run's rollback set, the pointer's copy is started and gated, exit 1 with `rollback: ok`; the store's file set and sums are unchanged | item 7, ruling 4 |
| T15 | `--plan` on activate and rollback prints the migration or restore and the flags; units, pointer, log and store unchanged. After an activate to C with `--migrate`, `rollback --plan` prints, line for line, the restore with the activation and `commits: the run takes away 2` | item 8 |
| T25 | sage-0778 ask 1: pointer on B, `activate S --plan` (S is on a side branch, so neither copy is an ancestor of the other) exits 3 with exactly `stack-release: refusing copies-unrelated: neither <B> nor <S> is an ancestor of the other, so the plan cannot list the commits`, empty stdout, no log, no lock | item 8, sage-0778 |
| T26 | sage-0778 ask 2: `activate C --plan`, with and without `--migrate`, prints the whole output line for line: the migration line, the `--migrate` need, the Jason line, then `commits: the run adds 2` and the two `row 904` and `row 903` lines, then the three install lines. `activate C1 --plan` prints `migration: none` | item 8, sage-0778 |
| T27 | the steps line: store at `3f`, `activate C --plan` prints `migration: 3f to 3h` then `migration steps: 3f to 3g, 3g to 3h`; `activate C1 --plan` (C1's fixture store.mjs has `from:` lines and no `to:` lines) prints `migration steps: cannot be read from <C1>/packages/bus/src/store.mjs` and still exits 0; `activate A --plan` (A's `to:` names `3x`, so the walk stops short of `3g`) prints the same for A | item 8, sage-0859 |
| T23 | sage-0745: the store at `3g` and stopped, the pointer on B, target C at `3h`. `activate C B --plan` exits 0 and prints `3g`, `3h`, `--migrate` and `Jason present ... decision 96 condition 2`. Afterwards the store's file set, mtime and sha256, the listing under the data root, the unit files and the wrapper are unchanged; there is no log file and no lock file. A second subtest with the bus active reads the version through the held read and prints the same lines | item 8, mos-0741 |
| T24 | `business-mismatch` (pointer for B, run for another business) and `capture-failed` (the rollback set's parent made read-only): each exits 3 with one `refused` line, no `stop` in `calls.log`; `capture-failed` follows a `started` line, so `status` shows no interrupted run afterwards. An activate ID that isn't 40 hex is a usage error, exit 4, no log | item 3, item 4.1 |
| T16 | `status` prints the pointer, five log lines, check, bus status and drift, and writes nothing | item 9 |
| T17 | in every `run` call: nothing under `backups/stack/`, `state/stack-rollback/`, `bus.post-attempt-*` or `copies/` is ever removed (the listing before the run is a subset of the one after). The store sums are checked in each test through its own fingerprint | item 10 |
| T18 | `stack-copy.test.mjs` still passes with `reachable` split out and the source guard | stack-copy.sh change |
| T19 | a `started` line with no final line: `status` shows the interrupted activation; `adopt`, `activate` and `rollback` each refuse `interrupted-run` naming it, with nothing changed | ruling 3 |
| T20 | a held `state/stack-release.lock`: `adopt`, `activate` and `rollback` refuse `run-locked` and change nothing; `status` and `--plan` still run | ruling 6 |
| T21 | a bus unit with no `MOSAIC_CLAUDE_BIN`: `adopt` and `activate` refuse `no-claude-pin` | ruling 5 |
| T22 | the store read (sage-0739), three subtests. A `bus.sqlite-wal` left after the stop refuses `store-unclean` naming the file, goes to item 6, and leaves every file under `bus/` with the same name, size, mtime and sha256. The same with a `bus.sqlite-shm`. The clean case: an activate reads the version, and afterwards `bus/` holds only `bus.sqlite`, with its mtime and sha256 the same as before the read | item 4.3, the store read |
## Mutants
One per behavior. Each is a single edit to a scratch copy of
`stack-release.sh` or `stack-copy.sh`, run against the file and expected
to fail the named test. The run script puts each mutant in its own
process group and kills the group on timeout.
| M | Mutation | Killed by |
|---|---|---|
| M1 | adopt skips the already-adopted check | T1 |
| M2 | adopt reads console flags without dropping `--checkout` | T1, T2 |
| M3 | activate installs without `--claude` | T2 |
| M4 | drift check ignores console flags | T3 |
| M5 | `instance-busy` check removed | T4 |
| M6 | `drop-in-override` check removed | T4 |
| M7 | `copy-unreachable` check removed | T4 |
| M8 | a refusal logs nothing | T4 |
| M9 | the bus stops before the Console | T5 |
| M10 | the Console starts before the bus | T5 |
| M11 | the migration runs without `--migrate` | T6 |
| M12 | the backup is taken after the migration | T7 |
| M13 | item 6 skips the store restore | T8 |
| M14 | `store-newer` treated as same | T9 |
| M15 to M19 | each gate check made to always pass | T10, one subtest each |
| M20 | the pointer is written before the gate | T10 |
| M21 | item 6 skips the wrapper restore | T10 |
| M22 | a failed rollback gate retries | T11 |
| M23 | rollback picks the newest `to` without skipping the current copy | T12 |
| M24 | `schema-differs` check removed | T13 |
| M25 | `--events` not compared | T14 |
| M26 | `--plan` writes a log line | T15 |
| M27 | restore deletes the post-attempt directory | T13, T8 (the run killed it there, not in T17) |
| M28 | `reachable` returns 0 always | T4, T18 |
| M29 | `instance-busy` ignores this business's transitional state | T4 (activating subtest) |
| M30 | the PM check accepts any live session started after step 5 | T10b (human launch subtest) |
| M31 | a `rollback --restore` gate failure goes through item 6 | T14b |
| M32 | the interrupted-run check removed | T19 |
| M33 | the run lock removed | T20 |
| M34 | the `no-claude-pin` check removed | T21 |
| M35 | `--since` given a bare local time instead of `@<epoch>` | T10b (pass subtest) |
| M36 | the PM check takes the first `launched` line instead of requiring exactly one | T10b (two-lines subtest) |
| M37 | the PM check skips the `startedAt` comparison | T10b (early-run subtest) |
| M38 | the `store-unclean` check removed | T22 (`-wal` and `-shm` subtests) |
| M39 | the store read uses `{ readOnly: true }` without immutable | T22 (clean subtest) |
| M40 | a `rollback` without `--restore` whose gate fails exits 2 without item 6 | T14c |
| M41 | `--plan` omits the `Jason present` line | T23 |
| M42 | `--plan` takes the run lock or logs `started` | T23, T20 |
| M43 | `--plan` reads the store with `{ readOnly: true }` while stopped, without immutable | T23 (file set) |
| M44 | a capture failure leaves the `started` line without a final line | T24 |
| M45 | the `copies-unrelated` refusal removed, or the `commits:` block left out | T25, T26 |
| M46 | `--plan` omits the migration line | T26, T23 |
| M47 | `--plan` omits the `migration steps:` line | T26, T23, T27 |
| M48 | `plan_steps` drops the odd-line-count check | T27 |
| M49 | `plan_steps` prints a partial walk that doesn't reach the target | T27 (the A case) |
## Candidate round 1
`r1/build.patch` against BASE `cfde62f9f` (sha256
`f5d28a751cb77c4054c598b36dd4dd996d7372bb47a0a8030c39bc8517b8a397`).
`r1/candidate-manifest.sha256` (sha256
`091693c51d137e4cd31e75ee22aef4f22c307e15abce6f91f6c518e3ff78fdd6`) lists
the 5 files in "Files". I applied `r1/build.patch` with `git apply
--index` to a fresh worktree at cfde62f9f: `sha256sum -c` passed on
every line, `git diff --cached --check` was clean, the staged names match
the manifest (5 files, +1604 -45), and the worktree was removed after.
`scripts/stack-release.sh` is mode 100755, the test file 100644. The
TOOLS.md diff is one hunk, inside "Pinned stack update".
### Checks round 1
On Sage's slot, 2026-10-18T19:25:10Z to 19:46:14Z, worktree at
cfde62f9f, TMPDIR `~/darkwing-scratch/r162/tmp`, each file run singly
with `timeout 600 node --test --test-reporter=spec`:
| Test file | Pass | Fail | Run |
|---|---|---|---|
| `packages/cli/tests/stack-release.test.mjs` | 58 | 0 | 19:43:19Z, after the steps change |
| `packages/cli/tests/stack-copy.test.mjs` | 12 | 0 | 19:45:27Z |
The first run in the slot, before the capture `mkdir` fix, was 56/57
with T24 failing (see "Notes"); the second was 57/57; the steps change
added T27. Mutants: all 47 killed against the 57-test file, then M41,
M45b, M46 and the new M47 to M49 killed against the 58-test file. The
mutant runner restores each file from a pristine copy and checks its
sha256 after every mutant. `sweep.sh` found no process of mine after.
Logs: `r1/stack-release-test.log`, `r1/stack-copy-test.log`,
`r1/mutants.summary`, `r1/mutants-steps.summary` and the runner
`r1/mutants.py`.
Not run here: the cli, bus and webui suites and the `scripts/test-*.sh`.
They are Sage's gate at landing.
## Candidate round 2 (sage-0882)
Row 125 landed as aa625565f, with its packet at df4d5702f (origin/refactor). It changed two of
this row's five paths, `docs/TOOLS.md` and `packages/cli/README.md`, so the r1 manifest no
longer described the tree this row would land on. Round 2 makes no source change.
I applied `r1/build.patch` with `git apply --index` to a fresh worktree at df4d5702f. It
applied without conflict, `git diff --cached --check` was clean, and the staged names are the
same 5 files (+1604 -45). The modes are unchanged: both scripts 100755, the test file 100644.
The worktree was removed after.
`r2/build.patch` against BASE `df4d5702f` (sha256
`336503b59a894c8ee0449dba5c36feb242a80404c9f262e96740def231fa541b`).
`r2/candidate-manifest.sha256` (sha256
`c213a801681b5dd9029520d606b3303cb30ed2fbfff8e320b0f7d79d483bd15c`).
What moved from r1:
- In `r2/build.patch`, only the two `index` lines for `docs/TOOLS.md` and
`packages/cli/README.md`, the base and result blob ids. Every hunk header, context line and
added or removed line is the same as in r1, because row 125's additions to both files sit
below this row's hunks.
- In the manifest, only the sha256 of those two files, since the files now carry row 125's
text too. `stack-release.test.mjs`, `stack-copy.sh` and `stack-release.sh` keep their r1
sha256s.
Nothing was rerun. `stack-release.test.mjs` and `stack-copy.test.mjs` read neither doc file,
and the scripts are byte-identical to r1, so the round 1 checks stand.
## Rulings (Sage, sage-0736, with sage-0738, sage-0739 and sage-0745)
1. `instance-busy`: approved as read. Refuse when another business's
`mosaic-bus@` instance is active, because the copy and wrapper switch
reach it too, or this business's is activating, deactivating or
reloading. The message names the instance and state. Test both cases.
2. PM check: approved without reading the business file, and it must
match the PM role: a live launch for B whose role is the PM, started at
or after this run's bus start. Don't name `launch.by`. sage-0738
accepted the journal reading in "The health gate" above, with an
unambiguous `--since`, exactly one line, and the one code
`pm-launch-unconfirmed` for every failure. Each refusal is tested
(T10b), and the stdout-line dependency is stated for the review. The
bus-side PM marker is a follow-up outside D1.
3. Logging: `started` and a final line in the jsonl, step detail in
`steps.log`. An unfinished `started` is an interrupted run: `status`
shows it and the next `adopt`, `activate` or `rollback` refuses
`interrupted-run`, naming the activation. Clearing it is Jason's or
Mos's. Tested.
4. A rollback whose own gate fails stops, prints both results, leaves the
stack stopped and writes `rollback-failed`. No further renames: a
second swap is a retry under another name. The post-attempt dir and
the restored bus stay for Jason. sage-0738 confirmed my reading: after
a restore, the unit files and wrapper stay as installed, and a
rollback that didn't restore goes through item 6 (T14b, T14c).
5. `no-claude-pin`: approved for both `adopt` and `activate`.
6. Run lock: approved, `flock -n` on `state/stack-release.lock`, code
`run-locked`. `status` and `--plan` take no lock.
SQLite probe: on Sage's slot (Rocko first, then me), on a fixture store
in my scratch, never the live store or a copy of it. Try `readOnly` and
`file:...?immutable=1`, and check no `-shm` or `-wal` appears and the
main file's mtime and hash hold. If neither works, the copy fallback.
Result in "The store read": readOnly left `-shm` and `-wal`, immutable
left nothing. sage-0739 approved immutable only with no `-wal` or `-shm`
present, `store-unclean` otherwise, no copy fallback, tested in T22.
sage-0745 (mos-0741): Mos takes the combined re-pin's `--plan` preview to
Jason, so `activate --plan` names the store's version and the target's
(3g to 3h there), that `--migrate` is needed, and that Jason must be
present under decision 96 condition 2. Tested in T23 with a fixture
store one version behind, which also checks that `--plan` writes
nothing. Base moved to 70504d23a, since row 124 touched
`packages/bus/src/store.mjs` and `schema.sql`.
## Notes
- T24 failed on the first run in the slot: `mkdir`'s "Permission denied"
reached stderr ahead of the `capture-failed` refusal. Both `mkdir`
calls in `capture` now send stderr to /dev/null; the refusal names the
path. The rerun passed.
- The install scripts render from the canonical checkout's templates. A
later rollback to an older copy renders that copy's units with today's
templates, and the in-run rollback restores the captured bytes. The
window plans had the same property.
- Nothing here prunes. Disk use grows by one rollback set per run and one
store copy per migrating run.
- The new TOOLS.md "Pinned stack update" section numbers the drop-in
removal as step 4, so the `drop-in-override` message says
`TOOLS.md step 4`. The brief's item 3 text still says step 3, from the
old seven-step list.
agents/darkwing/work/queue-162/followup-brief.md
+59
View File
@@ -0,0 +1,59 @@
# Follow-up row brief: stack-release.sh hardening after D1 (row 162 review notes)
Draft by Darkwing for Sage (sage-0882), from Filbert's five non-blocking notes on row 162
round 1, #1659 comment 28997. Line numbers are `scripts/stack-release.sh` and the brief as of
row 162's candidate; the script is byte-identical in r1 and r2. This row starts only after
row 162 lands, and it rebases on that landing.
Suggested gate: Filbert approves the candidate manifest; cli suite and every
`scripts/test-*.sh` green on Sage's gate. Suggested owner: Darkwing, reviewer Filbert, gate
owner Sage.
None of the five is a known failure today. Items 1 to 4 each fail closed or can't be reached
with the current pin. The fix is to make each one refuse earlier, or tell the truth sooner,
with codes row 162 already has. No new refusal code.
## Items
1. Precheck unreadable drop-ins. `drop_in_override` (:503) scans the same three directories as
`install_guard` in `scripts/bus-service.sh` (:100), but skips a directory or file it can't
read (`[ -d ]`, `grep 2>/dev/null`). `install_guard` dies on that same path, after the stop,
so the run goes through item 6 and the live stack stopped for nothing. The precheck refuses
`drop-in-override` with the path when a unit path, drop-in directory or `.conf` file exists
but can't be read, using `install_guard`'s own tests (`-r`, `-x`, a failed read). Test: a drop-in file with mode 000 (and a
directory with mode 000) refuses before any `systemctl stop` in the stand-in log, and the
units stay active.
2. Close the `is-active` to read race. `plan_store` (:573, check at :577) and the `--restore`
event count (:716) choose the held-store read or the stopped-store read from one
`is-active` call. If the bus stops between the check and the readOnly open, the open can
leave `-wal` or `-shm` beside a stopped store, and the next run refuses `store-unclean`.
Fix: after a held-store read, check `is-active` again. If the bus stopped meanwhile, report
it (`--plan`: say the store state changed during the read and print no migration line;
rollback: refuse `restore-mismatch` before anything stops) instead of using the count.
Test: a stand-in `systemctl` that answers active then inactive.
3. Compare the copy check with the target. The `copy-check` gate step (:373) runs
`stack-copy.sh check --business`, which proves the bus, Console and CLI agree on one copy,
not that the copy is the one this run installed. Pass the expected copy id (or compare the
check's reported id with the target) and fail the gate with `copy-check` on a mismatch.
Test: a stand-in check that reports agreement on another copy fails the gate.
4. Don't split a quoted `Environment=`. `read_units` (:148) reads the `systemctl show`
output (`show` at :144) and splits its `Environment=` line on spaces with `read -r -a`
(:157). systemd quotes a value that holds a space, so a `MOSAIC_CLAUDE_BIN` path with a
space reads as a prefix of itself. Smallest fix: `read_units` returns 1 when that line holds
a quote or backslash, so its callers refuse as they do for an unreadable unit
(`unit-flags-drift` at :535, `check-failed` at :561; the status view at :743 then skips its
unit flags line). Parsing systemd's
quoting is the larger option. The same split reads `ExecStart` argv (`argv_of`, :145, used
at :153 and :159); cover it the same way. Test: a stand-in unit with
`Environment="MOSAIC_CLAUDE_BIN=/a b/claude"` refuses by name and never reads `/a`.
5. Re-pin brief item 8. `docs/plans/2026-10-18_stack-release.md` item 8 (:180-185) says the
plan reads the schema "with no store read". sage-0745 and mos-0741 replaced that with the
held or immutable read that row 162 implements. Sage re-pins the brief text; no code.
## Out of scope
The bus-side PM marker (ruling 2's follow-up) and anything in Mos's live window.
agents/darkwing/work/queue-162/live-reads.md
+68
View File
@@ -0,0 +1,68 @@
# Row 162 live reads for the scratch-root plan (sage-0837, sage-0859)
One live read, at 2026-10-18T19:32:16Z, by Darkwing:
```
systemctl --user show [email protected] -p WorkingDirectory
```
Its output, as recorded in the session transcript:
```
WorkingDirectory=/home/jwoltje/.local/share/mosaic-stack/copies/52f35e7a0933e9fa845f59860a5de1709f9e5756
```
`systemctl show` reads the
unit's properties from the user manager. It writes nothing: no unit
file, no drop-in, no state under `~/.mosaic-dev` and no change to the
unit's state.
sage-0859 also allowed a read-only `systemctl --user is-active` on the
bus unit name. It was never called. The scratch data root has no store
file, so `plan_store` sets the store schema from the pointer before it
reaches the `is-active` branch.
The pointer's schema, `3f`, came from git in the canonical checkout, not
from anything live:
```
git show 52f35e7a0933e9fa845f59860a5de1709f9e5756:packages/bus/src/store.mjs
```
Nothing under `~/.mosaic-dev`, the live copies directory or the systemd
units was written. The scratch copy and its removal are in
`scratch-plan-steps.log`; the plan output is `scratch-plan.txt`.
## Rerun for target 8e7ea89a1 (sage-0871, mos-0810 D1-D2)
One more live read, at 2026-10-18T19:51:55Z, by Darkwing, at the start of
the sage-0871 slot:
```
systemctl --user show [email protected] -p WorkingDirectory
```
Its output:
```
WorkingDirectory=/home/jwoltje/.local/share/mosaic-stack/copies/52f35e7a0933e9fa845f59860a5de1709f9e5756
```
The live copy had not changed since the 19:32:16Z read. No `is-active`
read was made this time either. The pointer is hand-written as before:
copy `52f35e7a0`, schema `3f` from `git show` of that copy's store.mjs.
The script was `r1/plan-8e7.sh`, a copy of plan.sh with three changes:
the target is `8e7ea89a1` instead of `origin/refactor`, the scratch root
is `~/darkwing-scratch/r162/plan-8e7` and the output file is
`scratch-plan-8e7ea89a1.txt`. It ran from the r1 candidate worktree on
cfde62f9f, which matched `r1/candidate-manifest.sha256` at 19:51Z. The
scratch copy was made at 19:51:56Z and removed at 19:51:58Z, leaving 0
entries.
One defect in the step log: its "plan: written to" line names
`out/scratch-plan.txt`. My edit changed only the quoted output path, not
the unquoted one in that log line. The output went to
`out/scratch-plan-8e7ea89a1.txt`. The 606fafd43 output kept its sha256
(0586168e…), so the run did not touch it. The step log is kept as
written.
agents/darkwing/work/queue-162/r1/build.patch
+1706
View File
File diff suppressed because it is too large Load Diff
agents/darkwing/work/queue-162/r1/candidate-manifest.sha256
+5
View File
@@ -0,0 +1,5 @@
0e21117afe4acf0460fe10beecfddf4c90d1ceb30b6bb4f366b5bf5b3a5897c6 docs/TOOLS.md
7d5b4b7a7fcf075914438024fe30083d3ba03c11c834da729d87581b6d897419 packages/cli/README.md
73cdd6c2e2eceffa223d25c1af5f549c369d3cb68d906613ddd6f08c05db1bb0 packages/cli/tests/stack-release.test.mjs
19473e30b35c19cf2167fbf0d7aa0d3ed46562d50e44b738bb611d95cd123254 scripts/stack-copy.sh
6a2c828fefef0f467a1bea7c51f927790ee8990d58cfd26372c521ac64c53d7e scripts/stack-release.sh
agents/darkwing/work/queue-162/r1/mutants-steps.summary
+7
View File
@@ -0,0 +1,7 @@
M41 KILLED rc=1 tests=3 fail=3 1s failed=['T23 --plan reads the store without changing a file, stopped or held', 'held store', 'stopped store']
M45b KILLED rc=1 tests=1 fail=1 1s failed=['T26 --plan lists the commits an activate adds, after the migration lines']
M46 KILLED rc=1 tests=4 fail=4 2s failed=['T23 --plan reads the store without changing a file, stopped or held', 'T26 --plan lists the commits an activate adds, after the migration lines', 'held store', 'stopped store']
M47 KILLED rc=1 tests=5 fail=5 3s failed=['T23 --plan reads the store without changing a file, stopped or held', 'T26 --plan lists the commits an activate adds, after the migration lines', 'T27 --plan names each migration step, or says the steps cannot be read', 'held store', 'stopped store']
M48 KILLED rc=1 tests=1 fail=1 1s failed=['T27 --plan names each migration step, or says the steps cannot be read']
M49 KILLED rc=1 tests=1 fail=1 1s failed=['T27 --plan names each migration step, or says the steps cannot be read']
DONE 6 mutants, 6 killed, survived: []
agents/darkwing/work/queue-162/r1/mutants.py
+147
View File
@@ -0,0 +1,147 @@
#!/usr/bin/env python3
# Row 162 mutants M1-M46 (agents/darkwing/work/queue-162/BUILD.md). Each mutant is exact-string
# edits to one file in the r162 worktree; the named tests must fail. The file is restored from a
# pristine copy after every mutant and checked by sha256. Run with python3 -I.
import hashlib, os, re, shutil, signal, subprocess, sys, time
HOME = os.environ["HOME"]
R = f"{HOME}/darkwing-scratch/r162"
WT = f"{R}/wt"
OUT = f"{R}/out/mutants"
TEST = "packages/cli/tests/stack-release.test.mjs"
SR = "scripts/stack-release.sh"
SC = "scripts/stack-copy.sh"
def sha(p):
return hashlib.sha256(open(p, "rb").read()).hexdigest()
M = []
def mut(mid, pattern, edits, f=SR, expect=None):
M.append((mid, pattern, edits, f, expect))
mut("M1", "^T1 ", [(' [ ! -f "$POINTER" ] || { read_pointer; FROM="$P_COPY"; refuse already-adopted "$POINTER exists (copy $P_COPY)"; }\n', "")])
mut("M2", "^T[12] ", [(' if [ "${argv[i]}" = --checkout ]; then i=$((i + 1)); continue; fi\n', "")])
mut("M3", "^T2 ", [(' --claude "$claude" >> "$STEPS"', ' >> "$STEPS"')])
mut("M4", "^T3 ", [(' [ "${P_CONSOLE[*]}" = "${U_CONSOLE[*]}" ] || d+=$\'\\n\'" console:', ' true || d+=$\'\\n\'" console:')])
mut("M5", "^T4 ", [(" instance_busy\n drop_in_override\n", " drop_in_override\n")])
mut("M6", "^T4 ", [(" instance_busy\n drop_in_override\n", " instance_busy\n")])
mut("M7", "^T4 ", [('copy_ok "$TO" 2>/dev/null || refuse copy-unreachable', 'true || refuse copy-unreachable')])
mut("M8", "^T4 ", [('|| log_line result refused code "$code"', "|| true")])
mut("M9", "^T5 ", [(' systemctl --user stop "$CONSOLE_UNIT" || return 1\n systemctl --user stop "$BUS_UNIT" || return 1\n',
' systemctl --user stop "$BUS_UNIT" || return 1\n systemctl --user stop "$CONSOLE_UNIT" || return 1\n')])
mut("M10", "^T5 ", [('systemctl --user start "$BUS_UNIT" && systemctl --user start "$CONSOLE_UNIT"',
'systemctl --user start "$CONSOLE_UNIT" && systemctl --user start "$BUS_UNIT"')])
mut("M11", "^T6 ", [(' [ "$MIGRATE" = 1 ] || rollback_in_run migration-needs-flag\n', "")])
mut("M12", "^T7 ", [(' backup_store || rollback_in_run backup\n node "$COPIES/$TO/packages/bus/src/migrate-schema.mjs" "$DATA" >> "$STEPS" 2>&1 || rollback_in_run migrate-failed\n',
' node "$COPIES/$TO/packages/bus/src/migrate-schema.mjs" "$DATA" >> "$STEPS" 2>&1 || rollback_in_run migrate-failed\n backup_store || rollback_in_run backup\n')])
mut("M13", "^T8 ", [(' elif [ -n "$TAKEN" ] && ! restore_store "$TAKEN"; then rcode=restore\n', "")])
mut("M14", "^T9 ", [(' if [ "$STORE_SCHEMA" != "$T_SCHEMA" ]; then\n', ' if [ "$STORE_SCHEMA" != "$T_SCHEMA" ] && in_chain "$STORE_SCHEMA"; then\n')])
mut("M15", "^T10 ", [(' systemctl --user is-active --quiet "$BUS_UNIT" && systemctl --user is-active --quiet "$CONSOLE_UNIT" || return 1\n GATE=bus-status\n',
' true || return 1\n GATE=bus-status\n')], expect="units-inactive")
mut("M16", "^T10 ", [(" GATE=bus-status\n", " GATE=bus-status\n if false; then\n"),
(' [ -z "$GATE_WHY" ] || return 1\n GATE=healthz\n', ' fi\n GATE=healthz\n')], expect="bus-status")
mut("M17", "^T10 ", [('-- "$(console_port)" || return 1', '-- "$(console_port)" || true')], expect="healthz")
mut("M18", "^T10 ", [('--business "$BUSINESS" > /dev/null 2>&1 || return 1', '--business "$BUSINESS" > /dev/null 2>&1 || true')], expect="copy-check")
mut("M19", "^T10 ", [('pm_launched "$copy" || return 1', 'pm_launched "$copy" || true')], expect="pm-launch-unconfirmed")
mut("M20", "^T10 ", [(' gate "$TO" "$P_PM" || after_fail "$GATE"\n trap - INT TERM HUP\n write_pointer "$TO" "$P_PM" "$P_CLAUDE" "$T_SCHEMA" "${P_CONSOLE[@]}"\n',
' write_pointer "$TO" "$P_PM" "$P_CLAUDE" "$T_SCHEMA" "${P_CONSOLE[@]}"\n gate "$TO" "$P_PM" || after_fail "$GATE"\n trap - INT TERM HUP\n')])
mut("M21", "^T10 ", [(' tmp="$(mktemp "$BIN/.mosaic.XXXXXX")" && cp -p "$RS/wrapper" "$tmp" && chmod "$(cat "$RS/WRAPPER_MODE")" "$tmp" && mv -f "$tmp" "$BIN/mosaic" || return 1\n', ""),
(' [ "$(sha256sum < "$BIN/mosaic" | cut -d\' \' -f1)" = "$(sum_of "$RS" wrapper)" ] || return 1\n [ "$(stat -c %a "$BIN/mosaic")" = "$(cat "$RS/WRAPPER_MODE")" ] || return 1\n', "")])
mut("M22", "^T11 ", [(' elif ! gate "$P_COPY" "$P_PM"; then rcode="$GATE"\n',
' elif ! gate "$P_COPY" "$P_PM" && ! { start_both && gate "$P_COPY" "$P_PM"; }; then rcode="$GATE"\n')])
mut("M23", "^T12 ", [('if(mode==="previous"&&o.to!==current)', 'if(mode==="previous")')])
mut("M24", "^T13 ", [('[ "$P_SCHEMA" = "$T_SCHEMA" ] || [ -n "$RESTORE" ] || refuse schema-differs', 'true || refuse schema-differs')])
mut("M25", "^T14 ", [('[ "$COUNT" = 0 ] || [ "$EVENTS" = "$COUNT" ] || refuse post-backup-events', '[ "$COUNT" = 0 ] || [ -n "$EVENTS" ] || refuse post-backup-events')])
mut("M26", "^T15 ", [('if [ "$PLAN" = 1 ]; then check_target; plan; exit 0; fi', 'if [ "$PLAN" = 1 ]; then log_line result planned; check_target; plan; exit 0; fi', 2)])
mut("M27", "^T(8|13) ", [(' mv "$DATA/bus" "$POST" || return 1\n', ' mv "$DATA/bus" "$POST" && rm -rf "$POST" || return 1\n')])
mut("M28", "^T4 ", [('git -C "$REPO" merge-base --is-ancestor "$1" "$ref" || die', 'true || die')], f=SC)
mut("M29", "^T4 ", [('case "$state" in activating|deactivating|reloading) refuse instance-busy "$unit is $state" ;; esac', ":")])
mut("M30", "^T10b ", [(' [ "${#lines[@]}" -gt 0 ] || { GATE_WHY="no launched line since the start"; return 1; }\n',
' [ "${#lines[@]}" -gt 0 ] || lines=("launched x: run $("$1/scripts/mosaic" launches list --json 2>/dev/null | node -e \'const s=JSON.parse(require("fs").readFileSync(0,"utf8")).find((x)=>x.live);process.stdout.write(s?s.run:"none")\'), y")\n')],
expect="a human launch with no journal line")
mut("M31", "^T14b ", [(' [ -z "$RESTORED" ] || rollback_failed "$1" not-attempted\n', "")])
mut("M32", "^T19 ", [('[ -z "$open" ] || refuse interrupted-run', 'true || refuse interrupted-run')])
mut("M33", "^T20 ", [("flock -n 9 || refuse run-locked", "true || refuse run-locked")])
mut("M34", "^T21 ", [('[ -n "$U_CLAUDE" ] || refuse no-claude-pin', 'true || refuse no-claude-pin', 2)])
mut("M35", "^T10b ", [('--since "@$START_EPOCH"', '--since "$(date -d "@$START_EPOCH" \'+%Y-%m-%d %H:%M:%S\')"')], expect="one well-formed line and a live run pass")
mut("M36", "^T10b ", [(' [ "${#lines[@]}" = 1 ] || { GATE_WHY="${#lines[@]} launched lines since the start"; return 1; }\n', "")], expect="two lines")
mut("M37", "^T10b ", [(':!(Date.parse(s.startedAt)>=Number(since))?`run ${run} started before this start`', "")], expect="run started before the start")
mut("M38", "^T22 ", [('if [ -e "$f" ] || [ -L "$f" ]; then UNCLEAN="$f"; return 0; fi', ":")])
mut("M39", "^T22 ", [('new DatabaseSync(pathToFileURL(file).href+"?immutable=1",{readOnly:true})', "new DatabaseSync(file,{readOnly:true})")], expect="clean")
mut("M40", "^T14c ", [(' [ -z "$RESTORED" ] || rollback_failed "$1" not-attempted\n', ' [ -z "$RESTORED" ] && [ "$OP" != rollback ] || rollback_failed "$1" not-attempted\n')])
mut("M41", "^T23 ", [(' echo "needs: Jason present for this run (decision 96 condition 2)"\n', "")])
mut("M42", "^T(23|20) ", [(' [ "$PLAN" = 1 ] || lock\n', " lock\n", 2)])
mut("M43", "^T23 ", [(' S_SCHEMA="$(store_read version 2>/dev/null)" || S_SCHEMA=""\n', ' S_SCHEMA="$(node -e "$STORE_JS" -- heldversion "$STORE" 2>/dev/null)" || S_SCHEMA=""\n')])
mut("M44", "^T24 ", [(' capture || refuse capture-failed "could not write the rollback set $RS; nothing was stopped"\n',
' capture || { echo "stack-release: refusing capture-failed: could not write the rollback set $RS; nothing was stopped" >&2; exit 3; }\n')])
mut("M45a", "^T25 ", [(' else refuse copies-unrelated "neither $P_COPY nor $TO is an ancestor of the other, so the plan cannot list the commits"\n',
' else RANGE="$P_COPY..$TO" VERB=adds\n')])
mut("M45b", "^T26 ", [(" plan_commits\n", "")])
mut("M46", "^T(26|23) ", [(' echo "migration: $S_SCHEMA to $T_SCHEMA, after a backup of the stopped store"\n', "")])
mut("M47", "^T(26|23|27) ", [(' echo "migration steps: $(plan_steps)"\n', "")])
mut("M48", "^T27 ", [(' (( ${#l[@]} % 2 == 0 )) || { echo "cannot be read from $f"; return; }\n', "")])
mut("M49", "^T27 ", [(' if [ "$v" = "$T_SCHEMA" ] && [ -n "$out" ]; then', ' if [ -n "$out" ]; then')])
def main():
only = set(sys.argv[1:])
os.makedirs(OUT, exist_ok=True)
pristine = {}
for f in (SR, SC):
p = f"{R}/edit/pristine-{os.path.basename(f)}"
shutil.copy2(f"{WT}/{f}", p)
if sha(p) != sha(f"{WT}/{f}"):
sys.exit(f"ABORT: backup of {f} failed")
pristine[f] = (p, sha(p))
env = dict(os.environ, TMPDIR=f"{R}/tmp")
results = []
try:
for mid, pattern, edits, f, expect in M:
if only and mid not in only:
continue
src = open(pristine[f][0]).read()
text = src
for e in edits:
old, new = e[0], e[1]
n = e[2] if len(e) > 2 else 1
if text.count(old) != n:
sys.exit(f"ABORT {mid}: expected {n} match(es) of {old!r} in {f}, found {text.count(old)}")
text = text.replace(old, new)
if text == src:
sys.exit(f"ABORT {mid}: no change")
if os.environ.get("DRY"):
print(f"{mid} ok", flush=True); continue
open(f"{WT}/{f}", "w").write(text)
log = f"{OUT}/{mid}.log"
t0 = time.time()
with open(log, "w") as fh:
p = subprocess.Popen(["node", "--test", "--test-reporter=spec", f"--test-name-pattern={pattern}", TEST],
cwd=WT, env=env, stdout=fh, stderr=subprocess.STDOUT, start_new_session=True)
try:
rc = p.wait(timeout=600)
except subprocess.TimeoutExpired:
os.killpg(p.pid, signal.SIGKILL); p.wait(); rc = "timeout"
shutil.copy2(pristine[f][0], f"{WT}/{f}")
if sha(f"{WT}/{f}") != pristine[f][1]:
sys.exit(f"ABORT {mid}: restore of {f} failed")
body = open(log).read()
ran = re.search(r"^ℹ tests (\d+)", body, re.M)
fails = re.search(r"^ℹ fail (\d+)", body, re.M)
nfail = int(fails.group(1)) if fails else -1
failed_names = re.findall(r"^\s*✖ (.+?) \(\d", body, re.M)
killed = rc != 0 and nfail > 0
if expect and killed:
killed = any(n == expect for n in failed_names)
verdict = "KILLED" if killed else "SURVIVED"
line = f"{mid} {verdict} rc={rc} tests={ran.group(1) if ran else '?'} fail={nfail} {time.time()-t0:.0f}s failed={sorted(set(failed_names))[:6]}"
print(line, flush=True)
results.append((mid, verdict))
finally:
for f, (p, s) in pristine.items():
shutil.copy2(p, f"{WT}/{f}")
if sha(f"{WT}/{f}") != s:
print(f"RESTORE FAILED {f}", flush=True)
surv = [m for m, v in results if v != "KILLED"]
print(f"DONE {len(results)} mutants, {len(results)-len(surv)} killed, survived: {surv}", flush=True)
sys.exit(1 if surv else 0)
main()
agents/darkwing/work/queue-162/r1/mutants.summary
+48
View File
@@ -0,0 +1,48 @@
M1 KILLED rc=1 tests=1 fail=1 1s failed=['T1 adopt writes the pointer from the units once; a failing check refuses check-failed']
M2 KILLED rc=1 tests=2 fail=2 3s failed=['T1 adopt writes the pointer from the units once; a failing check refuses check-failed', "T2 activate installs exactly the pointer's pm, claude and Console flags"]
M3 KILLED rc=1 tests=1 fail=1 2s failed=["T2 activate installs exactly the pointer's pm, claude and Console flags"]
M4 KILLED rc=1 tests=4 fail=2 4s failed=['T3 a hand-edited unit refuses unit-flags-drift, names the difference and changes nothing', 'console']
M5 KILLED rc=1 tests=9 fail=3 7s failed=['T4 each item 3 refusal changes nothing and logs one line with its code', 'instance-busy (another business active)', 'instance-busy (this business activating)']
M6 KILLED rc=1 tests=9 fail=2 67s failed=['T4 each item 3 refusal changes nothing and logs one line with its code', 'drop-in-override']
M7 KILLED rc=1 tests=9 fail=2 6s failed=['T4 each item 3 refusal changes nothing and logs one line with its code', 'copy-unreachable']
M8 KILLED rc=1 tests=9 fail=9 5s failed=['T4 each item 3 refusal changes nothing and logs one line with its code', 'already-active', 'copy-missing', 'copy-schema-unknown', 'copy-unreachable', 'drop-in-override']
M9 KILLED rc=1 tests=1 fail=1 2s failed=['T5 an activate with no migration stops the Console then the bus, installs, starts the bus then the Console']
M10 KILLED rc=1 tests=1 fail=1 2s failed=['T5 an activate with no migration stops the Console then the bus, installs, starts the bus then the Console']
M11 KILLED rc=1 tests=1 fail=1 2s failed=['T6 a migration without --migrate rolls back with migration-needs-flag']
M12 KILLED rc=1 tests=1 fail=1 2s failed=['T7 --migrate takes the backup before the migration runs']
M13 KILLED rc=1 tests=1 fail=1 2s failed=['T8 a failed migration restores the backup and keeps the store it replaced']
M14 KILLED rc=1 tests=1 fail=1 2s failed=["T9 a store outside the target's chain rolls back with store-newer"]
M15 KILLED rc=1 tests=6 fail=2 25s failed=['T10 each gate check failing alone rolls back and puts every file back', 'units-inactive']
M16 KILLED rc=1 tests=6 fail=2 21s failed=['T10 each gate check failing alone rolls back and puts every file back', 'bus-status']
M17 KILLED rc=1 tests=6 fail=2 21s failed=['T10 each gate check failing alone rolls back and puts every file back', 'healthz']
M18 KILLED rc=1 tests=6 fail=2 21s failed=['T10 each gate check failing alone rolls back and puts every file back', 'copy-check']
M19 KILLED rc=1 tests=6 fail=2 22s failed=['T10 each gate check failing alone rolls back and puts every file back', 'pm-launch-unconfirmed']
M20 KILLED rc=1 tests=6 fail=6 25s failed=['T10 each gate check failing alone rolls back and puts every file back', 'bus-status', 'copy-check', 'healthz', 'pm-launch-unconfirmed', 'units-inactive']
M21 KILLED rc=1 tests=6 fail=6 39s failed=['T10 each gate check failing alone rolls back and puts every file back', 'bus-status', 'copy-check', 'healthz', 'pm-launch-unconfirmed', 'units-inactive']
M22 KILLED rc=1 tests=1 fail=1 12s failed=['T11 a rollback whose gate fails too: rollback-failed, exit 2, both units stopped, no retry']
M23 KILLED rc=1 tests=1 fail=1 2s failed=['T12 rollback takes the previous copy from the log']
M24 KILLED rc=1 tests=1 fail=1 3s failed=['T13 a rollback across a migration needs --restore, then restores and goes live']
M25 KILLED rc=1 tests=1 fail=1 3s failed=['T14 events written after the backup need --events N with the right N']
M26 KILLED rc=1 tests=1 fail=1 1s failed=['T15 --plan names the migration and the restore and changes nothing']
M27 KILLED rc=1 tests=2 fail=2 5s failed=['T13 a rollback across a migration needs --restore, then restores and goes live', 'T8 a failed migration restores the backup and keeps the store it replaced']
M28 KILLED rc=1 tests=9 fail=2 6s failed=['T4 each item 3 refusal changes nothing and logs one line with its code', 'copy-unreachable']
M29 KILLED rc=1 tests=9 fail=2 6s failed=['T4 each item 3 refusal changes nothing and logs one line with its code', 'instance-busy (this business activating)']
M30 KILLED rc=1 tests=10 fail=3 44s failed=['T10b the PM check needs exactly one launched line for a live run of this start', 'a human launch with no journal line', 'no launched line']
M31 KILLED rc=1 tests=1 fail=1 6s failed=['T14b a rollback --restore whose gate fails stops both units and skips item 6']
M32 KILLED rc=1 tests=1 fail=1 1s failed=['T19 a started line with no final line: status shows it and every changing verb refuses']
M33 KILLED rc=1 tests=1 fail=1 1s failed=['T20 a held run lock refuses run-locked without a log line; status and --plan still run']
M34 KILLED rc=1 tests=1 fail=1 1s failed=['T21 a bus unit with no MOSAIC_CLAUDE_BIN: adopt and activate refuse no-claude-pin']
M35 KILLED rc=1 tests=10 fail=10 183s failed=['T10b the PM check needs exactly one launched line for a live run of this start', 'a changed word', 'a human launch with no journal line', 'no launched line', 'one well-formed line and a live run pass', 'run for another business']
M36 KILLED rc=1 tests=10 fail=2 39s failed=['T10b the PM check needs exactly one launched line for a live run of this start', 'two lines']
M37 KILLED rc=1 tests=10 fail=2 42s failed=['T10b the PM check needs exactly one launched line for a live run of this start', 'run started before the start']
M38 KILLED rc=1 tests=4 fail=3 5s failed=['T22 the stopped store read: a -wal or -shm refuses store-unclean and is left alone; a clean read adds no file', 'bus.sqlite-shm', 'bus.sqlite-wal']
M39 KILLED rc=1 tests=4 fail=2 5s failed=['T22 the stopped store read: a -wal or -shm refuses store-unclean and is left alone; a clean read adds no file', 'clean']
M40 KILLED rc=1 tests=1 fail=1 6s failed=['T14c a rollback without --restore whose gate fails runs item 6']
M41 KILLED rc=1 tests=3 fail=3 1s failed=['T23 --plan reads the store without changing a file, stopped or held', 'held store', 'stopped store']
M42 KILLED rc=1 tests=4 fail=4 2s failed=['T20 a held run lock refuses run-locked without a log line; status and --plan still run', 'T23 --plan reads the store without changing a file, stopped or held', 'held store', 'stopped store']
M43 KILLED rc=1 tests=3 fail=2 1s failed=['T23 --plan reads the store without changing a file, stopped or held', 'stopped store']
M44 KILLED rc=1 tests=1 fail=1 1s failed=['T24 business-mismatch and capture-failed refuse with exit 3; a short id is a usage error']
M45a KILLED rc=1 tests=1 fail=1 1s failed=['T25 --plan refuses copies-unrelated when neither copy is an ancestor of the other']
M45b KILLED rc=1 tests=1 fail=1 1s failed=['T26 --plan lists the commits an activate adds, after the migration lines']
M46 KILLED rc=1 tests=4 fail=4 2s failed=['T23 --plan reads the store without changing a file, stopped or held', 'T26 --plan lists the commits an activate adds, after the migration lines', 'held store', 'stopped store']
DONE 47 mutants, 47 killed, survived: []
agents/darkwing/work/queue-162/r1/plan-8e7.sh
Executable
+45
View File
@@ -0,0 +1,45 @@
#!/usr/bin/env bash
# Row 162, sage-0837, rerun on sage-0871 (mos-0810 D1-D2, target 8e7ea89a1): activate --plan against a scratch data root with a hand-written pointer.
# Everything written lives under $P. The only live reads were made before this script, by hand, the second at the start of the sage-0871 slot:
# `systemctl --user show [email protected] -p WorkingDirectory` (19:32:16Z).
set -euo pipefail
LIVE=52f35e7a0933e9fa845f59860a5de1709f9e5756
TO="$(git -C /mnt/storage/src/mosaic-stack rev-parse --verify 8e7ea89a1^{commit})"
R="$HOME/darkwing-scratch/r162"
P="$R/plan-8e7"
WT="$R/wt"
export TMPDIR="$R/tmp"
[ ! -e "$P" ] || { echo "plan.sh: $P exists; remove it first" >&2; exit 1; }
mkdir -p "$P/cfg" "$P/data/state" "$P/data/bus" "$P/copies" "$P/bin"
step() { echo "$(date -u +%FT%TZ) $*" >> "$P/steps.log"; }
cat > "$P/cfg/config.json" <<EOF
{"configVersion":1,"environment":"development","dataRoot":"$P/data","execution":{"backend":"docker","provider":"zai","model":"glm-5.3-flash","adapter":"mock"}}
EOF
# The schema adopt would record for the live copy: SCHEMA_VERSION in its store.mjs, read from git.
SCHEMA="$(git -C /mnt/storage/src/mosaic-stack show "$LIVE:packages/bus/src/store.mjs" | sed -n "s/^export const SCHEMA_VERSION = '\([A-Za-z0-9]*\)';\$/\1/p")"
node -e '
const [copy,schema]=process.argv.slice(1);
process.stdout.write(JSON.stringify({copy,business:"mosaic-stack",pm:false,claude:"/hand-written/no-claude-pin-read",console:[],schema,activation:"hand-written",at:new Date().toISOString().replace(/\.\d+Z$/,"Z")})+"\n");' -- "$LIVE" "$SCHEMA" > "$P/data/state/stack-active.json"
step "pointer: hand-written, copy $LIVE, schema $SCHEMA"
step "copy: (cd $WT && scripts/stack-copy.sh copy $TO --copies $P/copies --allow-worktree)"
(cd "$WT" && scripts/stack-copy.sh copy "$TO" --copies "$P/copies" --allow-worktree) > "$P/copy.out" 2>&1 || { step "copy failed"; tail -20 "$P/copy.out"; exit 1; }
step "copy: $(tail -1 "$P/copy.out")"
{
echo "# SCRATCH ROOT, NOT LIVE. activate --plan run by darkwing at $(date -u +%FT%TZ) in the row 162 slot."
echo "# Data root: $P/data (config $P/cfg/config.json). There is no store file, so the store schema"
echo "# comes from the pointer. The pointer is HAND-WRITTEN: copy $LIVE is the live bus unit's"
echo "# WorkingDirectory; schema $SCHEMA is that copy's SCHEMA_VERSION read with git show. pm, claude"
echo "# and console are placeholders, so the bus and Console lines below are not the live flags."
echo "# Copies: $P/copies (a scratch copy of $TO). The real --plan against the live pointer runs"
echo "# again in Mos's window before any adopt."
echo "# Command: MOSAIC_CONFIG=$P/cfg/config.json scripts/stack-release.sh activate $TO mosaic-stack --plan --copies $P/copies --bin $P/bin"
set +e
(cd "$WT" && MOSAIC_CONFIG="$P/cfg/config.json" scripts/stack-release.sh activate "$TO" mosaic-stack --plan --copies "$P/copies" --bin "$P/bin" 2>&1)
echo "# exit $?"
} > "$R/out/scratch-plan-8e7ea89a1.txt"
step "plan: written to $R/out/scratch-plan.txt"
step "files under the data root after the plan: $(cd "$P/data" && find . | sort | tr '\n' ' ')"
chmod -R u+w "$P/copies/$TO"
rm -rf "$P/copies/$TO"
step "removed: chmod -R u+w $P/copies/$TO && rm -rf $P/copies/$TO; left: $(ls -A "$P/copies" | wc -l) entries"
cat "$P/steps.log"
agents/darkwing/work/queue-162/r1/stack-copy-test.log
+20
View File
@@ -0,0 +1,20 @@
✔ shared copy: npm ci, pushed reachability, readonly/existing copy, webui delegation (423.822635ms)
✔ wrapper preserves Q81 lease and marker, rejects foreign/symlink; check uses effective roots (443.799606ms)
✔ bus unit copy/pm pins, install refuses all busy states and failed reads, drop-in refusal (417.363696ms)
✔ missing package manifest refuses before npm; npm failure removes the unpublished temp copy (313.097858ms)
✔ real pinned install loads bus and Console; same-copy mosaic talk passes proof, other trees refuse (3467.083425ms)
✔ bus status reads effective copy/pm/drop-ins without writes; check refuses an overridden ExecStart (494.000458ms)
✔ drop-ins in either override field and malformed metadata refuse; uninstall also checks --no-reload (196.85006ms)
✔ install never shows a template name on systemd 261; template drop-ins still refuse (188.126705ms)
✔ unlisted instance drop-in is found in another unit lookup directory; path discovery errors refuse (172.544517ms)
✔ inherited mosaic-.service.d override refuses without changing the unit (193.256243ms)
✔ inherited service.d override refuses without changing the unit (196.067886ms)
✔ inherited drop-in unreadable directory or file refuses before writing (185.272649ms)
ℹ tests 12
ℹ suites 0
ℹ pass 12
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 6755.065602
agents/darkwing/work/queue-162/r1/stack-release-test.log
+72
View File
@@ -0,0 +1,72 @@
✔ T1 adopt writes the pointer from the units once; a failing check refuses check-failed (1073.366997ms)
✔ T2 activate installs exactly the pointer's pm, claude and Console flags (1516.662712ms)
▶ T3 a hand-edited unit refuses unit-flags-drift, names the difference and changes nothing
✔ pm (718.027528ms)
✔ claude (742.624633ms)
✔ console (718.380886ms)
✔ T3 a hand-edited unit refuses unit-flags-drift, names the difference and changes nothing (2179.878901ms)
▶ T4 each item 3 refusal changes nothing and logs one line with its code
✔ copy-missing (634.789281ms)
✔ copy-unreachable (640.551556ms)
✔ copy-schema-unknown (603.6274ms)
✔ already-active (721.483462ms)
✔ instance-busy (another business active) (639.947543ms)
✔ instance-busy (this business activating) (638.806649ms)
✔ drop-in-override (665.313306ms)
✔ no-pointer (347.822109ms)
✔ T4 each item 3 refusal changes nothing and logs one line with its code (4893.830749ms)
✔ T5 an activate with no migration stops the Console then the bus, installs, starts the bus then the Console (1516.133038ms)
✔ T6 a migration without --migrate rolls back with migration-needs-flag (1499.97179ms)
✔ T7 --migrate takes the backup before the migration runs (1611.488791ms)
✔ T8 a failed migration restores the backup and keeps the store it replaced (1693.972865ms)
✔ T9 a store outside the target's chain rolls back with store-newer (1624.299305ms)
▶ T10 each gate check failing alone rolls back and puts every file back
✔ units-inactive (4846.794113ms)
✔ bus-status (5224.664321ms)
✔ healthz (4427.925603ms)
✔ copy-check (4527.114561ms)
✔ pm-launch-unconfirmed (4676.65493ms)
✔ T10 each gate check failing alone rolls back and puts every file back (23704.096449ms)
▶ T10b the PM check needs exactly one launched line for a live run of this start
✔ no launched line (5937.72793ms)
✔ two lines (5852.839829ms)
✔ a changed word (4747.354719ms)
✔ run not in launches list (5206.082662ms)
✔ run started before the start (5014.009245ms)
✔ a human launch with no journal line (4636.092869ms)
✔ run for another business (4860.736291ms)
✔ run not live (4726.01952ms)
✔ one well-formed line and a live run pass (1585.348096ms)
✔ T10b the PM check needs exactly one launched line for a live run of this start (42567.84089ms)
✔ T11 a rollback whose gate fails too: rollback-failed, exit 2, both units stopped, no retry (6648.970375ms)
✔ T12 rollback takes the previous copy from the log (2541.277145ms)
✔ T13 a rollback across a migration needs --restore, then restores and goes live (3589.263326ms)
✔ T14 events written after the backup need --events N with the right N (3342.024838ms)
✔ T14b a rollback --restore whose gate fails stops both units and skips item 6 (6292.838156ms)
✔ T14c a rollback without --restore whose gate fails runs item 6 (6446.849123ms)
✔ T15 --plan names the migration and the restore and changes nothing (1895.703147ms)
✔ T26 --plan lists the commits an activate adds, after the migration lines (855.079889ms)
✔ T27 --plan names each migration step, or says the steps cannot be read (654.135105ms)
✔ T25 --plan refuses copies-unrelated when neither copy is an ancestor of the other (363.81159ms)
▶ T23 --plan reads the store without changing a file, stopped or held
✔ stopped store (466.507341ms)
✔ held store (449.905848ms)
✔ T23 --plan reads the store without changing a file, stopped or held (916.922098ms)
✔ T24 business-mismatch and capture-failed refuse with exit 3; a short id is a usage error (1060.156137ms)
✔ T16 status prints the pointer, the log, the check, bus status and drift, and writes nothing (735.52147ms)
✔ T19 a started line with no final line: status shows it and every changing verb refuses (974.128039ms)
✔ T20 a held run lock refuses run-locked without a log line; status and --plan still run (954.126012ms)
✔ T21 a bus unit with no MOSAIC_CLAUDE_BIN: adopt and activate refuse no-claude-pin (1266.249479ms)
▶ T22 the stopped store read: a -wal or -shm refuses store-unclean and is left alone; a clean read adds no file
✔ bus.sqlite-wal (1710.782106ms)
✔ bus.sqlite-shm (1859.932095ms)
✔ clean (1574.266875ms)
✔ T22 the stopped store read: a -wal or -shm refuses store-unclean and is left alone; a clean read adds no file (5145.708424ms)
ℹ tests 58
ℹ suites 0
ℹ pass 58
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 128039.528917
agents/darkwing/work/queue-162/r2/build.patch
+1706
View File
File diff suppressed because it is too large Load Diff
agents/darkwing/work/queue-162/r2/candidate-manifest.sha256
+5
View File
@@ -0,0 +1,5 @@
a97b359c89cdcc69c077518f93b8f5fa67b872722910803cf0d5c3da1789e284 docs/TOOLS.md
ee303cf4ad5fde793bdd248d7bc60503e83083cde17535dab2d539cb4d1dab74 packages/cli/README.md
73cdd6c2e2eceffa223d25c1af5f549c369d3cb68d906613ddd6f08c05db1bb0 packages/cli/tests/stack-release.test.mjs
19473e30b35c19cf2167fbf0d7aa0d3ed46562d50e44b738bb611d95cd123254 scripts/stack-copy.sh
6a2c828fefef0f467a1bea7c51f927790ee8990d58cfd26372c521ac64c53d7e scripts/stack-release.sh
agents/darkwing/work/queue-162/scratch-plan-8e7ea89a1-steps.log
+6
View File
@@ -0,0 +1,6 @@
2026-10-18T19:51:56Z pointer: hand-written, copy 52f35e7a0933e9fa845f59860a5de1709f9e5756, schema 3f
2026-10-18T19:51:56Z copy: (cd /home/jwoltje/darkwing-scratch/r162/wt && scripts/stack-copy.sh copy 8e7ea89a1e86447d26ec335e4b138acfd5130cbc --copies /home/jwoltje/darkwing-scratch/r162/plan-8e7/copies --allow-worktree)
2026-10-18T19:51:57Z copy: copied: /home/jwoltje/darkwing-scratch/r162/plan-8e7/copies/8e7ea89a1e86447d26ec335e4b138acfd5130cbc
2026-10-18T19:51:57Z plan: written to /home/jwoltje/darkwing-scratch/r162/out/scratch-plan.txt
2026-10-18T19:51:57Z files under the data root after the plan: . ./bus ./state ./state/stack-active.json
2026-10-18T19:51:58Z removed: chmod -R u+w /home/jwoltje/darkwing-scratch/r162/plan-8e7/copies/8e7ea89a1e86447d26ec335e4b138acfd5130cbc && rm -rf /home/jwoltje/darkwing-scratch/r162/plan-8e7/copies/8e7ea89a1e86447d26ec335e4b138acfd5130cbc; left: 0 entries
agents/darkwing/work/queue-162/scratch-plan-8e7ea89a1.txt
+114
View File
@@ -0,0 +1,114 @@
# SCRATCH ROOT, NOT LIVE. activate --plan run by darkwing at 2026-10-18T19:51:57Z in the row 162 slot.
# Data root: /home/jwoltje/darkwing-scratch/r162/plan-8e7/data (config /home/jwoltje/darkwing-scratch/r162/plan-8e7/cfg/config.json). There is no store file, so the store schema
# comes from the pointer. The pointer is HAND-WRITTEN: copy 52f35e7a0933e9fa845f59860a5de1709f9e5756 is the live bus unit's
# WorkingDirectory; schema 3f is that copy's SCHEMA_VERSION read with git show. pm, claude
# and console are placeholders, so the bus and Console lines below are not the live flags.
# Copies: /home/jwoltje/darkwing-scratch/r162/plan-8e7/copies (a scratch copy of 8e7ea89a1e86447d26ec335e4b138acfd5130cbc). The real --plan against the live pointer runs
# again in Mos's window before any adopt.
# Command: MOSAIC_CONFIG=/home/jwoltje/darkwing-scratch/r162/plan-8e7/cfg/config.json scripts/stack-release.sh activate 8e7ea89a1e86447d26ec335e4b138acfd5130cbc mosaic-stack --plan --copies /home/jwoltje/darkwing-scratch/r162/plan-8e7/copies --bin /home/jwoltje/darkwing-scratch/r162/plan-8e7/bin
plan: activate for mosaic-stack
from: 52f35e7a0933e9fa845f59860a5de1709f9e5756
to: 8e7ea89a1e86447d26ec335e4b138acfd5130cbc
store schema: 3f (from the pointer; there is no store file)
target schema: 3h
migration: 3f to 3h, after a backup of the stopped store
migration steps: 3f to 3g, 3g to 3h
needs: --migrate (not given; the run would roll back with migration-needs-flag)
needs: Jason present for this run (decision 96 condition 2)
commits: the run adds 92 (newest first, at most 200):
8e7ea89a1 sessions: Filbert row 129 r2 review (approve, comment 28950)
19fe85c3e queue: revs 1757-1790, row 124 done (Dewey r2 approve 1757), rows 125-127/129 re-pinned, 125 and 156 to dewey, 164 to rocko, row 149 done (Filbert r4 1771), 129 r1 changes (1780) and r2 in review (1787-1790), 154 phase 2 in review (Researcher 1782-1784, request 1785-1786)
81109fdd8 logs: Rocko row 129 r2 verification and row 164 accepted design picks (own lines)
c202d57a1 docs: Rocko channel design, cut-over and own log lines (sage-0766/0769)
537025145 sessions: Filbert row 129 r1 review (changes, comment 28933)
d2df4d934 logs: Rocko row 164 design handoff (own lines through temporary index)
c2f16240c docs: row 156 owner dewey, after row 125 (landing run brief)
03bbd2c98 roles, harness: security-reviewer and merge-gate roles, proactive contracts (row 149, #1643)
c682d5f7a docs: row 164 owner rocko, design packet before the claim (channel kind brief)
806ab89df sessions: Filbert row 149 r4 review (approve, comment 28929)
23f86feda logs: Rocko row 129 build, verification and review handoff (own lines via temporary index)
0f715a054 docs: lead decision 150, limit-check interface 2 with report (row 125, dewey-0593)
25eb9e976 sessions: Sage row 124 landing, rows 125-126 sequencing, row 162 rulings
5b203b614 docs: row 125 to dewey, row 126 after step 8 (PRD 1.2 build brief)
70504d23a bus, cli, webui: token accounting per run, schema 3h usage_records (row 124, #1618)
d52327229 sessions: Dewey row 124 round 2 review, approve (comment 28923, rev 1757)
7a13129be queue: revs 1731-1756, row 124 round 1 changes and round 2 in review (Rocko, Dewey 1738), row 129 claimed (1737), rows 157-161 re-pinned for the I4 checks, D1/D2 re-pinned, row 162 briefed to darkwing, row 164 channel kind added (decision 149)
cbe071125 logs: Rocko row 124 round 2 (own lines via temporary index)
810d1a191 sessions: Sage mos-0716 answers, D1 decision 96 conditions, decision 149 and row 164
c742ecec2 docs: lead decision 149 and the channel kind brief, after row 129 (Q156 item 4, Mos mos-0726, #1661)
cd9c071ae plans: D1 owner is darkwing
d503552ea plans: D1 brief carries decision 96's four conditions and records the claude pin and Console flags in the pointer (Mos mos-0716, Darkwing survey)
23533625d sessions: Dewey row 124 round 1 review, changes (comment 28920, rev 1738)
e51d8f204 plans: row 159 I4 tests add the container-address and network-membership checks (Mos, mos-0716)
5e0437659 logs: Rocko row 129 design and row 124 count correction (own lines)
1f3208d4e logs: Rocko row 124 build and verification (own lines via temp index)
640f4cef5 sessions: Sage row 155 ruling, D1/D2 rows 162-163, row 154 phase 1 to Mos, queue revs 1722-1730
6a2409f21 queue: revs 1722-1730, row 134 after 155, row 149 round 4 in review (Dewey 1723-1724), D1 and D2 rows 162-163 with P19 notes, row 157 after 163, row 154 brief re-pin
2b5c76eeb plans: row 154 brief owns two serve.mjs static routes and round-trips the goals review (Researcher phase 1)
0df05f2c5 plans: D1 and D2 brief, health-gated stack activate and rollback and the scratch-root rehearsal (#1659, #1660)
e27053867 sessions: Dewey row 149 round 4 in review (revs 1721-1724, comment 28916)
7dfda5182 sessions: Sage P19 334-335, queue revs 1713-1721, row 156 brief wake gap, row 149 round 4
9cc0ba0e1 plans: row 156 brief takes Filbert's three edits (task ref in landHolders, land and landing from one read, decision-consumed in flight)
957fbfd41 plans: row 156 brief wakes the landing run with a landing turn (Filbert, comment 28913)
a132b4605 queue: revisions 1713-1720 (rows 124, 154, 155 briefed and 124, 155 re-pinned by Sage; rocko 124 and darkwing 155 claims; filbert row 149 round 3 changes)
a3c56343a sessions: Filbert row 149 r3 review (changes, comment 28913)
9098c33f7 sessions: Sage row 87 rounds 5-8 and done, row 149 rulings, row 156, I2-I6, row 128 landing
6f85a5518 queue: revisions 1675-1712 (row 87 rounds 5-8 and done, I2-I6 as rows 157-161, row 156 added, row 128 round 1 and done, row 149 round 3 in review)
ade628c8b cli: mosaic makes projects and threads and talks in a named thread (row 128, #1622)
7de8b7270 sessions: Dewey row 149 round 3 build, slot runs and review request (comment 28902)
77b1a6e8a plans: row 87 follow-ups on the start limit, and decisions 147 (B1) and 148 (B2) (#1581)
db1519637 sessions: Filbert row 87 r8 and row 128 r1 reviews (approve, comments 28894 and 28895)
bb0766826 plans: brief for the host starting the landing run once a merge decision is approved (#1651)
917ef55e1 plans: row 87 round 8 counts a stop's no-such-container as gone, bounds each wait per container and gives I6 the unit comment (#1581)
3fd0b387f sessions: Filbert row 87 r7 review (approve, comment 28889)
4c1fee85a plans: row 87 round 7 makes the leftover rule wait by id until each core container is gone and names the start order (#1581)
e1f13a54d sessions: Filbert row 87 r6 review (approve, comment 28883)
80ebe2afc plans: row 87 round 6 gives the store one real-path spelling, starts each core fork on its first line and states the name-conflict refusal (#1581)
b04014e59 sessions: Filbert row 87 r5 review (approve, comment 28881)
75b4af0bc sessions: Filbert row 149 r2 review (changes, comment 28878)
232e05449 plans: row 87 round 5 keys the core container on the store, sets the stop timeout to cover both close waits, and defines connected and host.pids (#1581)
bf7d09c6c sessions: Sage row 87 rounds 3-4, row 128 brief, queue sweep
c95c437cb queue: revs 1659-1674, row 87 rounds 3 and 4 (round 4 approved by Filbert), row 128 assigned to Researcher, row 149 round 2
125531d5f sessions: Filbert row 87 r4 review (approve, comment 28876)
e5dbcf626 plans: row 87 round 4 adds the core container's lifecycle and bus status liveness, answering Darkwing's and Filbert's round 3 reviews (#1581)
d5025c8f2 sessions: Dewey row 149 round 2, rebase onto ac55e84fe
0f3eb25ea plans: refreshed row 128 brief, the bus project and thread verbs and talk --thread (#1622)
4af594fea sessions: Filbert row 87 r3 review (changes, comment 28870)
3e91d528f sessions: Researcher row 148 lines of 2026-10-16 (steps 1 and 1b, rounds 1 to 4)
2d6681a4d sessions: Researcher row 143 build, rounds 1 and 2, landing and worktree removal
ac55e84fe plans: row 87 round 3 adds the host channel and keeps bindLaunch, answering Darkwing's and Filbert's round 2 reviews (#1581)
e1964c854 docs: BUILD-LOG for the row 143 landing; Sage SESSIONS line
6e7fdf7ff queue: revisions 1645-1658 (row 143 done)
0278b9e0b bus, harness, business, roles: agents review Jason's spec commits, and the assistant summarizes (row 143, #1637)
970e88bf2 sessions: Filbert row 87 r2 review (changes, comment 28864)
80a2e442b sessions: Dewey row 143 r2 review (approve, comment 28862)
47b030d9b plans: row 87 round 2 answers Filbert's and Darkwing's reviews of the containerized install design (#1581)
8fcd0887e plans: row 143 round 2 adds the chief-of-staff relay bullet, and acceptance 314 needs row 128's thread (#1637)
c550efa27 queue: revisions 1632-1644 (re-pins of 149 and 143; row 143 in-review r1 for Researcher with the request 28849 and Dewey's changes 28853; row 87 in-review r1 with Filbert's 28851 and Darkwing's 28852 changes; row 149 in-review r1 for Dewey, request 28855)
8c0fdc807 sessions: Dewey row 149 in-review and row 143 r1 review (changes, comment 28853)
510382ea3 sessions: Filbert row 87 r1 review (changes, comment 28851)
203481725 plans: containerized install design and the I2 to I6 briefs (#1581)
c0af7f248 plans: no verb adds a role instance, so the 149 brief and acceptance 314 name Jason's business-file edit (#1643)
8517e5f45 queue: revisions 1629-1631 (rows 143 and 149 in progress for Researcher and Dewey, row 87 in progress for Sage)
8d47d6184 docs: acceptance 314 step 8 reads task trails, so mosaic trail gets its argument (#1524)
4b1795a8c queue: revisions 1627-1628 (briefs for 149 and 143 re-pinned to 3a16dfac8, row 149 carries the raise_decision domain parameter)
3a16dfac8 plans: row 149 carries Dewey's row 139 finding 1, the raise_decision domain parameter (#1643)
b9c1dcd82 docs: re-pin plan for rows 148 and 141 with the bus 3f to 3g migration (mos-0638)
d4c3b2838 docs: BUILD-LOG for the row 141 landing; Sage SESSIONS line; row 141 r2 packet; P19 follow-ups 321-322; row 155 carry line
28eeef8ea queue: revisions 1625-1626 (row 141 done)
de352dfaf bus, cli, business, harness: messages addressed by agent name (row 141, #1635)
2a85b41da docs: Filbert SESSIONS line for the row 141 round 2 rebase check
7e40fb83b docs: Filbert SESSIONS line for the row 141 round 1 review
d49da64a9 queue: revision 1624 (row 148 done)
625afe15f docs: BUILD-LOG for the row 148 landing; Sage SESSIONS line; P19 follow-ups 318-320 script and record
d4ea7e5a6 webui, bus, project: a Console editor for the project's spec and docs (row 148, #1642)
a4779d2a5 queue: revisions 1588-1623 (rows 137, 141, 148, 150)
fc5a8b74d docs: Filbert SESSIONS line for the row 148 round 3 review
e1e8dda0b docs: Dewey SESSIONS line for the row 148 round 3 review
862af3079 docs: BUILD-LOG for the 52f35e7a0 re-pin and the row 148 round 3 send-back; Sage SESSIONS line
dc12dc9c0 sage: window plan for the 52f35e7a0 re-pin, rows 135, 139, 150 and 137
22dbc2b28 dewey: row 137 live smoke plan packet (#1631)
bus: scripts/bus-service.sh install --copy 8e7ea89a1e86447d26ec335e4b138acfd5130cbc --copies /home/jwoltje/darkwing-scratch/r162/plan-8e7/copies --claude /hand-written/no-claude-pin-read
Console: scripts/webui-service.sh install --copy 8e7ea89a1e86447d26ec335e4b138acfd5130cbc --copies /home/jwoltje/darkwing-scratch/r162/plan-8e7/copies
CLI: scripts/stack-copy.sh wrapper 8e7ea89a1e86447d26ec335e4b138acfd5130cbc --copies /home/jwoltje/darkwing-scratch/r162/plan-8e7/copies --bin /home/jwoltje/darkwing-scratch/r162/plan-8e7/bin
# exit 0
agents/darkwing/work/queue-162/scratch-plan-steps.log
+6
View File
@@ -0,0 +1,6 @@
2026-10-18T19:45:54Z pointer: hand-written, copy 52f35e7a0933e9fa845f59860a5de1709f9e5756, schema 3f
2026-10-18T19:45:54Z copy: (cd /home/jwoltje/darkwing-scratch/r162/wt && scripts/stack-copy.sh copy 606fafd43154ded993a503933681198e737987cd --copies /home/jwoltje/darkwing-scratch/r162/plan/copies --allow-worktree)
2026-10-18T19:45:56Z copy: copied: /home/jwoltje/darkwing-scratch/r162/plan/copies/606fafd43154ded993a503933681198e737987cd
2026-10-18T19:45:56Z plan: written to /home/jwoltje/darkwing-scratch/r162/out/scratch-plan.txt
2026-10-18T19:45:56Z files under the data root after the plan: . ./bus ./state ./state/stack-active.json
2026-10-18T19:45:56Z removed: chmod -R u+w /home/jwoltje/darkwing-scratch/r162/plan/copies/606fafd43154ded993a503933681198e737987cd && rm -rf /home/jwoltje/darkwing-scratch/r162/plan/copies/606fafd43154ded993a503933681198e737987cd; left: 0 entries
agents/darkwing/work/queue-162/scratch-plan.txt
+134
View File
@@ -0,0 +1,134 @@
# SCRATCH ROOT, NOT LIVE. activate --plan run by darkwing at 2026-10-18T19:45:56Z in the row 162 slot.
# Data root: /home/jwoltje/darkwing-scratch/r162/plan/data (config /home/jwoltje/darkwing-scratch/r162/plan/cfg/config.json). There is no store file, so the store schema
# comes from the pointer. The pointer is HAND-WRITTEN: copy 52f35e7a0933e9fa845f59860a5de1709f9e5756 is the live bus unit's
# WorkingDirectory; schema 3f is that copy's SCHEMA_VERSION read with git show. pm, claude
# and console are placeholders, so the bus and Console lines below are not the live flags.
# Copies: /home/jwoltje/darkwing-scratch/r162/plan/copies (a scratch copy of 606fafd43154ded993a503933681198e737987cd). The real --plan against the live pointer runs
# again in Mos's window before any adopt.
# Command: MOSAIC_CONFIG=/home/jwoltje/darkwing-scratch/r162/plan/cfg/config.json scripts/stack-release.sh activate 606fafd43154ded993a503933681198e737987cd mosaic-stack --plan --copies /home/jwoltje/darkwing-scratch/r162/plan/copies --bin /home/jwoltje/darkwing-scratch/r162/plan/bin
plan: activate for mosaic-stack
from: 52f35e7a0933e9fa845f59860a5de1709f9e5756
to: 606fafd43154ded993a503933681198e737987cd
store schema: 3f (from the pointer; there is no store file)
target schema: 3h
migration: 3f to 3h, after a backup of the stopped store
migration steps: 3f to 3g, 3g to 3h
needs: --migrate (not given; the run would roll back with migration-needs-flag)
needs: Jason present for this run (decision 96 condition 2)
commits: the run adds 112 (newest first, at most 200):
606fafd43 plans: row 165 brief, Jason's Q4 confirmation relayed (mos-0805), hold lifted
cfde62f9f agents: Rocko row 164 r2 packet (#1661)
008746b6e plugins: channel kind for the CLI and Console chat paths (row 164, #1661)
e5b10d275 sessions: Filbert row 154 r3 review, approve
27f01be2e queue: revs 1805-1827, row 164 r1 in review (Rocko 1806-1807, request by Sage 1808-1809), r1 changes (Dewey 1814), r2 in review (Rocko 1816, 1820-1821, request by Sage 1824-1825) and approve (Dewey 1826); 125 r2 in review (Dewey 1805, 1811-1812) and approve (Filbert 1815); 154 r2 changes (Filbert 1810, Dewey 1813), r3 in review (Researcher 1817-1819, request by Sage 1822-1823) and Dewey approve (1827)
1dbf45610 sessions: Filbert row 125 r2 review, approve
6f963fd65 sessions: Filbert row 154 r2 review, changes
ed92e2e1e logs: correct Rocko row 164 resume stamp from recorded clock
3d527871c docs: Rocko row 164 candidate packet and own verification logs
370c4d897 sessions: Researcher row 154 phase 1, phase 2 and round 2 lines, row 128 mutants line (their own lines)
5ec790f14 queue: revs 1798-1804, row 165 added queued (Sage 1798, P19 332), 154 r2 in review (Researcher 1799-1801, request posted by Sage 1803-1804), 125 r1 changes (Filbert 1802)
7a7d7442d sessions: Filbert row 125 r1 review, changes
ab9f23d9e docs: row brief for adding and changing role instances (P19 332, #1662); I6 brief records Jason's P19 333 ruling
d62c61fbd logs: Rocko row 164 focused pass and full-gate fixture boundary (own lines)
fe7eef8fb queue: revs 1791-1797, row 129 r2 approve (Filbert 1791) and done (1792), 164 claimed (Rocko 1793), 154 r1 changes (Filbert 1794, Dewey 1797), 125 r2 in review (Dewey 1795-1796)
20b5030b5 sessions: Dewey row 125 in-review and row 154 r1 security review
fdb1d4b34 logs: Rocko row 164 test preparation and landed reference check (own lines)
e79ba6c94 sessions: Sage row 129 landing (850484a7d, #1623 closed)
3cbfbaac4 sessions: Filbert row 154 r1 review (changes, comment 28954)
850484a7d webui: Console chat area over the bus's project and thread verbs (row 129, #1623)
8e7ea89a1 sessions: Filbert row 129 r2 review (approve, comment 28950)
19fe85c3e queue: revs 1757-1790, row 124 done (Dewey r2 approve 1757), rows 125-127/129 re-pinned, 125 and 156 to dewey, 164 to rocko, row 149 done (Filbert r4 1771), 129 r1 changes (1780) and r2 in review (1787-1790), 154 phase 2 in review (Researcher 1782-1784, request 1785-1786)
81109fdd8 logs: Rocko row 129 r2 verification and row 164 accepted design picks (own lines)
c202d57a1 docs: Rocko channel design, cut-over and own log lines (sage-0766/0769)
537025145 sessions: Filbert row 129 r1 review (changes, comment 28933)
d2df4d934 logs: Rocko row 164 design handoff (own lines through temporary index)
c2f16240c docs: row 156 owner dewey, after row 125 (landing run brief)
03bbd2c98 roles, harness: security-reviewer and merge-gate roles, proactive contracts (row 149, #1643)
c682d5f7a docs: row 164 owner rocko, design packet before the claim (channel kind brief)
806ab89df sessions: Filbert row 149 r4 review (approve, comment 28929)
23f86feda logs: Rocko row 129 build, verification and review handoff (own lines via temporary index)
0f715a054 docs: lead decision 150, limit-check interface 2 with report (row 125, dewey-0593)
25eb9e976 sessions: Sage row 124 landing, rows 125-126 sequencing, row 162 rulings
5b203b614 docs: row 125 to dewey, row 126 after step 8 (PRD 1.2 build brief)
70504d23a bus, cli, webui: token accounting per run, schema 3h usage_records (row 124, #1618)
d52327229 sessions: Dewey row 124 round 2 review, approve (comment 28923, rev 1757)
7a13129be queue: revs 1731-1756, row 124 round 1 changes and round 2 in review (Rocko, Dewey 1738), row 129 claimed (1737), rows 157-161 re-pinned for the I4 checks, D1/D2 re-pinned, row 162 briefed to darkwing, row 164 channel kind added (decision 149)
cbe071125 logs: Rocko row 124 round 2 (own lines via temporary index)
810d1a191 sessions: Sage mos-0716 answers, D1 decision 96 conditions, decision 149 and row 164
c742ecec2 docs: lead decision 149 and the channel kind brief, after row 129 (Q156 item 4, Mos mos-0726, #1661)
cd9c071ae plans: D1 owner is darkwing
d503552ea plans: D1 brief carries decision 96's four conditions and records the claude pin and Console flags in the pointer (Mos mos-0716, Darkwing survey)
23533625d sessions: Dewey row 124 round 1 review, changes (comment 28920, rev 1738)
e51d8f204 plans: row 159 I4 tests add the container-address and network-membership checks (Mos, mos-0716)
5e0437659 logs: Rocko row 129 design and row 124 count correction (own lines)
1f3208d4e logs: Rocko row 124 build and verification (own lines via temp index)
640f4cef5 sessions: Sage row 155 ruling, D1/D2 rows 162-163, row 154 phase 1 to Mos, queue revs 1722-1730
6a2409f21 queue: revs 1722-1730, row 134 after 155, row 149 round 4 in review (Dewey 1723-1724), D1 and D2 rows 162-163 with P19 notes, row 157 after 163, row 154 brief re-pin
2b5c76eeb plans: row 154 brief owns two serve.mjs static routes and round-trips the goals review (Researcher phase 1)
0df05f2c5 plans: D1 and D2 brief, health-gated stack activate and rollback and the scratch-root rehearsal (#1659, #1660)
e27053867 sessions: Dewey row 149 round 4 in review (revs 1721-1724, comment 28916)
7dfda5182 sessions: Sage P19 334-335, queue revs 1713-1721, row 156 brief wake gap, row 149 round 4
9cc0ba0e1 plans: row 156 brief takes Filbert's three edits (task ref in landHolders, land and landing from one read, decision-consumed in flight)
957fbfd41 plans: row 156 brief wakes the landing run with a landing turn (Filbert, comment 28913)
a132b4605 queue: revisions 1713-1720 (rows 124, 154, 155 briefed and 124, 155 re-pinned by Sage; rocko 124 and darkwing 155 claims; filbert row 149 round 3 changes)
a3c56343a sessions: Filbert row 149 r3 review (changes, comment 28913)
9098c33f7 sessions: Sage row 87 rounds 5-8 and done, row 149 rulings, row 156, I2-I6, row 128 landing
6f85a5518 queue: revisions 1675-1712 (row 87 rounds 5-8 and done, I2-I6 as rows 157-161, row 156 added, row 128 round 1 and done, row 149 round 3 in review)
ade628c8b cli: mosaic makes projects and threads and talks in a named thread (row 128, #1622)
7de8b7270 sessions: Dewey row 149 round 3 build, slot runs and review request (comment 28902)
77b1a6e8a plans: row 87 follow-ups on the start limit, and decisions 147 (B1) and 148 (B2) (#1581)
db1519637 sessions: Filbert row 87 r8 and row 128 r1 reviews (approve, comments 28894 and 28895)
bb0766826 plans: brief for the host starting the landing run once a merge decision is approved (#1651)
917ef55e1 plans: row 87 round 8 counts a stop's no-such-container as gone, bounds each wait per container and gives I6 the unit comment (#1581)
3fd0b387f sessions: Filbert row 87 r7 review (approve, comment 28889)
4c1fee85a plans: row 87 round 7 makes the leftover rule wait by id until each core container is gone and names the start order (#1581)
e1f13a54d sessions: Filbert row 87 r6 review (approve, comment 28883)
80ebe2afc plans: row 87 round 6 gives the store one real-path spelling, starts each core fork on its first line and states the name-conflict refusal (#1581)
b04014e59 sessions: Filbert row 87 r5 review (approve, comment 28881)
75b4af0bc sessions: Filbert row 149 r2 review (changes, comment 28878)
232e05449 plans: row 87 round 5 keys the core container on the store, sets the stop timeout to cover both close waits, and defines connected and host.pids (#1581)
bf7d09c6c sessions: Sage row 87 rounds 3-4, row 128 brief, queue sweep
c95c437cb queue: revs 1659-1674, row 87 rounds 3 and 4 (round 4 approved by Filbert), row 128 assigned to Researcher, row 149 round 2
125531d5f sessions: Filbert row 87 r4 review (approve, comment 28876)
e5dbcf626 plans: row 87 round 4 adds the core container's lifecycle and bus status liveness, answering Darkwing's and Filbert's round 3 reviews (#1581)
d5025c8f2 sessions: Dewey row 149 round 2, rebase onto ac55e84fe
0f3eb25ea plans: refreshed row 128 brief, the bus project and thread verbs and talk --thread (#1622)
4af594fea sessions: Filbert row 87 r3 review (changes, comment 28870)
3e91d528f sessions: Researcher row 148 lines of 2026-10-16 (steps 1 and 1b, rounds 1 to 4)
2d6681a4d sessions: Researcher row 143 build, rounds 1 and 2, landing and worktree removal
ac55e84fe plans: row 87 round 3 adds the host channel and keeps bindLaunch, answering Darkwing's and Filbert's round 2 reviews (#1581)
e1964c854 docs: BUILD-LOG for the row 143 landing; Sage SESSIONS line
6e7fdf7ff queue: revisions 1645-1658 (row 143 done)
0278b9e0b bus, harness, business, roles: agents review Jason's spec commits, and the assistant summarizes (row 143, #1637)
970e88bf2 sessions: Filbert row 87 r2 review (changes, comment 28864)
80a2e442b sessions: Dewey row 143 r2 review (approve, comment 28862)
47b030d9b plans: row 87 round 2 answers Filbert's and Darkwing's reviews of the containerized install design (#1581)
8fcd0887e plans: row 143 round 2 adds the chief-of-staff relay bullet, and acceptance 314 needs row 128's thread (#1637)
c550efa27 queue: revisions 1632-1644 (re-pins of 149 and 143; row 143 in-review r1 for Researcher with the request 28849 and Dewey's changes 28853; row 87 in-review r1 with Filbert's 28851 and Darkwing's 28852 changes; row 149 in-review r1 for Dewey, request 28855)
8c0fdc807 sessions: Dewey row 149 in-review and row 143 r1 review (changes, comment 28853)
510382ea3 sessions: Filbert row 87 r1 review (changes, comment 28851)
203481725 plans: containerized install design and the I2 to I6 briefs (#1581)
c0af7f248 plans: no verb adds a role instance, so the 149 brief and acceptance 314 name Jason's business-file edit (#1643)
8517e5f45 queue: revisions 1629-1631 (rows 143 and 149 in progress for Researcher and Dewey, row 87 in progress for Sage)
8d47d6184 docs: acceptance 314 step 8 reads task trails, so mosaic trail gets its argument (#1524)
4b1795a8c queue: revisions 1627-1628 (briefs for 149 and 143 re-pinned to 3a16dfac8, row 149 carries the raise_decision domain parameter)
3a16dfac8 plans: row 149 carries Dewey's row 139 finding 1, the raise_decision domain parameter (#1643)
b9c1dcd82 docs: re-pin plan for rows 148 and 141 with the bus 3f to 3g migration (mos-0638)
d4c3b2838 docs: BUILD-LOG for the row 141 landing; Sage SESSIONS line; row 141 r2 packet; P19 follow-ups 321-322; row 155 carry line
28eeef8ea queue: revisions 1625-1626 (row 141 done)
de352dfaf bus, cli, business, harness: messages addressed by agent name (row 141, #1635)
2a85b41da docs: Filbert SESSIONS line for the row 141 round 2 rebase check
7e40fb83b docs: Filbert SESSIONS line for the row 141 round 1 review
d49da64a9 queue: revision 1624 (row 148 done)
625afe15f docs: BUILD-LOG for the row 148 landing; Sage SESSIONS line; P19 follow-ups 318-320 script and record
d4ea7e5a6 webui, bus, project: a Console editor for the project's spec and docs (row 148, #1642)
a4779d2a5 queue: revisions 1588-1623 (rows 137, 141, 148, 150)
fc5a8b74d docs: Filbert SESSIONS line for the row 148 round 3 review
e1e8dda0b docs: Dewey SESSIONS line for the row 148 round 3 review
862af3079 docs: BUILD-LOG for the 52f35e7a0 re-pin and the row 148 round 3 send-back; Sage SESSIONS line
dc12dc9c0 sage: window plan for the 52f35e7a0 re-pin, rows 135, 139, 150 and 137
22dbc2b28 dewey: row 137 live smoke plan packet (#1631)
bus: scripts/bus-service.sh install --copy 606fafd43154ded993a503933681198e737987cd --copies /home/jwoltje/darkwing-scratch/r162/plan/copies --claude /hand-written/no-claude-pin-read
Console: scripts/webui-service.sh install --copy 606fafd43154ded993a503933681198e737987cd --copies /home/jwoltje/darkwing-scratch/r162/plan/copies
CLI: scripts/stack-copy.sh wrapper 606fafd43154ded993a503933681198e737987cd --copies /home/jwoltje/darkwing-scratch/r162/plan/copies --bin /home/jwoltje/darkwing-scratch/r162/plan/bin
# exit 0