865 lines
50 KiB
Markdown
865 lines
50 KiB
Markdown
# Row 162 (D1, #1659): stack activate and rollback, health-gated
|
|
|
|
Owner: Darkwing. Reviewer: Filbert. Gate owner: Sage.
|
|
Brief: `docs/plans/2026-10-18_stack-release.md`, section "## Stack activate
|
|
and rollback, health-gated (D1)", commit cd9c071ae, blob cfc468c9df88.
|
|
Base: df4d5702f for round 2 (origin/refactor after row 125 landed; see
|
|
"Candidate round 2"). Round 1 was on cfde62f9f (origin/refactor after row 164 landed). The draft
|
|
started on cd9c071ae and moved up with `git checkout --detach`, through
|
|
70504d23a (row 124) to cfde62f9f. None of the 162 paths is among row
|
|
164's 21; the only base change on them is row 129's appended "Console
|
|
Chat" section in `docs/TOOLS.md`, and the TOOLS.md patch applied cleanly.
|
|
The store schema on refactor is 3h. The live copy 52f35e7a0 is at 3f,
|
|
not 3g as this packet said before 2026-10-18T19:40Z: 3g came with row
|
|
141 (de352dfaf), which isn't live yet.
|
|
Worktree: `~/darkwing-scratch/r162/wt`.
|
|
|
|
Status: built and tested on Sage's slot (19:25:10Z to 19:46:14Z on
|
|
2026-10-18). `stack-release.test.mjs` 58/58 and `stack-copy.test.mjs`
|
|
12/12, each run singly. 47 mutants killed before the steps change, then
|
|
M41, M45b, M46 and M47 to M49 killed after it. The scratch-root plan for
|
|
Mos is in `scratch-plan.txt` next to this file.
|
|
|
|
## Files
|
|
|
|
| File | Change |
|
|
|---|---|
|
|
| `scripts/stack-release.sh` | new: `adopt`, `activate`, `rollback`, `status` |
|
|
| `scripts/stack-copy.sh` | a `reachable` function split out of `copy_commit`, and a guard so the file can be sourced for its helpers |
|
|
| `packages/cli/tests/stack-release.test.mjs` | new |
|
|
| `docs/TOOLS.md` | the "Pinned stack update" section only |
|
|
| `packages/cli/README.md` | the pinned-stack paragraph only |
|
|
| this packet | |
|
|
|
|
Nothing else changes. `bus-service.sh` and `webui-service.sh` are called
|
|
as they are.
|
|
|
|
## Shape of the script
|
|
|
|
```
|
|
scripts/stack-release.sh adopt BUSINESS
|
|
scripts/stack-release.sh activate ID BUSINESS [--pm] [--migrate] [--timeout S] [--plan]
|
|
scripts/stack-release.sh rollback BUSINESS [--restore ACTIVATION [--events N]] [--timeout S] [--plan]
|
|
scripts/stack-release.sh status BUSINESS
|
|
common options: --copies DIR, --bin DIR (passed to stack-copy.sh and the install scripts)
|
|
```
|
|
|
|
Exit codes: 0 ok; 1 the gate failed and the rollback inside the run passed;
|
|
2 `rollback-failed`, the stack is left stopped; 3 a refusal, nothing
|
|
changed; 4 usage. Usage errors write no log line. Every other outcome
|
|
writes exactly one final line.
|
|
|
|
`--pm` on `activate` must match the pointer's `pm`. The brief has
|
|
`activate` install "exactly the pointer's `--pm`", so the flag is a check
|
|
the operator states, not a way to change it. A mismatch refuses
|
|
`unit-flags-drift`.
|
|
|
|
The header comment states decision 96's four conditions (item 10) and
|
|
points at decisions 94 and 102 for who starts the Console.
|
|
|
|
### Where things come from
|
|
|
|
- Data root: `load_config` from `scripts/common.sh`, run from the canonical
|
|
checkout (`MOSAIC_CONFIG` is honoured, as everywhere). The bus resolves
|
|
its root the same way, so both see one root.
|
|
- The canonical checkout: the `REPO` of this script. `bus-service.sh`,
|
|
`webui-service.sh` and `stack-copy.sh` are this checkout's, as in the
|
|
window plans. `webui-service.sh install` already refuses outside the
|
|
canonical checkout.
|
|
- Unit names: `[email protected]` and `mosaic-webui.service`.
|
|
Unit directory: `${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user`, the
|
|
default both install scripts write to.
|
|
- `claude`: `MOSAIC_CLAUDE_BIN=` from `systemctl --user show
|
|
mosaic-bus@B.service -p Environment`.
|
|
- `pm`: a `--pm` token after `bus start B` in the bus unit's `ExecStart`
|
|
`argv[]`.
|
|
- `console`: the `argv[]` tokens after `serve` in the Console unit's
|
|
`ExecStart`, minus `--checkout X`. The install script adds `--checkout`
|
|
itself.
|
|
- The copy's `SCHEMA_VERSION` and chain: text read from the copy's
|
|
`packages/bus/src/store.mjs`. `export const SCHEMA_VERSION = 'X';` must
|
|
match exactly once, and the chain is the `from: 'X'` lines in file
|
|
order. The copy's code is never imported or run for this. A copy where
|
|
either read fails refuses `copy-schema-unknown`. The migration file
|
|
names would also work, but `3b.sql` is a base, not a step, so the
|
|
store.mjs lines are the cleaner source.
|
|
|
|
### Sharing with stack-copy.sh
|
|
|
|
`copy_commit`'s reachability loop moves into `reachable FULL`, and
|
|
`copy_commit` calls it. The dispatch at the bottom gets
|
|
`[ "${BASH_SOURCE[0]}" = "$0" ] || return 0`, so sourcing defines the
|
|
functions and runs nothing. `stack-release.sh` sources it inside a
|
|
subshell function, so stack-copy's `die`, `parse` and globals never mix
|
|
with its own:
|
|
|
|
```
|
|
copy_ok() ( source "$REPO/scripts/stack-copy.sh"; ID="$1"; COPIES="$COPIES_DIR"; copy_root >/dev/null && reachable "$ID" )
|
|
```
|
|
|
|
`check` and `wrapper` run as commands, as an operator would run them.
|
|
|
|
## Pointer and log (item 1)
|
|
|
|
`<dataRoot>/state/stack-active.json`, written to a temp file in `state/`
|
|
and renamed:
|
|
|
|
```json
|
|
{"copy":"<40 hex>","business":"mosaic-stack","pm":true,
|
|
"claude":"/home/u/.local/bin/claude","console":["--port","7332","--business","mosaic-stack"],
|
|
"schema":"3g","activation":"20261018T170000Z-52f35e7a0abc","at":"2026-10-18T17:00:00Z"}
|
|
```
|
|
|
|
`schema` is the copy's `SCHEMA_VERSION` at the time it went live. On
|
|
`adopt` that is the adopted copy's, since a running host proves the store
|
|
is at its copy's schema.
|
|
|
|
`<dataRoot>/state/stack-activation-log.jsonl`, appended with `>>`, one
|
|
JSON object per line:
|
|
|
|
```json
|
|
{"at":"...","op":"activate","from":"<copy>","to":"<copy>","backup":null,"result":"ok"}
|
|
{"at":"...","op":"activate","from":"...","to":"...","backup":null,"result":"refused","code":"unit-flags-drift"}
|
|
{"at":"...","op":"activate","from":"...","to":"...","backup":"<activation>","result":"gate-failed","code":"healthz","rollback":"ok"}
|
|
{"at":"...","op":"activate","from":"...","to":"...","backup":"<activation>","result":"rollback-failed","code":"bus-status","rollback":"failed","rollbackCode":"healthz"}
|
|
```
|
|
|
|
Brief item 4 says "each step is logged". Ruling 3: a run that gets past
|
|
the refusals writes `result: "started"` before step 1 and one final line
|
|
at the end. The per-step detail goes to stdout and to `steps.log` inside
|
|
the run's rollback set. `rollback` reads final lines only.
|
|
|
|
A `started` line with no final line after it for the same activation is
|
|
an interrupted run. `status` shows it, and every later `adopt`,
|
|
`activate` or `rollback` refuses `interrupted-run`, naming the
|
|
activation. The script has no verb to clear it. Clearing it is Jason's or
|
|
Mos's call (ruling 3).
|
|
|
|
Every `adopt`, `activate` and `rollback` takes `flock -n` on
|
|
`state/stack-release.lock` first and refuses `run-locked` when another
|
|
run holds it. `status` and `--plan` take no lock (ruling 6).
|
|
|
|
Activation id: `<UTC %Y%m%dT%H%M%SZ>-<first 12 of the target copy>`. The
|
|
rollback set directory is made with `mkdir -m 0700` and no `-p`, so a
|
|
clash fails before anything changes.
|
|
|
|
## adopt (item 2)
|
|
|
|
1. The run lock, else `run-locked`. No `interrupted-run` in the log.
|
|
2. No pointer, else `already-adopted`.
|
|
3. `stack-copy.sh check --business B` passes, else `check-failed` with
|
|
its output.
|
|
4. Read `claude`, `pm` and `console` from `systemctl show`. No
|
|
`MOSAIC_CLAUDE_BIN` refuses `no-claude-pin` (ruling 5).
|
|
5. Write the pointer, then log `ok`.
|
|
|
|
It stops, starts and installs nothing.
|
|
|
|
## activate (items 3 to 6)
|
|
|
|
### Refusals before anything changes (item 3)
|
|
|
|
The checks run in this order. Each one gives exit 3, one log line with its
|
|
code, and no other write:
|
|
|
|
1. `run-locked`, then `interrupted-run`. `run-locked` writes no log
|
|
line, because the lock holder may be writing the log.
|
|
2. `no-pointer`, then `business-mismatch`: the pointer names another
|
|
business. The brief names no code for it; the pointer holds one
|
|
business, so a run for another one would install units the pointer
|
|
doesn't describe.
|
|
3. `copy-missing` (`copy_root` fails) or `copy-unreachable` (`reachable`
|
|
fails). The brief names no codes for these two, so these are mine.
|
|
4. `copy-schema-unknown`, from the store.mjs read above.
|
|
5. `already-active`: ID is the pointer's copy.
|
|
6. `instance-busy` (ruling 1): another business's `mosaic-bus@` instance
|
|
is in any state but inactive or failed, since the copy and wrapper
|
|
switch reach it too; or this business's instance is activating,
|
|
deactivating or reloading. The message names the instance and its
|
|
state. The read is `install_guard`'s `list-units` call. A literal
|
|
reading (refuse any running instance) would refuse every activate,
|
|
because the bus is running when activate starts.
|
|
7. `drop-in-override`: the `install_guard` scan of
|
|
`systemd-analyze --user unit-paths`, naming the file.
|
|
8. `no-claude-pin` (ruling 5): the effective bus unit has no
|
|
`MOSAIC_CLAUDE_BIN`.
|
|
9. `unit-flags-drift`: effective `pm`, `claude` or `console` differ from
|
|
the pointer's, or `--pm` differs from the pointer's `pm`. Prints
|
|
`pointer:` and `unit:` lines for each field that differs.
|
|
|
|
### The run (item 4)
|
|
|
|
1. Capture to `state/stack-rollback/<activation>/` (0700): the two unit
|
|
files from the unit directory, the drop-in directories
|
|
`[email protected]`, `mosaic-bus@B.service.d` and
|
|
`mosaic-webui.service.d` under that directory when present, the
|
|
wrapper `$BIN/mosaic` with `stat -c %a` in `WRAPPER_MODE`, the
|
|
`systemctl show` output for both units (it includes `DropInPaths`), and
|
|
`SHA256SUMS` over the captured files. `cp -p` throughout. This is
|
|
`capture-rollback.sh`. The `started` line is already written, so a
|
|
failure here writes the final line `refused` with code
|
|
`capture-failed` and exits 3. Nothing has stopped, and only the
|
|
partial rollback set exists.
|
|
2. `systemctl --user stop mosaic-webui.service`, then the bus unit. Poll
|
|
`is-active` for both to read inactive or failed, then the stopped
|
|
check: no `<dataRoot>/bus/writer.lock`, and no `/proc/*/fd` link
|
|
resolving under `<dataRoot>/bus/`. Bounded by `--timeout`. A timeout
|
|
goes to item 6 with code `stop-timeout`.
|
|
3. Schema read: see "The store read" below. A `-wal` or `-shm` beside
|
|
the store goes to item 6 with `store-unclean`.
|
|
- The same as the copy's `SCHEMA_VERSION`: no backup.
|
|
- In the chain and older, no `--migrate`: `migration-needs-flag`,
|
|
item 6.
|
|
- Older with `--migrate`: back up, then `node
|
|
<copy>/packages/bus/src/migrate-schema.mjs <dataRoot>`. A non-zero
|
|
exit goes to item 6 with code `migrate-failed` and the store
|
|
restored.
|
|
- Newer, or not in the chain: `store-newer`, item 6.
|
|
- No `bus.sqlite` at all (a fresh root): no backup, the new host
|
|
creates the store.
|
|
4. `bus-service.sh install --copy ID --copies DIR [--pm] --claude PATH`,
|
|
then `webui-service.sh install --copy ID --copies DIR <console
|
|
flags>`, then `stack-copy.sh wrapper ID --copies DIR --bin DIR`. Any
|
|
non-zero exit goes to item 6 with code `install-bus`, `install-console`
|
|
or `wrapper`.
|
|
5. Start the bus, then the Console.
|
|
|
|
From step 2 on, `trap` on INT, TERM and HUP sends the run to item 6, once.
|
|
A Ctrl-C mid-window leaves the old stack started, not a stopped one. The
|
|
one exception is after a `rollback --restore` has swapped the store; see
|
|
ruling 4 under rollback.
|
|
|
|
Most of the script's functions run as `f || handler`. Bash turns `set -e`
|
|
off inside such a call, so every step in them that can fail carries its
|
|
own `|| return 1` or sits in an `&&` chain.
|
|
|
|
### The backup
|
|
|
|
`<dataRoot>/backups/stack/<activation>/`, as `backup.sh` does it: umask
|
|
077, `mkdir -m 700` for the directory and `bus/`, `cp -p` of `bus.sqlite`
|
|
(only that file: a `-wal` or `-shm` was already refused as
|
|
`store-unclean`), `SOURCE.sha256` taken before the copy and
|
|
checked against `SHA256SUMS` after it, `MODES.txt`, `LISTING.txt`,
|
|
`DATA_ROOT`, `STAMP` (UTC ISO with milliseconds, the same form as
|
|
`events.at`), `SCHEMA` (the version read in step 3) and `ROLLBACK_SET`.
|
|
`backup.sh`, `restore.sh` and `stopped-check.sh` here mean Sage's
|
|
committed window scripts in `agents/sage/work/window-plan/repin-de352dfaf/`.
|
|
`stack-release.sh` carries its own copies of their steps and doesn't call
|
|
them. The stopped check runs again after the copy. Any failure goes to item 6
|
|
before the migration runs.
|
|
|
|
### The store read
|
|
|
|
`node -e` with `node:sqlite`:
|
|
`` new DatabaseSync(`${pathToFileURL(file).href}?immutable=1`, { readOnly: true }) ``,
|
|
then `SELECT value FROM meta WHERE key='schema_version'`, then close.
|
|
|
|
The probe settled the open mode. It ran on Sage's slot (sage-0738),
|
|
16:02:53Z to 16:03:08Z, node v26.8.1, on a WAL-mode fixture store in
|
|
`~/darkwing-scratch/r162/probe/store/`, never the live store or a copy of
|
|
it. The fixture was rebuilt before each trial and closed cleanly, so only
|
|
`bus.sqlite` was present at the start of each one.
|
|
|
|
| Open | Read | Files after | Main file mtime, sha256 |
|
|
|---|---|---|---|
|
|
| `{ readOnly: true }`, store stopped | version and count right | `bus.sqlite-shm` and `bus.sqlite-wal` created and left behind | unchanged |
|
|
| `file:...?immutable=1` with `readOnly`, store stopped | version and count right | none created | unchanged |
|
|
| `{ readOnly: true }` while a WAL connection holds the store | sees the holder's row still in the WAL | the holder's `-wal` and `-shm`, nothing new | not compared |
|
|
|
|
A plain read-only open of a stopped store writes two files next to it, so
|
|
it fails Sage's condition, and immutable is the mode. Immutable tells
|
|
SQLite the file can't change and makes it skip the WAL entirely. That is
|
|
sound only when there is no WAL to skip. So the rule (Sage approved it in
|
|
sage-0739):
|
|
|
|
- Before every stopped-store read, `bus.sqlite-wal` and `bus.sqlite-shm`
|
|
must both be absent. A clean close checkpoints and removes them. One
|
|
left behind means the bus did not close the store cleanly, and an
|
|
immutable read could return a stale version or count. The run refuses
|
|
`store-unclean`, naming the file, and goes to item 6. `stack-release.sh`
|
|
never opens, copies or moves that store. Whatever happens to it next is
|
|
Jason's or Mos's call. There is no copy fallback.
|
|
- With both absent, the read uses immutable.
|
|
- An unreadable version goes to item 6 with code `schema-unreadable`.
|
|
|
|
The `--restore` count in item 7 step 5 is taken before the stop, from a
|
|
store the running bus holds. There the `-wal` and `-shm` belong to the
|
|
bus, the bus unit must be active, and the read is `{ readOnly: true }`,
|
|
which the probe shows adds no file. If the bus unit isn't active, the
|
|
count is a stopped-store read under the rule above.
|
|
|
|
The no-write test below catches a read that creates a file: it compares
|
|
the file set and the sums under `bus/` before and after.
|
|
|
|
### The health gate (item 5)
|
|
|
|
Polled every second until all pass or `--timeout` (default 60) runs out.
|
|
The failing check named at the end is the first one still failing:
|
|
|
|
| Check | Code |
|
|
|---|---|
|
|
| `systemctl --user is-active` for both units | `units-inactive` |
|
|
| `<copy>/scripts/mosaic bus status --json` exits 0, `host.business` is B, `host.live` is true, `host.pid` is the bus unit's `MainPID`, `boot.ok` is true, `boot.business` is B, `boot.at` is at or after the moment step 5 began | `bus-status` |
|
|
| `GET http://127.0.0.1:<port>/healthz` returns 200, port from the `--port` flag in `console`, else 7332 (the `webui-service.sh` default) | `healthz` |
|
|
| `stack-copy.sh check --business B` exits 0 | `copy-check` |
|
|
| with `pm`: exactly one `launched` line in the bus unit's journal since step 5, and its run is live in `<copy>/scripts/mosaic launches list --json`, for B, with `startedAt` at or after the moment step 5 began | `pm-launch-unconfirmed` |
|
|
|
|
Ruling 2 (sage-0736, refined in sage-0738): the check must match the PM
|
|
role without reading the business file (Jason's org file) and without
|
|
naming `launch.by`. The session entries carry `instance`, but nothing in
|
|
them or in `launches/<business>.jsonl` marks a launch as the PM's. The
|
|
`bus start --pm` path is the only code that knows. It prints
|
|
`launched <instance>: run <run>, <harness> <model>` to stdout
|
|
(`packages/cli/src/cli.mjs:503`, the only place that prints such a line),
|
|
and under the unit that stdout is the journal.
|
|
|
|
So the gate reads `journalctl --user -u mosaic-bus@B.service --since
|
|
@<epoch seconds of step 5> -o cat`. The `@` form is unambiguous. This
|
|
host's journalctl reads a bare time as local CDT, and in Q14 an empty
|
|
window passed for a clean start that way. Of the lines it returns, it
|
|
keeps those starting with `launched `. It passes only when:
|
|
|
|
- there is exactly one such line;
|
|
- it matches `^launched (\S+): run (\S+), `;
|
|
- that run is live in `launches list --json` for B, with `startedAt` at
|
|
or after step 5.
|
|
|
|
Zero lines, two or more, a line that doesn't match, or a run that isn't
|
|
live all give the one code `pm-launch-unconfirmed`, and the message says
|
|
which. A human `mosaic launch` in the same window writes no line to the
|
|
unit's journal, so it can't pass for the PM.
|
|
|
|
The review should know that this check depends on the wording of that
|
|
stdout line. A change to it in `cli.mjs` makes the gate fail closed, not
|
|
pass. A test pins the dependency: T10b's unparseable-line subtest feeds a
|
|
line that still starts with `launched ` but has a later word changed
|
|
(`job` for `run`) and expects the refusal.
|
|
|
|
Follow-up, not in D1 (Sage files it if wanted): a bus-side PM marker, for
|
|
example a `pm: true` field on the launch event the `--pm` path writes,
|
|
would let the gate drop the journal read and the stdout-format
|
|
dependency.
|
|
|
|
`MainPID` equals `host.pid` because `scripts/mosaic` ends in `exec node
|
|
...cli.mjs`. Every `bus start` that loads the business calls
|
|
`recordBootOk` (`packages/cli/src/cli.mjs:494`), which writes `boot.at` as
|
|
the current time, so a `boot.at` from before step 5 means the status
|
|
describes an earlier start.
|
|
|
|
The healthz fetch is `node -e fetch(...)` with a 5 s abort, as
|
|
`webui-service.sh status` does it.
|
|
|
|
On a pass: rename the pointer into place, log `ok`, exit 0.
|
|
|
|
### Rollback inside the run (item 6)
|
|
|
|
1. Stop the Console, then the bus, and wait as in step 2.
|
|
2. Put back the two unit files and the drop-in directories from the
|
|
rollback set (temp file and rename each), and the wrapper bytes and
|
|
mode. Check them against `SHA256SUMS`. `daemon-reload`.
|
|
3. If step 3 of this run took a backup, restore it as `restore.sh` does
|
|
(a `rollback --restore` backup is never restored here): check
|
|
`DATA_ROOT`, the stopped check and `sha256sum -c`; rename `bus` to
|
|
`bus.post-attempt-<UTC stamp>`; `mkdir -m 700 bus`; copy in; check the
|
|
sums and that the file set matches exactly. Nothing is deleted.
|
|
4. Start the bus, then the Console. Run the gate against the pointer's
|
|
copy.
|
|
5. Log `gate-failed` (or the step's code) with `rollback: ok`, exit 1. If
|
|
the rollback's gate fails: print both results, stop both units, log
|
|
`rollback-failed` with both codes, exit 2. No retry.
|
|
|
|
## rollback (item 7)
|
|
|
|
1. `no-pointer`.
|
|
2. Target: scan the log newest first for `result: "ok"` lines with
|
|
`op` adopt, activate or rollback, and take the first `to` that isn't
|
|
the pointer's copy. None found refuses `no-previous`. This is
|
|
`release.sh`'s rule, and it shares that rule's property: two rollbacks
|
|
in a row go back and forth between two copies.
|
|
3. The item 3 refusals apply to the target (copy, schema, busy,
|
|
drop-ins, flag drift).
|
|
4. `schema-differs`: the pointer's `schema` is not the target's
|
|
`SCHEMA_VERSION` and there's no `--restore`. Decided from the pointer
|
|
before anything stops.
|
|
5. With `--restore ACTIVATION`: `backups/stack/ACTIVATION/` must exist
|
|
with `SHA256SUMS`, a `DATA_ROOT` equal to this root, and a `SCHEMA`
|
|
equal to the target's `SCHEMA_VERSION`, else `restore-mismatch`. Print
|
|
the post-attempt path the run will keep, and the count of events with
|
|
`at` after the backup's `STAMP`, from a read-only open of the store
|
|
before anything stops. Count above 0 and `--events N` not equal to it
|
|
refuses `post-backup-events`. No change.
|
|
6. Steps 4.1 to 4.5 with the target. In step 3, the stopped store's
|
|
version must equal the pointer's `schema`, else `store-differs`, item
|
|
6. With `--restore`, the count is read again after the stop. A
|
|
different count goes to item 6 with `post-backup-events`, since an
|
|
event landed between the check and the stop. Then the restore runs in
|
|
place of the migration.
|
|
7. The gate, as for activate. A pass renames the pointer (with the
|
|
target's schema) and logs `ok` with `backup: ACTIVATION`.
|
|
|
|
Ruling 4 (sage-0736, confirmed in sage-0738) has two cases:
|
|
|
|
- A `rollback --restore` that fails after the restore has moved the old
|
|
`bus/` aside: the run does not go through item 6. That covers the gate
|
|
and also an install, start or signal failure after the swap; the script
|
|
treats every one the same and logs `rollbackCode: "not-attempted"`. A
|
|
restore that fails before the move goes through item 6 like any other
|
|
step, since the store hasn't changed. Putting the newer store back would
|
|
be a second swap, which is a retry under another name. The run prints
|
|
both results, stops both units, logs `rollback-failed` and exits 2. The
|
|
post-attempt directory and the restored `bus` stay where they are for
|
|
Jason. The unit files and wrapper also stay as the run installed them,
|
|
since they match the restored store. Nothing is swapped back. Tested in
|
|
T14b.
|
|
- A `rollback` without `--restore` whose gate fails: the store was never
|
|
swapped, so the run goes through item 6 as `activate` does. It puts back
|
|
the unit files and wrapper from its rollback set, restarts the pointer's
|
|
copy and runs the gate on it. Exit 1 with `rollback: ok`, or exit 2 with
|
|
`rollback-failed` if that gate fails too. Tested in T14c.
|
|
|
|
## --plan (item 8)
|
|
|
|
It reads the pointer, the log, the target copy's store.mjs and the
|
|
store's schema version. It writes nothing: no lock, no log line, no file
|
|
under the data root.
|
|
|
|
Mos takes the `--plan` output to Jason before the combined re-pin
|
|
(mos-0741, sage-0745), so a migration must be stated plainly. For a
|
|
store at 3f and refactor at 3h, `activate --plan` prints:
|
|
|
|
```
|
|
store schema: 3f (read from the stopped store)
|
|
target schema: 3h
|
|
migration: 3f to 3h, after a backup of the stopped store
|
|
migration steps: 3f to 3g, 3g to 3h
|
|
needs: --migrate (not given; the run would roll back with migration-needs-flag)
|
|
needs: Jason present for this run (decision 96 condition 2)
|
|
```
|
|
|
|
I added the `migration steps:` line in the slot on 2026-10-18, before the
|
|
scratch-root run (see "The scratch-root plan for Mos"). It walks the target store.mjs's `from:` and
|
|
`to:` pairs from the store's version, as text, the same way
|
|
`copy_schema` reads the version, so nothing in the copy runs. When the
|
|
lines don't pair up, or the walk doesn't reach the target's version, it
|
|
prints `migration steps: cannot be read from <file>` and the rest of the
|
|
plan is unchanged. The run itself still calls `migrate-schema.mjs`, which
|
|
applies the copy's own chain; the steps line only names it.
|
|
|
|
The store version comes from the store itself. While the bus unit is
|
|
active the read is `{ readOnly: true }`, which adds no file next to a
|
|
store the bus holds (the probe). With the bus stopped and no `-wal` or
|
|
`-shm`, it is the immutable read. With no store file, a `-wal` or `-shm`
|
|
present, or a failed read, it falls back to the pointer's `schema` and
|
|
says which case applied. When the store and the pointer disagree, a
|
|
`note:` line says so; the run itself goes by the stopped store.
|
|
|
|
Other cases print `migration: none` (same version), a `store-newer`
|
|
warning (not in the target's chain) or, for rollback, `restore: ... needs
|
|
--restore ACTIVATION` with the activation the log's backup field names
|
|
for the pointer's copy. Then the exact `bus-service.sh`,
|
|
`webui-service.sh` and `stack-copy.sh wrapper` command lines. Exit 0.
|
|
The refusals it can see before a run (no-pointer, business-mismatch,
|
|
copy, already-active, no-previous) still print, with exit 3 and no log
|
|
line.
|
|
|
|
After the migration or restore lines comes the `commits:` block
|
|
(sage-0778): `commits: the run adds N` when the pointer's copy is an
|
|
ancestor of the target, `commits: the run takes away N` when the target
|
|
is an ancestor of the pointer's copy (a rollback), then the
|
|
`git log --oneline --no-merges` lines, newest first, at most 200. When
|
|
neither copy is an ancestor of the other, `--plan` refuses
|
|
`copies-unrelated`. My 16:53Z message to Sage said "refuse if FROM isn't
|
|
an ancestor of TO". The code is wider than that on purpose, so a
|
|
rollback still lists what it takes away. T25 and T26 test it.
|
|
|
|
### The scratch-root plan for Mos (sage-0837)
|
|
|
|
Mos asked (mos-0765) for the combined re-pin's `--plan` output with the
|
|
3g to 3h migration and rows 148, 141, 143, 128, 124 and 149. Before any
|
|
live adopt there is no pointer, so I run it against a scratch root under
|
|
`~/darkwing-scratch/r162/`: a data root, a config naming it, a
|
|
hand-written pointer and a copies directory, all scratch. Nothing under
|
|
`~/.mosaic-dev`, the live copies directory or the units is written.
|
|
|
|
- The live copy id: `systemctl --user show mosaic-bus@mosaic-stack.service
|
|
-p WorkingDirectory` at 2026-10-18T19:32:16Z gave
|
|
`.../copies/52f35e7a0933e9fa845f59860a5de1709f9e5756`. It reads the
|
|
unit's property and writes nothing. sage-0859 also allowed a read-only
|
|
`systemctl --user is-active` on the bus unit name; the run never made
|
|
that call, because with no store file `plan_store` doesn't reach it.
|
|
- The pointer's `schema` is that copy's `SCHEMA_VERSION`, read with
|
|
`git show 52f35e7a0:packages/bus/src/store.mjs`: `3f`. I had expected
|
|
3g. That is why the steps line exists: `migration: 3f to 3h` alone
|
|
doesn't show the 3g to 3h step Mos asked for, and hand-writing 3g would
|
|
have been a false pointer.
|
|
- The target copy, from `scratch-plan-steps.log`:
|
|
`(cd ~/darkwing-scratch/r162/wt && scripts/stack-copy.sh copy
|
|
606fafd43154ded993a503933681198e737987cd --copies
|
|
~/darkwing-scratch/r162/plan/copies --allow-worktree)` at 19:45:54Z,
|
|
copied by 19:45:56Z. Removed at 19:45:56Z with `chmod -R u+w` and
|
|
`rm -rf` of that one copy directory; the copies dir was left with 0
|
|
entries. 606fafd43 was origin/refactor at the run; it moved past
|
|
cfde62f9f after sage-0859.
|
|
- A header I put above the script's output names the scratch root and
|
|
says the pointer is hand-written. The script prints no such header
|
|
itself.
|
|
- The scratch root has no `bus/bus.sqlite`, so the store schema line
|
|
reads `from the pointer; there is no store file`. `--plan` runs only
|
|
`check_target` and `plan`, not the unit checks, so the only other reads
|
|
are git in the checkout and the copies directory. The data root held
|
|
only `state/stack-active.json` and an empty `bus/` after the run.
|
|
|
|
The result, `scratch-plan.txt` (134 lines, exit 0): `migration: 3f to
|
|
3h`, `migration steps: 3f to 3g, 3g to 3h`, both `needs:` lines, then
|
|
`commits: the run adds 112`. All six rows are in the list (grep hits per
|
|
row: 148 7, 141 5, 143 8, 128 9, 124 9, 149 18). The range also carries
|
|
row 164, the row 165 brief and everything else since 52f35e7a0.
|
|
|
|
Mos then narrowed the window target to `8e7ea89a1`, the six rows only
|
|
(mos-0810 D1-D2, sage-0871). I reran the same plan in a new slot with
|
|
`r1/plan-8e7.sh`, a copy of plan.sh with the target, the scratch root
|
|
(`plan-8e7`) and the output file changed. It ran from this worktree
|
|
while it still matched the r1 manifest. The result is
|
|
`scratch-plan-8e7ea89a1.txt` (114 lines, exit 0). The `migration` and
|
|
`migration steps` lines are unchanged. The plan lists 92 commits, the
|
|
same count as `git rev-list --count 52f35e7a0..8e7ea89a1`. Each of the
|
|
six landing commits appears once. Row 164's `008746b6e` and `850484a7d`
|
|
are not in it. Of the 92, only the six touch code paths. The step log
|
|
is `scratch-plan-8e7ea89a1-steps.log`. Its "plan: written to" line names
|
|
the wrong file, and `live-reads.md` explains why. Both 606fafd43 files
|
|
stay as they were, since this review uses them.
|
|
|
|
Sage forwards the output to Mos as a scratch-root plan. The real
|
|
`--plan` against the live pointer runs again in Mos's window before any
|
|
adopt.
|
|
|
|
## status (item 9)
|
|
|
|
The pointer, the last five log lines, any interrupted run,
|
|
`stack-copy.sh check` output and result, `<pointer copy>/scripts/mosaic
|
|
bus status`, whether the unit flags match the pointer, and the drift:
|
|
whether the copy is an ancestor of `refactor` and
|
|
`git rev-list --count COPY..refactor`. It only reads.
|
|
|
|
## Tests (`packages/cli/tests/stack-release.test.mjs`)
|
|
|
|
Fixture, built once per file:
|
|
|
|
- A temp canonical repo, made the way `stack-copy.test.mjs` `setup` does
|
|
it: `git ls-files -co` of scripts and packages copied in, `git init -b
|
|
refactor`, the light package.json. I copy the setup code rather than
|
|
import it, because importing a test file runs its tests too. The base
|
|
commit is F.
|
|
- On `refactor`, empty commits A, B, C1 and C (subjects `row 901: A` to
|
|
`row 904: C`). A side branch from A holds S and merges into `refactor`
|
|
as M, with `--no-ff`. A branch named `unreached` from A holds U.
|
|
`refactor` and `origin/refactor` both point at M.
|
|
- Read-only stub copies under a temp `copies/<id>/` for F, A, B, C1, C, S
|
|
and U. M has no copy, so it is the `copy-missing` case. Each copy has
|
|
`scripts/mosaic`, `packages/webui/src/cli.mjs` (present only, never
|
|
run), `packages/bus/src/store.mjs` and
|
|
`packages/bus/src/migrate-schema.mjs`. The store file in A, B, C1, S
|
|
and U says `3g` with a `from: '3f'` step. C says `3h` with `3f` and
|
|
`3g` steps. F has no `SCHEMA_VERSION`, which gives
|
|
`copy-schema-unknown`.
|
|
- The stub migrate records the `backups/stack/` entries it saw, and
|
|
whether each has `SHA256SUMS`, in `migrate.json`, then sets the store
|
|
to `3h` with `node:sqlite`. The `migrate-fails` fault sets
|
|
`3h-partial` and exits 1.
|
|
- The stand-ins `systemctl`, `journalctl` and `systemd-analyze` are node
|
|
scripts with a `#!<node>` shebang in a `sbin/` directory that comes
|
|
first on `PATH`. All three exist, so the real ones never run. A
|
|
`package.json` with `"type":"commonjs"` in the fixture's base
|
|
directory covers them, the holder and the copies' `scripts/mosaic`.
|
|
|
|
Per test: a temp `HOME`, `XDG_CONFIG_HOME`, `XDG_DATA_HOME`, data root,
|
|
`MOSAIC_CONFIG` (a valid config naming the temp data root), `bin/` with
|
|
the Q81 wrapper, `mosaic-old` and a fake `claude`, and a stand-in state
|
|
directory with a `faults/` directory. A fault is a file named for the
|
|
copy it hits, such as `healthz-<id>`, so one fault can fail the new copy
|
|
and leave the rollback target alone.
|
|
|
|
`seed` builds the stack the way an operator would: a real WAL-mode
|
|
`bus.sqlite` with a `meta` row and three events, then the real
|
|
`bus-service.sh install`, `webui-service.sh install` and
|
|
`stack-copy.sh wrapper` against the fixture copy, then a start of both
|
|
units. The pointer comes from a real `adopt`, or is written by hand with
|
|
no log (T23, T25), or is left out.
|
|
|
|
The stand-in `systemctl`:
|
|
|
|
- `show UNIT -p ...` reads the installed unit file and its drop-ins
|
|
(`[email protected]` and `<unit>.d`), fills in `%i` and `%h`, and
|
|
answers `LoadState`, `WorkingDirectory`, `ExecStart={ path=... ;
|
|
argv[]=... ; }`, `Environment`, `MainPID` and `DropInPaths`. The
|
|
`check-<id>` fault moves the Console's `WorkingDirectory`, so
|
|
`stack-copy.sh check` fails.
|
|
- `list-units`, `is-active`, `is-enabled`, `start`, `stop` and
|
|
`daemon-reload` keep unit state in `units.json` (written through a temp
|
|
file and a rename) and append every call to `calls.log`, so the order
|
|
is testable.
|
|
- `start` of the bus runs a holder process. The holder opens
|
|
`bus.sqlite` in WAL mode and keeps it open, writes `bus/writer.lock`
|
|
and a `host.json` with the business, its pid and a boot status, and
|
|
closes the store and removes the lock on SIGTERM. That gives the real
|
|
`-wal` and `-shm` files of a held store, and a clean close on stop. The
|
|
holder starts through `bash -c` with every fd above 2 closed, so it
|
|
never inherits the script's fd 9 lock. `SD_CLOCK_OFFSET` backdates the
|
|
seeded host and its PM run by two minutes, so they predate any run's
|
|
start.
|
|
- `start` of the bus with `--pm` in the unit's ExecStart appends the
|
|
`launched` line, as `cli.mjs:503` writes it, to the stand-in journal,
|
|
and adds the run to `launches.json`. A `pm-<id>` fault picks another
|
|
case: no line, two lines, a changed word, a run missing from the list,
|
|
an early `startedAt`, a human launch with no line, a run for another
|
|
business, a dead run.
|
|
- `stop` sends SIGTERM and waits for the holder to exit. The one-shot
|
|
`stop-leaves-wal` and `stop-leaves-shm` faults leave an empty `-wal` or
|
|
`-shm` beside the store, as an unclean close would. Each bus start and
|
|
stop records the name, size, mtime and sha256 of every file under
|
|
`bus/` in `snap.jsonl`, so T22 can compare the store before and after
|
|
the script's read.
|
|
- `start-dies-<id>` sets the bus to `failed`.
|
|
|
|
The stub `scripts/mosaic` prints `bus status --json` from `host.json`,
|
|
with `live` recomputed from `/proc`, and `launches list --json` from
|
|
`launches.json`. The `status-<id>` and `launches-<id>` faults make it
|
|
fail. The Console is an HTTP server inside the test process on a free
|
|
port. It answers 200 when the stand-in says the Console is active, and
|
|
500 when a `healthz-<id>` fault names the Console unit's current copy.
|
|
That is why the test runs the script with async `spawn`: `spawnSync`
|
|
would block the server.
|
|
|
|
`systemd-analyze --user unit-paths` prints the temp unit directory.
|
|
|
|
The stand-in `journalctl` logs the `--since` argument it got and prints
|
|
the stand-in journal lines for the unit at or after it. It accepts
|
|
`--since` only as `@<epoch seconds>` or `YYYY-MM-DD HH:MM:SS UTC`, and
|
|
exits 1 on any other form, so a bare local time gives no lines.
|
|
|
|
Every `run` call also checks T17: the listing of `backups/stack/`,
|
|
`state/stack-rollback/`, `bus.post-attempt-*` and `copies/` taken before
|
|
the run is a subset of the one after.
|
|
|
|
Clean-up kills only the holder pids the stand-in logged, after checking
|
|
each one's `/proc/<pid>/cmdline` names this file's holder.
|
|
|
|
Gate failure cases run with `--timeout 3`.
|
|
|
|
| # | Case | Brief |
|
|
|---|---|---|
|
|
| T1 | `adopt` writes the pointer (claude and console from `show`) and one `ok` line; a second adopt refuses `already-adopted`; a failing check refuses `check-failed` with no pointer | item 2 |
|
|
| T2 | `activate` installs exactly the pointer's `--pm`, `--claude` and Console flags (unit file text) | items 3, 4 |
|
|
| T3 | a hand-edited unit (each of pm, claude, a console flag) refuses `unit-flags-drift`, prints the difference, changes nothing | item 3 |
|
|
| T4 | each item 3 refusal: units, wrapper, pointer and store unchanged by sums, exactly one new log line with the code, no `start` or `stop` in `calls.log`; `instance-busy` has two subtests (another business's instance active, this one's activating) and both name the instance and state | item 3, ruling 1 |
|
|
| T5 | no-schema activate: `calls.log` shows stop Console, stop bus, daemon-reloads, start bus, start Console in order; pointer names C1; no `backups/stack/` | item 4 |
|
|
| T6 | B to C without `--migrate`: `migration-needs-flag`, old stack started, pointer unchanged | item 4.3 |
|
|
| T7 | B to C with `--migrate`: backup exists before the migrate stub ran (the stub records whether it saw the backup's sums), pointer schema `3h` | item 4.3 |
|
|
| T8 | B to C with `--migrate` and a failing migration: post-attempt dir exists, restored files match the backup's sums, old stack started | item 6 |
|
|
| T9 | `store-newer` (store at a version outside C's chain) | item 4.3 |
|
|
| T10 | each gate check failing alone (five subtests): rollback, pointer unchanged, unit files and wrapper back byte for byte, log names the check | items 5, 6 |
|
|
| T10b | the PM check, one subtest per case, each refusing `pm-launch-unconfirmed` with the case named: no `launched` line; two lines; a line that starts with `launched ` but has a later word changed (`launched mosaic-pm: job R, ...`); one line whose run isn't in `launches list`; one line whose run has `startedAt` before step 5; a live human `mosaic launch` session after step 5 with no journal line; a run for another business; a run that isn't live. A last subtest passes with one well-formed line and a live run, and checks the `--since` argument the stand-in received starts with `@` | item 5, ruling 2 |
|
|
| T11 | rollback gate fails too: `rollback-failed`, exit 2, both units stopped, no `start` after the failing one | item 6 |
|
|
| T12 | `rollback` finds the previous copy from the log | item 7 |
|
|
| T13 | `rollback` across a migration refuses `schema-differs`; with `--restore` it restores and goes live on the old copy | item 7 |
|
|
| T14 | `--restore` with post-backup events and no or a wrong `--events N` refuses `post-backup-events`; the right N proceeds | item 7 |
|
|
| T14b | a `rollback --restore` whose gate fails: `rollback-failed`, exit 2, both units stopped, the post-attempt dir and the restored `bus` in place, no rename after the restore | item 7, ruling 4 |
|
|
| T14c | a `rollback` without `--restore` whose gate fails: item 6 runs, the unit files and wrapper are back byte for byte from the run's rollback set, the pointer's copy is started and gated, exit 1 with `rollback: ok`; the store's file set and sums are unchanged | item 7, ruling 4 |
|
|
| T15 | `--plan` on activate and rollback prints the migration or restore and the flags; units, pointer, log and store unchanged. After an activate to C with `--migrate`, `rollback --plan` prints, line for line, the restore with the activation and `commits: the run takes away 2` | item 8 |
|
|
| T25 | sage-0778 ask 1: pointer on B, `activate S --plan` (S is on a side branch, so neither copy is an ancestor of the other) exits 3 with exactly `stack-release: refusing copies-unrelated: neither <B> nor <S> is an ancestor of the other, so the plan cannot list the commits`, empty stdout, no log, no lock | item 8, sage-0778 |
|
|
| T26 | sage-0778 ask 2: `activate C --plan`, with and without `--migrate`, prints the whole output line for line: the migration line, the `--migrate` need, the Jason line, then `commits: the run adds 2` and the two `row 904` and `row 903` lines, then the three install lines. `activate C1 --plan` prints `migration: none` | item 8, sage-0778 |
|
|
| T27 | the steps line: store at `3f`, `activate C --plan` prints `migration: 3f to 3h` then `migration steps: 3f to 3g, 3g to 3h`; `activate C1 --plan` (C1's fixture store.mjs has `from:` lines and no `to:` lines) prints `migration steps: cannot be read from <C1>/packages/bus/src/store.mjs` and still exits 0; `activate A --plan` (A's `to:` names `3x`, so the walk stops short of `3g`) prints the same for A | item 8, sage-0859 |
|
|
| T23 | sage-0745: the store at `3g` and stopped, the pointer on B, target C at `3h`. `activate C B --plan` exits 0 and prints `3g`, `3h`, `--migrate` and `Jason present ... decision 96 condition 2`. Afterwards the store's file set, mtime and sha256, the listing under the data root, the unit files and the wrapper are unchanged; there is no log file and no lock file. A second subtest with the bus active reads the version through the held read and prints the same lines | item 8, mos-0741 |
|
|
| T24 | `business-mismatch` (pointer for B, run for another business) and `capture-failed` (the rollback set's parent made read-only): each exits 3 with one `refused` line, no `stop` in `calls.log`; `capture-failed` follows a `started` line, so `status` shows no interrupted run afterwards. An activate ID that isn't 40 hex is a usage error, exit 4, no log | item 3, item 4.1 |
|
|
| T16 | `status` prints the pointer, five log lines, check, bus status and drift, and writes nothing | item 9 |
|
|
| T17 | in every `run` call: nothing under `backups/stack/`, `state/stack-rollback/`, `bus.post-attempt-*` or `copies/` is ever removed (the listing before the run is a subset of the one after). The store sums are checked in each test through its own fingerprint | item 10 |
|
|
| T18 | `stack-copy.test.mjs` still passes with `reachable` split out and the source guard | stack-copy.sh change |
|
|
| T19 | a `started` line with no final line: `status` shows the interrupted activation; `adopt`, `activate` and `rollback` each refuse `interrupted-run` naming it, with nothing changed | ruling 3 |
|
|
| T20 | a held `state/stack-release.lock`: `adopt`, `activate` and `rollback` refuse `run-locked` and change nothing; `status` and `--plan` still run | ruling 6 |
|
|
| T21 | a bus unit with no `MOSAIC_CLAUDE_BIN`: `adopt` and `activate` refuse `no-claude-pin` | ruling 5 |
|
|
| T22 | the store read (sage-0739), three subtests. A `bus.sqlite-wal` left after the stop refuses `store-unclean` naming the file, goes to item 6, and leaves every file under `bus/` with the same name, size, mtime and sha256. The same with a `bus.sqlite-shm`. The clean case: an activate reads the version, and afterwards `bus/` holds only `bus.sqlite`, with its mtime and sha256 the same as before the read | item 4.3, the store read |
|
|
|
|
## Mutants
|
|
|
|
One per behavior. Each is a single edit to a scratch copy of
|
|
`stack-release.sh` or `stack-copy.sh`, run against the file and expected
|
|
to fail the named test. The run script puts each mutant in its own
|
|
process group and kills the group on timeout.
|
|
|
|
| M | Mutation | Killed by |
|
|
|---|---|---|
|
|
| M1 | adopt skips the already-adopted check | T1 |
|
|
| M2 | adopt reads console flags without dropping `--checkout` | T1, T2 |
|
|
| M3 | activate installs without `--claude` | T2 |
|
|
| M4 | drift check ignores console flags | T3 |
|
|
| M5 | `instance-busy` check removed | T4 |
|
|
| M6 | `drop-in-override` check removed | T4 |
|
|
| M7 | `copy-unreachable` check removed | T4 |
|
|
| M8 | a refusal logs nothing | T4 |
|
|
| M9 | the bus stops before the Console | T5 |
|
|
| M10 | the Console starts before the bus | T5 |
|
|
| M11 | the migration runs without `--migrate` | T6 |
|
|
| M12 | the backup is taken after the migration | T7 |
|
|
| M13 | item 6 skips the store restore | T8 |
|
|
| M14 | `store-newer` treated as same | T9 |
|
|
| M15 to M19 | each gate check made to always pass | T10, one subtest each |
|
|
| M20 | the pointer is written before the gate | T10 |
|
|
| M21 | item 6 skips the wrapper restore | T10 |
|
|
| M22 | a failed rollback gate retries | T11 |
|
|
| M23 | rollback picks the newest `to` without skipping the current copy | T12 |
|
|
| M24 | `schema-differs` check removed | T13 |
|
|
| M25 | `--events` not compared | T14 |
|
|
| M26 | `--plan` writes a log line | T15 |
|
|
| M27 | restore deletes the post-attempt directory | T13, T8 (the run killed it there, not in T17) |
|
|
| M28 | `reachable` returns 0 always | T4, T18 |
|
|
| M29 | `instance-busy` ignores this business's transitional state | T4 (activating subtest) |
|
|
| M30 | the PM check accepts any live session started after step 5 | T10b (human launch subtest) |
|
|
| M31 | a `rollback --restore` gate failure goes through item 6 | T14b |
|
|
| M32 | the interrupted-run check removed | T19 |
|
|
| M33 | the run lock removed | T20 |
|
|
| M34 | the `no-claude-pin` check removed | T21 |
|
|
| M35 | `--since` given a bare local time instead of `@<epoch>` | T10b (pass subtest) |
|
|
| M36 | the PM check takes the first `launched` line instead of requiring exactly one | T10b (two-lines subtest) |
|
|
| M37 | the PM check skips the `startedAt` comparison | T10b (early-run subtest) |
|
|
| M38 | the `store-unclean` check removed | T22 (`-wal` and `-shm` subtests) |
|
|
| M39 | the store read uses `{ readOnly: true }` without immutable | T22 (clean subtest) |
|
|
| M40 | a `rollback` without `--restore` whose gate fails exits 2 without item 6 | T14c |
|
|
| M41 | `--plan` omits the `Jason present` line | T23 |
|
|
| M42 | `--plan` takes the run lock or logs `started` | T23, T20 |
|
|
| M43 | `--plan` reads the store with `{ readOnly: true }` while stopped, without immutable | T23 (file set) |
|
|
| M44 | a capture failure leaves the `started` line without a final line | T24 |
|
|
| M45 | the `copies-unrelated` refusal removed, or the `commits:` block left out | T25, T26 |
|
|
| M46 | `--plan` omits the migration line | T26, T23 |
|
|
| M47 | `--plan` omits the `migration steps:` line | T26, T23, T27 |
|
|
| M48 | `plan_steps` drops the odd-line-count check | T27 |
|
|
| M49 | `plan_steps` prints a partial walk that doesn't reach the target | T27 (the A case) |
|
|
|
|
## Candidate round 1
|
|
|
|
`r1/build.patch` against BASE `cfde62f9f` (sha256
|
|
`f5d28a751cb77c4054c598b36dd4dd996d7372bb47a0a8030c39bc8517b8a397`).
|
|
`r1/candidate-manifest.sha256` (sha256
|
|
`091693c51d137e4cd31e75ee22aef4f22c307e15abce6f91f6c518e3ff78fdd6`) lists
|
|
the 5 files in "Files". I applied `r1/build.patch` with `git apply
|
|
--index` to a fresh worktree at cfde62f9f: `sha256sum -c` passed on
|
|
every line, `git diff --cached --check` was clean, the staged names match
|
|
the manifest (5 files, +1604 -45), and the worktree was removed after.
|
|
`scripts/stack-release.sh` is mode 100755, the test file 100644. The
|
|
TOOLS.md diff is one hunk, inside "Pinned stack update".
|
|
|
|
### Checks round 1
|
|
|
|
On Sage's slot, 2026-10-18T19:25:10Z to 19:46:14Z, worktree at
|
|
cfde62f9f, TMPDIR `~/darkwing-scratch/r162/tmp`, each file run singly
|
|
with `timeout 600 node --test --test-reporter=spec`:
|
|
|
|
| Test file | Pass | Fail | Run |
|
|
|---|---|---|---|
|
|
| `packages/cli/tests/stack-release.test.mjs` | 58 | 0 | 19:43:19Z, after the steps change |
|
|
| `packages/cli/tests/stack-copy.test.mjs` | 12 | 0 | 19:45:27Z |
|
|
|
|
The first run in the slot, before the capture `mkdir` fix, was 56/57
|
|
with T24 failing (see "Notes"); the second was 57/57; the steps change
|
|
added T27. Mutants: all 47 killed against the 57-test file, then M41,
|
|
M45b, M46 and the new M47 to M49 killed against the 58-test file. The
|
|
mutant runner restores each file from a pristine copy and checks its
|
|
sha256 after every mutant. `sweep.sh` found no process of mine after.
|
|
|
|
Logs: `r1/stack-release-test.log`, `r1/stack-copy-test.log`,
|
|
`r1/mutants.summary`, `r1/mutants-steps.summary` and the runner
|
|
`r1/mutants.py`.
|
|
|
|
Not run here: the cli, bus and webui suites and the `scripts/test-*.sh`.
|
|
They are Sage's gate at landing.
|
|
|
|
## Candidate round 2 (sage-0882)
|
|
|
|
Row 125 landed as aa625565f, with its packet at df4d5702f (origin/refactor). It changed two of
|
|
this row's five paths, `docs/TOOLS.md` and `packages/cli/README.md`, so the r1 manifest no
|
|
longer described the tree this row would land on. Round 2 makes no source change.
|
|
|
|
I applied `r1/build.patch` with `git apply --index` to a fresh worktree at df4d5702f. It
|
|
applied without conflict, `git diff --cached --check` was clean, and the staged names are the
|
|
same 5 files (+1604 -45). The modes are unchanged: both scripts 100755, the test file 100644.
|
|
The worktree was removed after.
|
|
|
|
`r2/build.patch` against BASE `df4d5702f` (sha256
|
|
`336503b59a894c8ee0449dba5c36feb242a80404c9f262e96740def231fa541b`).
|
|
`r2/candidate-manifest.sha256` (sha256
|
|
`c213a801681b5dd9029520d606b3303cb30ed2fbfff8e320b0f7d79d483bd15c`).
|
|
|
|
What moved from r1:
|
|
|
|
- In `r2/build.patch`, only the two `index` lines for `docs/TOOLS.md` and
|
|
`packages/cli/README.md`, the base and result blob ids. Every hunk header, context line and
|
|
added or removed line is the same as in r1, because row 125's additions to both files sit
|
|
below this row's hunks.
|
|
- In the manifest, only the sha256 of those two files, since the files now carry row 125's
|
|
text too. `stack-release.test.mjs`, `stack-copy.sh` and `stack-release.sh` keep their r1
|
|
sha256s.
|
|
|
|
Nothing was rerun. `stack-release.test.mjs` and `stack-copy.test.mjs` read neither doc file,
|
|
and the scripts are byte-identical to r1, so the round 1 checks stand.
|
|
|
|
## Rulings (Sage, sage-0736, with sage-0738, sage-0739 and sage-0745)
|
|
|
|
1. `instance-busy`: approved as read. Refuse when another business's
|
|
`mosaic-bus@` instance is active, because the copy and wrapper switch
|
|
reach it too, or this business's is activating, deactivating or
|
|
reloading. The message names the instance and state. Test both cases.
|
|
2. PM check: approved without reading the business file, and it must
|
|
match the PM role: a live launch for B whose role is the PM, started at
|
|
or after this run's bus start. Don't name `launch.by`. sage-0738
|
|
accepted the journal reading in "The health gate" above, with an
|
|
unambiguous `--since`, exactly one line, and the one code
|
|
`pm-launch-unconfirmed` for every failure. Each refusal is tested
|
|
(T10b), and the stdout-line dependency is stated for the review. The
|
|
bus-side PM marker is a follow-up outside D1.
|
|
3. Logging: `started` and a final line in the jsonl, step detail in
|
|
`steps.log`. An unfinished `started` is an interrupted run: `status`
|
|
shows it and the next `adopt`, `activate` or `rollback` refuses
|
|
`interrupted-run`, naming the activation. Clearing it is Jason's or
|
|
Mos's. Tested.
|
|
4. A rollback whose own gate fails stops, prints both results, leaves the
|
|
stack stopped and writes `rollback-failed`. No further renames: a
|
|
second swap is a retry under another name. The post-attempt dir and
|
|
the restored bus stay for Jason. sage-0738 confirmed my reading: after
|
|
a restore, the unit files and wrapper stay as installed, and a
|
|
rollback that didn't restore goes through item 6 (T14b, T14c).
|
|
5. `no-claude-pin`: approved for both `adopt` and `activate`.
|
|
6. Run lock: approved, `flock -n` on `state/stack-release.lock`, code
|
|
`run-locked`. `status` and `--plan` take no lock.
|
|
|
|
SQLite probe: on Sage's slot (Rocko first, then me), on a fixture store
|
|
in my scratch, never the live store or a copy of it. Try `readOnly` and
|
|
`file:...?immutable=1`, and check no `-shm` or `-wal` appears and the
|
|
main file's mtime and hash hold. If neither works, the copy fallback.
|
|
Result in "The store read": readOnly left `-shm` and `-wal`, immutable
|
|
left nothing. sage-0739 approved immutable only with no `-wal` or `-shm`
|
|
present, `store-unclean` otherwise, no copy fallback, tested in T22.
|
|
|
|
sage-0745 (mos-0741): Mos takes the combined re-pin's `--plan` preview to
|
|
Jason, so `activate --plan` names the store's version and the target's
|
|
(3g to 3h there), that `--migrate` is needed, and that Jason must be
|
|
present under decision 96 condition 2. Tested in T23 with a fixture
|
|
store one version behind, which also checks that `--plan` writes
|
|
nothing. Base moved to 70504d23a, since row 124 touched
|
|
`packages/bus/src/store.mjs` and `schema.sql`.
|
|
|
|
## Notes
|
|
|
|
- T24 failed on the first run in the slot: `mkdir`'s "Permission denied"
|
|
reached stderr ahead of the `capture-failed` refusal. Both `mkdir`
|
|
calls in `capture` now send stderr to /dev/null; the refusal names the
|
|
path. The rerun passed.
|
|
- The install scripts render from the canonical checkout's templates. A
|
|
later rollback to an older copy renders that copy's units with today's
|
|
templates, and the in-run rollback restores the captured bytes. The
|
|
window plans had the same property.
|
|
- Nothing here prunes. Disk use grows by one rollback set per run and one
|
|
store copy per migrating run.
|
|
- The new TOOLS.md "Pinned stack update" section numbers the drop-in
|
|
removal as step 4, so the `drop-in-override` message says
|
|
`TOOLS.md step 4`. The brief's item 3 text still says step 3, from the
|
|
old seven-step list.
|