stack-upgrade: root's upgrade and rollback driver for root-owned units, with the host run sheet; stack-release --backup last and the external plan refusals (row 233, #1735, installer I3)
This commit is contained in:
1 parent
e65a3ab4d2
commit
aacfa2d7a4
7 files changed
+1397
-4
No files matched your search
@@ -0,0 +1,213 @@
|
||||
# Row 233 (I3): root's upgrade driver and the host run sheet
|
||||
|
||||
Brief: `docs/plans/2026-10-20_installer-rows.md` section I3, blob `e61291bcf1c6` (commit a0e3d7cfd), re-pinned
|
||||
by Sage (sage-1685, rev 2631) from `f35e7e57daee` (ad6f0be7a). The later blob only appends I4; the I3 text
|
||||
is the same. Gitea #1735, Vikunja P19
|
||||
#432. Owner Darkwing, reviewer Filbert, gate owner Sage. Design input: row 195 DESIGN r6,
|
||||
`agents/darkwing/work/queue-195/DESIGN.md` at b7239bb66, sections 2, 4 and 7. Where the brief and r6
|
||||
differ, the brief wins. Sage's rulings during the build: sage-1650 and sage-1652 (`--backup last`, the
|
||||
`--no-rehearsal` mapping, `--rehearse-port`, hostcheck), and sage-1653 (`from_log newest`, the two external
|
||||
plan refusals, the `BACKUP_STORE` line).
|
||||
|
||||
## Files
|
||||
|
||||
Base: e65a3ab4d, row 232's landing, on `darkwing/row233-r1`. The build started on ea1e85c7b
|
||||
(`darkwing/row233`, row 232's candidate e6ca230d2 and its BUILD.md) before row 232 landed. The three row 232
|
||||
files this row changes are identical on the two bases, so the uncommitted diff moved across unchanged.
|
||||
|
||||
- `scripts/stack-upgrade.sh` (new): `upgrade` and `rollback`, root's side of r6 A1-A5, A2f and R1-R4.
|
||||
- `packages/cli/tests/stack-upgrade.test.mjs` (new, 20 tests).
|
||||
- `docs/guides/host-run-sheet.md` (new).
|
||||
- `docs/TOOLS.md`: a new subsection "Root's upgrade driver (`scripts/stack-upgrade.sh`, row 233)", plus the
|
||||
row 232 lines below.
|
||||
- Row 232 files, changed on Sage's rulings (the brief's files list doesn't name them):
|
||||
- `scripts/stack-release.sh`:
|
||||
- `rehearse --backup last` takes the newest backup the log records (`from_log newest`) and prints
|
||||
`PASS backup-last`. With none it fails `no-backup`, exits 3 and makes no rehearsal directory.
|
||||
A newest backup that isn't an activation id aborts `backup-unreadable`, exit 1.
|
||||
- The rehearsal prints `BACKUP_STORE <activation id> schema <version>` before its migration step.
|
||||
- `--plan` under `--units external` refuses `open-activation` when any rollback set holds
|
||||
`AWAITING-UNITS`, and `migrate-required` for a migrating target without `--migrate`. Root reads
|
||||
only the plan's exit, so these two have to be refusals there. User mode prints the same bytes.
|
||||
- `packages/cli/tests/stack-release.test.mjs`: three new `row 233:` tests for `--backup last`.
|
||||
- `docs/TOOLS.md`: `--backup ACTIVATION|PATH|last` in the rehearse usage, a paragraph on `last`, and a
|
||||
paragraph on the two external plan refusals.
|
||||
- This file.
|
||||
|
||||
## The brief, item by item
|
||||
|
||||
| Brief item | Where |
|
||||
|---|---|
|
||||
| Root runs from a root-owned checkout, never a copy | `head-mismatch` (A1); TOOLS.md; run sheet section 6 |
|
||||
| I2's render with `/etc/mosaic-stack/<instance>.args`, `--copy` the one changed value | `read_args`, `args_with`; the args test |
|
||||
| The args file changes only after a confirmed run | `prepare_args` before A2, `mv` after A5; sequences 1 and 7, m6 |
|
||||
| `copy --bundle` as the stack's user before A1 | the `fetch`, `ff`, `copy` steps; sequence 1's sudo keys |
|
||||
| A rehearsal before A2, a failed one refuses with nothing stopped | `rehearse`; three rehearsal tests, m4 |
|
||||
| The root lock, held to the end, rollback included | `flock -n 9`; the lock test and sequence 3's probe, m1, m2 |
|
||||
| A1 refuses `open-activation` | stack-release.sh's external plan; the AWAITING-UNITS test, m3 |
|
||||
| `migrate-required` without `--migrate` | stack-release.sh's external plan; its test, m5 |
|
||||
| The step log, 0640, one line per step | `logl`, `step`; every test checks lines against `STEP` |
|
||||
| Exit codes 0-3 | `finish`, `refuse`; exit 4 is usage |
|
||||
| duckworth-0055's notes | the next section |
|
||||
| Root parses no stack user output beyond exits | the step log test; the one exception is sage-1653's `BACKUP_STORE` |
|
||||
| The host run sheet | `docs/guides/host-run-sheet.md` |
|
||||
|
||||
## duckworth-0055's notes on r6 (brief line 169)
|
||||
|
||||
- N1, A2's stop timeout under `gate --since`. Row 232's BUILD.md left two pieces to I3. The run sheet's
|
||||
A2f line is in section 6 ("When a run doesn't exit 0", exit 2). The test is `N1: a bus that outlives
|
||||
A2's timeout keeps its old start`: a bus that ignores `stop` keeps its old `ActiveEnterTimestamp`,
|
||||
`wait-down` logs `rc=1 ... state: active`, A2f restarts and gates, the gate fails `stale-start`, and the
|
||||
driver stops both units and exits 2. Sequence 4 is the other half: an A3 exit 3 goes to A2f and exits
|
||||
1 with the old copy's gate passing.
|
||||
- N2, the claude path under `/opt/mosaic-stack`. Run sheet section 3 installs it there as root with
|
||||
managed settings. The args file's `--claude` carries it, and `case2-unit.sh` already refuses a binary
|
||||
that isn't root's or sits under the home. The driver passes the args file through and adds no claude
|
||||
check of its own. F6 is I2's.
|
||||
|
||||
## Sage's rulings during the build
|
||||
|
||||
- sage-1650 and sage-1652. Root always runs the rehearsal and maps its exit:
|
||||
- 0 goes on, or refuses `rehearsal-required` with `--no-rehearsal`.
|
||||
- 3 refuses `no-backup`, or with `--no-rehearsal` logs `rehearsal-skipped` and goes on.
|
||||
- Anything else refuses `rehearsal-failed`.
|
||||
|
||||
`--rehearse-port N`, default 7339. Hostcheck: a missing file or a nonzero `--check` refuses
|
||||
`profile-drift`.
|
||||
- sage-1653. `from_log newest`, and the external plan refusals `open-activation` and `migrate-required`.
|
||||
Residual: the `BACKUP_STORE` id and schema land in root's step log, but nothing ties that backup to the
|
||||
one the activation's own backup step takes later. A stopped backup before the upgrade would close that
|
||||
gap; it's a later row, not this one.
|
||||
|
||||
## Tests
|
||||
|
||||
`stack-upgrade.test.mjs` runs the driver as the test's user against a scratch root, with the test seams
|
||||
`STACK_UPGRADE_*`. The stand-ins are a fake `sudo` (logs its argv, checks the driver's form, then runs the
|
||||
real command as the test's user, r6's B1 pattern), a fake `case2-unit.sh` render that installs and starts
|
||||
the fixture units, a fake `systemctl` for root's system units, and a shell `hostcheck.sh`. The stack's
|
||||
user's steps run the real `stack-release.sh` and `stack-copy.sh` from a clone made from a real bundle. The
|
||||
units, copies and store are `stack-release-fixture.mjs`'s.
|
||||
|
||||
| Test | Covers |
|
||||
|---|---|
|
||||
| sequence 1 | confirm fails healthz, rollback with `--cause A5`, exit 1; every sudo call in the driver's form |
|
||||
| sequence 2 | `activate --migrate` fails at the wrapper, store restored, `--cause A3`, exit 1 |
|
||||
| sequence 3 | render fails, `--cause A4`; a probe run during `render-old` refuses `upgrade-running` |
|
||||
| sequence 4 | stage exits 3, A2f restarts and gates the old units, exit 1 |
|
||||
| N1 | a unit that won't stop: A2f's gate fails `stale-start`, exit 2, units stopped |
|
||||
| sequence 5 | confirm of the wrong copy refused, `--cause A5`, exit 1 |
|
||||
| sequence 6 | revert fails, exit 2; the next run refuses `interrupted-run` at its plan |
|
||||
| sequence 7 | two upgrades and a `rollback --to` with `--restore`; the args file after each confirm; `to-not-previous` |
|
||||
| sequence 8 | a migration that writes and fails, store back byte for byte, exit 1 |
|
||||
| lock | a second run while the lock is held: one log line, no call |
|
||||
| open-activation | any `AWAITING-UNITS` refuses at A1, nothing stopped |
|
||||
| migrate-required | refused without `--migrate`, no rehearsal, nothing stopped |
|
||||
| rehearsal-failed | the rehearsal's argv, exit 3, nothing stopped |
|
||||
| `BACKUP_STORE` | the line after a match; four malformed lines log `BACKUP_STORE none` |
|
||||
| no-backup | the real rehearsal with no backup refuses; `--no-rehearsal` logs the skip and goes on |
|
||||
| backup-last | the real rehearsal names the newest backup; `--no-rehearsal` with one refuses `rehearsal-required` |
|
||||
| args file | group write, other write, two `--copy`, an unknown flag, a symlink, `args-differ` |
|
||||
| hostcheck | nonzero `--check` and a missing file refuse `profile-drift` before the plan |
|
||||
| head-mismatch | checkout not at `--sha`; a commit the clone can't fast-forward to refuses `fetch-failed` |
|
||||
| usage | twelve bad argvs exit 4 with no log and no lock |
|
||||
|
||||
The step log check (`STEP`) runs in every test that reads the log: each line must be a step line or the
|
||||
result line, and none may hold the stack user's output (`stack-release:`, `PASS `, `FAIL `, `REHEARSAL`,
|
||||
the fake rehearsal's text).
|
||||
|
||||
## Mutants
|
||||
|
||||
`~/darkwing-scratch/r233/mutants.sh`, not committed because it edits the worktree. It has the brief's six
|
||||
mutants and two for sage-1650's mapping. The method is row 232's: a checked backup, exact replacements, a
|
||||
syntax check, the named killers, then the file put back with its sha256 checked. One control runs first:
|
||||
the whole `stack-upgrade.test.mjs` with no change. `DRY=1` matched each replacement once on the candidate
|
||||
(2026-10-21).
|
||||
|
||||
| Mutant | File | Killers |
|
||||
|---|---|---|
|
||||
| m1 drop the flock | stack-upgrade.sh | `a second run while the lock is held`, `sequence 3:` |
|
||||
| m2 release the lock when the rollback starts | stack-upgrade.sh | `sequence 3:` |
|
||||
| m3 skip A1's open-activation check | stack-release.sh | `A1 refuses open-activation` |
|
||||
| m4 skip the rehearsal | stack-upgrade.sh | `the rehearsal runs after the plan`, `sequence 1:` |
|
||||
| m5 accept a migrating target without `--migrate` | stack-release.sh | `migrate-required` |
|
||||
| m6 write the args file before confirm | stack-upgrade.sh | `sequence 7:`, `sequence 1:` |
|
||||
| m7 `--no-rehearsal` accepted with a backup | stack-upgrade.sh | `refuses rehearsal-required` |
|
||||
| m8 drop the `rehearsal-skipped` line | stack-upgrade.sh | `a real rehearsal with no backup` |
|
||||
|
||||
## Deviations
|
||||
|
||||
D1. The brief has A1 refuse `open-activation` for an open `AWAITING-UNITS` or a non-empty `interrupted`
|
||||
set. The interrupted case refuses with row 232's existing code, `interrupted-run`; only `AWAITING-UNITS`
|
||||
gives `open-activation`. Both reach root as `plan-refused`, exit 3 (sequence 6, the open-activation test).
|
||||
|
||||
D2. Root copies one line of the stack user's output into its log, `BACKUP_STORE`, on sage-1653. It
|
||||
matches a fixed pattern first, takes the last match, logs `BACKUP_STORE none` otherwise, and decides
|
||||
nothing on it.
|
||||
|
||||
D3. The stack user's steps carry no `--instance`. The driver takes `--instance dev` only, row 190's default
|
||||
instance, which the stack user's commands select with no flag. A named instance's units are a later row.
|
||||
|
||||
D4. Most tests fake the rehearsal in the fake `sudo`: the fixture's stub copies can't pass a real one. Two
|
||||
tests run the real rehearsal (`faults/real-rehearse`): no-backup and backup-last. `stack-release.test.mjs`
|
||||
covers `--backup last` itself.
|
||||
|
||||
D5. Any nonzero `--plan` exit refuses `plan-refused`, exit 3, whatever the code. The stack user's output
|
||||
above it names the refusal; root doesn't parse it.
|
||||
|
||||
D6. `rollback --restore ACT` is accepted without `--events`, as `stack-release.sh rollback` accepts it.
|
||||
The brief's usage pairs them. `--events` without `--restore` is a usage error.
|
||||
|
||||
D7. The refusal of any `STACK_UPGRADE_*` seam when run as root has no test, because the tests don't run
|
||||
as root. It is three lines at the top of the script.
|
||||
|
||||
D8. The fake `sudo` keeps the test's environment, where real `sudo` resets it. The driver passes
|
||||
everything the stack user needs through `env PATH=...` and `-H`, and the fake checks that form.
|
||||
|
||||
D9. Exit 3 means nothing stopped, not nothing changed. A refusal after `fetch` leaves the stack user's
|
||||
clone fast-forwarded and the new copy made. Both are the user's own state, and the next run reuses them.
|
||||
TOOLS.md and the run sheet say so.
|
||||
|
||||
D10. The brief's usage has neither `--no-rehearsal` nor `--rehearse-port`. Both come from sage-1650 and
|
||||
sage-1652.
|
||||
|
||||
## Decisions
|
||||
|
||||
- `rollback` has no HEAD check. Its target is an older copy than root's checkout, which already exists.
|
||||
It fetches, copies and rehearses nothing (sequence 7).
|
||||
- The new args file is written next to the old one (`.dev.args.XXXXXX`, same mode) before A2, so a full
|
||||
`/etc` refuses before anything stops. It moves into place after A5. A2f and the rollback remove it; a failed move keeps it and names it.
|
||||
- Every stack user step gets `--copies` explicitly, so the copies directory comes from the args file's
|
||||
`--home`, never from the user's environment.
|
||||
- The stack's user does the fetch, the fast-forward and the copy, so root never runs git on a tree that
|
||||
user can write.
|
||||
- A4 is the render, and `case2-unit.sh` starts the bus, then the Console. The forward path has no
|
||||
`systemctl start` of its own; only A2f does.
|
||||
|
||||
## Run environment
|
||||
|
||||
Worktree `/mnt/storage/scratch/darkwing-r233`, `TMPDIR=/mnt/storage/scratch/tmp/r233`, node from PATH. One
|
||||
whole `stack-upgrade.test.mjs` run takes about 56 s.
|
||||
|
||||
## Results
|
||||
|
||||
### Singles, run 1, 2026-10-21, Sage's slot (sage-1689)
|
||||
|
||||
`~/darkwing-scratch/r233/singles.sh` on `darkwing/row233-r1` (base e65a3ab4d, uncommitted tree), 03:48:20Z
|
||||
to 03:59:00Z. These are the test files that read a changed script. `stack-instances.test.mjs` reads
|
||||
`stack-release.sh`, so it joined the three named in the plan.
|
||||
|
||||
| File | Result |
|
||||
|---|---|
|
||||
| `stack-upgrade.test.mjs` | 20 pass, 0 fail |
|
||||
| `stack-release.test.mjs` | 194 pass, 1 fail |
|
||||
| `stack-release-external.test.mjs` | 25 pass, 0 fail |
|
||||
| `stack-instances.test.mjs` | 23 pass, 0 fail |
|
||||
|
||||
The failure was my test, not the script. "row 233: --backup last takes the newest backup..." compares
|
||||
`untouched(e)` before and after the run, and `untouched` includes `existsSync(e.log)`. The test took its
|
||||
before snapshot ahead of `rlog()`, which writes the log, so that flag went from false to true. Every hash in
|
||||
the snapshot matched. The fix takes the snapshot after `rlog()`. No other row 233 test calls `untouched`.
|
||||
|
||||
### Singles, run 2
|
||||
|
||||
Pending: `stack-release.test.mjs` whole, after the fix. Then the gate and the mutants run in Sage's slot.
|
||||
Reference in new issue
Block a user