Compare commits
25
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4c90099ea9 | ||
|
|
c515273db7 | ||
|
|
da3051ccc6 | ||
|
|
060a17b549 | ||
|
|
ffd90e7fc4 | ||
|
|
02709125a6 | ||
|
|
c771a36f4b | ||
|
|
caa4b59440 | ||
|
|
57f59e0c39 | ||
|
|
5465fd52d5 | ||
|
|
b00d7d94d0 | ||
|
|
642a9b87de | ||
|
|
16019b80e6 | ||
|
|
9b53620185 | ||
|
|
7662b04902 | ||
|
|
04ce8dfd56 | ||
|
|
fc1cb856b0 | ||
|
|
c0c4907af9 | ||
|
|
5fb736b92d | ||
|
|
96bac588fb | ||
|
|
5d4a617762 | ||
|
|
e65a3ab4d2 | ||
|
|
0444acd084 | ||
|
|
052ba95c7c | ||
|
|
a52d64ed41 |
@@ -5175,3 +5175,5 @@ Before: managed project/thread verbs had no Console conversation area. After: fi
|
||||
|
||||
2026-10-20 05:10:34 UTC | Row 178 #1676 round 2 verified | Before: C1 render refusals lacked killing tests and row 181 base was pending. Rebased onto 931651f2d, kept all COUNTED/count code and tests, appended T178 tail; added independent owner/collision/absolute/charset/header and empty-loopback proof tests. Four files singly 156/156, all 24 mutants killed, metadata and sandbox fixture probes pass; scratch sweep empty. Packet agents/rocko/work/queue-178/r2/, candidate 9300a670, inventory d0dcde7b (59/59), scan 58 files no findings; in-review revs 2166-2167, expected no-token request failure. No live actions or commits.
|
||||
2026-10-21T01:28:05Z | Darkwing | row 232 #1734 installer I1 build, round 1 in review | Before: stack-release.sh managed only user units (systemctl --user), and stack-copy.sh had no bundle copy. After: --units external (adopt, activate, rollback --to, status, confirm, revert, gate --since, AWAITING-UNITS, stale-start, external-pm, --plan --expect-from) and stack-copy check --units external, copy --bundle FILE SHA, wrapper ID --create; user mode byte-identical to the base by test. First build 0af0c73c6 on 00d4cc1f6; rebased onto row 200 (cb5fdb064) as a0ffabc3f on darkwing/row232-r1; --sso refused under external (D9), row 200 config_checks run under external (D10). Gate on Sage's slot (sage-1618): cli 561/0, every scripts/test-*.sh green; 17/17 mutants killed. Results commit 84f1a91e6 (pushed, seat branch only). Manifest 7a3b4ea1 (9 paths), queue revs 2595-2596, request comment 29825, reviewer Filbert. Packet agents/darkwing/work/queue-232/BUILD.md.
|
||||
2026-10-21T04:50:44Z | Darkwing | row 233 #1735 installer I3 build, round 1 in review | Before: root had no upgrade driver for root-owned units and no host run sheet; stack-release.sh had no rehearse --backup last and no external plan refusals. After: scripts/stack-upgrade.sh upgrade and rollback (r6 A1-A5, A2f, R1-R4) with 20 tests, docs/guides/host-run-sheet.md, TOOLS.md lines, and on Sage's rulings stack-release.sh --backup last, BACKUP_STORE and the --plan refusals open-activation and migrate-required (3 tests). Base e65a3ab4d (row 232's landing). Gate on Sage's slot (sage-1704) on d99166ade: cli 616/0, every scripts/test-*.sh green, 1201/0 summed by awk; m1-m8 8/8 killed. Correction: the process sweep after the gate found an orphaned sleep 60 from the lock test's flock holder (N1); 08db6ac3f fixes it test only, Sage ruled the gate stands (sage-1707), rerun c0 20/0 and m1 killed with 0 left (sage-1708). Tip 1a1dc80ca (pushed, seat branch only). Manifest 464ce9eb (7 paths), queue revs 2644 and 2647, request comment 29901, reviewer Filbert. Packet agents/darkwing/work/queue-233/BUILD.md.
|
||||
2026-10-21T06:37:41Z | Darkwing | row 240 #1746 installer I4 build, round 1 in review | Before: root had no host check for the bwrap AppArmor profile, its pins or a boot unit, and stack-upgrade.sh exited 1 args-failed when the args move after A5 failed, while exit 1 means the old copy runs. After: scripts/case1-hostcheck.sh (seven items, --check, --enforce, --upstream), scripts/case1-pins.sh, mosaic-hostcheck.service and 16 tests; the args move failure exits 5 args-failed, documented in TOOLS.md and run sheet section 6 and tested; run sheet section 3 puts the pinned npm on PATH and names the bin/claude.exe --claude path; sections 8 and 10 have precheck lines. Verification: slot run r1 on 47ec9fe2f failed at the singles (the new N1 test's after hook threw ENOENT after the fixture cleanup removed the root); b45852daa drops the hook; r2 on b45852daa green: singles 16/0 and 21/0, c0 plus M1-M8 8/8 killed, cli 639/0, scripts 585/0. Candidate darkwing/row240-r3 ad1b23ab2 on c771a36f4, manifest 573ece95 (9 paths), revs 2694 and 2695, comment 29943.
|
||||
|
||||
@@ -0,0 +1,287 @@
|
||||
# Row 232 (I1): stack-release external units and bundle copies
|
||||
|
||||
Brief: `docs/plans/2026-10-20_installer-rows.md` section I1, blob `f35e7e57daee` (commit ad6f0be7a). Gitea
|
||||
#1734, Vikunja P19 #432. Owner Darkwing, reviewer Filbert, gate owner Sage. Design input: row 195 DESIGN r6,
|
||||
`agents/darkwing/work/queue-195/DESIGN.md` at b7239bb66 (approved in duckworth-0055). Where the brief and
|
||||
r6 differ, the brief wins. sage-1561 ruled where the stack user's `stack-release.sh` runs: from its own
|
||||
clone of the release bundle, so `check_target`, the plan's commit list and status drift are unchanged and
|
||||
nothing adds `safe.directory`.
|
||||
|
||||
## Files
|
||||
|
||||
Base: cb5fdb064 (row 199's landing, on top of row 200's 9e1085816). Branch `darkwing/row232-r1`. The first
|
||||
build, 0af0c73c6 on `darkwing/row232`, sat on 00d4cc1f6 (row 190); row 200 landed first, so this one is
|
||||
rebased onto it (see D9 and D10).
|
||||
|
||||
- `scripts/stack-release.sh`: `--units user|external`, the `sc` and `jctl` backend, `confirm`, `revert`,
|
||||
`gate --since`, `rollback --to`, `--plan --expect-from`, the external gate (start floor, PM check from
|
||||
`launches list`), `external-pm`, and the `awaiting units:` status line.
|
||||
- `scripts/stack-copy.sh`: `check --units external`, `copy --bundle FILE SHA`, `wrapper ID --create`.
|
||||
- `packages/cli/tests/stack-release-external.test.mjs` (new, 25 tests).
|
||||
- `packages/cli/tests/stack-release-fixture.mjs` (new): the helpers `stack-release.test.mjs` defined, moved
|
||||
so both files share them. See deviation D1.
|
||||
- `packages/cli/tests/stack-release.test.mjs`: the helpers leave and one import line comes in. The 74 tests
|
||||
have the same names and the same bodies as on the base.
|
||||
- `packages/cli/tests/stack-copy.test.mjs`: three new `row 232:` tests and a wider `node:fs` import.
|
||||
- `docs/TOOLS.md`: the stack-copy usage block and three paragraphs, the stack-release usage block, and a new
|
||||
subsection "Root-owned units (`--units external`, row 232)".
|
||||
- `packages/cli/README.md`: one paragraph.
|
||||
- This file.
|
||||
|
||||
The worktree's `node_modules` symlink is not part of the candidate.
|
||||
|
||||
## User mode is unchanged
|
||||
|
||||
Every `systemctl` call goes through `sc` and every `journalctl` call through `jctl`. Under `user` they
|
||||
are `systemctl --user "$@"` and `journalctl --user "$@"`. Under `external`, `sc` allows only `show`,
|
||||
`is-active` and `list-units`, and `jctl` returns 1, so an external run can't start, stop, install or read
|
||||
the journal even through a path the tests miss.
|
||||
|
||||
Two tests hold the brief's "every command line and every byte of output":
|
||||
|
||||
- `user mode: every systemctl call carries --user, and activate, rollback and check make the base's
|
||||
calls`. A fixture repo gets the base's two scripts from `git show cb5fdb064:`. The same activate,
|
||||
rollback and `stack-copy.sh check` run from both trees, and the recorded argv (with the fixture root and
|
||||
port replaced by placeholders) must match line for line. Every call carries `--user`.
|
||||
- `user mode: a default activate --plan prints the base's bytes`, for four plans.
|
||||
|
||||
The base output is produced live from the base blobs, not stored as a fixture file. See deviation D3.
|
||||
|
||||
## r6 section 7, test by test
|
||||
|
||||
All in `stack-release-external.test.mjs` unless named.
|
||||
|
||||
| r6 :742-812 item | Test |
|
||||
|---|---|
|
||||
| `units-running` (N4), store, log and TAKEN left | `units-running (N4) refuses while a unit is active...`; also checks that `failed` counts as down and that the Console is checked first |
|
||||
| activate leaves the pointer, writes `AWAITING-UNITS` and a backup | `activate stages: ...`, which also checks the set's file list, `UNITS.sha256`, no `--user`, no journal call and `status` |
|
||||
| `open-activation` with zero, two, the wrong copy, one not open | `confirm and revert refuse open-activation ...` |
|
||||
| `confirm` polls: 3 s fails at `--timeout 1`, passes at `--timeout 10` | `confirm polls: bus status good after 3 s fails ...` and `confirm polls to a pass ...` |
|
||||
| `stale-start` floor with `started` at .900 | `stale-start: @1792540799 fails ...` and `stale-start is a gate code ...` |
|
||||
| `confirm` pass writes pointer, ok, removes marker; timeout writes `FAILED` and exits 1 | the two polling tests |
|
||||
| `activation-failed` when `FAILED` exists | `confirm polls: ... FAILED then refuses activation-failed` |
|
||||
| lines carry the started op, from, to, activation and backup | `sameAsStarted` in every confirm test, and sequence 7 |
|
||||
| `gate` (N2) with and without `--since` | `gate (N2, A2f): ...` and the usage test |
|
||||
| `--plan --expect-from` (N5), activate and `rollback --to` | sequence 7 |
|
||||
| A3 order: capture-failed, marker-failed, wrapper failure after migrate | `A3 order: ...` and sequence 2 |
|
||||
| refusals log no activation; `interrupted` unchanged | `open-activation` test and sequence 5 |
|
||||
| PM check: one, none, two, early, not live, other business, other instance; no journal call | `the external PM check reads launches list ...` (modes '', coder, none, twolive, early, dead, other) and `journalCalls` in every external test |
|
||||
| `external-pm` | `pm-on and activate --pm-after-harness refuse external-pm` |
|
||||
| `revert` restores TAKEN only after a migration | `revert puts the wrapper back and leaves the store ...` and sequences 2 and 8 |
|
||||
| `confirm --rollback` per cause, `unrecorded`, `rollback-failed`, mismatch | sequences 1, 2, 3, 5 and 8, and `confirm --rollback refuses rollback-mismatch, and logs rollback-failed ...` |
|
||||
| sequences 1-8 | `sequence 1:` to `sequence 8:`, each ending with the `closed` check (one started and one result line, empty `interrupted`, no marker or `FAILED`, a following `--plan` that doesn't refuse `interrupted-run`) |
|
||||
| `rollback --to` refuses not-previous; `rollback --restore` finds an external backup | sequence 7, and `confirm --rollback after a --restore swap ...` |
|
||||
| `gate` writes nothing | the `gate (N2, A2f)` test and sequence 4 compare the log, the pointer and the state tree |
|
||||
| `confirm`, `revert`, `gate`, `--to` refuse without `--units external` | `confirm, revert, gate, --to and --expect-from need --units external ...` |
|
||||
| default path: every call carries `--user`; default `--plan` bytes | the two user-mode tests above |
|
||||
|
||||
The brief's additions, in `stack-copy.test.mjs`:
|
||||
|
||||
- `row 232: copy --bundle copies a SHA the bundle's refactor reaches, byte-identical to copy SHA, and
|
||||
refuses the rest`: a short SHA, a missing file, a symlink, a bundle that fails `git bundle verify`, a
|
||||
bundle with no `refs/heads/refactor`, a SHA not in the bundle, and a SHA the bundle carries on another
|
||||
branch that refactor doesn't reach. The bundle copy's tree matches `copy SHA`'s byte for byte, and the
|
||||
`release:` and `bundle sha256:` lines are checked.
|
||||
- `row 232: wrapper --create writes the first wrapper only, the same bytes wrapper ID writes, and refuses
|
||||
over any file`, a symlink included. Without `--create` a missing wrapper still refuses.
|
||||
- `row 232: check --units external reads the system units, the default reads the user units`.
|
||||
|
||||
## Mutants
|
||||
|
||||
`mutants.sh` in my scratch (not committed, it edits the worktree), 17 mutants: r6's re-pin list (:817-838)
|
||||
minus the three `case1-repin.sh` ones, plus the brief's two. Each one applies exact replacements after a
|
||||
checked backup, runs `bash -n`, runs its named killers, restores the file and checks its sha256. Two
|
||||
controls run first with no change: the whole external file, and stack-copy's `row 232:` tests.
|
||||
|
||||
| Mutant | Killers |
|
||||
|---|---|
|
||||
| m01 `confirm` writes the pointer on a gate failure | `confirm polls: bus status` |
|
||||
| m02 `gate_once` in place of `gate` | `confirm polls to a pass`, `stale-start is a gate code` |
|
||||
| m03 PM check counts any instance | `external PM check` |
|
||||
| m04 PM check drops the `startedAt` bound | `external PM check` |
|
||||
| m05 `confirm` takes the newest `AWAITING-UNITS` | `open-activation` |
|
||||
| m06 `rollback --to` skips the comparison | `sequence 7:` |
|
||||
| m07 the default path drops `--user` from one call (`is-active`) | `user mode: every systemctl` |
|
||||
| m08 the timeout writes r4's interim `rollback pending` line | `confirm polls: bus status`, `sequence 1:`, `sequence 5:` |
|
||||
| m09 `AWAITING-UNITS` last, after the wrapper | `sequence 2:` |
|
||||
| m10 a refusal names the open activation | `sequence 5:`, `open-activation` |
|
||||
| m11 `stale-start` is a refusal | `stale-start is a gate code` |
|
||||
| m12 `revert` restores without `MIGRATING` | `revert puts the wrapper back` |
|
||||
| m13 `confirm` logs op `confirm` | `sequence 7:`, `confirm polls to a pass` |
|
||||
| m14 `MIGRATING` after the migration | `sequence 8:` |
|
||||
| m15 `gate` takes its floor from the log without `--since` | `gate (N2` |
|
||||
| m16 `copy --bundle` skips the reachability check | stack-copy `row 232:` |
|
||||
| m17 `wrapper --create` overwrites | stack-copy `row 232:` |
|
||||
|
||||
## Deviations and choices
|
||||
|
||||
D1. The shared helpers moved to a new file, `packages/cli/tests/stack-release-fixture.mjs`. The brief's
|
||||
files list names `stack-release.test.mjs` "where a shared helper moves" but not a destination. Importing
|
||||
from the other test file would run its 74 tests twice, so the helpers need a module of their own.
|
||||
|
||||
D2. The fixture's T17 check (what a run may never remove) now exempts a rollback set's `AWAITING-UNITS` and
|
||||
`FAILED`. `confirm` removes both when it closes the activation (r6 A5 and R4). User mode writes neither, so
|
||||
user-mode runs check exactly what they did.
|
||||
|
||||
D3. The user-mode argv and `--plan` tests read the base's scripts with `git show cb5fdb064:` rather than
|
||||
storing their output as a fixture file. A later row that changes user-mode output on purpose moves the
|
||||
`BASE` constant in the test to its own base, or the test fails. I chose this over a stored fixture because
|
||||
a stored copy can drift from the base without anyone noticing.
|
||||
|
||||
D4. `marker-failed` is reached by making `state/stack-rollback` unwritable. r6 says "an existing `$RS`", but
|
||||
`next_activation` never picks a taken name, so no test can make the run meet an existing `$RS`. The
|
||||
exclusive `mkdir` is still what refuses.
|
||||
|
||||
D5. The stale-start polling case turns the enter time to `@1792540800`, not r6's `@1792540805`, with the
|
||||
started line at 1792540800.900. That also checks that the floor is the whole second, so .900 doesn't push it
|
||||
to 1792540801.
|
||||
|
||||
D6. No fake `sudo` or `case2-unit.sh`. r6's sequences ran under them because r6 had `case1-repin.sh` drive
|
||||
the steps, and that script is I3's. Here the test plays root: it stops and starts the fake system units and
|
||||
installs with the existing `bus-service.sh` and `webui-service.sh` stand-ins, between the real
|
||||
`stack-release.sh` calls, on one log.
|
||||
|
||||
D7. The fake marker in the `open-activation` test is named `20000101T000000Z-<C1 prefix>` so it sorts
|
||||
before the real one. A marker that sorted after it would leave m05 (newest marker) alive.
|
||||
|
||||
D8. m10 sets `ACT` inside `open_activation`, where r6's wording ("ACT set before the checks") lands in this
|
||||
code. `ACT` is set only after every check in `confirm`.
|
||||
|
||||
D9. Row 200's `--sso` is a usage error under `--units external`, with the proof flags, `--no-proof` and
|
||||
`--no-send`. It is a Console flag, and root's Console unit carries the Console flags; the script under
|
||||
external renders no Console line and never calls `console_for_target`, so `sso_checks` doesn't run there
|
||||
either. The usage text, the TOOLS.md bullet and the external usage test name it.
|
||||
|
||||
D10. Row 200's `config_checks` runs under `--units external` too, after `plugin_checks`, in both the
|
||||
`--plan` and run paths of `activate` and `rollback --to`. It checks the target's loader against the stack
|
||||
user's `config.json`, which the bus on a VM still reads, so a `console` key the target refuses would break
|
||||
root's bus start the same way. The cost is one more refusal (`config-console-key`) an external run can
|
||||
print; I took it over a bus that fails after root's start. The rebase changed no user-mode line: the
|
||||
conflicts were the `SSO=0` globals line, the `plan` body (row 200's `SSO_PLAN` line sits in the user
|
||||
branch) and the two `--plan` lines that gained `config_checks`. In the tests, row 200's fixture lines
|
||||
(the loaders, `sso-config.mjs` and the `--sso` cli.mjs line for C1 and C) moved into
|
||||
`stack-release-fixture.mjs` with the rest of the copy setup, and the `ALGS` import stays in
|
||||
`stack-release.test.mjs`, which T57 uses.
|
||||
|
||||
## duckworth-0055's notes on r6 (brief line 169)
|
||||
|
||||
Duckworth's r6 approval (REVIEW-r5.md at 23f5520b5) carried two notes.
|
||||
|
||||
- N1, A2's stop timeout under `gate --since`. A unit that never stopped keeps its old `ActiveEnterTimestamp`,
|
||||
so `gate --since` fails it with `stale-start`, and the run ends at exit 2 with root stopping the units.
|
||||
That is intended: a unit that won't stop is a fair reason to stop. In I1 this is the gate code and its
|
||||
polling (`stale-start: @1792540799 fails ...`, `gate (N2, A2f)`, m11). The run sheet's A2f line and the
|
||||
re-pin test that points "exits 1 with OLD's gate passing" at an A3 exit 3 belong to I3, which owns
|
||||
`scripts/stack-upgrade.sh` and `docs/guides/host-run-sheet.md`. I'll name both in the I3 build.
|
||||
- N2, the claude path under `/opt/mosaic-stack` (gosalyn-0058, sage-1522). I1 doesn't touch it. Under
|
||||
`--units external` the script checks no claude pin of its own (`no-claude-pin` applies only to user
|
||||
mode), because root's unit carries it. The `--claude` value, the managed-settings reasoning, F6 and the
|
||||
run sheet's claude step go to I2 and I3.
|
||||
|
||||
## Run environment
|
||||
|
||||
Worktree `/mnt/storage/scratch/darkwing-r232`, `TMPDIR=/mnt/storage/scratch/tmp/r232`, node from PATH.
|
||||
One external file run takes about 94 s.
|
||||
|
||||
## Results
|
||||
|
||||
Singles on the rebased candidate, before the gate:
|
||||
|
||||
| File | Tests | Pass | Fail |
|
||||
|---|---|---|---|
|
||||
| `packages/cli/tests/stack-release-external.test.mjs` | 25 | 25 | 0 |
|
||||
| `packages/cli/tests/stack-copy.test.mjs` | 16 | 16 | 0 |
|
||||
| `packages/cli/tests/stack-release.test.mjs` (row 200's T57 included) | 192 | 192 | 0 |
|
||||
| `packages/cli/tests/stack-instances.test.mjs` | 23 | 23 | 0 |
|
||||
| `packages/webui/tests/pinned-copy.test.mjs` | 10 | 10 | 0 |
|
||||
| `packages/cli/tests/channel-recovery.test.mjs` (imports `channel-fixture.mjs`, which names the scripts) | 1 | 1 | 0 |
|
||||
| `packages/cli/tests/host.test.mjs` (reads `packages/cli/README.md`) | 42 | 42 | 0 |
|
||||
|
||||
These are every test file that reads `stack-release.sh` or `stack-copy.sh` (a grep of `packages/*/tests` and
|
||||
`scripts/`). None of them pins a sha256 of a changed file. `packages/cli/tests/host.test.mjs` reads
|
||||
`packages/cli/README.md` for a unit file name; this row only adds a paragraph there. The grep and the runs
|
||||
were repeated on the rebased tree on 2026-10-21; the reader set is the same. A dry run of `mutants.sh`
|
||||
matches each of the 17 replacements once on the rebased scripts.
|
||||
|
||||
### Gate, 2026-10-21, Sage's slot (sage-1618)
|
||||
|
||||
Candidate a0ffabc3f on `darkwing/row232-r1`, run by `~/darkwing-scratch/r232/gate.sh`: the cli suite, then
|
||||
every `scripts/test-*.sh`, one after another, stopping at the first failure. busy.sh passed at 01:07:49Z
|
||||
after Launchpad's single `stack-release.test.mjs` run ended. The gate stopped at the first one, so the
|
||||
chain waited for that PID to exit and checked again. The EXIT trap removed the compose network; no
|
||||
container was left.
|
||||
|
||||
| Step | Time (UTC) | Result |
|
||||
|---|---|---|
|
||||
| `packages/cli`, `node --test tests/*.test.mjs` | 01:07:50 to 01:17:53 | 561 pass, 0 fail, 0 skipped |
|
||||
| `scripts/test-auth.sh` | 01:17:53 to 01:17:54 | 15 passed, 0 failed |
|
||||
| `scripts/test-conductor.sh` | 01:17:54 to 01:18:00 | 17 passed, 0 failed |
|
||||
| `scripts/test-config.sh` | 01:18:00 to 01:18:02 | 31 passed, 0 failed |
|
||||
| `scripts/test-discord.sh` | 01:18:02 to 01:18:12 | 78 passed, 0 failed |
|
||||
| `scripts/test-extension-package.sh` | 01:18:12 to 01:18:13 | 18 passed, 0 failed |
|
||||
| `scripts/test-foundation.sh` | 01:18:13 to 01:19:01 | 44 passed, 0 failed |
|
||||
| `scripts/test-land-work.sh` | 01:19:01 to 01:19:14 | 86 passed, 0 failed |
|
||||
| `scripts/test-queue.sh` | 01:19:14 to 01:19:37 | 156 node tests pass, queue suite 28 passed, 0 failed |
|
||||
| `scripts/test-release.sh` | 01:19:37 to 01:19:51 | 14 passed, 0 failed |
|
||||
| `scripts/test-task.sh` | 01:19:51 to 01:20:44 | 98 passed, 0 failed |
|
||||
|
||||
### Mutants, 2026-10-21, same slot
|
||||
|
||||
`~/darkwing-scratch/r232/mutants.sh`, right after the gate. Both controls (the external file whole, and
|
||||
stack-copy's `^row 232: ` tests) exited 0 on the unmutated tree. Each mutant was applied after a checked
|
||||
backup and a syntax check, ran its killers from the table above, and was put back with its sha256
|
||||
checked. The script stops on any restore mismatch, and it ran to its end.
|
||||
|
||||
| Mutant | Killer exit | Result |
|
||||
|---|---|---|
|
||||
| m01 pointer on gate failure | 1 | killed |
|
||||
| m02 gate once | 1 | killed |
|
||||
| m03 PM from any instance | 1 | killed |
|
||||
| m04 no startedAt check | 1 | killed |
|
||||
| m05 newest marker | 1 | killed |
|
||||
| m06 --to not compared | 1 | killed |
|
||||
| m07 one call without --user | 1 | killed |
|
||||
| m08 interim pending line | 1 | killed |
|
||||
| m09 marker written last | 1 | killed |
|
||||
| m10 refusal names the open activation | 1 | killed |
|
||||
| m11 stale-start refuses | 1 | killed |
|
||||
| m12 revert always restores | 1 | killed |
|
||||
| m13 confirm op | 1 | killed |
|
||||
| m14 MIGRATING after migrate | 1 | killed |
|
||||
| m15 gate floor from the log | 1 | killed |
|
||||
| m16 bundle reachability unchecked | 1 | killed |
|
||||
| m17 --create overwrites | 1 | killed |
|
||||
|
||||
17 of 17 killed, 01:22:16 to 01:24:32Z. I read the logs for m01 and m09, two of the quick kills: each failed on
|
||||
its named test's assertion (m09 on `AWAITING-UNITS` in sequence 2), not on a load error.
|
||||
|
||||
## Round 2: rebase onto 8fede5fc6 (sage-1673)
|
||||
|
||||
Rebase only. Before the landing gate, a check of BASE..HEAD on the manifest paths found that row 230
|
||||
(c8bc5cf4c) had changed one of them after the round 1 base, cb5fdb064. Sage ruled a re-round with a
|
||||
scoped review, the same shape as row 210 round 3.
|
||||
|
||||
Branch `darkwing/row232-r2`: the two round 1 commits, a0ffabc3f and 84f1a91e6, cherry-picked onto
|
||||
8fede5fc6 with no conflict, then this section. No code or test changed.
|
||||
|
||||
- `git merge-tree --write-tree origin/refactor darkwing/row232-r1` (refactor at 8fede5fc6) gives tree
|
||||
837bde79236c with no conflict. The cherry-picked tip before this section has the same tree.
|
||||
- `packages/cli/README.md` is a clean 3-way merge. Row 230's hunk is at lines 710-717 (the session credential
|
||||
paragraph); this row's is a new paragraph at 773. With `git patch-id --stable`, the README diff from r1 to
|
||||
r2 is 02b679e2, the same as row 230's README diff `cb5fdb064..8fede5fc6`. This row's README diff is
|
||||
c395ecd2 on both bases (`cb5fdb064..r1` and `8fede5fc6..r2`).
|
||||
- The seven code, test and doc paths other than the README hash the same as in manifest 7a3b4ea1, which
|
||||
Filbert approved in round 1: `docs/TOOLS.md`, `packages/cli/tests/stack-copy.test.mjs`,
|
||||
`packages/cli/tests/stack-release-external.test.mjs`, `packages/cli/tests/stack-release-fixture.mjs`,
|
||||
`packages/cli/tests/stack-release.test.mjs`, `scripts/stack-copy.sh`, `scripts/stack-release.sh`.
|
||||
This file also matched until this section was appended. Its other lines are unchanged.
|
||||
- The one test that reads the README, `host.test.mjs:855`, looks for the unit's `EnvironmentFile=` line.
|
||||
That line is unchanged context in row 230's hunk.
|
||||
- No row 230 test reads `stack-release.sh` or `stack-copy.sh`, and no test of this row reads a row 230 file.
|
||||
|
||||
Nothing was run for this round. The landing gate runs on the r2 tip on Sage's slot.
|
||||
|
||||
Follow-up, doc-only (sage-1673): `scripts/case2-unit.sh:108` and `scripts/case2-precheck.sh:157` cite
|
||||
`scripts/stack-copy.sh:8` and `:22` for `COPIES` and `copy_root`. With this row those lines are `:10` and
|
||||
`:26`. Only comments are affected and no test reads them. Row 233 doesn't touch those scripts, so the fix
|
||||
goes on the #1732 carry list.
|
||||
@@ -0,0 +1,249 @@
|
||||
# Row 233 (I3): root's upgrade driver and the host run sheet
|
||||
|
||||
Brief: `docs/plans/2026-10-20_installer-rows.md` section I3, blob `e61291bcf1c6` (commit a0e3d7cfd), re-pinned
|
||||
by Sage (sage-1685, rev 2631) from `f35e7e57daee` (ad6f0be7a). The later blob only appends I4; the I3 text
|
||||
is the same. Gitea #1735, Vikunja P19
|
||||
#432. Owner Darkwing, reviewer Filbert, gate owner Sage. Design input: row 195 DESIGN r6,
|
||||
`agents/darkwing/work/queue-195/DESIGN.md` at b7239bb66, sections 2, 4 and 7. Where the brief and r6
|
||||
differ, the brief wins. Sage's rulings during the build: sage-1650 and sage-1652 (`--backup last`, the
|
||||
`--no-rehearsal` mapping, `--rehearse-port`, hostcheck), and sage-1653 (`from_log newest`, the two external
|
||||
plan refusals, the `BACKUP_STORE` line).
|
||||
|
||||
## Files
|
||||
|
||||
Base: e65a3ab4d, row 232's landing, on `darkwing/row233-r1`. The build started on ea1e85c7b
|
||||
(`darkwing/row233`, row 232's candidate e6ca230d2 and its BUILD.md) before row 232 landed. The three row 232
|
||||
files this row changes are identical on the two bases, so the uncommitted diff moved across unchanged.
|
||||
|
||||
- `scripts/stack-upgrade.sh` (new): `upgrade` and `rollback`, root's side of r6 A1-A5, A2f and R1-R4.
|
||||
- `packages/cli/tests/stack-upgrade.test.mjs` (new, 20 tests).
|
||||
- `docs/guides/host-run-sheet.md` (new).
|
||||
- `docs/TOOLS.md`: a new subsection "Root's upgrade driver (`scripts/stack-upgrade.sh`, row 233)", plus the
|
||||
row 232 lines below.
|
||||
- Row 232 files, changed on Sage's rulings (the brief's files list doesn't name them):
|
||||
- `scripts/stack-release.sh`:
|
||||
- `rehearse --backup last` takes the newest backup the log records (`from_log newest`) and prints
|
||||
`PASS backup-last`. With none it fails `no-backup`, exits 3 and makes no rehearsal directory.
|
||||
A newest backup that isn't an activation id aborts `backup-unreadable`, exit 1.
|
||||
- The rehearsal prints `BACKUP_STORE <activation id> schema <version>` before its migration step.
|
||||
- `--plan` under `--units external` refuses `open-activation` when any rollback set holds
|
||||
`AWAITING-UNITS`, and `migrate-required` for a migrating target without `--migrate`. Root reads
|
||||
only the plan's exit, so these two have to be refusals there. User mode prints the same bytes.
|
||||
- `packages/cli/tests/stack-release.test.mjs`: three new `row 233:` tests for `--backup last`.
|
||||
- `docs/TOOLS.md`: `--backup ACTIVATION|PATH|last` in the rehearse usage, a paragraph on `last`, and a
|
||||
paragraph on the two external plan refusals.
|
||||
- This file.
|
||||
|
||||
## The brief, item by item
|
||||
|
||||
| Brief item | Where |
|
||||
|---|---|
|
||||
| Root runs from a root-owned checkout, never a copy | `head-mismatch` (A1); TOOLS.md; run sheet section 6 |
|
||||
| I2's render with `/etc/mosaic-stack/<instance>.args`, `--copy` the one changed value | `read_args`, `args_with`; the args test |
|
||||
| The args file changes only after a confirmed run | `prepare_args` before A2, `mv` after A5; sequences 1 and 7, m6 |
|
||||
| `copy --bundle` as the stack's user before A1 | the `fetch`, `ff`, `copy` steps; sequence 1's sudo keys |
|
||||
| A rehearsal before A2, a failed one refuses with nothing stopped | `rehearse`; three rehearsal tests, m4 |
|
||||
| The root lock, held to the end, rollback included | `flock -n 9`; the lock test and sequence 3's probe, m1, m2 |
|
||||
| A1 refuses `open-activation` | stack-release.sh's external plan; the AWAITING-UNITS test, m3 |
|
||||
| `migrate-required` without `--migrate` | stack-release.sh's external plan; its test, m5 |
|
||||
| The step log, 0640, one line per step | `logl`, `step`; every test checks lines against `STEP` |
|
||||
| Exit codes 0-3 | `finish`, `refuse`; exit 4 is usage |
|
||||
| duckworth-0055's notes | the next section |
|
||||
| Root parses no stack user output beyond exits | the step log test; the one exception is sage-1653's `BACKUP_STORE` |
|
||||
| The host run sheet | `docs/guides/host-run-sheet.md` |
|
||||
|
||||
## duckworth-0055's notes on r6 (brief line 169)
|
||||
|
||||
- N1, A2's stop timeout under `gate --since`. Row 232's BUILD.md left two pieces to I3. The run sheet's
|
||||
A2f line is in section 6 ("When a run doesn't exit 0", exit 2). The test is `N1: a bus that outlives
|
||||
A2's timeout keeps its old start`: a bus that ignores `stop` keeps its old `ActiveEnterTimestamp`,
|
||||
`wait-down` logs `rc=1 ... state: active`, A2f restarts and gates, the gate fails `stale-start`, and the
|
||||
driver stops both units and exits 2. Sequence 4 is the other half: an A3 exit 3 goes to A2f and exits
|
||||
1 with the old copy's gate passing.
|
||||
- N2, the claude path under `/opt/mosaic-stack`. Run sheet section 3 installs it there as root with
|
||||
managed settings. The args file's `--claude` carries it, and `case2-unit.sh` already refuses a binary
|
||||
that isn't root's or sits under the home. The driver passes the args file through and adds no claude
|
||||
check of its own. F6 is I2's.
|
||||
|
||||
## Sage's rulings during the build
|
||||
|
||||
- sage-1650 and sage-1652. Root always runs the rehearsal and maps its exit:
|
||||
- 0 goes on, or refuses `rehearsal-required` with `--no-rehearsal`.
|
||||
- 3 refuses `no-backup`, or with `--no-rehearsal` logs `rehearsal-skipped` and goes on.
|
||||
- Anything else refuses `rehearsal-failed`.
|
||||
|
||||
`--rehearse-port N`, default 7339. Hostcheck: a missing file or a nonzero `--check` refuses
|
||||
`profile-drift`.
|
||||
- sage-1653. `from_log newest`, and the external plan refusals `open-activation` and `migrate-required`.
|
||||
Residual: the `BACKUP_STORE` id and schema land in root's step log, but nothing ties that backup to the
|
||||
one the activation's own backup step takes later. A stopped backup before the upgrade would close that
|
||||
gap; it's a later row, not this one.
|
||||
|
||||
## Tests
|
||||
|
||||
`stack-upgrade.test.mjs` runs the driver as the test's user against a scratch root, with the test seams
|
||||
`STACK_UPGRADE_*`. The stand-ins are a fake `sudo` (logs its argv, checks the driver's form, then runs the
|
||||
real command as the test's user, r6's B1 pattern), a fake `case2-unit.sh` render that installs and starts
|
||||
the fixture units, a fake `systemctl` for root's system units, and a shell `hostcheck.sh`. The stack's
|
||||
user's steps run the real `stack-release.sh` and `stack-copy.sh` from a clone made from a real bundle. The
|
||||
units, copies and store are `stack-release-fixture.mjs`'s.
|
||||
|
||||
| Test | Covers |
|
||||
|---|---|
|
||||
| sequence 1 | confirm fails healthz, rollback with `--cause A5`, exit 1; every sudo call in the driver's form |
|
||||
| sequence 2 | `activate --migrate` fails at the wrapper, store restored, `--cause A3`, exit 1 |
|
||||
| sequence 3 | render fails, `--cause A4`; a probe run during `render-old` refuses `upgrade-running` |
|
||||
| sequence 4 | stage exits 3, A2f restarts and gates the old units, exit 1 |
|
||||
| N1 | a unit that won't stop: A2f's gate fails `stale-start`, exit 2, units stopped |
|
||||
| sequence 5 | confirm of the wrong copy refused, `--cause A5`, exit 1 |
|
||||
| sequence 6 | revert fails, exit 2; the next run refuses `interrupted-run` at its plan |
|
||||
| sequence 7 | two upgrades and a `rollback --to` with `--restore`; the args file after each confirm; `to-not-previous` |
|
||||
| sequence 8 | a migration that writes and fails, store back byte for byte, exit 1 |
|
||||
| lock | a second run while the lock is held: one log line, no call |
|
||||
| open-activation | any `AWAITING-UNITS` refuses at A1, nothing stopped |
|
||||
| migrate-required | refused without `--migrate`, no rehearsal, nothing stopped |
|
||||
| rehearsal-failed | the rehearsal's argv, exit 3, nothing stopped |
|
||||
| `BACKUP_STORE` | the line after a match; four malformed lines log `BACKUP_STORE none` |
|
||||
| no-backup | the real rehearsal with no backup refuses; `--no-rehearsal` logs the skip and goes on |
|
||||
| backup-last | the real rehearsal names the newest backup; `--no-rehearsal` with one refuses `rehearsal-required` |
|
||||
| args file | group write, other write, two `--copy`, an unknown flag, a symlink, `args-differ` |
|
||||
| hostcheck | nonzero `--check` and a missing file refuse `profile-drift` before the plan |
|
||||
| head-mismatch | checkout not at `--sha`; a commit the clone can't fast-forward to refuses `fetch-failed` |
|
||||
| usage | twelve bad argvs exit 4 with no log and no lock |
|
||||
|
||||
The step log check (`STEP`) runs in every test that reads the log: each line must be a step line or the
|
||||
result line, and none may hold the stack user's output (`stack-release:`, `PASS `, `FAIL `, `REHEARSAL`,
|
||||
the fake rehearsal's text).
|
||||
|
||||
## Mutants
|
||||
|
||||
`~/darkwing-scratch/r233/mutants.sh`, not committed because it edits the worktree. It has the brief's six
|
||||
mutants and two for sage-1650's mapping. The method is row 232's: a checked backup, exact replacements, a
|
||||
syntax check, the named killers, then the file put back with its sha256 checked. One control runs first:
|
||||
the whole `stack-upgrade.test.mjs` with no change. `DRY=1` matched each replacement once on the candidate
|
||||
(2026-10-21).
|
||||
|
||||
| Mutant | File | Killers |
|
||||
|---|---|---|
|
||||
| m1 drop the flock | stack-upgrade.sh | `a second run while the lock is held`, `sequence 3:` |
|
||||
| m2 release the lock when the rollback starts | stack-upgrade.sh | `sequence 3:` |
|
||||
| m3 skip A1's open-activation check | stack-release.sh | `A1 refuses open-activation` |
|
||||
| m4 skip the rehearsal | stack-upgrade.sh | `the rehearsal runs after the plan`, `sequence 1:` |
|
||||
| m5 accept a migrating target without `--migrate` | stack-release.sh | `migrate-required` |
|
||||
| m6 write the args file before confirm | stack-upgrade.sh | `sequence 7:`, `sequence 1:` |
|
||||
| m7 `--no-rehearsal` accepted with a backup | stack-upgrade.sh | `refuses rehearsal-required` |
|
||||
| m8 drop the `rehearsal-skipped` line | stack-upgrade.sh | `a real rehearsal with no backup` |
|
||||
|
||||
## Deviations
|
||||
|
||||
D1. The brief has A1 refuse `open-activation` for an open `AWAITING-UNITS` or a non-empty `interrupted`
|
||||
set. The interrupted case refuses with row 232's existing code, `interrupted-run`; only `AWAITING-UNITS`
|
||||
gives `open-activation`. Both reach root as `plan-refused`, exit 3 (sequence 6, the open-activation test).
|
||||
|
||||
D2. Root copies one line of the stack user's output into its log, `BACKUP_STORE`, on sage-1653. It
|
||||
matches a fixed pattern first, takes the last match, logs `BACKUP_STORE none` otherwise, and decides
|
||||
nothing on it.
|
||||
|
||||
D3. The stack user's steps carry no `--instance`. The driver takes `--instance dev` only, row 190's default
|
||||
instance, which the stack user's commands select with no flag. A named instance's units are a later row.
|
||||
|
||||
D4. Most tests fake the rehearsal in the fake `sudo`: the fixture's stub copies can't pass a real one. Two
|
||||
tests run the real rehearsal (`faults/real-rehearse`): no-backup and backup-last. `stack-release.test.mjs`
|
||||
covers `--backup last` itself.
|
||||
|
||||
D5. Any nonzero `--plan` exit refuses `plan-refused`, exit 3, whatever the code. The stack user's output
|
||||
above it names the refusal; root doesn't parse it.
|
||||
|
||||
D6. `rollback --restore ACT` is accepted without `--events`, as `stack-release.sh rollback` accepts it.
|
||||
The brief's usage pairs them. `--events` without `--restore` is a usage error.
|
||||
|
||||
D7. The refusal of any `STACK_UPGRADE_*` seam when run as root has no test, because the tests don't run
|
||||
as root. It is three lines at the top of the script.
|
||||
|
||||
D8. The fake `sudo` keeps the test's environment, where real `sudo` resets it. The driver passes
|
||||
everything the stack user needs through `env PATH=...` and `-H`, and the fake checks that form.
|
||||
|
||||
D9. Exit 3 means nothing stopped, not nothing changed. A refusal after `fetch` leaves the stack user's
|
||||
clone fast-forwarded and the new copy made. Both are the user's own state, and the next run reuses them.
|
||||
TOOLS.md and the run sheet say so.
|
||||
|
||||
D10. The brief's usage has neither `--no-rehearsal` nor `--rehearse-port`. Both come from sage-1650 and
|
||||
sage-1652.
|
||||
|
||||
## Decisions
|
||||
|
||||
- `rollback` has no HEAD check. Its target is an older copy than root's checkout, which already exists.
|
||||
It fetches, copies and rehearses nothing (sequence 7).
|
||||
- The new args file is written next to the old one (`.dev.args.XXXXXX`, same mode) before A2, so a full
|
||||
`/etc` refuses before anything stops. It moves into place after A5. A2f and the rollback remove it; a failed move keeps it and names it.
|
||||
- Every stack user step gets `--copies` explicitly, so the copies directory comes from the args file's
|
||||
`--home`, never from the user's environment.
|
||||
- The stack's user does the fetch, the fast-forward and the copy, so root never runs git on a tree that
|
||||
user can write.
|
||||
- A4 is the render, and `case2-unit.sh` starts the bus, then the Console. The forward path has no
|
||||
`systemctl start` of its own; only A2f does.
|
||||
|
||||
## Run environment
|
||||
|
||||
Worktree `/mnt/storage/scratch/darkwing-r233`, `TMPDIR=/mnt/storage/scratch/tmp/r233`, node from PATH. One
|
||||
whole `stack-upgrade.test.mjs` run takes about 56 s.
|
||||
|
||||
## Results
|
||||
|
||||
### Singles, run 1, 2026-10-21, Sage's slot (sage-1689)
|
||||
|
||||
`~/darkwing-scratch/r233/singles.sh` on `darkwing/row233-r1` (base e65a3ab4d, uncommitted tree), 03:48:20Z
|
||||
to 03:59:00Z. These are the test files that read a changed script. `stack-instances.test.mjs` reads
|
||||
`stack-release.sh`, so it joined the three named in the plan.
|
||||
|
||||
| File | Result |
|
||||
|---|---|
|
||||
| `stack-upgrade.test.mjs` | 20 pass, 0 fail |
|
||||
| `stack-release.test.mjs` | 194 pass, 1 fail |
|
||||
| `stack-release-external.test.mjs` | 25 pass, 0 fail |
|
||||
| `stack-instances.test.mjs` | 23 pass, 0 fail |
|
||||
|
||||
The failure was my test, not the script. "row 233: --backup last takes the newest backup..." compares
|
||||
`untouched(e)` before and after the run, and `untouched` includes `existsSync(e.log)`. The test took its
|
||||
before snapshot ahead of `rlog()`, which writes the log, so that flag went from false to true. Every hash in
|
||||
the snapshot matched. The fix takes the snapshot after `rlog()`. No other row 233 test calls `untouched`.
|
||||
|
||||
### Singles, run 2
|
||||
|
||||
`~/darkwing-scratch/r233/rerun.sh` on aacfa2d7a, Sage's slot (sage-1695), 04:13:05Z to 04:21:28Z:
|
||||
`stack-release.test.mjs` whole, 195 pass, 0 fail, 0 skipped. The fixed test passed. The other three files
|
||||
did not change after run 1. The gate and the mutants run next, in Sage's slot.
|
||||
|
||||
### Gate and mutants, 2026-10-21, Sage's slot (sage-1704)
|
||||
|
||||
`~/darkwing-scratch/r233/gate.sh` then `mutants.sh`, chained behind `busy.sh`, on d99166ade (code
|
||||
aacfa2d7a; d99166ade changes only this file). Counts are an awk sum over `gate-r1.out`.
|
||||
|
||||
| Part | Time (UTC) | Result |
|
||||
|---|---|---|
|
||||
| cli suite (`node --test tests/*.test.mjs`) | 04:29:01-04:37:27 | 616 pass, 0 fail, 0 skipped |
|
||||
| the 10 `scripts/test-*.sh` | 04:37:27-04:39:38 | selftests 429 passed, 0 failed; `test-queue`'s node part 156 pass, 0 fail |
|
||||
| gate total | | 1201 pass, 0 fail, 11 suites, `GATE_PASSED` |
|
||||
| c0 control | 04:39:38-04:40:35 | exit 0 |
|
||||
| m1-m8 | 04:40:35-04:41:25 | 8 of 8 killed, every one exit 1 |
|
||||
|
||||
Every restore passed its sha256 check, and `git diff` was clean at d99166ade afterwards. The EXIT trap
|
||||
removed the `darkwing-r233_default` network.
|
||||
|
||||
N1, found in the process sweep after the run: a `sleep 60` was left with its cwd in `packages/cli`.
|
||||
`stack-upgrade.test.mjs:312` spawns `flock LOCK sleep 60`, and its `t.after` SIGKILLs only the flock, so
|
||||
flock's child outlives the test by up to 60 s. It exited on its own. Row 232's T20
|
||||
(`stack-release.test.mjs:370`) spawns its holder `detached` and kills the group with `process.kill(-pid)`.
|
||||
The fix is the same two lines. That test is one of m1's killers.
|
||||
|
||||
Sage ruled it a fix now (sage-1707), and 08db6ac3f makes it, test only. After the gate's d99166ade
|
||||
the only change outside this file is that test:
|
||||
|
||||
```
|
||||
$ git diff --stat d99166ade 08db6ac3f
|
||||
packages/cli/tests/stack-upgrade.test.mjs | 4 ++--
|
||||
1 file changed, 2 insertions(+), 2 deletions(-)
|
||||
```
|
||||
|
||||
The gate stands on Sage's ruling. The rerun is the whole `stack-upgrade.test.mjs` singly plus m1, in
|
||||
Sage's slot, with a sweep for a leftover `sleep` afterwards.
|
||||
@@ -0,0 +1,173 @@
|
||||
# Row 240 (I4): root's host check, its pins and the boot unit
|
||||
|
||||
Brief: `docs/plans/2026-10-20_installer-rows.md` section I4, blob `4a5a65020f00` (commit b00d7d94d, rev 2665,
|
||||
the pin row 240 carries; the I3 carry items came in with it). Sage split it out of I3 (sage-1650). Gitea #1746, Vikunja P19 #432. Owner Darkwing, reviewer Filbert, gate owner Sage.
|
||||
Design: row 195 DESIGN r6 B2, `agents/darkwing/work/queue-195/DESIGN.md` at b7239bb66, :262-303 (the script,
|
||||
the three callers) and :839-860 (files, test cases, mutants). Where the brief and r6 differ, the brief wins.
|
||||
|
||||
## Files
|
||||
|
||||
Drafted on `darkwing/row240-r1` from d99166ade (row 233's candidate) while row 233's gate ran, then applied
|
||||
as one patch onto row 233's landing, 57f59e0c3, on `darkwing/row240-r2` (sage-1753). It applied cleanly; between
|
||||
the two bases the only change on this row's paths is 233's N1 test fix in `stack-upgrade.test.mjs`. Row 234
|
||||
then landed as c771a36f4 on 57f59e0c3. Sage preferred that the gate run with it in the tree (sage-1773), so both
|
||||
commits were picked onto c771a36f4 as `darkwing/row240-r3`. That was a clean pick: the 42 paths changed from
|
||||
57f59e0c3 to c771a36f4 share none with this row's 9.
|
||||
|
||||
- `scripts/case1-pins.sh` (new, 0644): `PIN_PROFILES_VERSION`, `PIN_PROFILE_SHA256`, `PIN_BUBBLEWRAP_VERSION`,
|
||||
r6 section 0's values.
|
||||
- `scripts/case1-hostcheck.sh` (new, 0755): the seven items and `--check`, `--enforce`, `--upstream`.
|
||||
- `packages/cli/systemd/mosaic-hostcheck.service` (new): the boot oneshot.
|
||||
- `packages/cli/tests/case1-hostcheck.test.mjs` (new, 16 tests). The cli suite's `tests/*.test.mjs` glob picks
|
||||
it up.
|
||||
- `docs/guides/host-run-sheet.md` section 4: the install lines that replace "Install them from row 240's run
|
||||
sheet lines once it lands", plus rollback, drift recovery and `--upstream` before a re-pin window. Sections
|
||||
3, 6, 8 and 10 for the I3 carry items below.
|
||||
- `scripts/stack-upgrade.sh` and `packages/cli/tests/stack-upgrade.test.mjs`: I3 N1 below.
|
||||
- `docs/TOOLS.md`: a new subsection "Root's host check (`scripts/case1-hostcheck.sh`, row 240)" after the
|
||||
upgrade driver's.
|
||||
|
||||
Brief line 257 said I3's `stack-upgrade.sh` doesn't change. Sage's ruling sage-1727 widened the row to it for
|
||||
N1 alone. Its default `STACK_UPGRADE_HOSTCHECK=/usr/local/lib/mosaic-stack/hostcheck.sh` is where the run
|
||||
sheet installs this script.
|
||||
|
||||
## Carried from I3
|
||||
|
||||
Filbert's row 233 approve (Gitea 29922) left five notes. Sage ruled N1, N4 and N5 into this row and N2 and N3
|
||||
to DEFERRED.md (sage-1727).
|
||||
|
||||
| Note | Change |
|
||||
|---|---|
|
||||
| N1: `stack-upgrade.sh` exits 1 `args-failed` when the args move after A5 fails, but exit 1 means the old copy runs | The move's failure exits 5 `args-failed`. The header, TOOLS.md's step 5 and exit line, and run sheet section 6's exit list name exit 5 and what root does: move the new file the result line names into place. Test `row 240: I3 N1: a failed args move after A5 exits 5 ...` makes `/etc/mosaic-stack` read-only at confirm and checks exit 5, the step names, `args rc=1`, the result line, both units on the new copy, the old args file's bytes and the new file beside it. The step log regex takes `exit=5`. Mutant M8. |
|
||||
| N4: section 3's `npm install` has no npm on root's PATH, and the `--claude` path is unnamed while `node_modules/.bin/claude` is a symlink `case2-unit.sh:140` refuses | The install line runs with section 2's pinned node first on PATH (the postinstall runs `node` too). The `--claude` path is `node_modules/@anthropic-ai/claude-code/bin/claude.exe`, and a `--version` line catches the postinstall's 500-byte stub. |
|
||||
| N5: sections 8 and 10 lack precheck lines | Each gets "Precheck line: none" and its missing line. |
|
||||
|
||||
Exit 5 is new rather than 2 because 2 means both units are stopped, and here both run. Exit 4 stays usage.
|
||||
|
||||
The `claude.exe` path comes from the 2.1.304 package itself: `npm view` gives `bin.claude` as `bin/claude.exe`,
|
||||
and its `install.cjs` hardlinks the platform binary over that placeholder, or copies it across devices, then
|
||||
sets 0755. I downloaded the tarball into `/mnt/storage/scratch/tmp/r240/cc304` and read it; nothing ran. The
|
||||
run sheet says a newer pin needs the same check.
|
||||
|
||||
## The brief, item by item
|
||||
|
||||
| Brief line | Where |
|
||||
|---|---|
|
||||
| 247 `case1-pins.sh` holds the three pins | `scripts/case1-pins.sh`; the pins test checks the values against r6's |
|
||||
| 248-249 seven items, `--check` exits 0 or 1 | `items()` and the final dispatch; the check tests |
|
||||
| 250-251 `--enforce`: `apparmor_parser -R`, `root:root 0700`, `DRIFT`, a log line, exit 0 | `enforce()` and `stop_bwrap()`; the four enforce tests |
|
||||
| 252 `--upstream` compares the current deb to the pin | `upstream()`; the upstream test |
|
||||
| 253 the boot oneshot | `mosaic-hostcheck.service`; the boot unit test |
|
||||
| 254-255 fakes for every host tool, r6's seven case groups, seven mutants | the test file; Tests and Mutants below |
|
||||
| 257 the I3 run sheet item points here | run sheet section 4 |
|
||||
|
||||
## Tests
|
||||
|
||||
Fakes on PATH, each logging its argv to `$HC/calls.log`: `aa-status`, `dpkg`, `dpkg-statoverride`,
|
||||
`apparmor_parser`, `systemd-run`, `stat`, `chown`, `chmod`, `apt-get`, `dpkg-deb`. `host(t)` builds a scratch
|
||||
root (`/etc`, `/proc/sys`, `/usr/bin/bwrap`, `/var/log`) and a lib dir with a copy of the script and fixture
|
||||
pins that hold the fixture profile's sha256. Every run passes `--root`.
|
||||
|
||||
r6's case groups (:843-856) against the tests:
|
||||
|
||||
| r6 case | Tests |
|
||||
|---|---|
|
||||
| `--check` exits 0 with all seven, 1 with each failing alone, its line naming it | `check: exits 0 ...`, `check: exits 1 with each item failing alone ...`, plus one test per item for the item's edge cases |
|
||||
| item 3: a second file naming `bwrap` as a word, `/{usr/,}bin/bwrap` included; passes with the profile alone | `check: item 3 ...` |
|
||||
| item 4: `unpriv_bwrap (enforce)` or a trailing space fails; the argv carries `-p NoNewPrivileges=yes` and the 2a' properties | `check: item 4 ...` checks the whole argv |
|
||||
| item 7: the floor at 732 fails | `check: item 6 ...; item 7 ...` (733 passes as the control) |
|
||||
| `--enforce` exits 0 on a pass and every failure; unload, close, `DRIFT`, one log line; `DRIFT` present with every item passing still closes | `enforce: a pass ...`, `enforce: each failure ...`, `enforce: with DRIFT present ...` |
|
||||
| `--enforce` never calls `dpkg-statoverride` with `--update`, `--add` or `--remove` | `enforce: no unload ...; never writes the statoverride` (every call is `--list /usr/bin/bwrap`) |
|
||||
| a bubblewrap version other than the pin is a line, not a failure | `check: a bubblewrap version other than the install pin ...` |
|
||||
| `--upstream` against a differing deb prints the difference, exits 0 | `upstream: ...` (also: a match, a deb with no profile, a failed download) |
|
||||
| the unit: `Type=oneshot`, `After=apparmor.service`, `[email protected]`, runs `--enforce` | `boot unit: ...` |
|
||||
|
||||
The last test also checks that the repo script reads `scripts/case1-pins.sh` beside it, that bad usage exits
|
||||
2, and that a non-root run without `--root` refuses.
|
||||
|
||||
Singles are the two files that run a changed script: `case1-hostcheck.test.mjs` and `stack-upgrade.test.mjs`.
|
||||
Sage asked (sage-1759) whether `stack-release.test.mjs` or `stack-release-external.test.mjs`, which share
|
||||
`stack-release-fixture.mjs`, run `stack-upgrade.sh` or `case1-pins.sh`. They don't. `grep -cF` for
|
||||
`stack-upgrade`, `case1-pins`, `case1-hostcheck` and `hostcheck` gives 0 in all three files; the control
|
||||
`stack-release.sh` gives 2 in the fixture and 8 in `stack-release.test.mjs`. `scripts/stack-release.sh` names
|
||||
`stack-upgrade.sh` once, in a comment (:2061). The gate's cli suite runs both files anyway.
|
||||
|
||||
## Mutants
|
||||
|
||||
r6's seven (:858-860) and one for I3 N1, scripted in `~/darkwing-scratch/r240/mutants.sh` with r233's method: a checked backup,
|
||||
an exact replacement that must match once, `bash -n`, the named killer by `--test-name-pattern`, the restore
|
||||
and a sha256 check. A dry run matched all eight once each. Results below.
|
||||
|
||||
| Mutant | Change | Killer |
|
||||
|---|---|---|
|
||||
| M1 | `--enforce` exits 1 after the stop | `enforce: each failure exits 0 ...` |
|
||||
| M2 | `[ ! -e "$DRIFT" ] \|\| had=1` becomes `true \|\| had=1` | `enforce: with DRIFT present ...` |
|
||||
| M3 | item 1's `dpkg -S` becomes `owner=""; rc=1` | `check: item 1 fails when a package owns ...` |
|
||||
| M4 | item 3's `grep -rlw bwrap` becomes `grep -rlF /usr/bin/bwrap` | `check: item 3 ...` |
|
||||
| M5 | item 4's `[ "$label" = "$LABEL" ]` becomes a substring match | `check: item 4 ...` |
|
||||
| M6 | item 4 drops `-p NoNewPrivileges=yes` | `check: item 4 ...` (argv) |
|
||||
| M7 | item 7's `-gt` becomes `-ge` | `check: item 6 ...; item 7 fails with the floor at 732 ...` |
|
||||
| M8 | `stack-upgrade.sh`'s `finish 5 args-failed` becomes `finish 1 args-failed` (I3 N1) | `row 240: I3 N1: a failed args move ...` |
|
||||
|
||||
## Deviations
|
||||
|
||||
- Repo name `scripts/case1-hostcheck.sh` (r6), installed as `/usr/local/lib/mosaic-stack/hostcheck.sh`, the
|
||||
name in brief line 239 and I3's `stack-upgrade.sh` default. r6's apt hook line says `case1-hostcheck.sh`;
|
||||
the run sheet's hook uses the installed name.
|
||||
- `stat` is a fake too. r6 lists seven fakes; a test can't make a root-owned file, so item 5's `stat` reads a
|
||||
fixture.
|
||||
- Item 7 with no Console unit installed reads the port as 1023 and says so ("1023, because no Console unit
|
||||
is installed"). Until I2 installs the unit the floor must be above 1023. A unit with no `--port` fails.
|
||||
- Item 4 runs under `timeout 60`. A hung `systemd-run` would otherwise hang the apt hook and the boot unit.
|
||||
|
||||
## Decisions
|
||||
|
||||
- Fail closed where r6 is silent:
|
||||
- pins with no 64-hex sha fail item 1;
|
||||
- `dpkg -S` exit 1 (no owner) passes, exit 0 fails with the owner, any other exit fails with the code;
|
||||
- a failed `aa-status` fails item 2 and counts as loaded, so `--enforce` tries the unload.
|
||||
- `--root DIR` is the test seam (r6 :840). Root refuses it; a non-root user needs it. Both exit 2. Root runs
|
||||
with `PATH=/usr/sbin:/usr/bin:/sbin:/bin`.
|
||||
- `DRIFT` is append-only (`<stamp> <failed line>`), and `--enforce` never removes it. The log line is written
|
||||
only when the stop runs: `<stamp> enforce stop items=<ids or none> drift-before=<0|1> unload=<rc=N|no-file|skipped> chown=N chmod=N`.
|
||||
- `--upstream` exits 1 when the download or the unpack fails, since it then compared nothing. A deb without
|
||||
the profile file is a difference (exit 0), per r6 :302-303.
|
||||
- The bubblewrap note reads the version from `dpkg -s bubblewrap`.
|
||||
- The run sheet's recovery steps (`chown root:mosaic`, `chmod 0750`, `apparmor_parser -r`, then `--check`
|
||||
before root removes `DRIFT`) are mine. r6 says only that root removes `DRIFT` in a reviewed step.
|
||||
|
||||
## Run environment
|
||||
|
||||
Worktree `/mnt/storage/scratch/darkwing-r240b`, branch `darkwing/row240-r3` on c771a36f4, with `node_modules`
|
||||
linked to the canonical checkout's. Node v26.8.1, kernel 7.2.2-1-cachyos, 16 CPUs, TMPDIR
|
||||
`/mnt/storage/scratch/tmp/r240`, compose project `darkwing-r240b`. Sage's slot (sage-1773, 06:16Z). Each run is
|
||||
one chain behind `busy.sh`, launched from `/`: `~/darkwing-scratch/r240/run-all.sh` checks the HEAD pin and a
|
||||
clean tree, runs the singles, then the mutants (8 killed, none surviving, the tree clean after), then the gate
|
||||
(the cli suite and every `scripts/test-*.sh`), each stopping the chain on failure, then sweeps `/proc` cwd links
|
||||
for anything left in the r240 trees.
|
||||
|
||||
## Results
|
||||
|
||||
Run r1, 06:17:10Z on 47ec9fe2f, failed at the singles. `case1-hostcheck.test.mjs` passed 16/0/0, and
|
||||
`stack-upgrade.test.mjs` went 20 pass, 1 fail. The failure was the N1 test's own `t.after` hook, which ran
|
||||
`chmodSync(e.etc, 0o755)` after the fixture's cleanup had already removed the scratch root, so it threw ENOENT.
|
||||
Every assertion in the test body had passed, exit 5 included. The fixture's cleanup runs `chmod -R u+w` on the
|
||||
root before `rmSync`, so the 0555 dir never needed the hook. b45852daa drops it and changes nothing else. The
|
||||
mutants and the gate didn't run. The sweep found nothing left. Sage counted r1 as this slot's one failure
|
||||
(sage-1788).
|
||||
|
||||
Run r2, 06:20:41Z to 06:33:28Z on b45852daa, passed. The counts come from awk over the .out files
|
||||
(`~/darkwing-scratch/r233/sum.sh`):
|
||||
|
||||
| Step | Result |
|
||||
|---|---|
|
||||
| `case1-hostcheck.test.mjs` | 16/0/0 |
|
||||
| `stack-upgrade.test.mjs` | 21/0/0 |
|
||||
| c0 control | exit 0 |
|
||||
| M1-M8 | 8/8 killed, none survived, every restore's sha256 matched, tree clean after |
|
||||
| `packages/cli` suite | 639/0/0 |
|
||||
| `scripts/test-*.sh` | auth 15, conductor 17, config 31, discord 78, extension-package 18, foundation 44, land-work 86, queue 156 node + 28 selftest, release 14, task 98; 585 pass, 0 fail |
|
||||
| Gate total | 11 suites, 1224 pass, 0 fail |
|
||||
| Sweep | nothing left in the r240 trees; the compose network was removed |
|
||||
|
||||
This commit adds only these BUILD.md lines. No test reads BUILD.md.
|
||||
@@ -273,3 +273,38 @@ harness spec-tools 2, raise-domain 2; business hosts 4; cli hosts 5; seat sandbo
|
||||
|
||||
Left: Sage runs the whole bus, business and cli suites and every `scripts/test-*.sh` at the landing gate,
|
||||
after Duckworth approves.
|
||||
|
||||
## Round 3 (sage-1742): two tests that count the schema
|
||||
|
||||
The landing gate failed `packages/bus/tests/schema.test.mjs:522`. It counts the tables and triggers that
|
||||
`schema.sql` creates, and row a adds two tables and a trigger, so the counts go from 16 to 18 tables and
|
||||
from 70 to 71 triggers. My singles didn't run it because I took it for a test of other rows' tables.
|
||||
|
||||
I grepped `packages/*/tests/*` for `sqlite_master` and `schema.sql`, with `schema.test.mjs:517` as the
|
||||
positive control. That turned up one more test that would fail: T46 in
|
||||
`packages/cli/tests/stack-release.test.mjs:910`. It puts every schema table in exactly one of four lists, and
|
||||
hosts and host_grants were in none. I'd left the stack-release tests out of the r2 singles on the claim that
|
||||
they don't load the bus, but T46 reads `schema.sql`. Both tables go in its untimed list (Sage's ruling,
|
||||
sage-1745: the message_threads pattern). Neither has an `at` column. Row a writes hosts at five lines,
|
||||
and each runs in one store transaction with a `grant.*` events row, which the restore guard counts.
|
||||
`#grantEvent` (broker.mjs:657) writes that row through `writeGrantEvent` (:201, INSERT INTO events at
|
||||
:212).
|
||||
|
||||
- :737 DELETE, in `#enrollFailed`, with grant.failed at :738. Both callers wrap it in a transaction,
|
||||
:816 and :849.
|
||||
- :763 INSERT, in the step 1 transaction (:754), with grant.host.enroll.step at :761.
|
||||
- :783 UPDATE installer_fingerprint, in the step 4 transaction (:780), with grant.host.enrolled at :784.
|
||||
- :932 DELETE, in the hosts.remove transaction (:927), with grant.host.removed at :933.
|
||||
- :950 UPDATE state ready, in the hosts.check transaction (:944), with grant.host.checked at :951.
|
||||
|
||||
Row a writes no host_grants rows. It only reads them (broker.mjs:695, :930, runtime.mjs:103). Row b adds
|
||||
the writes, and each must share a transaction with a `grant.*` event the same way. That goes on row
|
||||
235's carry list. The other hits pin an older schema
|
||||
(account, comment, name, spec, thread and usage migrations, chief-of-staff 3c) or count only their own
|
||||
tables (state.test.mjs:106 reads every table but pins no list, threads.test.mjs:129).
|
||||
|
||||
hosts and host_grants are mutable by design, so the refused-labels loop above `schema.test.mjs:517` gets no
|
||||
entry for them. hosts.check updates a host to ready. hosts.remove and a failed enrollment delete the
|
||||
record. Row b updates and deletes grant records. Neither table has append-only triggers. The one new
|
||||
trigger, events_grant_actor, guards events. grant-migration.test.mjs tests it, and it killed mutants
|
||||
A20 and A21.
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
# Row 238: the sso-config hint names the instance (row 200 follow-up N1)
|
||||
|
||||
Launchpad, 2026-10-21. Author Launchpad, reviewer Webby, gate owner Sage. Issue #1744. Brief "Brief: row 200
|
||||
follow-up N1, the sso-config hint names the instance" in `docs/plans/2026-10-20_stack-release-sso.md` (76134dfde),
|
||||
from Webby's row 200 round 3 note N1 (comment 29772), assigned in sage-1597.
|
||||
|
||||
Round 4 base c515273db, row 240's landing (`base.txt`). Branch `launchpad/row238-r4`, one commit.
|
||||
|
||||
- Round 1 was `launchpad/row238` bec7e499e on 011d41ba6, approved by Webby (webby-0049). It was built on
|
||||
76134dfde and rebased before its first push, after row 199 (cb5fdb064) added 6 lines to `docs/TOOLS.md`.
|
||||
- Rows 232 (e65a3ab4d) and 233 (57f59e0c3) then landed on all 3 paths (sage-1710, sage-1755). Round 2 was a
|
||||
local rebase onto 57f59e0c3 (ca3d30e5c, never pushed). The merge was clean. `sso_checks` moved from about
|
||||
:838 to :920 with its body unchanged except for this row's hunk. 233 moved the shared test helpers into
|
||||
`stack-release-fixture.mjs`, and the new subtest is still under T57.
|
||||
- Row 234 (c771a36f4) then added two lines to T46's `untimed` map (`host_grants`, `hosts`, now :919-920).
|
||||
Round 3 is the same patch on c771a36f4 with those lines kept. The code diff on the 3 paths has the same
|
||||
`git patch-id --stable` on both bases (b9d0bb0ee3c7). Pushed as `launchpad/row238-r3` a33597c33 and
|
||||
approved by Webby in round 2 (webby-0059, rev 2707).
|
||||
- Row 240 (c515273db) then changed `docs/TOOLS.md` at :837 and :854, away from this row's hunk at :599
|
||||
(sage-1803). Round 4 is the same patch on c515273db: `docs/TOOLS.md` is the landed blob plus the one
|
||||
hunk, and `scripts/stack-release.sh` (100755) and `stack-release.test.mjs` keep round 3's blobs. The
|
||||
patch-id is still b9d0bb0ee3c7. No test run for this doc-only merge; Sage's landing gate covers it.
|
||||
Row 240 also changed `stack-upgrade.test.mjs`, so the round 2 result for that file is from the older blob.
|
||||
`candidate-manifest.sha256` lists the 3 changed paths; the packet files are not in it.
|
||||
|
||||
No test touches `~/.mosaic-dev`, `~/.config/mosaic-dev`, `~/.mosaic`, a live unit or a live slot. The new
|
||||
subtest runs on row 190's `named()` scratch instance under the test's temp HOME.
|
||||
|
||||
## Files
|
||||
|
||||
| Path | Change |
|
||||
| --- | --- |
|
||||
| `scripts/stack-release.sh` | `sso_checks`: the N4 first step puts `${INSTANCE_FLAGS[*]}` after `activate $TO` in both commands. |
|
||||
| `packages/cli/tests/stack-release.test.mjs` | T57 subtest "row 238 (#1744)": the sso-config refusal and the `--plan` line on instance `alpha`. |
|
||||
| `docs/TOOLS.md` | The `sso-config` paragraph says both commands carry `--instance NAME` on a named instance. |
|
||||
|
||||
## What it does
|
||||
|
||||
On a named instance the hint now reads `activate ID --instance NAME without --sso first, then write
|
||||
console.sso, then run activate ID --instance NAME --sso`. `INSTANCE_FLAGS` is empty for the default instance,
|
||||
so `inst` is empty and the default text is unchanged byte for byte. T57's item 2 subtest still asserts the
|
||||
default text exactly. The flag goes right after `ID`, the same place the other entry points take it from
|
||||
`INSTANCE_FLAGS`. BUSINESS is not in the hint before or after this row.
|
||||
|
||||
## Tests run (this tree)
|
||||
|
||||
Logs are in `/mnt/storage/scratch/launchpad-r238/`.
|
||||
|
||||
- Round 1: `stack-release.test.mjs` whole, 193/193 on 76134dfde (`sr-whole.log`) and on 011d41ba6
|
||||
(`sr-whole-2.log`). `stack-instances.test.mjs` whole, 23/23 (`si.log`).
|
||||
- Round 2, on ca3d30e5c (57f59e0c3), each file singly on Sage's slot (sage-1761): `stack-release.test.mjs`
|
||||
196/196, `stack-release-external.test.mjs` 25/25, `stack-upgrade.test.mjs` 20/20 (`r2-*.log`, 241/0 by
|
||||
awk). Sage counts these as pre-merge (sage-1769).
|
||||
- Round 3, on this blob singly on Sage's slot (sage-1794): `stack-release.test.mjs` 196/196, 06:34:22Z to
|
||||
06:42:57Z (`r3-stack-release.log`). The other two files and
|
||||
`stack-release-fixture.mjs` are the same on 57f59e0c3 and c771a36f4, so their round 2 results stand.
|
||||
- A grep of `packages/*/tests/*` and `scripts/test-*.sh` finds the hint text only in `stack-release.test.mjs`
|
||||
(`:1538` default, `:1689` and `:1691` named) and no 64-hex pin on the 3 paths.
|
||||
- The cli suite, every `scripts/test-*.sh` and the mutant go to Sage's slot.
|
||||
|
||||
## Mutants (`mutants.py`, run as `python3 mutants.py <tree> [N ...]`, against T57)
|
||||
|
||||
| # | Mutant | Result |
|
||||
| --- | --- | --- |
|
||||
| 1 | the first step drops `--instance NAME` (`inst=""`) | not run, Sage's slot |
|
||||
|
||||
The pattern matches once on this tree.
|
||||
@@ -0,0 +1 @@
|
||||
c515273db7b364736527493ed170d0f088eb506e
|
||||
@@ -0,0 +1,3 @@
|
||||
abdd641ea5a411347e3644286821c6485a6220e2a3de0f5ee95dcc37ba2c75ca docs/TOOLS.md
|
||||
4eeae9f672aecccdf853c377fb10847ce2656dbc566720bddfece435c1755d3b packages/cli/tests/stack-release.test.mjs
|
||||
1896d6518a502d340dfe30d402102b8c5cd5d8520757387b018b636755eb5d36 scripts/stack-release.sh
|
||||
@@ -0,0 +1,37 @@
|
||||
# Row 238 (#1744) mutant: `python3 mutants.py <tree> [N ...]`. Each applies to the tree's scripts/stack-release.sh,
|
||||
# runs T57 in packages/cli/tests/stack-release.test.mjs and restores the file from a saved copy, checked by sha256.
|
||||
# Killed = test exit != 0.
|
||||
import hashlib, os, subprocess, sys, signal
|
||||
R = os.path.abspath(sys.argv[1])
|
||||
S = os.path.join(R, 'scripts', 'stack-release.sh')
|
||||
if not os.path.isfile(S): sys.exit(f'ABORT no scripts/stack-release.sh under {sys.argv[1]}')
|
||||
T, PAT = 'tests/stack-release.test.mjs', '^T57'
|
||||
M = [
|
||||
('the sso-config first step drops --instance NAME (row 238)', [(' local first="" inst="${INSTANCE_FLAGS[*]:+ ${INSTANCE_FLAGS[*]}}"', ' local first="" inst=""')]),
|
||||
]
|
||||
sha = lambda b: hashlib.sha256(b).hexdigest()
|
||||
only = sys.argv[2:]
|
||||
os.chdir(os.path.join(R, 'packages', 'cli'))
|
||||
survived = []
|
||||
for i, (name, pairs) in enumerate(M, 1):
|
||||
if only and str(i) not in only: continue
|
||||
orig = open(S, 'rb').read(); h = sha(orig)
|
||||
src = orig.decode()
|
||||
for a, b in pairs:
|
||||
if src.count(a) != 1: sys.exit(f'ABORT {i} {name}: pattern found {src.count(a)} times')
|
||||
src = src.replace(a, b)
|
||||
try:
|
||||
open(S, 'w').write(src); os.chmod(S, 0o755)
|
||||
p = subprocess.Popen(['node', '--test', f'--test-name-pattern={PAT}', T], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, start_new_session=True)
|
||||
try: out, _ = p.communicate(timeout=300)
|
||||
except subprocess.TimeoutExpired:
|
||||
os.killpg(p.pid, signal.SIGKILL); out, _ = p.communicate(); out += b'\nTIMEOUT'
|
||||
text = out.decode(errors='replace')
|
||||
ran = [l for l in text.splitlines() if l.startswith('ℹ fail ')]
|
||||
verdict = 'KILLED' if p.returncode != 0 else 'SURVIVED'
|
||||
if verdict == 'SURVIVED': survived.append(name)
|
||||
print(f'{i:2} {verdict} {name} [{ran[0] if ran else "no count"}]', flush=True)
|
||||
finally:
|
||||
open(S, 'wb').write(orig); os.chmod(S, 0o755)
|
||||
if sha(open(S, 'rb').read()) != h: sys.exit(f'ABORT restore failed for {S}')
|
||||
print('survived:', survived or 'none')
|
||||
@@ -0,0 +1,71 @@
|
||||
# Row 241: one decision lands once (#1747, P19 441)
|
||||
|
||||
Megavolt, 2026-10-21 (from `date -u`). Brief: `docs/plans/2026-10-20_agent-credentials.md`
|
||||
§ "Brief: row 237 follow-up, one decision lands once, with tests at the broker", c25785d3d. Source:
|
||||
row 237 residual 3, megavolt-0068, megavolt-0069 and sage-1651. Base: origin/refactor f5753c0e8, row 210's landing.
|
||||
Reviewer Duckworth, gate owner Sage. Tests only; no source changes.
|
||||
|
||||
## Why landing has no row 237 race
|
||||
|
||||
Row 237 fixed publish: two tickets on one decision both pushed, because the decision is spent only
|
||||
after the push, and a push to a remote fork can't be taken back. Landing has the same ticket shape
|
||||
(`checkLand` opens a ticket, `recordLand` spends the decision), but three guards keep one decision
|
||||
from landing twice:
|
||||
|
||||
1. **land-busy, the outer guard.** `doLand` in `packages/cli/src/git-socket.mjs:351` refuses
|
||||
`land-busy` while another landing runs. The slot is taken before the first await, and freed only
|
||||
when the landing ends. There is one git socket per launcher, and one host per dataRoot (host.lock).
|
||||
So through the socket two landings never overlap, and a later one stops at `checkLand` with
|
||||
`decision-consumed`. Even past that, applying the same reviewed patch to the new main fails
|
||||
`base-conflict` in `prepareLand`.
|
||||
2. **The CAS on main.** `finishLand` (`packages/seat/src/working.mjs:391`) moves main with
|
||||
`update-ref main <commit> <old>`, so a landing prepared on a tip that has since moved refuses
|
||||
`base-moved` and writes nothing.
|
||||
3. **The recordLand recheck.** `finishLand` calls `record()`, which is `recordLand`, under the same
|
||||
repository lock. `recordLand` runs `#landFacts` again (`packages/bus/src/broker.mjs:963`), which
|
||||
refuses `decision-consumed` once the decision is spent. When `record()` throws, `finishLand` moves
|
||||
main back.
|
||||
|
||||
Guards 2 and 3 can't be reached end to end, because guard 1 stops a second landing first. So they are
|
||||
pinned below the socket, and no test bypasses land-busy.
|
||||
|
||||
### The unrecorded-commit window
|
||||
|
||||
Between the `update-ref` and the end of `record()`, main names the squash, which isn't recorded yet.
|
||||
That window is under the repository's lock, so no other push, review or landing in this host sees
|
||||
it. If `record()` refuses, main moves back. If moving back fails, `finishLand` refuses
|
||||
`rollback-failed` and main keeps naming an unrecorded commit (tested at `working.test.mjs:555`, the
|
||||
main.lock case). That is existing behaviour, unchanged here.
|
||||
|
||||
## The test
|
||||
|
||||
`packages/bus/tests/land.test.mjs:312`. Two tickets pass `checkLand` on one decision before either
|
||||
records. `recordLand` on the first lands. `recordLand` on the second refuses `decision-consumed`
|
||||
(:319), and leaves one merge `action.allowed`, one RESULT and no `action.refused`. `landRefused`, as
|
||||
the socket would call it, then files the refusal and ends the ticket.
|
||||
|
||||
**The assert that pins the recordLand recheck is :319.** The `#spent` line is shared with
|
||||
`checkLand`, so dropping it also fails :144 (a `checkLand` after the decision is spent). Only :319
|
||||
shows the recheck at record time.
|
||||
|
||||
Single file on this branch: land.test.mjs 18/18.
|
||||
Whole suites on this branch before in-review: bus 500/500, seat 164/164, cli 565/565 (02:52 to 03:01Z).
|
||||
|
||||
## Mutants
|
||||
|
||||
| Mutant | Change | Killed by |
|
||||
|---|---|---|
|
||||
| M1 | drop the `#spent` line in `#landFacts` (broker.mjs:963) | land.test.mjs: :312 at :319, and :94 at :144 (16/2) |
|
||||
| M2 | drop `land.old` from the `update-ref` in `finishLand` (working.mjs:391) | working.test.mjs:555 at :567, the `base-moved` assert (25/1) |
|
||||
| M3 | drop the `land-busy` throw (git-socket.mjs:351) | git-socket.test.mjs:812 at :825, the `land-busy` assert (23/1) |
|
||||
|
||||
Each run was one test file on base f5753c0e8 (row 210's landing, which moved the `#spent` line from
|
||||
:945 to :963), 2026-10-21 02:51Z. Each assert failed with "Missing expected exception" or "rejection".
|
||||
`cmp` confirmed each source matched its saved copy after the restore. M2 is killed by the existing
|
||||
working.test:555, so per the brief working.test.mjs isn't changed. Logs: `/mnt/storage/scratch/megavolt-r241/`
|
||||
(m1.log to m3.log, with each diff).
|
||||
|
||||
## Gate
|
||||
|
||||
Per the brief: Duckworth approves the manifest; the bus, seat and cli suites and every
|
||||
`scripts/test-*.sh` are green on Sage's gate; the three mutants are killed.
|
||||
@@ -0,0 +1,108 @@
|
||||
# Row 242 build: a retry after publish-timeout pushes once
|
||||
|
||||
Row 242, #1748, P19 440. Brief: docs/plans/2026-10-20_agent-credentials.md, "Brief: row 237
|
||||
follow-up N1, a retry after publish-timeout pushes once" (9d165be8c). Base: origin/refactor
|
||||
8fede5fc6 (row 210 landed f5753c0e8).
|
||||
|
||||
## The defect
|
||||
|
||||
The git socket polls publishResult until publishDeadlineMs, then refuses publish-timeout
|
||||
(packages/cli/src/git-socket.mjs:433) while the bus process's push (runtime publishPush) can
|
||||
still be running. The socket then calls publishRefused, which went straight to
|
||||
broker.publishRefused: it filed the refusal and deleted the ticket. checkPublish no longer saw an
|
||||
open ticket on the decision, so a retry got a new ticket and pushed a second ref while the first
|
||||
push was still on its way to the fork.
|
||||
|
||||
## Fix chosen: hold the decision until the push ends, then record a push that reached the fork
|
||||
|
||||
packages/bus/src/runtime.mjs, `publishRefused` and `settle`:
|
||||
|
||||
- If the bus has no push for the ticket, publishRefused goes to the broker as before.
|
||||
- If the push is still running, the refusal code is held on the entry and the ticket stays open.
|
||||
checkPublish refuses another publish on the decision with publish-busy (row 237's check),
|
||||
the same as when the socket is still polling.
|
||||
- When the push ends, `settle` runs:
|
||||
- A push that reached the fork is recorded with broker.recordPublish, which spends the
|
||||
decision. The bus logs `pushed after the git socket refused <code>; recorded`.
|
||||
- A push that failed is filed as publishRefused with the socket's code.
|
||||
- If recordPublish refuses with a BusError, that code is filed instead.
|
||||
- If the push has already ended when the refusal arrives, it's settled at once. The sage-1440 test
|
||||
relies on this: its push fails fast, so publish-timeout is filed once, immediately, as before.
|
||||
|
||||
Why hold rather than kill. Killing the git client doesn't undo a push the fork has already
|
||||
received: receive-pack can update the ref after the client dies. After a kill the bus would have
|
||||
to ls-remote the fork to learn what landed, and the result could still race the server. Holding
|
||||
waits for the push's own outcome. Each git run in publishRemote already has its own limit
|
||||
(publishRemote defaults to checkout.mjs GIT_LIMITS), so a hold is bounded. Holding also reuses the existing publish-busy
|
||||
state, and the refusal text is unchanged.
|
||||
|
||||
Why record and not only hold. If the hold just filed publish-timeout once the push ended, the
|
||||
ticket would close and the decision would be free again with a ref on the fork. A retry would then
|
||||
push a second ref. Recording the late push spends the decision, so the retry gets
|
||||
decision-consumed.
|
||||
|
||||
The publish-timeout refusal text is unchanged (brief condition).
|
||||
|
||||
## Test
|
||||
|
||||
packages/cli/tests/publish-socket.test.mjs:302, "row 242: a retry after publish-timeout refuses
|
||||
publish-busy while the bus process's push runs; the late push is recorded and the decision pushes
|
||||
once".
|
||||
|
||||
- publishDeadlineMs 150 and a store token. The fork's pre-receive hook writes `held`, then loops
|
||||
until the test writes `release`.
|
||||
- The first publish gives publish-timeout. The test waits for `held`.
|
||||
- A retry to branch `other` gives publish-busy (:322). Refused events: ["publish-busy"]. No
|
||||
git.push.remote allowed event.
|
||||
- The test writes `release` and waits until there is exactly one allowed event, with ref
|
||||
refs/heads/coder/feat and the work commit.
|
||||
- The next retry gives decision-consumed (:330).
|
||||
- Exactly one git-receive-pack request reached the fake Gitea, and the fork has only
|
||||
refs/heads/coder/feat. Refused events: ["publish-busy", "decision-consumed"]. No TOKEN appears in
|
||||
events or logs.
|
||||
|
||||
Fail fast (sage-1681): the body is wrapped in try/finally, and the finally writes `release`. Under
|
||||
the first mutant run, the assert at the retry failed and the hook was never released. setup's
|
||||
teardown waits for the fake Gitea, which waited for the held push, so the file hung until I stopped
|
||||
it. Now a failed assert ends the hook and the run. Each wait is bounded at 10 s with an assert.fail.
|
||||
|
||||
## Mutants
|
||||
|
||||
Script: /mnt/storage/scratch/megavolt-r242/mutants.sh. Each mutant: save, one-line sed, run
|
||||
publish-socket.test.mjs whole, restore, cmp.
|
||||
|
||||
| Mutant | Change | Expected kill |
|
||||
|---|---|---|
|
||||
| M1 | export `publishRefused: (ticket, args) => broker.publishRefused(ticket, args)` again (removes the fix) | :302 at :322: the retry gets a new ticket and its own push, so publish-timeout, not publish-busy (seen in the stopped 03:06Z run) |
|
||||
| M2 | `settle` files publishRefused without trying recordPublish (hold only) | :302 at the wait for one allowed event, then :330 |
|
||||
|
||||
Results (Sage's slot, sage-1686, 2026-10-21T03:35:00Z to 03:35:15Z): both killed, both restored
|
||||
(cmp ok).
|
||||
|
||||
- M1: publish-socket 8/1, the row 242 test fails at :322 in 0.5 s.
|
||||
- M2: publish-socket 8/1, the row 242 test fails at :327, the wait for one allowed event, through
|
||||
the assert.fail inside `until` (:311) after its 10 s bound, in 10.3 s.
|
||||
|
||||
## Suites
|
||||
|
||||
On Sage's slot (sage-1686), on base 8fede5fc6 plus this row:
|
||||
|
||||
- bus `node --test tests/*.test.mjs`: 499/499, 03:35:20Z to 03:35:34Z. This is one fewer than
|
||||
Sage's 241 gate (500) because the base predates row 241's land.test case.
|
||||
- cli `node --test tests/*.test.mjs`: 566/566, 03:35:34Z to 03:43:47Z. That's 565 plus the
|
||||
row 242 test.
|
||||
|
||||
Singles before the slot: publish-socket.test.mjs 9/9, bus publish.test.mjs 13/13. The seat suite
|
||||
and scripts/test-*.sh are left for Sage's landing gate.
|
||||
|
||||
## Residuals
|
||||
|
||||
- The agent was told publish-timeout, but the push is recorded later and the run isn't
|
||||
notified. The allowed event and the bus log line are the only trace. A later publish on the
|
||||
decision gets decision-consumed, which tells the agent the decision was used. Whether the run
|
||||
should get a RESULT for a late record is a follow-up question for Sage.
|
||||
- A late push whose recordPublish refuses (Duckworth N2) is filed with that code: the ref stays on
|
||||
the fork and the decision is not spent, the same residual as an in-band push whose record
|
||||
refuses.
|
||||
- The hold lives in the bus process's memory, like the broker's open tickets. A bus restart during
|
||||
a hold drops both, the same as today for any publish in flight.
|
||||
@@ -20,12 +20,12 @@ so Mos doesn't check this draft. Draft 1 names NEW and goes to Mos for the P0 ch
|
||||
|
||||
- OLD, live: `4429a104f6d65ccf165f6c83b5d1dce3147dfde0`, schema 3i, pointer `pm: true` (record 99a406ed3).
|
||||
P1 re-reads the activation id and the Console flags from `status`. I don't copy them from the last window.
|
||||
- NEW: open. It's the row 234 landing, or a later code landing if Mos rules. Candidates in landing order:
|
||||
210 (r4 due 10-21), 237, 194, 234.
|
||||
- NEW: `c771a36f4bd0b6c359282425c668996a90072f93`, the row 234 landing (2026-10-21 06:16Z), unless Mos rules a
|
||||
later code landing. 194 is the open candidate. 210, 237, 232, 241, 242, 233 and 234 have landed.
|
||||
- REPO, `P`, `WIN=agents/sage/work/window-repin-<NEW12>/`, `SNAP`: as in `repin-4429a104f.md`, with the
|
||||
sha checks at P0 and before S1, S2 and any D3.
|
||||
|
||||
## What NEW carries over OLD (so far, refactor c8bc5cf4c)
|
||||
## What NEW carries over OLD (refactor c771a36f4)
|
||||
|
||||
`git log --first-parent 4429a104f..HEAD`, path-limited to packages, scripts, roles, contracts, adapters, skills
|
||||
and the package files:
|
||||
@@ -39,10 +39,14 @@ and the package files:
|
||||
| 199 | cb5fdb064 | Credential store per agent. Check: bus start with no store entries for the live agents changes nothing, and the chief's and pm's launches still pass. The P3 rehearsal covers start. |
|
||||
| N3 | 390ea5261 | README only. |
|
||||
| 230 | c8bc5cf4c | Case 2 scripts, tools only. Not run live. |
|
||||
| 210 | open | Host access. The live business has no `hosts`, so nothing is granted. Bus start runs the host-run sweep (M34d). P3 must show it removes nothing. |
|
||||
| 237 | open | Publish busy check (199 N1). |
|
||||
| 234 | open | Machine grant row a and the 3i to 3j schema step. `--plan` must say `migration: 3i -> 3j`. |
|
||||
| PRD 1.7+1.8 | open | `docs/prd/mosaic-stack.md` and `packages/tasks/requirements.json` move to version 1.8 (mos-1154, mos-1156, DECISIONS Q192). The live pm cites 1.6 until this window. Check after: the pm's requirements read answers version 1.8. |
|
||||
| 210 | f5753c0e8 | Host access. The live business has no `hosts`, so nothing is granted. Bus start runs the host-run sweep (M34d). P3 must show it removes nothing. |
|
||||
| 237 | d7d943199 | Publish busy check (199 N1). |
|
||||
| 232 | e65a3ab4d | Installer I1: `stack-release.sh --units external` and `stack-copy.sh copy --bundle`, `wrapper --create`. Both are opt-in, and this window uses neither. The default units path must render the same units, so the P2 unit diff covers it with row 190. P0 checks that no window command line carries `--units external`. |
|
||||
| 241 | 5d4a61776 | Test only (bus land.test.mjs). No live effect. |
|
||||
| 242 | 5fb736b92 | Bus publish hold: after publish-timeout the bus holds the decision until its push ends, then records a late push. The live business has no publish in flight at W1, and P3 shows none open. Its follow-up fc1cb856b is comments only (git-socket.mjs). |
|
||||
| 233 | 57f59e0c3 | Installer I3: new tool `scripts/stack-upgrade.sh` and a `stack-release.sh` change. This window doesn't run `stack-upgrade.sh`. The P2 unit diff covers the `stack-release.sh` change with rows 190 and 232. |
|
||||
| 234 | c771a36f4 | Machine grant row a and the 3i to 3j schema step (new tables `hosts` and `host_grants`). `--plan` must say `migration: 3i -> 3j`. |
|
||||
| PRD 1.7+1.8 | 0adf8ddfe | `docs/prd/mosaic-stack.md` and `packages/tasks/requirements.json` move to version 1.8 (mos-1154, mos-1156, DECISIONS Q192). The live pm cites 1.6 until this window. Check after: the pm's requirements read answers version 1.8. |
|
||||
|
||||
Service script drift OLD to HEAD is 415+/91- over the four scripts, mostly row 190. P5 holds them at NEW.
|
||||
|
||||
@@ -53,7 +57,14 @@ Service script drift OLD to HEAD is 415+/91- over the four scripts, mostly row 1
|
||||
- W0b: the business file copy, as in the last window.
|
||||
- Rollback after a migration isn't the plain rollback. OLD is 3i and the store is 3j, so going back needs
|
||||
`--restore <W5 activation>`, and that loses the events written after W5. The row 181 restore guard counts
|
||||
11 tables, and the untimed ones need the D3 snapshot. Mos decides any restore past W5, as before.
|
||||
11 tables, and the untimed ones need the D3 snapshot. Row 234 adds `hosts` and `host_grants` to the untimed
|
||||
list (T46), so at 3j the D3 snapshot covers them too. Draft 1 recounts the list from NEW. Mos decides any restore past W5, as before.
|
||||
- Grant look-alikes (row 234 A22b, gizmoduck-0107). `3i-3j.sql` adds `events_grant_actor` but doesn't check
|
||||
rows already in the store, so after the migration only the actor filter in `GRANT_EVIDENCE` (actor_role NULL,
|
||||
actor_run 'grant') ignores an old session-written row. P3, and again just before `--migrate`: on the
|
||||
backup copy, count the `action.allowed` events whose body.action or body.operation starts with `grant.`, or
|
||||
whose body has `hostLineLeft`. Expect 0, since no grant code has run live. Any nonzero count stops the window
|
||||
until I've read those rows.
|
||||
- P3 is the rehearsal with `--migrate` on a copy of the live backup. It must show the 3j step and the row
|
||||
counts before and after.
|
||||
|
||||
@@ -65,10 +76,15 @@ history unless the window fails.
|
||||
|
||||
## Open before draft 1
|
||||
|
||||
- Q1: NEW, once 234 lands. Do 194 and 237 go in if they land first? I recommend yes for 237 (small, bus
|
||||
only) and asking Mos about 194, since 194 changes the lifecycle verbs and roles/chief-of-staff.*.
|
||||
- Q1, ruled by mos-1161: draft 1 doesn't wait for 194. NEW is the latest code landing when I cut draft 1. If
|
||||
194 has landed with a green full gate (15 packages, ten script suites), NEW is the 194 landing and P3 also runs
|
||||
the lifecycle verbs once on the backup copy and gets one pm reply under the new roles/chief-of-staff files. A
|
||||
P3 failure that traces to 194 drops NEW back to c771a36f4 before anything touches live, and the window goes on.
|
||||
Otherwise NEW is c771a36f4 and 194 rides a later window.
|
||||
- Q2, settled by mos-1148: phase 2 is its own window on 10-23. Draft 1 takes OLD and the Console flags,
|
||||
`--sso` included, from `status` after that window.
|
||||
- Q3: who operates. If Darkwing is free, Darkwing operates and I check. Otherwise I operate as in the last
|
||||
window.
|
||||
- The P-step evidence list for rows 199 and 210 at bus start, written once both have landed.
|
||||
- The P-step evidence list for rows 199 and 210 at bus start. Both have landed (cb5fdb064, f5753c0e8), so
|
||||
it's due in draft 1. I take each log line and exit code from a P3 rehearsal run, not from the code.
|
||||
- Rows 241 and 242 (bus): P3 shows no publish or land decision open before W5.
|
||||
|
||||
@@ -1394,3 +1394,11 @@ are never rewritten or removed; corrections are new entries.
|
||||
2026-10-21T01:34:13Z | Filbert (Opus 5.5, 048f3c63) | row 232 installer I1 review r1 (#1734, manifest 7a3b4ea133cc) | APPROVE, Gitea 29829, queue rev 2600 (left for Sage's queue-commit); static review, gate and mutants are Darkwing's receipts; 3 non-blocking notes
|
||||
2026-10-21T01:49:55Z | Sage (Opus 5.5, 1ef1e4f8) | row 237 landed d7d943199 (#1743, gate green, M1 killed); PRD 1.7+1.8 landed 0adf8ddfe (Honker 2884ee496, Duckworth approve bf52628f9, tasks 97/0 bus 478/0 on the tip, DECISIONS Q192); rows 239 (#1745) and 240 (#1746) added; 233 briefed; queue revs 2540-2603 committed 584e2bd82 | landed, push next
|
||||
2026-10-21T02:51:46Z | Sage (Opus 5.5, 1ef1e4f8) | row 210 landing (r5 0485384e9, Duckworth r3 29851) | landed f5753c0e8, patch-id 723b1c83 = r5 diff, manifest 688d38b9 OK; gate 2796/0 packages, 429/0 scripts; mutants 52/53 (M22b equivalent); #1707 closed (29853), Vikunja 424 comment 429; queue revs 2604-2617 c084d8d42; row 234 briefed; Megavolt has the slot for 241 (sage-1668)
|
||||
2026-10-21T03:01:43Z | Filbert (Opus 5.5, 048f3c63) | row 232 installer I1 review r2, rebase scope (#1734, manifest 7af3c94308e0, da2da2f6c on 8fede5fc6) | APPROVE, Gitea 29865, queue rev 2622 (left for Sage's queue-commit); README = predicted 3-way blob 89a95b32, 7 blobs same as r1, BUILD.md +31/-0; corrected my r1 reader claim (host.test.mjs:855 reads cli README)
|
||||
2026-10-21T04:14:49Z | Sage (Opus 5.5, 1ef1e4f8) | lead: rows 241 and 242 landed, slot order | row 241 landed 5d4a61776 (#1747 closed 29883, M1-M3 killed, gate 2797/0 + 429/0); row 242 landed 5fb736b92 (#1748 closed 29894, Duckworth approve 29889, gate 2826/0 + 429/0); late-record notice ruled a follow-up (Vikunja 440 c432); row 233 brief re-pinned rev 2631; carry items on #1744 (29885) and #1734 (29886); queue revs 2628-2638 committed 96bac588f
|
||||
2026-10-21T04:50:44Z | Darkwing (Opus 5.5, 7525c8e4) | row 233 I3 gate, mutants and review request (sage-1704, sage-1707, sage-1708) | gate green on d99166ade (1201/0, cli 616/0) and m1-m8 8/8 killed; N1 flock-holder orphan fixed test-only in 08db6ac3f, rerun c0 20/0 + m1 killed, sweep 0 left; in review round 1, manifest 464ce9eb at 1a1dc80ca on darkwing/row233-r1, comment 29901, revs 2644 and 2647 (lock race resolved); Filbert woken
|
||||
2026-10-21T04:58:01Z | Filbert (Opus 5.5, 048f3c63) | row 233 (I3, #1735) review round 1 on manifest 464ce9eb177a (1a1dc80ca) | APPROVE, Gitea 29922, rev 2659 (left for Sage); static, no slot; 5 non-blocking notes N1-N5 (args-failed exit 1, REPO ownership, signals, run sheet npm/claude path, sections 8/10 precheck)
|
||||
2026-10-21T05:47:45Z | Sage (Opus 5.5, 1ef1e4f8) | rows 233+234 landing gate, row 233 landing, 240 brief, drift sweep (sage-1742..1758) | combined gate stopped on two row 234 failures (bus schema.test.mjs:522 counts, cli stack-release T46 restore-guard lists; mos-0823); 233 landed alone 57f59e0c3 (patch-id b16ac924, verify-commit ok, cli 617/0, scripts 585/0), pushed with b00d7d94d, 5465fd52d and riders 642a9b87d, 16019b80e; #1735 closed 29931, done rev 2675, Vikunja 432 comment 436; 240 briefed rev 2676; 234 r3 c783d5e39 rebuilt on the landing in Gizmoduck's slot; 197 and 238 drifted, rebase sent (sage-1754, 1755, 1757 stash fix)
|
||||
2026-10-21T06:20:30Z | Sage (Opus 5.5, 1ef1e4f8) | row 234 r3 landing gate (2890/0 packages, 585/0 scripts) and landing c771a36f4, pushed 06:16Z; #1740 closed 29938, Vikunja 436 c437; 235 briefed rev 2686, rotateBy ruling; 238/228/197 rebuilt on 234, slot order Darkwing 240 then 238, 228, 197; 10-24 draft NEW c771a36f4 (sage-1766..1787) | landed
|
||||
2026-10-21T06:37:41Z | Darkwing (Opus 5.5, 7525c8e4) | row 240 I4 build, slot run and review request (sage-1759, sage-1773, sage-1788, sage-1795) | r1 on 47ec9fe2f stopped at singles (stack-upgrade 20/1: the N1 test's own after hook threw ENOENT once the fixture removed the root), b45852daa drops it; r2 green: singles 37/0, M1-M8 8/8 killed, cli 639/0, scripts 585/0; darkwing/row240-r3 pushed at ad1b23ab2 on c771a36f4, manifest 573ece95 (9 paths), in-review revs 2694 and 2695, request comment 29943 on #1746, Filbert woken | in review
|
||||
2026-10-21T06:41:13Z | Filbert (Opus 5.5, 048f3c63) | row 240 (I4, #1746) round 1 review on manifest 573ece95 (ad1b23ab2) | approve, Gitea 29945, rev 2696; 233 carries N1/N4/N5 present; notes N1 boot unit Before= names one business, N2 doc-only, N3 info
|
||||
|
||||
+189
-7
@@ -408,8 +408,9 @@ exposure of a named Console waits for Jason after Q175 SSO.
|
||||
|
||||
```
|
||||
scripts/stack-copy.sh copy COMMIT [--copies DIR] [--allow-worktree]
|
||||
scripts/stack-copy.sh wrapper ID [--instance NAME] [--copies DIR] [--bin DIR]
|
||||
scripts/stack-copy.sh check [--instance NAME] [--copies DIR] [--bin DIR] [--business ID]
|
||||
scripts/stack-copy.sh copy --bundle FILE SHA [--copies DIR]
|
||||
scripts/stack-copy.sh wrapper ID [--instance NAME] [--copies DIR] [--bin DIR] [--create]
|
||||
scripts/stack-copy.sh check [--instance NAME] [--copies DIR] [--bin DIR] [--business ID] [--units user|external]
|
||||
```
|
||||
|
||||
One read-only tree supplies the bus, Console and human CLI (row 85,
|
||||
@@ -423,6 +424,14 @@ renames beside the target. A published existing copy stays unchanged.
|
||||
Dependencies use the exact lock and carry Pi's installed lock record.
|
||||
Failure publishes nothing and removes the temporary tree.
|
||||
|
||||
`copy --bundle FILE SHA` (row 232) is for a host with no refactor checkout.
|
||||
SHA is a full 40-character id. The bundle must pass `git bundle verify`,
|
||||
carry `refs/heads/refactor`, and reach SHA from it; each miss refuses and
|
||||
writes nothing. The copy is then made from SHA as `copy` makes it, and the
|
||||
script prints `release:` with the RELEASE value at SHA and `bundle sha256:`
|
||||
with the bundle's digest. That pair is the release identity on such a host.
|
||||
Without `--bundle`, `copy`'s reachability rule is unchanged.
|
||||
|
||||
`wrapper` is a live switching step, reserved for Jason or Mos. It accepts
|
||||
only an existing copy and a regular non-symlink mosaic wrapper with the
|
||||
Q81 marker on line 2 and its exact lease line to mosaic-old. It preserves
|
||||
@@ -430,11 +439,16 @@ both lines, adds comments naming the copy root and Q81's mosaic-old lease
|
||||
probe, and atomically replaces the wrapper's final exec with the
|
||||
copy's scripts/mosaic. --bin is a fixture override; the default is
|
||||
~/.npm-global/bin. Neither the Q81 worktree nor mosaic-old is removed.
|
||||
`wrapper ID --create` (row 232) writes the first wrapper on a new host, the
|
||||
same file `wrapper` writes. It refuses when anything is at the wrapper path,
|
||||
a symlink included, and never replaces it. Without `--create`, a missing
|
||||
wrapper still refuses.
|
||||
`check` changes nothing: it reads effective systemd WorkingDirectory and
|
||||
ExecStart (including drop-ins) for the bus and Console, plus the wrapper
|
||||
exec. It prints all three paths and succeeds only on the same existing
|
||||
copy; absent/failed reads and a mismatched ExecStart fail. Business defaults
|
||||
to mosaic-stack. Queue work continues on the canonical scripts/mosaic.
|
||||
to mosaic-stack. `check --units external` (row 232) reads root's system units
|
||||
with `systemctl show`, without `--user`. Queue work continues on the canonical scripts/mosaic.
|
||||
|
||||
Bus render/install accept --copy ID, --copies DIR, --pm and the unchanged
|
||||
--claude pin. Without --copy the rendered unit stays the checkout unit.
|
||||
@@ -455,7 +469,10 @@ scripts/stack-release.sh activate ID BUSINESS [--pm | --pm-after-harness] [--mig
|
||||
scripts/stack-release.sh pm-on BUSINESS [--timeout S]
|
||||
scripts/stack-release.sh rollback BUSINESS [--restore ACTIVATION [--events N]] [--timeout S] [--plan]
|
||||
scripts/stack-release.sh status BUSINESS
|
||||
common options: --copies DIR, --bin DIR, --instance NAME
|
||||
scripts/stack-release.sh confirm ID BUSINESS --units external [--rollback OLD --cause A3|A4|A5] [--timeout S]
|
||||
scripts/stack-release.sh revert ID BUSINESS --units external
|
||||
scripts/stack-release.sh gate ID BUSINESS --units external --since EPOCH [--timeout S]
|
||||
common options: --copies DIR, --bin DIR, --instance NAME, --units user|external
|
||||
--events N is the total of the per-table counts the post-backup-events refusal prints: events and
|
||||
every table in COUNTED.
|
||||
```
|
||||
@@ -582,7 +599,8 @@ error. Before anything stops, a run whose Console flags hold `--sso` refuses
|
||||
`sso-target` for a target Console without it, `sso-config` when the target's own
|
||||
`checkConsole` refuses the config's `console` key (or there is none, in which
|
||||
case, from another copy, it names the first step: activate the target without
|
||||
`--sso`, then write `console.sso`), and `sso-slot` for each provider's slot file
|
||||
`--sso`, then write `console.sso`; on a named instance both commands carry
|
||||
`--instance NAME`), and `sso-slot` for each provider's slot file
|
||||
that is missing, a symlink, not a regular file, not owned by this uid, has group
|
||||
or other bits, resolves inside the repository or the copies, or is empty. It
|
||||
uses `lstat`, `stat` and `realpath` on the slot and never reads it. `--plan`
|
||||
@@ -652,7 +670,7 @@ pinned copy](../packages/cli/README.md#one-pinned-copy-row-85-1579)).
|
||||
#### Rehearsal on a scratch root (D2, row 163)
|
||||
|
||||
```
|
||||
scripts/stack-release.sh rehearse ID BUSINESS --backup ACTIVATION|PATH --port N [--timeout S]
|
||||
scripts/stack-release.sh rehearse ID BUSINESS --backup ACTIVATION|PATH|last --port N [--timeout S]
|
||||
```
|
||||
|
||||
`rehearse` starts the target copy's bus host and Console on a copy of a
|
||||
@@ -663,7 +681,11 @@ runs it against live data. It stops and starts no unit and writes no pointer
|
||||
or log line.
|
||||
|
||||
`--backup` takes an activation id under `<dataRoot>/backups/stack/` or an
|
||||
absolute path. `--port` is the scratch Console's loopback port, 1024 to
|
||||
absolute path. `--backup last` (row 233) takes the newest backup the
|
||||
activation log records and prints `PASS backup-last` naming it. With none in
|
||||
the log it fails `no-backup`, exits 3 and makes no directory. After the
|
||||
backup's checks it prints `BACKUP_STORE <activation id> schema <version>`.
|
||||
`--port` is the scratch Console's loopback port, 1024 to
|
||||
65535. The run refuses `port-reserved` for 7332, 7333 and the pointer's
|
||||
`--port` and `--tester-port`, and `port-in-use` for a port something holds;
|
||||
it never picks another port. `--timeout` (default 60 seconds) bounds the
|
||||
@@ -711,6 +733,166 @@ Usage exits 4. Nothing prunes rehearsal directories.
|
||||
starts no thread ticker, and leaves an earlier host's `sessions.json` and
|
||||
its runs alone. No unit passes it, and `--pm` with it is a usage error.
|
||||
|
||||
#### Root-owned units (`--units external`, row 232)
|
||||
|
||||
On a host whose bus and Console are root's system units, the stack user runs
|
||||
`stack-release.sh` with `--units external` from its own clone of the release
|
||||
bundle (sage-1561). The script then calls `systemctl show`, `is-active` and
|
||||
`list-units` without `--user` and reads no journal. It starts, stops and
|
||||
installs nothing. Root does those steps, and the I3 driver calls the verbs
|
||||
below in order. Without `--units`, every command line and every byte of
|
||||
output is what it was before row 232. `confirm`, `revert`, `gate`, `--to`
|
||||
and `--expect-from` are usage errors without `--units external`.
|
||||
|
||||
- `adopt`, `status`, `activate ID` and `rollback --to ID` take the flag.
|
||||
`rollback` then needs `--to`, and `--to` must be the copy `rollback` alone
|
||||
would pick from the log, or it refuses `to-not-previous`.
|
||||
- `--plan --expect-from OLD` refuses `pointer-differs` when the pointer
|
||||
names a copy other than OLD.
|
||||
- `pm-on` and `activate --pm-after-harness` refuse `external-pm`; the PM
|
||||
flag is in root's bus unit. `--pm`, `--tester-port`, the proof flags,
|
||||
`--no-proof`, `--no-send` and `--sso` are usage errors; root's Console
|
||||
unit carries the Console flags.
|
||||
- `activate` and `rollback` refuse `units-running` unless both units read
|
||||
inactive or failed and the store is closed. They make the rollback set
|
||||
with an exclusive `mkdir`, write `AWAITING-UNITS` (the target copy) first,
|
||||
capture the pointer, the wrapper and both units' show output and unit file
|
||||
sums, back up and migrate or restore the store, and write the wrapper.
|
||||
They write no pointer and start nothing. The last line says the run is
|
||||
staged and names `confirm`. A failure after the capture writes `FAILED`
|
||||
with its code, logs nothing more and exits 1; the activation stays open.
|
||||
- `confirm ID BUSINESS` needs exactly one `AWAITING-UNITS` for copy ID whose
|
||||
activation the log holds open, or it refuses `open-activation`. It gates
|
||||
the start root made and writes the pointer and an `ok` line with the
|
||||
started line's op, from, to and backup.
|
||||
`confirm --rollback OLD --cause A3|A4|A5` gates OLD after root put the old
|
||||
units back and logs `failed` or `gate-failed` with `rollback ok` (exit 1)
|
||||
or `rollback-failed` (exit 2). Either removes `AWAITING-UNITS` and `FAILED`.
|
||||
- `revert ID BUSINESS` puts the saved wrapper back and, when the run had
|
||||
started a migration, restores the backup taken before it. It writes no log
|
||||
line and can run again.
|
||||
- `gate ID BUSINESS --since EPOCH` gates a copy with no open activation,
|
||||
takes no lock and writes nothing: 0 on a pass, 1 otherwise. EPOCH is root's
|
||||
clock before root started the units.
|
||||
|
||||
The gate under `--units external` replaces the journal read. The bus unit's
|
||||
`ActiveEnterTimestamp`, in whole seconds, must be at or after the started
|
||||
line's second (`confirm`) or `--since` (`gate`); an older start fails
|
||||
`stale-start`, so a unit that never stopped can't pass. With the PM on, the
|
||||
gate reads `launches list --json` for exactly one live launch of the PM's
|
||||
instance (the business's `launch.by`), for this business, whose run started at
|
||||
or after the floor. `status` adds an
|
||||
`awaiting units:` line for an open activation.
|
||||
|
||||
Under `--units external`, `activate --plan` also refuses `open-activation`
|
||||
while any rollback set holds `AWAITING-UNITS`, and `migrate-required` when
|
||||
the target migrates the store and the run lacks `--migrate` (row 233). Under
|
||||
`--units user` the plan's bytes are unchanged.
|
||||
|
||||
#### Root's upgrade driver (`scripts/stack-upgrade.sh`, row 233)
|
||||
|
||||
```
|
||||
scripts/stack-upgrade.sh upgrade --instance dev --bundle FILE --sha SHA [--migrate] [--no-rehearsal]
|
||||
[--rehearse-port N] [--timeout S]
|
||||
scripts/stack-upgrade.sh rollback --instance dev --to ID [--restore ACT [--events N]] [--timeout S]
|
||||
```
|
||||
|
||||
Root runs it from a root-owned checkout of the reviewed commit, never from a
|
||||
copy, because the stack's user can write the copies. Each run needs Mos's go,
|
||||
and `--migrate` needs Jason present (decision 96). The host must be prepared
|
||||
first (`docs/guides/host-run-sheet.md`). Only the default instance, `dev`, is
|
||||
accepted; a named instance's units are a later row.
|
||||
|
||||
The unit values come from `/etc/mosaic-stack/dev.args`, which I2's install
|
||||
writes: one `case2-unit.sh` argument per line, the serve flags after `--`,
|
||||
owned by root with no group or other write, exactly one `--copy`, and that
|
||||
copy the one both installed units run. Otherwise the run refuses
|
||||
`args-unreadable` or `args-differ`. The driver writes the new copy id into
|
||||
that file only after `confirm` passes.
|
||||
|
||||
`upgrade`, in order. Every stack user step runs as
|
||||
`sudo -u USER -H env PATH=<node dir>:/usr/local/bin:/usr/bin:/bin` from the
|
||||
user's clone `<home>/src/mosaic-stack`:
|
||||
|
||||
1. Takes `flock -n` on `/run/mosaic-stack/upgrade-dev.lock` and holds it to
|
||||
the end, rollback included. A second run refuses `upgrade-running`, and
|
||||
a lock it can't open refuses `lock-failed`.
|
||||
2. Fetches the bundle's `refactor` into the clone and fast-forwards it to
|
||||
SHA (`fetch-failed`), then `stack-copy.sh copy --bundle FILE SHA`
|
||||
(`copy-failed`).
|
||||
3. A1: this checkout's HEAD is SHA (`head-mismatch`), the copy exists
|
||||
(`copy-missing`), `/usr/local/lib/mosaic-stack/hostcheck.sh --check`
|
||||
exits 0 (`profile-drift`, row 240), both units name one copy
|
||||
(`units-unreadable`, `units-disagree`) and it is the args file's
|
||||
(`args-differ`), and
|
||||
`activate SHA BUSINESS --units external --plan --expect-from OLD` exits 0
|
||||
(`plan-refused`, which covers `open-activation`, `migrate-required` and
|
||||
`pointer-differs`).
|
||||
4. The rehearsal, `rehearse SHA BUSINESS --backup last --port 7339`
|
||||
(`--rehearse-port`). Exit 0 goes on. Exit 3 means no backup in the log
|
||||
and goes on only with `--no-rehearsal`, logging `rehearsal-skipped`;
|
||||
without the flag it refuses `no-backup`. Exit 0 with `--no-rehearsal`
|
||||
refuses `rehearsal-required`; any other exit refuses `rehearsal-failed`.
|
||||
5. A2 stops the Console, then the bus, and waits for both to read inactive
|
||||
or failed. A3 is `activate --units external`, A4 the render with the new
|
||||
`--copy` (which also starts the bus, then the Console), A5 `confirm`.
|
||||
The driver writes the new args file beside the old one before A2
|
||||
(`args-failed`, exit 3) and moves it into place after A5. A failed move
|
||||
exits 5 `args-failed`: the new copy is live and confirmed, the args file
|
||||
still names the old one, and root moves the new file (the result line
|
||||
names it) into place by hand.
|
||||
|
||||
A failure in A2 or a stage that exits 3 starts the old units again and gates
|
||||
them (A2f). A later failure stops both units, runs `revert`, renders the old
|
||||
copy and runs `confirm --rollback OLD --cause A3|A4|A5`. `rollback` is the
|
||||
same from A1 on, with `rollback BUSINESS --to ID`, no fetch, no copy and no
|
||||
rehearsal; `--restore` and `--events` pass through.
|
||||
|
||||
Root reads no output the stack user wrote except exit codes and the
|
||||
rehearsal's `BACKUP_STORE` line, which it copies into its log after a match
|
||||
against a fixed pattern (`BACKUP_STORE none` otherwise). The step log
|
||||
`/var/log/mosaic-stack/upgrade-dev.log`, 0640, gets one
|
||||
`<time> <step> rc=N argv: ...` line per step and a last
|
||||
`<time> result exit=N <code> <text>` line.
|
||||
|
||||
Exit 0 ok, 1 failed with the old stack running and gated, 2 rollback-failed
|
||||
with both units stopped, 3 refused with nothing stopped, 4 usage, 5
|
||||
args-failed with the new stack live and confirmed. A refusal
|
||||
after step 2 leaves the user's clone fast-forwarded and the new copy made. After exit
|
||||
2 the activation stays open, so the next run's plan refuses until root
|
||||
finishes it.
|
||||
|
||||
#### Root's host check (`scripts/case1-hostcheck.sh`, row 240)
|
||||
|
||||
```
|
||||
/usr/local/lib/mosaic-stack/hostcheck.sh --check | --enforce | --upstream
|
||||
```
|
||||
|
||||
Installed from `scripts/case1-hostcheck.sh` as `hostcheck.sh`, with
|
||||
`scripts/case1-pins.sh` beside it (`docs/guides/host-run-sheet.md` section 4).
|
||||
The pins are the profile's sha256, the `apparmor-profiles` version it came
|
||||
from and the bubblewrap install pin; changing one is a reviewed commit.
|
||||
Seven items, one `ok N` or `FAIL N` line each: the profile file matches the
|
||||
pin and no package owns it; `aa-status` has `bwrap` and `unpriv_bwrap` in
|
||||
enforce mode and no `mosaic-bwrap`; no other file under `/etc/apparmor.d`
|
||||
names `bwrap`; a bwrap child under the units' no-new-privs reads
|
||||
`bwrap//&unpriv_bwrap (enforce)`; `/usr/bin/bwrap` is `root mosaic 0750` by
|
||||
the statoverride and by `stat`; the user-namespace restriction and
|
||||
`fs.protected_hardlinks` are 1; `net.ipv4.ip_unprivileged_port_start` is
|
||||
above the Console unit's `--port`, or above 1023 with no unit. A `note` line
|
||||
says whether bubblewrap is the install pin and never fails.
|
||||
|
||||
`--check` exits 0 or 1; `stack-upgrade.sh` refuses `profile-drift` on 1.
|
||||
`--enforce` exits 0. On a failure, or while `/etc/mosaic-stack/DRIFT` exists,
|
||||
it unloads the profile if loaded, sets `/usr/bin/bwrap` to `root:root` 0700,
|
||||
appends the failed items to `DRIFT` and one line to
|
||||
`/var/log/mosaic-stack/hostcheck.log`. It never writes the statoverride. The
|
||||
apt hook runs it after every package run, and `mosaic-hostcheck.service`
|
||||
(`packages/cli/systemd/`) at boot before the bus. `--upstream` compares the
|
||||
`apparmor-profiles` candidate's profile to the pin, prints any difference
|
||||
and exits 0, or 1 when the download fails. Exit 2 is usage or a refusal:
|
||||
root takes no `--root` (the test seam), and other users run only with it.
|
||||
|
||||
## Console service (`scripts/webui-service.sh`)
|
||||
|
||||
```bash
|
||||
|
||||
@@ -0,0 +1,281 @@
|
||||
# Host run sheet: preparing a host for the stack
|
||||
|
||||
ops-01 runs this on a fresh host before install (I2), and once per host for enrollment. Jason places the
|
||||
credentials in section 9. Brief: `docs/plans/2026-10-20_installer-rows.md`, I3 (row 233, #1735). The
|
||||
host design is Darkwing's r6 (`agents/darkwing/work/queue-195/DESIGN.md`, section 2) as the brief
|
||||
changes it.
|
||||
|
||||
The hosts are .53 (profile `factory`) and .54 (profile `library`). Both get every section unless a
|
||||
section says otherwise. Every command runs as root unless it starts with `sudo -u mosaic`.
|
||||
|
||||
Each item names the `scripts/case2-precheck.sh` line that verifies it. Install's precheck runs as the
|
||||
stack's user with `--business ID --profile factory|library`. Where no precheck line checks an item, the
|
||||
item says "precheck line missing", and Gosalyn adds the line to I2. Today's precheck (C2-13a) has these
|
||||
gaps; C2-13b adds the library lines (`library.sync`, persona modes, no Discord binding).
|
||||
|
||||
## 1. The stack's user and its home
|
||||
|
||||
```sh
|
||||
useradd --system --user-group --home-dir /srv/mosaic --create-home --shell /usr/sbin/nologin mosaic
|
||||
chmod 0700 /srv/mosaic
|
||||
install -d -o root -g root -m 0755 /etc/mosaic-stack
|
||||
install -d -o root -g root -m 0750 /var/log/mosaic-stack
|
||||
```
|
||||
|
||||
`mosaic` gets no sudo rule and no supplementary group. Check with `sudo -l -U mosaic` (expect "not allowed
|
||||
to run sudo") and `id -G mosaic` (one number).
|
||||
|
||||
Precheck lines:
|
||||
- `ok HOME is /srv/mosaic`
|
||||
- `ok sudo -n true fails as mosaic`
|
||||
- `ok id -G prints one group (<gid>)`
|
||||
- `/srv/mosaic` at 0700: precheck line missing.
|
||||
|
||||
## 2. node, pinned under /opt
|
||||
|
||||
The pin is the version and sha256 the window names. `<v>` below is that version, for example `v24.x.y`.
|
||||
|
||||
```sh
|
||||
install -d -o root -g root -m 0755 /etc/mosaic-stack/node-keys /opt/mosaic-stack
|
||||
cd "$(mktemp -d)"
|
||||
curl -fsSLO https://nodejs.org/dist/<v>/node-<v>-linux-x64.tar.xz
|
||||
curl -fsSLO https://nodejs.org/dist/<v>/SHASUMS256.txt
|
||||
curl -fsSLO https://nodejs.org/dist/<v>/SHASUMS256.txt.asc
|
||||
gpg --no-default-keyring --keyring /etc/mosaic-stack/node-keys/pubring.kbx --verify SHASUMS256.txt.asc SHASUMS256.txt
|
||||
grep " node-<v>-linux-x64.tar.xz\$" SHASUMS256.txt | sha256sum -c -
|
||||
echo "<pinned sha256> node-<v>-linux-x64.tar.xz" | sha256sum -c -
|
||||
tar -xJf node-<v>-linux-x64.tar.xz --no-same-owner
|
||||
mv node-<v>-linux-x64 /opt/mosaic-stack/node-<v>
|
||||
chown -R root:root /opt/mosaic-stack/node-<v>
|
||||
chmod -R go-w /opt/mosaic-stack/node-<v>
|
||||
```
|
||||
|
||||
The keyring holds the Node release keys from the nodejs/release-keys repository, nothing else. Stop on any
|
||||
failed check; don't unpack. Rollback: `rm -r /opt/mosaic-stack/node-<v>`.
|
||||
|
||||
Precheck lines:
|
||||
- `ok node <v>` (version 24 or later only).
|
||||
- The `--node` path in the args file must be a root-owned regular file outside the home. `case2-unit.sh`
|
||||
refuses otherwise at render, which is not a precheck line.
|
||||
- The installed version equals the pin: precheck line missing.
|
||||
|
||||
## 3. claude, factory only (.53)
|
||||
|
||||
Root installs it under `/opt`, never under the stack's home (this replaces r6's `/srv/mosaic/.local/bin`).
|
||||
The pin is 2.1.304 unless the window names a newer one.
|
||||
|
||||
```sh
|
||||
install -d -o root -g root -m 0755 /opt/mosaic-stack/claude-<pin> /etc/claude-code
|
||||
PATH=/opt/mosaic-stack/node-<v>/bin:/usr/sbin:/usr/bin:/sbin:/bin \
|
||||
npm install --prefix /opt/mosaic-stack/claude-<pin> @anthropic-ai/claude-code@<pin>
|
||||
chown -R root:root /opt/mosaic-stack/claude-<pin>
|
||||
/opt/mosaic-stack/claude-<pin>/node_modules/@anthropic-ai/claude-code/bin/claude.exe --version
|
||||
printf '%s\n' '{"env":{"DISABLE_AUTOUPDATER":"1"}}' > /etc/claude-code/managed-settings.json
|
||||
chown root:root /etc/claude-code/managed-settings.json
|
||||
chmod 0644 /etc/claude-code/managed-settings.json
|
||||
```
|
||||
|
||||
Rollback: `rm -r /opt/mosaic-stack/claude-<pin> /etc/claude-code/managed-settings.json`.
|
||||
|
||||
Root's PATH has no npm, so the install line puts section 2's pinned node first; the package's postinstall
|
||||
also runs `node`. The `--version` line must print `<pin> (Claude Code)`. Anything else means the postinstall
|
||||
left its 500-byte stub in `bin/claude.exe` (no native package for the platform, or scripts were off): stop and
|
||||
roll back.
|
||||
|
||||
The args file's `--claude` is
|
||||
`/opt/mosaic-stack/claude-<pin>/node_modules/@anthropic-ai/claude-code/bin/claude.exe`, the regular file the
|
||||
postinstall hardlinks or copies the native binary to. Not `node_modules/.bin/claude`: that is npm's symlink
|
||||
to it, and `case2-unit.sh` refuses a symlink, as it does anything that isn't a root-owned regular file outside
|
||||
the home. Checked against the 2.1.304 package's `install.cjs`; a newer pin needs the same check.
|
||||
|
||||
Precheck lines:
|
||||
- `ok claude replaces its prompt with --system-prompt-file (<n> input tokens)`. It runs the claude the bus
|
||||
unit's `MOSAIC_CLAUDE_BIN` names, so it needs the model credential from section 9.
|
||||
- Managed settings present with the autoupdater off: precheck line missing.
|
||||
|
||||
## 4. bubblewrap, AppArmor and the drift stop
|
||||
|
||||
Both hosts. These are r6's items, unchanged.
|
||||
|
||||
```sh
|
||||
apt-get install bubblewrap=0.9.0-1ubuntu0.3
|
||||
stat -c '%U:%G %a' /usr/bin/bwrap > /etc/mosaic-stack/bwrap.orig # expect root:root 755
|
||||
|
||||
cd "$(mktemp -d)"
|
||||
apt-get download apparmor-profiles=4.0.1really4.0.1-0ubuntu0.24.04.9
|
||||
dpkg-deb --fsys-tarfile apparmor-profiles_*.deb | tar -x ./usr/share/apparmor/extra-profiles/bwrap-userns-restrict
|
||||
echo "11d39094f044f0cda0febb3ad517b830301da6b2ce929664af09ee9e4dd264f9 usr/share/apparmor/extra-profiles/bwrap-userns-restrict" | sha256sum -c -
|
||||
install -o root -g root -m 0644 usr/share/apparmor/extra-profiles/bwrap-userns-restrict /etc/apparmor.d/bwrap-userns-restrict
|
||||
apparmor_parser -r /etc/apparmor.d/bwrap-userns-restrict
|
||||
|
||||
dpkg-statoverride --update --add root mosaic 0750 /usr/bin/bwrap
|
||||
```
|
||||
|
||||
`aa-status` then lists `bwrap` and `unpriv_bwrap` in enforce mode. Install the one profile file, never the
|
||||
whole `apparmor-profiles` package, whose 20 other profiles would load at the next boot. Rollback, in
|
||||
reverse order: `dpkg-statoverride --remove /usr/bin/bwrap`, then `chown` and `chmod` to the values in
|
||||
`bwrap.orig`; `apparmor_parser -R /etc/apparmor.d/bwrap-userns-restrict`, then `rm` it;
|
||||
`apt-get remove bubblewrap`.
|
||||
|
||||
The no-new-privs test must print `bwrap//&unpriv_bwrap (enforce)`:
|
||||
|
||||
```sh
|
||||
systemd-run --wait --pipe --uid=mosaic --gid=mosaic -p NoNewPrivileges=yes -p ProtectSystem=strict \
|
||||
-p ProtectHome=yes -p PrivateTmp=yes \
|
||||
/usr/bin/bwrap --unshare-user --unshare-pid --ro-bind / / cat /proc/self/attr/current
|
||||
```
|
||||
|
||||
The P9 checks: another user's `/usr/bin/bwrap --version` fails with Permission denied, `getent group mosaic`
|
||||
lists no member, and `sysctl fs.protected_hardlinks` prints 1. `sysctl
|
||||
kernel.apparmor_restrict_unprivileged_userns` prints 1 too (Ubuntu 24.04's default); leave it, so nothing
|
||||
else on the host gains user namespaces.
|
||||
|
||||
The drift stop (row 240, I4, #1746) goes in last, after the profile and the statoverride. As root, from
|
||||
the root-owned checkout of the reviewed commit:
|
||||
|
||||
```sh
|
||||
install -d -o root -g root -m 0755 /usr/local/lib/mosaic-stack
|
||||
install -o root -g root -m 0755 scripts/case1-hostcheck.sh /usr/local/lib/mosaic-stack/hostcheck.sh
|
||||
install -o root -g root -m 0644 scripts/case1-pins.sh /usr/local/lib/mosaic-stack/case1-pins.sh
|
||||
printf '%s\n' 'DPkg::Post-Invoke { "/usr/local/lib/mosaic-stack/hostcheck.sh --enforce || true"; };' \
|
||||
> /etc/apt/apt.conf.d/80mosaic-hostcheck
|
||||
install -o root -g root -m 0644 packages/cli/systemd/mosaic-hostcheck.service /etc/systemd/system/
|
||||
systemctl daemon-reload && systemctl enable mosaic-hostcheck.service
|
||||
/usr/local/lib/mosaic-stack/hostcheck.sh --check
|
||||
```
|
||||
|
||||
`--check` prints one `ok` line for each of its seven items and exits 0. Item 7 reads 1023 as the port
|
||||
until I2 installs the Console unit (section 10), then the unit's `--port`. `stack-upgrade.sh` runs `--check`
|
||||
before its plan and refuses `profile-drift` on any exit but 0, a missing file included. Rollback:
|
||||
`systemctl disable mosaic-hostcheck.service`, then `rm` the unit, the hook file and
|
||||
`/usr/local/lib/mosaic-stack`.
|
||||
|
||||
On drift, `--enforce` (the apt hook after every package run, and the unit at boot) unloads the profile,
|
||||
closes `/usr/bin/bwrap` to `root:root` 0700, appends the failed items to `/etc/mosaic-stack/DRIFT` and logs
|
||||
a line in `/var/log/mosaic-stack/hostcheck.log`. Every role session then refuses to start. ops-01 reads the
|
||||
log in each STATE report and sends any `DRIFT` to Mos. Only root removes `DRIFT`, in a reviewed step with
|
||||
a new pin or a Mos go. After that, `chown root:mosaic /usr/bin/bwrap`, `chmod 0750 /usr/bin/bwrap` and
|
||||
`apparmor_parser -r /etc/apparmor.d/bwrap-userns-restrict` put bwrap back, and `--check` must exit 0.
|
||||
|
||||
Before each re-pin window, ops-01 runs `/usr/local/lib/mosaic-stack/hostcheck.sh --upstream`. It
|
||||
downloads the `apparmor-profiles` candidate and compares its `bwrap-userns-restrict` to the pin. A
|
||||
`DIFFERS` line goes to Mos as a new-pin request; it refuses nothing.
|
||||
|
||||
Precheck lines:
|
||||
- `ok bwrap --unshare-all runs`
|
||||
- The AppArmor profile, the statoverride, the no-new-privs result and the P9 checks: precheck line missing.
|
||||
`/usr/local/lib/mosaic-stack/hostcheck.sh --check` covers them (row 240).
|
||||
|
||||
## 5. The port floor
|
||||
|
||||
Leave `net.ipv4.ip_unprivileged_port_start` at its default (1024) or higher. The Console's port is below
|
||||
it and the unit gets `CAP_NET_BIND_SERVICE`, so no other `mosaic` process can bind that port.
|
||||
|
||||
Precheck line: `ok net.ipv4.ip_unprivileged_port_start <n> is above the Console's port <port>`.
|
||||
|
||||
## 6. The release transport
|
||||
|
||||
A release is a git bundle of `refs/heads/refactor` plus the 40-character commit id (I1, mos-1118 choice 2).
|
||||
Mos's go names both, with the bundle's sha256. `<sha>` below is that id.
|
||||
|
||||
First install:
|
||||
|
||||
```sh
|
||||
# Copy the bundle in, then check it before anything reads it.
|
||||
echo "<bundle sha256> /root/<sha>.bundle" | sha256sum -c -
|
||||
git bundle verify /root/<sha>.bundle
|
||||
|
||||
# Root's checkout. stack-upgrade.sh and case2-unit.sh run from here, never from a copy.
|
||||
git clone -q -b refactor /root/<sha>.bundle /root/mosaic-stack
|
||||
git -C /root/mosaic-stack checkout -q --detach <sha>
|
||||
git -C /root/mosaic-stack rev-parse HEAD # must print <sha>
|
||||
|
||||
# The stack's user's copy of the bundle, and its own clone.
|
||||
install -d -o mosaic -g mosaic -m 0700 /srv/mosaic/incoming /srv/mosaic/src
|
||||
install -o mosaic -g mosaic -m 0600 /root/<sha>.bundle /srv/mosaic/incoming/<sha>.bundle
|
||||
sudo -u mosaic -H git clone -q -b refactor /srv/mosaic/incoming/<sha>.bundle /srv/mosaic/src/mosaic-stack
|
||||
```
|
||||
|
||||
Each upgrade, before `stack-upgrade.sh upgrade`:
|
||||
|
||||
```sh
|
||||
echo "<bundle sha256> /root/<sha>.bundle" | sha256sum -c -
|
||||
git bundle verify /root/<sha>.bundle
|
||||
git -C /root/mosaic-stack fetch -q /root/<sha>.bundle refs/heads/refactor:refs/remotes/origin/refactor
|
||||
git -C /root/mosaic-stack checkout -q --detach <sha>
|
||||
install -o mosaic -g mosaic -m 0600 /root/<sha>.bundle /srv/mosaic/incoming/<sha>.bundle
|
||||
/root/mosaic-stack/scripts/stack-upgrade.sh upgrade --instance dev --bundle /srv/mosaic/incoming/<sha>.bundle --sha <sha>
|
||||
```
|
||||
|
||||
The driver fetches the bundle into the stack's user's clone and makes the copy itself. It refuses
|
||||
`head-mismatch` when root's checkout isn't at `<sha>`. Usage, exits and the step log are in
|
||||
`docs/TOOLS.md`, under "Root's upgrade driver". Each run needs Mos's go, and `--migrate` needs Jason present
|
||||
(decision 96).
|
||||
|
||||
When a run doesn't exit 0, send Mos the exit and the last lines of `/var/log/mosaic-stack/upgrade-dev.log`:
|
||||
- Exit 3: refused before anything stopped. The last line names the code. The stack's user's clone and the
|
||||
new copy may already hold the new commit; the units still run the old one.
|
||||
- Exit 1: the old copy runs and passed its gate. Either the driver restarted it after A2 or a refused A3
|
||||
(A2f), or it rolled back after A3, A4 or A5.
|
||||
- Exit 2: both units are stopped and the activation stays open, so the next run refuses at its plan until
|
||||
it is finished with `confirm` or `revert`. One way here: a unit that doesn't stop within `--timeout`
|
||||
keeps its old start time, A2f's gate fails it `stale-start`, and the driver stops both (duckworth-0055
|
||||
N1). That is on purpose; a unit that won't stop is a reason to stop.
|
||||
- Exit 5: `args-failed`. The new copy is live and passed confirm, but `/etc/mosaic-stack/dev.args` still names
|
||||
the old copy. The next run refuses `args-differ`, and a render from that file would put the old copy back.
|
||||
Move the file the result line names into place with `mv -f`. The two differ only in the `--copy` value.
|
||||
|
||||
Precheck lines: none. Root's checkout at `<sha>` and the stack's user's clone: precheck line missing.
|
||||
After install, `stack-copy.sh check --units external` reads the copy the units run.
|
||||
|
||||
## 7. The args file
|
||||
|
||||
`/etc/mosaic-stack/dev.args` holds the `case2-unit.sh` arguments, one per line, with the serve flags after a
|
||||
line holding `--`. I2's install writes it; ops-01 doesn't write it by hand. It must be root's, with no group
|
||||
or other write (`chmod 0644` or tighter). Before `--` it takes only `--user`, `--home`, `--business`,
|
||||
`--copy` (exactly once), `--node`, `--profile`, `--claude` and `--pm`. `stack-upgrade.sh` changes only the
|
||||
`--copy` line, and only after confirm passes.
|
||||
|
||||
Precheck line: none. `stack-upgrade.sh` refuses `args-unreadable` on any of these rules.
|
||||
|
||||
## 8. Enrollment, once per host
|
||||
|
||||
The machine-grant enrollment is Gizmoduck's design r6, commit f9039682e, section 3, steps E1-E4. ops-01's
|
||||
Q-O1 is answered (mos-1125). The host side is the `tools/host/` files: `mosaic-keys`, `mosaic-keys-write`,
|
||||
their sudoers line and the sshd drop-in `/etc/ssh/sshd_config.d/<nn>-mosaic-keys.conf`. The verbs that run
|
||||
it (`hosts.enroll`, the `host-enroll` decision) are rows 234 and 235, not yet on refactor. This item has no
|
||||
commands until they land.
|
||||
|
||||
Precheck line: none. The enrolled host's key and its `mosaic-keys` sudoers line: precheck line missing.
|
||||
|
||||
## 9. Credential slots
|
||||
|
||||
Jason places these, all in one sitting per host. No value appears here or in any message. Check a placed
|
||||
file with `stat -c '%U %a %n' PATH`, never `cat`. `<business>` is the host's business id.
|
||||
|
||||
### .53, factory
|
||||
|
||||
| Path | File | Mode | Owner | Profile | Precheck line |
|
||||
|---|---|---|---|---|---|
|
||||
| `/srv/mosaic/.config/mosaic-dev/secrets/` | `claude-session.env`, one line `CLAUDE_CODE_OAUTH_TOKEN=...` (Q185) | 0600 | mosaic | factory | `claude replaces its prompt with --system-prompt-file` runs claude with it. The file's mode and owner: precheck line missing. |
|
||||
| `/srv/mosaic/.config/mosaic-dev/secrets/agents/<business>/<agent>/` | one file per service | 0600, dirs 0700 | mosaic | both | none. Filled only by `mosaic credentials issue`, or Approve after #436. Never by hand. |
|
||||
|
||||
### .54, library
|
||||
|
||||
| Path | File | Mode | Owner | Profile | Precheck line |
|
||||
|---|---|---|---|---|---|
|
||||
| `/srv/mosaic/.config/mosaic-dev/secrets/` | `claude-session.env`, one line `ZAI_API_KEY=...` | 0600 | mosaic | library | `<path> sets ZAI_API_KEY` |
|
||||
| `/srv/mosaic/.config/mosaic-dev/secrets/` | the Console's proof file, the path the args file's `--proof-file` names | 0600 or 0400 | mosaic | library | none. `case2-unit.sh` refuses at render. Precheck line missing. |
|
||||
| `/srv/mosaic/.config/mosaic-dev/secrets/<business>/` | `library-gitea` | 0600 | mosaic | library | C2-13b's `library.sync` (not landed) |
|
||||
| `/srv/mosaic/.config/mosaic-dev/secrets/agents/<business>/<agent>/` | one file per service | 0600, dirs 0700 | mosaic | both | none. Filled only by `mosaic credentials issue`, or Approve after #436. Never by hand. |
|
||||
|
||||
The installer key, `<configDir>/secrets/hosts/<business>/<hostId>/installer`, is not a slot anyone places.
|
||||
Enrollment (section 8) writes it under the enrolling stack's config directory and deletes it when the host
|
||||
leaves.
|
||||
|
||||
## 10. Console start
|
||||
|
||||
None by hand. I2's `case2-unit.sh` enables both units, starts the bus, then the Console (Q189, mos-1122).
|
||||
`stack-upgrade.sh` stops and starts both units itself.
|
||||
|
||||
Precheck line: none. Both units active after the render: precheck line missing.
|
||||
@@ -256,6 +256,21 @@ r6 B2 as designed:
|
||||
|
||||
The I3 run sheet's hostcheck item points at this row. I3's A1 and its fake don't change.
|
||||
|
||||
### Carried from the I3 review (Filbert 29922 on #1735, sage-1727)
|
||||
|
||||
Row 233 lands with three non-blocking notes that must be fixed before ops-01. This row takes them, and that
|
||||
widens its paths to `scripts/stack-upgrade.sh` and `packages/cli/tests/stack-upgrade.test.mjs`:
|
||||
- N1: after A5 confirms, a failed rename of the new args file calls `finish 1 args-failed` while the new copy
|
||||
is live (`stack-upgrade.sh:280-282` at 1a1dc80ca). TOOLS.md and the run sheet define exit 1 as "the old copy
|
||||
runs". Give this case its own exit code, add it to the exit table in TOOLS.md and the run sheet, and add a
|
||||
test that makes the rename fail after A5 and expects the new code.
|
||||
- N4: run sheet section 3 doesn't put the pinned node's npm on root's PATH and doesn't name the `--claude`
|
||||
path. The npm symlink `node_modules/.bin/claude` is what `case2-unit.sh:140` refuses. The sheet must name
|
||||
the path the check accepts.
|
||||
- N5: run sheet sections 8 and 10 get their precheck lines.
|
||||
|
||||
Filbert's N2 and N3 (no `$REPO` check, no signal handling between A2 and A5) are in DEFERRED.md, not here.
|
||||
|
||||
### Gate
|
||||
|
||||
Filbert approves the manifest. The test file, the cli suite and every `scripts/test-*.sh` are green on
|
||||
|
||||
@@ -216,6 +216,49 @@ at every gate. Started 2026-09-12 during the control board MVP.
|
||||
READMEs, `docs/TOOLS.md` and Sage's suite recipe should use the glob
|
||||
form wherever Node 24 is a target. Darkwing, S1 packet. (2026-10-04,
|
||||
#1518)
|
||||
- **`stack-upgrade.sh` doesn't check `$REPO`.** Nothing confirms the
|
||||
source checkout is root-owned or that its tree is clean before the
|
||||
upgrade builds from it. Filbert N2 on row 233 (29922, #1735).
|
||||
(2026-10-21)
|
||||
- **`stack-upgrade.sh` has no signal handling between A2 and A5.** A HUP,
|
||||
INT or TERM in that stretch leaves the run failed closed, but recovery
|
||||
is by hand. A trap that writes the stop point and the recovery step to
|
||||
the log would make it a sheet step. Filbert N3 on row 233 (29922,
|
||||
#1735). (2026-10-21)
|
||||
- **`checkCreative` doesn't refuse `credentials`.** Row 213's
|
||||
`checkCreative` (packages/business/src/role.mjs:143 at 2d2ad2760)
|
||||
limits a creative role's tools, authority and messaging, but a role
|
||||
file that gave companion or research a credential would still load.
|
||||
The business file could then map a credential to that instance
|
||||
(business.mjs:157). Nobody has traced whether a mapped credential
|
||||
reaches the run's environment. The decision paths stay closed (M6),
|
||||
but a creative role with `web` holding a live token is a wider channel
|
||||
than design 4 means. The fix: refuse any `credentials` in
|
||||
`checkCreative`, with a role.test.mjs case. It goes after row 213
|
||||
lands, as a follow-up row or in the next row that edits role.mjs.
|
||||
Webby N1 on row 213 r3 (REVIEW-r3.md, 3ab3c010b), mos-1160.
|
||||
(2026-10-21)
|
||||
- **The boot host check orders only one business's bus.**
|
||||
`mosaic-hostcheck.service` (row 240) has
|
||||
`[email protected]`. `case2-unit.sh` enables the
|
||||
bus as `mosaic-bus@<business>.service`, so on a host whose business id
|
||||
isn't `mosaic-stack` nothing orders the two. The bus can start before
|
||||
the boot stop runs, and after a boot-time drift role sessions can use
|
||||
bwrap until it does. The fix: `After=mosaic-hostcheck.service` in
|
||||
`packages/cli/systemd/mosaic-bus.service.in`, which orders every
|
||||
instance and does nothing where the unit isn't installed. It goes in
|
||||
before the first install whose business id isn't `mosaic-stack`.
|
||||
Filbert N1 on row 240 (29945, #1746). (2026-10-21)
|
||||
- **The run sheet writes the apt hook before the first `--check`.**
|
||||
Section 4 installs `80mosaic-hostcheck`, then runs
|
||||
`hostcheck.sh --check`. If that check fails, the next package run
|
||||
closes bwrap before anyone has read the failure. It fails closed, but
|
||||
it surprises an installer. The fix: run `--check` first, or say what a
|
||||
failure at that step means. Doc-only, for the next row that edits
|
||||
`docs/guides/host-run-sheet.md`. Filbert N2 on row 240 (29945, #1746).
|
||||
Filbert's N3, that a closed bwrap appends to `DRIFT` on every later
|
||||
apt run and boot, is by design; ops-01's STATE read should expect the
|
||||
repeats. (2026-10-21)
|
||||
|
||||
## Queue
|
||||
|
||||
|
||||
+6
-6
@@ -33,7 +33,7 @@ Gaps found while working go to `docs/plans/DEFERRED.md`, not here.
|
||||
|
||||
<!-- mosaic-queue:begin -->
|
||||
|
||||
Generated from `docs/plans/queue.json` revision 2617 by `queue render`. Do not edit between the markers; change the queue with `scripts/mosaic queue`.
|
||||
Generated from `docs/plans/queue.json` revision 2641 by `queue render`. Do not edit between the markers; change the queue with `scripts/mosaic queue`.
|
||||
|
||||
| # | Piece | Owner | Issues | State | After | Gate | Brief | Note |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
@@ -268,17 +268,17 @@ Generated from `docs/plans/queue.json` revision 2617 by `queue render`. Do not e
|
||||
| 229 | Row B: per-instance pi agent dir for Chet's Codex login, design only, starts after 10-27 (#428) | gosalyn; reviewers webby | #1730 | queued | — | A live check, with Jason's or Mos's go, that two clients on one ChatGPT login don't revoke each other's refresh tokens. No build row starts before it passes. Held until after 10-27. (sage) | `docs/plans/2026-10-20_case2-rows.md` § Brief: row B, a per-instance pi agent dir for Chet's Codex login (#428) | — |
|
||||
| 230 | Case 2 C2-13a unit render and host precheck (case2-unit.sh, case2-precheck.sh) | gosalyn; reviewers webby | #1732 | done | — | Webby approves the manifest. The test file and the cli suite are green on Sage's gate, along with every scripts/test-*.sh. A run on a host is a separate window with Jason's go. (sage) | `docs/plans/2026-10-20_case2-rows.md` § C2-13a unit render and host precheck | Landed c8bc5cf4c on refactor (base 390ea5261), pushed. Patch-id f4b49eb68997 matches git diff 722136772..96768766c on the 5 manifest paths (manifest fbb0d7e42c67, Webby approve r2 rev 2563). packages/cli/README.md 3-way with row 199, hunks apart. Gosalyn gate on 96768766c: cli 541/0, scripts 429/0, mutants 10/10. Merged tree check: case2-unit 30/0, host 42/0. |
|
||||
| 231 | Case 2 C2-01b survey escapes, list item lines and attachment embeds (row 212 N1-N3) | gosalyn; reviewers webby | #1733 | briefed | 212 done | Webby approves the manifest. The library tests are green on Sage's gate, along with every scripts/test-*.sh. (sage) | `docs/plans/2026-10-20_case2-rows.md` § C2-01b survey escapes, list item lines and attachment embeds | — |
|
||||
| 232 | Installer I1: stack-release external units and stack-copy bundle copies (MVP, Q187, Vikunja #432) | darkwing; reviewers filbert | #1734 | in-review, round 1, request posted | — | Filbert approves the manifest. The cli suite and every scripts/test-*.sh are green on Sage's gate, with the mutants killed. (sage) | `docs/plans/2026-10-20_installer-rows.md` § I1 stack-release external units and bundle copies | — |
|
||||
| 233 | Installer I3: stack-upgrade.sh root driver and the ops-01 host run sheet (MVP, Q187, Vikunja #432) | darkwing; reviewers filbert | #1735 | briefed | 232 done | Filbert approves the manifest. The test file, the cli suite and every scripts/test-*.sh are green on Sage's gate, with the mutants killed. (sage) | `docs/plans/2026-10-20_installer-rows.md` § I3 the upgrade command and the host run sheet | — |
|
||||
| 234 | Machine grant row a: host enrollment, store records and migration 3i-3j (#436a, design r6 f9039682e, lands by 10-25 before the .53/.54 installs) | gizmoduck; reviewers duckworth | #1740 | briefed | 210 done | Duckworth approves the manifest. The bus, business and cli suites and every scripts/test-*.sh are green on Sage's gate. The design's section 10 row a tests and mutants pass, the migration test included. (sage) (sage) | `docs/plans/2026-10-20_host-credentials.md` § Machine grant row a: enrollment, store records and migration 3i-3j | — |
|
||||
| 232 | Installer I1: stack-release external units and stack-copy bundle copies (MVP, Q187, Vikunja #432) | darkwing; reviewers filbert | #1734 | done | — | Filbert approves the manifest. The cli suite and every scripts/test-*.sh are green on Sage's gate, with the mutants killed. (sage) | `docs/plans/2026-10-20_installer-rows.md` § I1 stack-release external units and bundle copies | Landed e65a3ab4d on refactor (base 0444acd08), pushed. Patch-id 5adfbab9 matches git diff 8fede5fc6..da2da2f6c on the 9 manifest paths; tree entries (modes too) equal da2da2f6c, files pass manifest 7af3c943. Filbert approve r1 29829, r2 29865. Darkwing's landing gate on da2da2f6c: cli 593/0, scripts 429/0 (+156 queue node tests). Mutants 17/17. |
|
||||
| 233 | Installer I3: stack-upgrade.sh root driver and the ops-01 host run sheet (MVP, Q187, Vikunja #432) | darkwing; reviewers filbert | #1735 | in-progress | 232 done | Filbert approves the manifest. The test file, the cli suite and every scripts/test-*.sh are green on Sage's gate, with the mutants killed. (sage) | `docs/plans/2026-10-20_installer-rows.md` § I3 the upgrade command and the host run sheet | — |
|
||||
| 234 | Machine grant row a: host enrollment, store records and migration 3i-3j (#436a, design r6 f9039682e, lands by 10-25 before the .53/.54 installs) | gizmoduck; reviewers duckworth | #1740 | in-progress | 210 done | Duckworth approves the manifest. The bus, business and cli suites and every scripts/test-*.sh are green on Sage's gate. The design's section 10 row a tests and mutants pass, the migration test included. (sage) (sage) | `docs/plans/2026-10-20_host-credentials.md` § Machine grant row a: enrollment, store records and migration 3i-3j | — |
|
||||
| 235 | Machine grant row b: raise binding, executor, reconcile and revoke (#436, design r6 f9039682e, C1, N1, G27, G28) | gizmoduck; reviewers duckworth | #1741 | queued | 234 done | Duckworth approves the manifest. The bus, business and cli suites and every scripts/test-*.sh are green on Sage's gate. Section 10's row b tests and mutants pass, G27, G28 and the upsert mutant included. (sage) (sage) | `docs/plans/2026-10-20_host-credentials.md` § Machine grant row b: raise binding, executor, reconcile and revoke | — |
|
||||
| 236 | #436b Gitea grant path build: seat accounts and tokens through an approved grant, schema 3j-3k (design r5 1be535aca, due 10-27 at risk; live waits for Jason's section 8 approval and bootstrap step) | megavolt; reviewers dewey | #1742 | queued | 210 done, 234 done | Dewey approves the manifest. The bus, business and cli suites and every scripts/test-*.sh are green on Sage's gate. Section 9's mutants are killed, N9's restart test included. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: #436b Gitea grant path, build (schema 3j-3k) | — |
|
||||
| 237 | Row 199 follow-up N1: one decision publishes once (publish-busy or #spent recheck) | megavolt; reviewers duckworth | #1743 | done | 199 done | Duckworth approves the manifest. The bus suite and every scripts/test-*.sh are green on Sage's gate, and the mutant is killed. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: row 199 follow-up N1, one decision publishes once | Landed d7d943199 on refactor (local, push pending), patch-id c5f4beb4 = e42415935, manifest bb777cb0 blobs checked. Sage gate 01:31-01:44Z on 816d48b03+4 paths: M1 killed (bus 1/13, cli 1/8), every package suite and scripts/test-*.sh green (bus 478, cli 564). |
|
||||
| 238 | row 200 follow-up N1: the sso-config hint carries --instance NAME (#1744) | launchpad; reviewers webby | #1744 | in-review, round 1, request posted | 200 done | Webby approves the manifest; stack-release.test.mjs whole, cli suite and every scripts/test-*.sh green on Sage's gate; the mutant killed (sage) | `docs/plans/2026-10-20_stack-release-sso.md` § Brief: row 200 follow-up N1, the sso-config hint names the instance | — |
|
||||
| 239 | Row 194 follow-up: the credential revoke runs under the writer's lock, refusals after a revoke say so, and a stale-revision fire test (duckworth-0074, mos-1158) | honker; reviewers duckworth | #1745 | queued | 194 done | Duckworth approves the manifest. The bus, business, cli and webui suites and every scripts/test-*.sh are green on Sage's gate, and the mutants are killed. (sage) (sage) | `docs/plans/2026-10-20_agent-lifecycle.md` § Brief: row 194 follow-up, the revoke under the writer's lock and a fire test | — |
|
||||
| 240 | Installer I4: hostcheck, the bwrap profile check, pins and boot unit, from r6 B2 (MVP, Q187, Vikunja #432; split from I3) | darkwing; reviewers filbert | #1746 | queued | 233 done | Filbert approves the manifest. The test file, the cli suite and every scripts/test-*.sh are green on Sage's gate, with the mutants killed. (sage) (sage) | `docs/plans/2026-10-20_installer-rows.md` § I4 hostcheck: the bwrap profile check, pins and boot unit | — |
|
||||
| 241 | Row 237 follow-up: one decision lands once, a two-ticket broker land test and three mutants on the landing guards (megavolt-0069, sage-1651, P19 441) | megavolt; reviewers duckworth | #1747 | in-progress | 237 done | Duckworth approves the manifest. The bus, seat and cli suites and every scripts/test-*.sh are green on Sage's gate, and the three mutants are killed. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: row 237 follow-up, one decision lands once, with tests at the broker | — |
|
||||
| 242 | Row 237 follow-up N1: a retry after publish-timeout pushes once; hold the decision or kill and wait for the push, a slow-push retry test and a mutant (duckworth-0069, P19 440) | megavolt; reviewers duckworth | #1748 | briefed | 237 done | Duckworth approves the manifest. The bus and cli suites and every scripts/test-*.sh are green on Sage's gate, and the mutant is killed. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: row 237 follow-up N1, a retry after publish-timeout pushes once | — |
|
||||
| 241 | Row 237 follow-up: one decision lands once, a two-ticket broker land test and three mutants on the landing guards (megavolt-0069, sage-1651, P19 441) | megavolt; reviewers duckworth | #1747 | done | 237 done | Duckworth approves the manifest. The bus, seat and cli suites and every scripts/test-*.sh are green on Sage's gate, and the three mutants are killed. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: row 237 follow-up, one decision lands once, with tests at the broker | Landed 5d4a61776 on refactor, pushed (base e65a3ab4d). Blobs from 3e3ed303e on 2 paths; patch-id a1649f80 equals git diff f5753c0e8..3e3ed303e on those paths; ls-tree modes equal; manifest e8266864 verifies on the landed files. Sage slot on bbf3b1ee8: M1-M3 killed; gate 15 packages 2797/0, 10 script suites 429/0. Duckworth approve 29870 rev 2628. |
|
||||
| 242 | Row 237 follow-up N1: a retry after publish-timeout pushes once; hold the decision or kill and wait for the push, a slow-push retry test and a mutant (duckworth-0069, P19 440) | megavolt; reviewers duckworth | #1748 | done | 237 done | Duckworth approves the manifest. The bus and cli suites and every scripts/test-*.sh are green on Sage's gate, and the mutant is killed. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: row 237 follow-up N1, a retry after publish-timeout pushes once | Landed 5fb736b92 on refactor, pushed (base 96bac588f). Blobs from 635c10924 on 3 paths; patch-id ceb741ee equals git diff 8fede5fc6..635c10924 on those paths; ls-tree modes equal; manifest 1fbfab6c verifies on the landed files. Megavolt M1/M2 killed on Sage's slot; Sage gate 15 packages 2826/0, 10 script suites 429/0. Duckworth approve 29889 rev 2639. |
|
||||
|
||||
<!-- mosaic-queue:end -->
|
||||
|
||||
|
||||
+762
-30
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"version": 1,
|
||||
"canonicalRoot": "/mnt/storage/src/mosaic-stack",
|
||||
"revision": 2617,
|
||||
"revision": 2641,
|
||||
"rows": [
|
||||
{
|
||||
"id": 1,
|
||||
@@ -22631,7 +22631,7 @@
|
||||
"closes": [
|
||||
1734
|
||||
],
|
||||
"state": "in-review",
|
||||
"state": "done",
|
||||
"previousState": null,
|
||||
"required": false,
|
||||
"requiredSince": null,
|
||||
@@ -22687,18 +22687,55 @@
|
||||
"candidate": "7a3b4ea133cc4ef7573cd05338a937523991c8aa838782bf525781f6b76bd9c4"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"n": 2,
|
||||
"op": "darkwing-232-inreview-r2",
|
||||
"by": "darkwing",
|
||||
"at": "2026-10-21T02:58:36.861Z",
|
||||
"issue": 1734,
|
||||
"candidate": {
|
||||
"kind": "manifest",
|
||||
"digest": "7af3c94308e0ce99d88fb40624ad26592305624b139a95a0582991bdd2666b50",
|
||||
"text": "cec97e006778ea2df1e2cbdd25f1641ed3e1965cb1ea91ce782aa22007263357 agents/darkwing/work/queue-232/BUILD.md\nea516da8a6ccd5fa13e371bce83066f7fc7a6bffe545f155b7d7b076682dd40d docs/TOOLS.md\nd21c25c62fcbb9cad185c25ca2302bbb6b3e6ec53db5f4f85d7c445fb454c798 packages/cli/README.md\n681ed534f6ee6188ba5b628e882b17b5a4d9e748d6a649be622a0777a2ee0e87 packages/cli/tests/stack-copy.test.mjs\ned72964534f2289996672b08bc3b75539ec891a62688cedf9ea18a436b7df103 packages/cli/tests/stack-release-external.test.mjs\n896399deb3d0dd07651f965a53ede6717629dcb574876d418184a02ba093ada0 packages/cli/tests/stack-release-fixture.mjs\n533a91460f249bf0f9458fe4269795b8438ab209be7c4e6290e1ea0e29dd98d8 packages/cli/tests/stack-release.test.mjs\nc111fb361bac4fda2d2fdb4c1304e818c79995653ec36f346e44a863846cedc3 scripts/stack-copy.sh\n61db23e9c1b49878f6edfa0c07805d770c20103f06947f0bfcf72da6568f94c7 scripts/stack-release.sh\n"
|
||||
},
|
||||
"request": "comment",
|
||||
"attempts": [
|
||||
{
|
||||
"op": "darkwing-232-inreview-r2",
|
||||
"by": "darkwing",
|
||||
"at": "2026-10-21T02:58:36.861Z",
|
||||
"state": "posted",
|
||||
"comment": 29860,
|
||||
"transport": {
|
||||
"outcome": "posted",
|
||||
"status": 201,
|
||||
"comment": 29860,
|
||||
"detail": "created"
|
||||
},
|
||||
"resolutions": []
|
||||
}
|
||||
],
|
||||
"duplicateRisk": false,
|
||||
"receipts": [
|
||||
{
|
||||
"reviewer": "filbert",
|
||||
"op": "filbert-232-record-r2",
|
||||
"at": "2026-10-21T03:01:22.609Z",
|
||||
"verdict": "approve",
|
||||
"comment": 29865,
|
||||
"candidate": "7af3c94308e0ce99d88fb40624ad26592305624b139a95a0582991bdd2666b50"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"claim": {
|
||||
"seat": "darkwing",
|
||||
"op": "darkwing-232-inprogress-r1"
|
||||
},
|
||||
"note": null,
|
||||
"claim": null,
|
||||
"note": "Landed e65a3ab4d on refactor (base 0444acd08), pushed. Patch-id 5adfbab9 matches git diff 8fede5fc6..da2da2f6c on the 9 manifest paths; tree entries (modes too) equal da2da2f6c, files pass manifest 7af3c943. Filbert approve r1 29829, r2 29865. Darkwing's landing gate on da2da2f6c: cli 593/0, scripts 429/0 (+156 queue node tests). Mutants 17/17.",
|
||||
"blockedReason": null,
|
||||
"createdAt": "2026-10-20T22:56:29.929Z",
|
||||
"updatedAt": "2026-10-21T01:33:53.697Z",
|
||||
"updatedBy": "filbert"
|
||||
"updatedAt": "2026-10-21T03:20:36.830Z",
|
||||
"updatedBy": "sage"
|
||||
},
|
||||
{
|
||||
"id": 233,
|
||||
@@ -22710,7 +22747,7 @@
|
||||
"closes": [
|
||||
1735
|
||||
],
|
||||
"state": "briefed",
|
||||
"state": "in-progress",
|
||||
"previousState": null,
|
||||
"required": false,
|
||||
"requiredSince": null,
|
||||
@@ -22719,7 +22756,7 @@
|
||||
"brief": {
|
||||
"path": "docs/plans/2026-10-20_installer-rows.md",
|
||||
"anchor": "I3 the upgrade command and the host run sheet",
|
||||
"blob": "f35e7e57daee39a760159e069994f213d733b167"
|
||||
"blob": "e61291bcf1c6e9f43f599ca94fe9c6eb0dce1383"
|
||||
},
|
||||
"after": [
|
||||
{
|
||||
@@ -22731,12 +22768,15 @@
|
||||
"filbert"
|
||||
],
|
||||
"review": null,
|
||||
"claim": null,
|
||||
"claim": {
|
||||
"seat": "darkwing",
|
||||
"op": "darkwing-233-inprogress-r1"
|
||||
},
|
||||
"note": null,
|
||||
"blockedReason": null,
|
||||
"createdAt": "2026-10-20T22:56:59.719Z",
|
||||
"updatedAt": "2026-10-21T01:30:41.511Z",
|
||||
"updatedBy": "sage"
|
||||
"updatedAt": "2026-10-21T03:25:49.757Z",
|
||||
"updatedBy": "darkwing"
|
||||
},
|
||||
{
|
||||
"id": 234,
|
||||
@@ -22748,7 +22788,7 @@
|
||||
"closes": [
|
||||
1740
|
||||
],
|
||||
"state": "briefed",
|
||||
"state": "in-progress",
|
||||
"previousState": null,
|
||||
"required": false,
|
||||
"requiredSince": null,
|
||||
@@ -22769,12 +22809,15 @@
|
||||
"duckworth"
|
||||
],
|
||||
"review": null,
|
||||
"claim": null,
|
||||
"claim": {
|
||||
"seat": "gizmoduck",
|
||||
"op": "gizmoduck-234-claim-1"
|
||||
},
|
||||
"note": null,
|
||||
"blockedReason": null,
|
||||
"createdAt": "2026-10-20T23:24:50.778Z",
|
||||
"updatedAt": "2026-10-21T02:49:40.096Z",
|
||||
"updatedBy": "sage"
|
||||
"updatedAt": "2026-10-21T02:53:18.435Z",
|
||||
"updatedBy": "gizmoduck"
|
||||
},
|
||||
{
|
||||
"id": 235,
|
||||
@@ -23135,7 +23178,7 @@
|
||||
"closes": [
|
||||
1747
|
||||
],
|
||||
"state": "in-progress",
|
||||
"state": "done",
|
||||
"previousState": null,
|
||||
"required": false,
|
||||
"requiredSince": null,
|
||||
@@ -23155,16 +23198,74 @@
|
||||
"reviewers": [
|
||||
"duckworth"
|
||||
],
|
||||
"review": null,
|
||||
"claim": {
|
||||
"seat": "megavolt",
|
||||
"op": "megavolt-241-start-1"
|
||||
"review": {
|
||||
"rounds": [
|
||||
{
|
||||
"n": 1,
|
||||
"op": "megavolt-241-inreview-r1",
|
||||
"by": "megavolt",
|
||||
"at": "2026-10-21T03:01:47.822Z",
|
||||
"issue": 1747,
|
||||
"candidate": {
|
||||
"kind": "manifest",
|
||||
"digest": "e82668642589eefca8b22433f1b0c7a24432f1993c0050f0a4b74c1146c93c2f",
|
||||
"text": "e94f98ce2bdefe85c1ea2e1af165473f9d9b04860700e84ed7382798e4b43523 agents/megavolt/work/queue-241/BUILD.md\n8351c9df4673c441f1ea8f2fdf2639e643b264180170fd1c93e0fca8d9359dae packages/bus/tests/land.test.mjs\n"
|
||||
},
|
||||
"request": "comment",
|
||||
"attempts": [
|
||||
{
|
||||
"op": "megavolt-241-inreview-r1",
|
||||
"by": "megavolt",
|
||||
"at": "2026-10-21T03:01:47.822Z",
|
||||
"state": "failed",
|
||||
"comment": null,
|
||||
"transport": {
|
||||
"outcome": "failed",
|
||||
"status": null,
|
||||
"comment": null,
|
||||
"detail": "pre-send: the credential or account check failed"
|
||||
},
|
||||
"resolutions": []
|
||||
},
|
||||
{
|
||||
"op": "sage-241-request-1",
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T03:02:56.461Z",
|
||||
"state": "posted",
|
||||
"comment": 29868,
|
||||
"transport": null,
|
||||
"resolutions": [
|
||||
{
|
||||
"verb": "resolve",
|
||||
"op": "sage-241-request-1-resolve",
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T03:04:33.826Z",
|
||||
"comment": 29868,
|
||||
"reason": null
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"duplicateRisk": false,
|
||||
"receipts": [
|
||||
{
|
||||
"reviewer": "duckworth",
|
||||
"op": "duckworth-r241-record-1",
|
||||
"at": "2026-10-21T03:06:16.441Z",
|
||||
"verdict": "approve",
|
||||
"comment": 29870,
|
||||
"candidate": "e82668642589eefca8b22433f1b0c7a24432f1993c0050f0a4b74c1146c93c2f"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"note": null,
|
||||
"claim": null,
|
||||
"note": "Landed 5d4a61776 on refactor, pushed (base e65a3ab4d). Blobs from 3e3ed303e on 2 paths; patch-id a1649f80 equals git diff f5753c0e8..3e3ed303e on those paths; ls-tree modes equal; manifest e8266864 verifies on the landed files. Sage slot on bbf3b1ee8: M1-M3 killed; gate 15 packages 2797/0, 10 script suites 429/0. Duckworth approve 29870 rev 2628.",
|
||||
"blockedReason": null,
|
||||
"createdAt": "2026-10-21T01:52:55.772Z",
|
||||
"updatedAt": "2026-10-21T01:54:02.058Z",
|
||||
"updatedBy": "megavolt"
|
||||
"updatedAt": "2026-10-21T03:35:52.657Z",
|
||||
"updatedBy": "sage"
|
||||
},
|
||||
{
|
||||
"id": 242,
|
||||
@@ -23176,7 +23277,7 @@
|
||||
"closes": [
|
||||
1748
|
||||
],
|
||||
"state": "briefed",
|
||||
"state": "done",
|
||||
"previousState": null,
|
||||
"required": false,
|
||||
"requiredSince": null,
|
||||
@@ -23196,12 +23297,69 @@
|
||||
"reviewers": [
|
||||
"duckworth"
|
||||
],
|
||||
"review": null,
|
||||
"review": {
|
||||
"rounds": [
|
||||
{
|
||||
"n": 1,
|
||||
"op": "megavolt-242-inreview-r1",
|
||||
"by": "megavolt",
|
||||
"at": "2026-10-21T03:44:17.331Z",
|
||||
"issue": 1748,
|
||||
"candidate": {
|
||||
"kind": "manifest",
|
||||
"digest": "1fbfab6c68e8d76d2d69b13c415257280d4fa8c5584e24d5dfd13dca1420a3da",
|
||||
"text": "d245f805bfc9ad8a0064a8898effa05148e9c3fb7deb22b85f7af499e0ded369 agents/megavolt/work/queue-242/BUILD.md\ncfecc67698ad18a94288ba4e852b372d92c8d1cb984494256f929b03a467baad packages/bus/src/runtime.mjs\na0783575b7da4e3b8e1ae161d7031fdf62a8a1a402a48ac898ef1b5c2d378d9b packages/cli/tests/publish-socket.test.mjs\n"
|
||||
},
|
||||
"request": "comment",
|
||||
"attempts": [
|
||||
{
|
||||
"op": "megavolt-242-inreview-r1",
|
||||
"by": "megavolt",
|
||||
"at": "2026-10-21T03:44:17.331Z",
|
||||
"state": "failed",
|
||||
"comment": null,
|
||||
"transport": {
|
||||
"outcome": "failed",
|
||||
"status": null,
|
||||
"comment": null,
|
||||
"detail": "pre-send: the credential or account check failed"
|
||||
},
|
||||
"resolutions": []
|
||||
},
|
||||
{
|
||||
"op": "sage-242-request-1",
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T03:45:41.063Z",
|
||||
"state": "posted",
|
||||
"comment": 29888,
|
||||
"transport": {
|
||||
"outcome": "posted",
|
||||
"status": 201,
|
||||
"comment": 29888,
|
||||
"detail": "created"
|
||||
},
|
||||
"resolutions": []
|
||||
}
|
||||
],
|
||||
"duplicateRisk": false,
|
||||
"receipts": [
|
||||
{
|
||||
"reviewer": "duckworth",
|
||||
"op": "duckworth-r242-record-1",
|
||||
"at": "2026-10-21T03:48:36.391Z",
|
||||
"verdict": "approve",
|
||||
"comment": 29889,
|
||||
"candidate": "1fbfab6c68e8d76d2d69b13c415257280d4fa8c5584e24d5dfd13dca1420a3da"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"claim": null,
|
||||
"note": null,
|
||||
"note": "Landed 5fb736b92 on refactor, pushed (base 96bac588f). Blobs from 635c10924 on 3 paths; patch-id ceb741ee equals git diff 8fede5fc6..635c10924 on those paths; ls-tree modes equal; manifest 1fbfab6c verifies on the landed files. Megavolt M1/M2 killed on Sage's slot; Sage gate 15 packages 2826/0, 10 script suites 429/0. Duckworth approve 29889 rev 2639.",
|
||||
"blockedReason": null,
|
||||
"createdAt": "2026-10-21T01:54:53.103Z",
|
||||
"updatedAt": "2026-10-21T01:55:17.835Z",
|
||||
"updatedAt": "2026-10-21T04:13:55.780Z",
|
||||
"updatedBy": "sage"
|
||||
}
|
||||
],
|
||||
@@ -92612,6 +92770,580 @@
|
||||
"receipt": "ok sage-234-briefed rev 2617 row 234 queued→briefed"
|
||||
},
|
||||
"viewSha": "99921e9a304f3fec10159d53cb04451ab1fd1a56a0bf4683e88f490aca94d5c8"
|
||||
},
|
||||
{
|
||||
"rev": 2618,
|
||||
"op": "gizmoduck-234-claim-1",
|
||||
"verb": "move",
|
||||
"args": {
|
||||
"id": 234,
|
||||
"to": "in-progress",
|
||||
"reason": null,
|
||||
"candidate": null,
|
||||
"evidence": null,
|
||||
"issue": null
|
||||
},
|
||||
"by": "gizmoduck",
|
||||
"at": "2026-10-21T02:53:18.435Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 234,
|
||||
"from": "briefed",
|
||||
"to": "in-progress",
|
||||
"round": null,
|
||||
"issue": null,
|
||||
"candidate": null,
|
||||
"receipt": "ok gizmoduck-234-claim-1 rev 2618 row 234 briefed→in-progress"
|
||||
},
|
||||
"viewSha": "056671c84c73a4175e57b541dc0b528910786ed10b2dd5e48138641fa6dc1212"
|
||||
},
|
||||
{
|
||||
"rev": 2619,
|
||||
"op": "darkwing-232-inprogress-r2",
|
||||
"verb": "move",
|
||||
"args": {
|
||||
"id": 232,
|
||||
"to": "in-progress",
|
||||
"reason": null,
|
||||
"candidate": null,
|
||||
"evidence": null,
|
||||
"issue": null
|
||||
},
|
||||
"by": "darkwing",
|
||||
"at": "2026-10-21T02:58:12.490Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 232,
|
||||
"from": "in-review",
|
||||
"to": "in-progress",
|
||||
"round": null,
|
||||
"issue": null,
|
||||
"candidate": null,
|
||||
"receipt": "ok darkwing-232-inprogress-r2 rev 2619 row 232 in-review→in-progress"
|
||||
},
|
||||
"viewSha": "3018370333dff232c9672ca379c57d1123163b47aeb3feb84977709ae3517ea3"
|
||||
},
|
||||
{
|
||||
"rev": 2620,
|
||||
"op": "darkwing-232-inreview-r2",
|
||||
"verb": "move",
|
||||
"args": {
|
||||
"id": 232,
|
||||
"to": "in-review",
|
||||
"reason": null,
|
||||
"candidate": "/home/jwoltje/darkwing-scratch/r232/candidate-manifest-r2.sha256",
|
||||
"evidence": null,
|
||||
"issue": null
|
||||
},
|
||||
"by": "darkwing",
|
||||
"at": "2026-10-21T02:58:36.861Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 232,
|
||||
"from": "in-progress",
|
||||
"to": "in-review",
|
||||
"round": 2,
|
||||
"issue": 1734,
|
||||
"candidate": {
|
||||
"kind": "manifest",
|
||||
"digest": "7af3c94308e0ce99d88fb40624ad26592305624b139a95a0582991bdd2666b50",
|
||||
"text": "cec97e006778ea2df1e2cbdd25f1641ed3e1965cb1ea91ce782aa22007263357 agents/darkwing/work/queue-232/BUILD.md\nea516da8a6ccd5fa13e371bce83066f7fc7a6bffe545f155b7d7b076682dd40d docs/TOOLS.md\nd21c25c62fcbb9cad185c25ca2302bbb6b3e6ec53db5f4f85d7c445fb454c798 packages/cli/README.md\n681ed534f6ee6188ba5b628e882b17b5a4d9e748d6a649be622a0777a2ee0e87 packages/cli/tests/stack-copy.test.mjs\ned72964534f2289996672b08bc3b75539ec891a62688cedf9ea18a436b7df103 packages/cli/tests/stack-release-external.test.mjs\n896399deb3d0dd07651f965a53ede6717629dcb574876d418184a02ba093ada0 packages/cli/tests/stack-release-fixture.mjs\n533a91460f249bf0f9458fe4269795b8438ab209be7c4e6290e1ea0e29dd98d8 packages/cli/tests/stack-release.test.mjs\nc111fb361bac4fda2d2fdb4c1304e818c79995653ec36f346e44a863846cedc3 scripts/stack-copy.sh\n61db23e9c1b49878f6edfa0c07805d770c20103f06947f0bfcf72da6568f94c7 scripts/stack-release.sh\n"
|
||||
},
|
||||
"receipt": "ok darkwing-232-inreview-r2 rev 2620 row 232 in-progress→in-review round 2 on #1734; request darkwing-232-inreview-r2 requesting"
|
||||
},
|
||||
"viewSha": "f2b47852ccea465a933c307c1ba35d529b0646c59c6f294113aa5bc21feae00d"
|
||||
},
|
||||
{
|
||||
"rev": 2621,
|
||||
"op": "darkwing-232-inreview-r2.outcome",
|
||||
"verb": "review-outcome",
|
||||
"args": {
|
||||
"id": 232,
|
||||
"attempt": "darkwing-232-inreview-r2",
|
||||
"outcome": "posted",
|
||||
"status": 201,
|
||||
"comment": 29860,
|
||||
"detail": "created"
|
||||
},
|
||||
"by": "darkwing",
|
||||
"at": "2026-10-21T02:59:02.511Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 232,
|
||||
"round": 2,
|
||||
"attempt": "darkwing-232-inreview-r2",
|
||||
"state": "posted",
|
||||
"receipt": "ok darkwing-232-inreview-r2.outcome rev 2621 row 232 round 2 request darkwing-232-inreview-r2 posted comment 29860"
|
||||
},
|
||||
"viewSha": "47603fa4a7e49e17ac510cfb8c2a031829362ce92ce7c094ca1a3c6e3afe418f"
|
||||
},
|
||||
{
|
||||
"rev": 2622,
|
||||
"op": "filbert-232-record-r2",
|
||||
"verb": "review-record",
|
||||
"args": {
|
||||
"id": 232,
|
||||
"verdict": "approve",
|
||||
"comment": 29865,
|
||||
"candidate": "7af3c94308e0ce99d88fb40624ad26592305624b139a95a0582991bdd2666b50"
|
||||
},
|
||||
"by": "filbert",
|
||||
"at": "2026-10-21T03:01:22.609Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 232,
|
||||
"round": 2,
|
||||
"verdict": "approve",
|
||||
"receipt": "ok filbert-232-record-r2 rev 2622 row 232 round 2 approve by filbert (comment 29865)"
|
||||
},
|
||||
"viewSha": "69cd7e81ad20bc0dedb61b4c90962c6867d41fd8002f862687644d870f7e27ea"
|
||||
},
|
||||
{
|
||||
"rev": 2623,
|
||||
"op": "megavolt-241-inreview-r1",
|
||||
"verb": "move",
|
||||
"args": {
|
||||
"id": 241,
|
||||
"to": "in-review",
|
||||
"reason": null,
|
||||
"candidate": "/mnt/storage/scratch/megavolt-r241/manifest-r1-3e3ed303e.txt",
|
||||
"evidence": null,
|
||||
"issue": null
|
||||
},
|
||||
"by": "megavolt",
|
||||
"at": "2026-10-21T03:01:47.822Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 241,
|
||||
"from": "in-progress",
|
||||
"to": "in-review",
|
||||
"round": 1,
|
||||
"issue": 1747,
|
||||
"candidate": {
|
||||
"kind": "manifest",
|
||||
"digest": "e82668642589eefca8b22433f1b0c7a24432f1993c0050f0a4b74c1146c93c2f",
|
||||
"text": "e94f98ce2bdefe85c1ea2e1af165473f9d9b04860700e84ed7382798e4b43523 agents/megavolt/work/queue-241/BUILD.md\n8351c9df4673c441f1ea8f2fdf2639e643b264180170fd1c93e0fca8d9359dae packages/bus/tests/land.test.mjs\n"
|
||||
},
|
||||
"receipt": "ok megavolt-241-inreview-r1 rev 2623 row 241 in-progress→in-review round 1 on #1747; request megavolt-241-inreview-r1 requesting"
|
||||
},
|
||||
"viewSha": "60d7b7151ebee0eb8e0eb8916cfda532e2ca7b15debe7f128129576b73087ad8"
|
||||
},
|
||||
{
|
||||
"rev": 2624,
|
||||
"op": "megavolt-241-inreview-r1.outcome",
|
||||
"verb": "review-outcome",
|
||||
"args": {
|
||||
"id": 241,
|
||||
"attempt": "megavolt-241-inreview-r1",
|
||||
"outcome": "failed",
|
||||
"status": null,
|
||||
"comment": null,
|
||||
"detail": "pre-send: the credential or account check failed"
|
||||
},
|
||||
"by": "megavolt",
|
||||
"at": "2026-10-21T03:02:12.573Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 241,
|
||||
"round": 1,
|
||||
"attempt": "megavolt-241-inreview-r1",
|
||||
"state": "failed",
|
||||
"receipt": "ok megavolt-241-inreview-r1.outcome rev 2624 row 241 round 1 request megavolt-241-inreview-r1 failed"
|
||||
},
|
||||
"viewSha": "5e4e5c177acdd24d83c6b094937b937d8f826b530c13fd6e2da926d318fc9f22"
|
||||
},
|
||||
{
|
||||
"rev": 2625,
|
||||
"op": "sage-241-request-1",
|
||||
"verb": "review-request",
|
||||
"args": {
|
||||
"id": 241
|
||||
},
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T03:02:56.461Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 241,
|
||||
"round": 1,
|
||||
"attempt": "sage-241-request-1",
|
||||
"state": "requesting",
|
||||
"receipt": "ok sage-241-request-1 rev 2625 row 241 round 1 request sage-241-request-1 requesting"
|
||||
},
|
||||
"viewSha": "29f9f090c272ecb96782f36db298a78c9b6a2fc268ce0895d9a414bcec4cad2f"
|
||||
},
|
||||
{
|
||||
"rev": 2626,
|
||||
"op": "megavolt-242-start-1",
|
||||
"verb": "move",
|
||||
"args": {
|
||||
"id": 242,
|
||||
"to": "in-progress",
|
||||
"reason": null,
|
||||
"candidate": null,
|
||||
"evidence": null,
|
||||
"issue": null
|
||||
},
|
||||
"by": "megavolt",
|
||||
"at": "2026-10-21T03:03:20.738Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 242,
|
||||
"from": "briefed",
|
||||
"to": "in-progress",
|
||||
"round": null,
|
||||
"issue": null,
|
||||
"candidate": null,
|
||||
"receipt": "ok megavolt-242-start-1 rev 2626 row 242 briefed→in-progress"
|
||||
},
|
||||
"viewSha": "9ed411c0f2ba8ecbc8ee58657a30a585770c8311861f41dd607686f1bcd98992"
|
||||
},
|
||||
{
|
||||
"rev": 2627,
|
||||
"op": "sage-241-request-1-resolve",
|
||||
"verb": "review-resolve",
|
||||
"args": {
|
||||
"id": 241,
|
||||
"attempt": "sage-241-request-1",
|
||||
"comment": 29868
|
||||
},
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T03:04:33.826Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 241,
|
||||
"round": 1,
|
||||
"attempt": "sage-241-request-1",
|
||||
"state": "posted",
|
||||
"receipt": "ok sage-241-request-1-resolve rev 2627 row 241 round 1 request sage-241-request-1 requesting→posted comment 29868"
|
||||
},
|
||||
"viewSha": "ec48b0f767195fb8ddaa8e67e9526a533ef4ddc789d6b11c0998befbf32433de"
|
||||
},
|
||||
{
|
||||
"rev": 2628,
|
||||
"op": "duckworth-r241-record-1",
|
||||
"verb": "review-record",
|
||||
"args": {
|
||||
"id": 241,
|
||||
"verdict": "approve",
|
||||
"comment": 29870,
|
||||
"candidate": "e82668642589eefca8b22433f1b0c7a24432f1993c0050f0a4b74c1146c93c2f"
|
||||
},
|
||||
"by": "duckworth",
|
||||
"at": "2026-10-21T03:06:16.441Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 241,
|
||||
"round": 1,
|
||||
"verdict": "approve",
|
||||
"receipt": "ok duckworth-r241-record-1 rev 2628 row 241 round 1 approve by duckworth (comment 29870)"
|
||||
},
|
||||
"viewSha": "094f036ccb430caef42be43e00643081c1ed2f7ec8253b865faba8dd28ad7298"
|
||||
},
|
||||
{
|
||||
"rev": 2629,
|
||||
"op": "sage-232-landing-note",
|
||||
"verb": "note",
|
||||
"args": {
|
||||
"id": 232,
|
||||
"text": "Landed e65a3ab4d on refactor (base 0444acd08), pushed. Patch-id 5adfbab9 matches git diff 8fede5fc6..da2da2f6c on the 9 manifest paths; tree entries (modes too) equal da2da2f6c, files pass manifest 7af3c943. Filbert approve r1 29829, r2 29865. Darkwing's landing gate on da2da2f6c: cli 593/0, scripts 429/0 (+156 queue node tests). Mutants 17/17."
|
||||
},
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T03:20:12.732Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 232,
|
||||
"field": "note",
|
||||
"receipt": "ok sage-232-landing-note rev 2629 row 232 note"
|
||||
},
|
||||
"viewSha": "e0f5ff1d8b8c1592a39c64a2df6b32d4d8f2a65e4ee095566a01989d491250ea"
|
||||
},
|
||||
{
|
||||
"rev": 2630,
|
||||
"op": "sage-232-done",
|
||||
"verb": "move",
|
||||
"args": {
|
||||
"id": 232,
|
||||
"to": "done",
|
||||
"reason": null,
|
||||
"candidate": null,
|
||||
"evidence": null,
|
||||
"issue": null
|
||||
},
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T03:20:36.830Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 232,
|
||||
"from": "in-review",
|
||||
"to": "done",
|
||||
"round": 2,
|
||||
"issue": null,
|
||||
"candidate": null,
|
||||
"receipt": "ok sage-232-done rev 2630 row 232 in-review→done round 2"
|
||||
},
|
||||
"viewSha": "8d3708d10c659ec2d626c28be1cb87089cd606574278e9b51b99fae515928050"
|
||||
},
|
||||
{
|
||||
"rev": 2631,
|
||||
"op": "sage-233-repin-brief-1",
|
||||
"verb": "set",
|
||||
"args": {
|
||||
"id": 233,
|
||||
"field": "brief",
|
||||
"value": "docs/plans/2026-10-20_installer-rows.md#I3 the upgrade command and the host run sheet",
|
||||
"reason": null
|
||||
},
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T03:25:06.360Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 233,
|
||||
"field": "brief",
|
||||
"from": {
|
||||
"path": "docs/plans/2026-10-20_installer-rows.md",
|
||||
"anchor": "I3 the upgrade command and the host run sheet",
|
||||
"blob": "f35e7e57daee39a760159e069994f213d733b167"
|
||||
},
|
||||
"to": {
|
||||
"path": "docs/plans/2026-10-20_installer-rows.md",
|
||||
"anchor": "I3 the upgrade command and the host run sheet",
|
||||
"blob": "e61291bcf1c6e9f43f599ca94fe9c6eb0dce1383"
|
||||
},
|
||||
"receipt": "ok sage-233-repin-brief-1 rev 2631 row 233 brief: docs/plans/2026-10-20_installer-rows.md § I3 the upgrade command and the host run sheet @f35e7e57daee→docs/plans/2026-10-20_installer-rows.md § I3 the upgrade command and the host run sheet @e61291bcf1c6"
|
||||
},
|
||||
"viewSha": "da749491f801e3ec47f908872aa4453fc3caa30b23e4194a205ddfd9d076241e"
|
||||
},
|
||||
{
|
||||
"rev": 2632,
|
||||
"op": "darkwing-233-inprogress-r1",
|
||||
"verb": "move",
|
||||
"args": {
|
||||
"id": 233,
|
||||
"to": "in-progress",
|
||||
"reason": null,
|
||||
"candidate": null,
|
||||
"evidence": null,
|
||||
"issue": null
|
||||
},
|
||||
"by": "darkwing",
|
||||
"at": "2026-10-21T03:25:49.757Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 233,
|
||||
"from": "briefed",
|
||||
"to": "in-progress",
|
||||
"round": null,
|
||||
"issue": null,
|
||||
"candidate": null,
|
||||
"receipt": "ok darkwing-233-inprogress-r1 rev 2632 row 233 briefed→in-progress"
|
||||
},
|
||||
"viewSha": "f77ca73f963f35d77733bfe6fa5e8ad3a4144d3ae6b7aa2fcba1d62eb3da4e12"
|
||||
},
|
||||
{
|
||||
"rev": 2633,
|
||||
"op": "sage-241-landed-note",
|
||||
"verb": "note",
|
||||
"args": {
|
||||
"id": 241,
|
||||
"text": "Landed 5d4a61776 on refactor, pushed (base e65a3ab4d). Blobs from 3e3ed303e on 2 paths; patch-id a1649f80 equals git diff f5753c0e8..3e3ed303e on those paths; ls-tree modes equal; manifest e8266864 verifies on the landed files. Sage slot on bbf3b1ee8: M1-M3 killed; gate 15 packages 2797/0, 10 script suites 429/0. Duckworth approve 29870 rev 2628."
|
||||
},
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T03:35:10.676Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 241,
|
||||
"field": "note",
|
||||
"receipt": "ok sage-241-landed-note rev 2633 row 241 note"
|
||||
},
|
||||
"viewSha": "60a6381f3fa3ff4ca5774baed90a9b383fd69b09bf88afc6c5aa123fa0379208"
|
||||
},
|
||||
{
|
||||
"rev": 2634,
|
||||
"op": "sage-241-done",
|
||||
"verb": "move",
|
||||
"args": {
|
||||
"id": 241,
|
||||
"to": "done",
|
||||
"reason": null,
|
||||
"candidate": null,
|
||||
"evidence": null,
|
||||
"issue": null
|
||||
},
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T03:35:52.657Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 241,
|
||||
"from": "in-review",
|
||||
"to": "done",
|
||||
"round": 1,
|
||||
"issue": null,
|
||||
"candidate": null,
|
||||
"receipt": "ok sage-241-done rev 2634 row 241 in-review→done round 1"
|
||||
},
|
||||
"viewSha": "369663313667edd50358fd170a5e652a06332ec394918089a02cc79b19eed63f"
|
||||
},
|
||||
{
|
||||
"rev": 2635,
|
||||
"op": "megavolt-242-inreview-r1",
|
||||
"verb": "move",
|
||||
"args": {
|
||||
"id": 242,
|
||||
"to": "in-review",
|
||||
"reason": null,
|
||||
"candidate": "/mnt/storage/scratch/megavolt-r242/manifest-r1-635c10924.txt",
|
||||
"evidence": null,
|
||||
"issue": null
|
||||
},
|
||||
"by": "megavolt",
|
||||
"at": "2026-10-21T03:44:17.331Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 242,
|
||||
"from": "in-progress",
|
||||
"to": "in-review",
|
||||
"round": 1,
|
||||
"issue": 1748,
|
||||
"candidate": {
|
||||
"kind": "manifest",
|
||||
"digest": "1fbfab6c68e8d76d2d69b13c415257280d4fa8c5584e24d5dfd13dca1420a3da",
|
||||
"text": "d245f805bfc9ad8a0064a8898effa05148e9c3fb7deb22b85f7af499e0ded369 agents/megavolt/work/queue-242/BUILD.md\ncfecc67698ad18a94288ba4e852b372d92c8d1cb984494256f929b03a467baad packages/bus/src/runtime.mjs\na0783575b7da4e3b8e1ae161d7031fdf62a8a1a402a48ac898ef1b5c2d378d9b packages/cli/tests/publish-socket.test.mjs\n"
|
||||
},
|
||||
"receipt": "ok megavolt-242-inreview-r1 rev 2635 row 242 in-progress→in-review round 1 on #1748; request megavolt-242-inreview-r1 requesting"
|
||||
},
|
||||
"viewSha": "432a911c27ffb0374b1ca49cb60215cb4211ce4c8a7f02bdfff91054f6a54e69"
|
||||
},
|
||||
{
|
||||
"rev": 2636,
|
||||
"op": "megavolt-242-inreview-r1.outcome",
|
||||
"verb": "review-outcome",
|
||||
"args": {
|
||||
"id": 242,
|
||||
"attempt": "megavolt-242-inreview-r1",
|
||||
"outcome": "failed",
|
||||
"status": null,
|
||||
"comment": null,
|
||||
"detail": "pre-send: the credential or account check failed"
|
||||
},
|
||||
"by": "megavolt",
|
||||
"at": "2026-10-21T03:44:42.695Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 242,
|
||||
"round": 1,
|
||||
"attempt": "megavolt-242-inreview-r1",
|
||||
"state": "failed",
|
||||
"receipt": "ok megavolt-242-inreview-r1.outcome rev 2636 row 242 round 1 request megavolt-242-inreview-r1 failed"
|
||||
},
|
||||
"viewSha": "42cd5e62d4b725838d45d409f48444e769c173db4825dcdc22e33c86a297ecda"
|
||||
},
|
||||
{
|
||||
"rev": 2637,
|
||||
"op": "sage-242-request-1",
|
||||
"verb": "review-request",
|
||||
"args": {
|
||||
"id": 242
|
||||
},
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T03:45:41.063Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 242,
|
||||
"round": 1,
|
||||
"attempt": "sage-242-request-1",
|
||||
"state": "requesting",
|
||||
"receipt": "ok sage-242-request-1 rev 2637 row 242 round 1 request sage-242-request-1 requesting"
|
||||
},
|
||||
"viewSha": "bc6018e64090bd29a7770b4f03511edc1314428a99d9114b3b274f2d248ce228"
|
||||
},
|
||||
{
|
||||
"rev": 2638,
|
||||
"op": "sage-242-request-1.outcome",
|
||||
"verb": "review-outcome",
|
||||
"args": {
|
||||
"id": 242,
|
||||
"attempt": "sage-242-request-1",
|
||||
"outcome": "posted",
|
||||
"status": 201,
|
||||
"comment": 29888,
|
||||
"detail": "created"
|
||||
},
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T03:46:05.721Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 242,
|
||||
"round": 1,
|
||||
"attempt": "sage-242-request-1",
|
||||
"state": "posted",
|
||||
"receipt": "ok sage-242-request-1.outcome rev 2638 row 242 round 1 request sage-242-request-1 posted comment 29888"
|
||||
},
|
||||
"viewSha": "d7bf764deb95140e4b2748863b838f5598f57162a0388adcfb24b4ca6cb90449"
|
||||
},
|
||||
{
|
||||
"rev": 2639,
|
||||
"op": "duckworth-r242-record-1",
|
||||
"verb": "review-record",
|
||||
"args": {
|
||||
"id": 242,
|
||||
"verdict": "approve",
|
||||
"comment": 29889,
|
||||
"candidate": "1fbfab6c68e8d76d2d69b13c415257280d4fa8c5584e24d5dfd13dca1420a3da"
|
||||
},
|
||||
"by": "duckworth",
|
||||
"at": "2026-10-21T03:48:36.391Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 242,
|
||||
"round": 1,
|
||||
"verdict": "approve",
|
||||
"receipt": "ok duckworth-r242-record-1 rev 2639 row 242 round 1 approve by duckworth (comment 29889)"
|
||||
},
|
||||
"viewSha": "80ea9e552b2edb1937fce5ff23bc6cf9b6b9e2381cbb2e5808d7c87acf3d4b45"
|
||||
},
|
||||
{
|
||||
"rev": 2640,
|
||||
"op": "sage-242-landed-note",
|
||||
"verb": "note",
|
||||
"args": {
|
||||
"id": 242,
|
||||
"text": "Landed 5fb736b92 on refactor, pushed (base 96bac588f). Blobs from 635c10924 on 3 paths; patch-id ceb741ee equals git diff 8fede5fc6..635c10924 on those paths; ls-tree modes equal; manifest 1fbfab6c verifies on the landed files. Megavolt M1/M2 killed on Sage's slot; Sage gate 15 packages 2826/0, 10 script suites 429/0. Duckworth approve 29889 rev 2639."
|
||||
},
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T04:13:30.071Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 242,
|
||||
"field": "note",
|
||||
"receipt": "ok sage-242-landed-note rev 2640 row 242 note"
|
||||
},
|
||||
"viewSha": "b0997e13a14bd17120aef27778c84deacdb08149688d0399e035fb8bc365bc37"
|
||||
},
|
||||
{
|
||||
"rev": 2641,
|
||||
"op": "sage-242-done",
|
||||
"verb": "move",
|
||||
"args": {
|
||||
"id": 242,
|
||||
"to": "done",
|
||||
"reason": null,
|
||||
"candidate": null,
|
||||
"evidence": null,
|
||||
"issue": null
|
||||
},
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T04:13:55.780Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 242,
|
||||
"from": "in-review",
|
||||
"to": "done",
|
||||
"round": 1,
|
||||
"issue": null,
|
||||
"candidate": null,
|
||||
"receipt": "ok sage-242-done rev 2641 row 242 in-review→done round 1"
|
||||
},
|
||||
"viewSha": "418b83170fd6c8ba7aa577b935f802b3a5dfead59be2260b11115206f324abbc"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -270,9 +270,42 @@ export async function startBroker({
|
||||
if (!(e instanceof BusError)) process.stderr.write(`bus: publish git-failed: ${JSON.stringify({ name: e?.name, code: e?.code })}\n`);
|
||||
Object.assign(entry, { state: 'refused', code: e instanceof BusError ? e.code : 'git-failed' });
|
||||
},
|
||||
);
|
||||
).then(() => {
|
||||
if (entry.held) settle(ticket, entry, entry.held);
|
||||
});
|
||||
return { state: 'running' };
|
||||
}
|
||||
// Row 242 (#1748): the git socket gave up on a publish (publish-timeout) whose push it never
|
||||
// collected. Its publishRefused waits for the push: until that ends the ticket stays open, so
|
||||
// checkPublish refuses another publish on the decision with publish-busy (row 237). A push that
|
||||
// reached the fork is then recorded, which spends the decision; one that didn't is filed with
|
||||
// the socket's code. Either way the decision can't push a second ref.
|
||||
function settle(ticket, entry, code) {
|
||||
remotes.delete(ticket);
|
||||
try {
|
||||
if (entry.state === 'pushed') {
|
||||
try {
|
||||
broker.recordPublish(ticket, { ref: entry.ref, commit: entry.commit, previous: entry.previous });
|
||||
process.stderr.write(`bus: publish ${ticket}: pushed after the git socket refused ${code}; recorded\n`);
|
||||
return;
|
||||
} catch (e) {
|
||||
if (!(e instanceof BusError)) throw e;
|
||||
code = e.code;
|
||||
}
|
||||
}
|
||||
broker.publishRefused(ticket, { code });
|
||||
} catch (e) {
|
||||
process.stderr.write(`bus: publish ${ticket}: not settled: ${JSON.stringify({ name: e?.name, code: e?.code })}\n`);
|
||||
}
|
||||
}
|
||||
function publishRefused(ticket, args) {
|
||||
const entry = remotes.get(ticket);
|
||||
if (!entry) return broker.publishRefused(ticket, args);
|
||||
if (!/^[a-z-]{1,64}$/.test(args?.code ?? '')) throw new BusError('invalid-request');
|
||||
if (entry.state === 'running') entry.held = args.code;
|
||||
else settle(ticket, entry, args.code);
|
||||
return args.code;
|
||||
}
|
||||
function publishResult(ticket) {
|
||||
const entry = remotes.get(ticket);
|
||||
if (!entry) throw new BusError('publish-unknown');
|
||||
@@ -309,7 +342,7 @@ export async function startBroker({
|
||||
publishPush,
|
||||
publishResult,
|
||||
recordPublish: (ticket, pushed) => broker.recordPublish(ticket, pushed),
|
||||
publishRefused: (ticket, args) => broker.publishRefused(ticket, args),
|
||||
publishRefused,
|
||||
landHeld: (record) => broker.landHeld(record),
|
||||
// Row 156 (#1651): the launcher's landing pass.
|
||||
landHolders: (business) => broker.landHolders(business),
|
||||
|
||||
@@ -306,6 +306,28 @@ test('a ticket outlives the run that asked, and recordLand checks the task again
|
||||
assert.throws(() => w.b.landRefused(ticket, { code: 'gate-failed' }), /land-unknown/);
|
||||
});
|
||||
|
||||
// Row 241 (#1747): one decision lands once. Two tickets on one decision can't both reach
|
||||
// recordLand through the git socket (land-busy, git-socket.mjs), so this pins the broker's own
|
||||
// guard: recordLand runs #landFacts again, and a decision spent by the first refuses the second.
|
||||
test('two tickets on one decision: the first recordLand lands, the second refuses decision-consumed and records nothing', (t) => {
|
||||
const w = setup(t);
|
||||
const { d } = w.ready();
|
||||
const first = w.check({ task_ref: TASK, decision: d.id }).ticket;
|
||||
const second = w.check({ task_ref: TASK, decision: d.id }).ticket;
|
||||
assert.notEqual(first, second);
|
||||
assert.deepEqual(w.b.recordLand(first, w.land), { task_ref: TASK, commit: B });
|
||||
assert.throws(() => w.b.recordLand(second, { ...w.land, commit: C, previous: B }), /decision-consumed/);
|
||||
assert.equal(w.merges().length, 1);
|
||||
assert.equal(w.merges()[0].body.commit, B);
|
||||
assert.equal(w.results().length, 1);
|
||||
assert.deepEqual(w.events('action.refused'), []);
|
||||
// The git socket then ends the second ticket with the broker's code, as it does any record refusal.
|
||||
assert.equal(w.b.landRefused(second, { code: 'decision-consumed', notify: true }), 'decision-consumed');
|
||||
assert.deepEqual(w.events('action.refused').map((e) => e.body), [{ code: 'decision-consumed' }]);
|
||||
assert.equal(w.merges().length, 1);
|
||||
assert.throws(() => w.b.recordLand(second, w.land), /land-unknown/);
|
||||
});
|
||||
|
||||
test('landRefused files the code on the task, tells the role only with notify, and ends the ticket', (t) => {
|
||||
const w = setup(t);
|
||||
const { d } = w.ready();
|
||||
|
||||
@@ -519,7 +519,8 @@ test('v3b prototype refusals, views and append-only mutations', (t) => {
|
||||
// Row 121 (#1615) adds 5 tables and 24 triggers for projects and threads; threads.test.mjs tests them.
|
||||
// Row 141 (#1635) adds instance_names and its 4 triggers; names.test.mjs tests them.
|
||||
// Row 124 adds usage_records and its three append-only triggers.
|
||||
assert.equal(count('table') - 1, 16);
|
||||
assert.equal(count('trigger'), 70);
|
||||
// Row 234 (#1740) adds hosts, host_grants and the events_grant_actor guard; grant-migration.test.mjs tests them.
|
||||
assert.equal(count('table') - 1, 18);
|
||||
assert.equal(count('trigger'), 71);
|
||||
assert.equal(count('view'), 5);
|
||||
});
|
||||
|
||||
@@ -774,6 +774,16 @@ Each live run requires Mos's go; agents build and test fixtures only, and
|
||||
so no PM turn may be in flight. Queue writes stay on the canonical
|
||||
scripts/mosaic.
|
||||
|
||||
On a host whose units are root's (row 232), `--units external` makes
|
||||
stack-release.sh read the system units and start or stop none. `activate`
|
||||
and `rollback --to` stage the copy and leave the `AWAITING-UNITS` marker;
|
||||
root installs and starts the units, then `confirm` gates the start and
|
||||
writes the pointer, `revert` undoes a staged run, and `gate --since` checks
|
||||
a copy with no open activation. stack-copy.sh adds `copy --bundle FILE SHA`
|
||||
for a host with no refactor checkout, `wrapper ID --create` for the first
|
||||
wrapper, and `check --units external`. Details are in
|
||||
[TOOLS.md, Root-owned units](../../docs/TOOLS.md#root-owned-units---units-external-row-232).
|
||||
|
||||
The human commands and `--tester` run only from the tree the bus runs from
|
||||
(row 130, #1625). The broker checks the path of the calling human-cli.mjs or
|
||||
tester-cli.mjs against its own. On a host with a pinned copy that tree is
|
||||
|
||||
@@ -38,7 +38,10 @@
|
||||
// one op at a time within 10 s. The poll ends at publishDeadlineMs
|
||||
// (publish-timeout), past git's own limit on each of the four git
|
||||
// runs. recordPublish files the push; any refusal after checkPublish
|
||||
// is filed once, by publishRefused.
|
||||
// is filed once, by publishRefused. After publish-timeout the bus
|
||||
// holds the decision until its push ends (a retry gets publish-busy),
|
||||
// then records a push that reached the fork or files the refusal
|
||||
// (row 242, #1748).
|
||||
//
|
||||
// A refusal is {"ok": false, "error": "<code>"}, and the connection ends.
|
||||
// The host checks, in order: identity (a role of this business), the request
|
||||
@@ -427,8 +430,9 @@ export function createGitSocket({ host, business: b, dataRoot, env = process.env
|
||||
await op({ op: "publishPush", ticket });
|
||||
const until = Date.now() + publishDeadlineMs;
|
||||
for (;;) {
|
||||
// The bus's git may still finish after this; the fork could then hold the commit with
|
||||
// no record, as when recordPublish refuses after a push.
|
||||
// The bus's git may still finish after this. The bus holds the decision until it does,
|
||||
// then records a push that reached the fork (row 242); the fork can still hold the
|
||||
// commit with no record if recordPublish refuses, as after an in-band push.
|
||||
if (Date.now() >= until) {
|
||||
log(`git socket: publish ${task_ref}: publish-timeout; the bus process's push may still reach the fork`);
|
||||
throw new Refusal("publish-timeout");
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
# Mosaic Case 1 host check at boot (row 240, #1746; r6 B2). Root installs it as
|
||||
# /etc/systemd/system/mosaic-hostcheck.service, root:root 0644, and enables it
|
||||
# (docs/guides/host-run-sheet.md). It runs the drift stop before the bus starts, which covers a kernel
|
||||
# or parser change that takes effect at boot. --enforce always exits 0; on drift it closes bwrap to
|
||||
# root, and every role session then refuses to start.
|
||||
[Unit]
|
||||
Description=Mosaic Case 1 host check: the bwrap profile, the statoverride and the sysctls
|
||||
After=apparmor.service
|
||||
Before=[email protected]
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/local/lib/mosaic-stack/hostcheck.sh --enforce
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -0,0 +1,335 @@
|
||||
// Row 240 (I4, #1746): scripts/case1-hostcheck.sh, the Case 1 host's drift stop for bwrap, and its
|
||||
// pins and boot unit. Brief: docs/plans/2026-10-20_installer-rows.md "I4 hostcheck"; design r6 B2
|
||||
// (agents/darkwing/work/queue-195/DESIGN.md at b7239bb66). Every case copies the script into a scratch
|
||||
// lib directory with a fixture case1-pins.sh beside it and runs it with --root on a scratch host tree.
|
||||
// Fake aa-status, dpkg, dpkg-statoverride, apparmor_parser, systemd-run, stat, chown, chmod, apt-get
|
||||
// and dpkg-deb on PATH record their argv. Nothing here reads or writes the real /etc, /proc or /usr.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { chmodSync, copyFileSync, existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { tmp } from "./helpers.mjs";
|
||||
|
||||
const REPO = fileURLToPath(new URL("../../../", import.meta.url));
|
||||
const SCRIPT = join(REPO, "scripts", "case1-hostcheck.sh");
|
||||
const PINS = join(REPO, "scripts", "case1-pins.sh");
|
||||
const UNIT = join(REPO, "packages", "cli", "systemd", "mosaic-hostcheck.service");
|
||||
const PROFILE = "/etc/apparmor.d/bwrap-userns-restrict";
|
||||
const LABEL = "bwrap//&unpriv_bwrap (enforce)";
|
||||
// A made-up profile; the fixture pins carry its sha256.
|
||||
const PROFILE_TEXT = "abi <abi/4.0>,\ninclude <tunables/global>\nprofile bwrap /usr/bin/bwrap flags=(attach_disconnected,mediate_deleted) {\n allow capability,\n change_profile /** -> &unpriv_bwrap,\n}\nprofile unpriv_bwrap flags=(attach_disconnected,mediate_deleted) {\n audit deny capability,\n}\n";
|
||||
const sha256 = (s) => createHash("sha256").update(s).digest("hex");
|
||||
const AA_OK = { version: "2", profiles: { "/usr/bin/man": "enforce", bwrap: "enforce", unpriv_bwrap: "enforce", "lsb_release": "enforce" }, processes: {} };
|
||||
const SYSTEMD_RUN = ["--wait", "--pipe", "--uid=mosaic", "--gid=mosaic", "-p", "NoNewPrivileges=yes", "-p", "ProtectSystem=strict",
|
||||
"-p", "ProtectHome=yes", "-p", "PrivateTmp=yes", "/usr/bin/bwrap", "--unshare-user", "--unshare-pid", "--ro-bind", "/", "/",
|
||||
"cat", "/proc/self/attr/current"];
|
||||
|
||||
function put(file, text, mode) {
|
||||
mkdirSync(dirname(file), { recursive: true });
|
||||
writeFileSync(file, text);
|
||||
if (mode) chmodSync(file, mode);
|
||||
}
|
||||
|
||||
// Each fake appends `name argv...` to calls.log and answers from a file in the state directory.
|
||||
const FAKES = {
|
||||
"aa-status": `[ -f "$HC/aa-rc" ] && exit "$(cat "$HC/aa-rc")"; cat "$HC/aa-status.json"`,
|
||||
dpkg: `case "$1" in
|
||||
-S) [ -f "$HC/owner" ] && cat "$HC/owner"; exit "$(cat "$HC/dpkg-S-rc")" ;;
|
||||
-s) [ "$2" = bubblewrap ] && printf 'Package: bubblewrap\\nStatus: install ok installed\\nVersion: %s\\n' "$(cat "$HC/bwrap-version")" ;;
|
||||
*) exit 2 ;;
|
||||
esac`,
|
||||
"dpkg-statoverride": `[ "$1" = --list ] && cat "$HC/statoverride"`,
|
||||
apparmor_parser: `exit 0`,
|
||||
"systemd-run": `cat "$HC/label"`,
|
||||
stat: `cat "$HC/stat"`,
|
||||
chown: `exit 0`,
|
||||
chmod: `exit 0`,
|
||||
"apt-get": `[ -f "$HC/apt-rc" ] && exit "$(cat "$HC/apt-rc")"; [ "$1 $2" = "download apparmor-profiles" ] && cp "$HC/deb" "./apparmor-profiles_9.9_all.deb"`,
|
||||
"dpkg-deb": `case "$1" in
|
||||
-f) [ "$3" = Version ] && echo 9.9-upstream ;;
|
||||
-x) mkdir -p "$3/usr/share/apparmor/extra-profiles"; [ ! -f "$HC/upstream-profile" ] || cp "$HC/upstream-profile" "$3/usr/share/apparmor/extra-profiles/bwrap-userns-restrict" ;;
|
||||
esac`,
|
||||
};
|
||||
|
||||
// A scratch host where all seven items hold.
|
||||
function host(t) {
|
||||
const dir = tmp(t, "case1-hostcheck-");
|
||||
const root = join(dir, "root"), hc = join(dir, "hc"), bin = join(dir, "bin"), lib = join(dir, "lib");
|
||||
put(join(root, PROFILE), PROFILE_TEXT);
|
||||
put(join(root, "etc", "apparmor.d", "abstractions", "base"), " /etc/ld.so.cache r,\n");
|
||||
put(join(root, "etc", "apparmor.d", "tunables", "global"), "@{HOME}=/home/*/\n");
|
||||
put(join(root, "proc", "sys", "kernel", "apparmor_restrict_unprivileged_userns"), "1\n");
|
||||
put(join(root, "proc", "sys", "fs", "protected_hardlinks"), "1\n");
|
||||
put(join(root, "proc", "sys", "net", "ipv4", "ip_unprivileged_port_start"), "1024\n");
|
||||
put(join(root, "etc", "systemd", "system", "mosaic-webui.service"),
|
||||
"[Service]\nExecStart=/opt/mosaic-stack/node/bin/node /srv/mosaic/copies/x/packages/webui/src/cli.mjs serve --port 732 --business mosaic-stack --tester-port 7333\n");
|
||||
put(join(root, "usr", "bin", "bwrap"), "", 0o755);
|
||||
put(join(hc, "aa-status.json"), JSON.stringify(AA_OK));
|
||||
put(join(hc, "dpkg-S-rc"), "1");
|
||||
put(join(hc, "bwrap-version"), "0.9.0-1ubuntu0.3");
|
||||
put(join(hc, "statoverride"), "root mosaic 0750 /usr/bin/bwrap\n");
|
||||
put(join(hc, "label"), `${LABEL}\n`);
|
||||
put(join(hc, "stat"), "root mosaic 750\n");
|
||||
put(join(hc, "deb"), "fake deb\n");
|
||||
put(join(hc, "upstream-profile"), PROFILE_TEXT);
|
||||
for (const [name, body] of Object.entries(FAKES)) {
|
||||
put(join(bin, name), `#!/usr/bin/env bash\nprintf '%s' ${JSON.stringify(name)} >> "$HC/calls.log"; printf ' %s' "$@" >> "$HC/calls.log"; echo >> "$HC/calls.log"\n${body}\n`, 0o755);
|
||||
}
|
||||
mkdirSync(lib);
|
||||
copyFileSync(SCRIPT, join(lib, "hostcheck.sh"));
|
||||
chmodSync(join(lib, "hostcheck.sh"), 0o755);
|
||||
put(join(lib, "case1-pins.sh"), `PIN_PROFILES_VERSION=4.0.1-fixture\nPIN_PROFILE_SHA256=${sha256(PROFILE_TEXT)}\nPIN_BUBBLEWRAP_VERSION=0.9.0-1ubuntu0.3\n`);
|
||||
return { dir, root, hc, bin, lib, script: join(lib, "hostcheck.sh") };
|
||||
}
|
||||
|
||||
function run(h, ...args) {
|
||||
const r = spawnSync(h.script, [...args, "--root", h.root], {
|
||||
encoding: "utf8",
|
||||
env: { ...process.env, PATH: `${h.bin}:${process.env.PATH}`, HC: h.hc, TMPDIR: h.dir },
|
||||
});
|
||||
return { status: r.status, out: r.stdout, err: r.stderr, lines: r.stdout.split("\n").filter(Boolean) };
|
||||
}
|
||||
const calls = (h, name) => (existsSync(join(h.hc, "calls.log")) ? readFileSync(join(h.hc, "calls.log"), "utf8") : "")
|
||||
.split("\n").filter((l) => l === name || l.startsWith(`${name} `)).map((l) => l.slice(name.length + 1));
|
||||
const fails = (r) => r.lines.filter((l) => l.startsWith("FAIL "));
|
||||
const drift = (h) => join(h.root, "etc", "mosaic-stack", "DRIFT");
|
||||
const hlog = (h) => join(h.root, "var", "log", "mosaic-stack", "hostcheck.log");
|
||||
const aa = (h, profiles) => put(join(h.hc, "aa-status.json"), JSON.stringify({ version: "2", profiles, processes: {} }));
|
||||
|
||||
// One fault per item; each makes that item, and only that one, fail.
|
||||
const FAULTS = {
|
||||
1: (h) => put(join(h.root, PROFILE), `${PROFILE_TEXT}# drift\n`),
|
||||
2: (h) => aa(h, { ...AA_OK.profiles, bwrap: "complain" }),
|
||||
3: (h) => put(join(h.root, "etc", "apparmor.d", "usr.bin.other"), "profile other /usr/bin/other {\n /usr/bin/bwrap ix,\n}\n"),
|
||||
4: (h) => put(join(h.hc, "label"), "unconfined\n"),
|
||||
5: (h) => put(join(h.hc, "statoverride"), "root root 0755 /usr/bin/bwrap\n"),
|
||||
6: (h) => put(join(h.root, "proc", "sys", "kernel", "apparmor_restrict_unprivileged_userns"), "0\n"),
|
||||
7: (h) => put(join(h.root, "proc", "sys", "net", "ipv4", "ip_unprivileged_port_start"), "700\n"),
|
||||
};
|
||||
|
||||
test("row 240 check: exits 0 with all seven items and a note that bubblewrap is the install pin", (t) => {
|
||||
const h = host(t);
|
||||
const r = run(h, "--check");
|
||||
assert.equal(r.status, 0, r.out + r.err);
|
||||
assert.deepEqual(r.lines.map((l) => l.split(" ").slice(0, 2).join(" ")), ["ok 1", "ok 2", "ok 3", "ok 4", "ok 5", "ok 6", "ok 7", "note bubblewrap"]);
|
||||
assert.match(r.lines[6], /is 1024, above the Console's --port 732$/);
|
||||
assert.equal(r.lines[7], "note bubblewrap 0.9.0-1ubuntu0.3 is the install pin");
|
||||
});
|
||||
|
||||
test("row 240 check: exits 1 with each item failing alone, and its line names it", (t) => {
|
||||
for (const [n, fault] of Object.entries(FAULTS)) {
|
||||
const h = host(t);
|
||||
fault(h);
|
||||
const r = run(h, "--check");
|
||||
assert.equal(r.status, 1, `item ${n}: ${r.out}`);
|
||||
assert.deepEqual(fails(r).map((l) => l.split(" ")[1]), [n], `item ${n}: ${r.out}`);
|
||||
}
|
||||
});
|
||||
|
||||
test("row 240 check: item 1 fails when a package owns the profile, when dpkg -S errs, or with no pin", (t) => {
|
||||
let h = host(t);
|
||||
put(join(h.hc, "dpkg-S-rc"), "0");
|
||||
put(join(h.hc, "owner"), `apparmor-profiles: ${PROFILE}\n`);
|
||||
let r = run(h, "--check");
|
||||
assert.equal(r.status, 1);
|
||||
assert.deepEqual(fails(r), [`FAIL 1 ${PROFILE} is owned by a package: apparmor-profiles: ${PROFILE}`]);
|
||||
assert.deepEqual(calls(h, "dpkg").filter((a) => a.startsWith("-S")), [`-S ${PROFILE}`]);
|
||||
h = host(t);
|
||||
put(join(h.hc, "dpkg-S-rc"), "2");
|
||||
r = run(h, "--check");
|
||||
assert.deepEqual(fails(r), [`FAIL 1 dpkg -S ${PROFILE} exited 2`]);
|
||||
h = host(t);
|
||||
rmSync(join(h.lib, "case1-pins.sh"));
|
||||
r = run(h, "--check");
|
||||
assert.equal(r.status, 1);
|
||||
assert.deepEqual(fails(r), [`FAIL 1 no profile pin in ${h.lib}/case1-pins.sh`]);
|
||||
});
|
||||
|
||||
test("row 240 check: item 2 fails on a loaded mosaic-bwrap, a missing unpriv_bwrap or a failed aa-status", (t) => {
|
||||
for (const [fault, text] of [
|
||||
[(h) => aa(h, { ...AA_OK.profiles, "mosaic-bwrap": "enforce" }), "a mosaic-bwrap profile"],
|
||||
[(h) => aa(h, { bwrap: "enforce" }), "no unpriv_bwrap profile in enforce mode"],
|
||||
[(h) => put(join(h.hc, "aa-rc"), "1"), "aa-status --json failed"],
|
||||
]) {
|
||||
const h = host(t);
|
||||
fault(h);
|
||||
const r = run(h, "--check");
|
||||
assert.equal(r.status, 1);
|
||||
assert.equal(fails(r).length, 1, r.out);
|
||||
assert.ok(fails(r)[0].startsWith("FAIL 2 ") && fails(r)[0].includes(text), r.out);
|
||||
}
|
||||
});
|
||||
|
||||
test("row 240 check: item 3 fails on a second file naming bwrap as a word, /{usr/,}bin/bwrap included", (t) => {
|
||||
for (const text of ["profile x /{usr/,}bin/bwrap {\n}\n", "# runs bwrap\n", "/usr/bin/bwrap ix,\n"]) {
|
||||
const h = host(t);
|
||||
put(join(h.root, "etc", "apparmor.d", "local", "extra"), text);
|
||||
const r = run(h, "--check");
|
||||
assert.equal(r.status, 1, text);
|
||||
assert.deepEqual(fails(r).map((l) => l.split(" ")[1]), ["3"], text);
|
||||
assert.ok(fails(r)[0].includes(join(h.root, "etc", "apparmor.d", "local", "extra")), r.out);
|
||||
}
|
||||
const h = host(t);
|
||||
put(join(h.root, "etc", "apparmor.d", "local", "extra"), "# bwrapper and unpriv-bwrapx are other words\n");
|
||||
assert.equal(run(h, "--check").status, 0);
|
||||
});
|
||||
|
||||
test("row 240 check: item 4 reads the label under the units' no-new-privs and matches it exactly", (t) => {
|
||||
let h = host(t);
|
||||
assert.equal(run(h, "--check").status, 0);
|
||||
assert.deepEqual(calls(h, "systemd-run"), [SYSTEMD_RUN.join(" ")]);
|
||||
for (const label of ["unpriv_bwrap (enforce)", `${LABEL} `, `x${LABEL}`]) {
|
||||
h = host(t);
|
||||
put(join(h.hc, "label"), `${label}\n`);
|
||||
const r = run(h, "--check");
|
||||
assert.equal(r.status, 1, JSON.stringify(label));
|
||||
assert.deepEqual(fails(r), [`FAIL 4 a bwrap child under no-new-privs has the label '${label}', not '${LABEL}'`]);
|
||||
}
|
||||
});
|
||||
|
||||
test("row 240 check: item 5 needs the statoverride and stat to agree", (t) => {
|
||||
const h = host(t);
|
||||
put(join(h.hc, "stat"), "root root 700\n");
|
||||
const r = run(h, "--check");
|
||||
assert.deepEqual(fails(r), ["FAIL 5 /usr/bin/bwrap is 'root root 700', not 'root mosaic 750'"]);
|
||||
assert.deepEqual(calls(h, "dpkg-statoverride"), ["--list /usr/bin/bwrap"]);
|
||||
assert.deepEqual(calls(h, "stat"), [`-c %U %G %a ${h.root}/usr/bin/bwrap`]);
|
||||
});
|
||||
|
||||
test("row 240 check: item 6 fails on protected_hardlinks 0; item 7 fails with the floor at 732 and reads 1023 with no unit", (t) => {
|
||||
let h = host(t);
|
||||
put(join(h.root, "proc", "sys", "fs", "protected_hardlinks"), "0\n");
|
||||
assert.deepEqual(fails(run(h, "--check")).map((l) => l.split(" ")[1]), ["6"]);
|
||||
h = host(t);
|
||||
put(join(h.root, "proc", "sys", "net", "ipv4", "ip_unprivileged_port_start"), "732\n");
|
||||
assert.deepEqual(fails(run(h, "--check")), ["FAIL 7 net.ipv4.ip_unprivileged_port_start is 732, not above the Console's --port 732"]);
|
||||
put(join(h.root, "proc", "sys", "net", "ipv4", "ip_unprivileged_port_start"), "733\n");
|
||||
assert.equal(run(h, "--check").status, 0);
|
||||
rmSync(join(h.root, "etc", "systemd", "system", "mosaic-webui.service"));
|
||||
assert.deepEqual(fails(run(h, "--check")), ["FAIL 7 net.ipv4.ip_unprivileged_port_start is 733, not above 1023, because no Console unit is installed"]);
|
||||
put(join(h.root, "proc", "sys", "net", "ipv4", "ip_unprivileged_port_start"), "1024\n");
|
||||
assert.equal(run(h, "--check").status, 0);
|
||||
put(join(h.root, "etc", "systemd", "system", "mosaic-webui.service"), "[Service]\nExecStart=/x serve --business mosaic-stack --tester-port 7333\n");
|
||||
assert.deepEqual(fails(run(h, "--check")), ["FAIL 7 /etc/systemd/system/mosaic-webui.service has no --port"]);
|
||||
});
|
||||
|
||||
test("row 240 check: a bubblewrap version other than the install pin is a note, not a failure", (t) => {
|
||||
const h = host(t);
|
||||
put(join(h.hc, "bwrap-version"), "0.9.0-1ubuntu0.4");
|
||||
const r = run(h, "--check");
|
||||
assert.equal(r.status, 0);
|
||||
assert.equal(r.lines.at(-1), "note bubblewrap 0.9.0-1ubuntu0.4 is not the install pin 0.9.0-1ubuntu0.3, which is not a drift item");
|
||||
});
|
||||
|
||||
test("row 240 enforce: a pass with no DRIFT changes nothing and exits 0", (t) => {
|
||||
const h = host(t);
|
||||
const r = run(h, "--enforce");
|
||||
assert.equal(r.status, 0);
|
||||
assert.equal(r.lines.at(-1), "hostcheck: all seven hold");
|
||||
assert.deepEqual([calls(h, "apparmor_parser"), calls(h, "chown"), calls(h, "chmod")], [[], [], []]);
|
||||
assert.equal(existsSync(drift(h)) || existsSync(hlog(h)), false);
|
||||
});
|
||||
|
||||
test("row 240 enforce: each failure exits 0, unloads the profile, closes bwrap, writes DRIFT and one log line", (t) => {
|
||||
for (const [n, fault] of Object.entries(FAULTS)) {
|
||||
const h = host(t);
|
||||
fault(h);
|
||||
const r = run(h, "--enforce");
|
||||
assert.equal(r.status, 0, `item ${n}: ${r.out}${r.err}`);
|
||||
assert.deepEqual(calls(h, "apparmor_parser"), [`-R ${h.root}${PROFILE}`], `item ${n}`);
|
||||
assert.deepEqual(calls(h, "chown"), [`root:root ${h.root}/usr/bin/bwrap`], `item ${n}`);
|
||||
assert.deepEqual(calls(h, "chmod"), [`0700 ${h.root}/usr/bin/bwrap`], `item ${n}`);
|
||||
const d = readFileSync(drift(h), "utf8").split("\n").filter(Boolean);
|
||||
assert.equal(d.length, 1, `item ${n}`);
|
||||
assert.match(d[0], new RegExp(`^\\d{4}-\\d\\d-\\d\\dT\\d\\d:\\d\\d:\\d\\dZ ${n} `), `item ${n}`);
|
||||
const l = readFileSync(hlog(h), "utf8").split("\n").filter(Boolean);
|
||||
assert.equal(l.length, 1, `item ${n}`);
|
||||
assert.match(l[0], new RegExp(`^\\d{4}-\\d\\d-\\d\\dT\\d\\d:\\d\\d:\\d\\dZ enforce stop items=${n} drift-before=0 unload=rc=0 chown=0 chmod=0$`), `item ${n}`);
|
||||
}
|
||||
});
|
||||
|
||||
test("row 240 enforce: with DRIFT present and every item passing, it still closes bwrap", (t) => {
|
||||
const h = host(t);
|
||||
put(drift(h), "2026-10-20T00:00:00Z 1 an earlier drift\n");
|
||||
const r = run(h, "--enforce");
|
||||
assert.equal(r.status, 0);
|
||||
assert.equal(fails(r).length, 0);
|
||||
assert.deepEqual(calls(h, "chown"), [`root:root ${h.root}/usr/bin/bwrap`]);
|
||||
assert.deepEqual(calls(h, "chmod"), [`0700 ${h.root}/usr/bin/bwrap`]);
|
||||
assert.equal(readFileSync(drift(h), "utf8"), "2026-10-20T00:00:00Z 1 an earlier drift\n");
|
||||
assert.match(readFileSync(hlog(h), "utf8"), /Z enforce stop items=none drift-before=1 unload=rc=0 chown=0 chmod=0\n$/);
|
||||
});
|
||||
|
||||
test("row 240 enforce: no unload when aa-status lists neither bwrap profile; never writes the statoverride", (t) => {
|
||||
const h = host(t);
|
||||
aa(h, { "/usr/bin/man": "enforce" });
|
||||
const r = run(h, "--enforce");
|
||||
assert.equal(r.status, 0);
|
||||
assert.deepEqual(calls(h, "apparmor_parser"), []);
|
||||
assert.deepEqual(calls(h, "chmod"), [`0700 ${h.root}/usr/bin/bwrap`]);
|
||||
assert.match(readFileSync(hlog(h), "utf8"), /Z enforce stop items=2 drift-before=0 unload=skipped chown=0 chmod=0\n$/);
|
||||
for (const [n, fault] of Object.entries(FAULTS)) {
|
||||
const g = host(t);
|
||||
fault(g);
|
||||
run(g, "--enforce");
|
||||
for (const a of calls(g, "dpkg-statoverride")) assert.doesNotMatch(a, /--(update|add|remove)/, `item ${n}`);
|
||||
assert.deepEqual(calls(g, "dpkg-statoverride"), ["--list /usr/bin/bwrap"], `item ${n}`);
|
||||
}
|
||||
});
|
||||
|
||||
test("row 240 upstream: a candidate whose profile differs prints the difference and exits 0", (t) => {
|
||||
let h = host(t);
|
||||
put(join(h.hc, "upstream-profile"), `${PROFILE_TEXT} # upstream change\n`);
|
||||
let r = run(h, "--upstream");
|
||||
assert.equal(r.status, 0, r.err);
|
||||
assert.match(r.out, new RegExp(`^upstream: DIFFERS: apparmor-profiles 9.9-upstream's bwrap-userns-restrict has sha256 ${sha256(`${PROFILE_TEXT} # upstream change\n`)}; the pin is ${sha256(PROFILE_TEXT)} from 4.0.1-fixture\\.`));
|
||||
assert.match(r.out, /^\+ {2}# upstream change$/m);
|
||||
assert.deepEqual(calls(h, "apt-get"), ["download apparmor-profiles"]);
|
||||
assert.deepEqual([calls(h, "chown"), calls(h, "chmod"), existsSync(drift(h))], [[], [], false]);
|
||||
h = host(t);
|
||||
r = run(h, "--upstream");
|
||||
assert.equal(r.status, 0);
|
||||
assert.equal(r.out, "upstream: apparmor-profiles 9.9-upstream's bwrap-userns-restrict matches the pin from 4.0.1-fixture\n");
|
||||
h = host(t);
|
||||
rmSync(join(h.hc, "upstream-profile"));
|
||||
r = run(h, "--upstream");
|
||||
assert.equal(r.status, 0);
|
||||
assert.match(r.out, /^upstream: DIFFERS: apparmor-profiles 9.9-upstream has no bwrap-userns-restrict;/);
|
||||
h = host(t);
|
||||
put(join(h.hc, "apt-rc"), "100");
|
||||
r = run(h, "--upstream");
|
||||
assert.equal(r.status, 1);
|
||||
assert.equal(r.err, "hostcheck: upstream: apt-get download apparmor-profiles failed\n");
|
||||
});
|
||||
|
||||
test("row 240 boot unit: a oneshot after apparmor and before the bus that runs --enforce", () => {
|
||||
const text = readFileSync(UNIT, "utf8");
|
||||
const lines = text.split("\n").filter((l) => l && !l.startsWith("#"));
|
||||
assert.deepEqual(lines.filter((l) => /^(Type|After|Before|ExecStart|WantedBy)=/.test(l)), [
|
||||
"After=apparmor.service",
|
||||
"[email protected]",
|
||||
"Type=oneshot",
|
||||
"ExecStart=/usr/local/lib/mosaic-stack/hostcheck.sh --enforce",
|
||||
"WantedBy=multi-user.target",
|
||||
]);
|
||||
});
|
||||
|
||||
test("row 240 pins and usage: the shipped pins are r6's, the script reads the pins beside it, and bad usage exits 2", (t) => {
|
||||
assert.equal(readFileSync(PINS, "utf8").split("\n").filter((l) => /^PIN_/.test(l)).join("\n"),
|
||||
"PIN_PROFILES_VERSION=4.0.1really4.0.1-0ubuntu0.24.04.9\nPIN_PROFILE_SHA256=11d39094f044f0cda0febb3ad517b830301da6b2ce929664af09ee9e4dd264f9\nPIN_BUBBLEWRAP_VERSION=0.9.0-1ubuntu0.3");
|
||||
const h = host(t);
|
||||
const r = run({ ...h, script: SCRIPT }, "--check");
|
||||
assert.equal(r.status, 1);
|
||||
assert.deepEqual(fails(r), [`FAIL 1 ${PROFILE} sha256 is ${sha256(PROFILE_TEXT)}, not the pin 11d39094f044f0cda0febb3ad517b830301da6b2ce929664af09ee9e4dd264f9`]);
|
||||
for (const args of [[], ["--check", "--enforce"], ["--check", "--root", "relative"], ["--status"]]) {
|
||||
const u = spawnSync(h.script, args, { encoding: "utf8", env: { ...process.env, PATH: `${h.bin}:${process.env.PATH}`, HC: h.hc } });
|
||||
assert.equal(u.status, 2, args.join(" "));
|
||||
}
|
||||
if (process.getuid() !== 0) {
|
||||
const u = spawnSync(h.script, ["--check"], { encoding: "utf8", env: { ...process.env, PATH: `${h.bin}:${process.env.PATH}`, HC: h.hc } });
|
||||
assert.deepEqual([u.status, u.stderr], [2, "hostcheck: refusing: run it as root\n"]);
|
||||
}
|
||||
});
|
||||
@@ -5,7 +5,7 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { chmodSync, lstatSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { chmodSync, existsSync, lstatSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { connect } from "node:net";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
@@ -298,3 +298,41 @@ test("sage-1440: the publishResult poll is bounded: publish-timeout, filed once,
|
||||
assert.equal(w.events("action.allowed").filter((e) => e.body.action === "git.push.remote" && e.body.operation !== "decision.raise").length, 0);
|
||||
assert.ok(w.logs.some((l) => l.includes("publish-timeout")));
|
||||
});
|
||||
|
||||
test("row 242: a retry after publish-timeout refuses publish-busy while the bus process's push runs; the late push is recorded and the decision pushes once", async (t) => {
|
||||
const w = await setup(t, { publishDeadlineMs: 150 });
|
||||
w.bus.credentials.issue("demo/coder", "gitea", TOKEN, "2099-01-01");
|
||||
// The fork holds the push in pre-receive until the test writes `release`; `held` says it's there.
|
||||
const held = join(w.root, "held");
|
||||
const release = join(w.root, "release");
|
||||
writeFileSync(join(w.forkRepo, "hooks", "pre-receive"), `#!/bin/sh\n: > '${held}'\nwhile [ ! -e '${release}' ]; do sleep 0.05; done\ncat > /dev/null\n`, { mode: 0o755 });
|
||||
const until = async (what, ok) => {
|
||||
for (const end = Date.now() + 10_000; !ok(); ) {
|
||||
if (Date.now() > end) assert.fail(`timed out waiting for ${what}`);
|
||||
await new Promise((r) => setTimeout(r, 20));
|
||||
}
|
||||
};
|
||||
const allowed = () => w.events("action.allowed").filter((e) => e.body.action === "git.push.remote" && e.body.operation !== "decision.raise");
|
||||
// A failed assert must still let the hook end: teardown waits for the fake Gitea, which waits for the push.
|
||||
try {
|
||||
const d = w.approve();
|
||||
assert.deepEqual(await w.publish(d), { ok: false, error: "publish-timeout" });
|
||||
await until("the push to reach pre-receive", () => existsSync(held));
|
||||
// The socket gave up, but the push still runs: the decision is held, and nothing is filed yet.
|
||||
assert.deepEqual(await w.publish(d, { branch: "other" }), { ok: false, error: "publish-busy" });
|
||||
assert.deepEqual(w.events("action.refused").map((e) => e.body.code), ["publish-busy"]);
|
||||
assert.equal(allowed().length, 0);
|
||||
writeFileSync(release, "");
|
||||
// The push reached the fork, so it's recorded, which spends the decision.
|
||||
await until("the late push to be recorded", () => allowed().length === 1);
|
||||
assert.equal(allowed()[0].body.ref, "refs/heads/coder/feat");
|
||||
assert.equal(allowed()[0].body.commit, w.work);
|
||||
assert.deepEqual(await w.publish(d, { branch: "other" }), { ok: false, error: "decision-consumed" });
|
||||
assert.equal(w.gitea.requests.filter((r) => r.path.endsWith("/git-receive-pack")).length, 1);
|
||||
assert.equal(git(w.root, `--git-dir=${w.forkRepo}`, "for-each-ref", "--format=%(refname)", "refs/heads/coder/"), "refs/heads/coder/feat");
|
||||
assert.deepEqual(w.events("action.refused").map((e) => e.body.code), ["publish-busy", "decision-consumed"]);
|
||||
} finally {
|
||||
writeFileSync(release, "");
|
||||
}
|
||||
assert.equal(JSON.stringify([w.events("action.allowed"), w.events("action.refused"), w.logs]).includes(TOKEN), false);
|
||||
});
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {execFileSync,spawnSync,spawn} from 'node:child_process';
|
||||
import {symlinkSync,unlinkSync,mkdtempSync,mkdirSync,writeFileSync,readFileSync,cpSync,rmSync,chmodSync,existsSync,statSync} from 'node:fs';
|
||||
import {symlinkSync,unlinkSync,mkdtempSync,mkdirSync,writeFileSync,readFileSync,cpSync,rmSync,chmodSync,existsSync,statSync,readdirSync,lstatSync,readlinkSync} from 'node:fs';
|
||||
import {createHash} from 'node:crypto';
|
||||
import {tmpdir} from 'node:os';
|
||||
import {join,dirname} from 'node:path';
|
||||
import {fileURLToPath} from 'node:url';
|
||||
@@ -227,3 +228,108 @@ test('inherited drop-in unreadable directory or file refuses before writing',t=>
|
||||
unlinkSync(dropdir);mkdirSync(dropdir);const file=join(dropdir,'broken.conf');symlinkSync(join(s.root,'absent-file'),file);
|
||||
const broken=s.run('bus-service.sh',['install','--dir',units,'--no-reload']);assert.equal(broken.status,1);assert.ok(broken.stderr.includes(file));assert.equal(readFileSync(target,'utf8'),'previous unit bytes');
|
||||
});
|
||||
// Row 232: every file of a copy with its mode and bytes, for the byte-identical comparison.
|
||||
function tree(dir){
|
||||
const out=[];
|
||||
const walk=rel=>{for(const e of readdirSync(join(dir,rel),{withFileTypes:true}).sort((a,b)=>a.name<b.name?-1:1)){
|
||||
const r=rel?`${rel}/${e.name}`:e.name,st=lstatSync(join(dir,r));
|
||||
if(e.isDirectory()){out.push(`d ${(st.mode&0o7777).toString(8)} ${r}`);walk(r);}
|
||||
else if(e.isSymbolicLink())out.push(`l ${r} ${readlinkSync(join(dir,r))}`);
|
||||
else out.push(`f ${(st.mode&0o7777).toString(8)} ${r} ${createHash('sha256').update(readFileSync(join(dir,r))).digest('hex')}`);
|
||||
}};
|
||||
walk('');return out;
|
||||
}
|
||||
test('row 232: copy --bundle copies a SHA the bundle\'s refactor reaches, byte-identical to copy SHA, and refuses the rest',t=>{
|
||||
const s=setup(t),tmp=join(s.root,'t');mkdirSync(tmp);s.env.TMPDIR=tmp;
|
||||
const good=join(s.root,'good.bundle');git(s.repo,'bundle','create','-q',good,'refactor');
|
||||
const a=join(s.root,'copies-a'),b=join(s.root,'copies-b');
|
||||
let r=s.run('stack-copy.sh',['copy',s.id,'--copies',a]);assert.equal(r.status,0,r.stderr);
|
||||
r=s.run('stack-copy.sh',['copy','--bundle',good,s.id,'--copies',b]);assert.equal(r.status,0,r.stderr);
|
||||
const sum=createHash('sha256').update(readFileSync(good)).digest('hex'),release=git(s.repo,'show',`${s.id}:RELEASE`);
|
||||
assert.equal(r.stdout,`copied: ${join(b,s.id)}\nrelease: ${release}\nbundle sha256: ${sum}\n`);
|
||||
assert.deepEqual(tree(join(b,s.id)),tree(join(a,s.id)));
|
||||
assert.equal(statSync(join(b,s.id)).mode&0o777,0o555);
|
||||
const left=()=>readdirSync(tmp).filter(f=>f.startsWith('tmp.'));
|
||||
assert.deepEqual(left(),[],'the bundle\'s temporary repository is removed');
|
||||
// A second run finds the copy and still names the release.
|
||||
r=s.run('stack-copy.sh',['copy','--bundle',good,s.id,'--copies',b]);assert.equal(r.status,0,r.stderr);
|
||||
assert.equal(r.stdout,`exists: ${join(b,s.id)}\nrelease: ${release}\nbundle sha256: ${sum}\n`);
|
||||
// A relative bundle path works from the caller's directory.
|
||||
const c=join(s.root,'copies-c');
|
||||
r=spawnSync(join(s.repo,'scripts','stack-copy.sh'),['copy','--bundle','good.bundle',s.id,'--copies',c],{cwd:s.root,env:s.env,encoding:'utf8'});
|
||||
assert.equal(r.status,0,r.stderr);assert.deepEqual(tree(join(c,s.id)),tree(join(a,s.id)));
|
||||
// A commit on another branch of the bundle: the bundle carries it, refactor does not reach it.
|
||||
const side=git(s.repo,'commit-tree','-p',s.id,'-m','side',`${s.id}^{tree}`);git(s.repo,'update-ref','refs/heads/side',side);
|
||||
const both=join(s.root,'both.bundle');git(s.repo,'bundle','create','-q',both,'refactor','side');
|
||||
const only=join(s.root,'side.bundle');git(s.repo,'bundle','create','-q',only,'side');
|
||||
const corrupt=join(s.root,'corrupt.bundle'),bytes=readFileSync(good);bytes[bytes.length-30]^=0xff;writeFileSync(corrupt,bytes);
|
||||
const cut=join(s.root,'cut.bundle');writeFileSync(cut,readFileSync(good).subarray(0,200));
|
||||
const junk=join(s.root,'junk.bundle');writeFileSync(junk,'# v2 git bundle\nnot a ref line\n\n');
|
||||
const d=join(s.root,'copies-d');
|
||||
for(const [label,bundle,id,re] of [
|
||||
['not reachable',both,side,/not reachable from the bundle's refs\/heads\/refactor/],
|
||||
['no refactor',only,side,/no refs\/heads\/refactor/],
|
||||
// verify reads the header and prerequisites; a damaged pack fails the fetch's own check.
|
||||
['not a bundle',junk,s.id,/git bundle verify failed/],
|
||||
['corrupt pack',corrupt,s.id,/cannot fetch/],
|
||||
['truncated pack',cut,s.id,/cannot fetch/],
|
||||
['absent',join(s.root,'none.bundle'),s.id,/bundle is absent/],
|
||||
['short id',good,s.id.slice(0,12),/full 40-character commit id/],
|
||||
]){
|
||||
r=s.run('stack-copy.sh',['copy','--bundle',bundle,id,'--copies',d]);
|
||||
assert.equal(r.status,1,label);assert.match(r.stderr,re,label);assert.equal(r.stdout,'',label);
|
||||
assert.ok(!existsSync(join(d,id)),label);assert.deepEqual(left(),[],label);
|
||||
if(existsSync(d))assert.deepEqual(readdirSync(d),[],label);
|
||||
}
|
||||
const link=join(s.root,'link.bundle');symlinkSync(good,link);
|
||||
r=s.run('stack-copy.sh',['copy','--bundle',link,s.id,'--copies',d]);assert.equal(r.status,1);assert.match(r.stderr,/symlink/);
|
||||
for(const args of [['copy','--bundle',good,s.id,'--allow-worktree'],['copy','--bundle',good],['copy','--bundle'],['wrapper','--bundle',good,s.id],['check','--units','system'],['copy',s.id,'--units','external'],['copy',s.id,'--create']])
|
||||
assert.equal(s.run('stack-copy.sh',args).status,4,args.join(' '));
|
||||
});
|
||||
test('row 232: wrapper --create writes the first wrapper only, the same bytes wrapper ID writes, and refuses over any file',t=>{
|
||||
const s=setup(t);assert.equal(s.run('stack-copy.sh',['copy',s.id,'--copies',s.copies]).status,0);
|
||||
const wrapper=join(s.bin,'mosaic'),root=join(s.copies,s.id);
|
||||
let r=s.run('stack-copy.sh',['wrapper',s.id,'--copies',s.copies,'--bin',s.bin]);
|
||||
assert.equal(r.status,1);assert.match(r.stderr,/wrapper is absent/,'no --create keeps the missing-wrapper refusal');assert.ok(!existsSync(wrapper));
|
||||
r=s.run('stack-copy.sh',['wrapper',s.id,'--copies',s.copies,'--bin',s.bin,'--create']);assert.equal(r.status,0,r.stderr);
|
||||
assert.equal(r.stdout,`wrapper: ${root}\n`);
|
||||
const created=readFileSync(wrapper,'utf8');assert.equal(statSync(wrapper).mode&0o777,0o755);
|
||||
r=s.run('stack-copy.sh',['wrapper',s.id,'--copies',s.copies,'--bin',s.bin]);assert.equal(r.status,0,r.stderr);
|
||||
assert.equal(r.stdout,`wrapper: ${root}\n`);assert.equal(readFileSync(wrapper,'utf8'),created,'wrapper ID writes the same bytes');
|
||||
const foreign='#!/bin/sh\n# not ours\n';
|
||||
for(const [label,make] of [['own wrapper',()=>{}],['foreign file',()=>writeFileSync(wrapper,foreign)],['symlink',()=>{rmSync(wrapper);symlinkSync(join(s.root,'elsewhere'),wrapper);}]]){
|
||||
make();const before=existsSync(wrapper)||label==='symlink'?(label==='symlink'?readlinkSync(wrapper):readFileSync(wrapper,'utf8')):null;
|
||||
r=s.run('stack-copy.sh',['wrapper',s.id,'--copies',s.copies,'--bin',s.bin,'--create']);
|
||||
assert.equal(r.status,1,label);assert.match(r.stderr,/--create and a wrapper exists/,label);
|
||||
assert.equal(label==='symlink'?readlinkSync(wrapper):readFileSync(wrapper,'utf8'),before,label);
|
||||
assert.deepEqual(readdirSync(s.bin).filter(f=>f.startsWith('.')),[],`${label}: no temporary file left`);
|
||||
}
|
||||
// A named instance: --create writes mosaic-NAME as wrapper --instance does, once.
|
||||
assert.equal(s.run('bootstrap.sh',['--instance','alpha']).status,0);
|
||||
const named=join(s.bin,'mosaic-alpha');
|
||||
r=s.run('stack-copy.sh',['wrapper',s.id,'--instance','alpha','--copies',s.copies,'--bin',s.bin,'--create']);assert.equal(r.status,0,r.stderr);
|
||||
const first=readFileSync(named,'utf8');
|
||||
r=s.run('stack-copy.sh',['wrapper',s.id,'--instance','alpha','--copies',s.copies,'--bin',s.bin]);assert.equal(r.status,0,r.stderr);
|
||||
assert.equal(readFileSync(named,'utf8'),first);
|
||||
r=s.run('stack-copy.sh',['wrapper',s.id,'--instance','alpha','--copies',s.copies,'--bin',s.bin,'--create']);
|
||||
assert.equal(r.status,1);assert.match(r.stderr,/--create and a wrapper exists/);assert.equal(readFileSync(named,'utf8'),first);
|
||||
});
|
||||
test('row 232: check --units external reads the system units, the default reads the user units',t=>{
|
||||
const s=setup(t);assert.equal(s.run('stack-copy.sh',['copy',s.id,'--copies',s.copies]).status,0);
|
||||
assert.equal(s.run('stack-copy.sh',['wrapper',s.id,'--copies',s.copies,'--bin',s.bin,'--create']).status,0);
|
||||
const copy=join(s.copies,s.id),argv=join(s.root,'argv.log');
|
||||
s.fake(`echo "$*" >> '${argv}'; echo 'LoadState=loaded'; echo 'WorkingDirectory=${copy}'; case "$*" in *mosaic-bus*) echo 'ExecStart={ path=${copy}/scripts/mosaic ; argv[]=${copy}/scripts/mosaic bus start mosaic-stack --pm ; }';; *) echo 'ExecStart={ path=/usr/bin/node ; argv[]=/usr/bin/node ${copy}/packages/webui/src/cli.mjs serve ; }';; esac\n`);
|
||||
const want=`bus: ${copy}\nConsole: ${copy}\nCLI: ${copy}\n`;
|
||||
let r=s.run('stack-copy.sh',['check','--copies',s.copies,'--bin',s.bin,'--units','external']);assert.equal(r.status,0,r.stderr);assert.equal(r.stdout,want);
|
||||
assert.deepEqual(readFileSync(argv,'utf8').trimEnd().split('\n'),[
|
||||
'show [email protected] -p LoadState -p WorkingDirectory -p ExecStart',
|
||||
'show mosaic-webui.service -p LoadState -p WorkingDirectory -p ExecStart']);
|
||||
rmSync(argv);
|
||||
for(const args of [[],['--units','user']]){
|
||||
r=s.run('stack-copy.sh',['check','--copies',s.copies,'--bin',s.bin,...args]);assert.equal(r.status,0,r.stderr);assert.equal(r.stdout,want);
|
||||
assert.deepEqual(readFileSync(argv,'utf8').trimEnd().split('\n'),[
|
||||
'--user show [email protected] -p LoadState -p WorkingDirectory -p ExecStart',
|
||||
'--user show mosaic-webui.service -p LoadState -p WorkingDirectory -p ExecStart']);
|
||||
rmSync(argv);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -0,0 +1,476 @@
|
||||
// scripts/stack-release.sh --units external (row 232, #1734). Design: r6 section 4 and the section 7
|
||||
// test list, agents/darkwing/work/queue-195/DESIGN.md; deviations: agents/darkwing/work/queue-232/BUILD.md.
|
||||
// The stand-ins and the module fixture are stack-release.test.mjs's (stack-release-fixture.mjs).
|
||||
// Root's part (stop, unit install, start) runs here as the stand-in systemctl and bus-service.sh and
|
||||
// webui-service.sh installs; no sudo and no real unit.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {execFileSync} from 'node:child_process';
|
||||
import {mkdirSync,writeFileSync,readFileSync,existsSync,readdirSync,rmSync,chmodSync,cpSync} from 'node:fs';
|
||||
import {DatabaseSync} from 'node:sqlite';
|
||||
import {join} from 'node:path';
|
||||
import {B,C,C1,COPIES,ROOT,SBIN,base,fault,lines,logLines,mk,pointer,repo,run,seed,sha,storeVersion,sysctl,tree,units} from './stack-release-fixture.mjs';
|
||||
|
||||
// The commit row 232 builds on: the user-mode tests compare the candidate's calls and --plan bytes
|
||||
// with its scripts. A later row that changes user mode on purpose moves this.
|
||||
const BASE='cb5fdb064fb4f12edd045374dd463dfdc25c2b42';
|
||||
|
||||
const X=(e)=>({...e,flags:[...e.flags,'--units','external']});
|
||||
const sh=(e,name,...a)=>execFileSync('bash',[join(repo,'scripts',name),...a],{env:e.env,encoding:'utf8',stdio:['ignore','pipe','pipe']});
|
||||
// Root's steps: R1/A2 stop, A4 install and start. offset is the start's SD_CLOCK_OFFSET.
|
||||
const down=(e)=>{sysctl(e,'stop',e.con);sysctl(e,'stop',e.bus)};
|
||||
const up=(e,copy,{pm=false,offset=0}={})=>{
|
||||
sh(e,'bus-service.sh','install','--copy',copy,'--copies',COPIES,...(pm?['--pm']:[]),'--claude',e.claude);
|
||||
sh(e,'webui-service.sh','install','--copy',copy,'--copies',COPIES,'--port',e.port,'--business','acme');
|
||||
execFileSync(join(SBIN,'systemctl'),['--user','start',e.bus],{env:{...e.env,SD_CLOCK_OFFSET:String(offset)}});
|
||||
sysctl(e,'start',e.con);
|
||||
};
|
||||
// The stack on B for acme, adopted under --units external, then stopped by root unless live.
|
||||
async function xseed(t,{pm=false,live=false}={}){
|
||||
const e=await mk(t);await seed(e,{pm,pointer:'none'});const x=X(e);
|
||||
const r=await run(x,'adopt','acme');assert.equal(r.code,0,r.err);
|
||||
if(!live)down(e);
|
||||
return {e,x};
|
||||
}
|
||||
const lastAct=(e)=>logLines(e).filter((o)=>o.result==='started').at(-1).activation;
|
||||
const rs=(e,a)=>join(e.state,'stack-rollback',a);
|
||||
const open=(e)=>{const m=new Set();for(const o of logLines(e)){if(!o.activation)continue;if(o.result==='started')m.add(o.activation);else m.delete(o.activation)}return [...m]};
|
||||
const mine=(e,a)=>logLines(e).filter((o)=>o.activation===a);
|
||||
const journalCalls=(e)=>lines(join(e.sd,'journal-since.log'));
|
||||
// A3: stop, stage TO. Returns the activation and the run.
|
||||
async function stage(e,x,to,...args){
|
||||
const r=await run(x,'activate',to,'acme',...args);
|
||||
return {r,a:r.code===0||r.code===1?lastAct(e):null};
|
||||
}
|
||||
// The started line's from, to and op on the activation's result line, and its backup the set's.
|
||||
function sameAsStarted(e,a){
|
||||
const [s,f,...more]=mine(e,a);assert.equal(more.length,0,JSON.stringify(mine(e,a)));
|
||||
assert.equal(s.result,'started');
|
||||
for(const k of ['op','from','to','activation'])assert.equal(f[k],s[k],k);
|
||||
assert.ok(['activate','rollback'].includes(f.op),f.op);
|
||||
assert.equal(f.units,'external');
|
||||
const b=join(rs(e,a),'BACKUP');assert.equal(f.backup,existsSync(b)?readFileSync(b,'utf8').trim():null);
|
||||
return f;
|
||||
}
|
||||
// B1's end state: one started and one result line, nothing open, no marker or FAILED, and a
|
||||
// following --plan that doesn't refuse interrupted-run.
|
||||
async function closed(e,x,a){
|
||||
sameAsStarted(e,a);
|
||||
assert.deepEqual(open(e),[]);
|
||||
for(const f of ['AWAITING-UNITS','FAILED'])assert.ok(!existsSync(join(rs(e,a),f)),f);
|
||||
const s=await run(x,'status','acme');assert.equal(s.code,0,s.err);
|
||||
assert.ok(!/interrupted run|awaiting units/.test(s.out),s.out);
|
||||
const p=await run(x,'activate',pointer(e).copy===C1?B:C1,'acme','--plan');
|
||||
assert.ok(!(p.out+p.err).includes('interrupted-run'),p.out+p.err);
|
||||
}
|
||||
const storeSha=(e)=>sha(e.store);
|
||||
const backupStore=(e,a)=>join(e.data,'backups','stack',a,'bus','bus.sqlite');
|
||||
|
||||
test('row 232: confirm, revert, gate, --to and --expect-from need --units external; the user-only flags are usage under it',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const logs=logLines(e).length;
|
||||
for(const a of [['confirm',C1,'acme'],['revert',C1,'acme'],['gate',B,'acme','--since','1'],['rollback','acme','--to',B],['activate',C1,'acme','--plan','--expect-from',B]]){
|
||||
const r=await run(e,...a);assert.equal(r.code,4,a.join(' '));assert.match(r.err,/^usage: stack-release.sh adopt BUSINESS\n/);
|
||||
}
|
||||
for(const a of [['activate',C1,'acme','--pm'],['activate',C1,'acme','--tester-port','7444'],['activate',C1,'acme','--no-proof'],['activate',C1,'acme','--no-send'],['activate',C1,'acme','--sso'],['activate',C1,'acme','--proof-header','X','--proof-file','/p'],['rollback','acme'],['gate',B,'acme'],['activate',C1,'acme','--expect-from',B],['confirm',C1,'acme','--rollback',B],['confirm',C1,'acme','--cause','A5']]){
|
||||
const r=await run(x,...a);assert.equal(r.code,4,a.join(' '));
|
||||
}
|
||||
assert.equal(logLines(e).length,logs);
|
||||
});
|
||||
|
||||
test('row 232: pm-on and activate --pm-after-harness refuse external-pm',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const p=await run(x,'pm-on','acme');assert.equal(p.code,3);
|
||||
assert.match(p.err,/^stack-release: refusing external-pm: under --units external the PM is a flag in root's bus unit/);
|
||||
const a=await run(x,'activate',C1,'acme','--pm-after-harness');assert.equal(a.code,3);
|
||||
assert.match(a.err,/^stack-release: refusing external-pm: /);
|
||||
const l=logLines(e).slice(-2);assert.deepEqual(l.map((o)=>[o.result,o.code,o.activation]),[['refused','external-pm',null],['refused','external-pm',null]]);
|
||||
assert.equal(readdirSync(e.state).includes('stack-rollback'),false);
|
||||
});
|
||||
|
||||
test('row 232: units-running (N4) refuses while a unit is active and leaves the store, the log and TAKEN; failed counts as down',async(t)=>{
|
||||
const {e,x}=await xseed(t,{live:true});
|
||||
sysctl(e,'stop',e.con);
|
||||
const before=storeSha(e),logs=logLines(e).length;
|
||||
const r=await run(x,'activate',C,'acme','--migrate');assert.equal(r.code,3,r.out+r.err);
|
||||
assert.equal(r.err,`stack-release: refusing units-running: ${e.bus} is active; root stops the units first\n`);
|
||||
assert.equal(storeSha(e),before);assert.equal(storeVersion(e.store,true),'3g');
|
||||
const l=logLines(e).slice(logs);assert.deepEqual(l.map((o)=>[o.result,o.code,o.activation,o.units]),[['refused','units-running',null,'external']]);
|
||||
assert.ok(!existsSync(join(e.data,'backups')));assert.ok(!existsSync(join(e.state,'stack-rollback')));
|
||||
// The Console active and the bus stopped refuses too.
|
||||
sysctl(e,'stop',e.bus);sysctl(e,'start',e.con);
|
||||
const c=await run(x,'activate',C1,'acme');assert.equal(c.code,3);assert.ok(c.err.startsWith(`stack-release: refusing units-running: ${e.con} is active`),c.err);
|
||||
// A bus unit root's stop left failed is down.
|
||||
sysctl(e,'stop',e.con);
|
||||
const s=units(e);s[e.bus].state='failed';writeFileSync(join(e.sd,'units.json'),JSON.stringify(s));
|
||||
const f=await run(x,'activate',C1,'acme');assert.equal(f.code,0,f.out+f.err);
|
||||
});
|
||||
|
||||
test('row 232: activate stages: the pointer stays, AWAITING-UNITS names TO, a backup of the stopped store, no directory in the set, no --user, no journal',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const p0=readFileSync(e.pointer,'utf8'),w0=sha(join(e.bin,'mosaic')),argv0=lines(join(e.sd,'argv.log')).length;
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
assert.equal(r.out.split('\n').at(-2),`stack-release: activate staged ${C1} for acme as ${a}; root installs and starts the units, then runs confirm ${C1} acme --units external`);
|
||||
assert.equal(readFileSync(e.pointer,'utf8'),p0);
|
||||
assert.equal(readFileSync(join(rs(e,a),'AWAITING-UNITS'),'utf8'),`${C1}\n`);
|
||||
assert.deepEqual(readdirSync(rs(e,a)).sort(),['AWAITING-UNITS','BACKUP','SHA256SUMS','UNITS.sha256','WRAPPER_MODE','pointer',`show-${e.bus}.txt`,`show-${e.con}.txt`,'steps.log','wrapper'].sort());
|
||||
assert.equal(readFileSync(join(rs(e,a),'BACKUP'),'utf8'),`${a}\n`);
|
||||
assert.equal(sha(backupStore(e,a)),storeSha(e));assert.equal(readFileSync(join(e.data,'backups','stack',a,'SCHEMA'),'utf8'),'3g\n');
|
||||
assert.equal(readFileSync(join(rs(e,a),'UNITS.sha256'),'utf8'),`${sha(join(e.units,e.tmpl))} ${join(e.units,e.tmpl)}\n${sha(join(e.units,e.con))} ${join(e.units,e.con)}\n`);
|
||||
assert.equal(sha(join(rs(e,a),'wrapper')),w0);assert.notEqual(sha(join(e.bin,'mosaic')),w0);
|
||||
assert.deepEqual(mine(e,a).map((o)=>o.result),['started']);
|
||||
assert.deepEqual([units(e)[e.bus].state,units(e)[e.con].state],['inactive','inactive']);
|
||||
const ext=lines(join(e.sd,'argv.log')).slice(argv0);
|
||||
assert.ok(ext.length>0);for(const c of ext)assert.match(c,/^(show|is-active|list-units) /,c);
|
||||
assert.deepEqual(journalCalls(e),[]);
|
||||
const s=await run(x,'status','acme');assert.equal(s.code,0,s.err);
|
||||
assert.ok(s.out.includes(`awaiting units: ${a} (copy ${C1})\n`),s.out);assert.ok(s.out.includes(`interrupted run: ${a} `),s.out);
|
||||
});
|
||||
|
||||
test('row 232: A3 order: capture-failed and marker-failed refuse with exit 3 and leave no marker or FAILED',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
fault(e,'no-fragment');
|
||||
const r=await run(x,'activate',C1,'acme');assert.equal(r.code,3,r.out+r.err);
|
||||
assert.match(r.err,/^stack-release: refusing capture-failed: could not write the rollback set .*; nothing was changed\n$/);
|
||||
const a=lastAct(e);
|
||||
assert.deepEqual(mine(e,a).map((o)=>[o.result,o.code]),[['started',undefined],['refused','capture-failed']]);
|
||||
for(const f of ['AWAITING-UNITS','FAILED'])assert.ok(!existsSync(join(rs(e,a),f)),f);
|
||||
assert.deepEqual(open(e),[]);
|
||||
rmSync(join(e.sd,'faults','no-fragment'));
|
||||
// The set can't be made: marker-failed, for that activation.
|
||||
chmodSync(join(e.state,'stack-rollback'),0o555);
|
||||
const m=await run(x,'activate',C1,'acme');assert.equal(m.code,3,m.out+m.err);
|
||||
assert.match(m.err,/^stack-release: refusing marker-failed: could not make the rollback set .*; nothing was changed\n$/);
|
||||
const b=lastAct(e);assert.notEqual(b,a);
|
||||
assert.deepEqual(mine(e,b).map((o)=>[o.result,o.code]),[['started',undefined],['refused','marker-failed']]);
|
||||
chmodSync(join(e.state,'stack-rollback'),0o755);
|
||||
assert.deepEqual(open(e),[]);
|
||||
});
|
||||
|
||||
test('row 232: confirm and revert refuse open-activation with no marker, two, the wrong copy, or one the log holds closed; refusals name no activation',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const tryBoth=async(id,why)=>{
|
||||
for(const v of ['confirm','revert']){
|
||||
const o=open(e),logs=logLines(e).length;
|
||||
const r=await run(x,v,id,'acme');assert.equal(r.code,3,`${v} ${why}: ${r.out}${r.err}`);
|
||||
assert.match(r.err,/^stack-release: refusing open-activation: /,why);
|
||||
assert.deepEqual(logLines(e).slice(logs).map((l)=>[l.result,l.code,l.activation,l.op]),[['refused','open-activation',null,v]],why);
|
||||
assert.deepEqual(open(e),o,why);
|
||||
}
|
||||
};
|
||||
await tryBoth(C1,'no marker');
|
||||
// One marker whose activation the log never started. It sorts before any real one, so a confirm
|
||||
// that took the newest of two markers would take the real one.
|
||||
const fake=join(e.state,'stack-rollback','20000101T000000Z-'+C1.slice(0,12));
|
||||
mkdirSync(fake,{recursive:true});writeFileSync(join(fake,'AWAITING-UNITS'),`${C1}\n`);
|
||||
await tryBoth(C1,'marker not open');
|
||||
rmSync(fake,{recursive:true});
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
await tryBoth(B,'wrong copy');
|
||||
mkdirSync(fake);writeFileSync(join(fake,'AWAITING-UNITS'),`${C1}\n`);
|
||||
await tryBoth(C1,'two markers');
|
||||
rmSync(fake,{recursive:true});
|
||||
// gate refuses nothing and writes nothing, open or not.
|
||||
const logs=readFileSync(e.log,'utf8');
|
||||
const g=await run(x,'gate',C1,'acme','--since','1','--timeout','1');assert.equal(g.code,1);
|
||||
assert.equal(readFileSync(e.log,'utf8'),logs);assert.deepEqual(open(e),[a]);
|
||||
});
|
||||
|
||||
test('row 232: confirm polls: bus status good after 3 s fails with --timeout 1, writes FAILED and leaves the rest; FAILED then refuses activation-failed',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
up(e,C1);
|
||||
const p0=readFileSync(e.pointer,'utf8'),l0=readFileSync(e.log,'utf8');
|
||||
fault(e,`status-until-${C1}`,String(Date.now()+3000));
|
||||
const c=await run(x,'confirm',C1,'acme','--timeout','1');assert.equal(c.code,1,c.out+c.err);
|
||||
assert.match(c.err,new RegExp(`^stack-release: activate failed with bus-status; ${a} stays open`));
|
||||
assert.equal(readFileSync(join(rs(e,a),'FAILED'),'utf8'),'bus-status\n');
|
||||
assert.equal(readFileSync(e.pointer,'utf8'),p0);assert.equal(readFileSync(e.log,'utf8'),l0);
|
||||
assert.ok(existsSync(join(rs(e,a),'AWAITING-UNITS')));
|
||||
const f=await run(x,'confirm',C1,'acme');assert.equal(f.code,3);
|
||||
assert.match(f.err,new RegExp(`^stack-release: refusing activation-failed: ${a} has a FAILED file`));
|
||||
assert.equal(logLines(e).at(-1).activation,null);assert.deepEqual(open(e),[a]);
|
||||
});
|
||||
|
||||
test('row 232: confirm polls to a pass with --timeout 10: the pointer, an ok line that reads as the started run, no marker',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
up(e,C1);
|
||||
fault(e,`status-until-${C1}`,String(Date.now()+3000));
|
||||
const c=await run(x,'confirm',C1,'acme','--timeout','10');assert.equal(c.code,0,c.out+c.err);
|
||||
assert.equal(c.out.split('\n').at(-2),`stack-release: activate ok: ${C1} is live for acme`);
|
||||
const p=pointer(e);assert.deepEqual([p.copy,p.activation,p.schema,p.pm,p.console],[C1,a,'3g',false,['--port',e.port,'--business','acme']]);
|
||||
const f=sameAsStarted(e,a);assert.deepEqual([f.result,f.op,f.from,f.to,f.backup],['ok','activate',B,C1,a]);
|
||||
assert.ok(!existsSync(join(rs(e,a),'AWAITING-UNITS')));
|
||||
assert.deepEqual(journalCalls(e),[]);
|
||||
});
|
||||
|
||||
// N4 with the brief's numbers: the started line at 1792540800.900 floors to 1792540800. The host's
|
||||
// clock runs so boot status is after that second.
|
||||
const T0=1792540800,offsetTo=(s)=>Math.floor(Date.now()/1000)-s;
|
||||
function startedAt(e,a,ms){
|
||||
const l=readFileSync(e.log,'utf8').split('\n').map((x)=>{if(!x)return x;const o=JSON.parse(x);if(o.activation===a&&o.result==='started')o.at=new Date(ms).toISOString();return JSON.stringify(o)});
|
||||
writeFileSync(e.log,l.join('\n'));
|
||||
}
|
||||
|
||||
test('row 232: stale-start: @1792540799 fails a start floor of 1792540800 with stale-start',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
startedAt(e,a,T0*1000+900);up(e,C1,{offset:offsetTo(T0+100)});
|
||||
fault(e,'enter',JSON.stringify({value:T0-1}));
|
||||
const c=await run(x,'confirm',C1,'acme','--timeout','1');assert.equal(c.code,1,c.out+c.err);
|
||||
assert.ok(c.out.includes(`gate failed on ${C1}: stale-start (the bus started at @${T0-1}, before @${T0})`),c.out);
|
||||
assert.equal(readFileSync(join(rs(e,a),'FAILED'),'utf8'),'stale-start\n');
|
||||
});
|
||||
|
||||
test('row 232: stale-start is a gate code: an enter time that turns to @1792540800 after 2 s passes with --timeout 10',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
startedAt(e,a,T0*1000+900);up(e,C1,{offset:offsetTo(T0+100)});
|
||||
fault(e,'enter',JSON.stringify({value:T0-1,at:Date.now()+2000,then:T0}));
|
||||
const c=await run(x,'confirm',C1,'acme','--timeout','10');assert.equal(c.code,0,c.out+c.err);
|
||||
assert.equal(pointer(e).copy,C1);
|
||||
});
|
||||
|
||||
test('row 232: gate (N2, A2f): --since 1792540800 passes @1792540800 and fails @1792540799; it writes nothing',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
up(e,B,{offset:offsetTo(T0+100)});
|
||||
const snap=()=>[readFileSync(e.log,'utf8'),readFileSync(e.pointer,'utf8'),tree(e.state).join('\n')];
|
||||
const s0=snap();
|
||||
fault(e,'enter',JSON.stringify({value:T0}));
|
||||
const g=await run(x,'gate',B,'acme','--since',String(T0));assert.equal(g.code,0,g.out+g.err);
|
||||
assert.equal(g.out,`stack-release: gate passed on ${B}\n`);
|
||||
fault(e,'enter',JSON.stringify({value:T0-1}));
|
||||
const f=await run(x,'gate',B,'acme','--since',String(T0),'--timeout','1');assert.equal(f.code,1,f.out+f.err);
|
||||
assert.ok(f.out.includes(`gate failed on ${B}: stale-start (the bus started at @${T0-1}, before @${T0})`),f.out);
|
||||
assert.deepEqual(snap(),s0);
|
||||
const u=await run(x,'gate',B,'acme');assert.equal(u.code,4);
|
||||
assert.deepEqual(journalCalls(e),[]);
|
||||
});
|
||||
|
||||
test('row 232: the external PM check reads launches list: one live run of launch.by since the start passes',async(t)=>{
|
||||
const {e,x}=await xseed(t,{pm:true});
|
||||
const cases=[['',0,''],['coder',0,''],['none',1,'no live launch of chief'],['twolive',1,'2 live launches of chief'],['early',1,'started before this start'],['dead',1,'no live launch of chief'],['other',1,'the launch of chief is for other']];
|
||||
for(const [mode,code,why] of cases){
|
||||
rmSync(join(e.sd,'launches.json'),{force:true});
|
||||
if(mode)fault(e,`pm-${B}`,mode);else rmSync(join(e.sd,'faults',`pm-${B}`),{force:true});
|
||||
const since=Math.floor(Date.now()/1000);
|
||||
up(e,B,{pm:true});
|
||||
const g=await run(x,'gate',B,'acme','--since',String(since),'--timeout','1');assert.equal(g.code,code,`${mode}: ${g.out}${g.err}`);
|
||||
if(why)assert.ok(g.out.includes(`gate failed on ${B}: pm-launch-unconfirmed (`)&&g.out.includes(why),`${mode}: ${g.out}`);
|
||||
down(e);
|
||||
}
|
||||
assert.deepEqual(journalCalls(e),[]);
|
||||
});
|
||||
|
||||
test('row 232: revert puts the wrapper back and leaves the store when nothing migrated',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const w0=sha(join(e.bin,'mosaic'));
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
// A write to the stopped store after A3: revert has no MIGRATING, so it is not its to undo.
|
||||
const db=new DatabaseSync(e.store);db.prepare('INSERT INTO events (at) VALUES (?)').run('2001-01-01T00:00:00.000Z');db.close();
|
||||
const s1=storeSha(e);assert.notEqual(s1,sha(backupStore(e,a)));
|
||||
const v=await run(x,'revert',C1,'acme');assert.equal(v.code,0,v.out+v.err);
|
||||
assert.equal(v.out.split('\n').at(-2),`stack-release: revert ok: ${a}'s wrapper is back; root installs the old units next`);
|
||||
assert.equal(sha(join(e.bin,'mosaic')),w0);assert.equal(storeSha(e),s1);
|
||||
assert.deepEqual(mine(e,a).map((o)=>o.result),['started']);assert.ok(existsSync(join(rs(e,a),'AWAITING-UNITS')));
|
||||
});
|
||||
|
||||
test('row 232: confirm --rollback refuses rollback-mismatch, and logs rollback-failed with exit 2 when OLD fails its gate',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
const v=await run(x,'revert',C1,'acme');assert.equal(v.code,0,v.out+v.err);
|
||||
const m=await run(x,'confirm',C1,'acme','--rollback',C,'--cause','A5');assert.equal(m.code,3);
|
||||
assert.equal(m.err,`stack-release: refusing rollback-mismatch: ${C} is not the copy ${a} started from, ${B}\n`);
|
||||
assert.equal(logLines(e).at(-1).activation,null);assert.deepEqual(open(e),[a]);
|
||||
fault(e,`healthz-${B}`);up(e,B);
|
||||
const f=await run(x,'confirm',C1,'acme','--rollback',B,'--cause','A5','--timeout','1');assert.equal(f.code,2,f.out+f.err);
|
||||
assert.equal(f.err,`stack-release: rollback-failed: the run failed with unrecorded and the rollback with healthz; root stops the units. Rollback set: ${rs(e,a)}, backup ${join(e.data,'backups','stack',a)}\n`);
|
||||
const l=sameAsStarted(e,a);assert.deepEqual([l.result,l.code,l.rollback,l.rollbackCode],['rollback-failed','unrecorded','failed','healthz']);
|
||||
for(const n of ['AWAITING-UNITS','FAILED'])assert.ok(!existsSync(join(rs(e,a),n)),n);
|
||||
assert.deepEqual(open(e),[]);
|
||||
});
|
||||
|
||||
test('row 232: confirm --rollback after a --restore swap logs rollbackCode not-attempted; rollback --restore finds an external backup',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C,'--migrate');assert.equal(r.code,0,r.out+r.err);
|
||||
up(e,C);assert.equal((await run(x,'confirm',C,'acme')).code,0);
|
||||
down(e);
|
||||
const b=await run(x,'rollback','acme','--to',B,'--restore',a);assert.equal(b.code,0,b.out+b.err);
|
||||
const ra=lastAct(e);
|
||||
assert.ok(b.out.includes(`restore: ${join(e.data,'backups','stack',a)}; `),b.out);
|
||||
assert.ok(existsSync(join(rs(e,ra),'RESTORED')));assert.equal(readFileSync(join(rs(e,ra),'BACKUP'),'utf8'),`${a}\n`);
|
||||
assert.equal(storeSha(e),sha(backupStore(e,a)));
|
||||
const f=await run(x,'confirm',B,'acme','--rollback',C,'--cause','A4');assert.equal(f.code,2,f.out+f.err);
|
||||
const l=sameAsStarted(e,ra);assert.deepEqual([l.op,l.result,l.code,l.rollbackCode,l.backup],['rollback','rollback-failed','units-start','not-attempted',a]);
|
||||
});
|
||||
|
||||
// --- B1: real sequences on one log ---
|
||||
|
||||
test('row 232: sequence 1: activate, confirm times out, revert, confirm --rollback --cause A5: gate-failed rollback ok',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const w0=sha(join(e.bin,'mosaic')),p0=readFileSync(e.pointer,'utf8');
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
fault(e,`healthz-${C1}`);up(e,C1);
|
||||
const c=await run(x,'confirm',C1,'acme','--timeout','1');assert.equal(c.code,1,c.out+c.err);
|
||||
assert.deepEqual(mine(e,a).map((o)=>o.result),['started']);
|
||||
down(e);
|
||||
assert.equal((await run(x,'revert',C1,'acme')).code,0);assert.equal(sha(join(e.bin,'mosaic')),w0);
|
||||
up(e,B);
|
||||
const k=await run(x,'confirm',C1,'acme','--rollback',B,'--cause','A5');assert.equal(k.code,1,k.out+k.err);
|
||||
assert.equal(k.err,`stack-release: healthz; rolled back to ${B}, which passed the gate\n`);
|
||||
const l=sameAsStarted(e,a);assert.deepEqual([l.result,l.code,l.rollback],['gate-failed','healthz','ok']);
|
||||
assert.equal(readFileSync(e.pointer,'utf8'),p0);
|
||||
await closed(e,x,a);
|
||||
});
|
||||
|
||||
test('row 232: sequence 2: activate --migrate, the wrapper install fails, revert restores TAKEN, confirm --rollback --cause A3',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const s0=storeSha(e);
|
||||
chmodSync(e.bin,0o555);
|
||||
const {r,a}=await stage(e,x,C,'--migrate');chmodSync(e.bin,0o755);
|
||||
assert.equal(r.code,1,r.out+r.err);assert.match(r.err,new RegExp(`^stack-release: activate failed with wrapper; ${a} stays open`));
|
||||
assert.deepEqual(mine(e,a).map((o)=>o.result),['started']);
|
||||
for(const n of ['AWAITING-UNITS','MIGRATING'])assert.ok(existsSync(join(rs(e,a),n)),n);
|
||||
assert.equal(readFileSync(join(rs(e,a),'FAILED'),'utf8'),'wrapper\n');
|
||||
assert.equal(storeVersion(e.store,false),'3h');assert.equal(sha(backupStore(e,a)),s0);
|
||||
const v=await run(x,'revert',C,'acme');assert.equal(v.code,0,v.out+v.err);
|
||||
assert.equal(v.out.split('\n').at(-2),`stack-release: revert ok: ${a}'s wrapper and store are back; root installs the old units next`);
|
||||
assert.equal(storeSha(e),s0);
|
||||
// revert runs again: the store already matches, and nothing moves.
|
||||
const posts=readdirSync(e.data).filter((n)=>n.startsWith('bus.post-attempt-'));
|
||||
assert.equal((await run(x,'revert',C,'acme')).code,0);
|
||||
assert.deepEqual(readdirSync(e.data).filter((n)=>n.startsWith('bus.post-attempt-')),posts);
|
||||
up(e,B);
|
||||
const k=await run(x,'confirm',C,'acme','--rollback',B,'--cause','A3');assert.equal(k.code,1,k.out+k.err);
|
||||
const l=sameAsStarted(e,a);assert.deepEqual([l.result,l.code,l.rollback],['failed','wrapper','ok']);
|
||||
await closed(e,x,a);
|
||||
});
|
||||
|
||||
test('row 232: sequence 3: activate, root\'s A4 fails, revert, confirm --rollback --cause A4: failed code units-start',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
assert.equal((await run(x,'revert',C1,'acme')).code,0);
|
||||
up(e,B);
|
||||
const k=await run(x,'confirm',C1,'acme','--rollback',B,'--cause','A4');assert.equal(k.code,1,k.out+k.err);
|
||||
assert.equal(k.err,`stack-release: units-start; rolled back to ${B}, which passed the gate\n`);
|
||||
const l=sameAsStarted(e,a);assert.deepEqual([l.result,l.code,l.rollback],['failed','units-start','ok']);
|
||||
await closed(e,x,a);
|
||||
});
|
||||
|
||||
test('row 232: sequence 4: a capture failure leaves nothing open, then A2f\'s gate OLD writes nothing',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
fault(e,'no-fragment');
|
||||
const r=await run(x,'activate',C1,'acme');assert.equal(r.code,3,r.out+r.err);rmSync(join(e.sd,'faults','no-fragment'));
|
||||
const a=lastAct(e);assert.deepEqual(mine(e,a).map((o)=>[o.result,o.code]),[['started',undefined],['refused','capture-failed']]);
|
||||
assert.deepEqual(open(e),[]);
|
||||
const since=Math.floor(Date.now()/1000);up(e,B);
|
||||
const s0=[readFileSync(e.log,'utf8'),readFileSync(e.pointer,'utf8'),tree(e.state).join('\n')];
|
||||
const g=await run(x,'gate',B,'acme','--since',String(since));assert.equal(g.code,0,g.out+g.err);
|
||||
assert.deepEqual([readFileSync(e.log,'utf8'),readFileSync(e.pointer,'utf8'),tree(e.state).join('\n')],s0);
|
||||
});
|
||||
|
||||
test('row 232: sequence 5: a confirm of the wrong copy is refused, and revert and confirm --rollback still close the activation',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
up(e,C1);
|
||||
const w=await run(x,'confirm',B,'acme');assert.equal(w.code,3);assert.match(w.err,/^stack-release: refusing open-activation: /);
|
||||
assert.equal(logLines(e).at(-1).activation,null);assert.deepEqual(open(e),[a]);
|
||||
down(e);
|
||||
assert.equal((await run(x,'revert',C1,'acme')).code,0);
|
||||
up(e,B);
|
||||
const k=await run(x,'confirm',C1,'acme','--rollback',B,'--cause','A5');assert.equal(k.code,1,k.out+k.err);
|
||||
const l=sameAsStarted(e,a);assert.deepEqual([l.result,l.code,l.rollback],['gate-failed','unrecorded','ok']);
|
||||
await closed(e,x,a);
|
||||
});
|
||||
|
||||
test('row 232: sequence 6: a revert that fails leaves the activation open, and the next --plan refuses interrupted-run',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
rmSync(join(rs(e,a),'wrapper'));
|
||||
const v=await run(x,'revert',C1,'acme');assert.equal(v.code,1,v.out+v.err);
|
||||
assert.equal(v.err,`stack-release: revert failed: the rollback set has no saved wrapper; ${a} stays open. Rollback set: ${rs(e,a)}\n`);
|
||||
assert.deepEqual(open(e),[a]);assert.ok(existsSync(join(rs(e,a),'AWAITING-UNITS')));
|
||||
const p=await run(x,'activate',C,'acme','--plan','--migrate');assert.equal(p.code,3,p.out+p.err);
|
||||
assert.ok(p.out.includes(`refuses interrupted-run: activation ${a} started and has no final line`),p.out);
|
||||
});
|
||||
|
||||
test('row 232: sequence 7: adopt, two confirmed activates, rollback --to the previous copy, confirmed, logs op rollback',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const one=await stage(e,x,C1);assert.equal(one.r.code,0,one.r.out+one.r.err);
|
||||
up(e,C1);assert.equal((await run(x,'confirm',C1,'acme')).code,0);down(e);
|
||||
const two=await stage(e,x,C,'--migrate');assert.equal(two.r.code,0,two.r.out+two.r.err);
|
||||
up(e,C);assert.equal((await run(x,'confirm',C,'acme')).code,0);down(e);
|
||||
const l1=sameAsStarted(e,one.a),l2=sameAsStarted(e,two.a);
|
||||
assert.deepEqual([l1.op,l1.result,l1.from,l1.to,l1.backup],['activate','ok',B,C1,one.a]);
|
||||
assert.deepEqual([l2.op,l2.result,l2.from,l2.to,l2.backup],['activate','ok',C1,C,two.a]);
|
||||
// N5: --expect-from names the copy root read from its units.
|
||||
const d1=await run(x,'activate',C1,'acme','--plan','--expect-from',C1);assert.equal(d1.code,3,d1.out+d1.err);
|
||||
assert.ok(d1.out.includes(`refuses pointer-differs: the pointer names ${C} and root's units name ${C1}; Jason or Mos settles it`),d1.out);
|
||||
const d2=await run(x,'rollback','acme','--to',C1,'--plan','--expect-from',B);assert.equal(d2.code,3,d2.out+d2.err);
|
||||
assert.ok(d2.out.includes(`refuses pointer-differs: the pointer names ${C} and root's units name ${B}`),d2.out);
|
||||
const ok=await run(x,'rollback','acme','--to',C1,'--restore',two.a,'--plan','--expect-from',C);assert.equal(ok.code,0,ok.out+ok.err);
|
||||
const logs=logLines(e).length;
|
||||
const no=await run(x,'rollback','acme','--to',B,'--plan');assert.equal(no.code,3);
|
||||
assert.equal(no.err,`stack-release: refusing to-not-previous: --to ${B} is not the previous copy; the log names ${C1}\n`);
|
||||
assert.equal(logLines(e).length,logs);
|
||||
const nr=await run(x,'rollback','acme','--to',B,'--restore',one.a);assert.equal(nr.code,3);assert.match(nr.err,/^stack-release: refusing to-not-previous: /);
|
||||
assert.deepEqual(logLines(e).slice(logs).map((o)=>[o.result,o.code,o.activation]),[['refused','to-not-previous',null]]);
|
||||
const rb=await run(x,'rollback','acme','--to',C1,'--restore',two.a);assert.equal(rb.code,0,rb.out+rb.err);
|
||||
const ra=lastAct(e);
|
||||
up(e,C1);
|
||||
const k=await run(x,'confirm',C1,'acme');assert.equal(k.code,0,k.out+k.err);
|
||||
assert.equal(k.out.split('\n').at(-2),`stack-release: rollback ok: ${C1} is live for acme`);
|
||||
const l3=sameAsStarted(e,ra);assert.deepEqual([l3.op,l3.result,l3.from,l3.to,l3.backup],['rollback','ok',C,C1,two.a]);
|
||||
assert.deepEqual([pointer(e).copy,pointer(e).schema],[C1,'3g']);
|
||||
const prev=await run(x,'rollback','acme','--to',B,'--plan');assert.match(prev.err,new RegExp(`the log names ${C}\\n$`));
|
||||
await closed(e,x,ra);
|
||||
});
|
||||
|
||||
test('row 232: sequence 8: a migration that writes and fails: revert restores the backup byte for byte, confirm --rollback --cause A3',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const s0=storeSha(e);
|
||||
fault(e,'migrate-fails');
|
||||
const {r,a}=await stage(e,x,C,'--migrate');assert.equal(r.code,1,r.out+r.err);rmSync(join(e.sd,'faults','migrate-fails'));
|
||||
assert.equal(readFileSync(join(rs(e,a),'FAILED'),'utf8'),'migrate-failed\n');assert.ok(existsSync(join(rs(e,a),'MIGRATING')));
|
||||
assert.equal(storeVersion(e.store,false),'3h-partial');
|
||||
assert.equal((await run(x,'revert',C,'acme')).code,0);
|
||||
assert.equal(storeSha(e),s0);assert.equal(storeSha(e),sha(backupStore(e,a)));
|
||||
assert.deepEqual(readdirSync(join(e.data,'bus')),readdirSync(join(e.data,'backups','stack',a,'bus')));
|
||||
up(e,B);
|
||||
const k=await run(x,'confirm',C,'acme','--rollback',B,'--cause','A3');assert.equal(k.code,1,k.out+k.err);
|
||||
const l=sameAsStarted(e,a);assert.deepEqual([l.result,l.code,l.rollback],['failed','migrate-failed','ok']);
|
||||
await closed(e,x,a);
|
||||
});
|
||||
|
||||
// --- User mode against the base ---
|
||||
|
||||
// The fixture repo with the base's two scripts: everything else is the same tree and history.
|
||||
const baseRepo=join(base,'base-repo');
|
||||
cpSync(repo,baseRepo,{recursive:true});
|
||||
for(const f of ['scripts/stack-release.sh','scripts/stack-copy.sh'])writeFileSync(join(baseRepo,f),execFileSync('git',['-C',ROOT,'show',`${BASE}:${f}`]),{mode:0o755});
|
||||
const BASE_SR=join(baseRepo,'scripts','stack-release.sh');
|
||||
|
||||
test('row 232: user mode: every systemctl call carries --user, and activate, rollback and check make the base\'s calls',async(t)=>{
|
||||
const calls=async(sr,dir)=>{
|
||||
const e=await mk(t);e.sr=sr;await seed(e);
|
||||
const n=lines(join(e.sd,'argv.log')).length;
|
||||
const a=await run(e,'activate',C1,'acme');assert.equal(a.code,0,a.out+a.err);
|
||||
const r=await run(e,'rollback','acme');assert.equal(r.code,0,r.out+r.err);
|
||||
execFileSync('bash',[join(dir,'scripts','stack-copy.sh'),'check','--copies',COPIES,'--bin',e.bin,'--business','acme'],{env:e.env,stdio:'ignore'});
|
||||
return lines(join(e.sd,'argv.log')).slice(n).map((l)=>l.split(e.root).join('<root>').split(e.port).join('<port>'));
|
||||
};
|
||||
const cand=await calls(undefined,repo),old=await calls(BASE_SR,baseRepo);
|
||||
assert.ok(cand.length>20);
|
||||
for(const c of cand)assert.match(c,/^--user /,c);
|
||||
assert.deepEqual(cand,old);
|
||||
});
|
||||
|
||||
test('row 232: user mode: a default activate --plan prints the base\'s bytes',async(t)=>{
|
||||
const e=await mk(t);await seed(e);
|
||||
for(const a of [['activate',C1,'acme','--plan'],['activate',C,'acme','--plan','--migrate'],['activate',C,'acme','--plan'],['rollback','acme','--plan']]){
|
||||
const cand=await run(e,...a),old=await run({...e,sr:BASE_SR},...a);
|
||||
assert.deepEqual(cand,old,a.join(' '));
|
||||
}
|
||||
assert.ok(!existsSync(join(e.state,'stack-rollback')));
|
||||
});
|
||||
@@ -0,0 +1,451 @@
|
||||
// The stack-release stand-ins and module fixture, shared by stack-release.test.mjs and
|
||||
// stack-release-external.test.mjs (row 232). Moved unchanged from stack-release.test.mjs, plus
|
||||
// what the external tests need; each test file runs in its own process, so each gets its own base.
|
||||
import test, {after} from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {execFileSync,spawn,spawnSync} from 'node:child_process';
|
||||
import {mkdtempSync,mkdirSync,writeFileSync,readFileSync,appendFileSync,cpSync,rmSync,chmodSync,existsSync,statSync,lstatSync,symlinkSync,linkSync,readdirSync,realpathSync} from 'node:fs';
|
||||
import {createServer} from 'node:http';
|
||||
import {createHash} from 'node:crypto';
|
||||
import {DatabaseSync} from 'node:sqlite';
|
||||
import {tmpdir} from 'node:os';
|
||||
import {join,dirname,basename} from 'node:path';
|
||||
import {fileURLToPath,pathToFileURL} from 'node:url';
|
||||
import {once} from 'node:events';
|
||||
|
||||
export const ROOT=fileURLToPath(new URL('../../..',import.meta.url)).replace(/\/$/,'');
|
||||
export const NODE=process.execPath;
|
||||
export const MARKER='# mosaic-cli wrapper (Q81, agents/sage/work/q81/mosaic-cli.sh)';
|
||||
export const BUS='[email protected]',CON='mosaic-webui.service';
|
||||
export const git=(cwd,...args)=>execFileSync('git',['-c','user.name=fixture','-c','user.email=fixture@invalid','-c','commit.gpgsign=false','-c','core.hooksPath=/dev/null','-C',cwd,...args],{encoding:'utf8'}).trim();
|
||||
export const script=(fn)=>`#!${NODE}\n(${fn})();\n`;
|
||||
|
||||
// --- Stand-ins. Each is serialized into its own file, so it uses only process and its own scope. ---
|
||||
|
||||
// systemctl --user: unit state in $SD_STATE/units.json, every call in calls.log. A bus start runs
|
||||
// holder.cjs (the store held open in WAL mode, writer.lock, host.json) and, with --pm in the unit's
|
||||
// ExecStart, writes the `launched` journal line and the launches entry unless a pm-<copy> fault
|
||||
// names another case. Faults are files in $SD_STATE/faults named for the copy they hit. Units of a
|
||||
// named instance (row 190) run on their own data root and host files beside the default's.
|
||||
export function systemctl(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),cp=process.getBuiltinModule('node:child_process'),crypto=process.getBuiltinModule('node:crypto');
|
||||
const sd=process.env.SD_STATE,data=process.env.SD_DATA,unitDir=path.join(process.env.XDG_CONFIG_HOME,'systemd','user');
|
||||
const args=process.argv.slice(2).filter((a)=>a!=='--user'),[verb,...rest]=args;
|
||||
fs.appendFileSync(path.join(sd,'calls.log'),args.join(' ')+'\n');
|
||||
// Row 232: argv.log keeps each call as given, --user and all, for the user/external argv tests.
|
||||
fs.appendFileSync(path.join(sd,'argv.log'),process.argv.slice(2).join(' ')+'\n');
|
||||
const faultFile=(n)=>path.join(sd,'faults',n),fault=(n)=>fs.existsSync(faultFile(n));
|
||||
const stFile=path.join(sd,'units.json');
|
||||
const load=()=>{try{return JSON.parse(fs.readFileSync(stFile,'utf8'))}catch{return {}}};
|
||||
const save=(s)=>{fs.writeFileSync(stFile+'.tmp',JSON.stringify(s));fs.renameSync(stFile+'.tmp',stFile)};
|
||||
const sleep=(ms)=>Atomics.wait(new Int32Array(new SharedArrayBuffer(4)),0,0,ms);
|
||||
const alive=(pid)=>{try{const s=fs.readFileSync(`/proc/${pid}/stat`,'utf8');return s[s.lastIndexOf(')')+2]!=='Z'}catch{return false}};
|
||||
// Row 190: a bus unit is the default's mosaic-bus@B or instance NAME's mosaic-bus-NAME@B; %i is B.
|
||||
const isBus=(u)=>/^mosaic-bus(-[a-z][a-z0-9]{1,15})?@[a-z0-9-]+\.service$/.test(u),inst=(u)=>u.slice(u.indexOf('@')+1,-'.service'.length);
|
||||
const unitFile=(u)=>{
|
||||
const main=isBus(u)?u.replace(/@[^@]*$/,'@.service'):u,file=path.join(unitDir,main);
|
||||
if(!fs.existsSync(file))return null;
|
||||
// The drop-in directories in systemd's order: service.d, the name cut after each hyphen, then
|
||||
// the template's and the unit's own.
|
||||
const stem=main.slice(0,main.search(/@?\.service$/)),dirs=['service.d'],drops=[];
|
||||
for(let i=0;i<stem.length;i++)if(stem[i]==='-')dirs.push(stem.slice(0,i+1)+'.service.d');
|
||||
for(const d of [...dirs,...(isBus(u)?[main+'.d',u+'.d']:[u+'.d'])]){const dir=path.join(unitDir,d);if(fs.existsSync(dir))for(const f of fs.readdirSync(dir).sort())if(f.endsWith('.conf'))drops.push(path.join(dir,f))}
|
||||
const r={file,wd:'',exec:'',env:[],drops};
|
||||
for(const f of [file,...drops])for(let line of fs.readFileSync(f,'utf8').split('\n')){
|
||||
line=line.replace(/%i/g,isBus(u)?inst(u):'').replace(/%h/g,process.env.HOME);
|
||||
const m=/^(WorkingDirectory|ExecStart|Environment)=(.*)$/.exec(line);if(!m)continue;
|
||||
if(m[1]==='WorkingDirectory')r.wd=m[2];else if(m[1]==='ExecStart'){if(m[2])r.exec=m[2]}else r.env.push(m[2]);
|
||||
}
|
||||
return r;
|
||||
};
|
||||
// A named instance's unit sets MOSAIC_CONFIG, and its bus works on that config's data root. A unit
|
||||
// without it is the default's, on $SD_DATA. Each data root has its own host and launches files.
|
||||
const dataOf=(f)=>{const c=f&&f.env.find((v)=>v.startsWith('MOSAIC_CONFIG='));return c?JSON.parse(fs.readFileSync(c.slice('MOSAIC_CONFIG='.length),'utf8')).dataRoot:data};
|
||||
const perData=(d,n)=>path.join(sd,d===data?`${n}.json`:`${n}-${path.basename(d)}.json`);
|
||||
const snap=(ev,d)=>{
|
||||
const dir=path.join(d,'bus'),files={};
|
||||
try{for(const n of fs.readdirSync(dir).sort()){const p=path.join(dir,n),st=fs.statSync(p);if(st.isFile())files[n]=[st.size,st.mtimeMs,crypto.createHash('sha256').update(fs.readFileSync(p)).digest('hex')]}}catch{}
|
||||
fs.appendFileSync(path.join(sd,'snap.jsonl'),JSON.stringify({ev,files})+'\n');
|
||||
};
|
||||
const pm=(unit,d,id,pid,off)=>{
|
||||
let mode='';try{mode=fs.readFileSync(faultFile(`pm-${id}`),'utf8').trim()}catch{}
|
||||
const b=inst(unit),now=Date.now()-off*1000,epoch=Math.floor(now/1000),at=new Date(now).toISOString(),run=`run-${pid}`;
|
||||
const line=(text)=>fs.appendFileSync(path.join(sd,'journal.jsonl'),JSON.stringify({epoch,unit,text})+'\n');
|
||||
const ok=()=>line(`launched pm-${b}: run ${run}, claude opus`);
|
||||
const lf=perData(d,'launches');let list=[];try{list=JSON.parse(fs.readFileSync(lf,'utf8'))}catch{}
|
||||
// instance is launch.by's, chief in the example business, as the external PM check reads (row 232).
|
||||
const entry=(o)=>list.push({run,instance:'chief',business:b,live:true,startedAt:at,...o});
|
||||
line(`bus host up: business ${b}, socket /fixture/bus.sock, notifier off`);
|
||||
switch(mode){
|
||||
case 'none':break;
|
||||
case 'two':ok();line(`launched pm-${b}: run ${run}b, claude opus`);entry({});break;
|
||||
case 'badword':line(`launched pm-${b}: job ${run}, claude opus`);entry({});break;
|
||||
case 'notlisted':ok();break;
|
||||
case 'early':ok();entry({startedAt:new Date(now-3600e3).toISOString()});break;
|
||||
case 'human':entry({run:`human-${pid}`});break;
|
||||
case 'other':ok();entry({business:'other'});break;
|
||||
case 'dead':ok();entry({live:false});break;
|
||||
// Row 232: two live entries for the PM's instance, and one for it beside a live coder run.
|
||||
case 'twolive':ok();entry({});entry({run:`${run}b`});break;
|
||||
case 'coder':ok();entry({});entry({run:`${run}c`,instance:'coder'});break;
|
||||
default:ok();entry({});
|
||||
}
|
||||
fs.writeFileSync(lf,JSON.stringify(list));
|
||||
};
|
||||
if(verb==='show'){
|
||||
// `show [--timestamp=unix] UNIT -p P...` and `show -p P UNIT`, as the external path calls it.
|
||||
let u='';const props=[];for(let i=0;i<rest.length;i++){if(rest[i]==='-p')props.push(rest[++i]);else if(!rest[i].startsWith('--'))u=rest[i]}
|
||||
const f=unitFile(u),st=load()[u]||{},id=f?path.basename(f.wd):'';
|
||||
// ActiveEnterTimestamp (row 232) is the start's second, or what the enter fault says: a JSON
|
||||
// {value} or {value,at,then}, which answers then from epoch ms at on.
|
||||
const enter=()=>{
|
||||
if(!fault('enter'))return st.enter;
|
||||
const o=JSON.parse(fs.readFileSync(faultFile('enter'),'utf8'));return o.at!==undefined&&Date.now()>=o.at?o.then:o.value;
|
||||
};
|
||||
for(const p of props){
|
||||
if(p==='LoadState')console.log(`LoadState=${f?'loaded':'not-found'}`);
|
||||
else if(p==='WorkingDirectory')console.log(`WorkingDirectory=${f?f.wd+(!isBus(u)&&fault(`check-${id}`)?'-moved':''):''}`);
|
||||
else if(p==='ExecStart')console.log(f&&f.exec?`ExecStart={ path=${f.exec.split(' ')[0]} ; argv[]=${f.exec} ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }`:'ExecStart=');
|
||||
else if(p==='Environment')console.log(`Environment=${f?f.env.join(' '):''}`);
|
||||
else if(p==='MainPID')console.log(`MainPID=${st.pid||0}`);
|
||||
else if(p==='DropInPaths')console.log(`DropInPaths=${f?f.drops.join(' '):''}`);
|
||||
else if(p==='FragmentPath')console.log(`FragmentPath=${f&&!fault('no-fragment')?f.file:''}`);
|
||||
else if(p==='ActiveEnterTimestamp'){const v=st.state==='active'?enter():undefined;console.log(`ActiveEnterTimestamp=${v===undefined?'':'@'+v}`)}
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
if(verb==='is-active'){
|
||||
const q=rest[0]==='--quiet',u=q?rest[1]:rest[0];let s=(load()[u]||{}).state||'inactive';
|
||||
// bus-stops-after N (T29): the next N bus checks answer the real state, every later one answers
|
||||
// inactive, and units.json stays as it is, as if the bus stopped between two checks.
|
||||
if(isBus(u)&&fault('bus-stops-after')){const n=Number(fs.readFileSync(faultFile('bus-stops-after'),'utf8'));if(n>0)fs.writeFileSync(faultFile('bus-stops-after'),String(n-1));else s='inactive'}
|
||||
if(!q)console.log(s);process.exit(s==='active'?0:3);
|
||||
}
|
||||
if(verb==='is-enabled'){console.log('disabled');process.exit(1)}
|
||||
if(verb==='daemon-reload')process.exit(0);
|
||||
if(verb==='list-units'){
|
||||
if(fault('list-fails'))process.exit(1);
|
||||
const pats=rest.filter((a)=>!a.startsWith('--')).map((p)=>new RegExp('^'+p.replace(/\./g,'\\.').replace(/\*/g,'.*')+'$'));
|
||||
for(const [u,s] of Object.entries(load()))if(pats.some((re)=>re.test(u)))console.log(`${u} loaded ${s.state} ${s.state==='active'?'running':'dead'} fixture`);
|
||||
process.exit(0);
|
||||
}
|
||||
if(verb==='start'){
|
||||
const u=rest[0],f=unitFile(u);if(!f){console.error(`Unit ${u} not found.`);process.exit(5)}
|
||||
const s=load(),id=path.basename(f.wd),d=dataOf(f);
|
||||
const enter=Math.floor((Date.now()-Number(process.env.SD_CLOCK_OFFSET||'0')*1000)/1000);
|
||||
if(!isBus(u)){s[u]={state:'active',pid:null,wd:f.wd,enter};save(s);process.exit(0)}
|
||||
if(fault(`start-dies-${id}`)){s[u]={state:'failed',pid:null,wd:f.wd};save(s);process.exit(0)}
|
||||
// Row 185 (T48): the copy's start writes one event before the holder opens the store, so the
|
||||
// store differs from a backup the run took before the start. The close leaves no -wal.
|
||||
if(fault(`start-writes-row-${id}`)){
|
||||
const db=new (process.getBuiltinModule('node:sqlite').DatabaseSync)(path.join(d,'bus','bus.sqlite'));
|
||||
db.prepare('INSERT INTO events (at) VALUES (?)').run('2999-01-01T00:00:00.000Z');db.close();
|
||||
}
|
||||
snap('start',d);
|
||||
const off=process.env.SD_CLOCK_OFFSET||'0';
|
||||
// The holder must not inherit stack-release.sh's lock (fd 9) or any other fd above 2.
|
||||
const close='for fd in /proc/self/fd/*; do n=${fd##*/}; [ "$n" -gt 2 ] && eval "exec $n>&-" 2>/dev/null; done; exec "$@"';
|
||||
const ch=cp.spawn('bash',['-c',close,'holder',process.execPath,path.join(__dirname,'..','holder.cjs'),d,inst(u),sd,off,perData(d,'host')],{detached:true,stdio:'ignore'});
|
||||
ch.unref();
|
||||
fs.appendFileSync(path.join(sd,'pids.log'),ch.pid+'\n');
|
||||
const ready=path.join(sd,`ready-${ch.pid}`),end=Date.now()+10000;
|
||||
while(!fs.existsSync(ready)){if(Date.now()>end){console.error('fixture holder did not start');process.exit(1)}sleep(20)}
|
||||
s[u]={state:'active',pid:ch.pid,wd:f.wd,enter};save(s);
|
||||
if(f.exec.split(' ').slice(1).includes('--pm'))pm(u,d,id,ch.pid,Number(off));
|
||||
process.exit(0);
|
||||
}
|
||||
if(verb==='stop'){
|
||||
const u=rest[0],s=load(),st=s[u];
|
||||
if(st&&st.pid){
|
||||
try{process.kill(st.pid,'SIGTERM')}catch{}
|
||||
const end=Date.now()+10000;while(alive(st.pid)&&Date.now()<end)sleep(20);
|
||||
if(alive(st.pid))try{process.kill(st.pid,'SIGKILL')}catch{}
|
||||
}
|
||||
if(st){s[u]={...st,state:'inactive',pid:null};save(s)}
|
||||
if(isBus(u)){
|
||||
const d=dataOf(unitFile(u));
|
||||
// One-shot: the stop leaves an empty -wal or -shm beside the store, as an unclean close would.
|
||||
for(const [name,ext] of [['stop-leaves-wal','-wal'],['stop-leaves-shm','-shm']])if(fault(name)){fs.rmSync(faultFile(name));fs.writeFileSync(path.join(d,'bus','bus.sqlite'+ext),'')}
|
||||
// One-shot (T45): the stop writes one row after any STAMP into the table the fault names, as a
|
||||
// write the bus made between rollback's count and its stop would. The close leaves no -wal.
|
||||
if(fault('stop-writes-row')){
|
||||
const t=fs.readFileSync(faultFile('stop-writes-row'),'utf8');fs.rmSync(faultFile('stop-writes-row'));
|
||||
const db=new (process.getBuiltinModule('node:sqlite').DatabaseSync)(path.join(d,'bus','bus.sqlite'));
|
||||
db.exec(`CREATE TABLE IF NOT EXISTS ${t} (seq INTEGER PRIMARY KEY, at TEXT NOT NULL)`);
|
||||
db.prepare(`INSERT INTO ${t} (at) VALUES (?)`).run('2999-01-01T00:00:00.000Z');db.close();
|
||||
}
|
||||
snap('stop',d);
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
console.error(`fixture systemctl: unsupported: ${args.join(' ')}`);process.exit(1);
|
||||
}
|
||||
|
||||
// journalctl --user -u UNIT --since @EPOCH -o cat. A bare local time gets exit 1, as M35 needs.
|
||||
export function journalctl(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path');
|
||||
const sd=process.env.SD_STATE,a=process.argv.slice(2);let unit='',since='';
|
||||
for(let i=0;i<a.length;i++){if(a[i]==='-u')unit=a[++i];else if(a[i]==='--since')since=a[++i]}
|
||||
fs.appendFileSync(path.join(sd,'journal-since.log'),since+'\n');
|
||||
if(fs.existsSync(path.join(sd,'faults','journal-fails')))process.exit(1);
|
||||
let from;
|
||||
if(/^@\d+$/.test(since))from=Number(since.slice(1));
|
||||
else{const m=/^(\d{4}-\d\d-\d\d) (\d\d:\d\d:\d\d) UTC$/.exec(since);if(!m)process.exit(1);from=Date.parse(`${m[1]}T${m[2]}Z`)/1000}
|
||||
let lines=[];try{lines=fs.readFileSync(path.join(sd,'journal.jsonl'),'utf8').split('\n').filter(Boolean).map((l)=>JSON.parse(l))}catch{}
|
||||
for(const o of lines)if(o.unit===unit&&o.epoch>=from)console.log(o.text);
|
||||
}
|
||||
|
||||
export function systemdAnalyze(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path');
|
||||
if(process.argv[2]!=='--user'||process.argv[3]!=='unit-paths')process.exit(1);
|
||||
console.log(path.join(process.env.XDG_CONFIG_HOME,'systemd','user'));
|
||||
// The unit-path fault adds its text as one more path, as T28's relative path needs.
|
||||
const extra=path.join(process.env.SD_STATE,'faults','unit-path');
|
||||
if(fs.existsSync(extra))console.log(fs.readFileSync(extra,'utf8'));
|
||||
}
|
||||
|
||||
// The bus host: holds the store open in WAL mode until SIGTERM, as the broker does. It writes its
|
||||
// status to the host file systemctl names for its data root.
|
||||
export function holder(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),{DatabaseSync}=process.getBuiltinModule('node:sqlite');
|
||||
const [data,business,sd,offset,host]=process.argv.slice(2),bus=path.join(data,'bus');
|
||||
fs.mkdirSync(bus,{recursive:true,mode:0o700});
|
||||
const db=new DatabaseSync(path.join(bus,'bus.sqlite'));
|
||||
if(!db.prepare("SELECT count(*) AS n FROM sqlite_master WHERE name='meta'").get().n){
|
||||
db.exec('PRAGMA journal_mode=WAL');
|
||||
db.exec('CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL)');
|
||||
db.exec('CREATE TABLE events (id INTEGER PRIMARY KEY, at TEXT NOT NULL)');
|
||||
db.prepare("INSERT INTO meta VALUES ('schema_version', '3g')").run();
|
||||
}
|
||||
fs.writeFileSync(path.join(bus,'writer.lock'),String(process.pid));
|
||||
const at=new Date(Date.now()-Number(offset||0)*1000).toISOString();
|
||||
fs.writeFileSync(host+'.tmp',JSON.stringify({host:{business,pid:process.pid,live:true,startedAt:at,notifier:'ok'},boot:{ok:true,business,at}}));
|
||||
fs.renameSync(host+'.tmp',host);
|
||||
process.on('SIGTERM',()=>{db.close();fs.rmSync(path.join(bus,'writer.lock'),{force:true});process.exit(0)});
|
||||
fs.writeFileSync(path.join(sd,`ready-${process.pid}`),'');
|
||||
setInterval(()=>{},1<<30);
|
||||
}
|
||||
|
||||
// A copy's scripts/mosaic: bus status and launches list from what the holder and systemctl wrote
|
||||
// for the data root of the config it runs under (row 190), the default's being $SD_DATA.
|
||||
export function mosaicStub(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path');
|
||||
const id=path.basename(path.resolve(__dirname,'..')),sd=process.env.SD_STATE,a=process.argv.slice(2).join(' ');
|
||||
const fault=(n)=>fs.existsSync(path.join(sd,'faults',n));
|
||||
let d=process.env.SD_DATA;try{d=JSON.parse(fs.readFileSync(process.env.MOSAIC_CONFIG,'utf8')).dataRoot}catch{}
|
||||
const perData=(n)=>path.join(sd,d===process.env.SD_DATA?`${n}.json`:`${n}-${path.basename(d)}.json`);
|
||||
const alive=(pid)=>{try{const s=fs.readFileSync(`/proc/${pid}/stat`,'utf8');return s[s.lastIndexOf(')')+2]!=='Z'}catch{return false}};
|
||||
if(a==='bus status --json'){
|
||||
if(fault(`status-${id}`))process.exit(1);
|
||||
// Row 232: status-until-<copy> fails bus status until the epoch ms it holds.
|
||||
const until=path.join(sd,'faults',`status-until-${id}`);
|
||||
if(fs.existsSync(until)&&Date.now()<Number(fs.readFileSync(until,'utf8')))process.exit(1);
|
||||
let s={host:null,boot:null};
|
||||
try{s=JSON.parse(fs.readFileSync(perData('host'),'utf8'));s.host.live=alive(s.host.pid)}catch{}
|
||||
console.log(JSON.stringify(s));
|
||||
}else if(a==='bus status')console.log(`bus: fixture status for ${id}`);
|
||||
else if(a==='launches list --json'){
|
||||
if(fault(`launches-${id}`))process.exit(1);
|
||||
let l=[];try{l=JSON.parse(fs.readFileSync(perData('launches'),'utf8'))}catch{}
|
||||
console.log(JSON.stringify(l));
|
||||
}else{console.error(`fixture mosaic: ${a}`);process.exit(4)}
|
||||
}
|
||||
|
||||
// A copy's migrate-schema.mjs: records the backups it saw, then sets 3h, or 3h-partial and exit 1.
|
||||
export function migrateStub(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),{DatabaseSync}=process.getBuiltinModule('node:sqlite');
|
||||
const data=process.argv[2],sd=process.env.SD_STATE,dir=path.join(data,'backups','stack');
|
||||
const backups=fs.existsSync(dir)?fs.readdirSync(dir).sort().map((name)=>({name,sums:fs.existsSync(path.join(dir,name,'SHA256SUMS'))})):[];
|
||||
fs.writeFileSync(path.join(sd,'migrate.json'),JSON.stringify({backups}));
|
||||
const fail=fs.existsSync(path.join(sd,'faults','migrate-fails'));
|
||||
const db=new DatabaseSync(path.join(data,'bus','bus.sqlite'));
|
||||
db.prepare("UPDATE meta SET value=? WHERE key='schema_version'").run(fail?'3h-partial':'3h');db.close();
|
||||
if(fail)process.exit(1);
|
||||
console.log('fixture migrate: 3h');
|
||||
}
|
||||
|
||||
// --- The module fixture: one repo, its copies and the stand-ins, shared by every test. ---
|
||||
|
||||
export const base=realpathSync(mkdtempSync(join(tmpdir(),'stack-release-')));
|
||||
after(()=>{execFileSync('chmod',['-R','u+w',base]);rmSync(base,{recursive:true,force:true})});
|
||||
export const repo=join(base,'repo'),COPIES=join(base,'copies'),SBIN=join(base,'sbin');
|
||||
mkdirSync(repo);mkdirSync(COPIES);mkdirSync(SBIN);
|
||||
writeFileSync(join(base,'package.json'),'{"type":"commonjs"}\n');
|
||||
export const tracked=execFileSync('git',['-C',ROOT,'ls-files','-co','--exclude-standard','--','scripts','packages','adapters','roles','RELEASE','package.json','package-lock.json',':!packages/*/tests',':!scripts/foundation'],{encoding:'utf8'}).trim().split('\n');
|
||||
for(const f of tracked)if(existsSync(join(ROOT,f)))cpSync(join(ROOT,f),join(repo,f));
|
||||
export const pkg={name:'copy-fixture',version:'1.0.0'};
|
||||
writeFileSync(join(repo,'package.json'),JSON.stringify(pkg));
|
||||
writeFileSync(join(repo,'package-lock.json'),JSON.stringify({name:pkg.name,version:pkg.version,lockfileVersion:3,packages:{'':pkg}}));
|
||||
git(repo,'init','-q','-b','refactor');git(repo,'config','gc.auto','0');git(repo,'add','.');git(repo,'commit','-qm','fixture');
|
||||
export const commit=(msg)=>{git(repo,'commit','-q','--allow-empty','-m',msg);return git(repo,'rev-parse','HEAD')};
|
||||
// refactor: F A B C1 C, then M merging side branch S (from A). U sits on a branch nothing reaches.
|
||||
export const F=git(repo,'rev-parse','HEAD'),A=commit('row 901: A'),B=commit('row 902: B'),C1=commit('row 903: C1'),C=commit('row 904: C');
|
||||
git(repo,'checkout','-q','-b','side',A);export const S=commit('row 905: S');
|
||||
git(repo,'checkout','-q','-b','unreached',A);export const U=commit('row 906: U');
|
||||
git(repo,'checkout','-q','refactor');git(repo,'merge','-q','--no-ff','-m','merge side','side');
|
||||
export const M=git(repo,'rev-parse','HEAD');git(repo,'update-ref','refs/remotes/origin/refactor',M);
|
||||
// `to` false leaves out the `to:` lines (C1); a string puts that value in each one (A). The --plan
|
||||
// steps line can't be read from either (T27).
|
||||
export const storeText=(v,froms,to=true)=>`export const SCHEMA_VERSION = '${v}';\nexport const MIGRATIONS = [\n${froms.map((f,i)=>` {\n from: '${f}',\n${to===false?'':` to: '${to===true?froms[i+1]??v:to}',\n`} run() {},\n },\n`).join('')}];\n`;
|
||||
// Plug-in kinds (row 169): C1 and C know the harness kind, as a copy at or past c216f47db does. The
|
||||
// rest have it under another name, as ca15d73e9 has no harness kind. Every copy has the real business
|
||||
// loader and roles, which the tool imports to read the business file.
|
||||
export const KINDS_NEW=readFileSync(join(repo,'packages','plugins','src','kinds.mjs'),'utf8');
|
||||
export const KINDS_OLD=KINDS_NEW.replace('\n harness: Object.freeze({','\n "harness-before-c216f47db": Object.freeze({');
|
||||
assert.notEqual(KINDS_OLD,KINDS_NEW);
|
||||
// Row 200 (#1698): C1 and C take --sso and have the row 192 loader and checkConsole; C's checker
|
||||
// refuses RS256, a stand-in for a target whose rules differ from the repository's. B has the loader
|
||||
// from before row 192, which refuses a `console` key. The other copies have no loader.
|
||||
const LOADER=readFileSync(join(repo,'scripts','mosaic-config.mjs'),'utf8'),SSO_CONFIG=readFileSync(join(repo,'packages','webui','src','sso-config.mjs'),'utf8');
|
||||
const LOADER_OLD=LOADER.replace('["configVersion", "environment", "dataRoot", "execution", "console"]','["configVersion", "environment", "dataRoot", "execution"]');
|
||||
const SSO_CONFIG_C=SSO_CONFIG.replace("'ES256', 'RS256',","'ES256',");
|
||||
assert.notEqual(LOADER_OLD,LOADER);assert.notEqual(SSO_CONFIG_C,SSO_CONFIG);
|
||||
const ssoFiles=(id)=>id===B?[['scripts/mosaic-config.mjs',LOADER_OLD]]:[C1,C].includes(id)?[['scripts/mosaic-config.mjs',LOADER],['packages/webui/src/sso-config.mjs',id===C?SSO_CONFIG_C:SSO_CONFIG]]:[];
|
||||
for(const [id,store] of [[F,'// a copy from before SCHEMA_VERSION\n'],[A,storeText('3g',['3f'],'3x')],[B,storeText('3g',['3f'])],[C1,storeText('3g',['3f'],false)],[C,storeText('3h',['3f','3g'])],[S,storeText('3g',['3f'])],[U,storeText('3g',['3f'])]]){
|
||||
const d=join(COPIES,id);
|
||||
for(const [f,text,mode] of [['scripts/mosaic',script(mosaicStub),0o755],['packages/webui/src/cli.mjs',[B,C1,C].includes(id)?`// fixture Console, never run\n// serve accepts '--tester-port'\n// serve accepts '--proof-header' (row 178)\n${[C1,C].includes(id)?"// serve accepts '--sso' (row 192)\n":''}`:'// fixture Console before tester listener, never run\n',0o644],['packages/bus/src/store.mjs',store,0o644],['packages/bus/src/migrate-schema.mjs',`(${migrateStub})();\n`,0o644],['packages/plugins/src/kinds.mjs',[C1,C].includes(id)?KINDS_NEW:KINDS_OLD,0o644],...ssoFiles(id).map(([f,text])=>[f,text,0o644])]){
|
||||
mkdirSync(dirname(join(d,f)),{recursive:true});writeFileSync(join(d,f),text,{mode});
|
||||
}
|
||||
for(const p of ['packages/business/src','roles'])cpSync(join(repo,p),join(d,p),{recursive:true});
|
||||
execFileSync('chmod',['-R','a-w',d]);
|
||||
}
|
||||
for(const [name,fn] of [['systemctl',systemctl],['journalctl',journalctl],['systemd-analyze',systemdAnalyze]])writeFileSync(join(SBIN,name),script(fn),{mode:0o755});
|
||||
writeFileSync(join(base,'holder.cjs'),`(${holder})();\n`);
|
||||
export const SR=join(repo,'scripts','stack-release.sh');
|
||||
|
||||
// --- Per-test roots and helpers ---
|
||||
|
||||
export const sha=(f)=>createHash('sha256').update(readFileSync(f)).digest('hex');
|
||||
export const lines=(f)=>existsSync(f)?readFileSync(f,'utf8').split('\n').filter(Boolean):[];
|
||||
export const calls=(e)=>lines(join(e.sd,'calls.log'));
|
||||
export const startsStops=(e,from)=>calls(e).slice(from).filter((c)=>/^(start|stop) /.test(c));
|
||||
export const logLines=(e)=>lines(e.log).map((l)=>JSON.parse(l));
|
||||
export const pointer=(e)=>existsSync(e.pointer)?JSON.parse(readFileSync(e.pointer,'utf8')):null;
|
||||
export const units=(e)=>JSON.parse(readFileSync(join(e.sd,'units.json'),'utf8'));
|
||||
export const snaps=(e)=>lines(join(e.sd,'snap.jsonl')).map((l)=>JSON.parse(l));
|
||||
export const fault=(e,name,text='')=>writeFileSync(join(e.sd,'faults',name),text);
|
||||
export const sysctl=(e,...a)=>execFileSync(join(SBIN,'systemctl'),['--user',...a],{env:e.env,encoding:'utf8'});
|
||||
export const edit=(f,from,to)=>{const s=readFileSync(f,'utf8');assert.ok(s.includes(from),`${f} has no ${from}`);writeFileSync(f,s.replace(from,to))};
|
||||
export const posts=(e)=>readdirSync(e.data).filter((n)=>n.startsWith('bus.post-attempt-'));
|
||||
export const backupSum=(e,act)=>readFileSync(join(e.data,'backups','stack',act,'SHA256SUMS'),'utf8').split(' ')[0];
|
||||
export const commitLines=(range)=>git(repo,'log','--oneline','--no-merges','--max-count=200',range).split('\n').map((l)=>` ${l}`);
|
||||
// The units, the wrapper, the pointer and the store, by sha256.
|
||||
export const fp=(e)=>({bus:sha(join(e.units,e.tmpl)),console:sha(join(e.units,e.con)),wrapper:sha(join(e.bin,e.wrapper)),pointer:existsSync(e.pointer)?sha(e.pointer):null,store:sha(e.store)});
|
||||
// The schema version: readOnly while the holder has the store open, immutable when it is stopped.
|
||||
export function storeVersion(file,held){
|
||||
const db=held?new DatabaseSync(file,{readOnly:true}):new DatabaseSync(pathToFileURL(file).href+'?immutable=1',{readOnly:true});
|
||||
try{return db.prepare("SELECT value FROM meta WHERE key='schema_version'").get().value}finally{db.close()}
|
||||
}
|
||||
// Every file and directory under p with size and mtime: T23 and T16 compare it before and after.
|
||||
export function tree(p){
|
||||
const out=[];
|
||||
const walk=(q)=>{let st;try{st=statSync(q)}catch{return}out.push(st.isDirectory()?`${q}/`:`${q} ${st.size} ${st.mtimeMs}`);if(st.isDirectory())for(const n of readdirSync(q).sort())walk(join(q,n))};
|
||||
walk(p);return out;
|
||||
}
|
||||
// T17: what a run may never remove. Under --units external confirm removes a set's AWAITING-UNITS
|
||||
// and FAILED when it closes the activation (row 232, r6 A5 and R4); user mode writes neither.
|
||||
const markers=/\/(AWAITING-UNITS|FAILED) /;
|
||||
export const kept=(e)=>[...tree(join(e.data,'backups','stack')),...tree(join(e.state,'stack-rollback')).filter((l)=>!markers.test(l)),...posts(e).flatMap((n)=>tree(join(e.data,n))),...tree(COPIES)].map((l)=>l.replace(/ .*/,''));
|
||||
|
||||
// The business file (row 169): the committed example made loadable, as acme, with `plugins` when given.
|
||||
export const EXAMPLE=readFileSync(join(repo,'packages','business','examples','mosaic-stack.example.json'),'utf8');
|
||||
export function biz(e,plugins){
|
||||
const b=JSON.parse(EXAMPLE);let n=0;
|
||||
const walk=(o)=>{for(const [k,v] of Object.entries(o)){if(k==='botId')o[k]=++n;else if(v==='YYYY-MM-DD')o[k]='2027-01-01';else if(v&&typeof v==='object')walk(v)}};
|
||||
walk(b);b.id='acme';if(plugins!==undefined)b.plugins=plugins;
|
||||
const f=join(dirname(e.cfg),'businesses','acme.json');mkdirSync(dirname(f),{recursive:true});
|
||||
writeFileSync(f,JSON.stringify(b,null,2)+'\n',{mode:0o600});chmodSync(f,0o600);
|
||||
}
|
||||
|
||||
export async function mk(t){
|
||||
const root=realpathSync(mkdtempSync(join(tmpdir(),'sr-')));
|
||||
const e={root,data:join(root,'data'),bin:join(root,'bin'),sd:join(root,'sd'),units:join(root,'xcfg','systemd','user'),cfg:join(root,'cfg','config.json'),claude:join(root,'claude-bin','claude')};
|
||||
// The default instance's names; named() sets NAME's (row 190).
|
||||
Object.assign(e,{flags:[],bus:BUS,tmpl:'[email protected]',con:CON,wrapper:'mosaic'});
|
||||
Object.assign(e,{state:join(e.data,'state'),pointer:join(e.data,'state','stack-active.json'),log:join(e.data,'state','stack-activation-log.jsonl'),store:join(e.data,'bus','bus.sqlite'),lock:join(e.data,'state','stack-release.lock')});
|
||||
for(const d of [join(root,'home'),dirname(e.cfg),join(e.data,'bus'),e.bin,dirname(e.claude),join(e.sd,'faults'),e.units,join(root,'xdata')])mkdirSync(d,{recursive:true});
|
||||
chmodSync(join(e.data,'bus'),0o700);
|
||||
writeFileSync(e.cfg,JSON.stringify({configVersion:1,environment:'development',dataRoot:e.data,execution:{backend:'docker',provider:'zai',model:'glm-5.3-flash',adapter:'mock'}}));
|
||||
biz(e);
|
||||
writeFileSync(join(e.bin,'mosaic'),`#!/usr/bin/env bash\n${MARKER}\n[ "\${1:-}" = __lease-capability ] && exec "${e.bin}/mosaic-old" "$@"\nexec "/old/scripts/mosaic" "$@"\n`,{mode:0o755});
|
||||
writeFileSync(join(e.bin,'mosaic-old'),'#!/bin/sh\nexit 0\n',{mode:0o755});
|
||||
writeFileSync(e.claude,'#!/bin/sh\necho "9.9.9 (Claude Code fixture)"\n',{mode:0o755});
|
||||
e.env={PATH:`${SBIN}:${dirname(NODE)}:/usr/bin:/bin`,HOME:join(root,'home'),XDG_CONFIG_HOME:join(root,'xcfg'),XDG_DATA_HOME:join(root,'xdata'),MOSAIC_CONFIG:e.cfg,SD_STATE:e.sd,SD_DATA:e.data,TMPDIR:tmpdir()};
|
||||
const server=await healthz(e);
|
||||
t.after(()=>{
|
||||
for(const pid of lines(join(e.sd,'pids.log')).map(Number)){
|
||||
try{if(readFileSync(`/proc/${pid}/cmdline`,'utf8').includes(join(base,'holder.cjs')))process.kill(pid,'SIGKILL')}catch{}
|
||||
}
|
||||
for(const s of [server,...(e.servers??[])])s.close();
|
||||
execFileSync('chmod',['-R','u+w',root]);rmSync(root,{recursive:true,force:true});
|
||||
});
|
||||
return e;
|
||||
}
|
||||
|
||||
// The Console's /healthz on e.port: 200 while the stand-in has e.con active and no healthz-<copy>
|
||||
// fault names its copy.
|
||||
export async function healthz(e){
|
||||
const server=createServer((q,r)=>{
|
||||
let st={};try{st=units(e)[e.con]||{}}catch{}
|
||||
const ok=q.url==='/healthz'&&st.state==='active'&&!existsSync(join(e.sd,'faults',`healthz-${basename(st.wd||'')}`));
|
||||
r.writeHead(ok?200:500);r.end();
|
||||
});
|
||||
server.listen(0,'127.0.0.1');await once(server,'listening');e.port=String(server.address().port);
|
||||
return server;
|
||||
}
|
||||
|
||||
// Row 190: stack instance NAME on e's host. It shares e's stand-ins, unit directory and $BIN, and has
|
||||
// its own config and data root from bootstrap --instance, its own Console port and business acme.
|
||||
// Its runs pass --instance NAME and no MOSAIC_CONFIG.
|
||||
export async function named(e,name){
|
||||
const env={...e.env};delete env.MOSAIC_CONFIG;
|
||||
execFileSync('bash',[join(repo,'scripts','bootstrap.sh'),'--instance',name],{env,stdio:'ignore'});
|
||||
const data=join(env.HOME,`.mosaic-${name}`);
|
||||
const n={...e,env,data,cfg:join(env.HOME,'.config',`mosaic-${name}`,'config.json'),flags:['--instance',name],bus:`mosaic-bus-${name}@acme.service`,tmpl:`mosaic-bus-${name}@.service`,con:`mosaic-webui-${name}.service`,wrapper:`mosaic-${name}`,name};
|
||||
Object.assign(n,{state:join(data,'state'),pointer:join(data,'state','stack-active.json'),log:join(data,'state','stack-activation-log.jsonl'),store:join(data,'bus','bus.sqlite'),lock:join(data,'state','stack-release.lock')});
|
||||
mkdirSync(join(data,'bus'),{mode:0o700});
|
||||
biz(n);
|
||||
(e.servers??=[]).push(await healthz(n));
|
||||
return n;
|
||||
}
|
||||
|
||||
export async function run(e,...args){
|
||||
const before=existsSync(e.data)?kept(e):[];
|
||||
const ch=spawn('bash',[e.sr??SR,...args,...e.flags,'--copies',COPIES,'--bin',e.bin],{env:e.env,stdio:['ignore','pipe','pipe']});
|
||||
let out='',err='';ch.stdout.on('data',(d)=>{out+=d});ch.stderr.on('data',(d)=>{err+=d});
|
||||
const timer=setTimeout(()=>ch.kill('SIGKILL'),120000);
|
||||
const [code]=await once(ch,'close');clearTimeout(timer);
|
||||
const now=new Set(kept(e));
|
||||
for(const p of before)assert.ok(now.has(p),`T17: ${args.join(' ')} removed ${p}`);
|
||||
return {code,out,err};
|
||||
}
|
||||
|
||||
// The stack on copy `copy` for business acme, both units active. pointer: adopt (a real adopt and
|
||||
// its log line), hand (written here, no log) or none. Snapshots start after the seed.
|
||||
export async function seed(e,{copy=B,pm=false,store='3g',pointer:how='adopt'}={}){
|
||||
const db=new DatabaseSync(e.store);
|
||||
db.exec('PRAGMA journal_mode=WAL');
|
||||
db.exec('CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL)');
|
||||
db.exec('CREATE TABLE events (id INTEGER PRIMARY KEY, at TEXT NOT NULL)');
|
||||
db.prepare("INSERT INTO meta VALUES ('schema_version', ?)").run(store);
|
||||
for(let i=0;i<3;i++)db.prepare('INSERT INTO events (at) VALUES (?)').run(`2000-01-01T00:00:0${i}.000Z`);
|
||||
db.close();
|
||||
const sh=(name,...a)=>execFileSync('bash',[join(repo,'scripts',name),...a],{env:e.env,encoding:'utf8',stdio:['ignore','pipe','pipe']});
|
||||
sh('bus-service.sh','install',...e.flags,'--copy',copy,'--copies',COPIES,...(pm?['--pm']:[]),'--claude',e.claude);
|
||||
sh('webui-service.sh','install',...e.flags,'--copy',copy,'--copies',COPIES,'--port',e.port,'--business','acme');
|
||||
sh('stack-copy.sh','wrapper',copy,...e.flags,'--copies',COPIES,'--bin',e.bin);
|
||||
// The seeded host started two minutes ago, so its boot status and PM run predate any run's start.
|
||||
execFileSync(join(SBIN,'systemctl'),['--user','start',e.bus],{env:{...e.env,SD_CLOCK_OFFSET:'120'}});
|
||||
sysctl(e,'start',e.con);
|
||||
if(how==='adopt'){const r=await run(e,'adopt','acme');assert.equal(r.code,0,r.err)}
|
||||
else if(how==='hand'){
|
||||
mkdirSync(e.state,{recursive:true});
|
||||
writeFileSync(e.pointer,JSON.stringify({copy,business:'acme',pm,claude:e.claude,console:['--port',e.port,'--business','acme'],schema:'3g',activation:'manual',at:'2026-10-18T00:00:00Z'})+'\n');
|
||||
}
|
||||
writeFileSync(join(e.sd,'snap.jsonl'),'');
|
||||
}
|
||||
|
||||
export const rolledBack=(code,to=B)=>`stack-release: ${code}; rolled back to ${to}, which passed the gate\n`;
|
||||
export const planBackup=(e)=>`backup: the run backs up the stopped store to ${e.data}/backups/stack/<activation>/ before any migration`;
|
||||
export const planNoBackup='backup: none; the run rolls back before the backup';
|
||||
export const planTail=(e,to,pm=false)=>[`bus: scripts/bus-service.sh install --copy ${to} --copies ${COPIES}${pm?' --pm':''} --claude ${e.claude}`,'proof: off','send: off',`Console: scripts/webui-service.sh install --copy ${to} --copies ${COPIES} --port ${e.port} --business acme`,`CLI: scripts/stack-copy.sh wrapper ${to} --copies ${COPIES} --bin ${e.bin}`];
|
||||
@@ -13,422 +13,7 @@ import {join,dirname,basename} from 'node:path';
|
||||
import {fileURLToPath,pathToFileURL} from 'node:url';
|
||||
import {once} from 'node:events';
|
||||
import {ALGS} from '../../webui/src/sso-config.mjs';
|
||||
|
||||
const ROOT=fileURLToPath(new URL('../../..',import.meta.url)).replace(/\/$/,'');
|
||||
const NODE=process.execPath;
|
||||
const MARKER='# mosaic-cli wrapper (Q81, agents/sage/work/q81/mosaic-cli.sh)';
|
||||
const BUS='[email protected]',CON='mosaic-webui.service';
|
||||
const git=(cwd,...args)=>execFileSync('git',['-c','user.name=fixture','-c','user.email=fixture@invalid','-c','commit.gpgsign=false','-c','core.hooksPath=/dev/null','-C',cwd,...args],{encoding:'utf8'}).trim();
|
||||
const script=(fn)=>`#!${NODE}\n(${fn})();\n`;
|
||||
|
||||
// --- Stand-ins. Each is serialized into its own file, so it uses only process and its own scope. ---
|
||||
|
||||
// systemctl --user: unit state in $SD_STATE/units.json, every call in calls.log. A bus start runs
|
||||
// holder.cjs (the store held open in WAL mode, writer.lock, host.json) and, with --pm in the unit's
|
||||
// ExecStart, writes the `launched` journal line and the launches entry unless a pm-<copy> fault
|
||||
// names another case. Faults are files in $SD_STATE/faults named for the copy they hit. Units of a
|
||||
// named instance (row 190) run on their own data root and host files beside the default's.
|
||||
function systemctl(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),cp=process.getBuiltinModule('node:child_process'),crypto=process.getBuiltinModule('node:crypto');
|
||||
const sd=process.env.SD_STATE,data=process.env.SD_DATA,unitDir=path.join(process.env.XDG_CONFIG_HOME,'systemd','user');
|
||||
const args=process.argv.slice(2).filter((a)=>a!=='--user'),[verb,...rest]=args;
|
||||
fs.appendFileSync(path.join(sd,'calls.log'),args.join(' ')+'\n');
|
||||
const faultFile=(n)=>path.join(sd,'faults',n),fault=(n)=>fs.existsSync(faultFile(n));
|
||||
const stFile=path.join(sd,'units.json');
|
||||
const load=()=>{try{return JSON.parse(fs.readFileSync(stFile,'utf8'))}catch{return {}}};
|
||||
const save=(s)=>{fs.writeFileSync(stFile+'.tmp',JSON.stringify(s));fs.renameSync(stFile+'.tmp',stFile)};
|
||||
const sleep=(ms)=>Atomics.wait(new Int32Array(new SharedArrayBuffer(4)),0,0,ms);
|
||||
const alive=(pid)=>{try{const s=fs.readFileSync(`/proc/${pid}/stat`,'utf8');return s[s.lastIndexOf(')')+2]!=='Z'}catch{return false}};
|
||||
// Row 190: a bus unit is the default's mosaic-bus@B or instance NAME's mosaic-bus-NAME@B; %i is B.
|
||||
const isBus=(u)=>/^mosaic-bus(-[a-z][a-z0-9]{1,15})?@[a-z0-9-]+\.service$/.test(u),inst=(u)=>u.slice(u.indexOf('@')+1,-'.service'.length);
|
||||
const unitFile=(u)=>{
|
||||
const main=isBus(u)?u.replace(/@[^@]*$/,'@.service'):u,file=path.join(unitDir,main);
|
||||
if(!fs.existsSync(file))return null;
|
||||
// The drop-in directories in systemd's order: service.d, the name cut after each hyphen, then
|
||||
// the template's and the unit's own.
|
||||
const stem=main.slice(0,main.search(/@?\.service$/)),dirs=['service.d'],drops=[];
|
||||
for(let i=0;i<stem.length;i++)if(stem[i]==='-')dirs.push(stem.slice(0,i+1)+'.service.d');
|
||||
for(const d of [...dirs,...(isBus(u)?[main+'.d',u+'.d']:[u+'.d'])]){const dir=path.join(unitDir,d);if(fs.existsSync(dir))for(const f of fs.readdirSync(dir).sort())if(f.endsWith('.conf'))drops.push(path.join(dir,f))}
|
||||
const r={wd:'',exec:'',env:[],drops};
|
||||
for(const f of [file,...drops])for(let line of fs.readFileSync(f,'utf8').split('\n')){
|
||||
line=line.replace(/%i/g,isBus(u)?inst(u):'').replace(/%h/g,process.env.HOME);
|
||||
const m=/^(WorkingDirectory|ExecStart|Environment)=(.*)$/.exec(line);if(!m)continue;
|
||||
if(m[1]==='WorkingDirectory')r.wd=m[2];else if(m[1]==='ExecStart'){if(m[2])r.exec=m[2]}else r.env.push(m[2]);
|
||||
}
|
||||
return r;
|
||||
};
|
||||
// A named instance's unit sets MOSAIC_CONFIG, and its bus works on that config's data root. A unit
|
||||
// without it is the default's, on $SD_DATA. Each data root has its own host and launches files.
|
||||
const dataOf=(f)=>{const c=f&&f.env.find((v)=>v.startsWith('MOSAIC_CONFIG='));return c?JSON.parse(fs.readFileSync(c.slice('MOSAIC_CONFIG='.length),'utf8')).dataRoot:data};
|
||||
const perData=(d,n)=>path.join(sd,d===data?`${n}.json`:`${n}-${path.basename(d)}.json`);
|
||||
const snap=(ev,d)=>{
|
||||
const dir=path.join(d,'bus'),files={};
|
||||
try{for(const n of fs.readdirSync(dir).sort()){const p=path.join(dir,n),st=fs.statSync(p);if(st.isFile())files[n]=[st.size,st.mtimeMs,crypto.createHash('sha256').update(fs.readFileSync(p)).digest('hex')]}}catch{}
|
||||
fs.appendFileSync(path.join(sd,'snap.jsonl'),JSON.stringify({ev,files})+'\n');
|
||||
};
|
||||
const pm=(unit,d,id,pid,off)=>{
|
||||
let mode='';try{mode=fs.readFileSync(faultFile(`pm-${id}`),'utf8').trim()}catch{}
|
||||
const b=inst(unit),now=Date.now()-off*1000,epoch=Math.floor(now/1000),at=new Date(now).toISOString(),run=`run-${pid}`;
|
||||
const line=(text)=>fs.appendFileSync(path.join(sd,'journal.jsonl'),JSON.stringify({epoch,unit,text})+'\n');
|
||||
const ok=()=>line(`launched pm-${b}: run ${run}, claude opus`);
|
||||
const lf=perData(d,'launches');let list=[];try{list=JSON.parse(fs.readFileSync(lf,'utf8'))}catch{}
|
||||
const entry=(o)=>list.push({run,business:b,live:true,startedAt:at,...o});
|
||||
line(`bus host up: business ${b}, socket /fixture/bus.sock, notifier off`);
|
||||
switch(mode){
|
||||
case 'none':break;
|
||||
case 'two':ok();line(`launched pm-${b}: run ${run}b, claude opus`);entry({});break;
|
||||
case 'badword':line(`launched pm-${b}: job ${run}, claude opus`);entry({});break;
|
||||
case 'notlisted':ok();break;
|
||||
case 'early':ok();entry({startedAt:new Date(now-3600e3).toISOString()});break;
|
||||
case 'human':entry({run:`human-${pid}`});break;
|
||||
case 'other':ok();entry({business:'other'});break;
|
||||
case 'dead':ok();entry({live:false});break;
|
||||
default:ok();entry({});
|
||||
}
|
||||
fs.writeFileSync(lf,JSON.stringify(list));
|
||||
};
|
||||
if(verb==='show'){
|
||||
const u=rest[0],props=[];for(let i=1;i<rest.length;i++)if(rest[i]==='-p')props.push(rest[++i]);
|
||||
const f=unitFile(u),st=load()[u]||{},id=f?path.basename(f.wd):'';
|
||||
for(const p of props){
|
||||
if(p==='LoadState')console.log(`LoadState=${f?'loaded':'not-found'}`);
|
||||
else if(p==='WorkingDirectory')console.log(`WorkingDirectory=${f?f.wd+(!isBus(u)&&fault(`check-${id}`)?'-moved':''):''}`);
|
||||
else if(p==='ExecStart')console.log(f&&f.exec?`ExecStart={ path=${f.exec.split(' ')[0]} ; argv[]=${f.exec} ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }`:'ExecStart=');
|
||||
else if(p==='Environment')console.log(`Environment=${f?f.env.join(' '):''}`);
|
||||
else if(p==='MainPID')console.log(`MainPID=${st.pid||0}`);
|
||||
else if(p==='DropInPaths')console.log(`DropInPaths=${f?f.drops.join(' '):''}`);
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
if(verb==='is-active'){
|
||||
const q=rest[0]==='--quiet',u=q?rest[1]:rest[0];let s=(load()[u]||{}).state||'inactive';
|
||||
// bus-stops-after N (T29): the next N bus checks answer the real state, every later one answers
|
||||
// inactive, and units.json stays as it is, as if the bus stopped between two checks.
|
||||
if(isBus(u)&&fault('bus-stops-after')){const n=Number(fs.readFileSync(faultFile('bus-stops-after'),'utf8'));if(n>0)fs.writeFileSync(faultFile('bus-stops-after'),String(n-1));else s='inactive'}
|
||||
if(!q)console.log(s);process.exit(s==='active'?0:3);
|
||||
}
|
||||
if(verb==='is-enabled'){console.log('disabled');process.exit(1)}
|
||||
if(verb==='daemon-reload')process.exit(0);
|
||||
if(verb==='list-units'){
|
||||
if(fault('list-fails'))process.exit(1);
|
||||
const pats=rest.filter((a)=>!a.startsWith('--')).map((p)=>new RegExp('^'+p.replace(/\./g,'\\.').replace(/\*/g,'.*')+'$'));
|
||||
for(const [u,s] of Object.entries(load()))if(pats.some((re)=>re.test(u)))console.log(`${u} loaded ${s.state} ${s.state==='active'?'running':'dead'} fixture`);
|
||||
process.exit(0);
|
||||
}
|
||||
if(verb==='start'){
|
||||
const u=rest[0],f=unitFile(u);if(!f){console.error(`Unit ${u} not found.`);process.exit(5)}
|
||||
const s=load(),id=path.basename(f.wd),d=dataOf(f);
|
||||
if(!isBus(u)){s[u]={state:'active',pid:null,wd:f.wd};save(s);process.exit(0)}
|
||||
if(fault(`start-dies-${id}`)){s[u]={state:'failed',pid:null,wd:f.wd};save(s);process.exit(0)}
|
||||
// Row 185 (T48): the copy's start writes one event before the holder opens the store, so the
|
||||
// store differs from a backup the run took before the start. The close leaves no -wal.
|
||||
if(fault(`start-writes-row-${id}`)){
|
||||
const db=new (process.getBuiltinModule('node:sqlite').DatabaseSync)(path.join(d,'bus','bus.sqlite'));
|
||||
db.prepare('INSERT INTO events (at) VALUES (?)').run('2999-01-01T00:00:00.000Z');db.close();
|
||||
}
|
||||
snap('start',d);
|
||||
const off=process.env.SD_CLOCK_OFFSET||'0';
|
||||
// The holder must not inherit stack-release.sh's lock (fd 9) or any other fd above 2.
|
||||
const close='for fd in /proc/self/fd/*; do n=${fd##*/}; [ "$n" -gt 2 ] && eval "exec $n>&-" 2>/dev/null; done; exec "$@"';
|
||||
const ch=cp.spawn('bash',['-c',close,'holder',process.execPath,path.join(__dirname,'..','holder.cjs'),d,inst(u),sd,off,perData(d,'host')],{detached:true,stdio:'ignore'});
|
||||
ch.unref();
|
||||
fs.appendFileSync(path.join(sd,'pids.log'),ch.pid+'\n');
|
||||
const ready=path.join(sd,`ready-${ch.pid}`),end=Date.now()+10000;
|
||||
while(!fs.existsSync(ready)){if(Date.now()>end){console.error('fixture holder did not start');process.exit(1)}sleep(20)}
|
||||
s[u]={state:'active',pid:ch.pid,wd:f.wd};save(s);
|
||||
if(f.exec.split(' ').slice(1).includes('--pm'))pm(u,d,id,ch.pid,Number(off));
|
||||
process.exit(0);
|
||||
}
|
||||
if(verb==='stop'){
|
||||
const u=rest[0],s=load(),st=s[u];
|
||||
if(st&&st.pid){
|
||||
try{process.kill(st.pid,'SIGTERM')}catch{}
|
||||
const end=Date.now()+10000;while(alive(st.pid)&&Date.now()<end)sleep(20);
|
||||
if(alive(st.pid))try{process.kill(st.pid,'SIGKILL')}catch{}
|
||||
}
|
||||
if(st){s[u]={...st,state:'inactive',pid:null};save(s)}
|
||||
if(isBus(u)){
|
||||
const d=dataOf(unitFile(u));
|
||||
// One-shot: the stop leaves an empty -wal or -shm beside the store, as an unclean close would.
|
||||
for(const [name,ext] of [['stop-leaves-wal','-wal'],['stop-leaves-shm','-shm']])if(fault(name)){fs.rmSync(faultFile(name));fs.writeFileSync(path.join(d,'bus','bus.sqlite'+ext),'')}
|
||||
// One-shot (T45): the stop writes one row after any STAMP into the table the fault names, as a
|
||||
// write the bus made between rollback's count and its stop would. The close leaves no -wal.
|
||||
if(fault('stop-writes-row')){
|
||||
const t=fs.readFileSync(faultFile('stop-writes-row'),'utf8');fs.rmSync(faultFile('stop-writes-row'));
|
||||
const db=new (process.getBuiltinModule('node:sqlite').DatabaseSync)(path.join(d,'bus','bus.sqlite'));
|
||||
db.exec(`CREATE TABLE IF NOT EXISTS ${t} (seq INTEGER PRIMARY KEY, at TEXT NOT NULL)`);
|
||||
db.prepare(`INSERT INTO ${t} (at) VALUES (?)`).run('2999-01-01T00:00:00.000Z');db.close();
|
||||
}
|
||||
snap('stop',d);
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
console.error(`fixture systemctl: unsupported: ${args.join(' ')}`);process.exit(1);
|
||||
}
|
||||
|
||||
// journalctl --user -u UNIT --since @EPOCH -o cat. A bare local time gets exit 1, as M35 needs.
|
||||
function journalctl(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path');
|
||||
const sd=process.env.SD_STATE,a=process.argv.slice(2);let unit='',since='';
|
||||
for(let i=0;i<a.length;i++){if(a[i]==='-u')unit=a[++i];else if(a[i]==='--since')since=a[++i]}
|
||||
fs.appendFileSync(path.join(sd,'journal-since.log'),since+'\n');
|
||||
if(fs.existsSync(path.join(sd,'faults','journal-fails')))process.exit(1);
|
||||
let from;
|
||||
if(/^@\d+$/.test(since))from=Number(since.slice(1));
|
||||
else{const m=/^(\d{4}-\d\d-\d\d) (\d\d:\d\d:\d\d) UTC$/.exec(since);if(!m)process.exit(1);from=Date.parse(`${m[1]}T${m[2]}Z`)/1000}
|
||||
let lines=[];try{lines=fs.readFileSync(path.join(sd,'journal.jsonl'),'utf8').split('\n').filter(Boolean).map((l)=>JSON.parse(l))}catch{}
|
||||
for(const o of lines)if(o.unit===unit&&o.epoch>=from)console.log(o.text);
|
||||
}
|
||||
|
||||
function systemdAnalyze(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path');
|
||||
if(process.argv[2]!=='--user'||process.argv[3]!=='unit-paths')process.exit(1);
|
||||
console.log(path.join(process.env.XDG_CONFIG_HOME,'systemd','user'));
|
||||
// The unit-path fault adds its text as one more path, as T28's relative path needs.
|
||||
const extra=path.join(process.env.SD_STATE,'faults','unit-path');
|
||||
if(fs.existsSync(extra))console.log(fs.readFileSync(extra,'utf8'));
|
||||
}
|
||||
|
||||
// The bus host: holds the store open in WAL mode until SIGTERM, as the broker does. It writes its
|
||||
// status to the host file systemctl names for its data root.
|
||||
function holder(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),{DatabaseSync}=process.getBuiltinModule('node:sqlite');
|
||||
const [data,business,sd,offset,host]=process.argv.slice(2),bus=path.join(data,'bus');
|
||||
fs.mkdirSync(bus,{recursive:true,mode:0o700});
|
||||
const db=new DatabaseSync(path.join(bus,'bus.sqlite'));
|
||||
if(!db.prepare("SELECT count(*) AS n FROM sqlite_master WHERE name='meta'").get().n){
|
||||
db.exec('PRAGMA journal_mode=WAL');
|
||||
db.exec('CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL)');
|
||||
db.exec('CREATE TABLE events (id INTEGER PRIMARY KEY, at TEXT NOT NULL)');
|
||||
db.prepare("INSERT INTO meta VALUES ('schema_version', '3g')").run();
|
||||
}
|
||||
fs.writeFileSync(path.join(bus,'writer.lock'),String(process.pid));
|
||||
const at=new Date(Date.now()-Number(offset||0)*1000).toISOString();
|
||||
fs.writeFileSync(host+'.tmp',JSON.stringify({host:{business,pid:process.pid,live:true,startedAt:at,notifier:'ok'},boot:{ok:true,business,at}}));
|
||||
fs.renameSync(host+'.tmp',host);
|
||||
process.on('SIGTERM',()=>{db.close();fs.rmSync(path.join(bus,'writer.lock'),{force:true});process.exit(0)});
|
||||
fs.writeFileSync(path.join(sd,`ready-${process.pid}`),'');
|
||||
setInterval(()=>{},1<<30);
|
||||
}
|
||||
|
||||
// A copy's scripts/mosaic: bus status and launches list from what the holder and systemctl wrote
|
||||
// for the data root of the config it runs under (row 190), the default's being $SD_DATA.
|
||||
function mosaicStub(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path');
|
||||
const id=path.basename(path.resolve(__dirname,'..')),sd=process.env.SD_STATE,a=process.argv.slice(2).join(' ');
|
||||
const fault=(n)=>fs.existsSync(path.join(sd,'faults',n));
|
||||
let d=process.env.SD_DATA;try{d=JSON.parse(fs.readFileSync(process.env.MOSAIC_CONFIG,'utf8')).dataRoot}catch{}
|
||||
const perData=(n)=>path.join(sd,d===process.env.SD_DATA?`${n}.json`:`${n}-${path.basename(d)}.json`);
|
||||
const alive=(pid)=>{try{const s=fs.readFileSync(`/proc/${pid}/stat`,'utf8');return s[s.lastIndexOf(')')+2]!=='Z'}catch{return false}};
|
||||
if(a==='bus status --json'){
|
||||
if(fault(`status-${id}`))process.exit(1);
|
||||
let s={host:null,boot:null};
|
||||
try{s=JSON.parse(fs.readFileSync(perData('host'),'utf8'));s.host.live=alive(s.host.pid)}catch{}
|
||||
console.log(JSON.stringify(s));
|
||||
}else if(a==='bus status')console.log(`bus: fixture status for ${id}`);
|
||||
else if(a==='launches list --json'){
|
||||
if(fault(`launches-${id}`))process.exit(1);
|
||||
let l=[];try{l=JSON.parse(fs.readFileSync(perData('launches'),'utf8'))}catch{}
|
||||
console.log(JSON.stringify(l));
|
||||
}else{console.error(`fixture mosaic: ${a}`);process.exit(4)}
|
||||
}
|
||||
|
||||
// A copy's migrate-schema.mjs: records the backups it saw, then sets 3h, or 3h-partial and exit 1.
|
||||
function migrateStub(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),{DatabaseSync}=process.getBuiltinModule('node:sqlite');
|
||||
const data=process.argv[2],sd=process.env.SD_STATE,dir=path.join(data,'backups','stack');
|
||||
const backups=fs.existsSync(dir)?fs.readdirSync(dir).sort().map((name)=>({name,sums:fs.existsSync(path.join(dir,name,'SHA256SUMS'))})):[];
|
||||
fs.writeFileSync(path.join(sd,'migrate.json'),JSON.stringify({backups}));
|
||||
const fail=fs.existsSync(path.join(sd,'faults','migrate-fails'));
|
||||
const db=new DatabaseSync(path.join(data,'bus','bus.sqlite'));
|
||||
db.prepare("UPDATE meta SET value=? WHERE key='schema_version'").run(fail?'3h-partial':'3h');db.close();
|
||||
if(fail)process.exit(1);
|
||||
console.log('fixture migrate: 3h');
|
||||
}
|
||||
|
||||
// --- The module fixture: one repo, its copies and the stand-ins, shared by every test. ---
|
||||
|
||||
const base=realpathSync(mkdtempSync(join(tmpdir(),'stack-release-')));
|
||||
after(()=>{execFileSync('chmod',['-R','u+w',base]);rmSync(base,{recursive:true,force:true})});
|
||||
const repo=join(base,'repo'),COPIES=join(base,'copies'),SBIN=join(base,'sbin');
|
||||
mkdirSync(repo);mkdirSync(COPIES);mkdirSync(SBIN);
|
||||
writeFileSync(join(base,'package.json'),'{"type":"commonjs"}\n');
|
||||
const tracked=execFileSync('git',['-C',ROOT,'ls-files','-co','--exclude-standard','--','scripts','packages','adapters','roles','RELEASE','package.json','package-lock.json',':!packages/*/tests',':!scripts/foundation'],{encoding:'utf8'}).trim().split('\n');
|
||||
for(const f of tracked)if(existsSync(join(ROOT,f)))cpSync(join(ROOT,f),join(repo,f));
|
||||
const pkg={name:'copy-fixture',version:'1.0.0'};
|
||||
writeFileSync(join(repo,'package.json'),JSON.stringify(pkg));
|
||||
writeFileSync(join(repo,'package-lock.json'),JSON.stringify({name:pkg.name,version:pkg.version,lockfileVersion:3,packages:{'':pkg}}));
|
||||
git(repo,'init','-q','-b','refactor');git(repo,'config','gc.auto','0');git(repo,'add','.');git(repo,'commit','-qm','fixture');
|
||||
const commit=(msg)=>{git(repo,'commit','-q','--allow-empty','-m',msg);return git(repo,'rev-parse','HEAD')};
|
||||
// refactor: F A B C1 C, then M merging side branch S (from A). U sits on a branch nothing reaches.
|
||||
const F=git(repo,'rev-parse','HEAD'),A=commit('row 901: A'),B=commit('row 902: B'),C1=commit('row 903: C1'),C=commit('row 904: C');
|
||||
git(repo,'checkout','-q','-b','side',A);const S=commit('row 905: S');
|
||||
git(repo,'checkout','-q','-b','unreached',A);const U=commit('row 906: U');
|
||||
git(repo,'checkout','-q','refactor');git(repo,'merge','-q','--no-ff','-m','merge side','side');
|
||||
const M=git(repo,'rev-parse','HEAD');git(repo,'update-ref','refs/remotes/origin/refactor',M);
|
||||
// `to` false leaves out the `to:` lines (C1); a string puts that value in each one (A). The --plan
|
||||
// steps line can't be read from either (T27).
|
||||
const storeText=(v,froms,to=true)=>`export const SCHEMA_VERSION = '${v}';\nexport const MIGRATIONS = [\n${froms.map((f,i)=>` {\n from: '${f}',\n${to===false?'':` to: '${to===true?froms[i+1]??v:to}',\n`} run() {},\n },\n`).join('')}];\n`;
|
||||
// Plug-in kinds (row 169): C1 and C know the harness kind, as a copy at or past c216f47db does. The
|
||||
// rest have it under another name, as ca15d73e9 has no harness kind. Every copy has the real business
|
||||
// loader and roles, which the tool imports to read the business file.
|
||||
const KINDS_NEW=readFileSync(join(repo,'packages','plugins','src','kinds.mjs'),'utf8');
|
||||
const KINDS_OLD=KINDS_NEW.replace('\n harness: Object.freeze({','\n "harness-before-c216f47db": Object.freeze({');
|
||||
assert.notEqual(KINDS_OLD,KINDS_NEW);
|
||||
// Row 200 (#1698): C1 and C take --sso and have the row 192 loader and checkConsole; C's checker
|
||||
// refuses RS256, a stand-in for a target whose rules differ from the repository's. B has the loader
|
||||
// from before row 192, which refuses a `console` key. The other copies have no loader.
|
||||
const LOADER=readFileSync(join(repo,'scripts','mosaic-config.mjs'),'utf8'),SSO_CONFIG=readFileSync(join(repo,'packages','webui','src','sso-config.mjs'),'utf8');
|
||||
const LOADER_OLD=LOADER.replace('["configVersion", "environment", "dataRoot", "execution", "console"]','["configVersion", "environment", "dataRoot", "execution"]');
|
||||
const SSO_CONFIG_C=SSO_CONFIG.replace("'ES256', 'RS256',","'ES256',");
|
||||
assert.notEqual(LOADER_OLD,LOADER);assert.notEqual(SSO_CONFIG_C,SSO_CONFIG);
|
||||
const ssoFiles=(id)=>id===B?[['scripts/mosaic-config.mjs',LOADER_OLD]]:[C1,C].includes(id)?[['scripts/mosaic-config.mjs',LOADER],['packages/webui/src/sso-config.mjs',id===C?SSO_CONFIG_C:SSO_CONFIG]]:[];
|
||||
for(const [id,store] of [[F,'// a copy from before SCHEMA_VERSION\n'],[A,storeText('3g',['3f'],'3x')],[B,storeText('3g',['3f'])],[C1,storeText('3g',['3f'],false)],[C,storeText('3h',['3f','3g'])],[S,storeText('3g',['3f'])],[U,storeText('3g',['3f'])]]){
|
||||
const d=join(COPIES,id);
|
||||
for(const [f,text,mode] of [['scripts/mosaic',script(mosaicStub),0o755],['packages/webui/src/cli.mjs',[B,C1,C].includes(id)?`// fixture Console, never run\n// serve accepts '--tester-port'\n// serve accepts '--proof-header' (row 178)\n${[C1,C].includes(id)?"// serve accepts '--sso' (row 192)\n":''}`:'// fixture Console before tester listener, never run\n',0o644],['packages/bus/src/store.mjs',store,0o644],['packages/bus/src/migrate-schema.mjs',`(${migrateStub})();\n`,0o644],['packages/plugins/src/kinds.mjs',[C1,C].includes(id)?KINDS_NEW:KINDS_OLD,0o644],...ssoFiles(id).map(([f,text])=>[f,text,0o644])]){
|
||||
mkdirSync(dirname(join(d,f)),{recursive:true});writeFileSync(join(d,f),text,{mode});
|
||||
}
|
||||
for(const p of ['packages/business/src','roles'])cpSync(join(repo,p),join(d,p),{recursive:true});
|
||||
execFileSync('chmod',['-R','a-w',d]);
|
||||
}
|
||||
for(const [name,fn] of [['systemctl',systemctl],['journalctl',journalctl],['systemd-analyze',systemdAnalyze]])writeFileSync(join(SBIN,name),script(fn),{mode:0o755});
|
||||
writeFileSync(join(base,'holder.cjs'),`(${holder})();\n`);
|
||||
const SR=join(repo,'scripts','stack-release.sh');
|
||||
|
||||
// --- Per-test roots and helpers ---
|
||||
|
||||
const sha=(f)=>createHash('sha256').update(readFileSync(f)).digest('hex');
|
||||
const lines=(f)=>existsSync(f)?readFileSync(f,'utf8').split('\n').filter(Boolean):[];
|
||||
const calls=(e)=>lines(join(e.sd,'calls.log'));
|
||||
const startsStops=(e,from)=>calls(e).slice(from).filter((c)=>/^(start|stop) /.test(c));
|
||||
const logLines=(e)=>lines(e.log).map((l)=>JSON.parse(l));
|
||||
const pointer=(e)=>existsSync(e.pointer)?JSON.parse(readFileSync(e.pointer,'utf8')):null;
|
||||
const units=(e)=>JSON.parse(readFileSync(join(e.sd,'units.json'),'utf8'));
|
||||
const snaps=(e)=>lines(join(e.sd,'snap.jsonl')).map((l)=>JSON.parse(l));
|
||||
const fault=(e,name,text='')=>writeFileSync(join(e.sd,'faults',name),text);
|
||||
const sysctl=(e,...a)=>execFileSync(join(SBIN,'systemctl'),['--user',...a],{env:e.env,encoding:'utf8'});
|
||||
const edit=(f,from,to)=>{const s=readFileSync(f,'utf8');assert.ok(s.includes(from),`${f} has no ${from}`);writeFileSync(f,s.replace(from,to))};
|
||||
const posts=(e)=>readdirSync(e.data).filter((n)=>n.startsWith('bus.post-attempt-'));
|
||||
const backupSum=(e,act)=>readFileSync(join(e.data,'backups','stack',act,'SHA256SUMS'),'utf8').split(' ')[0];
|
||||
const commitLines=(range)=>git(repo,'log','--oneline','--no-merges','--max-count=200',range).split('\n').map((l)=>` ${l}`);
|
||||
// The units, the wrapper, the pointer and the store, by sha256.
|
||||
const fp=(e)=>({bus:sha(join(e.units,e.tmpl)),console:sha(join(e.units,e.con)),wrapper:sha(join(e.bin,e.wrapper)),pointer:existsSync(e.pointer)?sha(e.pointer):null,store:sha(e.store)});
|
||||
// The schema version: readOnly while the holder has the store open, immutable when it is stopped.
|
||||
function storeVersion(file,held){
|
||||
const db=held?new DatabaseSync(file,{readOnly:true}):new DatabaseSync(pathToFileURL(file).href+'?immutable=1',{readOnly:true});
|
||||
try{return db.prepare("SELECT value FROM meta WHERE key='schema_version'").get().value}finally{db.close()}
|
||||
}
|
||||
// Every file and directory under p with size and mtime: T23 and T16 compare it before and after.
|
||||
function tree(p){
|
||||
const out=[];
|
||||
const walk=(q)=>{let st;try{st=statSync(q)}catch{return}out.push(st.isDirectory()?`${q}/`:`${q} ${st.size} ${st.mtimeMs}`);if(st.isDirectory())for(const n of readdirSync(q).sort())walk(join(q,n))};
|
||||
walk(p);return out;
|
||||
}
|
||||
// T17: what a run may never remove.
|
||||
const kept=(e)=>[...tree(join(e.data,'backups','stack')),...tree(join(e.state,'stack-rollback')),...posts(e).flatMap((n)=>tree(join(e.data,n))),...tree(COPIES)].map((l)=>l.replace(/ .*/,''));
|
||||
|
||||
// The business file (row 169): the committed example made loadable, as acme, with `plugins` when given.
|
||||
const EXAMPLE=readFileSync(join(repo,'packages','business','examples','mosaic-stack.example.json'),'utf8');
|
||||
function biz(e,plugins){
|
||||
const b=JSON.parse(EXAMPLE);let n=0;
|
||||
const walk=(o)=>{for(const [k,v] of Object.entries(o)){if(k==='botId')o[k]=++n;else if(v==='YYYY-MM-DD')o[k]='2027-01-01';else if(v&&typeof v==='object')walk(v)}};
|
||||
walk(b);b.id='acme';if(plugins!==undefined)b.plugins=plugins;
|
||||
const f=join(dirname(e.cfg),'businesses','acme.json');mkdirSync(dirname(f),{recursive:true});
|
||||
writeFileSync(f,JSON.stringify(b,null,2)+'\n',{mode:0o600});chmodSync(f,0o600);
|
||||
}
|
||||
|
||||
async function mk(t){
|
||||
const root=realpathSync(mkdtempSync(join(tmpdir(),'sr-')));
|
||||
const e={root,data:join(root,'data'),bin:join(root,'bin'),sd:join(root,'sd'),units:join(root,'xcfg','systemd','user'),cfg:join(root,'cfg','config.json'),claude:join(root,'claude-bin','claude')};
|
||||
// The default instance's names; named() sets NAME's (row 190).
|
||||
Object.assign(e,{flags:[],bus:BUS,tmpl:'[email protected]',con:CON,wrapper:'mosaic'});
|
||||
Object.assign(e,{state:join(e.data,'state'),pointer:join(e.data,'state','stack-active.json'),log:join(e.data,'state','stack-activation-log.jsonl'),store:join(e.data,'bus','bus.sqlite'),lock:join(e.data,'state','stack-release.lock')});
|
||||
for(const d of [join(root,'home'),dirname(e.cfg),join(e.data,'bus'),e.bin,dirname(e.claude),join(e.sd,'faults'),e.units,join(root,'xdata')])mkdirSync(d,{recursive:true});
|
||||
chmodSync(join(e.data,'bus'),0o700);
|
||||
writeFileSync(e.cfg,JSON.stringify({configVersion:1,environment:'development',dataRoot:e.data,execution:{backend:'docker',provider:'zai',model:'glm-5.3-flash',adapter:'mock'}}));
|
||||
biz(e);
|
||||
writeFileSync(join(e.bin,'mosaic'),`#!/usr/bin/env bash\n${MARKER}\n[ "\${1:-}" = __lease-capability ] && exec "${e.bin}/mosaic-old" "$@"\nexec "/old/scripts/mosaic" "$@"\n`,{mode:0o755});
|
||||
writeFileSync(join(e.bin,'mosaic-old'),'#!/bin/sh\nexit 0\n',{mode:0o755});
|
||||
writeFileSync(e.claude,'#!/bin/sh\necho "9.9.9 (Claude Code fixture)"\n',{mode:0o755});
|
||||
e.env={PATH:`${SBIN}:${dirname(NODE)}:/usr/bin:/bin`,HOME:join(root,'home'),XDG_CONFIG_HOME:join(root,'xcfg'),XDG_DATA_HOME:join(root,'xdata'),MOSAIC_CONFIG:e.cfg,SD_STATE:e.sd,SD_DATA:e.data,TMPDIR:tmpdir()};
|
||||
const server=await healthz(e);
|
||||
t.after(()=>{
|
||||
for(const pid of lines(join(e.sd,'pids.log')).map(Number)){
|
||||
try{if(readFileSync(`/proc/${pid}/cmdline`,'utf8').includes(join(base,'holder.cjs')))process.kill(pid,'SIGKILL')}catch{}
|
||||
}
|
||||
for(const s of [server,...(e.servers??[])])s.close();
|
||||
execFileSync('chmod',['-R','u+w',root]);rmSync(root,{recursive:true,force:true});
|
||||
});
|
||||
return e;
|
||||
}
|
||||
|
||||
// The Console's /healthz on e.port: 200 while the stand-in has e.con active and no healthz-<copy>
|
||||
// fault names its copy.
|
||||
async function healthz(e){
|
||||
const server=createServer((q,r)=>{
|
||||
let st={};try{st=units(e)[e.con]||{}}catch{}
|
||||
const ok=q.url==='/healthz'&&st.state==='active'&&!existsSync(join(e.sd,'faults',`healthz-${basename(st.wd||'')}`));
|
||||
r.writeHead(ok?200:500);r.end();
|
||||
});
|
||||
server.listen(0,'127.0.0.1');await once(server,'listening');e.port=String(server.address().port);
|
||||
return server;
|
||||
}
|
||||
|
||||
// Row 190: stack instance NAME on e's host. It shares e's stand-ins, unit directory and $BIN, and has
|
||||
// its own config and data root from bootstrap --instance, its own Console port and business acme.
|
||||
// Its runs pass --instance NAME and no MOSAIC_CONFIG.
|
||||
async function named(e,name){
|
||||
const env={...e.env};delete env.MOSAIC_CONFIG;
|
||||
execFileSync('bash',[join(repo,'scripts','bootstrap.sh'),'--instance',name],{env,stdio:'ignore'});
|
||||
const data=join(env.HOME,`.mosaic-${name}`);
|
||||
const n={...e,env,data,cfg:join(env.HOME,'.config',`mosaic-${name}`,'config.json'),flags:['--instance',name],bus:`mosaic-bus-${name}@acme.service`,tmpl:`mosaic-bus-${name}@.service`,con:`mosaic-webui-${name}.service`,wrapper:`mosaic-${name}`,name};
|
||||
Object.assign(n,{state:join(data,'state'),pointer:join(data,'state','stack-active.json'),log:join(data,'state','stack-activation-log.jsonl'),store:join(data,'bus','bus.sqlite'),lock:join(data,'state','stack-release.lock')});
|
||||
mkdirSync(join(data,'bus'),{mode:0o700});
|
||||
biz(n);
|
||||
(e.servers??=[]).push(await healthz(n));
|
||||
return n;
|
||||
}
|
||||
|
||||
async function run(e,...args){
|
||||
const before=existsSync(e.data)?kept(e):[];
|
||||
const ch=spawn('bash',[e.sr??SR,...args,...e.flags,'--copies',COPIES,'--bin',e.bin],{env:e.env,stdio:['ignore','pipe','pipe']});
|
||||
let out='',err='';ch.stdout.on('data',(d)=>{out+=d});ch.stderr.on('data',(d)=>{err+=d});
|
||||
const timer=setTimeout(()=>ch.kill('SIGKILL'),120000);
|
||||
const [code]=await once(ch,'close');clearTimeout(timer);
|
||||
const now=new Set(kept(e));
|
||||
for(const p of before)assert.ok(now.has(p),`T17: ${args.join(' ')} removed ${p}`);
|
||||
return {code,out,err};
|
||||
}
|
||||
|
||||
// The stack on copy `copy` for business acme, both units active. pointer: adopt (a real adopt and
|
||||
// its log line), hand (written here, no log) or none. Snapshots start after the seed.
|
||||
async function seed(e,{copy=B,pm=false,store='3g',pointer:how='adopt'}={}){
|
||||
const db=new DatabaseSync(e.store);
|
||||
db.exec('PRAGMA journal_mode=WAL');
|
||||
db.exec('CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL)');
|
||||
db.exec('CREATE TABLE events (id INTEGER PRIMARY KEY, at TEXT NOT NULL)');
|
||||
db.prepare("INSERT INTO meta VALUES ('schema_version', ?)").run(store);
|
||||
for(let i=0;i<3;i++)db.prepare('INSERT INTO events (at) VALUES (?)').run(`2000-01-01T00:00:0${i}.000Z`);
|
||||
db.close();
|
||||
const sh=(name,...a)=>execFileSync('bash',[join(repo,'scripts',name),...a],{env:e.env,encoding:'utf8',stdio:['ignore','pipe','pipe']});
|
||||
sh('bus-service.sh','install',...e.flags,'--copy',copy,'--copies',COPIES,...(pm?['--pm']:[]),'--claude',e.claude);
|
||||
sh('webui-service.sh','install',...e.flags,'--copy',copy,'--copies',COPIES,'--port',e.port,'--business','acme');
|
||||
sh('stack-copy.sh','wrapper',copy,...e.flags,'--copies',COPIES,'--bin',e.bin);
|
||||
// The seeded host started two minutes ago, so its boot status and PM run predate any run's start.
|
||||
execFileSync(join(SBIN,'systemctl'),['--user','start',e.bus],{env:{...e.env,SD_CLOCK_OFFSET:'120'}});
|
||||
sysctl(e,'start',e.con);
|
||||
if(how==='adopt'){const r=await run(e,'adopt','acme');assert.equal(r.code,0,r.err)}
|
||||
else if(how==='hand'){
|
||||
mkdirSync(e.state,{recursive:true});
|
||||
writeFileSync(e.pointer,JSON.stringify({copy,business:'acme',pm,claude:e.claude,console:['--port',e.port,'--business','acme'],schema:'3g',activation:'manual',at:'2026-10-18T00:00:00Z'})+'\n');
|
||||
}
|
||||
writeFileSync(join(e.sd,'snap.jsonl'),'');
|
||||
}
|
||||
|
||||
const rolledBack=(code,to=B)=>`stack-release: ${code}; rolled back to ${to}, which passed the gate\n`;
|
||||
const planBackup=(e)=>`backup: the run backs up the stopped store to ${e.data}/backups/stack/<activation>/ before any migration`;
|
||||
const planNoBackup='backup: none; the run rolls back before the backup';
|
||||
const planTail=(e,to,pm=false)=>[`bus: scripts/bus-service.sh install --copy ${to} --copies ${COPIES}${pm?' --pm':''} --claude ${e.claude}`,'proof: off','send: off',`Console: scripts/webui-service.sh install --copy ${to} --copies ${COPIES} --port ${e.port} --business acme`,`CLI: scripts/stack-copy.sh wrapper ${to} --copies ${COPIES} --bin ${e.bin}`];
|
||||
import {A,B,BUS,C,C1,CON,COPIES,EXAMPLE,F,KINDS_NEW,KINDS_OLD,M,MARKER,NODE,ROOT,S,SBIN,SR,backupSum,base,biz,calls,commit,commitLines,edit,fault,fp,git,healthz,holder,journalctl,kept,lines,logLines,migrateStub,mk,mosaicStub,named,pkg,planBackup,planNoBackup,planTail,pointer,posts,repo,rolledBack,run,script,seed,sha,snaps,startsStops,storeText,storeVersion,sysctl,systemctl,systemdAnalyze,tracked,tree,U,units} from './stack-release-fixture.mjs';
|
||||
|
||||
// --- The T table ---
|
||||
|
||||
@@ -1331,6 +916,8 @@ test('T46 row 181 item 4: every table in the bus schema is in exactly one of fou
|
||||
const hasAt=(n)=>db.prepare(`SELECT count(*) AS n FROM pragma_table_info(?) WHERE name='at'`).get(n).n===1;
|
||||
const always={decisions:'decision.raise writes action.allowed in the same transaction'};
|
||||
const untimed={
|
||||
host_grants:'no time column; row 234: each write goes in one transaction with a grant.* events row, which is counted',
|
||||
hosts:'no time column; row 234: each write goes in one transaction with a grant.* events row, which is counted',
|
||||
meta:'key and value only; its one UPDATE is the migration step',
|
||||
message_threads:'no time column; each row goes with a messages row, which is counted',
|
||||
sqlite_sequence:'SQLite\'s own AUTOINCREMENT table',
|
||||
@@ -2096,6 +1683,14 @@ test('T57 row 200: activate --sso, its refusals before any stop, and the rollbac
|
||||
assert.ok(stepText(a).includes(`Z drop --sso; ${B}'s Console does not support it, so the Console flags come from ${set}/${show}\n`));
|
||||
await noLeak(a);
|
||||
});
|
||||
await t.test('row 238 (#1744): on a named instance the sso-config first step names --instance NAME in both commands',async(t)=>{
|
||||
const e=await mk(t),a=await named(e,'alpha');await seed(a);
|
||||
assert.equal(JSON.parse(readFileSync(a.cfg,'utf8')).console,undefined);
|
||||
await refused(a,['activate',C1,'acme','--sso'],'sso-config',`${C1}'s Console would refuse its --sso start: there is no console.sso in the system config; activate ${C1} --instance alpha without --sso first, then write console.sso, then run activate ${C1} --instance alpha --sso`);
|
||||
const out=await planOnly(a,3,'activate',C1,'acme','--sso');
|
||||
assert.ok(out.includes(`refuses sso-config: ${C1}'s Console would refuse its --sso start: there is no console.sso in the system config; activate ${C1} --instance alpha without --sso first, then write console.sso, then run activate ${C1} --instance alpha --sso`),out.join('\n'));
|
||||
await noLeak(a);
|
||||
});
|
||||
});
|
||||
|
||||
// --- rehearse (D2, row 163; the brief's "Stack rehearsal on a scratch root") ---
|
||||
@@ -2477,6 +2072,49 @@ test('R15 a rehearsal stopped by TERM, INT or HUP fails signal and still stops t
|
||||
}
|
||||
});
|
||||
|
||||
// --- Row 233 (#1735): --backup last, the newest backup the log records (sage-1650, sage-1653) ---
|
||||
|
||||
// The log's lines, as the runs write them, with only the fields from_log reads set.
|
||||
function rlog(e,...ls){
|
||||
mkdirSync(e.state,{recursive:true});
|
||||
writeFileSync(e.log,ls.map(([op,to,backup],i)=>JSON.stringify({at:`2026-10-20T00:00:0${i}Z`,op,from:null,to,backup,activation:`20261020T00000${i}Z-${to.slice(0,12)}`,result:'ok'})+'\n').join(''));
|
||||
}
|
||||
|
||||
test('row 233: --backup last takes the newest backup the log records; a later line with backup null does not hide it, and the store line names it',async(t)=>{
|
||||
const e=await rmk(t);
|
||||
// An older backup that isn't on disk: a run that took it would fail backup-missing.
|
||||
rlog(e,['activate',C1,`20261019T000000Z-${C1.slice(0,12)}`],['activate',B,e.act],['rollback',B,null]);
|
||||
const l0=readFileSync(e.log,'utf8'),before=untouched(e);
|
||||
const r=await rehearse(e,B,'--backup','last','--port',await freePort(),'--timeout','10');
|
||||
assert.equal(r.code,0,r.out+r.err);
|
||||
assert.deepEqual(r.names,['PASS backup-last',...PRE,'PASS migration','PASS gate','PASS no-session',...POST]);
|
||||
assert.ok(r.out.startsWith(`PASS backup-last: the log's newest backup is ${e.act}\n`),r.out);
|
||||
assert.ok(r.out.includes(`\nBACKUP_STORE ${e.act} schema 3g\n`),r.out);
|
||||
assert.equal(readFileSync(e.log,'utf8'),l0);assert.equal(untouched(e),before);
|
||||
// The rehearsal's own steps.log has the PASS and FAIL lines only.
|
||||
assert.deepEqual(lines(join(r.dir,'steps.log')).map((l)=>l.replace(/^\S+ /,'')),r.out.split('\n').filter((l)=>/^(PASS|FAIL) |^REHEARSAL_/.test(l)));
|
||||
});
|
||||
|
||||
test('row 233: --backup last with no backup in the log, or no log, exits 3 with no-backup and makes no rehearsal directory',async(t)=>{
|
||||
const e=await rmk(t);
|
||||
for(const how of ['no log','adopt only']){
|
||||
if(how==='adopt only')rlog(e,['adopt',B,null]);
|
||||
const r=await rehearse(e,B,'--backup','last','--port',await freePort(),'--timeout','3');
|
||||
assert.equal(r.code,3,how+r.out+r.err);
|
||||
assert.equal(r.out,'FAIL no-backup: the log records no backup\nREHEARSAL_FAIL\n',how);
|
||||
assert.equal(r.dir,null);assert.deepEqual(rDirs(e),[],how);assert.ok(!existsSync(join(e.data,'rehearsals')),how);
|
||||
}
|
||||
});
|
||||
|
||||
test('row 233: --backup last refuses a newest backup that is not an activation id, with exit 1',async(t)=>{
|
||||
const e=await rmk(t);
|
||||
rlog(e,['activate',B,'../../etc']);
|
||||
const r=await rehearse(e,B,'--backup','last','--port',await freePort(),'--timeout','3');
|
||||
assert.equal(r.code,1,r.out+r.err);
|
||||
assert.equal(r.out,'FAIL backup-unreadable: the log\'s newest backup is not an activation id\nREHEARSAL_FAIL\n');
|
||||
assert.deepEqual(rDirs(e),[]);
|
||||
});
|
||||
|
||||
// --- Row 190 (#1688): stack instances on one host. Design and the T-I table:
|
||||
// agents/darkwing/work/queue-190/DESIGN.md. T-I1, T-I2, T-I5, T-I10 and the loader half of T-I7 are in
|
||||
// stack-instances.test.mjs; these need the units, the stand-ins and stack-release.sh. Every existing
|
||||
|
||||
@@ -0,0 +1,464 @@
|
||||
// scripts/stack-upgrade.sh (row 233, I3, #1735): root's upgrade and rollback of a stack whose units are
|
||||
// root's, driving stack-release.sh --units external (row 232). Design: r6 sections 4 and 7,
|
||||
// agents/darkwing/work/queue-195/DESIGN.md, as the I3 brief changes them; deviations:
|
||||
// agents/darkwing/work/queue-233/BUILD.md. The stand-ins and the module fixture are
|
||||
// stack-release-fixture.mjs's. Root's sudo, case2-unit.sh render and row 240 hostcheck are stand-ins
|
||||
// here; the stack's user's steps are the real stack-release.sh from the user's clone, except the
|
||||
// rehearsal, which the fixture's stub copies can't pass (its own tests are in stack-release.test.mjs).
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {execFileSync,spawn} from 'node:child_process';
|
||||
import {mkdirSync,writeFileSync,readFileSync,existsSync,readdirSync,rmSync,chmodSync,copyFileSync,symlinkSync,statSync} from 'node:fs';
|
||||
import {DatabaseSync} from 'node:sqlite';
|
||||
import {join,dirname} from 'node:path';
|
||||
import {once} from 'node:events';
|
||||
import {B,C,C1,U,COPIES,NODE,SBIN,base,fault,git,lines,logLines,mk,pointer,repo,sha,storeVersion,sysctl,units} from './stack-release-fixture.mjs';
|
||||
|
||||
// --- Root's stand-ins. Each is serialized into its own file with K, its baked paths. ---
|
||||
|
||||
// sudo -u USER -H env PATH=P CMD...: the form the driver must use, logged to sudo.log, then CMD with
|
||||
// SBIN before P. A one-shot hook arm/<key>.sh runs first and arm/<key>.id replaces CMD's ID. The
|
||||
// rehearsal prints a fixed BACKUP_STORE line and exits faults/rehearse-rc unless faults/real-rehearse.
|
||||
function fakeSudo(K){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),cp=process.getBuiltinModule('node:child_process');
|
||||
const a=process.argv.slice(2),sd=process.env.SD_STATE,f=(n)=>path.join(sd,'faults',n);
|
||||
if(a[0]!=='-u'||!a[1]||a[2]!=='-H'||a[3]!=='env'||!/^PATH=/.test(a[4]||'')||a.length<6){console.error(`fake sudo: not the driver's form: ${a.join(' ')}`);process.exit(99)}
|
||||
const P=a[4].slice(5),cmd=a.slice(5);
|
||||
let key=path.basename(cmd[0]);
|
||||
if(cmd[0].endsWith('/scripts/stack-release.sh')){
|
||||
key=cmd[1];
|
||||
if(['activate','rollback'].includes(key)&&cmd.includes('--plan'))key+='-plan';
|
||||
if(key==='confirm'&&cmd.includes('--rollback'))key='confirm-rollback';
|
||||
}else if(cmd[0]==='git')key=`git-${cmd[3]}`;
|
||||
fs.appendFileSync(path.join(sd,'sudo.log'),JSON.stringify({user:a[1],path:P,key,cmd})+'\n');
|
||||
const env={...process.env,PATH:`${K.sbin}:${P}`},arm=(n)=>path.join(sd,'arm',`${key}.${n}`);
|
||||
if(fs.existsSync(arm('sh'))){const run=arm('sh')+'.ran';fs.renameSync(arm('sh'),run);cp.spawnSync('bash',[run],{env,stdio:'inherit'})}
|
||||
if(fs.existsSync(arm('id'))){cmd[2]=fs.readFileSync(arm('id'),'utf8').trim();fs.rmSync(arm('id'))}
|
||||
if(key==='rehearse'&&!fs.existsSync(f('real-rehearse'))){
|
||||
console.log(`fixture rehearse: ${cmd.slice(1).join(' ')}`);
|
||||
let line='BACKUP_STORE 20261020T000000Z-0123456789ab schema 3g';try{line=fs.readFileSync(f('rehearse-store-line'),'utf8').replace(/\n$/,'')}catch{}
|
||||
if(line)console.log(line);
|
||||
let rc=0;try{rc=Number(fs.readFileSync(f('rehearse-rc'),'utf8'))}catch{}
|
||||
process.exit(rc);
|
||||
}
|
||||
const r=cp.spawnSync(cmd[0],cmd.slice(1),{env,stdio:'inherit'});process.exit(r.status??1);
|
||||
}
|
||||
|
||||
// case2-unit.sh: logs its argv, then installs the copy's units as the fixture's user units (its
|
||||
// WorkingDirectory under the stack's copies) and starts the bus, then the Console, as case2-unit.sh
|
||||
// does. faults/render-fails-<copy> exits 1 first; faults/render-probe-<copy> (one-shot) first runs
|
||||
// probe.sh, a second driver, and records its exit and stderr.
|
||||
function fakeRender(K){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),cp=process.getBuiltinModule('node:child_process');
|
||||
const a=process.argv.slice(2),sd=process.env.SD_STATE,f=(n)=>path.join(sd,'faults',n);
|
||||
fs.appendFileSync(path.join(sd,'render.log'),JSON.stringify(a)+'\n');
|
||||
const cut=a.indexOf('--'),pre=a.slice(0,cut),serve=a.slice(cut+1),get=(k)=>pre[pre.indexOf(k)+1];
|
||||
const copy=get('--copy'),copies=path.join(get('--home'),'.local','share','mosaic-stack','copies');
|
||||
if(fs.existsSync(f(`render-probe-${copy}`))){
|
||||
fs.rmSync(f(`render-probe-${copy}`));
|
||||
const p=cp.spawnSync('bash',[path.join(sd,'probe.sh')],{encoding:'utf8'});
|
||||
fs.writeFileSync(path.join(sd,'probe.json'),JSON.stringify({code:p.status,err:p.stderr}));
|
||||
}
|
||||
if(fs.existsSync(f(`render-fails-${copy}`))){console.error(`fixture render: ${copy} fails`);process.exit(1)}
|
||||
const sh=(n,...x)=>cp.execFileSync('bash',[path.join(K.repo,'scripts',n),...x],{stdio:['ignore','ignore','inherit']});
|
||||
sh('bus-service.sh','install','--copy',copy,'--copies',copies,...(pre.includes('--pm')?['--pm']:[]),'--claude',get('--claude'));
|
||||
sh('webui-service.sh','install','--copy',copy,'--copies',copies,...serve);
|
||||
for(const u of [`mosaic-bus@${get('--business')}.service`,'mosaic-webui.service'])cp.execFileSync(path.join(K.sbin,'systemctl'),['start',u],{stdio:'inherit'});
|
||||
}
|
||||
|
||||
// Root's systemctl: the fixture's, with systemd's no-op start of an active unit, and a one-shot
|
||||
// faults/stop-noop-<unit> that leaves the unit running, as a stop past its timeout would.
|
||||
function fakeSystemctl(K){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),cp=process.getBuiltinModule('node:child_process');
|
||||
const a=process.argv.slice(2),sd=process.env.SD_STATE;
|
||||
fs.appendFileSync(path.join(sd,'root-systemctl.log'),a.join(' ')+'\n');
|
||||
const noop=path.join(sd,'faults',`stop-noop-${a[1]}`);
|
||||
if(a[0]==='stop'&&fs.existsSync(noop)){fs.rmSync(noop);process.exit(0)}
|
||||
if(a[0]==='start'){let s={};try{s=JSON.parse(fs.readFileSync(path.join(sd,'units.json'),'utf8'))}catch{}if((s[a[1]]||{}).state==='active')process.exit(0)}
|
||||
const r=cp.spawnSync(path.join(K.sbin,'systemctl'),a,{stdio:'inherit'});process.exit(r.status??1);
|
||||
}
|
||||
|
||||
const FAKES=join(base,'upgrade-fakes');mkdirSync(FAKES);
|
||||
const K={sbin:SBIN,repo};
|
||||
for(const [n,fn] of [['sudo',fakeSudo],['render',fakeRender],['systemctl',fakeSystemctl]])writeFileSync(join(FAKES,n),`#!${NODE}\n(${fn})(${JSON.stringify(K)});\n`,{mode:0o755});
|
||||
writeFileSync(join(FAKES,'hostcheck'),'#!/bin/sh\necho "$*" >> "$SD_STATE/hostcheck.log"\nexit "$(cat "$SD_STATE/faults/hostcheck-rc" 2>/dev/null || echo 0)"\n',{mode:0o755});
|
||||
const BUNDLE=join(base,'fixture.bundle');git(repo,'bundle','create','-q',BUNDLE,'refactor');
|
||||
// Root's checkout of a reviewed commit: a clone of the fixture repo, detached at it.
|
||||
const checkouts=new Map();
|
||||
const checkout=(id)=>{
|
||||
if(!checkouts.has(id)){const d=join(base,`root-${id.slice(0,12)}`);git(base,'clone','-q','--no-checkout',repo,d);git(d,'checkout','-q','--detach',id);checkouts.set(id,d)}
|
||||
return checkouts.get(id);
|
||||
};
|
||||
const TO=['--timeout','3'];
|
||||
|
||||
// --- Per-test stack ---
|
||||
|
||||
// The stack's user's home: its copies (the fixture's, through a link), its clone, its bin. Both units
|
||||
// installed on `copy` and active, adopted under --units external, and the I2 args file naming it.
|
||||
async function useed(t,{copy=B}={}){
|
||||
const e=await mk(t),home=e.env.HOME;
|
||||
e.hc=join(home,'.local','share','mosaic-stack','copies');mkdirSync(dirname(e.hc),{recursive:true});symlinkSync(COPIES,e.hc);
|
||||
const bin=join(home,'.npm-global','bin');mkdirSync(bin,{recursive:true});
|
||||
writeFileSync(join(bin,'mosaic'),readFileSync(join(e.bin,'mosaic'),'utf8').replaceAll(`${e.bin}/`,`${bin}/`),{mode:0o755});
|
||||
copyFileSync(join(e.bin,'mosaic-old'),join(bin,'mosaic-old'));chmodSync(join(bin,'mosaic-old'),0o755);
|
||||
e.bin=bin;e.clone=join(home,'src','mosaic-stack');git(base,'clone','-q',repo,e.clone);
|
||||
const db=new DatabaseSync(e.store);
|
||||
db.exec('PRAGMA journal_mode=WAL');
|
||||
db.exec('CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL)');
|
||||
db.exec('CREATE TABLE events (id INTEGER PRIMARY KEY, at TEXT NOT NULL)');
|
||||
db.prepare("INSERT INTO meta VALUES ('schema_version', '3g')").run();
|
||||
for(let i=0;i<3;i++)db.prepare('INSERT INTO events (at) VALUES (?)').run(`2000-01-01T00:00:0${i}.000Z`);
|
||||
db.close();
|
||||
const sh=(n,...a)=>execFileSync('bash',[join(e.clone,'scripts',n),...a],{env:e.env,encoding:'utf8',stdio:['ignore','pipe','pipe']});
|
||||
sh('bus-service.sh','install','--copy',copy,'--copies',e.hc,'--claude',e.claude);
|
||||
sh('webui-service.sh','install','--copy',copy,'--copies',e.hc,'--port',e.port,'--business','acme');
|
||||
sh('stack-copy.sh','wrapper',copy,'--copies',e.hc,'--bin',e.bin);
|
||||
execFileSync(join(SBIN,'systemctl'),['start',e.bus],{env:{...e.env,SD_CLOCK_OFFSET:'120'}});
|
||||
sysctl(e,'start',e.con);
|
||||
sh('stack-release.sh','adopt','acme','--units','external','--copies',e.hc,'--bin',e.bin);
|
||||
e.etc=join(e.root,'etc');mkdirSync(e.etc);e.args=join(e.etc,'dev.args');
|
||||
e.argsText=`--user\nfixture\n--home\n${home}\n--business\nacme\n--copy\n${copy}\n--node\n${NODE}\n--claude\n${e.claude}\n--\n--port\n${e.port}\n--business\nacme\n`;
|
||||
writeFileSync(e.args,e.argsText);chmodSync(e.args,0o640);
|
||||
e.run=join(e.root,'run');e.slog=join(e.root,'log','upgrade-dev.log');e.lockf=join(e.run,'upgrade-dev.lock');
|
||||
mkdirSync(join(e.sd,'arm'));
|
||||
e.denv={...e.env,STACK_UPGRADE_ETC:e.etc,STACK_UPGRADE_RUN:e.run,STACK_UPGRADE_LOG_DIR:dirname(e.slog),STACK_UPGRADE_UNIT_DIR:e.units,STACK_UPGRADE_HOSTCHECK:join(FAKES,'hostcheck'),STACK_UPGRADE_RENDER:join(FAKES,'render'),STACK_UPGRADE_SUDO:join(FAKES,'sudo'),STACK_UPGRADE_SYSTEMCTL:join(FAKES,'systemctl')};
|
||||
writeFileSync(join(e.sd,'probe.sh'),`exec bash ${join(checkout(C1),'scripts','stack-upgrade.sh')} upgrade --instance dev --bundle ${BUNDLE} --sha ${C1} --timeout 3\n`);
|
||||
writeFileSync(join(e.sd,'snap.jsonl'),'');
|
||||
return e;
|
||||
}
|
||||
|
||||
// The driver from root's checkout of `at`.
|
||||
async function drive(e,at,...args){
|
||||
const ch=spawn('bash',[join(checkout(at),'scripts','stack-upgrade.sh'),...args],{env:e.denv,stdio:['ignore','pipe','pipe']});
|
||||
let out='',err='';ch.stdout.on('data',(d)=>{out+=d});ch.stderr.on('data',(d)=>{err+=d});
|
||||
const timer=setTimeout(()=>ch.kill('SIGKILL'),180000);
|
||||
const [code]=await once(ch,'close');clearTimeout(timer);
|
||||
return {code,out,err,all:out+err};
|
||||
}
|
||||
const upgrade=(e,id,...a)=>drive(e,id,'upgrade','--instance','dev','--bundle',BUNDLE,'--sha',id,...TO,...a);
|
||||
const arm=(e,key,text)=>writeFileSync(join(e.sd,'arm',key),text);
|
||||
|
||||
// The step log: every line `<time> <step> rc=N ...` or the result line, root:root 0640 (here the
|
||||
// test user's), and nothing the stack's user printed. Returns the lines without the time.
|
||||
const STEP=/^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ (start|fetch|ff|copy|hostcheck|units|plan|rehearse|rehearse-store|rehearsal-skipped|stop-console|stop-bus|wait-down|stage|render|confirm|args|revert|render-old|confirm-rollback|gate|start-bus|start-console|stop-units) rc=\d+ |^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ result exit=[0-35] [a-z-]+ /;
|
||||
function slog(e){
|
||||
const l=lines(e.slog);
|
||||
for(const x of l){assert.match(x,STEP);assert.ok(!/stack-release:|fixture rehearse|PASS |FAIL |REHEARSAL/.test(x),x)}
|
||||
assert.equal(statSync(e.slog).mode&0o777,0o640);
|
||||
return l.map((x)=>x.slice(21));
|
||||
}
|
||||
const names=(e)=>slog(e).map((x)=>x.split(' ')[0]);
|
||||
const result=(e)=>slog(e).at(-1);
|
||||
const sudoKeys=(e)=>lines(join(e.sd,'sudo.log')).map((l)=>JSON.parse(l).key);
|
||||
const lastAct=(e)=>logLines(e).filter((o)=>o.result==='started').at(-1).activation;
|
||||
const mine=(e,a)=>logLines(e).filter((o)=>o.activation===a);
|
||||
const open=(e)=>{const m=new Set();for(const o of logLines(e)){if(!o.activation)continue;if(o.result==='started')m.add(o.activation);else m.delete(o.activation)}return [...m]};
|
||||
const final=(e,a)=>{const [s,f,...more]=mine(e,a);assert.equal(s.result,'started');assert.equal(more.length,0);return f};
|
||||
const live=(e)=>[units(e)[e.bus].state,units(e)[e.con].state,units(e)[e.bus].wd.split('/').at(-1),units(e)[e.con].wd.split('/').at(-1)];
|
||||
// Nothing the run did reached the args file: the same bytes and mode, and no temporary file left.
|
||||
const argsKept=(e)=>{assert.equal(readFileSync(e.args,'utf8'),e.argsText);assert.equal(statSync(e.args).mode&0o777,0o640);assert.deepEqual(readdirSync(e.etc),['dev.args'])};
|
||||
const argsCopy=(e)=>{const l=readFileSync(e.args,'utf8').split('\n');return l[l.indexOf('--copy')+1]};
|
||||
const PRE=['start','fetch','ff','copy','hostcheck','units','plan','rehearse','rehearse-store'];
|
||||
const A2=['stop-console','stop-bus','wait-down','wait-down'];
|
||||
const R=[...A2,'revert','render-old','confirm-rollback'];
|
||||
|
||||
// --- r6 sequences 1-8, through the driver ---
|
||||
|
||||
test('row 233: sequence 1: confirm fails healthz, the driver reverts, renders the old copy and confirm --rollback --cause A5 passes: exit 1',async(t)=>{
|
||||
const e=await useed(t);
|
||||
fault(e,`healthz-${C1}`);
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,1,r.all);
|
||||
assert.equal(r.err.split('\n').at(-2),`stack-upgrade: failed: A5 failed; rolled back to ${B}, which passed the gate`);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage','render','confirm',...R,'result']);
|
||||
assert.equal(result(e),`result exit=1 failed A5 failed; rolled back to ${B}, which passed the gate`);
|
||||
const a=lastAct(e),l=final(e,a);assert.deepEqual([l.op,l.result,l.code,l.rollback,l.from,l.to],['activate','gate-failed','healthz','ok',B,C1]);
|
||||
assert.deepEqual(open(e),[]);assert.deepEqual(live(e),['active','active',B,B]);
|
||||
argsKept(e);
|
||||
// The stack's user's steps went through sudo in the driver's form, as fixture, the pinned node first.
|
||||
for(const l of lines(join(e.sd,'sudo.log')).map((x)=>JSON.parse(x)))assert.deepEqual([l.user,l.path],['fixture',`${dirname(NODE)}:/usr/local/bin:/usr/bin:/bin`]);
|
||||
assert.deepEqual(sudoKeys(e),['git-fetch','git-merge','stack-copy.sh','activate-plan','rehearse','activate','confirm','revert','confirm-rollback']);
|
||||
});
|
||||
|
||||
test('row 233: sequence 2: activate --migrate fails at the wrapper, revert restores the store, confirm --rollback --cause A3: exit 1',async(t)=>{
|
||||
const e=await useed(t);
|
||||
const s0=sha(e.store);
|
||||
arm(e,'activate.sh',`chmod 0555 ${e.bin}\n`);arm(e,'revert.sh',`chmod 0755 ${e.bin}\n`);
|
||||
const r=await upgrade(e,C,'--migrate');assert.equal(r.code,1,r.all);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage',...R,'result']);
|
||||
assert.equal(result(e),`result exit=1 failed A3 failed; rolled back to ${B}, which passed the gate`);
|
||||
const l=final(e,lastAct(e));assert.deepEqual([l.result,l.code,l.rollback],['failed','wrapper','ok']);
|
||||
assert.equal(storeVersion(e.store,true),'3g');assert.deepEqual(live(e),['active','active',B,B]);
|
||||
// The bus holds the store open again; its bytes are the backup's once it stops.
|
||||
sysctl(e,'stop',e.bus);assert.equal(sha(e.store),s0);
|
||||
argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: sequence 3: root\'s render of the new copy fails, the driver rolls back with --cause A4, and holds the lock to the end: exit 1',async(t)=>{
|
||||
const e=await useed(t);
|
||||
fault(e,`render-fails-${C1}`);fault(e,`render-probe-${B}`);
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,1,r.all);
|
||||
// The probe ran inside render-old, during the rollback, and its one line is its refusal.
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage','render',...A2,'revert','result','render-old','confirm-rollback','result']);
|
||||
const p=JSON.parse(readFileSync(join(e.sd,'probe.json'),'utf8'));assert.equal(p.code,3,p.err);
|
||||
assert.equal(p.err,`stack-upgrade: upgrade-running: another stack-upgrade.sh run holds ${e.lockf} (nothing was stopped)\n`);
|
||||
assert.match(slog(e).at(-4),/^result exit=3 upgrade-running /);
|
||||
assert.match(slog(e).find((x)=>x.startsWith('render ')),/^render rc=1 /);
|
||||
const l=final(e,lastAct(e));assert.deepEqual([l.result,l.code,l.rollback],['failed','units-start','ok']);
|
||||
assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: sequence 4: a capture failure is exit 3 from the stage, and A2f restarts the old units and gates them: exit 1',async(t)=>{
|
||||
const e=await useed(t);
|
||||
arm(e,'activate.sh',`: > ${join(e.sd,'faults','no-fragment')}\n`);arm(e,'gate.sh',`rm ${join(e.sd,'faults','no-fragment')}\n`);
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,1,r.all);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage','start-bus','start-console','gate','result']);
|
||||
assert.match(slog(e).find((x)=>x.startsWith('stage ')),/^stage rc=3 /);
|
||||
assert.equal(result(e),`result exit=1 failed A3 failed; ${B} is running again and passed the gate`);
|
||||
const a=lastAct(e);assert.deepEqual(mine(e,a).map((o)=>[o.result,o.code]),[['started',undefined],['refused','capture-failed']]);
|
||||
assert.deepEqual(open(e),[]);assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: N1: a bus that outlives A2\'s timeout keeps its old start, so A2f\'s gate fails stale-start: exit 2, both units stopped',async(t)=>{
|
||||
const e=await useed(t);
|
||||
fault(e,`stop-noop-${e.bus}`);
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,2,r.all);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'start-bus','start-console','gate','stop-units','result']);
|
||||
assert.ok(slog(e).includes(`wait-down rc=1 unit: ${e.bus} state: active`),slog(e).join('\n'));
|
||||
assert.ok(r.out.includes(`gate failed on ${B}: stale-start`),r.out);
|
||||
assert.equal(result(e),`result exit=2 rollback-failed A2 failed and ${B} failed its gate after the restart; both units are stopped`);
|
||||
assert.deepEqual(live(e).slice(0,2),['inactive','inactive']);
|
||||
assert.ok(!sudoKeys(e).includes('activate'));argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: sequence 5: confirm of the wrong copy is refused, and the driver rolls back with --cause A5: exit 1',async(t)=>{
|
||||
const e=await useed(t);
|
||||
arm(e,'confirm.id',B);
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,1,r.all);
|
||||
assert.ok(r.out.includes('refusing open-activation')||r.err.includes('refusing open-activation'),r.all);
|
||||
assert.match(slog(e).find((x)=>x.startsWith('confirm ')),/^confirm rc=3 /);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage','render','confirm',...R,'result']);
|
||||
const l=final(e,lastAct(e));assert.deepEqual([l.result,l.code,l.rollback],['gate-failed','unrecorded','ok']);
|
||||
assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: sequence 6: a revert that fails stops both units at exit 2, and the next run refuses at its --plan with nothing stopped',async(t)=>{
|
||||
const e=await useed(t);
|
||||
fault(e,`render-fails-${C1}`);
|
||||
arm(e,'revert.sh',`for d in ${join(e.state,'stack-rollback')}/*/; do [ -f "$d/AWAITING-UNITS" ] && rm "$d/wrapper"; done\n`);
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,2,r.all);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage','render',...A2,'revert','stop-units','result']);
|
||||
assert.equal(result(e),`result exit=2 rollback-failed A4 failed and revert failed; both units are stopped`);
|
||||
const a=lastAct(e);assert.deepEqual(open(e),[a]);assert.deepEqual(live(e).slice(0,2),['inactive','inactive']);
|
||||
argsKept(e);
|
||||
const n=slog(e).length,sc=lines(join(e.sd,'root-systemctl.log')).length;
|
||||
rmSync(join(e.sd,'faults',`render-fails-${C1}`));
|
||||
const again=await upgrade(e,C1);assert.equal(again.code,3,again.all);
|
||||
assert.ok(again.out.includes(`refuses interrupted-run: activation ${a} started and has no final line`),again.out);
|
||||
assert.deepEqual(names(e).slice(n),PRE.slice(0,7).concat('result'));
|
||||
assert.match(result(e),/^result exit=3 plan-refused /);
|
||||
assert.equal(lines(join(e.sd,'root-systemctl.log')).length,sc);argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: sequence 7: two upgrades and a rollback --to the previous copy with --restore, the args file following each confirm',async(t)=>{
|
||||
const e=await useed(t);
|
||||
// The args file at confirm still names the old copy; the new one moves in after it.
|
||||
arm(e,'confirm.sh',`cp ${e.args} ${join(e.sd,'args-at-confirm')}\n`);
|
||||
const one=await upgrade(e,C1);assert.equal(one.code,0,one.all);
|
||||
assert.equal(one.out.split('\n').at(-2),`stack-upgrade: upgrade ok: ${C1} is live for acme`);
|
||||
assert.equal(readFileSync(join(e.sd,'args-at-confirm'),'utf8'),e.argsText);
|
||||
assert.equal(readFileSync(e.args,'utf8'),e.argsText.replace(`--copy\n${B}\n`,`--copy\n${C1}\n`));
|
||||
assert.equal(statSync(e.args).mode&0o777,0o640);assert.deepEqual(readdirSync(e.etc),['dev.args']);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage','render','confirm','args','result']);
|
||||
assert.ok(slog(e).includes(`args rc=0 copy: ${C1}`));
|
||||
const a1=lastAct(e);
|
||||
const two=await upgrade(e,C,'--migrate');assert.equal(two.code,0,two.all);
|
||||
const a2=lastAct(e);assert.equal(argsCopy(e),C);
|
||||
const l1=final(e,a1),l2=final(e,a2);
|
||||
assert.deepEqual([l1.op,l1.result,l1.from,l1.to],['activate','ok',B,C1]);
|
||||
assert.deepEqual([l2.op,l2.result,l2.from,l2.to,l2.backup],['activate','ok',C1,C,a2]);
|
||||
assert.equal(storeVersion(e.store,true),'3h');
|
||||
const n=slog(e).length;
|
||||
const rb=await drive(e,C,'rollback','--instance','dev','--to',C1,'--restore',a2,...TO);assert.equal(rb.code,0,rb.all);
|
||||
assert.equal(rb.out.split('\n').at(-2),`stack-upgrade: rollback ok: ${C1} is live for acme`);
|
||||
// A rollback fetches, copies and rehearses nothing.
|
||||
assert.deepEqual(names(e).slice(n),['start','hostcheck','units','plan',...A2,'stage','render','confirm','args','result']);
|
||||
const ra=lastAct(e),l3=final(e,ra);assert.deepEqual([l3.op,l3.result,l3.from,l3.to,l3.backup],['rollback','ok',C,C1,a2]);
|
||||
assert.deepEqual([pointer(e).copy,pointer(e).schema],[C1,'3g']);assert.equal(storeVersion(e.store,true),'3g');
|
||||
assert.equal(argsCopy(e),C1);assert.deepEqual(live(e),['active','active',C1,C1]);
|
||||
const m=slog(e).length,sc=lines(join(e.sd,'root-systemctl.log')).length;
|
||||
const no=await drive(e,C,'rollback','--instance','dev','--to',B,...TO);assert.equal(no.code,3,no.all);
|
||||
assert.ok(no.all.includes(`refusing to-not-previous: --to ${B} is not the previous copy; the log names ${C}`),no.all);
|
||||
assert.deepEqual(names(e).slice(m),['start','hostcheck','units','plan','result']);
|
||||
assert.equal(lines(join(e.sd,'root-systemctl.log')).length,sc);assert.equal(argsCopy(e),C1);
|
||||
});
|
||||
|
||||
test('row 240: I3 N1: a failed args move after A5 exits 5 args-failed, the new copy live and confirmed and both args files kept',async(t)=>{
|
||||
const e=await useed(t);
|
||||
// A read-only /etc/mosaic-stack at confirm: the stage and the render are done, and the move fails.
|
||||
// The fixture's cleanup runs chmod -R u+w before it removes the root, so the 0555 dir needs no hook here.
|
||||
arm(e,'confirm.sh',`chmod 0555 ${e.etc}\n`);
|
||||
const l=await upgrade(e,C1);assert.equal(l.code,5,l.all);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage','render','confirm','args','result']);
|
||||
assert.ok(slog(e).includes(`args rc=1 copy: ${C1}`));
|
||||
assert.match(result(e),new RegExp(`^result exit=5 args-failed ${C1} is live and confirmed, but ${e.args} still names ${B}; put ${e.etc}/\\.dev\\.args\\.\\w{6} in its place$`));
|
||||
assert.deepEqual(open(e),[]);assert.deepEqual(live(e),['active','active',C1,C1]);
|
||||
assert.equal(readFileSync(e.args,'utf8'),e.argsText);
|
||||
const left=readdirSync(e.etc).filter((n)=>n!=='dev.args');assert.equal(left.length,1);
|
||||
assert.equal(readFileSync(join(e.etc,left[0]),'utf8'),e.argsText.replace(`--copy\n${B}\n`,`--copy\n${C1}\n`));
|
||||
});
|
||||
|
||||
test('row 233: sequence 8: a migration that writes and fails: the driver reverts to the backup byte for byte, --cause A3: exit 1',async(t)=>{
|
||||
const e=await useed(t);
|
||||
const s0=sha(e.store);
|
||||
fault(e,'migrate-fails');
|
||||
const r=await upgrade(e,C,'--migrate');assert.equal(r.code,1,r.all);
|
||||
assert.match(slog(e).find((x)=>x.startsWith('stage ')),/^stage rc=1 /);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage',...R,'result']);
|
||||
const a=lastAct(e),l=final(e,a);assert.deepEqual([l.result,l.code,l.rollback],['failed','migrate-failed','ok']);
|
||||
sysctl(e,'stop',e.bus);assert.equal(sha(e.store),s0);assert.equal(storeVersion(e.store,false),'3g');
|
||||
argsKept(e);
|
||||
});
|
||||
|
||||
// --- The lock, A1's refusals and the rehearsal ---
|
||||
|
||||
test('row 233: a second run while the lock is held refuses upgrade-running with one log line and no call',async(t)=>{
|
||||
const e=await useed(t);
|
||||
mkdirSync(e.run,{recursive:true});
|
||||
const holder=spawn('flock',[e.lockf,'sleep','60'],{detached:true,stdio:'ignore'});
|
||||
t.after(()=>{try{process.kill(-holder.pid,'SIGKILL')}catch{}});
|
||||
for(let i=0;i<100&&execFileSync('bash',['-c','flock -n "$1" true && echo free || echo held','_',e.lockf],{encoding:'utf8'}).trim()==='free';i++)await new Promise((r)=>setTimeout(r,50));
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,3,r.all);
|
||||
assert.equal(r.err,`stack-upgrade: upgrade-running: another stack-upgrade.sh run holds ${e.lockf} (nothing was stopped)\n`);
|
||||
assert.deepEqual(names(e),['result']);
|
||||
assert.ok(!existsSync(join(e.sd,'sudo.log')));assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));
|
||||
assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: A1 refuses open-activation on any AWAITING-UNITS file, through the stack\'s user\'s --plan, with nothing stopped',async(t)=>{
|
||||
const e=await useed(t);
|
||||
const fake=join(e.state,'stack-rollback',`20000101T000000Z-${C1.slice(0,12)}`);mkdirSync(fake,{recursive:true});writeFileSync(join(fake,'AWAITING-UNITS'),`${C1}\n`);
|
||||
const logs=logLines(e).length;
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,3,r.all);
|
||||
assert.ok(r.out.includes('refuses open-activation'),r.out);
|
||||
assert.deepEqual(names(e),[...PRE.slice(0,7),'result']);assert.match(result(e),/^result exit=3 plan-refused /);
|
||||
assert.equal(logLines(e).length,logs);assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));
|
||||
assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: a target that migrates refuses migrate-required without --migrate, with nothing stopped',async(t)=>{
|
||||
const e=await useed(t);
|
||||
const r=await upgrade(e,C);assert.equal(r.code,3,r.all);
|
||||
assert.ok(r.out.includes('refuses migrate-required'),r.out);
|
||||
assert.deepEqual(names(e),[...PRE.slice(0,7),'result']);assert.match(result(e),/^result exit=3 plan-refused /);
|
||||
assert.ok(!sudoKeys(e).includes('rehearse'));assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));
|
||||
assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: the rehearsal runs after the plan and before A2; a failed one refuses rehearsal-failed with nothing stopped',async(t)=>{
|
||||
const e=await useed(t);
|
||||
fault(e,'rehearse-rc','1');
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,3,r.all);
|
||||
assert.ok(r.out.includes('fixture rehearse: '),r.out);
|
||||
assert.deepEqual(names(e),[...PRE,'result']);
|
||||
assert.equal(result(e),'result exit=3 rehearsal-failed the rehearsal exited 1 (nothing was stopped)');
|
||||
const rh=lines(join(e.sd,'sudo.log')).map((l)=>JSON.parse(l)).find((l)=>l.key==='rehearse');
|
||||
assert.deepEqual(rh.cmd.slice(1),['rehearse',C1,'acme','--copies',e.hc,'--backup','last','--port','7339','--timeout','3']);
|
||||
assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: the rehearsal\'s BACKUP_STORE line goes to the step log after a match, a malformed one as none, the rest of its output never',async(t)=>{
|
||||
const e=await useed(t);
|
||||
fault(e,'rehearse-rc','1');
|
||||
const r=await upgrade(e,C1,'--rehearse-port','7449');assert.equal(r.code,3,r.all);
|
||||
assert.ok(slog(e).includes('rehearse-store rc=0 BACKUP_STORE 20261020T000000Z-0123456789ab schema 3g'),slog(e).join('\n'));
|
||||
assert.ok(slog(e).find((x)=>x.startsWith('rehearse ')).includes(' --port 7449 '));
|
||||
for(const bad of ['BACKUP_STORE ../../etc schema 3g','BACKUP_STORE 20261020T000000Z-0123456789ab schema 3g; rm -rf /','stack-release: BACKUP_STORE 20261020T000000Z-0123456789ab schema 3g','']){
|
||||
fault(e,'rehearse-store-line',bad);
|
||||
const n=slog(e).length;
|
||||
const b=await upgrade(e,C1);assert.equal(b.code,3,b.all);
|
||||
assert.deepEqual(slog(e).slice(n).filter((x)=>x.startsWith('rehearse-store ')),['rehearse-store rc=0 BACKUP_STORE none'],bad);
|
||||
}
|
||||
});
|
||||
|
||||
test('row 233: a real rehearsal with no backup in the log refuses no-backup, and goes on only with --no-rehearsal, which logs the skip',async(t)=>{
|
||||
const e=await useed(t);
|
||||
fault(e,'real-rehearse');
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,3,r.all);
|
||||
assert.ok(r.out.includes('FAIL no-backup: the log records no backup\nREHEARSAL_FAIL\n'),r.out);
|
||||
assert.equal(result(e),'result exit=3 no-backup the log records no backup to rehearse against; only a go that names --no-rehearsal goes on (nothing was stopped)');
|
||||
assert.ok(slog(e).includes('rehearse-store rc=0 BACKUP_STORE none'));
|
||||
assert.ok(!existsSync(join(e.data,'rehearsals')));assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));
|
||||
const n=slog(e).length;
|
||||
const s=await upgrade(e,C1,'--no-rehearsal');assert.equal(s.code,0,s.all);
|
||||
assert.deepEqual(names(e).slice(n),[...PRE,'rehearsal-skipped',...A2,'stage','render','confirm','args','result']);
|
||||
assert.equal(argsCopy(e),C1);assert.deepEqual(live(e),['active','active',C1,C1]);
|
||||
});
|
||||
|
||||
test('row 233: a real rehearsal after an upgrade names the log\'s newest backup; --no-rehearsal with a backup refuses rehearsal-required',async(t)=>{
|
||||
const e=await useed(t);
|
||||
const one=await upgrade(e,C1);assert.equal(one.code,0,one.all);
|
||||
const a1=lastAct(e);
|
||||
fault(e,'real-rehearse');
|
||||
const n=slog(e).length;
|
||||
// The stub copies fail the rehearsal right after it names the backup (copy-without-no-launch).
|
||||
const r=await upgrade(e,C,'--migrate');assert.equal(r.code,3,r.all);
|
||||
assert.ok(r.out.includes(`PASS backup-last: the log's newest backup is ${a1}\n`),r.out);
|
||||
assert.deepEqual(names(e).slice(n),[...PRE,'result']);assert.match(result(e),/^result exit=3 rehearsal-failed the rehearsal exited 1 /);
|
||||
rmSync(join(e.sd,'faults','real-rehearse'));
|
||||
const m=slog(e).length;
|
||||
const q=await upgrade(e,C,'--migrate','--no-rehearsal');assert.equal(q.code,3,q.all);
|
||||
assert.deepEqual(names(e).slice(m),[...PRE,'result']);
|
||||
assert.equal(result(e),'result exit=3 rehearsal-required the log records a backup and the rehearsal ran; --no-rehearsal is only for a stack with none (nothing was stopped)');
|
||||
assert.equal(argsCopy(e),C1);assert.deepEqual(live(e),['active','active',C1,C1]);
|
||||
});
|
||||
|
||||
// --- Before A1 and the args file ---
|
||||
|
||||
test('row 233: the args file must be the running user\'s, with no group or other write, one --copy, and the copy the units run',async(t)=>{
|
||||
const e=await useed(t);
|
||||
const tryArgs=async(text,mode,code,why)=>{
|
||||
writeFileSync(e.args,text);chmodSync(e.args,mode);
|
||||
const n=existsSync(e.slog)?slog(e).length:0;
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,3,`${why}: ${r.all}`);
|
||||
assert.match(slog(e).at(-1),new RegExp(`^result exit=3 ${code} `),why);
|
||||
assert.ok(!slog(e).slice(n).some((x)=>/^(stop|stage|render|confirm)/.test(x)),why);
|
||||
};
|
||||
await tryArgs(e.argsText,0o660,'args-unreadable','group write');
|
||||
await tryArgs(e.argsText,0o642,'args-unreadable','other write');
|
||||
await tryArgs(e.argsText.replace('--\n',`--copy\n${C1}\n--\n`),0o640,'args-unreadable','two --copy');
|
||||
await tryArgs(e.argsText.replace('--\n','--tester\n--\n'),0o640,'args-unreadable','an unknown flag');
|
||||
await tryArgs(e.argsText.replace(`--copy\n${B}\n`,`--copy\n${C1}\n`),0o640,'args-differ','the units run B');
|
||||
rmSync(e.args);symlinkSync(join(e.sd,'probe.sh'),e.args);
|
||||
const l=await upgrade(e,C1);assert.equal(l.code,3);assert.match(result(e),/^result exit=3 args-unreadable /);
|
||||
assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));assert.deepEqual(live(e),['active','active',B,B]);
|
||||
});
|
||||
|
||||
test('row 233: hostcheck (row 240): a nonzero --check or a missing file refuses profile-drift before the plan',async(t)=>{
|
||||
const e=await useed(t);
|
||||
fault(e,'hostcheck-rc','1');
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,3,r.all);
|
||||
assert.match(result(e),/^result exit=3 profile-drift /);
|
||||
assert.deepEqual(lines(join(e.sd,'hostcheck.log')),['--check']);
|
||||
assert.deepEqual(sudoKeys(e),['git-fetch','git-merge','stack-copy.sh']);
|
||||
e.denv.STACK_UPGRADE_HOSTCHECK=join(e.root,'no-hostcheck');
|
||||
const m=await upgrade(e,C1);assert.equal(m.code,3,m.all);
|
||||
assert.equal(result(e),`result exit=3 profile-drift no executable ${join(e.root,'no-hostcheck')} (row 240) (nothing was stopped)`);
|
||||
assert.ok(!sudoKeys(e).includes('activate-plan'));assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: the checkout must be at --sha (head-mismatch), and a commit the clone can\'t fast-forward to refuses fetch-failed',async(t)=>{
|
||||
const e=await useed(t);
|
||||
const r=await drive(e,C1,'upgrade','--instance','dev','--bundle',BUNDLE,'--sha',C,'--migrate',...TO);assert.equal(r.code,3,r.all);
|
||||
assert.equal(result(e),`result exit=3 head-mismatch this checkout, ${checkout(C1)}, is at ${C1}, not ${C} (nothing was stopped)`);
|
||||
const u=await drive(e,U,'upgrade','--instance','dev','--bundle',BUNDLE,'--sha',U,...TO);assert.equal(u.code,3,u.all);
|
||||
assert.match(result(e),/^result exit=3 fetch-failed .* fast-forward /);
|
||||
assert.ok(!sudoKeys(e).includes('activate-plan'));assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: usage exits 4 before any log or lock',async(t)=>{
|
||||
const e=await useed(t);
|
||||
for(const a of [[],['status'],['upgrade','--instance','dev','--bundle',BUNDLE],['upgrade','--instance','other','--bundle',BUNDLE,'--sha',C1],['upgrade','--instance','dev','--bundle','rel/x','--sha',C1],['upgrade','--instance','dev','--bundle',BUNDLE,'--sha',C1.slice(0,12)],['upgrade','--instance','dev','--bundle',BUNDLE,'--sha',C1,'--to',B],['upgrade','--instance','dev','--bundle',BUNDLE,'--sha',C1,'--rehearse-port','80'],['rollback','--instance','dev','--to',B,'--migrate'],['rollback','--instance','dev','--to',B,'--events','3'],['rollback','--instance','dev'],['rollback','--instance','dev','--to',B,'--restore','x']]){
|
||||
const r=await drive(e,C1,...a);assert.equal(r.code,4,a.join(' '));assert.match(r.err,/^usage: stack-upgrade.sh upgrade /,a.join(' '));
|
||||
}
|
||||
assert.ok(!existsSync(dirname(e.slog)));assert.ok(!existsSync(e.run));
|
||||
});
|
||||
Executable
+216
@@ -0,0 +1,216 @@
|
||||
#!/usr/bin/env bash
|
||||
# Row 240 (I4, #1746): the Case 1 host's drift stop for bwrap, r6 B2
|
||||
# (agents/darkwing/work/queue-195/DESIGN.md at b7239bb66, as docs/plans/2026-10-20_installer-rows.md
|
||||
# "I4 hostcheck" ships it). Deviations and decisions: agents/darkwing/work/queue-240/BUILD.md.
|
||||
#
|
||||
# case1-hostcheck.sh --check | --enforce | --upstream
|
||||
#
|
||||
# Installed as /usr/local/lib/mosaic-stack/hostcheck.sh, root:root 0755, with case1-pins.sh beside it,
|
||||
# root:root 0644. Three callers: stack-upgrade.sh A1 runs --check and refuses profile-drift on any
|
||||
# exit but 0; the apt hook /etc/apt/apt.conf.d/80mosaic-hostcheck runs --enforce after every dpkg run;
|
||||
# mosaic-hostcheck.service runs --enforce at boot, before the bus.
|
||||
#
|
||||
# Seven items, one line each, `ok N ...` or `FAIL N ...`:
|
||||
# 1. /etc/apparmor.d/bwrap-userns-restrict has the pinned sha256, and `dpkg -S` finds no owner.
|
||||
# 2. `aa-status --json` lists bwrap and unpriv_bwrap in enforce mode, and no mosaic-bwrap.
|
||||
# 3. `grep -rlw bwrap /etc/apparmor.d` lists that one file and nothing else.
|
||||
# 4. A bwrap child started as mosaic under the units' no-new-privs reads the label
|
||||
# `bwrap//&unpriv_bwrap (enforce)`, exactly.
|
||||
# 5. The statoverride on /usr/bin/bwrap reads `root mosaic 0750`, and stat agrees.
|
||||
# 6. kernel.apparmor_restrict_unprivileged_userns and fs.protected_hardlinks are 1.
|
||||
# 7. net.ipv4.ip_unprivileged_port_start is above the Console's --port in the installed unit, or
|
||||
# above 1023, the highest --port case2-unit.sh takes, while no unit is installed.
|
||||
# A `note` line says whether bubblewrap is the install pin; it is never a failure.
|
||||
#
|
||||
# --check exits 0 when all seven hold and 1 otherwise. --enforce runs the same items and exits 0. On
|
||||
# a failure, or while /etc/mosaic-stack/DRIFT exists, it stops bwrap: `apparmor_parser -R` on the
|
||||
# profile if it is loaded, then root:root 0700 on /usr/bin/bwrap. It appends the failed items to DRIFT
|
||||
# and one line to /var/log/mosaic-stack/hostcheck.log. Only root removes DRIFT, in a reviewed step.
|
||||
# It never writes the statoverride, because dpkg's lock may be held under apt's (r6).
|
||||
# --upstream downloads the apparmor-profiles candidate into a temp dir and compares its
|
||||
# bwrap-userns-restrict to the pin. A difference is printed and exits 0, because it refuses nothing;
|
||||
# it exits 1 when the download or the unpack fails.
|
||||
#
|
||||
# Exit 2 on usage or a refusal. Root takes no --root; without root it runs only with --root.
|
||||
#
|
||||
# Test seam, for packages/cli/tests/case1-hostcheck.test.mjs only:
|
||||
# --root DIR prefixes /etc, /proc/sys, /var/log and /usr/bin/bwrap
|
||||
set -uo pipefail
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
usage() { echo "usage: case1-hostcheck.sh --check | --enforce | --upstream" >&2; exit 2; }
|
||||
MODE="" ROOT=""
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--check|--enforce|--upstream) [ -z "$MODE" ] || usage; MODE="${1#--}"; shift ;;
|
||||
--root) [ $# -ge 2 ] && [[ "$2" =~ ^/[A-Za-z0-9/._+-]+$ ]] && [ -d "$2" ] || usage; ROOT="${2%/}"; shift 2 ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
done
|
||||
[ -n "$MODE" ] || usage
|
||||
if [ "$(id -u)" = 0 ]; then
|
||||
[ -z "$ROOT" ] || { echo "hostcheck: refusing: --root is a test seam, and root takes none" >&2; exit 2; }
|
||||
PATH=/usr/sbin:/usr/bin:/sbin:/bin
|
||||
else
|
||||
[ -n "$ROOT" ] || { echo "hostcheck: refusing: run it as root" >&2; exit 2; }
|
||||
fi
|
||||
|
||||
PIN_PROFILES_VERSION="" PIN_PROFILE_SHA256="" PIN_BUBBLEWRAP_VERSION=""
|
||||
# shellcheck source=case1-pins.sh
|
||||
[ -f "$HERE/case1-pins.sh" ] && . "$HERE/case1-pins.sh"
|
||||
PROFILE=/etc/apparmor.d/bwrap-userns-restrict
|
||||
BWRAP=/usr/bin/bwrap
|
||||
DRIFT="$ROOT/etc/mosaic-stack/DRIFT"
|
||||
LOGF="$ROOT/var/log/mosaic-stack/hostcheck.log"
|
||||
LABEL='bwrap//&unpriv_bwrap (enforce)'
|
||||
stamp() { date -u +%Y-%m-%dT%H:%M:%SZ; }
|
||||
|
||||
FAILED=() LOADED=1
|
||||
pass() { echo "ok $*"; }
|
||||
fail() { echo "FAIL $*"; FAILED+=("$*"); }
|
||||
|
||||
item1() {
|
||||
local sum owner rc
|
||||
[[ "$PIN_PROFILE_SHA256" =~ ^[0-9a-f]{64}$ ]] || { fail "1 no profile pin in $HERE/case1-pins.sh"; return; }
|
||||
sum="$(sha256sum < "$ROOT$PROFILE" 2>/dev/null | cut -d' ' -f1)"
|
||||
[ "$sum" = "$PIN_PROFILE_SHA256" ] || { fail "1 $PROFILE sha256 is ${sum:-unreadable}, not the pin $PIN_PROFILE_SHA256"; return; }
|
||||
owner="$(dpkg -S "$PROFILE" 2>/dev/null)"; rc=$?
|
||||
case "$rc" in
|
||||
1) pass "1 $PROFILE matches the pin from apparmor-profiles $PIN_PROFILES_VERSION, and no package owns it" ;;
|
||||
0) fail "1 $PROFILE is owned by a package: $owner" ;;
|
||||
*) fail "1 dpkg -S $PROFILE exited $rc" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
item2() {
|
||||
local json list
|
||||
if ! json="$(aa-status --json 2>/dev/null)"; then fail "2 aa-status --json failed"; return; fi
|
||||
# The profiles object holds name-to-mode pairs and nothing nested; one pair per line.
|
||||
list="$(sed -n 's/.*"profiles": *{\([^}]*\)}.*/\1/p' <<< "$json" | tr ',' '\n' | sed 's/^ *//; s/ *$//')"
|
||||
grep -Eq '^"(bwrap|unpriv_bwrap)":' <<< "$list" || LOADED=0
|
||||
grep -qxE '"bwrap": *"enforce"' <<< "$list" || { fail "2 aa-status lists no bwrap profile in enforce mode"; return; }
|
||||
grep -qxE '"unpriv_bwrap": *"enforce"' <<< "$list" || { fail "2 aa-status lists no unpriv_bwrap profile in enforce mode"; return; }
|
||||
! grep -qE '^"mosaic-bwrap":' <<< "$list" || { fail "2 aa-status lists a mosaic-bwrap profile"; return; }
|
||||
pass "2 bwrap and unpriv_bwrap are in enforce mode, and no mosaic-bwrap is loaded"
|
||||
}
|
||||
|
||||
item3() {
|
||||
local found
|
||||
found="$(grep -rlw bwrap "$ROOT/etc/apparmor.d" 2>/dev/null | sort)"
|
||||
[ "$found" = "$ROOT$PROFILE" ] || { fail "3 the files under /etc/apparmor.d that name bwrap are not $PROFILE alone: $(tr '\n' ' ' <<< "$found")"; return; }
|
||||
pass "3 $PROFILE is the only file under /etc/apparmor.d that names bwrap"
|
||||
}
|
||||
|
||||
item4() {
|
||||
local label
|
||||
label="$(timeout 60 systemd-run --wait --pipe --uid=mosaic --gid=mosaic -p NoNewPrivileges=yes \
|
||||
-p ProtectSystem=strict -p ProtectHome=yes -p PrivateTmp=yes \
|
||||
"$BWRAP" --unshare-user --unshare-pid --ro-bind / / cat /proc/self/attr/current 2>/dev/null)"
|
||||
[ "$label" = "$LABEL" ] || { fail "4 a bwrap child under no-new-privs has the label '$label', not '$LABEL'"; return; }
|
||||
pass "4 a bwrap child under no-new-privs has the label '$LABEL'"
|
||||
}
|
||||
|
||||
item5() {
|
||||
local so st
|
||||
so="$(dpkg-statoverride --list "$BWRAP" 2>/dev/null)"
|
||||
st="$(stat -c '%U %G %a' "$ROOT$BWRAP" 2>/dev/null)"
|
||||
[ "$so" = "root mosaic 0750 $BWRAP" ] || { fail "5 the statoverride on $BWRAP reads '$so', not 'root mosaic 0750'"; return; }
|
||||
[ "$st" = "root mosaic 750" ] || { fail "5 $BWRAP is '$st', not 'root mosaic 750'"; return; }
|
||||
pass "5 $BWRAP is root mosaic 0750, by the statoverride and by stat"
|
||||
}
|
||||
|
||||
item6() {
|
||||
local userns links
|
||||
userns="$(cat "$ROOT/proc/sys/kernel/apparmor_restrict_unprivileged_userns" 2>/dev/null)"
|
||||
links="$(cat "$ROOT/proc/sys/fs/protected_hardlinks" 2>/dev/null)"
|
||||
[ "$userns" = 1 ] && [ "$links" = 1 ] ||
|
||||
{ fail "6 kernel.apparmor_restrict_unprivileged_userns is '$userns' and fs.protected_hardlinks is '$links', not 1 and 1"; return; }
|
||||
pass "6 kernel.apparmor_restrict_unprivileged_userns and fs.protected_hardlinks are 1"
|
||||
}
|
||||
|
||||
item7() {
|
||||
local floor unit="$ROOT/etc/systemd/system/mosaic-webui.service" ports port=0 p from
|
||||
floor="$(cat "$ROOT/proc/sys/net/ipv4/ip_unprivileged_port_start" 2>/dev/null)"
|
||||
[[ "$floor" =~ ^[0-9]{1,5}$ ]] || { fail "7 net.ipv4.ip_unprivileged_port_start is '$floor', not a number"; return; }
|
||||
if [ -e "$unit" ]; then
|
||||
ports="$(sed -n 's/^ExecStart=//p' "$unit" 2>/dev/null | grep -oE ' --port [0-9]{1,5}( |$)' | tr -dc '0-9\n')"
|
||||
[ -n "$ports" ] || { fail "7 /etc/systemd/system/mosaic-webui.service has no --port"; return; }
|
||||
for p in $ports; do [ "$((10#$p))" -le "$port" ] || port="$((10#$p))"; done
|
||||
from="the Console's --port $port"
|
||||
else
|
||||
port=1023 from="1023, because no Console unit is installed"
|
||||
fi
|
||||
[ "$((10#$floor))" -gt "$port" ] || { fail "7 net.ipv4.ip_unprivileged_port_start is $floor, not above $from"; return; }
|
||||
pass "7 net.ipv4.ip_unprivileged_port_start is $floor, above $from"
|
||||
}
|
||||
|
||||
items() {
|
||||
local v
|
||||
item1; item2; item3; item4; item5; item6; item7
|
||||
v="$(dpkg -s bubblewrap 2>/dev/null | sed -n 's/^Version: //p')"
|
||||
if [ "$v" = "$PIN_BUBBLEWRAP_VERSION" ]; then
|
||||
echo "note bubblewrap $v is the install pin"
|
||||
else
|
||||
echo "note bubblewrap ${v:-unknown} is not the install pin ${PIN_BUBBLEWRAP_VERSION:-unknown}, which is not a drift item"
|
||||
fi
|
||||
}
|
||||
|
||||
# The stop: unload the profile if it is loaded, then close bwrap to root. Prints the results.
|
||||
stop_bwrap() {
|
||||
local unload=skipped chown_rc=0 chmod_rc=0
|
||||
if [ "$LOADED" = 1 ]; then
|
||||
if [ -f "$ROOT$PROFILE" ]; then apparmor_parser -R "$ROOT$PROFILE" >/dev/null 2>&1; unload="rc=$?"; else unload=no-file; fi
|
||||
fi
|
||||
chown root:root "$ROOT$BWRAP" || chown_rc=$?
|
||||
chmod 0700 "$ROOT$BWRAP" || chmod_rc=$?
|
||||
echo "unload=$unload chown=$chown_rc chmod=$chmod_rc"
|
||||
}
|
||||
|
||||
enforce() {
|
||||
local had=0 now result f ids=""
|
||||
[ ! -e "$DRIFT" ] || had=1
|
||||
items
|
||||
[ ${#FAILED[@]} -gt 0 ] || [ "$had" = 1 ] || { echo "hostcheck: all seven hold"; exit 0; }
|
||||
umask 027
|
||||
now="$(stamp)"
|
||||
if [ ${#FAILED[@]} -gt 0 ]; then
|
||||
mkdir -p "$(dirname "$DRIFT")" && for f in "${FAILED[@]}"; do echo "$now $f"; done >> "$DRIFT" ||
|
||||
echo "hostcheck: cannot write $DRIFT" >&2
|
||||
fi
|
||||
for f in "${FAILED[@]}"; do ids+="${ids:+,}${f%% *}"; done
|
||||
result="$(stop_bwrap)"
|
||||
mkdir -p "$(dirname "$LOGF")" && echo "$now enforce stop items=${ids:-none} drift-before=$had $result" >> "$LOGF" ||
|
||||
echo "hostcheck: cannot write $LOGF" >&2
|
||||
echo "hostcheck: bwrap stopped ($result); only root removes $DRIFT"
|
||||
exit 0
|
||||
}
|
||||
|
||||
upstream() {
|
||||
local tmp deb ver f sum
|
||||
tmp="$(mktemp -d)" || exit 1
|
||||
trap 'rm -rf "$tmp"' EXIT
|
||||
(cd "$tmp" && apt-get download apparmor-profiles) >/dev/null 2>&1 || { echo "hostcheck: upstream: apt-get download apparmor-profiles failed" >&2; exit 1; }
|
||||
deb=("$tmp"/apparmor-profiles_*.deb)
|
||||
[ ${#deb[@]} = 1 ] && [ -f "${deb[0]}" ] || { echo "hostcheck: upstream: no single apparmor-profiles deb in the download" >&2; exit 1; }
|
||||
ver="$(dpkg-deb -f "${deb[0]}" Version 2>/dev/null)" && dpkg-deb -x "${deb[0]}" "$tmp/x" >/dev/null 2>&1 ||
|
||||
{ echo "hostcheck: upstream: cannot unpack ${deb[0]##*/}" >&2; exit 1; }
|
||||
f="$tmp/x/usr/share/apparmor/extra-profiles/bwrap-userns-restrict"
|
||||
if [ ! -f "$f" ]; then
|
||||
echo "upstream: DIFFERS: apparmor-profiles $ver has no bwrap-userns-restrict; the pin is $PIN_PROFILE_SHA256 from $PIN_PROFILES_VERSION. Send it to Mos as a new-pin request."
|
||||
exit 0
|
||||
fi
|
||||
sum="$(sha256sum < "$f" | cut -d' ' -f1)"
|
||||
if [ "$sum" = "$PIN_PROFILE_SHA256" ]; then
|
||||
echo "upstream: apparmor-profiles $ver's bwrap-userns-restrict matches the pin from $PIN_PROFILES_VERSION"
|
||||
else
|
||||
echo "upstream: DIFFERS: apparmor-profiles $ver's bwrap-userns-restrict has sha256 $sum; the pin is $PIN_PROFILE_SHA256 from $PIN_PROFILES_VERSION. Send it to Mos as a new-pin request."
|
||||
diff -u --label "installed $PROFILE" --label "apparmor-profiles $ver" "$ROOT$PROFILE" "$f" || true
|
||||
fi
|
||||
exit 0
|
||||
}
|
||||
|
||||
case "$MODE" in
|
||||
check) items; [ ${#FAILED[@]} -eq 0 ] || exit 1; exit 0 ;;
|
||||
enforce) enforce ;;
|
||||
upstream) upstream ;;
|
||||
esac
|
||||
@@ -0,0 +1,12 @@
|
||||
# Row 240 (I4, #1746): the Case 1 host pins, sourced by case1-hostcheck.sh, which is installed as
|
||||
# /usr/local/lib/mosaic-stack/hostcheck.sh with this file beside it. r6 B2
|
||||
# (agents/darkwing/work/queue-195/DESIGN.md at b7239bb66). Changing a pin is a reviewed commit and
|
||||
# its own round.
|
||||
#
|
||||
# The AppArmor profile 2a' installs as /etc/apparmor.d/bwrap-userns-restrict, taken from
|
||||
# /usr/share/apparmor/extra-profiles/bwrap-userns-restrict in this apparmor-profiles version.
|
||||
PIN_PROFILES_VERSION=4.0.1really4.0.1-0ubuntu0.24.04.9
|
||||
PIN_PROFILE_SHA256=11d39094f044f0cda0febb3ad517b830301da6b2ce929664af09ee9e4dd264f9
|
||||
# The bubblewrap version 2a' installs. An install pin, not a drift item: a security update keeps the
|
||||
# statoverride, and the profile attaches by path. hostcheck prints a line when they differ.
|
||||
PIN_BUBBLEWRAP_VERSION=0.9.0-1ubuntu0.3
|
||||
+62
-13
@@ -3,21 +3,25 @@
|
||||
# copy creates files only; wrapper is a separately Mos-gated live step.
|
||||
# wrapper and check take --instance NAME (row 190): the units mosaic-bus-NAME@ and mosaic-webui-NAME
|
||||
# and the wrapper $BIN/mosaic-NAME, which sets NAME's MOSAIC_CONFIG. check needs --business for NAME.
|
||||
# Row 232: check --units external reads root's system units. copy --bundle makes the copy from a git
|
||||
# bundle on a host with no checkout, and wrapper --create writes the first wrapper on a new host.
|
||||
set -euo pipefail
|
||||
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
COPIES="${XDG_DATA_HOME:-$HOME/.local/share}/mosaic-stack/copies"
|
||||
BIN="$HOME/.npm-global/bin"
|
||||
BUSINESS=mosaic-stack BUSINESS_SET=0
|
||||
ALLOW_WORKTREE=0
|
||||
ALLOW_WORKTREE=0 UNITS=user BUNDLE="" CREATE=0 CLEAN=""
|
||||
MARKER='# mosaic-cli wrapper (Q81, agents/sage/work/q81/mosaic-cli.sh)'
|
||||
usage() { echo 'usage: stack-copy.sh copy COMMIT [--copies DIR] [--allow-worktree] | wrapper ID [--instance NAME] [--copies DIR] [--bin DIR] | check [--instance NAME] [--copies DIR] [--bin DIR] [--business ID]' >&2; exit "${1:-4}"; }
|
||||
usage() { echo 'usage: stack-copy.sh copy COMMIT [--copies DIR] [--allow-worktree] | copy --bundle FILE SHA [--copies DIR] | wrapper ID [--instance NAME] [--copies DIR] [--bin DIR] [--create] | check [--instance NAME] [--copies DIR] [--bin DIR] [--business ID] [--units user|external]' >&2; exit "${1:-4}"; }
|
||||
die() { echo "stack-copy: $*" >&2; exit 1; }
|
||||
plain_path() { case "$1" in /*) ;; *) die "$2 must be an absolute path: $1" ;; esac; case "$1" in *[!A-Za-z0-9/._+-]*) die "$2 contains a character the unit cannot carry: $1" ;; esac; }
|
||||
parse() { while [ $# -gt 0 ]; do case "$1" in
|
||||
--copies) [ $# -ge 2 ] || usage; COPIES="$2"; shift 2;;
|
||||
--bin) [ "$CMD" != copy ] && [ $# -ge 2 ] || usage; BIN="$2";shift 2;;
|
||||
--business) [ "$CMD" = check ] && [ $# -ge 2 ] || usage; BUSINESS="$2";BUSINESS_SET=1;shift 2;;
|
||||
--allow-worktree) [ "$CMD" = copy ] || usage; ALLOW_WORKTREE=1;shift;;
|
||||
--allow-worktree) [ "$CMD" = copy ] && [ -z "$BUNDLE" ] || usage; ALLOW_WORKTREE=1;shift;;
|
||||
--units) [ "$CMD" = check ] && [ $# -ge 2 ] || usage; case "$2" in user|external) UNITS="$2";; *) usage;; esac; shift 2;;
|
||||
--create) [ "$CMD" = wrapper ] || usage; CREATE=1;shift;;
|
||||
*) usage;; esac; done; plain_path "$COPIES" 'copies directory'; }
|
||||
copy_root() { [[ "$ID" =~ ^[0-9a-f]{40}$ ]] || die '--copy must be a full 40-character commit id'; [ -f "$COPIES/$ID/scripts/mosaic" ] || die "no copy $ID in $COPIES"; echo "$COPIES/$ID"; }
|
||||
# Closure of scripts/mosaic, bus process/notifier/launcher, Console serve,
|
||||
@@ -44,16 +48,22 @@ reachable() {
|
||||
copy_commit() {
|
||||
[ -d "$REPO/.git" ] || [ "$ALLOW_WORKTREE" = 1 ] || die "refusing: $REPO is not the canonical checkout (its .git is not a directory); run copy from the canonical checkout"
|
||||
[[ "$ID" =~ ^[A-Za-z0-9][A-Za-z0-9._/-]*$ ]] || die "commit must be a commit id or branch name: $ID"
|
||||
local full target tmp
|
||||
local full
|
||||
full="$(git -C "$REPO" rev-parse --verify --quiet "$ID^{commit}")" || die "not a commit: $ID"
|
||||
reachable "$full"
|
||||
copy_tree "$REPO" "$full"
|
||||
}
|
||||
# The copy of commit $2 from repository $1. CLEAN names a bundle's temporary repository, which the
|
||||
# trap removes with the temporary copy.
|
||||
copy_tree() {
|
||||
local full="$2" target tmp
|
||||
target="$COPIES/$full"
|
||||
[ ! -L "$target" ] || die "refusing: copy target is a symlink: $target"
|
||||
if [ -e "$target" ]; then echo "exists: $target"; return; fi
|
||||
mkdir -p "$COPIES"
|
||||
tmp="$(mktemp -d "$COPIES/.$full.XXXXXX")"
|
||||
trap "chmod -R u+w '$tmp' 2>/dev/null || true; rm -rf '$tmp'" EXIT
|
||||
git -C "$REPO" archive --format=tar "$full" -- "${COPY_PATHS[@]}" | tar -xf - -C "$tmp" || die "git archive failed; no copy written"
|
||||
trap "chmod -R u+w '$tmp' 2>/dev/null || true; rm -rf '$tmp'$CLEAN" EXIT
|
||||
git -C "$1" archive --format=tar "$full" -- "${COPY_PATHS[@]}" | tar -xf - -C "$tmp" || die "git archive failed; no copy written"
|
||||
[ -f "$tmp/package.json" ] && [ -f "$tmp/package-lock.json" ] || die 'package.json or package-lock.json missing in temp copy; npm ci not run'
|
||||
(cd "$tmp" && npm ci --no-audit --no-fund) >&2 || die 'npm ci failed; no copy written'
|
||||
find "$tmp" -type f -exec chmod a-w {} +
|
||||
@@ -65,19 +75,53 @@ copy_commit() {
|
||||
trap - EXIT
|
||||
echo "copied: $target"
|
||||
}
|
||||
# Row 232: the copy of SHA from a git bundle, for a host with no refactor checkout. The bundle must
|
||||
# verify, carry refs/heads/refactor, and reach SHA from it, as reachable asks of a checkout. Prints
|
||||
# the RELEASE value at SHA and the bundle's sha256, the release identity on such a host.
|
||||
copy_bundle() {
|
||||
[[ "$ID" =~ ^[0-9a-f]{40}$ ]] || die "--bundle needs a full 40-character commit id: $ID"
|
||||
[ -f "$BUNDLE" ] && [ ! -L "$BUNDLE" ] || die "refusing: bundle is absent, not regular, or a symlink: $BUNDLE"
|
||||
local g sum release heads
|
||||
case "$BUNDLE" in /*) ;; *) BUNDLE="$PWD/$BUNDLE";; esac
|
||||
g="$(mktemp -d)"
|
||||
CLEAN=" '$g'"
|
||||
trap "rm -rf '$g'" EXIT
|
||||
git init -q --bare "$g" || die 'git init failed; no copy written'
|
||||
git -C "$g" bundle verify -q "$BUNDLE" >/dev/null 2>&1 || die "refusing: git bundle verify failed: $BUNDLE"
|
||||
heads="$(git -C "$g" bundle list-heads "$BUNDLE" refs/heads/refactor)" || die "refusing: cannot list the bundle's heads: $BUNDLE"
|
||||
grep -qx '[0-9a-f]\{40\} refs/heads/refactor' <<< "$heads" || die "refusing: the bundle has no refs/heads/refactor: $BUNDLE"
|
||||
# Every ref, so a SHA the bundle carries on another branch reaches the reachability check.
|
||||
git -C "$g" fetch -q "$BUNDLE" 'refs/*:refs/*' 2>/dev/null || die "refusing: cannot fetch from $BUNDLE"
|
||||
git -C "$g" cat-file -e "$ID^{commit}" 2>/dev/null || die "refusing: $ID is not in the bundle"
|
||||
git -C "$g" merge-base --is-ancestor "$ID" refs/heads/refactor || die "refusing: $ID is not reachable from the bundle's refs/heads/refactor"
|
||||
release="$(git -C "$g" show "$ID:RELEASE")" || die "no RELEASE at $ID"
|
||||
sum="$(sha256sum < "$BUNDLE" | cut -d' ' -f1)"
|
||||
copy_tree "$g" "$ID"
|
||||
rm -rf "$g"; trap - EXIT
|
||||
echo "release: $release"
|
||||
echo "bundle sha256: $sum"
|
||||
}
|
||||
# Row 232: --create writes the wrapper only where none is. ln refuses any existing path, a symlink
|
||||
# included, so this is the one check, and a wrapper that appears mid-run is never replaced.
|
||||
place() {
|
||||
if [ "$CREATE" = 1 ]; then ln "$1" "$2" 2>/dev/null || die "refusing: --create and a wrapper exists: $2"; rm -f "$1"
|
||||
else mv -f "$1" "$2"; fi
|
||||
}
|
||||
wrapper() {
|
||||
[ -z "$MOSAIC_UNIT_SUFFIX" ] || { wrapper_instance; return; }
|
||||
plain_path "$BIN" 'bin directory'
|
||||
local root file lease tmp
|
||||
root="$(copy_root)";file="$BIN/mosaic"
|
||||
[ -f "$file" ] && [ ! -L "$file" ] || die "refusing: wrapper is absent, not regular, or a symlink: $file"
|
||||
[ "$(sed -n '2p' "$file")" = "$MARKER" ] || die "refusing: foreign wrapper (missing Q81 marker on line 2): $file"
|
||||
lease='[ "${1:-}" = __lease-capability ] && exec "'"$BIN"'/mosaic-old" "$@"'
|
||||
[ "$(grep -Fxc "$lease" "$file" || true)" = 1 ] || die 'refusing: Q81 lease line missing or ambiguous'
|
||||
if [ "$CREATE" = 0 ]; then
|
||||
[ -f "$file" ] && [ ! -L "$file" ] || die "refusing: wrapper is absent, not regular, or a symlink: $file"
|
||||
[ "$(sed -n '2p' "$file")" = "$MARKER" ] || die "refusing: foreign wrapper (missing Q81 marker on line 2): $file"
|
||||
[ "$(grep -Fxc "$lease" "$file" || true)" = 1 ] || die 'refusing: Q81 lease line missing or ambiguous'
|
||||
fi
|
||||
tmp="$(mktemp "$BIN/.mosaic.XXXXXX")"
|
||||
trap 'rm -f "$tmp"' EXIT
|
||||
printf '#!/usr/bin/env bash\n%s\n# CLI runs from pinned copy: %s\n# The Q81 lease probe still answers at mosaic-old.\n%s\nexec "%s/scripts/mosaic" "$@"\n' "$MARKER" "$root" "$lease" "$root" > "$tmp"
|
||||
chmod 0755 "$tmp";mv -f "$tmp" "$file";trap - EXIT
|
||||
chmod 0755 "$tmp";place "$tmp" "$file";trap - EXIT
|
||||
echo "wrapper: $root"
|
||||
}
|
||||
# Row 190: $BIN/mosaic-NAME. It creates the file or replaces one with its own marker, never another.
|
||||
@@ -93,12 +137,14 @@ wrapper_instance() {
|
||||
tmp="$(mktemp "$BIN/.$WRAPPER_NAME.XXXXXX")"
|
||||
trap 'rm -f "$tmp"' EXIT
|
||||
printf '#!/usr/bin/env bash\n%s\n# CLI runs from pinned copy: %s\nexport MOSAIC_CONFIG='"'"'%s'"'"'\nexec "%s/scripts/mosaic" "$@"\n' "$INSTANCE_MARKER" "$root" "$MOSAIC_INSTANCE_CONFIG" "$root" > "$tmp"
|
||||
chmod 0755 "$tmp";mv -f "$tmp" "$file";trap - EXIT
|
||||
chmod 0755 "$tmp";place "$tmp" "$file";trap - EXIT
|
||||
echo "wrapper: $root"
|
||||
}
|
||||
# --units external reads root's system units, which the stack user can read and not change.
|
||||
units_show() { if [ "$UNITS" = external ]; then systemctl show "$@"; else systemctl --user show "$@"; fi; }
|
||||
unit_root() {
|
||||
local out load root exec_start
|
||||
out="$(systemctl --user show "$1" -p LoadState -p WorkingDirectory -p ExecStart 2>/dev/null)" || { echo 'absent (systemctl read failed)';return; }
|
||||
out="$(units_show "$1" -p LoadState -p WorkingDirectory -p ExecStart 2>/dev/null)" || { echo 'absent (systemctl read failed)';return; }
|
||||
load="$(sed -n 's/^LoadState=//p' <<< "$out")"; root="$(sed -n 's/^WorkingDirectory=//p' <<< "$out")"
|
||||
exec_start="$(sed -n 's/^ExecStart=//p' <<< "$out")"
|
||||
if [ "$load" != loaded ] || [ -z "$root" ];then echo absent;return;fi
|
||||
@@ -133,7 +179,10 @@ CMD="$1";shift
|
||||
case "$CMD" in check|wrapper) instance_select;; *) [ -z "$INSTANCE_FLAG" ] || usage;; esac
|
||||
INSTANCE_MARKER="# mosaic-cli wrapper, instance ${MOSAIC_INSTANCE:-dev} (row 190)"
|
||||
case "$CMD" in
|
||||
copy|wrapper) [ $# -gt 0 ] || usage;ID="$1";shift;case "$ID" in -*)usage;;esac;parse "$@";if [ "$CMD" = copy ];then copy_commit;else wrapper;fi;;
|
||||
copy|wrapper) [ $# -gt 0 ] || usage
|
||||
if [ "$CMD" = copy ] && [ "$1" = --bundle ];then [ $# -ge 3 ] && [ -n "$2" ] || usage;BUNDLE="$2";shift 2;fi
|
||||
ID="$1";shift;case "$ID" in -*)usage;;esac;parse "$@"
|
||||
if [ -n "$BUNDLE" ];then copy_bundle;elif [ "$CMD" = copy ];then copy_commit;else wrapper;fi;;
|
||||
check) parse "$@";[ -z "$MOSAIC_UNIT_SUFFIX" ] || [ "$BUSINESS_SET" = 1 ] || usage;check;;
|
||||
-h|--help) usage 0;;
|
||||
*) usage;;esac
|
||||
|
||||
+487
-63
@@ -19,6 +19,13 @@
|
||||
# rehearse (D2, row 163) stops and starts no unit and writes only its own rehearsal directory, but
|
||||
# it reads the live config and a backup of the live store, so Jason or Mos runs it.
|
||||
#
|
||||
# --units external (row 232, #1734; r6 section 4 at agents/darkwing/work/queue-195/DESIGN.md) is
|
||||
# for a host whose units are root's system units. This script then reads those units and never
|
||||
# starts or stops one: root stops them, activate or rollback stages the copy (A3), root installs
|
||||
# and starts them, and confirm gates them and writes the pointer (A5). revert and confirm
|
||||
# --rollback put the old copy back when a step fails (R2, R4), and gate checks a copy root
|
||||
# restarted with no activation open (A2f). Without it, everything runs as before.
|
||||
#
|
||||
# Most functions below run as `f || handler`, and bash ignores set -e inside such a call, so
|
||||
# every step that can fail carries its own `|| return 1`.
|
||||
set -euo pipefail
|
||||
@@ -32,6 +39,7 @@ BIN="$HOME/.npm-global/bin"
|
||||
UNIT_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user"
|
||||
TIMEOUT=60 WANT_PM=0 AFTER_HARNESS=0 MIGRATE=0 PLAN=0 RESTORE="" EVENTS="" TESTER_PORT=""
|
||||
PROOF_HEADER="" PROOF_FILE="" NO_PROOF=0 NO_SEND=0 SSO=0 R_BACKUP="" R_PORT=""
|
||||
UNITS=user TO_FLAG="" EXPECT_FROM="" SINCE="" ROLLBACK_OLD="" CAUSE=""
|
||||
|
||||
usage() {
|
||||
cat >&2 <<'USAGE'
|
||||
@@ -42,11 +50,18 @@ usage: stack-release.sh adopt BUSINESS
|
||||
stack-release.sh rollback BUSINESS [--restore ACTIVATION [--events N]] [--timeout S] [--plan]
|
||||
stack-release.sh status BUSINESS
|
||||
stack-release.sh rehearse ID BUSINESS --backup ACTIVATION|PATH --port N [--timeout S]
|
||||
common options: --copies DIR, --bin DIR, --instance NAME
|
||||
stack-release.sh confirm ID BUSINESS --units external [--rollback OLD --cause A3|A4|A5] [--timeout S]
|
||||
stack-release.sh revert ID BUSINESS --units external
|
||||
stack-release.sh gate ID BUSINESS --units external --since EPOCH [--timeout S]
|
||||
common options: --copies DIR, --bin DIR, --instance NAME, --units user|external
|
||||
--instance NAME (row 190) runs on stack instance NAME: its config, data root, units and the wrapper
|
||||
mosaic-NAME. Without it, the default instance, as before.
|
||||
--events N is the total of the per-table counts the post-backup-events refusal prints: events and
|
||||
every table in COUNTED.
|
||||
--units external (row 232) reads root's system units and starts or stops none. adopt, activate,
|
||||
rollback, status, confirm, revert and gate take it; rollback then needs --to ID, the previous copy.
|
||||
activate and rollback --plan take --expect-from OLD. --pm, --pm-after-harness, --tester-port, the
|
||||
proof flags, --no-proof, --no-send, --sso and pm-on are not for it.
|
||||
USAGE
|
||||
exit "${1:-4}"
|
||||
}
|
||||
@@ -58,7 +73,27 @@ now_ms() { date -u +%Y-%m-%dT%H:%M:%S.%3NZ; }
|
||||
jline() {
|
||||
node -e 'const a=process.argv.slice(1),o={};for(let i=0;i<a.length;i+=2)o[a[i]]=a[i+1]==="@null"?null:a[i+1];process.stdout.write(JSON.stringify(o)+"\n")' -- "$@"
|
||||
}
|
||||
log_line() { jline at "$(now)" op "$OP" from "${FROM:-@null}" to "${TO:-@null}" backup "${BACKUP:-@null}" activation "${ACT:-@null}" "$@" >> "$LOG"; }
|
||||
log_line() {
|
||||
[ "$UNITS" = user ] || set -- "$@" units external
|
||||
jline at "$(now)" op "$OP" from "${FROM:-@null}" to "${TO:-@null}" backup "${BACKUP:-@null}" activation "${ACT:-@null}" "$@" >> "$LOG"
|
||||
}
|
||||
|
||||
# --- The units backend (row 232) ---
|
||||
|
||||
# Every systemctl, journalctl and systemd-analyze call goes through these three. user is the
|
||||
# systemd user manager, as before. external reads root's system units, which needs no privilege,
|
||||
# and refuses any verb that would change one.
|
||||
sc() {
|
||||
if [ "$UNITS" = user ]; then systemctl --user "$@"; return; fi
|
||||
case "$1" in
|
||||
show|is-active|list-units) systemctl "$@" ;;
|
||||
*) echo "stack-release: --units external never runs systemctl $1; root does" >&2; return 1 ;;
|
||||
esac
|
||||
}
|
||||
# The bus unit's journal is a user journal only: under external it lands in the system journal,
|
||||
# which the stack's user can't read (r6 section 4), so the external PM check reads launches list.
|
||||
jctl() { [ "$UNITS" = user ] || return 1; journalctl --user "$@"; }
|
||||
analyze() { [ "$UNITS" = user ] || return 1; systemd-analyze --user "$@"; }
|
||||
|
||||
# A refusal before anything stopped: exit 3, and one log line with the code, except for
|
||||
# run-locked (the lock holder may be writing the log) and in --plan, which writes nothing.
|
||||
@@ -83,7 +118,7 @@ parse_opts() {
|
||||
case "$1" in
|
||||
--copies) [ $# -ge 2 ] || usage; COPIES="$2"; shift 2 ;;
|
||||
--bin) [ $# -ge 2 ] || usage; BIN="$2"; shift 2 ;;
|
||||
--timeout) [ "$OP" != status ] && [ "$OP" != adopt ] && [ $# -ge 2 ] && [[ "$2" =~ ^[1-9][0-9]{0,4}$ ]] || usage; TIMEOUT="$2"; shift 2 ;;
|
||||
--timeout) [ "$OP" != status ] && [ "$OP" != adopt ] && [ "$OP" != revert ] && [ $# -ge 2 ] && [[ "$2" =~ ^[1-9][0-9]{0,4}$ ]] || usage; TIMEOUT="$2"; shift 2 ;;
|
||||
--plan) [ "$OP" = activate ] || [ "$OP" = rollback ] || usage; PLAN=1; shift ;;
|
||||
--pm) [ "$OP" = activate ] || usage; WANT_PM=1; shift ;;
|
||||
--pm-after-harness) [ "$OP" = activate ] || usage; AFTER_HARNESS=1; shift ;;
|
||||
@@ -95,12 +130,30 @@ parse_opts() {
|
||||
--sso) [ "$OP" = activate ] || usage; SSO=1; shift ;;
|
||||
--migrate) [ "$OP" = activate ] || usage; MIGRATE=1; shift ;;
|
||||
--restore) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])?$ ]] || usage; RESTORE="$2"; shift 2 ;;
|
||||
--backup) [ "$OP" = rehearse ] && [ $# -ge 2 ] && { [[ "$2" =~ ^[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])?$ ]] || [[ "$2" =~ ^/[A-Za-z0-9/._+-]*$ ]]; } || usage; R_BACKUP="$2"; shift 2 ;;
|
||||
--backup) [ "$OP" = rehearse ] && [ $# -ge 2 ] && { [ "$2" = last ] || [[ "$2" =~ ^[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])?$ ]] || [[ "$2" =~ ^/[A-Za-z0-9/._+-]*$ ]]; } || usage; R_BACKUP="$2"; shift 2 ;;
|
||||
--port) [ "$OP" = rehearse ] && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9]{1,5}$ ]] && [ "$((10#$2))" -ge 1024 ] && [ "$((10#$2))" -le 65535 ] || usage; R_PORT="$((10#$2))"; shift 2 ;;
|
||||
--events) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ ^(0|[1-9][0-9]*)$ ]] || usage; EVENTS="$2"; shift 2 ;;
|
||||
--units) [ "$OP" != rehearse ] && [ $# -ge 2 ] && { [ "$2" = user ] || [ "$2" = external ]; } || usage; UNITS="$2"; shift 2 ;;
|
||||
--to) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9a-f]{40}$ ]] || usage; TO_FLAG="$2"; shift 2 ;;
|
||||
--expect-from) { [ "$OP" = activate ] || [ "$OP" = rollback ]; } && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9a-f]{40}$ ]] || usage; EXPECT_FROM="$2"; shift 2 ;;
|
||||
--since) [ "$OP" = gate ] && [ $# -ge 2 ] && [[ "$2" =~ ^(0|[1-9][0-9]{0,11})$ ]] || usage; SINCE="$2"; shift 2 ;;
|
||||
--rollback) [ "$OP" = confirm ] && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9a-f]{40}$ ]] || usage; ROLLBACK_OLD="$2"; shift 2 ;;
|
||||
--cause) [ "$OP" = confirm ] && [ $# -ge 2 ] && [[ "$2" =~ ^A[345]$ ]] || usage; CAUSE="$2"; shift 2 ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
done
|
||||
# Row 232: confirm, revert, gate, --to and --expect-from are external only, and an external
|
||||
# rollback names its target. gate needs --since, its start floor (r6 N2), and --expect-from
|
||||
# goes with --plan. The flags that change a unit's render or the PM are root's under external.
|
||||
if [ "$UNITS" = user ]; then
|
||||
[ "$OP" != confirm ] && [ "$OP" != revert ] && [ "$OP" != gate ] && [ -z "$TO_FLAG" ] && [ -z "$EXPECT_FROM" ] || usage
|
||||
else
|
||||
[ "$OP" != rollback ] || [ -n "$TO_FLAG" ] || usage
|
||||
[ "$WANT_PM" = 0 ] && [ -z "$TESTER_PORT" ] && [ -z "$PROOF_HEADER" ] && [ -z "$PROOF_FILE" ] && [ "$NO_PROOF" = 0 ] && [ "$NO_SEND" = 0 ] && [ "$SSO" = 0 ] || usage
|
||||
fi
|
||||
[ "$OP" != gate ] || [ -n "$SINCE" ] || usage
|
||||
[ -z "$EXPECT_FROM" ] || [ "$PLAN" = 1 ] || usage
|
||||
[ "${ROLLBACK_OLD:+1}" = "${CAUSE:+1}" ] || usage
|
||||
[ -z "$EVENTS" ] || [ -n "$RESTORE" ] || usage
|
||||
[ "$OP" != rehearse ] || { [ -n "$R_BACKUP" ] && [ -n "$R_PORT" ]; } || usage
|
||||
# A bad tester port is a usage error; --plan lists it as tester-port-invalid instead (row 180).
|
||||
@@ -167,7 +220,8 @@ for(const k of open.keys())process.stdout.write(k+"\n");' -- "$LOG"
|
||||
|
||||
# Reads the log's ok lines, newest first. `previous` prints rollback's target: the first `to`
|
||||
# that is not the pointer's copy (release.sh's rule). `backup` prints the backup field of the
|
||||
# line that made the pointer's copy live, the backup a rollback would restore. `arrival` prints the
|
||||
# line that made the pointer's copy live, the backup a rollback would restore. `newest` prints the
|
||||
# backup field of the newest line that has one, rehearse's --backup last (row 233). `arrival` prints the
|
||||
# activation and `from` of the line that moved the stack onto the pointer's copy from another one,
|
||||
# past any same-copy re-pin after it (row 200, sage-1389).
|
||||
from_log() {
|
||||
@@ -180,6 +234,7 @@ for(const l of lines.reverse()){
|
||||
if(o.result!=="ok"||!["adopt","activate","rollback"].includes(o.op)||!o.to)continue;
|
||||
if(mode==="previous"&&o.to!==current){process.stdout.write(o.to+"\n");process.exit(0)}
|
||||
if(mode==="backup"&&o.to===current){if(o.backup)process.stdout.write(o.backup+"\n");process.exit(o.backup?0:1)}
|
||||
if(mode==="newest"&&o.backup){process.stdout.write(o.backup+"\n");process.exit(0)}
|
||||
if(mode==="arrival"&&o.to===current&&o.from!==current){process.stdout.write(`${o.activation??""} ${o.from??""}\n`);process.exit(0)}
|
||||
}
|
||||
process.exit(1);' -- "$LOG" "$1" "$P_COPY"
|
||||
@@ -187,7 +242,7 @@ process.exit(1);' -- "$LOG" "$1" "$P_COPY"
|
||||
|
||||
# --- Reading the units ---
|
||||
|
||||
show() { systemctl --user show "$1" -p LoadState -p WorkingDirectory -p ExecStart -p Environment -p MainPID -p DropInPaths 2>/dev/null; }
|
||||
show() { sc show "$1" -p LoadState -p WorkingDirectory -p ExecStart -p Environment -p MainPID -p DropInPaths 2>/dev/null; }
|
||||
argv_of() { sed -n 's/^ExecStart=.*argv\[\]=\([^;]*\) ;.*/\1/p' <<< "$1"; }
|
||||
# systemctl show quotes a value that holds a space and escapes with a backslash, and the read -a
|
||||
# split below can't undo either, so such a line is unreadable rather than read wrong (row 166 item 4).
|
||||
@@ -286,12 +341,20 @@ check_target() {
|
||||
# has KIND exits 0 when the copy's KINDS has KIND
|
||||
# unknown prints the business file's plugins keys the copy's KINDS lacks, from a plain JSON read
|
||||
# harness prints plugins.harness as JSON, or `none`, from the copy's loader
|
||||
# launchby prints launch.by, the PM's instance, from a plain JSON read (row 232); it imports
|
||||
# only business.mjs, for the business file's path
|
||||
# A failure exits 1 and prints its first line.
|
||||
PLUGINS_JS='
|
||||
const {readFileSync}=require("node:fs");const {pathToFileURL}=require("node:url");
|
||||
const [mode,copy,id,kind]=process.argv.slice(1);
|
||||
const imp=(p)=>import(pathToFileURL(`${copy}/packages/${p}`).href);
|
||||
(async()=>{
|
||||
if(mode==="launchby"){
|
||||
const {businessFilePath}=await imp("business/src/business.mjs");
|
||||
const by=JSON.parse(readFileSync(businessFilePath(id),"utf8")).launch?.by;
|
||||
if(typeof by!=="string"||!by)throw new Error("the business file has no launch.by");
|
||||
return process.stdout.write(by);
|
||||
}
|
||||
const {KINDS}=await imp("plugins/src/kinds.mjs");
|
||||
if(mode==="has")process.exit(Object.hasOwn(KINDS,kind)?0:1);
|
||||
const b=await imp("business/src/business.mjs");
|
||||
@@ -455,13 +518,13 @@ capture() {
|
||||
|
||||
# Stops the Console, then the bus, and waits for both and the stopped check, within --timeout.
|
||||
stop_both() {
|
||||
systemctl --user stop "$CONSOLE_UNIT" || return 1
|
||||
systemctl --user stop "$BUS_UNIT" || return 1
|
||||
sc stop "$CONSOLE_UNIT" || return 1
|
||||
sc stop "$BUS_UNIT" || return 1
|
||||
local deadline=$((SECONDS + TIMEOUT)) u s down
|
||||
while :; do
|
||||
down=1
|
||||
for u in "$CONSOLE_UNIT" "$BUS_UNIT"; do
|
||||
s="$(systemctl --user is-active "$u" 2>/dev/null || true)"
|
||||
s="$(sc is-active "$u" 2>/dev/null || true)"
|
||||
case "$s" in inactive|failed) ;; *) down=0 ;; esac
|
||||
done
|
||||
if [ "$down" = 1 ] && stopped; then say "stopped both units"; return 0; fi
|
||||
@@ -472,7 +535,7 @@ stop_both() {
|
||||
|
||||
start_both() {
|
||||
START_EPOCH="$(date +%s)" && START_MS="$(date +%s%3N)" || return 1
|
||||
systemctl --user start "$BUS_UNIT" && systemctl --user start "$CONSOLE_UNIT"
|
||||
sc start "$BUS_UNIT" && sc start "$CONSOLE_UNIT"
|
||||
}
|
||||
|
||||
# Step 4: installs copy $1 with pm $2, claude $3 and the Console flags after them. Sets FAIL.
|
||||
@@ -493,9 +556,19 @@ install_stack() {
|
||||
gate_once() {
|
||||
local copy="$COPIES/$1" pm="$2" main out
|
||||
GATE=units-inactive GATE_WHY=""
|
||||
systemctl --user is-active --quiet "$BUS_UNIT" && systemctl --user is-active --quiet "$CONSOLE_UNIT" || return 1
|
||||
sc is-active --quiet "$BUS_UNIT" && sc is-active --quiet "$CONSOLE_UNIT" || return 1
|
||||
if [ "$UNITS" = external ]; then
|
||||
# r6 N4: root started the units, so this start's time is the bus unit's, in whole seconds,
|
||||
# at or after FLOOR. It is a gate code, so polling goes on: Restart=on-failure can bring a
|
||||
# fresh start.
|
||||
GATE=stale-start
|
||||
out="$(sc show --timestamp=unix -p ActiveEnterTimestamp "$BUS_UNIT" 2>/dev/null | sed -n 's/^ActiveEnterTimestamp=@//p')" || return 1
|
||||
[[ "$out" =~ ^[0-9]{1,12}$ ]] || { GATE_WHY="ActiveEnterTimestamp is unreadable"; return 1; }
|
||||
[ "$out" -ge "$FLOOR" ] || { GATE_WHY="the bus started at @$out, before @$FLOOR"; return 1; }
|
||||
START_MS=$((out * 1000))
|
||||
fi
|
||||
GATE=bus-status
|
||||
main="$(systemctl --user show "$BUS_UNIT" -p MainPID | sed -n 's/^MainPID=//p')" || return 1
|
||||
main="$(sc show "$BUS_UNIT" -p MainPID | sed -n 's/^MainPID=//p')" || return 1
|
||||
out="$("$copy/scripts/mosaic" bus status --json 2>/dev/null)" || { GATE_WHY="bus status failed"; return 1; }
|
||||
# Every `bus start` writes boot.at (packages/cli/src/boot-status.mjs), so a value from before
|
||||
# this start means the host on the socket is not the one this run started.
|
||||
@@ -508,13 +581,13 @@ process.stdout.write(why);' -- "$out" "$BUSINESS" "$main" "$START_MS" 2>&1)" ||
|
||||
node -e '
|
||||
fetch(`http://127.0.0.1:${process.argv[1]}/healthz`,{signal:AbortSignal.timeout(5000)}).then((r)=>process.exit(r.status===200?0:1),()=>process.exit(1));' -- "$(console_port)" || return 1
|
||||
GATE=copy-check
|
||||
out="$(scripts/stack-copy.sh check "${INSTANCE_FLAGS[@]}" --copies "$COPIES" --bin "$BIN" --business "$BUSINESS" 2>/dev/null)" || return 1
|
||||
out="$(scripts/stack-copy.sh check "${INSTANCE_FLAGS[@]}" "${UNITS_FLAGS[@]}" --copies "$COPIES" --bin "$BIN" --business "$BUSINESS" 2>/dev/null)" || return 1
|
||||
# check passes when the three agree on one copy; this run needs it to be the target (row 166 item 3).
|
||||
out="$(sed -n 's/^bus: //p' <<< "$out")"
|
||||
[ "$out" = "$copy" ] || { GATE_WHY="the bus, Console and CLI agree on $out, not $copy"; return 1; }
|
||||
if [ "$pm" = 1 ]; then
|
||||
GATE=pm-launch-unconfirmed
|
||||
pm_launched "$copy" || return 1
|
||||
if [ "$UNITS" = user ]; then pm_launched "$copy" || return 1; else pm_external "$1" || return 1; fi
|
||||
fi
|
||||
GATE=""
|
||||
}
|
||||
@@ -526,7 +599,7 @@ fetch(`http://127.0.0.1:${process.argv[1]}/healthz`,{signal:AbortSignal.timeout(
|
||||
# because this host's journalctl reads a bare time as local time.
|
||||
pm_launched() {
|
||||
local journal run list
|
||||
journal="$(journalctl --user -u "$BUS_UNIT" --since "@$START_EPOCH" -o cat 2>/dev/null)" || { GATE_WHY="journal read failed"; return 1; }
|
||||
journal="$(jctl -u "$BUS_UNIT" --since "@$START_EPOCH" -o cat 2>/dev/null)" || { GATE_WHY="journal read failed"; return 1; }
|
||||
local -a lines=()
|
||||
mapfile -t lines < <(grep '^launched ' <<< "$journal" || true)
|
||||
[ "${#lines[@]}" -gt 0 ] || { GATE_WHY="no launched line since the start"; return 1; }
|
||||
@@ -540,6 +613,20 @@ process.stdout.write(!s?`run ${run} is not in launches list`:s.business!==b?`run
|
||||
[ -z "$GATE_WHY" ]
|
||||
}
|
||||
|
||||
# r6 section 4: the PM check under --units external, where the bus unit's journal is the system
|
||||
# journal and unreadable here. It passes when exactly one live launches list entry is for the
|
||||
# PM's instance (launch.by, packages/cli/src/launcher.mjs:675), and that entry is for BUSINESS and
|
||||
# started at or after this start. A run the PM starts for another role has another instance.
|
||||
pm_external() {
|
||||
local by list
|
||||
by="$(plugins launchby "$1" 2>&1)" || { GATE_WHY="cannot read launch.by: $by"; return 1; }
|
||||
list="$("$COPIES/$1/scripts/mosaic" launches list --json 2>/dev/null)" || { GATE_WHY="launches list failed"; return 1; }
|
||||
GATE_WHY="$(node -e '
|
||||
const [list,by,b,since]=process.argv.slice(1),l=JSON.parse(list).filter((x)=>x.instance===by&&x.live===true),s=l[0];
|
||||
process.stdout.write(l.length===0?`no live launch of ${by}`:l.length>1?`${l.length} live launches of ${by}`:s.business!==b?`the launch of ${by} is for ${s.business}`:!(Date.parse(s.startedAt)>=Number(since))?`run ${s.run} started before this start`:"");' -- "$list" "$by" "$BUSINESS" "$START_MS" 2>&1)" || { GATE_WHY="launches list unreadable"; return 1; }
|
||||
[ -z "$GATE_WHY" ]
|
||||
}
|
||||
|
||||
# Polls every second until every check passes or --timeout runs out.
|
||||
gate() {
|
||||
local deadline=$((SECONDS + TIMEOUT))
|
||||
@@ -573,7 +660,7 @@ put_back() {
|
||||
done
|
||||
[ "$(sha256sum < "$BIN/$WRAPPER_NAME" | cut -d' ' -f1)" = "$(sum_of "$RS" wrapper)" ] || return 1
|
||||
[ "$(stat -c %a "$BIN/$WRAPPER_NAME")" = "$(cat "$RS/WRAPPER_MODE")" ] || return 1
|
||||
systemctl --user daemon-reload || return 1
|
||||
sc daemon-reload || return 1
|
||||
say "put back the unit files and the wrapper from $RS"
|
||||
}
|
||||
|
||||
@@ -608,8 +695,8 @@ after_fail() {
|
||||
# Stop both units, print both results, log rollback-failed, exit 2. No retry.
|
||||
rollback_failed() {
|
||||
trap - INT TERM HUP
|
||||
systemctl --user stop "$CONSOLE_UNIT" || true
|
||||
systemctl --user stop "$BUS_UNIT" || true
|
||||
sc stop "$CONSOLE_UNIT" || true
|
||||
sc stop "$BUS_UNIT" || true
|
||||
say "rollback-failed: run $1, rollback $2"
|
||||
log_line result rollback-failed code "$1" rollback failed rollbackCode "$2"
|
||||
echo "stack-release: rollback-failed: the run failed with $1 and the rollback with $2; both units are stopped. Rollback set: $RS${BACKUP:+, backup $DATA/backups/stack/$BACKUP}${RESTORED:+, post-attempt store $POST}" >&2
|
||||
@@ -618,10 +705,15 @@ rollback_failed() {
|
||||
|
||||
# --- Refusals (item 3) ---
|
||||
|
||||
lock() {
|
||||
# confirm and revert act on the open activation by design, so they take the lock alone (r6 section 4).
|
||||
lock_only() {
|
||||
mkdir -p "$STATE"
|
||||
exec 9> "$STATE/stack-release.lock"
|
||||
flock -n 9 || refuse run-locked "another stack-release run holds $STATE/stack-release.lock"
|
||||
}
|
||||
|
||||
lock() {
|
||||
lock_only
|
||||
local open
|
||||
open="$(interrupted | paste -sd' ' -)"
|
||||
[ -z "$open" ] || refuse interrupted-run "activation $open started and has no final line; Jason or Mos settles it (status shows it)"
|
||||
@@ -629,7 +721,7 @@ lock() {
|
||||
|
||||
instance_busy() {
|
||||
local out unit load state rest
|
||||
out="$(systemctl --user list-units --all --no-legend --plain "$BUS_GLOB" 2>/dev/null)" || refuse instance-busy "systemctl --user list-units failed, so the bus state is unknown"
|
||||
out="$(sc list-units --all --no-legend --plain "$BUS_GLOB" 2>/dev/null)" || refuse instance-busy "$SC_NAME list-units failed, so the bus state is unknown"
|
||||
while read -r unit load state rest; do
|
||||
[[ "$unit" == $BUS_GLOB ]] || continue
|
||||
if [ "$unit" = "$BUS_UNIT" ]; then
|
||||
@@ -645,7 +737,7 @@ instance_busy() {
|
||||
drop_in_override() {
|
||||
local paths directory dropdir file content glob
|
||||
local -a dropdirs
|
||||
paths="$(systemd-analyze --user unit-paths 2>/dev/null)" && [ -n "$paths" ] || refuse drop-in-override "systemd unit paths are unknown"
|
||||
paths="$(analyze unit-paths 2>/dev/null)" && [ -n "$paths" ] || refuse drop-in-override "systemd unit paths are unknown"
|
||||
while IFS= read -r directory; do
|
||||
[ -n "$directory" ] || continue
|
||||
case "$directory" in /*) ;; *) refuse drop-in-override "invalid systemd unit path $directory" ;; esac
|
||||
@@ -679,7 +771,8 @@ flags_drift() {
|
||||
if [ "$AFTER_HARNESS" = 1 ]; then
|
||||
[ "$P_PM" = 1 ] || d+=$'\n'" pm: pointer $P_PM, --pm-after-harness needs a pointer with pm 1"
|
||||
else
|
||||
[ "$OP" != activate ] || [ "$WANT_PM" = "$P_PM" ] || d+=$'\n'" pm: pointer $P_PM, --pm given $WANT_PM"
|
||||
# Under --units external the PM is a flag in root's unit, so there is no --pm to compare.
|
||||
[ "$UNITS" != user ] || [ "$OP" != activate ] || [ "$WANT_PM" = "$P_PM" ] || d+=$'\n'" pm: pointer $P_PM, --pm given $WANT_PM"
|
||||
fi
|
||||
[ -z "$d" ] || refuse unit-flags-drift "the units differ from the pointer; settle it first:$d"
|
||||
}
|
||||
@@ -836,12 +929,13 @@ sso_checks() {
|
||||
sso_supported || { check_refuse sso-target "$TO's Console does not support --sso"; return 0; }
|
||||
local cfg="${MOSAIC_CONFIG:-$HOME/.config/mosaic-dev/config.json}" out id slot path why
|
||||
# N4 (webby-0019, sage-1403): a pre-phase-1 loader refuses a console key, so with the key not yet
|
||||
# written and the stack on another copy, the first step is the re-pin without --sso.
|
||||
local first=""
|
||||
# written and the stack on another copy, the first step is the re-pin without --sso. On a named
|
||||
# instance both commands carry --instance NAME (row 238).
|
||||
local first="" inst="${INSTANCE_FLAGS[*]:+ ${INSTANCE_FLAGS[*]}}"
|
||||
if ! out="$(node -e "$SSO_JS" -- "$COPIES/$TO" "$cfg" 2>&1)"; then
|
||||
out="${out%%$'\n'*}"
|
||||
[ "$out" != "there is no console.sso in the system config" ] || [ "$P_COPY" = "$TO" ] ||
|
||||
first="; activate $TO without --sso first, then write console.sso, then run activate $TO --sso"
|
||||
first="; activate $TO$inst without --sso first, then write console.sso, then run activate $TO$inst --sso"
|
||||
check_refuse sso-config "$TO's Console would refuse its --sso start: $out$first"
|
||||
return 0
|
||||
fi
|
||||
@@ -1075,11 +1169,12 @@ do_adopt() {
|
||||
lock
|
||||
[ ! -f "$POINTER" ] || { read_pointer; FROM="$P_COPY"; refuse already-adopted "$POINTER exists (copy $P_COPY)"; }
|
||||
local out
|
||||
out="$(scripts/stack-copy.sh check "${INSTANCE_FLAGS[@]}" --copies "$COPIES" --bin "$BIN" --business "$BUSINESS" 2>&1)" || refuse check-failed "stack-copy.sh check did not pass:"$'\n'"$out"
|
||||
out="$(scripts/stack-copy.sh check "${INSTANCE_FLAGS[@]}" "${UNITS_FLAGS[@]}" --copies "$COPIES" --bin "$BIN" --business "$BUSINESS" 2>&1)" || refuse check-failed "stack-copy.sh check did not pass:"$'\n'"$out"
|
||||
TO="$(sed -n 's/^bus: //p' <<< "$out")"
|
||||
TO="${TO#"$COPIES/"}"
|
||||
read_units || refuse check-failed "cannot read $BUS_UNIT or $CONSOLE_UNIT"
|
||||
[ -n "$U_CLAUDE" ] || refuse no-claude-pin "$BUS_UNIT has no MOSAIC_CLAUDE_BIN"
|
||||
# Under --units external the claude pin is root's render, so adopt records what the unit has.
|
||||
[ "$UNITS" = external ] || [ -n "$U_CLAUDE" ] || refuse no-claude-pin "$BUS_UNIT has no MOSAIC_CLAUDE_BIN"
|
||||
target_schema "$TO" || refuse copy-schema-unknown "cannot read SCHEMA_VERSION from $COPIES/$TO/packages/bus/src/store.mjs"
|
||||
next_activation
|
||||
write_pointer "$TO" "$U_PM" 0 "$U_CLAUDE" "$T_SCHEMA" "${U_CONSOLE[@]}"
|
||||
@@ -1094,12 +1189,12 @@ plan_store() {
|
||||
local src="" why=""
|
||||
S_SCHEMA="" S_FROM="" S_CHANGED=""
|
||||
if [ ! -e "$STORE" ]; then why="there is no store file"
|
||||
elif systemctl --user is-active --quiet "$BUS_UNIT"; then
|
||||
elif sc is-active --quiet "$BUS_UNIT"; then
|
||||
src="the store the running bus holds"
|
||||
S_SCHEMA="$(node -e "$STORE_JS" -- heldversion "$STORE" 2>/dev/null)" || S_SCHEMA=""
|
||||
# A bus that stops during the readOnly open can leave -wal or -shm beside the stopped
|
||||
# store, so the version read can't be planned on (row 166 item 2).
|
||||
systemctl --user is-active --quiet "$BUS_UNIT" || S_CHANGED=1
|
||||
sc is-active --quiet "$BUS_UNIT" || S_CHANGED=1
|
||||
elif store_unclean; then why="the store was not read: $UNCLEAN is present"
|
||||
else
|
||||
src="the stopped store"
|
||||
@@ -1170,6 +1265,8 @@ plan() {
|
||||
echo "needs: --migrate$([ "$MIGRATE" = 1 ] && echo ' (given)' || echo ' (not given; the run would roll back with migration-needs-flag)')"
|
||||
echo "needs: Jason present for this run (decision 96 condition 2)"
|
||||
[ "$MIGRATE" = 1 ] || early=1
|
||||
# Row 233: root reads only the plan's exit, so under external a missing --migrate refuses here.
|
||||
[ "$UNITS" = user ] || [ "$MIGRATE" = 1 ] || PLAN_REFUSE+=("migrate-required: the store is $S_SCHEMA and $TO migrates it to $T_SCHEMA; the go names --migrate")
|
||||
elif [ "$OP" = activate ]; then
|
||||
echo "migration: none possible; $S_SCHEMA is not in $T_SCHEMA's migration chain, so the run would roll back with store-newer"
|
||||
early=1
|
||||
@@ -1186,15 +1283,19 @@ plan() {
|
||||
echo "backup: the run backs up the stopped store to $DATA/backups/stack/<activation>/ before any migration"
|
||||
fi
|
||||
plan_commits
|
||||
[ "$P_PENDING" = 0 ] || echo "pm: the pointer is pending (activate --pm-after-harness); the bus runs without its PM"
|
||||
[ "$RUN_PENDING" = 0 ] || echo "pm: the bus starts without --pm and the PM stays pending; next: Jason's plugins set of harness, then pm-on"
|
||||
local flags="${INSTANCE_FLAGS[*]:+${INSTANCE_FLAGS[*]} }"
|
||||
echo "bus: scripts/bus-service.sh install $flags--copy $TO --copies $COPIES$([ "$RUN_PM" = 0 ] || echo ' --pm') --claude $P_CLAUDE"
|
||||
[ -z "$DROP_TESTER" ] || echo "drop $DROP_TESTER; the target Console does not support it"
|
||||
echo "$PROOF_PLAN"
|
||||
echo "$SEND_PLAN"
|
||||
[ -z "$SSO_PLAN" ] || echo "$SSO_PLAN"
|
||||
echo "Console: scripts/webui-service.sh install $flags--copy $TO --copies $COPIES${RUN_CONSOLE[*]:+ ${RUN_CONSOLE[*]}}"
|
||||
if [ "$UNITS" = external ]; then
|
||||
echo "units: external; root installs the units for $TO and starts them, then confirm gates them"
|
||||
else
|
||||
[ "$P_PENDING" = 0 ] || echo "pm: the pointer is pending (activate --pm-after-harness); the bus runs without its PM"
|
||||
[ "$RUN_PENDING" = 0 ] || echo "pm: the bus starts without --pm and the PM stays pending; next: Jason's plugins set of harness, then pm-on"
|
||||
echo "bus: scripts/bus-service.sh install $flags--copy $TO --copies $COPIES$([ "$RUN_PM" = 0 ] || echo ' --pm') --claude $P_CLAUDE"
|
||||
[ -z "$DROP_TESTER" ] || echo "drop $DROP_TESTER; the target Console does not support it"
|
||||
echo "$PROOF_PLAN"
|
||||
echo "$SEND_PLAN"
|
||||
[ -z "$SSO_PLAN" ] || echo "$SSO_PLAN"
|
||||
echo "Console: scripts/webui-service.sh install $flags--copy $TO --copies $COPIES${RUN_CONSOLE[*]:+ ${RUN_CONSOLE[*]}}"
|
||||
fi
|
||||
echo "CLI: scripts/stack-copy.sh wrapper $TO $flags--copies $COPIES --bin $BIN"
|
||||
local r
|
||||
for r in "${PLAN_REFUSE[@]}"; do echo "refuses $r"; done
|
||||
@@ -1268,6 +1369,16 @@ do_activate() {
|
||||
pointer_for_business
|
||||
run_pm
|
||||
[ "$P_PENDING" = 0 ] || check_refuse pm-pending "the pointer is pending (activate --pm-after-harness); run pm-on or rollback first"
|
||||
if [ "$UNITS" = external ]; then
|
||||
ext_pointer_checks
|
||||
if [ "$PLAN" = 1 ]; then check_target; plugin_checks; config_checks; plan; exit 0; fi
|
||||
ext_precheck
|
||||
plugin_checks
|
||||
config_checks
|
||||
new_activation
|
||||
run_external migrate
|
||||
return
|
||||
fi
|
||||
if [ "$PLAN" = 1 ]; then check_target; console_for_target; ports_unclaimed; plugin_checks; config_checks; plan; exit 0; fi
|
||||
precheck
|
||||
plugin_checks
|
||||
@@ -1281,43 +1392,328 @@ do_rollback() {
|
||||
pointer_for_business
|
||||
TO="$(from_log previous)" || refuse no-previous "the log names no earlier copy than $P_COPY"
|
||||
run_pm
|
||||
if [ "$UNITS" = external ]; then
|
||||
# r6: root names the target from the go, and it must be the one rollback reads from the log.
|
||||
[ "$TO_FLAG" = "$TO" ] || refuse to-not-previous "--to $TO_FLAG is not the previous copy; the log names $TO"
|
||||
ext_pointer_checks
|
||||
if [ "$PLAN" = 1 ]; then
|
||||
check_target; plugin_checks; config_checks
|
||||
[ -z "$RESTORE" ] || { next_activation; restore_checks; }
|
||||
plan; exit 0
|
||||
fi
|
||||
ext_precheck
|
||||
plugin_checks
|
||||
config_checks
|
||||
[ "$P_SCHEMA" = "$T_SCHEMA" ] || [ -n "$RESTORE" ] || refuse schema-differs "the store is at $P_SCHEMA and $TO expects $T_SCHEMA; pass --restore ACTIVATION"
|
||||
new_activation
|
||||
[ -z "$RESTORE" ] || restore_checks
|
||||
run_external restore
|
||||
return
|
||||
fi
|
||||
if [ "$PLAN" = 1 ]; then check_target; console_for_target; ports_unclaimed; plugin_checks; config_checks; plan; exit 0; fi
|
||||
precheck
|
||||
plugin_checks
|
||||
config_checks
|
||||
[ "$P_SCHEMA" = "$T_SCHEMA" ] || [ -n "$RESTORE" ] || refuse schema-differs "the store is at $P_SCHEMA and $TO expects $T_SCHEMA; pass --restore ACTIVATION"
|
||||
new_activation
|
||||
if [ -n "$RESTORE" ]; then
|
||||
local o="$DATA/backups/stack/$RESTORE" rc=0 c="" text
|
||||
[ -f "$o/SHA256SUMS" ] && [ "$(cat "$o/DATA_ROOT" 2>/dev/null)" = "$DATA" ] && [ "$(cat "$o/SCHEMA" 2>/dev/null)" = "$T_SCHEMA" ] && [ -f "$o/STAMP" ] \
|
||||
|| refuse restore-mismatch "$o is not a backup of this root at $T_SCHEMA"
|
||||
# Before anything stops. A store the running bus holds is read with readOnly, which adds
|
||||
# no file (the probe); a stopped one falls under the store_unclean rule.
|
||||
if systemctl --user is-active --quiet "$BUS_UNIT"; then
|
||||
c="$(node -e "$STORE_JS" -- heldcount "$STORE" "$(cat "$o/STAMP")" "$COUNTED")" || rc=1
|
||||
# plan_store's recheck (row 166 item 2), before anything stops.
|
||||
systemctl --user is-active --quiet "$BUS_UNIT" || refuse restore-mismatch "the bus stopped while the events after $(cat "$o/STAMP") were counted, so the store state changed; nothing was stopped"
|
||||
else
|
||||
! store_unclean || refuse store-unclean "$UNCLEAN is present; the bus did not close the store cleanly"
|
||||
c="$(store_read count "$(cat "$o/STAMP")" "$COUNTED")" || rc=1
|
||||
fi
|
||||
[ "$rc" = 0 ] || refuse restore-mismatch "cannot count the rows after $(cat "$o/STAMP")"
|
||||
COUNT="${c%%$'\t'*}" text="${c#*$'\t'}"
|
||||
echo "restore: $o; the store now in place moves to $POST and is kept; $text were written after the backup"
|
||||
[ "$COUNT" = 0 ] || [ "$EVENTS" = "$COUNT" ] || refuse post-backup-events "$text were written after the backup; Jason decides, then --events $COUNT"
|
||||
BACKUP="$RESTORE"
|
||||
fi
|
||||
[ -z "$RESTORE" ] || restore_checks
|
||||
run_steps restore
|
||||
}
|
||||
|
||||
# rollback --restore's checks on the backup and the rows written after it, before anything stops.
|
||||
# Sets COUNT and BACKUP; POST comes from new_activation.
|
||||
restore_checks() {
|
||||
local o="$DATA/backups/stack/$RESTORE" rc=0 c="" text
|
||||
[ -f "$o/SHA256SUMS" ] && [ "$(cat "$o/DATA_ROOT" 2>/dev/null)" = "$DATA" ] && [ "$(cat "$o/SCHEMA" 2>/dev/null)" = "$T_SCHEMA" ] && [ -f "$o/STAMP" ] \
|
||||
|| refuse restore-mismatch "$o is not a backup of this root at $T_SCHEMA"
|
||||
# Before anything stops. A store the running bus holds is read with readOnly, which adds
|
||||
# no file (the probe); a stopped one falls under the store_unclean rule.
|
||||
if sc is-active --quiet "$BUS_UNIT"; then
|
||||
c="$(node -e "$STORE_JS" -- heldcount "$STORE" "$(cat "$o/STAMP")" "$COUNTED")" || rc=1
|
||||
# plan_store's recheck (row 166 item 2), before anything stops.
|
||||
sc is-active --quiet "$BUS_UNIT" || refuse restore-mismatch "the bus stopped while the events after $(cat "$o/STAMP") were counted, so the store state changed; nothing was stopped"
|
||||
else
|
||||
! store_unclean || refuse store-unclean "$UNCLEAN is present; the bus did not close the store cleanly"
|
||||
c="$(store_read count "$(cat "$o/STAMP")" "$COUNTED")" || rc=1
|
||||
fi
|
||||
[ "$rc" = 0 ] || refuse restore-mismatch "cannot count the rows after $(cat "$o/STAMP")"
|
||||
COUNT="${c%%$'\t'*}" text="${c#*$'\t'}"
|
||||
echo "restore: $o; the store now in place moves to $POST and is kept; $text were written after the backup"
|
||||
[ "$COUNT" = 0 ] || [ "$EVENTS" = "$COUNT" ] || refuse post-backup-events "$text were written after the backup; Jason decides, then --events $COUNT"
|
||||
BACKUP="$RESTORE"
|
||||
}
|
||||
|
||||
# --- --units external (row 232, r6 section 4) ---
|
||||
|
||||
# After the pointer, for activate and rollback, in --plan and in the run. OLD is the copy root read
|
||||
# from the unit files it installed (r6 A1, N5), so a pointer that names another copy stops the
|
||||
# re-pin before root stops anything. The PM is a flag in root's unit, so --pm-after-harness has
|
||||
# nothing to wait on. The run's own lock refuses an open activation; --plan takes no lock, so it
|
||||
# lists one here.
|
||||
ext_pointer_checks() {
|
||||
[ -z "$EXPECT_FROM" ] || [ "$EXPECT_FROM" = "$P_COPY" ] || check_refuse pointer-differs "the pointer names $P_COPY and root's units name $EXPECT_FROM; Jason or Mos settles it"
|
||||
[ "$AFTER_HARNESS" = 0 ] || check_refuse external-pm "under --units external the PM is a flag in root's bus unit; changing it is a re-install with its own go"
|
||||
[ "$PLAN" = 1 ] || return 0
|
||||
local open
|
||||
open="$(interrupted | paste -sd' ' -)"
|
||||
[ -z "$open" ] || check_refuse interrupted-run "activation $open started and has no final line; Jason or Mos settles it (status shows it)"
|
||||
# Row 233: root can't read the stack's state, so the plan lists any rollback set still waiting
|
||||
# for units, whatever the log says.
|
||||
local m sets=""
|
||||
local -a markers
|
||||
mapfile -t markers < <(compgen -G "$STATE/stack-rollback/*/AWAITING-UNITS" || true)
|
||||
for m in "${markers[@]}"; do sets+=" $(basename "$(dirname "$m")")"; done
|
||||
[ -z "$sets" ] || check_refuse open-activation "rollback set(s)$sets hold AWAITING-UNITS; root finishes them with confirm or revert first"
|
||||
}
|
||||
|
||||
# A3's checks before `started`. Root has stopped the units and installs them after this run, so
|
||||
# there is no Console flag, port, drop-in or claude pin of this script's to check.
|
||||
ext_precheck() {
|
||||
check_target
|
||||
instance_busy
|
||||
read_units || refuse unit-flags-drift "cannot read $BUS_UNIT or $CONSOLE_UNIT"
|
||||
flags_drift
|
||||
units_down
|
||||
}
|
||||
|
||||
# r6 A3 (N4): each unit reads inactive, or failed as stop_both accepts, and the store is closed,
|
||||
# so nothing below runs against a store a live bus holds.
|
||||
units_down() {
|
||||
local u s
|
||||
for u in "$CONSOLE_UNIT" "$BUS_UNIT"; do
|
||||
s="$(sc is-active "$u" 2>/dev/null || true)"
|
||||
case "$s" in inactive|failed) ;; *) refuse units-running "$u is ${s:-unknown}; root stops the units first" ;; esac
|
||||
done
|
||||
stopped || refuse units-running "the store under $DATA/bus is still open (writer.lock or an open file)"
|
||||
}
|
||||
|
||||
# A3's capture into $RS, which run_external made: the pointer, the wrapper and its mode, and for each
|
||||
# unit its show output and the sha256 of its unit file. It reads root's units and makes no directory.
|
||||
capture_units() {
|
||||
local u frag
|
||||
[ ! -f "$POINTER" ] || cp -p "$POINTER" "$RS/pointer" || return 1
|
||||
[ -f "$BIN/$WRAPPER_NAME" ] || { say "capture: no wrapper $BIN/$WRAPPER_NAME"; return 1; }
|
||||
cp -p "$BIN/$WRAPPER_NAME" "$RS/wrapper" || return 1
|
||||
stat -c %a "$BIN/$WRAPPER_NAME" > "$RS/WRAPPER_MODE" || return 1
|
||||
: > "$RS/UNITS.sha256" || return 1
|
||||
for u in "$BUS_UNIT" "$CONSOLE_UNIT"; do
|
||||
show "$u" > "$RS/show-$u.txt" || return 1
|
||||
frag="$(sc show "$u" -p FragmentPath 2>/dev/null | sed -n 's/^FragmentPath=//p')" && [ -n "$frag" ] || { say "capture: no FragmentPath for $u"; return 1; }
|
||||
sha256sum "$frag" >> "$RS/UNITS.sha256" || return 1
|
||||
done
|
||||
sums "$RS" > "$RS/SHA256SUMS" || return 1
|
||||
say "captured the rollback set: $RS"
|
||||
}
|
||||
|
||||
# A failure after the capture: FAILED holds the code, nothing is logged or cleaned up, exit 1.
|
||||
# Root runs R1-R4, and revert is the one cleanup path (r6 A3).
|
||||
ext_fail() {
|
||||
trap - INT TERM HUP
|
||||
{ echo "$1" > "$RS/FAILED"; } 2>/dev/null || true
|
||||
say "failed: $1"
|
||||
echo "stack-release: $OP failed with $1; $ACT stays open for root's revert and confirm --rollback. Rollback set: $RS" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
# A3 for target $TO. $1 is the store step: migrate or restore. The marker goes in first, so every
|
||||
# failure after it leaves what revert needs. Writes no pointer and starts nothing.
|
||||
run_external() {
|
||||
log_line result started
|
||||
{ mkdir -p -m 0700 "$STATE/stack-rollback" && mkdir -m 0700 "$RS"; } 2>/dev/null || refuse marker-failed "could not make the rollback set $RS; nothing was changed"
|
||||
{ echo "$TO" > "$RS/AWAITING-UNITS"; } 2>/dev/null || { rm -f "$RS/AWAITING-UNITS"; refuse marker-failed "could not write $RS/AWAITING-UNITS; nothing was changed"; }
|
||||
STEPS="$RS/steps.log"
|
||||
capture_units || { rm -f "$RS/AWAITING-UNITS"; refuse capture-failed "could not write the rollback set $RS; nothing was changed"; }
|
||||
trap 'ext_fail signal' INT TERM HUP
|
||||
if [ -e "$STORE" ]; then
|
||||
if store_unclean; then say "$UNCLEAN is present; the store is left as it is for Jason or Mos"; ext_fail store-unclean; fi
|
||||
STORE_SCHEMA="$(store_read version)" && [ -n "$STORE_SCHEMA" ] || ext_fail schema-unreadable
|
||||
say "stopped store schema: $STORE_SCHEMA"
|
||||
if [ "$1" = migrate ]; then
|
||||
if [ "$STORE_SCHEMA" != "$T_SCHEMA" ]; then
|
||||
in_chain "$STORE_SCHEMA" || ext_fail store-newer
|
||||
[ "$MIGRATE" = 1 ] || ext_fail migration-needs-flag
|
||||
fi
|
||||
backup_store || ext_fail backup
|
||||
echo "$BACKUP" > "$RS/BACKUP" || ext_fail backup
|
||||
if [ "$STORE_SCHEMA" != "$T_SCHEMA" ]; then
|
||||
# MIGRATING and TAKEN before the migration, so revert restores a partial one (r6 B2).
|
||||
: > "$RS/MIGRATING" || ext_fail migrate-failed
|
||||
TAKEN="$ACT"
|
||||
node "$COPIES/$TO/packages/bus/src/migrate-schema.mjs" "$DATA" >> "$STEPS" 2>&1 || ext_fail migrate-failed
|
||||
say "migrated the store to $T_SCHEMA"
|
||||
fi
|
||||
else
|
||||
[ "$STORE_SCHEMA" = "$P_SCHEMA" ] || ext_fail store-differs
|
||||
if [ -n "$RESTORE" ]; then
|
||||
local n
|
||||
n="$(store_read count "$(cat "$DATA/backups/stack/$RESTORE/STAMP")" "$COUNTED")" || ext_fail schema-unreadable
|
||||
[ "${n%%$'\t'*}" = "$COUNT" ] || ext_fail post-backup-events
|
||||
echo "$RESTORE" > "$RS/BACKUP" || ext_fail restore
|
||||
# RESTORED tells confirm --rollback the store was swapped, so it skips OLD's gate
|
||||
# and logs rollbackCode not-attempted (ruling 4, T14b).
|
||||
if ! restore_store "$RESTORE"; then
|
||||
[ -z "$RESTORED" ] || : > "$RS/RESTORED" || true
|
||||
ext_fail restore
|
||||
fi
|
||||
: > "$RS/RESTORED" || ext_fail restore
|
||||
fi
|
||||
fi
|
||||
elif [ -n "$RESTORE" ]; then
|
||||
ext_fail store-differs
|
||||
fi
|
||||
scripts/stack-copy.sh wrapper "$TO" "${INSTANCE_FLAGS[@]}" --copies "$COPIES" --bin "$BIN" >> "$STEPS" 2>&1 || ext_fail wrapper
|
||||
trap - INT TERM HUP
|
||||
say "staged $TO"
|
||||
echo "stack-release: $OP staged $TO for $BUSINESS as $ACT; root installs and starts the units, then runs confirm $TO $BUSINESS --units external"
|
||||
}
|
||||
|
||||
# r6 "The open activation": exactly one AWAITING-UNITS file, its activation the one the log holds
|
||||
# open, holding the copy root passed. Sets O_ACT. Anything else refuses open-activation, and ACT
|
||||
# stays unset, so the refused line names no activation and the open one stays open.
|
||||
open_activation() {
|
||||
local -a markers open
|
||||
local m found="" seen=""
|
||||
mapfile -t markers < <(compgen -G "$STATE/stack-rollback/*/AWAITING-UNITS" || true)
|
||||
mapfile -t open < <(interrupted)
|
||||
for m in "${markers[@]}"; do
|
||||
seen+=" $(basename "$(dirname "$m")") (copy $(cat "$m" 2>/dev/null || echo unreadable))"
|
||||
done
|
||||
if [ "${#markers[@]}" = 1 ] && [ "${#open[@]}" = 1 ]; then
|
||||
m="$(basename "$(dirname "${markers[0]}")")"
|
||||
[ "$m" != "${open[0]}" ] || [ "$(cat "${markers[0]}" 2>/dev/null)" != "$ID" ] || found="$m"
|
||||
fi
|
||||
[ -n "$found" ] || refuse open-activation "need one AWAITING-UNITS for copy $ID whose activation the log holds open; markers:${seen:- none}; open: ${open[*]:-none}"
|
||||
O_ACT="$found"
|
||||
}
|
||||
|
||||
# The op, from, to and start second of O_ACT's `started` line, one per line.
|
||||
started_of() {
|
||||
node -e '
|
||||
const [file,act]=process.argv.slice(1);
|
||||
let s;
|
||||
for(const l of require("fs").readFileSync(file,"utf8").split("\n")){
|
||||
if(!l.trim())continue;
|
||||
let o;try{o=JSON.parse(l)}catch{continue}
|
||||
if(o.activation===act&&o.result==="started")s=o;
|
||||
}
|
||||
const at=s&&Math.floor(Date.parse(s.at)/1000);
|
||||
if(!s||!["activate","rollback"].includes(s.op)||!Number.isFinite(at))process.exit(1);
|
||||
process.stdout.write([s.op,s.from??"",s.to??"",at].join("\n")+"\n");' -- "$LOG" "$O_ACT"
|
||||
}
|
||||
|
||||
# A5, and R4 with --rollback. The lines it writes carry the open activation's op, from, to and
|
||||
# activation, and the backup A3 recorded, so from_log reads them as a dragon-lin run's. Every check
|
||||
# runs before ACT is set.
|
||||
do_confirm() {
|
||||
lock_only
|
||||
open_activation
|
||||
local s
|
||||
local -a f
|
||||
s="$(started_of)" || refuse open-activation "the log's started line for $O_ACT is unreadable"
|
||||
mapfile -t f <<< "$s"
|
||||
[ "${f[2]}" = "$ID" ] || refuse open-activation "$O_ACT started for ${f[2]}, not $ID"
|
||||
if read_pointer; then [ "$P_BUSINESS" = "$BUSINESS" ] || refuse business-mismatch "the pointer is for $P_BUSINESS"; fi
|
||||
if [ -n "$ROLLBACK_OLD" ]; then
|
||||
[ "$ROLLBACK_OLD" = "${f[1]}" ] || refuse rollback-mismatch "$ROLLBACK_OLD is not the copy $O_ACT started from, ${f[1]:-none}"
|
||||
else
|
||||
[ ! -e "$STATE/stack-rollback/$O_ACT/FAILED" ] || refuse activation-failed "$O_ACT has a FAILED file; root runs revert and confirm --rollback"
|
||||
target_schema "$ID" || refuse copy-schema-unknown "cannot read SCHEMA_VERSION from $COPIES/$ID/packages/bus/src/store.mjs"
|
||||
fi
|
||||
OP="${f[0]}" FROM="${f[1]}" TO="${f[2]}" FLOOR="${f[3]}" ACT="$O_ACT" RS="$STATE/stack-rollback/$O_ACT"
|
||||
STEPS="$RS/steps.log"
|
||||
BACKUP="$(cat "$RS/BACKUP" 2>/dev/null || true)"
|
||||
if [ -z "$ROLLBACK_OLD" ]; then
|
||||
read_units || ext_fail units-unreadable
|
||||
P_CONSOLE=("${U_CONSOLE[@]}")
|
||||
gate "$TO" "$U_PM" || ext_fail "$GATE"
|
||||
write_pointer "$TO" "$U_PM" 0 "$U_CLAUDE" "$T_SCHEMA" "${U_CONSOLE[@]}"
|
||||
log_line result ok
|
||||
rm -f "$RS/AWAITING-UNITS"
|
||||
echo "stack-release: $OP ok: $TO is live for $BUSINESS"
|
||||
return
|
||||
fi
|
||||
local code rcode=""
|
||||
if [ "$CAUSE" = A4 ]; then code=units-start
|
||||
else code="$(head -n 1 "$RS/FAILED" 2>/dev/null || true)"; [ -n "$code" ] || code=unrecorded
|
||||
fi
|
||||
if [ -e "$RS/RESTORED" ]; then rcode=not-attempted
|
||||
elif ! read_units; then rcode=units-unreadable
|
||||
else
|
||||
P_CONSOLE=("${U_CONSOLE[@]}")
|
||||
gate "$ROLLBACK_OLD" "$U_PM" || rcode="$GATE"
|
||||
fi
|
||||
if [ -z "$rcode" ]; then
|
||||
local r=failed
|
||||
[ "$CAUSE" != A5 ] || r=gate-failed
|
||||
log_line result "$r" code "$code" rollback ok
|
||||
rm -f "$RS/AWAITING-UNITS" "$RS/FAILED"
|
||||
echo "stack-release: $code; rolled back to $ROLLBACK_OLD, which passed the gate" >&2
|
||||
exit 1
|
||||
fi
|
||||
say "rollback-failed: run $code, rollback $rcode"
|
||||
log_line result rollback-failed code "$code" rollback failed rollbackCode "$rcode"
|
||||
rm -f "$RS/AWAITING-UNITS" "$RS/FAILED"
|
||||
echo "stack-release: rollback-failed: the run failed with $code and the rollback with $rcode; root stops the units. Rollback set: $RS${BACKUP:+, backup $DATA/backups/stack/$BACKUP}" >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
revert_fail() {
|
||||
say "revert failed: $1"
|
||||
echo "stack-release: revert failed: $1; $O_ACT stays open. Rollback set: $RS" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
# The store already holds backup $1 when every file checks against its SHA256SUMS and the listing
|
||||
# matches, as after a revert that restored it.
|
||||
store_matches() {
|
||||
local o="$DATA/backups/stack/$1"
|
||||
[ -d "$DATA/bus" ] && (cd "$DATA/bus" && sha256sum -c --quiet "$o/SHA256SUMS") >/dev/null 2>&1 &&
|
||||
[ "$(ls -1A "$DATA/bus")" = "$(ls -1A "$o/bus")" ]
|
||||
}
|
||||
|
||||
# R2: puts the wrapper back and, after a migration, the backup taken before it. Each part can run
|
||||
# again, so it serves an A3 that stopped partway. A rollback's --restore swap is confirm's to
|
||||
# report, not revert's to undo. Writes no log line.
|
||||
do_revert() {
|
||||
lock_only
|
||||
open_activation
|
||||
RS="$STATE/stack-rollback/$O_ACT" STEPS="$STATE/stack-rollback/$O_ACT/steps.log"
|
||||
[[ "$O_ACT" =~ ^([0-9]{8}T[0-9]{6}Z)-[0-9a-f]{12}(-[1-9])?$ ]] || revert_fail "$O_ACT is not an activation id"
|
||||
POST="$DATA/bus.post-attempt-${BASH_REMATCH[1]}${BASH_REMATCH[2]}"
|
||||
[ -f "$RS/wrapper" ] && [ -f "$RS/WRAPPER_MODE" ] || revert_fail "the rollback set has no saved wrapper"
|
||||
local tmp
|
||||
tmp="$(mktemp "$BIN/.$WRAPPER_NAME.XXXXXX")" && cp -p "$RS/wrapper" "$tmp" && chmod "$(cat "$RS/WRAPPER_MODE")" "$tmp" && mv -f "$tmp" "$BIN/$WRAPPER_NAME" ||
|
||||
revert_fail "could not put the wrapper back"
|
||||
[ "$(sha256sum < "$BIN/$WRAPPER_NAME" | cut -d' ' -f1)" = "$(sum_of "$RS" wrapper)" ] &&
|
||||
[ "$(stat -c %a "$BIN/$WRAPPER_NAME")" = "$(cat "$RS/WRAPPER_MODE")" ] || revert_fail "the wrapper does not match the rollback set"
|
||||
say "put back the wrapper from $RS"
|
||||
if [ -e "$RS/MIGRATING" ]; then
|
||||
TAKEN="$(cat "$RS/BACKUP" 2>/dev/null || true)"
|
||||
[ -n "$TAKEN" ] || revert_fail "MIGRATING is set and BACKUP names no backup"
|
||||
if store_matches "$TAKEN"; then say "the store already matches backup $TAKEN"
|
||||
else restore_store "$TAKEN" || revert_fail "could not restore backup $TAKEN"
|
||||
fi
|
||||
fi
|
||||
echo "stack-release: revert ok: $O_ACT's wrapper$([ -e "$RS/MIGRATING" ] && echo ' and store are' || echo ' is') back; root installs the old units next"
|
||||
}
|
||||
|
||||
# A2f: gates copy ID with no open activation, on a start at or after --since. Takes no lock and
|
||||
# writes nothing, so it refuses nothing: 0 on a pass, 1 otherwise.
|
||||
do_gate() {
|
||||
FLOOR="$SINCE"
|
||||
read_units || { echo "stack-release: gate: cannot read $BUS_UNIT or $CONSOLE_UNIT" >&2; exit 1; }
|
||||
P_CONSOLE=("${U_CONSOLE[@]}")
|
||||
gate "$ID" "$U_PM" || exit 1
|
||||
}
|
||||
|
||||
# --- pm-on (row 169 item 3) ---
|
||||
|
||||
# Stops the bus alone and waits for it and the stopped check, within --timeout. The Console runs on.
|
||||
stop_bus() {
|
||||
systemctl --user stop "$BUS_UNIT" || return 1
|
||||
sc stop "$BUS_UNIT" || return 1
|
||||
local deadline=$((SECONDS + TIMEOUT)) s
|
||||
while :; do
|
||||
s="$(systemctl --user is-active "$BUS_UNIT" 2>/dev/null || true)"
|
||||
s="$(sc is-active "$BUS_UNIT" 2>/dev/null || true)"
|
||||
case "$s" in inactive|failed) if stopped; then say "stopped $BUS_UNIT"; return 0; fi ;; esac
|
||||
[ "$SECONDS" -lt "$deadline" ] || return 1
|
||||
sleep 1
|
||||
@@ -1326,7 +1722,7 @@ stop_bus() {
|
||||
|
||||
start_bus() {
|
||||
START_EPOCH="$(date +%s)" && START_MS="$(date +%s%3N)" || return 1
|
||||
systemctl --user start "$BUS_UNIT"
|
||||
sc start "$BUS_UNIT"
|
||||
}
|
||||
|
||||
# mos-0848 point 3: put the phase-1 bus unit back (no --pm), restart it and rerun the phase-1 gate.
|
||||
@@ -1348,8 +1744,8 @@ pm_on_fail() {
|
||||
exit 1
|
||||
fi
|
||||
local pwhy="$GATE_WHY"
|
||||
systemctl --user stop "$CONSOLE_UNIT" || true
|
||||
systemctl --user stop "$BUS_UNIT" || true
|
||||
sc stop "$CONSOLE_UNIT" || true
|
||||
sc stop "$BUS_UNIT" || true
|
||||
say "pm-on-down: pm-on $code, phase 1 $pcode"
|
||||
log_line result pm-on-down code "$code" why "${why:-@null}" phase1 "$pcode" phase1Why "${pwhy:-@null}"
|
||||
echo "stack-release: pm-on-down: pm-on failed with $code${why:+ ($why)} and the phase-1 bus with $pcode${pwhy:+ ($pwhy)}; both units are stopped. Rollback set: $RS" >&2
|
||||
@@ -1359,6 +1755,7 @@ pm_on_fail() {
|
||||
# Phase 2: the bus unit alone, on the pointer's copy with --pm, gated with pm_launched.
|
||||
do_pm_on() {
|
||||
lock
|
||||
[ "$UNITS" = user ] || refuse external-pm "under --units external the PM is a flag in root's bus unit; changing it is a re-install with its own go"
|
||||
pointer_for_business
|
||||
[ "$P_PENDING" = 1 ] || refuse not-pending "the pointer is not pending; pm-on follows activate --pm-after-harness"
|
||||
local out
|
||||
@@ -1395,8 +1792,16 @@ do_status() {
|
||||
local open
|
||||
open="$(interrupted | paste -sd' ' -)"
|
||||
[ -z "$open" ] || echo "interrupted run: $open (started, no final line; Jason or Mos settles it)"
|
||||
if [ "$UNITS" = external ]; then
|
||||
local m
|
||||
local -a markers
|
||||
mapfile -t markers < <(compgen -G "$STATE/stack-rollback/*/AWAITING-UNITS" || true)
|
||||
for m in "${markers[@]}"; do
|
||||
echo "awaiting units: $(basename "$(dirname "$m")") (copy $(cat "$m" 2>/dev/null || echo unreadable))"
|
||||
done
|
||||
fi
|
||||
echo "check:"
|
||||
if scripts/stack-copy.sh check "${INSTANCE_FLAGS[@]}" --copies "$COPIES" --bin "$BIN" --business "$BUSINESS" 2>&1 | sed 's/^/ /'; then echo " result: pass"; else echo " result: FAIL"; fi
|
||||
if scripts/stack-copy.sh check "${INSTANCE_FLAGS[@]}" "${UNITS_FLAGS[@]}" --copies "$COPIES" --bin "$BIN" --business "$BUSINESS" 2>&1 | sed 's/^/ /'; then echo " result: pass"; else echo " result: FAIL"; fi
|
||||
[ -n "$P_COPY" ] || return 0
|
||||
echo "bus status:"
|
||||
"$COPIES/$P_COPY/scripts/mosaic" bus status 2>&1 | sed 's/^/ /' || true
|
||||
@@ -1580,6 +1985,9 @@ r_env() { env -i HOME="$HOME" PATH="$PATH" MOSAIC_CONFIG="$R/config/config.json"
|
||||
r_migration() {
|
||||
local copy="$COPIES/$TO" old="$COPIES/$P_COPY" v out rc
|
||||
v="$(node -e "$STORE_JS" -- version "$R/data/bus/bus.sqlite" 2>&1)" || r_abort store-unreadable "$v"
|
||||
# Row 233 (sage-1653): which store the rehearsal ran against, for root's step log. Output only:
|
||||
# the rehearsal's own steps.log has the backup's path on its backup line.
|
||||
echo "BACKUP_STORE $R_BACKUP schema $v"
|
||||
if [ "$v" = "$T_SCHEMA" ]; then r_pass migration "not needed: the backup is already $v"; return 0; fi
|
||||
in_chain "$v" || r_abort store-newer "the backup is $v, which $TO's migration chain does not start from"
|
||||
(umask 077; mkdir "$R/refuse" "$R/refuse/bus" && cp "$R_BK"/bus/* "$R/refuse/bus/") || r_abort scratch-build "cannot copy the backup to $R/refuse"
|
||||
@@ -1650,6 +2058,13 @@ do_rehearse() {
|
||||
trap 'r_fail signal "the rehearsal was interrupted"; r_end' INT TERM HUP
|
||||
read_pointer || r_abort no-pointer "no $POINTER; the pointer's copy is the old copy the checks use"
|
||||
[ "$P_BUSINESS" = "$BUSINESS" ] || r_abort business-mismatch "the pointer is for $P_BUSINESS"
|
||||
# Row 233 (sage-1650, sage-1653): last is the newest backup the log records. With none, exit 3
|
||||
# and no directory, the one rehearse refusal stack-upgrade.sh reads as no-backup.
|
||||
if [ "$R_BACKUP" = last ]; then
|
||||
R_BACKUP="$(from_log newest)" || { r_fail no-backup "the log records no backup"; R_RC=3; r_end; }
|
||||
[[ "$R_BACKUP" =~ ^[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])?$ ]] || r_abort backup-unreadable "the log's newest backup is not an activation id"
|
||||
r_pass backup-last "the log's newest backup is $R_BACKUP"
|
||||
fi
|
||||
copy_exists "$TO" 2>/dev/null || r_abort copy-missing "no copy $TO in $COPIES"
|
||||
copy_ok "$TO" 2>/dev/null || r_abort copy-unreachable "$TO is not reachable from refactor and origin/refactor"
|
||||
target_schema "$TO" || r_abort copy-schema-unknown "cannot read SCHEMA_VERSION and its chain from $COPIES/$TO/packages/bus/src/store.mjs"
|
||||
@@ -1722,11 +2137,16 @@ ID=""
|
||||
case "$OP" in
|
||||
activate|rehearse) [ $# -ge 2 ] || usage; ID="$1"; BUSINESS="$2"; shift 2; [[ "$ID" =~ ^[0-9a-f]{40}$ ]] || usage ;;
|
||||
adopt|pm-on|rollback|status) [ $# -ge 1 ] || usage; BUSINESS="$1"; shift ;;
|
||||
confirm|revert|gate) [ $# -ge 2 ] || usage; ID="$1"; BUSINESS="$2"; shift 2; [[ "$ID" =~ ^[0-9a-f]{40}$ ]] || usage ;;
|
||||
-h|--help) usage 0 ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
[[ "$BUSINESS" =~ ^[a-z0-9][a-z0-9-]*$ ]] || usage
|
||||
parse_opts "$@"
|
||||
# Row 232: the stack-copy.sh check flags and the systemctl the instance-busy message names.
|
||||
UNITS_FLAGS=()
|
||||
SC_NAME="systemctl --user"
|
||||
if [ "$UNITS" = external ]; then UNITS_FLAGS=(--units external); SC_NAME=systemctl; fi
|
||||
COPIES_DIR="$COPIES"
|
||||
instance_select
|
||||
load_config
|
||||
@@ -1741,6 +2161,7 @@ STORE_SCHEMA="" START_EPOCH="" START_MS=0 GATE="" GATE_WHY="" S_CHANGED=""
|
||||
P_COPY="" P_PM=0 P_PENDING=0 P_SCHEMA="" P_CLAUDE="" P_CONSOLE=() RUN_PM=0 RUN_PENDING=0 RUN_CONSOLE=() DROP_TESTER="" PROOF_PLAN="" PROOF_STEP="" SEND_PLAN="" SEND_STEP="" PLAN_REFUSE=()
|
||||
BASE_CONSOLE=() SERVE_FLAGS=() SSO_PLAN="" SSO_STEP="" SSO_DROP="" SSO_NO_TARGET=""
|
||||
R="" R_RC=0 R_EARLY=() R_BK="" R_HOST_PID="" R_CONSOLE_PID="" R_STARTED="" LIVE_CFG="" LIVE_BIZ=""
|
||||
O_ACT="" FLOOR=0
|
||||
case "$OP" in
|
||||
adopt) do_adopt ;;
|
||||
activate) do_activate ;;
|
||||
@@ -1748,4 +2169,7 @@ case "$OP" in
|
||||
rollback) do_rollback ;;
|
||||
status) do_status ;;
|
||||
rehearse) do_rehearse ;;
|
||||
confirm) do_confirm ;;
|
||||
revert) do_revert ;;
|
||||
gate) do_gate ;;
|
||||
esac
|
||||
|
||||
Executable
+348
@@ -0,0 +1,348 @@
|
||||
#!/usr/bin/env bash
|
||||
# Row 233 (I3, #1735): root's upgrade and rollback for a stack whose units are root's system units
|
||||
# (stack-release.sh --units external, row 232). r6's activate A1-A5, A2f, R1-R4 and rollback
|
||||
# (agents/darkwing/work/queue-195/DESIGN.md), as the I3 brief changes them
|
||||
# (docs/plans/2026-10-20_installer-rows.md). Deviations and decisions:
|
||||
# agents/darkwing/work/queue-233/BUILD.md.
|
||||
#
|
||||
# stack-upgrade.sh upgrade --instance dev --bundle FILE --sha SHA [--migrate] [--no-rehearsal]
|
||||
# [--rehearse-port N] [--timeout S]
|
||||
# stack-upgrade.sh rollback --instance dev --to ID [--restore ACT [--events N]] [--timeout S]
|
||||
#
|
||||
# Root runs it from a root-owned checkout of the reviewed commit, never from a copy, because the
|
||||
# stack's user can write the copies. The unit values come from /etc/mosaic-stack/<instance>.args,
|
||||
# which I2's install writes: one case2-unit.sh argument per line, the serve flags after `--`. Only
|
||||
# --copy changes, and the file changes only after confirm passes. The stack's user runs its steps
|
||||
# from its own clone, <home>/src/mosaic-stack, through sudo. Root reads none of that user's output
|
||||
# but exit codes and the rehearsal's BACKUP_STORE line (sage-1653), which it copies into its step
|
||||
# log after a match against a fixed pattern and decides nothing on.
|
||||
#
|
||||
# --no-rehearsal is for a stack whose log records no backup. The rehearsal runs either way: exit 3
|
||||
# from it (no-backup) goes on only with the flag, and exit 0 with the flag refuses
|
||||
# rehearsal-required (sage-1650, sage-1652).
|
||||
#
|
||||
# The lock /run/mosaic-stack/upgrade-<instance>.lock is held from before A1 to the end of the run,
|
||||
# rollback included; every child runs with the lock's descriptor closed. The step log
|
||||
# /var/log/mosaic-stack/upgrade-<instance>.log gets one line per step, `<time> <step> rc=N argv:
|
||||
# <argv>`, and a last `<time> result exit=N <code> <text>` line.
|
||||
#
|
||||
# Exit 0 ok, 1 failed with the old stack running, 2 rollback-failed with the units stopped,
|
||||
# 3 refused with nothing stopped, 4 usage, 5 args-failed: the new stack is live and confirmed, but
|
||||
# the args file still names the old copy.
|
||||
#
|
||||
# Test seams, for packages/cli/tests/stack-upgrade.test.mjs only. Run as root, it refuses any:
|
||||
# STACK_UPGRADE_ETC default /etc/mosaic-stack
|
||||
# STACK_UPGRADE_RUN default /run/mosaic-stack
|
||||
# STACK_UPGRADE_LOG_DIR default /var/log/mosaic-stack
|
||||
# STACK_UPGRADE_UNIT_DIR default /etc/systemd/system
|
||||
# STACK_UPGRADE_HOSTCHECK default /usr/local/lib/mosaic-stack/hostcheck.sh (row 240)
|
||||
# STACK_UPGRADE_RENDER default scripts/case2-unit.sh in this checkout
|
||||
# STACK_UPGRADE_SUDO default sudo
|
||||
# STACK_UPGRADE_SYSTEMCTL default systemctl
|
||||
set -uo pipefail
|
||||
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
|
||||
usage() {
|
||||
cat >&2 <<'USAGE'
|
||||
usage: stack-upgrade.sh upgrade --instance dev --bundle FILE --sha SHA [--migrate] [--no-rehearsal] [--rehearse-port N] [--timeout S]
|
||||
stack-upgrade.sh rollback --instance dev --to ID [--restore ACT [--events N]] [--timeout S]
|
||||
USAGE
|
||||
exit 4
|
||||
}
|
||||
|
||||
if [ "$(id -u)" = 0 ]; then
|
||||
for v in ${!STACK_UPGRADE_@}; do echo "stack-upgrade: refusing: $v is a test seam, and root takes none" >&2; exit 3; done
|
||||
fi
|
||||
ETC="${STACK_UPGRADE_ETC:-/etc/mosaic-stack}"
|
||||
RUN_DIR="${STACK_UPGRADE_RUN:-/run/mosaic-stack}"
|
||||
LOG_DIR="${STACK_UPGRADE_LOG_DIR:-/var/log/mosaic-stack}"
|
||||
UNIT_DIR="${STACK_UPGRADE_UNIT_DIR:-/etc/systemd/system}"
|
||||
HOSTCHECK="${STACK_UPGRADE_HOSTCHECK:-/usr/local/lib/mosaic-stack/hostcheck.sh}"
|
||||
RENDER="${STACK_UPGRADE_RENDER:-$REPO/scripts/case2-unit.sh}"
|
||||
SUDO="${STACK_UPGRADE_SUDO:-sudo}"
|
||||
SYSTEMCTL="${STACK_UPGRADE_SYSTEMCTL:-systemctl}"
|
||||
ID_RE='^[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])?$'
|
||||
|
||||
ARGV=("$@")
|
||||
[ $# -gt 0 ] || usage
|
||||
OP="$1"; shift
|
||||
case "$OP" in upgrade|rollback) ;; *) usage ;; esac
|
||||
INSTANCE="" BUNDLE="" SHA="" MIGRATE=0 NO_REHEARSAL=0 R_PORT=7339 TIMEOUT=60 TO="" RESTORE="" EVENTS=""
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--instance) [ $# -ge 2 ] || usage; INSTANCE="$2"; shift 2 ;;
|
||||
--bundle) [ "$OP" = upgrade ] && [ $# -ge 2 ] && [[ "$2" =~ ^/[A-Za-z0-9/._+-]+$ ]] || usage; BUNDLE="$2"; shift 2 ;;
|
||||
--sha) [ "$OP" = upgrade ] && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9a-f]{40}$ ]] || usage; SHA="$2"; shift 2 ;;
|
||||
--migrate) [ "$OP" = upgrade ] || usage; MIGRATE=1; shift ;;
|
||||
--no-rehearsal) [ "$OP" = upgrade ] || usage; NO_REHEARSAL=1; shift ;;
|
||||
--rehearse-port) [ "$OP" = upgrade ] && [ $# -ge 2 ] && [[ "$2" =~ ^[1-9][0-9]{3,4}$ ]] && [ "$2" -le 65535 ] || usage; R_PORT="$2"; shift 2 ;;
|
||||
--timeout) [ $# -ge 2 ] && [[ "$2" =~ ^[1-9][0-9]{0,4}$ ]] || usage; TIMEOUT="$2"; shift 2 ;;
|
||||
--to) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9a-f]{40}$ ]] || usage; TO="$2"; shift 2 ;;
|
||||
--restore) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ $ID_RE ]] || usage; RESTORE="$2"; shift 2 ;;
|
||||
--events) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ ^(0|[1-9][0-9]*)$ ]] || usage; EVENTS="$2"; shift 2 ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
done
|
||||
# The default instance only, whose name is dev (row 190); a named instance's units are a later row.
|
||||
[ "$INSTANCE" = dev ] || usage
|
||||
if [ "$OP" = upgrade ]; then [ -n "$BUNDLE" ] && [ -n "$SHA" ] || usage; else [ -n "$TO" ] || usage; fi
|
||||
[ -z "$EVENTS" ] || [ -n "$RESTORE" ] || usage
|
||||
|
||||
# --- The step log and the lock ---
|
||||
|
||||
umask 027
|
||||
LOG="$LOG_DIR/upgrade-$INSTANCE.log"
|
||||
LOCK="$RUN_DIR/upgrade-$INSTANCE.lock"
|
||||
{ mkdir -p "$LOG_DIR" "$RUN_DIR" && touch "$LOG" && chmod 0640 "$LOG"; } 2>/dev/null ||
|
||||
{ echo "stack-upgrade: refusing log-failed: cannot write $LOG or make $RUN_DIR; nothing was changed" >&2; exit 3; }
|
||||
logl() { printf '%s %s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$*" >> "$LOG" || true; }
|
||||
# The run's last line, in the log and on the terminal, then the exit.
|
||||
finish() {
|
||||
local rc="$1" code="$2"; shift 2
|
||||
logl "result exit=$rc $code $*"
|
||||
if [ "$rc" = 0 ]; then echo "stack-upgrade: $*"; else echo "stack-upgrade: $code: $*" >&2; fi
|
||||
exit "$rc"
|
||||
}
|
||||
refuse() { local code="$1"; shift; finish 3 "$code" "$* (nothing was stopped)"; }
|
||||
# One step: $1 its name, the rest the command, run with the lock's descriptor closed. One log line
|
||||
# with its exit and argv; the command's output goes to the terminal only.
|
||||
step() {
|
||||
local name="$1" rc=0; shift
|
||||
"$@" 9>&- || rc=$?
|
||||
logl "$name rc=$rc argv:$(printf ' %q' "$@")"
|
||||
return "$rc"
|
||||
}
|
||||
|
||||
exec 9>> "$LOCK" || refuse lock-failed "cannot open $LOCK"
|
||||
flock -n 9 || refuse upgrade-running "another stack-upgrade.sh run holds $LOCK"
|
||||
logl "start rc=0 argv:$(printf ' %q' "$0" "${ARGV[@]}")"
|
||||
|
||||
# --- The args file and the stack's user ---
|
||||
|
||||
# I2's file: one case2-unit.sh argument per line, owned by root with no group or other write. Sets
|
||||
# ARGS (the lines), S_USER, S_HOME, BUSINESS, NODE and ARGS_COPY, and the paths derived from them.
|
||||
read_args() {
|
||||
local f="$ETC/$INSTANCE.args" st i copies=0 v
|
||||
[ -f "$f" ] && [ ! -L "$f" ] || refuse args-unreadable "no regular file $f"
|
||||
st="$(stat -c '%u %a' "$f")" || refuse args-unreadable "cannot stat $f"
|
||||
[ "${st% *}" = "$(id -u)" ] && (( (8#${st#* } & 8#022) == 0 )) || refuse args-unreadable "$f must be $(id -un)'s with no group or other write (it is uid ${st% *}, mode ${st#* })"
|
||||
mapfile -t ARGS < "$f" || refuse args-unreadable "cannot read $f"
|
||||
S_USER="" S_HOME="" BUSINESS="" NODE="" ARGS_COPY=""
|
||||
for ((i = 0; i < ${#ARGS[@]}; i++)); do
|
||||
case "${ARGS[i]}" in
|
||||
--) break ;;
|
||||
--pm) ;;
|
||||
--user|--home|--business|--copy|--node|--profile|--claude)
|
||||
v="${ARGS[i + 1]:-}"
|
||||
case "${ARGS[i]}" in
|
||||
--user) S_USER="$v" ;; --home) S_HOME="$v" ;; --business) BUSINESS="$v" ;; --node) NODE="$v" ;;
|
||||
--copy) ARGS_COPY="$v"; copies=$((copies + 1)) ;;
|
||||
esac
|
||||
i=$((i + 1)) ;;
|
||||
*) refuse args-unreadable "$f has ${ARGS[i]} before --, which case2-unit.sh does not take" ;;
|
||||
esac
|
||||
done
|
||||
[ "$copies" = 1 ] || refuse args-unreadable "$f needs exactly one --copy before --, not $copies"
|
||||
[[ "$S_USER" =~ ^[a-z_][a-z0-9_-]*$ ]] && [[ "$S_HOME" =~ ^/[A-Za-z0-9/._+-]+$ ]] && [[ "$BUSINESS" =~ ^[a-z0-9][a-z0-9-]*$ ]] &&
|
||||
[[ "$NODE" =~ ^/[A-Za-z0-9/._+-]+$ ]] && [[ "$ARGS_COPY" =~ ^[0-9a-f]{40}$ ]] ||
|
||||
refuse args-unreadable "$f lacks a plain --user, --home, --business, --node or --copy"
|
||||
COPIES_DIR="$S_HOME/.local/share/mosaic-stack/copies"
|
||||
CLONE="$S_HOME/src/mosaic-stack"
|
||||
SR="$CLONE/scripts/stack-release.sh"
|
||||
BUS_UNIT="mosaic-bus@$BUSINESS.service" CON_UNIT=mosaic-webui.service
|
||||
# The stack's user's steps: its own HOME and default config, the pinned node first on PATH.
|
||||
AS_USER=("$SUDO" -u "$S_USER" -H env "PATH=$(dirname "$NODE"):/usr/local/bin:/usr/bin:/bin")
|
||||
}
|
||||
|
||||
# RENDER_ARGS: the args file's lines with copy $1 in place of --copy's value.
|
||||
args_with() {
|
||||
local i past=""
|
||||
RENDER_ARGS=()
|
||||
for ((i = 0; i < ${#ARGS[@]}; i++)); do
|
||||
RENDER_ARGS+=("${ARGS[i]}")
|
||||
[ "${ARGS[i]}" != -- ] || past=1
|
||||
if [ -z "$past" ] && [ "${ARGS[i]}" = --copy ]; then RENDER_ARGS+=("$1"); i=$((i + 1)); fi
|
||||
done
|
||||
}
|
||||
|
||||
# The new args file, written beside the old one before anything stops, moved into place only after
|
||||
# confirm passes. $1 is the copy.
|
||||
prepare_args() {
|
||||
args_with "$1"
|
||||
NEW_ARGS="$(mktemp "$ETC/.$INSTANCE.args.XXXXXX")" &&
|
||||
printf '%s\n' "${RENDER_ARGS[@]}" > "$NEW_ARGS" && chmod --reference="$ETC/$INSTANCE.args" "$NEW_ARGS" ||
|
||||
{ rm -f "${NEW_ARGS:-}"; refuse args-failed "cannot write the new args file in $ETC"; }
|
||||
}
|
||||
drop_args() { rm -f "${NEW_ARGS:-}"; }
|
||||
|
||||
# --- A1 ---
|
||||
|
||||
# OLD: the copy both installed units run, from their WorkingDirectory under the stack's copies.
|
||||
read_old() {
|
||||
local f ids=() id n
|
||||
for f in "$UNIT_DIR/[email protected]" "$UNIT_DIR/$CON_UNIT"; do
|
||||
[ -f "$f" ] && [ ! -L "$f" ] || refuse units-unreadable "no regular file $f"
|
||||
n="$(grep -c '^WorkingDirectory=' "$f")"
|
||||
id="$(awk -v c="WorkingDirectory=$COPIES_DIR/" 'index($0, c) == 1 { print substr($0, length(c) + 1) }' "$f")"
|
||||
[ "$n" = 1 ] && [[ "$id" =~ ^[0-9a-f]{40}$ ]] || refuse units-unreadable "$f names no copy under $COPIES_DIR"
|
||||
ids+=("$id")
|
||||
done
|
||||
[ "${ids[0]}" = "${ids[1]}" ] || refuse units-disagree "$UNIT_DIR's units name ${ids[0]} and ${ids[1]}"
|
||||
OLD="${ids[0]}"
|
||||
logl "units rc=0 old: $OLD"
|
||||
[ "$ARGS_COPY" = "$OLD" ] || refuse args-differ "$ETC/$INSTANCE.args names $ARGS_COPY and the units name $OLD"
|
||||
}
|
||||
|
||||
# r6 A1 and row 240: the host's profile still matches, on every host.
|
||||
hostcheck() {
|
||||
[ -f "$HOSTCHECK" ] && [ -x "$HOSTCHECK" ] || refuse profile-drift "no executable $HOSTCHECK (row 240)"
|
||||
step hostcheck "$HOSTCHECK" --check || refuse profile-drift "$HOSTCHECK --check failed"
|
||||
}
|
||||
|
||||
# The stack's user's --plan for the run, $@ its op and arguments; any refusal stops here.
|
||||
plan() {
|
||||
local rc=0
|
||||
step plan "${AS_USER[@]}" "$SR" "$@" --units external --copies "$COPIES_DIR" --plan --expect-from "$OLD" || rc=$?
|
||||
[ "$rc" = 0 ] || refuse plan-refused "the stack's user's $1 --plan exited $rc; its output above names the refusal"
|
||||
}
|
||||
|
||||
# --- The units ---
|
||||
|
||||
stop_wait() {
|
||||
local u s deadline=$((SECONDS + TIMEOUT)) rc=0
|
||||
step stop-console "$SYSTEMCTL" stop "$CON_UNIT" || rc=1
|
||||
step stop-bus "$SYSTEMCTL" stop "$BUS_UNIT" || rc=1
|
||||
for u in "$CON_UNIT" "$BUS_UNIT"; do
|
||||
while :; do
|
||||
s="$("$SYSTEMCTL" is-active "$u" 9>&- 2>/dev/null)"
|
||||
case "$s" in inactive|failed) break ;; esac
|
||||
[ "$SECONDS" -lt "$deadline" ] || { rc=1; break; }
|
||||
sleep 1
|
||||
done
|
||||
case "$s" in inactive|failed) logl "wait-down rc=0 unit: $u state: $s" ;; *) logl "wait-down rc=1 unit: $u state: ${s:-unknown}" ;; esac
|
||||
done
|
||||
return "$rc"
|
||||
}
|
||||
stop_units() { "$SYSTEMCTL" stop "$CON_UNIT" 9>&- || true; "$SYSTEMCTL" stop "$BUS_UNIT" 9>&- || true; logl "stop-units rc=0 units: $CON_UNIT $BUS_UNIT"; }
|
||||
|
||||
# A2f: the old units back on the copy they never left, gated on a start at or after this second.
|
||||
# An A2 that timed out leaves a unit on its old start, which the gate refuses as stale-start.
|
||||
a2f() {
|
||||
local cause="$1" since
|
||||
drop_args
|
||||
since="$(date -u +%s)"
|
||||
step start-bus "$SYSTEMCTL" start "$BUS_UNIT"
|
||||
step start-console "$SYSTEMCTL" start "$CON_UNIT"
|
||||
if step gate "${AS_USER[@]}" "$SR" gate "$OLD" "$BUSINESS" --units external --copies "$COPIES_DIR" --since "$since" --timeout "$TIMEOUT"; then
|
||||
finish 1 failed "$cause failed; $OLD is running again and passed the gate"
|
||||
fi
|
||||
stop_units
|
||||
finish 2 rollback-failed "$cause failed and $OLD failed its gate after the restart; both units are stopped"
|
||||
}
|
||||
|
||||
# R1-R4: back to OLD after the stack's user staged $1. $2 is the cause: A3, A4 or A5.
|
||||
rollback_to_old() {
|
||||
local id="$1" cause="$2"
|
||||
drop_args
|
||||
stop_wait || { stop_units; finish 2 rollback-failed "$cause failed and the units did not stop for the rollback; both units are stopped"; }
|
||||
step revert "${AS_USER[@]}" "$SR" revert "$id" "$BUSINESS" --units external --copies "$COPIES_DIR" ||
|
||||
{ stop_units; finish 2 rollback-failed "$cause failed and revert failed; both units are stopped"; }
|
||||
args_with "$OLD"
|
||||
step render-old "$RENDER" "${RENDER_ARGS[@]}" ||
|
||||
{ stop_units; finish 2 rollback-failed "$cause failed and the render of $OLD failed; both units are stopped"; }
|
||||
# confirm --rollback exits 1 when OLD passed its gate and 2 when it failed; anything else is a
|
||||
# refusal, with OLD's gate never run.
|
||||
local rc=0
|
||||
step confirm-rollback "${AS_USER[@]}" "$SR" confirm "$id" "$BUSINESS" --units external --copies "$COPIES_DIR" --rollback "$OLD" --cause "$cause" --timeout "$TIMEOUT" || rc=$?
|
||||
[ "$rc" != 1 ] || finish 1 failed "$cause failed; rolled back to $OLD, which passed the gate"
|
||||
stop_units
|
||||
finish 2 rollback-failed "$cause failed and the rollback to $OLD did not pass (confirm --rollback exited $rc); both units are stopped"
|
||||
}
|
||||
|
||||
# A2 to A5 for copy $1, once A1 and any rehearsal passed. $@ after it is the stack's user's op.
|
||||
run_steps() {
|
||||
local id="$1" rc=0; shift
|
||||
prepare_args "$id"
|
||||
# A2
|
||||
stop_wait || a2f A2
|
||||
# A3
|
||||
step stage "${AS_USER[@]}" "$SR" "$@" --units external --copies "$COPIES_DIR" || rc=$?
|
||||
case "$rc" in
|
||||
0) ;;
|
||||
3) a2f A3 ;;
|
||||
*) rollback_to_old "$id" A3 ;;
|
||||
esac
|
||||
# A4
|
||||
args_with "$id"
|
||||
step render "$RENDER" "${RENDER_ARGS[@]}" || rollback_to_old "$id" A4
|
||||
# A5
|
||||
step confirm "${AS_USER[@]}" "$SR" confirm "$id" "$BUSINESS" --units external --copies "$COPIES_DIR" --timeout "$TIMEOUT" ||
|
||||
rollback_to_old "$id" A5
|
||||
if mv -f "$NEW_ARGS" "$ETC/$INSTANCE.args"; then logl "args rc=0 copy: $id"
|
||||
else logl "args rc=1 copy: $id"; finish 5 args-failed "$id is live and confirmed, but $ETC/$INSTANCE.args still names $OLD; put $NEW_ARGS in its place"
|
||||
fi
|
||||
}
|
||||
|
||||
do_upgrade() {
|
||||
read_args
|
||||
# Before A1: the bundle into the stack's user's clone, then the copy from it (I1).
|
||||
step fetch "${AS_USER[@]}" git -C "$CLONE" fetch -q "$BUNDLE" refs/heads/refactor:refs/remotes/origin/refactor ||
|
||||
refuse fetch-failed "the stack's user could not fetch $BUNDLE into $CLONE"
|
||||
step ff "${AS_USER[@]}" git -C "$CLONE" merge -q --ff-only "$SHA" ||
|
||||
refuse fetch-failed "the stack's user could not fast-forward $CLONE's refactor to $SHA"
|
||||
step copy "${AS_USER[@]}" "$CLONE/scripts/stack-copy.sh" copy --bundle "$BUNDLE" "$SHA" --copies "$COPIES_DIR" ||
|
||||
refuse copy-failed "the stack's user could not make copy $SHA from $BUNDLE"
|
||||
# A1
|
||||
local head
|
||||
head="$(git -C "$REPO" rev-parse HEAD 2>/dev/null)"
|
||||
[ "$head" = "$SHA" ] || refuse head-mismatch "this checkout, $REPO, is at ${head:-unknown}, not $SHA"
|
||||
[ -x "$COPIES_DIR/$SHA/scripts/mosaic" ] || refuse copy-missing "no $COPIES_DIR/$SHA/scripts/mosaic"
|
||||
hostcheck
|
||||
read_old
|
||||
local -a migrate=()
|
||||
[ "$MIGRATE" = 0 ] || migrate=(--migrate)
|
||||
plan activate "$SHA" "$BUSINESS" "${migrate[@]}"
|
||||
rehearse
|
||||
run_steps "$SHA" activate "$SHA" "$BUSINESS" "${migrate[@]}"
|
||||
finish 0 ok "upgrade ok: $SHA is live for $BUSINESS"
|
||||
}
|
||||
|
||||
# The rehearsal against the log's newest backup, before anything stops. Its output goes to the
|
||||
# terminal; root keeps only the BACKUP_STORE line, so the go and the log show which store it used.
|
||||
rehearse() {
|
||||
local cap rc bs
|
||||
cap="$(mktemp "$RUN_DIR/rehearse-$INSTANCE.XXXXXX")" || refuse rehearsal-failed "cannot make a capture file in $RUN_DIR"
|
||||
local -a cmd=("${AS_USER[@]}" "$SR" rehearse "$SHA" "$BUSINESS" --copies "$COPIES_DIR" --backup last --port "$R_PORT" --timeout "$TIMEOUT")
|
||||
"${cmd[@]}" 9>&- 2>&1 | tee "$cap" 9>&-
|
||||
rc="${PIPESTATUS[0]}"
|
||||
logl "rehearse rc=$rc argv:$(printf ' %q' "${cmd[@]}")"
|
||||
bs="$(grep -E '^BACKUP_STORE [0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])? schema [0-9a-z]{1,16}$' "$cap" | tail -n 1)"
|
||||
rm -f "$cap"
|
||||
logl "rehearse-store rc=0 ${bs:-BACKUP_STORE none}"
|
||||
case "$rc" in
|
||||
0) [ "$NO_REHEARSAL" = 0 ] || refuse rehearsal-required "the log records a backup and the rehearsal ran; --no-rehearsal is only for a stack with none" ;;
|
||||
3) [ "$NO_REHEARSAL" = 1 ] || refuse no-backup "the log records no backup to rehearse against; only a go that names --no-rehearsal goes on"
|
||||
logl "rehearsal-skipped rc=0 the log records no backup and the go names --no-rehearsal" ;;
|
||||
*) refuse rehearsal-failed "the rehearsal exited $rc" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# r6 rollback: to the log's previous copy, which the stack's user checks against --to. No HEAD
|
||||
# check: the target is an older copy than this checkout, and no rehearsal.
|
||||
do_rollback() {
|
||||
read_args
|
||||
[ -x "$COPIES_DIR/$TO/scripts/mosaic" ] || refuse copy-missing "no $COPIES_DIR/$TO/scripts/mosaic"
|
||||
hostcheck
|
||||
read_old
|
||||
local -a restore=()
|
||||
[ -z "$RESTORE" ] || restore=(--restore "$RESTORE")
|
||||
[ -z "$EVENTS" ] || restore+=(--events "$EVENTS")
|
||||
plan rollback "$BUSINESS" --to "$TO" "${restore[@]}"
|
||||
run_steps "$TO" rollback "$BUSINESS" --to "$TO" "${restore[@]}"
|
||||
finish 0 ok "rollback ok: $TO is live for $BUSINESS"
|
||||
}
|
||||
|
||||
case "$OP" in
|
||||
upgrade) do_upgrade ;;
|
||||
rollback) do_rollback ;;
|
||||
esac
|
||||
Reference in New Issue
Block a user