Compare commits
8
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 jason.woltje
|
1a1dc80caa | ||
|
jason.woltje
|
08db6ac3f5 | ||
|
jason.woltje
|
d99166ade2 | ||
|
jason.woltje
|
aacfa2d7a4 | ||
|
jason.woltje
|
e65a3ab4d2 | ||
|
jason.woltje
|
0444acd084 | ||
|
jason.woltje
|
052ba95c7c | ||
|
jason.woltje
|
a52d64ed41 |
@@ -0,0 +1,287 @@
|
||||
# Row 232 (I1): stack-release external units and bundle copies
|
||||
|
||||
Brief: `docs/plans/2026-10-20_installer-rows.md` section I1, blob `f35e7e57daee` (commit ad6f0be7a). Gitea
|
||||
#1734, Vikunja P19 #432. Owner Darkwing, reviewer Filbert, gate owner Sage. Design input: row 195 DESIGN r6,
|
||||
`agents/darkwing/work/queue-195/DESIGN.md` at b7239bb66 (approved in duckworth-0055). Where the brief and
|
||||
r6 differ, the brief wins. sage-1561 ruled where the stack user's `stack-release.sh` runs: from its own
|
||||
clone of the release bundle, so `check_target`, the plan's commit list and status drift are unchanged and
|
||||
nothing adds `safe.directory`.
|
||||
|
||||
## Files
|
||||
|
||||
Base: cb5fdb064 (row 199's landing, on top of row 200's 9e1085816). Branch `darkwing/row232-r1`. The first
|
||||
build, 0af0c73c6 on `darkwing/row232`, sat on 00d4cc1f6 (row 190); row 200 landed first, so this one is
|
||||
rebased onto it (see D9 and D10).
|
||||
|
||||
- `scripts/stack-release.sh`: `--units user|external`, the `sc` and `jctl` backend, `confirm`, `revert`,
|
||||
`gate --since`, `rollback --to`, `--plan --expect-from`, the external gate (start floor, PM check from
|
||||
`launches list`), `external-pm`, and the `awaiting units:` status line.
|
||||
- `scripts/stack-copy.sh`: `check --units external`, `copy --bundle FILE SHA`, `wrapper ID --create`.
|
||||
- `packages/cli/tests/stack-release-external.test.mjs` (new, 25 tests).
|
||||
- `packages/cli/tests/stack-release-fixture.mjs` (new): the helpers `stack-release.test.mjs` defined, moved
|
||||
so both files share them. See deviation D1.
|
||||
- `packages/cli/tests/stack-release.test.mjs`: the helpers leave and one import line comes in. The 74 tests
|
||||
have the same names and the same bodies as on the base.
|
||||
- `packages/cli/tests/stack-copy.test.mjs`: three new `row 232:` tests and a wider `node:fs` import.
|
||||
- `docs/TOOLS.md`: the stack-copy usage block and three paragraphs, the stack-release usage block, and a new
|
||||
subsection "Root-owned units (`--units external`, row 232)".
|
||||
- `packages/cli/README.md`: one paragraph.
|
||||
- This file.
|
||||
|
||||
The worktree's `node_modules` symlink is not part of the candidate.
|
||||
|
||||
## User mode is unchanged
|
||||
|
||||
Every `systemctl` call goes through `sc` and every `journalctl` call through `jctl`. Under `user` they
|
||||
are `systemctl --user "$@"` and `journalctl --user "$@"`. Under `external`, `sc` allows only `show`,
|
||||
`is-active` and `list-units`, and `jctl` returns 1, so an external run can't start, stop, install or read
|
||||
the journal even through a path the tests miss.
|
||||
|
||||
Two tests hold the brief's "every command line and every byte of output":
|
||||
|
||||
- `user mode: every systemctl call carries --user, and activate, rollback and check make the base's
|
||||
calls`. A fixture repo gets the base's two scripts from `git show cb5fdb064:`. The same activate,
|
||||
rollback and `stack-copy.sh check` run from both trees, and the recorded argv (with the fixture root and
|
||||
port replaced by placeholders) must match line for line. Every call carries `--user`.
|
||||
- `user mode: a default activate --plan prints the base's bytes`, for four plans.
|
||||
|
||||
The base output is produced live from the base blobs, not stored as a fixture file. See deviation D3.
|
||||
|
||||
## r6 section 7, test by test
|
||||
|
||||
All in `stack-release-external.test.mjs` unless named.
|
||||
|
||||
| r6 :742-812 item | Test |
|
||||
|---|---|
|
||||
| `units-running` (N4), store, log and TAKEN left | `units-running (N4) refuses while a unit is active...`; also checks that `failed` counts as down and that the Console is checked first |
|
||||
| activate leaves the pointer, writes `AWAITING-UNITS` and a backup | `activate stages: ...`, which also checks the set's file list, `UNITS.sha256`, no `--user`, no journal call and `status` |
|
||||
| `open-activation` with zero, two, the wrong copy, one not open | `confirm and revert refuse open-activation ...` |
|
||||
| `confirm` polls: 3 s fails at `--timeout 1`, passes at `--timeout 10` | `confirm polls: bus status good after 3 s fails ...` and `confirm polls to a pass ...` |
|
||||
| `stale-start` floor with `started` at .900 | `stale-start: @1792540799 fails ...` and `stale-start is a gate code ...` |
|
||||
| `confirm` pass writes pointer, ok, removes marker; timeout writes `FAILED` and exits 1 | the two polling tests |
|
||||
| `activation-failed` when `FAILED` exists | `confirm polls: ... FAILED then refuses activation-failed` |
|
||||
| lines carry the started op, from, to, activation and backup | `sameAsStarted` in every confirm test, and sequence 7 |
|
||||
| `gate` (N2) with and without `--since` | `gate (N2, A2f): ...` and the usage test |
|
||||
| `--plan --expect-from` (N5), activate and `rollback --to` | sequence 7 |
|
||||
| A3 order: capture-failed, marker-failed, wrapper failure after migrate | `A3 order: ...` and sequence 2 |
|
||||
| refusals log no activation; `interrupted` unchanged | `open-activation` test and sequence 5 |
|
||||
| PM check: one, none, two, early, not live, other business, other instance; no journal call | `the external PM check reads launches list ...` (modes '', coder, none, twolive, early, dead, other) and `journalCalls` in every external test |
|
||||
| `external-pm` | `pm-on and activate --pm-after-harness refuse external-pm` |
|
||||
| `revert` restores TAKEN only after a migration | `revert puts the wrapper back and leaves the store ...` and sequences 2 and 8 |
|
||||
| `confirm --rollback` per cause, `unrecorded`, `rollback-failed`, mismatch | sequences 1, 2, 3, 5 and 8, and `confirm --rollback refuses rollback-mismatch, and logs rollback-failed ...` |
|
||||
| sequences 1-8 | `sequence 1:` to `sequence 8:`, each ending with the `closed` check (one started and one result line, empty `interrupted`, no marker or `FAILED`, a following `--plan` that doesn't refuse `interrupted-run`) |
|
||||
| `rollback --to` refuses not-previous; `rollback --restore` finds an external backup | sequence 7, and `confirm --rollback after a --restore swap ...` |
|
||||
| `gate` writes nothing | the `gate (N2, A2f)` test and sequence 4 compare the log, the pointer and the state tree |
|
||||
| `confirm`, `revert`, `gate`, `--to` refuse without `--units external` | `confirm, revert, gate, --to and --expect-from need --units external ...` |
|
||||
| default path: every call carries `--user`; default `--plan` bytes | the two user-mode tests above |
|
||||
|
||||
The brief's additions, in `stack-copy.test.mjs`:
|
||||
|
||||
- `row 232: copy --bundle copies a SHA the bundle's refactor reaches, byte-identical to copy SHA, and
|
||||
refuses the rest`: a short SHA, a missing file, a symlink, a bundle that fails `git bundle verify`, a
|
||||
bundle with no `refs/heads/refactor`, a SHA not in the bundle, and a SHA the bundle carries on another
|
||||
branch that refactor doesn't reach. The bundle copy's tree matches `copy SHA`'s byte for byte, and the
|
||||
`release:` and `bundle sha256:` lines are checked.
|
||||
- `row 232: wrapper --create writes the first wrapper only, the same bytes wrapper ID writes, and refuses
|
||||
over any file`, a symlink included. Without `--create` a missing wrapper still refuses.
|
||||
- `row 232: check --units external reads the system units, the default reads the user units`.
|
||||
|
||||
## Mutants
|
||||
|
||||
`mutants.sh` in my scratch (not committed, it edits the worktree), 17 mutants: r6's re-pin list (:817-838)
|
||||
minus the three `case1-repin.sh` ones, plus the brief's two. Each one applies exact replacements after a
|
||||
checked backup, runs `bash -n`, runs its named killers, restores the file and checks its sha256. Two
|
||||
controls run first with no change: the whole external file, and stack-copy's `row 232:` tests.
|
||||
|
||||
| Mutant | Killers |
|
||||
|---|---|
|
||||
| m01 `confirm` writes the pointer on a gate failure | `confirm polls: bus status` |
|
||||
| m02 `gate_once` in place of `gate` | `confirm polls to a pass`, `stale-start is a gate code` |
|
||||
| m03 PM check counts any instance | `external PM check` |
|
||||
| m04 PM check drops the `startedAt` bound | `external PM check` |
|
||||
| m05 `confirm` takes the newest `AWAITING-UNITS` | `open-activation` |
|
||||
| m06 `rollback --to` skips the comparison | `sequence 7:` |
|
||||
| m07 the default path drops `--user` from one call (`is-active`) | `user mode: every systemctl` |
|
||||
| m08 the timeout writes r4's interim `rollback pending` line | `confirm polls: bus status`, `sequence 1:`, `sequence 5:` |
|
||||
| m09 `AWAITING-UNITS` last, after the wrapper | `sequence 2:` |
|
||||
| m10 a refusal names the open activation | `sequence 5:`, `open-activation` |
|
||||
| m11 `stale-start` is a refusal | `stale-start is a gate code` |
|
||||
| m12 `revert` restores without `MIGRATING` | `revert puts the wrapper back` |
|
||||
| m13 `confirm` logs op `confirm` | `sequence 7:`, `confirm polls to a pass` |
|
||||
| m14 `MIGRATING` after the migration | `sequence 8:` |
|
||||
| m15 `gate` takes its floor from the log without `--since` | `gate (N2` |
|
||||
| m16 `copy --bundle` skips the reachability check | stack-copy `row 232:` |
|
||||
| m17 `wrapper --create` overwrites | stack-copy `row 232:` |
|
||||
|
||||
## Deviations and choices
|
||||
|
||||
D1. The shared helpers moved to a new file, `packages/cli/tests/stack-release-fixture.mjs`. The brief's
|
||||
files list names `stack-release.test.mjs` "where a shared helper moves" but not a destination. Importing
|
||||
from the other test file would run its 74 tests twice, so the helpers need a module of their own.
|
||||
|
||||
D2. The fixture's T17 check (what a run may never remove) now exempts a rollback set's `AWAITING-UNITS` and
|
||||
`FAILED`. `confirm` removes both when it closes the activation (r6 A5 and R4). User mode writes neither, so
|
||||
user-mode runs check exactly what they did.
|
||||
|
||||
D3. The user-mode argv and `--plan` tests read the base's scripts with `git show cb5fdb064:` rather than
|
||||
storing their output as a fixture file. A later row that changes user-mode output on purpose moves the
|
||||
`BASE` constant in the test to its own base, or the test fails. I chose this over a stored fixture because
|
||||
a stored copy can drift from the base without anyone noticing.
|
||||
|
||||
D4. `marker-failed` is reached by making `state/stack-rollback` unwritable. r6 says "an existing `$RS`", but
|
||||
`next_activation` never picks a taken name, so no test can make the run meet an existing `$RS`. The
|
||||
exclusive `mkdir` is still what refuses.
|
||||
|
||||
D5. The stale-start polling case turns the enter time to `@1792540800`, not r6's `@1792540805`, with the
|
||||
started line at 1792540800.900. That also checks that the floor is the whole second, so .900 doesn't push it
|
||||
to 1792540801.
|
||||
|
||||
D6. No fake `sudo` or `case2-unit.sh`. r6's sequences ran under them because r6 had `case1-repin.sh` drive
|
||||
the steps, and that script is I3's. Here the test plays root: it stops and starts the fake system units and
|
||||
installs with the existing `bus-service.sh` and `webui-service.sh` stand-ins, between the real
|
||||
`stack-release.sh` calls, on one log.
|
||||
|
||||
D7. The fake marker in the `open-activation` test is named `20000101T000000Z-<C1 prefix>` so it sorts
|
||||
before the real one. A marker that sorted after it would leave m05 (newest marker) alive.
|
||||
|
||||
D8. m10 sets `ACT` inside `open_activation`, where r6's wording ("ACT set before the checks") lands in this
|
||||
code. `ACT` is set only after every check in `confirm`.
|
||||
|
||||
D9. Row 200's `--sso` is a usage error under `--units external`, with the proof flags, `--no-proof` and
|
||||
`--no-send`. It is a Console flag, and root's Console unit carries the Console flags; the script under
|
||||
external renders no Console line and never calls `console_for_target`, so `sso_checks` doesn't run there
|
||||
either. The usage text, the TOOLS.md bullet and the external usage test name it.
|
||||
|
||||
D10. Row 200's `config_checks` runs under `--units external` too, after `plugin_checks`, in both the
|
||||
`--plan` and run paths of `activate` and `rollback --to`. It checks the target's loader against the stack
|
||||
user's `config.json`, which the bus on a VM still reads, so a `console` key the target refuses would break
|
||||
root's bus start the same way. The cost is one more refusal (`config-console-key`) an external run can
|
||||
print; I took it over a bus that fails after root's start. The rebase changed no user-mode line: the
|
||||
conflicts were the `SSO=0` globals line, the `plan` body (row 200's `SSO_PLAN` line sits in the user
|
||||
branch) and the two `--plan` lines that gained `config_checks`. In the tests, row 200's fixture lines
|
||||
(the loaders, `sso-config.mjs` and the `--sso` cli.mjs line for C1 and C) moved into
|
||||
`stack-release-fixture.mjs` with the rest of the copy setup, and the `ALGS` import stays in
|
||||
`stack-release.test.mjs`, which T57 uses.
|
||||
|
||||
## duckworth-0055's notes on r6 (brief line 169)
|
||||
|
||||
Duckworth's r6 approval (REVIEW-r5.md at 23f5520b5) carried two notes.
|
||||
|
||||
- N1, A2's stop timeout under `gate --since`. A unit that never stopped keeps its old `ActiveEnterTimestamp`,
|
||||
so `gate --since` fails it with `stale-start`, and the run ends at exit 2 with root stopping the units.
|
||||
That is intended: a unit that won't stop is a fair reason to stop. In I1 this is the gate code and its
|
||||
polling (`stale-start: @1792540799 fails ...`, `gate (N2, A2f)`, m11). The run sheet's A2f line and the
|
||||
re-pin test that points "exits 1 with OLD's gate passing" at an A3 exit 3 belong to I3, which owns
|
||||
`scripts/stack-upgrade.sh` and `docs/guides/host-run-sheet.md`. I'll name both in the I3 build.
|
||||
- N2, the claude path under `/opt/mosaic-stack` (gosalyn-0058, sage-1522). I1 doesn't touch it. Under
|
||||
`--units external` the script checks no claude pin of its own (`no-claude-pin` applies only to user
|
||||
mode), because root's unit carries it. The `--claude` value, the managed-settings reasoning, F6 and the
|
||||
run sheet's claude step go to I2 and I3.
|
||||
|
||||
## Run environment
|
||||
|
||||
Worktree `/mnt/storage/scratch/darkwing-r232`, `TMPDIR=/mnt/storage/scratch/tmp/r232`, node from PATH.
|
||||
One external file run takes about 94 s.
|
||||
|
||||
## Results
|
||||
|
||||
Singles on the rebased candidate, before the gate:
|
||||
|
||||
| File | Tests | Pass | Fail |
|
||||
|---|---|---|---|
|
||||
| `packages/cli/tests/stack-release-external.test.mjs` | 25 | 25 | 0 |
|
||||
| `packages/cli/tests/stack-copy.test.mjs` | 16 | 16 | 0 |
|
||||
| `packages/cli/tests/stack-release.test.mjs` (row 200's T57 included) | 192 | 192 | 0 |
|
||||
| `packages/cli/tests/stack-instances.test.mjs` | 23 | 23 | 0 |
|
||||
| `packages/webui/tests/pinned-copy.test.mjs` | 10 | 10 | 0 |
|
||||
| `packages/cli/tests/channel-recovery.test.mjs` (imports `channel-fixture.mjs`, which names the scripts) | 1 | 1 | 0 |
|
||||
| `packages/cli/tests/host.test.mjs` (reads `packages/cli/README.md`) | 42 | 42 | 0 |
|
||||
|
||||
These are every test file that reads `stack-release.sh` or `stack-copy.sh` (a grep of `packages/*/tests` and
|
||||
`scripts/`). None of them pins a sha256 of a changed file. `packages/cli/tests/host.test.mjs` reads
|
||||
`packages/cli/README.md` for a unit file name; this row only adds a paragraph there. The grep and the runs
|
||||
were repeated on the rebased tree on 2026-10-21; the reader set is the same. A dry run of `mutants.sh`
|
||||
matches each of the 17 replacements once on the rebased scripts.
|
||||
|
||||
### Gate, 2026-10-21, Sage's slot (sage-1618)
|
||||
|
||||
Candidate a0ffabc3f on `darkwing/row232-r1`, run by `~/darkwing-scratch/r232/gate.sh`: the cli suite, then
|
||||
every `scripts/test-*.sh`, one after another, stopping at the first failure. busy.sh passed at 01:07:49Z
|
||||
after Launchpad's single `stack-release.test.mjs` run ended. The gate stopped at the first one, so the
|
||||
chain waited for that PID to exit and checked again. The EXIT trap removed the compose network; no
|
||||
container was left.
|
||||
|
||||
| Step | Time (UTC) | Result |
|
||||
|---|---|---|
|
||||
| `packages/cli`, `node --test tests/*.test.mjs` | 01:07:50 to 01:17:53 | 561 pass, 0 fail, 0 skipped |
|
||||
| `scripts/test-auth.sh` | 01:17:53 to 01:17:54 | 15 passed, 0 failed |
|
||||
| `scripts/test-conductor.sh` | 01:17:54 to 01:18:00 | 17 passed, 0 failed |
|
||||
| `scripts/test-config.sh` | 01:18:00 to 01:18:02 | 31 passed, 0 failed |
|
||||
| `scripts/test-discord.sh` | 01:18:02 to 01:18:12 | 78 passed, 0 failed |
|
||||
| `scripts/test-extension-package.sh` | 01:18:12 to 01:18:13 | 18 passed, 0 failed |
|
||||
| `scripts/test-foundation.sh` | 01:18:13 to 01:19:01 | 44 passed, 0 failed |
|
||||
| `scripts/test-land-work.sh` | 01:19:01 to 01:19:14 | 86 passed, 0 failed |
|
||||
| `scripts/test-queue.sh` | 01:19:14 to 01:19:37 | 156 node tests pass, queue suite 28 passed, 0 failed |
|
||||
| `scripts/test-release.sh` | 01:19:37 to 01:19:51 | 14 passed, 0 failed |
|
||||
| `scripts/test-task.sh` | 01:19:51 to 01:20:44 | 98 passed, 0 failed |
|
||||
|
||||
### Mutants, 2026-10-21, same slot
|
||||
|
||||
`~/darkwing-scratch/r232/mutants.sh`, right after the gate. Both controls (the external file whole, and
|
||||
stack-copy's `^row 232: ` tests) exited 0 on the unmutated tree. Each mutant was applied after a checked
|
||||
backup and a syntax check, ran its killers from the table above, and was put back with its sha256
|
||||
checked. The script stops on any restore mismatch, and it ran to its end.
|
||||
|
||||
| Mutant | Killer exit | Result |
|
||||
|---|---|---|
|
||||
| m01 pointer on gate failure | 1 | killed |
|
||||
| m02 gate once | 1 | killed |
|
||||
| m03 PM from any instance | 1 | killed |
|
||||
| m04 no startedAt check | 1 | killed |
|
||||
| m05 newest marker | 1 | killed |
|
||||
| m06 --to not compared | 1 | killed |
|
||||
| m07 one call without --user | 1 | killed |
|
||||
| m08 interim pending line | 1 | killed |
|
||||
| m09 marker written last | 1 | killed |
|
||||
| m10 refusal names the open activation | 1 | killed |
|
||||
| m11 stale-start refuses | 1 | killed |
|
||||
| m12 revert always restores | 1 | killed |
|
||||
| m13 confirm op | 1 | killed |
|
||||
| m14 MIGRATING after migrate | 1 | killed |
|
||||
| m15 gate floor from the log | 1 | killed |
|
||||
| m16 bundle reachability unchecked | 1 | killed |
|
||||
| m17 --create overwrites | 1 | killed |
|
||||
|
||||
17 of 17 killed, 01:22:16 to 01:24:32Z. I read the logs for m01 and m09, two of the quick kills: each failed on
|
||||
its named test's assertion (m09 on `AWAITING-UNITS` in sequence 2), not on a load error.
|
||||
|
||||
## Round 2: rebase onto 8fede5fc6 (sage-1673)
|
||||
|
||||
Rebase only. Before the landing gate, a check of BASE..HEAD on the manifest paths found that row 230
|
||||
(c8bc5cf4c) had changed one of them after the round 1 base, cb5fdb064. Sage ruled a re-round with a
|
||||
scoped review, the same shape as row 210 round 3.
|
||||
|
||||
Branch `darkwing/row232-r2`: the two round 1 commits, a0ffabc3f and 84f1a91e6, cherry-picked onto
|
||||
8fede5fc6 with no conflict, then this section. No code or test changed.
|
||||
|
||||
- `git merge-tree --write-tree origin/refactor darkwing/row232-r1` (refactor at 8fede5fc6) gives tree
|
||||
837bde79236c with no conflict. The cherry-picked tip before this section has the same tree.
|
||||
- `packages/cli/README.md` is a clean 3-way merge. Row 230's hunk is at lines 710-717 (the session credential
|
||||
paragraph); this row's is a new paragraph at 773. With `git patch-id --stable`, the README diff from r1 to
|
||||
r2 is 02b679e2, the same as row 230's README diff `cb5fdb064..8fede5fc6`. This row's README diff is
|
||||
c395ecd2 on both bases (`cb5fdb064..r1` and `8fede5fc6..r2`).
|
||||
- The seven code, test and doc paths other than the README hash the same as in manifest 7a3b4ea1, which
|
||||
Filbert approved in round 1: `docs/TOOLS.md`, `packages/cli/tests/stack-copy.test.mjs`,
|
||||
`packages/cli/tests/stack-release-external.test.mjs`, `packages/cli/tests/stack-release-fixture.mjs`,
|
||||
`packages/cli/tests/stack-release.test.mjs`, `scripts/stack-copy.sh`, `scripts/stack-release.sh`.
|
||||
This file also matched until this section was appended. Its other lines are unchanged.
|
||||
- The one test that reads the README, `host.test.mjs:855`, looks for the unit's `EnvironmentFile=` line.
|
||||
That line is unchanged context in row 230's hunk.
|
||||
- No row 230 test reads `stack-release.sh` or `stack-copy.sh`, and no test of this row reads a row 230 file.
|
||||
|
||||
Nothing was run for this round. The landing gate runs on the r2 tip on Sage's slot.
|
||||
|
||||
Follow-up, doc-only (sage-1673): `scripts/case2-unit.sh:108` and `scripts/case2-precheck.sh:157` cite
|
||||
`scripts/stack-copy.sh:8` and `:22` for `COPIES` and `copy_root`. With this row those lines are `:10` and
|
||||
`:26`. Only comments are affected and no test reads them. Row 233 doesn't touch those scripts, so the fix
|
||||
goes on the #1732 carry list.
|
||||
@@ -0,0 +1,249 @@
|
||||
# Row 233 (I3): root's upgrade driver and the host run sheet
|
||||
|
||||
Brief: `docs/plans/2026-10-20_installer-rows.md` section I3, blob `e61291bcf1c6` (commit a0e3d7cfd), re-pinned
|
||||
by Sage (sage-1685, rev 2631) from `f35e7e57daee` (ad6f0be7a). The later blob only appends I4; the I3 text
|
||||
is the same. Gitea #1735, Vikunja P19
|
||||
#432. Owner Darkwing, reviewer Filbert, gate owner Sage. Design input: row 195 DESIGN r6,
|
||||
`agents/darkwing/work/queue-195/DESIGN.md` at b7239bb66, sections 2, 4 and 7. Where the brief and r6
|
||||
differ, the brief wins. Sage's rulings during the build: sage-1650 and sage-1652 (`--backup last`, the
|
||||
`--no-rehearsal` mapping, `--rehearse-port`, hostcheck), and sage-1653 (`from_log newest`, the two external
|
||||
plan refusals, the `BACKUP_STORE` line).
|
||||
|
||||
## Files
|
||||
|
||||
Base: e65a3ab4d, row 232's landing, on `darkwing/row233-r1`. The build started on ea1e85c7b
|
||||
(`darkwing/row233`, row 232's candidate e6ca230d2 and its BUILD.md) before row 232 landed. The three row 232
|
||||
files this row changes are identical on the two bases, so the uncommitted diff moved across unchanged.
|
||||
|
||||
- `scripts/stack-upgrade.sh` (new): `upgrade` and `rollback`, root's side of r6 A1-A5, A2f and R1-R4.
|
||||
- `packages/cli/tests/stack-upgrade.test.mjs` (new, 20 tests).
|
||||
- `docs/guides/host-run-sheet.md` (new).
|
||||
- `docs/TOOLS.md`: a new subsection "Root's upgrade driver (`scripts/stack-upgrade.sh`, row 233)", plus the
|
||||
row 232 lines below.
|
||||
- Row 232 files, changed on Sage's rulings (the brief's files list doesn't name them):
|
||||
- `scripts/stack-release.sh`:
|
||||
- `rehearse --backup last` takes the newest backup the log records (`from_log newest`) and prints
|
||||
`PASS backup-last`. With none it fails `no-backup`, exits 3 and makes no rehearsal directory.
|
||||
A newest backup that isn't an activation id aborts `backup-unreadable`, exit 1.
|
||||
- The rehearsal prints `BACKUP_STORE <activation id> schema <version>` before its migration step.
|
||||
- `--plan` under `--units external` refuses `open-activation` when any rollback set holds
|
||||
`AWAITING-UNITS`, and `migrate-required` for a migrating target without `--migrate`. Root reads
|
||||
only the plan's exit, so these two have to be refusals there. User mode prints the same bytes.
|
||||
- `packages/cli/tests/stack-release.test.mjs`: three new `row 233:` tests for `--backup last`.
|
||||
- `docs/TOOLS.md`: `--backup ACTIVATION|PATH|last` in the rehearse usage, a paragraph on `last`, and a
|
||||
paragraph on the two external plan refusals.
|
||||
- This file.
|
||||
|
||||
## The brief, item by item
|
||||
|
||||
| Brief item | Where |
|
||||
|---|---|
|
||||
| Root runs from a root-owned checkout, never a copy | `head-mismatch` (A1); TOOLS.md; run sheet section 6 |
|
||||
| I2's render with `/etc/mosaic-stack/<instance>.args`, `--copy` the one changed value | `read_args`, `args_with`; the args test |
|
||||
| The args file changes only after a confirmed run | `prepare_args` before A2, `mv` after A5; sequences 1 and 7, m6 |
|
||||
| `copy --bundle` as the stack's user before A1 | the `fetch`, `ff`, `copy` steps; sequence 1's sudo keys |
|
||||
| A rehearsal before A2, a failed one refuses with nothing stopped | `rehearse`; three rehearsal tests, m4 |
|
||||
| The root lock, held to the end, rollback included | `flock -n 9`; the lock test and sequence 3's probe, m1, m2 |
|
||||
| A1 refuses `open-activation` | stack-release.sh's external plan; the AWAITING-UNITS test, m3 |
|
||||
| `migrate-required` without `--migrate` | stack-release.sh's external plan; its test, m5 |
|
||||
| The step log, 0640, one line per step | `logl`, `step`; every test checks lines against `STEP` |
|
||||
| Exit codes 0-3 | `finish`, `refuse`; exit 4 is usage |
|
||||
| duckworth-0055's notes | the next section |
|
||||
| Root parses no stack user output beyond exits | the step log test; the one exception is sage-1653's `BACKUP_STORE` |
|
||||
| The host run sheet | `docs/guides/host-run-sheet.md` |
|
||||
|
||||
## duckworth-0055's notes on r6 (brief line 169)
|
||||
|
||||
- N1, A2's stop timeout under `gate --since`. Row 232's BUILD.md left two pieces to I3. The run sheet's
|
||||
A2f line is in section 6 ("When a run doesn't exit 0", exit 2). The test is `N1: a bus that outlives
|
||||
A2's timeout keeps its old start`: a bus that ignores `stop` keeps its old `ActiveEnterTimestamp`,
|
||||
`wait-down` logs `rc=1 ... state: active`, A2f restarts and gates, the gate fails `stale-start`, and the
|
||||
driver stops both units and exits 2. Sequence 4 is the other half: an A3 exit 3 goes to A2f and exits
|
||||
1 with the old copy's gate passing.
|
||||
- N2, the claude path under `/opt/mosaic-stack`. Run sheet section 3 installs it there as root with
|
||||
managed settings. The args file's `--claude` carries it, and `case2-unit.sh` already refuses a binary
|
||||
that isn't root's or sits under the home. The driver passes the args file through and adds no claude
|
||||
check of its own. F6 is I2's.
|
||||
|
||||
## Sage's rulings during the build
|
||||
|
||||
- sage-1650 and sage-1652. Root always runs the rehearsal and maps its exit:
|
||||
- 0 goes on, or refuses `rehearsal-required` with `--no-rehearsal`.
|
||||
- 3 refuses `no-backup`, or with `--no-rehearsal` logs `rehearsal-skipped` and goes on.
|
||||
- Anything else refuses `rehearsal-failed`.
|
||||
|
||||
`--rehearse-port N`, default 7339. Hostcheck: a missing file or a nonzero `--check` refuses
|
||||
`profile-drift`.
|
||||
- sage-1653. `from_log newest`, and the external plan refusals `open-activation` and `migrate-required`.
|
||||
Residual: the `BACKUP_STORE` id and schema land in root's step log, but nothing ties that backup to the
|
||||
one the activation's own backup step takes later. A stopped backup before the upgrade would close that
|
||||
gap; it's a later row, not this one.
|
||||
|
||||
## Tests
|
||||
|
||||
`stack-upgrade.test.mjs` runs the driver as the test's user against a scratch root, with the test seams
|
||||
`STACK_UPGRADE_*`. The stand-ins are a fake `sudo` (logs its argv, checks the driver's form, then runs the
|
||||
real command as the test's user, r6's B1 pattern), a fake `case2-unit.sh` render that installs and starts
|
||||
the fixture units, a fake `systemctl` for root's system units, and a shell `hostcheck.sh`. The stack's
|
||||
user's steps run the real `stack-release.sh` and `stack-copy.sh` from a clone made from a real bundle. The
|
||||
units, copies and store are `stack-release-fixture.mjs`'s.
|
||||
|
||||
| Test | Covers |
|
||||
|---|---|
|
||||
| sequence 1 | confirm fails healthz, rollback with `--cause A5`, exit 1; every sudo call in the driver's form |
|
||||
| sequence 2 | `activate --migrate` fails at the wrapper, store restored, `--cause A3`, exit 1 |
|
||||
| sequence 3 | render fails, `--cause A4`; a probe run during `render-old` refuses `upgrade-running` |
|
||||
| sequence 4 | stage exits 3, A2f restarts and gates the old units, exit 1 |
|
||||
| N1 | a unit that won't stop: A2f's gate fails `stale-start`, exit 2, units stopped |
|
||||
| sequence 5 | confirm of the wrong copy refused, `--cause A5`, exit 1 |
|
||||
| sequence 6 | revert fails, exit 2; the next run refuses `interrupted-run` at its plan |
|
||||
| sequence 7 | two upgrades and a `rollback --to` with `--restore`; the args file after each confirm; `to-not-previous` |
|
||||
| sequence 8 | a migration that writes and fails, store back byte for byte, exit 1 |
|
||||
| lock | a second run while the lock is held: one log line, no call |
|
||||
| open-activation | any `AWAITING-UNITS` refuses at A1, nothing stopped |
|
||||
| migrate-required | refused without `--migrate`, no rehearsal, nothing stopped |
|
||||
| rehearsal-failed | the rehearsal's argv, exit 3, nothing stopped |
|
||||
| `BACKUP_STORE` | the line after a match; four malformed lines log `BACKUP_STORE none` |
|
||||
| no-backup | the real rehearsal with no backup refuses; `--no-rehearsal` logs the skip and goes on |
|
||||
| backup-last | the real rehearsal names the newest backup; `--no-rehearsal` with one refuses `rehearsal-required` |
|
||||
| args file | group write, other write, two `--copy`, an unknown flag, a symlink, `args-differ` |
|
||||
| hostcheck | nonzero `--check` and a missing file refuse `profile-drift` before the plan |
|
||||
| head-mismatch | checkout not at `--sha`; a commit the clone can't fast-forward to refuses `fetch-failed` |
|
||||
| usage | twelve bad argvs exit 4 with no log and no lock |
|
||||
|
||||
The step log check (`STEP`) runs in every test that reads the log: each line must be a step line or the
|
||||
result line, and none may hold the stack user's output (`stack-release:`, `PASS `, `FAIL `, `REHEARSAL`,
|
||||
the fake rehearsal's text).
|
||||
|
||||
## Mutants
|
||||
|
||||
`~/darkwing-scratch/r233/mutants.sh`, not committed because it edits the worktree. It has the brief's six
|
||||
mutants and two for sage-1650's mapping. The method is row 232's: a checked backup, exact replacements, a
|
||||
syntax check, the named killers, then the file put back with its sha256 checked. One control runs first:
|
||||
the whole `stack-upgrade.test.mjs` with no change. `DRY=1` matched each replacement once on the candidate
|
||||
(2026-10-21).
|
||||
|
||||
| Mutant | File | Killers |
|
||||
|---|---|---|
|
||||
| m1 drop the flock | stack-upgrade.sh | `a second run while the lock is held`, `sequence 3:` |
|
||||
| m2 release the lock when the rollback starts | stack-upgrade.sh | `sequence 3:` |
|
||||
| m3 skip A1's open-activation check | stack-release.sh | `A1 refuses open-activation` |
|
||||
| m4 skip the rehearsal | stack-upgrade.sh | `the rehearsal runs after the plan`, `sequence 1:` |
|
||||
| m5 accept a migrating target without `--migrate` | stack-release.sh | `migrate-required` |
|
||||
| m6 write the args file before confirm | stack-upgrade.sh | `sequence 7:`, `sequence 1:` |
|
||||
| m7 `--no-rehearsal` accepted with a backup | stack-upgrade.sh | `refuses rehearsal-required` |
|
||||
| m8 drop the `rehearsal-skipped` line | stack-upgrade.sh | `a real rehearsal with no backup` |
|
||||
|
||||
## Deviations
|
||||
|
||||
D1. The brief has A1 refuse `open-activation` for an open `AWAITING-UNITS` or a non-empty `interrupted`
|
||||
set. The interrupted case refuses with row 232's existing code, `interrupted-run`; only `AWAITING-UNITS`
|
||||
gives `open-activation`. Both reach root as `plan-refused`, exit 3 (sequence 6, the open-activation test).
|
||||
|
||||
D2. Root copies one line of the stack user's output into its log, `BACKUP_STORE`, on sage-1653. It
|
||||
matches a fixed pattern first, takes the last match, logs `BACKUP_STORE none` otherwise, and decides
|
||||
nothing on it.
|
||||
|
||||
D3. The stack user's steps carry no `--instance`. The driver takes `--instance dev` only, row 190's default
|
||||
instance, which the stack user's commands select with no flag. A named instance's units are a later row.
|
||||
|
||||
D4. Most tests fake the rehearsal in the fake `sudo`: the fixture's stub copies can't pass a real one. Two
|
||||
tests run the real rehearsal (`faults/real-rehearse`): no-backup and backup-last. `stack-release.test.mjs`
|
||||
covers `--backup last` itself.
|
||||
|
||||
D5. Any nonzero `--plan` exit refuses `plan-refused`, exit 3, whatever the code. The stack user's output
|
||||
above it names the refusal; root doesn't parse it.
|
||||
|
||||
D6. `rollback --restore ACT` is accepted without `--events`, as `stack-release.sh rollback` accepts it.
|
||||
The brief's usage pairs them. `--events` without `--restore` is a usage error.
|
||||
|
||||
D7. The refusal of any `STACK_UPGRADE_*` seam when run as root has no test, because the tests don't run
|
||||
as root. It is three lines at the top of the script.
|
||||
|
||||
D8. The fake `sudo` keeps the test's environment, where real `sudo` resets it. The driver passes
|
||||
everything the stack user needs through `env PATH=...` and `-H`, and the fake checks that form.
|
||||
|
||||
D9. Exit 3 means nothing stopped, not nothing changed. A refusal after `fetch` leaves the stack user's
|
||||
clone fast-forwarded and the new copy made. Both are the user's own state, and the next run reuses them.
|
||||
TOOLS.md and the run sheet say so.
|
||||
|
||||
D10. The brief's usage has neither `--no-rehearsal` nor `--rehearse-port`. Both come from sage-1650 and
|
||||
sage-1652.
|
||||
|
||||
## Decisions
|
||||
|
||||
- `rollback` has no HEAD check. Its target is an older copy than root's checkout, which already exists.
|
||||
It fetches, copies and rehearses nothing (sequence 7).
|
||||
- The new args file is written next to the old one (`.dev.args.XXXXXX`, same mode) before A2, so a full
|
||||
`/etc` refuses before anything stops. It moves into place after A5. A2f and the rollback remove it; a failed move keeps it and names it.
|
||||
- Every stack user step gets `--copies` explicitly, so the copies directory comes from the args file's
|
||||
`--home`, never from the user's environment.
|
||||
- The stack's user does the fetch, the fast-forward and the copy, so root never runs git on a tree that
|
||||
user can write.
|
||||
- A4 is the render, and `case2-unit.sh` starts the bus, then the Console. The forward path has no
|
||||
`systemctl start` of its own; only A2f does.
|
||||
|
||||
## Run environment
|
||||
|
||||
Worktree `/mnt/storage/scratch/darkwing-r233`, `TMPDIR=/mnt/storage/scratch/tmp/r233`, node from PATH. One
|
||||
whole `stack-upgrade.test.mjs` run takes about 56 s.
|
||||
|
||||
## Results
|
||||
|
||||
### Singles, run 1, 2026-10-21, Sage's slot (sage-1689)
|
||||
|
||||
`~/darkwing-scratch/r233/singles.sh` on `darkwing/row233-r1` (base e65a3ab4d, uncommitted tree), 03:48:20Z
|
||||
to 03:59:00Z. These are the test files that read a changed script. `stack-instances.test.mjs` reads
|
||||
`stack-release.sh`, so it joined the three named in the plan.
|
||||
|
||||
| File | Result |
|
||||
|---|---|
|
||||
| `stack-upgrade.test.mjs` | 20 pass, 0 fail |
|
||||
| `stack-release.test.mjs` | 194 pass, 1 fail |
|
||||
| `stack-release-external.test.mjs` | 25 pass, 0 fail |
|
||||
| `stack-instances.test.mjs` | 23 pass, 0 fail |
|
||||
|
||||
The failure was my test, not the script. "row 233: --backup last takes the newest backup..." compares
|
||||
`untouched(e)` before and after the run, and `untouched` includes `existsSync(e.log)`. The test took its
|
||||
before snapshot ahead of `rlog()`, which writes the log, so that flag went from false to true. Every hash in
|
||||
the snapshot matched. The fix takes the snapshot after `rlog()`. No other row 233 test calls `untouched`.
|
||||
|
||||
### Singles, run 2
|
||||
|
||||
`~/darkwing-scratch/r233/rerun.sh` on aacfa2d7a, Sage's slot (sage-1695), 04:13:05Z to 04:21:28Z:
|
||||
`stack-release.test.mjs` whole, 195 pass, 0 fail, 0 skipped. The fixed test passed. The other three files
|
||||
did not change after run 1. The gate and the mutants run next, in Sage's slot.
|
||||
|
||||
### Gate and mutants, 2026-10-21, Sage's slot (sage-1704)
|
||||
|
||||
`~/darkwing-scratch/r233/gate.sh` then `mutants.sh`, chained behind `busy.sh`, on d99166ade (code
|
||||
aacfa2d7a; d99166ade changes only this file). Counts are an awk sum over `gate-r1.out`.
|
||||
|
||||
| Part | Time (UTC) | Result |
|
||||
|---|---|---|
|
||||
| cli suite (`node --test tests/*.test.mjs`) | 04:29:01-04:37:27 | 616 pass, 0 fail, 0 skipped |
|
||||
| the 10 `scripts/test-*.sh` | 04:37:27-04:39:38 | selftests 429 passed, 0 failed; `test-queue`'s node part 156 pass, 0 fail |
|
||||
| gate total | | 1201 pass, 0 fail, 11 suites, `GATE_PASSED` |
|
||||
| c0 control | 04:39:38-04:40:35 | exit 0 |
|
||||
| m1-m8 | 04:40:35-04:41:25 | 8 of 8 killed, every one exit 1 |
|
||||
|
||||
Every restore passed its sha256 check, and `git diff` was clean at d99166ade afterwards. The EXIT trap
|
||||
removed the `darkwing-r233_default` network.
|
||||
|
||||
N1, found in the process sweep after the run: a `sleep 60` was left with its cwd in `packages/cli`.
|
||||
`stack-upgrade.test.mjs:312` spawns `flock LOCK sleep 60`, and its `t.after` SIGKILLs only the flock, so
|
||||
flock's child outlives the test by up to 60 s. It exited on its own. Row 232's T20
|
||||
(`stack-release.test.mjs:370`) spawns its holder `detached` and kills the group with `process.kill(-pid)`.
|
||||
The fix is the same two lines. That test is one of m1's killers.
|
||||
|
||||
Sage ruled it a fix now (sage-1707), and 08db6ac3f makes it, test only. After the gate's d99166ade
|
||||
the only change outside this file is that test:
|
||||
|
||||
```
|
||||
$ git diff --stat d99166ade 08db6ac3f
|
||||
packages/cli/tests/stack-upgrade.test.mjs | 4 ++--
|
||||
1 file changed, 2 insertions(+), 2 deletions(-)
|
||||
```
|
||||
|
||||
The gate stands on Sage's ruling. The rerun is the whole `stack-upgrade.test.mjs` singly plus m1, in
|
||||
Sage's slot, with a sweep for a leftover `sleep` afterwards.
|
||||
@@ -1,275 +0,0 @@
|
||||
# Row 234 (#436a row a, Gitea #1740): host enrollment, store records and bus schema 3i to 3j
|
||||
|
||||
Gizmoduck, 2026-10-21. Reviewer Duckworth, gate owner Sage. Lands by 10-25, before the .53 and .54 installs
|
||||
(mos-1134).
|
||||
|
||||
- Brief: `docs/plans/2026-10-20_host-credentials.md`, section "Machine grant row a: enrollment, store records
|
||||
and migration 3i-3j", blob 64451be5 (queue rev 2617).
|
||||
- Design: `agents/gizmoduck/work/machine-grant/DESIGN.md` r6, f9039682e on `gizmoduck/machine-grant-design`,
|
||||
approved by Duckworth (duckworth-0064, REVIEW-r6.md at 8ea792256). Not reopened here. Cites "r6:" are lines
|
||||
of that file.
|
||||
- Grant event rule: #436b section 7, fa5658941 on `megavolt/row199`,
|
||||
`agents/megavolt/work/436b/DESIGN.md:431-490`, file sha256
|
||||
96510b83778e79a71b33d0d5f6fd0c806dfe74f6de315ff4a9f1fbe5620f6c69, approved by Duckworth (duckworth-0067).
|
||||
- Base: origin/refactor 8fede5fc6, with rows 199 (cb5fdb064) and 210 (f5753c0e8) landed. Branch
|
||||
`gizmoduck/row234`. Code cites are `path:line` at 8fede5fc6 unless they say otherwise.
|
||||
|
||||
## Names: r6 against section 7
|
||||
|
||||
r6's `grant.*` and `host.enroll.*` names are `body.action` values on `action.allowed` events, not event
|
||||
kinds. The `events` kind CHECK isn't rebuilt. Four r6 names move under `grant.`:
|
||||
|
||||
| r6 | This row |
|
||||
|---|---|
|
||||
| `host.enroll.step` | `grant.host.enroll.step` |
|
||||
| `host.enrolled` | `grant.host.enrolled` |
|
||||
| `host.checked` | `grant.host.checked` |
|
||||
| `host.removed` | `grant.host.removed` |
|
||||
|
||||
Only the consume record carries `decision`: `{operation: 'grant.host-enroll', decision}`. Every other
|
||||
guarded body names the decision `grantDecision`, so r6's `decision` field in step, enrolled and failed bodies
|
||||
is `grantDecision` here. The actor of every guarded event is `actor_role` NULL with `actor_run` 'grant',
|
||||
and `body.by` names the caller the broker checked.
|
||||
|
||||
## Split (r6 section 12 Q-S4)
|
||||
|
||||
Row a builds:
|
||||
|
||||
1. **Schema 3i to 3j.** `migrations/3i-3j.sql` adds `hosts` and `host_grants` (r6 S1, primary keys
|
||||
(business, host) and (business, agent, host)) and the guard trigger: an `action.allowed` whose
|
||||
`body.action` or `body.operation` GLOBs `'grant.*'` aborts unless the actor is (NULL, 'grant').
|
||||
`SCHEMA_VERSION` 3j, a migration test with a step that fails partway.
|
||||
2. **Grant event writer.** A private session-less broker writer for guarded events. `#event` doesn't change.
|
||||
`recordEvent` refuses `reserved-event` for the guarded namespace.
|
||||
3. **The `grant` raise frame.** `decision.raise` takes `grant` only with `credential.mint`, only from the chief
|
||||
(`grant-chief-only`), and in row a only with `method: 'host-enroll'`. Any other method refuses
|
||||
`invalid-request` until row b adds `host`. E1's checks, the `ssh-keyscan` set compare before the raise
|
||||
transaction, and the rendered block.
|
||||
4. **Enrollment executor.** `#consumeGrant`, the start on the human's cli yes (X1), E2 steps 1 to 4, the
|
||||
installer key at `<configDir>/secrets/hosts/<business>/<hostId>/installer`, the failure cleanup, and the
|
||||
start reconcile for enrollments: the `registry` step undo, and a resolved-yes unspent enrollment ending
|
||||
`grant.failed interrupted` (X3).
|
||||
5. **`hosts.check` and `hosts.remove`** (E4), with `mosaic hosts status|check|remove`.
|
||||
6. **`tools/host/`**: `mosaic-keys`, `mosaic-keys-write`, `mosaic-keys.sudoers`, `mosaic-keys.sshd.conf` (E3).
|
||||
7. **`host.run` on the store** (X4 first and third bullets): it reads the `hosts` and `host_grants` records
|
||||
and the Approve record check, and at start the runtime gives `Credentials` a reference per `host_grants`
|
||||
record. The loader refuses `hosts` at the top level and in a role (S1). Row 210's tests move to store
|
||||
fixtures.
|
||||
8. **Reader filter** (duckworth-0067 N1): every grant-evidence reader in this row (status, the start
|
||||
reconcile, `openGrantDecisions`) filters `kind = 'action.allowed' AND actor_role IS NULL AND actor_run =
|
||||
'grant'`, with a test that an `action.refused` look-alike and a session-written one are ignored.
|
||||
`hosts.remove`'s `hostLineLeft` reader is one of them (duckworth-0079 M4).
|
||||
|
||||
Row b keeps the `host` method's raise binding (R4), the grant executor (X2), the grant rows of X3, X5,
|
||||
`Credentials.register`/`claim`/`abortClaim`, revoke (section 6) and `sweep`. So row a has no way to make a
|
||||
`host_grants` record outside a test fixture, and from its landing to row b's no agent holds a host grant
|
||||
(r6 N9, the run sheet strips row 210's keys).
|
||||
|
||||
Tests in row a (r6 section 10): G1 and G5 for the `host-enroll` raise, G6 and G10 for the enrollment
|
||||
executor, G11's enrollment cases, G14, G15, G16, G17, G18, G20, G21 (its `host-line-left` case reads an
|
||||
event fixture until row b writes `hostLineLeft`), G25, the migration test, the trigger cases, the reader
|
||||
filter test, and row 210's T-tests on store fixtures. Row b: G2 to G4, G7 to G9, G11's grant cases, G12,
|
||||
G13, G19, G22 to G24, G26 to G28.
|
||||
|
||||
## Early check items (duckworth-0079, REVIEW-build-split.md 10672772e on `duckworth/reviews`)
|
||||
|
||||
Moves into row a:
|
||||
- M1. G4 for the `host-enroll` block (r6:240-242): a question can't forge the block, and the 4096-character
|
||||
limit holds with the block appended. G4 for the `host` block stays in row b.
|
||||
- M2. The tag function (r6:265-266), because E2 step 2 tags the installer line. G24's first sentence moves
|
||||
here: names of 64 characters with `.` and `_` give distinct tags of the fixed length. The `sweep` half
|
||||
stays in row b.
|
||||
- M3. A store-fixture test that `host.run` refuses `host-not-granted` for no record, an unresolved decision,
|
||||
a decision resolved by an arbiter, the tester or an agent or not via cli, a consume record with operation
|
||||
`grant.host-enroll` in place of `grant.host`, and an unspent decision (X4 first bullet, r6:593-596). A
|
||||
restart test that a `host_grants` fixture gives `Credentials` a reference (X4 third bullet).
|
||||
|
||||
G21's `host-line-left` case:
|
||||
- F1. The fixture goes through this row's private grant writer in the real shape: `action.allowed`, actor
|
||||
(NULL, 'grant'), `body.action` 'grant.failed', `body.host`, `body.hostLineLeft`. No raw SQL around the
|
||||
trigger.
|
||||
- F2. `hosts.remove` reads `body.hostLineLeft` from any guarded body, not only `grant.failed`.
|
||||
- F3. Decided: a guarded body that carries `hostLineLeft` also carries `host`. Row b adds `host` to
|
||||
`grant.revoke.closed`, so its body is `{tag, host, hostLineLeft}` (r6:716 has `{tag, hostLineLeft}`).
|
||||
The reader matches on `body.host`, and doesn't map tags back to hosts through `grant.step`. A direct
|
||||
field is one equality in SQL, and a tag lookup would add a second reader that the N1 filter also has to
|
||||
cover.
|
||||
- F4. Row b carry: "no later `sweep` cleared it" (r6:403) needs `grant.sweep`, which only row b writes. Row
|
||||
b adds the sweep clear to `hosts.remove` with a G21 case, and its G8 or G13 shows `hosts.remove`
|
||||
refusing after a real `hostLineLeft` write, not only after the fixture. Until then row a's check is "any
|
||||
guarded `hostLineLeft` for this host refuses".
|
||||
|
||||
## Mutants (r6:838-883, numbered here)
|
||||
|
||||
Row a, killed on this row's tree:
|
||||
|
||||
| # | Mutant |
|
||||
|---|---|
|
||||
| A1 | drop the chief check |
|
||||
| A2 | start the executor on a `credential.mint` with no `grant` |
|
||||
| A3 | start it on an arbiter resolve |
|
||||
| A3b | start it on a `cli` yes by someone other than the human (duckworth-0083 N8) |
|
||||
| A4 | start it on `no` |
|
||||
| A5 | skip the keyscan compare |
|
||||
| A6 | compare only one key |
|
||||
| A7 | write the private key into the event body |
|
||||
| A8 | drop the "not yet enrolled" check in E2 step 1 |
|
||||
| A9 | drop the `hosts` primary key |
|
||||
| A10 | skip the start check for a resolved, unspent enrollment decision |
|
||||
| A11 | `host.run` reads the business file instead of the grant record |
|
||||
| A12 | build the tag without the business |
|
||||
| A13 | drop `from=` in the writer |
|
||||
| A14 | let the writer touch a line outside the tag pattern |
|
||||
| A15 | drop `O_NOFOLLOW` on the user file |
|
||||
| A16 | drop one group from the writer's refused list |
|
||||
| A17 | drop the user pattern check before the path use |
|
||||
| A18 | treat a `privileged` entry as enough without `users` |
|
||||
| A19 | let the writer accept a sudo user not in `privileged` |
|
||||
| A20 | trigger: drop the `body.operation` arm, so the consume record passes from a session |
|
||||
| A21 | trigger: accept actor_role NULL with any actor_run |
|
||||
| A22 | a grant reader drops the `kind = 'action.allowed'` or actor filter. The runner splits it: A22a drops the kind, and G21's `action.refused` look-alikes kill it. A22b drops the actor, which the trigger can't stand in for, because 3i-3j adds the trigger without checking old rows. Its killer is the grant-migration test "A22b, sage-1703": a session-written `grant.host.removed` with `hostLineLeft` and a `grant.host-enroll` row from 3i, migrated to 3j, where `hosts.remove` still removes the host and `reconcileGrants` ends nothing |
|
||||
| A23 | `host.run` skips the Approve record check (any `host_grants` record is enough) |
|
||||
|
||||
Row b keeps the rest of r6's list: the `ready` check, the three hash compares, the definition check, the
|
||||
install and record order, `remove` on a verify failure, both start sweeps, the reconcile's remote `remove`,
|
||||
the `grant.step` before install, revoke leaving the record, `register` before `issue`, the abort check and
|
||||
the `await` before `issue`, revoke and an unreferenced claim, `sweep` and any `mosaic-` line, revoke's host
|
||||
step order, claim before consume, the resolved unspent grant decision, an async `commit`, `hosts.revoke`
|
||||
through `credential-verbs.mjs`, the revoke marker reconcile, the upsert, the undo past `grant.revoked`, the
|
||||
abort after step 5 and a missing `bus.tester` as dev.
|
||||
|
||||
## Broker functions this row touches (sage-1677)
|
||||
|
||||
For row 236 (Megavolt), which rebuilds on this landing. Old lines are `packages/bus/src/broker.mjs` at
|
||||
8fede5fc6, new lines are at the last commit that changed this file (they moved by 10 after `runEnrollment`
|
||||
went in). The planned list is kept in this file's history (827b89ea6).
|
||||
|
||||
Changed:
|
||||
- `constructor` (:279): one more option, `hostGrant`, the object `host-grant.mjs` builds.
|
||||
- `recordEvent` (old :1243-1251, new :1606): one more refusal. An `action.allowed` whose `body.action` or
|
||||
`body.operation` is a `grant.*` name refuses `reserved-event`.
|
||||
- `request()` (old :1362, new :1726): two routes ahead of the sync path, a `decision.raise` with `grant` to
|
||||
`#raiseGrant` and the `hosts.*` verbs to `#requestHosts`. Both are async.
|
||||
- `requestHost` (old :1712-1795, new :2079): the business-file lookups (old :1730-1736) become one
|
||||
`#hostAccess` call, in the same transaction as `#agent`. `commands` and `entry` come from it.
|
||||
- `#dispatch` (old :1796, new :2160) gets a fifth parameter, `enroll`, which only `#raiseGrant` passes.
|
||||
- `decision.raise` (new :2244): `grant` joins the allowed keys, and `grant` without `enroll` (or the
|
||||
reverse) refuses `invalid-request`. The `host.sudo` block reads `#hostAccess` in place of `b.hosts`
|
||||
and the role's `hosts`, and an undefined `target` still refuses `host-unknown`. `validArgv` and
|
||||
`validStdin` now run after the access check, so a malformed argv for an ungranted host refuses
|
||||
`host-not-granted` first. The enrollment block is appended to the question before `string(question)`,
|
||||
so the 4096 cap counts it (M1). The decision's raise body carries `grant: enroll`.
|
||||
- `decision.resolve` (new :2409): when `op` is `resolved`, `#approvedGrant` reads the decision, and a
|
||||
`host-enroll` grant starts `#runEnrollment` from `setImmediate`, after the transaction commits. The
|
||||
result adds `grant: 'started'`.
|
||||
|
||||
New, all private but two:
|
||||
- `#grantEvent` (:657), the writer of guarded events. It runs the secret check, then `writeGrantEvent`.
|
||||
- `#approvedGrant` (:667), `#consumeGrant` (:678). `#consumeGrant` sits after `#approvedGrant`, not next
|
||||
to `#consumeAuthority` as planned.
|
||||
- `#hostRecord` (:685), `#hostAccess` (:692), `#grantNotice` (:708), `#enrollFailed` (:736),
|
||||
`#runEnrollment` (:750), `#openGrantDecisions` (:823).
|
||||
- `runEnrollment` (:745), public, a trusted entry to `#runEnrollment` for G10. No verb reaches it.
|
||||
- `reconcileGrants` (:833), public, which `runtime.mjs` calls once at start.
|
||||
- `#raiseGrant` (:866), `#hostsCaller` (:886), `#requestHosts` (:900), `#hostLinesLeft` (:964).
|
||||
|
||||
Module level, new (:184-230): `GRANT_RUN`, `grantName`, `GRANT_EVIDENCE` (the N1 filter as SQL),
|
||||
`registrySha256`, `writeGrantEvent` (exported, the one writer of guarded events, which test fixtures use
|
||||
for F1), `HOSTS_VERBS`, `fingerprintOf`, `INSTALLER_COMMENT`.
|
||||
|
||||
Unchanged: `#event`, `#spent`, `#consumeAuthority`.
|
||||
|
||||
Other `src` files:
|
||||
- `bus/src/host-grant.mjs`, new: the keyscan, the installer key pair, and `hosts.check`'s `list`.
|
||||
- `bus/src/runtime.mjs`: builds `hostGrant`, calls `reconcileGrants` before the server starts, and adds
|
||||
the `host_grants` references before `new Credentials` (X4 third bullet).
|
||||
- `bus/src/host-run.mjs`: `runHost` takes `command`, sent as is in place of the sudo argv, for the
|
||||
installer's `list`.
|
||||
- `bus/src/server.mjs`: the `hosts.*` verbs and a raise with `grant` go down the async path.
|
||||
- `bus/src/store.mjs`: the 3i-3j step, `SCHEMA_VERSION` 3j. `bus/src/credentials.mjs` exports
|
||||
`privateDir`.
|
||||
- `business/src/business.mjs` and `business/src/hosts.mjs`: the loader refuses `hosts` (S1).
|
||||
`checkHosts` and `checkHostGrants` are gone.
|
||||
- `cli/src/hosts.mjs`, new, routed from `cli.mjs` and `scripts/mosaic`.
|
||||
|
||||
## Decisions taken in the build
|
||||
|
||||
- **`#hostAccess` binds the agent and the host.** X4 says the Approve record check passes when the
|
||||
decision is an approved `host` grant with a `grant.host` consume. The check also requires the grant's
|
||||
`agent` to equal the session's role and its `host` to equal the requested host. Without that, an
|
||||
approved grant for one agent and host would vouch for any `host_grants` row that names its decision.
|
||||
Cost: none for row b, whose executor writes the row from the same grant.
|
||||
- **Store fixtures are written the real way.** `bus/tests/host-fixtures.mjs` inserts the decision rows by
|
||||
SQL, but every guarded event goes through `writeGrantEvent`, so the trigger sees it (F1). An enrollment
|
||||
fixture writes `grant.host.enrolled`, or the start reconcile reads it as interrupted.
|
||||
- **The resolver test for host kinds is gone.** `business/tests/hosts.test.mjs` lost its D5 case, because
|
||||
the loader now refuses `host.<id>` credential refs. The `isHostKind` filter in `business/src/resolve.mjs`
|
||||
can't be reached now. I left it in, since row b may want it back, and it's one line.
|
||||
- **Store before Credentials.** The runtime already opened the store before `Credentials`. The
|
||||
`host_grants` references depend on that order, and the comment there says so.
|
||||
- **CLI.** `mosaic hosts` prints tab lines, and `--json` prints the bus rows. Usage errors exit 4 and bus
|
||||
refusals exit 2, as `mosaic credentials` does. There is no enroll command: enrollment is the chief's raise.
|
||||
- **Host tools tests live in the bus suite.** `bus/tests/host-tools.test.mjs` loads the writer by path
|
||||
through a Python harness under `python3 -B -I`. The harness passes its own key dir, config dir, owner
|
||||
uid and user, group and sudo lookups, and runs `mosaic-keys` from a copy with stub `sudo` and `logger`.
|
||||
The gate runs the bus suite, so G16 and G25 run there. The real `pwd`, `grp` and `sudo -l` paths are
|
||||
checked on the #433 VM, not here. The test also runs `visudo -c` on the sudoers file and `sshd -T` on
|
||||
the drop-in.
|
||||
- **Writer details the design left open.** The `from` file must hold exactly one value matching
|
||||
`[0-9A-Za-z.:/*?!_-]+(,[0-9A-Za-z.:/*?!_-]+)*`, so it can't close the quote. `users`, `privileged` and `from` must be
|
||||
regular, owned by the owner uid and not group or world writable, and are opened without following a
|
||||
symlink. Lines starting with `#` are comments. A refusal exits 1 with its code on stderr and in syslog.
|
||||
`add` of a tag that is already there exits 0 and writes nothing.
|
||||
- **`runEnrollment` is public for G10 (duckworth-0081 B1).** G10 needs a second start for a spent
|
||||
decision, and the resolve path can't give one: a second resolve refuses `decision-closed`
|
||||
(broker.mjs:2416) before the executor runs. So
|
||||
`#runEnrollment` returns its code (`enrolled`, `host-enrolled` or the refusal), and a public
|
||||
`runEnrollment` calls it, as `reconcileGrants` is public. The test checks a spent
|
||||
decision gives `decision-consumed`, and an open or agent-resolved one gives `not-authorized`, with
|
||||
nothing written. That second case also reaches the `via` and `by` checks in `#approvedGrant`, so A3 is
|
||||
no longer vacuous (N1).
|
||||
Who can call it (sage-1692): only code holding the `Broker` object, which is the bus process and the
|
||||
tests. The one `new Broker` in `src` is `runtime.mjs:126`, and no `src` file calls `runEnrollment`;
|
||||
the resolve path calls the private `#runEnrollment` from `setImmediate` (broker.mjs:2445). A seat
|
||||
holds no `Broker`. It reaches the bus only over `broker.sock`, where `server.mjs` calls a fixed set of
|
||||
methods by name (`request` at :134 and :140, `identity`, `disconnect`). `request()` routes on the verb
|
||||
through the `#dispatch` switch, and neither file looks a method up by name (no `this[` in either), so no
|
||||
verb string can reach `runEnrollment`.
|
||||
- **A `sudo -l` that times out refuses `user-sudo` (N2).** `_sudo` returns an empty answer on
|
||||
`TimeoutExpired`, which isn't the expected line, so `main` logs the refusal and exits 1.
|
||||
- **Keyscan comment (N3).** A scan that can't start refuses `host-failed`. One that runs and finds no key
|
||||
returns no lines, and the raise refuses `host-key-mismatch`. The comment in `host-grant.mjs` now says so.
|
||||
- **No tag on in-run `grant.failed` (N4).** r6:578 tags only the `interrupted` one, so the code follows it.
|
||||
- **Run sheet (N5).** The writer refuses `config-privileged` when the file is missing, so each host needs an
|
||||
empty, root-owned `/etc/mosaic-keys/privileged` (0644) before the first enrollment.
|
||||
|
||||
## Mutant results (Sage's slot, 2026-10-21)
|
||||
|
||||
All 25 killed: A1 to A23, with A3b added for duckworth-0083 N8 and A22 split into A22a and A22b. The
|
||||
runner is `mutants.py` outside the repo. Each mutant gets its own session, a 420 s limit with a group kill,
|
||||
and a byte restore checked against git status. None hung. Run 1 (04:24Z, on 95f966b1c) killed A1 to A22a
|
||||
and stopped at A22b, which survived. Run 2 (04:43Z, on 097892741) killed A22b and A23.
|
||||
|
||||
- Each kill comes from a named assertion in the mutant's own test files, except A9. Dropping the `hosts`
|
||||
primary key leaves `host_grants`' foreign key with no parent key, so the store refuses to open
|
||||
("foreign key mismatch") and 28 of 29 tests fail at setup. That's a structural kill, not the S1 test.
|
||||
The schema can't exist without the key (sage-1709).
|
||||
- A22b drops the actor filter from `GRANT_EVIDENCE`. The trigger can't stand in for it, because a 3i store
|
||||
can hold a session-written grant row that the 3i-3j step keeps. The grant-migration test "A22b,
|
||||
sage-1703" kills it (see the A22 row above).
|
||||
|
||||
Follow-up, not in row a: G18's control calls `runHost` directly, so nothing runs `hg.check` end to end
|
||||
with a good pin (duckworth-0083 N9). The #433 VM run covers it.
|
||||
|
||||
## Status at 097892741
|
||||
|
||||
Done: every Split item, 1 to 8, duckworth-0081 B1 to B3, duckworth-0083 N8, and all 25 mutants killed.
|
||||
|
||||
Single files in the slot, all exit 0, 178 pass and 0 fail by an awk sum over 23 logs: bus grant-enroll
|
||||
24, host-run 24, grant-migration 6, host-tools 6, account-migration 3, chief-of-staff 18,
|
||||
comment-migration 5, hard-waits 13, name-migration 1, project-runtime 3, spec-migration 1,
|
||||
thread-migration 3, usage-migration 1, project-files 8, spec-socket 6, spec-review 12, land 17, spec 4;
|
||||
harness spec-tools 2, raise-domain 2; business hosts 4; cli hosts 5; seat sandbox 10.
|
||||
|
||||
Left: Sage runs the whole bus, business and cli suites and every `scripts/test-*.sh` at the landing gate,
|
||||
after Duckworth approves.
|
||||
@@ -21,11 +21,11 @@ so Mos doesn't check this draft. Draft 1 names NEW and goes to Mos for the P0 ch
|
||||
- OLD, live: `4429a104f6d65ccf165f6c83b5d1dce3147dfde0`, schema 3i, pointer `pm: true` (record 99a406ed3).
|
||||
P1 re-reads the activation id and the Console flags from `status`. I don't copy them from the last window.
|
||||
- NEW: open. It's the row 234 landing, or a later code landing if Mos rules. Candidates in landing order:
|
||||
210 (r4 due 10-21), 237, 194, 234.
|
||||
194, 234 (210 and 237 have landed).
|
||||
- REPO, `P`, `WIN=agents/sage/work/window-repin-<NEW12>/`, `SNAP`: as in `repin-4429a104f.md`, with the
|
||||
sha checks at P0 and before S1, S2 and any D3.
|
||||
|
||||
## What NEW carries over OLD (so far, refactor c8bc5cf4c)
|
||||
## What NEW carries over OLD (so far, refactor 8fede5fc6)
|
||||
|
||||
`git log --first-parent 4429a104f..HEAD`, path-limited to packages, scripts, roles, contracts, adapters, skills
|
||||
and the package files:
|
||||
@@ -39,10 +39,10 @@ and the package files:
|
||||
| 199 | cb5fdb064 | Credential store per agent. Check: bus start with no store entries for the live agents changes nothing, and the chief's and pm's launches still pass. The P3 rehearsal covers start. |
|
||||
| N3 | 390ea5261 | README only. |
|
||||
| 230 | c8bc5cf4c | Case 2 scripts, tools only. Not run live. |
|
||||
| 210 | open | Host access. The live business has no `hosts`, so nothing is granted. Bus start runs the host-run sweep (M34d). P3 must show it removes nothing. |
|
||||
| 237 | open | Publish busy check (199 N1). |
|
||||
| 210 | f5753c0e8 | Host access. The live business has no `hosts`, so nothing is granted. Bus start runs the host-run sweep (M34d). P3 must show it removes nothing. |
|
||||
| 237 | d7d943199 | Publish busy check (199 N1). |
|
||||
| 234 | open | Machine grant row a and the 3i to 3j schema step. `--plan` must say `migration: 3i -> 3j`. |
|
||||
| PRD 1.7+1.8 | open | `docs/prd/mosaic-stack.md` and `packages/tasks/requirements.json` move to version 1.8 (mos-1154, mos-1156, DECISIONS Q192). The live pm cites 1.6 until this window. Check after: the pm's requirements read answers version 1.8. |
|
||||
| PRD 1.7+1.8 | 0adf8ddfe | `docs/prd/mosaic-stack.md` and `packages/tasks/requirements.json` move to version 1.8 (mos-1154, mos-1156, DECISIONS Q192). The live pm cites 1.6 until this window. Check after: the pm's requirements read answers version 1.8. |
|
||||
|
||||
Service script drift OLD to HEAD is 415+/91- over the four scripts, mostly row 190. P5 holds them at NEW.
|
||||
|
||||
|
||||
@@ -1394,3 +1394,4 @@ are never rewritten or removed; corrections are new entries.
|
||||
2026-10-21T01:34:13Z | Filbert (Opus 5.5, 048f3c63) | row 232 installer I1 review r1 (#1734, manifest 7a3b4ea133cc) | APPROVE, Gitea 29829, queue rev 2600 (left for Sage's queue-commit); static review, gate and mutants are Darkwing's receipts; 3 non-blocking notes
|
||||
2026-10-21T01:49:55Z | Sage (Opus 5.5, 1ef1e4f8) | row 237 landed d7d943199 (#1743, gate green, M1 killed); PRD 1.7+1.8 landed 0adf8ddfe (Honker 2884ee496, Duckworth approve bf52628f9, tasks 97/0 bus 478/0 on the tip, DECISIONS Q192); rows 239 (#1745) and 240 (#1746) added; 233 briefed; queue revs 2540-2603 committed 584e2bd82 | landed, push next
|
||||
2026-10-21T02:51:46Z | Sage (Opus 5.5, 1ef1e4f8) | row 210 landing (r5 0485384e9, Duckworth r3 29851) | landed f5753c0e8, patch-id 723b1c83 = r5 diff, manifest 688d38b9 OK; gate 2796/0 packages, 429/0 scripts; mutants 52/53 (M22b equivalent); #1707 closed (29853), Vikunja 424 comment 429; queue revs 2604-2617 c084d8d42; row 234 briefed; Megavolt has the slot for 241 (sage-1668)
|
||||
2026-10-21T03:01:43Z | Filbert (Opus 5.5, 048f3c63) | row 232 installer I1 review r2, rebase scope (#1734, manifest 7af3c94308e0, da2da2f6c on 8fede5fc6) | APPROVE, Gitea 29865, queue rev 2622 (left for Sage's queue-commit); README = predicted 3-way blob 89a95b32, 7 blobs same as r1, BUILD.md +31/-0; corrected my r1 reader claim (host.test.mjs:855 reads cli README)
|
||||
|
||||
+154
-6
@@ -408,8 +408,9 @@ exposure of a named Console waits for Jason after Q175 SSO.
|
||||
|
||||
```
|
||||
scripts/stack-copy.sh copy COMMIT [--copies DIR] [--allow-worktree]
|
||||
scripts/stack-copy.sh wrapper ID [--instance NAME] [--copies DIR] [--bin DIR]
|
||||
scripts/stack-copy.sh check [--instance NAME] [--copies DIR] [--bin DIR] [--business ID]
|
||||
scripts/stack-copy.sh copy --bundle FILE SHA [--copies DIR]
|
||||
scripts/stack-copy.sh wrapper ID [--instance NAME] [--copies DIR] [--bin DIR] [--create]
|
||||
scripts/stack-copy.sh check [--instance NAME] [--copies DIR] [--bin DIR] [--business ID] [--units user|external]
|
||||
```
|
||||
|
||||
One read-only tree supplies the bus, Console and human CLI (row 85,
|
||||
@@ -423,6 +424,14 @@ renames beside the target. A published existing copy stays unchanged.
|
||||
Dependencies use the exact lock and carry Pi's installed lock record.
|
||||
Failure publishes nothing and removes the temporary tree.
|
||||
|
||||
`copy --bundle FILE SHA` (row 232) is for a host with no refactor checkout.
|
||||
SHA is a full 40-character id. The bundle must pass `git bundle verify`,
|
||||
carry `refs/heads/refactor`, and reach SHA from it; each miss refuses and
|
||||
writes nothing. The copy is then made from SHA as `copy` makes it, and the
|
||||
script prints `release:` with the RELEASE value at SHA and `bundle sha256:`
|
||||
with the bundle's digest. That pair is the release identity on such a host.
|
||||
Without `--bundle`, `copy`'s reachability rule is unchanged.
|
||||
|
||||
`wrapper` is a live switching step, reserved for Jason or Mos. It accepts
|
||||
only an existing copy and a regular non-symlink mosaic wrapper with the
|
||||
Q81 marker on line 2 and its exact lease line to mosaic-old. It preserves
|
||||
@@ -430,11 +439,16 @@ both lines, adds comments naming the copy root and Q81's mosaic-old lease
|
||||
probe, and atomically replaces the wrapper's final exec with the
|
||||
copy's scripts/mosaic. --bin is a fixture override; the default is
|
||||
~/.npm-global/bin. Neither the Q81 worktree nor mosaic-old is removed.
|
||||
`wrapper ID --create` (row 232) writes the first wrapper on a new host, the
|
||||
same file `wrapper` writes. It refuses when anything is at the wrapper path,
|
||||
a symlink included, and never replaces it. Without `--create`, a missing
|
||||
wrapper still refuses.
|
||||
`check` changes nothing: it reads effective systemd WorkingDirectory and
|
||||
ExecStart (including drop-ins) for the bus and Console, plus the wrapper
|
||||
exec. It prints all three paths and succeeds only on the same existing
|
||||
copy; absent/failed reads and a mismatched ExecStart fail. Business defaults
|
||||
to mosaic-stack. Queue work continues on the canonical scripts/mosaic.
|
||||
to mosaic-stack. `check --units external` (row 232) reads root's system units
|
||||
with `systemctl show`, without `--user`. Queue work continues on the canonical scripts/mosaic.
|
||||
|
||||
Bus render/install accept --copy ID, --copies DIR, --pm and the unchanged
|
||||
--claude pin. Without --copy the rendered unit stays the checkout unit.
|
||||
@@ -455,7 +469,10 @@ scripts/stack-release.sh activate ID BUSINESS [--pm | --pm-after-harness] [--mig
|
||||
scripts/stack-release.sh pm-on BUSINESS [--timeout S]
|
||||
scripts/stack-release.sh rollback BUSINESS [--restore ACTIVATION [--events N]] [--timeout S] [--plan]
|
||||
scripts/stack-release.sh status BUSINESS
|
||||
common options: --copies DIR, --bin DIR, --instance NAME
|
||||
scripts/stack-release.sh confirm ID BUSINESS --units external [--rollback OLD --cause A3|A4|A5] [--timeout S]
|
||||
scripts/stack-release.sh revert ID BUSINESS --units external
|
||||
scripts/stack-release.sh gate ID BUSINESS --units external --since EPOCH [--timeout S]
|
||||
common options: --copies DIR, --bin DIR, --instance NAME, --units user|external
|
||||
--events N is the total of the per-table counts the post-backup-events refusal prints: events and
|
||||
every table in COUNTED.
|
||||
```
|
||||
@@ -652,7 +669,7 @@ pinned copy](../packages/cli/README.md#one-pinned-copy-row-85-1579)).
|
||||
#### Rehearsal on a scratch root (D2, row 163)
|
||||
|
||||
```
|
||||
scripts/stack-release.sh rehearse ID BUSINESS --backup ACTIVATION|PATH --port N [--timeout S]
|
||||
scripts/stack-release.sh rehearse ID BUSINESS --backup ACTIVATION|PATH|last --port N [--timeout S]
|
||||
```
|
||||
|
||||
`rehearse` starts the target copy's bus host and Console on a copy of a
|
||||
@@ -663,7 +680,11 @@ runs it against live data. It stops and starts no unit and writes no pointer
|
||||
or log line.
|
||||
|
||||
`--backup` takes an activation id under `<dataRoot>/backups/stack/` or an
|
||||
absolute path. `--port` is the scratch Console's loopback port, 1024 to
|
||||
absolute path. `--backup last` (row 233) takes the newest backup the
|
||||
activation log records and prints `PASS backup-last` naming it. With none in
|
||||
the log it fails `no-backup`, exits 3 and makes no directory. After the
|
||||
backup's checks it prints `BACKUP_STORE <activation id> schema <version>`.
|
||||
`--port` is the scratch Console's loopback port, 1024 to
|
||||
65535. The run refuses `port-reserved` for 7332, 7333 and the pointer's
|
||||
`--port` and `--tester-port`, and `port-in-use` for a port something holds;
|
||||
it never picks another port. `--timeout` (default 60 seconds) bounds the
|
||||
@@ -711,6 +732,133 @@ Usage exits 4. Nothing prunes rehearsal directories.
|
||||
starts no thread ticker, and leaves an earlier host's `sessions.json` and
|
||||
its runs alone. No unit passes it, and `--pm` with it is a usage error.
|
||||
|
||||
#### Root-owned units (`--units external`, row 232)
|
||||
|
||||
On a host whose bus and Console are root's system units, the stack user runs
|
||||
`stack-release.sh` with `--units external` from its own clone of the release
|
||||
bundle (sage-1561). The script then calls `systemctl show`, `is-active` and
|
||||
`list-units` without `--user` and reads no journal. It starts, stops and
|
||||
installs nothing. Root does those steps, and the I3 driver calls the verbs
|
||||
below in order. Without `--units`, every command line and every byte of
|
||||
output is what it was before row 232. `confirm`, `revert`, `gate`, `--to`
|
||||
and `--expect-from` are usage errors without `--units external`.
|
||||
|
||||
- `adopt`, `status`, `activate ID` and `rollback --to ID` take the flag.
|
||||
`rollback` then needs `--to`, and `--to` must be the copy `rollback` alone
|
||||
would pick from the log, or it refuses `to-not-previous`.
|
||||
- `--plan --expect-from OLD` refuses `pointer-differs` when the pointer
|
||||
names a copy other than OLD.
|
||||
- `pm-on` and `activate --pm-after-harness` refuse `external-pm`; the PM
|
||||
flag is in root's bus unit. `--pm`, `--tester-port`, the proof flags,
|
||||
`--no-proof`, `--no-send` and `--sso` are usage errors; root's Console
|
||||
unit carries the Console flags.
|
||||
- `activate` and `rollback` refuse `units-running` unless both units read
|
||||
inactive or failed and the store is closed. They make the rollback set
|
||||
with an exclusive `mkdir`, write `AWAITING-UNITS` (the target copy) first,
|
||||
capture the pointer, the wrapper and both units' show output and unit file
|
||||
sums, back up and migrate or restore the store, and write the wrapper.
|
||||
They write no pointer and start nothing. The last line says the run is
|
||||
staged and names `confirm`. A failure after the capture writes `FAILED`
|
||||
with its code, logs nothing more and exits 1; the activation stays open.
|
||||
- `confirm ID BUSINESS` needs exactly one `AWAITING-UNITS` for copy ID whose
|
||||
activation the log holds open, or it refuses `open-activation`. It gates
|
||||
the start root made and writes the pointer and an `ok` line with the
|
||||
started line's op, from, to and backup.
|
||||
`confirm --rollback OLD --cause A3|A4|A5` gates OLD after root put the old
|
||||
units back and logs `failed` or `gate-failed` with `rollback ok` (exit 1)
|
||||
or `rollback-failed` (exit 2). Either removes `AWAITING-UNITS` and `FAILED`.
|
||||
- `revert ID BUSINESS` puts the saved wrapper back and, when the run had
|
||||
started a migration, restores the backup taken before it. It writes no log
|
||||
line and can run again.
|
||||
- `gate ID BUSINESS --since EPOCH` gates a copy with no open activation,
|
||||
takes no lock and writes nothing: 0 on a pass, 1 otherwise. EPOCH is root's
|
||||
clock before root started the units.
|
||||
|
||||
The gate under `--units external` replaces the journal read. The bus unit's
|
||||
`ActiveEnterTimestamp`, in whole seconds, must be at or after the started
|
||||
line's second (`confirm`) or `--since` (`gate`); an older start fails
|
||||
`stale-start`, so a unit that never stopped can't pass. With the PM on, the
|
||||
gate reads `launches list --json` for exactly one live launch of the PM's
|
||||
instance (the business's `launch.by`), for this business, whose run started at
|
||||
or after the floor. `status` adds an
|
||||
`awaiting units:` line for an open activation.
|
||||
|
||||
Under `--units external`, `activate --plan` also refuses `open-activation`
|
||||
while any rollback set holds `AWAITING-UNITS`, and `migrate-required` when
|
||||
the target migrates the store and the run lacks `--migrate` (row 233). Under
|
||||
`--units user` the plan's bytes are unchanged.
|
||||
|
||||
#### Root's upgrade driver (`scripts/stack-upgrade.sh`, row 233)
|
||||
|
||||
```
|
||||
scripts/stack-upgrade.sh upgrade --instance dev --bundle FILE --sha SHA [--migrate] [--no-rehearsal]
|
||||
[--rehearse-port N] [--timeout S]
|
||||
scripts/stack-upgrade.sh rollback --instance dev --to ID [--restore ACT [--events N]] [--timeout S]
|
||||
```
|
||||
|
||||
Root runs it from a root-owned checkout of the reviewed commit, never from a
|
||||
copy, because the stack's user can write the copies. Each run needs Mos's go,
|
||||
and `--migrate` needs Jason present (decision 96). The host must be prepared
|
||||
first (`docs/guides/host-run-sheet.md`). Only the default instance, `dev`, is
|
||||
accepted; a named instance's units are a later row.
|
||||
|
||||
The unit values come from `/etc/mosaic-stack/dev.args`, which I2's install
|
||||
writes: one `case2-unit.sh` argument per line, the serve flags after `--`,
|
||||
owned by root with no group or other write, exactly one `--copy`, and that
|
||||
copy the one both installed units run. Otherwise the run refuses
|
||||
`args-unreadable` or `args-differ`. The driver writes the new copy id into
|
||||
that file only after `confirm` passes.
|
||||
|
||||
`upgrade`, in order. Every stack user step runs as
|
||||
`sudo -u USER -H env PATH=<node dir>:/usr/local/bin:/usr/bin:/bin` from the
|
||||
user's clone `<home>/src/mosaic-stack`:
|
||||
|
||||
1. Takes `flock -n` on `/run/mosaic-stack/upgrade-dev.lock` and holds it to
|
||||
the end, rollback included. A second run refuses `upgrade-running`, and
|
||||
a lock it can't open refuses `lock-failed`.
|
||||
2. Fetches the bundle's `refactor` into the clone and fast-forwards it to
|
||||
SHA (`fetch-failed`), then `stack-copy.sh copy --bundle FILE SHA`
|
||||
(`copy-failed`).
|
||||
3. A1: this checkout's HEAD is SHA (`head-mismatch`), the copy exists
|
||||
(`copy-missing`), `/usr/local/lib/mosaic-stack/hostcheck.sh --check`
|
||||
exits 0 (`profile-drift`, row 240), both units name one copy
|
||||
(`units-unreadable`, `units-disagree`) and it is the args file's
|
||||
(`args-differ`), and
|
||||
`activate SHA BUSINESS --units external --plan --expect-from OLD` exits 0
|
||||
(`plan-refused`, which covers `open-activation`, `migrate-required` and
|
||||
`pointer-differs`).
|
||||
4. The rehearsal, `rehearse SHA BUSINESS --backup last --port 7339`
|
||||
(`--rehearse-port`). Exit 0 goes on. Exit 3 means no backup in the log
|
||||
and goes on only with `--no-rehearsal`, logging `rehearsal-skipped`;
|
||||
without the flag it refuses `no-backup`. Exit 0 with `--no-rehearsal`
|
||||
refuses `rehearsal-required`; any other exit refuses `rehearsal-failed`.
|
||||
5. A2 stops the Console, then the bus, and waits for both to read inactive
|
||||
or failed. A3 is `activate --units external`, A4 the render with the new
|
||||
`--copy` (which also starts the bus, then the Console), A5 `confirm`.
|
||||
The driver writes the new args file beside the old one before A2
|
||||
(`args-failed`, exit 3) and moves it into place after A5. A failed move
|
||||
exits 1 `args-failed`: the new copy is live and confirmed, and root puts
|
||||
the file in place by hand.
|
||||
|
||||
A failure in A2 or a stage that exits 3 starts the old units again and gates
|
||||
them (A2f). A later failure stops both units, runs `revert`, renders the old
|
||||
copy and runs `confirm --rollback OLD --cause A3|A4|A5`. `rollback` is the
|
||||
same from A1 on, with `rollback BUSINESS --to ID`, no fetch, no copy and no
|
||||
rehearsal; `--restore` and `--events` pass through.
|
||||
|
||||
Root reads no output the stack user wrote except exit codes and the
|
||||
rehearsal's `BACKUP_STORE` line, which it copies into its log after a match
|
||||
against a fixed pattern (`BACKUP_STORE none` otherwise). The step log
|
||||
`/var/log/mosaic-stack/upgrade-dev.log`, 0640, gets one
|
||||
`<time> <step> rc=N argv: ...` line per step and a last
|
||||
`<time> result exit=N <code> <text>` line.
|
||||
|
||||
Exit 0 ok, 1 failed with the old stack running and gated, 2 rollback-failed
|
||||
with both units stopped, 3 refused with nothing stopped, 4 usage. A refusal
|
||||
after step 2 leaves the user's clone fast-forwarded and the new copy made. After exit
|
||||
2 the activation stays open, so the next run's plan refuses until root
|
||||
finishes it.
|
||||
|
||||
## Console service (`scripts/webui-service.sh`)
|
||||
|
||||
```bash
|
||||
|
||||
@@ -0,0 +1,240 @@
|
||||
# Host run sheet: preparing a host for the stack
|
||||
|
||||
ops-01 runs this on a fresh host before install (I2), and once per host for enrollment. Jason places the
|
||||
credentials in section 9. Brief: `docs/plans/2026-10-20_installer-rows.md`, I3 (row 233, #1735). The
|
||||
host design is Darkwing's r6 (`agents/darkwing/work/queue-195/DESIGN.md`, section 2) as the brief
|
||||
changes it.
|
||||
|
||||
The hosts are .53 (profile `factory`) and .54 (profile `library`). Both get every section unless a
|
||||
section says otherwise. Every command runs as root unless it starts with `sudo -u mosaic`.
|
||||
|
||||
Each item names the `scripts/case2-precheck.sh` line that verifies it. Install's precheck runs as the
|
||||
stack's user with `--business ID --profile factory|library`. Where no precheck line checks an item, the
|
||||
item says "precheck line missing", and Gosalyn adds the line to I2. Today's precheck (C2-13a) has these
|
||||
gaps; C2-13b adds the library lines (`library.sync`, persona modes, no Discord binding).
|
||||
|
||||
## 1. The stack's user and its home
|
||||
|
||||
```sh
|
||||
useradd --system --user-group --home-dir /srv/mosaic --create-home --shell /usr/sbin/nologin mosaic
|
||||
chmod 0700 /srv/mosaic
|
||||
install -d -o root -g root -m 0755 /etc/mosaic-stack
|
||||
install -d -o root -g root -m 0750 /var/log/mosaic-stack
|
||||
```
|
||||
|
||||
`mosaic` gets no sudo rule and no supplementary group. Check with `sudo -l -U mosaic` (expect "not allowed
|
||||
to run sudo") and `id -G mosaic` (one number).
|
||||
|
||||
Precheck lines:
|
||||
- `ok HOME is /srv/mosaic`
|
||||
- `ok sudo -n true fails as mosaic`
|
||||
- `ok id -G prints one group (<gid>)`
|
||||
- `/srv/mosaic` at 0700: precheck line missing.
|
||||
|
||||
## 2. node, pinned under /opt
|
||||
|
||||
The pin is the version and sha256 the window names. `<v>` below is that version, for example `v24.x.y`.
|
||||
|
||||
```sh
|
||||
install -d -o root -g root -m 0755 /etc/mosaic-stack/node-keys /opt/mosaic-stack
|
||||
cd "$(mktemp -d)"
|
||||
curl -fsSLO https://nodejs.org/dist/<v>/node-<v>-linux-x64.tar.xz
|
||||
curl -fsSLO https://nodejs.org/dist/<v>/SHASUMS256.txt
|
||||
curl -fsSLO https://nodejs.org/dist/<v>/SHASUMS256.txt.asc
|
||||
gpg --no-default-keyring --keyring /etc/mosaic-stack/node-keys/pubring.kbx --verify SHASUMS256.txt.asc SHASUMS256.txt
|
||||
grep " node-<v>-linux-x64.tar.xz\$" SHASUMS256.txt | sha256sum -c -
|
||||
echo "<pinned sha256> node-<v>-linux-x64.tar.xz" | sha256sum -c -
|
||||
tar -xJf node-<v>-linux-x64.tar.xz --no-same-owner
|
||||
mv node-<v>-linux-x64 /opt/mosaic-stack/node-<v>
|
||||
chown -R root:root /opt/mosaic-stack/node-<v>
|
||||
chmod -R go-w /opt/mosaic-stack/node-<v>
|
||||
```
|
||||
|
||||
The keyring holds the Node release keys from the nodejs/release-keys repository, nothing else. Stop on any
|
||||
failed check; don't unpack. Rollback: `rm -r /opt/mosaic-stack/node-<v>`.
|
||||
|
||||
Precheck lines:
|
||||
- `ok node <v>` (version 24 or later only).
|
||||
- The `--node` path in the args file must be a root-owned regular file outside the home. `case2-unit.sh`
|
||||
refuses otherwise at render, which is not a precheck line.
|
||||
- The installed version equals the pin: precheck line missing.
|
||||
|
||||
## 3. claude, factory only (.53)
|
||||
|
||||
Root installs it under `/opt`, never under the stack's home (this replaces r6's `/srv/mosaic/.local/bin`).
|
||||
The pin is 2.1.304 unless the window names a newer one.
|
||||
|
||||
```sh
|
||||
install -d -o root -g root -m 0755 /opt/mosaic-stack/claude-<pin> /etc/claude-code
|
||||
npm install --prefix /opt/mosaic-stack/claude-<pin> @anthropic-ai/claude-code@<pin>
|
||||
chown -R root:root /opt/mosaic-stack/claude-<pin>
|
||||
printf '%s\n' '{"env":{"DISABLE_AUTOUPDATER":"1"}}' > /etc/claude-code/managed-settings.json
|
||||
chown root:root /etc/claude-code/managed-settings.json
|
||||
chmod 0644 /etc/claude-code/managed-settings.json
|
||||
```
|
||||
|
||||
Rollback: `rm -r /opt/mosaic-stack/claude-<pin> /etc/claude-code/managed-settings.json`.
|
||||
|
||||
The args file's `--claude` names the binary under `/opt/mosaic-stack/claude-<pin>`. `case2-unit.sh`
|
||||
refuses one that isn't a root-owned regular file outside the home.
|
||||
|
||||
Precheck lines:
|
||||
- `ok claude replaces its prompt with --system-prompt-file (<n> input tokens)`. It runs the claude the bus
|
||||
unit's `MOSAIC_CLAUDE_BIN` names, so it needs the model credential from section 9.
|
||||
- Managed settings present with the autoupdater off: precheck line missing.
|
||||
|
||||
## 4. bubblewrap, AppArmor and the drift stop
|
||||
|
||||
Both hosts. These are r6's items, unchanged.
|
||||
|
||||
```sh
|
||||
apt-get install bubblewrap=0.9.0-1ubuntu0.3
|
||||
stat -c '%U:%G %a' /usr/bin/bwrap > /etc/mosaic-stack/bwrap.orig # expect root:root 755
|
||||
|
||||
cd "$(mktemp -d)"
|
||||
apt-get download apparmor-profiles=4.0.1really4.0.1-0ubuntu0.24.04.9
|
||||
dpkg-deb --fsys-tarfile apparmor-profiles_*.deb | tar -x ./usr/share/apparmor/extra-profiles/bwrap-userns-restrict
|
||||
echo "11d39094f044f0cda0febb3ad517b830301da6b2ce929664af09ee9e4dd264f9 usr/share/apparmor/extra-profiles/bwrap-userns-restrict" | sha256sum -c -
|
||||
install -o root -g root -m 0644 usr/share/apparmor/extra-profiles/bwrap-userns-restrict /etc/apparmor.d/bwrap-userns-restrict
|
||||
apparmor_parser -r /etc/apparmor.d/bwrap-userns-restrict
|
||||
|
||||
dpkg-statoverride --update --add root mosaic 0750 /usr/bin/bwrap
|
||||
```
|
||||
|
||||
`aa-status` then lists `bwrap` and `unpriv_bwrap` in enforce mode. Install the one profile file, never the
|
||||
whole `apparmor-profiles` package, whose 20 other profiles would load at the next boot. Rollback, in
|
||||
reverse order: `dpkg-statoverride --remove /usr/bin/bwrap`, then `chown` and `chmod` to the values in
|
||||
`bwrap.orig`; `apparmor_parser -R /etc/apparmor.d/bwrap-userns-restrict`, then `rm` it;
|
||||
`apt-get remove bubblewrap`.
|
||||
|
||||
The no-new-privs test must print `bwrap//&unpriv_bwrap (enforce)`:
|
||||
|
||||
```sh
|
||||
systemd-run --wait --pipe --uid=mosaic --gid=mosaic -p NoNewPrivileges=yes -p ProtectSystem=strict \
|
||||
-p ProtectHome=yes -p PrivateTmp=yes \
|
||||
/usr/bin/bwrap --unshare-user --unshare-pid --ro-bind / / cat /proc/self/attr/current
|
||||
```
|
||||
|
||||
The P9 checks: another user's `/usr/bin/bwrap --version` fails with Permission denied, `getent group mosaic`
|
||||
lists no member, and `sysctl fs.protected_hardlinks` prints 1. `sysctl
|
||||
kernel.apparmor_restrict_unprivileged_userns` prints 1 too (Ubuntu 24.04's default); leave it, so nothing
|
||||
else on the host gains user namespaces.
|
||||
|
||||
The drift stop is row 240's (I4, #1746): `hostcheck.sh` and its pins in `/usr/local/lib/mosaic-stack/`, the
|
||||
apt hook `/etc/apt/apt.conf.d/80mosaic-hostcheck`
|
||||
(`DPkg::Post-Invoke { "/usr/local/lib/mosaic-stack/hostcheck.sh --enforce || true"; };`) and the boot unit
|
||||
`mosaic-hostcheck.service`. Install them from row 240's run sheet lines once it lands. Until then
|
||||
`stack-upgrade.sh` refuses `profile-drift` on every run, because it runs `hostcheck.sh --check` and a
|
||||
missing file refuses.
|
||||
|
||||
Precheck lines:
|
||||
- `ok bwrap --unshare-all runs`
|
||||
- The AppArmor profile, the statoverride, the no-new-privs result and the P9 checks: precheck line missing.
|
||||
A line that runs `/usr/local/lib/mosaic-stack/hostcheck.sh --check` would cover them once row 240 lands.
|
||||
|
||||
## 5. The port floor
|
||||
|
||||
Leave `net.ipv4.ip_unprivileged_port_start` at its default (1024) or higher. The Console's port is below
|
||||
it and the unit gets `CAP_NET_BIND_SERVICE`, so no other `mosaic` process can bind that port.
|
||||
|
||||
Precheck line: `ok net.ipv4.ip_unprivileged_port_start <n> is above the Console's port <port>`.
|
||||
|
||||
## 6. The release transport
|
||||
|
||||
A release is a git bundle of `refs/heads/refactor` plus the 40-character commit id (I1, mos-1118 choice 2).
|
||||
Mos's go names both, with the bundle's sha256. `<sha>` below is that id.
|
||||
|
||||
First install:
|
||||
|
||||
```sh
|
||||
# Copy the bundle in, then check it before anything reads it.
|
||||
echo "<bundle sha256> /root/<sha>.bundle" | sha256sum -c -
|
||||
git bundle verify /root/<sha>.bundle
|
||||
|
||||
# Root's checkout. stack-upgrade.sh and case2-unit.sh run from here, never from a copy.
|
||||
git clone -q -b refactor /root/<sha>.bundle /root/mosaic-stack
|
||||
git -C /root/mosaic-stack checkout -q --detach <sha>
|
||||
git -C /root/mosaic-stack rev-parse HEAD # must print <sha>
|
||||
|
||||
# The stack's user's copy of the bundle, and its own clone.
|
||||
install -d -o mosaic -g mosaic -m 0700 /srv/mosaic/incoming /srv/mosaic/src
|
||||
install -o mosaic -g mosaic -m 0600 /root/<sha>.bundle /srv/mosaic/incoming/<sha>.bundle
|
||||
sudo -u mosaic -H git clone -q -b refactor /srv/mosaic/incoming/<sha>.bundle /srv/mosaic/src/mosaic-stack
|
||||
```
|
||||
|
||||
Each upgrade, before `stack-upgrade.sh upgrade`:
|
||||
|
||||
```sh
|
||||
echo "<bundle sha256> /root/<sha>.bundle" | sha256sum -c -
|
||||
git bundle verify /root/<sha>.bundle
|
||||
git -C /root/mosaic-stack fetch -q /root/<sha>.bundle refs/heads/refactor:refs/remotes/origin/refactor
|
||||
git -C /root/mosaic-stack checkout -q --detach <sha>
|
||||
install -o mosaic -g mosaic -m 0600 /root/<sha>.bundle /srv/mosaic/incoming/<sha>.bundle
|
||||
/root/mosaic-stack/scripts/stack-upgrade.sh upgrade --instance dev --bundle /srv/mosaic/incoming/<sha>.bundle --sha <sha>
|
||||
```
|
||||
|
||||
The driver fetches the bundle into the stack's user's clone and makes the copy itself. It refuses
|
||||
`head-mismatch` when root's checkout isn't at `<sha>`. Usage, exits and the step log are in
|
||||
`docs/TOOLS.md`, under "Root's upgrade driver". Each run needs Mos's go, and `--migrate` needs Jason present
|
||||
(decision 96).
|
||||
|
||||
When a run doesn't exit 0, send Mos the exit and the last lines of `/var/log/mosaic-stack/upgrade-dev.log`:
|
||||
- Exit 3: refused before anything stopped. The last line names the code. The stack's user's clone and the
|
||||
new copy may already hold the new commit; the units still run the old one.
|
||||
- Exit 1: the old copy runs and passed its gate. Either the driver restarted it after A2 or a refused A3
|
||||
(A2f), or it rolled back after A3, A4 or A5.
|
||||
- Exit 2: both units are stopped and the activation stays open, so the next run refuses at its plan until
|
||||
it is finished with `confirm` or `revert`. One way here: a unit that doesn't stop within `--timeout`
|
||||
keeps its old start time, A2f's gate fails it `stale-start`, and the driver stops both (duckworth-0055
|
||||
N1). That is on purpose; a unit that won't stop is a reason to stop.
|
||||
|
||||
Precheck lines: none. Root's checkout at `<sha>` and the stack's user's clone: precheck line missing.
|
||||
After install, `stack-copy.sh check --units external` reads the copy the units run.
|
||||
|
||||
## 7. The args file
|
||||
|
||||
`/etc/mosaic-stack/dev.args` holds the `case2-unit.sh` arguments, one per line, with the serve flags after a
|
||||
line holding `--`. I2's install writes it; ops-01 doesn't write it by hand. It must be root's, with no group
|
||||
or other write (`chmod 0644` or tighter). Before `--` it takes only `--user`, `--home`, `--business`,
|
||||
`--copy` (exactly once), `--node`, `--profile`, `--claude` and `--pm`. `stack-upgrade.sh` changes only the
|
||||
`--copy` line, and only after confirm passes.
|
||||
|
||||
Precheck line: none. `stack-upgrade.sh` refuses `args-unreadable` on any of these rules.
|
||||
|
||||
## 8. Enrollment, once per host
|
||||
|
||||
The machine-grant enrollment is Gizmoduck's design r6, commit f9039682e, section 3, steps E1-E4. ops-01's
|
||||
Q-O1 is answered (mos-1125). The host side is the `tools/host/` files: `mosaic-keys`, `mosaic-keys-write`,
|
||||
their sudoers line and the sshd drop-in `/etc/ssh/sshd_config.d/<nn>-mosaic-keys.conf`. The verbs that run
|
||||
it (`hosts.enroll`, the `host-enroll` decision) are rows 234 and 235, not yet on refactor. This item has no
|
||||
commands until they land.
|
||||
|
||||
## 9. Credential slots
|
||||
|
||||
Jason places these, all in one sitting per host. No value appears here or in any message. Check a placed
|
||||
file with `stat -c '%U %a %n' PATH`, never `cat`. `<business>` is the host's business id.
|
||||
|
||||
### .53, factory
|
||||
|
||||
| Path | File | Mode | Owner | Profile | Precheck line |
|
||||
|---|---|---|---|---|---|
|
||||
| `/srv/mosaic/.config/mosaic-dev/secrets/` | `claude-session.env`, one line `CLAUDE_CODE_OAUTH_TOKEN=...` (Q185) | 0600 | mosaic | factory | `claude replaces its prompt with --system-prompt-file` runs claude with it. The file's mode and owner: precheck line missing. |
|
||||
| `/srv/mosaic/.config/mosaic-dev/secrets/agents/<business>/<agent>/` | one file per service | 0600, dirs 0700 | mosaic | both | none. Filled only by `mosaic credentials issue`, or Approve after #436. Never by hand. |
|
||||
|
||||
### .54, library
|
||||
|
||||
| Path | File | Mode | Owner | Profile | Precheck line |
|
||||
|---|---|---|---|---|---|
|
||||
| `/srv/mosaic/.config/mosaic-dev/secrets/` | `claude-session.env`, one line `ZAI_API_KEY=...` | 0600 | mosaic | library | `<path> sets ZAI_API_KEY` |
|
||||
| `/srv/mosaic/.config/mosaic-dev/secrets/` | the Console's proof file, the path the args file's `--proof-file` names | 0600 or 0400 | mosaic | library | none. `case2-unit.sh` refuses at render. Precheck line missing. |
|
||||
| `/srv/mosaic/.config/mosaic-dev/secrets/<business>/` | `library-gitea` | 0600 | mosaic | library | C2-13b's `library.sync` (not landed) |
|
||||
| `/srv/mosaic/.config/mosaic-dev/secrets/agents/<business>/<agent>/` | one file per service | 0600, dirs 0700 | mosaic | both | none. Filled only by `mosaic credentials issue`, or Approve after #436. Never by hand. |
|
||||
|
||||
The installer key, `<configDir>/secrets/hosts/<business>/<hostId>/installer`, is not a slot anyone places.
|
||||
Enrollment (section 8) writes it under the enrolling stack's config directory and deletes it when the host
|
||||
leaves.
|
||||
|
||||
## 10. Console start
|
||||
|
||||
None by hand. I2's `case2-unit.sh` enables both units, starts the bus, then the Console (Q189, mos-1122).
|
||||
`stack-upgrade.sh` stops and starts both units itself.
|
||||
+5
-5
@@ -33,7 +33,7 @@ Gaps found while working go to `docs/plans/DEFERRED.md`, not here.
|
||||
|
||||
<!-- mosaic-queue:begin -->
|
||||
|
||||
Generated from `docs/plans/queue.json` revision 2617 by `queue render`. Do not edit between the markers; change the queue with `scripts/mosaic queue`.
|
||||
Generated from `docs/plans/queue.json` revision 2627 by `queue render`. Do not edit between the markers; change the queue with `scripts/mosaic queue`.
|
||||
|
||||
| # | Piece | Owner | Issues | State | After | Gate | Brief | Note |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
@@ -268,17 +268,17 @@ Generated from `docs/plans/queue.json` revision 2617 by `queue render`. Do not e
|
||||
| 229 | Row B: per-instance pi agent dir for Chet's Codex login, design only, starts after 10-27 (#428) | gosalyn; reviewers webby | #1730 | queued | — | A live check, with Jason's or Mos's go, that two clients on one ChatGPT login don't revoke each other's refresh tokens. No build row starts before it passes. Held until after 10-27. (sage) | `docs/plans/2026-10-20_case2-rows.md` § Brief: row B, a per-instance pi agent dir for Chet's Codex login (#428) | — |
|
||||
| 230 | Case 2 C2-13a unit render and host precheck (case2-unit.sh, case2-precheck.sh) | gosalyn; reviewers webby | #1732 | done | — | Webby approves the manifest. The test file and the cli suite are green on Sage's gate, along with every scripts/test-*.sh. A run on a host is a separate window with Jason's go. (sage) | `docs/plans/2026-10-20_case2-rows.md` § C2-13a unit render and host precheck | Landed c8bc5cf4c on refactor (base 390ea5261), pushed. Patch-id f4b49eb68997 matches git diff 722136772..96768766c on the 5 manifest paths (manifest fbb0d7e42c67, Webby approve r2 rev 2563). packages/cli/README.md 3-way with row 199, hunks apart. Gosalyn gate on 96768766c: cli 541/0, scripts 429/0, mutants 10/10. Merged tree check: case2-unit 30/0, host 42/0. |
|
||||
| 231 | Case 2 C2-01b survey escapes, list item lines and attachment embeds (row 212 N1-N3) | gosalyn; reviewers webby | #1733 | briefed | 212 done | Webby approves the manifest. The library tests are green on Sage's gate, along with every scripts/test-*.sh. (sage) | `docs/plans/2026-10-20_case2-rows.md` § C2-01b survey escapes, list item lines and attachment embeds | — |
|
||||
| 232 | Installer I1: stack-release external units and stack-copy bundle copies (MVP, Q187, Vikunja #432) | darkwing; reviewers filbert | #1734 | in-review, round 1, request posted | — | Filbert approves the manifest. The cli suite and every scripts/test-*.sh are green on Sage's gate, with the mutants killed. (sage) | `docs/plans/2026-10-20_installer-rows.md` § I1 stack-release external units and bundle copies | — |
|
||||
| 232 | Installer I1: stack-release external units and stack-copy bundle copies (MVP, Q187, Vikunja #432) | darkwing; reviewers filbert | #1734 | in-review, round 2, request posted | — | Filbert approves the manifest. The cli suite and every scripts/test-*.sh are green on Sage's gate, with the mutants killed. (sage) | `docs/plans/2026-10-20_installer-rows.md` § I1 stack-release external units and bundle copies | — |
|
||||
| 233 | Installer I3: stack-upgrade.sh root driver and the ops-01 host run sheet (MVP, Q187, Vikunja #432) | darkwing; reviewers filbert | #1735 | briefed | 232 done | Filbert approves the manifest. The test file, the cli suite and every scripts/test-*.sh are green on Sage's gate, with the mutants killed. (sage) | `docs/plans/2026-10-20_installer-rows.md` § I3 the upgrade command and the host run sheet | — |
|
||||
| 234 | Machine grant row a: host enrollment, store records and migration 3i-3j (#436a, design r6 f9039682e, lands by 10-25 before the .53/.54 installs) | gizmoduck; reviewers duckworth | #1740 | briefed | 210 done | Duckworth approves the manifest. The bus, business and cli suites and every scripts/test-*.sh are green on Sage's gate. The design's section 10 row a tests and mutants pass, the migration test included. (sage) (sage) | `docs/plans/2026-10-20_host-credentials.md` § Machine grant row a: enrollment, store records and migration 3i-3j | — |
|
||||
| 234 | Machine grant row a: host enrollment, store records and migration 3i-3j (#436a, design r6 f9039682e, lands by 10-25 before the .53/.54 installs) | gizmoduck; reviewers duckworth | #1740 | in-progress | 210 done | Duckworth approves the manifest. The bus, business and cli suites and every scripts/test-*.sh are green on Sage's gate. The design's section 10 row a tests and mutants pass, the migration test included. (sage) (sage) | `docs/plans/2026-10-20_host-credentials.md` § Machine grant row a: enrollment, store records and migration 3i-3j | — |
|
||||
| 235 | Machine grant row b: raise binding, executor, reconcile and revoke (#436, design r6 f9039682e, C1, N1, G27, G28) | gizmoduck; reviewers duckworth | #1741 | queued | 234 done | Duckworth approves the manifest. The bus, business and cli suites and every scripts/test-*.sh are green on Sage's gate. Section 10's row b tests and mutants pass, G27, G28 and the upsert mutant included. (sage) (sage) | `docs/plans/2026-10-20_host-credentials.md` § Machine grant row b: raise binding, executor, reconcile and revoke | — |
|
||||
| 236 | #436b Gitea grant path build: seat accounts and tokens through an approved grant, schema 3j-3k (design r5 1be535aca, due 10-27 at risk; live waits for Jason's section 8 approval and bootstrap step) | megavolt; reviewers dewey | #1742 | queued | 210 done, 234 done | Dewey approves the manifest. The bus, business and cli suites and every scripts/test-*.sh are green on Sage's gate. Section 9's mutants are killed, N9's restart test included. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: #436b Gitea grant path, build (schema 3j-3k) | — |
|
||||
| 237 | Row 199 follow-up N1: one decision publishes once (publish-busy or #spent recheck) | megavolt; reviewers duckworth | #1743 | done | 199 done | Duckworth approves the manifest. The bus suite and every scripts/test-*.sh are green on Sage's gate, and the mutant is killed. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: row 199 follow-up N1, one decision publishes once | Landed d7d943199 on refactor (local, push pending), patch-id c5f4beb4 = e42415935, manifest bb777cb0 blobs checked. Sage gate 01:31-01:44Z on 816d48b03+4 paths: M1 killed (bus 1/13, cli 1/8), every package suite and scripts/test-*.sh green (bus 478, cli 564). |
|
||||
| 238 | row 200 follow-up N1: the sso-config hint carries --instance NAME (#1744) | launchpad; reviewers webby | #1744 | in-review, round 1, request posted | 200 done | Webby approves the manifest; stack-release.test.mjs whole, cli suite and every scripts/test-*.sh green on Sage's gate; the mutant killed (sage) | `docs/plans/2026-10-20_stack-release-sso.md` § Brief: row 200 follow-up N1, the sso-config hint names the instance | — |
|
||||
| 239 | Row 194 follow-up: the credential revoke runs under the writer's lock, refusals after a revoke say so, and a stale-revision fire test (duckworth-0074, mos-1158) | honker; reviewers duckworth | #1745 | queued | 194 done | Duckworth approves the manifest. The bus, business, cli and webui suites and every scripts/test-*.sh are green on Sage's gate, and the mutants are killed. (sage) (sage) | `docs/plans/2026-10-20_agent-lifecycle.md` § Brief: row 194 follow-up, the revoke under the writer's lock and a fire test | — |
|
||||
| 240 | Installer I4: hostcheck, the bwrap profile check, pins and boot unit, from r6 B2 (MVP, Q187, Vikunja #432; split from I3) | darkwing; reviewers filbert | #1746 | queued | 233 done | Filbert approves the manifest. The test file, the cli suite and every scripts/test-*.sh are green on Sage's gate, with the mutants killed. (sage) (sage) | `docs/plans/2026-10-20_installer-rows.md` § I4 hostcheck: the bwrap profile check, pins and boot unit | — |
|
||||
| 241 | Row 237 follow-up: one decision lands once, a two-ticket broker land test and three mutants on the landing guards (megavolt-0069, sage-1651, P19 441) | megavolt; reviewers duckworth | #1747 | in-progress | 237 done | Duckworth approves the manifest. The bus, seat and cli suites and every scripts/test-*.sh are green on Sage's gate, and the three mutants are killed. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: row 237 follow-up, one decision lands once, with tests at the broker | — |
|
||||
| 242 | Row 237 follow-up N1: a retry after publish-timeout pushes once; hold the decision or kill and wait for the push, a slow-push retry test and a mutant (duckworth-0069, P19 440) | megavolt; reviewers duckworth | #1748 | briefed | 237 done | Duckworth approves the manifest. The bus and cli suites and every scripts/test-*.sh are green on Sage's gate, and the mutant is killed. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: row 237 follow-up N1, a retry after publish-timeout pushes once | — |
|
||||
| 241 | Row 237 follow-up: one decision lands once, a two-ticket broker land test and three mutants on the landing guards (megavolt-0069, sage-1651, P19 441) | megavolt; reviewers duckworth | #1747 | in-review, round 1, request posted | 237 done | Duckworth approves the manifest. The bus, seat and cli suites and every scripts/test-*.sh are green on Sage's gate, and the three mutants are killed. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: row 237 follow-up, one decision lands once, with tests at the broker | — |
|
||||
| 242 | Row 237 follow-up N1: a retry after publish-timeout pushes once; hold the decision or kill and wait for the push, a slow-push retry test and a mutant (duckworth-0069, P19 440) | megavolt; reviewers duckworth | #1748 | in-progress | 237 done | Duckworth approves the manifest. The bus and cli suites and every scripts/test-*.sh are green on Sage's gate, and the mutant is killed. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: row 237 follow-up N1, a retry after publish-timeout pushes once | — |
|
||||
|
||||
<!-- mosaic-queue:end -->
|
||||
|
||||
|
||||
+359
-14
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"version": 1,
|
||||
"canonicalRoot": "/mnt/storage/src/mosaic-stack",
|
||||
"revision": 2617,
|
||||
"revision": 2627,
|
||||
"rows": [
|
||||
{
|
||||
"id": 1,
|
||||
@@ -22687,6 +22687,46 @@
|
||||
"candidate": "7a3b4ea133cc4ef7573cd05338a937523991c8aa838782bf525781f6b76bd9c4"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"n": 2,
|
||||
"op": "darkwing-232-inreview-r2",
|
||||
"by": "darkwing",
|
||||
"at": "2026-10-21T02:58:36.861Z",
|
||||
"issue": 1734,
|
||||
"candidate": {
|
||||
"kind": "manifest",
|
||||
"digest": "7af3c94308e0ce99d88fb40624ad26592305624b139a95a0582991bdd2666b50",
|
||||
"text": "cec97e006778ea2df1e2cbdd25f1641ed3e1965cb1ea91ce782aa22007263357 agents/darkwing/work/queue-232/BUILD.md\nea516da8a6ccd5fa13e371bce83066f7fc7a6bffe545f155b7d7b076682dd40d docs/TOOLS.md\nd21c25c62fcbb9cad185c25ca2302bbb6b3e6ec53db5f4f85d7c445fb454c798 packages/cli/README.md\n681ed534f6ee6188ba5b628e882b17b5a4d9e748d6a649be622a0777a2ee0e87 packages/cli/tests/stack-copy.test.mjs\ned72964534f2289996672b08bc3b75539ec891a62688cedf9ea18a436b7df103 packages/cli/tests/stack-release-external.test.mjs\n896399deb3d0dd07651f965a53ede6717629dcb574876d418184a02ba093ada0 packages/cli/tests/stack-release-fixture.mjs\n533a91460f249bf0f9458fe4269795b8438ab209be7c4e6290e1ea0e29dd98d8 packages/cli/tests/stack-release.test.mjs\nc111fb361bac4fda2d2fdb4c1304e818c79995653ec36f346e44a863846cedc3 scripts/stack-copy.sh\n61db23e9c1b49878f6edfa0c07805d770c20103f06947f0bfcf72da6568f94c7 scripts/stack-release.sh\n"
|
||||
},
|
||||
"request": "comment",
|
||||
"attempts": [
|
||||
{
|
||||
"op": "darkwing-232-inreview-r2",
|
||||
"by": "darkwing",
|
||||
"at": "2026-10-21T02:58:36.861Z",
|
||||
"state": "posted",
|
||||
"comment": 29860,
|
||||
"transport": {
|
||||
"outcome": "posted",
|
||||
"status": 201,
|
||||
"comment": 29860,
|
||||
"detail": "created"
|
||||
},
|
||||
"resolutions": []
|
||||
}
|
||||
],
|
||||
"duplicateRisk": false,
|
||||
"receipts": [
|
||||
{
|
||||
"reviewer": "filbert",
|
||||
"op": "filbert-232-record-r2",
|
||||
"at": "2026-10-21T03:01:22.609Z",
|
||||
"verdict": "approve",
|
||||
"comment": 29865,
|
||||
"candidate": "7af3c94308e0ce99d88fb40624ad26592305624b139a95a0582991bdd2666b50"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -22697,7 +22737,7 @@
|
||||
"note": null,
|
||||
"blockedReason": null,
|
||||
"createdAt": "2026-10-20T22:56:29.929Z",
|
||||
"updatedAt": "2026-10-21T01:33:53.697Z",
|
||||
"updatedAt": "2026-10-21T03:01:22.609Z",
|
||||
"updatedBy": "filbert"
|
||||
},
|
||||
{
|
||||
@@ -22748,7 +22788,7 @@
|
||||
"closes": [
|
||||
1740
|
||||
],
|
||||
"state": "briefed",
|
||||
"state": "in-progress",
|
||||
"previousState": null,
|
||||
"required": false,
|
||||
"requiredSince": null,
|
||||
@@ -22769,12 +22809,15 @@
|
||||
"duckworth"
|
||||
],
|
||||
"review": null,
|
||||
"claim": null,
|
||||
"claim": {
|
||||
"seat": "gizmoduck",
|
||||
"op": "gizmoduck-234-claim-1"
|
||||
},
|
||||
"note": null,
|
||||
"blockedReason": null,
|
||||
"createdAt": "2026-10-20T23:24:50.778Z",
|
||||
"updatedAt": "2026-10-21T02:49:40.096Z",
|
||||
"updatedBy": "sage"
|
||||
"updatedAt": "2026-10-21T02:53:18.435Z",
|
||||
"updatedBy": "gizmoduck"
|
||||
},
|
||||
{
|
||||
"id": 235,
|
||||
@@ -23135,7 +23178,7 @@
|
||||
"closes": [
|
||||
1747
|
||||
],
|
||||
"state": "in-progress",
|
||||
"state": "in-review",
|
||||
"previousState": null,
|
||||
"required": false,
|
||||
"requiredSince": null,
|
||||
@@ -23155,7 +23198,59 @@
|
||||
"reviewers": [
|
||||
"duckworth"
|
||||
],
|
||||
"review": null,
|
||||
"review": {
|
||||
"rounds": [
|
||||
{
|
||||
"n": 1,
|
||||
"op": "megavolt-241-inreview-r1",
|
||||
"by": "megavolt",
|
||||
"at": "2026-10-21T03:01:47.822Z",
|
||||
"issue": 1747,
|
||||
"candidate": {
|
||||
"kind": "manifest",
|
||||
"digest": "e82668642589eefca8b22433f1b0c7a24432f1993c0050f0a4b74c1146c93c2f",
|
||||
"text": "e94f98ce2bdefe85c1ea2e1af165473f9d9b04860700e84ed7382798e4b43523 agents/megavolt/work/queue-241/BUILD.md\n8351c9df4673c441f1ea8f2fdf2639e643b264180170fd1c93e0fca8d9359dae packages/bus/tests/land.test.mjs\n"
|
||||
},
|
||||
"request": "comment",
|
||||
"attempts": [
|
||||
{
|
||||
"op": "megavolt-241-inreview-r1",
|
||||
"by": "megavolt",
|
||||
"at": "2026-10-21T03:01:47.822Z",
|
||||
"state": "failed",
|
||||
"comment": null,
|
||||
"transport": {
|
||||
"outcome": "failed",
|
||||
"status": null,
|
||||
"comment": null,
|
||||
"detail": "pre-send: the credential or account check failed"
|
||||
},
|
||||
"resolutions": []
|
||||
},
|
||||
{
|
||||
"op": "sage-241-request-1",
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T03:02:56.461Z",
|
||||
"state": "posted",
|
||||
"comment": 29868,
|
||||
"transport": null,
|
||||
"resolutions": [
|
||||
{
|
||||
"verb": "resolve",
|
||||
"op": "sage-241-request-1-resolve",
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T03:04:33.826Z",
|
||||
"comment": 29868,
|
||||
"reason": null
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"duplicateRisk": false,
|
||||
"receipts": []
|
||||
}
|
||||
]
|
||||
},
|
||||
"claim": {
|
||||
"seat": "megavolt",
|
||||
"op": "megavolt-241-start-1"
|
||||
@@ -23163,8 +23258,8 @@
|
||||
"note": null,
|
||||
"blockedReason": null,
|
||||
"createdAt": "2026-10-21T01:52:55.772Z",
|
||||
"updatedAt": "2026-10-21T01:54:02.058Z",
|
||||
"updatedBy": "megavolt"
|
||||
"updatedAt": "2026-10-21T03:04:33.826Z",
|
||||
"updatedBy": "sage"
|
||||
},
|
||||
{
|
||||
"id": 242,
|
||||
@@ -23176,7 +23271,7 @@
|
||||
"closes": [
|
||||
1748
|
||||
],
|
||||
"state": "briefed",
|
||||
"state": "in-progress",
|
||||
"previousState": null,
|
||||
"required": false,
|
||||
"requiredSince": null,
|
||||
@@ -23197,12 +23292,15 @@
|
||||
"duckworth"
|
||||
],
|
||||
"review": null,
|
||||
"claim": null,
|
||||
"claim": {
|
||||
"seat": "megavolt",
|
||||
"op": "megavolt-242-start-1"
|
||||
},
|
||||
"note": null,
|
||||
"blockedReason": null,
|
||||
"createdAt": "2026-10-21T01:54:53.103Z",
|
||||
"updatedAt": "2026-10-21T01:55:17.835Z",
|
||||
"updatedBy": "sage"
|
||||
"updatedAt": "2026-10-21T03:03:20.738Z",
|
||||
"updatedBy": "megavolt"
|
||||
}
|
||||
],
|
||||
"log": [
|
||||
@@ -92612,6 +92710,253 @@
|
||||
"receipt": "ok sage-234-briefed rev 2617 row 234 queued→briefed"
|
||||
},
|
||||
"viewSha": "99921e9a304f3fec10159d53cb04451ab1fd1a56a0bf4683e88f490aca94d5c8"
|
||||
},
|
||||
{
|
||||
"rev": 2618,
|
||||
"op": "gizmoduck-234-claim-1",
|
||||
"verb": "move",
|
||||
"args": {
|
||||
"id": 234,
|
||||
"to": "in-progress",
|
||||
"reason": null,
|
||||
"candidate": null,
|
||||
"evidence": null,
|
||||
"issue": null
|
||||
},
|
||||
"by": "gizmoduck",
|
||||
"at": "2026-10-21T02:53:18.435Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 234,
|
||||
"from": "briefed",
|
||||
"to": "in-progress",
|
||||
"round": null,
|
||||
"issue": null,
|
||||
"candidate": null,
|
||||
"receipt": "ok gizmoduck-234-claim-1 rev 2618 row 234 briefed→in-progress"
|
||||
},
|
||||
"viewSha": "056671c84c73a4175e57b541dc0b528910786ed10b2dd5e48138641fa6dc1212"
|
||||
},
|
||||
{
|
||||
"rev": 2619,
|
||||
"op": "darkwing-232-inprogress-r2",
|
||||
"verb": "move",
|
||||
"args": {
|
||||
"id": 232,
|
||||
"to": "in-progress",
|
||||
"reason": null,
|
||||
"candidate": null,
|
||||
"evidence": null,
|
||||
"issue": null
|
||||
},
|
||||
"by": "darkwing",
|
||||
"at": "2026-10-21T02:58:12.490Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 232,
|
||||
"from": "in-review",
|
||||
"to": "in-progress",
|
||||
"round": null,
|
||||
"issue": null,
|
||||
"candidate": null,
|
||||
"receipt": "ok darkwing-232-inprogress-r2 rev 2619 row 232 in-review→in-progress"
|
||||
},
|
||||
"viewSha": "3018370333dff232c9672ca379c57d1123163b47aeb3feb84977709ae3517ea3"
|
||||
},
|
||||
{
|
||||
"rev": 2620,
|
||||
"op": "darkwing-232-inreview-r2",
|
||||
"verb": "move",
|
||||
"args": {
|
||||
"id": 232,
|
||||
"to": "in-review",
|
||||
"reason": null,
|
||||
"candidate": "/home/jwoltje/darkwing-scratch/r232/candidate-manifest-r2.sha256",
|
||||
"evidence": null,
|
||||
"issue": null
|
||||
},
|
||||
"by": "darkwing",
|
||||
"at": "2026-10-21T02:58:36.861Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 232,
|
||||
"from": "in-progress",
|
||||
"to": "in-review",
|
||||
"round": 2,
|
||||
"issue": 1734,
|
||||
"candidate": {
|
||||
"kind": "manifest",
|
||||
"digest": "7af3c94308e0ce99d88fb40624ad26592305624b139a95a0582991bdd2666b50",
|
||||
"text": "cec97e006778ea2df1e2cbdd25f1641ed3e1965cb1ea91ce782aa22007263357 agents/darkwing/work/queue-232/BUILD.md\nea516da8a6ccd5fa13e371bce83066f7fc7a6bffe545f155b7d7b076682dd40d docs/TOOLS.md\nd21c25c62fcbb9cad185c25ca2302bbb6b3e6ec53db5f4f85d7c445fb454c798 packages/cli/README.md\n681ed534f6ee6188ba5b628e882b17b5a4d9e748d6a649be622a0777a2ee0e87 packages/cli/tests/stack-copy.test.mjs\ned72964534f2289996672b08bc3b75539ec891a62688cedf9ea18a436b7df103 packages/cli/tests/stack-release-external.test.mjs\n896399deb3d0dd07651f965a53ede6717629dcb574876d418184a02ba093ada0 packages/cli/tests/stack-release-fixture.mjs\n533a91460f249bf0f9458fe4269795b8438ab209be7c4e6290e1ea0e29dd98d8 packages/cli/tests/stack-release.test.mjs\nc111fb361bac4fda2d2fdb4c1304e818c79995653ec36f346e44a863846cedc3 scripts/stack-copy.sh\n61db23e9c1b49878f6edfa0c07805d770c20103f06947f0bfcf72da6568f94c7 scripts/stack-release.sh\n"
|
||||
},
|
||||
"receipt": "ok darkwing-232-inreview-r2 rev 2620 row 232 in-progress→in-review round 2 on #1734; request darkwing-232-inreview-r2 requesting"
|
||||
},
|
||||
"viewSha": "f2b47852ccea465a933c307c1ba35d529b0646c59c6f294113aa5bc21feae00d"
|
||||
},
|
||||
{
|
||||
"rev": 2621,
|
||||
"op": "darkwing-232-inreview-r2.outcome",
|
||||
"verb": "review-outcome",
|
||||
"args": {
|
||||
"id": 232,
|
||||
"attempt": "darkwing-232-inreview-r2",
|
||||
"outcome": "posted",
|
||||
"status": 201,
|
||||
"comment": 29860,
|
||||
"detail": "created"
|
||||
},
|
||||
"by": "darkwing",
|
||||
"at": "2026-10-21T02:59:02.511Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 232,
|
||||
"round": 2,
|
||||
"attempt": "darkwing-232-inreview-r2",
|
||||
"state": "posted",
|
||||
"receipt": "ok darkwing-232-inreview-r2.outcome rev 2621 row 232 round 2 request darkwing-232-inreview-r2 posted comment 29860"
|
||||
},
|
||||
"viewSha": "47603fa4a7e49e17ac510cfb8c2a031829362ce92ce7c094ca1a3c6e3afe418f"
|
||||
},
|
||||
{
|
||||
"rev": 2622,
|
||||
"op": "filbert-232-record-r2",
|
||||
"verb": "review-record",
|
||||
"args": {
|
||||
"id": 232,
|
||||
"verdict": "approve",
|
||||
"comment": 29865,
|
||||
"candidate": "7af3c94308e0ce99d88fb40624ad26592305624b139a95a0582991bdd2666b50"
|
||||
},
|
||||
"by": "filbert",
|
||||
"at": "2026-10-21T03:01:22.609Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 232,
|
||||
"round": 2,
|
||||
"verdict": "approve",
|
||||
"receipt": "ok filbert-232-record-r2 rev 2622 row 232 round 2 approve by filbert (comment 29865)"
|
||||
},
|
||||
"viewSha": "69cd7e81ad20bc0dedb61b4c90962c6867d41fd8002f862687644d870f7e27ea"
|
||||
},
|
||||
{
|
||||
"rev": 2623,
|
||||
"op": "megavolt-241-inreview-r1",
|
||||
"verb": "move",
|
||||
"args": {
|
||||
"id": 241,
|
||||
"to": "in-review",
|
||||
"reason": null,
|
||||
"candidate": "/mnt/storage/scratch/megavolt-r241/manifest-r1-3e3ed303e.txt",
|
||||
"evidence": null,
|
||||
"issue": null
|
||||
},
|
||||
"by": "megavolt",
|
||||
"at": "2026-10-21T03:01:47.822Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 241,
|
||||
"from": "in-progress",
|
||||
"to": "in-review",
|
||||
"round": 1,
|
||||
"issue": 1747,
|
||||
"candidate": {
|
||||
"kind": "manifest",
|
||||
"digest": "e82668642589eefca8b22433f1b0c7a24432f1993c0050f0a4b74c1146c93c2f",
|
||||
"text": "e94f98ce2bdefe85c1ea2e1af165473f9d9b04860700e84ed7382798e4b43523 agents/megavolt/work/queue-241/BUILD.md\n8351c9df4673c441f1ea8f2fdf2639e643b264180170fd1c93e0fca8d9359dae packages/bus/tests/land.test.mjs\n"
|
||||
},
|
||||
"receipt": "ok megavolt-241-inreview-r1 rev 2623 row 241 in-progress→in-review round 1 on #1747; request megavolt-241-inreview-r1 requesting"
|
||||
},
|
||||
"viewSha": "60d7b7151ebee0eb8e0eb8916cfda532e2ca7b15debe7f128129576b73087ad8"
|
||||
},
|
||||
{
|
||||
"rev": 2624,
|
||||
"op": "megavolt-241-inreview-r1.outcome",
|
||||
"verb": "review-outcome",
|
||||
"args": {
|
||||
"id": 241,
|
||||
"attempt": "megavolt-241-inreview-r1",
|
||||
"outcome": "failed",
|
||||
"status": null,
|
||||
"comment": null,
|
||||
"detail": "pre-send: the credential or account check failed"
|
||||
},
|
||||
"by": "megavolt",
|
||||
"at": "2026-10-21T03:02:12.573Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 241,
|
||||
"round": 1,
|
||||
"attempt": "megavolt-241-inreview-r1",
|
||||
"state": "failed",
|
||||
"receipt": "ok megavolt-241-inreview-r1.outcome rev 2624 row 241 round 1 request megavolt-241-inreview-r1 failed"
|
||||
},
|
||||
"viewSha": "5e4e5c177acdd24d83c6b094937b937d8f826b530c13fd6e2da926d318fc9f22"
|
||||
},
|
||||
{
|
||||
"rev": 2625,
|
||||
"op": "sage-241-request-1",
|
||||
"verb": "review-request",
|
||||
"args": {
|
||||
"id": 241
|
||||
},
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T03:02:56.461Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 241,
|
||||
"round": 1,
|
||||
"attempt": "sage-241-request-1",
|
||||
"state": "requesting",
|
||||
"receipt": "ok sage-241-request-1 rev 2625 row 241 round 1 request sage-241-request-1 requesting"
|
||||
},
|
||||
"viewSha": "29f9f090c272ecb96782f36db298a78c9b6a2fc268ce0895d9a414bcec4cad2f"
|
||||
},
|
||||
{
|
||||
"rev": 2626,
|
||||
"op": "megavolt-242-start-1",
|
||||
"verb": "move",
|
||||
"args": {
|
||||
"id": 242,
|
||||
"to": "in-progress",
|
||||
"reason": null,
|
||||
"candidate": null,
|
||||
"evidence": null,
|
||||
"issue": null
|
||||
},
|
||||
"by": "megavolt",
|
||||
"at": "2026-10-21T03:03:20.738Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 242,
|
||||
"from": "briefed",
|
||||
"to": "in-progress",
|
||||
"round": null,
|
||||
"issue": null,
|
||||
"candidate": null,
|
||||
"receipt": "ok megavolt-242-start-1 rev 2626 row 242 briefed→in-progress"
|
||||
},
|
||||
"viewSha": "9ed411c0f2ba8ecbc8ee58657a30a585770c8311861f41dd607686f1bcd98992"
|
||||
},
|
||||
{
|
||||
"rev": 2627,
|
||||
"op": "sage-241-request-1-resolve",
|
||||
"verb": "review-resolve",
|
||||
"args": {
|
||||
"id": 241,
|
||||
"attempt": "sage-241-request-1",
|
||||
"comment": 29868
|
||||
},
|
||||
"by": "sage",
|
||||
"at": "2026-10-21T03:04:33.826Z",
|
||||
"semantics": 2,
|
||||
"result": {
|
||||
"row": 241,
|
||||
"round": 1,
|
||||
"attempt": "sage-241-request-1",
|
||||
"state": "posted",
|
||||
"receipt": "ok sage-241-request-1-resolve rev 2627 row 241 round 1 request sage-241-request-1 requesting→posted comment 29868"
|
||||
},
|
||||
"viewSha": "ec48b0f767195fb8ddaa8e67e9526a533ef4ddc789d6b11c0998befbf32433de"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
+4
-15
@@ -218,9 +218,6 @@ an operator inspect content and delivery evidence after an uncertain result.
|
||||
| `credentials.issue` | `{agent,service,token,date}`; human only (row 199, #1697). Writes the token to `<configDir>/secrets/agents/<business>/<agent>/<service>` (0600, dirs 0700: tmp, fsync, rename), rewrites the agent's reference in the business file to `{"store":true,<rotateBy or expires>:date}`, then holds the token in memory, with no restart. Over an issued entry it is a rotate: the old value is gone from disk and memory when it returns. `service` is `gitea` or `vikunja` and the business file must already name it for the agent (`credential-unknown`). A token the bus already holds is `credential-leak`; a bad token `credential-format`; a bad date `credential-date`. Returns `{agent,verb,service,date,state}`, `verb` `issue` or `rotate`. The tester gets `tester-refused`. |
|
||||
| `credentials.revoke` | `{agent,service?,reason}`; human, or the tester (Q173). Drops the entry from memory, then unlinks a store file; a `file` or `env` reference keeps its file and becomes `store`, so a restart loads nothing. Without `service`, every kind the agent has. Returns `{agent,verb,services:[{service,state}]}`. |
|
||||
| `credentials.delete` | `{agent,reason}`; human only. `credentials.revoke` for every kind, then removes the agent's store dir; a file there other than a token is `credential-file`. While the agent or one of its threads holds a claim, `agent-running`. |
|
||||
| `hosts.status` | No args; the human, the tester or the business's chief (row 234, #1740). `[{host,address,port,user,sudo,state,hostKeys,installerFingerprint,checkedAt,hostLineLeft}]` from the `hosts` records, with host keys as fingerprints. Never a key. |
|
||||
| `hosts.check` | `{host}`; same callers. Runs the host's `list <user>` as `mosaic-installer` with the installer key and the pinned host keys, then sets `state` `ready` and `checkedAt` and writes `grant.host.checked`. A nonzero exit is `host-failed`, a record that changed during the run `host-changed`, a record with no installer key `host-not-enrolled`. |
|
||||
| `hosts.remove` | `{host}`; the human or the tester. Deletes the record, writes `grant.host.removed` and removes the installer key. A `host_grants` record for the host is `host-in-use`, a guarded event naming a host line left on the host `host-line-left`. |
|
||||
|
||||
Each of the three leaves one receipt in the organization's `changes.jsonl`: the actor and a
|
||||
`detail` of `{verb,agent,kind,date or reason}`, never a value. A change to the store alone writes
|
||||
@@ -707,10 +704,10 @@ scope probes and rotation/expiry notifications; S2 never probes real services.
|
||||
|
||||
## Schema and verification
|
||||
|
||||
`schema.sql` is v3j, SHA-256
|
||||
`0a7b633ce7eec9d0bb37c3255f4244d97bd0c8eac38db91a11b4ef6cc982ec8c`.
|
||||
It has 18 tables, 72 triggers and five views. Source digest and a trusted full
|
||||
sqlite_master digest are checked at open, alongside schema version `3j`, WAL,
|
||||
`schema.sql` is v3i, SHA-256
|
||||
`6cde5232c57ac04dc6073112c0830c89cfc5e953d7a5727e7716263747fe8737`.
|
||||
It has 16 tables, 71 triggers and five views. Source digest and a trusted full
|
||||
sqlite_master digest are checked at open, alongside schema version `3i`, WAL,
|
||||
quick_check and foreign-key integrity. Missing metadata refuses; it never
|
||||
reinitializes an existing file. Full synchronous WAL transactions either commit
|
||||
or roll back. SQL UPDATE, DELETE and INSERT OR REPLACE refuse on evidence tables.
|
||||
@@ -870,14 +867,6 @@ one transaction; a failure leaves the store at 3h. It runs only through
|
||||
refuses a 3i store. The 3h fixture, `tests/fixtures/schema-3h.sql`, is pinned from
|
||||
81f26551b's tree; no live store is used.
|
||||
|
||||
Schema 3j (row 234, #1740) adds only new objects: the `hosts` and `host_grants` tables, which
|
||||
hold enrolled hosts and host grants, and the `events_grant_actor` trigger. The trigger refuses an
|
||||
`action.allowed` event whose `body.action` or `body.operation` matches `grant.*` unless its actor
|
||||
is `actor_role` NULL with `actor_run` 'grant', the broker's grant writer (#436b section 7). The
|
||||
step runs in one transaction, and a failure leaves the store at 3i. The 3i fixture,
|
||||
`tests/fixtures/schema-3i.sql`, is pinned from 8fede5fc6's tree, and `tests/grant-migration.test.mjs`
|
||||
covers the step and the trigger.
|
||||
|
||||
The managed runner calls usage.begin before an actual harness turn and usage.end
|
||||
after it, including a failed/stopped turn. Each takes only {turn: deliveredMessageId}.
|
||||
The broker requires the bound role holder and its delivery, binds run identity,
|
||||
|
||||
@@ -363,34 +363,3 @@ CREATE TABLE usage_records (
|
||||
CREATE TRIGGER usage_records_no_update BEFORE UPDATE ON usage_records BEGIN SELECT RAISE(ABORT, 'usage_records is append-only'); END;
|
||||
CREATE TRIGGER usage_records_no_delete BEFORE DELETE ON usage_records BEGIN SELECT RAISE(ABORT, 'usage_records is append-only'); END;
|
||||
CREATE TRIGGER usage_records_no_replace BEFORE INSERT ON usage_records WHEN EXISTS (SELECT 1 FROM usage_records WHERE seq=NEW.seq OR (business=NEW.business AND run=NEW.run AND turn=NEW.turn AND source=NEW.source AND phase=NEW.phase)) BEGIN SELECT RAISE(ABORT, 'usage_records is append-only'); END;
|
||||
CREATE TABLE hosts (
|
||||
business TEXT NOT NULL,
|
||||
host TEXT NOT NULL,
|
||||
address TEXT NOT NULL,
|
||||
port INTEGER NOT NULL CHECK (port BETWEEN 1 AND 65535),
|
||||
user TEXT NOT NULL,
|
||||
sudo INTEGER NOT NULL CHECK (sudo IN (0,1)),
|
||||
host_keys TEXT NOT NULL CHECK (json_valid(host_keys) AND json_type(host_keys) = 'array' AND json_array_length(host_keys) > 0),
|
||||
installer_fingerprint TEXT,
|
||||
state TEXT NOT NULL CHECK (state IN ('pending-install','ready')),
|
||||
decision TEXT NOT NULL REFERENCES decisions(id),
|
||||
checked_at TEXT,
|
||||
PRIMARY KEY (business, host),
|
||||
CHECK (state = 'pending-install' OR (installer_fingerprint IS NOT NULL AND checked_at IS NOT NULL))
|
||||
) STRICT;
|
||||
CREATE TABLE host_grants (
|
||||
business TEXT NOT NULL,
|
||||
agent TEXT NOT NULL,
|
||||
host TEXT NOT NULL,
|
||||
commands TEXT NOT NULL CHECK (json_valid(commands) AND json_type(commands) = 'array' AND json_array_length(commands) > 0),
|
||||
rotate_by TEXT NOT NULL,
|
||||
tag TEXT NOT NULL CHECK (length(tag) = 39 AND tag GLOB 'mosaic-*' AND NOT substr(tag, 8) GLOB '*[^0-9a-f]*'),
|
||||
decision TEXT NOT NULL REFERENCES decisions(id),
|
||||
PRIMARY KEY (business, agent, host),
|
||||
FOREIGN KEY (business, host) REFERENCES hosts(business, host)
|
||||
) STRICT;
|
||||
CREATE TRIGGER events_grant_actor BEFORE INSERT ON events
|
||||
WHEN NEW.kind = 'action.allowed'
|
||||
AND (json_extract(NEW.body, '$.action') GLOB 'grant.*' OR json_extract(NEW.body, '$.operation') GLOB 'grant.*')
|
||||
AND NOT (NEW.actor_role IS NULL AND NEW.actor_run IS 'grant')
|
||||
BEGIN SELECT RAISE(ABORT, 'only the grant writer records grant events'); END;
|
||||
|
||||
+17
-400
@@ -3,7 +3,7 @@ import { isAbsolute } from 'node:path';
|
||||
import { NAME as HARNESS_NAME } from '../../plugins/src/kinds.mjs';
|
||||
import { loadRequirements } from '../../tasks/src/requirements.mjs';
|
||||
import { MERGE, LandingRefusal, readLanding } from './landing.mjs';
|
||||
import { ID_PATTERN, THREAD_DEFINITION, END_PROTECTED, forkProblem, forkUrl, credentialKind, validArgv, commandAllowed, HOST_ID } from '../../business/src/vocabulary.mjs';
|
||||
import { ID_PATTERN, THREAD_DEFINITION, END_PROTECTED, forkProblem, forkUrl, credentialKind, validArgv, commandAllowed } from '../../business/src/vocabulary.mjs';
|
||||
|
||||
// Kept closed while S1 lands; its vocabulary is the integration authority.
|
||||
export const ACTIONS = Object.freeze([
|
||||
@@ -182,52 +182,6 @@ const STAFF_COUNT = 20;
|
||||
// Row 93 (#1587): the review loop's evidence, written only by the verbs below. A push is an
|
||||
// action.allowed for git.push.working that names its commit, written only by recordPush.
|
||||
const REVIEW_KINDS = ['review.requested', 'review.verdict'];
|
||||
// Row 234 (#1740, #436b section 7): grant evidence is an action.allowed whose body.action or
|
||||
// body.operation starts with grant., written only by #grantEvent under this actor. No session has a
|
||||
// NULL role with a non-NULL run, and the 3j trigger refuses any other actor.
|
||||
const GRANT_RUN = 'grant';
|
||||
const grantName = (body) => [body?.action, body?.operation].some(n => typeof n === 'string' && n.startsWith('grant.'));
|
||||
// Every grant-evidence read filters on this (duckworth-0067 N1): a look-alike under another kind or
|
||||
// actor is not evidence.
|
||||
const GRANT_EVIDENCE = "kind='action.allowed' AND actor_role IS NULL AND actor_run='grant'";
|
||||
// r6 R4: the host record's identity, hashed at raise and at each later check.
|
||||
export const registrySha256 = ({ address, port, user, sudo, hostKeys }) =>
|
||||
createHash('sha256').update(JSON.stringify({ address, port, user, sudo: !!sudo, hostKeys })).digest('hex');
|
||||
// Row 234 (#1740, #436b section 7): the one writer of grant evidence, session-less, as actor (NULL,
|
||||
// 'grant'). The broker's #grantEvent calls it after its secret check, and the tests write their fixtures
|
||||
// through it (duckworth-0079 F1). No verb reaches it. Only the consume record {operation, decision}
|
||||
// names the decision as `decision`, so #spent counts it and nothing else. Every other body has a
|
||||
// grant.* action, names the decision grantDecision, and body.by names the caller the broker checked.
|
||||
export function writeGrantEvent(store, at, business, body, subject = null) {
|
||||
const consume = body?.operation !== undefined;
|
||||
if (
|
||||
!grantName(body) ||
|
||||
(consume
|
||||
? Object.keys(body).length !== 2 || typeof body.decision !== 'string' || !body.operation.startsWith('grant.')
|
||||
: typeof body.action !== 'string' || !body.action.startsWith('grant.') || 'decision' in body)
|
||||
)
|
||||
throw new Error('grant event shape');
|
||||
const eid = randomUUID();
|
||||
store.run(
|
||||
'INSERT INTO events(id,at,business,kind,actor_role,actor_run,subject,body) VALUES(?,?,?,?,?,?,?,?)',
|
||||
eid,
|
||||
at,
|
||||
business,
|
||||
'action.allowed',
|
||||
null,
|
||||
GRANT_RUN,
|
||||
subject,
|
||||
JSON.stringify(body),
|
||||
);
|
||||
return eid;
|
||||
}
|
||||
export const HOSTS_VERBS = Object.freeze(['hosts.status', 'hosts.check', 'hosts.remove']);
|
||||
// The OpenSSH fingerprint of a "<type> <base64>" host key line. host-grant.mjs has the same; the broker
|
||||
// doesn't import it, so the sandbox's RUNNER_FILES stay as they are.
|
||||
const fingerprintOf = (line) =>
|
||||
'SHA256:' + createHash('sha256').update(Buffer.from(line.split(' ')[1], 'base64')).digest('base64').replace(/=+$/, '');
|
||||
// r6 E3: the installer line's comment, then the tag's 32 hex digits.
|
||||
const INSTALLER_COMMENT = 'mosaic-installer-';
|
||||
// Row 121 (#1615): an instance of THREAD_DEFINITION (vocabulary.mjs since row 155) talks with the
|
||||
// user in threads. Its runs claim a thread, not the role, and every message to or from it names its
|
||||
// thread (REQ-MSG-1).
|
||||
@@ -270,13 +224,11 @@ export class Broker {
|
||||
#usage;
|
||||
#limits;
|
||||
#harnessOn;
|
||||
// Row 234 (#1740): the trusted enrollment half (host-grant.mjs), or null where hosts are off.
|
||||
#hostGrant;
|
||||
// Row 125 (#1619, REQ-LIMIT-4): a run a limit stops, by business:run, until its next receive or state.
|
||||
#stops = new Map();
|
||||
#clock = 0;
|
||||
// credentialStatus is the runtime's metadata-only Credentials.status(); the broker never gets use().
|
||||
constructor({ store, businesses, secretCheck = () => {}, credentialStatus = () => [], organization = null, project = null, usage = null, limits = null, harnessOn = null, credentialVerbs = null, hostGrant = null }) {
|
||||
constructor({ store, businesses, secretCheck = () => {}, credentialStatus = () => [], organization = null, project = null, usage = null, limits = null, harnessOn = null, credentialVerbs = null }) {
|
||||
if (!object(businesses) || !Object.keys(businesses).length) fail('invalid-business');
|
||||
this.#businesses = structuredClone(businesses);
|
||||
this.#store = store;
|
||||
@@ -285,7 +237,6 @@ export class Broker {
|
||||
this.#organization = organization;
|
||||
// Row 199: the runtime's credential lifecycle (credential-verbs.mjs); it holds the store, not the broker.
|
||||
this.#credentialVerbs = credentialVerbs;
|
||||
this.#hostGrant = hostGrant;
|
||||
this.#project = project;
|
||||
this.#usage = usage;
|
||||
this.#limits = limits;
|
||||
@@ -653,323 +604,9 @@ export class Broker {
|
||||
);
|
||||
return eid;
|
||||
}
|
||||
// Row 234 (#1740): grant evidence, through writeGrantEvent. Caller owns the transaction.
|
||||
#grantEvent(business, body, subject = null) {
|
||||
this.#secretCheck({ kind: 'action.allowed', body, subject });
|
||||
return writeGrantEvent(this.#store, this.#now(), business, body, subject);
|
||||
}
|
||||
// PRD 1.6 REQ-WAIT-1: a hard wait goes to the human at every class, cited or not. Jason's ruling
|
||||
// Q169 makes a role.revoke whose target's definition is in END_PROTECTED one, so no arbiter
|
||||
// resolves a revoke of the assistant.
|
||||
// Row 234 (#1740, r6 X1): the raise's grant binding, if the human's cli resolve chose its approval
|
||||
// option on a credential.mint routed to the human. Null otherwise: an arbiter, the tester or an agent
|
||||
// closes with via 'broker'.
|
||||
#approvedGrant(business, decision) {
|
||||
const d = this.#store.get('SELECT action,route_to FROM decisions WHERE business=? AND id=?', business, decision);
|
||||
const raise = this.#raised(business, decision);
|
||||
if (d?.action !== 'credential.mint' || d.route_to !== 'human' || !raise?.grant) return null;
|
||||
const r = this.#closed(decision);
|
||||
if (r?.op !== 'resolved' || r.choice !== raise.approvalChoice || r.via !== 'cli' || r.by !== this.#business(business).human)
|
||||
return null;
|
||||
return raise.grant;
|
||||
}
|
||||
// r6 X1: the executor's own spend. The caller owns the transaction. The consume record is the only
|
||||
// guarded body with `decision` (#436b section 7), so #spent counts it.
|
||||
#consumeGrant(business, decision, method) {
|
||||
const grant = this.#approvedGrant(business, decision);
|
||||
if (grant?.method !== method) fail('not-authorized');
|
||||
if (this.#spent(business, decision)) fail('decision-consumed');
|
||||
this.#grantEvent(business, { operation: 'grant.' + method, decision }, decision);
|
||||
return grant;
|
||||
}
|
||||
#hostRecord(business, host) {
|
||||
const r = this.#store.get('SELECT * FROM hosts WHERE business=? AND host=?', business, host);
|
||||
return r && { ...r, sudo: r.sudo === 1, hostKeys: JSON.parse(r.host_keys) };
|
||||
}
|
||||
// r6 X4 first bullet, the Approve record check: the host record and the session's grant record, whose
|
||||
// decision is a credential.mint with a `host` grant, resolved yes by the human via cli and spent by
|
||||
// #consumeGrant as grant.host. Read at each call, so a grant, a revoke and a hosts.remove need no reload.
|
||||
#hostAccess(s, host) {
|
||||
const r = this.#hostRecord(s.business, host);
|
||||
if (!r) fail('host-unknown');
|
||||
const g = this.#store.get('SELECT * FROM host_grants WHERE business=? AND agent=? AND host=?', s.business, s.role, host);
|
||||
const spent = g && this.#store.get(
|
||||
`SELECT 1 FROM events WHERE business=? AND ${GRANT_EVIDENCE} AND json_extract(body,'$.operation')='grant.host' AND json_extract(body,'$.decision')=? LIMIT 1`,
|
||||
s.business,
|
||||
g.decision,
|
||||
);
|
||||
// The grant must name this agent and host too, so one approval can't stand behind another record.
|
||||
const grant = spent && this.#approvedGrant(s.business, g.decision);
|
||||
if (grant?.method !== 'host' || grant.agent !== s.role || grant.host !== host) fail('host-not-granted');
|
||||
return { entry: { address: r.address, port: r.port, user: r.user, hostKeys: r.hostKeys }, commands: JSON.parse(g.commands) };
|
||||
}
|
||||
// A grant outcome, from the bus, to the human's inbox and to the raising role, threaded as a
|
||||
// close is.
|
||||
#grantNotice(business, decision, body) {
|
||||
const d = this.#store.get('SELECT raised_by_role,raised_by_run FROM decisions WHERE id=?', decision);
|
||||
const thread = this.#threaded(business, d.raised_by_role)
|
||||
? (this.#store.get(
|
||||
"SELECT thread FROM thread_claims WHERE business=? AND holder_run=? AND op='claim' ORDER BY seq DESC LIMIT 1",
|
||||
business,
|
||||
d.raised_by_run,
|
||||
)?.thread ?? null)
|
||||
: null;
|
||||
for (const [to, t] of [['human', null], [d.raised_by_role, thread]]) {
|
||||
const mid = randomUUID();
|
||||
this.#store.run(
|
||||
'INSERT INTO messages(id,at,business,from_role,from_run,to_role,class,in_reply_to,decision,corrects,body) VALUES(?,?,?,?,?,?,?,?,?,?,?)',
|
||||
mid,
|
||||
this.#now(),
|
||||
business,
|
||||
NOTICE_ROLE,
|
||||
NOTICE_RUN,
|
||||
to,
|
||||
'RESULT',
|
||||
null,
|
||||
decision,
|
||||
null,
|
||||
body,
|
||||
);
|
||||
this.#route(business, mid, to, t);
|
||||
}
|
||||
}
|
||||
#enrollFailed(business, decision, host, code, extra = {}) {
|
||||
this.#store.run('DELETE FROM hosts WHERE business=? AND host=? AND decision=?', business, host, decision);
|
||||
this.#grantEvent(business, { action: 'grant.failed', grantDecision: decision, host, code, ...extra }, host);
|
||||
this.#grantNotice(business, decision, `Enrollment of host ${host} (decision ${decision}) failed: ${code}.`);
|
||||
}
|
||||
// Trusted host API, as reconcileGrants is, and no verb reaches it: the executor's entry for one
|
||||
// decision. Step 1 checks everything again, so a second start for a spent decision ends
|
||||
// decision-consumed, and an unresolved or agent-resolved one ends not-authorized, with nothing
|
||||
// written (r6 G10).
|
||||
runEnrollment(business, decision) {
|
||||
return this.#runEnrollment(business, decision);
|
||||
}
|
||||
// r6 E2, started by the human's cli yes. Never throws: the outcome is in the store and the notices,
|
||||
// and the return value is its code, 'enrolled' or the refusal.
|
||||
async #runEnrollment(business, decision) {
|
||||
let grant;
|
||||
try {
|
||||
// Step 1, one transaction: spend, the not-enrolled check, the registry step and the record.
|
||||
grant = this.#store.transaction(() => {
|
||||
const g = this.#consumeGrant(business, decision, 'host-enroll');
|
||||
if (this.#hostRecord(business, g.host)) {
|
||||
this.#grantEvent(business, { action: 'grant.failed', grantDecision: decision, host: g.host, code: 'host-enrolled' }, g.host);
|
||||
this.#grantNotice(business, decision, `Enrollment of host ${g.host} (decision ${decision}) failed: host-enrolled.`);
|
||||
return null;
|
||||
}
|
||||
this.#grantEvent(business, { action: 'grant.host.enroll.step', grantDecision: decision, host: g.host, step: 'registry' }, g.host);
|
||||
this.#store.run(
|
||||
'INSERT INTO hosts(business,host,address,port,user,sudo,host_keys,installer_fingerprint,state,decision,checked_at) VALUES(?,?,?,?,?,?,?,?,?,?,?)',
|
||||
business, g.host, g.address, g.port, g.user, Number(g.sudo), JSON.stringify(g.hostKeys), null, 'pending-install', decision, null,
|
||||
);
|
||||
return g;
|
||||
});
|
||||
} catch (e) {
|
||||
// Spent, not approved or no store: nothing was written. The start reconcile ends a resolved,
|
||||
// unspent decision.
|
||||
return e instanceof BusError ? e.code : 'host-failed';
|
||||
}
|
||||
if (!grant) return 'host-enrolled';
|
||||
const host = grant.host;
|
||||
try {
|
||||
if (!this.#hostGrant) fail('host-not-configured');
|
||||
// Steps 2 and 3: the key pair, and the private half to its file.
|
||||
const key = await this.#hostGrant.enroll(business, host);
|
||||
// Step 4.
|
||||
this.#store.transaction(() => {
|
||||
const r = this.#hostRecord(business, host);
|
||||
if (r?.decision !== decision) fail('host-changed');
|
||||
this.#store.run('UPDATE hosts SET installer_fingerprint=? WHERE business=? AND host=?', key.fingerprint, business, host);
|
||||
this.#grantEvent(
|
||||
business,
|
||||
{
|
||||
action: 'grant.host.enrolled',
|
||||
grantDecision: decision,
|
||||
host,
|
||||
registrySha256: registrySha256(r),
|
||||
installerFingerprint: key.fingerprint,
|
||||
user: r.user,
|
||||
sudo: r.sudo,
|
||||
},
|
||||
host,
|
||||
);
|
||||
this.#grantNotice(
|
||||
business,
|
||||
decision,
|
||||
`Host ${host} is enrolled (decision ${decision}) and pending install. ops-01 writes this line to ` +
|
||||
`/etc/ssh/mosaic-keys/mosaic-installer, with <from> the value in /etc/mosaic-keys/from:\n` +
|
||||
`restrict,command="/usr/local/sbin/mosaic-keys",from="<from>" ${key.publicLine} ` +
|
||||
`${INSTALLER_COMMENT}${key.tag.slice('mosaic-'.length)}\n` +
|
||||
`/etc/mosaic-keys/users lists ${r.user}.` +
|
||||
(r.sudo ? ` /etc/mosaic-keys/privileged lists ${r.user}.` : ` /etc/mosaic-keys/privileged doesn't list ${r.user}.`) +
|
||||
`\nThen run: mosaic hosts check ${host}`,
|
||||
);
|
||||
});
|
||||
return 'enrolled';
|
||||
} catch (e) {
|
||||
const code = e instanceof BusError ? e.code : 'host-failed';
|
||||
try {
|
||||
this.#hostGrant?.unenroll(business, host);
|
||||
} catch {}
|
||||
try {
|
||||
this.#store.transaction(() => this.#enrollFailed(business, decision, host, code));
|
||||
} catch {}
|
||||
return code;
|
||||
}
|
||||
}
|
||||
// r6 X3 and duckworth-0067 N1: credential.mint decisions for `method`, resolved yes by the human via
|
||||
// cli and never spent.
|
||||
#openGrantDecisions(business, method) {
|
||||
return this.#store
|
||||
.all("SELECT id FROM decisions WHERE business=? AND action='credential.mint' AND route_to='human' ORDER BY at,id", business)
|
||||
.map((d) => d.id)
|
||||
.filter((id) => this.#approvedGrant(business, id)?.method === method && !this.#spent(business, id));
|
||||
}
|
||||
// Trusted host API, at start before the socket opens (r6 X3). An enrollment spent with no
|
||||
// grant.host.enrolled or grant.failed died after its registry step: the key file and the record
|
||||
// go, and it ends grant.failed interrupted. A resolved, unspent one is spent and ends the same way.
|
||||
// Neither runs late.
|
||||
reconcileGrants() {
|
||||
const ended = [];
|
||||
for (const business of Object.keys(this.#businesses)) {
|
||||
const spent = this.#store.all(
|
||||
`SELECT json_extract(body,'$.decision') AS decision FROM events WHERE business=? AND ${GRANT_EVIDENCE} AND json_extract(body,'$.operation')='grant.host-enroll' ORDER BY seq`,
|
||||
business,
|
||||
);
|
||||
for (const { decision } of spent) {
|
||||
const end = this.#store.get(
|
||||
`SELECT 1 FROM events WHERE business=? AND ${GRANT_EVIDENCE} AND json_extract(body,'$.grantDecision')=? AND json_extract(body,'$.action') IN ('grant.host.enrolled','grant.failed') LIMIT 1`,
|
||||
business,
|
||||
decision,
|
||||
);
|
||||
if (end) continue;
|
||||
const host = this.#raised(business, decision).grant.host;
|
||||
this.#hostGrant?.unenroll(business, host);
|
||||
this.#store.transaction(() =>
|
||||
this.#enrollFailed(business, decision, host, 'interrupted', { tag: this.#hostGrant?.installerTag(business, host) ?? null }),
|
||||
);
|
||||
ended.push(decision);
|
||||
}
|
||||
for (const decision of this.#openGrantDecisions(business, 'host-enroll')) {
|
||||
this.#store.transaction(() => {
|
||||
const g = this.#consumeGrant(business, decision, 'host-enroll');
|
||||
this.#grantEvent(business, { action: 'grant.failed', grantDecision: decision, host: g.host, code: 'interrupted', tag: this.#hostGrant?.installerTag(business, g.host) ?? null }, g.host);
|
||||
this.#grantNotice(business, decision, `Enrollment of host ${g.host} (decision ${decision}) failed: interrupted.`);
|
||||
});
|
||||
ended.push(decision);
|
||||
}
|
||||
}
|
||||
return ended;
|
||||
}
|
||||
// r6 E1, R3: the raise checks and the keyscan, then the raise transaction.
|
||||
async #raiseGrant(s, a) {
|
||||
try {
|
||||
this.#store.transaction(() => this.#agent(s));
|
||||
if (a.action !== 'credential.mint' || 'spec_path' in a || 'spec_commit' in a) fail('invalid-request');
|
||||
if (this.#business(s.business).roles[s.role]?.definition !== STAFF_TO) fail('grant-chief-only');
|
||||
if (!this.#hostGrant) fail('host-not-configured');
|
||||
const enroll = this.#hostGrant.enrollCheck(a.grant);
|
||||
if (this.#hostRecord(s.business, enroll.host)) fail('host-enrolled');
|
||||
const hostKeys = await this.#hostGrant.keyscan(enroll.address, enroll.port);
|
||||
if (!this.#hostGrant.sameFingerprints(hostKeys, enroll.fingerprints)) fail('host-key-mismatch');
|
||||
return this.#store.transaction(() => {
|
||||
if (this.#hostRecord(s.business, enroll.host)) fail('host-enrolled');
|
||||
const result = this.#dispatch(s, 'decision.raise', a, null, { ...enroll, hostKeys });
|
||||
this.#secretCheck(result);
|
||||
return result;
|
||||
});
|
||||
} catch (e) {
|
||||
throw this.#refused(s, e, { verb: 'decision.raise', args: a });
|
||||
}
|
||||
}
|
||||
#hostsCaller(s, verb) {
|
||||
if (s.reader) fail('read-only');
|
||||
if (s.tester) return 'tester';
|
||||
if (s.human) {
|
||||
this.#human(s);
|
||||
return s.human;
|
||||
}
|
||||
// hosts.remove takes access away, as a revoke does: the human or the tester (r6 E4).
|
||||
if (verb === 'hosts.remove') fail('human-required');
|
||||
this.#store.transaction(() => this.#agent(s));
|
||||
if (this.#business(s.business).roles[s.role]?.definition !== STAFF_TO) fail('grant-chief-only');
|
||||
return s.role;
|
||||
}
|
||||
// r6 E4: hosts.status, hosts.check and hosts.remove. Async, because check runs ssh.
|
||||
async #requestHosts(s, verb, a) {
|
||||
try {
|
||||
const by = this.#hostsCaller(s, verb);
|
||||
if (verb === 'hosts.status') {
|
||||
keys(a, []);
|
||||
return this.#store
|
||||
.all('SELECT host FROM hosts WHERE business=? ORDER BY host', s.business)
|
||||
.map(({ host }) => {
|
||||
const r = this.#hostRecord(s.business, host);
|
||||
return {
|
||||
host,
|
||||
address: r.address,
|
||||
port: r.port,
|
||||
user: r.user,
|
||||
sudo: r.sudo,
|
||||
state: r.state,
|
||||
hostKeys: r.hostKeys.map(fingerprintOf),
|
||||
installerFingerprint: r.installer_fingerprint,
|
||||
checkedAt: r.checked_at,
|
||||
hostLineLeft: this.#hostLinesLeft(s.business, host),
|
||||
};
|
||||
});
|
||||
}
|
||||
keys(a, ['host'], ['host']);
|
||||
if (typeof a.host !== 'string' || !HOST_ID.test(a.host)) fail('invalid-request');
|
||||
if (!this.#hostGrant) fail('host-not-configured');
|
||||
if (verb === 'hosts.remove') {
|
||||
return this.#store.transaction(() => {
|
||||
const r = this.#hostRecord(s.business, a.host);
|
||||
if (!r) fail('host-unknown');
|
||||
if (this.#store.get('SELECT 1 FROM host_grants WHERE business=? AND host=? LIMIT 1', s.business, a.host)) fail('host-in-use');
|
||||
if (this.#hostLinesLeft(s.business, a.host).length) fail('host-line-left');
|
||||
this.#store.run('DELETE FROM hosts WHERE business=? AND host=?', s.business, a.host);
|
||||
this.#grantEvent(s.business, { action: 'grant.host.removed', host: a.host, registrySha256: registrySha256(r), by }, a.host);
|
||||
// Last, so a key file that won't go rolls the record and the event back.
|
||||
this.#hostGrant.unenroll(s.business, a.host);
|
||||
return { host: a.host, removed: true };
|
||||
});
|
||||
}
|
||||
const before = this.#hostRecord(s.business, a.host);
|
||||
if (!before) fail('host-unknown');
|
||||
if (!before.installer_fingerprint) fail('host-not-enrolled');
|
||||
const out = await this.#hostGrant.check(s.business, a.host, before);
|
||||
if (out.exit !== 0) fail('host-failed');
|
||||
return this.#store.transaction(() => {
|
||||
const r = this.#hostRecord(s.business, a.host);
|
||||
const sha = r && registrySha256(r);
|
||||
if (!r || r.decision !== before.decision || sha !== registrySha256(before) || out.installerFingerprint !== r.installer_fingerprint)
|
||||
fail('host-changed');
|
||||
const at = this.#now();
|
||||
this.#store.run("UPDATE hosts SET state='ready',checked_at=? WHERE business=? AND host=?", at, s.business, a.host);
|
||||
this.#grantEvent(
|
||||
s.business,
|
||||
{ action: 'grant.host.checked', host: a.host, registrySha256: sha, installerFingerprint: r.installer_fingerprint, by },
|
||||
a.host,
|
||||
);
|
||||
return { host: a.host, state: 'ready', checkedAt: at };
|
||||
});
|
||||
} catch (e) {
|
||||
throw this.#refused(s, e, { verb, args: a });
|
||||
}
|
||||
}
|
||||
// r6 E4, duckworth-0079 F2 and F3: every guarded body that left a line on the host names the host.
|
||||
// Row b adds the sweep clear (F4).
|
||||
#hostLinesLeft(business, host) {
|
||||
return this.#store
|
||||
.all(
|
||||
`SELECT json_extract(body,'$.hostLineLeft') AS tag FROM events WHERE business=? AND ${GRANT_EVIDENCE} AND json_extract(body,'$.host')=? AND json_extract(body,'$.hostLineLeft') IS NOT NULL ORDER BY seq`,
|
||||
business,
|
||||
host,
|
||||
)
|
||||
.map((e) => e.tag);
|
||||
}
|
||||
#hardWait(s, action, target) {
|
||||
if (GATED.has(action)) return true;
|
||||
return action === 'role.revoke' && END_PROTECTED.includes(this.#business(s.business).roles[target]?.definition);
|
||||
@@ -1609,7 +1246,6 @@ export class Broker {
|
||||
this.#agent(s);
|
||||
if (['human.input', 'launch.revoked', 'launch.restored', ...REVIEW_KINDS].includes(kind)) fail('reserved-event');
|
||||
if (kind === 'action.allowed' && ['git.push.working', MERGE].includes(body?.action)) fail('reserved-event');
|
||||
if (kind === 'action.allowed' && grantName(body)) fail('reserved-event');
|
||||
return this.#event(s, kind, body, subject);
|
||||
});
|
||||
}
|
||||
@@ -1735,9 +1371,6 @@ export class Broker {
|
||||
return this.#requestUsage(s, request.verb, structuredClone(args));
|
||||
if ((request.verb === 'message.receive' || request.verb === 'state') && this.#stops.has(s.business + ':' + s.run))
|
||||
this.#limitStop(s);
|
||||
// Row 234 (#1740, r6 E1): a raise with `grant` runs its keyscan before the raise transaction.
|
||||
if (request.verb === 'decision.raise' && 'grant' in args) return this.#raiseGrant(s, structuredClone(args));
|
||||
if (HOSTS_VERBS.includes(request.verb)) return this.#requestHosts(s, request.verb, structuredClone(args));
|
||||
if (request.verb === 'spec.read' || request.verb === 'record.write' ||
|
||||
(request.verb === 'decision.raise' && ('spec_path' in args || 'spec_commit' in args)))
|
||||
return this.#requestProject(s, { verb: request.verb, args: structuredClone(args) });
|
||||
@@ -2094,18 +1727,21 @@ export class Broker {
|
||||
a = request.args ?? {};
|
||||
keys(a, ['host', 'argv', 'stdin', 'timeout', 'decision'], ['host', 'argv']);
|
||||
this.#secretCheck(a);
|
||||
this.#store.transaction(() => this.#agent(s));
|
||||
const b = this.#business(s.business);
|
||||
identifier(a.host);
|
||||
// Row 234 (#1740, r6 X4): the store records, not the business file.
|
||||
const access = this.#store.transaction(() => (this.#agent(s), this.#hostAccess(s, a.host)));
|
||||
if (!Object.hasOwn(b.hosts ?? {}, a.host)) fail('host-unknown');
|
||||
const grant = Object.hasOwn(b.roles[s.role]?.hosts ?? {}, a.host) ? b.roles[s.role].hosts[a.host] : null;
|
||||
if (!grant) fail('host-not-granted');
|
||||
if (!validArgv(a.argv) || !validStdin(a.stdin)) fail('invalid-request');
|
||||
if (!commandAllowed(a.argv, access.commands)) fail('command-not-granted');
|
||||
if (!commandAllowed(a.argv, grant.commands)) fail('command-not-granted');
|
||||
timeout = a.timeout ?? HOST_TIMEOUT_DEFAULT;
|
||||
if (!Number.isInteger(timeout) || timeout < 1 || timeout > HOST_TIMEOUT_MAX) fail('invalid-request');
|
||||
if (a.decision !== undefined) identifier(a.decision);
|
||||
if (!executor) fail('host-not-configured');
|
||||
// A busy host refuses before the decision is spent; the executor checks again under its lock.
|
||||
if (executor.busy(s, a.host)) fail('host-busy');
|
||||
entry = access.entry;
|
||||
entry = b.hosts[a.host];
|
||||
bound = hostRun(a.host, a.argv, a.stdin);
|
||||
this.#store.transaction(() =>
|
||||
this.#consumeAuthority(
|
||||
@@ -2157,7 +1793,7 @@ export class Broker {
|
||||
});
|
||||
return out;
|
||||
}
|
||||
#dispatch(s, verb, a, spec = null, enroll = null) {
|
||||
#dispatch(s, verb, a, spec = null) {
|
||||
const b = this.#business(s.business),
|
||||
by = s.human || s.run;
|
||||
switch (verb) {
|
||||
@@ -2262,12 +1898,9 @@ export class Broker {
|
||||
'spec_commit',
|
||||
'argv',
|
||||
'stdin',
|
||||
'grant',
|
||||
],
|
||||
['action', 'question', 'options', 'recommendation', 'blocking'],
|
||||
);
|
||||
// Only #raiseGrant passes `enroll`, after its checks and keyscan.
|
||||
if (('grant' in a) !== (enroll !== null)) fail('invalid-request');
|
||||
this.#agent(s);
|
||||
if ((a.spec_path !== undefined || a.spec_commit !== undefined) && !spec) fail('invalid-spec-citation');
|
||||
const cls = this.#classification(s, a.action, a.target);
|
||||
@@ -2328,29 +1961,18 @@ export class Broker {
|
||||
if (!sudo && (a.argv !== undefined || a.stdin !== undefined)) fail('invalid-request');
|
||||
let question = pushCommit ? `${a.question} (commit ${pushCommit})` : a.question;
|
||||
if (sudo) {
|
||||
if (a.target === undefined) fail('host-unknown');
|
||||
// The human is asked only about a call host.run would take: the raiser's own grant and prefix (N1).
|
||||
// Row 234 (r6 X4): read from the store records, with the Approve record check.
|
||||
const access = this.#hostAccess(s, a.target);
|
||||
if (a.target === undefined || !Object.hasOwn(b.hosts ?? {}, a.target)) fail('host-unknown');
|
||||
if (!validArgv(a.argv) || !validStdin(a.stdin)) fail('invalid-request');
|
||||
if (!commandAllowed(a.argv, access.commands)) fail('command-not-granted');
|
||||
const h = access.entry;
|
||||
// The human is asked only about a call host.run would take: the raiser's own grant and prefix (N1).
|
||||
const grant = Object.hasOwn(b.roles[s.role]?.hosts ?? {}, a.target) ? b.roles[s.role].hosts[a.target] : null;
|
||||
if (!grant) fail('host-not-granted');
|
||||
if (!commandAllowed(a.argv, grant.commands)) fail('command-not-granted');
|
||||
const h = b.hosts[a.target];
|
||||
question +=
|
||||
`\n\nHost ${a.target}, ${h.user} at ${h.address} port ${h.port}, runs as root:` +
|
||||
`\nargv: ${a.argv.map((x) => JSON.stringify(x)).join(' ')}` +
|
||||
`\nstdin: ${JSON.stringify(a.stdin ?? '')}`;
|
||||
}
|
||||
// Row 234 (#1740, r6 E1): the human sees each scanned key with its fingerprint, and the sudo
|
||||
// mark in plain words. The chief can't shape this block.
|
||||
if (enroll) {
|
||||
question +=
|
||||
`\n\nEnroll host ${enroll.host}: user ${enroll.user} at ${enroll.address} port ${enroll.port}.` +
|
||||
`\nHost keys from ssh-keyscan, equal to the requested fingerprints:` +
|
||||
enroll.hostKeys.map((k) => `\n${fingerprintOf(k)} ${k}`).join('') +
|
||||
(enroll.sudo
|
||||
? `\n${enroll.user} has sudo on this host. Approving lets later grants for ${enroll.user} be installed, and section 9's first limit applies to ${enroll.user}.`
|
||||
: `\n${enroll.user} has no sudo on this host.`);
|
||||
}
|
||||
string(question);
|
||||
const did = randomUUID();
|
||||
this.#store.run(
|
||||
@@ -2388,7 +2010,6 @@ export class Broker {
|
||||
a.action === 'role.revoke' ? (this.#holder(s.business, a.target)?.holder_run ?? null) : null,
|
||||
...(pushCommit ? { pushCommit } : {}),
|
||||
...(sudo ? { hostRun: hostRun(a.target, a.argv, a.stdin) } : {}),
|
||||
...(enroll ? { grant: enroll } : {}),
|
||||
},
|
||||
a.task_ref ?? (taskRef(a.target) ? a.target : did),
|
||||
);
|
||||
@@ -2439,10 +2060,6 @@ export class Broker {
|
||||
a.note ?? null,
|
||||
s.human ? 'cli' : 'broker',
|
||||
);
|
||||
// Row 234 (#1740, r6 X1): the human's cli approve of a grant mint starts the executor once this
|
||||
// transaction commits. It rechecks all of this itself, so a rollback starts nothing.
|
||||
const grant = op === 'resolved' ? this.#approvedGrant(s.business, d.id) : null;
|
||||
if (grant?.method === 'host-enroll') setImmediate(() => this.#runEnrollment(s.business, d.id));
|
||||
if (s.human)
|
||||
this.#event(
|
||||
s,
|
||||
@@ -2486,7 +2103,7 @@ export class Broker {
|
||||
);
|
||||
this.#route(s.business, mid, d.raised_by_role, thread);
|
||||
}
|
||||
return grant ? { id: d.id, op, grant: 'started' } : { id: d.id, op };
|
||||
return { id: d.id, op };
|
||||
}
|
||||
case 'message.send': {
|
||||
keys(a, ['to', 'body', 'class', 'in_reply_to', 'corrects', 'decision'], ['to', 'body']);
|
||||
|
||||
@@ -98,7 +98,7 @@ export function assertCleanKey(value, key) {
|
||||
if (key.needles.some((n) => encoded.includes(n))) deny('credential-leak');
|
||||
}
|
||||
// Row 199 (#1697, design D1, D4): a store dir is the stack instance's user's own and nobody else's.
|
||||
export const privateDir = (path, { missing = false } = {}) => {
|
||||
const privateDir = (path, { missing = false } = {}) => {
|
||||
let st;
|
||||
try {
|
||||
st = lstatSync(path);
|
||||
|
||||
@@ -1,217 +0,0 @@
|
||||
// Row 234 (#1740, #436a row a): the trusted half of host enrollment, run in the bus child. The broker
|
||||
// gets it as `hostGrant` and holds every store write. This module holds what touches the network or
|
||||
// a key file: the raise's ssh-keyscan (E1), the installer key pair (E2 steps 2 and 3) and the installer's
|
||||
// `list` for hosts.check (E4).
|
||||
//
|
||||
// The installer key lives at <configDir>/secrets/hosts/<business>/<hostId>/installer, 0600 in 0700 dirs,
|
||||
// written as 210's issue writes (tmp, fsync, rename). It is never read into an event or a message.
|
||||
import { spawn } from 'node:child_process';
|
||||
import { createHash } from 'node:crypto';
|
||||
import {
|
||||
closeSync, constants, fsyncSync, mkdirSync, mkdtempSync, openSync, readFileSync, renameSync, rmSync, rmdirSync,
|
||||
unlinkSync, writeSync,
|
||||
} from 'node:fs';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { BusError } from './broker.mjs';
|
||||
import { parseHostKey, privateDir } from './credentials.mjs';
|
||||
import { runHost, HOST_LIMITS } from './host-run.mjs';
|
||||
import { checkAddress, HOST_KEY, USER } from '../../business/src/hosts.mjs';
|
||||
import { HOST_ID } from '../../business/src/vocabulary.mjs';
|
||||
|
||||
const deny = (code) => {
|
||||
throw new BusError(code);
|
||||
};
|
||||
const PATH = '/usr/bin:/bin';
|
||||
const KILL_WITH_PARENT = ['/usr/bin/setpriv', '--pdeathsig', 'KILL', '--'];
|
||||
// E1: the users no enrollment may name. The host writer refuses them again (tools/host/mosaic-keys-write).
|
||||
export const RESERVED_USERS = Object.freeze(['root', 'localadmin', 'mosaic-installer']);
|
||||
const FINGERPRINT = /^SHA256:[A-Za-z0-9+/]{43}$/;
|
||||
// E1: the three key types the keyscan asks for, so any other fingerprint can't match.
|
||||
const SCAN_TYPES = 'ed25519,ecdsa,rsa';
|
||||
export const INSTALLER_USER = 'mosaic-installer';
|
||||
const INSTALLER_AGENT = '@installer';
|
||||
const SCAN_MS = 10000;
|
||||
|
||||
// X2 step 2: `mosaic-` and the first 32 hex digits of sha256 over business, agent and host id joined
|
||||
// with NUL. Names match ID_PATTERN or HOST_ID, neither holds NUL, so the join is one to one.
|
||||
export function grantTag(business, agent, hostId) {
|
||||
return 'mosaic-' + createHash('sha256').update([business, agent, hostId].join('\0')).digest('hex').slice(0, 32);
|
||||
}
|
||||
export const installerTag = (business, hostId) => grantTag(business, INSTALLER_AGENT, hostId);
|
||||
|
||||
// The OpenSSH fingerprint of a "<type> <base64>" line: sha256 over the decoded blob.
|
||||
export function fingerprint(line) {
|
||||
const blob = Buffer.from(line.split(' ')[1], 'base64');
|
||||
return 'SHA256:' + createHash('sha256').update(blob).digest('base64').replace(/=+$/, '');
|
||||
}
|
||||
|
||||
// E1's binding. Anything off refuses invalid-request; the broker adds the chief, action and enrolled
|
||||
// checks. Returns a fresh object with only the bound keys.
|
||||
export function enrollCheck(grant) {
|
||||
const allowed = ['method', 'host', 'address', 'port', 'user', 'sudo', 'fingerprints'];
|
||||
if (!grant || typeof grant !== 'object' || Array.isArray(grant)) deny('invalid-request');
|
||||
const own = Object.keys(grant);
|
||||
if (own.some((k) => !allowed.includes(k)) || allowed.some((k) => !own.includes(k))) deny('invalid-request');
|
||||
if (grant.method !== 'host-enroll') deny('invalid-request');
|
||||
if (typeof grant.host !== 'string' || !HOST_ID.test(grant.host)) deny('invalid-request');
|
||||
try {
|
||||
checkAddress(grant.address, 'grant.address');
|
||||
} catch {
|
||||
deny('invalid-request');
|
||||
}
|
||||
if (!Number.isInteger(grant.port) || grant.port < 1 || grant.port > 65535) deny('invalid-request');
|
||||
if (typeof grant.user !== 'string' || !USER.test(grant.user) || RESERVED_USERS.includes(grant.user)) deny('invalid-request');
|
||||
if (typeof grant.sudo !== 'boolean') deny('invalid-request');
|
||||
const prints = grant.fingerprints;
|
||||
if (!Array.isArray(prints) || prints.length < 1 || prints.length > 3) deny('invalid-request');
|
||||
if (prints.some((f) => typeof f !== 'string' || !FINGERPRINT.test(f)) || new Set(prints).size !== prints.length) deny('invalid-request');
|
||||
return { method: 'host-enroll', host: grant.host, address: grant.address, port: grant.port, user: grant.user, sudo: grant.sudo, fingerprints: [...prints] };
|
||||
}
|
||||
|
||||
// The keyscan's output lines, "<host> <type> <base64>", as "<type> <base64>". Comment lines and any
|
||||
// other type drop out. A key listed twice counts once.
|
||||
export function scanLines(stdout) {
|
||||
const keys = [];
|
||||
for (const line of stdout.split('\n')) {
|
||||
if (!line || line.startsWith('#')) continue;
|
||||
const parts = line.trim().split(/\s+/);
|
||||
if (parts.length !== 3) continue;
|
||||
const key = parts[1] + ' ' + parts[2];
|
||||
if (HOST_KEY.test(key) && !keys.includes(key)) keys.push(key);
|
||||
}
|
||||
return keys;
|
||||
}
|
||||
|
||||
// E1: `ssh-keyscan -T 5 -p <port> -t ed25519,ecdsa,rsa <address>` from the stack host. It reads public
|
||||
// keys and logs in to nothing. A scan that can't start refuses host-failed. One that runs and gets no key,
|
||||
// as from a host that doesn't answer, returns no lines, so the raise refuses host-key-mismatch
|
||||
// (duckworth-0081 N3).
|
||||
export function keyscan(address, port, { bin = 'ssh-keyscan', limitMs = SCAN_MS } = {}) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const child = spawn(KILL_WITH_PARENT[0], [...KILL_WITH_PARENT.slice(1), bin, '-T', '5', '-p', String(port), '-t', SCAN_TYPES, '--', address], {
|
||||
env: { PATH },
|
||||
detached: true,
|
||||
stdio: ['ignore', 'pipe', 'ignore'],
|
||||
});
|
||||
const out = [];
|
||||
let size = 0;
|
||||
child.stdout.on('data', (chunk) => {
|
||||
size += chunk.length;
|
||||
if (size <= 65536) out.push(chunk);
|
||||
});
|
||||
const timer = setTimeout(() => {
|
||||
try {
|
||||
process.kill(-child.pid, 'SIGKILL');
|
||||
} catch {}
|
||||
}, limitMs);
|
||||
child.on('error', () => (clearTimeout(timer), reject(new BusError('host-failed'))));
|
||||
child.on('close', () => (clearTimeout(timer), resolve(scanLines(Buffer.concat(out).toString('utf8')))));
|
||||
});
|
||||
}
|
||||
|
||||
// E1: the scanned set equals the requested set, or host-key-mismatch.
|
||||
export function sameFingerprints(lines, prints) {
|
||||
const scanned = new Set(lines.map(fingerprint));
|
||||
return scanned.size === prints.length && prints.every((f) => scanned.has(f));
|
||||
}
|
||||
|
||||
const run = (bin, args) =>
|
||||
new Promise((resolve, reject) => {
|
||||
const child = spawn(bin, args, { env: { PATH }, stdio: ['ignore', 'ignore', 'ignore'] });
|
||||
child.on('error', () => reject(new BusError('host-failed')));
|
||||
child.on('close', (code) => (code === 0 ? resolve() : reject(new BusError('host-failed'))));
|
||||
});
|
||||
|
||||
export function hostGrant({ configDir, runRoot, limits = HOST_LIMITS, scan = keyscan, keygen = 'ssh-keygen' }) {
|
||||
const keyRoot = join(configDir, 'secrets', 'hosts');
|
||||
const keyPath = (business, hostId) => join(keyRoot, business, hostId, 'installer');
|
||||
const readKey = (business, hostId) => {
|
||||
const file = keyPath(business, hostId);
|
||||
for (const dir of [keyRoot, dirname(dirname(file)), dirname(file)]) privateDir(dir);
|
||||
let fd;
|
||||
try {
|
||||
fd = openSync(file, constants.O_RDONLY | constants.O_NOFOLLOW);
|
||||
return parseHostKey(readFileSync(fd, 'utf8'));
|
||||
} catch (e) {
|
||||
if (e instanceof BusError) throw e;
|
||||
deny('credential-file');
|
||||
} finally {
|
||||
if (fd !== undefined) closeSync(fd);
|
||||
}
|
||||
};
|
||||
return {
|
||||
enrollCheck,
|
||||
sameFingerprints,
|
||||
fingerprint,
|
||||
installerTag,
|
||||
keyscan: (address, port) => scan(address, port),
|
||||
// E2 steps 2 and 3. Returns the public line and fingerprint; the private half only reaches the file.
|
||||
async enroll(business, hostId) {
|
||||
mkdirSync(runRoot, { recursive: true, mode: 0o700 });
|
||||
privateDir(runRoot);
|
||||
const dir = mkdtempSync(join(runRoot, 'h-'));
|
||||
try {
|
||||
const tag = installerTag(business, hostId);
|
||||
await run(keygen, ['-q', '-t', 'ed25519', '-N', '', '-C', INSTALLER_USER + '-' + tag.slice('mosaic-'.length), '-f', join(dir, 'key')]);
|
||||
const key = parseHostKey(readFileSync(join(dir, 'key'), 'utf8'));
|
||||
const file = keyPath(business, hostId);
|
||||
mkdirSync(keyRoot, { recursive: true, mode: 0o700 });
|
||||
privateDir(keyRoot);
|
||||
for (const d of [dirname(dirname(file)), dirname(file)]) {
|
||||
if (!privateDir(d, { missing: true })) mkdirSync(d, { mode: 0o700 });
|
||||
privateDir(d);
|
||||
}
|
||||
const tmp = file + '.tmp';
|
||||
try {
|
||||
unlinkSync(tmp);
|
||||
} catch (e) {
|
||||
if (e.code !== 'ENOENT') deny('credential-file');
|
||||
}
|
||||
let fd;
|
||||
try {
|
||||
fd = openSync(tmp, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW, 0o600);
|
||||
writeSync(fd, key.text);
|
||||
fsyncSync(fd);
|
||||
} finally {
|
||||
if (fd !== undefined) closeSync(fd);
|
||||
}
|
||||
renameSync(tmp, file);
|
||||
const dirFd = openSync(dirname(file), constants.O_RDONLY);
|
||||
try {
|
||||
fsyncSync(dirFd);
|
||||
} finally {
|
||||
closeSync(dirFd);
|
||||
}
|
||||
return { publicLine: key.publicLine, fingerprint: key.fingerprint, tag };
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
},
|
||||
// The X3 undo and hosts.remove: the installer key and its two dirs, if present.
|
||||
unenroll(business, hostId) {
|
||||
const file = keyPath(business, hostId);
|
||||
for (const f of [file + '.tmp', file]) {
|
||||
try {
|
||||
unlinkSync(f);
|
||||
} catch (e) {
|
||||
if (e.code !== 'ENOENT') deny('credential-file');
|
||||
}
|
||||
}
|
||||
for (const d of [dirname(file), dirname(dirname(file))]) {
|
||||
try {
|
||||
rmdirSync(d);
|
||||
} catch (e) {
|
||||
if (!['ENOENT', 'ENOTEMPTY'].includes(e.code)) deny('credential-file');
|
||||
}
|
||||
}
|
||||
},
|
||||
// E4: `list <user>` as mosaic-installer with the installer key and the pinned host keys. A wrong
|
||||
// pin refuses host-key-mismatch (runHost). Returns the installer fingerprint and the exit.
|
||||
async check(business, hostId, record) {
|
||||
const key = readKey(business, hostId);
|
||||
const entry = { address: record.address, port: record.port, user: INSTALLER_USER, hostKeys: record.hostKeys };
|
||||
const out = await runHost({ root: runRoot, hostId, entry, key, timeout: 30, limits, command: `list ${record.user}` });
|
||||
return { installerFingerprint: key.fingerprint, exit: out.exit };
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -159,9 +159,7 @@ function runSsh(args, stdin, timeoutMs, signal, limits, onSpawn) {
|
||||
|
||||
// D7: one call. `key` is the parsed entry from credentials.mjs (text and publicLine); `entry` is
|
||||
// the registry's { address, port, user, hostKeys }. `root` is <configDir>/secrets/run.
|
||||
// Row 234 (#1740): `command`, when set, is sent as is in place of the sudo argv. The installer key's
|
||||
// forced command reads it (tools/host/mosaic-keys).
|
||||
export async function runHost({ root, hostId, entry, key, argv, stdin, timeout, signal = null, limits = HOST_LIMITS, command = null }) {
|
||||
export async function runHost({ root, hostId, entry, key, argv, stdin, timeout, signal = null, limits = HOST_LIMITS }) {
|
||||
if (signal?.aborted) deny('credential-unavailable');
|
||||
const dir = mkdtempSync(join(runRoot(root), 'h-'));
|
||||
const sock = join(dir, 'a.sock');
|
||||
@@ -180,7 +178,7 @@ export async function runHost({ root, hostId, entry, key, argv, stdin, timeout,
|
||||
await addKey(sock, key.text, timeout + 30, { limitMs: limits.agentWaitMs, signal });
|
||||
if (signal?.aborted) deny('credential-unavailable');
|
||||
const out = await runSsh(
|
||||
[...sshArgs({ hostId, ...entry, dir }), command ?? remoteCommand(argv)],
|
||||
[...sshArgs({ hostId, ...entry, dir }), remoteCommand(argv)],
|
||||
stdin,
|
||||
timeout * 1000,
|
||||
signal,
|
||||
|
||||
@@ -1,35 +0,0 @@
|
||||
-- Row 234 (#436a row a, #1740): enrolled hosts and host grants live in the store, written only by
|
||||
-- the grant executor, hosts.check, hosts.remove and revoke, each in the transaction of its event.
|
||||
-- A grant event is an action.allowed whose body.action or body.operation is grant.*; the trigger
|
||||
-- refuses one unless the actor is (NULL, 'grant'), the broker's own grant writer (#436b section 7).
|
||||
CREATE TABLE hosts (
|
||||
business TEXT NOT NULL,
|
||||
host TEXT NOT NULL,
|
||||
address TEXT NOT NULL,
|
||||
port INTEGER NOT NULL CHECK (port BETWEEN 1 AND 65535),
|
||||
user TEXT NOT NULL,
|
||||
sudo INTEGER NOT NULL CHECK (sudo IN (0,1)),
|
||||
host_keys TEXT NOT NULL CHECK (json_valid(host_keys) AND json_type(host_keys) = 'array' AND json_array_length(host_keys) > 0),
|
||||
installer_fingerprint TEXT,
|
||||
state TEXT NOT NULL CHECK (state IN ('pending-install','ready')),
|
||||
decision TEXT NOT NULL REFERENCES decisions(id),
|
||||
checked_at TEXT,
|
||||
PRIMARY KEY (business, host),
|
||||
CHECK (state = 'pending-install' OR (installer_fingerprint IS NOT NULL AND checked_at IS NOT NULL))
|
||||
) STRICT;
|
||||
CREATE TABLE host_grants (
|
||||
business TEXT NOT NULL,
|
||||
agent TEXT NOT NULL,
|
||||
host TEXT NOT NULL,
|
||||
commands TEXT NOT NULL CHECK (json_valid(commands) AND json_type(commands) = 'array' AND json_array_length(commands) > 0),
|
||||
rotate_by TEXT NOT NULL,
|
||||
tag TEXT NOT NULL CHECK (length(tag) = 39 AND tag GLOB 'mosaic-*' AND NOT substr(tag, 8) GLOB '*[^0-9a-f]*'),
|
||||
decision TEXT NOT NULL REFERENCES decisions(id),
|
||||
PRIMARY KEY (business, agent, host),
|
||||
FOREIGN KEY (business, host) REFERENCES hosts(business, host)
|
||||
) STRICT;
|
||||
CREATE TRIGGER events_grant_actor BEFORE INSERT ON events
|
||||
WHEN NEW.kind = 'action.allowed'
|
||||
AND (json_extract(NEW.body, '$.action') GLOB 'grant.*' OR json_extract(NEW.body, '$.operation') GLOB 'grant.*')
|
||||
AND NOT (NEW.actor_role IS NULL AND NEW.actor_run IS 'grant')
|
||||
BEGIN SELECT RAISE(ABORT, 'only the grant writer records grant events'); END;
|
||||
@@ -16,7 +16,6 @@ import { organizationWriter } from '../../business/src/organization.mjs';
|
||||
import { credentialVerbs } from './credential-verbs.mjs';
|
||||
import { webService } from './web.mjs';
|
||||
import { hostExecutor, sweepRunDir } from './host-run.mjs';
|
||||
import { hostGrant } from './host-grant.mjs';
|
||||
import { verifyHuman } from './human.mjs';
|
||||
import { createUsage } from './usage.mjs';
|
||||
import { createLimits } from './limits.mjs';
|
||||
@@ -96,16 +95,8 @@ export async function startBroker({
|
||||
];
|
||||
// Row 199 (#1697, design D1): the per-agent store is <configDir>/secrets/agents.
|
||||
const storeRoot = organization ? join(organization.dir, 'secrets', 'agents') : null;
|
||||
store = new Store(dataRoot);
|
||||
// Row 234 (#1740, r6 X4 third bullet): each host grant record gives its agent a store reference to
|
||||
// host.<id>, so a restart finds the keys the grant executor wrote. The loader refuses host.<id> in a
|
||||
// role's credentials, so these never meet a business-file reference.
|
||||
for (const g of store.all('SELECT business, agent, host, rotate_by FROM host_grants ORDER BY business, agent, host')) {
|
||||
if (!Object.hasOwn(businesses, g.business)) continue;
|
||||
const refs = (references[g.business + '/' + g.agent] = { ...(references[g.business + '/' + g.agent] ?? {}) });
|
||||
refs['host.' + g.host] = { store: true, file: storeRoot ? join(storeRoot, g.business, g.agent, 'host.' + g.host) : null, rotateBy: g.rotate_by };
|
||||
}
|
||||
credentials = new Credentials({ references, repoRoots: protectedRoots, dataRoot, storeRoot });
|
||||
store = new Store(dataRoot);
|
||||
// vikunja:19/305: the plug-in switch through the same human-only path as the limits: the verbs
|
||||
// plugins and plugins.set, logged in changes.jsonl with the actor, a missing module refused.
|
||||
const plugins=organization?pluginSettings({dir:organization.dir,rolesDir:organization.rolesDir,exists:hasModule}):null;
|
||||
@@ -121,10 +112,7 @@ export async function startBroker({
|
||||
const limits = createLimits({ store, read: id => settings ? settings.current(id) : businesses[id], log: limitLog });
|
||||
for (const id of Object.keys(businesses)) await limits.refresh(id);
|
||||
const withTokens = (id, result) => ({ ...result, tokens: limits.report(id) });
|
||||
// Row 210 (#1707, design D7 step 6) and row 234 (#1740, r6 E2): the run dir and the host enrollment half.
|
||||
const runRoot = organization ? join(organization.dir, 'secrets', 'run') : null;
|
||||
const broker = new Broker({
|
||||
hostGrant: runRoot ? hostGrant({ configDir: organization.dir, runRoot }) : null,
|
||||
usage,
|
||||
limits,
|
||||
store,
|
||||
@@ -193,9 +181,8 @@ export async function startBroker({
|
||||
}
|
||||
const web = await webServices(businesses);
|
||||
// Row 210 (#1707, design D7 step 6): clear what a killed bus child left, before anyone can call.
|
||||
const runRoot = organization ? join(organization.dir, 'secrets', 'run') : null;
|
||||
if (runRoot) sweepRunDir(runRoot);
|
||||
// Row 234 (r6 X3): end the enrollments a killed bus child left, before anyone can call.
|
||||
broker.reconcileGrants();
|
||||
const path = join(store.directory, 'broker.sock');
|
||||
server = await serve({
|
||||
web: (business) => web.get(business) ?? null,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { createServer } from 'node:net';
|
||||
import { lstatSync, chmodSync, unlinkSync } from 'node:fs';
|
||||
import { BusError, TASK_VERBS, WEB_VERBS, HOST_VERBS, HOSTS_VERBS, PROJECT_FILE_VERBS, PROJECT_SAVE_VERB, PROJECT_SAVE_REQUEST_MAX } from './broker.mjs';
|
||||
import { BusError, TASK_VERBS, WEB_VERBS, HOST_VERBS, PROJECT_FILE_VERBS, PROJECT_SAVE_VERB, PROJECT_SAVE_REQUEST_MAX } from './broker.mjs';
|
||||
const LIMIT = 65536;
|
||||
// Row 148: a document save may carry up to 1 MiB of text. Only a request that names that verb near
|
||||
// its start (the proof object comes first) may pass LIMIT; the parsed verb is checked again below.
|
||||
@@ -123,10 +123,7 @@ export async function serve({ broker, path, authenticateHuman = null, authentica
|
||||
// Row138: only project Git effects need an await; preserve every other sync path below.
|
||||
if (r.verb === 'usage.begin' || r.verb === 'usage.end' ||
|
||||
r.verb === 'spec.read' || r.verb === 'record.write' || PROJECT_FILE_VERBS.includes(r.verb) ||
|
||||
// Row 234 (#1740): a grant raise awaits its keyscan, and hosts.check its ssh.
|
||||
HOSTS_VERBS.includes(r.verb) ||
|
||||
(r.verb === 'decision.raise' && (Object.hasOwn(r.args ?? {}, 'spec_path') || Object.hasOwn(r.args ?? {}, 'spec_commit') ||
|
||||
Object.hasOwn(r.args ?? {}, 'grant')))) {
|
||||
(r.verb === 'decision.raise' && (Object.hasOwn(r.args ?? {}, 'spec_path') || Object.hasOwn(r.args ?? {}, 'spec_commit')))) {
|
||||
const own = transient;
|
||||
transient = null;
|
||||
// Local Git may take 100 seconds; keep its result beyond the ordinary idle deadline.
|
||||
|
||||
@@ -13,8 +13,8 @@ import { join, isAbsolute } from 'node:path';
|
||||
import { createHash } from 'node:crypto';
|
||||
|
||||
const ddl = readFileSync(new URL('../schema.sql', import.meta.url), 'utf8');
|
||||
export const SCHEMA_SHA256 = '0a7b633ce7eec9d0bb37c3255f4244d97bd0c8eac38db91a11b4ef6cc982ec8c';
|
||||
export const SCHEMA_VERSION = '3j';
|
||||
export const SCHEMA_SHA256 = '6cde5232c57ac04dc6073112c0830c89cfc5e953d7a5727e7716263747fe8737';
|
||||
export const SCHEMA_VERSION = '3i';
|
||||
// Ordered, named offline steps. New schema changes append a step and keep prior DDL pinned.
|
||||
const legacyDDL = readFileSync(new URL('./migrations/3b.sql', import.meta.url), 'utf8');
|
||||
const LEGACY_SHA256 = '179ffe356d4ff19a49b5ebad39b6c6bfd7771deb8e1b7c55b5e746f039d69e65';
|
||||
@@ -56,11 +56,6 @@ const migrations = [{
|
||||
from: '3h',
|
||||
to: '3i',
|
||||
sql: readFileSync(new URL('./migrations/3h-3i.sql', import.meta.url), 'utf8'),
|
||||
}, {
|
||||
// Row 234 (#1740): enrolled hosts, host grants and the grant event guard. Only new objects.
|
||||
from: '3i',
|
||||
to: '3j',
|
||||
sql: readFileSync(new URL('./migrations/3i-3j.sql', import.meta.url), 'utf8'),
|
||||
}];
|
||||
const timedTables = [
|
||||
'events',
|
||||
|
||||
@@ -69,7 +69,6 @@ async function engine(root, name, { store, schema, step = (s) => s }) {
|
||||
for (const file of ['3b.sql', '3b-3c.sql', '3c-3d.sql', '3d-3e.sql', '3e-3f.sql', '3f-3g.sql', '3g-3h.sql'])
|
||||
copyFileSync(here('../src/migrations/' + file), join(moduleRoot, 'src/migrations', file));
|
||||
writeFileSync(join(moduleRoot, 'src/migrations/3h-3i.sql'), step(readFileSync(here('../src/migrations/3h-3i.sql'), 'utf8')));
|
||||
copyFileSync(here('../src/migrations/3i-3j.sql'), join(moduleRoot, 'src/migrations/3i-3j.sql'));
|
||||
return (await import(pathToFileURL(join(moduleRoot, 'src/store.mjs')))).Store;
|
||||
}
|
||||
|
||||
@@ -77,7 +76,7 @@ test('REQ-LIMIT-4 3h copy migrates to 3i: every row keeps its seq, the sequence
|
||||
const { root, path } = source3h(t), before = snapshot(path), was = master(path);
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
assert.equal(version(path), '3j');
|
||||
assert.equal(version(path), '3i');
|
||||
// Every table's rows as they were, usage_records with its seq values and sqlite_sequence with 50.
|
||||
assert.deepEqual(snapshot(path, before.columns).rows, before.rows);
|
||||
assert.deepEqual(before.rows.usage_records.map(r => r.seq), [1, 2, 3]);
|
||||
@@ -85,9 +84,7 @@ test('REQ-LIMIT-4 3h copy migrates to 3i: every row keeps its seq, the sequence
|
||||
// sqlite_master as at 3h, the account line aside: same tables, indexes and triggers, same text.
|
||||
const old = ' account TEXT CHECK (account IS NULL),\n';
|
||||
assert.equal(was.filter(r => r.sql?.includes(old)).map(r => r.name).join(), 'usage_records');
|
||||
// Row 234 (#1740) adds 3i→3j's own objects after this step. Its test is grant-migration.test.mjs.
|
||||
const later = ['hosts', 'host_grants', 'events_grant_actor', 'sqlite_autoindex_hosts_1', 'sqlite_autoindex_host_grants_1'];
|
||||
assert.deepEqual(master(path).filter(r => !later.includes(r.name)), was.map(r => r.name === 'usage_records' ? { ...r, sql: r.sql.replace(old, ' account TEXT,\n') } : r));
|
||||
assert.deepEqual(master(path), was.map(r => r.name === 'usage_records' ? { ...r, sql: r.sql.replace(old, ' account TEXT,\n') } : r));
|
||||
assert.equal(was.filter(r => r.type === 'trigger' && r.tbl_name === 'usage_records').length, 3);
|
||||
const store = new Store(root);
|
||||
try {
|
||||
@@ -125,11 +122,11 @@ test('REQ-LIMIT-4 a 3h→3i step that fails partway leaves the store at 3h with
|
||||
assert.deepEqual(snapshot(path, before.columns).rows, before.rows);
|
||||
// Still a whole 3h store: the real step migrates it.
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
assert.equal(version(path), '3j');
|
||||
assert.equal(version(path), '3i');
|
||||
assert.deepEqual(snapshot(path, before.columns).rows, before.rows);
|
||||
});
|
||||
|
||||
test('REQ-LIMIT-4 the 3h engine refuses a migrated store, migrating or not, and leaves it as it was', async t => {
|
||||
test('REQ-LIMIT-4 the 3h engine refuses a 3i store, migrating or not, and leaves it at 3i', async t => {
|
||||
const { root, path } = source3h(t);
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
const after = snapshot(path), was = master(path);
|
||||
@@ -137,14 +134,14 @@ test('REQ-LIMIT-4 the 3h engine refuses a migrated store, migrating or not, and
|
||||
schema: ddl3h,
|
||||
store: (source) => source
|
||||
.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3h).digest('hex')}';`)
|
||||
.replace("export const SCHEMA_VERSION = '3j';", "export const SCHEMA_VERSION = '3h';")
|
||||
.replace("export const SCHEMA_VERSION = '3i';", "export const SCHEMA_VERSION = '3h';")
|
||||
.replace(/\}, \{\n {2}\/\/ Row 126 \(#1620\)[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables'),
|
||||
});
|
||||
const pinned = readFileSync(join(root, 'engine-3h/src/store.mjs'), 'utf8');
|
||||
assert.ok(pinned.includes("export const SCHEMA_VERSION = '3h';") && !pinned.includes('3h-3i'), 'the 3h engine has no 3h→3i step');
|
||||
assert.throws(() => new Store3h(root), /schema-mismatch/);
|
||||
assert.throws(() => new Store3h(root, { migrateSchema: true }), /schema-mismatch/);
|
||||
assert.equal(version(path), '3j');
|
||||
assert.equal(version(path), '3i');
|
||||
assert.deepEqual(master(path), was);
|
||||
assert.deepEqual(snapshot(path, after.columns).rows, after.rows);
|
||||
});
|
||||
|
||||
@@ -606,7 +606,7 @@ async function database3c(t) {
|
||||
let source = readFileSync(here('../src/store.mjs'), 'utf8');
|
||||
const pinned = source
|
||||
.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3c).digest('hex')}';`)
|
||||
.replace("export const SCHEMA_VERSION = '3j';", "export const SCHEMA_VERSION = '3c';")
|
||||
.replace("export const SCHEMA_VERSION = '3i';", "export const SCHEMA_VERSION = '3c';")
|
||||
.replace(/\}, \{\n {2}from: '3c',[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables');
|
||||
assert.notEqual(pinned, source);
|
||||
assert.ok(!pinned.includes('3c-3d'), 'the 3c engine has no 3c→3d step');
|
||||
@@ -630,7 +630,7 @@ test('REQ-ROLE-2, REQ-COS-1: the 3c→3d migration lets the chief of staff end a
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
const store = new Store(root, { migrateSchema: true });
|
||||
t.after(() => store.close());
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3j');
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3i');
|
||||
assert.equal(store.get("SELECT body FROM events WHERE id='preserved'").body, '{"fixture":true}');
|
||||
assert.equal(store.get("SELECT holder_run FROM role_claims WHERE role='coder'").holder_run, 'coder-run');
|
||||
const b = new Broker({ store, businesses: demo(), organization: null });
|
||||
|
||||
@@ -33,7 +33,7 @@ test('explicit 3b copy migration preserves evidence and opens as 3g; normal star
|
||||
const original = readFileSync(source);
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
const store = new Store(root, { migrateSchema: true });
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3j');
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3i');
|
||||
assert.equal(store.get("SELECT body FROM events WHERE id='preserved'").body, '{"fixture":true}');
|
||||
assert.equal(store.get('SELECT count(*) n FROM task_comment_cursors').n, 0);
|
||||
const epoch = store.get("SELECT value FROM meta WHERE key='comment_cursor_epoch'").value;
|
||||
@@ -75,15 +75,15 @@ test('named migration registry chains 3b through 3g to a fixture-only future ver
|
||||
const sha = createHash('sha256').update(futureDDL).digest('hex');
|
||||
let source = readFileSync(new URL('../src/store.mjs', import.meta.url), 'utf8');
|
||||
source = source.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${sha}';`)
|
||||
.replace("export const SCHEMA_VERSION = '3j';", "export const SCHEMA_VERSION = '3k';")
|
||||
.replace('}];\nconst timedTables', `}, { from: '3j', to: '3k', sql: ${JSON.stringify(futureSQL)} }];\nconst timedTables`);
|
||||
.replace("export const SCHEMA_VERSION = '3i';", "export const SCHEMA_VERSION = '3j';")
|
||||
.replace('}];\nconst timedTables', `}, { from: '3i', to: '3j', sql: ${JSON.stringify(futureSQL)} }];\nconst timedTables`);
|
||||
writeFileSync(join(moduleRoot, 'src/store.mjs'), source);
|
||||
writeFileSync(join(moduleRoot, 'schema.sql'), futureDDL);
|
||||
for (const name of ['3b.sql', '3b-3c.sql', '3c-3d.sql', '3d-3e.sql', '3e-3f.sql', '3f-3g.sql', '3g-3h.sql', '3h-3i.sql', '3i-3j.sql'])
|
||||
for (const name of ['3b.sql', '3b-3c.sql', '3c-3d.sql', '3d-3e.sql', '3e-3f.sql', '3f-3g.sql', '3g-3h.sql', '3h-3i.sql'])
|
||||
copyFileSync(new URL('../src/migrations/' + name, import.meta.url), join(moduleRoot, 'src/migrations', name));
|
||||
const { Store: FutureStore } = await import(pathToFileURL(join(moduleRoot, 'src/store.mjs')));
|
||||
const future = new FutureStore(root, { migrateSchema: true });
|
||||
assert.equal(future.get("SELECT value FROM meta WHERE key='schema_version'").value, '3k');
|
||||
assert.equal(future.get("SELECT value FROM meta WHERE key='schema_version'").value, '3j');
|
||||
assert.equal(future.get("SELECT name FROM sqlite_master WHERE name='future_fixture'").name, 'future_fixture');
|
||||
assert.equal(future.get("SELECT count(*) n FROM task_comment_cursors").n, 0);
|
||||
assert.equal(future.get("SELECT id FROM events").id, 'preserved');
|
||||
@@ -109,7 +109,7 @@ test('offline command migrates only the supplied fixture root and reports usage
|
||||
const command = new URL('../src/migrate-schema.mjs', import.meta.url).pathname;
|
||||
const migrated = spawnSync(process.execPath, [command, root], { encoding: 'utf8' });
|
||||
assert.equal(migrated.status, 0, migrated.stderr);
|
||||
assert.match(migrated.stdout, /bus schema 3j ready/);
|
||||
assert.match(migrated.stdout, /bus schema 3i ready/);
|
||||
new Store(root).close();
|
||||
const usage = spawnSync(process.execPath, [command], { encoding: 'utf8' });
|
||||
assert.equal(usage.status, 2);
|
||||
|
||||
-365
@@ -1,365 +0,0 @@
|
||||
PRAGMA journal_mode = WAL;
|
||||
PRAGMA foreign_keys = ON;
|
||||
CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL) STRICT;
|
||||
CREATE TABLE events (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
id TEXT NOT NULL UNIQUE,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
kind TEXT NOT NULL CHECK (kind IN (
|
||||
'session.launched',
|
||||
'session.ended',
|
||||
'action.allowed',
|
||||
'action.refused',
|
||||
'task.created',
|
||||
'task.assigned',
|
||||
'task.state',
|
||||
'task.closed',
|
||||
'task.changed.external',
|
||||
'task.conflict',
|
||||
'task.missing',
|
||||
'review.requested',
|
||||
'review.verdict',
|
||||
'human.input',
|
||||
'config.refused',
|
||||
'credential.expiring',
|
||||
'credential.expired',
|
||||
'credential.changed',
|
||||
'launch.revoked',
|
||||
'launch.restored',
|
||||
'digest.sent')),
|
||||
actor_role TEXT, actor_run TEXT,
|
||||
subject TEXT,
|
||||
corrects TEXT REFERENCES events(id),
|
||||
body TEXT NOT NULL CHECK (json_valid(body))
|
||||
) STRICT;
|
||||
CREATE TABLE role_claims (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL, role TEXT NOT NULL,
|
||||
op TEXT NOT NULL CHECK (op IN ('claim','release','revoke')),
|
||||
holder_run TEXT NOT NULL,
|
||||
harness TEXT NOT NULL, address TEXT,
|
||||
by TEXT NOT NULL, reason TEXT,
|
||||
decision TEXT
|
||||
) STRICT;
|
||||
CREATE TABLE decisions (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
id TEXT NOT NULL UNIQUE,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL, project TEXT,
|
||||
raised_by_role TEXT NOT NULL, raised_by_run TEXT NOT NULL,
|
||||
class TEXT NOT NULL CHECK (class IN ('routine','within-role','cross-role','gated')),
|
||||
action TEXT NOT NULL,
|
||||
route_to TEXT NOT NULL,
|
||||
question TEXT NOT NULL,
|
||||
options TEXT NOT NULL CHECK (json_valid(options) AND json_array_length(options) BETWEEN 2 AND 9),
|
||||
recommendation TEXT NOT NULL,
|
||||
task_ref TEXT CHECK (task_ref IS NULL OR (task_ref GLOB 'vikunja:[1-9]*/[1-9]*' AND NOT substr(task_ref, 9) GLOB '*[^0-9/]*' AND NOT substr(task_ref, 9) GLOB '*/*/*')), requirement_ref TEXT,
|
||||
blocking INTEGER NOT NULL CHECK (blocking IN (0,1)),
|
||||
supersedes TEXT REFERENCES decisions(id)
|
||||
, spec_path TEXT, spec_commit TEXT) STRICT;
|
||||
CREATE TABLE decision_events (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
decision TEXT NOT NULL REFERENCES decisions(id),
|
||||
at TEXT NOT NULL,
|
||||
op TEXT NOT NULL CHECK (op IN ('seen','resolved','withdrawn','expired')),
|
||||
by TEXT NOT NULL,
|
||||
choice TEXT, note TEXT, via TEXT
|
||||
) STRICT;
|
||||
CREATE TABLE messages (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
id TEXT NOT NULL UNIQUE,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
from_role TEXT NOT NULL, from_run TEXT NOT NULL,
|
||||
to_role TEXT NOT NULL,
|
||||
class TEXT NOT NULL,
|
||||
in_reply_to TEXT REFERENCES messages(id),
|
||||
decision TEXT REFERENCES decisions(id),
|
||||
corrects TEXT REFERENCES messages(id),
|
||||
body TEXT NOT NULL
|
||||
, to_name TEXT) STRICT;
|
||||
CREATE TABLE deliveries (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
message TEXT NOT NULL REFERENCES messages(id),
|
||||
at TEXT NOT NULL,
|
||||
op TEXT NOT NULL CHECK (op IN ('routed','delivered','failed','read')),
|
||||
holder_run TEXT, transport TEXT, address TEXT, detail TEXT
|
||||
) STRICT;
|
||||
CREATE TABLE task_snapshots (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
task_ref TEXT NOT NULL CHECK (task_ref GLOB 'vikunja:[1-9]*/[1-9]*' AND NOT substr(task_ref, 9) GLOB '*[^0-9/]*' AND NOT substr(task_ref, 9) GLOB '*/*/*'),
|
||||
updated TEXT NOT NULL,
|
||||
etag TEXT,
|
||||
digest TEXT NOT NULL CHECK (length(digest) = 64 AND NOT digest GLOB '*[^0-9a-f]*'),
|
||||
fields TEXT NOT NULL CHECK (json_valid(fields)),
|
||||
source TEXT NOT NULL CHECK (source IN ('self','poll')),
|
||||
via TEXT CHECK (via IN ('board','cursor','task','reconcile')),
|
||||
read_at TEXT,
|
||||
role TEXT, run TEXT,
|
||||
CHECK ((source = 'self') = (role IS NOT NULL AND run IS NOT NULL)),
|
||||
CHECK ((source = 'poll') = (via IS NOT NULL AND read_at IS NOT NULL)),
|
||||
CHECK (json_type(fields, '$.bucket') IS 'integer'
|
||||
OR (source IS 'poll' AND via IS 'task' AND json_type(fields, '$.gone') IS 'text'))
|
||||
) STRICT;
|
||||
CREATE INDEX task_snapshots_ref ON task_snapshots (business, task_ref, seq);
|
||||
CREATE TRIGGER decisions_resolve_once BEFORE INSERT ON decision_events
|
||||
WHEN NEW.op IN ('resolved','withdrawn','expired') AND EXISTS (
|
||||
SELECT 1 FROM decision_events WHERE decision = NEW.decision AND op IN ('resolved','withdrawn','expired'))
|
||||
BEGIN SELECT RAISE(ABORT, 'decision already closed'); END;
|
||||
CREATE TRIGGER decisions_resolved_choice BEFORE INSERT ON decision_events
|
||||
WHEN NEW.op = 'resolved' AND (NEW.choice IS NULL OR NOT EXISTS (
|
||||
SELECT 1 FROM decisions d, json_each(d.options) o WHERE d.id = NEW.decision AND json_extract(o.value,'$.key') = NEW.choice))
|
||||
BEGIN SELECT RAISE(ABORT, 'resolution must name one of the options'); END;
|
||||
CREATE TRIGGER role_one_holder BEFORE INSERT ON role_claims
|
||||
WHEN NEW.op = 'claim' AND (SELECT op FROM role_claims WHERE business = NEW.business AND role = NEW.role ORDER BY seq DESC LIMIT 1) = 'claim'
|
||||
BEGIN SELECT RAISE(ABORT, 'role already held'); END;
|
||||
CREATE TRIGGER role_release_by_holder BEFORE INSERT ON role_claims
|
||||
WHEN NEW.op IN ('release','revoke') AND COALESCE((SELECT op FROM role_claims WHERE business = NEW.business AND role = NEW.role ORDER BY seq DESC LIMIT 1),'') <> 'claim'
|
||||
BEGIN SELECT RAISE(ABORT, 'role is not held'); END;
|
||||
CREATE TRIGGER role_release_same_run BEFORE INSERT ON role_claims
|
||||
WHEN NEW.op = 'release' AND (SELECT holder_run FROM role_claims WHERE business = NEW.business AND role = NEW.role ORDER BY seq DESC LIMIT 1) <> NEW.holder_run
|
||||
BEGIN SELECT RAISE(ABORT, 'only the holder releases; others revoke'); END;
|
||||
CREATE TRIGGER role_revoke_needs_decision BEFORE INSERT ON role_claims
|
||||
WHEN NEW.op = 'revoke' AND NEW.decision IS NULL AND NEW.reason IS NULL
|
||||
BEGIN SELECT RAISE(ABORT, 'revoke needs a resolved decision or a reason'); END;
|
||||
CREATE TRIGGER meta_no_update BEFORE UPDATE ON meta BEGIN SELECT RAISE(ABORT, 'meta is append-only'); END;
|
||||
CREATE TRIGGER meta_no_delete BEFORE DELETE ON meta BEGIN SELECT RAISE(ABORT, 'meta is append-only'); END;
|
||||
CREATE TRIGGER events_no_update BEFORE UPDATE ON events BEGIN SELECT RAISE(ABORT, 'events is append-only'); END;
|
||||
CREATE TRIGGER events_no_delete BEFORE DELETE ON events BEGIN SELECT RAISE(ABORT, 'events is append-only'); END;
|
||||
CREATE TRIGGER role_claims_no_update BEFORE UPDATE ON role_claims BEGIN SELECT RAISE(ABORT, 'role_claims is append-only'); END;
|
||||
CREATE TRIGGER role_claims_no_delete BEFORE DELETE ON role_claims BEGIN SELECT RAISE(ABORT, 'role_claims is append-only'); END;
|
||||
CREATE TRIGGER decisions_no_update BEFORE UPDATE ON decisions BEGIN SELECT RAISE(ABORT, 'decisions is append-only'); END;
|
||||
CREATE TRIGGER decisions_no_delete BEFORE DELETE ON decisions BEGIN SELECT RAISE(ABORT, 'decisions is append-only'); END;
|
||||
CREATE TRIGGER decision_events_no_update BEFORE UPDATE ON decision_events BEGIN SELECT RAISE(ABORT, 'decision_events is append-only'); END;
|
||||
CREATE TRIGGER decision_events_no_delete BEFORE DELETE ON decision_events BEGIN SELECT RAISE(ABORT, 'decision_events is append-only'); END;
|
||||
CREATE TRIGGER messages_no_update BEFORE UPDATE ON messages BEGIN SELECT RAISE(ABORT, 'messages is append-only'); END;
|
||||
CREATE TRIGGER messages_no_delete BEFORE DELETE ON messages BEGIN SELECT RAISE(ABORT, 'messages is append-only'); END;
|
||||
CREATE TRIGGER deliveries_no_update BEFORE UPDATE ON deliveries BEGIN SELECT RAISE(ABORT, 'deliveries is append-only'); END;
|
||||
CREATE TRIGGER deliveries_no_delete BEFORE DELETE ON deliveries BEGIN SELECT RAISE(ABORT, 'deliveries is append-only'); END;
|
||||
CREATE TRIGGER meta_no_replace BEFORE INSERT ON meta WHEN EXISTS (SELECT 1 FROM meta WHERE key = NEW.key) BEGIN SELECT RAISE(ABORT, 'meta is append-only'); END;
|
||||
CREATE TRIGGER events_no_replace BEFORE INSERT ON events WHEN EXISTS (SELECT 1 FROM events WHERE seq = NEW.seq OR id = NEW.id) BEGIN SELECT RAISE(ABORT, 'events is append-only'); END;
|
||||
CREATE TRIGGER role_claims_no_replace BEFORE INSERT ON role_claims WHEN EXISTS (SELECT 1 FROM role_claims WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'role_claims is append-only'); END;
|
||||
CREATE TRIGGER decisions_no_replace BEFORE INSERT ON decisions WHEN EXISTS (SELECT 1 FROM decisions WHERE seq = NEW.seq OR id = NEW.id) BEGIN SELECT RAISE(ABORT, 'decisions is append-only'); END;
|
||||
CREATE TRIGGER decision_events_no_replace BEFORE INSERT ON decision_events WHEN EXISTS (SELECT 1 FROM decision_events WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'decision_events is append-only'); END;
|
||||
CREATE TRIGGER messages_no_replace BEFORE INSERT ON messages WHEN EXISTS (SELECT 1 FROM messages WHERE seq = NEW.seq OR id = NEW.id) BEGIN SELECT RAISE(ABORT, 'messages is append-only'); END;
|
||||
CREATE TRIGGER deliveries_no_replace BEFORE INSERT ON deliveries WHEN EXISTS (SELECT 1 FROM deliveries WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'deliveries is append-only'); END;
|
||||
CREATE TRIGGER task_snapshots_no_update BEFORE UPDATE ON task_snapshots BEGIN SELECT RAISE(ABORT, 'task_snapshots is append-only'); END;
|
||||
CREATE TRIGGER task_snapshots_no_delete BEFORE DELETE ON task_snapshots BEGIN SELECT RAISE(ABORT, 'task_snapshots is append-only'); END;
|
||||
CREATE TRIGGER task_snapshots_no_replace BEFORE INSERT ON task_snapshots WHEN EXISTS (SELECT 1 FROM task_snapshots WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'task_snapshots is append-only'); END;
|
||||
CREATE TRIGGER decisions_blocking_needs_task BEFORE INSERT ON decisions
|
||||
WHEN NEW.blocking = 1 AND NEW.task_ref IS NULL
|
||||
BEGIN SELECT RAISE(ABORT, 'a blocking decision cites the task it blocks'); END;
|
||||
CREATE TRIGGER events_launch_by_human BEFORE INSERT ON events
|
||||
WHEN NEW.kind IN ('launch.revoked','launch.restored') AND (NEW.actor_role IS NOT NULL OR NEW.actor_run IS NOT NULL)
|
||||
BEGIN SELECT RAISE(ABORT, 'only the human revokes or restores launching'); END;
|
||||
CREATE TRIGGER events_credential_body BEFORE INSERT ON events
|
||||
WHEN NEW.kind GLOB 'credential.*' AND (
|
||||
json_extract(NEW.body, '$.service') IS NULL OR json_extract(NEW.body, '$.service') NOT IN ('gitea','vikunja')
|
||||
OR json_extract(NEW.body, '$.instance') IS NULL)
|
||||
BEGIN SELECT RAISE(ABORT, 'credential events name a service and a role instance'); END;
|
||||
CREATE TRIGGER events_task_missing_body BEFORE INSERT ON events
|
||||
WHEN NEW.kind = 'task.missing' AND (
|
||||
json_extract(NEW.body, '$.reason') IS NULL OR json_extract(NEW.body, '$.reason') NOT IN ('moved','not-found','no-access')
|
||||
OR (json_extract(NEW.body, '$.reason') = 'moved' AND json_type(NEW.body, '$.project') IS NOT 'integer'))
|
||||
BEGIN SELECT RAISE(ABORT, 'task.missing carries a reason, and the new project when moved'); END;
|
||||
CREATE TRIGGER events_task_subject BEFORE INSERT ON events
|
||||
WHEN NEW.kind GLOB 'task.*' AND (NEW.subject IS NULL OR NOT (NEW.subject GLOB 'vikunja:[1-9]*/[1-9]*' AND NOT substr(NEW.subject, 9) GLOB '*[^0-9/]*' AND NOT substr(NEW.subject, 9) GLOB '*/*/*'))
|
||||
BEGIN SELECT RAISE(ABORT, 'a task event names its task in subject'); END;
|
||||
CREATE TRIGGER events_task_created_body BEFORE INSERT ON events
|
||||
WHEN NEW.kind = 'task.created' AND (
|
||||
json_type(NEW.body, '$.request') IS NOT 'text'
|
||||
OR NOT EXISTS (SELECT 1 FROM events WHERE id = json_extract(NEW.body, '$.request')
|
||||
AND kind = 'human.input' AND business = NEW.business)
|
||||
OR json_type(NEW.body, '$.requirement') IS NOT 'text'
|
||||
OR NOT json_extract(NEW.body, '$.requirement') GLOB 'REQ-[A-Z]*-[1-9]*'
|
||||
OR json_extract(NEW.body, '$.requirement') GLOB 'REQ-*[^A-Z0-9-]*')
|
||||
BEGIN SELECT RAISE(ABORT, 'task.created cites the human.input that asked for it and a requirement id'); END;
|
||||
CREATE VIEW launch_state AS
|
||||
SELECT business, CASE kind WHEN 'launch.revoked' THEN 'revoked' ELSE 'allowed' END AS state, at
|
||||
FROM events e WHERE kind IN ('launch.revoked','launch.restored')
|
||||
AND seq = (SELECT max(seq) FROM events WHERE business = e.business AND kind IN ('launch.revoked','launch.restored'));
|
||||
CREATE VIEW task_external_changes AS
|
||||
SELECT p.business, p.task_ref, p.seq, p.via, p.updated, p.digest, ls.digest AS self_digest
|
||||
FROM task_snapshots p
|
||||
LEFT JOIN task_snapshots ls ON ls.seq = (SELECT max(seq) FROM task_snapshots
|
||||
WHERE business = p.business AND task_ref = p.task_ref AND source = 'self')
|
||||
WHERE p.source = 'poll'
|
||||
AND p.seq = (SELECT max(seq) FROM task_snapshots WHERE business = p.business AND task_ref = p.task_ref)
|
||||
AND (ls.seq IS NULL OR (p.updated >= ls.updated AND p.read_at > ls.at AND p.digest <> ls.digest));
|
||||
CREATE VIEW task_current AS
|
||||
SELECT s.seq, s.at, s.business, s.task_ref, s.updated, s.etag, s.digest, s.fields, s.source, s.via, s.read_at, s.role, s.run
|
||||
FROM task_snapshots s
|
||||
WHERE s.seq = (SELECT max(c.seq) FROM task_snapshots c
|
||||
WHERE c.business = s.business AND c.task_ref = s.task_ref
|
||||
AND (c.source = 'self'
|
||||
OR c.read_at > coalesce((SELECT ls.at FROM task_snapshots ls
|
||||
WHERE ls.business = c.business AND ls.task_ref = c.task_ref AND ls.source = 'self'
|
||||
ORDER BY ls.seq DESC LIMIT 1), '')));
|
||||
CREATE VIEW tasks_open AS
|
||||
SELECT s.business, s.task_ref, json_extract(s.fields, '$.bucket') AS bucket, s.seq
|
||||
FROM task_current s
|
||||
WHERE json_type(s.fields, '$.gone') IS NULL
|
||||
AND json_extract(s.fields, '$.done') = 0;
|
||||
CREATE VIEW urgent_inbox AS
|
||||
SELECT d.id, d.business, d.task_ref, d.question, d.at
|
||||
FROM decisions d
|
||||
WHERE d.class = 'gated' AND d.blocking = 1
|
||||
AND NOT EXISTS (SELECT 1 FROM decision_events x WHERE x.decision = d.id AND x.op IN ('resolved','withdrawn','expired'));
|
||||
CREATE TABLE task_comment_cursors (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
business TEXT NOT NULL,
|
||||
task_id INTEGER NOT NULL CHECK (task_id > 0),
|
||||
last_comment INTEGER NOT NULL CHECK (last_comment > 0)
|
||||
) STRICT;
|
||||
CREATE INDEX task_comment_cursors_latest ON task_comment_cursors (business, task_id, seq);
|
||||
CREATE TRIGGER task_comment_cursors_no_update BEFORE UPDATE ON task_comment_cursors
|
||||
BEGIN SELECT RAISE(ABORT, 'task_comment_cursors is append-only'); END;
|
||||
CREATE TRIGGER task_comment_cursors_no_delete BEFORE DELETE ON task_comment_cursors
|
||||
BEGIN SELECT RAISE(ABORT, 'task_comment_cursors is append-only'); END;
|
||||
CREATE TRIGGER task_comment_cursors_no_replace BEFORE INSERT ON task_comment_cursors
|
||||
WHEN EXISTS (SELECT 1 FROM task_comment_cursors WHERE seq = NEW.seq)
|
||||
BEGIN SELECT RAISE(ABORT, 'task_comment_cursors is append-only'); END;
|
||||
CREATE TRIGGER task_comment_cursors_forward BEFORE INSERT ON task_comment_cursors
|
||||
WHEN NEW.last_comment <= COALESCE((SELECT last_comment FROM task_comment_cursors
|
||||
WHERE business = NEW.business AND task_id = NEW.task_id ORDER BY seq DESC LIMIT 1), 0)
|
||||
BEGIN SELECT RAISE(ABORT, 'comment cursor must advance'); END;
|
||||
CREATE TABLE projects (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
id TEXT NOT NULL UNIQUE,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
repo TEXT,
|
||||
created_by TEXT NOT NULL,
|
||||
UNIQUE (business, name)
|
||||
) STRICT;
|
||||
CREATE TABLE threads (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
id TEXT NOT NULL UNIQUE,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
project TEXT NOT NULL REFERENCES projects(id),
|
||||
instance TEXT NOT NULL,
|
||||
owner TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
UNIQUE (project, name)
|
||||
) STRICT;
|
||||
CREATE TABLE thread_claims (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
thread TEXT NOT NULL REFERENCES threads(id),
|
||||
op TEXT NOT NULL CHECK (op IN ('claim','release','revoke')),
|
||||
holder_run TEXT NOT NULL,
|
||||
harness TEXT NOT NULL, address TEXT,
|
||||
by TEXT NOT NULL, reason TEXT,
|
||||
decision TEXT
|
||||
) STRICT;
|
||||
CREATE TABLE message_threads (
|
||||
message TEXT PRIMARY KEY REFERENCES messages(id),
|
||||
thread TEXT NOT NULL REFERENCES threads(id)
|
||||
) STRICT;
|
||||
CREATE TABLE thread_opens (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
thread TEXT NOT NULL REFERENCES threads(id),
|
||||
by TEXT NOT NULL
|
||||
) STRICT;
|
||||
CREATE INDEX thread_claims_thread ON thread_claims (thread, seq);
|
||||
CREATE INDEX message_threads_thread ON message_threads (thread);
|
||||
CREATE TRIGGER threads_project_business BEFORE INSERT ON threads
|
||||
WHEN (SELECT business FROM projects WHERE id = NEW.project) IS NOT NEW.business
|
||||
BEGIN SELECT RAISE(ABORT, 'a thread belongs to its project''s business'); END;
|
||||
CREATE TRIGGER thread_one_holder BEFORE INSERT ON thread_claims
|
||||
WHEN NEW.op = 'claim' AND (SELECT op FROM thread_claims WHERE thread = NEW.thread ORDER BY seq DESC LIMIT 1) = 'claim'
|
||||
BEGIN SELECT RAISE(ABORT, 'thread already held'); END;
|
||||
CREATE TRIGGER thread_one_per_run BEFORE INSERT ON thread_claims
|
||||
WHEN NEW.op = 'claim' AND EXISTS (SELECT 1 FROM thread_claims c WHERE c.holder_run = NEW.holder_run AND c.business = NEW.business
|
||||
AND c.seq = (SELECT max(seq) FROM thread_claims WHERE thread = c.thread) AND c.op = 'claim')
|
||||
BEGIN SELECT RAISE(ABORT, 'run already holds a thread'); END;
|
||||
CREATE TRIGGER thread_release_by_holder BEFORE INSERT ON thread_claims
|
||||
WHEN NEW.op IN ('release','revoke') AND COALESCE((SELECT op FROM thread_claims WHERE thread = NEW.thread ORDER BY seq DESC LIMIT 1),'') <> 'claim'
|
||||
BEGIN SELECT RAISE(ABORT, 'thread is not held'); END;
|
||||
CREATE TRIGGER thread_release_same_run BEFORE INSERT ON thread_claims
|
||||
WHEN NEW.op = 'release' AND (SELECT holder_run FROM thread_claims WHERE thread = NEW.thread ORDER BY seq DESC LIMIT 1) <> NEW.holder_run
|
||||
BEGIN SELECT RAISE(ABORT, 'only the holder releases; others revoke'); END;
|
||||
CREATE TRIGGER thread_revoke_needs_decision BEFORE INSERT ON thread_claims
|
||||
WHEN NEW.op = 'revoke' AND NEW.decision IS NULL AND NEW.reason IS NULL
|
||||
BEGIN SELECT RAISE(ABORT, 'revoke needs a resolved decision or a reason'); END;
|
||||
CREATE TRIGGER thread_claims_business BEFORE INSERT ON thread_claims
|
||||
WHEN (SELECT business FROM threads WHERE id = NEW.thread) IS NOT NEW.business
|
||||
BEGIN SELECT RAISE(ABORT, 'a thread claim names the thread''s business'); END;
|
||||
CREATE TRIGGER message_threads_match BEFORE INSERT ON message_threads
|
||||
WHEN NOT EXISTS (SELECT 1 FROM messages m JOIN threads t ON t.id = NEW.thread
|
||||
WHERE m.id = NEW.message AND m.business = t.business AND t.instance IN (m.to_role, m.from_role))
|
||||
BEGIN SELECT RAISE(ABORT, 'a threaded message is to or from the thread''s instance'); END;
|
||||
CREATE TRIGGER delivery_same_thread BEFORE INSERT ON deliveries
|
||||
WHEN NEW.op IN ('routed','delivered') AND (NEW.op = 'delivered' OR NEW.holder_run IS NOT NULL)
|
||||
AND EXISTS (SELECT 1 FROM message_threads mt JOIN messages m ON m.id = mt.message JOIN threads t ON t.id = mt.thread
|
||||
WHERE mt.message = NEW.message AND m.to_role = t.instance
|
||||
AND NOT EXISTS (SELECT 1 FROM thread_claims c WHERE c.seq = (SELECT max(seq) FROM thread_claims WHERE thread = t.id)
|
||||
AND c.op = 'claim' AND c.holder_run IS NEW.holder_run))
|
||||
BEGIN SELECT RAISE(ABORT, 'a threaded message goes only to its thread''s holder'); END;
|
||||
CREATE TRIGGER projects_no_update BEFORE UPDATE ON projects BEGIN SELECT RAISE(ABORT, 'projects is append-only'); END;
|
||||
CREATE TRIGGER projects_no_delete BEFORE DELETE ON projects BEGIN SELECT RAISE(ABORT, 'projects is append-only'); END;
|
||||
CREATE TRIGGER projects_no_replace BEFORE INSERT ON projects WHEN EXISTS (SELECT 1 FROM projects WHERE seq = NEW.seq OR id = NEW.id) BEGIN SELECT RAISE(ABORT, 'projects is append-only'); END;
|
||||
CREATE TRIGGER threads_no_update BEFORE UPDATE ON threads BEGIN SELECT RAISE(ABORT, 'threads is append-only'); END;
|
||||
CREATE TRIGGER threads_no_delete BEFORE DELETE ON threads BEGIN SELECT RAISE(ABORT, 'threads is append-only'); END;
|
||||
CREATE TRIGGER threads_no_replace BEFORE INSERT ON threads WHEN EXISTS (SELECT 1 FROM threads WHERE seq = NEW.seq OR id = NEW.id) BEGIN SELECT RAISE(ABORT, 'threads is append-only'); END;
|
||||
CREATE TRIGGER thread_claims_no_update BEFORE UPDATE ON thread_claims BEGIN SELECT RAISE(ABORT, 'thread_claims is append-only'); END;
|
||||
CREATE TRIGGER thread_claims_no_delete BEFORE DELETE ON thread_claims BEGIN SELECT RAISE(ABORT, 'thread_claims is append-only'); END;
|
||||
CREATE TRIGGER thread_claims_no_replace BEFORE INSERT ON thread_claims WHEN EXISTS (SELECT 1 FROM thread_claims WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'thread_claims is append-only'); END;
|
||||
CREATE TRIGGER message_threads_no_update BEFORE UPDATE ON message_threads BEGIN SELECT RAISE(ABORT, 'message_threads is append-only'); END;
|
||||
CREATE TRIGGER message_threads_no_delete BEFORE DELETE ON message_threads BEGIN SELECT RAISE(ABORT, 'message_threads is append-only'); END;
|
||||
CREATE TRIGGER message_threads_no_replace BEFORE INSERT ON message_threads WHEN EXISTS (SELECT 1 FROM message_threads WHERE message = NEW.message) BEGIN SELECT RAISE(ABORT, 'message_threads is append-only'); END;
|
||||
CREATE TRIGGER thread_opens_no_update BEFORE UPDATE ON thread_opens BEGIN SELECT RAISE(ABORT, 'thread_opens is append-only'); END;
|
||||
CREATE TRIGGER thread_opens_no_delete BEFORE DELETE ON thread_opens BEGIN SELECT RAISE(ABORT, 'thread_opens is append-only'); END;
|
||||
CREATE TRIGGER thread_opens_no_replace BEFORE INSERT ON thread_opens WHEN EXISTS (SELECT 1 FROM thread_opens WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'thread_opens is append-only'); END;
|
||||
CREATE TABLE instance_names (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
instance TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
by TEXT NOT NULL
|
||||
) STRICT;
|
||||
CREATE TRIGGER instance_names_unique BEFORE INSERT ON instance_names
|
||||
WHEN EXISTS (SELECT 1 FROM instance_names n WHERE n.business = NEW.business AND n.name = NEW.name
|
||||
AND n.instance <> NEW.instance
|
||||
AND n.seq = (SELECT max(seq) FROM instance_names WHERE business = n.business AND instance = n.instance))
|
||||
BEGIN SELECT RAISE(ABORT, 'an instance name is unique in its business'); END;
|
||||
CREATE TRIGGER instance_names_no_update BEFORE UPDATE ON instance_names BEGIN SELECT RAISE(ABORT, 'instance_names is append-only'); END;
|
||||
CREATE TRIGGER instance_names_no_delete BEFORE DELETE ON instance_names BEGIN SELECT RAISE(ABORT, 'instance_names is append-only'); END;
|
||||
CREATE TRIGGER instance_names_no_replace BEFORE INSERT ON instance_names WHEN EXISTS (SELECT 1 FROM instance_names WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'instance_names is append-only'); END;
|
||||
-- Row 124 (#1618): immutable source snapshots and turn outcomes, never transcript text.
|
||||
CREATE TABLE usage_records (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
at TEXT NOT NULL,
|
||||
business TEXT NOT NULL,
|
||||
run TEXT NOT NULL,
|
||||
instance TEXT NOT NULL,
|
||||
provider TEXT,
|
||||
model TEXT,
|
||||
account TEXT,
|
||||
turn TEXT NOT NULL,
|
||||
source TEXT NOT NULL,
|
||||
phase TEXT NOT NULL CHECK (phase IN ('begin','end')),
|
||||
status TEXT NOT NULL CHECK (status IN ('known','unknown')),
|
||||
code TEXT,
|
||||
cursor TEXT CHECK (cursor IS NULL OR json_valid(cursor)),
|
||||
input INTEGER,
|
||||
output INTEGER,
|
||||
cache_write INTEGER,
|
||||
cache_read INTEGER,
|
||||
UNIQUE (business,run,turn,source,phase),
|
||||
CHECK ((status='unknown' AND input IS NULL AND output IS NULL AND cache_write IS NULL AND cache_read IS NULL)
|
||||
OR (status='known' AND input >= 0 AND output >= 0 AND cache_write >= 0 AND cache_read >= 0
|
||||
AND input IS NOT NULL AND output IS NOT NULL AND cache_write IS NOT NULL AND cache_read IS NOT NULL))
|
||||
) STRICT;
|
||||
CREATE TRIGGER usage_records_no_update BEFORE UPDATE ON usage_records BEGIN SELECT RAISE(ABORT, 'usage_records is append-only'); END;
|
||||
CREATE TRIGGER usage_records_no_delete BEFORE DELETE ON usage_records BEGIN SELECT RAISE(ABORT, 'usage_records is append-only'); END;
|
||||
CREATE TRIGGER usage_records_no_replace BEFORE INSERT ON usage_records WHEN EXISTS (SELECT 1 FROM usage_records WHERE seq=NEW.seq OR (business=NEW.business AND run=NEW.run AND turn=NEW.turn AND source=NEW.source AND phase=NEW.phase)) BEGIN SELECT RAISE(ABORT, 'usage_records is append-only'); END;
|
||||
@@ -1,489 +0,0 @@
|
||||
// Row 234 (#1740, #436a row a): host enrollment through a credential.mint raise with a `grant` binding
|
||||
// (r6 E1, E2, E4, X1, X3). The keyscan is a fake that returns key lines this test makes, ssh-keygen is
|
||||
// real, and the key files go under a scratch config dir. Nothing connects to any host.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { existsSync, mkdtempSync, readFileSync, readdirSync, rmSync, statSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { Broker, BusError, writeGrantEvent, registrySha256 } from '../src/broker.mjs';
|
||||
import { Store } from '../src/store.mjs';
|
||||
import { hostGrant, fingerprint, grantTag, installerTag } from '../src/host-grant.mjs';
|
||||
import { grantDecision } from './host-fixtures.mjs';
|
||||
|
||||
const options = [{ key: 'yes', text: 'Yes' }, { key: 'no', text: 'No' }];
|
||||
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
|
||||
const keyLine = (dir, name) => {
|
||||
execFileSync('ssh-keygen', ['-q', '-t', 'ed25519', '-N', '', '-C', 'x', '-f', join(dir, name)]);
|
||||
return readFileSync(join(dir, name + '.pub'), 'utf8').trim().split(' ').slice(0, 2).join(' ');
|
||||
};
|
||||
const refusesWith = async (p, code) => assert.rejects(Promise.resolve().then(() => p), (e) => (assert.equal(e.code, code), true));
|
||||
const refusesSync = (fn, code) => assert.throws(fn, (e) => (assert.equal(e.code, code), true));
|
||||
|
||||
function setup(t, { keygen, check, hang = false } = {}) {
|
||||
const root = mkdtempSync(join(tmpdir(), 'g234-'));
|
||||
const store = new Store(root);
|
||||
t.after(() => {
|
||||
store.close();
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
const hostKeys = [keyLine(root, 'hk1'), keyLine(root, 'hk2')];
|
||||
const scans = [];
|
||||
const scan = { lines: hostKeys };
|
||||
const configDir = join(root, 'config');
|
||||
const real = hostGrant({
|
||||
configDir,
|
||||
runRoot: join(configDir, 'secrets', 'run'),
|
||||
scan: async (address, port) => (scans.push([address, port]), scan.lines),
|
||||
...(keygen ? { keygen } : {}),
|
||||
});
|
||||
const checks = [];
|
||||
// `hang`: steps 2 and 3 run, then the call never returns, as when the child dies before step 4.
|
||||
const enroll = hang ? async (...args) => (await real.enroll(...args), new Promise(() => {})) : real.enroll;
|
||||
const hg = { ...real, enroll, check: async (...args) => (checks.push(args), check ? check(...args) : { installerFingerprint: args[2].installer_fingerprint, exit: 0 }) };
|
||||
const business = {
|
||||
id: 'demo',
|
||||
human: 'jason',
|
||||
arbiters: { technical: 'pm', delivery: 'pm' },
|
||||
roles: {
|
||||
chief: { definition: 'chief-of-staff', authority: { withinRole: ['message.send'], crossRole: [] } },
|
||||
pm: { definition: 'pm', authority: { withinRole: ['message.send'], crossRole: [] } },
|
||||
},
|
||||
vars: { 'bus.tester': true },
|
||||
};
|
||||
const make = (h = hg) => new Broker({ store, businesses: { demo: business }, hostGrant: h });
|
||||
const broker = make();
|
||||
const caps = {};
|
||||
const bind = (b) => {
|
||||
for (const role of ['chief', 'pm']) {
|
||||
caps[role] = b.bindLaunch({ business: 'demo', role, run: role + '-run-' + Math.random(), harness: 'pi', address: role });
|
||||
b.request(caps[role], { verb: 'role.claim' });
|
||||
}
|
||||
caps.human = b.bindHuman({ business: 'demo', human: 'jason', via: 'cli', outsideAgent: true });
|
||||
caps.tester = b.bindTester({ business: 'demo' });
|
||||
caps.reader = b.bindReader({ business: 'demo' });
|
||||
};
|
||||
bind(broker);
|
||||
const grant = (over = {}) => ({
|
||||
method: 'host-enroll',
|
||||
host: 'web1',
|
||||
address: '192.0.2.10',
|
||||
port: 22,
|
||||
user: 'jarvis',
|
||||
sudo: true,
|
||||
fingerprints: hostKeys.map(fingerprint),
|
||||
...over,
|
||||
});
|
||||
const call = (who, verb, args = {}, b = broker) => b.request(caps[who], { verb, args });
|
||||
const raise = (over = {}, who = 'chief', extra = {}) =>
|
||||
call(who, 'decision.raise', {
|
||||
action: 'credential.mint',
|
||||
question: 'Enroll web1?',
|
||||
options,
|
||||
recommendation: 'yes',
|
||||
blocking: false,
|
||||
grant: grant(over),
|
||||
...extra,
|
||||
});
|
||||
const events = () =>
|
||||
store
|
||||
.all("SELECT kind, actor_role, actor_run, body FROM events WHERE business='demo' ORDER BY seq")
|
||||
.map((e) => ({ kind: e.kind, actor_role: e.actor_role, actor_run: e.actor_run, ...JSON.parse(e.body) }));
|
||||
const grantEvents = () => events().filter((e) => e.actor_run === 'grant');
|
||||
const settle = async (decision) => {
|
||||
for (let i = 0; i < 250; i++) {
|
||||
if (grantEvents().some((e) => e.grantDecision === decision && ['grant.host.enrolled', 'grant.failed'].includes(e.action))) return;
|
||||
await sleep(20);
|
||||
}
|
||||
throw new Error('enrollment did not end');
|
||||
};
|
||||
const keyFile = (host = 'web1') => join(configDir, 'secrets', 'hosts', 'demo', host, 'installer');
|
||||
const hostRow = (host = 'web1') => store.get("SELECT * FROM hosts WHERE business='demo' AND host=?", host);
|
||||
const messages = () => store.all("SELECT from_role, to_role, class, decision, body FROM messages WHERE business='demo' ORDER BY rowid");
|
||||
return { root, store, broker, make, bind, caps, call, raise, grant, hostKeys, scan, scans, checks, events, grantEvents, settle, keyFile, hostRow, messages, configDir };
|
||||
}
|
||||
const approve = async (f, over) => {
|
||||
const d = await f.raise(over);
|
||||
const r = f.call('human', 'decision.resolve', { id: d.id, choice: 'yes' });
|
||||
await f.settle(d.id);
|
||||
return { d, r };
|
||||
};
|
||||
|
||||
test('G1, A1: only the chief raises a grant, and a refused raise stores nothing and runs no keyscan', async (t) => {
|
||||
const f = setup(t);
|
||||
await refusesWith(f.raise({}, 'pm'), 'grant-chief-only');
|
||||
await refusesWith(f.call('human', 'decision.raise', { action: 'credential.mint', question: 'q', options, recommendation: 'yes', blocking: false, grant: f.grant() }), 'agent-required');
|
||||
await refusesWith(f.call('tester', 'decision.raise', { action: 'credential.mint', question: 'q', options, recommendation: 'yes', blocking: false, grant: f.grant() }), 'tester-refused');
|
||||
assert.equal(f.scans.length, 0);
|
||||
assert.equal(f.store.get('SELECT count(*) AS n FROM decisions').n, 0);
|
||||
assert.deepEqual(f.events().filter((e) => e.kind === 'action.refused').map((e) => e.code), ['grant-chief-only', 'agent-required', 'tester-refused']);
|
||||
});
|
||||
|
||||
test('G1: grant goes only with credential.mint, row a takes only host-enroll, and E1 refuses each bad field', async (t) => {
|
||||
const f = setup(t);
|
||||
await refusesWith(f.raise({}, 'chief', { action: 'deploy' }), 'invalid-request');
|
||||
for (const over of [
|
||||
{ method: 'host' },
|
||||
{ user: 'root' },
|
||||
{ user: 'localadmin' },
|
||||
{ user: 'mosaic-installer' },
|
||||
{ user: 'Bad' },
|
||||
{ sudo: 'yes' },
|
||||
{ port: 0 },
|
||||
{ port: 65536 },
|
||||
{ host: 'Web_1' },
|
||||
{ address: '127.0.0.1' },
|
||||
{ address: 'localhost' },
|
||||
{ fingerprints: [] },
|
||||
{ fingerprints: ['SHA256:short'] },
|
||||
{ fingerprints: [f.grant().fingerprints[0], f.grant().fingerprints[0]] },
|
||||
{ extra: 1 },
|
||||
])
|
||||
await refusesWith(f.raise(over), 'invalid-request');
|
||||
const { method, ...noMethod } = f.grant();
|
||||
await refusesWith(
|
||||
f.call('chief', 'decision.raise', { action: 'credential.mint', question: 'q', options, recommendation: 'yes', blocking: false, grant: noMethod }),
|
||||
'invalid-request',
|
||||
);
|
||||
assert.equal(f.scans.length, 0);
|
||||
assert.equal(f.store.get('SELECT count(*) AS n FROM decisions').n, 0);
|
||||
});
|
||||
|
||||
test('G5, A5, A6: the scanned key set must equal the requested set, or host-key-mismatch and nothing stored', async (t) => {
|
||||
const f = setup(t);
|
||||
const other = keyLine(f.root, 'hk3');
|
||||
for (const lines of [[f.hostKeys[0]], [...f.hostKeys, other], [f.hostKeys[0], other], []]) {
|
||||
f.scan.lines = lines;
|
||||
await refusesWith(f.raise(), 'host-key-mismatch');
|
||||
}
|
||||
f.scan.lines = f.hostKeys;
|
||||
await refusesWith(f.raise({ fingerprints: [fingerprint(f.hostKeys[0])] }), 'host-key-mismatch');
|
||||
assert.deepEqual(f.scans.at(-1), ['192.0.2.10', 22]);
|
||||
assert.equal(f.store.get('SELECT count(*) AS n FROM decisions').n, 0);
|
||||
});
|
||||
|
||||
test('G4 (host-enroll block, M1): the broker appends the keys, fingerprints and sudo line, and the cap counts the block', async (t) => {
|
||||
const f = setup(t);
|
||||
const d = await f.raise();
|
||||
assert.equal(d.route_to ?? f.store.get('SELECT route_to FROM decisions WHERE id=?', d.id).route_to, 'human');
|
||||
const q = f.store.get('SELECT question FROM decisions WHERE id=?', d.id).question;
|
||||
assert.ok(q.startsWith('Enroll web1?\n\nEnroll host web1: user jarvis at 192.0.2.10 port 22.\n'));
|
||||
for (const k of f.hostKeys) assert.ok(q.includes(`\n${fingerprint(k)} ${k}`));
|
||||
assert.ok(q.endsWith('\njarvis has sudo on this host. Approving lets later grants for jarvis be installed, and section 9\'s first limit applies to jarvis.'));
|
||||
const raiseBody = f.events().find((e) => e.operation === 'decision.raise');
|
||||
assert.deepEqual(raiseBody.grant, { ...f.grant(), hostKeys: f.hostKeys });
|
||||
// A question that carries a forged block still gets the broker's block after it.
|
||||
const forged = 'x\n\nEnroll host web1: user other at 192.0.2.99 port 22.\nother has no sudo on this host.';
|
||||
const d2 = await f.raise({ host: 'web2', sudo: false }, 'chief', { question: forged });
|
||||
const q2 = f.store.get('SELECT question FROM decisions WHERE id=?', d2.id).question;
|
||||
assert.ok(q2.startsWith(forged + '\n\nEnroll host web2: user jarvis at 192.0.2.10 port 22.\n'));
|
||||
assert.ok(q2.endsWith('\njarvis has no sudo on this host.'));
|
||||
// The 4096 cap holds with the block appended.
|
||||
await refusesWith(f.raise({ host: 'web3' }, 'chief', { question: 'q'.repeat(4000) }), 'invalid-request');
|
||||
assert.equal(f.store.get("SELECT count(*) AS n FROM decisions").n, 2);
|
||||
});
|
||||
|
||||
test('G6, X1: the human\'s cli yes starts enrollment, which writes the record, the key file and the events in order', async (t) => {
|
||||
const f = setup(t);
|
||||
const { d, r } = await approve(f);
|
||||
assert.deepEqual(r, { id: d.id, op: 'resolved', grant: 'started' });
|
||||
const row = f.hostRow();
|
||||
assert.equal(row.state, 'pending-install');
|
||||
assert.equal(row.decision, d.id);
|
||||
assert.equal(row.user, 'jarvis');
|
||||
assert.equal(row.sudo, 1);
|
||||
assert.deepEqual(JSON.parse(row.host_keys), f.hostKeys);
|
||||
assert.match(row.installer_fingerprint, /^SHA256:/);
|
||||
// E2 step 3: 0600 in 0700 dirs.
|
||||
assert.equal(statSync(f.keyFile()).mode & 0o777, 0o600);
|
||||
for (const dir of [join(f.keyFile(), '..'), join(f.keyFile(), '..', '..')]) assert.equal(statSync(dir).mode & 0o777, 0o700);
|
||||
const g = f.grantEvents();
|
||||
assert.deepEqual(g.map((e) => e.operation ?? e.action), ['grant.host-enroll', 'grant.host.enroll.step', 'grant.host.enrolled']);
|
||||
for (const e of g) assert.deepEqual([e.kind, e.actor_role, e.actor_run], ['action.allowed', null, 'grant']);
|
||||
assert.deepEqual(g[0], { kind: 'action.allowed', actor_role: null, actor_run: 'grant', operation: 'grant.host-enroll', decision: d.id });
|
||||
assert.equal(g[1].grantDecision, d.id);
|
||||
assert.equal(g[1].step, 'registry');
|
||||
const enrolled = g[2];
|
||||
assert.equal('decision' in enrolled, false);
|
||||
assert.equal(enrolled.registrySha256, registrySha256({ address: '192.0.2.10', port: 22, user: 'jarvis', sudo: true, hostKeys: f.hostKeys }));
|
||||
assert.equal(enrolled.installerFingerprint, row.installer_fingerprint);
|
||||
assert.deepEqual([enrolled.user, enrolled.sudo], ['jarvis', true]);
|
||||
// E2 step 4's message to the human and the chief: the installer line, public half only.
|
||||
const tag = installerTag('demo', 'web1');
|
||||
const pub = execFileSync('ssh-keygen', ['-y', '-f', f.keyFile()], { encoding: 'utf8' }).trim().split(' ').slice(0, 2).join(' ');
|
||||
const notes = f.messages().filter((m) => m.from_role === '_bus' && m.decision === d.id);
|
||||
assert.deepEqual(notes.map((m) => [m.to_role, m.class]), [['human', 'RESULT'], ['chief', 'RESULT']]);
|
||||
for (const m of notes)
|
||||
assert.ok(m.body.includes(`\nrestrict,command="/usr/local/sbin/mosaic-keys",from="<from>" ${pub} mosaic-installer-${tag.slice(7)}\n`));
|
||||
// A7: no part of the private key reaches the store.
|
||||
const secret = readFileSync(f.keyFile(), 'utf8').split('\n').filter((l) => l.length >= 24 && !l.startsWith('-----'));
|
||||
const dump = JSON.stringify([f.store.all('SELECT body FROM events'), f.store.all('SELECT body FROM messages'), f.store.all('SELECT * FROM hosts')]);
|
||||
for (const l of secret) assert.equal(dump.includes(l), false);
|
||||
// The tmp dir under the run root is gone.
|
||||
assert.deepEqual(readdirSync(join(f.configDir, 'secrets', 'run')), []);
|
||||
});
|
||||
|
||||
test('A2, A4: a credential.mint without grant, and a no, start nothing', async (t) => {
|
||||
const f = setup(t);
|
||||
const plain = f.call('chief', 'decision.raise', { action: 'credential.mint', question: 'Hand-made key?', options, recommendation: 'yes', blocking: false });
|
||||
assert.deepEqual(f.call('human', 'decision.resolve', { id: plain.id, choice: 'yes' }), { id: plain.id, op: 'resolved' });
|
||||
const d = await f.raise();
|
||||
assert.deepEqual(f.call('human', 'decision.resolve', { id: d.id, choice: 'no' }), { id: d.id, op: 'resolved' });
|
||||
await sleep(200);
|
||||
assert.deepEqual(f.grantEvents(), []);
|
||||
assert.equal(f.hostRow(), undefined);
|
||||
});
|
||||
|
||||
test('A3, X1: a resolve by the tester or an agent starts nothing', async (t) => {
|
||||
const f = setup(t);
|
||||
const d = await f.raise();
|
||||
for (const who of ['tester', 'pm', 'chief']) {
|
||||
try {
|
||||
f.call(who, 'decision.resolve', { id: d.id, choice: 'yes' });
|
||||
} catch {}
|
||||
}
|
||||
await sleep(200);
|
||||
assert.deepEqual(f.grantEvents(), []);
|
||||
assert.equal(f.hostRow(), undefined);
|
||||
});
|
||||
|
||||
test('G6 (duckworth-0081 B3): a withdraw by the chief or the human, and an arbiter resolve, start nothing', async (t) => {
|
||||
const f = setup(t);
|
||||
for (const who of ['chief', 'human']) {
|
||||
const d = await f.raise();
|
||||
assert.deepEqual(f.call(who, 'decision.withdraw', { id: d.id }), { id: d.id, op: 'withdrawn' });
|
||||
}
|
||||
// A grant raise always routes to the human, so the fixture writes an open one routed to the pm
|
||||
// arbiter, which then resolves it yes through the verb.
|
||||
const d = { id: grantDecision(f.store, { route: 'pm', grant: f.grant(), close: null }) };
|
||||
assert.deepEqual(f.call('pm', 'decision.resolve', { id: d.id, choice: 'yes' }), { id: d.id, op: 'resolved' });
|
||||
await sleep(200);
|
||||
assert.deepEqual(f.grantEvents(), []);
|
||||
assert.equal(f.hostRow(), undefined);
|
||||
assert.deepEqual(f.broker.reconcileGrants(), []);
|
||||
assert.equal(await f.broker.runEnrollment('demo', d.id), 'not-authorized');
|
||||
assert.deepEqual(f.grantEvents(), []);
|
||||
});
|
||||
|
||||
test('G10 (duckworth-0081 B1): a second start for a spent decision refuses decision-consumed, and an unresolved or agent-resolved one not-authorized', async (t) => {
|
||||
const f = setup(t);
|
||||
const { d } = await approve(f);
|
||||
const before = { events: f.grantEvents(), row: f.hostRow(), messages: f.messages().length };
|
||||
assert.equal(await f.broker.runEnrollment('demo', d.id), 'decision-consumed');
|
||||
assert.deepEqual(f.grantEvents(), before.events);
|
||||
assert.deepEqual(f.hostRow(), before.row);
|
||||
assert.equal(f.messages().length, before.messages);
|
||||
// Unresolved, then resolved by an agent: written by SQL, since the verb refuses the agent first.
|
||||
const open = await f.raise({ host: 'web2' });
|
||||
assert.equal(await f.broker.runEnrollment('demo', open.id), 'not-authorized');
|
||||
f.store.run('INSERT INTO decision_events(decision,at,op,by,choice,via) VALUES(?,?,?,?,?,?)', open.id, new Date().toISOString(), 'resolved', 'chief', 'yes', 'broker');
|
||||
assert.equal(await f.broker.runEnrollment('demo', open.id), 'not-authorized');
|
||||
// A cli yes by someone other than the human, so the by check alone refuses it (duckworth-0083 N8, A3b).
|
||||
const other = await f.raise({ host: 'web3' });
|
||||
f.store.run('INSERT INTO decision_events(decision,at,op,by,choice,via) VALUES(?,?,?,?,?,?)', other.id, new Date().toISOString(), 'resolved', 'chief', 'yes', 'cli');
|
||||
assert.equal(await f.broker.runEnrollment('demo', other.id), 'not-authorized');
|
||||
assert.deepEqual(f.broker.reconcileGrants(), []);
|
||||
assert.deepEqual(f.grantEvents(), before.events);
|
||||
assert.equal(f.hostRow('web2'), undefined);
|
||||
assert.equal(existsSync(f.keyFile('web2')), false);
|
||||
assert.equal(f.hostRow('web3'), undefined);
|
||||
assert.equal(existsSync(f.keyFile('web3')), false);
|
||||
});
|
||||
|
||||
test('A8, S1: two approved enrollments for one host: the second ends host-enrolled and the pin stays', async (t) => {
|
||||
const f = setup(t);
|
||||
const d1 = await f.raise();
|
||||
const d2 = await f.raise({ user: 'melanie', sudo: false });
|
||||
f.call('human', 'decision.resolve', { id: d1.id, choice: 'yes' });
|
||||
await f.settle(d1.id);
|
||||
const before = f.hostRow();
|
||||
f.call('human', 'decision.resolve', { id: d2.id, choice: 'yes' });
|
||||
await f.settle(d2.id);
|
||||
assert.deepEqual(f.hostRow(), before);
|
||||
const failed = f.grantEvents().find((e) => e.grantDecision === d2.id && e.action === 'grant.failed');
|
||||
assert.equal(failed.code, 'host-enrolled');
|
||||
assert.ok(f.grantEvents().some((e) => e.operation === 'grant.host-enroll' && e.decision === d2.id));
|
||||
assert.equal(f.grantEvents().filter((e) => e.grantDecision === d2.id && e.action === 'grant.host.enroll.step').length, 0);
|
||||
// A raise for an enrolled host refuses before the keyscan.
|
||||
const scans = f.scans.length;
|
||||
await refusesWith(f.raise(), 'host-enrolled');
|
||||
assert.equal(f.scans.length, scans);
|
||||
});
|
||||
|
||||
test('A9: the hosts primary key refuses a second record for (business, host)', (t) => {
|
||||
const f = setup(t);
|
||||
const d = f.call('chief', 'decision.raise', { action: 'credential.mint', question: 'q', options, recommendation: 'yes', blocking: false });
|
||||
const insert = () =>
|
||||
f.store.run(
|
||||
"INSERT INTO hosts(business,host,address,port,user,sudo,host_keys,state,decision) VALUES('demo','web1','192.0.2.1',22,'u',0,'[\"k\"]','pending-install',?)",
|
||||
d.id,
|
||||
);
|
||||
insert();
|
||||
assert.throws(insert, /UNIQUE|PRIMARY/);
|
||||
});
|
||||
|
||||
test('X3: a failure at step 2 deletes the record again and ends grant.failed, with no key file', async (t) => {
|
||||
const f = setup(t, { keygen: '/nonexistent/ssh-keygen' });
|
||||
const { d } = await approve(f);
|
||||
assert.equal(f.hostRow(), undefined);
|
||||
assert.equal(existsSync(f.keyFile()), false);
|
||||
const failed = f.grantEvents().find((e) => e.action === 'grant.failed');
|
||||
assert.deepEqual([failed.grantDecision, failed.host, failed.code], [d.id, 'web1', 'host-failed']);
|
||||
const notes = f.messages().filter((m) => m.from_role === '_bus' && m.decision === d.id);
|
||||
assert.deepEqual(notes.map((m) => m.to_role), ['human', 'chief']);
|
||||
assert.ok(notes[0].body.includes('failed: host-failed'));
|
||||
});
|
||||
|
||||
test('A10, X3: at start a resolved, unspent enrollment is spent and ends interrupted, and never runs late', async (t) => {
|
||||
const f = setup(t);
|
||||
const d = await f.raise();
|
||||
// The resolve commits, and the child dies before step 1: the resolve row alone, written here.
|
||||
f.store.run("INSERT INTO decision_events(decision,at,op,by,choice,via) VALUES(?,?,?,?,?,?)", d.id, new Date().toISOString(), 'resolved', 'jason', 'yes', 'cli');
|
||||
const restarted = f.make();
|
||||
assert.deepEqual(restarted.reconcileGrants(), [d.id]);
|
||||
const g = f.grantEvents();
|
||||
assert.deepEqual(g.map((e) => e.operation ?? e.action), ['grant.host-enroll', 'grant.failed']);
|
||||
assert.deepEqual([g[1].code, g[1].host, g[1].tag], ['interrupted', 'web1', installerTag('demo', 'web1')]);
|
||||
assert.equal(f.hostRow(), undefined);
|
||||
assert.deepEqual(restarted.reconcileGrants(), []);
|
||||
});
|
||||
|
||||
test('X3: at start an enrollment that stopped after its registry step loses its record and key file', async (t) => {
|
||||
const f = setup(t, { hang: true });
|
||||
const d = await f.raise();
|
||||
f.call('human', 'decision.resolve', { id: d.id, choice: 'yes' });
|
||||
for (let i = 0; i < 250 && !existsSync(f.keyFile()); i++) await sleep(20);
|
||||
assert.equal(f.hostRow().state, 'pending-install');
|
||||
assert.ok(existsSync(f.keyFile()));
|
||||
const restarted = f.make();
|
||||
assert.deepEqual(restarted.reconcileGrants(), [d.id]);
|
||||
assert.equal(f.hostRow(), undefined);
|
||||
assert.equal(existsSync(f.keyFile()), false);
|
||||
const failed = f.grantEvents().find((e) => e.grantDecision === d.id && e.action === 'grant.failed');
|
||||
assert.deepEqual([failed.code, failed.host, failed.tag], ['interrupted', 'web1', installerTag('demo', 'web1')]);
|
||||
assert.deepEqual(f.messages().filter((m) => m.decision === d.id && m.from_role === '_bus').map((m) => m.to_role), ['human', 'chief']);
|
||||
assert.deepEqual(restarted.reconcileGrants(), []);
|
||||
});
|
||||
|
||||
test('G24 first sentence, M2, A12: tags are mosaic- and 32 hex, and differ by business, agent and host', () => {
|
||||
const long = 'a'.repeat(30) + '.' + '_'.repeat(2) + 'b'.repeat(31);
|
||||
assert.equal(long.length, 64);
|
||||
const tags = [grantTag('demo', long, 'web1'), grantTag('demo', long + 'x', 'web1'), grantTag('other', long, 'web1'), grantTag('demo', long, 'web2'), installerTag('demo', 'web1'), installerTag('other', 'web1')];
|
||||
for (const t of tags) assert.match(t, /^mosaic-[0-9a-f]{32}$/);
|
||||
assert.equal(new Set(tags).size, tags.length);
|
||||
});
|
||||
|
||||
test('E4: hosts.check sets the record ready and records grant.host.checked; a non-zero list leaves it pending', async (t) => {
|
||||
const exits = [1, 0];
|
||||
const f = setup(t, { check: (b, h, r) => ({ installerFingerprint: r.installer_fingerprint, exit: exits.shift() }) });
|
||||
await approve(f);
|
||||
await refusesWith(f.call('chief', 'hosts.check', { host: 'web1' }), 'host-failed');
|
||||
assert.equal(f.hostRow().state, 'pending-install');
|
||||
await refusesWith(f.call('pm', 'hosts.check', { host: 'web1' }), 'grant-chief-only');
|
||||
await refusesWith(f.call('chief', 'hosts.check', { host: 'nope' }), 'host-unknown');
|
||||
const r = await f.call('chief', 'hosts.check', { host: 'web1' });
|
||||
assert.equal(r.state, 'ready');
|
||||
assert.equal(f.hostRow().state, 'ready');
|
||||
assert.equal(f.hostRow().checked_at, r.checkedAt);
|
||||
const checked = f.grantEvents().find((e) => e.action === 'grant.host.checked');
|
||||
assert.deepEqual([checked.host, checked.by, checked.installerFingerprint], ['web1', 'chief', f.hostRow().installer_fingerprint]);
|
||||
assert.equal(checked.registrySha256, registrySha256({ address: '192.0.2.10', port: 22, user: 'jarvis', sudo: true, hostKeys: f.hostKeys }));
|
||||
const status = await f.call('human', 'hosts.status');
|
||||
assert.deepEqual(status.map((h) => [h.host, h.state, h.hostKeys, h.hostLineLeft]), [['web1', 'ready', f.hostKeys.map(fingerprint), []]]);
|
||||
});
|
||||
|
||||
test('G20 (duckworth-0081 B3): the human\'s hosts.check marks the host ready, and the reader refuses', async (t) => {
|
||||
const f = setup(t);
|
||||
await approve(f);
|
||||
await refusesWith(f.call('reader', 'hosts.check', { host: 'web1' }), 'read-only');
|
||||
const r = await f.call('human', 'hosts.check', { host: 'web1' });
|
||||
assert.equal(r.state, 'ready');
|
||||
assert.equal(f.hostRow().state, 'ready');
|
||||
const checked = f.grantEvents().filter((e) => e.action === 'grant.host.checked');
|
||||
assert.deepEqual(checked.map((e) => e.by), ['jason']);
|
||||
});
|
||||
|
||||
test('G18 (duckworth-0081 B2), broker half: a check refused host-key-mismatch leaves the host pending-install with no grant.host.checked', async (t) => {
|
||||
const f = setup(t, { check: () => { throw new BusError('host-key-mismatch'); } });
|
||||
await approve(f);
|
||||
await refusesWith(f.call('chief', 'hosts.check', { host: 'web1' }), 'host-key-mismatch');
|
||||
assert.equal(f.checks.length, 1);
|
||||
assert.equal(f.hostRow().state, 'pending-install');
|
||||
assert.equal(f.hostRow().checked_at, null);
|
||||
assert.deepEqual(f.grantEvents().filter((e) => e.action === 'grant.host.checked'), []);
|
||||
});
|
||||
|
||||
test('E4: a check whose installer fingerprint differs from the record refuses host-changed', async (t) => {
|
||||
const f = setup(t, { check: () => ({ installerFingerprint: 'SHA256:' + 'A'.repeat(43), exit: 0 }) });
|
||||
await approve(f);
|
||||
await refusesWith(f.call('tester', 'hosts.check', { host: 'web1' }), 'host-changed');
|
||||
assert.equal(f.hostRow().state, 'pending-install');
|
||||
});
|
||||
|
||||
test('E4, G21: hosts.remove is the human\'s or the tester\'s, deletes the key and the record, and records grant.host.removed', async (t) => {
|
||||
const f = setup(t);
|
||||
await approve(f);
|
||||
await refusesWith(f.call('chief', 'hosts.remove', { host: 'web1' }), 'human-required');
|
||||
await refusesWith(f.call('reader', 'hosts.remove', { host: 'web1' }), 'read-only');
|
||||
assert.deepEqual(await f.call('human', 'hosts.remove', { host: 'web1' }), { host: 'web1', removed: true });
|
||||
assert.equal(f.hostRow(), undefined);
|
||||
assert.equal(existsSync(f.keyFile()), false);
|
||||
assert.equal(existsSync(join(f.keyFile(), '..')), false);
|
||||
const removed = f.grantEvents().find((e) => e.action === 'grant.host.removed');
|
||||
assert.deepEqual([removed.host, removed.by], ['web1', 'jason']);
|
||||
await refusesWith(f.call('tester', 'hosts.remove', { host: 'web1' }), 'host-unknown');
|
||||
});
|
||||
|
||||
test('G21, F1, F2, M4, A22: hosts.remove refuses host-line-left on a guarded hostLineLeft, and ignores look-alikes', async (t) => {
|
||||
const f = setup(t);
|
||||
const { d } = await approve(f);
|
||||
const at = new Date().toISOString();
|
||||
// Look-alikes: the same body under action.refused, and under another actor of another kind.
|
||||
const lookAlike = JSON.stringify({ action: 'grant.failed', grantDecision: d.id, host: 'web1', hostLineLeft: installerTag('demo', 'web1') });
|
||||
f.store.run("INSERT INTO events(id,at,business,kind,actor_role,actor_run,subject,body) VALUES('la1',?,'demo','action.refused',NULL,'grant','web1',?)", at, lookAlike);
|
||||
f.store.run("INSERT INTO events(id,at,business,kind,actor_role,actor_run,subject,body) VALUES('la2',?,'demo','action.refused','chief','chief-run','web1',?)", at, lookAlike);
|
||||
// The trigger refuses the look-alike as an action.allowed from a session.
|
||||
assert.throws(() =>
|
||||
f.store.run("INSERT INTO events(id,at,business,kind,actor_role,actor_run,subject,body) VALUES('la3',?,'demo','action.allowed','chief','chief-run','web1',?)", at, lookAlike),
|
||||
);
|
||||
assert.equal((await f.call('human', 'hosts.status'))[0].hostLineLeft.length, 0);
|
||||
// F1: the real shape, through the grant writer. F2: any guarded body, not only grant.failed.
|
||||
writeGrantEvent(f.store, at, 'demo', { action: 'grant.revoke.closed', tag: 'mosaic-' + '0'.repeat(32), host: 'web1', hostLineLeft: 'mosaic-' + '0'.repeat(32) }, 'web1');
|
||||
assert.deepEqual((await f.call('human', 'hosts.status'))[0].hostLineLeft, ['mosaic-' + '0'.repeat(32)]);
|
||||
await refusesWith(f.call('human', 'hosts.remove', { host: 'web1' }), 'host-line-left');
|
||||
assert.ok(f.hostRow());
|
||||
assert.ok(existsSync(f.keyFile()));
|
||||
});
|
||||
|
||||
test('hosts.remove refuses host-in-use while a grant record names the host', async (t) => {
|
||||
const f = setup(t);
|
||||
const { d } = await approve(f);
|
||||
f.store.run(
|
||||
"INSERT INTO host_grants(business,agent,host,commands,rotate_by,tag,decision) VALUES('demo','pm','web1','[[\"true\"]]','2099-01-01',?,?)",
|
||||
grantTag('demo', 'pm', 'web1'),
|
||||
d.id,
|
||||
);
|
||||
await refusesWith(f.call('human', 'hosts.remove', { host: 'web1' }), 'host-in-use');
|
||||
assert.ok(f.hostRow());
|
||||
});
|
||||
|
||||
test('the grant writer refuses a body outside section 7\'s shape', (t) => {
|
||||
const f = setup(t);
|
||||
const at = new Date().toISOString();
|
||||
for (const body of [
|
||||
{ action: 'host.sudo' },
|
||||
{ action: 'grant.failed', decision: 'd' },
|
||||
{ operation: 'grant.host-enroll', decision: 'd', host: 'web1' },
|
||||
{ operation: 'grant.host-enroll' },
|
||||
{ operation: 'decision.raise', decision: 'd' },
|
||||
])
|
||||
assert.throws(() => writeGrantEvent(f.store, at, 'demo', body), /grant event shape/);
|
||||
});
|
||||
|
||||
test('recordEvent refuses reserved-event for a grant.* action or operation', (t) => {
|
||||
const f = setup(t);
|
||||
for (const body of [{ action: 'grant.failed' }, { operation: 'grant.host-enroll', decision: 'x' }])
|
||||
refusesSync(() => f.broker.recordEvent(f.caps.chief, { kind: 'action.allowed', body }), 'reserved-event');
|
||||
assert.equal(f.store.get("SELECT count(*) AS n FROM events WHERE json_extract(body,'$.action') GLOB 'grant.*'").n, 0);
|
||||
});
|
||||
@@ -1,212 +0,0 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { chmodSync, copyFileSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { pathToFileURL } from 'node:url';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { DatabaseSync } from 'node:sqlite';
|
||||
import { Store, schemaDigest } from '../src/store.mjs';
|
||||
import { Broker } from '../src/broker.mjs';
|
||||
// Row 234 (#1740, #436a row a): the 3i→3j step adds the hosts and host_grants tables and the grant
|
||||
// event guard (#436b section 7 at fa5658941). It runs on a 3i store built here from the pinned 3i
|
||||
// schema (8fede5fc6's schema.sql) and fixture rows. No live store is read or copied.
|
||||
const AT = '2026-10-21T00:00:00.000Z';
|
||||
const TAG = 'mosaic-' + 'a'.repeat(32);
|
||||
const ddl3i = readFileSync(new URL('./fixtures/schema-3i.sql', import.meta.url), 'utf8');
|
||||
const here = (p) => new URL(p, import.meta.url);
|
||||
const NEW = ['hosts', 'host_grants', 'events_grant_actor', 'sqlite_autoindex_hosts_1', 'sqlite_autoindex_host_grants_1'];
|
||||
function source3i(t) {
|
||||
const root = mkdtempSync(join(tmpdir(), 'grant-migrate-'));
|
||||
t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
mkdirSync(join(root, 'bus'), { mode: 0o700 });
|
||||
const path = join(root, 'bus/bus.sqlite'), db = new DatabaseSync(path);
|
||||
assert.equal(createHash('sha256').update(ddl3i).digest('hex'), '6cde5232c57ac04dc6073112c0830c89cfc5e953d7a5727e7716263747fe8737');
|
||||
db.exec(ddl3i);
|
||||
const run = (sql, ...args) => db.prepare(sql).run(...args);
|
||||
run('INSERT INTO meta VALUES (?,?)', 'schema_version', '3i');
|
||||
run('INSERT INTO meta VALUES (?,?)', 'schema_digest', schemaDigest(db));
|
||||
run('INSERT INTO meta VALUES (?,?)', 'comment_cursor_epoch', AT);
|
||||
run('INSERT INTO events(id,at,business,kind,actor_role,actor_run,body) VALUES(?,?,?,?,?,?,?)', 'e1', AT, 'demo', 'action.allowed', 'coder', 'run1', '{"action":"host.sudo"}');
|
||||
run('INSERT INTO events(id,at,business,kind,body) VALUES(?,?,?,?,?)', 'e2', AT, 'demo', 'action.allowed', '{}');
|
||||
db.close(); chmodSync(path, 0o600);
|
||||
return { root, path };
|
||||
}
|
||||
function rows(path) {
|
||||
const db = new DatabaseSync(path, { readOnly: true });
|
||||
try {
|
||||
return Object.fromEntries(db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name NOT IN ('hosts','host_grants')").all()
|
||||
.map(({ name }) => [name, db.prepare(`SELECT * FROM ${name} ORDER BY rowid`).all()
|
||||
.filter(r => name !== 'meta' || !['schema_version', 'schema_digest'].includes(r.key)).map(r => ({ ...r }))]));
|
||||
} finally { db.close(); }
|
||||
}
|
||||
function master(path) {
|
||||
const db = new DatabaseSync(path, { readOnly: true });
|
||||
try {
|
||||
return db.prepare('SELECT type,name,tbl_name,sql FROM sqlite_master ORDER BY type,name').all().map(r => ({ ...r }));
|
||||
} finally { db.close(); }
|
||||
}
|
||||
const version = (path) => {
|
||||
const db = new DatabaseSync(path, { readOnly: true });
|
||||
try { return db.prepare("SELECT value FROM meta WHERE key='schema_version'").get().value; } finally { db.close(); }
|
||||
};
|
||||
// A copy of the bus module whose store.mjs, schema.sql and 3i→3j step the caller edits.
|
||||
async function engine(root, name, { store, schema, step = (s) => s }) {
|
||||
const moduleRoot = join(root, name);
|
||||
mkdirSync(join(moduleRoot, 'src/migrations'), { recursive: true });
|
||||
const source = readFileSync(here('../src/store.mjs'), 'utf8'), edited = store ? store(source) : source;
|
||||
if (store) assert.notEqual(edited, source);
|
||||
writeFileSync(join(moduleRoot, 'src/store.mjs'), edited);
|
||||
writeFileSync(join(moduleRoot, 'schema.sql'), schema ?? readFileSync(here('../schema.sql'), 'utf8'));
|
||||
for (const file of ['3b.sql', '3b-3c.sql', '3c-3d.sql', '3d-3e.sql', '3e-3f.sql', '3f-3g.sql', '3g-3h.sql', '3h-3i.sql'])
|
||||
copyFileSync(here('../src/migrations/' + file), join(moduleRoot, 'src/migrations', file));
|
||||
writeFileSync(join(moduleRoot, 'src/migrations/3i-3j.sql'), step(readFileSync(here('../src/migrations/3i-3j.sql'), 'utf8')));
|
||||
return (await import(pathToFileURL(join(moduleRoot, 'src/store.mjs')))).Store;
|
||||
}
|
||||
function fresh(t) {
|
||||
const root = mkdtempSync(join(tmpdir(), 'grant-store-'));
|
||||
const store = new Store(root);
|
||||
t.after(() => { store.close(); rmSync(root, { recursive: true, force: true }); });
|
||||
store.run(`INSERT INTO decisions(id,at,business,raised_by_role,raised_by_run,class,action,route_to,question,options,recommendation,blocking)
|
||||
VALUES('d1',?,'demo','chief','run1','gated','credential.mint','human','q','["yes","no"]','yes',0)`, AT);
|
||||
return store;
|
||||
}
|
||||
const event = (store, actor, body, kind = 'action.allowed') => store.run(
|
||||
'INSERT INTO events(id,at,business,kind,actor_role,actor_run,body) VALUES(?,?,?,?,?,?,?)',
|
||||
'e' + Math.random().toString(16).slice(2), AT, 'demo', kind, actor[0], actor[1], JSON.stringify(body));
|
||||
const host = (store, values = {}) => store.run(
|
||||
`INSERT INTO hosts(business,host,address,port,user,sudo,host_keys,installer_fingerprint,state,decision,checked_at)
|
||||
VALUES(:business,:host,:address,:port,:user,:sudo,:host_keys,:fp,:state,:decision,:checked)`,
|
||||
{ business: 'demo', host: 'web1', address: '192.0.2.10', port: 22, user: 'deploy', sudo: 0,
|
||||
host_keys: '["ssh-ed25519 AAAA"]', fp: null, state: 'pending-install', decision: 'd1', checked: null, ...values });
|
||||
|
||||
test('3i copy migrates to 3j: every row kept, and sqlite_master gains only the two tables, their keys and the guard', t => {
|
||||
const { root, path } = source3i(t), before = rows(path), was = master(path);
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
assert.equal(version(path), '3j');
|
||||
assert.deepEqual(rows(path), before);
|
||||
const now = master(path);
|
||||
assert.deepEqual(now.filter(r => !NEW.includes(r.name)), was);
|
||||
assert.deepEqual(now.filter(r => NEW.includes(r.name)).map(r => r.name).sort(), [...NEW].sort());
|
||||
const store = new Store(root);
|
||||
try {
|
||||
assert.equal(store.get('PRAGMA quick_check').quick_check, 'ok');
|
||||
assert.deepEqual(store.all('PRAGMA foreign_key_check'), []);
|
||||
assert.equal(store.get('SELECT count(*) AS n FROM hosts').n, 0);
|
||||
// An event row 210 wrote before the step stays, and a session can still write host.sudo.
|
||||
event(store, ['coder', 'run2'], { action: 'host.sudo', phase: 'requested' });
|
||||
} finally { store.close(); }
|
||||
});
|
||||
|
||||
test('a 3i→3j step that fails partway leaves the store at 3i with every row and the schema intact', async t => {
|
||||
const { root, path } = source3i(t), before = rows(path), was = master(path);
|
||||
// The real step plus a statement that fails only when events has a row, so the empty replay at
|
||||
// import passes and the store's run fails after both tables and the trigger exist.
|
||||
const Failing = await engine(root, 'engine-failing', { step: (sql) => sql + 'INSERT INTO events SELECT * FROM events LIMIT 1;\n' });
|
||||
assert.throws(() => new Failing(root, { migrateSchema: true }), /events is append-only/);
|
||||
assert.equal(version(path), '3i');
|
||||
assert.deepEqual(master(path), was);
|
||||
assert.deepEqual(rows(path), before);
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
assert.equal(version(path), '3j');
|
||||
});
|
||||
|
||||
test('the 3i engine refuses a 3j store, migrating or not, and leaves it at 3j', async t => {
|
||||
const { root, path } = source3i(t);
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
const after = rows(path), was = master(path);
|
||||
const Store3i = await engine(root, 'engine-3i', {
|
||||
schema: ddl3i,
|
||||
store: (source) => source
|
||||
.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3i).digest('hex')}';`)
|
||||
.replace("export const SCHEMA_VERSION = '3j';", "export const SCHEMA_VERSION = '3i';")
|
||||
.replace(/\}, \{\n {2}\/\/ Row 234 \(#1740\)[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables'),
|
||||
});
|
||||
const pinned = readFileSync(join(root, 'engine-3i/src/store.mjs'), 'utf8');
|
||||
assert.ok(pinned.includes("export const SCHEMA_VERSION = '3i';") && !pinned.includes('3i-3j'), 'the 3i engine has no 3i→3j step');
|
||||
assert.throws(() => new Store3i(root), /schema-mismatch/);
|
||||
assert.throws(() => new Store3i(root, { migrateSchema: true }), /schema-mismatch/);
|
||||
assert.equal(version(path), '3j');
|
||||
assert.deepEqual(master(path), was);
|
||||
assert.deepEqual(rows(path), after);
|
||||
});
|
||||
|
||||
test('A22b, sage-1703: a session-written grant.* row from before the step survives it, and the grant readers ignore it', async t => {
|
||||
// The step adds the guard but leaves old rows. At 3i recordEvent had no reserved-event refusal, so a
|
||||
// session could write these, and after the step only GRANT_EVIDENCE's actor filter keeps them out.
|
||||
const { root, path } = source3i(t);
|
||||
const db = new DatabaseSync(path);
|
||||
const run = (sql, ...args) => db.prepare(sql).run(...args);
|
||||
run(`INSERT INTO decisions(id,at,business,raised_by_role,raised_by_run,class,action,route_to,question,options,recommendation,blocking)
|
||||
VALUES('d1',?,'demo','chief','run1','gated','credential.mint','human','q','["yes","no"]','yes',0)`, AT);
|
||||
const legacy = [{ action: 'grant.host.removed', host: 'web1', hostLineLeft: TAG }, { operation: 'grant.host-enroll', decision: 'd1' }];
|
||||
for (const [i, body] of legacy.entries())
|
||||
run('INSERT INTO events(id,at,business,kind,actor_role,actor_run,body) VALUES(?,?,?,?,?,?,?)', 'old' + i, AT, 'demo', 'action.allowed', 'chief', 'run1', JSON.stringify(body));
|
||||
db.close();
|
||||
new Store(root, { migrateSchema: true }).close();
|
||||
const store = new Store(root);
|
||||
t.after(() => store.close());
|
||||
assert.equal(store.get("SELECT count(*) AS n FROM events WHERE id IN ('old0','old1')").n, 2);
|
||||
host(store);
|
||||
const business = {
|
||||
id: 'demo',
|
||||
human: 'jason',
|
||||
arbiters: { technical: 'pm', delivery: 'pm' },
|
||||
roles: {
|
||||
chief: { definition: 'chief-of-staff', authority: { withinRole: ['message.send'], crossRole: [] } },
|
||||
pm: { definition: 'pm', authority: { withinRole: ['message.send'], crossRole: [] } },
|
||||
},
|
||||
vars: {},
|
||||
};
|
||||
const unenrolled = [];
|
||||
const broker = new Broker({ store, businesses: { demo: business }, hostGrant: { unenroll: (b, h) => unenrolled.push([b, h]), installerTag: () => null } });
|
||||
// The enrollment look-alike is not a spent decision, so nothing ends interrupted.
|
||||
assert.deepEqual(broker.reconcileGrants(), []);
|
||||
assert.equal(store.get("SELECT count(*) AS n FROM events WHERE actor_run='grant'").n, 0);
|
||||
// The hostLineLeft look-alike is not a line left on the host.
|
||||
const human = broker.bindHuman({ business: 'demo', human: 'jason', via: 'cli', outsideAgent: true });
|
||||
assert.deepEqual(await broker.request(human, { verb: 'hosts.remove', args: { host: 'web1' } }), { host: 'web1', removed: true });
|
||||
assert.deepEqual(unenrolled, [['demo', 'web1']]);
|
||||
});
|
||||
|
||||
test('the guard: a grant.* action or operation on action.allowed needs the (NULL, grant) actor', t => {
|
||||
const store = fresh(t);
|
||||
const refused = /only the grant writer records grant events/;
|
||||
for (const body of [{ action: 'grant.step' }, { action: 'grant.host.enrolled' }, { operation: 'grant.host-enroll', decision: 'd1' }]) {
|
||||
assert.throws(() => event(store, ['chief', 'run1'], body), refused, JSON.stringify(body));
|
||||
assert.throws(() => event(store, ['tester', 'run1'], body), refused);
|
||||
assert.throws(() => event(store, [null, null], body), refused, 'the human and session-less pair is not the grant actor');
|
||||
assert.throws(() => event(store, [null, 'run1'], body), refused);
|
||||
assert.throws(() => event(store, ['grant', 'grant'], body), refused);
|
||||
event(store, [null, 'grant'], body);
|
||||
}
|
||||
// Row 210's host.sudo and host.run stay outside the namespace, and so does any other kind.
|
||||
event(store, ['coder', 'run1'], { action: 'host.sudo', phase: 'completed' });
|
||||
event(store, ['coder', 'run1'], { action: 'grant.step' }, 'action.refused');
|
||||
event(store, ['coder', 'run1'], { action: 'granted.step', operation: 'grantx' });
|
||||
assert.equal(store.get("SELECT count(*) AS n FROM events WHERE actor_run='grant'").n, 3);
|
||||
});
|
||||
|
||||
test('hosts: one record per business and host, a ready host has its installer and check, grants need the host', t => {
|
||||
const store = fresh(t);
|
||||
host(store);
|
||||
assert.throws(() => host(store, { address: '192.0.2.11' }), /UNIQUE constraint failed: hosts\.business, hosts\.host/);
|
||||
host(store, { business: 'other' });
|
||||
assert.throws(() => host(store, { host: 'web2', state: 'ready' }), /CHECK constraint failed/);
|
||||
assert.throws(() => host(store, { host: 'web2', state: 'ready', fp: 'SHA256:x' }), /CHECK constraint failed/);
|
||||
host(store, { host: 'web2', state: 'ready', fp: 'SHA256:x', checked: AT });
|
||||
assert.throws(() => host(store, { host: 'web3', state: 'live' }), /CHECK constraint failed/);
|
||||
assert.throws(() => host(store, { host: 'web3', host_keys: '[]' }), /CHECK constraint failed/);
|
||||
assert.throws(() => host(store, { host: 'web3', decision: 'nope' }), /FOREIGN KEY constraint failed/);
|
||||
const grant = (values = {}) => store.run(
|
||||
'INSERT INTO host_grants(business,agent,host,commands,rotate_by,tag,decision) VALUES(:business,:agent,:host,:commands,:rotate,:tag,:decision)',
|
||||
{ business: 'demo', agent: 'coder', host: 'web2', commands: '[["true"]]', rotate: '2026-12-01', tag: TAG, decision: 'd1', ...values });
|
||||
assert.throws(() => grant({ host: 'web9' }), /FOREIGN KEY constraint failed/);
|
||||
for (const tag of ['mosaic-' + 'A'.repeat(32), 'mosaic-' + 'a'.repeat(31), 'mosaix-' + 'a'.repeat(32), 'mosaic-' + 'a'.repeat(31) + 'g'])
|
||||
assert.throws(() => grant({ tag }), /CHECK constraint failed/, tag);
|
||||
assert.throws(() => grant({ commands: '[]' }), /CHECK constraint failed/);
|
||||
grant();
|
||||
assert.throws(() => grant({ tag: 'mosaic-' + 'b'.repeat(32) }), /UNIQUE constraint failed/);
|
||||
assert.throws(() => store.run("DELETE FROM hosts WHERE business='demo' AND host='web2'"), /FOREIGN KEY constraint failed/);
|
||||
});
|
||||
@@ -7,7 +7,6 @@ import { Store } from '../src/store.mjs';
|
||||
import { Broker, ACTIONS, HARD_WAITS } from '../src/broker.mjs';
|
||||
import { ACTIONS as VOCABULARY, GATED_ONLY } from '../../business/src/vocabulary.mjs';
|
||||
import { fixture } from './spec-fixtures.mjs';
|
||||
import { hostGrantRecord, hostRecord } from './host-fixtures.mjs';
|
||||
|
||||
// PRD 1.5 REQ-WAIT-1, REQ-WAIT-2 and REQ-DEC-2 (row 139, #1633).
|
||||
const WAITS = ['spend', 'data.delete', 'network.expose', 'account.create', 'credential.mint', 'message.external',
|
||||
@@ -38,11 +37,10 @@ test('REQ-WAIT-2: a business granting a hard wait at either class is refused; th
|
||||
|
||||
test('REQ-DEC-2: every hard wait goes to the human, cited or not, and the arbiter cannot resolve it', async t => {
|
||||
const f = await fixture(t, { roles: { coder: { definition: 'coder',
|
||||
authority: { withinRole: ['message.send', 'record.write', 'git.push.working'], crossRole: [] } } } });
|
||||
// Row 210: a host.sudo raise names a registered host the raiser is granted and an argv its grant allows.
|
||||
// Row 234 (#1740): both are store records.
|
||||
hostRecord(f.store, { host: 'web1', entry: { address: '10.0.0.5', port: 22, user: 'admin', hostKeys: ['ssh-ed25519 AAAA'] } });
|
||||
hostGrantRecord(f.store, { agent: 'coder', host: 'web1', commands: [['systemctl', 'status']] });
|
||||
authority: { withinRole: ['message.send', 'record.write', 'git.push.working'], crossRole: [] },
|
||||
hosts: { web1: { commands: [['systemctl', 'status']] } } } },
|
||||
// Row 210: a host.sudo raise names a registered host the raiser is granted and an argv its grant allows.
|
||||
hosts: { web1: { address: '10.0.0.5', port: 22, user: 'admin', hostKeys: ['ssh-ed25519 AAAA'] } } });
|
||||
// Row 199: a git.push.remote raise binds the task's latest push, so the task has one.
|
||||
f.broker.recordPush(f.coder, { task_ref: 'vikunja:1/2', project: 'stack', branch: 'feat', ref: 'work/vikunja-stack-2',
|
||||
commit: 'a'.repeat(40), previous: null });
|
||||
|
||||
@@ -1,62 +0,0 @@
|
||||
// Row 234 (#1740, r6 X4): host and grant store records for tests. Row a has no way to make a host_grants
|
||||
// record outside a fixture, because row b's grant executor writes them. The decision, its raise event and
|
||||
// its close go in by SQL. The grant evidence goes through writeGrantEvent (F1), so the trigger sees the
|
||||
// real shape.
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { registrySha256, writeGrantEvent } from '../src/broker.mjs';
|
||||
import { grantTag } from '../src/host-grant.mjs';
|
||||
|
||||
const AT = '2026-10-21T00:00:00.000Z';
|
||||
const OPTIONS = JSON.stringify([{ key: 'yes', text: 'Yes' }, { key: 'no', text: 'No' }]);
|
||||
export const HUMAN_YES = { op: 'resolved', by: 'jason', choice: 'yes', via: 'cli' };
|
||||
|
||||
// A credential.mint decision with `grant` in its raise body. `close` is the decision_events row, or null
|
||||
// for an open decision.
|
||||
export function grantDecision(store, { business = 'demo', action = 'credential.mint', route = 'human', grant, close = HUMAN_YES }) {
|
||||
const id = randomUUID();
|
||||
store.run(
|
||||
'INSERT INTO decisions(id,at,business,raised_by_role,raised_by_run,class,action,route_to,question,options,recommendation,blocking) VALUES(?,?,?,?,?,?,?,?,?,?,?,?)',
|
||||
id, AT, business, 'chief', 'chief-run', 'gated', action, route, 'Grant?', OPTIONS, 'yes', 0,
|
||||
);
|
||||
store.run(
|
||||
"INSERT INTO events(id,at,business,kind,actor_role,actor_run,subject,body) VALUES(?,?,?,'action.allowed',?,?,?,?)",
|
||||
randomUUID(), AT, business, 'chief', 'chief-run', id,
|
||||
JSON.stringify({ decision: id, action, operation: 'decision.raise', approvalChoice: 'yes', ...(grant ? { grant } : {}) }),
|
||||
);
|
||||
if (close)
|
||||
store.run('INSERT INTO decision_events(decision,at,op,by,choice,via) VALUES(?,?,?,?,?,?)', id, AT, close.op, close.by, close.choice ?? null, close.via ?? null);
|
||||
return id;
|
||||
}
|
||||
|
||||
// A ready host record from an approved, spent and finished enrollment. grant.host.enrolled keeps the
|
||||
// start reconcile from reading the enrollment as interrupted.
|
||||
export function hostRecord(store, { business = 'demo', host = 'web1', entry, sudo = true }) {
|
||||
const grant = { method: 'host-enroll', host, address: entry.address, port: entry.port, user: entry.user, sudo, fingerprints: [] };
|
||||
const decision = grantDecision(store, { business, grant });
|
||||
writeGrantEvent(store, AT, business, { operation: 'grant.host-enroll', decision }, decision);
|
||||
const installerFingerprint = 'SHA256:' + 'A'.repeat(43);
|
||||
writeGrantEvent(store, AT, business, {
|
||||
action: 'grant.host.enrolled', grantDecision: decision, host, installerFingerprint, user: entry.user, sudo,
|
||||
registrySha256: registrySha256({ ...entry, sudo }),
|
||||
}, decision);
|
||||
store.run(
|
||||
'INSERT INTO hosts(business,host,address,port,user,sudo,host_keys,installer_fingerprint,state,decision,checked_at) VALUES(?,?,?,?,?,?,?,?,?,?,?)',
|
||||
business, host, entry.address, entry.port, entry.user, sudo ? 1 : 0, JSON.stringify(entry.hostKeys), installerFingerprint, 'ready', decision, AT,
|
||||
);
|
||||
return decision;
|
||||
}
|
||||
|
||||
// One agent's grant record on a host. By default its decision is a `host` grant for the same agent and
|
||||
// host, resolved yes by the human via cli and spent as grant.host. Each option breaks one part.
|
||||
export function hostGrantRecord(store, { business = 'demo', agent, host = 'web1', commands, rotateBy = '2099-01-01', grant = {}, close = HUMAN_YES, spend = 'grant.host', action }) {
|
||||
const decision = grantDecision(store, {
|
||||
business, close, action,
|
||||
grant: grant === null ? undefined : { method: 'host', agent, host, commands, rotateBy, ...grant },
|
||||
});
|
||||
if (spend) writeGrantEvent(store, AT, business, { operation: spend, decision }, decision);
|
||||
store.run(
|
||||
'INSERT INTO host_grants(business,agent,host,commands,rotate_by,tag,decision) VALUES(?,?,?,?,?,?,?)',
|
||||
business, agent, host, JSON.stringify(commands), rotateBy, grantTag(business, agent, host), decision,
|
||||
);
|
||||
return decision;
|
||||
}
|
||||
@@ -12,10 +12,8 @@ import { fileURLToPath } from 'node:url';
|
||||
import { Credentials, parseHostKey } from '../src/credentials.mjs';
|
||||
import { addKey, hostExecutor, runHost, sshArgs, startTime, sweepRunDir, quote, remoteCommand } from '../src/host-run.mjs';
|
||||
import { fixture } from './spec-fixtures.mjs';
|
||||
import { HUMAN_YES, hostGrantRecord, hostRecord } from './host-fixtures.mjs';
|
||||
import { Broker, ORGANIZATION_KEYS } from '../src/broker.mjs';
|
||||
import { Store } from '../src/store.mjs';
|
||||
import { hostGrant } from '../src/host-grant.mjs';
|
||||
import { startBroker } from '../src/runtime.mjs';
|
||||
import { businessDoc, rolesCopy, systemFor, writeJson } from '../../business/tests/helpers.mjs';
|
||||
|
||||
@@ -110,12 +108,13 @@ async function hostFixture(t, { commands = GRANT, reviewerHosts, wrongHostKey =
|
||||
const executor = hostExecutor({ credentials, root });
|
||||
const entry = { address: '127.0.0.1', port, user: USER, hostKeys: [pubLine(join(dir, wrongHostKey ? 'otherhost' : 'hostkey'))] };
|
||||
const authority = { withinRole: ['message.send', 'record.write'], crossRole: [] };
|
||||
const f = await fixture(t, { roles: { coder: { definition: 'coder', authority } } });
|
||||
// Row 234 (#1740, r6 X4): host.run reads store records, so the hosts and grants are fixtures there.
|
||||
for (const host of ['web1', 'web2']) hostRecord(f.store, { host, entry });
|
||||
hostGrantRecord(f.store, { agent: 'coder', host: 'web1', commands });
|
||||
if (coderWeb2) hostGrantRecord(f.store, { agent: 'coder', host: 'web2', commands });
|
||||
for (const [host, g] of Object.entries(reviewerHosts ?? {})) hostGrantRecord(f.store, { agent: 'reviewer', host, commands: g.commands });
|
||||
const f = await fixture(t, {
|
||||
roles: {
|
||||
coder: { definition: 'coder', authority, hosts: { web1: { commands }, ...(coderWeb2 ? { web2: { commands } } : {}) } },
|
||||
reviewer: { definition: 'reviewer', authority: { withinRole: ['message.send'], crossRole: [] }, ...(reviewerHosts ? { hosts: reviewerHosts } : {}) },
|
||||
},
|
||||
hosts: { web1: entry, web2: { ...entry } },
|
||||
});
|
||||
const approve = async (argv, stdin, host = 'web1') => {
|
||||
const d = await f.raise({ action: 'host.sudo', target: host, argv, ...(stdin !== undefined ? { stdin } : {}) });
|
||||
f.call(f.human, 'decision.resolve', { id: d.id, choice: 'yes' });
|
||||
@@ -292,26 +291,6 @@ test('T14: a host key not in the registry refuses and nothing runs', async (t) =
|
||||
assert.deepEqual(f.argvSeen(), []);
|
||||
});
|
||||
|
||||
// Row 234 (#1740, r6 G18, duckworth-0081 B2): hosts.check's `list` goes through runHost with the record's
|
||||
// pinned keys. A stub `list` on the scratch sshd's PATH records each run.
|
||||
test('G18: hosts.check with a wrong pin refuses host-key-mismatch and runs nothing on the host', async (t) => {
|
||||
const f = await hostFixture(t);
|
||||
writeFileSync(join(f.dir, 'fake', 'list'), `#!/bin/sh\necho "$@" >> '${f.dir}/list.log'\n`, { mode: 0o755 });
|
||||
const configDir = join(f.dir, 'cfg');
|
||||
const keyDir = join(configDir, 'secrets', 'hosts', 'demo', 'web1');
|
||||
mkdirSync(keyDir, { recursive: true, mode: 0o700 });
|
||||
for (const d of [join(configDir, 'secrets'), join(configDir, 'secrets', 'hosts'), join(configDir, 'secrets', 'hosts', 'demo'), keyDir]) chmodSync(d, 0o700);
|
||||
writeFileSync(join(keyDir, 'installer'), readFileSync(f.keyFile), { mode: 0o600 });
|
||||
const hg = hostGrant({ configDir, runRoot: f.root });
|
||||
const record = { address: '127.0.0.1', port: f.port, user: 'jarvis', hostKeys: [pubLine(join(f.dir, 'otherhost'))] };
|
||||
await refusesWith(hg.check('demo', 'web1', record), 'host-key-mismatch');
|
||||
assert.equal(existsSync(join(f.dir, 'list.log')), false);
|
||||
// Control: with the right pin the same command reaches the stub.
|
||||
const out = await runHost({ root: f.root, hostId: 'web1', entry: f.entry, key: f.key, timeout: 10, command: 'list jarvis' });
|
||||
assert.equal(out.exit, 0);
|
||||
assert.equal(readFileSync(join(f.dir, 'list.log'), 'utf8'), 'jarvis\n');
|
||||
});
|
||||
|
||||
test('N3: a remote exit 255 that prints the pin refusal text is the command result, and an ssh failure keeps its message', async (t) => {
|
||||
const f = await hostFixture(t, { commands: [['fail255']] });
|
||||
const out = await f.call(['fail255']);
|
||||
@@ -553,90 +532,3 @@ test('T27: the bus start sweeps <configDir>/secrets/run before it listens, and a
|
||||
// A bus that starts anyway is closed, so a failure here ends the file instead of holding it open.
|
||||
await assert.rejects(async () => { const again = await start(); await again.close(); }, /credential-file/);
|
||||
});
|
||||
|
||||
// Row 234 (#1740, duckworth-0079 M3, r6 X4 first bullet): host.run takes a grant record only behind the
|
||||
// Approve record check. Each case breaks one part of it. The refusals come before the executor, so none
|
||||
// needs sshd. The control passes the check and stops at host-not-configured.
|
||||
test('M3: host.run refuses host-not-granted unless the grant record has an approved, spent host grant', async (t) => {
|
||||
const f = await fixture(t, { roles: { coder: { definition: 'coder', authority: { withinRole: ['message.send'], crossRole: [] } } } });
|
||||
const entry = { address: '10.0.0.5', port: 22, user: 'admin', hostKeys: ['ssh-ed25519 AAAA'] };
|
||||
for (const host of ['web1', 'web2']) hostRecord(f.store, { host, entry });
|
||||
const run = (cap = f.coder, host = 'web1') => f.broker.requestHost(cap, { verb: 'host.run', args: { host, argv: ['true'] } }, null);
|
||||
const commands = [['true']];
|
||||
const cases = {
|
||||
'no record': null,
|
||||
unresolved: { close: null },
|
||||
'resolved no': { close: { ...HUMAN_YES, choice: 'no' } },
|
||||
withdrawn: { close: { op: 'withdrawn', by: 'jason', via: 'cli' } },
|
||||
'an arbiter': { close: { ...HUMAN_YES, by: 'reviewer', via: 'broker' } },
|
||||
'the tester': { close: { ...HUMAN_YES, by: 'tester' } },
|
||||
'an agent': { close: { ...HUMAN_YES, by: 'coder', via: 'broker' } },
|
||||
'not via cli': { close: { ...HUMAN_YES, via: 'web' } },
|
||||
'consumed as grant.host-enroll': { spend: 'grant.host-enroll' },
|
||||
unspent: { spend: null },
|
||||
'not a credential.mint': { action: 'deploy' },
|
||||
'no grant in the raise': { grant: null },
|
||||
'a host-enroll grant': { grant: { method: 'host-enroll' } },
|
||||
'a grant for another agent': { grant: { agent: 'reviewer' } },
|
||||
'a grant for another host': { grant: { host: 'web2' } },
|
||||
};
|
||||
for (const [name, opts] of Object.entries(cases)) {
|
||||
f.store.run('DELETE FROM host_grants');
|
||||
if (opts) hostGrantRecord(f.store, { agent: 'coder', host: 'web1', commands, ...opts });
|
||||
await refusesWith(run(), 'host-not-granted').catch((e) => assert.fail(`${name}: ${e.message}`));
|
||||
}
|
||||
f.store.run('DELETE FROM host_grants');
|
||||
hostGrantRecord(f.store, { agent: 'coder', host: 'web1', commands });
|
||||
await refusesWith(run(), 'host-not-configured');
|
||||
// The record is per agent: the reviewer holds none on web1, and web9 has no host record.
|
||||
await refusesWith(run(f.reviewer), 'host-not-granted');
|
||||
await refusesWith(run(f.coder, 'web9'), 'host-unknown');
|
||||
// The host.sudo raise takes the same check (N1).
|
||||
f.store.run('DELETE FROM host_grants');
|
||||
hostGrantRecord(f.store, { agent: 'coder', host: 'web1', commands, spend: null });
|
||||
assert.throws(() => f.raise({ action: 'host.sudo', target: 'web1', argv: ['true'] }), /host-not-granted/);
|
||||
});
|
||||
|
||||
// Row 234 (#1740, M3, r6 X4 third bullet): at start each host_grants record gives Credentials a store
|
||||
// reference to host.<id>, so a restart finds the key the grant executor wrote.
|
||||
test('M3: a host_grants record gives Credentials a store reference at start, and a restart reads its key', async (t) => {
|
||||
const root = mkdtempSync(join(tmpdir(), 'h234-'));
|
||||
t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
const rolesDir = rolesCopy(root);
|
||||
mkdirSync(join(root, 'data'), { mode: 0o700 });
|
||||
mkdirSync(join(root, 'secrets', 'agents'), { recursive: true, mode: 0o700 });
|
||||
chmodSync(join(root, 'secrets'), 0o700);
|
||||
const demo = {
|
||||
id: 'demo', human: 'jason', arbiters: { technical: 'coder', delivery: 'coder' },
|
||||
roles: { coder: { authority: { withinRole: ['message.send'], crossRole: [] }, credentials: {} } },
|
||||
};
|
||||
const start = () => startBroker({ dataRoot: join(root, 'data'), businesses: { demo }, organization: { dir: root, rolesDir, system: systemFor(root) } });
|
||||
const hostKeys = (bus) => bus.credentials.status().filter((c) => c.service.startsWith('host.'));
|
||||
let bus = await start();
|
||||
assert.deepEqual(hostKeys(bus), []);
|
||||
await bus.close();
|
||||
const store = new Store(join(root, 'data'));
|
||||
hostRecord(store, { host: 'web1', entry: { address: '10.0.0.5', port: 22, user: 'admin', hostKeys: ['ssh-ed25519 AAAA'] } });
|
||||
hostGrantRecord(store, { agent: 'coder', host: 'web1', commands: [['true']], rotateBy: '2099-02-03' });
|
||||
store.close();
|
||||
bus = await start();
|
||||
try {
|
||||
assert.deepEqual(hostKeys(bus).map(({ instance, service, date, state }) => ({ instance, service, date, state })), [
|
||||
{ instance: 'demo/coder', service: 'host.web1', date: '2099-02-03', state: 'not-issued' },
|
||||
]);
|
||||
} finally {
|
||||
await bus.close();
|
||||
}
|
||||
const dir = join(root, 'secrets', 'agents', 'demo', 'coder');
|
||||
mkdirSync(dir, { recursive: true, mode: 0o700 });
|
||||
chmodSync(join(root, 'secrets', 'agents', 'demo'), 0o700);
|
||||
keygen(join(dir, 'host.web1'));
|
||||
bus = await start();
|
||||
try {
|
||||
const [key] = hostKeys(bus);
|
||||
assert.equal(key.publicKey, pubLine(join(dir, 'host.web1')));
|
||||
assert.notEqual(key.state, 'not-issued');
|
||||
} finally {
|
||||
await bus.close();
|
||||
}
|
||||
});
|
||||
|
||||
@@ -1,358 +0,0 @@
|
||||
// Row 234 (#1740, #436a row a), design r6 E3: the four host files in tools/host, as G16 and G25.
|
||||
// The writer runs as the test user through a harness that loads it by path and passes its own dirs,
|
||||
// owner uid and user, group and sudo lookups, so nothing runs as root and nothing under /etc is read.
|
||||
// mosaic-keys runs from a copy whose sudo and logger paths point at stubs in a scratch dir.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { execFileSync, spawnSync } from 'node:child_process';
|
||||
import { createHash, randomBytes } from 'node:crypto';
|
||||
import {
|
||||
chmodSync, existsSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, statSync, symlinkSync,
|
||||
writeFileSync,
|
||||
} from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const REPO = fileURLToPath(new URL('../../../', import.meta.url));
|
||||
const TOOLS = join(REPO, 'tools', 'host');
|
||||
const WRITER = join(TOOLS, 'mosaic-keys-write');
|
||||
const REFUSED_GROUPS = ['sudo', 'admin', 'wheel', 'docker', 'lxd', 'disk', 'libvirt', 'incus-admin'];
|
||||
const tag = (n) => 'mosaic-' + createHash('sha256').update(String(n)).digest('hex').slice(0, 32);
|
||||
const u32 = (n) => {
|
||||
const b = Buffer.alloc(4);
|
||||
b.writeUInt32BE(n);
|
||||
return b;
|
||||
};
|
||||
const keyBlob = () => Buffer.concat([u32(11), Buffer.from('ssh-ed25519'), u32(32), randomBytes(32)]).toString('base64');
|
||||
const fingerprint = (b64) => 'SHA256:' + createHash('sha256').update(Buffer.from(b64, 'base64')).digest('base64').replace(/=+$/, '');
|
||||
const notAllowed = (u) => `User ${u} is not allowed to run sudo on testhost.\n`;
|
||||
|
||||
const HARNESS = `import importlib.machinery, importlib.util, json, os, sys
|
||||
sys.dont_write_bytecode = True
|
||||
cfg = json.loads(sys.argv[1])
|
||||
loader = importlib.machinery.SourceFileLoader('mosaic_keys_write', cfg['writer'])
|
||||
m = importlib.util.module_from_spec(importlib.util.spec_from_loader('mosaic_keys_write', loader))
|
||||
loader.exec_module(m)
|
||||
users = cfg['users']
|
||||
def log(msg):
|
||||
with open(cfg['log'], 'a') as f:
|
||||
f.write(msg + '\\n')
|
||||
def sudo(u):
|
||||
with open(cfg['sudoLog'], 'a') as f:
|
||||
f.write(u + '\\n')
|
||||
return users[u]['sudo'] if 'sudo' in users[u] else 'User %s is not allowed to run sudo on testhost.\\n' % u
|
||||
host = m.Host(key_dir=cfg['keyDir'], conf_dir=cfg['confDir'], owner=os.getuid(),
|
||||
passwd=lambda u: users[u]['uid'] if u in users else None,
|
||||
groups=lambda u: users[u].get('groups', []), sudo=sudo, hostname='testhost', log=log)
|
||||
sys.exit(m.main(sys.argv[2:], host))
|
||||
`;
|
||||
|
||||
// The users the harness's lookups know. users lists all of them but dave and eve. bob and dave are
|
||||
// privileged. carol has a sudo line. Each gX is in one refused group.
|
||||
function userTable() {
|
||||
const users = {
|
||||
alice: { uid: 1500 },
|
||||
bob: { uid: 1501, groups: ['sudo', 'docker'], sudo: 'User bob may run the following commands on testhost:\n (ALL) ALL\n' },
|
||||
carol: { uid: 1502, sudo: 'User carol may run the following commands on testhost:\n (root) /usr/bin/true\n' },
|
||||
dave: { uid: 1503 },
|
||||
eve: { uid: 1504 },
|
||||
frank: { uid: 1505, groups: ['users', 'adm'] },
|
||||
'../x': { uid: 1506 },
|
||||
root: { uid: 0 },
|
||||
toor: { uid: 0 },
|
||||
daemon2: { uid: 999 },
|
||||
localadmin: { uid: 1000 },
|
||||
'mosaic-installer': { uid: 1001 },
|
||||
};
|
||||
for (const [i, g] of REFUSED_GROUPS.entries()) users['g' + g] = { uid: 1600 + i, groups: ['users', g] };
|
||||
return users;
|
||||
}
|
||||
|
||||
function hostDirs() {
|
||||
const root = mkdtempSync(join(tmpdir(), 'hk234-'));
|
||||
const keyDir = join(root, 'keys');
|
||||
const confDir = join(root, 'conf');
|
||||
mkdirSync(keyDir, { mode: 0o755 });
|
||||
mkdirSync(confDir, { mode: 0o755 });
|
||||
chmodSync(keyDir, 0o755);
|
||||
const users = userTable();
|
||||
const conf = (name, text) => {
|
||||
writeFileSync(join(confDir, name), text, { mode: 0o644 });
|
||||
chmodSync(join(confDir, name), 0o644);
|
||||
};
|
||||
conf('users', '# users the writer may touch\n' + Object.keys(users).filter((u) => !['dave', 'eve'].includes(u)).join('\n') + '\n');
|
||||
conf('privileged', 'bob\ndave\n');
|
||||
conf('from', '10.1.2.3/32\n');
|
||||
const harness = join(root, 'harness.py');
|
||||
writeFileSync(harness, HARNESS);
|
||||
const cfg = { writer: WRITER, keyDir, confDir, users, log: join(root, 'syslog'), sudoLog: join(root, 'sudo.log') };
|
||||
const run = (...args) => {
|
||||
const r = spawnSync('python3', ['-B', '-I', harness, JSON.stringify(cfg), ...args], { encoding: 'utf8' });
|
||||
return { status: r.status, stdout: r.stdout, stderr: r.stderr };
|
||||
};
|
||||
const read = (file) => readFileSync(join(keyDir, file), 'utf8');
|
||||
const lastLog = () => readFileSync(cfg.log, 'utf8').trim().split('\n').at(-1);
|
||||
const sudoCalls = () => (existsSync(cfg.sudoLog) ? readFileSync(cfg.sudoLog, 'utf8').trim().split('\n') : []);
|
||||
return { root, keyDir, confDir, conf, run, read, lastLog, sudoCalls, done: () => rmSync(root, { recursive: true, force: true }) };
|
||||
}
|
||||
|
||||
const refused = (r, code) => {
|
||||
assert.equal(r.status, 1, r.stderr);
|
||||
assert.equal(r.stderr, `mosaic-keys-write: refused ${code}\n`);
|
||||
};
|
||||
|
||||
test('G16: the writer refuses each user, tag and key the design names, before any key file', () => {
|
||||
const h = hostDirs();
|
||||
try {
|
||||
const key = keyBlob();
|
||||
const t = tag(1);
|
||||
const add = (user) => h.run('add', user, 'ssh-ed25519', key, t);
|
||||
// The user pattern stops `../x` before any path, though the lookups and users know it.
|
||||
for (const user of ['../x', 'Alice', 'a'.repeat(33), '1alice', 'al ice', '']) {
|
||||
refused(add(user), 'user-pattern');
|
||||
refused(h.run('list', user), 'user-pattern');
|
||||
}
|
||||
// users is always required: privileged alone allows nothing.
|
||||
refused(add('dave'), 'user-not-listed');
|
||||
refused(add('eve'), 'user-not-listed');
|
||||
for (const user of ['root', 'localadmin', 'mosaic-installer']) refused(add(user), 'user-not-listed');
|
||||
refused(add('toor'), 'user-system');
|
||||
refused(add('daemon2'), 'user-system');
|
||||
refused(add('ghost'), 'user-not-listed');
|
||||
for (const g of REFUSED_GROUPS) {
|
||||
refused(add('g' + g), 'user-group');
|
||||
refused(h.run('remove', 'g' + g, t), 'user-group');
|
||||
}
|
||||
refused(add('carol'), 'user-sudo');
|
||||
refused(h.run('list', 'carol'), 'user-sudo');
|
||||
assert.equal(h.lastLog(), 'result=refused user-sudo user=carol');
|
||||
for (const bad of ['mosaic-' + 'a'.repeat(31), 'mosaic-' + 'A'.repeat(32), 'mosaic-installer-' + 'a'.repeat(32), 'mosaic-' + 'g'.repeat(32)]) {
|
||||
refused(h.run('add', 'alice', 'ssh-ed25519', key, bad), 'tag-pattern');
|
||||
refused(h.run('remove', 'alice', bad), 'tag-pattern');
|
||||
}
|
||||
refused(h.run('add', 'alice', 'ssh-dss', key, t), 'key-format');
|
||||
refused(h.run('add', 'alice', 'ssh-ed25519', key.slice(0, -2) + '!!', t), 'key-format');
|
||||
refused(h.run('add', 'alice', 'ssh-ed25519', 'A', t), 'key-format');
|
||||
refused(h.run('add', 'alice', 'ssh-ed25519', key + ' x', t), 'key-format');
|
||||
for (const args of [[], ['list'], ['list', 'alice', 'bob'], ['add', 'alice', 'ssh-ed25519', key], ['remove', 'alice'], ['rm', 'alice', t]])
|
||||
refused(h.run(...args), 'request');
|
||||
// No refusal reached the key dir, so not even the lock exists.
|
||||
assert.deepEqual(readdirSync(h.keyDir), []);
|
||||
// Only users past the groups check reach sudo -l, and the privileged never do.
|
||||
assert.deepEqual([...new Set(h.sudoCalls())], ['carol']);
|
||||
} finally {
|
||||
h.done();
|
||||
}
|
||||
});
|
||||
|
||||
test('G16: from= comes from the host file, a privileged user skips the group and sudo checks, and the line is exact', () => {
|
||||
const h = hostDirs();
|
||||
try {
|
||||
const k1 = keyBlob();
|
||||
const k2 = keyBlob();
|
||||
assert.equal(h.run('add', 'alice', 'ssh-ed25519', k1, tag(1)).status, 0);
|
||||
assert.equal(h.lastLog(), `result=added user=alice tag=${tag(1)} key=${fingerprint(k1)}`);
|
||||
h.conf('from', '# the stack host\n10.9.9.9,192.168.1.0/24\n');
|
||||
assert.equal(h.run('add', 'alice', 'ssh-ed25519', k2, tag(2)).status, 0);
|
||||
assert.equal(
|
||||
h.read('alice'),
|
||||
`from="10.1.2.3/32",restrict ssh-ed25519 ${k1} ${tag(1)}\nfrom="10.9.9.9,192.168.1.0/24",restrict ssh-ed25519 ${k2} ${tag(2)}\n`,
|
||||
);
|
||||
// A tag already there changes nothing and exits 0.
|
||||
assert.equal(h.run('add', 'alice', 'ssh-ed25519', k2, tag(1)).status, 0);
|
||||
assert.equal(h.lastLog(), `result=present user=alice tag=${tag(1)} key=${fingerprint(k2)}`);
|
||||
assert.equal(h.read('alice').split('\n').length, 3);
|
||||
// bob is in sudo and docker with a sudo line, and privileged.
|
||||
assert.equal(h.run('add', 'bob', 'ssh-ed25519', k1, tag(3)).status, 0);
|
||||
// frank's groups are none of the refused ones.
|
||||
assert.equal(h.run('add', 'frank', 'ssh-ed25519', k1, tag(4)).status, 0);
|
||||
assert.deepEqual(h.sudoCalls(), ['alice', 'alice', 'alice', 'frank']);
|
||||
// A from value that could close the quote, two values, or none, refuses and writes nothing.
|
||||
for (const text of ['10.0.0.1",command="/bin/sh\n', '10.0.0.1\n10.0.0.2\n', '# none\n', '10.0.0.1 10.0.0.2\n']) {
|
||||
h.conf('from', text);
|
||||
refused(h.run('add', 'frank', 'ssh-ed25519', k2, tag(5)), 'config-from');
|
||||
}
|
||||
rmSync(join(h.confDir, 'from'));
|
||||
refused(h.run('add', 'frank', 'ssh-ed25519', k2, tag(5)), 'config-from');
|
||||
assert.ok(!h.read('frank').includes(tag(5)));
|
||||
// list and remove don't need from.
|
||||
assert.equal(h.run('list', 'frank').stdout, `from="10.9.9.9,192.168.1.0/24",restrict ssh-ed25519 ${k1} ${tag(4)}\n`);
|
||||
for (const name of ['users', 'privileged']) {
|
||||
chmodSync(join(h.confDir, name), 0o664);
|
||||
refused(h.run('list', 'alice'), 'not-root-owned');
|
||||
chmodSync(join(h.confDir, name), 0o644);
|
||||
}
|
||||
rmSync(join(h.confDir, 'users'));
|
||||
refused(h.run('list', 'alice'), 'config-users');
|
||||
} finally {
|
||||
h.done();
|
||||
}
|
||||
});
|
||||
|
||||
test('G16: list shows tagged lines only, and add and remove never touch a line outside the tag pattern', () => {
|
||||
const h = hostDirs();
|
||||
try {
|
||||
const k = keyBlob();
|
||||
const hand = [
|
||||
'# written by hand',
|
||||
`ssh-ed25519 ${k} mosaic-installer-${'a'.repeat(32)}`,
|
||||
'',
|
||||
`ssh-ed25519 ${k} ${tag(9).toUpperCase()}`,
|
||||
`ssh-ed25519 ${k} ${tag(1)} trailing`,
|
||||
];
|
||||
const tagged = `from="10.1.2.3/32",restrict ssh-ed25519 ${k} ${tag(7)}`;
|
||||
writeFileSync(join(h.keyDir, 'alice'), [hand[0], tagged, ...hand.slice(1)].join('\n') + '\n', { mode: 0o644 });
|
||||
assert.equal(h.run('list', 'alice').stdout, tagged + '\n');
|
||||
// The last field is the tag, so `... mosaic-<1> trailing` isn't tag 1.
|
||||
assert.equal(h.run('remove', 'alice', tag(1)).status, 0);
|
||||
assert.equal(h.lastLog(), `result=absent user=alice tag=${tag(1)}`);
|
||||
assert.equal(h.run('add', 'alice', 'ssh-ed25519', k, tag(8)).status, 0);
|
||||
assert.equal(h.run('remove', 'alice', tag(7)).status, 0);
|
||||
assert.equal(h.lastLog(), `result=removed user=alice tag=${tag(7)}`);
|
||||
assert.equal(h.read('alice'), [...hand, `from="10.1.2.3/32",restrict ssh-ed25519 ${k} ${tag(8)}`].join('\n') + '\n');
|
||||
assert.equal(h.run('remove', 'alice', tag(8)).status, 0);
|
||||
assert.equal(h.read('alice'), hand.join('\n') + '\n');
|
||||
// A missing file lists nothing and a remove there changes nothing.
|
||||
assert.equal(h.run('list', 'frank').stdout, '');
|
||||
assert.equal(h.run('remove', 'frank', tag(1)).status, 0);
|
||||
assert.ok(!existsSync(join(h.keyDir, 'frank')));
|
||||
} finally {
|
||||
h.done();
|
||||
}
|
||||
});
|
||||
|
||||
test('G25: the writer refuses a symlinked dir or user file, a group-writable dir and an odd tmp, and writes 0644', () => {
|
||||
const h = hostDirs();
|
||||
try {
|
||||
const k = keyBlob();
|
||||
const add = (n, user = 'alice') => h.run('add', user, 'ssh-ed25519', k, tag(n));
|
||||
// A first add creates the file, regular, 0644, owned by the owner uid, with no tmp left.
|
||||
assert.equal(add(1).status, 0);
|
||||
const st = statSync(join(h.keyDir, 'alice'));
|
||||
assert.equal(st.mode & 0o777, 0o644);
|
||||
assert.equal(st.uid, process.getuid());
|
||||
assert.deepEqual(readdirSync(h.keyDir).sort(), ['.lock', 'alice']);
|
||||
// A leftover regular tmp is unlinked and the write goes ahead.
|
||||
writeFileSync(join(h.keyDir, '.alice.tmp'), 'half a write');
|
||||
assert.equal(add(2).status, 0);
|
||||
assert.ok(h.read('alice').includes(tag(2)));
|
||||
assert.ok(!existsSync(join(h.keyDir, '.alice.tmp')));
|
||||
// A tmp that is a symlink, a dir or a fifo refuses, and nothing is written.
|
||||
const decoy = join(h.root, 'decoy');
|
||||
writeFileSync(decoy, 'decoy\n');
|
||||
const before = h.read('alice');
|
||||
for (const make of [
|
||||
(p) => symlinkSync(decoy, p),
|
||||
(p) => mkdirSync(p),
|
||||
(p) => execFileSync('mkfifo', [p]),
|
||||
]) {
|
||||
make(join(h.keyDir, '.alice.tmp'));
|
||||
refused(add(3), 'tmp-file');
|
||||
assert.equal(h.read('alice'), before);
|
||||
assert.equal(readFileSync(decoy, 'utf8'), 'decoy\n');
|
||||
rmSync(join(h.keyDir, '.alice.tmp'), { recursive: true });
|
||||
}
|
||||
// A symlinked user file refuses, though its target is a regular file the owner owns.
|
||||
symlinkSync(decoy, join(h.keyDir, 'frank'));
|
||||
refused(add(4, 'frank'), 'user-file');
|
||||
refused(h.run('list', 'frank'), 'user-file');
|
||||
assert.equal(readFileSync(decoy, 'utf8'), 'decoy\n');
|
||||
assert.ok(lstatSync(join(h.keyDir, 'frank')).isSymbolicLink());
|
||||
// A user "file" that is a dir refuses.
|
||||
mkdirSync(join(h.keyDir, 'bob'));
|
||||
refused(add(5, 'bob'), 'not-root-owned');
|
||||
// A group-writable dir refuses.
|
||||
chmodSync(h.keyDir, 0o775);
|
||||
refused(add(6), 'not-root-owned');
|
||||
chmodSync(h.keyDir, 0o755);
|
||||
// A symlinked key dir refuses, and its target is untouched.
|
||||
const real = join(h.root, 'real-keys');
|
||||
mkdirSync(real, { mode: 0o755 });
|
||||
rmSync(h.keyDir, { recursive: true });
|
||||
symlinkSync(real, h.keyDir);
|
||||
refused(add(7), 'key-dir');
|
||||
assert.deepEqual(readdirSync(real), []);
|
||||
} finally {
|
||||
h.done();
|
||||
}
|
||||
});
|
||||
|
||||
// mosaic-keys from a copy with its sudo and logger paths on stubs. The stub sudo records its argv,
|
||||
// prints a line and exits 3, so the test sees the exit and output pass through.
|
||||
function forcedCommand() {
|
||||
const root = mkdtempSync(join(tmpdir(), 'hk234-'));
|
||||
const sudo = join(root, 'sudo');
|
||||
const logger = join(root, 'logger');
|
||||
writeFileSync(sudo, `#!/bin/bash\nprintf '%s\\n' "$@" > ${root}/sudo.argv\necho from-writer\nexit 3\n`, { mode: 0o755 });
|
||||
writeFileSync(logger, `#!/bin/bash\nprintf '%s\\n' "$*" >> ${root}/logger.log\n`, { mode: 0o755 });
|
||||
const source = readFileSync(join(TOOLS, 'mosaic-keys'), 'utf8');
|
||||
assert.equal(source.split('/usr/bin/sudo').length, 2);
|
||||
assert.equal(source.split('/usr/bin/logger').length, 2);
|
||||
const copy = join(root, 'mosaic-keys');
|
||||
writeFileSync(copy, source.replace('/usr/bin/sudo', sudo).replace('/usr/bin/logger', logger), { mode: 0o755 });
|
||||
const run = (request) => {
|
||||
rmSync(join(root, 'sudo.argv'), { force: true });
|
||||
const env = { PATH: '/usr/bin:/bin', SSH_CONNECTION: '10.1.2.3 50000 10.1.2.9 22' };
|
||||
if (request !== undefined) env.SSH_ORIGINAL_COMMAND = request;
|
||||
const r = spawnSync(copy, [], { encoding: 'utf8', env });
|
||||
const argv = existsSync(join(root, 'sudo.argv')) ? readFileSync(join(root, 'sudo.argv'), 'utf8').trim().split('\n') : null;
|
||||
return { status: r.status, stdout: r.stdout, stderr: r.stderr, argv };
|
||||
};
|
||||
const logged = () => readFileSync(join(root, 'logger.log'), 'utf8').trim().split('\n');
|
||||
return { run, logged, done: () => rmSync(root, { recursive: true, force: true }) };
|
||||
}
|
||||
|
||||
test('G16: mosaic-keys passes the three requests to the writer through sudo and refuses anything else without it', () => {
|
||||
const f = forcedCommand();
|
||||
try {
|
||||
const k = keyBlob();
|
||||
const t = tag(1);
|
||||
const writer = ['-n', '/usr/local/sbin/mosaic-keys-write'];
|
||||
for (const words of [['list', 'alice'], ['add', 'alice', 'ssh-ed25519', k, t], ['add', 'alice', 'ecdsa-sha2-nistp256', 'AAAA', t], ['remove', 'alice', t]]) {
|
||||
const r = f.run(words.join(' '));
|
||||
assert.deepEqual(r.argv, [...writer, ...words]);
|
||||
assert.equal(r.status, 3);
|
||||
assert.equal(r.stdout, 'from-writer\n');
|
||||
}
|
||||
const bad = [
|
||||
undefined, '', 'list', 'list ../x', 'list Alice', 'list alice;id', 'list alice ', ' list alice', 'list alice', 'list\talice',
|
||||
'list alice\nlist bob', 'list $(id)', 'list alice`id`', 'list *', 'rm alice', 'list ' + 'a'.repeat(33),
|
||||
`add alice ssh-dss ${k} ${t}`, `add alice ssh-ed25519 ${k}! ${t}`, `add alice ssh-ed25519 ${k}`, `add alice ssh-ed25519 ${k} ${t} x`,
|
||||
`remove alice mosaic-${'a'.repeat(31)}`, `remove alice ${t.toUpperCase()}`, `remove alice ${t}\n`, `remove alice ${t}; rm -rf /`,
|
||||
];
|
||||
for (const request of bad) {
|
||||
const r = f.run(request);
|
||||
assert.equal(r.argv, null, JSON.stringify(request));
|
||||
assert.equal(r.status, 1);
|
||||
assert.equal(r.stderr, 'mosaic-keys: refused\n');
|
||||
}
|
||||
// Two bad requests hold a newline, so count the logger calls, not the lines.
|
||||
const logged = f.logged().filter((l) => l.startsWith('-t mosaic-keys -- '));
|
||||
assert.equal(logged.length, 4 + bad.length);
|
||||
assert.equal(logged[0], '-t mosaic-keys -- connection=10.1.2.3 50000 10.1.2.9 22 request=list alice');
|
||||
} finally {
|
||||
f.done();
|
||||
}
|
||||
});
|
||||
|
||||
test('E3: the sudoers line and the sshd drop-in are the design lines and parse', () => {
|
||||
const lines = (file) => readFileSync(join(TOOLS, file), 'utf8').split('\n').filter((l) => l && !l.startsWith('#'));
|
||||
assert.deepEqual(lines('mosaic-keys.sudoers'), ['mosaic-installer ALL=(root) NOPASSWD: /usr/local/sbin/mosaic-keys-write']);
|
||||
assert.deepEqual(lines('mosaic-keys.sshd.conf'), ['AuthorizedKeysFile .ssh/authorized_keys .ssh/authorized_keys2 /etc/ssh/mosaic-keys/%u']);
|
||||
assert.ok(existsSync('/usr/bin/visudo'), 'visudo on the test host');
|
||||
execFileSync('/usr/bin/visudo', ['-c', '-q', '-f', join(TOOLS, 'mosaic-keys.sudoers')]);
|
||||
const dir = mkdtempSync(join(tmpdir(), 'hk234-'));
|
||||
try {
|
||||
execFileSync('ssh-keygen', ['-q', '-t', 'ed25519', '-N', '', '-f', join(dir, 'hk')]);
|
||||
writeFileSync(join(dir, 'cfg'), `HostKey ${join(dir, 'hk')}\nInclude ${join(TOOLS, 'mosaic-keys.sshd.conf')}\n`);
|
||||
const out = execFileSync('/usr/bin/sshd', ['-T', '-f', join(dir, 'cfg'), '-C', 'user=alice,host=x,addr=127.0.0.1'], { encoding: 'utf8' });
|
||||
// sshd -T prints this key as AuthorizedKeysFile, unlike most.
|
||||
const line = out.split('\n').filter((l) => /^authorizedkeysfile /i.test(l));
|
||||
assert.deepEqual(line.map((l) => l.split(' ').slice(1).join(' ')), ['.ssh/authorized_keys .ssh/authorized_keys2 /etc/ssh/mosaic-keys/%u']);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
for (const file of ['mosaic-keys', 'mosaic-keys-write']) assert.equal(statSync(join(TOOLS, file)).mode & 0o111, 0o111, file);
|
||||
assert.equal(readFileSync(WRITER, 'utf8').split('\n')[0], '#!/usr/bin/python3 -I');
|
||||
});
|
||||
@@ -55,7 +55,7 @@ test('3f copy migrates to 3g preserving every old column/row, with an empty inst
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
const store = new Store(root, { migrateSchema: true });
|
||||
try {
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3j');
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3i');
|
||||
assert.deepEqual({ ...store.get('SELECT to_name FROM messages') }, { to_name: null });
|
||||
assert.equal(store.get('SELECT count(*) AS n FROM instance_names').n, 0);
|
||||
assert.deepEqual({ ...store.get('SELECT spec_path,spec_commit FROM decisions') }, { spec_path: 'docs/spec.md', spec_commit: 'c'.repeat(40) });
|
||||
|
||||
@@ -25,8 +25,7 @@ test('Broker keeps the sandbox client import closure: only the listed imports, n
|
||||
// RUNNER_FILES and imports nothing. Row 155 (#1649) reads THREAD_DEFINITION and END_PROTECTED
|
||||
// from it too. Row 199 (#1697) reads the D9c fork parse, forkProblem and forkUrl, from it.
|
||||
// Row 210 (#1707) reads credentialKind and the D6 argv check, validArgv and commandAllowed, from it.
|
||||
// Row 234 (#1740) reads HOST_ID for the hosts verbs.
|
||||
"import { ID_PATTERN, THREAD_DEFINITION, END_PROTECTED, forkProblem, forkUrl, credentialKind, validArgv, commandAllowed, HOST_ID } from '../../business/src/vocabulary.mjs';",
|
||||
"import { ID_PATTERN, THREAD_DEFINITION, END_PROTECTED, forkProblem, forkUrl, credentialKind, validArgv, commandAllowed } from '../../business/src/vocabulary.mjs';",
|
||||
]);
|
||||
assert.doesNotMatch(source, /\bimport\s*\(/);
|
||||
assert.doesNotMatch(readFileSync(new URL('../src/landing.mjs', import.meta.url), 'utf8'), /\bimport\b/);
|
||||
|
||||
@@ -7,7 +7,7 @@ import { Broker } from '../src/broker.mjs';
|
||||
import { initProject, readSpec, writeRecord, listProjectFiles, readProjectFile, writeHumanFile, ProjectError } from '../../project/src/index.mjs';
|
||||
export const git = (repo, ...args) => execFileSync('git', ['-c', 'core.hooksPath=/dev/null', '-C', repo, ...args], { encoding: 'utf8' }).trim();
|
||||
const options = [{ key: 'yes', text: 'Yes' }, { key: 'no', text: 'No' }];
|
||||
export async function fixture(t, { adapter = true, roles = {}, arbiters = { technical: 'reviewer', delivery: 'reviewer' } } = {}) {
|
||||
export async function fixture(t, { adapter = true, roles = {}, arbiters = { technical: 'reviewer', delivery: 'reviewer' }, hosts } = {}) {
|
||||
const root = mkdtempSync(join(tmpdir(), 'bus-spec-'));
|
||||
const store = new Store(root);
|
||||
t.after(() => { store.close(); rmSync(root, { recursive: true, force: true }); });
|
||||
@@ -17,7 +17,7 @@ export async function fixture(t, { adapter = true, roles = {}, arbiters = { tech
|
||||
coder: { definition: 'coder', authority: authority(['message.send', 'record.write']) },
|
||||
reviewer: { definition: 'reviewer', authority: authority(['message.send']) },
|
||||
...roles,
|
||||
} };
|
||||
}, ...(hosts ? { hosts } : {}) };
|
||||
const { repo, commit } = await initProject({ business, project: 'stack', dataRoot: root });
|
||||
const broker = new Broker({ store, businesses: { demo: business }, project: adapter ? {
|
||||
readSpec: context => readSpec({ ...context, dataRoot: root }),
|
||||
|
||||
@@ -53,7 +53,7 @@ test('3e copy migrates through 3f to 3g preserving every old column/row and addi
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
const store = new Store(root, { migrateSchema: true });
|
||||
try {
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3j');
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3i');
|
||||
assert.deepEqual({ ...store.get('SELECT spec_path,spec_commit FROM decisions') }, { spec_path: null, spec_commit: null });
|
||||
assert.equal(store.get('PRAGMA quick_check').quick_check, 'ok');
|
||||
assert.deepEqual(store.all('PRAGMA foreign_key_check'), []);
|
||||
|
||||
@@ -49,7 +49,7 @@ async function database3d(t) {
|
||||
const source = readFileSync(here('../src/store.mjs'), 'utf8');
|
||||
const pinned = source
|
||||
.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3d).digest('hex')}';`)
|
||||
.replace("export const SCHEMA_VERSION = '3j';", "export const SCHEMA_VERSION = '3d';")
|
||||
.replace("export const SCHEMA_VERSION = '3i';", "export const SCHEMA_VERSION = '3d';")
|
||||
.replace(/\}, \{\n {2}from: '3d',[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables');
|
||||
assert.ok(!pinned.includes('3d-3e'), 'the 3d engine has no 3d→3e step');
|
||||
writeFileSync(join(moduleRoot, 'src/store.mjs'), pinned);
|
||||
@@ -101,7 +101,7 @@ test('REQ-THREAD-1 REQ-ORG-2: the 3d→3e migration adds the thread tables and l
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
const store = new Store(root, { migrateSchema: true });
|
||||
t.after(() => store.close());
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3j');
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3i');
|
||||
for (const table of ['projects', 'threads', 'thread_claims', 'message_threads', 'thread_opens'])
|
||||
assert.equal(store.get(`SELECT count(*) n FROM ${table}`).n, 0, table);
|
||||
// Parked: the undelivered ones are counted per business and role, and none is changed or given a thread.
|
||||
@@ -161,5 +161,5 @@ test('REQ-THREAD-1: the offline command prints the kept-message counts before th
|
||||
const command = new URL('../src/migrate-schema.mjs', import.meta.url).pathname;
|
||||
const migrated = spawnSync(process.execPath, [command, root], { encoding: 'utf8' });
|
||||
assert.equal(migrated.status, 0, migrated.stderr);
|
||||
assert.equal(migrated.stdout, [...REPORT, 'bus schema 3j ready', ''].join('\n'));
|
||||
assert.equal(migrated.stdout, [...REPORT, 'bus schema 3i ready', ''].join('\n'));
|
||||
});
|
||||
|
||||
@@ -51,12 +51,12 @@ function snapshot(path, columns) {
|
||||
return { columns, rows };
|
||||
} finally { db.close(); }
|
||||
}
|
||||
test('REQ-LIMIT-4 3g copy migrates through 3h and 3i to 3j preserving every old column, row and sequence; new usage table is empty', t => {
|
||||
test('REQ-LIMIT-4 3g copy migrates through 3h to 3i preserving every old column, row and sequence; new usage table is empty', t => {
|
||||
const { root, path, copy } = source3g(t), before = snapshot(path), untouched = readFileSync(copy);
|
||||
assert.throws(() => new Store(root), /schema-mismatch/);
|
||||
const store = new Store(root, { migrateSchema: true });
|
||||
try {
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3j');
|
||||
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3i');
|
||||
assert.deepEqual({ ...store.get('SELECT to_name FROM messages') }, { to_name: null });
|
||||
assert.equal(store.get('SELECT count(*) AS n FROM usage_records').n, 0);
|
||||
assert.equal(store.get('SELECT name FROM instance_names').name, 'Builder');
|
||||
|
||||
@@ -10,7 +10,8 @@ import { checkVars } from "./vars.mjs";
|
||||
import { parseCredentialRef } from "./credentials.mjs";
|
||||
import { loadRole } from "./role.mjs";
|
||||
import { checkPlugins } from "../../plugins/src/kinds.mjs";
|
||||
import { forkProblem, giteaName, parseGiteaUrl, HOST_SERVICE } from "./vocabulary.mjs";
|
||||
import { forkProblem, giteaName, parseGiteaUrl, HOST_SERVICE, HOST_PREFIX } from "./vocabulary.mjs";
|
||||
import { checkHosts, checkHostGrants } from "./hosts.mjs";
|
||||
import {
|
||||
requireObject, rejectUnknownKeys, requireId, requirePositiveInt, requireString, requireDistinctList, readJsonFile, deepFreeze,
|
||||
} from "./util.mjs";
|
||||
@@ -102,11 +103,7 @@ function checkTracker(tracker, file) {
|
||||
return { sync, labels };
|
||||
}
|
||||
|
||||
// Row 234 (#1740, r6 S1): hosts and host grants are bus store records made by a human-approved grant,
|
||||
// so the business file can't name them. The key stays known so the refusal says where they went.
|
||||
const HOSTS_MOVED = "lives in the bus store now: enroll a host with a host-enroll grant (mosaic hosts status)";
|
||||
|
||||
function checkRoles(roles, file, rolesDir, business) {
|
||||
function checkRoles(roles, file, rolesDir, business, hosts) {
|
||||
const where = `${file} roles`;
|
||||
requireObject(roles, where);
|
||||
if (Object.keys(roles).length === 0) refuse(`${where} must declare at least one role instance`);
|
||||
@@ -124,9 +121,14 @@ function checkRoles(roles, file, rolesDir, business) {
|
||||
definitions[entry.definition] = definition;
|
||||
if (definition.roleVersion !== 2) refuse(`${at}: role definition ${entry.definition} is version 1 and can't back a business role instance`);
|
||||
const services = definition.credentials.map((c) => c.service);
|
||||
if (entry.hosts !== undefined) refuse(`${at}.hosts ${HOSTS_MOVED}`);
|
||||
// A host.<id> key reference comes from the agent's grant record at bus start (r6 X4), never from here.
|
||||
const kinds = services.filter((s) => s !== HOST_SERVICE);
|
||||
// Row 210 (#1707, design D3 steps 1 and 3): a host grant needs a definition that declares the
|
||||
// host credential, and each granted host needs its own host.<id> key reference.
|
||||
let hostGrants = null;
|
||||
if (entry.hosts !== undefined) {
|
||||
if (!services.includes(HOST_SERVICE)) refuse(`${at}.hosts needs a role definition that declares the host credential`);
|
||||
hostGrants = checkHostGrants(entry.hosts, hosts, `${at}.hosts`);
|
||||
}
|
||||
const kinds = [...services.filter((s) => s !== HOST_SERVICE), ...Object.keys(hostGrants ?? {}).map((id) => HOST_PREFIX + id)];
|
||||
let tracker = null;
|
||||
if (services.includes("vikunja")) {
|
||||
if (entry.tracker === undefined) refuse(`${at} needs "tracker" with its Vikunja bot, because ${entry.definition} uses vikunja`);
|
||||
@@ -149,6 +151,7 @@ function checkRoles(roles, file, rolesDir, business) {
|
||||
// Row 199 (#1697, design D8, D9 step 2): identity and publish are optional, so today's files
|
||||
// load unchanged. publish needs both an identity and a gitea credential; the project ids it
|
||||
// names are checked against projects once those are read.
|
||||
if (hostGrants) out[instance].hosts = hostGrants;
|
||||
if (entry.identity !== undefined) out[instance].identity = checkIdentity(entry.identity, `${at}.identity`);
|
||||
if (entry.publish !== undefined) {
|
||||
// N10: one message for both, naming the grant.
|
||||
@@ -204,8 +207,8 @@ export function validateBusinessDocument(document, file, { rolesDir }) {
|
||||
}
|
||||
const human = requireId(document.human, "business human");
|
||||
const vars = checkVars(document.vars ?? {}, "business", `${file} vars`);
|
||||
if (document.hosts !== undefined) refuse(`${file} hosts ${HOSTS_MOVED}`);
|
||||
const { roles, definitions } = checkRoles(document.roles, file, rolesDir, document.id);
|
||||
const hosts = checkHosts(document.hosts, `${file} hosts`);
|
||||
const { roles, definitions } = checkRoles(document.roles, file, rolesDir, document.id, hosts);
|
||||
|
||||
requireObject(document.arbiters, `${file} arbiters`);
|
||||
rejectUnknownKeys(document.arbiters, ["delivery", "technical"], `${file} arbiters`);
|
||||
@@ -265,6 +268,7 @@ export function validateBusinessDocument(document, file, { rolesDir }) {
|
||||
|
||||
return deepFreeze({
|
||||
businessVersion: 1, id: document.id, file, human, arbiters, projects, vars, tracker, roles, launch, plugins, definitions,
|
||||
...(document.hosts === undefined ? {} : { hosts }),
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -1,19 +1,19 @@
|
||||
// Row 210 (#1707, design D3 and D6): the address, user and host key checks, and the argv allowlist the
|
||||
// broker matches each host.run against. Row 234 (#1740, r6 S1) moved the host registry and the grants
|
||||
// out of the business file into bus store records. The enrollment raise checks its fields with these,
|
||||
// and row b's host grant raise checks its commands with checkCommands.
|
||||
// Row 210 (#1707, design D3 and D6): the host registry in the business file, an agent's host
|
||||
// grants, and the argv allowlist the broker matches each host.run against. Hand edits only: the
|
||||
// chief's organization.set and the human's instances.set take neither key.
|
||||
|
||||
import { isIP } from "node:net";
|
||||
import { refuse } from "./errors.mjs";
|
||||
import { ARGV_MAX } from "./vocabulary.mjs";
|
||||
import { HOST_ID, ARGV_MAX } from "./vocabulary.mjs";
|
||||
// Row 210 r4 (#1707): the argv check lives in vocabulary.mjs, which the worker sandbox mounts
|
||||
// and which imports nothing, so the broker can read it. Re-exported so the D6 checks keep one
|
||||
// business home here.
|
||||
export { ARGV_MAX, ARG_BYTES, ARGV_BYTES, validArgv, commandAllowed } from "./vocabulary.mjs";
|
||||
import { requireObject, rejectUnknownKeys, isPlainObject } from "./util.mjs";
|
||||
|
||||
const DNS = /^[a-z0-9]([a-z0-9.-]{0,251}[a-z0-9])?$/;
|
||||
export const USER = /^[a-z_][a-z0-9_-]{0,31}$/;
|
||||
export const HOST_KEY = /^(ssh-ed25519|ecdsa-sha2-nistp256|ssh-rsa) [A-Za-z0-9+/]+={0,2}$/;
|
||||
const USER = /^[a-z_][a-z0-9_-]{0,31}$/;
|
||||
const HOST_KEY = /^(ssh-ed25519|ecdsa-sha2-nistp256|ssh-rsa) [A-Za-z0-9+/]+={0,2}$/;
|
||||
|
||||
// D6: a prefix starting with one of these, by basename, would turn the grant back into "any".
|
||||
// The list can't be complete. Under PRD 1.7 the control is the human's decision on each call (D4).
|
||||
@@ -49,7 +49,7 @@ export function ownMachine(address) {
|
||||
return address === "localhost" || address.endsWith(".localhost");
|
||||
}
|
||||
|
||||
export function checkAddress(address, where) {
|
||||
function checkAddress(address, where) {
|
||||
if (typeof address !== "string") refuse(`${where} must be an IP address or a DNS name`);
|
||||
const version = isIP(address);
|
||||
if (version === 6 && address.includes("%")) refuse(`${where} must not carry an IPv6 zone`);
|
||||
@@ -63,6 +63,32 @@ export function checkAddress(address, where) {
|
||||
return address;
|
||||
}
|
||||
|
||||
// The top-level "hosts" key: { <hostId>: { address, port, user, hostKeys } }.
|
||||
export function checkHosts(hosts, where) {
|
||||
if (hosts === undefined) return {};
|
||||
requireObject(hosts, where);
|
||||
const out = {};
|
||||
for (const [id, entry] of Object.entries(hosts)) {
|
||||
if (!HOST_ID.test(id)) refuse(`${where} host id ${JSON.stringify(id)} must match ${HOST_ID}`);
|
||||
const at = `${where}.${id}`;
|
||||
requireObject(entry, at);
|
||||
rejectUnknownKeys(entry, ["address", "port", "user", "hostKeys"], at);
|
||||
const address = checkAddress(entry.address, `${at}.address`);
|
||||
if (!Number.isInteger(entry.port) || entry.port < 1 || entry.port > 65535) refuse(`${at}.port must be 1 to 65535`);
|
||||
if (typeof entry.user !== "string" || !USER.test(entry.user)) refuse(`${at}.user must match ${USER}`);
|
||||
if (!Array.isArray(entry.hostKeys) || entry.hostKeys.length === 0) refuse(`${at}.hostKeys must be a non-empty list`);
|
||||
const keys = entry.hostKeys.map((key, i) => {
|
||||
if (typeof key !== "string" || !HOST_KEY.test(key)) {
|
||||
refuse(`${at}.hostKeys[${i}] must be "<type> <base64>" with type ssh-ed25519, ecdsa-sha2-nistp256 or ssh-rsa, and no comment`);
|
||||
}
|
||||
return key;
|
||||
});
|
||||
if (new Set(keys).size !== keys.length) refuse(`${at}.hostKeys lists a key twice`);
|
||||
out[id] = { address, port: entry.port, user: entry.user, hostKeys: keys };
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// D6: an argv allowlist and nothing else. No wildcard: "any", "*", {} and [] all refuse.
|
||||
export function checkCommands(commands, where) {
|
||||
if (!Array.isArray(commands)) refuse(`${where} must be a list of argv prefixes; there is no wildcard grant`);
|
||||
@@ -84,3 +110,18 @@ export function checkCommands(commands, where) {
|
||||
return [...prefix];
|
||||
});
|
||||
}
|
||||
|
||||
// A role entry's "hosts": { <hostId>: { commands } }, each id in the registry.
|
||||
export function checkHostGrants(grants, registry, where) {
|
||||
requireObject(grants, where);
|
||||
if (Object.keys(grants).length === 0) refuse(`${where} must name at least one host`);
|
||||
const out = {};
|
||||
for (const [id, grant] of Object.entries(grants)) {
|
||||
const at = `${where}.${id}`;
|
||||
if (!Object.hasOwn(registry, id)) refuse(`${at}: host ${id} isn't in the business file's hosts`);
|
||||
if (!isPlainObject(grant)) refuse(`${at} must be an object`);
|
||||
rejectUnknownKeys(grant, ["commands"], at);
|
||||
out[id] = { commands: checkCommands(grant.commands, `${at}.commands`) };
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
@@ -1,12 +1,13 @@
|
||||
// Row 210 (#1707, design D3 and D6): the address checks and the argv allowlist. Row 234 (#1740, r6 S1)
|
||||
// moved the registry and the grants to bus store records, so the business file refuses them.
|
||||
// Row 210 (#1707, design D3 and D6): the host registry, an agent's host grant and its argv
|
||||
// allowlist, checked at business load. Tests T1, T2 and T26 of the design.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { BusinessError, validateBusinessDocument } from "../src/index.mjs";
|
||||
import { HOST_KEY as HOST_KEY_PATTERN, REFUSED_COMMANDS, USER, checkAddress, checkCommands, commandAllowed, ownMachine, validArgv } from "../src/hosts.mjs";
|
||||
import { businessDoc, rolesCopy, tmp, tokenFile, FAR_DATE } from "./helpers.mjs";
|
||||
import { resolveInstance } from "../src/resolve.mjs";
|
||||
import { REFUSED_COMMANDS, commandAllowed, ownMachine, validArgv } from "../src/hosts.mjs";
|
||||
import { businessDoc, rolesCopy, systemFor, tmp, tokenFile, FAR_DATE } from "./helpers.mjs";
|
||||
|
||||
const HOST_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOv6Rq3Mvo9w2j3eYvYbqX0J1pJ0x9m8Zr0w1kJ8p0Qn";
|
||||
|
||||
@@ -18,7 +19,8 @@ function refuses(fn, pattern) {
|
||||
});
|
||||
}
|
||||
|
||||
// The fixture with coder's definition declaring the host credential and no hosts.
|
||||
// The fixture with coder's definition declaring the host credential, one registered host, and
|
||||
// coder granted it with one command.
|
||||
function check(mutate, { declare = true } = {}) {
|
||||
const root = tmp();
|
||||
const rolesDir = rolesCopy(root);
|
||||
@@ -29,44 +31,62 @@ function check(mutate, { declare = true } = {}) {
|
||||
writeFileSync(file, JSON.stringify(coder));
|
||||
}
|
||||
const doc = businessDoc(root);
|
||||
doc.hosts = { web1: { address: "10.1.1.53", port: 22, user: "localadmin", hostKeys: [HOST_KEY] } };
|
||||
doc.roles.coder.hosts = { web1: { commands: [["systemctl", "status"]] } };
|
||||
doc.roles.coder.credentials["host.web1"] = { file: tokenFile(join(root, "secrets"), "coder-host.key"), rotateBy: FAR_DATE };
|
||||
mutate?.(doc, root);
|
||||
return validateBusinessDocument(doc, join(root, "businesses", "acme.json"), { rolesDir });
|
||||
}
|
||||
|
||||
// Row 234 (#1740, r6 S1): hosts and grants are bus store records, so the business file refuses both keys
|
||||
// and any host.<id> key reference. The control loads.
|
||||
test("S1: the business file refuses hosts at the top level and in a role, and host.<id> references", () => {
|
||||
assert.equal("hosts" in check(), false);
|
||||
assert.equal("hosts" in check().roles.coder, false);
|
||||
const entry = { address: "10.1.1.53", port: 22, user: "localadmin", hostKeys: [HOST_KEY] };
|
||||
refuses(() => check((d) => { d.hosts = { web1: entry }; }), /acme\.json hosts lives in the bus store/);
|
||||
refuses(() => check((d) => { d.hosts = {}; }), /hosts lives in the bus store/);
|
||||
refuses(() => check((d) => { d.roles.coder.hosts = { web1: { commands: [["systemctl", "status"]] } }; }), /roles\.coder\.hosts lives in the bus store/);
|
||||
refuses(() => check((d) => { d.roles.coder.hosts = {}; }, { declare: false }), /roles\.coder\.hosts lives in the bus store/);
|
||||
const key = (d, root) => { d.roles.coder.credentials["host.web1"] = { file: tokenFile(join(root, "secrets"), "coder-host.key"), rotateBy: FAR_DATE }; };
|
||||
refuses(() => check(key), /host\.web1/);
|
||||
test("T1: a registry and a grant load, and the resolved session credentials leave the host key out", () => {
|
||||
const business = check();
|
||||
assert.deepEqual(business.hosts.web1, { address: "10.1.1.53", port: 22, user: "localadmin", hostKeys: [HOST_KEY] });
|
||||
assert.deepEqual(business.roles.coder.hosts, { web1: { commands: [["systemctl", "status"]] } });
|
||||
assert.ok(business.roles.coder.credentials["host.web1"]);
|
||||
assert.equal(check((d) => { d.hosts.web1.address = "host.example"; }).hosts.web1.address, "host.example");
|
||||
// A file without hosts loads with no hosts key at all.
|
||||
assert.equal("hosts" in check((d) => { delete d.hosts; delete d.roles.coder.hosts; delete d.roles.coder.credentials["host.web1"]; }), false);
|
||||
});
|
||||
|
||||
test("T1: address, user and host key checks, which the enrollment raise uses", () => {
|
||||
test("T1: registry field checks name the key path", () => {
|
||||
refuses(() => check((d) => { d.hosts = []; }), /hosts/);
|
||||
refuses(() => check((d) => { d.hosts = { "Web 1": d.hosts.web1 }; }), /host id/);
|
||||
refuses(() => check((d) => { d.hosts.web1.extra = 1; }), /hosts\.web1/);
|
||||
for (const port of [0, 65536, "22", 1.5]) refuses(() => check((d) => { d.hosts.web1.port = port; }), /hosts\.web1\.port/);
|
||||
for (const user of ["Root", "a b", "", 7]) refuses(() => check((d) => { d.hosts.web1.user = user; }), /hosts\.web1\.user/);
|
||||
refuses(() => check((d) => { d.hosts.web1.hostKeys = []; }), /hosts\.web1\.hostKeys/);
|
||||
refuses(() => check((d) => { d.hosts.web1.hostKeys = [`${HOST_KEY} root@web1`]; }), /hosts\.web1\.hostKeys\[0\]/);
|
||||
refuses(() => check((d) => { d.hosts.web1.hostKeys = ["ssh-dss AAAA"]; }), /hosts\.web1\.hostKeys\[0\]/);
|
||||
refuses(() => check((d) => { d.hosts.web1.hostKeys = [HOST_KEY, HOST_KEY]; }), /twice/);
|
||||
for (const address of ["bad_name", "a..b", "-a.b", "fe80::1%eth0", 7])
|
||||
refuses(() => check((d) => { d.hosts.web1.address = address; }), /hosts\.web1\.address/);
|
||||
});
|
||||
|
||||
test("T1: every own-machine form refuses, and the controls load", () => {
|
||||
const own = ["127.0.0.2", "0.0.0.0", "::1", "::", "0:0:0:0:0:0:0:1", "::ffff:127.0.0.1", "::ffff:0.0.0.0", "localhost", "a.localhost",
|
||||
"127.1", "2130706433", "0177.0.0.1", "0x7f.1", "bad_name", "a..b", "-a.b", "fe80::1%eth0", 7];
|
||||
for (const address of own) refuses(() => checkAddress(address, "grant.address"), /grant\.address/);
|
||||
for (const address of ["10.1.1.53", "host.example", "2001:db8::1"]) assert.equal(checkAddress(address, "grant.address"), address);
|
||||
"127.1", "2130706433", "0177.0.0.1", "0x7f.1"];
|
||||
for (const address of own) refuses(() => check((d) => { d.hosts.web1.address = address; }), /hosts\.web1\.address/);
|
||||
for (const address of ["10.1.1.53", "host.example", "2001:db8::1"]) assert.equal(check((d) => { d.hosts.web1.address = address; }).hosts.web1.address, address);
|
||||
assert.equal(ownMachine("::ffff:7f00:1"), true);
|
||||
assert.equal(ownMachine("::ffff:10.0.0.1"), false);
|
||||
for (const user of ["Root", "a b", ""]) assert.equal(USER.test(user), false);
|
||||
assert.equal(USER.test("localadmin"), true);
|
||||
for (const key of [`${HOST_KEY} root@web1`, "ssh-dss AAAA"]) assert.equal(HOST_KEY_PATTERN.test(key), false);
|
||||
assert.equal(HOST_KEY_PATTERN.test(HOST_KEY), true);
|
||||
});
|
||||
|
||||
test("T2: a grant naming an unknown host, or on a definition without host, refuses at roles.<agent>.hosts", () => {
|
||||
refuses(() => check((d) => { d.roles.coder.hosts = { web2: { commands: [["ls"]] } }; }), /roles\.coder\.hosts\.web2: host web2 isn't/);
|
||||
refuses(() => check(null, { declare: false }), /roles\.coder\.hosts needs a role definition that declares the host credential/);
|
||||
refuses(() => check((d) => { d.roles.coder.hosts = {}; }), /roles\.coder\.hosts must name at least one host/);
|
||||
refuses(() => check((d) => { d.roles.coder.hosts.web1.extra = 1; }), /roles\.coder\.hosts\.web1/);
|
||||
// The key reference is for a granted host only.
|
||||
refuses(() => check((d) => { d.roles.coder.credentials["host.web2"] = d.roles.coder.credentials["host.web1"]; }), /host\.web2/);
|
||||
});
|
||||
|
||||
test("T26: no wildcard and no shell, runner or interpreter as a prefix", () => {
|
||||
const at = /grant\.commands/;
|
||||
const at = /roles\.coder\.hosts\.web1\.commands/;
|
||||
for (const commands of ["any", "*", {}, [], [[]], [[7]], [["ls", ""]], [["ls"], ["ls"]], [["ls", "*"]], [["ls", "a?"]], [["ls", "[a]"]], [["bin/ls"]], [["ls\0"]]])
|
||||
refuses(() => checkCommands(commands, "grant.commands"), at);
|
||||
refuses(() => check((d) => { d.roles.coder.hosts.web1.commands = commands; }), at);
|
||||
const names = [...REFUSED_COMMANDS, "python", "python3", "python3.12", "awk", "gawk", "mawk", "nawk", "php", "php8.2", "/bin/bash", "/usr/bin/env"];
|
||||
for (const name of names) refuses(() => checkCommands([[name, "x"]], "grant.commands"), at);
|
||||
for (const name of ["systemctl", "/usr/bin/journalctl", "tee"]) assert.deepEqual(checkCommands([[name]], "grant.commands"), [[name]]);
|
||||
for (const name of names) refuses(() => check((d) => { d.roles.coder.hosts.web1.commands = [[name, "x"]]; }), at);
|
||||
for (const name of ["systemctl", "/usr/bin/journalctl", "tee"]) assert.ok(check((d) => { d.roles.coder.hosts.web1.commands = [[name]]; }));
|
||||
});
|
||||
|
||||
test("D6: argv shape and the literal prefix match", () => {
|
||||
@@ -81,3 +101,11 @@ test("D6: argv shape and the literal prefix match", () => {
|
||||
assert.equal(commandAllowed(["systemctl", "restart", "nginx"], commands), false);
|
||||
assert.equal(commandAllowed(["systemctl", "status*"], commands), false);
|
||||
});
|
||||
|
||||
test("D5: resolveInstance leaves host.<id> out of the session's credentials", () => {
|
||||
let root;
|
||||
const business = check((d, r) => { root = r; });
|
||||
const resolved = resolveInstance({ system: systemFor(root), business, instance: "coder" });
|
||||
assert.equal("host.web1" in resolved.credentials, false);
|
||||
assert.ok(resolved.credentials.gitea);
|
||||
});
|
||||
|
||||
+10
-9
@@ -774,6 +774,16 @@ Each live run requires Mos's go; agents build and test fixtures only, and
|
||||
so no PM turn may be in flight. Queue writes stay on the canonical
|
||||
scripts/mosaic.
|
||||
|
||||
On a host whose units are root's (row 232), `--units external` makes
|
||||
stack-release.sh read the system units and start or stop none. `activate`
|
||||
and `rollback --to` stage the copy and leave the `AWAITING-UNITS` marker;
|
||||
root installs and starts the units, then `confirm` gates the start and
|
||||
writes the pointer, `revert` undoes a staged run, and `gate --since` checks
|
||||
a copy with no open activation. stack-copy.sh adds `copy --bundle FILE SHA`
|
||||
for a host with no refactor checkout, `wrapper ID --create` for the first
|
||||
wrapper, and `check --units external`. Details are in
|
||||
[TOOLS.md, Root-owned units](../../docs/TOOLS.md#root-owned-units---units-external-row-232).
|
||||
|
||||
The human commands and `--tester` run only from the tree the bus runs from
|
||||
(row 130, #1625). The broker checks the path of the calling human-cli.mjs or
|
||||
tester-cli.mjs against its own. On a host with a pinned copy that tree is
|
||||
@@ -1036,15 +1046,6 @@ entries; `delete` refuses `agent-running` while the agent holds a claim. With
|
||||
`--tester` only `status` and `revoke` run. After a rotate or revoke, delete
|
||||
the old token in Gitea or Vikunja yourself; the bus only forgets it.
|
||||
|
||||
## hosts (row 234)
|
||||
|
||||
`mosaic hosts [status] [--json] [--tester] [--business ID]` prints one tab-separated line per
|
||||
enrolled host: id, state, `user@address:port`, `sudo` or `no-sudo`, the last check time or
|
||||
`unchecked`, and any host lines a failed step left. `--json` prints the bus rows. `check HOST`
|
||||
runs the bus's `hosts.check` and `remove HOST` its `hosts.remove`, each printing the JSON result.
|
||||
Enrollment itself is a `host-enroll` grant the chief raises and the human approves, not a CLI
|
||||
command. A usage error exits 4 and a bus refusal 2.
|
||||
|
||||
## Token usage
|
||||
|
||||
`mosaic usage [--business ID] [--json] [--tester]` reads the bus's accounting, without
|
||||
|
||||
@@ -18,7 +18,6 @@ import { resolveThread, threadCommand, threadMembers, threadsCommand, THREAD_TAL
|
||||
import { limitsCommand, LIMITS_USAGE } from "./limits.mjs";
|
||||
import { pluginsCommand, PLUGINS_USAGE } from "./plugins.mjs";
|
||||
import { credentialsCommand, CREDENTIALS_USAGE } from "./credentials.mjs";
|
||||
import { hostsCommand, HOSTS_USAGE } from "./hosts.mjs";
|
||||
import { instancesCommand, INSTANCES_USAGE } from "./instances.mjs";
|
||||
import { createChannelCaller } from "../../plugins/src/channel.mjs";
|
||||
import { PluginError } from "../../plugins/src/kinds.mjs";
|
||||
@@ -41,7 +40,6 @@ export const USAGE = `usage:
|
||||
${LIMITS_USAGE}
|
||||
${PLUGINS_USAGE}
|
||||
${CREDENTIALS_USAGE}
|
||||
${HOSTS_USAGE}
|
||||
${INSTANCES_USAGE}
|
||||
${PROJECT_USAGE}
|
||||
${PROJECT_BUS_USAGE}
|
||||
@@ -582,8 +580,6 @@ export async function main(argv, io = { env: process.env, stdin: process.stdin,
|
||||
if (verb === "plugins") return pluginsCommand(rest, io, (p) => callFor(p, io, deps));
|
||||
// Row 199 (#1697): issue and rotate read the token on stdin.
|
||||
if (verb === "credentials") return credentialsCommand(rest, io, (p) => callFor(p, io, deps));
|
||||
// Row 234 (#1740, r6 E4): the host records.
|
||||
if (verb === "hosts") return hostsCommand(rest, io, (p) => callFor(p, io, deps));
|
||||
// Row 165 (#1662): scripts/mosaic sends only `business instances` and `business instance` here.
|
||||
if (verb === "business") return instancesCommand(rest, io, (p) => callFor(p, io, deps));
|
||||
if (verb === "talk") return talk(rest, io, deps);
|
||||
|
||||
@@ -1,44 +0,0 @@
|
||||
// `mosaic hosts` (row 234, #1740, r6 E4): the broker's hosts.status, hosts.check and hosts.remove. The
|
||||
// human runs all three. With --tester the tester's session runs them. check logs in as mosaic-installer
|
||||
// and runs `list` on the host, then marks the host ready. remove drops a host no grant uses. Nothing here
|
||||
// reads a key: the answers carry fingerprints only.
|
||||
import { CliError } from "./errors.mjs";
|
||||
|
||||
export const HOSTS_USAGE = "mosaic hosts [status] [--json] [--tester] [--business ID] | check|remove HOST [--tester] [--business ID]";
|
||||
|
||||
function parse(args) {
|
||||
const out = { positional: [], flags: new Set(), values: {} };
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
const arg = args[i];
|
||||
if (arg === "--json" || arg === "--tester") out.flags.add(arg);
|
||||
else if (arg === "--business") {
|
||||
if (!args[i + 1] || Object.hasOwn(out.values, arg)) throw new CliError(HOSTS_USAGE, 4);
|
||||
out.values[arg] = args[++i];
|
||||
} else if (arg.startsWith("--")) throw new CliError(HOSTS_USAGE, 4);
|
||||
else out.positional.push(arg);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
const line = (h) =>
|
||||
[h.host, h.state, `${h.user}@${h.address}:${h.port}`, h.sudo ? "sudo" : "no-sudo", h.checkedAt ?? "unchecked",
|
||||
...(h.hostLineLeft.length ? [`line left: ${h.hostLineLeft.join(" ")}`] : [])].join("\t");
|
||||
|
||||
export async function hostsCommand(args, io, callFor) {
|
||||
const parsed = parse(args);
|
||||
const [verb = "status", host, ...extra] = parsed.positional;
|
||||
const shape = { status: host === undefined, check: host !== undefined, remove: host !== undefined };
|
||||
if (extra.length || !shape[verb] || (parsed.flags.has("--json") && verb !== "status")) throw new CliError(HOSTS_USAGE, 4);
|
||||
try {
|
||||
const call = callFor(parsed);
|
||||
if (verb === "status") {
|
||||
const rows = await call("hosts.status", {});
|
||||
io.stdout.write(parsed.flags.has("--json") ? JSON.stringify(rows, null, 2) + "\n" : rows.map(line).join("\n") + (rows.length ? "\n" : ""));
|
||||
return;
|
||||
}
|
||||
io.stdout.write(JSON.stringify(await call(`hosts.${verb}`, { host }), null, 2) + "\n");
|
||||
} catch (error) {
|
||||
if (error instanceof CliError) throw error;
|
||||
throw new CliError(error.message, 2);
|
||||
}
|
||||
}
|
||||
@@ -1,74 +0,0 @@
|
||||
// Row 234 (#1740, r6 E4): `mosaic hosts status|check|remove`. A fake transport records the calls; no bus
|
||||
// runs here. The bus side of each verb is in packages/bus/tests/grant-enroll.test.mjs.
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { main } from "../src/cli.mjs";
|
||||
import { io, tmp } from "./helpers.mjs";
|
||||
|
||||
const MOSAIC = fileURLToPath(new URL("../../../scripts/mosaic", import.meta.url));
|
||||
const ROWS = [
|
||||
{ host: "web1", address: "10.1.1.53", port: 22, user: "deploy", sudo: true, state: "ready", hostKeys: ["SHA256:a"],
|
||||
installerFingerprint: "SHA256:b", checkedAt: "2026-10-21T00:00:00.000Z", hostLineLeft: [] },
|
||||
{ host: "web2", address: "10.1.1.54", port: 2222, user: "app", sudo: false, state: "pending-install", hostKeys: ["SHA256:c"],
|
||||
installerFingerprint: "SHA256:d", checkedAt: null, hostLineLeft: ["mosaic-" + "0".repeat(32)] },
|
||||
];
|
||||
|
||||
function deps(calls) {
|
||||
const answer = (who) => (business) => async (verb, args) => {
|
||||
calls.push({ who, business, verb, args });
|
||||
if (verb === "hosts.status") return ROWS;
|
||||
if (verb === "hosts.check") return { host: args.host, state: "ready", checkedAt: "2026-10-21T00:00:00.000Z" };
|
||||
return { host: args.host, removed: true };
|
||||
};
|
||||
return { system: { dataRoot: "/nonexistent" }, transport: answer("human"), testerTransport: answer("tester") };
|
||||
}
|
||||
|
||||
test("row 234: hosts status prints one line per host, and --json the rows", async () => {
|
||||
const calls = [];
|
||||
const x = io();
|
||||
await main(["hosts", "--business", "acme"], x, deps(calls));
|
||||
assert.deepEqual(calls, [{ who: "human", business: "acme", verb: "hosts.status", args: {} }]);
|
||||
assert.equal(x.out.text,
|
||||
"web1\tready\[email protected]:22\tsudo\t2026-10-21T00:00:00.000Z\n" +
|
||||
`web2\tpending-install\t[email protected]:2222\tno-sudo\tunchecked\tline left: mosaic-${"0".repeat(32)}\n`);
|
||||
const y = io();
|
||||
await main(["hosts", "status", "--json", "--tester", "--business", "acme"], y, deps(calls));
|
||||
assert.deepEqual(JSON.parse(y.out.text), ROWS);
|
||||
assert.equal(calls[1].who, "tester");
|
||||
});
|
||||
|
||||
test("row 234: hosts check and remove send the host id, as the human or the tester", async () => {
|
||||
const calls = [];
|
||||
const x = io();
|
||||
await main(["hosts", "check", "web1", "--business", "acme"], x, deps(calls));
|
||||
await main(["hosts", "remove", "web2", "--tester", "--business", "acme"], x, deps(calls));
|
||||
assert.deepEqual(calls.map(({ who, verb, args }) => ({ who, verb, args })), [
|
||||
{ who: "human", verb: "hosts.check", args: { host: "web1" } },
|
||||
{ who: "tester", verb: "hosts.remove", args: { host: "web2" } },
|
||||
]);
|
||||
});
|
||||
|
||||
test("row 234: a malformed hosts command is a usage error and calls nothing", async () => {
|
||||
const calls = [];
|
||||
for (const argv of [["hosts", "check"], ["hosts", "remove"], ["hosts", "status", "web1"], ["hosts", "check", "web1", "web2"],
|
||||
["hosts", "purge", "web1"], ["hosts", "check", "web1", "--json"], ["hosts", "--business"], ["hosts", "--force"]])
|
||||
await assert.rejects(main([...argv], io(), deps(calls)), (e) => (assert.equal(e.exitCode, 4, argv.join(" ")), assert.match(e.message, /mosaic hosts/), true));
|
||||
assert.equal(calls.length, 0);
|
||||
});
|
||||
|
||||
test("row 234: a bus refusal exits 2 with its code", async () => {
|
||||
const refusing = { system: { dataRoot: "/nonexistent" }, transport: () => async () => { throw new Error("host-in-use"); } };
|
||||
await assert.rejects(main(["hosts", "remove", "web1", "--business", "acme"], io(), refusing), (e) => (assert.equal(e.exitCode, 2), assert.match(e.message, /host-in-use/), true));
|
||||
});
|
||||
|
||||
test("row 234: scripts/mosaic sends hosts to packages/cli, not the seat CLI", (t) => {
|
||||
const root = tmp(t, "mosaic-cli-hosts-route-");
|
||||
const env = { PATH: process.env.PATH, HOME: join(root, "home"), MOSAIC_CONFIG: join(root, "none.json") };
|
||||
const r = spawnSync(MOSAIC, ["hosts", "purge", "web1"], { env, encoding: "utf8" });
|
||||
assert.equal(r.status, 4);
|
||||
assert.match(r.stderr, /^mosaic: mosaic hosts \[status\]/);
|
||||
assert.doesNotMatch(r.stderr, /mosaic launch </);
|
||||
});
|
||||
@@ -1,7 +1,8 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {execFileSync,spawnSync,spawn} from 'node:child_process';
|
||||
import {symlinkSync,unlinkSync,mkdtempSync,mkdirSync,writeFileSync,readFileSync,cpSync,rmSync,chmodSync,existsSync,statSync} from 'node:fs';
|
||||
import {symlinkSync,unlinkSync,mkdtempSync,mkdirSync,writeFileSync,readFileSync,cpSync,rmSync,chmodSync,existsSync,statSync,readdirSync,lstatSync,readlinkSync} from 'node:fs';
|
||||
import {createHash} from 'node:crypto';
|
||||
import {tmpdir} from 'node:os';
|
||||
import {join,dirname} from 'node:path';
|
||||
import {fileURLToPath} from 'node:url';
|
||||
@@ -227,3 +228,108 @@ test('inherited drop-in unreadable directory or file refuses before writing',t=>
|
||||
unlinkSync(dropdir);mkdirSync(dropdir);const file=join(dropdir,'broken.conf');symlinkSync(join(s.root,'absent-file'),file);
|
||||
const broken=s.run('bus-service.sh',['install','--dir',units,'--no-reload']);assert.equal(broken.status,1);assert.ok(broken.stderr.includes(file));assert.equal(readFileSync(target,'utf8'),'previous unit bytes');
|
||||
});
|
||||
// Row 232: every file of a copy with its mode and bytes, for the byte-identical comparison.
|
||||
function tree(dir){
|
||||
const out=[];
|
||||
const walk=rel=>{for(const e of readdirSync(join(dir,rel),{withFileTypes:true}).sort((a,b)=>a.name<b.name?-1:1)){
|
||||
const r=rel?`${rel}/${e.name}`:e.name,st=lstatSync(join(dir,r));
|
||||
if(e.isDirectory()){out.push(`d ${(st.mode&0o7777).toString(8)} ${r}`);walk(r);}
|
||||
else if(e.isSymbolicLink())out.push(`l ${r} ${readlinkSync(join(dir,r))}`);
|
||||
else out.push(`f ${(st.mode&0o7777).toString(8)} ${r} ${createHash('sha256').update(readFileSync(join(dir,r))).digest('hex')}`);
|
||||
}};
|
||||
walk('');return out;
|
||||
}
|
||||
test('row 232: copy --bundle copies a SHA the bundle\'s refactor reaches, byte-identical to copy SHA, and refuses the rest',t=>{
|
||||
const s=setup(t),tmp=join(s.root,'t');mkdirSync(tmp);s.env.TMPDIR=tmp;
|
||||
const good=join(s.root,'good.bundle');git(s.repo,'bundle','create','-q',good,'refactor');
|
||||
const a=join(s.root,'copies-a'),b=join(s.root,'copies-b');
|
||||
let r=s.run('stack-copy.sh',['copy',s.id,'--copies',a]);assert.equal(r.status,0,r.stderr);
|
||||
r=s.run('stack-copy.sh',['copy','--bundle',good,s.id,'--copies',b]);assert.equal(r.status,0,r.stderr);
|
||||
const sum=createHash('sha256').update(readFileSync(good)).digest('hex'),release=git(s.repo,'show',`${s.id}:RELEASE`);
|
||||
assert.equal(r.stdout,`copied: ${join(b,s.id)}\nrelease: ${release}\nbundle sha256: ${sum}\n`);
|
||||
assert.deepEqual(tree(join(b,s.id)),tree(join(a,s.id)));
|
||||
assert.equal(statSync(join(b,s.id)).mode&0o777,0o555);
|
||||
const left=()=>readdirSync(tmp).filter(f=>f.startsWith('tmp.'));
|
||||
assert.deepEqual(left(),[],'the bundle\'s temporary repository is removed');
|
||||
// A second run finds the copy and still names the release.
|
||||
r=s.run('stack-copy.sh',['copy','--bundle',good,s.id,'--copies',b]);assert.equal(r.status,0,r.stderr);
|
||||
assert.equal(r.stdout,`exists: ${join(b,s.id)}\nrelease: ${release}\nbundle sha256: ${sum}\n`);
|
||||
// A relative bundle path works from the caller's directory.
|
||||
const c=join(s.root,'copies-c');
|
||||
r=spawnSync(join(s.repo,'scripts','stack-copy.sh'),['copy','--bundle','good.bundle',s.id,'--copies',c],{cwd:s.root,env:s.env,encoding:'utf8'});
|
||||
assert.equal(r.status,0,r.stderr);assert.deepEqual(tree(join(c,s.id)),tree(join(a,s.id)));
|
||||
// A commit on another branch of the bundle: the bundle carries it, refactor does not reach it.
|
||||
const side=git(s.repo,'commit-tree','-p',s.id,'-m','side',`${s.id}^{tree}`);git(s.repo,'update-ref','refs/heads/side',side);
|
||||
const both=join(s.root,'both.bundle');git(s.repo,'bundle','create','-q',both,'refactor','side');
|
||||
const only=join(s.root,'side.bundle');git(s.repo,'bundle','create','-q',only,'side');
|
||||
const corrupt=join(s.root,'corrupt.bundle'),bytes=readFileSync(good);bytes[bytes.length-30]^=0xff;writeFileSync(corrupt,bytes);
|
||||
const cut=join(s.root,'cut.bundle');writeFileSync(cut,readFileSync(good).subarray(0,200));
|
||||
const junk=join(s.root,'junk.bundle');writeFileSync(junk,'# v2 git bundle\nnot a ref line\n\n');
|
||||
const d=join(s.root,'copies-d');
|
||||
for(const [label,bundle,id,re] of [
|
||||
['not reachable',both,side,/not reachable from the bundle's refs\/heads\/refactor/],
|
||||
['no refactor',only,side,/no refs\/heads\/refactor/],
|
||||
// verify reads the header and prerequisites; a damaged pack fails the fetch's own check.
|
||||
['not a bundle',junk,s.id,/git bundle verify failed/],
|
||||
['corrupt pack',corrupt,s.id,/cannot fetch/],
|
||||
['truncated pack',cut,s.id,/cannot fetch/],
|
||||
['absent',join(s.root,'none.bundle'),s.id,/bundle is absent/],
|
||||
['short id',good,s.id.slice(0,12),/full 40-character commit id/],
|
||||
]){
|
||||
r=s.run('stack-copy.sh',['copy','--bundle',bundle,id,'--copies',d]);
|
||||
assert.equal(r.status,1,label);assert.match(r.stderr,re,label);assert.equal(r.stdout,'',label);
|
||||
assert.ok(!existsSync(join(d,id)),label);assert.deepEqual(left(),[],label);
|
||||
if(existsSync(d))assert.deepEqual(readdirSync(d),[],label);
|
||||
}
|
||||
const link=join(s.root,'link.bundle');symlinkSync(good,link);
|
||||
r=s.run('stack-copy.sh',['copy','--bundle',link,s.id,'--copies',d]);assert.equal(r.status,1);assert.match(r.stderr,/symlink/);
|
||||
for(const args of [['copy','--bundle',good,s.id,'--allow-worktree'],['copy','--bundle',good],['copy','--bundle'],['wrapper','--bundle',good,s.id],['check','--units','system'],['copy',s.id,'--units','external'],['copy',s.id,'--create']])
|
||||
assert.equal(s.run('stack-copy.sh',args).status,4,args.join(' '));
|
||||
});
|
||||
test('row 232: wrapper --create writes the first wrapper only, the same bytes wrapper ID writes, and refuses over any file',t=>{
|
||||
const s=setup(t);assert.equal(s.run('stack-copy.sh',['copy',s.id,'--copies',s.copies]).status,0);
|
||||
const wrapper=join(s.bin,'mosaic'),root=join(s.copies,s.id);
|
||||
let r=s.run('stack-copy.sh',['wrapper',s.id,'--copies',s.copies,'--bin',s.bin]);
|
||||
assert.equal(r.status,1);assert.match(r.stderr,/wrapper is absent/,'no --create keeps the missing-wrapper refusal');assert.ok(!existsSync(wrapper));
|
||||
r=s.run('stack-copy.sh',['wrapper',s.id,'--copies',s.copies,'--bin',s.bin,'--create']);assert.equal(r.status,0,r.stderr);
|
||||
assert.equal(r.stdout,`wrapper: ${root}\n`);
|
||||
const created=readFileSync(wrapper,'utf8');assert.equal(statSync(wrapper).mode&0o777,0o755);
|
||||
r=s.run('stack-copy.sh',['wrapper',s.id,'--copies',s.copies,'--bin',s.bin]);assert.equal(r.status,0,r.stderr);
|
||||
assert.equal(r.stdout,`wrapper: ${root}\n`);assert.equal(readFileSync(wrapper,'utf8'),created,'wrapper ID writes the same bytes');
|
||||
const foreign='#!/bin/sh\n# not ours\n';
|
||||
for(const [label,make] of [['own wrapper',()=>{}],['foreign file',()=>writeFileSync(wrapper,foreign)],['symlink',()=>{rmSync(wrapper);symlinkSync(join(s.root,'elsewhere'),wrapper);}]]){
|
||||
make();const before=existsSync(wrapper)||label==='symlink'?(label==='symlink'?readlinkSync(wrapper):readFileSync(wrapper,'utf8')):null;
|
||||
r=s.run('stack-copy.sh',['wrapper',s.id,'--copies',s.copies,'--bin',s.bin,'--create']);
|
||||
assert.equal(r.status,1,label);assert.match(r.stderr,/--create and a wrapper exists/,label);
|
||||
assert.equal(label==='symlink'?readlinkSync(wrapper):readFileSync(wrapper,'utf8'),before,label);
|
||||
assert.deepEqual(readdirSync(s.bin).filter(f=>f.startsWith('.')),[],`${label}: no temporary file left`);
|
||||
}
|
||||
// A named instance: --create writes mosaic-NAME as wrapper --instance does, once.
|
||||
assert.equal(s.run('bootstrap.sh',['--instance','alpha']).status,0);
|
||||
const named=join(s.bin,'mosaic-alpha');
|
||||
r=s.run('stack-copy.sh',['wrapper',s.id,'--instance','alpha','--copies',s.copies,'--bin',s.bin,'--create']);assert.equal(r.status,0,r.stderr);
|
||||
const first=readFileSync(named,'utf8');
|
||||
r=s.run('stack-copy.sh',['wrapper',s.id,'--instance','alpha','--copies',s.copies,'--bin',s.bin]);assert.equal(r.status,0,r.stderr);
|
||||
assert.equal(readFileSync(named,'utf8'),first);
|
||||
r=s.run('stack-copy.sh',['wrapper',s.id,'--instance','alpha','--copies',s.copies,'--bin',s.bin,'--create']);
|
||||
assert.equal(r.status,1);assert.match(r.stderr,/--create and a wrapper exists/);assert.equal(readFileSync(named,'utf8'),first);
|
||||
});
|
||||
test('row 232: check --units external reads the system units, the default reads the user units',t=>{
|
||||
const s=setup(t);assert.equal(s.run('stack-copy.sh',['copy',s.id,'--copies',s.copies]).status,0);
|
||||
assert.equal(s.run('stack-copy.sh',['wrapper',s.id,'--copies',s.copies,'--bin',s.bin,'--create']).status,0);
|
||||
const copy=join(s.copies,s.id),argv=join(s.root,'argv.log');
|
||||
s.fake(`echo "$*" >> '${argv}'; echo 'LoadState=loaded'; echo 'WorkingDirectory=${copy}'; case "$*" in *mosaic-bus*) echo 'ExecStart={ path=${copy}/scripts/mosaic ; argv[]=${copy}/scripts/mosaic bus start mosaic-stack --pm ; }';; *) echo 'ExecStart={ path=/usr/bin/node ; argv[]=/usr/bin/node ${copy}/packages/webui/src/cli.mjs serve ; }';; esac\n`);
|
||||
const want=`bus: ${copy}\nConsole: ${copy}\nCLI: ${copy}\n`;
|
||||
let r=s.run('stack-copy.sh',['check','--copies',s.copies,'--bin',s.bin,'--units','external']);assert.equal(r.status,0,r.stderr);assert.equal(r.stdout,want);
|
||||
assert.deepEqual(readFileSync(argv,'utf8').trimEnd().split('\n'),[
|
||||
'show [email protected] -p LoadState -p WorkingDirectory -p ExecStart',
|
||||
'show mosaic-webui.service -p LoadState -p WorkingDirectory -p ExecStart']);
|
||||
rmSync(argv);
|
||||
for(const args of [[],['--units','user']]){
|
||||
r=s.run('stack-copy.sh',['check','--copies',s.copies,'--bin',s.bin,...args]);assert.equal(r.status,0,r.stderr);assert.equal(r.stdout,want);
|
||||
assert.deepEqual(readFileSync(argv,'utf8').trimEnd().split('\n'),[
|
||||
'--user show [email protected] -p LoadState -p WorkingDirectory -p ExecStart',
|
||||
'--user show mosaic-webui.service -p LoadState -p WorkingDirectory -p ExecStart']);
|
||||
rmSync(argv);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -0,0 +1,476 @@
|
||||
// scripts/stack-release.sh --units external (row 232, #1734). Design: r6 section 4 and the section 7
|
||||
// test list, agents/darkwing/work/queue-195/DESIGN.md; deviations: agents/darkwing/work/queue-232/BUILD.md.
|
||||
// The stand-ins and the module fixture are stack-release.test.mjs's (stack-release-fixture.mjs).
|
||||
// Root's part (stop, unit install, start) runs here as the stand-in systemctl and bus-service.sh and
|
||||
// webui-service.sh installs; no sudo and no real unit.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {execFileSync} from 'node:child_process';
|
||||
import {mkdirSync,writeFileSync,readFileSync,existsSync,readdirSync,rmSync,chmodSync,cpSync} from 'node:fs';
|
||||
import {DatabaseSync} from 'node:sqlite';
|
||||
import {join} from 'node:path';
|
||||
import {B,C,C1,COPIES,ROOT,SBIN,base,fault,lines,logLines,mk,pointer,repo,run,seed,sha,storeVersion,sysctl,tree,units} from './stack-release-fixture.mjs';
|
||||
|
||||
// The commit row 232 builds on: the user-mode tests compare the candidate's calls and --plan bytes
|
||||
// with its scripts. A later row that changes user mode on purpose moves this.
|
||||
const BASE='cb5fdb064fb4f12edd045374dd463dfdc25c2b42';
|
||||
|
||||
const X=(e)=>({...e,flags:[...e.flags,'--units','external']});
|
||||
const sh=(e,name,...a)=>execFileSync('bash',[join(repo,'scripts',name),...a],{env:e.env,encoding:'utf8',stdio:['ignore','pipe','pipe']});
|
||||
// Root's steps: R1/A2 stop, A4 install and start. offset is the start's SD_CLOCK_OFFSET.
|
||||
const down=(e)=>{sysctl(e,'stop',e.con);sysctl(e,'stop',e.bus)};
|
||||
const up=(e,copy,{pm=false,offset=0}={})=>{
|
||||
sh(e,'bus-service.sh','install','--copy',copy,'--copies',COPIES,...(pm?['--pm']:[]),'--claude',e.claude);
|
||||
sh(e,'webui-service.sh','install','--copy',copy,'--copies',COPIES,'--port',e.port,'--business','acme');
|
||||
execFileSync(join(SBIN,'systemctl'),['--user','start',e.bus],{env:{...e.env,SD_CLOCK_OFFSET:String(offset)}});
|
||||
sysctl(e,'start',e.con);
|
||||
};
|
||||
// The stack on B for acme, adopted under --units external, then stopped by root unless live.
|
||||
async function xseed(t,{pm=false,live=false}={}){
|
||||
const e=await mk(t);await seed(e,{pm,pointer:'none'});const x=X(e);
|
||||
const r=await run(x,'adopt','acme');assert.equal(r.code,0,r.err);
|
||||
if(!live)down(e);
|
||||
return {e,x};
|
||||
}
|
||||
const lastAct=(e)=>logLines(e).filter((o)=>o.result==='started').at(-1).activation;
|
||||
const rs=(e,a)=>join(e.state,'stack-rollback',a);
|
||||
const open=(e)=>{const m=new Set();for(const o of logLines(e)){if(!o.activation)continue;if(o.result==='started')m.add(o.activation);else m.delete(o.activation)}return [...m]};
|
||||
const mine=(e,a)=>logLines(e).filter((o)=>o.activation===a);
|
||||
const journalCalls=(e)=>lines(join(e.sd,'journal-since.log'));
|
||||
// A3: stop, stage TO. Returns the activation and the run.
|
||||
async function stage(e,x,to,...args){
|
||||
const r=await run(x,'activate',to,'acme',...args);
|
||||
return {r,a:r.code===0||r.code===1?lastAct(e):null};
|
||||
}
|
||||
// The started line's from, to and op on the activation's result line, and its backup the set's.
|
||||
function sameAsStarted(e,a){
|
||||
const [s,f,...more]=mine(e,a);assert.equal(more.length,0,JSON.stringify(mine(e,a)));
|
||||
assert.equal(s.result,'started');
|
||||
for(const k of ['op','from','to','activation'])assert.equal(f[k],s[k],k);
|
||||
assert.ok(['activate','rollback'].includes(f.op),f.op);
|
||||
assert.equal(f.units,'external');
|
||||
const b=join(rs(e,a),'BACKUP');assert.equal(f.backup,existsSync(b)?readFileSync(b,'utf8').trim():null);
|
||||
return f;
|
||||
}
|
||||
// B1's end state: one started and one result line, nothing open, no marker or FAILED, and a
|
||||
// following --plan that doesn't refuse interrupted-run.
|
||||
async function closed(e,x,a){
|
||||
sameAsStarted(e,a);
|
||||
assert.deepEqual(open(e),[]);
|
||||
for(const f of ['AWAITING-UNITS','FAILED'])assert.ok(!existsSync(join(rs(e,a),f)),f);
|
||||
const s=await run(x,'status','acme');assert.equal(s.code,0,s.err);
|
||||
assert.ok(!/interrupted run|awaiting units/.test(s.out),s.out);
|
||||
const p=await run(x,'activate',pointer(e).copy===C1?B:C1,'acme','--plan');
|
||||
assert.ok(!(p.out+p.err).includes('interrupted-run'),p.out+p.err);
|
||||
}
|
||||
const storeSha=(e)=>sha(e.store);
|
||||
const backupStore=(e,a)=>join(e.data,'backups','stack',a,'bus','bus.sqlite');
|
||||
|
||||
test('row 232: confirm, revert, gate, --to and --expect-from need --units external; the user-only flags are usage under it',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const logs=logLines(e).length;
|
||||
for(const a of [['confirm',C1,'acme'],['revert',C1,'acme'],['gate',B,'acme','--since','1'],['rollback','acme','--to',B],['activate',C1,'acme','--plan','--expect-from',B]]){
|
||||
const r=await run(e,...a);assert.equal(r.code,4,a.join(' '));assert.match(r.err,/^usage: stack-release.sh adopt BUSINESS\n/);
|
||||
}
|
||||
for(const a of [['activate',C1,'acme','--pm'],['activate',C1,'acme','--tester-port','7444'],['activate',C1,'acme','--no-proof'],['activate',C1,'acme','--no-send'],['activate',C1,'acme','--sso'],['activate',C1,'acme','--proof-header','X','--proof-file','/p'],['rollback','acme'],['gate',B,'acme'],['activate',C1,'acme','--expect-from',B],['confirm',C1,'acme','--rollback',B],['confirm',C1,'acme','--cause','A5']]){
|
||||
const r=await run(x,...a);assert.equal(r.code,4,a.join(' '));
|
||||
}
|
||||
assert.equal(logLines(e).length,logs);
|
||||
});
|
||||
|
||||
test('row 232: pm-on and activate --pm-after-harness refuse external-pm',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const p=await run(x,'pm-on','acme');assert.equal(p.code,3);
|
||||
assert.match(p.err,/^stack-release: refusing external-pm: under --units external the PM is a flag in root's bus unit/);
|
||||
const a=await run(x,'activate',C1,'acme','--pm-after-harness');assert.equal(a.code,3);
|
||||
assert.match(a.err,/^stack-release: refusing external-pm: /);
|
||||
const l=logLines(e).slice(-2);assert.deepEqual(l.map((o)=>[o.result,o.code,o.activation]),[['refused','external-pm',null],['refused','external-pm',null]]);
|
||||
assert.equal(readdirSync(e.state).includes('stack-rollback'),false);
|
||||
});
|
||||
|
||||
test('row 232: units-running (N4) refuses while a unit is active and leaves the store, the log and TAKEN; failed counts as down',async(t)=>{
|
||||
const {e,x}=await xseed(t,{live:true});
|
||||
sysctl(e,'stop',e.con);
|
||||
const before=storeSha(e),logs=logLines(e).length;
|
||||
const r=await run(x,'activate',C,'acme','--migrate');assert.equal(r.code,3,r.out+r.err);
|
||||
assert.equal(r.err,`stack-release: refusing units-running: ${e.bus} is active; root stops the units first\n`);
|
||||
assert.equal(storeSha(e),before);assert.equal(storeVersion(e.store,true),'3g');
|
||||
const l=logLines(e).slice(logs);assert.deepEqual(l.map((o)=>[o.result,o.code,o.activation,o.units]),[['refused','units-running',null,'external']]);
|
||||
assert.ok(!existsSync(join(e.data,'backups')));assert.ok(!existsSync(join(e.state,'stack-rollback')));
|
||||
// The Console active and the bus stopped refuses too.
|
||||
sysctl(e,'stop',e.bus);sysctl(e,'start',e.con);
|
||||
const c=await run(x,'activate',C1,'acme');assert.equal(c.code,3);assert.ok(c.err.startsWith(`stack-release: refusing units-running: ${e.con} is active`),c.err);
|
||||
// A bus unit root's stop left failed is down.
|
||||
sysctl(e,'stop',e.con);
|
||||
const s=units(e);s[e.bus].state='failed';writeFileSync(join(e.sd,'units.json'),JSON.stringify(s));
|
||||
const f=await run(x,'activate',C1,'acme');assert.equal(f.code,0,f.out+f.err);
|
||||
});
|
||||
|
||||
test('row 232: activate stages: the pointer stays, AWAITING-UNITS names TO, a backup of the stopped store, no directory in the set, no --user, no journal',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const p0=readFileSync(e.pointer,'utf8'),w0=sha(join(e.bin,'mosaic')),argv0=lines(join(e.sd,'argv.log')).length;
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
assert.equal(r.out.split('\n').at(-2),`stack-release: activate staged ${C1} for acme as ${a}; root installs and starts the units, then runs confirm ${C1} acme --units external`);
|
||||
assert.equal(readFileSync(e.pointer,'utf8'),p0);
|
||||
assert.equal(readFileSync(join(rs(e,a),'AWAITING-UNITS'),'utf8'),`${C1}\n`);
|
||||
assert.deepEqual(readdirSync(rs(e,a)).sort(),['AWAITING-UNITS','BACKUP','SHA256SUMS','UNITS.sha256','WRAPPER_MODE','pointer',`show-${e.bus}.txt`,`show-${e.con}.txt`,'steps.log','wrapper'].sort());
|
||||
assert.equal(readFileSync(join(rs(e,a),'BACKUP'),'utf8'),`${a}\n`);
|
||||
assert.equal(sha(backupStore(e,a)),storeSha(e));assert.equal(readFileSync(join(e.data,'backups','stack',a,'SCHEMA'),'utf8'),'3g\n');
|
||||
assert.equal(readFileSync(join(rs(e,a),'UNITS.sha256'),'utf8'),`${sha(join(e.units,e.tmpl))} ${join(e.units,e.tmpl)}\n${sha(join(e.units,e.con))} ${join(e.units,e.con)}\n`);
|
||||
assert.equal(sha(join(rs(e,a),'wrapper')),w0);assert.notEqual(sha(join(e.bin,'mosaic')),w0);
|
||||
assert.deepEqual(mine(e,a).map((o)=>o.result),['started']);
|
||||
assert.deepEqual([units(e)[e.bus].state,units(e)[e.con].state],['inactive','inactive']);
|
||||
const ext=lines(join(e.sd,'argv.log')).slice(argv0);
|
||||
assert.ok(ext.length>0);for(const c of ext)assert.match(c,/^(show|is-active|list-units) /,c);
|
||||
assert.deepEqual(journalCalls(e),[]);
|
||||
const s=await run(x,'status','acme');assert.equal(s.code,0,s.err);
|
||||
assert.ok(s.out.includes(`awaiting units: ${a} (copy ${C1})\n`),s.out);assert.ok(s.out.includes(`interrupted run: ${a} `),s.out);
|
||||
});
|
||||
|
||||
test('row 232: A3 order: capture-failed and marker-failed refuse with exit 3 and leave no marker or FAILED',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
fault(e,'no-fragment');
|
||||
const r=await run(x,'activate',C1,'acme');assert.equal(r.code,3,r.out+r.err);
|
||||
assert.match(r.err,/^stack-release: refusing capture-failed: could not write the rollback set .*; nothing was changed\n$/);
|
||||
const a=lastAct(e);
|
||||
assert.deepEqual(mine(e,a).map((o)=>[o.result,o.code]),[['started',undefined],['refused','capture-failed']]);
|
||||
for(const f of ['AWAITING-UNITS','FAILED'])assert.ok(!existsSync(join(rs(e,a),f)),f);
|
||||
assert.deepEqual(open(e),[]);
|
||||
rmSync(join(e.sd,'faults','no-fragment'));
|
||||
// The set can't be made: marker-failed, for that activation.
|
||||
chmodSync(join(e.state,'stack-rollback'),0o555);
|
||||
const m=await run(x,'activate',C1,'acme');assert.equal(m.code,3,m.out+m.err);
|
||||
assert.match(m.err,/^stack-release: refusing marker-failed: could not make the rollback set .*; nothing was changed\n$/);
|
||||
const b=lastAct(e);assert.notEqual(b,a);
|
||||
assert.deepEqual(mine(e,b).map((o)=>[o.result,o.code]),[['started',undefined],['refused','marker-failed']]);
|
||||
chmodSync(join(e.state,'stack-rollback'),0o755);
|
||||
assert.deepEqual(open(e),[]);
|
||||
});
|
||||
|
||||
test('row 232: confirm and revert refuse open-activation with no marker, two, the wrong copy, or one the log holds closed; refusals name no activation',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const tryBoth=async(id,why)=>{
|
||||
for(const v of ['confirm','revert']){
|
||||
const o=open(e),logs=logLines(e).length;
|
||||
const r=await run(x,v,id,'acme');assert.equal(r.code,3,`${v} ${why}: ${r.out}${r.err}`);
|
||||
assert.match(r.err,/^stack-release: refusing open-activation: /,why);
|
||||
assert.deepEqual(logLines(e).slice(logs).map((l)=>[l.result,l.code,l.activation,l.op]),[['refused','open-activation',null,v]],why);
|
||||
assert.deepEqual(open(e),o,why);
|
||||
}
|
||||
};
|
||||
await tryBoth(C1,'no marker');
|
||||
// One marker whose activation the log never started. It sorts before any real one, so a confirm
|
||||
// that took the newest of two markers would take the real one.
|
||||
const fake=join(e.state,'stack-rollback','20000101T000000Z-'+C1.slice(0,12));
|
||||
mkdirSync(fake,{recursive:true});writeFileSync(join(fake,'AWAITING-UNITS'),`${C1}\n`);
|
||||
await tryBoth(C1,'marker not open');
|
||||
rmSync(fake,{recursive:true});
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
await tryBoth(B,'wrong copy');
|
||||
mkdirSync(fake);writeFileSync(join(fake,'AWAITING-UNITS'),`${C1}\n`);
|
||||
await tryBoth(C1,'two markers');
|
||||
rmSync(fake,{recursive:true});
|
||||
// gate refuses nothing and writes nothing, open or not.
|
||||
const logs=readFileSync(e.log,'utf8');
|
||||
const g=await run(x,'gate',C1,'acme','--since','1','--timeout','1');assert.equal(g.code,1);
|
||||
assert.equal(readFileSync(e.log,'utf8'),logs);assert.deepEqual(open(e),[a]);
|
||||
});
|
||||
|
||||
test('row 232: confirm polls: bus status good after 3 s fails with --timeout 1, writes FAILED and leaves the rest; FAILED then refuses activation-failed',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
up(e,C1);
|
||||
const p0=readFileSync(e.pointer,'utf8'),l0=readFileSync(e.log,'utf8');
|
||||
fault(e,`status-until-${C1}`,String(Date.now()+3000));
|
||||
const c=await run(x,'confirm',C1,'acme','--timeout','1');assert.equal(c.code,1,c.out+c.err);
|
||||
assert.match(c.err,new RegExp(`^stack-release: activate failed with bus-status; ${a} stays open`));
|
||||
assert.equal(readFileSync(join(rs(e,a),'FAILED'),'utf8'),'bus-status\n');
|
||||
assert.equal(readFileSync(e.pointer,'utf8'),p0);assert.equal(readFileSync(e.log,'utf8'),l0);
|
||||
assert.ok(existsSync(join(rs(e,a),'AWAITING-UNITS')));
|
||||
const f=await run(x,'confirm',C1,'acme');assert.equal(f.code,3);
|
||||
assert.match(f.err,new RegExp(`^stack-release: refusing activation-failed: ${a} has a FAILED file`));
|
||||
assert.equal(logLines(e).at(-1).activation,null);assert.deepEqual(open(e),[a]);
|
||||
});
|
||||
|
||||
test('row 232: confirm polls to a pass with --timeout 10: the pointer, an ok line that reads as the started run, no marker',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
up(e,C1);
|
||||
fault(e,`status-until-${C1}`,String(Date.now()+3000));
|
||||
const c=await run(x,'confirm',C1,'acme','--timeout','10');assert.equal(c.code,0,c.out+c.err);
|
||||
assert.equal(c.out.split('\n').at(-2),`stack-release: activate ok: ${C1} is live for acme`);
|
||||
const p=pointer(e);assert.deepEqual([p.copy,p.activation,p.schema,p.pm,p.console],[C1,a,'3g',false,['--port',e.port,'--business','acme']]);
|
||||
const f=sameAsStarted(e,a);assert.deepEqual([f.result,f.op,f.from,f.to,f.backup],['ok','activate',B,C1,a]);
|
||||
assert.ok(!existsSync(join(rs(e,a),'AWAITING-UNITS')));
|
||||
assert.deepEqual(journalCalls(e),[]);
|
||||
});
|
||||
|
||||
// N4 with the brief's numbers: the started line at 1792540800.900 floors to 1792540800. The host's
|
||||
// clock runs so boot status is after that second.
|
||||
const T0=1792540800,offsetTo=(s)=>Math.floor(Date.now()/1000)-s;
|
||||
function startedAt(e,a,ms){
|
||||
const l=readFileSync(e.log,'utf8').split('\n').map((x)=>{if(!x)return x;const o=JSON.parse(x);if(o.activation===a&&o.result==='started')o.at=new Date(ms).toISOString();return JSON.stringify(o)});
|
||||
writeFileSync(e.log,l.join('\n'));
|
||||
}
|
||||
|
||||
test('row 232: stale-start: @1792540799 fails a start floor of 1792540800 with stale-start',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
startedAt(e,a,T0*1000+900);up(e,C1,{offset:offsetTo(T0+100)});
|
||||
fault(e,'enter',JSON.stringify({value:T0-1}));
|
||||
const c=await run(x,'confirm',C1,'acme','--timeout','1');assert.equal(c.code,1,c.out+c.err);
|
||||
assert.ok(c.out.includes(`gate failed on ${C1}: stale-start (the bus started at @${T0-1}, before @${T0})`),c.out);
|
||||
assert.equal(readFileSync(join(rs(e,a),'FAILED'),'utf8'),'stale-start\n');
|
||||
});
|
||||
|
||||
test('row 232: stale-start is a gate code: an enter time that turns to @1792540800 after 2 s passes with --timeout 10',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
startedAt(e,a,T0*1000+900);up(e,C1,{offset:offsetTo(T0+100)});
|
||||
fault(e,'enter',JSON.stringify({value:T0-1,at:Date.now()+2000,then:T0}));
|
||||
const c=await run(x,'confirm',C1,'acme','--timeout','10');assert.equal(c.code,0,c.out+c.err);
|
||||
assert.equal(pointer(e).copy,C1);
|
||||
});
|
||||
|
||||
test('row 232: gate (N2, A2f): --since 1792540800 passes @1792540800 and fails @1792540799; it writes nothing',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
up(e,B,{offset:offsetTo(T0+100)});
|
||||
const snap=()=>[readFileSync(e.log,'utf8'),readFileSync(e.pointer,'utf8'),tree(e.state).join('\n')];
|
||||
const s0=snap();
|
||||
fault(e,'enter',JSON.stringify({value:T0}));
|
||||
const g=await run(x,'gate',B,'acme','--since',String(T0));assert.equal(g.code,0,g.out+g.err);
|
||||
assert.equal(g.out,`stack-release: gate passed on ${B}\n`);
|
||||
fault(e,'enter',JSON.stringify({value:T0-1}));
|
||||
const f=await run(x,'gate',B,'acme','--since',String(T0),'--timeout','1');assert.equal(f.code,1,f.out+f.err);
|
||||
assert.ok(f.out.includes(`gate failed on ${B}: stale-start (the bus started at @${T0-1}, before @${T0})`),f.out);
|
||||
assert.deepEqual(snap(),s0);
|
||||
const u=await run(x,'gate',B,'acme');assert.equal(u.code,4);
|
||||
assert.deepEqual(journalCalls(e),[]);
|
||||
});
|
||||
|
||||
test('row 232: the external PM check reads launches list: one live run of launch.by since the start passes',async(t)=>{
|
||||
const {e,x}=await xseed(t,{pm:true});
|
||||
const cases=[['',0,''],['coder',0,''],['none',1,'no live launch of chief'],['twolive',1,'2 live launches of chief'],['early',1,'started before this start'],['dead',1,'no live launch of chief'],['other',1,'the launch of chief is for other']];
|
||||
for(const [mode,code,why] of cases){
|
||||
rmSync(join(e.sd,'launches.json'),{force:true});
|
||||
if(mode)fault(e,`pm-${B}`,mode);else rmSync(join(e.sd,'faults',`pm-${B}`),{force:true});
|
||||
const since=Math.floor(Date.now()/1000);
|
||||
up(e,B,{pm:true});
|
||||
const g=await run(x,'gate',B,'acme','--since',String(since),'--timeout','1');assert.equal(g.code,code,`${mode}: ${g.out}${g.err}`);
|
||||
if(why)assert.ok(g.out.includes(`gate failed on ${B}: pm-launch-unconfirmed (`)&&g.out.includes(why),`${mode}: ${g.out}`);
|
||||
down(e);
|
||||
}
|
||||
assert.deepEqual(journalCalls(e),[]);
|
||||
});
|
||||
|
||||
test('row 232: revert puts the wrapper back and leaves the store when nothing migrated',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const w0=sha(join(e.bin,'mosaic'));
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
// A write to the stopped store after A3: revert has no MIGRATING, so it is not its to undo.
|
||||
const db=new DatabaseSync(e.store);db.prepare('INSERT INTO events (at) VALUES (?)').run('2001-01-01T00:00:00.000Z');db.close();
|
||||
const s1=storeSha(e);assert.notEqual(s1,sha(backupStore(e,a)));
|
||||
const v=await run(x,'revert',C1,'acme');assert.equal(v.code,0,v.out+v.err);
|
||||
assert.equal(v.out.split('\n').at(-2),`stack-release: revert ok: ${a}'s wrapper is back; root installs the old units next`);
|
||||
assert.equal(sha(join(e.bin,'mosaic')),w0);assert.equal(storeSha(e),s1);
|
||||
assert.deepEqual(mine(e,a).map((o)=>o.result),['started']);assert.ok(existsSync(join(rs(e,a),'AWAITING-UNITS')));
|
||||
});
|
||||
|
||||
test('row 232: confirm --rollback refuses rollback-mismatch, and logs rollback-failed with exit 2 when OLD fails its gate',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
const v=await run(x,'revert',C1,'acme');assert.equal(v.code,0,v.out+v.err);
|
||||
const m=await run(x,'confirm',C1,'acme','--rollback',C,'--cause','A5');assert.equal(m.code,3);
|
||||
assert.equal(m.err,`stack-release: refusing rollback-mismatch: ${C} is not the copy ${a} started from, ${B}\n`);
|
||||
assert.equal(logLines(e).at(-1).activation,null);assert.deepEqual(open(e),[a]);
|
||||
fault(e,`healthz-${B}`);up(e,B);
|
||||
const f=await run(x,'confirm',C1,'acme','--rollback',B,'--cause','A5','--timeout','1');assert.equal(f.code,2,f.out+f.err);
|
||||
assert.equal(f.err,`stack-release: rollback-failed: the run failed with unrecorded and the rollback with healthz; root stops the units. Rollback set: ${rs(e,a)}, backup ${join(e.data,'backups','stack',a)}\n`);
|
||||
const l=sameAsStarted(e,a);assert.deepEqual([l.result,l.code,l.rollback,l.rollbackCode],['rollback-failed','unrecorded','failed','healthz']);
|
||||
for(const n of ['AWAITING-UNITS','FAILED'])assert.ok(!existsSync(join(rs(e,a),n)),n);
|
||||
assert.deepEqual(open(e),[]);
|
||||
});
|
||||
|
||||
test('row 232: confirm --rollback after a --restore swap logs rollbackCode not-attempted; rollback --restore finds an external backup',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C,'--migrate');assert.equal(r.code,0,r.out+r.err);
|
||||
up(e,C);assert.equal((await run(x,'confirm',C,'acme')).code,0);
|
||||
down(e);
|
||||
const b=await run(x,'rollback','acme','--to',B,'--restore',a);assert.equal(b.code,0,b.out+b.err);
|
||||
const ra=lastAct(e);
|
||||
assert.ok(b.out.includes(`restore: ${join(e.data,'backups','stack',a)}; `),b.out);
|
||||
assert.ok(existsSync(join(rs(e,ra),'RESTORED')));assert.equal(readFileSync(join(rs(e,ra),'BACKUP'),'utf8'),`${a}\n`);
|
||||
assert.equal(storeSha(e),sha(backupStore(e,a)));
|
||||
const f=await run(x,'confirm',B,'acme','--rollback',C,'--cause','A4');assert.equal(f.code,2,f.out+f.err);
|
||||
const l=sameAsStarted(e,ra);assert.deepEqual([l.op,l.result,l.code,l.rollbackCode,l.backup],['rollback','rollback-failed','units-start','not-attempted',a]);
|
||||
});
|
||||
|
||||
// --- B1: real sequences on one log ---
|
||||
|
||||
test('row 232: sequence 1: activate, confirm times out, revert, confirm --rollback --cause A5: gate-failed rollback ok',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const w0=sha(join(e.bin,'mosaic')),p0=readFileSync(e.pointer,'utf8');
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
fault(e,`healthz-${C1}`);up(e,C1);
|
||||
const c=await run(x,'confirm',C1,'acme','--timeout','1');assert.equal(c.code,1,c.out+c.err);
|
||||
assert.deepEqual(mine(e,a).map((o)=>o.result),['started']);
|
||||
down(e);
|
||||
assert.equal((await run(x,'revert',C1,'acme')).code,0);assert.equal(sha(join(e.bin,'mosaic')),w0);
|
||||
up(e,B);
|
||||
const k=await run(x,'confirm',C1,'acme','--rollback',B,'--cause','A5');assert.equal(k.code,1,k.out+k.err);
|
||||
assert.equal(k.err,`stack-release: healthz; rolled back to ${B}, which passed the gate\n`);
|
||||
const l=sameAsStarted(e,a);assert.deepEqual([l.result,l.code,l.rollback],['gate-failed','healthz','ok']);
|
||||
assert.equal(readFileSync(e.pointer,'utf8'),p0);
|
||||
await closed(e,x,a);
|
||||
});
|
||||
|
||||
test('row 232: sequence 2: activate --migrate, the wrapper install fails, revert restores TAKEN, confirm --rollback --cause A3',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const s0=storeSha(e);
|
||||
chmodSync(e.bin,0o555);
|
||||
const {r,a}=await stage(e,x,C,'--migrate');chmodSync(e.bin,0o755);
|
||||
assert.equal(r.code,1,r.out+r.err);assert.match(r.err,new RegExp(`^stack-release: activate failed with wrapper; ${a} stays open`));
|
||||
assert.deepEqual(mine(e,a).map((o)=>o.result),['started']);
|
||||
for(const n of ['AWAITING-UNITS','MIGRATING'])assert.ok(existsSync(join(rs(e,a),n)),n);
|
||||
assert.equal(readFileSync(join(rs(e,a),'FAILED'),'utf8'),'wrapper\n');
|
||||
assert.equal(storeVersion(e.store,false),'3h');assert.equal(sha(backupStore(e,a)),s0);
|
||||
const v=await run(x,'revert',C,'acme');assert.equal(v.code,0,v.out+v.err);
|
||||
assert.equal(v.out.split('\n').at(-2),`stack-release: revert ok: ${a}'s wrapper and store are back; root installs the old units next`);
|
||||
assert.equal(storeSha(e),s0);
|
||||
// revert runs again: the store already matches, and nothing moves.
|
||||
const posts=readdirSync(e.data).filter((n)=>n.startsWith('bus.post-attempt-'));
|
||||
assert.equal((await run(x,'revert',C,'acme')).code,0);
|
||||
assert.deepEqual(readdirSync(e.data).filter((n)=>n.startsWith('bus.post-attempt-')),posts);
|
||||
up(e,B);
|
||||
const k=await run(x,'confirm',C,'acme','--rollback',B,'--cause','A3');assert.equal(k.code,1,k.out+k.err);
|
||||
const l=sameAsStarted(e,a);assert.deepEqual([l.result,l.code,l.rollback],['failed','wrapper','ok']);
|
||||
await closed(e,x,a);
|
||||
});
|
||||
|
||||
test('row 232: sequence 3: activate, root\'s A4 fails, revert, confirm --rollback --cause A4: failed code units-start',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
assert.equal((await run(x,'revert',C1,'acme')).code,0);
|
||||
up(e,B);
|
||||
const k=await run(x,'confirm',C1,'acme','--rollback',B,'--cause','A4');assert.equal(k.code,1,k.out+k.err);
|
||||
assert.equal(k.err,`stack-release: units-start; rolled back to ${B}, which passed the gate\n`);
|
||||
const l=sameAsStarted(e,a);assert.deepEqual([l.result,l.code,l.rollback],['failed','units-start','ok']);
|
||||
await closed(e,x,a);
|
||||
});
|
||||
|
||||
test('row 232: sequence 4: a capture failure leaves nothing open, then A2f\'s gate OLD writes nothing',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
fault(e,'no-fragment');
|
||||
const r=await run(x,'activate',C1,'acme');assert.equal(r.code,3,r.out+r.err);rmSync(join(e.sd,'faults','no-fragment'));
|
||||
const a=lastAct(e);assert.deepEqual(mine(e,a).map((o)=>[o.result,o.code]),[['started',undefined],['refused','capture-failed']]);
|
||||
assert.deepEqual(open(e),[]);
|
||||
const since=Math.floor(Date.now()/1000);up(e,B);
|
||||
const s0=[readFileSync(e.log,'utf8'),readFileSync(e.pointer,'utf8'),tree(e.state).join('\n')];
|
||||
const g=await run(x,'gate',B,'acme','--since',String(since));assert.equal(g.code,0,g.out+g.err);
|
||||
assert.deepEqual([readFileSync(e.log,'utf8'),readFileSync(e.pointer,'utf8'),tree(e.state).join('\n')],s0);
|
||||
});
|
||||
|
||||
test('row 232: sequence 5: a confirm of the wrong copy is refused, and revert and confirm --rollback still close the activation',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
up(e,C1);
|
||||
const w=await run(x,'confirm',B,'acme');assert.equal(w.code,3);assert.match(w.err,/^stack-release: refusing open-activation: /);
|
||||
assert.equal(logLines(e).at(-1).activation,null);assert.deepEqual(open(e),[a]);
|
||||
down(e);
|
||||
assert.equal((await run(x,'revert',C1,'acme')).code,0);
|
||||
up(e,B);
|
||||
const k=await run(x,'confirm',C1,'acme','--rollback',B,'--cause','A5');assert.equal(k.code,1,k.out+k.err);
|
||||
const l=sameAsStarted(e,a);assert.deepEqual([l.result,l.code,l.rollback],['gate-failed','unrecorded','ok']);
|
||||
await closed(e,x,a);
|
||||
});
|
||||
|
||||
test('row 232: sequence 6: a revert that fails leaves the activation open, and the next --plan refuses interrupted-run',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const {r,a}=await stage(e,x,C1);assert.equal(r.code,0,r.out+r.err);
|
||||
rmSync(join(rs(e,a),'wrapper'));
|
||||
const v=await run(x,'revert',C1,'acme');assert.equal(v.code,1,v.out+v.err);
|
||||
assert.equal(v.err,`stack-release: revert failed: the rollback set has no saved wrapper; ${a} stays open. Rollback set: ${rs(e,a)}\n`);
|
||||
assert.deepEqual(open(e),[a]);assert.ok(existsSync(join(rs(e,a),'AWAITING-UNITS')));
|
||||
const p=await run(x,'activate',C,'acme','--plan','--migrate');assert.equal(p.code,3,p.out+p.err);
|
||||
assert.ok(p.out.includes(`refuses interrupted-run: activation ${a} started and has no final line`),p.out);
|
||||
});
|
||||
|
||||
test('row 232: sequence 7: adopt, two confirmed activates, rollback --to the previous copy, confirmed, logs op rollback',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const one=await stage(e,x,C1);assert.equal(one.r.code,0,one.r.out+one.r.err);
|
||||
up(e,C1);assert.equal((await run(x,'confirm',C1,'acme')).code,0);down(e);
|
||||
const two=await stage(e,x,C,'--migrate');assert.equal(two.r.code,0,two.r.out+two.r.err);
|
||||
up(e,C);assert.equal((await run(x,'confirm',C,'acme')).code,0);down(e);
|
||||
const l1=sameAsStarted(e,one.a),l2=sameAsStarted(e,two.a);
|
||||
assert.deepEqual([l1.op,l1.result,l1.from,l1.to,l1.backup],['activate','ok',B,C1,one.a]);
|
||||
assert.deepEqual([l2.op,l2.result,l2.from,l2.to,l2.backup],['activate','ok',C1,C,two.a]);
|
||||
// N5: --expect-from names the copy root read from its units.
|
||||
const d1=await run(x,'activate',C1,'acme','--plan','--expect-from',C1);assert.equal(d1.code,3,d1.out+d1.err);
|
||||
assert.ok(d1.out.includes(`refuses pointer-differs: the pointer names ${C} and root's units name ${C1}; Jason or Mos settles it`),d1.out);
|
||||
const d2=await run(x,'rollback','acme','--to',C1,'--plan','--expect-from',B);assert.equal(d2.code,3,d2.out+d2.err);
|
||||
assert.ok(d2.out.includes(`refuses pointer-differs: the pointer names ${C} and root's units name ${B}`),d2.out);
|
||||
const ok=await run(x,'rollback','acme','--to',C1,'--restore',two.a,'--plan','--expect-from',C);assert.equal(ok.code,0,ok.out+ok.err);
|
||||
const logs=logLines(e).length;
|
||||
const no=await run(x,'rollback','acme','--to',B,'--plan');assert.equal(no.code,3);
|
||||
assert.equal(no.err,`stack-release: refusing to-not-previous: --to ${B} is not the previous copy; the log names ${C1}\n`);
|
||||
assert.equal(logLines(e).length,logs);
|
||||
const nr=await run(x,'rollback','acme','--to',B,'--restore',one.a);assert.equal(nr.code,3);assert.match(nr.err,/^stack-release: refusing to-not-previous: /);
|
||||
assert.deepEqual(logLines(e).slice(logs).map((o)=>[o.result,o.code,o.activation]),[['refused','to-not-previous',null]]);
|
||||
const rb=await run(x,'rollback','acme','--to',C1,'--restore',two.a);assert.equal(rb.code,0,rb.out+rb.err);
|
||||
const ra=lastAct(e);
|
||||
up(e,C1);
|
||||
const k=await run(x,'confirm',C1,'acme');assert.equal(k.code,0,k.out+k.err);
|
||||
assert.equal(k.out.split('\n').at(-2),`stack-release: rollback ok: ${C1} is live for acme`);
|
||||
const l3=sameAsStarted(e,ra);assert.deepEqual([l3.op,l3.result,l3.from,l3.to,l3.backup],['rollback','ok',C,C1,two.a]);
|
||||
assert.deepEqual([pointer(e).copy,pointer(e).schema],[C1,'3g']);
|
||||
const prev=await run(x,'rollback','acme','--to',B,'--plan');assert.match(prev.err,new RegExp(`the log names ${C}\\n$`));
|
||||
await closed(e,x,ra);
|
||||
});
|
||||
|
||||
test('row 232: sequence 8: a migration that writes and fails: revert restores the backup byte for byte, confirm --rollback --cause A3',async(t)=>{
|
||||
const {e,x}=await xseed(t);
|
||||
const s0=storeSha(e);
|
||||
fault(e,'migrate-fails');
|
||||
const {r,a}=await stage(e,x,C,'--migrate');assert.equal(r.code,1,r.out+r.err);rmSync(join(e.sd,'faults','migrate-fails'));
|
||||
assert.equal(readFileSync(join(rs(e,a),'FAILED'),'utf8'),'migrate-failed\n');assert.ok(existsSync(join(rs(e,a),'MIGRATING')));
|
||||
assert.equal(storeVersion(e.store,false),'3h-partial');
|
||||
assert.equal((await run(x,'revert',C,'acme')).code,0);
|
||||
assert.equal(storeSha(e),s0);assert.equal(storeSha(e),sha(backupStore(e,a)));
|
||||
assert.deepEqual(readdirSync(join(e.data,'bus')),readdirSync(join(e.data,'backups','stack',a,'bus')));
|
||||
up(e,B);
|
||||
const k=await run(x,'confirm',C,'acme','--rollback',B,'--cause','A3');assert.equal(k.code,1,k.out+k.err);
|
||||
const l=sameAsStarted(e,a);assert.deepEqual([l.result,l.code,l.rollback],['failed','migrate-failed','ok']);
|
||||
await closed(e,x,a);
|
||||
});
|
||||
|
||||
// --- User mode against the base ---
|
||||
|
||||
// The fixture repo with the base's two scripts: everything else is the same tree and history.
|
||||
const baseRepo=join(base,'base-repo');
|
||||
cpSync(repo,baseRepo,{recursive:true});
|
||||
for(const f of ['scripts/stack-release.sh','scripts/stack-copy.sh'])writeFileSync(join(baseRepo,f),execFileSync('git',['-C',ROOT,'show',`${BASE}:${f}`]),{mode:0o755});
|
||||
const BASE_SR=join(baseRepo,'scripts','stack-release.sh');
|
||||
|
||||
test('row 232: user mode: every systemctl call carries --user, and activate, rollback and check make the base\'s calls',async(t)=>{
|
||||
const calls=async(sr,dir)=>{
|
||||
const e=await mk(t);e.sr=sr;await seed(e);
|
||||
const n=lines(join(e.sd,'argv.log')).length;
|
||||
const a=await run(e,'activate',C1,'acme');assert.equal(a.code,0,a.out+a.err);
|
||||
const r=await run(e,'rollback','acme');assert.equal(r.code,0,r.out+r.err);
|
||||
execFileSync('bash',[join(dir,'scripts','stack-copy.sh'),'check','--copies',COPIES,'--bin',e.bin,'--business','acme'],{env:e.env,stdio:'ignore'});
|
||||
return lines(join(e.sd,'argv.log')).slice(n).map((l)=>l.split(e.root).join('<root>').split(e.port).join('<port>'));
|
||||
};
|
||||
const cand=await calls(undefined,repo),old=await calls(BASE_SR,baseRepo);
|
||||
assert.ok(cand.length>20);
|
||||
for(const c of cand)assert.match(c,/^--user /,c);
|
||||
assert.deepEqual(cand,old);
|
||||
});
|
||||
|
||||
test('row 232: user mode: a default activate --plan prints the base\'s bytes',async(t)=>{
|
||||
const e=await mk(t);await seed(e);
|
||||
for(const a of [['activate',C1,'acme','--plan'],['activate',C,'acme','--plan','--migrate'],['activate',C,'acme','--plan'],['rollback','acme','--plan']]){
|
||||
const cand=await run(e,...a),old=await run({...e,sr:BASE_SR},...a);
|
||||
assert.deepEqual(cand,old,a.join(' '));
|
||||
}
|
||||
assert.ok(!existsSync(join(e.state,'stack-rollback')));
|
||||
});
|
||||
@@ -0,0 +1,451 @@
|
||||
// The stack-release stand-ins and module fixture, shared by stack-release.test.mjs and
|
||||
// stack-release-external.test.mjs (row 232). Moved unchanged from stack-release.test.mjs, plus
|
||||
// what the external tests need; each test file runs in its own process, so each gets its own base.
|
||||
import test, {after} from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {execFileSync,spawn,spawnSync} from 'node:child_process';
|
||||
import {mkdtempSync,mkdirSync,writeFileSync,readFileSync,appendFileSync,cpSync,rmSync,chmodSync,existsSync,statSync,lstatSync,symlinkSync,linkSync,readdirSync,realpathSync} from 'node:fs';
|
||||
import {createServer} from 'node:http';
|
||||
import {createHash} from 'node:crypto';
|
||||
import {DatabaseSync} from 'node:sqlite';
|
||||
import {tmpdir} from 'node:os';
|
||||
import {join,dirname,basename} from 'node:path';
|
||||
import {fileURLToPath,pathToFileURL} from 'node:url';
|
||||
import {once} from 'node:events';
|
||||
|
||||
export const ROOT=fileURLToPath(new URL('../../..',import.meta.url)).replace(/\/$/,'');
|
||||
export const NODE=process.execPath;
|
||||
export const MARKER='# mosaic-cli wrapper (Q81, agents/sage/work/q81/mosaic-cli.sh)';
|
||||
export const BUS='[email protected]',CON='mosaic-webui.service';
|
||||
export const git=(cwd,...args)=>execFileSync('git',['-c','user.name=fixture','-c','user.email=fixture@invalid','-c','commit.gpgsign=false','-c','core.hooksPath=/dev/null','-C',cwd,...args],{encoding:'utf8'}).trim();
|
||||
export const script=(fn)=>`#!${NODE}\n(${fn})();\n`;
|
||||
|
||||
// --- Stand-ins. Each is serialized into its own file, so it uses only process and its own scope. ---
|
||||
|
||||
// systemctl --user: unit state in $SD_STATE/units.json, every call in calls.log. A bus start runs
|
||||
// holder.cjs (the store held open in WAL mode, writer.lock, host.json) and, with --pm in the unit's
|
||||
// ExecStart, writes the `launched` journal line and the launches entry unless a pm-<copy> fault
|
||||
// names another case. Faults are files in $SD_STATE/faults named for the copy they hit. Units of a
|
||||
// named instance (row 190) run on their own data root and host files beside the default's.
|
||||
export function systemctl(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),cp=process.getBuiltinModule('node:child_process'),crypto=process.getBuiltinModule('node:crypto');
|
||||
const sd=process.env.SD_STATE,data=process.env.SD_DATA,unitDir=path.join(process.env.XDG_CONFIG_HOME,'systemd','user');
|
||||
const args=process.argv.slice(2).filter((a)=>a!=='--user'),[verb,...rest]=args;
|
||||
fs.appendFileSync(path.join(sd,'calls.log'),args.join(' ')+'\n');
|
||||
// Row 232: argv.log keeps each call as given, --user and all, for the user/external argv tests.
|
||||
fs.appendFileSync(path.join(sd,'argv.log'),process.argv.slice(2).join(' ')+'\n');
|
||||
const faultFile=(n)=>path.join(sd,'faults',n),fault=(n)=>fs.existsSync(faultFile(n));
|
||||
const stFile=path.join(sd,'units.json');
|
||||
const load=()=>{try{return JSON.parse(fs.readFileSync(stFile,'utf8'))}catch{return {}}};
|
||||
const save=(s)=>{fs.writeFileSync(stFile+'.tmp',JSON.stringify(s));fs.renameSync(stFile+'.tmp',stFile)};
|
||||
const sleep=(ms)=>Atomics.wait(new Int32Array(new SharedArrayBuffer(4)),0,0,ms);
|
||||
const alive=(pid)=>{try{const s=fs.readFileSync(`/proc/${pid}/stat`,'utf8');return s[s.lastIndexOf(')')+2]!=='Z'}catch{return false}};
|
||||
// Row 190: a bus unit is the default's mosaic-bus@B or instance NAME's mosaic-bus-NAME@B; %i is B.
|
||||
const isBus=(u)=>/^mosaic-bus(-[a-z][a-z0-9]{1,15})?@[a-z0-9-]+\.service$/.test(u),inst=(u)=>u.slice(u.indexOf('@')+1,-'.service'.length);
|
||||
const unitFile=(u)=>{
|
||||
const main=isBus(u)?u.replace(/@[^@]*$/,'@.service'):u,file=path.join(unitDir,main);
|
||||
if(!fs.existsSync(file))return null;
|
||||
// The drop-in directories in systemd's order: service.d, the name cut after each hyphen, then
|
||||
// the template's and the unit's own.
|
||||
const stem=main.slice(0,main.search(/@?\.service$/)),dirs=['service.d'],drops=[];
|
||||
for(let i=0;i<stem.length;i++)if(stem[i]==='-')dirs.push(stem.slice(0,i+1)+'.service.d');
|
||||
for(const d of [...dirs,...(isBus(u)?[main+'.d',u+'.d']:[u+'.d'])]){const dir=path.join(unitDir,d);if(fs.existsSync(dir))for(const f of fs.readdirSync(dir).sort())if(f.endsWith('.conf'))drops.push(path.join(dir,f))}
|
||||
const r={file,wd:'',exec:'',env:[],drops};
|
||||
for(const f of [file,...drops])for(let line of fs.readFileSync(f,'utf8').split('\n')){
|
||||
line=line.replace(/%i/g,isBus(u)?inst(u):'').replace(/%h/g,process.env.HOME);
|
||||
const m=/^(WorkingDirectory|ExecStart|Environment)=(.*)$/.exec(line);if(!m)continue;
|
||||
if(m[1]==='WorkingDirectory')r.wd=m[2];else if(m[1]==='ExecStart'){if(m[2])r.exec=m[2]}else r.env.push(m[2]);
|
||||
}
|
||||
return r;
|
||||
};
|
||||
// A named instance's unit sets MOSAIC_CONFIG, and its bus works on that config's data root. A unit
|
||||
// without it is the default's, on $SD_DATA. Each data root has its own host and launches files.
|
||||
const dataOf=(f)=>{const c=f&&f.env.find((v)=>v.startsWith('MOSAIC_CONFIG='));return c?JSON.parse(fs.readFileSync(c.slice('MOSAIC_CONFIG='.length),'utf8')).dataRoot:data};
|
||||
const perData=(d,n)=>path.join(sd,d===data?`${n}.json`:`${n}-${path.basename(d)}.json`);
|
||||
const snap=(ev,d)=>{
|
||||
const dir=path.join(d,'bus'),files={};
|
||||
try{for(const n of fs.readdirSync(dir).sort()){const p=path.join(dir,n),st=fs.statSync(p);if(st.isFile())files[n]=[st.size,st.mtimeMs,crypto.createHash('sha256').update(fs.readFileSync(p)).digest('hex')]}}catch{}
|
||||
fs.appendFileSync(path.join(sd,'snap.jsonl'),JSON.stringify({ev,files})+'\n');
|
||||
};
|
||||
const pm=(unit,d,id,pid,off)=>{
|
||||
let mode='';try{mode=fs.readFileSync(faultFile(`pm-${id}`),'utf8').trim()}catch{}
|
||||
const b=inst(unit),now=Date.now()-off*1000,epoch=Math.floor(now/1000),at=new Date(now).toISOString(),run=`run-${pid}`;
|
||||
const line=(text)=>fs.appendFileSync(path.join(sd,'journal.jsonl'),JSON.stringify({epoch,unit,text})+'\n');
|
||||
const ok=()=>line(`launched pm-${b}: run ${run}, claude opus`);
|
||||
const lf=perData(d,'launches');let list=[];try{list=JSON.parse(fs.readFileSync(lf,'utf8'))}catch{}
|
||||
// instance is launch.by's, chief in the example business, as the external PM check reads (row 232).
|
||||
const entry=(o)=>list.push({run,instance:'chief',business:b,live:true,startedAt:at,...o});
|
||||
line(`bus host up: business ${b}, socket /fixture/bus.sock, notifier off`);
|
||||
switch(mode){
|
||||
case 'none':break;
|
||||
case 'two':ok();line(`launched pm-${b}: run ${run}b, claude opus`);entry({});break;
|
||||
case 'badword':line(`launched pm-${b}: job ${run}, claude opus`);entry({});break;
|
||||
case 'notlisted':ok();break;
|
||||
case 'early':ok();entry({startedAt:new Date(now-3600e3).toISOString()});break;
|
||||
case 'human':entry({run:`human-${pid}`});break;
|
||||
case 'other':ok();entry({business:'other'});break;
|
||||
case 'dead':ok();entry({live:false});break;
|
||||
// Row 232: two live entries for the PM's instance, and one for it beside a live coder run.
|
||||
case 'twolive':ok();entry({});entry({run:`${run}b`});break;
|
||||
case 'coder':ok();entry({});entry({run:`${run}c`,instance:'coder'});break;
|
||||
default:ok();entry({});
|
||||
}
|
||||
fs.writeFileSync(lf,JSON.stringify(list));
|
||||
};
|
||||
if(verb==='show'){
|
||||
// `show [--timestamp=unix] UNIT -p P...` and `show -p P UNIT`, as the external path calls it.
|
||||
let u='';const props=[];for(let i=0;i<rest.length;i++){if(rest[i]==='-p')props.push(rest[++i]);else if(!rest[i].startsWith('--'))u=rest[i]}
|
||||
const f=unitFile(u),st=load()[u]||{},id=f?path.basename(f.wd):'';
|
||||
// ActiveEnterTimestamp (row 232) is the start's second, or what the enter fault says: a JSON
|
||||
// {value} or {value,at,then}, which answers then from epoch ms at on.
|
||||
const enter=()=>{
|
||||
if(!fault('enter'))return st.enter;
|
||||
const o=JSON.parse(fs.readFileSync(faultFile('enter'),'utf8'));return o.at!==undefined&&Date.now()>=o.at?o.then:o.value;
|
||||
};
|
||||
for(const p of props){
|
||||
if(p==='LoadState')console.log(`LoadState=${f?'loaded':'not-found'}`);
|
||||
else if(p==='WorkingDirectory')console.log(`WorkingDirectory=${f?f.wd+(!isBus(u)&&fault(`check-${id}`)?'-moved':''):''}`);
|
||||
else if(p==='ExecStart')console.log(f&&f.exec?`ExecStart={ path=${f.exec.split(' ')[0]} ; argv[]=${f.exec} ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }`:'ExecStart=');
|
||||
else if(p==='Environment')console.log(`Environment=${f?f.env.join(' '):''}`);
|
||||
else if(p==='MainPID')console.log(`MainPID=${st.pid||0}`);
|
||||
else if(p==='DropInPaths')console.log(`DropInPaths=${f?f.drops.join(' '):''}`);
|
||||
else if(p==='FragmentPath')console.log(`FragmentPath=${f&&!fault('no-fragment')?f.file:''}`);
|
||||
else if(p==='ActiveEnterTimestamp'){const v=st.state==='active'?enter():undefined;console.log(`ActiveEnterTimestamp=${v===undefined?'':'@'+v}`)}
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
if(verb==='is-active'){
|
||||
const q=rest[0]==='--quiet',u=q?rest[1]:rest[0];let s=(load()[u]||{}).state||'inactive';
|
||||
// bus-stops-after N (T29): the next N bus checks answer the real state, every later one answers
|
||||
// inactive, and units.json stays as it is, as if the bus stopped between two checks.
|
||||
if(isBus(u)&&fault('bus-stops-after')){const n=Number(fs.readFileSync(faultFile('bus-stops-after'),'utf8'));if(n>0)fs.writeFileSync(faultFile('bus-stops-after'),String(n-1));else s='inactive'}
|
||||
if(!q)console.log(s);process.exit(s==='active'?0:3);
|
||||
}
|
||||
if(verb==='is-enabled'){console.log('disabled');process.exit(1)}
|
||||
if(verb==='daemon-reload')process.exit(0);
|
||||
if(verb==='list-units'){
|
||||
if(fault('list-fails'))process.exit(1);
|
||||
const pats=rest.filter((a)=>!a.startsWith('--')).map((p)=>new RegExp('^'+p.replace(/\./g,'\\.').replace(/\*/g,'.*')+'$'));
|
||||
for(const [u,s] of Object.entries(load()))if(pats.some((re)=>re.test(u)))console.log(`${u} loaded ${s.state} ${s.state==='active'?'running':'dead'} fixture`);
|
||||
process.exit(0);
|
||||
}
|
||||
if(verb==='start'){
|
||||
const u=rest[0],f=unitFile(u);if(!f){console.error(`Unit ${u} not found.`);process.exit(5)}
|
||||
const s=load(),id=path.basename(f.wd),d=dataOf(f);
|
||||
const enter=Math.floor((Date.now()-Number(process.env.SD_CLOCK_OFFSET||'0')*1000)/1000);
|
||||
if(!isBus(u)){s[u]={state:'active',pid:null,wd:f.wd,enter};save(s);process.exit(0)}
|
||||
if(fault(`start-dies-${id}`)){s[u]={state:'failed',pid:null,wd:f.wd};save(s);process.exit(0)}
|
||||
// Row 185 (T48): the copy's start writes one event before the holder opens the store, so the
|
||||
// store differs from a backup the run took before the start. The close leaves no -wal.
|
||||
if(fault(`start-writes-row-${id}`)){
|
||||
const db=new (process.getBuiltinModule('node:sqlite').DatabaseSync)(path.join(d,'bus','bus.sqlite'));
|
||||
db.prepare('INSERT INTO events (at) VALUES (?)').run('2999-01-01T00:00:00.000Z');db.close();
|
||||
}
|
||||
snap('start',d);
|
||||
const off=process.env.SD_CLOCK_OFFSET||'0';
|
||||
// The holder must not inherit stack-release.sh's lock (fd 9) or any other fd above 2.
|
||||
const close='for fd in /proc/self/fd/*; do n=${fd##*/}; [ "$n" -gt 2 ] && eval "exec $n>&-" 2>/dev/null; done; exec "$@"';
|
||||
const ch=cp.spawn('bash',['-c',close,'holder',process.execPath,path.join(__dirname,'..','holder.cjs'),d,inst(u),sd,off,perData(d,'host')],{detached:true,stdio:'ignore'});
|
||||
ch.unref();
|
||||
fs.appendFileSync(path.join(sd,'pids.log'),ch.pid+'\n');
|
||||
const ready=path.join(sd,`ready-${ch.pid}`),end=Date.now()+10000;
|
||||
while(!fs.existsSync(ready)){if(Date.now()>end){console.error('fixture holder did not start');process.exit(1)}sleep(20)}
|
||||
s[u]={state:'active',pid:ch.pid,wd:f.wd,enter};save(s);
|
||||
if(f.exec.split(' ').slice(1).includes('--pm'))pm(u,d,id,ch.pid,Number(off));
|
||||
process.exit(0);
|
||||
}
|
||||
if(verb==='stop'){
|
||||
const u=rest[0],s=load(),st=s[u];
|
||||
if(st&&st.pid){
|
||||
try{process.kill(st.pid,'SIGTERM')}catch{}
|
||||
const end=Date.now()+10000;while(alive(st.pid)&&Date.now()<end)sleep(20);
|
||||
if(alive(st.pid))try{process.kill(st.pid,'SIGKILL')}catch{}
|
||||
}
|
||||
if(st){s[u]={...st,state:'inactive',pid:null};save(s)}
|
||||
if(isBus(u)){
|
||||
const d=dataOf(unitFile(u));
|
||||
// One-shot: the stop leaves an empty -wal or -shm beside the store, as an unclean close would.
|
||||
for(const [name,ext] of [['stop-leaves-wal','-wal'],['stop-leaves-shm','-shm']])if(fault(name)){fs.rmSync(faultFile(name));fs.writeFileSync(path.join(d,'bus','bus.sqlite'+ext),'')}
|
||||
// One-shot (T45): the stop writes one row after any STAMP into the table the fault names, as a
|
||||
// write the bus made between rollback's count and its stop would. The close leaves no -wal.
|
||||
if(fault('stop-writes-row')){
|
||||
const t=fs.readFileSync(faultFile('stop-writes-row'),'utf8');fs.rmSync(faultFile('stop-writes-row'));
|
||||
const db=new (process.getBuiltinModule('node:sqlite').DatabaseSync)(path.join(d,'bus','bus.sqlite'));
|
||||
db.exec(`CREATE TABLE IF NOT EXISTS ${t} (seq INTEGER PRIMARY KEY, at TEXT NOT NULL)`);
|
||||
db.prepare(`INSERT INTO ${t} (at) VALUES (?)`).run('2999-01-01T00:00:00.000Z');db.close();
|
||||
}
|
||||
snap('stop',d);
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
console.error(`fixture systemctl: unsupported: ${args.join(' ')}`);process.exit(1);
|
||||
}
|
||||
|
||||
// journalctl --user -u UNIT --since @EPOCH -o cat. A bare local time gets exit 1, as M35 needs.
|
||||
export function journalctl(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path');
|
||||
const sd=process.env.SD_STATE,a=process.argv.slice(2);let unit='',since='';
|
||||
for(let i=0;i<a.length;i++){if(a[i]==='-u')unit=a[++i];else if(a[i]==='--since')since=a[++i]}
|
||||
fs.appendFileSync(path.join(sd,'journal-since.log'),since+'\n');
|
||||
if(fs.existsSync(path.join(sd,'faults','journal-fails')))process.exit(1);
|
||||
let from;
|
||||
if(/^@\d+$/.test(since))from=Number(since.slice(1));
|
||||
else{const m=/^(\d{4}-\d\d-\d\d) (\d\d:\d\d:\d\d) UTC$/.exec(since);if(!m)process.exit(1);from=Date.parse(`${m[1]}T${m[2]}Z`)/1000}
|
||||
let lines=[];try{lines=fs.readFileSync(path.join(sd,'journal.jsonl'),'utf8').split('\n').filter(Boolean).map((l)=>JSON.parse(l))}catch{}
|
||||
for(const o of lines)if(o.unit===unit&&o.epoch>=from)console.log(o.text);
|
||||
}
|
||||
|
||||
export function systemdAnalyze(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path');
|
||||
if(process.argv[2]!=='--user'||process.argv[3]!=='unit-paths')process.exit(1);
|
||||
console.log(path.join(process.env.XDG_CONFIG_HOME,'systemd','user'));
|
||||
// The unit-path fault adds its text as one more path, as T28's relative path needs.
|
||||
const extra=path.join(process.env.SD_STATE,'faults','unit-path');
|
||||
if(fs.existsSync(extra))console.log(fs.readFileSync(extra,'utf8'));
|
||||
}
|
||||
|
||||
// The bus host: holds the store open in WAL mode until SIGTERM, as the broker does. It writes its
|
||||
// status to the host file systemctl names for its data root.
|
||||
export function holder(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),{DatabaseSync}=process.getBuiltinModule('node:sqlite');
|
||||
const [data,business,sd,offset,host]=process.argv.slice(2),bus=path.join(data,'bus');
|
||||
fs.mkdirSync(bus,{recursive:true,mode:0o700});
|
||||
const db=new DatabaseSync(path.join(bus,'bus.sqlite'));
|
||||
if(!db.prepare("SELECT count(*) AS n FROM sqlite_master WHERE name='meta'").get().n){
|
||||
db.exec('PRAGMA journal_mode=WAL');
|
||||
db.exec('CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL)');
|
||||
db.exec('CREATE TABLE events (id INTEGER PRIMARY KEY, at TEXT NOT NULL)');
|
||||
db.prepare("INSERT INTO meta VALUES ('schema_version', '3g')").run();
|
||||
}
|
||||
fs.writeFileSync(path.join(bus,'writer.lock'),String(process.pid));
|
||||
const at=new Date(Date.now()-Number(offset||0)*1000).toISOString();
|
||||
fs.writeFileSync(host+'.tmp',JSON.stringify({host:{business,pid:process.pid,live:true,startedAt:at,notifier:'ok'},boot:{ok:true,business,at}}));
|
||||
fs.renameSync(host+'.tmp',host);
|
||||
process.on('SIGTERM',()=>{db.close();fs.rmSync(path.join(bus,'writer.lock'),{force:true});process.exit(0)});
|
||||
fs.writeFileSync(path.join(sd,`ready-${process.pid}`),'');
|
||||
setInterval(()=>{},1<<30);
|
||||
}
|
||||
|
||||
// A copy's scripts/mosaic: bus status and launches list from what the holder and systemctl wrote
|
||||
// for the data root of the config it runs under (row 190), the default's being $SD_DATA.
|
||||
export function mosaicStub(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path');
|
||||
const id=path.basename(path.resolve(__dirname,'..')),sd=process.env.SD_STATE,a=process.argv.slice(2).join(' ');
|
||||
const fault=(n)=>fs.existsSync(path.join(sd,'faults',n));
|
||||
let d=process.env.SD_DATA;try{d=JSON.parse(fs.readFileSync(process.env.MOSAIC_CONFIG,'utf8')).dataRoot}catch{}
|
||||
const perData=(n)=>path.join(sd,d===process.env.SD_DATA?`${n}.json`:`${n}-${path.basename(d)}.json`);
|
||||
const alive=(pid)=>{try{const s=fs.readFileSync(`/proc/${pid}/stat`,'utf8');return s[s.lastIndexOf(')')+2]!=='Z'}catch{return false}};
|
||||
if(a==='bus status --json'){
|
||||
if(fault(`status-${id}`))process.exit(1);
|
||||
// Row 232: status-until-<copy> fails bus status until the epoch ms it holds.
|
||||
const until=path.join(sd,'faults',`status-until-${id}`);
|
||||
if(fs.existsSync(until)&&Date.now()<Number(fs.readFileSync(until,'utf8')))process.exit(1);
|
||||
let s={host:null,boot:null};
|
||||
try{s=JSON.parse(fs.readFileSync(perData('host'),'utf8'));s.host.live=alive(s.host.pid)}catch{}
|
||||
console.log(JSON.stringify(s));
|
||||
}else if(a==='bus status')console.log(`bus: fixture status for ${id}`);
|
||||
else if(a==='launches list --json'){
|
||||
if(fault(`launches-${id}`))process.exit(1);
|
||||
let l=[];try{l=JSON.parse(fs.readFileSync(perData('launches'),'utf8'))}catch{}
|
||||
console.log(JSON.stringify(l));
|
||||
}else{console.error(`fixture mosaic: ${a}`);process.exit(4)}
|
||||
}
|
||||
|
||||
// A copy's migrate-schema.mjs: records the backups it saw, then sets 3h, or 3h-partial and exit 1.
|
||||
export function migrateStub(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),{DatabaseSync}=process.getBuiltinModule('node:sqlite');
|
||||
const data=process.argv[2],sd=process.env.SD_STATE,dir=path.join(data,'backups','stack');
|
||||
const backups=fs.existsSync(dir)?fs.readdirSync(dir).sort().map((name)=>({name,sums:fs.existsSync(path.join(dir,name,'SHA256SUMS'))})):[];
|
||||
fs.writeFileSync(path.join(sd,'migrate.json'),JSON.stringify({backups}));
|
||||
const fail=fs.existsSync(path.join(sd,'faults','migrate-fails'));
|
||||
const db=new DatabaseSync(path.join(data,'bus','bus.sqlite'));
|
||||
db.prepare("UPDATE meta SET value=? WHERE key='schema_version'").run(fail?'3h-partial':'3h');db.close();
|
||||
if(fail)process.exit(1);
|
||||
console.log('fixture migrate: 3h');
|
||||
}
|
||||
|
||||
// --- The module fixture: one repo, its copies and the stand-ins, shared by every test. ---
|
||||
|
||||
export const base=realpathSync(mkdtempSync(join(tmpdir(),'stack-release-')));
|
||||
after(()=>{execFileSync('chmod',['-R','u+w',base]);rmSync(base,{recursive:true,force:true})});
|
||||
export const repo=join(base,'repo'),COPIES=join(base,'copies'),SBIN=join(base,'sbin');
|
||||
mkdirSync(repo);mkdirSync(COPIES);mkdirSync(SBIN);
|
||||
writeFileSync(join(base,'package.json'),'{"type":"commonjs"}\n');
|
||||
export const tracked=execFileSync('git',['-C',ROOT,'ls-files','-co','--exclude-standard','--','scripts','packages','adapters','roles','RELEASE','package.json','package-lock.json',':!packages/*/tests',':!scripts/foundation'],{encoding:'utf8'}).trim().split('\n');
|
||||
for(const f of tracked)if(existsSync(join(ROOT,f)))cpSync(join(ROOT,f),join(repo,f));
|
||||
export const pkg={name:'copy-fixture',version:'1.0.0'};
|
||||
writeFileSync(join(repo,'package.json'),JSON.stringify(pkg));
|
||||
writeFileSync(join(repo,'package-lock.json'),JSON.stringify({name:pkg.name,version:pkg.version,lockfileVersion:3,packages:{'':pkg}}));
|
||||
git(repo,'init','-q','-b','refactor');git(repo,'config','gc.auto','0');git(repo,'add','.');git(repo,'commit','-qm','fixture');
|
||||
export const commit=(msg)=>{git(repo,'commit','-q','--allow-empty','-m',msg);return git(repo,'rev-parse','HEAD')};
|
||||
// refactor: F A B C1 C, then M merging side branch S (from A). U sits on a branch nothing reaches.
|
||||
export const F=git(repo,'rev-parse','HEAD'),A=commit('row 901: A'),B=commit('row 902: B'),C1=commit('row 903: C1'),C=commit('row 904: C');
|
||||
git(repo,'checkout','-q','-b','side',A);export const S=commit('row 905: S');
|
||||
git(repo,'checkout','-q','-b','unreached',A);export const U=commit('row 906: U');
|
||||
git(repo,'checkout','-q','refactor');git(repo,'merge','-q','--no-ff','-m','merge side','side');
|
||||
export const M=git(repo,'rev-parse','HEAD');git(repo,'update-ref','refs/remotes/origin/refactor',M);
|
||||
// `to` false leaves out the `to:` lines (C1); a string puts that value in each one (A). The --plan
|
||||
// steps line can't be read from either (T27).
|
||||
export const storeText=(v,froms,to=true)=>`export const SCHEMA_VERSION = '${v}';\nexport const MIGRATIONS = [\n${froms.map((f,i)=>` {\n from: '${f}',\n${to===false?'':` to: '${to===true?froms[i+1]??v:to}',\n`} run() {},\n },\n`).join('')}];\n`;
|
||||
// Plug-in kinds (row 169): C1 and C know the harness kind, as a copy at or past c216f47db does. The
|
||||
// rest have it under another name, as ca15d73e9 has no harness kind. Every copy has the real business
|
||||
// loader and roles, which the tool imports to read the business file.
|
||||
export const KINDS_NEW=readFileSync(join(repo,'packages','plugins','src','kinds.mjs'),'utf8');
|
||||
export const KINDS_OLD=KINDS_NEW.replace('\n harness: Object.freeze({','\n "harness-before-c216f47db": Object.freeze({');
|
||||
assert.notEqual(KINDS_OLD,KINDS_NEW);
|
||||
// Row 200 (#1698): C1 and C take --sso and have the row 192 loader and checkConsole; C's checker
|
||||
// refuses RS256, a stand-in for a target whose rules differ from the repository's. B has the loader
|
||||
// from before row 192, which refuses a `console` key. The other copies have no loader.
|
||||
const LOADER=readFileSync(join(repo,'scripts','mosaic-config.mjs'),'utf8'),SSO_CONFIG=readFileSync(join(repo,'packages','webui','src','sso-config.mjs'),'utf8');
|
||||
const LOADER_OLD=LOADER.replace('["configVersion", "environment", "dataRoot", "execution", "console"]','["configVersion", "environment", "dataRoot", "execution"]');
|
||||
const SSO_CONFIG_C=SSO_CONFIG.replace("'ES256', 'RS256',","'ES256',");
|
||||
assert.notEqual(LOADER_OLD,LOADER);assert.notEqual(SSO_CONFIG_C,SSO_CONFIG);
|
||||
const ssoFiles=(id)=>id===B?[['scripts/mosaic-config.mjs',LOADER_OLD]]:[C1,C].includes(id)?[['scripts/mosaic-config.mjs',LOADER],['packages/webui/src/sso-config.mjs',id===C?SSO_CONFIG_C:SSO_CONFIG]]:[];
|
||||
for(const [id,store] of [[F,'// a copy from before SCHEMA_VERSION\n'],[A,storeText('3g',['3f'],'3x')],[B,storeText('3g',['3f'])],[C1,storeText('3g',['3f'],false)],[C,storeText('3h',['3f','3g'])],[S,storeText('3g',['3f'])],[U,storeText('3g',['3f'])]]){
|
||||
const d=join(COPIES,id);
|
||||
for(const [f,text,mode] of [['scripts/mosaic',script(mosaicStub),0o755],['packages/webui/src/cli.mjs',[B,C1,C].includes(id)?`// fixture Console, never run\n// serve accepts '--tester-port'\n// serve accepts '--proof-header' (row 178)\n${[C1,C].includes(id)?"// serve accepts '--sso' (row 192)\n":''}`:'// fixture Console before tester listener, never run\n',0o644],['packages/bus/src/store.mjs',store,0o644],['packages/bus/src/migrate-schema.mjs',`(${migrateStub})();\n`,0o644],['packages/plugins/src/kinds.mjs',[C1,C].includes(id)?KINDS_NEW:KINDS_OLD,0o644],...ssoFiles(id).map(([f,text])=>[f,text,0o644])]){
|
||||
mkdirSync(dirname(join(d,f)),{recursive:true});writeFileSync(join(d,f),text,{mode});
|
||||
}
|
||||
for(const p of ['packages/business/src','roles'])cpSync(join(repo,p),join(d,p),{recursive:true});
|
||||
execFileSync('chmod',['-R','a-w',d]);
|
||||
}
|
||||
for(const [name,fn] of [['systemctl',systemctl],['journalctl',journalctl],['systemd-analyze',systemdAnalyze]])writeFileSync(join(SBIN,name),script(fn),{mode:0o755});
|
||||
writeFileSync(join(base,'holder.cjs'),`(${holder})();\n`);
|
||||
export const SR=join(repo,'scripts','stack-release.sh');
|
||||
|
||||
// --- Per-test roots and helpers ---
|
||||
|
||||
export const sha=(f)=>createHash('sha256').update(readFileSync(f)).digest('hex');
|
||||
export const lines=(f)=>existsSync(f)?readFileSync(f,'utf8').split('\n').filter(Boolean):[];
|
||||
export const calls=(e)=>lines(join(e.sd,'calls.log'));
|
||||
export const startsStops=(e,from)=>calls(e).slice(from).filter((c)=>/^(start|stop) /.test(c));
|
||||
export const logLines=(e)=>lines(e.log).map((l)=>JSON.parse(l));
|
||||
export const pointer=(e)=>existsSync(e.pointer)?JSON.parse(readFileSync(e.pointer,'utf8')):null;
|
||||
export const units=(e)=>JSON.parse(readFileSync(join(e.sd,'units.json'),'utf8'));
|
||||
export const snaps=(e)=>lines(join(e.sd,'snap.jsonl')).map((l)=>JSON.parse(l));
|
||||
export const fault=(e,name,text='')=>writeFileSync(join(e.sd,'faults',name),text);
|
||||
export const sysctl=(e,...a)=>execFileSync(join(SBIN,'systemctl'),['--user',...a],{env:e.env,encoding:'utf8'});
|
||||
export const edit=(f,from,to)=>{const s=readFileSync(f,'utf8');assert.ok(s.includes(from),`${f} has no ${from}`);writeFileSync(f,s.replace(from,to))};
|
||||
export const posts=(e)=>readdirSync(e.data).filter((n)=>n.startsWith('bus.post-attempt-'));
|
||||
export const backupSum=(e,act)=>readFileSync(join(e.data,'backups','stack',act,'SHA256SUMS'),'utf8').split(' ')[0];
|
||||
export const commitLines=(range)=>git(repo,'log','--oneline','--no-merges','--max-count=200',range).split('\n').map((l)=>` ${l}`);
|
||||
// The units, the wrapper, the pointer and the store, by sha256.
|
||||
export const fp=(e)=>({bus:sha(join(e.units,e.tmpl)),console:sha(join(e.units,e.con)),wrapper:sha(join(e.bin,e.wrapper)),pointer:existsSync(e.pointer)?sha(e.pointer):null,store:sha(e.store)});
|
||||
// The schema version: readOnly while the holder has the store open, immutable when it is stopped.
|
||||
export function storeVersion(file,held){
|
||||
const db=held?new DatabaseSync(file,{readOnly:true}):new DatabaseSync(pathToFileURL(file).href+'?immutable=1',{readOnly:true});
|
||||
try{return db.prepare("SELECT value FROM meta WHERE key='schema_version'").get().value}finally{db.close()}
|
||||
}
|
||||
// Every file and directory under p with size and mtime: T23 and T16 compare it before and after.
|
||||
export function tree(p){
|
||||
const out=[];
|
||||
const walk=(q)=>{let st;try{st=statSync(q)}catch{return}out.push(st.isDirectory()?`${q}/`:`${q} ${st.size} ${st.mtimeMs}`);if(st.isDirectory())for(const n of readdirSync(q).sort())walk(join(q,n))};
|
||||
walk(p);return out;
|
||||
}
|
||||
// T17: what a run may never remove. Under --units external confirm removes a set's AWAITING-UNITS
|
||||
// and FAILED when it closes the activation (row 232, r6 A5 and R4); user mode writes neither.
|
||||
const markers=/\/(AWAITING-UNITS|FAILED) /;
|
||||
export const kept=(e)=>[...tree(join(e.data,'backups','stack')),...tree(join(e.state,'stack-rollback')).filter((l)=>!markers.test(l)),...posts(e).flatMap((n)=>tree(join(e.data,n))),...tree(COPIES)].map((l)=>l.replace(/ .*/,''));
|
||||
|
||||
// The business file (row 169): the committed example made loadable, as acme, with `plugins` when given.
|
||||
export const EXAMPLE=readFileSync(join(repo,'packages','business','examples','mosaic-stack.example.json'),'utf8');
|
||||
export function biz(e,plugins){
|
||||
const b=JSON.parse(EXAMPLE);let n=0;
|
||||
const walk=(o)=>{for(const [k,v] of Object.entries(o)){if(k==='botId')o[k]=++n;else if(v==='YYYY-MM-DD')o[k]='2027-01-01';else if(v&&typeof v==='object')walk(v)}};
|
||||
walk(b);b.id='acme';if(plugins!==undefined)b.plugins=plugins;
|
||||
const f=join(dirname(e.cfg),'businesses','acme.json');mkdirSync(dirname(f),{recursive:true});
|
||||
writeFileSync(f,JSON.stringify(b,null,2)+'\n',{mode:0o600});chmodSync(f,0o600);
|
||||
}
|
||||
|
||||
export async function mk(t){
|
||||
const root=realpathSync(mkdtempSync(join(tmpdir(),'sr-')));
|
||||
const e={root,data:join(root,'data'),bin:join(root,'bin'),sd:join(root,'sd'),units:join(root,'xcfg','systemd','user'),cfg:join(root,'cfg','config.json'),claude:join(root,'claude-bin','claude')};
|
||||
// The default instance's names; named() sets NAME's (row 190).
|
||||
Object.assign(e,{flags:[],bus:BUS,tmpl:'[email protected]',con:CON,wrapper:'mosaic'});
|
||||
Object.assign(e,{state:join(e.data,'state'),pointer:join(e.data,'state','stack-active.json'),log:join(e.data,'state','stack-activation-log.jsonl'),store:join(e.data,'bus','bus.sqlite'),lock:join(e.data,'state','stack-release.lock')});
|
||||
for(const d of [join(root,'home'),dirname(e.cfg),join(e.data,'bus'),e.bin,dirname(e.claude),join(e.sd,'faults'),e.units,join(root,'xdata')])mkdirSync(d,{recursive:true});
|
||||
chmodSync(join(e.data,'bus'),0o700);
|
||||
writeFileSync(e.cfg,JSON.stringify({configVersion:1,environment:'development',dataRoot:e.data,execution:{backend:'docker',provider:'zai',model:'glm-5.3-flash',adapter:'mock'}}));
|
||||
biz(e);
|
||||
writeFileSync(join(e.bin,'mosaic'),`#!/usr/bin/env bash\n${MARKER}\n[ "\${1:-}" = __lease-capability ] && exec "${e.bin}/mosaic-old" "$@"\nexec "/old/scripts/mosaic" "$@"\n`,{mode:0o755});
|
||||
writeFileSync(join(e.bin,'mosaic-old'),'#!/bin/sh\nexit 0\n',{mode:0o755});
|
||||
writeFileSync(e.claude,'#!/bin/sh\necho "9.9.9 (Claude Code fixture)"\n',{mode:0o755});
|
||||
e.env={PATH:`${SBIN}:${dirname(NODE)}:/usr/bin:/bin`,HOME:join(root,'home'),XDG_CONFIG_HOME:join(root,'xcfg'),XDG_DATA_HOME:join(root,'xdata'),MOSAIC_CONFIG:e.cfg,SD_STATE:e.sd,SD_DATA:e.data,TMPDIR:tmpdir()};
|
||||
const server=await healthz(e);
|
||||
t.after(()=>{
|
||||
for(const pid of lines(join(e.sd,'pids.log')).map(Number)){
|
||||
try{if(readFileSync(`/proc/${pid}/cmdline`,'utf8').includes(join(base,'holder.cjs')))process.kill(pid,'SIGKILL')}catch{}
|
||||
}
|
||||
for(const s of [server,...(e.servers??[])])s.close();
|
||||
execFileSync('chmod',['-R','u+w',root]);rmSync(root,{recursive:true,force:true});
|
||||
});
|
||||
return e;
|
||||
}
|
||||
|
||||
// The Console's /healthz on e.port: 200 while the stand-in has e.con active and no healthz-<copy>
|
||||
// fault names its copy.
|
||||
export async function healthz(e){
|
||||
const server=createServer((q,r)=>{
|
||||
let st={};try{st=units(e)[e.con]||{}}catch{}
|
||||
const ok=q.url==='/healthz'&&st.state==='active'&&!existsSync(join(e.sd,'faults',`healthz-${basename(st.wd||'')}`));
|
||||
r.writeHead(ok?200:500);r.end();
|
||||
});
|
||||
server.listen(0,'127.0.0.1');await once(server,'listening');e.port=String(server.address().port);
|
||||
return server;
|
||||
}
|
||||
|
||||
// Row 190: stack instance NAME on e's host. It shares e's stand-ins, unit directory and $BIN, and has
|
||||
// its own config and data root from bootstrap --instance, its own Console port and business acme.
|
||||
// Its runs pass --instance NAME and no MOSAIC_CONFIG.
|
||||
export async function named(e,name){
|
||||
const env={...e.env};delete env.MOSAIC_CONFIG;
|
||||
execFileSync('bash',[join(repo,'scripts','bootstrap.sh'),'--instance',name],{env,stdio:'ignore'});
|
||||
const data=join(env.HOME,`.mosaic-${name}`);
|
||||
const n={...e,env,data,cfg:join(env.HOME,'.config',`mosaic-${name}`,'config.json'),flags:['--instance',name],bus:`mosaic-bus-${name}@acme.service`,tmpl:`mosaic-bus-${name}@.service`,con:`mosaic-webui-${name}.service`,wrapper:`mosaic-${name}`,name};
|
||||
Object.assign(n,{state:join(data,'state'),pointer:join(data,'state','stack-active.json'),log:join(data,'state','stack-activation-log.jsonl'),store:join(data,'bus','bus.sqlite'),lock:join(data,'state','stack-release.lock')});
|
||||
mkdirSync(join(data,'bus'),{mode:0o700});
|
||||
biz(n);
|
||||
(e.servers??=[]).push(await healthz(n));
|
||||
return n;
|
||||
}
|
||||
|
||||
export async function run(e,...args){
|
||||
const before=existsSync(e.data)?kept(e):[];
|
||||
const ch=spawn('bash',[e.sr??SR,...args,...e.flags,'--copies',COPIES,'--bin',e.bin],{env:e.env,stdio:['ignore','pipe','pipe']});
|
||||
let out='',err='';ch.stdout.on('data',(d)=>{out+=d});ch.stderr.on('data',(d)=>{err+=d});
|
||||
const timer=setTimeout(()=>ch.kill('SIGKILL'),120000);
|
||||
const [code]=await once(ch,'close');clearTimeout(timer);
|
||||
const now=new Set(kept(e));
|
||||
for(const p of before)assert.ok(now.has(p),`T17: ${args.join(' ')} removed ${p}`);
|
||||
return {code,out,err};
|
||||
}
|
||||
|
||||
// The stack on copy `copy` for business acme, both units active. pointer: adopt (a real adopt and
|
||||
// its log line), hand (written here, no log) or none. Snapshots start after the seed.
|
||||
export async function seed(e,{copy=B,pm=false,store='3g',pointer:how='adopt'}={}){
|
||||
const db=new DatabaseSync(e.store);
|
||||
db.exec('PRAGMA journal_mode=WAL');
|
||||
db.exec('CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL)');
|
||||
db.exec('CREATE TABLE events (id INTEGER PRIMARY KEY, at TEXT NOT NULL)');
|
||||
db.prepare("INSERT INTO meta VALUES ('schema_version', ?)").run(store);
|
||||
for(let i=0;i<3;i++)db.prepare('INSERT INTO events (at) VALUES (?)').run(`2000-01-01T00:00:0${i}.000Z`);
|
||||
db.close();
|
||||
const sh=(name,...a)=>execFileSync('bash',[join(repo,'scripts',name),...a],{env:e.env,encoding:'utf8',stdio:['ignore','pipe','pipe']});
|
||||
sh('bus-service.sh','install',...e.flags,'--copy',copy,'--copies',COPIES,...(pm?['--pm']:[]),'--claude',e.claude);
|
||||
sh('webui-service.sh','install',...e.flags,'--copy',copy,'--copies',COPIES,'--port',e.port,'--business','acme');
|
||||
sh('stack-copy.sh','wrapper',copy,...e.flags,'--copies',COPIES,'--bin',e.bin);
|
||||
// The seeded host started two minutes ago, so its boot status and PM run predate any run's start.
|
||||
execFileSync(join(SBIN,'systemctl'),['--user','start',e.bus],{env:{...e.env,SD_CLOCK_OFFSET:'120'}});
|
||||
sysctl(e,'start',e.con);
|
||||
if(how==='adopt'){const r=await run(e,'adopt','acme');assert.equal(r.code,0,r.err)}
|
||||
else if(how==='hand'){
|
||||
mkdirSync(e.state,{recursive:true});
|
||||
writeFileSync(e.pointer,JSON.stringify({copy,business:'acme',pm,claude:e.claude,console:['--port',e.port,'--business','acme'],schema:'3g',activation:'manual',at:'2026-10-18T00:00:00Z'})+'\n');
|
||||
}
|
||||
writeFileSync(join(e.sd,'snap.jsonl'),'');
|
||||
}
|
||||
|
||||
export const rolledBack=(code,to=B)=>`stack-release: ${code}; rolled back to ${to}, which passed the gate\n`;
|
||||
export const planBackup=(e)=>`backup: the run backs up the stopped store to ${e.data}/backups/stack/<activation>/ before any migration`;
|
||||
export const planNoBackup='backup: none; the run rolls back before the backup';
|
||||
export const planTail=(e,to,pm=false)=>[`bus: scripts/bus-service.sh install --copy ${to} --copies ${COPIES}${pm?' --pm':''} --claude ${e.claude}`,'proof: off','send: off',`Console: scripts/webui-service.sh install --copy ${to} --copies ${COPIES} --port ${e.port} --business acme`,`CLI: scripts/stack-copy.sh wrapper ${to} --copies ${COPIES} --bin ${e.bin}`];
|
||||
@@ -13,422 +13,7 @@ import {join,dirname,basename} from 'node:path';
|
||||
import {fileURLToPath,pathToFileURL} from 'node:url';
|
||||
import {once} from 'node:events';
|
||||
import {ALGS} from '../../webui/src/sso-config.mjs';
|
||||
|
||||
const ROOT=fileURLToPath(new URL('../../..',import.meta.url)).replace(/\/$/,'');
|
||||
const NODE=process.execPath;
|
||||
const MARKER='# mosaic-cli wrapper (Q81, agents/sage/work/q81/mosaic-cli.sh)';
|
||||
const BUS='[email protected]',CON='mosaic-webui.service';
|
||||
const git=(cwd,...args)=>execFileSync('git',['-c','user.name=fixture','-c','user.email=fixture@invalid','-c','commit.gpgsign=false','-c','core.hooksPath=/dev/null','-C',cwd,...args],{encoding:'utf8'}).trim();
|
||||
const script=(fn)=>`#!${NODE}\n(${fn})();\n`;
|
||||
|
||||
// --- Stand-ins. Each is serialized into its own file, so it uses only process and its own scope. ---
|
||||
|
||||
// systemctl --user: unit state in $SD_STATE/units.json, every call in calls.log. A bus start runs
|
||||
// holder.cjs (the store held open in WAL mode, writer.lock, host.json) and, with --pm in the unit's
|
||||
// ExecStart, writes the `launched` journal line and the launches entry unless a pm-<copy> fault
|
||||
// names another case. Faults are files in $SD_STATE/faults named for the copy they hit. Units of a
|
||||
// named instance (row 190) run on their own data root and host files beside the default's.
|
||||
function systemctl(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),cp=process.getBuiltinModule('node:child_process'),crypto=process.getBuiltinModule('node:crypto');
|
||||
const sd=process.env.SD_STATE,data=process.env.SD_DATA,unitDir=path.join(process.env.XDG_CONFIG_HOME,'systemd','user');
|
||||
const args=process.argv.slice(2).filter((a)=>a!=='--user'),[verb,...rest]=args;
|
||||
fs.appendFileSync(path.join(sd,'calls.log'),args.join(' ')+'\n');
|
||||
const faultFile=(n)=>path.join(sd,'faults',n),fault=(n)=>fs.existsSync(faultFile(n));
|
||||
const stFile=path.join(sd,'units.json');
|
||||
const load=()=>{try{return JSON.parse(fs.readFileSync(stFile,'utf8'))}catch{return {}}};
|
||||
const save=(s)=>{fs.writeFileSync(stFile+'.tmp',JSON.stringify(s));fs.renameSync(stFile+'.tmp',stFile)};
|
||||
const sleep=(ms)=>Atomics.wait(new Int32Array(new SharedArrayBuffer(4)),0,0,ms);
|
||||
const alive=(pid)=>{try{const s=fs.readFileSync(`/proc/${pid}/stat`,'utf8');return s[s.lastIndexOf(')')+2]!=='Z'}catch{return false}};
|
||||
// Row 190: a bus unit is the default's mosaic-bus@B or instance NAME's mosaic-bus-NAME@B; %i is B.
|
||||
const isBus=(u)=>/^mosaic-bus(-[a-z][a-z0-9]{1,15})?@[a-z0-9-]+\.service$/.test(u),inst=(u)=>u.slice(u.indexOf('@')+1,-'.service'.length);
|
||||
const unitFile=(u)=>{
|
||||
const main=isBus(u)?u.replace(/@[^@]*$/,'@.service'):u,file=path.join(unitDir,main);
|
||||
if(!fs.existsSync(file))return null;
|
||||
// The drop-in directories in systemd's order: service.d, the name cut after each hyphen, then
|
||||
// the template's and the unit's own.
|
||||
const stem=main.slice(0,main.search(/@?\.service$/)),dirs=['service.d'],drops=[];
|
||||
for(let i=0;i<stem.length;i++)if(stem[i]==='-')dirs.push(stem.slice(0,i+1)+'.service.d');
|
||||
for(const d of [...dirs,...(isBus(u)?[main+'.d',u+'.d']:[u+'.d'])]){const dir=path.join(unitDir,d);if(fs.existsSync(dir))for(const f of fs.readdirSync(dir).sort())if(f.endsWith('.conf'))drops.push(path.join(dir,f))}
|
||||
const r={wd:'',exec:'',env:[],drops};
|
||||
for(const f of [file,...drops])for(let line of fs.readFileSync(f,'utf8').split('\n')){
|
||||
line=line.replace(/%i/g,isBus(u)?inst(u):'').replace(/%h/g,process.env.HOME);
|
||||
const m=/^(WorkingDirectory|ExecStart|Environment)=(.*)$/.exec(line);if(!m)continue;
|
||||
if(m[1]==='WorkingDirectory')r.wd=m[2];else if(m[1]==='ExecStart'){if(m[2])r.exec=m[2]}else r.env.push(m[2]);
|
||||
}
|
||||
return r;
|
||||
};
|
||||
// A named instance's unit sets MOSAIC_CONFIG, and its bus works on that config's data root. A unit
|
||||
// without it is the default's, on $SD_DATA. Each data root has its own host and launches files.
|
||||
const dataOf=(f)=>{const c=f&&f.env.find((v)=>v.startsWith('MOSAIC_CONFIG='));return c?JSON.parse(fs.readFileSync(c.slice('MOSAIC_CONFIG='.length),'utf8')).dataRoot:data};
|
||||
const perData=(d,n)=>path.join(sd,d===data?`${n}.json`:`${n}-${path.basename(d)}.json`);
|
||||
const snap=(ev,d)=>{
|
||||
const dir=path.join(d,'bus'),files={};
|
||||
try{for(const n of fs.readdirSync(dir).sort()){const p=path.join(dir,n),st=fs.statSync(p);if(st.isFile())files[n]=[st.size,st.mtimeMs,crypto.createHash('sha256').update(fs.readFileSync(p)).digest('hex')]}}catch{}
|
||||
fs.appendFileSync(path.join(sd,'snap.jsonl'),JSON.stringify({ev,files})+'\n');
|
||||
};
|
||||
const pm=(unit,d,id,pid,off)=>{
|
||||
let mode='';try{mode=fs.readFileSync(faultFile(`pm-${id}`),'utf8').trim()}catch{}
|
||||
const b=inst(unit),now=Date.now()-off*1000,epoch=Math.floor(now/1000),at=new Date(now).toISOString(),run=`run-${pid}`;
|
||||
const line=(text)=>fs.appendFileSync(path.join(sd,'journal.jsonl'),JSON.stringify({epoch,unit,text})+'\n');
|
||||
const ok=()=>line(`launched pm-${b}: run ${run}, claude opus`);
|
||||
const lf=perData(d,'launches');let list=[];try{list=JSON.parse(fs.readFileSync(lf,'utf8'))}catch{}
|
||||
const entry=(o)=>list.push({run,business:b,live:true,startedAt:at,...o});
|
||||
line(`bus host up: business ${b}, socket /fixture/bus.sock, notifier off`);
|
||||
switch(mode){
|
||||
case 'none':break;
|
||||
case 'two':ok();line(`launched pm-${b}: run ${run}b, claude opus`);entry({});break;
|
||||
case 'badword':line(`launched pm-${b}: job ${run}, claude opus`);entry({});break;
|
||||
case 'notlisted':ok();break;
|
||||
case 'early':ok();entry({startedAt:new Date(now-3600e3).toISOString()});break;
|
||||
case 'human':entry({run:`human-${pid}`});break;
|
||||
case 'other':ok();entry({business:'other'});break;
|
||||
case 'dead':ok();entry({live:false});break;
|
||||
default:ok();entry({});
|
||||
}
|
||||
fs.writeFileSync(lf,JSON.stringify(list));
|
||||
};
|
||||
if(verb==='show'){
|
||||
const u=rest[0],props=[];for(let i=1;i<rest.length;i++)if(rest[i]==='-p')props.push(rest[++i]);
|
||||
const f=unitFile(u),st=load()[u]||{},id=f?path.basename(f.wd):'';
|
||||
for(const p of props){
|
||||
if(p==='LoadState')console.log(`LoadState=${f?'loaded':'not-found'}`);
|
||||
else if(p==='WorkingDirectory')console.log(`WorkingDirectory=${f?f.wd+(!isBus(u)&&fault(`check-${id}`)?'-moved':''):''}`);
|
||||
else if(p==='ExecStart')console.log(f&&f.exec?`ExecStart={ path=${f.exec.split(' ')[0]} ; argv[]=${f.exec} ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }`:'ExecStart=');
|
||||
else if(p==='Environment')console.log(`Environment=${f?f.env.join(' '):''}`);
|
||||
else if(p==='MainPID')console.log(`MainPID=${st.pid||0}`);
|
||||
else if(p==='DropInPaths')console.log(`DropInPaths=${f?f.drops.join(' '):''}`);
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
if(verb==='is-active'){
|
||||
const q=rest[0]==='--quiet',u=q?rest[1]:rest[0];let s=(load()[u]||{}).state||'inactive';
|
||||
// bus-stops-after N (T29): the next N bus checks answer the real state, every later one answers
|
||||
// inactive, and units.json stays as it is, as if the bus stopped between two checks.
|
||||
if(isBus(u)&&fault('bus-stops-after')){const n=Number(fs.readFileSync(faultFile('bus-stops-after'),'utf8'));if(n>0)fs.writeFileSync(faultFile('bus-stops-after'),String(n-1));else s='inactive'}
|
||||
if(!q)console.log(s);process.exit(s==='active'?0:3);
|
||||
}
|
||||
if(verb==='is-enabled'){console.log('disabled');process.exit(1)}
|
||||
if(verb==='daemon-reload')process.exit(0);
|
||||
if(verb==='list-units'){
|
||||
if(fault('list-fails'))process.exit(1);
|
||||
const pats=rest.filter((a)=>!a.startsWith('--')).map((p)=>new RegExp('^'+p.replace(/\./g,'\\.').replace(/\*/g,'.*')+'$'));
|
||||
for(const [u,s] of Object.entries(load()))if(pats.some((re)=>re.test(u)))console.log(`${u} loaded ${s.state} ${s.state==='active'?'running':'dead'} fixture`);
|
||||
process.exit(0);
|
||||
}
|
||||
if(verb==='start'){
|
||||
const u=rest[0],f=unitFile(u);if(!f){console.error(`Unit ${u} not found.`);process.exit(5)}
|
||||
const s=load(),id=path.basename(f.wd),d=dataOf(f);
|
||||
if(!isBus(u)){s[u]={state:'active',pid:null,wd:f.wd};save(s);process.exit(0)}
|
||||
if(fault(`start-dies-${id}`)){s[u]={state:'failed',pid:null,wd:f.wd};save(s);process.exit(0)}
|
||||
// Row 185 (T48): the copy's start writes one event before the holder opens the store, so the
|
||||
// store differs from a backup the run took before the start. The close leaves no -wal.
|
||||
if(fault(`start-writes-row-${id}`)){
|
||||
const db=new (process.getBuiltinModule('node:sqlite').DatabaseSync)(path.join(d,'bus','bus.sqlite'));
|
||||
db.prepare('INSERT INTO events (at) VALUES (?)').run('2999-01-01T00:00:00.000Z');db.close();
|
||||
}
|
||||
snap('start',d);
|
||||
const off=process.env.SD_CLOCK_OFFSET||'0';
|
||||
// The holder must not inherit stack-release.sh's lock (fd 9) or any other fd above 2.
|
||||
const close='for fd in /proc/self/fd/*; do n=${fd##*/}; [ "$n" -gt 2 ] && eval "exec $n>&-" 2>/dev/null; done; exec "$@"';
|
||||
const ch=cp.spawn('bash',['-c',close,'holder',process.execPath,path.join(__dirname,'..','holder.cjs'),d,inst(u),sd,off,perData(d,'host')],{detached:true,stdio:'ignore'});
|
||||
ch.unref();
|
||||
fs.appendFileSync(path.join(sd,'pids.log'),ch.pid+'\n');
|
||||
const ready=path.join(sd,`ready-${ch.pid}`),end=Date.now()+10000;
|
||||
while(!fs.existsSync(ready)){if(Date.now()>end){console.error('fixture holder did not start');process.exit(1)}sleep(20)}
|
||||
s[u]={state:'active',pid:ch.pid,wd:f.wd};save(s);
|
||||
if(f.exec.split(' ').slice(1).includes('--pm'))pm(u,d,id,ch.pid,Number(off));
|
||||
process.exit(0);
|
||||
}
|
||||
if(verb==='stop'){
|
||||
const u=rest[0],s=load(),st=s[u];
|
||||
if(st&&st.pid){
|
||||
try{process.kill(st.pid,'SIGTERM')}catch{}
|
||||
const end=Date.now()+10000;while(alive(st.pid)&&Date.now()<end)sleep(20);
|
||||
if(alive(st.pid))try{process.kill(st.pid,'SIGKILL')}catch{}
|
||||
}
|
||||
if(st){s[u]={...st,state:'inactive',pid:null};save(s)}
|
||||
if(isBus(u)){
|
||||
const d=dataOf(unitFile(u));
|
||||
// One-shot: the stop leaves an empty -wal or -shm beside the store, as an unclean close would.
|
||||
for(const [name,ext] of [['stop-leaves-wal','-wal'],['stop-leaves-shm','-shm']])if(fault(name)){fs.rmSync(faultFile(name));fs.writeFileSync(path.join(d,'bus','bus.sqlite'+ext),'')}
|
||||
// One-shot (T45): the stop writes one row after any STAMP into the table the fault names, as a
|
||||
// write the bus made between rollback's count and its stop would. The close leaves no -wal.
|
||||
if(fault('stop-writes-row')){
|
||||
const t=fs.readFileSync(faultFile('stop-writes-row'),'utf8');fs.rmSync(faultFile('stop-writes-row'));
|
||||
const db=new (process.getBuiltinModule('node:sqlite').DatabaseSync)(path.join(d,'bus','bus.sqlite'));
|
||||
db.exec(`CREATE TABLE IF NOT EXISTS ${t} (seq INTEGER PRIMARY KEY, at TEXT NOT NULL)`);
|
||||
db.prepare(`INSERT INTO ${t} (at) VALUES (?)`).run('2999-01-01T00:00:00.000Z');db.close();
|
||||
}
|
||||
snap('stop',d);
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
console.error(`fixture systemctl: unsupported: ${args.join(' ')}`);process.exit(1);
|
||||
}
|
||||
|
||||
// journalctl --user -u UNIT --since @EPOCH -o cat. A bare local time gets exit 1, as M35 needs.
|
||||
function journalctl(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path');
|
||||
const sd=process.env.SD_STATE,a=process.argv.slice(2);let unit='',since='';
|
||||
for(let i=0;i<a.length;i++){if(a[i]==='-u')unit=a[++i];else if(a[i]==='--since')since=a[++i]}
|
||||
fs.appendFileSync(path.join(sd,'journal-since.log'),since+'\n');
|
||||
if(fs.existsSync(path.join(sd,'faults','journal-fails')))process.exit(1);
|
||||
let from;
|
||||
if(/^@\d+$/.test(since))from=Number(since.slice(1));
|
||||
else{const m=/^(\d{4}-\d\d-\d\d) (\d\d:\d\d:\d\d) UTC$/.exec(since);if(!m)process.exit(1);from=Date.parse(`${m[1]}T${m[2]}Z`)/1000}
|
||||
let lines=[];try{lines=fs.readFileSync(path.join(sd,'journal.jsonl'),'utf8').split('\n').filter(Boolean).map((l)=>JSON.parse(l))}catch{}
|
||||
for(const o of lines)if(o.unit===unit&&o.epoch>=from)console.log(o.text);
|
||||
}
|
||||
|
||||
function systemdAnalyze(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path');
|
||||
if(process.argv[2]!=='--user'||process.argv[3]!=='unit-paths')process.exit(1);
|
||||
console.log(path.join(process.env.XDG_CONFIG_HOME,'systemd','user'));
|
||||
// The unit-path fault adds its text as one more path, as T28's relative path needs.
|
||||
const extra=path.join(process.env.SD_STATE,'faults','unit-path');
|
||||
if(fs.existsSync(extra))console.log(fs.readFileSync(extra,'utf8'));
|
||||
}
|
||||
|
||||
// The bus host: holds the store open in WAL mode until SIGTERM, as the broker does. It writes its
|
||||
// status to the host file systemctl names for its data root.
|
||||
function holder(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),{DatabaseSync}=process.getBuiltinModule('node:sqlite');
|
||||
const [data,business,sd,offset,host]=process.argv.slice(2),bus=path.join(data,'bus');
|
||||
fs.mkdirSync(bus,{recursive:true,mode:0o700});
|
||||
const db=new DatabaseSync(path.join(bus,'bus.sqlite'));
|
||||
if(!db.prepare("SELECT count(*) AS n FROM sqlite_master WHERE name='meta'").get().n){
|
||||
db.exec('PRAGMA journal_mode=WAL');
|
||||
db.exec('CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL)');
|
||||
db.exec('CREATE TABLE events (id INTEGER PRIMARY KEY, at TEXT NOT NULL)');
|
||||
db.prepare("INSERT INTO meta VALUES ('schema_version', '3g')").run();
|
||||
}
|
||||
fs.writeFileSync(path.join(bus,'writer.lock'),String(process.pid));
|
||||
const at=new Date(Date.now()-Number(offset||0)*1000).toISOString();
|
||||
fs.writeFileSync(host+'.tmp',JSON.stringify({host:{business,pid:process.pid,live:true,startedAt:at,notifier:'ok'},boot:{ok:true,business,at}}));
|
||||
fs.renameSync(host+'.tmp',host);
|
||||
process.on('SIGTERM',()=>{db.close();fs.rmSync(path.join(bus,'writer.lock'),{force:true});process.exit(0)});
|
||||
fs.writeFileSync(path.join(sd,`ready-${process.pid}`),'');
|
||||
setInterval(()=>{},1<<30);
|
||||
}
|
||||
|
||||
// A copy's scripts/mosaic: bus status and launches list from what the holder and systemctl wrote
|
||||
// for the data root of the config it runs under (row 190), the default's being $SD_DATA.
|
||||
function mosaicStub(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path');
|
||||
const id=path.basename(path.resolve(__dirname,'..')),sd=process.env.SD_STATE,a=process.argv.slice(2).join(' ');
|
||||
const fault=(n)=>fs.existsSync(path.join(sd,'faults',n));
|
||||
let d=process.env.SD_DATA;try{d=JSON.parse(fs.readFileSync(process.env.MOSAIC_CONFIG,'utf8')).dataRoot}catch{}
|
||||
const perData=(n)=>path.join(sd,d===process.env.SD_DATA?`${n}.json`:`${n}-${path.basename(d)}.json`);
|
||||
const alive=(pid)=>{try{const s=fs.readFileSync(`/proc/${pid}/stat`,'utf8');return s[s.lastIndexOf(')')+2]!=='Z'}catch{return false}};
|
||||
if(a==='bus status --json'){
|
||||
if(fault(`status-${id}`))process.exit(1);
|
||||
let s={host:null,boot:null};
|
||||
try{s=JSON.parse(fs.readFileSync(perData('host'),'utf8'));s.host.live=alive(s.host.pid)}catch{}
|
||||
console.log(JSON.stringify(s));
|
||||
}else if(a==='bus status')console.log(`bus: fixture status for ${id}`);
|
||||
else if(a==='launches list --json'){
|
||||
if(fault(`launches-${id}`))process.exit(1);
|
||||
let l=[];try{l=JSON.parse(fs.readFileSync(perData('launches'),'utf8'))}catch{}
|
||||
console.log(JSON.stringify(l));
|
||||
}else{console.error(`fixture mosaic: ${a}`);process.exit(4)}
|
||||
}
|
||||
|
||||
// A copy's migrate-schema.mjs: records the backups it saw, then sets 3h, or 3h-partial and exit 1.
|
||||
function migrateStub(){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),{DatabaseSync}=process.getBuiltinModule('node:sqlite');
|
||||
const data=process.argv[2],sd=process.env.SD_STATE,dir=path.join(data,'backups','stack');
|
||||
const backups=fs.existsSync(dir)?fs.readdirSync(dir).sort().map((name)=>({name,sums:fs.existsSync(path.join(dir,name,'SHA256SUMS'))})):[];
|
||||
fs.writeFileSync(path.join(sd,'migrate.json'),JSON.stringify({backups}));
|
||||
const fail=fs.existsSync(path.join(sd,'faults','migrate-fails'));
|
||||
const db=new DatabaseSync(path.join(data,'bus','bus.sqlite'));
|
||||
db.prepare("UPDATE meta SET value=? WHERE key='schema_version'").run(fail?'3h-partial':'3h');db.close();
|
||||
if(fail)process.exit(1);
|
||||
console.log('fixture migrate: 3h');
|
||||
}
|
||||
|
||||
// --- The module fixture: one repo, its copies and the stand-ins, shared by every test. ---
|
||||
|
||||
const base=realpathSync(mkdtempSync(join(tmpdir(),'stack-release-')));
|
||||
after(()=>{execFileSync('chmod',['-R','u+w',base]);rmSync(base,{recursive:true,force:true})});
|
||||
const repo=join(base,'repo'),COPIES=join(base,'copies'),SBIN=join(base,'sbin');
|
||||
mkdirSync(repo);mkdirSync(COPIES);mkdirSync(SBIN);
|
||||
writeFileSync(join(base,'package.json'),'{"type":"commonjs"}\n');
|
||||
const tracked=execFileSync('git',['-C',ROOT,'ls-files','-co','--exclude-standard','--','scripts','packages','adapters','roles','RELEASE','package.json','package-lock.json',':!packages/*/tests',':!scripts/foundation'],{encoding:'utf8'}).trim().split('\n');
|
||||
for(const f of tracked)if(existsSync(join(ROOT,f)))cpSync(join(ROOT,f),join(repo,f));
|
||||
const pkg={name:'copy-fixture',version:'1.0.0'};
|
||||
writeFileSync(join(repo,'package.json'),JSON.stringify(pkg));
|
||||
writeFileSync(join(repo,'package-lock.json'),JSON.stringify({name:pkg.name,version:pkg.version,lockfileVersion:3,packages:{'':pkg}}));
|
||||
git(repo,'init','-q','-b','refactor');git(repo,'config','gc.auto','0');git(repo,'add','.');git(repo,'commit','-qm','fixture');
|
||||
const commit=(msg)=>{git(repo,'commit','-q','--allow-empty','-m',msg);return git(repo,'rev-parse','HEAD')};
|
||||
// refactor: F A B C1 C, then M merging side branch S (from A). U sits on a branch nothing reaches.
|
||||
const F=git(repo,'rev-parse','HEAD'),A=commit('row 901: A'),B=commit('row 902: B'),C1=commit('row 903: C1'),C=commit('row 904: C');
|
||||
git(repo,'checkout','-q','-b','side',A);const S=commit('row 905: S');
|
||||
git(repo,'checkout','-q','-b','unreached',A);const U=commit('row 906: U');
|
||||
git(repo,'checkout','-q','refactor');git(repo,'merge','-q','--no-ff','-m','merge side','side');
|
||||
const M=git(repo,'rev-parse','HEAD');git(repo,'update-ref','refs/remotes/origin/refactor',M);
|
||||
// `to` false leaves out the `to:` lines (C1); a string puts that value in each one (A). The --plan
|
||||
// steps line can't be read from either (T27).
|
||||
const storeText=(v,froms,to=true)=>`export const SCHEMA_VERSION = '${v}';\nexport const MIGRATIONS = [\n${froms.map((f,i)=>` {\n from: '${f}',\n${to===false?'':` to: '${to===true?froms[i+1]??v:to}',\n`} run() {},\n },\n`).join('')}];\n`;
|
||||
// Plug-in kinds (row 169): C1 and C know the harness kind, as a copy at or past c216f47db does. The
|
||||
// rest have it under another name, as ca15d73e9 has no harness kind. Every copy has the real business
|
||||
// loader and roles, which the tool imports to read the business file.
|
||||
const KINDS_NEW=readFileSync(join(repo,'packages','plugins','src','kinds.mjs'),'utf8');
|
||||
const KINDS_OLD=KINDS_NEW.replace('\n harness: Object.freeze({','\n "harness-before-c216f47db": Object.freeze({');
|
||||
assert.notEqual(KINDS_OLD,KINDS_NEW);
|
||||
// Row 200 (#1698): C1 and C take --sso and have the row 192 loader and checkConsole; C's checker
|
||||
// refuses RS256, a stand-in for a target whose rules differ from the repository's. B has the loader
|
||||
// from before row 192, which refuses a `console` key. The other copies have no loader.
|
||||
const LOADER=readFileSync(join(repo,'scripts','mosaic-config.mjs'),'utf8'),SSO_CONFIG=readFileSync(join(repo,'packages','webui','src','sso-config.mjs'),'utf8');
|
||||
const LOADER_OLD=LOADER.replace('["configVersion", "environment", "dataRoot", "execution", "console"]','["configVersion", "environment", "dataRoot", "execution"]');
|
||||
const SSO_CONFIG_C=SSO_CONFIG.replace("'ES256', 'RS256',","'ES256',");
|
||||
assert.notEqual(LOADER_OLD,LOADER);assert.notEqual(SSO_CONFIG_C,SSO_CONFIG);
|
||||
const ssoFiles=(id)=>id===B?[['scripts/mosaic-config.mjs',LOADER_OLD]]:[C1,C].includes(id)?[['scripts/mosaic-config.mjs',LOADER],['packages/webui/src/sso-config.mjs',id===C?SSO_CONFIG_C:SSO_CONFIG]]:[];
|
||||
for(const [id,store] of [[F,'// a copy from before SCHEMA_VERSION\n'],[A,storeText('3g',['3f'],'3x')],[B,storeText('3g',['3f'])],[C1,storeText('3g',['3f'],false)],[C,storeText('3h',['3f','3g'])],[S,storeText('3g',['3f'])],[U,storeText('3g',['3f'])]]){
|
||||
const d=join(COPIES,id);
|
||||
for(const [f,text,mode] of [['scripts/mosaic',script(mosaicStub),0o755],['packages/webui/src/cli.mjs',[B,C1,C].includes(id)?`// fixture Console, never run\n// serve accepts '--tester-port'\n// serve accepts '--proof-header' (row 178)\n${[C1,C].includes(id)?"// serve accepts '--sso' (row 192)\n":''}`:'// fixture Console before tester listener, never run\n',0o644],['packages/bus/src/store.mjs',store,0o644],['packages/bus/src/migrate-schema.mjs',`(${migrateStub})();\n`,0o644],['packages/plugins/src/kinds.mjs',[C1,C].includes(id)?KINDS_NEW:KINDS_OLD,0o644],...ssoFiles(id).map(([f,text])=>[f,text,0o644])]){
|
||||
mkdirSync(dirname(join(d,f)),{recursive:true});writeFileSync(join(d,f),text,{mode});
|
||||
}
|
||||
for(const p of ['packages/business/src','roles'])cpSync(join(repo,p),join(d,p),{recursive:true});
|
||||
execFileSync('chmod',['-R','a-w',d]);
|
||||
}
|
||||
for(const [name,fn] of [['systemctl',systemctl],['journalctl',journalctl],['systemd-analyze',systemdAnalyze]])writeFileSync(join(SBIN,name),script(fn),{mode:0o755});
|
||||
writeFileSync(join(base,'holder.cjs'),`(${holder})();\n`);
|
||||
const SR=join(repo,'scripts','stack-release.sh');
|
||||
|
||||
// --- Per-test roots and helpers ---
|
||||
|
||||
const sha=(f)=>createHash('sha256').update(readFileSync(f)).digest('hex');
|
||||
const lines=(f)=>existsSync(f)?readFileSync(f,'utf8').split('\n').filter(Boolean):[];
|
||||
const calls=(e)=>lines(join(e.sd,'calls.log'));
|
||||
const startsStops=(e,from)=>calls(e).slice(from).filter((c)=>/^(start|stop) /.test(c));
|
||||
const logLines=(e)=>lines(e.log).map((l)=>JSON.parse(l));
|
||||
const pointer=(e)=>existsSync(e.pointer)?JSON.parse(readFileSync(e.pointer,'utf8')):null;
|
||||
const units=(e)=>JSON.parse(readFileSync(join(e.sd,'units.json'),'utf8'));
|
||||
const snaps=(e)=>lines(join(e.sd,'snap.jsonl')).map((l)=>JSON.parse(l));
|
||||
const fault=(e,name,text='')=>writeFileSync(join(e.sd,'faults',name),text);
|
||||
const sysctl=(e,...a)=>execFileSync(join(SBIN,'systemctl'),['--user',...a],{env:e.env,encoding:'utf8'});
|
||||
const edit=(f,from,to)=>{const s=readFileSync(f,'utf8');assert.ok(s.includes(from),`${f} has no ${from}`);writeFileSync(f,s.replace(from,to))};
|
||||
const posts=(e)=>readdirSync(e.data).filter((n)=>n.startsWith('bus.post-attempt-'));
|
||||
const backupSum=(e,act)=>readFileSync(join(e.data,'backups','stack',act,'SHA256SUMS'),'utf8').split(' ')[0];
|
||||
const commitLines=(range)=>git(repo,'log','--oneline','--no-merges','--max-count=200',range).split('\n').map((l)=>` ${l}`);
|
||||
// The units, the wrapper, the pointer and the store, by sha256.
|
||||
const fp=(e)=>({bus:sha(join(e.units,e.tmpl)),console:sha(join(e.units,e.con)),wrapper:sha(join(e.bin,e.wrapper)),pointer:existsSync(e.pointer)?sha(e.pointer):null,store:sha(e.store)});
|
||||
// The schema version: readOnly while the holder has the store open, immutable when it is stopped.
|
||||
function storeVersion(file,held){
|
||||
const db=held?new DatabaseSync(file,{readOnly:true}):new DatabaseSync(pathToFileURL(file).href+'?immutable=1',{readOnly:true});
|
||||
try{return db.prepare("SELECT value FROM meta WHERE key='schema_version'").get().value}finally{db.close()}
|
||||
}
|
||||
// Every file and directory under p with size and mtime: T23 and T16 compare it before and after.
|
||||
function tree(p){
|
||||
const out=[];
|
||||
const walk=(q)=>{let st;try{st=statSync(q)}catch{return}out.push(st.isDirectory()?`${q}/`:`${q} ${st.size} ${st.mtimeMs}`);if(st.isDirectory())for(const n of readdirSync(q).sort())walk(join(q,n))};
|
||||
walk(p);return out;
|
||||
}
|
||||
// T17: what a run may never remove.
|
||||
const kept=(e)=>[...tree(join(e.data,'backups','stack')),...tree(join(e.state,'stack-rollback')),...posts(e).flatMap((n)=>tree(join(e.data,n))),...tree(COPIES)].map((l)=>l.replace(/ .*/,''));
|
||||
|
||||
// The business file (row 169): the committed example made loadable, as acme, with `plugins` when given.
|
||||
const EXAMPLE=readFileSync(join(repo,'packages','business','examples','mosaic-stack.example.json'),'utf8');
|
||||
function biz(e,plugins){
|
||||
const b=JSON.parse(EXAMPLE);let n=0;
|
||||
const walk=(o)=>{for(const [k,v] of Object.entries(o)){if(k==='botId')o[k]=++n;else if(v==='YYYY-MM-DD')o[k]='2027-01-01';else if(v&&typeof v==='object')walk(v)}};
|
||||
walk(b);b.id='acme';if(plugins!==undefined)b.plugins=plugins;
|
||||
const f=join(dirname(e.cfg),'businesses','acme.json');mkdirSync(dirname(f),{recursive:true});
|
||||
writeFileSync(f,JSON.stringify(b,null,2)+'\n',{mode:0o600});chmodSync(f,0o600);
|
||||
}
|
||||
|
||||
async function mk(t){
|
||||
const root=realpathSync(mkdtempSync(join(tmpdir(),'sr-')));
|
||||
const e={root,data:join(root,'data'),bin:join(root,'bin'),sd:join(root,'sd'),units:join(root,'xcfg','systemd','user'),cfg:join(root,'cfg','config.json'),claude:join(root,'claude-bin','claude')};
|
||||
// The default instance's names; named() sets NAME's (row 190).
|
||||
Object.assign(e,{flags:[],bus:BUS,tmpl:'[email protected]',con:CON,wrapper:'mosaic'});
|
||||
Object.assign(e,{state:join(e.data,'state'),pointer:join(e.data,'state','stack-active.json'),log:join(e.data,'state','stack-activation-log.jsonl'),store:join(e.data,'bus','bus.sqlite'),lock:join(e.data,'state','stack-release.lock')});
|
||||
for(const d of [join(root,'home'),dirname(e.cfg),join(e.data,'bus'),e.bin,dirname(e.claude),join(e.sd,'faults'),e.units,join(root,'xdata')])mkdirSync(d,{recursive:true});
|
||||
chmodSync(join(e.data,'bus'),0o700);
|
||||
writeFileSync(e.cfg,JSON.stringify({configVersion:1,environment:'development',dataRoot:e.data,execution:{backend:'docker',provider:'zai',model:'glm-5.3-flash',adapter:'mock'}}));
|
||||
biz(e);
|
||||
writeFileSync(join(e.bin,'mosaic'),`#!/usr/bin/env bash\n${MARKER}\n[ "\${1:-}" = __lease-capability ] && exec "${e.bin}/mosaic-old" "$@"\nexec "/old/scripts/mosaic" "$@"\n`,{mode:0o755});
|
||||
writeFileSync(join(e.bin,'mosaic-old'),'#!/bin/sh\nexit 0\n',{mode:0o755});
|
||||
writeFileSync(e.claude,'#!/bin/sh\necho "9.9.9 (Claude Code fixture)"\n',{mode:0o755});
|
||||
e.env={PATH:`${SBIN}:${dirname(NODE)}:/usr/bin:/bin`,HOME:join(root,'home'),XDG_CONFIG_HOME:join(root,'xcfg'),XDG_DATA_HOME:join(root,'xdata'),MOSAIC_CONFIG:e.cfg,SD_STATE:e.sd,SD_DATA:e.data,TMPDIR:tmpdir()};
|
||||
const server=await healthz(e);
|
||||
t.after(()=>{
|
||||
for(const pid of lines(join(e.sd,'pids.log')).map(Number)){
|
||||
try{if(readFileSync(`/proc/${pid}/cmdline`,'utf8').includes(join(base,'holder.cjs')))process.kill(pid,'SIGKILL')}catch{}
|
||||
}
|
||||
for(const s of [server,...(e.servers??[])])s.close();
|
||||
execFileSync('chmod',['-R','u+w',root]);rmSync(root,{recursive:true,force:true});
|
||||
});
|
||||
return e;
|
||||
}
|
||||
|
||||
// The Console's /healthz on e.port: 200 while the stand-in has e.con active and no healthz-<copy>
|
||||
// fault names its copy.
|
||||
async function healthz(e){
|
||||
const server=createServer((q,r)=>{
|
||||
let st={};try{st=units(e)[e.con]||{}}catch{}
|
||||
const ok=q.url==='/healthz'&&st.state==='active'&&!existsSync(join(e.sd,'faults',`healthz-${basename(st.wd||'')}`));
|
||||
r.writeHead(ok?200:500);r.end();
|
||||
});
|
||||
server.listen(0,'127.0.0.1');await once(server,'listening');e.port=String(server.address().port);
|
||||
return server;
|
||||
}
|
||||
|
||||
// Row 190: stack instance NAME on e's host. It shares e's stand-ins, unit directory and $BIN, and has
|
||||
// its own config and data root from bootstrap --instance, its own Console port and business acme.
|
||||
// Its runs pass --instance NAME and no MOSAIC_CONFIG.
|
||||
async function named(e,name){
|
||||
const env={...e.env};delete env.MOSAIC_CONFIG;
|
||||
execFileSync('bash',[join(repo,'scripts','bootstrap.sh'),'--instance',name],{env,stdio:'ignore'});
|
||||
const data=join(env.HOME,`.mosaic-${name}`);
|
||||
const n={...e,env,data,cfg:join(env.HOME,'.config',`mosaic-${name}`,'config.json'),flags:['--instance',name],bus:`mosaic-bus-${name}@acme.service`,tmpl:`mosaic-bus-${name}@.service`,con:`mosaic-webui-${name}.service`,wrapper:`mosaic-${name}`,name};
|
||||
Object.assign(n,{state:join(data,'state'),pointer:join(data,'state','stack-active.json'),log:join(data,'state','stack-activation-log.jsonl'),store:join(data,'bus','bus.sqlite'),lock:join(data,'state','stack-release.lock')});
|
||||
mkdirSync(join(data,'bus'),{mode:0o700});
|
||||
biz(n);
|
||||
(e.servers??=[]).push(await healthz(n));
|
||||
return n;
|
||||
}
|
||||
|
||||
async function run(e,...args){
|
||||
const before=existsSync(e.data)?kept(e):[];
|
||||
const ch=spawn('bash',[e.sr??SR,...args,...e.flags,'--copies',COPIES,'--bin',e.bin],{env:e.env,stdio:['ignore','pipe','pipe']});
|
||||
let out='',err='';ch.stdout.on('data',(d)=>{out+=d});ch.stderr.on('data',(d)=>{err+=d});
|
||||
const timer=setTimeout(()=>ch.kill('SIGKILL'),120000);
|
||||
const [code]=await once(ch,'close');clearTimeout(timer);
|
||||
const now=new Set(kept(e));
|
||||
for(const p of before)assert.ok(now.has(p),`T17: ${args.join(' ')} removed ${p}`);
|
||||
return {code,out,err};
|
||||
}
|
||||
|
||||
// The stack on copy `copy` for business acme, both units active. pointer: adopt (a real adopt and
|
||||
// its log line), hand (written here, no log) or none. Snapshots start after the seed.
|
||||
async function seed(e,{copy=B,pm=false,store='3g',pointer:how='adopt'}={}){
|
||||
const db=new DatabaseSync(e.store);
|
||||
db.exec('PRAGMA journal_mode=WAL');
|
||||
db.exec('CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL)');
|
||||
db.exec('CREATE TABLE events (id INTEGER PRIMARY KEY, at TEXT NOT NULL)');
|
||||
db.prepare("INSERT INTO meta VALUES ('schema_version', ?)").run(store);
|
||||
for(let i=0;i<3;i++)db.prepare('INSERT INTO events (at) VALUES (?)').run(`2000-01-01T00:00:0${i}.000Z`);
|
||||
db.close();
|
||||
const sh=(name,...a)=>execFileSync('bash',[join(repo,'scripts',name),...a],{env:e.env,encoding:'utf8',stdio:['ignore','pipe','pipe']});
|
||||
sh('bus-service.sh','install',...e.flags,'--copy',copy,'--copies',COPIES,...(pm?['--pm']:[]),'--claude',e.claude);
|
||||
sh('webui-service.sh','install',...e.flags,'--copy',copy,'--copies',COPIES,'--port',e.port,'--business','acme');
|
||||
sh('stack-copy.sh','wrapper',copy,...e.flags,'--copies',COPIES,'--bin',e.bin);
|
||||
// The seeded host started two minutes ago, so its boot status and PM run predate any run's start.
|
||||
execFileSync(join(SBIN,'systemctl'),['--user','start',e.bus],{env:{...e.env,SD_CLOCK_OFFSET:'120'}});
|
||||
sysctl(e,'start',e.con);
|
||||
if(how==='adopt'){const r=await run(e,'adopt','acme');assert.equal(r.code,0,r.err)}
|
||||
else if(how==='hand'){
|
||||
mkdirSync(e.state,{recursive:true});
|
||||
writeFileSync(e.pointer,JSON.stringify({copy,business:'acme',pm,claude:e.claude,console:['--port',e.port,'--business','acme'],schema:'3g',activation:'manual',at:'2026-10-18T00:00:00Z'})+'\n');
|
||||
}
|
||||
writeFileSync(join(e.sd,'snap.jsonl'),'');
|
||||
}
|
||||
|
||||
const rolledBack=(code,to=B)=>`stack-release: ${code}; rolled back to ${to}, which passed the gate\n`;
|
||||
const planBackup=(e)=>`backup: the run backs up the stopped store to ${e.data}/backups/stack/<activation>/ before any migration`;
|
||||
const planNoBackup='backup: none; the run rolls back before the backup';
|
||||
const planTail=(e,to,pm=false)=>[`bus: scripts/bus-service.sh install --copy ${to} --copies ${COPIES}${pm?' --pm':''} --claude ${e.claude}`,'proof: off','send: off',`Console: scripts/webui-service.sh install --copy ${to} --copies ${COPIES} --port ${e.port} --business acme`,`CLI: scripts/stack-copy.sh wrapper ${to} --copies ${COPIES} --bin ${e.bin}`];
|
||||
import {A,B,BUS,C,C1,CON,COPIES,EXAMPLE,F,KINDS_NEW,KINDS_OLD,M,MARKER,NODE,ROOT,S,SBIN,SR,backupSum,base,biz,calls,commit,commitLines,edit,fault,fp,git,healthz,holder,journalctl,kept,lines,logLines,migrateStub,mk,mosaicStub,named,pkg,planBackup,planNoBackup,planTail,pointer,posts,repo,rolledBack,run,script,seed,sha,snaps,startsStops,storeText,storeVersion,sysctl,systemctl,systemdAnalyze,tracked,tree,U,units} from './stack-release-fixture.mjs';
|
||||
|
||||
// --- The T table ---
|
||||
|
||||
@@ -2477,6 +2062,49 @@ test('R15 a rehearsal stopped by TERM, INT or HUP fails signal and still stops t
|
||||
}
|
||||
});
|
||||
|
||||
// --- Row 233 (#1735): --backup last, the newest backup the log records (sage-1650, sage-1653) ---
|
||||
|
||||
// The log's lines, as the runs write them, with only the fields from_log reads set.
|
||||
function rlog(e,...ls){
|
||||
mkdirSync(e.state,{recursive:true});
|
||||
writeFileSync(e.log,ls.map(([op,to,backup],i)=>JSON.stringify({at:`2026-10-20T00:00:0${i}Z`,op,from:null,to,backup,activation:`20261020T00000${i}Z-${to.slice(0,12)}`,result:'ok'})+'\n').join(''));
|
||||
}
|
||||
|
||||
test('row 233: --backup last takes the newest backup the log records; a later line with backup null does not hide it, and the store line names it',async(t)=>{
|
||||
const e=await rmk(t);
|
||||
// An older backup that isn't on disk: a run that took it would fail backup-missing.
|
||||
rlog(e,['activate',C1,`20261019T000000Z-${C1.slice(0,12)}`],['activate',B,e.act],['rollback',B,null]);
|
||||
const l0=readFileSync(e.log,'utf8'),before=untouched(e);
|
||||
const r=await rehearse(e,B,'--backup','last','--port',await freePort(),'--timeout','10');
|
||||
assert.equal(r.code,0,r.out+r.err);
|
||||
assert.deepEqual(r.names,['PASS backup-last',...PRE,'PASS migration','PASS gate','PASS no-session',...POST]);
|
||||
assert.ok(r.out.startsWith(`PASS backup-last: the log's newest backup is ${e.act}\n`),r.out);
|
||||
assert.ok(r.out.includes(`\nBACKUP_STORE ${e.act} schema 3g\n`),r.out);
|
||||
assert.equal(readFileSync(e.log,'utf8'),l0);assert.equal(untouched(e),before);
|
||||
// The rehearsal's own steps.log has the PASS and FAIL lines only.
|
||||
assert.deepEqual(lines(join(r.dir,'steps.log')).map((l)=>l.replace(/^\S+ /,'')),r.out.split('\n').filter((l)=>/^(PASS|FAIL) |^REHEARSAL_/.test(l)));
|
||||
});
|
||||
|
||||
test('row 233: --backup last with no backup in the log, or no log, exits 3 with no-backup and makes no rehearsal directory',async(t)=>{
|
||||
const e=await rmk(t);
|
||||
for(const how of ['no log','adopt only']){
|
||||
if(how==='adopt only')rlog(e,['adopt',B,null]);
|
||||
const r=await rehearse(e,B,'--backup','last','--port',await freePort(),'--timeout','3');
|
||||
assert.equal(r.code,3,how+r.out+r.err);
|
||||
assert.equal(r.out,'FAIL no-backup: the log records no backup\nREHEARSAL_FAIL\n',how);
|
||||
assert.equal(r.dir,null);assert.deepEqual(rDirs(e),[],how);assert.ok(!existsSync(join(e.data,'rehearsals')),how);
|
||||
}
|
||||
});
|
||||
|
||||
test('row 233: --backup last refuses a newest backup that is not an activation id, with exit 1',async(t)=>{
|
||||
const e=await rmk(t);
|
||||
rlog(e,['activate',B,'../../etc']);
|
||||
const r=await rehearse(e,B,'--backup','last','--port',await freePort(),'--timeout','3');
|
||||
assert.equal(r.code,1,r.out+r.err);
|
||||
assert.equal(r.out,'FAIL backup-unreadable: the log\'s newest backup is not an activation id\nREHEARSAL_FAIL\n');
|
||||
assert.deepEqual(rDirs(e),[]);
|
||||
});
|
||||
|
||||
// --- Row 190 (#1688): stack instances on one host. Design and the T-I table:
|
||||
// agents/darkwing/work/queue-190/DESIGN.md. T-I1, T-I2, T-I5, T-I10 and the loader half of T-I7 are in
|
||||
// stack-instances.test.mjs; these need the units, the stand-ins and stack-release.sh. Every existing
|
||||
|
||||
@@ -0,0 +1,449 @@
|
||||
// scripts/stack-upgrade.sh (row 233, I3, #1735): root's upgrade and rollback of a stack whose units are
|
||||
// root's, driving stack-release.sh --units external (row 232). Design: r6 sections 4 and 7,
|
||||
// agents/darkwing/work/queue-195/DESIGN.md, as the I3 brief changes them; deviations:
|
||||
// agents/darkwing/work/queue-233/BUILD.md. The stand-ins and the module fixture are
|
||||
// stack-release-fixture.mjs's. Root's sudo, case2-unit.sh render and row 240 hostcheck are stand-ins
|
||||
// here; the stack's user's steps are the real stack-release.sh from the user's clone, except the
|
||||
// rehearsal, which the fixture's stub copies can't pass (its own tests are in stack-release.test.mjs).
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {execFileSync,spawn} from 'node:child_process';
|
||||
import {mkdirSync,writeFileSync,readFileSync,existsSync,readdirSync,rmSync,chmodSync,copyFileSync,symlinkSync,statSync} from 'node:fs';
|
||||
import {DatabaseSync} from 'node:sqlite';
|
||||
import {join,dirname} from 'node:path';
|
||||
import {once} from 'node:events';
|
||||
import {B,C,C1,U,COPIES,NODE,SBIN,base,fault,git,lines,logLines,mk,pointer,repo,sha,storeVersion,sysctl,units} from './stack-release-fixture.mjs';
|
||||
|
||||
// --- Root's stand-ins. Each is serialized into its own file with K, its baked paths. ---
|
||||
|
||||
// sudo -u USER -H env PATH=P CMD...: the form the driver must use, logged to sudo.log, then CMD with
|
||||
// SBIN before P. A one-shot hook arm/<key>.sh runs first and arm/<key>.id replaces CMD's ID. The
|
||||
// rehearsal prints a fixed BACKUP_STORE line and exits faults/rehearse-rc unless faults/real-rehearse.
|
||||
function fakeSudo(K){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),cp=process.getBuiltinModule('node:child_process');
|
||||
const a=process.argv.slice(2),sd=process.env.SD_STATE,f=(n)=>path.join(sd,'faults',n);
|
||||
if(a[0]!=='-u'||!a[1]||a[2]!=='-H'||a[3]!=='env'||!/^PATH=/.test(a[4]||'')||a.length<6){console.error(`fake sudo: not the driver's form: ${a.join(' ')}`);process.exit(99)}
|
||||
const P=a[4].slice(5),cmd=a.slice(5);
|
||||
let key=path.basename(cmd[0]);
|
||||
if(cmd[0].endsWith('/scripts/stack-release.sh')){
|
||||
key=cmd[1];
|
||||
if(['activate','rollback'].includes(key)&&cmd.includes('--plan'))key+='-plan';
|
||||
if(key==='confirm'&&cmd.includes('--rollback'))key='confirm-rollback';
|
||||
}else if(cmd[0]==='git')key=`git-${cmd[3]}`;
|
||||
fs.appendFileSync(path.join(sd,'sudo.log'),JSON.stringify({user:a[1],path:P,key,cmd})+'\n');
|
||||
const env={...process.env,PATH:`${K.sbin}:${P}`},arm=(n)=>path.join(sd,'arm',`${key}.${n}`);
|
||||
if(fs.existsSync(arm('sh'))){const run=arm('sh')+'.ran';fs.renameSync(arm('sh'),run);cp.spawnSync('bash',[run],{env,stdio:'inherit'})}
|
||||
if(fs.existsSync(arm('id'))){cmd[2]=fs.readFileSync(arm('id'),'utf8').trim();fs.rmSync(arm('id'))}
|
||||
if(key==='rehearse'&&!fs.existsSync(f('real-rehearse'))){
|
||||
console.log(`fixture rehearse: ${cmd.slice(1).join(' ')}`);
|
||||
let line='BACKUP_STORE 20261020T000000Z-0123456789ab schema 3g';try{line=fs.readFileSync(f('rehearse-store-line'),'utf8').replace(/\n$/,'')}catch{}
|
||||
if(line)console.log(line);
|
||||
let rc=0;try{rc=Number(fs.readFileSync(f('rehearse-rc'),'utf8'))}catch{}
|
||||
process.exit(rc);
|
||||
}
|
||||
const r=cp.spawnSync(cmd[0],cmd.slice(1),{env,stdio:'inherit'});process.exit(r.status??1);
|
||||
}
|
||||
|
||||
// case2-unit.sh: logs its argv, then installs the copy's units as the fixture's user units (its
|
||||
// WorkingDirectory under the stack's copies) and starts the bus, then the Console, as case2-unit.sh
|
||||
// does. faults/render-fails-<copy> exits 1 first; faults/render-probe-<copy> (one-shot) first runs
|
||||
// probe.sh, a second driver, and records its exit and stderr.
|
||||
function fakeRender(K){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),cp=process.getBuiltinModule('node:child_process');
|
||||
const a=process.argv.slice(2),sd=process.env.SD_STATE,f=(n)=>path.join(sd,'faults',n);
|
||||
fs.appendFileSync(path.join(sd,'render.log'),JSON.stringify(a)+'\n');
|
||||
const cut=a.indexOf('--'),pre=a.slice(0,cut),serve=a.slice(cut+1),get=(k)=>pre[pre.indexOf(k)+1];
|
||||
const copy=get('--copy'),copies=path.join(get('--home'),'.local','share','mosaic-stack','copies');
|
||||
if(fs.existsSync(f(`render-probe-${copy}`))){
|
||||
fs.rmSync(f(`render-probe-${copy}`));
|
||||
const p=cp.spawnSync('bash',[path.join(sd,'probe.sh')],{encoding:'utf8'});
|
||||
fs.writeFileSync(path.join(sd,'probe.json'),JSON.stringify({code:p.status,err:p.stderr}));
|
||||
}
|
||||
if(fs.existsSync(f(`render-fails-${copy}`))){console.error(`fixture render: ${copy} fails`);process.exit(1)}
|
||||
const sh=(n,...x)=>cp.execFileSync('bash',[path.join(K.repo,'scripts',n),...x],{stdio:['ignore','ignore','inherit']});
|
||||
sh('bus-service.sh','install','--copy',copy,'--copies',copies,...(pre.includes('--pm')?['--pm']:[]),'--claude',get('--claude'));
|
||||
sh('webui-service.sh','install','--copy',copy,'--copies',copies,...serve);
|
||||
for(const u of [`mosaic-bus@${get('--business')}.service`,'mosaic-webui.service'])cp.execFileSync(path.join(K.sbin,'systemctl'),['start',u],{stdio:'inherit'});
|
||||
}
|
||||
|
||||
// Root's systemctl: the fixture's, with systemd's no-op start of an active unit, and a one-shot
|
||||
// faults/stop-noop-<unit> that leaves the unit running, as a stop past its timeout would.
|
||||
function fakeSystemctl(K){
|
||||
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),cp=process.getBuiltinModule('node:child_process');
|
||||
const a=process.argv.slice(2),sd=process.env.SD_STATE;
|
||||
fs.appendFileSync(path.join(sd,'root-systemctl.log'),a.join(' ')+'\n');
|
||||
const noop=path.join(sd,'faults',`stop-noop-${a[1]}`);
|
||||
if(a[0]==='stop'&&fs.existsSync(noop)){fs.rmSync(noop);process.exit(0)}
|
||||
if(a[0]==='start'){let s={};try{s=JSON.parse(fs.readFileSync(path.join(sd,'units.json'),'utf8'))}catch{}if((s[a[1]]||{}).state==='active')process.exit(0)}
|
||||
const r=cp.spawnSync(path.join(K.sbin,'systemctl'),a,{stdio:'inherit'});process.exit(r.status??1);
|
||||
}
|
||||
|
||||
const FAKES=join(base,'upgrade-fakes');mkdirSync(FAKES);
|
||||
const K={sbin:SBIN,repo};
|
||||
for(const [n,fn] of [['sudo',fakeSudo],['render',fakeRender],['systemctl',fakeSystemctl]])writeFileSync(join(FAKES,n),`#!${NODE}\n(${fn})(${JSON.stringify(K)});\n`,{mode:0o755});
|
||||
writeFileSync(join(FAKES,'hostcheck'),'#!/bin/sh\necho "$*" >> "$SD_STATE/hostcheck.log"\nexit "$(cat "$SD_STATE/faults/hostcheck-rc" 2>/dev/null || echo 0)"\n',{mode:0o755});
|
||||
const BUNDLE=join(base,'fixture.bundle');git(repo,'bundle','create','-q',BUNDLE,'refactor');
|
||||
// Root's checkout of a reviewed commit: a clone of the fixture repo, detached at it.
|
||||
const checkouts=new Map();
|
||||
const checkout=(id)=>{
|
||||
if(!checkouts.has(id)){const d=join(base,`root-${id.slice(0,12)}`);git(base,'clone','-q','--no-checkout',repo,d);git(d,'checkout','-q','--detach',id);checkouts.set(id,d)}
|
||||
return checkouts.get(id);
|
||||
};
|
||||
const TO=['--timeout','3'];
|
||||
|
||||
// --- Per-test stack ---
|
||||
|
||||
// The stack's user's home: its copies (the fixture's, through a link), its clone, its bin. Both units
|
||||
// installed on `copy` and active, adopted under --units external, and the I2 args file naming it.
|
||||
async function useed(t,{copy=B}={}){
|
||||
const e=await mk(t),home=e.env.HOME;
|
||||
e.hc=join(home,'.local','share','mosaic-stack','copies');mkdirSync(dirname(e.hc),{recursive:true});symlinkSync(COPIES,e.hc);
|
||||
const bin=join(home,'.npm-global','bin');mkdirSync(bin,{recursive:true});
|
||||
writeFileSync(join(bin,'mosaic'),readFileSync(join(e.bin,'mosaic'),'utf8').replaceAll(`${e.bin}/`,`${bin}/`),{mode:0o755});
|
||||
copyFileSync(join(e.bin,'mosaic-old'),join(bin,'mosaic-old'));chmodSync(join(bin,'mosaic-old'),0o755);
|
||||
e.bin=bin;e.clone=join(home,'src','mosaic-stack');git(base,'clone','-q',repo,e.clone);
|
||||
const db=new DatabaseSync(e.store);
|
||||
db.exec('PRAGMA journal_mode=WAL');
|
||||
db.exec('CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL)');
|
||||
db.exec('CREATE TABLE events (id INTEGER PRIMARY KEY, at TEXT NOT NULL)');
|
||||
db.prepare("INSERT INTO meta VALUES ('schema_version', '3g')").run();
|
||||
for(let i=0;i<3;i++)db.prepare('INSERT INTO events (at) VALUES (?)').run(`2000-01-01T00:00:0${i}.000Z`);
|
||||
db.close();
|
||||
const sh=(n,...a)=>execFileSync('bash',[join(e.clone,'scripts',n),...a],{env:e.env,encoding:'utf8',stdio:['ignore','pipe','pipe']});
|
||||
sh('bus-service.sh','install','--copy',copy,'--copies',e.hc,'--claude',e.claude);
|
||||
sh('webui-service.sh','install','--copy',copy,'--copies',e.hc,'--port',e.port,'--business','acme');
|
||||
sh('stack-copy.sh','wrapper',copy,'--copies',e.hc,'--bin',e.bin);
|
||||
execFileSync(join(SBIN,'systemctl'),['start',e.bus],{env:{...e.env,SD_CLOCK_OFFSET:'120'}});
|
||||
sysctl(e,'start',e.con);
|
||||
sh('stack-release.sh','adopt','acme','--units','external','--copies',e.hc,'--bin',e.bin);
|
||||
e.etc=join(e.root,'etc');mkdirSync(e.etc);e.args=join(e.etc,'dev.args');
|
||||
e.argsText=`--user\nfixture\n--home\n${home}\n--business\nacme\n--copy\n${copy}\n--node\n${NODE}\n--claude\n${e.claude}\n--\n--port\n${e.port}\n--business\nacme\n`;
|
||||
writeFileSync(e.args,e.argsText);chmodSync(e.args,0o640);
|
||||
e.run=join(e.root,'run');e.slog=join(e.root,'log','upgrade-dev.log');e.lockf=join(e.run,'upgrade-dev.lock');
|
||||
mkdirSync(join(e.sd,'arm'));
|
||||
e.denv={...e.env,STACK_UPGRADE_ETC:e.etc,STACK_UPGRADE_RUN:e.run,STACK_UPGRADE_LOG_DIR:dirname(e.slog),STACK_UPGRADE_UNIT_DIR:e.units,STACK_UPGRADE_HOSTCHECK:join(FAKES,'hostcheck'),STACK_UPGRADE_RENDER:join(FAKES,'render'),STACK_UPGRADE_SUDO:join(FAKES,'sudo'),STACK_UPGRADE_SYSTEMCTL:join(FAKES,'systemctl')};
|
||||
writeFileSync(join(e.sd,'probe.sh'),`exec bash ${join(checkout(C1),'scripts','stack-upgrade.sh')} upgrade --instance dev --bundle ${BUNDLE} --sha ${C1} --timeout 3\n`);
|
||||
writeFileSync(join(e.sd,'snap.jsonl'),'');
|
||||
return e;
|
||||
}
|
||||
|
||||
// The driver from root's checkout of `at`.
|
||||
async function drive(e,at,...args){
|
||||
const ch=spawn('bash',[join(checkout(at),'scripts','stack-upgrade.sh'),...args],{env:e.denv,stdio:['ignore','pipe','pipe']});
|
||||
let out='',err='';ch.stdout.on('data',(d)=>{out+=d});ch.stderr.on('data',(d)=>{err+=d});
|
||||
const timer=setTimeout(()=>ch.kill('SIGKILL'),180000);
|
||||
const [code]=await once(ch,'close');clearTimeout(timer);
|
||||
return {code,out,err,all:out+err};
|
||||
}
|
||||
const upgrade=(e,id,...a)=>drive(e,id,'upgrade','--instance','dev','--bundle',BUNDLE,'--sha',id,...TO,...a);
|
||||
const arm=(e,key,text)=>writeFileSync(join(e.sd,'arm',key),text);
|
||||
|
||||
// The step log: every line `<time> <step> rc=N ...` or the result line, root:root 0640 (here the
|
||||
// test user's), and nothing the stack's user printed. Returns the lines without the time.
|
||||
const STEP=/^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ (start|fetch|ff|copy|hostcheck|units|plan|rehearse|rehearse-store|rehearsal-skipped|stop-console|stop-bus|wait-down|stage|render|confirm|args|revert|render-old|confirm-rollback|gate|start-bus|start-console|stop-units) rc=\d+ |^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ result exit=[0-3] [a-z-]+ /;
|
||||
function slog(e){
|
||||
const l=lines(e.slog);
|
||||
for(const x of l){assert.match(x,STEP);assert.ok(!/stack-release:|fixture rehearse|PASS |FAIL |REHEARSAL/.test(x),x)}
|
||||
assert.equal(statSync(e.slog).mode&0o777,0o640);
|
||||
return l.map((x)=>x.slice(21));
|
||||
}
|
||||
const names=(e)=>slog(e).map((x)=>x.split(' ')[0]);
|
||||
const result=(e)=>slog(e).at(-1);
|
||||
const sudoKeys=(e)=>lines(join(e.sd,'sudo.log')).map((l)=>JSON.parse(l).key);
|
||||
const lastAct=(e)=>logLines(e).filter((o)=>o.result==='started').at(-1).activation;
|
||||
const mine=(e,a)=>logLines(e).filter((o)=>o.activation===a);
|
||||
const open=(e)=>{const m=new Set();for(const o of logLines(e)){if(!o.activation)continue;if(o.result==='started')m.add(o.activation);else m.delete(o.activation)}return [...m]};
|
||||
const final=(e,a)=>{const [s,f,...more]=mine(e,a);assert.equal(s.result,'started');assert.equal(more.length,0);return f};
|
||||
const live=(e)=>[units(e)[e.bus].state,units(e)[e.con].state,units(e)[e.bus].wd.split('/').at(-1),units(e)[e.con].wd.split('/').at(-1)];
|
||||
// Nothing the run did reached the args file: the same bytes and mode, and no temporary file left.
|
||||
const argsKept=(e)=>{assert.equal(readFileSync(e.args,'utf8'),e.argsText);assert.equal(statSync(e.args).mode&0o777,0o640);assert.deepEqual(readdirSync(e.etc),['dev.args'])};
|
||||
const argsCopy=(e)=>{const l=readFileSync(e.args,'utf8').split('\n');return l[l.indexOf('--copy')+1]};
|
||||
const PRE=['start','fetch','ff','copy','hostcheck','units','plan','rehearse','rehearse-store'];
|
||||
const A2=['stop-console','stop-bus','wait-down','wait-down'];
|
||||
const R=[...A2,'revert','render-old','confirm-rollback'];
|
||||
|
||||
// --- r6 sequences 1-8, through the driver ---
|
||||
|
||||
test('row 233: sequence 1: confirm fails healthz, the driver reverts, renders the old copy and confirm --rollback --cause A5 passes: exit 1',async(t)=>{
|
||||
const e=await useed(t);
|
||||
fault(e,`healthz-${C1}`);
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,1,r.all);
|
||||
assert.equal(r.err.split('\n').at(-2),`stack-upgrade: failed: A5 failed; rolled back to ${B}, which passed the gate`);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage','render','confirm',...R,'result']);
|
||||
assert.equal(result(e),`result exit=1 failed A5 failed; rolled back to ${B}, which passed the gate`);
|
||||
const a=lastAct(e),l=final(e,a);assert.deepEqual([l.op,l.result,l.code,l.rollback,l.from,l.to],['activate','gate-failed','healthz','ok',B,C1]);
|
||||
assert.deepEqual(open(e),[]);assert.deepEqual(live(e),['active','active',B,B]);
|
||||
argsKept(e);
|
||||
// The stack's user's steps went through sudo in the driver's form, as fixture, the pinned node first.
|
||||
for(const l of lines(join(e.sd,'sudo.log')).map((x)=>JSON.parse(x)))assert.deepEqual([l.user,l.path],['fixture',`${dirname(NODE)}:/usr/local/bin:/usr/bin:/bin`]);
|
||||
assert.deepEqual(sudoKeys(e),['git-fetch','git-merge','stack-copy.sh','activate-plan','rehearse','activate','confirm','revert','confirm-rollback']);
|
||||
});
|
||||
|
||||
test('row 233: sequence 2: activate --migrate fails at the wrapper, revert restores the store, confirm --rollback --cause A3: exit 1',async(t)=>{
|
||||
const e=await useed(t);
|
||||
const s0=sha(e.store);
|
||||
arm(e,'activate.sh',`chmod 0555 ${e.bin}\n`);arm(e,'revert.sh',`chmod 0755 ${e.bin}\n`);
|
||||
const r=await upgrade(e,C,'--migrate');assert.equal(r.code,1,r.all);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage',...R,'result']);
|
||||
assert.equal(result(e),`result exit=1 failed A3 failed; rolled back to ${B}, which passed the gate`);
|
||||
const l=final(e,lastAct(e));assert.deepEqual([l.result,l.code,l.rollback],['failed','wrapper','ok']);
|
||||
assert.equal(storeVersion(e.store,true),'3g');assert.deepEqual(live(e),['active','active',B,B]);
|
||||
// The bus holds the store open again; its bytes are the backup's once it stops.
|
||||
sysctl(e,'stop',e.bus);assert.equal(sha(e.store),s0);
|
||||
argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: sequence 3: root\'s render of the new copy fails, the driver rolls back with --cause A4, and holds the lock to the end: exit 1',async(t)=>{
|
||||
const e=await useed(t);
|
||||
fault(e,`render-fails-${C1}`);fault(e,`render-probe-${B}`);
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,1,r.all);
|
||||
// The probe ran inside render-old, during the rollback, and its one line is its refusal.
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage','render',...A2,'revert','result','render-old','confirm-rollback','result']);
|
||||
const p=JSON.parse(readFileSync(join(e.sd,'probe.json'),'utf8'));assert.equal(p.code,3,p.err);
|
||||
assert.equal(p.err,`stack-upgrade: upgrade-running: another stack-upgrade.sh run holds ${e.lockf} (nothing was stopped)\n`);
|
||||
assert.match(slog(e).at(-4),/^result exit=3 upgrade-running /);
|
||||
assert.match(slog(e).find((x)=>x.startsWith('render ')),/^render rc=1 /);
|
||||
const l=final(e,lastAct(e));assert.deepEqual([l.result,l.code,l.rollback],['failed','units-start','ok']);
|
||||
assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: sequence 4: a capture failure is exit 3 from the stage, and A2f restarts the old units and gates them: exit 1',async(t)=>{
|
||||
const e=await useed(t);
|
||||
arm(e,'activate.sh',`: > ${join(e.sd,'faults','no-fragment')}\n`);arm(e,'gate.sh',`rm ${join(e.sd,'faults','no-fragment')}\n`);
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,1,r.all);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage','start-bus','start-console','gate','result']);
|
||||
assert.match(slog(e).find((x)=>x.startsWith('stage ')),/^stage rc=3 /);
|
||||
assert.equal(result(e),`result exit=1 failed A3 failed; ${B} is running again and passed the gate`);
|
||||
const a=lastAct(e);assert.deepEqual(mine(e,a).map((o)=>[o.result,o.code]),[['started',undefined],['refused','capture-failed']]);
|
||||
assert.deepEqual(open(e),[]);assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: N1: a bus that outlives A2\'s timeout keeps its old start, so A2f\'s gate fails stale-start: exit 2, both units stopped',async(t)=>{
|
||||
const e=await useed(t);
|
||||
fault(e,`stop-noop-${e.bus}`);
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,2,r.all);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'start-bus','start-console','gate','stop-units','result']);
|
||||
assert.ok(slog(e).includes(`wait-down rc=1 unit: ${e.bus} state: active`),slog(e).join('\n'));
|
||||
assert.ok(r.out.includes(`gate failed on ${B}: stale-start`),r.out);
|
||||
assert.equal(result(e),`result exit=2 rollback-failed A2 failed and ${B} failed its gate after the restart; both units are stopped`);
|
||||
assert.deepEqual(live(e).slice(0,2),['inactive','inactive']);
|
||||
assert.ok(!sudoKeys(e).includes('activate'));argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: sequence 5: confirm of the wrong copy is refused, and the driver rolls back with --cause A5: exit 1',async(t)=>{
|
||||
const e=await useed(t);
|
||||
arm(e,'confirm.id',B);
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,1,r.all);
|
||||
assert.ok(r.out.includes('refusing open-activation')||r.err.includes('refusing open-activation'),r.all);
|
||||
assert.match(slog(e).find((x)=>x.startsWith('confirm ')),/^confirm rc=3 /);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage','render','confirm',...R,'result']);
|
||||
const l=final(e,lastAct(e));assert.deepEqual([l.result,l.code,l.rollback],['gate-failed','unrecorded','ok']);
|
||||
assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: sequence 6: a revert that fails stops both units at exit 2, and the next run refuses at its --plan with nothing stopped',async(t)=>{
|
||||
const e=await useed(t);
|
||||
fault(e,`render-fails-${C1}`);
|
||||
arm(e,'revert.sh',`for d in ${join(e.state,'stack-rollback')}/*/; do [ -f "$d/AWAITING-UNITS" ] && rm "$d/wrapper"; done\n`);
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,2,r.all);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage','render',...A2,'revert','stop-units','result']);
|
||||
assert.equal(result(e),`result exit=2 rollback-failed A4 failed and revert failed; both units are stopped`);
|
||||
const a=lastAct(e);assert.deepEqual(open(e),[a]);assert.deepEqual(live(e).slice(0,2),['inactive','inactive']);
|
||||
argsKept(e);
|
||||
const n=slog(e).length,sc=lines(join(e.sd,'root-systemctl.log')).length;
|
||||
rmSync(join(e.sd,'faults',`render-fails-${C1}`));
|
||||
const again=await upgrade(e,C1);assert.equal(again.code,3,again.all);
|
||||
assert.ok(again.out.includes(`refuses interrupted-run: activation ${a} started and has no final line`),again.out);
|
||||
assert.deepEqual(names(e).slice(n),PRE.slice(0,7).concat('result'));
|
||||
assert.match(result(e),/^result exit=3 plan-refused /);
|
||||
assert.equal(lines(join(e.sd,'root-systemctl.log')).length,sc);argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: sequence 7: two upgrades and a rollback --to the previous copy with --restore, the args file following each confirm',async(t)=>{
|
||||
const e=await useed(t);
|
||||
// The args file at confirm still names the old copy; the new one moves in after it.
|
||||
arm(e,'confirm.sh',`cp ${e.args} ${join(e.sd,'args-at-confirm')}\n`);
|
||||
const one=await upgrade(e,C1);assert.equal(one.code,0,one.all);
|
||||
assert.equal(one.out.split('\n').at(-2),`stack-upgrade: upgrade ok: ${C1} is live for acme`);
|
||||
assert.equal(readFileSync(join(e.sd,'args-at-confirm'),'utf8'),e.argsText);
|
||||
assert.equal(readFileSync(e.args,'utf8'),e.argsText.replace(`--copy\n${B}\n`,`--copy\n${C1}\n`));
|
||||
assert.equal(statSync(e.args).mode&0o777,0o640);assert.deepEqual(readdirSync(e.etc),['dev.args']);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage','render','confirm','args','result']);
|
||||
assert.ok(slog(e).includes(`args rc=0 copy: ${C1}`));
|
||||
const a1=lastAct(e);
|
||||
const two=await upgrade(e,C,'--migrate');assert.equal(two.code,0,two.all);
|
||||
const a2=lastAct(e);assert.equal(argsCopy(e),C);
|
||||
const l1=final(e,a1),l2=final(e,a2);
|
||||
assert.deepEqual([l1.op,l1.result,l1.from,l1.to],['activate','ok',B,C1]);
|
||||
assert.deepEqual([l2.op,l2.result,l2.from,l2.to,l2.backup],['activate','ok',C1,C,a2]);
|
||||
assert.equal(storeVersion(e.store,true),'3h');
|
||||
const n=slog(e).length;
|
||||
const rb=await drive(e,C,'rollback','--instance','dev','--to',C1,'--restore',a2,...TO);assert.equal(rb.code,0,rb.all);
|
||||
assert.equal(rb.out.split('\n').at(-2),`stack-upgrade: rollback ok: ${C1} is live for acme`);
|
||||
// A rollback fetches, copies and rehearses nothing.
|
||||
assert.deepEqual(names(e).slice(n),['start','hostcheck','units','plan',...A2,'stage','render','confirm','args','result']);
|
||||
const ra=lastAct(e),l3=final(e,ra);assert.deepEqual([l3.op,l3.result,l3.from,l3.to,l3.backup],['rollback','ok',C,C1,a2]);
|
||||
assert.deepEqual([pointer(e).copy,pointer(e).schema],[C1,'3g']);assert.equal(storeVersion(e.store,true),'3g');
|
||||
assert.equal(argsCopy(e),C1);assert.deepEqual(live(e),['active','active',C1,C1]);
|
||||
const m=slog(e).length,sc=lines(join(e.sd,'root-systemctl.log')).length;
|
||||
const no=await drive(e,C,'rollback','--instance','dev','--to',B,...TO);assert.equal(no.code,3,no.all);
|
||||
assert.ok(no.all.includes(`refusing to-not-previous: --to ${B} is not the previous copy; the log names ${C}`),no.all);
|
||||
assert.deepEqual(names(e).slice(m),['start','hostcheck','units','plan','result']);
|
||||
assert.equal(lines(join(e.sd,'root-systemctl.log')).length,sc);assert.equal(argsCopy(e),C1);
|
||||
});
|
||||
|
||||
test('row 233: sequence 8: a migration that writes and fails: the driver reverts to the backup byte for byte, --cause A3: exit 1',async(t)=>{
|
||||
const e=await useed(t);
|
||||
const s0=sha(e.store);
|
||||
fault(e,'migrate-fails');
|
||||
const r=await upgrade(e,C,'--migrate');assert.equal(r.code,1,r.all);
|
||||
assert.match(slog(e).find((x)=>x.startsWith('stage ')),/^stage rc=1 /);
|
||||
assert.deepEqual(names(e),[...PRE,...A2,'stage',...R,'result']);
|
||||
const a=lastAct(e),l=final(e,a);assert.deepEqual([l.result,l.code,l.rollback],['failed','migrate-failed','ok']);
|
||||
sysctl(e,'stop',e.bus);assert.equal(sha(e.store),s0);assert.equal(storeVersion(e.store,false),'3g');
|
||||
argsKept(e);
|
||||
});
|
||||
|
||||
// --- The lock, A1's refusals and the rehearsal ---
|
||||
|
||||
test('row 233: a second run while the lock is held refuses upgrade-running with one log line and no call',async(t)=>{
|
||||
const e=await useed(t);
|
||||
mkdirSync(e.run,{recursive:true});
|
||||
const holder=spawn('flock',[e.lockf,'sleep','60'],{detached:true,stdio:'ignore'});
|
||||
t.after(()=>{try{process.kill(-holder.pid,'SIGKILL')}catch{}});
|
||||
for(let i=0;i<100&&execFileSync('bash',['-c','flock -n "$1" true && echo free || echo held','_',e.lockf],{encoding:'utf8'}).trim()==='free';i++)await new Promise((r)=>setTimeout(r,50));
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,3,r.all);
|
||||
assert.equal(r.err,`stack-upgrade: upgrade-running: another stack-upgrade.sh run holds ${e.lockf} (nothing was stopped)\n`);
|
||||
assert.deepEqual(names(e),['result']);
|
||||
assert.ok(!existsSync(join(e.sd,'sudo.log')));assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));
|
||||
assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: A1 refuses open-activation on any AWAITING-UNITS file, through the stack\'s user\'s --plan, with nothing stopped',async(t)=>{
|
||||
const e=await useed(t);
|
||||
const fake=join(e.state,'stack-rollback',`20000101T000000Z-${C1.slice(0,12)}`);mkdirSync(fake,{recursive:true});writeFileSync(join(fake,'AWAITING-UNITS'),`${C1}\n`);
|
||||
const logs=logLines(e).length;
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,3,r.all);
|
||||
assert.ok(r.out.includes('refuses open-activation'),r.out);
|
||||
assert.deepEqual(names(e),[...PRE.slice(0,7),'result']);assert.match(result(e),/^result exit=3 plan-refused /);
|
||||
assert.equal(logLines(e).length,logs);assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));
|
||||
assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: a target that migrates refuses migrate-required without --migrate, with nothing stopped',async(t)=>{
|
||||
const e=await useed(t);
|
||||
const r=await upgrade(e,C);assert.equal(r.code,3,r.all);
|
||||
assert.ok(r.out.includes('refuses migrate-required'),r.out);
|
||||
assert.deepEqual(names(e),[...PRE.slice(0,7),'result']);assert.match(result(e),/^result exit=3 plan-refused /);
|
||||
assert.ok(!sudoKeys(e).includes('rehearse'));assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));
|
||||
assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: the rehearsal runs after the plan and before A2; a failed one refuses rehearsal-failed with nothing stopped',async(t)=>{
|
||||
const e=await useed(t);
|
||||
fault(e,'rehearse-rc','1');
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,3,r.all);
|
||||
assert.ok(r.out.includes('fixture rehearse: '),r.out);
|
||||
assert.deepEqual(names(e),[...PRE,'result']);
|
||||
assert.equal(result(e),'result exit=3 rehearsal-failed the rehearsal exited 1 (nothing was stopped)');
|
||||
const rh=lines(join(e.sd,'sudo.log')).map((l)=>JSON.parse(l)).find((l)=>l.key==='rehearse');
|
||||
assert.deepEqual(rh.cmd.slice(1),['rehearse',C1,'acme','--copies',e.hc,'--backup','last','--port','7339','--timeout','3']);
|
||||
assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: the rehearsal\'s BACKUP_STORE line goes to the step log after a match, a malformed one as none, the rest of its output never',async(t)=>{
|
||||
const e=await useed(t);
|
||||
fault(e,'rehearse-rc','1');
|
||||
const r=await upgrade(e,C1,'--rehearse-port','7449');assert.equal(r.code,3,r.all);
|
||||
assert.ok(slog(e).includes('rehearse-store rc=0 BACKUP_STORE 20261020T000000Z-0123456789ab schema 3g'),slog(e).join('\n'));
|
||||
assert.ok(slog(e).find((x)=>x.startsWith('rehearse ')).includes(' --port 7449 '));
|
||||
for(const bad of ['BACKUP_STORE ../../etc schema 3g','BACKUP_STORE 20261020T000000Z-0123456789ab schema 3g; rm -rf /','stack-release: BACKUP_STORE 20261020T000000Z-0123456789ab schema 3g','']){
|
||||
fault(e,'rehearse-store-line',bad);
|
||||
const n=slog(e).length;
|
||||
const b=await upgrade(e,C1);assert.equal(b.code,3,b.all);
|
||||
assert.deepEqual(slog(e).slice(n).filter((x)=>x.startsWith('rehearse-store ')),['rehearse-store rc=0 BACKUP_STORE none'],bad);
|
||||
}
|
||||
});
|
||||
|
||||
test('row 233: a real rehearsal with no backup in the log refuses no-backup, and goes on only with --no-rehearsal, which logs the skip',async(t)=>{
|
||||
const e=await useed(t);
|
||||
fault(e,'real-rehearse');
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,3,r.all);
|
||||
assert.ok(r.out.includes('FAIL no-backup: the log records no backup\nREHEARSAL_FAIL\n'),r.out);
|
||||
assert.equal(result(e),'result exit=3 no-backup the log records no backup to rehearse against; only a go that names --no-rehearsal goes on (nothing was stopped)');
|
||||
assert.ok(slog(e).includes('rehearse-store rc=0 BACKUP_STORE none'));
|
||||
assert.ok(!existsSync(join(e.data,'rehearsals')));assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));
|
||||
const n=slog(e).length;
|
||||
const s=await upgrade(e,C1,'--no-rehearsal');assert.equal(s.code,0,s.all);
|
||||
assert.deepEqual(names(e).slice(n),[...PRE,'rehearsal-skipped',...A2,'stage','render','confirm','args','result']);
|
||||
assert.equal(argsCopy(e),C1);assert.deepEqual(live(e),['active','active',C1,C1]);
|
||||
});
|
||||
|
||||
test('row 233: a real rehearsal after an upgrade names the log\'s newest backup; --no-rehearsal with a backup refuses rehearsal-required',async(t)=>{
|
||||
const e=await useed(t);
|
||||
const one=await upgrade(e,C1);assert.equal(one.code,0,one.all);
|
||||
const a1=lastAct(e);
|
||||
fault(e,'real-rehearse');
|
||||
const n=slog(e).length;
|
||||
// The stub copies fail the rehearsal right after it names the backup (copy-without-no-launch).
|
||||
const r=await upgrade(e,C,'--migrate');assert.equal(r.code,3,r.all);
|
||||
assert.ok(r.out.includes(`PASS backup-last: the log's newest backup is ${a1}\n`),r.out);
|
||||
assert.deepEqual(names(e).slice(n),[...PRE,'result']);assert.match(result(e),/^result exit=3 rehearsal-failed the rehearsal exited 1 /);
|
||||
rmSync(join(e.sd,'faults','real-rehearse'));
|
||||
const m=slog(e).length;
|
||||
const q=await upgrade(e,C,'--migrate','--no-rehearsal');assert.equal(q.code,3,q.all);
|
||||
assert.deepEqual(names(e).slice(m),[...PRE,'result']);
|
||||
assert.equal(result(e),'result exit=3 rehearsal-required the log records a backup and the rehearsal ran; --no-rehearsal is only for a stack with none (nothing was stopped)');
|
||||
assert.equal(argsCopy(e),C1);assert.deepEqual(live(e),['active','active',C1,C1]);
|
||||
});
|
||||
|
||||
// --- Before A1 and the args file ---
|
||||
|
||||
test('row 233: the args file must be the running user\'s, with no group or other write, one --copy, and the copy the units run',async(t)=>{
|
||||
const e=await useed(t);
|
||||
const tryArgs=async(text,mode,code,why)=>{
|
||||
writeFileSync(e.args,text);chmodSync(e.args,mode);
|
||||
const n=existsSync(e.slog)?slog(e).length:0;
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,3,`${why}: ${r.all}`);
|
||||
assert.match(slog(e).at(-1),new RegExp(`^result exit=3 ${code} `),why);
|
||||
assert.ok(!slog(e).slice(n).some((x)=>/^(stop|stage|render|confirm)/.test(x)),why);
|
||||
};
|
||||
await tryArgs(e.argsText,0o660,'args-unreadable','group write');
|
||||
await tryArgs(e.argsText,0o642,'args-unreadable','other write');
|
||||
await tryArgs(e.argsText.replace('--\n',`--copy\n${C1}\n--\n`),0o640,'args-unreadable','two --copy');
|
||||
await tryArgs(e.argsText.replace('--\n','--tester\n--\n'),0o640,'args-unreadable','an unknown flag');
|
||||
await tryArgs(e.argsText.replace(`--copy\n${B}\n`,`--copy\n${C1}\n`),0o640,'args-differ','the units run B');
|
||||
rmSync(e.args);symlinkSync(join(e.sd,'probe.sh'),e.args);
|
||||
const l=await upgrade(e,C1);assert.equal(l.code,3);assert.match(result(e),/^result exit=3 args-unreadable /);
|
||||
assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));assert.deepEqual(live(e),['active','active',B,B]);
|
||||
});
|
||||
|
||||
test('row 233: hostcheck (row 240): a nonzero --check or a missing file refuses profile-drift before the plan',async(t)=>{
|
||||
const e=await useed(t);
|
||||
fault(e,'hostcheck-rc','1');
|
||||
const r=await upgrade(e,C1);assert.equal(r.code,3,r.all);
|
||||
assert.match(result(e),/^result exit=3 profile-drift /);
|
||||
assert.deepEqual(lines(join(e.sd,'hostcheck.log')),['--check']);
|
||||
assert.deepEqual(sudoKeys(e),['git-fetch','git-merge','stack-copy.sh']);
|
||||
e.denv.STACK_UPGRADE_HOSTCHECK=join(e.root,'no-hostcheck');
|
||||
const m=await upgrade(e,C1);assert.equal(m.code,3,m.all);
|
||||
assert.equal(result(e),`result exit=3 profile-drift no executable ${join(e.root,'no-hostcheck')} (row 240) (nothing was stopped)`);
|
||||
assert.ok(!sudoKeys(e).includes('activate-plan'));assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: the checkout must be at --sha (head-mismatch), and a commit the clone can\'t fast-forward to refuses fetch-failed',async(t)=>{
|
||||
const e=await useed(t);
|
||||
const r=await drive(e,C1,'upgrade','--instance','dev','--bundle',BUNDLE,'--sha',C,'--migrate',...TO);assert.equal(r.code,3,r.all);
|
||||
assert.equal(result(e),`result exit=3 head-mismatch this checkout, ${checkout(C1)}, is at ${C1}, not ${C} (nothing was stopped)`);
|
||||
const u=await drive(e,U,'upgrade','--instance','dev','--bundle',BUNDLE,'--sha',U,...TO);assert.equal(u.code,3,u.all);
|
||||
assert.match(result(e),/^result exit=3 fetch-failed .* fast-forward /);
|
||||
assert.ok(!sudoKeys(e).includes('activate-plan'));assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));argsKept(e);
|
||||
});
|
||||
|
||||
test('row 233: usage exits 4 before any log or lock',async(t)=>{
|
||||
const e=await useed(t);
|
||||
for(const a of [[],['status'],['upgrade','--instance','dev','--bundle',BUNDLE],['upgrade','--instance','other','--bundle',BUNDLE,'--sha',C1],['upgrade','--instance','dev','--bundle','rel/x','--sha',C1],['upgrade','--instance','dev','--bundle',BUNDLE,'--sha',C1.slice(0,12)],['upgrade','--instance','dev','--bundle',BUNDLE,'--sha',C1,'--to',B],['upgrade','--instance','dev','--bundle',BUNDLE,'--sha',C1,'--rehearse-port','80'],['rollback','--instance','dev','--to',B,'--migrate'],['rollback','--instance','dev','--to',B,'--events','3'],['rollback','--instance','dev'],['rollback','--instance','dev','--to',B,'--restore','x']]){
|
||||
const r=await drive(e,C1,...a);assert.equal(r.code,4,a.join(' '));assert.match(r.err,/^usage: stack-upgrade.sh upgrade /,a.join(' '));
|
||||
}
|
||||
assert.ok(!existsSync(dirname(e.slog)));assert.ok(!existsSync(e.run));
|
||||
});
|
||||
+1
-1
@@ -35,6 +35,6 @@ if [ "${1:-}" = business ]; then
|
||||
exec node "$REPO/packages/business/src/cli.mjs" "$@"
|
||||
fi
|
||||
case "${1:-}" in
|
||||
usage|project|limits|inbox|decide|tasks|agents|trail|talk|stop|launches|bus|plugins|credentials|hosts|name|names|follow|threads|thread) exec node "$REPO/packages/cli/src/cli.mjs" "$@" ;;
|
||||
usage|project|limits|inbox|decide|tasks|agents|trail|talk|stop|launches|bus|plugins|credentials|name|names|follow|threads|thread) exec node "$REPO/packages/cli/src/cli.mjs" "$@" ;;
|
||||
esac
|
||||
exec node "$REPO/packages/seat/src/cli.mjs" "$@"
|
||||
|
||||
+62
-13
@@ -3,21 +3,25 @@
|
||||
# copy creates files only; wrapper is a separately Mos-gated live step.
|
||||
# wrapper and check take --instance NAME (row 190): the units mosaic-bus-NAME@ and mosaic-webui-NAME
|
||||
# and the wrapper $BIN/mosaic-NAME, which sets NAME's MOSAIC_CONFIG. check needs --business for NAME.
|
||||
# Row 232: check --units external reads root's system units. copy --bundle makes the copy from a git
|
||||
# bundle on a host with no checkout, and wrapper --create writes the first wrapper on a new host.
|
||||
set -euo pipefail
|
||||
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
COPIES="${XDG_DATA_HOME:-$HOME/.local/share}/mosaic-stack/copies"
|
||||
BIN="$HOME/.npm-global/bin"
|
||||
BUSINESS=mosaic-stack BUSINESS_SET=0
|
||||
ALLOW_WORKTREE=0
|
||||
ALLOW_WORKTREE=0 UNITS=user BUNDLE="" CREATE=0 CLEAN=""
|
||||
MARKER='# mosaic-cli wrapper (Q81, agents/sage/work/q81/mosaic-cli.sh)'
|
||||
usage() { echo 'usage: stack-copy.sh copy COMMIT [--copies DIR] [--allow-worktree] | wrapper ID [--instance NAME] [--copies DIR] [--bin DIR] | check [--instance NAME] [--copies DIR] [--bin DIR] [--business ID]' >&2; exit "${1:-4}"; }
|
||||
usage() { echo 'usage: stack-copy.sh copy COMMIT [--copies DIR] [--allow-worktree] | copy --bundle FILE SHA [--copies DIR] | wrapper ID [--instance NAME] [--copies DIR] [--bin DIR] [--create] | check [--instance NAME] [--copies DIR] [--bin DIR] [--business ID] [--units user|external]' >&2; exit "${1:-4}"; }
|
||||
die() { echo "stack-copy: $*" >&2; exit 1; }
|
||||
plain_path() { case "$1" in /*) ;; *) die "$2 must be an absolute path: $1" ;; esac; case "$1" in *[!A-Za-z0-9/._+-]*) die "$2 contains a character the unit cannot carry: $1" ;; esac; }
|
||||
parse() { while [ $# -gt 0 ]; do case "$1" in
|
||||
--copies) [ $# -ge 2 ] || usage; COPIES="$2"; shift 2;;
|
||||
--bin) [ "$CMD" != copy ] && [ $# -ge 2 ] || usage; BIN="$2";shift 2;;
|
||||
--business) [ "$CMD" = check ] && [ $# -ge 2 ] || usage; BUSINESS="$2";BUSINESS_SET=1;shift 2;;
|
||||
--allow-worktree) [ "$CMD" = copy ] || usage; ALLOW_WORKTREE=1;shift;;
|
||||
--allow-worktree) [ "$CMD" = copy ] && [ -z "$BUNDLE" ] || usage; ALLOW_WORKTREE=1;shift;;
|
||||
--units) [ "$CMD" = check ] && [ $# -ge 2 ] || usage; case "$2" in user|external) UNITS="$2";; *) usage;; esac; shift 2;;
|
||||
--create) [ "$CMD" = wrapper ] || usage; CREATE=1;shift;;
|
||||
*) usage;; esac; done; plain_path "$COPIES" 'copies directory'; }
|
||||
copy_root() { [[ "$ID" =~ ^[0-9a-f]{40}$ ]] || die '--copy must be a full 40-character commit id'; [ -f "$COPIES/$ID/scripts/mosaic" ] || die "no copy $ID in $COPIES"; echo "$COPIES/$ID"; }
|
||||
# Closure of scripts/mosaic, bus process/notifier/launcher, Console serve,
|
||||
@@ -44,16 +48,22 @@ reachable() {
|
||||
copy_commit() {
|
||||
[ -d "$REPO/.git" ] || [ "$ALLOW_WORKTREE" = 1 ] || die "refusing: $REPO is not the canonical checkout (its .git is not a directory); run copy from the canonical checkout"
|
||||
[[ "$ID" =~ ^[A-Za-z0-9][A-Za-z0-9._/-]*$ ]] || die "commit must be a commit id or branch name: $ID"
|
||||
local full target tmp
|
||||
local full
|
||||
full="$(git -C "$REPO" rev-parse --verify --quiet "$ID^{commit}")" || die "not a commit: $ID"
|
||||
reachable "$full"
|
||||
copy_tree "$REPO" "$full"
|
||||
}
|
||||
# The copy of commit $2 from repository $1. CLEAN names a bundle's temporary repository, which the
|
||||
# trap removes with the temporary copy.
|
||||
copy_tree() {
|
||||
local full="$2" target tmp
|
||||
target="$COPIES/$full"
|
||||
[ ! -L "$target" ] || die "refusing: copy target is a symlink: $target"
|
||||
if [ -e "$target" ]; then echo "exists: $target"; return; fi
|
||||
mkdir -p "$COPIES"
|
||||
tmp="$(mktemp -d "$COPIES/.$full.XXXXXX")"
|
||||
trap "chmod -R u+w '$tmp' 2>/dev/null || true; rm -rf '$tmp'" EXIT
|
||||
git -C "$REPO" archive --format=tar "$full" -- "${COPY_PATHS[@]}" | tar -xf - -C "$tmp" || die "git archive failed; no copy written"
|
||||
trap "chmod -R u+w '$tmp' 2>/dev/null || true; rm -rf '$tmp'$CLEAN" EXIT
|
||||
git -C "$1" archive --format=tar "$full" -- "${COPY_PATHS[@]}" | tar -xf - -C "$tmp" || die "git archive failed; no copy written"
|
||||
[ -f "$tmp/package.json" ] && [ -f "$tmp/package-lock.json" ] || die 'package.json or package-lock.json missing in temp copy; npm ci not run'
|
||||
(cd "$tmp" && npm ci --no-audit --no-fund) >&2 || die 'npm ci failed; no copy written'
|
||||
find "$tmp" -type f -exec chmod a-w {} +
|
||||
@@ -65,19 +75,53 @@ copy_commit() {
|
||||
trap - EXIT
|
||||
echo "copied: $target"
|
||||
}
|
||||
# Row 232: the copy of SHA from a git bundle, for a host with no refactor checkout. The bundle must
|
||||
# verify, carry refs/heads/refactor, and reach SHA from it, as reachable asks of a checkout. Prints
|
||||
# the RELEASE value at SHA and the bundle's sha256, the release identity on such a host.
|
||||
copy_bundle() {
|
||||
[[ "$ID" =~ ^[0-9a-f]{40}$ ]] || die "--bundle needs a full 40-character commit id: $ID"
|
||||
[ -f "$BUNDLE" ] && [ ! -L "$BUNDLE" ] || die "refusing: bundle is absent, not regular, or a symlink: $BUNDLE"
|
||||
local g sum release heads
|
||||
case "$BUNDLE" in /*) ;; *) BUNDLE="$PWD/$BUNDLE";; esac
|
||||
g="$(mktemp -d)"
|
||||
CLEAN=" '$g'"
|
||||
trap "rm -rf '$g'" EXIT
|
||||
git init -q --bare "$g" || die 'git init failed; no copy written'
|
||||
git -C "$g" bundle verify -q "$BUNDLE" >/dev/null 2>&1 || die "refusing: git bundle verify failed: $BUNDLE"
|
||||
heads="$(git -C "$g" bundle list-heads "$BUNDLE" refs/heads/refactor)" || die "refusing: cannot list the bundle's heads: $BUNDLE"
|
||||
grep -qx '[0-9a-f]\{40\} refs/heads/refactor' <<< "$heads" || die "refusing: the bundle has no refs/heads/refactor: $BUNDLE"
|
||||
# Every ref, so a SHA the bundle carries on another branch reaches the reachability check.
|
||||
git -C "$g" fetch -q "$BUNDLE" 'refs/*:refs/*' 2>/dev/null || die "refusing: cannot fetch from $BUNDLE"
|
||||
git -C "$g" cat-file -e "$ID^{commit}" 2>/dev/null || die "refusing: $ID is not in the bundle"
|
||||
git -C "$g" merge-base --is-ancestor "$ID" refs/heads/refactor || die "refusing: $ID is not reachable from the bundle's refs/heads/refactor"
|
||||
release="$(git -C "$g" show "$ID:RELEASE")" || die "no RELEASE at $ID"
|
||||
sum="$(sha256sum < "$BUNDLE" | cut -d' ' -f1)"
|
||||
copy_tree "$g" "$ID"
|
||||
rm -rf "$g"; trap - EXIT
|
||||
echo "release: $release"
|
||||
echo "bundle sha256: $sum"
|
||||
}
|
||||
# Row 232: --create writes the wrapper only where none is. ln refuses any existing path, a symlink
|
||||
# included, so this is the one check, and a wrapper that appears mid-run is never replaced.
|
||||
place() {
|
||||
if [ "$CREATE" = 1 ]; then ln "$1" "$2" 2>/dev/null || die "refusing: --create and a wrapper exists: $2"; rm -f "$1"
|
||||
else mv -f "$1" "$2"; fi
|
||||
}
|
||||
wrapper() {
|
||||
[ -z "$MOSAIC_UNIT_SUFFIX" ] || { wrapper_instance; return; }
|
||||
plain_path "$BIN" 'bin directory'
|
||||
local root file lease tmp
|
||||
root="$(copy_root)";file="$BIN/mosaic"
|
||||
[ -f "$file" ] && [ ! -L "$file" ] || die "refusing: wrapper is absent, not regular, or a symlink: $file"
|
||||
[ "$(sed -n '2p' "$file")" = "$MARKER" ] || die "refusing: foreign wrapper (missing Q81 marker on line 2): $file"
|
||||
lease='[ "${1:-}" = __lease-capability ] && exec "'"$BIN"'/mosaic-old" "$@"'
|
||||
[ "$(grep -Fxc "$lease" "$file" || true)" = 1 ] || die 'refusing: Q81 lease line missing or ambiguous'
|
||||
if [ "$CREATE" = 0 ]; then
|
||||
[ -f "$file" ] && [ ! -L "$file" ] || die "refusing: wrapper is absent, not regular, or a symlink: $file"
|
||||
[ "$(sed -n '2p' "$file")" = "$MARKER" ] || die "refusing: foreign wrapper (missing Q81 marker on line 2): $file"
|
||||
[ "$(grep -Fxc "$lease" "$file" || true)" = 1 ] || die 'refusing: Q81 lease line missing or ambiguous'
|
||||
fi
|
||||
tmp="$(mktemp "$BIN/.mosaic.XXXXXX")"
|
||||
trap 'rm -f "$tmp"' EXIT
|
||||
printf '#!/usr/bin/env bash\n%s\n# CLI runs from pinned copy: %s\n# The Q81 lease probe still answers at mosaic-old.\n%s\nexec "%s/scripts/mosaic" "$@"\n' "$MARKER" "$root" "$lease" "$root" > "$tmp"
|
||||
chmod 0755 "$tmp";mv -f "$tmp" "$file";trap - EXIT
|
||||
chmod 0755 "$tmp";place "$tmp" "$file";trap - EXIT
|
||||
echo "wrapper: $root"
|
||||
}
|
||||
# Row 190: $BIN/mosaic-NAME. It creates the file or replaces one with its own marker, never another.
|
||||
@@ -93,12 +137,14 @@ wrapper_instance() {
|
||||
tmp="$(mktemp "$BIN/.$WRAPPER_NAME.XXXXXX")"
|
||||
trap 'rm -f "$tmp"' EXIT
|
||||
printf '#!/usr/bin/env bash\n%s\n# CLI runs from pinned copy: %s\nexport MOSAIC_CONFIG='"'"'%s'"'"'\nexec "%s/scripts/mosaic" "$@"\n' "$INSTANCE_MARKER" "$root" "$MOSAIC_INSTANCE_CONFIG" "$root" > "$tmp"
|
||||
chmod 0755 "$tmp";mv -f "$tmp" "$file";trap - EXIT
|
||||
chmod 0755 "$tmp";place "$tmp" "$file";trap - EXIT
|
||||
echo "wrapper: $root"
|
||||
}
|
||||
# --units external reads root's system units, which the stack user can read and not change.
|
||||
units_show() { if [ "$UNITS" = external ]; then systemctl show "$@"; else systemctl --user show "$@"; fi; }
|
||||
unit_root() {
|
||||
local out load root exec_start
|
||||
out="$(systemctl --user show "$1" -p LoadState -p WorkingDirectory -p ExecStart 2>/dev/null)" || { echo 'absent (systemctl read failed)';return; }
|
||||
out="$(units_show "$1" -p LoadState -p WorkingDirectory -p ExecStart 2>/dev/null)" || { echo 'absent (systemctl read failed)';return; }
|
||||
load="$(sed -n 's/^LoadState=//p' <<< "$out")"; root="$(sed -n 's/^WorkingDirectory=//p' <<< "$out")"
|
||||
exec_start="$(sed -n 's/^ExecStart=//p' <<< "$out")"
|
||||
if [ "$load" != loaded ] || [ -z "$root" ];then echo absent;return;fi
|
||||
@@ -133,7 +179,10 @@ CMD="$1";shift
|
||||
case "$CMD" in check|wrapper) instance_select;; *) [ -z "$INSTANCE_FLAG" ] || usage;; esac
|
||||
INSTANCE_MARKER="# mosaic-cli wrapper, instance ${MOSAIC_INSTANCE:-dev} (row 190)"
|
||||
case "$CMD" in
|
||||
copy|wrapper) [ $# -gt 0 ] || usage;ID="$1";shift;case "$ID" in -*)usage;;esac;parse "$@";if [ "$CMD" = copy ];then copy_commit;else wrapper;fi;;
|
||||
copy|wrapper) [ $# -gt 0 ] || usage
|
||||
if [ "$CMD" = copy ] && [ "$1" = --bundle ];then [ $# -ge 3 ] && [ -n "$2" ] || usage;BUNDLE="$2";shift 2;fi
|
||||
ID="$1";shift;case "$ID" in -*)usage;;esac;parse "$@"
|
||||
if [ -n "$BUNDLE" ];then copy_bundle;elif [ "$CMD" = copy ];then copy_commit;else wrapper;fi;;
|
||||
check) parse "$@";[ -z "$MOSAIC_UNIT_SUFFIX" ] || [ "$BUSINESS_SET" = 1 ] || usage;check;;
|
||||
-h|--help) usage 0;;
|
||||
*) usage;;esac
|
||||
|
||||
+483
-60
@@ -19,6 +19,13 @@
|
||||
# rehearse (D2, row 163) stops and starts no unit and writes only its own rehearsal directory, but
|
||||
# it reads the live config and a backup of the live store, so Jason or Mos runs it.
|
||||
#
|
||||
# --units external (row 232, #1734; r6 section 4 at agents/darkwing/work/queue-195/DESIGN.md) is
|
||||
# for a host whose units are root's system units. This script then reads those units and never
|
||||
# starts or stops one: root stops them, activate or rollback stages the copy (A3), root installs
|
||||
# and starts them, and confirm gates them and writes the pointer (A5). revert and confirm
|
||||
# --rollback put the old copy back when a step fails (R2, R4), and gate checks a copy root
|
||||
# restarted with no activation open (A2f). Without it, everything runs as before.
|
||||
#
|
||||
# Most functions below run as `f || handler`, and bash ignores set -e inside such a call, so
|
||||
# every step that can fail carries its own `|| return 1`.
|
||||
set -euo pipefail
|
||||
@@ -32,6 +39,7 @@ BIN="$HOME/.npm-global/bin"
|
||||
UNIT_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user"
|
||||
TIMEOUT=60 WANT_PM=0 AFTER_HARNESS=0 MIGRATE=0 PLAN=0 RESTORE="" EVENTS="" TESTER_PORT=""
|
||||
PROOF_HEADER="" PROOF_FILE="" NO_PROOF=0 NO_SEND=0 SSO=0 R_BACKUP="" R_PORT=""
|
||||
UNITS=user TO_FLAG="" EXPECT_FROM="" SINCE="" ROLLBACK_OLD="" CAUSE=""
|
||||
|
||||
usage() {
|
||||
cat >&2 <<'USAGE'
|
||||
@@ -42,11 +50,18 @@ usage: stack-release.sh adopt BUSINESS
|
||||
stack-release.sh rollback BUSINESS [--restore ACTIVATION [--events N]] [--timeout S] [--plan]
|
||||
stack-release.sh status BUSINESS
|
||||
stack-release.sh rehearse ID BUSINESS --backup ACTIVATION|PATH --port N [--timeout S]
|
||||
common options: --copies DIR, --bin DIR, --instance NAME
|
||||
stack-release.sh confirm ID BUSINESS --units external [--rollback OLD --cause A3|A4|A5] [--timeout S]
|
||||
stack-release.sh revert ID BUSINESS --units external
|
||||
stack-release.sh gate ID BUSINESS --units external --since EPOCH [--timeout S]
|
||||
common options: --copies DIR, --bin DIR, --instance NAME, --units user|external
|
||||
--instance NAME (row 190) runs on stack instance NAME: its config, data root, units and the wrapper
|
||||
mosaic-NAME. Without it, the default instance, as before.
|
||||
--events N is the total of the per-table counts the post-backup-events refusal prints: events and
|
||||
every table in COUNTED.
|
||||
--units external (row 232) reads root's system units and starts or stops none. adopt, activate,
|
||||
rollback, status, confirm, revert and gate take it; rollback then needs --to ID, the previous copy.
|
||||
activate and rollback --plan take --expect-from OLD. --pm, --pm-after-harness, --tester-port, the
|
||||
proof flags, --no-proof, --no-send, --sso and pm-on are not for it.
|
||||
USAGE
|
||||
exit "${1:-4}"
|
||||
}
|
||||
@@ -58,7 +73,27 @@ now_ms() { date -u +%Y-%m-%dT%H:%M:%S.%3NZ; }
|
||||
jline() {
|
||||
node -e 'const a=process.argv.slice(1),o={};for(let i=0;i<a.length;i+=2)o[a[i]]=a[i+1]==="@null"?null:a[i+1];process.stdout.write(JSON.stringify(o)+"\n")' -- "$@"
|
||||
}
|
||||
log_line() { jline at "$(now)" op "$OP" from "${FROM:-@null}" to "${TO:-@null}" backup "${BACKUP:-@null}" activation "${ACT:-@null}" "$@" >> "$LOG"; }
|
||||
log_line() {
|
||||
[ "$UNITS" = user ] || set -- "$@" units external
|
||||
jline at "$(now)" op "$OP" from "${FROM:-@null}" to "${TO:-@null}" backup "${BACKUP:-@null}" activation "${ACT:-@null}" "$@" >> "$LOG"
|
||||
}
|
||||
|
||||
# --- The units backend (row 232) ---
|
||||
|
||||
# Every systemctl, journalctl and systemd-analyze call goes through these three. user is the
|
||||
# systemd user manager, as before. external reads root's system units, which needs no privilege,
|
||||
# and refuses any verb that would change one.
|
||||
sc() {
|
||||
if [ "$UNITS" = user ]; then systemctl --user "$@"; return; fi
|
||||
case "$1" in
|
||||
show|is-active|list-units) systemctl "$@" ;;
|
||||
*) echo "stack-release: --units external never runs systemctl $1; root does" >&2; return 1 ;;
|
||||
esac
|
||||
}
|
||||
# The bus unit's journal is a user journal only: under external it lands in the system journal,
|
||||
# which the stack's user can't read (r6 section 4), so the external PM check reads launches list.
|
||||
jctl() { [ "$UNITS" = user ] || return 1; journalctl --user "$@"; }
|
||||
analyze() { [ "$UNITS" = user ] || return 1; systemd-analyze --user "$@"; }
|
||||
|
||||
# A refusal before anything stopped: exit 3, and one log line with the code, except for
|
||||
# run-locked (the lock holder may be writing the log) and in --plan, which writes nothing.
|
||||
@@ -83,7 +118,7 @@ parse_opts() {
|
||||
case "$1" in
|
||||
--copies) [ $# -ge 2 ] || usage; COPIES="$2"; shift 2 ;;
|
||||
--bin) [ $# -ge 2 ] || usage; BIN="$2"; shift 2 ;;
|
||||
--timeout) [ "$OP" != status ] && [ "$OP" != adopt ] && [ $# -ge 2 ] && [[ "$2" =~ ^[1-9][0-9]{0,4}$ ]] || usage; TIMEOUT="$2"; shift 2 ;;
|
||||
--timeout) [ "$OP" != status ] && [ "$OP" != adopt ] && [ "$OP" != revert ] && [ $# -ge 2 ] && [[ "$2" =~ ^[1-9][0-9]{0,4}$ ]] || usage; TIMEOUT="$2"; shift 2 ;;
|
||||
--plan) [ "$OP" = activate ] || [ "$OP" = rollback ] || usage; PLAN=1; shift ;;
|
||||
--pm) [ "$OP" = activate ] || usage; WANT_PM=1; shift ;;
|
||||
--pm-after-harness) [ "$OP" = activate ] || usage; AFTER_HARNESS=1; shift ;;
|
||||
@@ -95,12 +130,30 @@ parse_opts() {
|
||||
--sso) [ "$OP" = activate ] || usage; SSO=1; shift ;;
|
||||
--migrate) [ "$OP" = activate ] || usage; MIGRATE=1; shift ;;
|
||||
--restore) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])?$ ]] || usage; RESTORE="$2"; shift 2 ;;
|
||||
--backup) [ "$OP" = rehearse ] && [ $# -ge 2 ] && { [[ "$2" =~ ^[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])?$ ]] || [[ "$2" =~ ^/[A-Za-z0-9/._+-]*$ ]]; } || usage; R_BACKUP="$2"; shift 2 ;;
|
||||
--backup) [ "$OP" = rehearse ] && [ $# -ge 2 ] && { [ "$2" = last ] || [[ "$2" =~ ^[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])?$ ]] || [[ "$2" =~ ^/[A-Za-z0-9/._+-]*$ ]]; } || usage; R_BACKUP="$2"; shift 2 ;;
|
||||
--port) [ "$OP" = rehearse ] && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9]{1,5}$ ]] && [ "$((10#$2))" -ge 1024 ] && [ "$((10#$2))" -le 65535 ] || usage; R_PORT="$((10#$2))"; shift 2 ;;
|
||||
--events) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ ^(0|[1-9][0-9]*)$ ]] || usage; EVENTS="$2"; shift 2 ;;
|
||||
--units) [ "$OP" != rehearse ] && [ $# -ge 2 ] && { [ "$2" = user ] || [ "$2" = external ]; } || usage; UNITS="$2"; shift 2 ;;
|
||||
--to) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9a-f]{40}$ ]] || usage; TO_FLAG="$2"; shift 2 ;;
|
||||
--expect-from) { [ "$OP" = activate ] || [ "$OP" = rollback ]; } && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9a-f]{40}$ ]] || usage; EXPECT_FROM="$2"; shift 2 ;;
|
||||
--since) [ "$OP" = gate ] && [ $# -ge 2 ] && [[ "$2" =~ ^(0|[1-9][0-9]{0,11})$ ]] || usage; SINCE="$2"; shift 2 ;;
|
||||
--rollback) [ "$OP" = confirm ] && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9a-f]{40}$ ]] || usage; ROLLBACK_OLD="$2"; shift 2 ;;
|
||||
--cause) [ "$OP" = confirm ] && [ $# -ge 2 ] && [[ "$2" =~ ^A[345]$ ]] || usage; CAUSE="$2"; shift 2 ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
done
|
||||
# Row 232: confirm, revert, gate, --to and --expect-from are external only, and an external
|
||||
# rollback names its target. gate needs --since, its start floor (r6 N2), and --expect-from
|
||||
# goes with --plan. The flags that change a unit's render or the PM are root's under external.
|
||||
if [ "$UNITS" = user ]; then
|
||||
[ "$OP" != confirm ] && [ "$OP" != revert ] && [ "$OP" != gate ] && [ -z "$TO_FLAG" ] && [ -z "$EXPECT_FROM" ] || usage
|
||||
else
|
||||
[ "$OP" != rollback ] || [ -n "$TO_FLAG" ] || usage
|
||||
[ "$WANT_PM" = 0 ] && [ -z "$TESTER_PORT" ] && [ -z "$PROOF_HEADER" ] && [ -z "$PROOF_FILE" ] && [ "$NO_PROOF" = 0 ] && [ "$NO_SEND" = 0 ] && [ "$SSO" = 0 ] || usage
|
||||
fi
|
||||
[ "$OP" != gate ] || [ -n "$SINCE" ] || usage
|
||||
[ -z "$EXPECT_FROM" ] || [ "$PLAN" = 1 ] || usage
|
||||
[ "${ROLLBACK_OLD:+1}" = "${CAUSE:+1}" ] || usage
|
||||
[ -z "$EVENTS" ] || [ -n "$RESTORE" ] || usage
|
||||
[ "$OP" != rehearse ] || { [ -n "$R_BACKUP" ] && [ -n "$R_PORT" ]; } || usage
|
||||
# A bad tester port is a usage error; --plan lists it as tester-port-invalid instead (row 180).
|
||||
@@ -167,7 +220,8 @@ for(const k of open.keys())process.stdout.write(k+"\n");' -- "$LOG"
|
||||
|
||||
# Reads the log's ok lines, newest first. `previous` prints rollback's target: the first `to`
|
||||
# that is not the pointer's copy (release.sh's rule). `backup` prints the backup field of the
|
||||
# line that made the pointer's copy live, the backup a rollback would restore. `arrival` prints the
|
||||
# line that made the pointer's copy live, the backup a rollback would restore. `newest` prints the
|
||||
# backup field of the newest line that has one, rehearse's --backup last (row 233). `arrival` prints the
|
||||
# activation and `from` of the line that moved the stack onto the pointer's copy from another one,
|
||||
# past any same-copy re-pin after it (row 200, sage-1389).
|
||||
from_log() {
|
||||
@@ -180,6 +234,7 @@ for(const l of lines.reverse()){
|
||||
if(o.result!=="ok"||!["adopt","activate","rollback"].includes(o.op)||!o.to)continue;
|
||||
if(mode==="previous"&&o.to!==current){process.stdout.write(o.to+"\n");process.exit(0)}
|
||||
if(mode==="backup"&&o.to===current){if(o.backup)process.stdout.write(o.backup+"\n");process.exit(o.backup?0:1)}
|
||||
if(mode==="newest"&&o.backup){process.stdout.write(o.backup+"\n");process.exit(0)}
|
||||
if(mode==="arrival"&&o.to===current&&o.from!==current){process.stdout.write(`${o.activation??""} ${o.from??""}\n`);process.exit(0)}
|
||||
}
|
||||
process.exit(1);' -- "$LOG" "$1" "$P_COPY"
|
||||
@@ -187,7 +242,7 @@ process.exit(1);' -- "$LOG" "$1" "$P_COPY"
|
||||
|
||||
# --- Reading the units ---
|
||||
|
||||
show() { systemctl --user show "$1" -p LoadState -p WorkingDirectory -p ExecStart -p Environment -p MainPID -p DropInPaths 2>/dev/null; }
|
||||
show() { sc show "$1" -p LoadState -p WorkingDirectory -p ExecStart -p Environment -p MainPID -p DropInPaths 2>/dev/null; }
|
||||
argv_of() { sed -n 's/^ExecStart=.*argv\[\]=\([^;]*\) ;.*/\1/p' <<< "$1"; }
|
||||
# systemctl show quotes a value that holds a space and escapes with a backslash, and the read -a
|
||||
# split below can't undo either, so such a line is unreadable rather than read wrong (row 166 item 4).
|
||||
@@ -286,12 +341,20 @@ check_target() {
|
||||
# has KIND exits 0 when the copy's KINDS has KIND
|
||||
# unknown prints the business file's plugins keys the copy's KINDS lacks, from a plain JSON read
|
||||
# harness prints plugins.harness as JSON, or `none`, from the copy's loader
|
||||
# launchby prints launch.by, the PM's instance, from a plain JSON read (row 232); it imports
|
||||
# only business.mjs, for the business file's path
|
||||
# A failure exits 1 and prints its first line.
|
||||
PLUGINS_JS='
|
||||
const {readFileSync}=require("node:fs");const {pathToFileURL}=require("node:url");
|
||||
const [mode,copy,id,kind]=process.argv.slice(1);
|
||||
const imp=(p)=>import(pathToFileURL(`${copy}/packages/${p}`).href);
|
||||
(async()=>{
|
||||
if(mode==="launchby"){
|
||||
const {businessFilePath}=await imp("business/src/business.mjs");
|
||||
const by=JSON.parse(readFileSync(businessFilePath(id),"utf8")).launch?.by;
|
||||
if(typeof by!=="string"||!by)throw new Error("the business file has no launch.by");
|
||||
return process.stdout.write(by);
|
||||
}
|
||||
const {KINDS}=await imp("plugins/src/kinds.mjs");
|
||||
if(mode==="has")process.exit(Object.hasOwn(KINDS,kind)?0:1);
|
||||
const b=await imp("business/src/business.mjs");
|
||||
@@ -455,13 +518,13 @@ capture() {
|
||||
|
||||
# Stops the Console, then the bus, and waits for both and the stopped check, within --timeout.
|
||||
stop_both() {
|
||||
systemctl --user stop "$CONSOLE_UNIT" || return 1
|
||||
systemctl --user stop "$BUS_UNIT" || return 1
|
||||
sc stop "$CONSOLE_UNIT" || return 1
|
||||
sc stop "$BUS_UNIT" || return 1
|
||||
local deadline=$((SECONDS + TIMEOUT)) u s down
|
||||
while :; do
|
||||
down=1
|
||||
for u in "$CONSOLE_UNIT" "$BUS_UNIT"; do
|
||||
s="$(systemctl --user is-active "$u" 2>/dev/null || true)"
|
||||
s="$(sc is-active "$u" 2>/dev/null || true)"
|
||||
case "$s" in inactive|failed) ;; *) down=0 ;; esac
|
||||
done
|
||||
if [ "$down" = 1 ] && stopped; then say "stopped both units"; return 0; fi
|
||||
@@ -472,7 +535,7 @@ stop_both() {
|
||||
|
||||
start_both() {
|
||||
START_EPOCH="$(date +%s)" && START_MS="$(date +%s%3N)" || return 1
|
||||
systemctl --user start "$BUS_UNIT" && systemctl --user start "$CONSOLE_UNIT"
|
||||
sc start "$BUS_UNIT" && sc start "$CONSOLE_UNIT"
|
||||
}
|
||||
|
||||
# Step 4: installs copy $1 with pm $2, claude $3 and the Console flags after them. Sets FAIL.
|
||||
@@ -493,9 +556,19 @@ install_stack() {
|
||||
gate_once() {
|
||||
local copy="$COPIES/$1" pm="$2" main out
|
||||
GATE=units-inactive GATE_WHY=""
|
||||
systemctl --user is-active --quiet "$BUS_UNIT" && systemctl --user is-active --quiet "$CONSOLE_UNIT" || return 1
|
||||
sc is-active --quiet "$BUS_UNIT" && sc is-active --quiet "$CONSOLE_UNIT" || return 1
|
||||
if [ "$UNITS" = external ]; then
|
||||
# r6 N4: root started the units, so this start's time is the bus unit's, in whole seconds,
|
||||
# at or after FLOOR. It is a gate code, so polling goes on: Restart=on-failure can bring a
|
||||
# fresh start.
|
||||
GATE=stale-start
|
||||
out="$(sc show --timestamp=unix -p ActiveEnterTimestamp "$BUS_UNIT" 2>/dev/null | sed -n 's/^ActiveEnterTimestamp=@//p')" || return 1
|
||||
[[ "$out" =~ ^[0-9]{1,12}$ ]] || { GATE_WHY="ActiveEnterTimestamp is unreadable"; return 1; }
|
||||
[ "$out" -ge "$FLOOR" ] || { GATE_WHY="the bus started at @$out, before @$FLOOR"; return 1; }
|
||||
START_MS=$((out * 1000))
|
||||
fi
|
||||
GATE=bus-status
|
||||
main="$(systemctl --user show "$BUS_UNIT" -p MainPID | sed -n 's/^MainPID=//p')" || return 1
|
||||
main="$(sc show "$BUS_UNIT" -p MainPID | sed -n 's/^MainPID=//p')" || return 1
|
||||
out="$("$copy/scripts/mosaic" bus status --json 2>/dev/null)" || { GATE_WHY="bus status failed"; return 1; }
|
||||
# Every `bus start` writes boot.at (packages/cli/src/boot-status.mjs), so a value from before
|
||||
# this start means the host on the socket is not the one this run started.
|
||||
@@ -508,13 +581,13 @@ process.stdout.write(why);' -- "$out" "$BUSINESS" "$main" "$START_MS" 2>&1)" ||
|
||||
node -e '
|
||||
fetch(`http://127.0.0.1:${process.argv[1]}/healthz`,{signal:AbortSignal.timeout(5000)}).then((r)=>process.exit(r.status===200?0:1),()=>process.exit(1));' -- "$(console_port)" || return 1
|
||||
GATE=copy-check
|
||||
out="$(scripts/stack-copy.sh check "${INSTANCE_FLAGS[@]}" --copies "$COPIES" --bin "$BIN" --business "$BUSINESS" 2>/dev/null)" || return 1
|
||||
out="$(scripts/stack-copy.sh check "${INSTANCE_FLAGS[@]}" "${UNITS_FLAGS[@]}" --copies "$COPIES" --bin "$BIN" --business "$BUSINESS" 2>/dev/null)" || return 1
|
||||
# check passes when the three agree on one copy; this run needs it to be the target (row 166 item 3).
|
||||
out="$(sed -n 's/^bus: //p' <<< "$out")"
|
||||
[ "$out" = "$copy" ] || { GATE_WHY="the bus, Console and CLI agree on $out, not $copy"; return 1; }
|
||||
if [ "$pm" = 1 ]; then
|
||||
GATE=pm-launch-unconfirmed
|
||||
pm_launched "$copy" || return 1
|
||||
if [ "$UNITS" = user ]; then pm_launched "$copy" || return 1; else pm_external "$1" || return 1; fi
|
||||
fi
|
||||
GATE=""
|
||||
}
|
||||
@@ -526,7 +599,7 @@ fetch(`http://127.0.0.1:${process.argv[1]}/healthz`,{signal:AbortSignal.timeout(
|
||||
# because this host's journalctl reads a bare time as local time.
|
||||
pm_launched() {
|
||||
local journal run list
|
||||
journal="$(journalctl --user -u "$BUS_UNIT" --since "@$START_EPOCH" -o cat 2>/dev/null)" || { GATE_WHY="journal read failed"; return 1; }
|
||||
journal="$(jctl -u "$BUS_UNIT" --since "@$START_EPOCH" -o cat 2>/dev/null)" || { GATE_WHY="journal read failed"; return 1; }
|
||||
local -a lines=()
|
||||
mapfile -t lines < <(grep '^launched ' <<< "$journal" || true)
|
||||
[ "${#lines[@]}" -gt 0 ] || { GATE_WHY="no launched line since the start"; return 1; }
|
||||
@@ -540,6 +613,20 @@ process.stdout.write(!s?`run ${run} is not in launches list`:s.business!==b?`run
|
||||
[ -z "$GATE_WHY" ]
|
||||
}
|
||||
|
||||
# r6 section 4: the PM check under --units external, where the bus unit's journal is the system
|
||||
# journal and unreadable here. It passes when exactly one live launches list entry is for the
|
||||
# PM's instance (launch.by, packages/cli/src/launcher.mjs:675), and that entry is for BUSINESS and
|
||||
# started at or after this start. A run the PM starts for another role has another instance.
|
||||
pm_external() {
|
||||
local by list
|
||||
by="$(plugins launchby "$1" 2>&1)" || { GATE_WHY="cannot read launch.by: $by"; return 1; }
|
||||
list="$("$COPIES/$1/scripts/mosaic" launches list --json 2>/dev/null)" || { GATE_WHY="launches list failed"; return 1; }
|
||||
GATE_WHY="$(node -e '
|
||||
const [list,by,b,since]=process.argv.slice(1),l=JSON.parse(list).filter((x)=>x.instance===by&&x.live===true),s=l[0];
|
||||
process.stdout.write(l.length===0?`no live launch of ${by}`:l.length>1?`${l.length} live launches of ${by}`:s.business!==b?`the launch of ${by} is for ${s.business}`:!(Date.parse(s.startedAt)>=Number(since))?`run ${s.run} started before this start`:"");' -- "$list" "$by" "$BUSINESS" "$START_MS" 2>&1)" || { GATE_WHY="launches list unreadable"; return 1; }
|
||||
[ -z "$GATE_WHY" ]
|
||||
}
|
||||
|
||||
# Polls every second until every check passes or --timeout runs out.
|
||||
gate() {
|
||||
local deadline=$((SECONDS + TIMEOUT))
|
||||
@@ -573,7 +660,7 @@ put_back() {
|
||||
done
|
||||
[ "$(sha256sum < "$BIN/$WRAPPER_NAME" | cut -d' ' -f1)" = "$(sum_of "$RS" wrapper)" ] || return 1
|
||||
[ "$(stat -c %a "$BIN/$WRAPPER_NAME")" = "$(cat "$RS/WRAPPER_MODE")" ] || return 1
|
||||
systemctl --user daemon-reload || return 1
|
||||
sc daemon-reload || return 1
|
||||
say "put back the unit files and the wrapper from $RS"
|
||||
}
|
||||
|
||||
@@ -608,8 +695,8 @@ after_fail() {
|
||||
# Stop both units, print both results, log rollback-failed, exit 2. No retry.
|
||||
rollback_failed() {
|
||||
trap - INT TERM HUP
|
||||
systemctl --user stop "$CONSOLE_UNIT" || true
|
||||
systemctl --user stop "$BUS_UNIT" || true
|
||||
sc stop "$CONSOLE_UNIT" || true
|
||||
sc stop "$BUS_UNIT" || true
|
||||
say "rollback-failed: run $1, rollback $2"
|
||||
log_line result rollback-failed code "$1" rollback failed rollbackCode "$2"
|
||||
echo "stack-release: rollback-failed: the run failed with $1 and the rollback with $2; both units are stopped. Rollback set: $RS${BACKUP:+, backup $DATA/backups/stack/$BACKUP}${RESTORED:+, post-attempt store $POST}" >&2
|
||||
@@ -618,10 +705,15 @@ rollback_failed() {
|
||||
|
||||
# --- Refusals (item 3) ---
|
||||
|
||||
lock() {
|
||||
# confirm and revert act on the open activation by design, so they take the lock alone (r6 section 4).
|
||||
lock_only() {
|
||||
mkdir -p "$STATE"
|
||||
exec 9> "$STATE/stack-release.lock"
|
||||
flock -n 9 || refuse run-locked "another stack-release run holds $STATE/stack-release.lock"
|
||||
}
|
||||
|
||||
lock() {
|
||||
lock_only
|
||||
local open
|
||||
open="$(interrupted | paste -sd' ' -)"
|
||||
[ -z "$open" ] || refuse interrupted-run "activation $open started and has no final line; Jason or Mos settles it (status shows it)"
|
||||
@@ -629,7 +721,7 @@ lock() {
|
||||
|
||||
instance_busy() {
|
||||
local out unit load state rest
|
||||
out="$(systemctl --user list-units --all --no-legend --plain "$BUS_GLOB" 2>/dev/null)" || refuse instance-busy "systemctl --user list-units failed, so the bus state is unknown"
|
||||
out="$(sc list-units --all --no-legend --plain "$BUS_GLOB" 2>/dev/null)" || refuse instance-busy "$SC_NAME list-units failed, so the bus state is unknown"
|
||||
while read -r unit load state rest; do
|
||||
[[ "$unit" == $BUS_GLOB ]] || continue
|
||||
if [ "$unit" = "$BUS_UNIT" ]; then
|
||||
@@ -645,7 +737,7 @@ instance_busy() {
|
||||
drop_in_override() {
|
||||
local paths directory dropdir file content glob
|
||||
local -a dropdirs
|
||||
paths="$(systemd-analyze --user unit-paths 2>/dev/null)" && [ -n "$paths" ] || refuse drop-in-override "systemd unit paths are unknown"
|
||||
paths="$(analyze unit-paths 2>/dev/null)" && [ -n "$paths" ] || refuse drop-in-override "systemd unit paths are unknown"
|
||||
while IFS= read -r directory; do
|
||||
[ -n "$directory" ] || continue
|
||||
case "$directory" in /*) ;; *) refuse drop-in-override "invalid systemd unit path $directory" ;; esac
|
||||
@@ -679,7 +771,8 @@ flags_drift() {
|
||||
if [ "$AFTER_HARNESS" = 1 ]; then
|
||||
[ "$P_PM" = 1 ] || d+=$'\n'" pm: pointer $P_PM, --pm-after-harness needs a pointer with pm 1"
|
||||
else
|
||||
[ "$OP" != activate ] || [ "$WANT_PM" = "$P_PM" ] || d+=$'\n'" pm: pointer $P_PM, --pm given $WANT_PM"
|
||||
# Under --units external the PM is a flag in root's unit, so there is no --pm to compare.
|
||||
[ "$UNITS" != user ] || [ "$OP" != activate ] || [ "$WANT_PM" = "$P_PM" ] || d+=$'\n'" pm: pointer $P_PM, --pm given $WANT_PM"
|
||||
fi
|
||||
[ -z "$d" ] || refuse unit-flags-drift "the units differ from the pointer; settle it first:$d"
|
||||
}
|
||||
@@ -1075,11 +1168,12 @@ do_adopt() {
|
||||
lock
|
||||
[ ! -f "$POINTER" ] || { read_pointer; FROM="$P_COPY"; refuse already-adopted "$POINTER exists (copy $P_COPY)"; }
|
||||
local out
|
||||
out="$(scripts/stack-copy.sh check "${INSTANCE_FLAGS[@]}" --copies "$COPIES" --bin "$BIN" --business "$BUSINESS" 2>&1)" || refuse check-failed "stack-copy.sh check did not pass:"$'\n'"$out"
|
||||
out="$(scripts/stack-copy.sh check "${INSTANCE_FLAGS[@]}" "${UNITS_FLAGS[@]}" --copies "$COPIES" --bin "$BIN" --business "$BUSINESS" 2>&1)" || refuse check-failed "stack-copy.sh check did not pass:"$'\n'"$out"
|
||||
TO="$(sed -n 's/^bus: //p' <<< "$out")"
|
||||
TO="${TO#"$COPIES/"}"
|
||||
read_units || refuse check-failed "cannot read $BUS_UNIT or $CONSOLE_UNIT"
|
||||
[ -n "$U_CLAUDE" ] || refuse no-claude-pin "$BUS_UNIT has no MOSAIC_CLAUDE_BIN"
|
||||
# Under --units external the claude pin is root's render, so adopt records what the unit has.
|
||||
[ "$UNITS" = external ] || [ -n "$U_CLAUDE" ] || refuse no-claude-pin "$BUS_UNIT has no MOSAIC_CLAUDE_BIN"
|
||||
target_schema "$TO" || refuse copy-schema-unknown "cannot read SCHEMA_VERSION from $COPIES/$TO/packages/bus/src/store.mjs"
|
||||
next_activation
|
||||
write_pointer "$TO" "$U_PM" 0 "$U_CLAUDE" "$T_SCHEMA" "${U_CONSOLE[@]}"
|
||||
@@ -1094,12 +1188,12 @@ plan_store() {
|
||||
local src="" why=""
|
||||
S_SCHEMA="" S_FROM="" S_CHANGED=""
|
||||
if [ ! -e "$STORE" ]; then why="there is no store file"
|
||||
elif systemctl --user is-active --quiet "$BUS_UNIT"; then
|
||||
elif sc is-active --quiet "$BUS_UNIT"; then
|
||||
src="the store the running bus holds"
|
||||
S_SCHEMA="$(node -e "$STORE_JS" -- heldversion "$STORE" 2>/dev/null)" || S_SCHEMA=""
|
||||
# A bus that stops during the readOnly open can leave -wal or -shm beside the stopped
|
||||
# store, so the version read can't be planned on (row 166 item 2).
|
||||
systemctl --user is-active --quiet "$BUS_UNIT" || S_CHANGED=1
|
||||
sc is-active --quiet "$BUS_UNIT" || S_CHANGED=1
|
||||
elif store_unclean; then why="the store was not read: $UNCLEAN is present"
|
||||
else
|
||||
src="the stopped store"
|
||||
@@ -1170,6 +1264,8 @@ plan() {
|
||||
echo "needs: --migrate$([ "$MIGRATE" = 1 ] && echo ' (given)' || echo ' (not given; the run would roll back with migration-needs-flag)')"
|
||||
echo "needs: Jason present for this run (decision 96 condition 2)"
|
||||
[ "$MIGRATE" = 1 ] || early=1
|
||||
# Row 233: root reads only the plan's exit, so under external a missing --migrate refuses here.
|
||||
[ "$UNITS" = user ] || [ "$MIGRATE" = 1 ] || PLAN_REFUSE+=("migrate-required: the store is $S_SCHEMA and $TO migrates it to $T_SCHEMA; the go names --migrate")
|
||||
elif [ "$OP" = activate ]; then
|
||||
echo "migration: none possible; $S_SCHEMA is not in $T_SCHEMA's migration chain, so the run would roll back with store-newer"
|
||||
early=1
|
||||
@@ -1186,15 +1282,19 @@ plan() {
|
||||
echo "backup: the run backs up the stopped store to $DATA/backups/stack/<activation>/ before any migration"
|
||||
fi
|
||||
plan_commits
|
||||
[ "$P_PENDING" = 0 ] || echo "pm: the pointer is pending (activate --pm-after-harness); the bus runs without its PM"
|
||||
[ "$RUN_PENDING" = 0 ] || echo "pm: the bus starts without --pm and the PM stays pending; next: Jason's plugins set of harness, then pm-on"
|
||||
local flags="${INSTANCE_FLAGS[*]:+${INSTANCE_FLAGS[*]} }"
|
||||
echo "bus: scripts/bus-service.sh install $flags--copy $TO --copies $COPIES$([ "$RUN_PM" = 0 ] || echo ' --pm') --claude $P_CLAUDE"
|
||||
[ -z "$DROP_TESTER" ] || echo "drop $DROP_TESTER; the target Console does not support it"
|
||||
echo "$PROOF_PLAN"
|
||||
echo "$SEND_PLAN"
|
||||
[ -z "$SSO_PLAN" ] || echo "$SSO_PLAN"
|
||||
echo "Console: scripts/webui-service.sh install $flags--copy $TO --copies $COPIES${RUN_CONSOLE[*]:+ ${RUN_CONSOLE[*]}}"
|
||||
if [ "$UNITS" = external ]; then
|
||||
echo "units: external; root installs the units for $TO and starts them, then confirm gates them"
|
||||
else
|
||||
[ "$P_PENDING" = 0 ] || echo "pm: the pointer is pending (activate --pm-after-harness); the bus runs without its PM"
|
||||
[ "$RUN_PENDING" = 0 ] || echo "pm: the bus starts without --pm and the PM stays pending; next: Jason's plugins set of harness, then pm-on"
|
||||
echo "bus: scripts/bus-service.sh install $flags--copy $TO --copies $COPIES$([ "$RUN_PM" = 0 ] || echo ' --pm') --claude $P_CLAUDE"
|
||||
[ -z "$DROP_TESTER" ] || echo "drop $DROP_TESTER; the target Console does not support it"
|
||||
echo "$PROOF_PLAN"
|
||||
echo "$SEND_PLAN"
|
||||
[ -z "$SSO_PLAN" ] || echo "$SSO_PLAN"
|
||||
echo "Console: scripts/webui-service.sh install $flags--copy $TO --copies $COPIES${RUN_CONSOLE[*]:+ ${RUN_CONSOLE[*]}}"
|
||||
fi
|
||||
echo "CLI: scripts/stack-copy.sh wrapper $TO $flags--copies $COPIES --bin $BIN"
|
||||
local r
|
||||
for r in "${PLAN_REFUSE[@]}"; do echo "refuses $r"; done
|
||||
@@ -1268,6 +1368,16 @@ do_activate() {
|
||||
pointer_for_business
|
||||
run_pm
|
||||
[ "$P_PENDING" = 0 ] || check_refuse pm-pending "the pointer is pending (activate --pm-after-harness); run pm-on or rollback first"
|
||||
if [ "$UNITS" = external ]; then
|
||||
ext_pointer_checks
|
||||
if [ "$PLAN" = 1 ]; then check_target; plugin_checks; config_checks; plan; exit 0; fi
|
||||
ext_precheck
|
||||
plugin_checks
|
||||
config_checks
|
||||
new_activation
|
||||
run_external migrate
|
||||
return
|
||||
fi
|
||||
if [ "$PLAN" = 1 ]; then check_target; console_for_target; ports_unclaimed; plugin_checks; config_checks; plan; exit 0; fi
|
||||
precheck
|
||||
plugin_checks
|
||||
@@ -1281,43 +1391,328 @@ do_rollback() {
|
||||
pointer_for_business
|
||||
TO="$(from_log previous)" || refuse no-previous "the log names no earlier copy than $P_COPY"
|
||||
run_pm
|
||||
if [ "$UNITS" = external ]; then
|
||||
# r6: root names the target from the go, and it must be the one rollback reads from the log.
|
||||
[ "$TO_FLAG" = "$TO" ] || refuse to-not-previous "--to $TO_FLAG is not the previous copy; the log names $TO"
|
||||
ext_pointer_checks
|
||||
if [ "$PLAN" = 1 ]; then
|
||||
check_target; plugin_checks; config_checks
|
||||
[ -z "$RESTORE" ] || { next_activation; restore_checks; }
|
||||
plan; exit 0
|
||||
fi
|
||||
ext_precheck
|
||||
plugin_checks
|
||||
config_checks
|
||||
[ "$P_SCHEMA" = "$T_SCHEMA" ] || [ -n "$RESTORE" ] || refuse schema-differs "the store is at $P_SCHEMA and $TO expects $T_SCHEMA; pass --restore ACTIVATION"
|
||||
new_activation
|
||||
[ -z "$RESTORE" ] || restore_checks
|
||||
run_external restore
|
||||
return
|
||||
fi
|
||||
if [ "$PLAN" = 1 ]; then check_target; console_for_target; ports_unclaimed; plugin_checks; config_checks; plan; exit 0; fi
|
||||
precheck
|
||||
plugin_checks
|
||||
config_checks
|
||||
[ "$P_SCHEMA" = "$T_SCHEMA" ] || [ -n "$RESTORE" ] || refuse schema-differs "the store is at $P_SCHEMA and $TO expects $T_SCHEMA; pass --restore ACTIVATION"
|
||||
new_activation
|
||||
if [ -n "$RESTORE" ]; then
|
||||
local o="$DATA/backups/stack/$RESTORE" rc=0 c="" text
|
||||
[ -f "$o/SHA256SUMS" ] && [ "$(cat "$o/DATA_ROOT" 2>/dev/null)" = "$DATA" ] && [ "$(cat "$o/SCHEMA" 2>/dev/null)" = "$T_SCHEMA" ] && [ -f "$o/STAMP" ] \
|
||||
|| refuse restore-mismatch "$o is not a backup of this root at $T_SCHEMA"
|
||||
# Before anything stops. A store the running bus holds is read with readOnly, which adds
|
||||
# no file (the probe); a stopped one falls under the store_unclean rule.
|
||||
if systemctl --user is-active --quiet "$BUS_UNIT"; then
|
||||
c="$(node -e "$STORE_JS" -- heldcount "$STORE" "$(cat "$o/STAMP")" "$COUNTED")" || rc=1
|
||||
# plan_store's recheck (row 166 item 2), before anything stops.
|
||||
systemctl --user is-active --quiet "$BUS_UNIT" || refuse restore-mismatch "the bus stopped while the events after $(cat "$o/STAMP") were counted, so the store state changed; nothing was stopped"
|
||||
else
|
||||
! store_unclean || refuse store-unclean "$UNCLEAN is present; the bus did not close the store cleanly"
|
||||
c="$(store_read count "$(cat "$o/STAMP")" "$COUNTED")" || rc=1
|
||||
fi
|
||||
[ "$rc" = 0 ] || refuse restore-mismatch "cannot count the rows after $(cat "$o/STAMP")"
|
||||
COUNT="${c%%$'\t'*}" text="${c#*$'\t'}"
|
||||
echo "restore: $o; the store now in place moves to $POST and is kept; $text were written after the backup"
|
||||
[ "$COUNT" = 0 ] || [ "$EVENTS" = "$COUNT" ] || refuse post-backup-events "$text were written after the backup; Jason decides, then --events $COUNT"
|
||||
BACKUP="$RESTORE"
|
||||
fi
|
||||
[ -z "$RESTORE" ] || restore_checks
|
||||
run_steps restore
|
||||
}
|
||||
|
||||
# rollback --restore's checks on the backup and the rows written after it, before anything stops.
|
||||
# Sets COUNT and BACKUP; POST comes from new_activation.
|
||||
restore_checks() {
|
||||
local o="$DATA/backups/stack/$RESTORE" rc=0 c="" text
|
||||
[ -f "$o/SHA256SUMS" ] && [ "$(cat "$o/DATA_ROOT" 2>/dev/null)" = "$DATA" ] && [ "$(cat "$o/SCHEMA" 2>/dev/null)" = "$T_SCHEMA" ] && [ -f "$o/STAMP" ] \
|
||||
|| refuse restore-mismatch "$o is not a backup of this root at $T_SCHEMA"
|
||||
# Before anything stops. A store the running bus holds is read with readOnly, which adds
|
||||
# no file (the probe); a stopped one falls under the store_unclean rule.
|
||||
if sc is-active --quiet "$BUS_UNIT"; then
|
||||
c="$(node -e "$STORE_JS" -- heldcount "$STORE" "$(cat "$o/STAMP")" "$COUNTED")" || rc=1
|
||||
# plan_store's recheck (row 166 item 2), before anything stops.
|
||||
sc is-active --quiet "$BUS_UNIT" || refuse restore-mismatch "the bus stopped while the events after $(cat "$o/STAMP") were counted, so the store state changed; nothing was stopped"
|
||||
else
|
||||
! store_unclean || refuse store-unclean "$UNCLEAN is present; the bus did not close the store cleanly"
|
||||
c="$(store_read count "$(cat "$o/STAMP")" "$COUNTED")" || rc=1
|
||||
fi
|
||||
[ "$rc" = 0 ] || refuse restore-mismatch "cannot count the rows after $(cat "$o/STAMP")"
|
||||
COUNT="${c%%$'\t'*}" text="${c#*$'\t'}"
|
||||
echo "restore: $o; the store now in place moves to $POST and is kept; $text were written after the backup"
|
||||
[ "$COUNT" = 0 ] || [ "$EVENTS" = "$COUNT" ] || refuse post-backup-events "$text were written after the backup; Jason decides, then --events $COUNT"
|
||||
BACKUP="$RESTORE"
|
||||
}
|
||||
|
||||
# --- --units external (row 232, r6 section 4) ---
|
||||
|
||||
# After the pointer, for activate and rollback, in --plan and in the run. OLD is the copy root read
|
||||
# from the unit files it installed (r6 A1, N5), so a pointer that names another copy stops the
|
||||
# re-pin before root stops anything. The PM is a flag in root's unit, so --pm-after-harness has
|
||||
# nothing to wait on. The run's own lock refuses an open activation; --plan takes no lock, so it
|
||||
# lists one here.
|
||||
ext_pointer_checks() {
|
||||
[ -z "$EXPECT_FROM" ] || [ "$EXPECT_FROM" = "$P_COPY" ] || check_refuse pointer-differs "the pointer names $P_COPY and root's units name $EXPECT_FROM; Jason or Mos settles it"
|
||||
[ "$AFTER_HARNESS" = 0 ] || check_refuse external-pm "under --units external the PM is a flag in root's bus unit; changing it is a re-install with its own go"
|
||||
[ "$PLAN" = 1 ] || return 0
|
||||
local open
|
||||
open="$(interrupted | paste -sd' ' -)"
|
||||
[ -z "$open" ] || check_refuse interrupted-run "activation $open started and has no final line; Jason or Mos settles it (status shows it)"
|
||||
# Row 233: root can't read the stack's state, so the plan lists any rollback set still waiting
|
||||
# for units, whatever the log says.
|
||||
local m sets=""
|
||||
local -a markers
|
||||
mapfile -t markers < <(compgen -G "$STATE/stack-rollback/*/AWAITING-UNITS" || true)
|
||||
for m in "${markers[@]}"; do sets+=" $(basename "$(dirname "$m")")"; done
|
||||
[ -z "$sets" ] || check_refuse open-activation "rollback set(s)$sets hold AWAITING-UNITS; root finishes them with confirm or revert first"
|
||||
}
|
||||
|
||||
# A3's checks before `started`. Root has stopped the units and installs them after this run, so
|
||||
# there is no Console flag, port, drop-in or claude pin of this script's to check.
|
||||
ext_precheck() {
|
||||
check_target
|
||||
instance_busy
|
||||
read_units || refuse unit-flags-drift "cannot read $BUS_UNIT or $CONSOLE_UNIT"
|
||||
flags_drift
|
||||
units_down
|
||||
}
|
||||
|
||||
# r6 A3 (N4): each unit reads inactive, or failed as stop_both accepts, and the store is closed,
|
||||
# so nothing below runs against a store a live bus holds.
|
||||
units_down() {
|
||||
local u s
|
||||
for u in "$CONSOLE_UNIT" "$BUS_UNIT"; do
|
||||
s="$(sc is-active "$u" 2>/dev/null || true)"
|
||||
case "$s" in inactive|failed) ;; *) refuse units-running "$u is ${s:-unknown}; root stops the units first" ;; esac
|
||||
done
|
||||
stopped || refuse units-running "the store under $DATA/bus is still open (writer.lock or an open file)"
|
||||
}
|
||||
|
||||
# A3's capture into $RS, which run_external made: the pointer, the wrapper and its mode, and for each
|
||||
# unit its show output and the sha256 of its unit file. It reads root's units and makes no directory.
|
||||
capture_units() {
|
||||
local u frag
|
||||
[ ! -f "$POINTER" ] || cp -p "$POINTER" "$RS/pointer" || return 1
|
||||
[ -f "$BIN/$WRAPPER_NAME" ] || { say "capture: no wrapper $BIN/$WRAPPER_NAME"; return 1; }
|
||||
cp -p "$BIN/$WRAPPER_NAME" "$RS/wrapper" || return 1
|
||||
stat -c %a "$BIN/$WRAPPER_NAME" > "$RS/WRAPPER_MODE" || return 1
|
||||
: > "$RS/UNITS.sha256" || return 1
|
||||
for u in "$BUS_UNIT" "$CONSOLE_UNIT"; do
|
||||
show "$u" > "$RS/show-$u.txt" || return 1
|
||||
frag="$(sc show "$u" -p FragmentPath 2>/dev/null | sed -n 's/^FragmentPath=//p')" && [ -n "$frag" ] || { say "capture: no FragmentPath for $u"; return 1; }
|
||||
sha256sum "$frag" >> "$RS/UNITS.sha256" || return 1
|
||||
done
|
||||
sums "$RS" > "$RS/SHA256SUMS" || return 1
|
||||
say "captured the rollback set: $RS"
|
||||
}
|
||||
|
||||
# A failure after the capture: FAILED holds the code, nothing is logged or cleaned up, exit 1.
|
||||
# Root runs R1-R4, and revert is the one cleanup path (r6 A3).
|
||||
ext_fail() {
|
||||
trap - INT TERM HUP
|
||||
{ echo "$1" > "$RS/FAILED"; } 2>/dev/null || true
|
||||
say "failed: $1"
|
||||
echo "stack-release: $OP failed with $1; $ACT stays open for root's revert and confirm --rollback. Rollback set: $RS" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
# A3 for target $TO. $1 is the store step: migrate or restore. The marker goes in first, so every
|
||||
# failure after it leaves what revert needs. Writes no pointer and starts nothing.
|
||||
run_external() {
|
||||
log_line result started
|
||||
{ mkdir -p -m 0700 "$STATE/stack-rollback" && mkdir -m 0700 "$RS"; } 2>/dev/null || refuse marker-failed "could not make the rollback set $RS; nothing was changed"
|
||||
{ echo "$TO" > "$RS/AWAITING-UNITS"; } 2>/dev/null || { rm -f "$RS/AWAITING-UNITS"; refuse marker-failed "could not write $RS/AWAITING-UNITS; nothing was changed"; }
|
||||
STEPS="$RS/steps.log"
|
||||
capture_units || { rm -f "$RS/AWAITING-UNITS"; refuse capture-failed "could not write the rollback set $RS; nothing was changed"; }
|
||||
trap 'ext_fail signal' INT TERM HUP
|
||||
if [ -e "$STORE" ]; then
|
||||
if store_unclean; then say "$UNCLEAN is present; the store is left as it is for Jason or Mos"; ext_fail store-unclean; fi
|
||||
STORE_SCHEMA="$(store_read version)" && [ -n "$STORE_SCHEMA" ] || ext_fail schema-unreadable
|
||||
say "stopped store schema: $STORE_SCHEMA"
|
||||
if [ "$1" = migrate ]; then
|
||||
if [ "$STORE_SCHEMA" != "$T_SCHEMA" ]; then
|
||||
in_chain "$STORE_SCHEMA" || ext_fail store-newer
|
||||
[ "$MIGRATE" = 1 ] || ext_fail migration-needs-flag
|
||||
fi
|
||||
backup_store || ext_fail backup
|
||||
echo "$BACKUP" > "$RS/BACKUP" || ext_fail backup
|
||||
if [ "$STORE_SCHEMA" != "$T_SCHEMA" ]; then
|
||||
# MIGRATING and TAKEN before the migration, so revert restores a partial one (r6 B2).
|
||||
: > "$RS/MIGRATING" || ext_fail migrate-failed
|
||||
TAKEN="$ACT"
|
||||
node "$COPIES/$TO/packages/bus/src/migrate-schema.mjs" "$DATA" >> "$STEPS" 2>&1 || ext_fail migrate-failed
|
||||
say "migrated the store to $T_SCHEMA"
|
||||
fi
|
||||
else
|
||||
[ "$STORE_SCHEMA" = "$P_SCHEMA" ] || ext_fail store-differs
|
||||
if [ -n "$RESTORE" ]; then
|
||||
local n
|
||||
n="$(store_read count "$(cat "$DATA/backups/stack/$RESTORE/STAMP")" "$COUNTED")" || ext_fail schema-unreadable
|
||||
[ "${n%%$'\t'*}" = "$COUNT" ] || ext_fail post-backup-events
|
||||
echo "$RESTORE" > "$RS/BACKUP" || ext_fail restore
|
||||
# RESTORED tells confirm --rollback the store was swapped, so it skips OLD's gate
|
||||
# and logs rollbackCode not-attempted (ruling 4, T14b).
|
||||
if ! restore_store "$RESTORE"; then
|
||||
[ -z "$RESTORED" ] || : > "$RS/RESTORED" || true
|
||||
ext_fail restore
|
||||
fi
|
||||
: > "$RS/RESTORED" || ext_fail restore
|
||||
fi
|
||||
fi
|
||||
elif [ -n "$RESTORE" ]; then
|
||||
ext_fail store-differs
|
||||
fi
|
||||
scripts/stack-copy.sh wrapper "$TO" "${INSTANCE_FLAGS[@]}" --copies "$COPIES" --bin "$BIN" >> "$STEPS" 2>&1 || ext_fail wrapper
|
||||
trap - INT TERM HUP
|
||||
say "staged $TO"
|
||||
echo "stack-release: $OP staged $TO for $BUSINESS as $ACT; root installs and starts the units, then runs confirm $TO $BUSINESS --units external"
|
||||
}
|
||||
|
||||
# r6 "The open activation": exactly one AWAITING-UNITS file, its activation the one the log holds
|
||||
# open, holding the copy root passed. Sets O_ACT. Anything else refuses open-activation, and ACT
|
||||
# stays unset, so the refused line names no activation and the open one stays open.
|
||||
open_activation() {
|
||||
local -a markers open
|
||||
local m found="" seen=""
|
||||
mapfile -t markers < <(compgen -G "$STATE/stack-rollback/*/AWAITING-UNITS" || true)
|
||||
mapfile -t open < <(interrupted)
|
||||
for m in "${markers[@]}"; do
|
||||
seen+=" $(basename "$(dirname "$m")") (copy $(cat "$m" 2>/dev/null || echo unreadable))"
|
||||
done
|
||||
if [ "${#markers[@]}" = 1 ] && [ "${#open[@]}" = 1 ]; then
|
||||
m="$(basename "$(dirname "${markers[0]}")")"
|
||||
[ "$m" != "${open[0]}" ] || [ "$(cat "${markers[0]}" 2>/dev/null)" != "$ID" ] || found="$m"
|
||||
fi
|
||||
[ -n "$found" ] || refuse open-activation "need one AWAITING-UNITS for copy $ID whose activation the log holds open; markers:${seen:- none}; open: ${open[*]:-none}"
|
||||
O_ACT="$found"
|
||||
}
|
||||
|
||||
# The op, from, to and start second of O_ACT's `started` line, one per line.
|
||||
started_of() {
|
||||
node -e '
|
||||
const [file,act]=process.argv.slice(1);
|
||||
let s;
|
||||
for(const l of require("fs").readFileSync(file,"utf8").split("\n")){
|
||||
if(!l.trim())continue;
|
||||
let o;try{o=JSON.parse(l)}catch{continue}
|
||||
if(o.activation===act&&o.result==="started")s=o;
|
||||
}
|
||||
const at=s&&Math.floor(Date.parse(s.at)/1000);
|
||||
if(!s||!["activate","rollback"].includes(s.op)||!Number.isFinite(at))process.exit(1);
|
||||
process.stdout.write([s.op,s.from??"",s.to??"",at].join("\n")+"\n");' -- "$LOG" "$O_ACT"
|
||||
}
|
||||
|
||||
# A5, and R4 with --rollback. The lines it writes carry the open activation's op, from, to and
|
||||
# activation, and the backup A3 recorded, so from_log reads them as a dragon-lin run's. Every check
|
||||
# runs before ACT is set.
|
||||
do_confirm() {
|
||||
lock_only
|
||||
open_activation
|
||||
local s
|
||||
local -a f
|
||||
s="$(started_of)" || refuse open-activation "the log's started line for $O_ACT is unreadable"
|
||||
mapfile -t f <<< "$s"
|
||||
[ "${f[2]}" = "$ID" ] || refuse open-activation "$O_ACT started for ${f[2]}, not $ID"
|
||||
if read_pointer; then [ "$P_BUSINESS" = "$BUSINESS" ] || refuse business-mismatch "the pointer is for $P_BUSINESS"; fi
|
||||
if [ -n "$ROLLBACK_OLD" ]; then
|
||||
[ "$ROLLBACK_OLD" = "${f[1]}" ] || refuse rollback-mismatch "$ROLLBACK_OLD is not the copy $O_ACT started from, ${f[1]:-none}"
|
||||
else
|
||||
[ ! -e "$STATE/stack-rollback/$O_ACT/FAILED" ] || refuse activation-failed "$O_ACT has a FAILED file; root runs revert and confirm --rollback"
|
||||
target_schema "$ID" || refuse copy-schema-unknown "cannot read SCHEMA_VERSION from $COPIES/$ID/packages/bus/src/store.mjs"
|
||||
fi
|
||||
OP="${f[0]}" FROM="${f[1]}" TO="${f[2]}" FLOOR="${f[3]}" ACT="$O_ACT" RS="$STATE/stack-rollback/$O_ACT"
|
||||
STEPS="$RS/steps.log"
|
||||
BACKUP="$(cat "$RS/BACKUP" 2>/dev/null || true)"
|
||||
if [ -z "$ROLLBACK_OLD" ]; then
|
||||
read_units || ext_fail units-unreadable
|
||||
P_CONSOLE=("${U_CONSOLE[@]}")
|
||||
gate "$TO" "$U_PM" || ext_fail "$GATE"
|
||||
write_pointer "$TO" "$U_PM" 0 "$U_CLAUDE" "$T_SCHEMA" "${U_CONSOLE[@]}"
|
||||
log_line result ok
|
||||
rm -f "$RS/AWAITING-UNITS"
|
||||
echo "stack-release: $OP ok: $TO is live for $BUSINESS"
|
||||
return
|
||||
fi
|
||||
local code rcode=""
|
||||
if [ "$CAUSE" = A4 ]; then code=units-start
|
||||
else code="$(head -n 1 "$RS/FAILED" 2>/dev/null || true)"; [ -n "$code" ] || code=unrecorded
|
||||
fi
|
||||
if [ -e "$RS/RESTORED" ]; then rcode=not-attempted
|
||||
elif ! read_units; then rcode=units-unreadable
|
||||
else
|
||||
P_CONSOLE=("${U_CONSOLE[@]}")
|
||||
gate "$ROLLBACK_OLD" "$U_PM" || rcode="$GATE"
|
||||
fi
|
||||
if [ -z "$rcode" ]; then
|
||||
local r=failed
|
||||
[ "$CAUSE" != A5 ] || r=gate-failed
|
||||
log_line result "$r" code "$code" rollback ok
|
||||
rm -f "$RS/AWAITING-UNITS" "$RS/FAILED"
|
||||
echo "stack-release: $code; rolled back to $ROLLBACK_OLD, which passed the gate" >&2
|
||||
exit 1
|
||||
fi
|
||||
say "rollback-failed: run $code, rollback $rcode"
|
||||
log_line result rollback-failed code "$code" rollback failed rollbackCode "$rcode"
|
||||
rm -f "$RS/AWAITING-UNITS" "$RS/FAILED"
|
||||
echo "stack-release: rollback-failed: the run failed with $code and the rollback with $rcode; root stops the units. Rollback set: $RS${BACKUP:+, backup $DATA/backups/stack/$BACKUP}" >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
revert_fail() {
|
||||
say "revert failed: $1"
|
||||
echo "stack-release: revert failed: $1; $O_ACT stays open. Rollback set: $RS" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
# The store already holds backup $1 when every file checks against its SHA256SUMS and the listing
|
||||
# matches, as after a revert that restored it.
|
||||
store_matches() {
|
||||
local o="$DATA/backups/stack/$1"
|
||||
[ -d "$DATA/bus" ] && (cd "$DATA/bus" && sha256sum -c --quiet "$o/SHA256SUMS") >/dev/null 2>&1 &&
|
||||
[ "$(ls -1A "$DATA/bus")" = "$(ls -1A "$o/bus")" ]
|
||||
}
|
||||
|
||||
# R2: puts the wrapper back and, after a migration, the backup taken before it. Each part can run
|
||||
# again, so it serves an A3 that stopped partway. A rollback's --restore swap is confirm's to
|
||||
# report, not revert's to undo. Writes no log line.
|
||||
do_revert() {
|
||||
lock_only
|
||||
open_activation
|
||||
RS="$STATE/stack-rollback/$O_ACT" STEPS="$STATE/stack-rollback/$O_ACT/steps.log"
|
||||
[[ "$O_ACT" =~ ^([0-9]{8}T[0-9]{6}Z)-[0-9a-f]{12}(-[1-9])?$ ]] || revert_fail "$O_ACT is not an activation id"
|
||||
POST="$DATA/bus.post-attempt-${BASH_REMATCH[1]}${BASH_REMATCH[2]}"
|
||||
[ -f "$RS/wrapper" ] && [ -f "$RS/WRAPPER_MODE" ] || revert_fail "the rollback set has no saved wrapper"
|
||||
local tmp
|
||||
tmp="$(mktemp "$BIN/.$WRAPPER_NAME.XXXXXX")" && cp -p "$RS/wrapper" "$tmp" && chmod "$(cat "$RS/WRAPPER_MODE")" "$tmp" && mv -f "$tmp" "$BIN/$WRAPPER_NAME" ||
|
||||
revert_fail "could not put the wrapper back"
|
||||
[ "$(sha256sum < "$BIN/$WRAPPER_NAME" | cut -d' ' -f1)" = "$(sum_of "$RS" wrapper)" ] &&
|
||||
[ "$(stat -c %a "$BIN/$WRAPPER_NAME")" = "$(cat "$RS/WRAPPER_MODE")" ] || revert_fail "the wrapper does not match the rollback set"
|
||||
say "put back the wrapper from $RS"
|
||||
if [ -e "$RS/MIGRATING" ]; then
|
||||
TAKEN="$(cat "$RS/BACKUP" 2>/dev/null || true)"
|
||||
[ -n "$TAKEN" ] || revert_fail "MIGRATING is set and BACKUP names no backup"
|
||||
if store_matches "$TAKEN"; then say "the store already matches backup $TAKEN"
|
||||
else restore_store "$TAKEN" || revert_fail "could not restore backup $TAKEN"
|
||||
fi
|
||||
fi
|
||||
echo "stack-release: revert ok: $O_ACT's wrapper$([ -e "$RS/MIGRATING" ] && echo ' and store are' || echo ' is') back; root installs the old units next"
|
||||
}
|
||||
|
||||
# A2f: gates copy ID with no open activation, on a start at or after --since. Takes no lock and
|
||||
# writes nothing, so it refuses nothing: 0 on a pass, 1 otherwise.
|
||||
do_gate() {
|
||||
FLOOR="$SINCE"
|
||||
read_units || { echo "stack-release: gate: cannot read $BUS_UNIT or $CONSOLE_UNIT" >&2; exit 1; }
|
||||
P_CONSOLE=("${U_CONSOLE[@]}")
|
||||
gate "$ID" "$U_PM" || exit 1
|
||||
}
|
||||
|
||||
# --- pm-on (row 169 item 3) ---
|
||||
|
||||
# Stops the bus alone and waits for it and the stopped check, within --timeout. The Console runs on.
|
||||
stop_bus() {
|
||||
systemctl --user stop "$BUS_UNIT" || return 1
|
||||
sc stop "$BUS_UNIT" || return 1
|
||||
local deadline=$((SECONDS + TIMEOUT)) s
|
||||
while :; do
|
||||
s="$(systemctl --user is-active "$BUS_UNIT" 2>/dev/null || true)"
|
||||
s="$(sc is-active "$BUS_UNIT" 2>/dev/null || true)"
|
||||
case "$s" in inactive|failed) if stopped; then say "stopped $BUS_UNIT"; return 0; fi ;; esac
|
||||
[ "$SECONDS" -lt "$deadline" ] || return 1
|
||||
sleep 1
|
||||
@@ -1326,7 +1721,7 @@ stop_bus() {
|
||||
|
||||
start_bus() {
|
||||
START_EPOCH="$(date +%s)" && START_MS="$(date +%s%3N)" || return 1
|
||||
systemctl --user start "$BUS_UNIT"
|
||||
sc start "$BUS_UNIT"
|
||||
}
|
||||
|
||||
# mos-0848 point 3: put the phase-1 bus unit back (no --pm), restart it and rerun the phase-1 gate.
|
||||
@@ -1348,8 +1743,8 @@ pm_on_fail() {
|
||||
exit 1
|
||||
fi
|
||||
local pwhy="$GATE_WHY"
|
||||
systemctl --user stop "$CONSOLE_UNIT" || true
|
||||
systemctl --user stop "$BUS_UNIT" || true
|
||||
sc stop "$CONSOLE_UNIT" || true
|
||||
sc stop "$BUS_UNIT" || true
|
||||
say "pm-on-down: pm-on $code, phase 1 $pcode"
|
||||
log_line result pm-on-down code "$code" why "${why:-@null}" phase1 "$pcode" phase1Why "${pwhy:-@null}"
|
||||
echo "stack-release: pm-on-down: pm-on failed with $code${why:+ ($why)} and the phase-1 bus with $pcode${pwhy:+ ($pwhy)}; both units are stopped. Rollback set: $RS" >&2
|
||||
@@ -1359,6 +1754,7 @@ pm_on_fail() {
|
||||
# Phase 2: the bus unit alone, on the pointer's copy with --pm, gated with pm_launched.
|
||||
do_pm_on() {
|
||||
lock
|
||||
[ "$UNITS" = user ] || refuse external-pm "under --units external the PM is a flag in root's bus unit; changing it is a re-install with its own go"
|
||||
pointer_for_business
|
||||
[ "$P_PENDING" = 1 ] || refuse not-pending "the pointer is not pending; pm-on follows activate --pm-after-harness"
|
||||
local out
|
||||
@@ -1395,8 +1791,16 @@ do_status() {
|
||||
local open
|
||||
open="$(interrupted | paste -sd' ' -)"
|
||||
[ -z "$open" ] || echo "interrupted run: $open (started, no final line; Jason or Mos settles it)"
|
||||
if [ "$UNITS" = external ]; then
|
||||
local m
|
||||
local -a markers
|
||||
mapfile -t markers < <(compgen -G "$STATE/stack-rollback/*/AWAITING-UNITS" || true)
|
||||
for m in "${markers[@]}"; do
|
||||
echo "awaiting units: $(basename "$(dirname "$m")") (copy $(cat "$m" 2>/dev/null || echo unreadable))"
|
||||
done
|
||||
fi
|
||||
echo "check:"
|
||||
if scripts/stack-copy.sh check "${INSTANCE_FLAGS[@]}" --copies "$COPIES" --bin "$BIN" --business "$BUSINESS" 2>&1 | sed 's/^/ /'; then echo " result: pass"; else echo " result: FAIL"; fi
|
||||
if scripts/stack-copy.sh check "${INSTANCE_FLAGS[@]}" "${UNITS_FLAGS[@]}" --copies "$COPIES" --bin "$BIN" --business "$BUSINESS" 2>&1 | sed 's/^/ /'; then echo " result: pass"; else echo " result: FAIL"; fi
|
||||
[ -n "$P_COPY" ] || return 0
|
||||
echo "bus status:"
|
||||
"$COPIES/$P_COPY/scripts/mosaic" bus status 2>&1 | sed 's/^/ /' || true
|
||||
@@ -1580,6 +1984,9 @@ r_env() { env -i HOME="$HOME" PATH="$PATH" MOSAIC_CONFIG="$R/config/config.json"
|
||||
r_migration() {
|
||||
local copy="$COPIES/$TO" old="$COPIES/$P_COPY" v out rc
|
||||
v="$(node -e "$STORE_JS" -- version "$R/data/bus/bus.sqlite" 2>&1)" || r_abort store-unreadable "$v"
|
||||
# Row 233 (sage-1653): which store the rehearsal ran against, for root's step log. Output only:
|
||||
# the rehearsal's own steps.log has the backup's path on its backup line.
|
||||
echo "BACKUP_STORE $R_BACKUP schema $v"
|
||||
if [ "$v" = "$T_SCHEMA" ]; then r_pass migration "not needed: the backup is already $v"; return 0; fi
|
||||
in_chain "$v" || r_abort store-newer "the backup is $v, which $TO's migration chain does not start from"
|
||||
(umask 077; mkdir "$R/refuse" "$R/refuse/bus" && cp "$R_BK"/bus/* "$R/refuse/bus/") || r_abort scratch-build "cannot copy the backup to $R/refuse"
|
||||
@@ -1650,6 +2057,13 @@ do_rehearse() {
|
||||
trap 'r_fail signal "the rehearsal was interrupted"; r_end' INT TERM HUP
|
||||
read_pointer || r_abort no-pointer "no $POINTER; the pointer's copy is the old copy the checks use"
|
||||
[ "$P_BUSINESS" = "$BUSINESS" ] || r_abort business-mismatch "the pointer is for $P_BUSINESS"
|
||||
# Row 233 (sage-1650, sage-1653): last is the newest backup the log records. With none, exit 3
|
||||
# and no directory, the one rehearse refusal stack-upgrade.sh reads as no-backup.
|
||||
if [ "$R_BACKUP" = last ]; then
|
||||
R_BACKUP="$(from_log newest)" || { r_fail no-backup "the log records no backup"; R_RC=3; r_end; }
|
||||
[[ "$R_BACKUP" =~ ^[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])?$ ]] || r_abort backup-unreadable "the log's newest backup is not an activation id"
|
||||
r_pass backup-last "the log's newest backup is $R_BACKUP"
|
||||
fi
|
||||
copy_exists "$TO" 2>/dev/null || r_abort copy-missing "no copy $TO in $COPIES"
|
||||
copy_ok "$TO" 2>/dev/null || r_abort copy-unreachable "$TO is not reachable from refactor and origin/refactor"
|
||||
target_schema "$TO" || r_abort copy-schema-unknown "cannot read SCHEMA_VERSION and its chain from $COPIES/$TO/packages/bus/src/store.mjs"
|
||||
@@ -1722,11 +2136,16 @@ ID=""
|
||||
case "$OP" in
|
||||
activate|rehearse) [ $# -ge 2 ] || usage; ID="$1"; BUSINESS="$2"; shift 2; [[ "$ID" =~ ^[0-9a-f]{40}$ ]] || usage ;;
|
||||
adopt|pm-on|rollback|status) [ $# -ge 1 ] || usage; BUSINESS="$1"; shift ;;
|
||||
confirm|revert|gate) [ $# -ge 2 ] || usage; ID="$1"; BUSINESS="$2"; shift 2; [[ "$ID" =~ ^[0-9a-f]{40}$ ]] || usage ;;
|
||||
-h|--help) usage 0 ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
[[ "$BUSINESS" =~ ^[a-z0-9][a-z0-9-]*$ ]] || usage
|
||||
parse_opts "$@"
|
||||
# Row 232: the stack-copy.sh check flags and the systemctl the instance-busy message names.
|
||||
UNITS_FLAGS=()
|
||||
SC_NAME="systemctl --user"
|
||||
if [ "$UNITS" = external ]; then UNITS_FLAGS=(--units external); SC_NAME=systemctl; fi
|
||||
COPIES_DIR="$COPIES"
|
||||
instance_select
|
||||
load_config
|
||||
@@ -1741,6 +2160,7 @@ STORE_SCHEMA="" START_EPOCH="" START_MS=0 GATE="" GATE_WHY="" S_CHANGED=""
|
||||
P_COPY="" P_PM=0 P_PENDING=0 P_SCHEMA="" P_CLAUDE="" P_CONSOLE=() RUN_PM=0 RUN_PENDING=0 RUN_CONSOLE=() DROP_TESTER="" PROOF_PLAN="" PROOF_STEP="" SEND_PLAN="" SEND_STEP="" PLAN_REFUSE=()
|
||||
BASE_CONSOLE=() SERVE_FLAGS=() SSO_PLAN="" SSO_STEP="" SSO_DROP="" SSO_NO_TARGET=""
|
||||
R="" R_RC=0 R_EARLY=() R_BK="" R_HOST_PID="" R_CONSOLE_PID="" R_STARTED="" LIVE_CFG="" LIVE_BIZ=""
|
||||
O_ACT="" FLOOR=0
|
||||
case "$OP" in
|
||||
adopt) do_adopt ;;
|
||||
activate) do_activate ;;
|
||||
@@ -1748,4 +2168,7 @@ case "$OP" in
|
||||
rollback) do_rollback ;;
|
||||
status) do_status ;;
|
||||
rehearse) do_rehearse ;;
|
||||
confirm) do_confirm ;;
|
||||
revert) do_revert ;;
|
||||
gate) do_gate ;;
|
||||
esac
|
||||
|
||||
Executable
+347
@@ -0,0 +1,347 @@
|
||||
#!/usr/bin/env bash
|
||||
# Row 233 (I3, #1735): root's upgrade and rollback for a stack whose units are root's system units
|
||||
# (stack-release.sh --units external, row 232). r6's activate A1-A5, A2f, R1-R4 and rollback
|
||||
# (agents/darkwing/work/queue-195/DESIGN.md), as the I3 brief changes them
|
||||
# (docs/plans/2026-10-20_installer-rows.md). Deviations and decisions:
|
||||
# agents/darkwing/work/queue-233/BUILD.md.
|
||||
#
|
||||
# stack-upgrade.sh upgrade --instance dev --bundle FILE --sha SHA [--migrate] [--no-rehearsal]
|
||||
# [--rehearse-port N] [--timeout S]
|
||||
# stack-upgrade.sh rollback --instance dev --to ID [--restore ACT [--events N]] [--timeout S]
|
||||
#
|
||||
# Root runs it from a root-owned checkout of the reviewed commit, never from a copy, because the
|
||||
# stack's user can write the copies. The unit values come from /etc/mosaic-stack/<instance>.args,
|
||||
# which I2's install writes: one case2-unit.sh argument per line, the serve flags after `--`. Only
|
||||
# --copy changes, and the file changes only after confirm passes. The stack's user runs its steps
|
||||
# from its own clone, <home>/src/mosaic-stack, through sudo. Root reads none of that user's output
|
||||
# but exit codes and the rehearsal's BACKUP_STORE line (sage-1653), which it copies into its step
|
||||
# log after a match against a fixed pattern and decides nothing on.
|
||||
#
|
||||
# --no-rehearsal is for a stack whose log records no backup. The rehearsal runs either way: exit 3
|
||||
# from it (no-backup) goes on only with the flag, and exit 0 with the flag refuses
|
||||
# rehearsal-required (sage-1650, sage-1652).
|
||||
#
|
||||
# The lock /run/mosaic-stack/upgrade-<instance>.lock is held from before A1 to the end of the run,
|
||||
# rollback included; every child runs with the lock's descriptor closed. The step log
|
||||
# /var/log/mosaic-stack/upgrade-<instance>.log gets one line per step, `<time> <step> rc=N argv:
|
||||
# <argv>`, and a last `<time> result exit=N <code> <text>` line.
|
||||
#
|
||||
# Exit 0 ok, 1 failed with the old stack running, 2 rollback-failed with the units stopped,
|
||||
# 3 refused with nothing stopped, 4 usage.
|
||||
#
|
||||
# Test seams, for packages/cli/tests/stack-upgrade.test.mjs only. Run as root, it refuses any:
|
||||
# STACK_UPGRADE_ETC default /etc/mosaic-stack
|
||||
# STACK_UPGRADE_RUN default /run/mosaic-stack
|
||||
# STACK_UPGRADE_LOG_DIR default /var/log/mosaic-stack
|
||||
# STACK_UPGRADE_UNIT_DIR default /etc/systemd/system
|
||||
# STACK_UPGRADE_HOSTCHECK default /usr/local/lib/mosaic-stack/hostcheck.sh (row 240)
|
||||
# STACK_UPGRADE_RENDER default scripts/case2-unit.sh in this checkout
|
||||
# STACK_UPGRADE_SUDO default sudo
|
||||
# STACK_UPGRADE_SYSTEMCTL default systemctl
|
||||
set -uo pipefail
|
||||
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
|
||||
usage() {
|
||||
cat >&2 <<'USAGE'
|
||||
usage: stack-upgrade.sh upgrade --instance dev --bundle FILE --sha SHA [--migrate] [--no-rehearsal] [--rehearse-port N] [--timeout S]
|
||||
stack-upgrade.sh rollback --instance dev --to ID [--restore ACT [--events N]] [--timeout S]
|
||||
USAGE
|
||||
exit 4
|
||||
}
|
||||
|
||||
if [ "$(id -u)" = 0 ]; then
|
||||
for v in ${!STACK_UPGRADE_@}; do echo "stack-upgrade: refusing: $v is a test seam, and root takes none" >&2; exit 3; done
|
||||
fi
|
||||
ETC="${STACK_UPGRADE_ETC:-/etc/mosaic-stack}"
|
||||
RUN_DIR="${STACK_UPGRADE_RUN:-/run/mosaic-stack}"
|
||||
LOG_DIR="${STACK_UPGRADE_LOG_DIR:-/var/log/mosaic-stack}"
|
||||
UNIT_DIR="${STACK_UPGRADE_UNIT_DIR:-/etc/systemd/system}"
|
||||
HOSTCHECK="${STACK_UPGRADE_HOSTCHECK:-/usr/local/lib/mosaic-stack/hostcheck.sh}"
|
||||
RENDER="${STACK_UPGRADE_RENDER:-$REPO/scripts/case2-unit.sh}"
|
||||
SUDO="${STACK_UPGRADE_SUDO:-sudo}"
|
||||
SYSTEMCTL="${STACK_UPGRADE_SYSTEMCTL:-systemctl}"
|
||||
ID_RE='^[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])?$'
|
||||
|
||||
ARGV=("$@")
|
||||
[ $# -gt 0 ] || usage
|
||||
OP="$1"; shift
|
||||
case "$OP" in upgrade|rollback) ;; *) usage ;; esac
|
||||
INSTANCE="" BUNDLE="" SHA="" MIGRATE=0 NO_REHEARSAL=0 R_PORT=7339 TIMEOUT=60 TO="" RESTORE="" EVENTS=""
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--instance) [ $# -ge 2 ] || usage; INSTANCE="$2"; shift 2 ;;
|
||||
--bundle) [ "$OP" = upgrade ] && [ $# -ge 2 ] && [[ "$2" =~ ^/[A-Za-z0-9/._+-]+$ ]] || usage; BUNDLE="$2"; shift 2 ;;
|
||||
--sha) [ "$OP" = upgrade ] && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9a-f]{40}$ ]] || usage; SHA="$2"; shift 2 ;;
|
||||
--migrate) [ "$OP" = upgrade ] || usage; MIGRATE=1; shift ;;
|
||||
--no-rehearsal) [ "$OP" = upgrade ] || usage; NO_REHEARSAL=1; shift ;;
|
||||
--rehearse-port) [ "$OP" = upgrade ] && [ $# -ge 2 ] && [[ "$2" =~ ^[1-9][0-9]{3,4}$ ]] && [ "$2" -le 65535 ] || usage; R_PORT="$2"; shift 2 ;;
|
||||
--timeout) [ $# -ge 2 ] && [[ "$2" =~ ^[1-9][0-9]{0,4}$ ]] || usage; TIMEOUT="$2"; shift 2 ;;
|
||||
--to) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9a-f]{40}$ ]] || usage; TO="$2"; shift 2 ;;
|
||||
--restore) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ $ID_RE ]] || usage; RESTORE="$2"; shift 2 ;;
|
||||
--events) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ ^(0|[1-9][0-9]*)$ ]] || usage; EVENTS="$2"; shift 2 ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
done
|
||||
# The default instance only, whose name is dev (row 190); a named instance's units are a later row.
|
||||
[ "$INSTANCE" = dev ] || usage
|
||||
if [ "$OP" = upgrade ]; then [ -n "$BUNDLE" ] && [ -n "$SHA" ] || usage; else [ -n "$TO" ] || usage; fi
|
||||
[ -z "$EVENTS" ] || [ -n "$RESTORE" ] || usage
|
||||
|
||||
# --- The step log and the lock ---
|
||||
|
||||
umask 027
|
||||
LOG="$LOG_DIR/upgrade-$INSTANCE.log"
|
||||
LOCK="$RUN_DIR/upgrade-$INSTANCE.lock"
|
||||
{ mkdir -p "$LOG_DIR" "$RUN_DIR" && touch "$LOG" && chmod 0640 "$LOG"; } 2>/dev/null ||
|
||||
{ echo "stack-upgrade: refusing log-failed: cannot write $LOG or make $RUN_DIR; nothing was changed" >&2; exit 3; }
|
||||
logl() { printf '%s %s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$*" >> "$LOG" || true; }
|
||||
# The run's last line, in the log and on the terminal, then the exit.
|
||||
finish() {
|
||||
local rc="$1" code="$2"; shift 2
|
||||
logl "result exit=$rc $code $*"
|
||||
if [ "$rc" = 0 ]; then echo "stack-upgrade: $*"; else echo "stack-upgrade: $code: $*" >&2; fi
|
||||
exit "$rc"
|
||||
}
|
||||
refuse() { local code="$1"; shift; finish 3 "$code" "$* (nothing was stopped)"; }
|
||||
# One step: $1 its name, the rest the command, run with the lock's descriptor closed. One log line
|
||||
# with its exit and argv; the command's output goes to the terminal only.
|
||||
step() {
|
||||
local name="$1" rc=0; shift
|
||||
"$@" 9>&- || rc=$?
|
||||
logl "$name rc=$rc argv:$(printf ' %q' "$@")"
|
||||
return "$rc"
|
||||
}
|
||||
|
||||
exec 9>> "$LOCK" || refuse lock-failed "cannot open $LOCK"
|
||||
flock -n 9 || refuse upgrade-running "another stack-upgrade.sh run holds $LOCK"
|
||||
logl "start rc=0 argv:$(printf ' %q' "$0" "${ARGV[@]}")"
|
||||
|
||||
# --- The args file and the stack's user ---
|
||||
|
||||
# I2's file: one case2-unit.sh argument per line, owned by root with no group or other write. Sets
|
||||
# ARGS (the lines), S_USER, S_HOME, BUSINESS, NODE and ARGS_COPY, and the paths derived from them.
|
||||
read_args() {
|
||||
local f="$ETC/$INSTANCE.args" st i copies=0 v
|
||||
[ -f "$f" ] && [ ! -L "$f" ] || refuse args-unreadable "no regular file $f"
|
||||
st="$(stat -c '%u %a' "$f")" || refuse args-unreadable "cannot stat $f"
|
||||
[ "${st% *}" = "$(id -u)" ] && (( (8#${st#* } & 8#022) == 0 )) || refuse args-unreadable "$f must be $(id -un)'s with no group or other write (it is uid ${st% *}, mode ${st#* })"
|
||||
mapfile -t ARGS < "$f" || refuse args-unreadable "cannot read $f"
|
||||
S_USER="" S_HOME="" BUSINESS="" NODE="" ARGS_COPY=""
|
||||
for ((i = 0; i < ${#ARGS[@]}; i++)); do
|
||||
case "${ARGS[i]}" in
|
||||
--) break ;;
|
||||
--pm) ;;
|
||||
--user|--home|--business|--copy|--node|--profile|--claude)
|
||||
v="${ARGS[i + 1]:-}"
|
||||
case "${ARGS[i]}" in
|
||||
--user) S_USER="$v" ;; --home) S_HOME="$v" ;; --business) BUSINESS="$v" ;; --node) NODE="$v" ;;
|
||||
--copy) ARGS_COPY="$v"; copies=$((copies + 1)) ;;
|
||||
esac
|
||||
i=$((i + 1)) ;;
|
||||
*) refuse args-unreadable "$f has ${ARGS[i]} before --, which case2-unit.sh does not take" ;;
|
||||
esac
|
||||
done
|
||||
[ "$copies" = 1 ] || refuse args-unreadable "$f needs exactly one --copy before --, not $copies"
|
||||
[[ "$S_USER" =~ ^[a-z_][a-z0-9_-]*$ ]] && [[ "$S_HOME" =~ ^/[A-Za-z0-9/._+-]+$ ]] && [[ "$BUSINESS" =~ ^[a-z0-9][a-z0-9-]*$ ]] &&
|
||||
[[ "$NODE" =~ ^/[A-Za-z0-9/._+-]+$ ]] && [[ "$ARGS_COPY" =~ ^[0-9a-f]{40}$ ]] ||
|
||||
refuse args-unreadable "$f lacks a plain --user, --home, --business, --node or --copy"
|
||||
COPIES_DIR="$S_HOME/.local/share/mosaic-stack/copies"
|
||||
CLONE="$S_HOME/src/mosaic-stack"
|
||||
SR="$CLONE/scripts/stack-release.sh"
|
||||
BUS_UNIT="mosaic-bus@$BUSINESS.service" CON_UNIT=mosaic-webui.service
|
||||
# The stack's user's steps: its own HOME and default config, the pinned node first on PATH.
|
||||
AS_USER=("$SUDO" -u "$S_USER" -H env "PATH=$(dirname "$NODE"):/usr/local/bin:/usr/bin:/bin")
|
||||
}
|
||||
|
||||
# RENDER_ARGS: the args file's lines with copy $1 in place of --copy's value.
|
||||
args_with() {
|
||||
local i past=""
|
||||
RENDER_ARGS=()
|
||||
for ((i = 0; i < ${#ARGS[@]}; i++)); do
|
||||
RENDER_ARGS+=("${ARGS[i]}")
|
||||
[ "${ARGS[i]}" != -- ] || past=1
|
||||
if [ -z "$past" ] && [ "${ARGS[i]}" = --copy ]; then RENDER_ARGS+=("$1"); i=$((i + 1)); fi
|
||||
done
|
||||
}
|
||||
|
||||
# The new args file, written beside the old one before anything stops, moved into place only after
|
||||
# confirm passes. $1 is the copy.
|
||||
prepare_args() {
|
||||
args_with "$1"
|
||||
NEW_ARGS="$(mktemp "$ETC/.$INSTANCE.args.XXXXXX")" &&
|
||||
printf '%s\n' "${RENDER_ARGS[@]}" > "$NEW_ARGS" && chmod --reference="$ETC/$INSTANCE.args" "$NEW_ARGS" ||
|
||||
{ rm -f "${NEW_ARGS:-}"; refuse args-failed "cannot write the new args file in $ETC"; }
|
||||
}
|
||||
drop_args() { rm -f "${NEW_ARGS:-}"; }
|
||||
|
||||
# --- A1 ---
|
||||
|
||||
# OLD: the copy both installed units run, from their WorkingDirectory under the stack's copies.
|
||||
read_old() {
|
||||
local f ids=() id n
|
||||
for f in "$UNIT_DIR/[email protected]" "$UNIT_DIR/$CON_UNIT"; do
|
||||
[ -f "$f" ] && [ ! -L "$f" ] || refuse units-unreadable "no regular file $f"
|
||||
n="$(grep -c '^WorkingDirectory=' "$f")"
|
||||
id="$(awk -v c="WorkingDirectory=$COPIES_DIR/" 'index($0, c) == 1 { print substr($0, length(c) + 1) }' "$f")"
|
||||
[ "$n" = 1 ] && [[ "$id" =~ ^[0-9a-f]{40}$ ]] || refuse units-unreadable "$f names no copy under $COPIES_DIR"
|
||||
ids+=("$id")
|
||||
done
|
||||
[ "${ids[0]}" = "${ids[1]}" ] || refuse units-disagree "$UNIT_DIR's units name ${ids[0]} and ${ids[1]}"
|
||||
OLD="${ids[0]}"
|
||||
logl "units rc=0 old: $OLD"
|
||||
[ "$ARGS_COPY" = "$OLD" ] || refuse args-differ "$ETC/$INSTANCE.args names $ARGS_COPY and the units name $OLD"
|
||||
}
|
||||
|
||||
# r6 A1 and row 240: the host's profile still matches, on every host.
|
||||
hostcheck() {
|
||||
[ -f "$HOSTCHECK" ] && [ -x "$HOSTCHECK" ] || refuse profile-drift "no executable $HOSTCHECK (row 240)"
|
||||
step hostcheck "$HOSTCHECK" --check || refuse profile-drift "$HOSTCHECK --check failed"
|
||||
}
|
||||
|
||||
# The stack's user's --plan for the run, $@ its op and arguments; any refusal stops here.
|
||||
plan() {
|
||||
local rc=0
|
||||
step plan "${AS_USER[@]}" "$SR" "$@" --units external --copies "$COPIES_DIR" --plan --expect-from "$OLD" || rc=$?
|
||||
[ "$rc" = 0 ] || refuse plan-refused "the stack's user's $1 --plan exited $rc; its output above names the refusal"
|
||||
}
|
||||
|
||||
# --- The units ---
|
||||
|
||||
stop_wait() {
|
||||
local u s deadline=$((SECONDS + TIMEOUT)) rc=0
|
||||
step stop-console "$SYSTEMCTL" stop "$CON_UNIT" || rc=1
|
||||
step stop-bus "$SYSTEMCTL" stop "$BUS_UNIT" || rc=1
|
||||
for u in "$CON_UNIT" "$BUS_UNIT"; do
|
||||
while :; do
|
||||
s="$("$SYSTEMCTL" is-active "$u" 9>&- 2>/dev/null)"
|
||||
case "$s" in inactive|failed) break ;; esac
|
||||
[ "$SECONDS" -lt "$deadline" ] || { rc=1; break; }
|
||||
sleep 1
|
||||
done
|
||||
case "$s" in inactive|failed) logl "wait-down rc=0 unit: $u state: $s" ;; *) logl "wait-down rc=1 unit: $u state: ${s:-unknown}" ;; esac
|
||||
done
|
||||
return "$rc"
|
||||
}
|
||||
stop_units() { "$SYSTEMCTL" stop "$CON_UNIT" 9>&- || true; "$SYSTEMCTL" stop "$BUS_UNIT" 9>&- || true; logl "stop-units rc=0 units: $CON_UNIT $BUS_UNIT"; }
|
||||
|
||||
# A2f: the old units back on the copy they never left, gated on a start at or after this second.
|
||||
# An A2 that timed out leaves a unit on its old start, which the gate refuses as stale-start.
|
||||
a2f() {
|
||||
local cause="$1" since
|
||||
drop_args
|
||||
since="$(date -u +%s)"
|
||||
step start-bus "$SYSTEMCTL" start "$BUS_UNIT"
|
||||
step start-console "$SYSTEMCTL" start "$CON_UNIT"
|
||||
if step gate "${AS_USER[@]}" "$SR" gate "$OLD" "$BUSINESS" --units external --copies "$COPIES_DIR" --since "$since" --timeout "$TIMEOUT"; then
|
||||
finish 1 failed "$cause failed; $OLD is running again and passed the gate"
|
||||
fi
|
||||
stop_units
|
||||
finish 2 rollback-failed "$cause failed and $OLD failed its gate after the restart; both units are stopped"
|
||||
}
|
||||
|
||||
# R1-R4: back to OLD after the stack's user staged $1. $2 is the cause: A3, A4 or A5.
|
||||
rollback_to_old() {
|
||||
local id="$1" cause="$2"
|
||||
drop_args
|
||||
stop_wait || { stop_units; finish 2 rollback-failed "$cause failed and the units did not stop for the rollback; both units are stopped"; }
|
||||
step revert "${AS_USER[@]}" "$SR" revert "$id" "$BUSINESS" --units external --copies "$COPIES_DIR" ||
|
||||
{ stop_units; finish 2 rollback-failed "$cause failed and revert failed; both units are stopped"; }
|
||||
args_with "$OLD"
|
||||
step render-old "$RENDER" "${RENDER_ARGS[@]}" ||
|
||||
{ stop_units; finish 2 rollback-failed "$cause failed and the render of $OLD failed; both units are stopped"; }
|
||||
# confirm --rollback exits 1 when OLD passed its gate and 2 when it failed; anything else is a
|
||||
# refusal, with OLD's gate never run.
|
||||
local rc=0
|
||||
step confirm-rollback "${AS_USER[@]}" "$SR" confirm "$id" "$BUSINESS" --units external --copies "$COPIES_DIR" --rollback "$OLD" --cause "$cause" --timeout "$TIMEOUT" || rc=$?
|
||||
[ "$rc" != 1 ] || finish 1 failed "$cause failed; rolled back to $OLD, which passed the gate"
|
||||
stop_units
|
||||
finish 2 rollback-failed "$cause failed and the rollback to $OLD did not pass (confirm --rollback exited $rc); both units are stopped"
|
||||
}
|
||||
|
||||
# A2 to A5 for copy $1, once A1 and any rehearsal passed. $@ after it is the stack's user's op.
|
||||
run_steps() {
|
||||
local id="$1" rc=0; shift
|
||||
prepare_args "$id"
|
||||
# A2
|
||||
stop_wait || a2f A2
|
||||
# A3
|
||||
step stage "${AS_USER[@]}" "$SR" "$@" --units external --copies "$COPIES_DIR" || rc=$?
|
||||
case "$rc" in
|
||||
0) ;;
|
||||
3) a2f A3 ;;
|
||||
*) rollback_to_old "$id" A3 ;;
|
||||
esac
|
||||
# A4
|
||||
args_with "$id"
|
||||
step render "$RENDER" "${RENDER_ARGS[@]}" || rollback_to_old "$id" A4
|
||||
# A5
|
||||
step confirm "${AS_USER[@]}" "$SR" confirm "$id" "$BUSINESS" --units external --copies "$COPIES_DIR" --timeout "$TIMEOUT" ||
|
||||
rollback_to_old "$id" A5
|
||||
if mv -f "$NEW_ARGS" "$ETC/$INSTANCE.args"; then logl "args rc=0 copy: $id"
|
||||
else logl "args rc=1 copy: $id"; finish 1 args-failed "$id is live and confirmed, but $ETC/$INSTANCE.args still names $OLD; put $NEW_ARGS in its place"
|
||||
fi
|
||||
}
|
||||
|
||||
do_upgrade() {
|
||||
read_args
|
||||
# Before A1: the bundle into the stack's user's clone, then the copy from it (I1).
|
||||
step fetch "${AS_USER[@]}" git -C "$CLONE" fetch -q "$BUNDLE" refs/heads/refactor:refs/remotes/origin/refactor ||
|
||||
refuse fetch-failed "the stack's user could not fetch $BUNDLE into $CLONE"
|
||||
step ff "${AS_USER[@]}" git -C "$CLONE" merge -q --ff-only "$SHA" ||
|
||||
refuse fetch-failed "the stack's user could not fast-forward $CLONE's refactor to $SHA"
|
||||
step copy "${AS_USER[@]}" "$CLONE/scripts/stack-copy.sh" copy --bundle "$BUNDLE" "$SHA" --copies "$COPIES_DIR" ||
|
||||
refuse copy-failed "the stack's user could not make copy $SHA from $BUNDLE"
|
||||
# A1
|
||||
local head
|
||||
head="$(git -C "$REPO" rev-parse HEAD 2>/dev/null)"
|
||||
[ "$head" = "$SHA" ] || refuse head-mismatch "this checkout, $REPO, is at ${head:-unknown}, not $SHA"
|
||||
[ -x "$COPIES_DIR/$SHA/scripts/mosaic" ] || refuse copy-missing "no $COPIES_DIR/$SHA/scripts/mosaic"
|
||||
hostcheck
|
||||
read_old
|
||||
local -a migrate=()
|
||||
[ "$MIGRATE" = 0 ] || migrate=(--migrate)
|
||||
plan activate "$SHA" "$BUSINESS" "${migrate[@]}"
|
||||
rehearse
|
||||
run_steps "$SHA" activate "$SHA" "$BUSINESS" "${migrate[@]}"
|
||||
finish 0 ok "upgrade ok: $SHA is live for $BUSINESS"
|
||||
}
|
||||
|
||||
# The rehearsal against the log's newest backup, before anything stops. Its output goes to the
|
||||
# terminal; root keeps only the BACKUP_STORE line, so the go and the log show which store it used.
|
||||
rehearse() {
|
||||
local cap rc bs
|
||||
cap="$(mktemp "$RUN_DIR/rehearse-$INSTANCE.XXXXXX")" || refuse rehearsal-failed "cannot make a capture file in $RUN_DIR"
|
||||
local -a cmd=("${AS_USER[@]}" "$SR" rehearse "$SHA" "$BUSINESS" --copies "$COPIES_DIR" --backup last --port "$R_PORT" --timeout "$TIMEOUT")
|
||||
"${cmd[@]}" 9>&- 2>&1 | tee "$cap" 9>&-
|
||||
rc="${PIPESTATUS[0]}"
|
||||
logl "rehearse rc=$rc argv:$(printf ' %q' "${cmd[@]}")"
|
||||
bs="$(grep -E '^BACKUP_STORE [0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])? schema [0-9a-z]{1,16}$' "$cap" | tail -n 1)"
|
||||
rm -f "$cap"
|
||||
logl "rehearse-store rc=0 ${bs:-BACKUP_STORE none}"
|
||||
case "$rc" in
|
||||
0) [ "$NO_REHEARSAL" = 0 ] || refuse rehearsal-required "the log records a backup and the rehearsal ran; --no-rehearsal is only for a stack with none" ;;
|
||||
3) [ "$NO_REHEARSAL" = 1 ] || refuse no-backup "the log records no backup to rehearse against; only a go that names --no-rehearsal goes on"
|
||||
logl "rehearsal-skipped rc=0 the log records no backup and the go names --no-rehearsal" ;;
|
||||
*) refuse rehearsal-failed "the rehearsal exited $rc" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# r6 rollback: to the log's previous copy, which the stack's user checks against --to. No HEAD
|
||||
# check: the target is an older copy than this checkout, and no rehearsal.
|
||||
do_rollback() {
|
||||
read_args
|
||||
[ -x "$COPIES_DIR/$TO/scripts/mosaic" ] || refuse copy-missing "no $COPIES_DIR/$TO/scripts/mosaic"
|
||||
hostcheck
|
||||
read_old
|
||||
local -a restore=()
|
||||
[ -z "$RESTORE" ] || restore=(--restore "$RESTORE")
|
||||
[ -z "$EVENTS" ] || restore+=(--events "$EVENTS")
|
||||
plan rollback "$BUSINESS" --to "$TO" "${restore[@]}"
|
||||
run_steps "$TO" rollback "$BUSINESS" --to "$TO" "${restore[@]}"
|
||||
finish 0 ok "rollback ok: $TO is live for $BUSINESS"
|
||||
}
|
||||
|
||||
case "$OP" in
|
||||
upgrade) do_upgrade ;;
|
||||
rollback) do_rollback ;;
|
||||
esac
|
||||
@@ -1,30 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Row 234 (#1740, #436a row a), design r6 E3: the installer key's forced command.
|
||||
#
|
||||
# Installed as /usr/local/sbin/mosaic-keys, root, 0755, and named by mosaic-installer's own line in
|
||||
# /etc/ssh/mosaic-keys/mosaic-installer:
|
||||
# restrict,command="/usr/local/sbin/mosaic-keys",from="<from>" ssh-ed25519 <base64> mosaic-installer-<tag>
|
||||
#
|
||||
# It matches SSH_ORIGINAL_COMMAND against three fixed patterns and hands the words to
|
||||
# mosaic-keys-write through sudo. Nothing else runs. sudo's env_reset drops SSH_CONNECTION, so the
|
||||
# connection and the request go to syslog here, before sudo. The writer checks everything again.
|
||||
set -u
|
||||
export PATH=/usr/bin:/bin LC_ALL=C
|
||||
|
||||
request=${SSH_ORIGINAL_COMMAND-}
|
||||
/usr/bin/logger -t mosaic-keys -- "connection=${SSH_CONNECTION-} request=${request:0:1024}"
|
||||
|
||||
user='[a-z_][a-z0-9_-]{0,31}'
|
||||
tag='mosaic-[0-9a-f]{32}'
|
||||
key='(ssh-ed25519|ecdsa-sha2-nistp256|ssh-rsa) [A-Za-z0-9+/]+={0,2}'
|
||||
list_re="^list $user\$"
|
||||
add_re="^add $user $key $tag\$"
|
||||
remove_re="^remove $user $tag\$"
|
||||
|
||||
if [[ $request =~ $list_re || $request =~ $add_re || $request =~ $remove_re ]]; then
|
||||
# The patterns hold no space, glob or quote characters, so a split on spaces gives the words back.
|
||||
read -r -a words <<<"$request"
|
||||
exec /usr/bin/sudo -n /usr/local/sbin/mosaic-keys-write "${words[@]}"
|
||||
fi
|
||||
echo 'mosaic-keys: refused' >&2
|
||||
exit 1
|
||||
@@ -1,285 +0,0 @@
|
||||
#!/usr/bin/python3 -I
|
||||
# Row 234 (#1740, #436a row a), design r6 E3: the root half of host enrollment.
|
||||
#
|
||||
# Installed as /usr/local/sbin/mosaic-keys-write, root, 0755. mosaic-keys (the installer key's forced
|
||||
# command) runs it through `sudo -n` with one request:
|
||||
#
|
||||
# list <user> each line of /etc/ssh/mosaic-keys/<user> with a mosaic- tag
|
||||
# add <user> <type> <base64> <tag> appends from="<from>",restrict <type> <base64> <tag>, unless the tag exists
|
||||
# remove <user> <tag> drops the line with that exact tag; a missing tag changes nothing
|
||||
#
|
||||
# It checks the request again, whatever mosaic-keys did. Host config is root-owned and written by ops:
|
||||
# /etc/mosaic-keys/users (the users it may touch), /etc/mosaic-keys/privileged (users with sudo that
|
||||
# Jason approved by name) and /etc/mosaic-keys/from (the from= value). Standard library only. Exit 0
|
||||
# on success, 1 on any refusal, with the reason on stderr and in syslog.
|
||||
import base64
|
||||
import fcntl
|
||||
import grp
|
||||
import hashlib
|
||||
import os
|
||||
import pwd
|
||||
import re
|
||||
import socket
|
||||
import stat
|
||||
import subprocess
|
||||
import sys
|
||||
import syslog
|
||||
|
||||
KEY_DIR = '/etc/ssh/mosaic-keys'
|
||||
CONF_DIR = '/etc/mosaic-keys'
|
||||
USER = re.compile(r'[a-z_][a-z0-9_-]{0,31}')
|
||||
TAG = re.compile(r'mosaic-[0-9a-f]{32}')
|
||||
KEY_TYPE = re.compile(r'ssh-ed25519|ecdsa-sha2-nistp256|ssh-rsa')
|
||||
BASE64 = re.compile(r'[A-Za-z0-9+/]+={0,2}')
|
||||
FROM = re.compile(r'[0-9A-Za-z.:/*?!_-]+(,[0-9A-Za-z.:/*?!_-]+)*')
|
||||
# In the code, not in a file: no request may name these, whatever users says.
|
||||
RESERVED_USERS = ('root', 'localadmin', 'mosaic-installer')
|
||||
MIN_UID = 1000
|
||||
# Each is root by another route. disk reads and writes raw block devices, libvirt and incus-admin start
|
||||
# guests with host mounts (r6 N5). privileged lifts this check and nothing else.
|
||||
REFUSED_GROUPS = ('sudo', 'admin', 'wheel', 'docker', 'lxd', 'disk', 'libvirt', 'incus-admin')
|
||||
LOCK = '.lock'
|
||||
|
||||
|
||||
class Refused(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def _passwd(user):
|
||||
try:
|
||||
return pwd.getpwnam(user).pw_uid
|
||||
except KeyError:
|
||||
return None
|
||||
|
||||
|
||||
def _groups(user):
|
||||
entry = pwd.getpwnam(user)
|
||||
names = set()
|
||||
for gid in os.getgrouplist(user, entry.pw_gid):
|
||||
try:
|
||||
names.add(grp.getgrgid(gid).gr_name)
|
||||
except KeyError:
|
||||
pass
|
||||
return names
|
||||
|
||||
|
||||
def _sudo(user):
|
||||
try:
|
||||
out = subprocess.run(
|
||||
['/usr/bin/sudo', '-l', '-U', user],
|
||||
env={'LC_ALL': 'C', 'PATH': '/usr/bin:/bin'},
|
||||
stdin=subprocess.DEVNULL,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
timeout=10,
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
# No answer is not the expected line, so the caller refuses user-sudo and logs it (duckworth-0081 N2).
|
||||
return ''
|
||||
return out.stdout.decode('utf-8', 'replace')
|
||||
|
||||
|
||||
def _log(message):
|
||||
syslog.openlog('mosaic-keys-write', 0, syslog.LOG_AUTH)
|
||||
syslog.syslog(syslog.LOG_INFO, message)
|
||||
|
||||
|
||||
class Host:
|
||||
# What the writer reads from the machine. Tests pass their own dirs, owner and lookups.
|
||||
def __init__(self, key_dir=KEY_DIR, conf_dir=CONF_DIR, owner=0, passwd=_passwd, groups=_groups,
|
||||
sudo=_sudo, hostname=None, log=_log):
|
||||
self.key_dir = key_dir
|
||||
self.conf_dir = conf_dir
|
||||
self.owner = owner
|
||||
self.passwd = passwd
|
||||
self.groups = groups
|
||||
self.sudo = sudo
|
||||
self.hostname = hostname if hostname is not None else socket.gethostname().split('.')[0]
|
||||
self.log = log
|
||||
|
||||
|
||||
def fingerprint(blob64):
|
||||
digest = hashlib.sha256(base64.b64decode(blob64, validate=True)).digest()
|
||||
return 'SHA256:' + base64.b64encode(digest).decode().rstrip('=')
|
||||
|
||||
|
||||
def parse(argv):
|
||||
# The request's shape and every pattern, before any name reaches a path.
|
||||
if not argv:
|
||||
raise Refused('request')
|
||||
verb, args = argv[0], argv[1:]
|
||||
shapes = {'list': 1, 'add': 4, 'remove': 2}
|
||||
if verb not in shapes or len(args) != shapes[verb]:
|
||||
raise Refused('request')
|
||||
if not USER.fullmatch(args[0]):
|
||||
raise Refused('user-pattern')
|
||||
req = {'verb': verb, 'user': args[0]}
|
||||
if verb == 'add':
|
||||
if not KEY_TYPE.fullmatch(args[1]) or not BASE64.fullmatch(args[2]):
|
||||
raise Refused('key-format')
|
||||
try:
|
||||
req['fingerprint'] = fingerprint(args[2])
|
||||
except ValueError:
|
||||
raise Refused('key-format')
|
||||
req['type'], req['base64'] = args[1], args[2]
|
||||
if verb in ('add', 'remove'):
|
||||
if not TAG.fullmatch(args[-1]):
|
||||
raise Refused('tag-pattern')
|
||||
req['tag'] = args[-1]
|
||||
return req
|
||||
|
||||
|
||||
def _owned(st, host, kind):
|
||||
if st.st_uid != host.owner or st.st_mode & 0o022 or not kind(st.st_mode):
|
||||
raise Refused('not-root-owned')
|
||||
|
||||
|
||||
def read_conf(host, name):
|
||||
# A root-owned regular file, not group or world writable, opened without following a symlink.
|
||||
try:
|
||||
fd = os.open(os.path.join(host.conf_dir, name), os.O_RDONLY | os.O_NOFOLLOW)
|
||||
except OSError:
|
||||
raise Refused('config-' + name)
|
||||
try:
|
||||
_owned(os.fstat(fd), host, stat.S_ISREG)
|
||||
with os.fdopen(os.dup(fd), 'r', encoding='utf-8') as f:
|
||||
text = f.read()
|
||||
finally:
|
||||
os.close(fd)
|
||||
return [line.strip() for line in text.splitlines() if line.strip() and not line.strip().startswith('#')]
|
||||
|
||||
|
||||
def check_user(req, host):
|
||||
user = req['user']
|
||||
users = read_conf(host, 'users')
|
||||
privileged = read_conf(host, 'privileged')
|
||||
if user in RESERVED_USERS or user not in users:
|
||||
raise Refused('user-not-listed')
|
||||
uid = host.passwd(user)
|
||||
if uid is None:
|
||||
raise Refused('user-unknown')
|
||||
if uid == 0 or uid < MIN_UID:
|
||||
raise Refused('user-system')
|
||||
if user in privileged:
|
||||
return
|
||||
if set(REFUSED_GROUPS) & set(host.groups(user)):
|
||||
raise Refused('user-group')
|
||||
expected = 'User %s is not allowed to run sudo on %s.' % (user, host.hostname)
|
||||
out = host.sudo(user)
|
||||
if out not in (expected, expected + '\n'):
|
||||
raise Refused('user-sudo')
|
||||
|
||||
|
||||
def read_from(host):
|
||||
lines = read_conf(host, 'from')
|
||||
if len(lines) != 1 or not FROM.fullmatch(lines[0]):
|
||||
raise Refused('config-from')
|
||||
return lines[0]
|
||||
|
||||
|
||||
def tag_of(line):
|
||||
parts = line.split()
|
||||
return parts[-1] if parts and TAG.fullmatch(parts[-1]) else None
|
||||
|
||||
|
||||
def read_lines(dir_fd, user, host):
|
||||
# ENOENT means an empty list: a first add finds no file (r6 N6).
|
||||
try:
|
||||
fd = os.open(user, os.O_RDONLY | os.O_NOFOLLOW, dir_fd=dir_fd)
|
||||
except FileNotFoundError:
|
||||
return []
|
||||
except OSError:
|
||||
raise Refused('user-file')
|
||||
try:
|
||||
_owned(os.fstat(fd), host, stat.S_ISREG)
|
||||
with os.fdopen(os.dup(fd), 'r', encoding='utf-8') as f:
|
||||
return f.read().splitlines()
|
||||
finally:
|
||||
os.close(fd)
|
||||
|
||||
|
||||
def write_lines(dir_fd, user, lines, host):
|
||||
tmp = '.' + user + '.tmp'
|
||||
try:
|
||||
st = os.stat(tmp, dir_fd=dir_fd, follow_symlinks=False)
|
||||
except FileNotFoundError:
|
||||
st = None
|
||||
if st is not None:
|
||||
# Under the flock a leftover tmp can only come from a writer that crashed.
|
||||
if not stat.S_ISREG(st.st_mode) or st.st_uid != host.owner:
|
||||
raise Refused('tmp-file')
|
||||
os.unlink(tmp, dir_fd=dir_fd)
|
||||
fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o644, dir_fd=dir_fd)
|
||||
try:
|
||||
os.fchmod(fd, 0o644)
|
||||
os.write(fd, ''.join(line + '\n' for line in lines).encode('utf-8'))
|
||||
os.fsync(fd)
|
||||
finally:
|
||||
os.close(fd)
|
||||
os.rename(tmp, user, src_dir_fd=dir_fd, dst_dir_fd=dir_fd)
|
||||
os.fsync(dir_fd)
|
||||
|
||||
|
||||
def run(req, host, out):
|
||||
check_user(req, host)
|
||||
line_from = read_from(host) if req['verb'] == 'add' else None
|
||||
try:
|
||||
dir_fd = os.open(host.key_dir, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
|
||||
except OSError:
|
||||
raise Refused('key-dir')
|
||||
try:
|
||||
_owned(os.fstat(dir_fd), host, stat.S_ISDIR)
|
||||
lock = os.open(LOCK, os.O_RDWR | os.O_CREAT | os.O_NOFOLLOW, 0o600, dir_fd=dir_fd)
|
||||
try:
|
||||
fcntl.flock(lock, fcntl.LOCK_EX)
|
||||
user = req['user']
|
||||
lines = read_lines(dir_fd, user, host)
|
||||
if req['verb'] == 'list':
|
||||
for line in lines:
|
||||
if tag_of(line):
|
||||
out.write(line + '\n')
|
||||
return 'listed'
|
||||
if req['verb'] == 'add':
|
||||
if any(tag_of(line) == req['tag'] for line in lines):
|
||||
return 'present'
|
||||
entry = 'from="%s",restrict %s %s %s' % (line_from, req['type'], req['base64'], req['tag'])
|
||||
write_lines(dir_fd, user, lines + [entry], host)
|
||||
return 'added'
|
||||
kept = [line for line in lines if tag_of(line) != req['tag']]
|
||||
if len(kept) == len(lines):
|
||||
return 'absent'
|
||||
write_lines(dir_fd, user, kept, host)
|
||||
return 'removed'
|
||||
finally:
|
||||
os.close(lock)
|
||||
finally:
|
||||
os.close(dir_fd)
|
||||
|
||||
|
||||
def main(argv, host, out=sys.stdout, err=sys.stderr):
|
||||
req = None
|
||||
try:
|
||||
req = parse(argv)
|
||||
result = run(req, host, out)
|
||||
code = 0
|
||||
except Refused as e:
|
||||
result = 'refused ' + str(e)
|
||||
code = 1
|
||||
except OSError as e:
|
||||
result = 'refused os-' + (e.__class__.__name__)
|
||||
code = 1
|
||||
fields = ['result=' + result]
|
||||
if req:
|
||||
fields.append('user=' + req['user'])
|
||||
if 'tag' in req:
|
||||
fields.append('tag=' + req['tag'])
|
||||
if 'fingerprint' in req:
|
||||
fields.append('key=' + req['fingerprint'])
|
||||
host.log(' '.join(fields))
|
||||
if code:
|
||||
err.write('mosaic-keys-write: ' + result + '\n')
|
||||
return code
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main(sys.argv[1:], Host()))
|
||||
@@ -1,3 +0,0 @@
|
||||
# Row 234 (#1740), design r6 E3. Installed as /etc/ssh/sshd_config.d/<nn>-mosaic-keys.conf, root, 0644.
|
||||
# Keeps both paths the hosts use today and adds the root-owned per-user files mosaic-keys-write keeps.
|
||||
AuthorizedKeysFile .ssh/authorized_keys .ssh/authorized_keys2 /etc/ssh/mosaic-keys/%u
|
||||
@@ -1,2 +0,0 @@
|
||||
# Row 234 (#1740), design r6 E3. Installed as /etc/sudoers.d/mosaic-keys, root, 0440.
|
||||
mosaic-installer ALL=(root) NOPASSWD: /usr/local/sbin/mosaic-keys-write
|
||||
Reference in New Issue
Block a user