Compare commits

...
Author SHA1 Message Date
jason.woltje b8461266cd test: clean proof temp roots and catch preparation signals (#1695) 2026-10-21 03:29:20 -05:00
Herb 8ae9d8acd9 test: add row 197 proof evidence driver for review (#1695) 2026-10-21 03:11:34 -05:00
Herb 9aa6d52d94 test: merge approved gate lanes onto row 238 (#1695) 2026-10-21 02:29:19 -05:00
jason.woltje e7078e6a85 deferred: Webby's row 238 N1, the sso-config first-step hint has no business (#1744) 2026-10-21 02:27:43 -05:00
jason.woltje d49b1f2bd3 row 238: stack-release's sso-config first step carries --instance NAME on a named instance (#1744)
Lands Launchpad's round 3 candidate, manifest 1bf831f3 (4c90099ea on launchpad/row238-r4, base
c515273db), approved by Webby (29956, rev 2717). Three paths: scripts/stack-release.sh, packages/cli/tests/
stack-release.test.mjs and docs/TOOLS.md. It is row 200's follow-up N1. Round 3 only merges docs/TOOLS.md
onto the row 240 landing: the landed blob plus the one hunk at :599. The 3-path patch-id, b9d0bb0ee3c7,
is the same in all three rounds, and stack-release.sh stays 100755.

Evidence: Launchpad's slot run on round 2, 2afbfbb01: stack-release.test.mjs 196/0 (sage-1794). Sage's
combined gate with row 228 on c515273db plus both rows' paths (07:11:30Z to 07:23:34Z): packages/seat 165 pass, 0 fail; packages/cli 640 pass, 0 fail; all ten scripts/test-*.sh 585 pass, 0 fail (out/, run.log). Row 238's mutant, the sso-config first step dropping --instance NAME, was killed (2 fails), survived: none, and stack-release.sh was restored to its blob.
`queue review verify-commit 238` matches the manifest on all 3 paths. Round 1's N1 stays on the #1744
carry list.
2026-10-21 02:24:25 -05:00
jason.woltje 86249ac552 row 228: ZAI_API_KEY in the session env allowlist (#1729)
Lands Researcher's round 3 candidate, manifest 82cf98c6 (cf18c42b9 on researcher/row228-r3, base
c771a36f4), approved by Webby (29949, rev 2708). Three paths: packages/seat/src/session.mjs (ZAI_API_KEY
in ENV_ALLOW), packages/seat/tests/session.test.mjs (the pass-through test) and packages/cli/README.md
(the line, with the round 1 and round 2 N1 trims at :711 and :718). Round 3 is the round 2 rebase onto
the row 234 landing; the two seat files carry the blobs approved in rounds 1 and 2.

Evidence: Researcher's slot run on cf18c42b9: session.test.mjs 13/0, host.test.mjs 42/0 (sage-1796).
Round 2's seat and cli runs, 94/0, and its killed mutant stand, since the seat blobs are unchanged.
Sage's combined gate with row 238 on c515273db plus both rows' paths (07:11:30Z to 07:23:34Z):
packages/seat 165 pass, 0 fail; packages/cli 640 pass, 0 fail; all ten scripts/test-*.sh 585 pass, 0 fail (out/, run.log). bus runtime.mjs and webui reads.mjs import only readSessions from session.mjs, which this
change doesn't touch. `queue review verify-commit 228` matches the manifest on all 3 paths.
2026-10-21 02:24:25 -05:00
jason.woltje 59f817c321 sessions: Darkwing, row 195 DESIGN r7 pushed (5425ea2f3), Duckworth review requested (Gitea 29960), row 226 items to Gosalyn 2026-10-21 02:15:49 -05:00
jason.woltje c515273db7 row 240: installer I4, root's host check, its pins and the boot unit; I3's N1, N4 and N5 (#1746, Vikunja #432)
Lands Darkwing's round 1 candidate, manifest 573ece95a4a9 (ad1b23ab2 on darkwing/row240-r3, base
c771a36f4), approved by Filbert (29945, rev 2696). Nine paths: scripts/case1-hostcheck.sh (new),
scripts/case1-pins.sh (new), packages/cli/systemd/mosaic-hostcheck.service (new),
packages/cli/tests/case1-hostcheck.test.mjs (new), scripts/stack-upgrade.sh,
packages/cli/tests/stack-upgrade.test.mjs, docs/guides/host-run-sheet.md, docs/TOOLS.md and
agents/darkwing/work/queue-240/BUILD.md. File modes match the candidate. case1-hostcheck.sh and
stack-upgrade.sh are 100755. case1-pins.sh is 100644, because the host check sources it.

Evidence: Darkwing's first slot run failed one test, the N1 test's after hook (ENOENT once the
fixture had removed its root). The fix b45852daa is test-only. The second run on b45852daa was green:
singles 37/0, mutants M1-M8 8 of 8 killed, cli 639/0, scripts 585/0. Since c771a36f4 no commit
touches the nine paths. Sage's gate on 060a17b54 plus the nine paths (06:45:05Z to 06:56:21Z):
packages/cli 639 pass, 0 fail, and all ten scripts/test-*.sh 585 pass, 0 fail (out/, run.log). `queue review verify-commit 240` matches the manifest on all 9 paths.

Filbert's notes: N1 (the boot unit orders only mosaic-bus@mosaic-stack) and N2 (the run sheet writes
the apt hook before the first --check) are in docs/plans/DEFERRED.md. N3, repeated DRIFT lines after
bwrap is closed, is by design and is noted with N2.
2026-10-21 01:56:47 -05:00
jason.woltje da3051ccc6 deferred: Filbert's row 240 N1 and N2 (29945, #1746); 10-24 window draft Q1 takes the mos-1161 ruling 2026-10-21 01:50:34 -05:00
jason.woltje 060a17b549 sessions: Filbert, row 240 round 1 approve (29945, rev 2696) 2026-10-21 01:41:13 -05:00
jason.woltje ffd90e7fc4 sessions: Darkwing, row 240 in review round 1 (ad1b23ab2, manifest 573ece95, revs 2694 and 2695), with its BUILD-LOG entry 2026-10-21 01:37:57 -05:00
jason.woltje 02709125a6 sessions: Sage, row 234 landing c771a36f4, 235 briefed, rebuilt-row slot order, 10-24 draft NEW (sage-1766..1787)
Doc only. The 10-24 window draft 0 now names NEW c771a36f4 and adds row 233 and the T46 untimed tables.
2026-10-21 01:20:30 -05:00
jason.woltje c771a36f4b row 234: machine grant row a, host enrollment, store records and bus schema 3i to 3j (#1740, Vikunja #436)
Lands Gizmoduck's round 3 candidate, manifest 4782c8b5ecb7 (c783d5e39 on gizmoduck/row234-r3, one commit
on 57f59e0c3), approved by Duckworth: content on round 1 (861182fa6, 29919), the round 2 rebase check
(29928, rev 2670) and round 3 (REVIEW-r3.md 7c8c38b69, posted 29935, rev 2683). 41 paths.

Round 3 fixes the two failures in Sage's combined gate with row 233: bus schema.test.mjs now counts 18
tables and 71 triggers for the new hosts and host_grants tables, and stack-release.test.mjs T46 puts both
tables in the untimed restore-guard list, since every write shares a transaction with a grant.* event.
Round 3 rebuilt on the row 233 landing as one commit. 38 of the 39 round 2 blobs are unchanged and BUILD.md
is append-only. Gizmoduck's singles on round 3: stack-release 196/0, schema 1/0.

Evidence: Gizmoduck's slot runs, 25 of 25 mutants killed (A22b and A23 included, A9 structural), singles on
r2 387/0 over 47 files. Sage's landing gate on the c783d5e39 tree: 06:02:12Z to 06:15:26Z, every package except queue 2890 pass, 0 fail (15 packages; bus 539, cli 622), and all ten scripts/test-*.sh 585 pass, 0 fail.

This is code only. The live stack runs from a pinned copy at schema 3i. The 3i to 3j migration runs in the
10-24 window (mos-1145), agent-run under Q173, with a backup first. Duckworth's N10 (hosts.mjs
Object.hasOwn) and N12 (doc-only) go to row 235's carry list.
2026-10-21 01:15:43 -05:00
jason.woltje caa4b59440 sessions: Sage, row 233 landing 57f59e0c3, 234 gate stop and r3, 240 briefed, 197/238 drift (sage-1742..1758) 2026-10-21 00:47:45 -05:00
jason.woltje 57f59e0c39 row 233: installer I3, stack-upgrade.sh and the host run sheet (#1735, Vikunja #432)
Lands Darkwing's round 1 candidate, manifest 464ce9eb177a (1a1dc80ca on darkwing's branch), approved by
Filbert (29922, rev 2659). Seven paths: scripts/stack-upgrade.sh (new), scripts/stack-release.sh,
docs/guides/host-run-sheet.md (new), docs/TOOLS.md, packages/cli/tests/stack-upgrade.test.mjs (new),
packages/cli/tests/stack-release.test.mjs and agents/darkwing/work/queue-233/BUILD.md.

Evidence: Darkwing's gate on d99166ade (base e65a3ab4d) 1201/0, mutants m1-m8 8 of 8 killed. The N1 fix
08db6ac3f (flock sleep leak in stack-upgrade.test.mjs) reran c0 20/0 with m1 killed (sage-1708); BUILD.md
doesn't record that rerun. Since e65a3ab4d no commit touches the seven paths. Rows 241 and 242 changed bus
and cli files. Only cli tests read the seven paths, so Sage ran the cli suite on 5465fd52d plus the seven
paths: 617 pass, 0 fail (out233/pkg-cli.log, 05:26:33Z to 05:35:00Z). A combined gate with row 234 failed bus schema.test.mjs:522 on 234's new tables, and
Gizmoduck found a second 234 failure (stack-release.test.mjs T46), so Sage stopped it. Row 234 round 3 fixes
both and rebuilds on this landing, since its T46 lines go in a file this commit changes.

Filbert's N1, N4 and N5 go to row 240 (brief b00d7d94d). N2 and N3 are in DEFERRED.md.
2026-10-21 00:35:18 -05:00
jason.woltje 5465fd52d5 sage: DEFERRED, checkCreative doesn't refuse credentials (row 213 r3 Webby N1, mos-1160) 2026-10-21 00:12:52 -05:00
jason.woltje b00d7d94d0 sage: row 240 (I4) carries Filbert N1, N4 and N5 from the 233 review; N2 and N3 go to DEFERRED (29922, sage-1727) 2026-10-21 00:01:26 -05:00
jason.woltje 642a9b87de sessions: Filbert, row 233 round 1 approve (29922, rev 2659) 2026-10-20 23:58:01 -05:00
jason.woltje 16019b80e6 sessions: Darkwing, row 233 in review round 1 (1a1dc80ca, manifest 464ce9eb, revs 2644 and 2647), with its BUILD-LOG entry 2026-10-20 23:51:01 -05:00
jason.woltje 9b53620185 sage: 10-24 window draft 0 counts grant look-alike events before --migrate (row 234 A22b, gizmoduck-0107) 2026-10-20 23:29:34 -05:00
jason.woltje 7662b04902 sage: 10-24 window draft 0 adds rows 232 (e65a3ab4d), 241 (5d4a61776) and 242 (5fb736b92, fc1cb856b); 199 and 210 evidence list due in draft 1 2026-10-20 23:24:00 -05:00
jason.woltje 04ce8dfd56 queue: revs 2639-2641 (row 242 Duckworth approve record, landing note and done) 2026-10-20 23:22:41 -05:00
jason.woltje fc1cb856b0 publish: row 242 follow-up docs, the git socket comments name the hold and the late record; BUILD.md residual and :311 cite (Duckworth N1, N2, N4, #1748)
Megavolt's c5a125f4b on megavolt/row242-docs, parent 5fb736b92 (the row 242
landing). Comments and .md only: git-socket.mjs changes comment lines at
:39-44 and :433-435, BUILD.md records N2 and N4. No test pins either path's
hash. Landed without a gate under the doc-only rule.
2026-10-20 23:17:14 -05:00
jason.woltje c0c4907af9 sessions: Sage, rows 241 (5d4a61776) and 242 (5fb736b92) landed, queue revs 2628-2641 2026-10-20 23:14:49 -05:00
jason.woltje 5fb736b92d publish: a retry after publish-timeout pushes once; the bus holds the decision until its push ends and records a late push (row 242, #1748, P19 440)
Megavolt's candidate 635c10924 on megavolt/row242, manifest 1fbfab6c (3 paths:
agents/megavolt/work/queue-242/BUILD.md, packages/bus/src/runtime.mjs,
packages/cli/tests/publish-socket.test.mjs). Duckworth approved (comment 29889, rev 2639).

Mutants M1 and M2, run by Megavolt on Sage's slot: both killed (publish-socket 8/1 each).
Sage's landing gate on 12b8538e0 (96bac588f plus these 3 paths): 15 packages 2826 pass 0 fail (bus 500, cli 594, seat 164), 10 scripts/test-*.sh suites 429 pass 0 fail plus 156 node tests in the queue suite, 03:59Z to 04:12:44Z

The 3 paths are unchanged on refactor since the candidate's base 8fede5fc6. Follow-ups:
Duckworth's doc notes N1, N2 and N4 go in a later doc-only commit; the late-record notice
to the run is Vikunja 440 comment 432. No live change.
2026-10-20 23:13:06 -05:00
jason.woltje 96bac588fb queue: revs 2628-2638 (row 241 approve record, landing note and done; row 232 landing note and done; row 233 brief re-pin and Darkwing claim; row 242 in review round 1, request 29888) 2026-10-20 22:47:48 -05:00
jason.woltje 5d4a617762 bus: row 241 one decision lands once; a broker test with two checkLand tickets on one decision, where the second recordLand refuses decision-consumed and records nothing (row 241, #1747, P19 441)
Megavolt's candidate 3e3ed303e on megavolt/row241, manifest e8266864 (2 paths:
agents/megavolt/work/queue-241/BUILD.md, packages/bus/tests/land.test.mjs). Tests and
packet only, no source change. Duckworth approved (comment 29870, rev 2628).

Sage's slot on bbf3b1ee8 (0444acd08 plus these 2 paths):
- mutants M1 (#spent recheck, broker.mjs:963), M2 (CAS old value, seat working.mjs:391),
  M3 (land-busy, cli git-socket.mjs:351): all killed
- full gate: 15 packages 2797 pass 0 fail (bus 500, cli 565), 10 scripts/test-*.sh suites 429 pass 0 fail plus 156 node tests in the queue suite, 03:21:13Z to 03:34:16Z

Row 232's landing e65a3ab4d sits between that base and this commit. Its 9 paths are
disjoint from these 2 and touch no bus code. No live change.
2026-10-20 22:34:30 -05:00
100 changed files with 8534 additions and 289 deletions
+2
View File
@@ -5175,3 +5175,5 @@ Before: managed project/thread verbs had no Console conversation area. After: fi
2026-10-20 05:10:34 UTC | Row 178 #1676 round 2 verified | Before: C1 render refusals lacked killing tests and row 181 base was pending. Rebased onto 931651f2d, kept all COUNTED/count code and tests, appended T178 tail; added independent owner/collision/absolute/charset/header and empty-loopback proof tests. Four files singly 156/156, all 24 mutants killed, metadata and sandbox fixture probes pass; scratch sweep empty. Packet agents/rocko/work/queue-178/r2/, candidate 9300a670, inventory d0dcde7b (59/59), scan 58 files no findings; in-review revs 2166-2167, expected no-token request failure. No live actions or commits.
2026-10-21T01:28:05Z | Darkwing | row 232 #1734 installer I1 build, round 1 in review | Before: stack-release.sh managed only user units (systemctl --user), and stack-copy.sh had no bundle copy. After: --units external (adopt, activate, rollback --to, status, confirm, revert, gate --since, AWAITING-UNITS, stale-start, external-pm, --plan --expect-from) and stack-copy check --units external, copy --bundle FILE SHA, wrapper ID --create; user mode byte-identical to the base by test. First build 0af0c73c6 on 00d4cc1f6; rebased onto row 200 (cb5fdb064) as a0ffabc3f on darkwing/row232-r1; --sso refused under external (D9), row 200 config_checks run under external (D10). Gate on Sage's slot (sage-1618): cli 561/0, every scripts/test-*.sh green; 17/17 mutants killed. Results commit 84f1a91e6 (pushed, seat branch only). Manifest 7a3b4ea1 (9 paths), queue revs 2595-2596, request comment 29825, reviewer Filbert. Packet agents/darkwing/work/queue-232/BUILD.md.
2026-10-21T04:50:44Z | Darkwing | row 233 #1735 installer I3 build, round 1 in review | Before: root had no upgrade driver for root-owned units and no host run sheet; stack-release.sh had no rehearse --backup last and no external plan refusals. After: scripts/stack-upgrade.sh upgrade and rollback (r6 A1-A5, A2f, R1-R4) with 20 tests, docs/guides/host-run-sheet.md, TOOLS.md lines, and on Sage's rulings stack-release.sh --backup last, BACKUP_STORE and the --plan refusals open-activation and migrate-required (3 tests). Base e65a3ab4d (row 232's landing). Gate on Sage's slot (sage-1704) on d99166ade: cli 616/0, every scripts/test-*.sh green, 1201/0 summed by awk; m1-m8 8/8 killed. Correction: the process sweep after the gate found an orphaned sleep 60 from the lock test's flock holder (N1); 08db6ac3f fixes it test only, Sage ruled the gate stands (sage-1707), rerun c0 20/0 and m1 killed with 0 left (sage-1708). Tip 1a1dc80ca (pushed, seat branch only). Manifest 464ce9eb (7 paths), queue revs 2644 and 2647, request comment 29901, reviewer Filbert. Packet agents/darkwing/work/queue-233/BUILD.md.
2026-10-21T06:37:41Z | Darkwing | row 240 #1746 installer I4 build, round 1 in review | Before: root had no host check for the bwrap AppArmor profile, its pins or a boot unit, and stack-upgrade.sh exited 1 args-failed when the args move after A5 failed, while exit 1 means the old copy runs. After: scripts/case1-hostcheck.sh (seven items, --check, --enforce, --upstream), scripts/case1-pins.sh, mosaic-hostcheck.service and 16 tests; the args move failure exits 5 args-failed, documented in TOOLS.md and run sheet section 6 and tested; run sheet section 3 puts the pinned npm on PATH and names the bin/claude.exe --claude path; sections 8 and 10 have precheck lines. Verification: slot run r1 on 47ec9fe2f failed at the singles (the new N1 test's after hook threw ENOENT after the fixture cleanup removed the root); b45852daa drops the hook; r2 on b45852daa green: singles 16/0 and 21/0, c0 plus M1-M8 8/8 killed, cli 639/0, scripts 585/0. Candidate darkwing/row240-r3 ad1b23ab2 on c771a36f4, manifest 573ece95 (9 paths), revs 2694 and 2695, comment 29943.
+249
View File
@@ -0,0 +1,249 @@
# Row 233 (I3): root's upgrade driver and the host run sheet
Brief: `docs/plans/2026-10-20_installer-rows.md` section I3, blob `e61291bcf1c6` (commit a0e3d7cfd), re-pinned
by Sage (sage-1685, rev 2631) from `f35e7e57daee` (ad6f0be7a). The later blob only appends I4; the I3 text
is the same. Gitea #1735, Vikunja P19
#432. Owner Darkwing, reviewer Filbert, gate owner Sage. Design input: row 195 DESIGN r6,
`agents/darkwing/work/queue-195/DESIGN.md` at b7239bb66, sections 2, 4 and 7. Where the brief and r6
differ, the brief wins. Sage's rulings during the build: sage-1650 and sage-1652 (`--backup last`, the
`--no-rehearsal` mapping, `--rehearse-port`, hostcheck), and sage-1653 (`from_log newest`, the two external
plan refusals, the `BACKUP_STORE` line).
## Files
Base: e65a3ab4d, row 232's landing, on `darkwing/row233-r1`. The build started on ea1e85c7b
(`darkwing/row233`, row 232's candidate e6ca230d2 and its BUILD.md) before row 232 landed. The three row 232
files this row changes are identical on the two bases, so the uncommitted diff moved across unchanged.
- `scripts/stack-upgrade.sh` (new): `upgrade` and `rollback`, root's side of r6 A1-A5, A2f and R1-R4.
- `packages/cli/tests/stack-upgrade.test.mjs` (new, 20 tests).
- `docs/guides/host-run-sheet.md` (new).
- `docs/TOOLS.md`: a new subsection "Root's upgrade driver (`scripts/stack-upgrade.sh`, row 233)", plus the
row 232 lines below.
- Row 232 files, changed on Sage's rulings (the brief's files list doesn't name them):
- `scripts/stack-release.sh`:
- `rehearse --backup last` takes the newest backup the log records (`from_log newest`) and prints
`PASS backup-last`. With none it fails `no-backup`, exits 3 and makes no rehearsal directory.
A newest backup that isn't an activation id aborts `backup-unreadable`, exit 1.
- The rehearsal prints `BACKUP_STORE <activation id> schema <version>` before its migration step.
- `--plan` under `--units external` refuses `open-activation` when any rollback set holds
`AWAITING-UNITS`, and `migrate-required` for a migrating target without `--migrate`. Root reads
only the plan's exit, so these two have to be refusals there. User mode prints the same bytes.
- `packages/cli/tests/stack-release.test.mjs`: three new `row 233:` tests for `--backup last`.
- `docs/TOOLS.md`: `--backup ACTIVATION|PATH|last` in the rehearse usage, a paragraph on `last`, and a
paragraph on the two external plan refusals.
- This file.
## The brief, item by item
| Brief item | Where |
|---|---|
| Root runs from a root-owned checkout, never a copy | `head-mismatch` (A1); TOOLS.md; run sheet section 6 |
| I2's render with `/etc/mosaic-stack/<instance>.args`, `--copy` the one changed value | `read_args`, `args_with`; the args test |
| The args file changes only after a confirmed run | `prepare_args` before A2, `mv` after A5; sequences 1 and 7, m6 |
| `copy --bundle` as the stack's user before A1 | the `fetch`, `ff`, `copy` steps; sequence 1's sudo keys |
| A rehearsal before A2, a failed one refuses with nothing stopped | `rehearse`; three rehearsal tests, m4 |
| The root lock, held to the end, rollback included | `flock -n 9`; the lock test and sequence 3's probe, m1, m2 |
| A1 refuses `open-activation` | stack-release.sh's external plan; the AWAITING-UNITS test, m3 |
| `migrate-required` without `--migrate` | stack-release.sh's external plan; its test, m5 |
| The step log, 0640, one line per step | `logl`, `step`; every test checks lines against `STEP` |
| Exit codes 0-3 | `finish`, `refuse`; exit 4 is usage |
| duckworth-0055's notes | the next section |
| Root parses no stack user output beyond exits | the step log test; the one exception is sage-1653's `BACKUP_STORE` |
| The host run sheet | `docs/guides/host-run-sheet.md` |
## duckworth-0055's notes on r6 (brief line 169)
- N1, A2's stop timeout under `gate --since`. Row 232's BUILD.md left two pieces to I3. The run sheet's
A2f line is in section 6 ("When a run doesn't exit 0", exit 2). The test is `N1: a bus that outlives
A2's timeout keeps its old start`: a bus that ignores `stop` keeps its old `ActiveEnterTimestamp`,
`wait-down` logs `rc=1 ... state: active`, A2f restarts and gates, the gate fails `stale-start`, and the
driver stops both units and exits 2. Sequence 4 is the other half: an A3 exit 3 goes to A2f and exits
1 with the old copy's gate passing.
- N2, the claude path under `/opt/mosaic-stack`. Run sheet section 3 installs it there as root with
managed settings. The args file's `--claude` carries it, and `case2-unit.sh` already refuses a binary
that isn't root's or sits under the home. The driver passes the args file through and adds no claude
check of its own. F6 is I2's.
## Sage's rulings during the build
- sage-1650 and sage-1652. Root always runs the rehearsal and maps its exit:
- 0 goes on, or refuses `rehearsal-required` with `--no-rehearsal`.
- 3 refuses `no-backup`, or with `--no-rehearsal` logs `rehearsal-skipped` and goes on.
- Anything else refuses `rehearsal-failed`.
`--rehearse-port N`, default 7339. Hostcheck: a missing file or a nonzero `--check` refuses
`profile-drift`.
- sage-1653. `from_log newest`, and the external plan refusals `open-activation` and `migrate-required`.
Residual: the `BACKUP_STORE` id and schema land in root's step log, but nothing ties that backup to the
one the activation's own backup step takes later. A stopped backup before the upgrade would close that
gap; it's a later row, not this one.
## Tests
`stack-upgrade.test.mjs` runs the driver as the test's user against a scratch root, with the test seams
`STACK_UPGRADE_*`. The stand-ins are a fake `sudo` (logs its argv, checks the driver's form, then runs the
real command as the test's user, r6's B1 pattern), a fake `case2-unit.sh` render that installs and starts
the fixture units, a fake `systemctl` for root's system units, and a shell `hostcheck.sh`. The stack's
user's steps run the real `stack-release.sh` and `stack-copy.sh` from a clone made from a real bundle. The
units, copies and store are `stack-release-fixture.mjs`'s.
| Test | Covers |
|---|---|
| sequence 1 | confirm fails healthz, rollback with `--cause A5`, exit 1; every sudo call in the driver's form |
| sequence 2 | `activate --migrate` fails at the wrapper, store restored, `--cause A3`, exit 1 |
| sequence 3 | render fails, `--cause A4`; a probe run during `render-old` refuses `upgrade-running` |
| sequence 4 | stage exits 3, A2f restarts and gates the old units, exit 1 |
| N1 | a unit that won't stop: A2f's gate fails `stale-start`, exit 2, units stopped |
| sequence 5 | confirm of the wrong copy refused, `--cause A5`, exit 1 |
| sequence 6 | revert fails, exit 2; the next run refuses `interrupted-run` at its plan |
| sequence 7 | two upgrades and a `rollback --to` with `--restore`; the args file after each confirm; `to-not-previous` |
| sequence 8 | a migration that writes and fails, store back byte for byte, exit 1 |
| lock | a second run while the lock is held: one log line, no call |
| open-activation | any `AWAITING-UNITS` refuses at A1, nothing stopped |
| migrate-required | refused without `--migrate`, no rehearsal, nothing stopped |
| rehearsal-failed | the rehearsal's argv, exit 3, nothing stopped |
| `BACKUP_STORE` | the line after a match; four malformed lines log `BACKUP_STORE none` |
| no-backup | the real rehearsal with no backup refuses; `--no-rehearsal` logs the skip and goes on |
| backup-last | the real rehearsal names the newest backup; `--no-rehearsal` with one refuses `rehearsal-required` |
| args file | group write, other write, two `--copy`, an unknown flag, a symlink, `args-differ` |
| hostcheck | nonzero `--check` and a missing file refuse `profile-drift` before the plan |
| head-mismatch | checkout not at `--sha`; a commit the clone can't fast-forward to refuses `fetch-failed` |
| usage | twelve bad argvs exit 4 with no log and no lock |
The step log check (`STEP`) runs in every test that reads the log: each line must be a step line or the
result line, and none may hold the stack user's output (`stack-release:`, `PASS `, `FAIL `, `REHEARSAL`,
the fake rehearsal's text).
## Mutants
`~/darkwing-scratch/r233/mutants.sh`, not committed because it edits the worktree. It has the brief's six
mutants and two for sage-1650's mapping. The method is row 232's: a checked backup, exact replacements, a
syntax check, the named killers, then the file put back with its sha256 checked. One control runs first:
the whole `stack-upgrade.test.mjs` with no change. `DRY=1` matched each replacement once on the candidate
(2026-10-21).
| Mutant | File | Killers |
|---|---|---|
| m1 drop the flock | stack-upgrade.sh | `a second run while the lock is held`, `sequence 3:` |
| m2 release the lock when the rollback starts | stack-upgrade.sh | `sequence 3:` |
| m3 skip A1's open-activation check | stack-release.sh | `A1 refuses open-activation` |
| m4 skip the rehearsal | stack-upgrade.sh | `the rehearsal runs after the plan`, `sequence 1:` |
| m5 accept a migrating target without `--migrate` | stack-release.sh | `migrate-required` |
| m6 write the args file before confirm | stack-upgrade.sh | `sequence 7:`, `sequence 1:` |
| m7 `--no-rehearsal` accepted with a backup | stack-upgrade.sh | `refuses rehearsal-required` |
| m8 drop the `rehearsal-skipped` line | stack-upgrade.sh | `a real rehearsal with no backup` |
## Deviations
D1. The brief has A1 refuse `open-activation` for an open `AWAITING-UNITS` or a non-empty `interrupted`
set. The interrupted case refuses with row 232's existing code, `interrupted-run`; only `AWAITING-UNITS`
gives `open-activation`. Both reach root as `plan-refused`, exit 3 (sequence 6, the open-activation test).
D2. Root copies one line of the stack user's output into its log, `BACKUP_STORE`, on sage-1653. It
matches a fixed pattern first, takes the last match, logs `BACKUP_STORE none` otherwise, and decides
nothing on it.
D3. The stack user's steps carry no `--instance`. The driver takes `--instance dev` only, row 190's default
instance, which the stack user's commands select with no flag. A named instance's units are a later row.
D4. Most tests fake the rehearsal in the fake `sudo`: the fixture's stub copies can't pass a real one. Two
tests run the real rehearsal (`faults/real-rehearse`): no-backup and backup-last. `stack-release.test.mjs`
covers `--backup last` itself.
D5. Any nonzero `--plan` exit refuses `plan-refused`, exit 3, whatever the code. The stack user's output
above it names the refusal; root doesn't parse it.
D6. `rollback --restore ACT` is accepted without `--events`, as `stack-release.sh rollback` accepts it.
The brief's usage pairs them. `--events` without `--restore` is a usage error.
D7. The refusal of any `STACK_UPGRADE_*` seam when run as root has no test, because the tests don't run
as root. It is three lines at the top of the script.
D8. The fake `sudo` keeps the test's environment, where real `sudo` resets it. The driver passes
everything the stack user needs through `env PATH=...` and `-H`, and the fake checks that form.
D9. Exit 3 means nothing stopped, not nothing changed. A refusal after `fetch` leaves the stack user's
clone fast-forwarded and the new copy made. Both are the user's own state, and the next run reuses them.
TOOLS.md and the run sheet say so.
D10. The brief's usage has neither `--no-rehearsal` nor `--rehearse-port`. Both come from sage-1650 and
sage-1652.
## Decisions
- `rollback` has no HEAD check. Its target is an older copy than root's checkout, which already exists.
It fetches, copies and rehearses nothing (sequence 7).
- The new args file is written next to the old one (`.dev.args.XXXXXX`, same mode) before A2, so a full
`/etc` refuses before anything stops. It moves into place after A5. A2f and the rollback remove it; a failed move keeps it and names it.
- Every stack user step gets `--copies` explicitly, so the copies directory comes from the args file's
`--home`, never from the user's environment.
- The stack's user does the fetch, the fast-forward and the copy, so root never runs git on a tree that
user can write.
- A4 is the render, and `case2-unit.sh` starts the bus, then the Console. The forward path has no
`systemctl start` of its own; only A2f does.
## Run environment
Worktree `/mnt/storage/scratch/darkwing-r233`, `TMPDIR=/mnt/storage/scratch/tmp/r233`, node from PATH. One
whole `stack-upgrade.test.mjs` run takes about 56 s.
## Results
### Singles, run 1, 2026-10-21, Sage's slot (sage-1689)
`~/darkwing-scratch/r233/singles.sh` on `darkwing/row233-r1` (base e65a3ab4d, uncommitted tree), 03:48:20Z
to 03:59:00Z. These are the test files that read a changed script. `stack-instances.test.mjs` reads
`stack-release.sh`, so it joined the three named in the plan.
| File | Result |
|---|---|
| `stack-upgrade.test.mjs` | 20 pass, 0 fail |
| `stack-release.test.mjs` | 194 pass, 1 fail |
| `stack-release-external.test.mjs` | 25 pass, 0 fail |
| `stack-instances.test.mjs` | 23 pass, 0 fail |
The failure was my test, not the script. "row 233: --backup last takes the newest backup..." compares
`untouched(e)` before and after the run, and `untouched` includes `existsSync(e.log)`. The test took its
before snapshot ahead of `rlog()`, which writes the log, so that flag went from false to true. Every hash in
the snapshot matched. The fix takes the snapshot after `rlog()`. No other row 233 test calls `untouched`.
### Singles, run 2
`~/darkwing-scratch/r233/rerun.sh` on aacfa2d7a, Sage's slot (sage-1695), 04:13:05Z to 04:21:28Z:
`stack-release.test.mjs` whole, 195 pass, 0 fail, 0 skipped. The fixed test passed. The other three files
did not change after run 1. The gate and the mutants run next, in Sage's slot.
### Gate and mutants, 2026-10-21, Sage's slot (sage-1704)
`~/darkwing-scratch/r233/gate.sh` then `mutants.sh`, chained behind `busy.sh`, on d99166ade (code
aacfa2d7a; d99166ade changes only this file). Counts are an awk sum over `gate-r1.out`.
| Part | Time (UTC) | Result |
|---|---|---|
| cli suite (`node --test tests/*.test.mjs`) | 04:29:01-04:37:27 | 616 pass, 0 fail, 0 skipped |
| the 10 `scripts/test-*.sh` | 04:37:27-04:39:38 | selftests 429 passed, 0 failed; `test-queue`'s node part 156 pass, 0 fail |
| gate total | | 1201 pass, 0 fail, 11 suites, `GATE_PASSED` |
| c0 control | 04:39:38-04:40:35 | exit 0 |
| m1-m8 | 04:40:35-04:41:25 | 8 of 8 killed, every one exit 1 |
Every restore passed its sha256 check, and `git diff` was clean at d99166ade afterwards. The EXIT trap
removed the `darkwing-r233_default` network.
N1, found in the process sweep after the run: a `sleep 60` was left with its cwd in `packages/cli`.
`stack-upgrade.test.mjs:312` spawns `flock LOCK sleep 60`, and its `t.after` SIGKILLs only the flock, so
flock's child outlives the test by up to 60 s. It exited on its own. Row 232's T20
(`stack-release.test.mjs:370`) spawns its holder `detached` and kills the group with `process.kill(-pid)`.
The fix is the same two lines. That test is one of m1's killers.
Sage ruled it a fix now (sage-1707), and 08db6ac3f makes it, test only. After the gate's d99166ade
the only change outside this file is that test:
```
$ git diff --stat d99166ade 08db6ac3f
packages/cli/tests/stack-upgrade.test.mjs | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
```
The gate stands on Sage's ruling. The rerun is the whole `stack-upgrade.test.mjs` singly plus m1, in
Sage's slot, with a sweep for a leftover `sleep` afterwards.
+173
View File
@@ -0,0 +1,173 @@
# Row 240 (I4): root's host check, its pins and the boot unit
Brief: `docs/plans/2026-10-20_installer-rows.md` section I4, blob `4a5a65020f00` (commit b00d7d94d, rev 2665,
the pin row 240 carries; the I3 carry items came in with it). Sage split it out of I3 (sage-1650). Gitea #1746, Vikunja P19 #432. Owner Darkwing, reviewer Filbert, gate owner Sage.
Design: row 195 DESIGN r6 B2, `agents/darkwing/work/queue-195/DESIGN.md` at b7239bb66, :262-303 (the script,
the three callers) and :839-860 (files, test cases, mutants). Where the brief and r6 differ, the brief wins.
## Files
Drafted on `darkwing/row240-r1` from d99166ade (row 233's candidate) while row 233's gate ran, then applied
as one patch onto row 233's landing, 57f59e0c3, on `darkwing/row240-r2` (sage-1753). It applied cleanly; between
the two bases the only change on this row's paths is 233's N1 test fix in `stack-upgrade.test.mjs`. Row 234
then landed as c771a36f4 on 57f59e0c3. Sage preferred that the gate run with it in the tree (sage-1773), so both
commits were picked onto c771a36f4 as `darkwing/row240-r3`. That was a clean pick: the 42 paths changed from
57f59e0c3 to c771a36f4 share none with this row's 9.
- `scripts/case1-pins.sh` (new, 0644): `PIN_PROFILES_VERSION`, `PIN_PROFILE_SHA256`, `PIN_BUBBLEWRAP_VERSION`,
r6 section 0's values.
- `scripts/case1-hostcheck.sh` (new, 0755): the seven items and `--check`, `--enforce`, `--upstream`.
- `packages/cli/systemd/mosaic-hostcheck.service` (new): the boot oneshot.
- `packages/cli/tests/case1-hostcheck.test.mjs` (new, 16 tests). The cli suite's `tests/*.test.mjs` glob picks
it up.
- `docs/guides/host-run-sheet.md` section 4: the install lines that replace "Install them from row 240's run
sheet lines once it lands", plus rollback, drift recovery and `--upstream` before a re-pin window. Sections
3, 6, 8 and 10 for the I3 carry items below.
- `scripts/stack-upgrade.sh` and `packages/cli/tests/stack-upgrade.test.mjs`: I3 N1 below.
- `docs/TOOLS.md`: a new subsection "Root's host check (`scripts/case1-hostcheck.sh`, row 240)" after the
upgrade driver's.
Brief line 257 said I3's `stack-upgrade.sh` doesn't change. Sage's ruling sage-1727 widened the row to it for
N1 alone. Its default `STACK_UPGRADE_HOSTCHECK=/usr/local/lib/mosaic-stack/hostcheck.sh` is where the run
sheet installs this script.
## Carried from I3
Filbert's row 233 approve (Gitea 29922) left five notes. Sage ruled N1, N4 and N5 into this row and N2 and N3
to DEFERRED.md (sage-1727).
| Note | Change |
|---|---|
| N1: `stack-upgrade.sh` exits 1 `args-failed` when the args move after A5 fails, but exit 1 means the old copy runs | The move's failure exits 5 `args-failed`. The header, TOOLS.md's step 5 and exit line, and run sheet section 6's exit list name exit 5 and what root does: move the new file the result line names into place. Test `row 240: I3 N1: a failed args move after A5 exits 5 ...` makes `/etc/mosaic-stack` read-only at confirm and checks exit 5, the step names, `args rc=1`, the result line, both units on the new copy, the old args file's bytes and the new file beside it. The step log regex takes `exit=5`. Mutant M8. |
| N4: section 3's `npm install` has no npm on root's PATH, and the `--claude` path is unnamed while `node_modules/.bin/claude` is a symlink `case2-unit.sh:140` refuses | The install line runs with section 2's pinned node first on PATH (the postinstall runs `node` too). The `--claude` path is `node_modules/@anthropic-ai/claude-code/bin/claude.exe`, and a `--version` line catches the postinstall's 500-byte stub. |
| N5: sections 8 and 10 lack precheck lines | Each gets "Precheck line: none" and its missing line. |
Exit 5 is new rather than 2 because 2 means both units are stopped, and here both run. Exit 4 stays usage.
The `claude.exe` path comes from the 2.1.304 package itself: `npm view` gives `bin.claude` as `bin/claude.exe`,
and its `install.cjs` hardlinks the platform binary over that placeholder, or copies it across devices, then
sets 0755. I downloaded the tarball into `/mnt/storage/scratch/tmp/r240/cc304` and read it; nothing ran. The
run sheet says a newer pin needs the same check.
## The brief, item by item
| Brief line | Where |
|---|---|
| 247 `case1-pins.sh` holds the three pins | `scripts/case1-pins.sh`; the pins test checks the values against r6's |
| 248-249 seven items, `--check` exits 0 or 1 | `items()` and the final dispatch; the check tests |
| 250-251 `--enforce`: `apparmor_parser -R`, `root:root 0700`, `DRIFT`, a log line, exit 0 | `enforce()` and `stop_bwrap()`; the four enforce tests |
| 252 `--upstream` compares the current deb to the pin | `upstream()`; the upstream test |
| 253 the boot oneshot | `mosaic-hostcheck.service`; the boot unit test |
| 254-255 fakes for every host tool, r6's seven case groups, seven mutants | the test file; Tests and Mutants below |
| 257 the I3 run sheet item points here | run sheet section 4 |
## Tests
Fakes on PATH, each logging its argv to `$HC/calls.log`: `aa-status`, `dpkg`, `dpkg-statoverride`,
`apparmor_parser`, `systemd-run`, `stat`, `chown`, `chmod`, `apt-get`, `dpkg-deb`. `host(t)` builds a scratch
root (`/etc`, `/proc/sys`, `/usr/bin/bwrap`, `/var/log`) and a lib dir with a copy of the script and fixture
pins that hold the fixture profile's sha256. Every run passes `--root`.
r6's case groups (:843-856) against the tests:
| r6 case | Tests |
|---|---|
| `--check` exits 0 with all seven, 1 with each failing alone, its line naming it | `check: exits 0 ...`, `check: exits 1 with each item failing alone ...`, plus one test per item for the item's edge cases |
| item 3: a second file naming `bwrap` as a word, `/{usr/,}bin/bwrap` included; passes with the profile alone | `check: item 3 ...` |
| item 4: `unpriv_bwrap (enforce)` or a trailing space fails; the argv carries `-p NoNewPrivileges=yes` and the 2a' properties | `check: item 4 ...` checks the whole argv |
| item 7: the floor at 732 fails | `check: item 6 ...; item 7 ...` (733 passes as the control) |
| `--enforce` exits 0 on a pass and every failure; unload, close, `DRIFT`, one log line; `DRIFT` present with every item passing still closes | `enforce: a pass ...`, `enforce: each failure ...`, `enforce: with DRIFT present ...` |
| `--enforce` never calls `dpkg-statoverride` with `--update`, `--add` or `--remove` | `enforce: no unload ...; never writes the statoverride` (every call is `--list /usr/bin/bwrap`) |
| a bubblewrap version other than the pin is a line, not a failure | `check: a bubblewrap version other than the install pin ...` |
| `--upstream` against a differing deb prints the difference, exits 0 | `upstream: ...` (also: a match, a deb with no profile, a failed download) |
| the unit: `Type=oneshot`, `After=apparmor.service`, `[email protected]`, runs `--enforce` | `boot unit: ...` |
The last test also checks that the repo script reads `scripts/case1-pins.sh` beside it, that bad usage exits
2, and that a non-root run without `--root` refuses.
Singles are the two files that run a changed script: `case1-hostcheck.test.mjs` and `stack-upgrade.test.mjs`.
Sage asked (sage-1759) whether `stack-release.test.mjs` or `stack-release-external.test.mjs`, which share
`stack-release-fixture.mjs`, run `stack-upgrade.sh` or `case1-pins.sh`. They don't. `grep -cF` for
`stack-upgrade`, `case1-pins`, `case1-hostcheck` and `hostcheck` gives 0 in all three files; the control
`stack-release.sh` gives 2 in the fixture and 8 in `stack-release.test.mjs`. `scripts/stack-release.sh` names
`stack-upgrade.sh` once, in a comment (:2061). The gate's cli suite runs both files anyway.
## Mutants
r6's seven (:858-860) and one for I3 N1, scripted in `~/darkwing-scratch/r240/mutants.sh` with r233's method: a checked backup,
an exact replacement that must match once, `bash -n`, the named killer by `--test-name-pattern`, the restore
and a sha256 check. A dry run matched all eight once each. Results below.
| Mutant | Change | Killer |
|---|---|---|
| M1 | `--enforce` exits 1 after the stop | `enforce: each failure exits 0 ...` |
| M2 | `[ ! -e "$DRIFT" ] \|\| had=1` becomes `true \|\| had=1` | `enforce: with DRIFT present ...` |
| M3 | item 1's `dpkg -S` becomes `owner=""; rc=1` | `check: item 1 fails when a package owns ...` |
| M4 | item 3's `grep -rlw bwrap` becomes `grep -rlF /usr/bin/bwrap` | `check: item 3 ...` |
| M5 | item 4's `[ "$label" = "$LABEL" ]` becomes a substring match | `check: item 4 ...` |
| M6 | item 4 drops `-p NoNewPrivileges=yes` | `check: item 4 ...` (argv) |
| M7 | item 7's `-gt` becomes `-ge` | `check: item 6 ...; item 7 fails with the floor at 732 ...` |
| M8 | `stack-upgrade.sh`'s `finish 5 args-failed` becomes `finish 1 args-failed` (I3 N1) | `row 240: I3 N1: a failed args move ...` |
## Deviations
- Repo name `scripts/case1-hostcheck.sh` (r6), installed as `/usr/local/lib/mosaic-stack/hostcheck.sh`, the
name in brief line 239 and I3's `stack-upgrade.sh` default. r6's apt hook line says `case1-hostcheck.sh`;
the run sheet's hook uses the installed name.
- `stat` is a fake too. r6 lists seven fakes; a test can't make a root-owned file, so item 5's `stat` reads a
fixture.
- Item 7 with no Console unit installed reads the port as 1023 and says so ("1023, because no Console unit
is installed"). Until I2 installs the unit the floor must be above 1023. A unit with no `--port` fails.
- Item 4 runs under `timeout 60`. A hung `systemd-run` would otherwise hang the apt hook and the boot unit.
## Decisions
- Fail closed where r6 is silent:
- pins with no 64-hex sha fail item 1;
- `dpkg -S` exit 1 (no owner) passes, exit 0 fails with the owner, any other exit fails with the code;
- a failed `aa-status` fails item 2 and counts as loaded, so `--enforce` tries the unload.
- `--root DIR` is the test seam (r6 :840). Root refuses it; a non-root user needs it. Both exit 2. Root runs
with `PATH=/usr/sbin:/usr/bin:/sbin:/bin`.
- `DRIFT` is append-only (`<stamp> <failed line>`), and `--enforce` never removes it. The log line is written
only when the stop runs: `<stamp> enforce stop items=<ids or none> drift-before=<0|1> unload=<rc=N|no-file|skipped> chown=N chmod=N`.
- `--upstream` exits 1 when the download or the unpack fails, since it then compared nothing. A deb without
the profile file is a difference (exit 0), per r6 :302-303.
- The bubblewrap note reads the version from `dpkg -s bubblewrap`.
- The run sheet's recovery steps (`chown root:mosaic`, `chmod 0750`, `apparmor_parser -r`, then `--check`
before root removes `DRIFT`) are mine. r6 says only that root removes `DRIFT` in a reviewed step.
## Run environment
Worktree `/mnt/storage/scratch/darkwing-r240b`, branch `darkwing/row240-r3` on c771a36f4, with `node_modules`
linked to the canonical checkout's. Node v26.8.1, kernel 7.2.2-1-cachyos, 16 CPUs, TMPDIR
`/mnt/storage/scratch/tmp/r240`, compose project `darkwing-r240b`. Sage's slot (sage-1773, 06:16Z). Each run is
one chain behind `busy.sh`, launched from `/`: `~/darkwing-scratch/r240/run-all.sh` checks the HEAD pin and a
clean tree, runs the singles, then the mutants (8 killed, none surviving, the tree clean after), then the gate
(the cli suite and every `scripts/test-*.sh`), each stopping the chain on failure, then sweeps `/proc` cwd links
for anything left in the r240 trees.
## Results
Run r1, 06:17:10Z on 47ec9fe2f, failed at the singles. `case1-hostcheck.test.mjs` passed 16/0/0, and
`stack-upgrade.test.mjs` went 20 pass, 1 fail. The failure was the N1 test's own `t.after` hook, which ran
`chmodSync(e.etc, 0o755)` after the fixture's cleanup had already removed the scratch root, so it threw ENOENT.
Every assertion in the test body had passed, exit 5 included. The fixture's cleanup runs `chmod -R u+w` on the
root before `rmSync`, so the 0555 dir never needed the hook. b45852daa drops it and changes nothing else. The
mutants and the gate didn't run. The sweep found nothing left. Sage counted r1 as this slot's one failure
(sage-1788).
Run r2, 06:20:41Z to 06:33:28Z on b45852daa, passed. The counts come from awk over the .out files
(`~/darkwing-scratch/r233/sum.sh`):
| Step | Result |
|---|---|
| `case1-hostcheck.test.mjs` | 16/0/0 |
| `stack-upgrade.test.mjs` | 21/0/0 |
| c0 control | exit 0 |
| M1-M8 | 8/8 killed, none survived, every restore's sha256 matched, tree clean after |
| `packages/cli` suite | 639/0/0 |
| `scripts/test-*.sh` | auth 15, conductor 17, config 31, discord 78, extension-package 18, foundation 44, land-work 86, queue 156 node + 28 selftest, release 14, task 98; 585 pass, 0 fail |
| Gate total | 11 suites, 1224 pass, 0 fail |
| Sweep | nothing left in the r240 trees; the compose network was removed |
This commit adds only these BUILD.md lines. No test reads BUILD.md.
+310
View File
@@ -0,0 +1,310 @@
# Row 234 (#436a row a, Gitea #1740): host enrollment, store records and bus schema 3i to 3j
Gizmoduck, 2026-10-21. Reviewer Duckworth, gate owner Sage. Lands by 10-25, before the .53 and .54 installs
(mos-1134).
- Brief: `docs/plans/2026-10-20_host-credentials.md`, section "Machine grant row a: enrollment, store records
and migration 3i-3j", blob 64451be5 (queue rev 2617).
- Design: `agents/gizmoduck/work/machine-grant/DESIGN.md` r6, f9039682e on `gizmoduck/machine-grant-design`,
approved by Duckworth (duckworth-0064, REVIEW-r6.md at 8ea792256). Not reopened here. Cites "r6:" are lines
of that file.
- Grant event rule: #436b section 7, fa5658941 on `megavolt/row199`,
`agents/megavolt/work/436b/DESIGN.md:431-490`, file sha256
96510b83778e79a71b33d0d5f6fd0c806dfe74f6de315ff4a9f1fbe5620f6c69, approved by Duckworth (duckworth-0067).
- Base: origin/refactor 8fede5fc6, with rows 199 (cb5fdb064) and 210 (f5753c0e8) landed. Branch
`gizmoduck/row234`. Code cites are `path:line` at 8fede5fc6 unless they say otherwise.
## Names: r6 against section 7
r6's `grant.*` and `host.enroll.*` names are `body.action` values on `action.allowed` events, not event
kinds. The `events` kind CHECK isn't rebuilt. Four r6 names move under `grant.`:
| r6 | This row |
|---|---|
| `host.enroll.step` | `grant.host.enroll.step` |
| `host.enrolled` | `grant.host.enrolled` |
| `host.checked` | `grant.host.checked` |
| `host.removed` | `grant.host.removed` |
Only the consume record carries `decision`: `{operation: 'grant.host-enroll', decision}`. Every other
guarded body names the decision `grantDecision`, so r6's `decision` field in step, enrolled and failed bodies
is `grantDecision` here. The actor of every guarded event is `actor_role` NULL with `actor_run` 'grant',
and `body.by` names the caller the broker checked.
## Split (r6 section 12 Q-S4)
Row a builds:
1. **Schema 3i to 3j.** `migrations/3i-3j.sql` adds `hosts` and `host_grants` (r6 S1, primary keys
(business, host) and (business, agent, host)) and the guard trigger: an `action.allowed` whose
`body.action` or `body.operation` GLOBs `'grant.*'` aborts unless the actor is (NULL, 'grant').
`SCHEMA_VERSION` 3j, a migration test with a step that fails partway.
2. **Grant event writer.** A private session-less broker writer for guarded events. `#event` doesn't change.
`recordEvent` refuses `reserved-event` for the guarded namespace.
3. **The `grant` raise frame.** `decision.raise` takes `grant` only with `credential.mint`, only from the chief
(`grant-chief-only`), and in row a only with `method: 'host-enroll'`. Any other method refuses
`invalid-request` until row b adds `host`. E1's checks, the `ssh-keyscan` set compare before the raise
transaction, and the rendered block.
4. **Enrollment executor.** `#consumeGrant`, the start on the human's cli yes (X1), E2 steps 1 to 4, the
installer key at `<configDir>/secrets/hosts/<business>/<hostId>/installer`, the failure cleanup, and the
start reconcile for enrollments: the `registry` step undo, and a resolved-yes unspent enrollment ending
`grant.failed interrupted` (X3).
5. **`hosts.check` and `hosts.remove`** (E4), with `mosaic hosts status|check|remove`.
6. **`tools/host/`**: `mosaic-keys`, `mosaic-keys-write`, `mosaic-keys.sudoers`, `mosaic-keys.sshd.conf` (E3).
7. **`host.run` on the store** (X4 first and third bullets): it reads the `hosts` and `host_grants` records
and the Approve record check, and at start the runtime gives `Credentials` a reference per `host_grants`
record. The loader refuses `hosts` at the top level and in a role (S1). Row 210's tests move to store
fixtures.
8. **Reader filter** (duckworth-0067 N1): every grant-evidence reader in this row (status, the start
reconcile, `openGrantDecisions`) filters `kind = 'action.allowed' AND actor_role IS NULL AND actor_run =
'grant'`, with a test that an `action.refused` look-alike and a session-written one are ignored.
`hosts.remove`'s `hostLineLeft` reader is one of them (duckworth-0079 M4).
Row b keeps the `host` method's raise binding (R4), the grant executor (X2), the grant rows of X3, X5,
`Credentials.register`/`claim`/`abortClaim`, revoke (section 6) and `sweep`. So row a has no way to make a
`host_grants` record outside a test fixture, and from its landing to row b's no agent holds a host grant
(r6 N9, the run sheet strips row 210's keys).
Tests in row a (r6 section 10): G1 and G5 for the `host-enroll` raise, G6 and G10 for the enrollment
executor, G11's enrollment cases, G14, G15, G16, G17, G18, G20, G21 (its `host-line-left` case reads an
event fixture until row b writes `hostLineLeft`), G25, the migration test, the trigger cases, the reader
filter test, and row 210's T-tests on store fixtures. Row b: G2 to G4, G7 to G9, G11's grant cases, G12,
G13, G19, G22 to G24, G26 to G28.
## Early check items (duckworth-0079, REVIEW-build-split.md 10672772e on `duckworth/reviews`)
Moves into row a:
- M1. G4 for the `host-enroll` block (r6:240-242): a question can't forge the block, and the 4096-character
limit holds with the block appended. G4 for the `host` block stays in row b.
- M2. The tag function (r6:265-266), because E2 step 2 tags the installer line. G24's first sentence moves
here: names of 64 characters with `.` and `_` give distinct tags of the fixed length. The `sweep` half
stays in row b.
- M3. A store-fixture test that `host.run` refuses `host-not-granted` for no record, an unresolved decision,
a decision resolved by an arbiter, the tester or an agent or not via cli, a consume record with operation
`grant.host-enroll` in place of `grant.host`, and an unspent decision (X4 first bullet, r6:593-596). A
restart test that a `host_grants` fixture gives `Credentials` a reference (X4 third bullet).
G21's `host-line-left` case:
- F1. The fixture goes through this row's private grant writer in the real shape: `action.allowed`, actor
(NULL, 'grant'), `body.action` 'grant.failed', `body.host`, `body.hostLineLeft`. No raw SQL around the
trigger.
- F2. `hosts.remove` reads `body.hostLineLeft` from any guarded body, not only `grant.failed`.
- F3. Decided: a guarded body that carries `hostLineLeft` also carries `host`. Row b adds `host` to
`grant.revoke.closed`, so its body is `{tag, host, hostLineLeft}` (r6:716 has `{tag, hostLineLeft}`).
The reader matches on `body.host`, and doesn't map tags back to hosts through `grant.step`. A direct
field is one equality in SQL, and a tag lookup would add a second reader that the N1 filter also has to
cover.
- F4. Row b carry: "no later `sweep` cleared it" (r6:403) needs `grant.sweep`, which only row b writes. Row
b adds the sweep clear to `hosts.remove` with a G21 case, and its G8 or G13 shows `hosts.remove`
refusing after a real `hostLineLeft` write, not only after the fixture. Until then row a's check is "any
guarded `hostLineLeft` for this host refuses".
## Mutants (r6:838-883, numbered here)
Row a, killed on this row's tree:
| # | Mutant |
|---|---|
| A1 | drop the chief check |
| A2 | start the executor on a `credential.mint` with no `grant` |
| A3 | start it on an arbiter resolve |
| A3b | start it on a `cli` yes by someone other than the human (duckworth-0083 N8) |
| A4 | start it on `no` |
| A5 | skip the keyscan compare |
| A6 | compare only one key |
| A7 | write the private key into the event body |
| A8 | drop the "not yet enrolled" check in E2 step 1 |
| A9 | drop the `hosts` primary key |
| A10 | skip the start check for a resolved, unspent enrollment decision |
| A11 | `host.run` reads the business file instead of the grant record |
| A12 | build the tag without the business |
| A13 | drop `from=` in the writer |
| A14 | let the writer touch a line outside the tag pattern |
| A15 | drop `O_NOFOLLOW` on the user file |
| A16 | drop one group from the writer's refused list |
| A17 | drop the user pattern check before the path use |
| A18 | treat a `privileged` entry as enough without `users` |
| A19 | let the writer accept a sudo user not in `privileged` |
| A20 | trigger: drop the `body.operation` arm, so the consume record passes from a session |
| A21 | trigger: accept actor_role NULL with any actor_run |
| A22 | a grant reader drops the `kind = 'action.allowed'` or actor filter. The runner splits it: A22a drops the kind, and G21's `action.refused` look-alikes kill it. A22b drops the actor, which the trigger can't stand in for, because 3i-3j adds the trigger without checking old rows. Its killer is the grant-migration test "A22b, sage-1703": a session-written `grant.host.removed` with `hostLineLeft` and a `grant.host-enroll` row from 3i, migrated to 3j, where `hosts.remove` still removes the host and `reconcileGrants` ends nothing |
| A23 | `host.run` skips the Approve record check (any `host_grants` record is enough) |
Row b keeps the rest of r6's list: the `ready` check, the three hash compares, the definition check, the
install and record order, `remove` on a verify failure, both start sweeps, the reconcile's remote `remove`,
the `grant.step` before install, revoke leaving the record, `register` before `issue`, the abort check and
the `await` before `issue`, revoke and an unreferenced claim, `sweep` and any `mosaic-` line, revoke's host
step order, claim before consume, the resolved unspent grant decision, an async `commit`, `hosts.revoke`
through `credential-verbs.mjs`, the revoke marker reconcile, the upsert, the undo past `grant.revoked`, the
abort after step 5 and a missing `bus.tester` as dev.
## Broker functions this row touches (sage-1677)
For row 236 (Megavolt), which rebuilds on this landing. Old lines are `packages/bus/src/broker.mjs` at
8fede5fc6, new lines are at the last commit that changed this file (they moved by 10 after `runEnrollment`
went in). The planned list is kept in this file's history (827b89ea6).
Changed:
- `constructor` (:279): one more option, `hostGrant`, the object `host-grant.mjs` builds.
- `recordEvent` (old :1243-1251, new :1606): one more refusal. An `action.allowed` whose `body.action` or
`body.operation` is a `grant.*` name refuses `reserved-event`.
- `request()` (old :1362, new :1726): two routes ahead of the sync path, a `decision.raise` with `grant` to
`#raiseGrant` and the `hosts.*` verbs to `#requestHosts`. Both are async.
- `requestHost` (old :1712-1795, new :2079): the business-file lookups (old :1730-1736) become one
`#hostAccess` call, in the same transaction as `#agent`. `commands` and `entry` come from it.
- `#dispatch` (old :1796, new :2160) gets a fifth parameter, `enroll`, which only `#raiseGrant` passes.
- `decision.raise` (new :2244): `grant` joins the allowed keys, and `grant` without `enroll` (or the
reverse) refuses `invalid-request`. The `host.sudo` block reads `#hostAccess` in place of `b.hosts`
and the role's `hosts`, and an undefined `target` still refuses `host-unknown`. `validArgv` and
`validStdin` now run after the access check, so a malformed argv for an ungranted host refuses
`host-not-granted` first. The enrollment block is appended to the question before `string(question)`,
so the 4096 cap counts it (M1). The decision's raise body carries `grant: enroll`.
- `decision.resolve` (new :2409): when `op` is `resolved`, `#approvedGrant` reads the decision, and a
`host-enroll` grant starts `#runEnrollment` from `setImmediate`, after the transaction commits. The
result adds `grant: 'started'`.
New, all private but two:
- `#grantEvent` (:657), the writer of guarded events. It runs the secret check, then `writeGrantEvent`.
- `#approvedGrant` (:667), `#consumeGrant` (:678). `#consumeGrant` sits after `#approvedGrant`, not next
to `#consumeAuthority` as planned.
- `#hostRecord` (:685), `#hostAccess` (:692), `#grantNotice` (:708), `#enrollFailed` (:736),
`#runEnrollment` (:750), `#openGrantDecisions` (:823).
- `runEnrollment` (:745), public, a trusted entry to `#runEnrollment` for G10. No verb reaches it.
- `reconcileGrants` (:833), public, which `runtime.mjs` calls once at start.
- `#raiseGrant` (:866), `#hostsCaller` (:886), `#requestHosts` (:900), `#hostLinesLeft` (:964).
Module level, new (:184-230): `GRANT_RUN`, `grantName`, `GRANT_EVIDENCE` (the N1 filter as SQL),
`registrySha256`, `writeGrantEvent` (exported, the one writer of guarded events, which test fixtures use
for F1), `HOSTS_VERBS`, `fingerprintOf`, `INSTALLER_COMMENT`.
Unchanged: `#event`, `#spent`, `#consumeAuthority`.
Other `src` files:
- `bus/src/host-grant.mjs`, new: the keyscan, the installer key pair, and `hosts.check`'s `list`.
- `bus/src/runtime.mjs`: builds `hostGrant`, calls `reconcileGrants` before the server starts, and adds
the `host_grants` references before `new Credentials` (X4 third bullet).
- `bus/src/host-run.mjs`: `runHost` takes `command`, sent as is in place of the sudo argv, for the
installer's `list`.
- `bus/src/server.mjs`: the `hosts.*` verbs and a raise with `grant` go down the async path.
- `bus/src/store.mjs`: the 3i-3j step, `SCHEMA_VERSION` 3j. `bus/src/credentials.mjs` exports
`privateDir`.
- `business/src/business.mjs` and `business/src/hosts.mjs`: the loader refuses `hosts` (S1).
`checkHosts` and `checkHostGrants` are gone.
- `cli/src/hosts.mjs`, new, routed from `cli.mjs` and `scripts/mosaic`.
## Decisions taken in the build
- **`#hostAccess` binds the agent and the host.** X4 says the Approve record check passes when the
decision is an approved `host` grant with a `grant.host` consume. The check also requires the grant's
`agent` to equal the session's role and its `host` to equal the requested host. Without that, an
approved grant for one agent and host would vouch for any `host_grants` row that names its decision.
Cost: none for row b, whose executor writes the row from the same grant.
- **Store fixtures are written the real way.** `bus/tests/host-fixtures.mjs` inserts the decision rows by
SQL, but every guarded event goes through `writeGrantEvent`, so the trigger sees it (F1). An enrollment
fixture writes `grant.host.enrolled`, or the start reconcile reads it as interrupted.
- **The resolver test for host kinds is gone.** `business/tests/hosts.test.mjs` lost its D5 case, because
the loader now refuses `host.<id>` credential refs. The `isHostKind` filter in `business/src/resolve.mjs`
can't be reached now. I left it in, since row b may want it back, and it's one line.
- **Store before Credentials.** The runtime already opened the store before `Credentials`. The
`host_grants` references depend on that order, and the comment there says so.
- **CLI.** `mosaic hosts` prints tab lines, and `--json` prints the bus rows. Usage errors exit 4 and bus
refusals exit 2, as `mosaic credentials` does. There is no enroll command: enrollment is the chief's raise.
- **Host tools tests live in the bus suite.** `bus/tests/host-tools.test.mjs` loads the writer by path
through a Python harness under `python3 -B -I`. The harness passes its own key dir, config dir, owner
uid and user, group and sudo lookups, and runs `mosaic-keys` from a copy with stub `sudo` and `logger`.
The gate runs the bus suite, so G16 and G25 run there. The real `pwd`, `grp` and `sudo -l` paths are
checked on the #433 VM, not here. The test also runs `visudo -c` on the sudoers file and `sshd -T` on
the drop-in.
- **Writer details the design left open.** The `from` file must hold exactly one value matching
`[0-9A-Za-z.:/*?!_-]+(,[0-9A-Za-z.:/*?!_-]+)*`, so it can't close the quote. `users`, `privileged` and `from` must be
regular, owned by the owner uid and not group or world writable, and are opened without following a
symlink. Lines starting with `#` are comments. A refusal exits 1 with its code on stderr and in syslog.
`add` of a tag that is already there exits 0 and writes nothing.
- **`runEnrollment` is public for G10 (duckworth-0081 B1).** G10 needs a second start for a spent
decision, and the resolve path can't give one: a second resolve refuses `decision-closed`
(broker.mjs:2416) before the executor runs. So
`#runEnrollment` returns its code (`enrolled`, `host-enrolled` or the refusal), and a public
`runEnrollment` calls it, as `reconcileGrants` is public. The test checks a spent
decision gives `decision-consumed`, and an open or agent-resolved one gives `not-authorized`, with
nothing written. That second case also reaches the `via` and `by` checks in `#approvedGrant`, so A3 is
no longer vacuous (N1).
Who can call it (sage-1692): only code holding the `Broker` object, which is the bus process and the
tests. The one `new Broker` in `src` is `runtime.mjs:126`, and no `src` file calls `runEnrollment`;
the resolve path calls the private `#runEnrollment` from `setImmediate` (broker.mjs:2445). A seat
holds no `Broker`. It reaches the bus only over `broker.sock`, where `server.mjs` calls a fixed set of
methods by name (`request` at :134 and :140, `identity`, `disconnect`). `request()` routes on the verb
through the `#dispatch` switch, and neither file looks a method up by name (no `this[` in either), so no
verb string can reach `runEnrollment`.
- **A `sudo -l` that times out refuses `user-sudo` (N2).** `_sudo` returns an empty answer on
`TimeoutExpired`, which isn't the expected line, so `main` logs the refusal and exits 1.
- **Keyscan comment (N3).** A scan that can't start refuses `host-failed`. One that runs and finds no key
returns no lines, and the raise refuses `host-key-mismatch`. The comment in `host-grant.mjs` now says so.
- **No tag on in-run `grant.failed` (N4).** r6:578 tags only the `interrupted` one, so the code follows it.
- **Run sheet (N5).** The writer refuses `config-privileged` when the file is missing, so each host needs an
empty, root-owned `/etc/mosaic-keys/privileged` (0644) before the first enrollment.
## Mutant results (Sage's slot, 2026-10-21)
All 25 killed: A1 to A23, with A3b added for duckworth-0083 N8 and A22 split into A22a and A22b. The
runner is `mutants.py` outside the repo. Each mutant gets its own session, a 420 s limit with a group kill,
and a byte restore checked against git status. None hung. Run 1 (04:24Z, on 95f966b1c) killed A1 to A22a
and stopped at A22b, which survived. Run 2 (04:43Z, on 097892741) killed A22b and A23.
- Each kill comes from a named assertion in the mutant's own test files, except A9. Dropping the `hosts`
primary key leaves `host_grants`' foreign key with no parent key, so the store refuses to open
("foreign key mismatch") and 28 of 29 tests fail at setup. That's a structural kill, not the S1 test.
The schema can't exist without the key (sage-1709).
- A22b drops the actor filter from `GRANT_EVIDENCE`. The trigger can't stand in for it, because a 3i store
can hold a session-written grant row that the 3i-3j step keeps. The grant-migration test "A22b,
sage-1703" kills it (see the A22 row above).
Follow-up, not in row a: G18's control calls `runHost` directly, so nothing runs `hg.check` end to end
with a good pin (duckworth-0083 N9). The #433 VM run covers it.
## Status at 097892741
Done: every Split item, 1 to 8, duckworth-0081 B1 to B3, duckworth-0083 N8, and all 25 mutants killed.
Single files in the slot, all exit 0, 178 pass and 0 fail by an awk sum over 23 logs: bus grant-enroll
24, host-run 24, grant-migration 6, host-tools 6, account-migration 3, chief-of-staff 18,
comment-migration 5, hard-waits 13, name-migration 1, project-runtime 3, spec-migration 1,
thread-migration 3, usage-migration 1, project-files 8, spec-socket 6, spec-review 12, land 17, spec 4;
harness spec-tools 2, raise-domain 2; business hosts 4; cli hosts 5; seat sandbox 10.
Left: Sage runs the whole bus, business and cli suites and every `scripts/test-*.sh` at the landing gate,
after Duckworth approves.
## Round 3 (sage-1742): two tests that count the schema
The landing gate failed `packages/bus/tests/schema.test.mjs:522`. It counts the tables and triggers that
`schema.sql` creates, and row a adds two tables and a trigger, so the counts go from 16 to 18 tables and
from 70 to 71 triggers. My singles didn't run it because I took it for a test of other rows' tables.
I grepped `packages/*/tests/*` for `sqlite_master` and `schema.sql`, with `schema.test.mjs:517` as the
positive control. That turned up one more test that would fail: T46 in
`packages/cli/tests/stack-release.test.mjs:910`. It puts every schema table in exactly one of four lists, and
hosts and host_grants were in none. I'd left the stack-release tests out of the r2 singles on the claim that
they don't load the bus, but T46 reads `schema.sql`. Both tables go in its untimed list (Sage's ruling,
sage-1745: the message_threads pattern). Neither has an `at` column. Row a writes hosts at five lines,
and each runs in one store transaction with a `grant.*` events row, which the restore guard counts.
`#grantEvent` (broker.mjs:657) writes that row through `writeGrantEvent` (:201, INSERT INTO events at
:212).
- :737 DELETE, in `#enrollFailed`, with grant.failed at :738. Both callers wrap it in a transaction,
:816 and :849.
- :763 INSERT, in the step 1 transaction (:754), with grant.host.enroll.step at :761.
- :783 UPDATE installer_fingerprint, in the step 4 transaction (:780), with grant.host.enrolled at :784.
- :932 DELETE, in the hosts.remove transaction (:927), with grant.host.removed at :933.
- :950 UPDATE state ready, in the hosts.check transaction (:944), with grant.host.checked at :951.
Row a writes no host_grants rows. It only reads them (broker.mjs:695, :930, runtime.mjs:103). Row b adds
the writes, and each must share a transaction with a `grant.*` event the same way. That goes on row
235's carry list. The other hits pin an older schema
(account, comment, name, spec, thread and usage migrations, chief-of-staff 3c) or count only their own
tables (state.test.mjs:106 reads every table but pins no list, threads.test.mjs:129).
hosts and host_grants are mutable by design, so the refused-labels loop above `schema.test.mjs:517` gets no
entry for them. hosts.check updates a host to ready. hosts.remove and a failed enrollment delete the
record. Row b updates and deletes grant records. Neither table has append-only triggers. The one new
trigger, events_grant_actor, guards events. grant-migration.test.mjs tests it, and it killed mutants
A20 and A21.
+294
View File
@@ -0,0 +1,294 @@
# Row 197 execution ledger
Plan: MANIFEST.md r2, approved by Webby ca6b32d66.
Build grant: sage-1390. Suites, image builds and paired proof are not granted.
Rebased five documentation commits onto origin/refactor aa1bd209e before code.
Existing worktree herb/row197 and plain node_modules symlink retained.
- T1 preflight/steps: in progress.
- T2 locks/status/recovery: pending.
- T3 image/Docker: pending.
- T4 fixtures/timing: pending.
- T5 evidence/acceptance: historical timing done; implementation pending.
Preflight interfaces: T1 RunPaths supplies T2 ownership and T3 image checks.
T2 run receipt supplies T4 lane/port state and T5 attribution. T3 image identity
is checked before Docker-bearing steps. T4 timings supply T5 pair summaries.
No interface conflict found. No product API or assertion changes are authorized.
Ruling: use this approved BUILD.md as the execution ledger, rather than adding
skill workspace paths outside the manifest. This preserves the reviewed path
boundary. Missing ledger detail would cost recovery time, not product behavior.
## Implementation checkpoint 2026-10-20T20:11:58Z
T1-T5 source is present within the approved manifest. Full acceptance is pending.
Fresh single-file checks: preflight/supervisor 6/6; state/recovery 11/11; image and
Docker guard 4/4; port leases/attribution/proof summary 3/3. No skips or failures.
Existing files checked singly: board serve; WebUI serve, preview, proof-cli,
tester-cli and service; harness pi and Claude. Stack-release R1 selected single
passed 1/1. WebUI service also passed 31/31 with lane-a leases and a 30-byte root.
All existing assert lines are unchanged. Shell syntax and diff checks pass.
Observed defects fixed during the single-file cycles: cancellation incorrectly
returned zero when a TERM handler exited zero; recovery did not supersede an
unclean terminal; status omitted the lock-before-receipt window. Evidence logs
retain the failing and passing observations. Source still has no commit because
applicable suites have not run. No image build, proof, push or live operation.
Ruling: use an instrumented single-lane author gate on a frozen dirty-source
clone before committing source. The new wrapper requires a clean candidate and
proven image receipt, while repository rules require green suites before source
commits. The author gate retains every suite and assertion, caps concurrency at
two, uses own port/state/temp/project roots, and guards Docker against builds.
Cost if wrong: the author gate fails; clean-candidate wrapper baselines and the
paired proof remain mandatory after source is verified and committed.
Ruling: publish the initial run directory atomically after the start receipt is
complete. This avoids a second lane observing a partially written start. A
crash before publication has started no test or mutable Docker operation.
Cost if wrong: a lane refuses or quarantines; no protected state is bypassed.
## Implementation review corrections, 2026-10-20T20:33Z
Webby-0029 reviewed the first implementation packet at f942bd7d4.
B1 identified missing manifest tests. B2 identified a test that leased both real
bands through a private lock namespace. The old author-gate clear was withdrawn
in herb-0019. SOURCE.json, BUILD.patch and gate.sh still describe the old frozen
clone. They are not a clear for the edited working tree. No author gate ran.
B2 correction: the helper test uses only the active lane and MOSAIC_GATE_STATE.
It compares the other band arithmetically and has no unasserted listener. The
occupied-port probe first obtains its own real lease, binds that port, and
asserts that its sentinel listener survives refusal. No test reserves another
lane's band. Exhaustion narrows only the test process's band to one owned port.
B1 coverage added within the existing approved paths. No case is dropped:
| Manifest case | Added check |
| --- | --- |
| T1 canonical and builder checkout, existing output | gate-lane.test: direct validateRun refusals |
| T1 92-byte socket and over-107-byte socket | exact design example with a distinct gt- test root; 108-byte refusal |
| T1 both candidates carry tooling | validateTooling compares each launcher file, including missing and changed cases |
| T1 duplicates and NODE_TEST_CONTEXT | repeated package/script fixture is retained; sanitized environment is exercised by both supervised children |
| T2 one-winner race and inherited FDs | two actual shell wrappers race; a controlled supervisor verifies held FDs, and its child checks that no FD points to either lock file |
| T2 same checkout and legacy claim refusal | activeConflicts uses synthetic process observations with a live lane and a sibling landing gate |
| T2 surviving child and interrupted recovery | locked recovery kills a child proven by the private root, preserves a foreign child, is killed during fake Docker cleanup, leaves quarantine, and resumes under a new fixture grant |
| T3 missing receipt and missing image | verifyImage refuses absent provenance and failing inspection |
| T3 tag changed mid-gate and at gate end | runGate now uses the same executePinnedSteps boundary exercised by the tests; the next step never runs after a changed ID |
| T3 mutable-tag rebuild | fake preparation returns a new image ID with identical input digest; the old receipt refuses it |
| T3 cold and warm counters | preparation records one build and its time; warm execution records zero builds; repeated verification never builds |
| T4 parallel test-file leases | two generated Node test files hold ports in the same lane/state; locks are checked during child execution and shutdown |
| T4 exhausted band, occupied port and foreign listener | a one-port process-local band refuses both a held lease and an existing sentinel listener; the listener survives |
| T4 owner death | a killed allocator releases only its own holder; a foreign lease remains held |
| T4 lease through child shutdown | parent probes the lock while the fixture test runs and after its child has stopped |
| T4 fixtureCacheEnv | replacement child environment writes only under separate tmp, jiti and cache directories |
| T5 unrelated session append | a separate process waits for a running-step barrier, appends under a fixture HOME, and remains unrelated in the resulting metadata attribution |
N1 removes the unused closed binding. All leases reserve an unbound endpoint;
existing closed options need no call-site change. N2 makes child TMPDIR a tmp
subdirectory. N3 normalizes ss interface scopes before checking port 9. N4 uses
gt- for unit run roots. N5 documents the canonical node_modules dependency.
N6 is recorded: row 200 also edits stack-release.test.mjs; the second landing
must rebase and rerun that file singly. Sage was told in herb-0019.
Before Sage's hold, expanded preflight tests passed 11/11, state tests 14/14,
image tests 7/7, and the pi/Claude singles passed 8/8 each with separate tmp
subdirectories. Later source/test edits still need verification. In particular,
the new parallel test-file probe had a cleanup-order defect and is unverified.
Its root cleanup ran before child shutdown. Cleanup now stops children before
removing their root. Output is retained for a useful failure report. The initial
FD probe also assumed Node did not reopen FD numbers; it now checks the actual
lock targets. The early passing fixture file covered child allocators, before
they were changed to actual generated test files. It is not acceptance of the
latest file.
Sage-1448 stopped all node --test runs at 20:30Z for Rocko's browser slot.
The remaining owned process tree was stopped at 20:30:32Z. Inspection found no
remaining owned test processes afterward. Edits and static inspection continue.
20 module syntax checks, shell syntax and git diff checks pass. Existing-file
assertion lines remain identical to HEAD. No test is run during the hold.
Next: after Sage passes Rocko's clear, verify the corrected files singly,
including a lane-env helper-file check. Then refresh the frozen clone and all
three packet artifacts, send their hashes, and request a new author-gate clear.
Suites, image builds and paired proofs still need their own grants. No source
commit, push, suite, real Docker build or paired proof has run in this phase.
## Refrozen review packet, 2026-10-20T20:41Z
Sage-1454 lifted the single-file hold. Fresh runs passed without failures or
skips: review-t1-resume.log 11/11, review-t2-resume.log 14/14,
review-t3-resume.log 8/8, review-t4-resume.log 8/8. The helper file also passed
8/8 with lane-a environment in review-t4-lane-a.log. Its retained state had only
an a port-lock namespace, no b directory, and every lock was released. Generated
fixture Node test entries explicitly use --test-concurrency=2. The child-test
probe now finishes and cleans up before removing its root. All 41 new tests pass.
B1 and B2 fixes are ready for Webby's review. No manifest case was dropped.
All 25 source paths were copied to a new frozen clone and compared by mode and
SHA-256. This supersedes the earlier withdrawn source packet:
- SOURCE.json: 91ba149f2fc039a8bed0e6d1ad5da44b80fb8cbec84d0fa4e38616040138596b
- gate.sh: 48e06e23e45d3484755f8cd79ad5b6dfde5ef4f960081e7c455a2540e707fed1
- BUILD.patch: 0b3871bd9848ca223c7cda50266debc04643e5d986eb7f0fece096aaf25ff43e
- Frozen checkout: /mnt/storage/scratch/mg197-a-d22dacd92735/wt
- Output: /mnt/storage/scratch/mg197-a-d22dacd92735/out
- Temporary root: /mnt/storage/scratch/ga-VPXhG7 (30 bytes)
- Compose project: mg197-a-d22dacd92735
After an explicit author-gate grant, run this command with Sage's actual ID:
```sh
env MOSAIC_GATE_GRANT=ACTUAL_SAGE_GRANT_ID bash /home/jwoltje/.t3/worktrees/mosaic-stack/herb-row197/agents/herb/work/queue-197/gate.sh > /mnt/storage/scratch/mg197-a-d22dacd92735/out/gate.log 2>&1
```
No gate or suite has started. Queue order from sage-1454 is 192, 201, 204,
200, Gander, then 197. No branch deletion is authorized or attempted. Any later
rebase uses a new branch name, and pr-merge.sh -d stays off under #426(a).
Whichever of rows 197 and 200 lands second must rerun stack-release singly.
Clean-candidate wrapper baselines, image preparation and paired proofs remain
separately gated acceptance work. There is still no source commit or push.
## Implementation approval and final notes, 2026-10-20T20:47Z
Webby-0031 approved r2 in REVIEW-impl-r2.md at 1f404c973. Their static review
verified every source hash/mode and patch result, and all B1/B2 cases. No queue
review request comment exists yet; Sage owns creating it if needed.
Folded all three optional notes into the same approved paths. The helper file's
standalone fallback now shares /mnt/storage/scratch/mosaic-gate-lanes with real
lane-a leases and leaves its stable lock files in place. Band assertions use the
imported BANDS. Both T5 polling children fail after five seconds if their barrier
is missed. README records the shared standalone namespace and deadlines.
Fresh single-file logs review-r3-fixtures.log and review-r3-lane-a.log both pass
8/8 with zero skips. The lane-a run has no b lock directory, and every lock was
released. Syntax and diff checks pass. No existing test assertions or timeouts
changed. Remaining source matches the 11/14/8-file checks from r2.
Refreshed all 25 source rows in the existing frozen checkout. New digests replace
the preceding r2 packet. gate.sh and its command are unchanged:
- SOURCE.json: 3584763642506e81a532c839bdd07b111d25daa87e35dbe441d8e8c01a2c2aab
- gate.sh: 48e06e23e45d3484755f8cd79ad5b6dfde5ef4f960081e7c455a2540e707fed1
- BUILD.patch: 55fe390da17ab926a0593d39e79561abe52756a1b1a2c756ec57f45d90f10fde
Author gate remains unstarted and queued for Sage's explicit grant. No source
commit, suite, image build, paired proof, push or branch deletion ran.
## R3 approval receipt
Webby-0033 approved r3 in REVIEW-impl-r3.md at a5643f9bd. Static review verified
the three packet hashes, all 25 source rows, and the patch result. Their review
accepts N1-N3; no additional hang-guard test is requested. No queue review request
comment exists yet, so Webby has not recorded a queue review.
Read-only verification after receipt confirms that the approved packet hashes
and all 25 worktree/frozen source hashes and modes still match. Source and gate
script stay frozen. The author gate has not started and still needs Sage's grant.
## Queue review round 1
Sage-1476 authorized moving row 197 to in-review before the author gate.
Created CANDIDATE.sha256 from the same 25 reviewed SOURCE.json rows. Its digest
is 6318f0d0bba9ee90ccaa3503cac8c2e04c6397f2ef6c3564f687fbc2092989e3.
Ran the queue move from the canonical checkout with op herb-197-review-r3.
The move succeeded at revision 2395, opening round 1 on #1695. The request post
failed before send because MOSAIC_GITEA_CREDENTIAL_FILE is not set; revision 2396
records the failed outcome. Exit code 1 describes the failed post, not a failed
row transition. Read-back confirms state in-review and the correct candidate.
Sage will post as jarvis and give Webby the request comment number. No credential
lookup, alternate post or retry was attempted here.
SOURCE.json, BUILD.patch, gate.sh and all 25 frozen source rows remain unchanged.
Latest slot order is row 190 r2, 201, 204, 200, Gander, then Herb. No gate starts
before Sage sends an explicit grant ID.
## Follow-on assignment
Sage-1503 assigns row 215, Case 2 C2-04, to Herb after row 197 (queue rev 2455,
Gosalyn design r11). Read docs/plans/2026-10-20_case2-rows.md, C2-04 section,
at 5b8def2f1 when starting that row. Its required base is origin/refactor
305857c8f. This assignment does not replace row 197 or authorize its gate.
Keep row 197's approved source frozen while awaiting Sage's grant. Start row
215 only after row 197, using a new branch name under the existing fleet rule.
## Round 2 merge preparation on row 233
Sage-1757 replaces history replay with one candidate commit on 57f59e0c3.
The attempted history replay on herb/row197-r2-233 had already finished when
that instruction arrived. Its only conflict was the append-only BUILD-LOG tail.
That branch remains for history; none of its replayed commits is in this round.
The current branch is herb/row197-r3-233, directly from
57f59e0c399ce29b1bb85ca6af686fdc10eb195d.
Stash correction: shared refs/stash was used incorrectly. The exact entry
2767b88549dfd8c7e810908b5416876fa50cb571 is now kept on
herb/row197-r3-wip. After verifying stash@{0} still named it, only that entry was
dropped. The two other entries remain. No further stash creation, force push,
branch deletion or refs/stash mutation is planned. A separate working-copy
backup is /mnt/storage/scratch/herb197-merge-W4oiLg.
Sage-1760 requires preserving every landed hunk on every changed manifest path.
Checked each of the 25 r1 paths with git diff aa1bd209e 57f59e0c3 -- PATH.
Exactly three changed upstream. Their r2 diff against 57f59e0c3 is captured in
r2-merge-only.patch:
| Merged path | Only row 197 changes against landed base | Landed content retained |
| --- | --- | --- |
| packages/cli/tests/stack-release.test.mjs | leasePort and laneAfter imports; freePort now calls the lease helper | All landed tests, including row 233's tests after R15 and BACKUP_STORE checks |
| packages/webui/tests/service.test.mjs | leasePort and laneAfter imports; closedPort now calls the lease helper | SEND_REFUSAL permits --sso; default unit rendering removes the @CONFIG_ENV@ line |
| scripts/test-discord.sh | fixture PI_CODING_AGENT_DIR setup; optional two-worker nested Node cap | Row 190's absent scratch MOSAIC_CONFIG selection and caller-config restoration |
The other 22 paths equal the r1 reviewed source bytes and modes. r2-MERGE.json
lists every path, its r1/r2 digest, mode, upstream-change flag and equality result.
Source copies came from the hash-verified r1 frozen checkout. Original r1 digest:
6318f0d0bba9ee90ccaa3503cac8c2e04c6397f2ef6c3564f687fbc2092989e3.
No existing assertion or timeout line differs from the landed base. Static
checks only: 20 module syntax checks, four shell syntax checks and git diff check
pass. BUILD-LOG.md equals 57f59e0c3 byte for byte and is outside this commit.
Sage explicitly requested the local candidate SHA before granting tests. This
commit is an untested preparation checkpoint, not gate evidence. No tests or
push are authorized yet. r2-TEST-FILES.txt lists the requested grant scope:
16 Node test files singly and scripts/test-discord.sh, test-foundation.sh and
test-queue.sh. The four stack-release consumers are stack-release.test.mjs,
stack-release-external.test.mjs, stack-upgrade.test.mjs and stack-instances.test.mjs.
The shared stack-release-fixture.mjs is exercised by its importing test files,
not run as a standalone test. This covers all 13 touched Node test files plus
three additional stack-release consumers and all three touched shell suites.
Current slot order from sage-1760: Gizmoduck, Launchpad, Darkwing, then Herb,
around 06:30Z. Wait for an explicit grant. After granted checks and authorized
push, build the new candidate manifest from that pushed tip and open the
merge-scoped round 2. Old SOURCE.json, CANDIDATE.sha256 and gate.sh remain r1
historical artifacts and are not a clear to run the merged candidate.
## Round 2 rebuilt on row 234
Sage-1776 names landing c771a36f4bd0b6c359282425c668996a90072f93.
Built a new branch, herb/row197-r4-234, directly from that landing. No history
replay, stash, force push, branch deletion, test run or push is involved.
The prior candidate a3b7cbde8 remains untested and is not the queued candidate.
Checked git diff --name-only 57f59e0c3 c771a36f4 against all 25 source paths.
Only packages/cli/tests/stack-release.test.mjs overlaps; the other 24 do not.
The full comparison is in r2-on234-OVERLAP.txt. The stack-release file starts
from the landing and keeps both T46 untimed entries, host_grants and hosts.
Its only changes against the landing are the leasePort and laneAfter imports
and freePort's call to the lease helper. r2-on234-merge-only.patch records those
hunks. Existing assertions and timeouts match the landing exactly.
Every other source path equals a3b7cbde8 byte for byte and retains its mode.
This includes the previously merged service and test-discord files, preserving
all landed --sso, CONFIG_ENV and scratch-config handling. The earlier evidence
r2-MERGE.json still records those three pre-234 merge cases. The new
r2-on234-MERGE.json records each current digest, prior digest, mode and equality
result. BUILD-LOG.md equals the new landing byte for byte and is excluded.
Static check of the changed Node file and the candidate diff pass. No tests
were run. The scope stays r2-TEST-FILES.txt: 16 Node files singly plus the three
shell suites. No manifest or queue round is rebuilt until granted checks and
authorized push. Old SOURCE.json, CANDIDATE.sha256 and gate.sh remain historical
r1 evidence. The test slot follows Darkwing 240 and needs Sage's explicit grant.
+126
View File
@@ -0,0 +1,126 @@
# Two-lane proof command and schedule
This evidence stays outside r5's 32 manifest paths. No proof or image build has
run. The driver and example config are committed on herb/row197-proof, rooted
at 9aa6d52d9. Sage runs the committed copy from their own checkout.
Driver: `agents/herb/work/queue-197/proof/run-proof.mjs`.
The commit message to Sage carries its exact SHA-256. Check it with
`git show <proof-commit>:agents/herb/work/queue-197/proof/run-proof.mjs | sha256sum`.
After Sage fills config.json from config.example.json and grants the whole block:
```sh
MOSAIC_PROOF_GRANT=ACTUAL_SAGE_GRANT_ID node /SAGE_PROOF_CHECKOUT/agents/herb/work/queue-197/proof/run-proof.mjs /SAGE_OWNED_CONFIG/config.json
```
The example config pins the launcher and candidate A to 197 r5:
9aa6d52d9431efd215aa31d748d7e273eb321474. The launcher is a separate clean
checkout owned by Sage, so later row 215 work cannot change it. Fill candidateB with
Sage's approved combined 212+197 commit, imageReceipt with the granted pinned
image-provenance receipt, and runRoot with a fresh private scratch path. Both
candidates must carry identical lane tooling and image inputs and use the same
pinned image ID. The driver refuses missing parameters, equal candidate SHAs,
dirty launcher, changed tooling, mismatched image inputs or image ID.
D5 order is exactly two solo gates, then three pairs:
| Wave | Candidate A | Candidate B |
| --- | --- | --- |
| 1 | solo, lane a | idle |
| 2 | idle | solo, lane b |
| 3 | paired, lane a | paired, lane b |
| 4 | paired, lane b | paired, lane a |
| 5 | paired, lane a | paired, lane b |
Every invocation gets its own clean clone, output, short temp root and random
Compose project. The wrapper discovers all package and shell steps, with two
Node test-file workers per lane. The summary requires three overlaps, the lane
swap, successful clean receipts and zero warm image builds. Raw logs and per-step
receipts remain available. No failed gate or series is retried automatically.
A live monitor stops both gates on the first failure line, records lane, step,
source file when the runner emits it, test name and preserved log, then enriches
that report after shutdown. If the runner gives no source path, the report says
unresolved and points to the log rather than guessing. It stops earlier than the
two-failure limit. TERM/INT/HUP stop owned supervisors; cleanup waits for them.
Container and network names come from Compose's random recorded project. The
project and service configuration have no fixed global names. Exact-label Docker
event streams record create/destroy IDs and names, including --rm containers.
An exit trap inventories names/labels, removes only exact-project objects with
the approved cleanup helper, verifies emptiness and stops its own event streams.
The driver's Docker inspection never reads Config.Env or credential contents, and never builds, pulls,
retags or prunes. Missing resource-name evidence, an ended event stream, unknown
ownership, leftover objects or quarantined lanes fail the proof. A SIGKILL cannot
run an exit trap; durable wrapper receipts then require granted recovery.
Run this as its own whole-slot phase. A still-running legacy landing.mjs gate or
gate.sh is LEGACY BUSY, including a parent process. The driver does not suppress
that refusal. The ordinary landing gate must exit before this proof phase starts.
Planning runtime from sage-1837 is 12-15 minutes per wave, so reserve 60-75 minutes
for all five waves, plus any separately granted image preparation. This is an
estimate, not a measurement. Measured proof runtime is unavailable until this
first run. Historical row 189 full gate was 659 seconds; the r4 focused 19-file
run was 761.33 seconds and is not a full solo baseline. The resulting summary
records actual per-gate times, pair makespans, slowdowns and total wall time.
No image provenance receipt or combined integration SHA is available to Herb yet.
The driver remains unexecuted. Sage will read the exact hashed driver and request
Webby's static review before the landing/proof grant.
Static node --check passed after sage-1839 single cleared. This is syntax-only;
the driver has not run a proof or Docker command. Its runtime behavior awaits
Sage and Webby review and the separately granted proof block.
## Driver review fixes, sage-1857
Webby's first review was CHANGES, B1, at bba870ee9. The revised driver takes
tempRoot only from immutable start records for the plan's checkouts. After
every owned gate exits, Docker cleanup and the final status/process checks
run. The driver requires the exact short ga-/gb- scratch spelling, a matching
lane, an owned real directory and no live owner, child group or process with
the run paths in its argv before removal. It also requires a clean terminal
receipt. A successful gate's root is removed recursively. A failed gate's
root stays for diagnosis; an unverified cleanup also stays and fails the
proof. docker-summary.json records removed, kept and error entries. Any
cleanup error makes the driver fail. No unrelated scratch directory is read.
Signal handlers are installed before configuration and preparation. Clone
preparation is inside the try/finally, and yields between synchronous clones
so a pending signal stops it before another invocation. A preparation signal
writes failure evidence and reaches the cleanup path. Partial clones remain
under runRoot as permitted by Sage; they are never confused with lane roots.
SIGKILL cannot be trapped and still needs granted recovery.
At 2026-10-21T08:26:41Z, Herb removed four earlier roots recorded in their
own packet: ga-VPXhG7, gb-XeTAUS, ga-35Zo9Y and ga-R5IOT3. Each was an owned
real directory with no process argv/cwd reference before removal. Names
ga-AMtfI3 and ga-Vd9Ef2 were visible in the scratch directory listing but had
no ownership evidence in Herb's packet; neither was opened or changed.
## N4 read-only trace, pre-existing and out of scope
Sage ruled in sage-1859 that N4 does not block 197. Legacy suites already
reach Compose's read-only host auth mount, and the lane adds no exposure.
Sage owns a DEFERRED entry for fixture PI_AUTH_FILE coverage. No fix is made
in 197, and this procedure does not claim that the full gate is secret-free.
At 9aa6d52d9:
- gate-lane.mjs:109-111 keeps HOME real but neither keeps nor sets
PI_AUTH_FILE. compose.yaml:43 therefore selects its host auth-file default.
- script-task is discovered by gate-lane.mjs:36. test-task.sh has no
PI_AUTH_FILE assignment. Calls at :149-150 (mock), :444, :460, :480, :483
and :520 (live) reach run-task. mosaic-task.mjs:321 copies the environment
and :395-400 invokes compose run mosaic-agent without changing that path.
- script-release calls release activate at test-release.sh:69,85,93.
release.sh:41-54 health_check calls run-task with no PI_AUTH_FILE override,
then reaches the same Compose call. These tests use scratch MOSAIC_CONFIG;
that does not override the separate auth-file bind mount.
- test-auth.sh:48-92 sets explicit fixtures, but calls auth.sh instead of
Compose. Package runs/task-cli.test.mjs exercises list/show, not run.
This was a static source trace only. No auth file was opened, no credential
value was read, and no Docker command, image build or proof was executed.
@@ -0,0 +1,10 @@
{
"repo": "/mnt/storage/src/mosaic-stack",
"launcherRoot": "SAGE_OWNED_CLEAN_LAUNCHER_CHECKOUT",
"launcherCandidate": "9aa6d52d9431efd215aa31d748d7e273eb321474",
"candidateA": "9aa6d52d9431efd215aa31d748d7e273eb321474",
"candidateB": "SAGE_APPROVED_212_PLUS_197_COMMIT",
"candidateDifference": "A is approved 197 r5; B must be the approved combined 212+197 integration commit",
"imageReceipt": "SAGE_PINNED_IMAGE_PROVENANCE_RECEIPT",
"runRoot": "/mnt/storage/scratch/mg197-proof-REPLACE_WITH_UNIQUE_NONCE"
}
@@ -0,0 +1,201 @@
// Evidence driver only. Runs no test without Sage's whole-slot grant.
import { readFileSync, writeFileSync, appendFileSync, mkdirSync, existsSync, realpathSync, openSync, closeSync, symlinkSync, readdirSync, statSync, lstatSync, readSync, rmSync } from 'node:fs';
import { resolve, join, dirname, relative, isAbsolute } from 'node:path';
import { pathToFileURL } from 'node:url';
import { execFileSync, spawn } from 'node:child_process';
const proofStart=performance.now();
let interrupted=false,abortOnSignal=()=>{};
const signal=()=>{interrupted=true;abortOnSignal();};
for(const name of ['SIGINT','SIGTERM','SIGHUP'])process.on(name,signal);
const fail=m=>{throw Error(m);};
const within=(root,path)=>{const p=relative(root,path);return p===''||!p.startsWith('..')&&!isAbsolute(p);};
const grant=process.env.MOSAIC_PROOF_GRANT;
if(!/^sage-[0-9]+$/.test(grant??''))fail('explicit whole-slot MOSAIC_PROOF_GRANT required');
const config=JSON.parse(readFileSync(process.argv[2],'utf8'));
for(const key of ['candidateA','candidateB'])if(!/^[a-f0-9]{40}$/.test(config[key]??''))fail('full approved candidate SHA required: '+key);
if(config.candidateA===config.candidateB)fail('two distinct approved candidates required');
if(!config.candidateDifference?.trim())fail('record why the approved candidates differ');
const launcher=realpathSync(config.launcherRoot),repo=realpathSync(config.repo),receipt=realpathSync(config.imageReceipt);
const git=(root,...args)=>execFileSync('git',['-C',root,...args],{encoding:'utf8',stdio:['ignore','pipe','pipe']}).trim();
if(git(launcher,'rev-parse','HEAD')!==config.launcherCandidate)fail('launcher pin differs');
if(git(launcher,'status','--porcelain').split('\n').some(l=>l&&l!=='?? node_modules'))fail('launcher checkout dirty');
const root=resolve(config.runRoot);
if(existsSync(root)||!within('/mnt/storage/scratch',realpathSync(dirname(root))))fail('fresh scratch runRoot required');
const {validateTooling}=await import(pathToFileURL(join(launcher,'scripts/testing/gate-lane.mjs')));
const {verifyImage}=await import(pathToFileURL(join(launcher,'scripts/testing/lane-image.mjs')));
const {runRecords,status,processInventory,sameProcess}=await import(pathToFileURL(join(launcher,'scripts/testing/lane-state.mjs')));
const {summarizePairs}=await import(pathToFileURL(join(launcher,'scripts/testing/lane-evidence.mjs')));
const {cleanupProject}=await import(pathToFileURL(join(launcher,'scripts/testing/lane-docker.mjs')));
const docker=execFileSync('which',['docker'],{encoding:'utf8'}).trim();
const image=JSON.parse(readFileSync(receipt,'utf8'));
mkdirSync(root,{mode:0o700});const state=join(root,'state');mkdirSync(state,{mode:0o700});
const put=(name,value)=>writeFileSync(join(root,name),JSON.stringify(value,null,2)+'\n',{mode:0o600,flag:'wx'});
const waves=[
[['solo-a','a',config.candidateA]],
[['solo-b','b',config.candidateB]],
[['pair1-a','a',config.candidateA],['pair1-b','b',config.candidateB]],
[['pair2-b','b',config.candidateA],['pair2-a','a',config.candidateB]],
[['pair3-a','a',config.candidateA],['pair3-b','b',config.candidateB]],
];
const plan=waves.map(w=>w.map(([label,lane,candidate])=>({label,lane,candidate,dir:join(root,label),checkout:join(root,label,'wt'),out:join(root,label,'out')})));
put('plan.json',{grant,...config,plan,image,started:new Date().toISOString()});
const active=new Set(),watchers=new Map(),objects=new Map();let firstFailure=null;
const eventError=[];
const knownRuns=new Map();let abortTimer;
const ownedRuns=()=>runRecords(state).flatMap(r=>r.records.filter(e=>e.kind==='start'&&plan.flat().some(p=>p.checkout===e.checkout)));
function logObject(project,kind,id,name,action){
if(!/^mg197-[ab]-[a-f0-9]{12}$/.test(project)||!['container','network'].includes(kind)||!/^[a-f0-9]{12,64}$/.test(id))fail('invalid Docker ownership metadata');
if(name&&!String(name).replace(/^\//,'').startsWith(project))fail('Docker name outside its recorded project');
const key=kind+':'+id,old=objects.get(key);const value={project,kind,id,name:name??old?.name??null,created:old?.created||action==='create',destroyed:old?.destroyed||action==='destroy'};
objects.set(key,value);appendFileSync(join(root,'docker-objects.jsonl'),JSON.stringify({...value,action,at:new Date().toISOString()})+'\n',{mode:0o600});
}
function ensureWatchers(final=false){
if(final||[...active].some(j=>!knownRuns.has(j.run.checkout)))for(const run of ownedRuns())knownRuns.set(run.checkout,run);
for(const run of knownRuns.values())if(!watchers.has(run.project)){
// Replay the few seconds before the immutable start was noticed; the random
// exact project label excludes all foreign resources. No raw events are saved.
const child=spawn(docker,['events','--since',String(Math.floor(Date.now()/1000)-5),'--filter','label=com.docker.compose.project='+run.project,'--filter','type=container','--filter','type=network','--filter','event=create','--filter','event=destroy','--format','{{json .}}'],{stdio:['ignore','pipe','pipe']});
const watcher={child,stopping:false,buffer:''};watchers.set(run.project,watcher);
child.stdout.on('data',bytes=>{watcher.buffer+=bytes;let end;while((end=watcher.buffer.indexOf('\n'))>=0){const line=watcher.buffer.slice(0,end);watcher.buffer=watcher.buffer.slice(end+1);if(!line)continue;try{const event=JSON.parse(line),kind=event.Type,action=event.Action;const attrs=event.Actor?.Attributes??{};if(attrs['com.docker.compose.project']&&attrs['com.docker.compose.project']!==run.project)throw Error('foreign project event');if(['container','network'].includes(kind)&&['create','destroy'].includes(action))logObject(run.project,kind,event.Actor.ID,attrs.name,action);}catch(error){eventError.push(error.message);}}});
child.stderr.resume();watcher.done=new Promise(resolve=>{child.once('error',error=>{eventError.push(error.message);resolve();});child.once('close',code=>{if(!watcher.stopping)eventError.push('Docker event stream ended early: '+code);resolve();});});
}
}
function inventory(project){
const found=[];
for(const kind of ['container','network']){
const query=kind==='container'?['ps','-aq']:['network','ls','-q'];
const ids=execFileSync(docker,[...query,'--filter','label=com.docker.compose.project='+project],{encoding:'utf8'}).trim().split(/\s+/).filter(Boolean);
for(const id of ids){
if(!/^[a-f0-9]{12,64}$/.test(id))fail('invalid inventory ID');
// Read only name and ownership label, never Config.Env or credentials.
const labels=kind==='container'?'Config.Labels':'Labels';
const format='{"name":{{json .Name}},"project":{{json (index .'+labels+' "com.docker.compose.project")}}}';
const item=JSON.parse(execFileSync(docker,[kind,'inspect',id,'--format',format],{encoding:'utf8'}));
if(item.project!==project)fail('foreign Docker object');
logObject(project,kind,id,item.name,'inventory');found.push({kind,id,name:item.name});
}
}
return found;
}
function failureDetails(job){
const events=runRecords(state).find(r=>r.records.some(e=>e.kind==='start'&&e.checkout===job.run.checkout))?.records??[];
const steps=events.filter(e=>e.kind==='step');
const paths=firstFailure?.gate===job.run.label&&firstFailure.log?[firstFailure.log]:existsSync(job.run.out)?readdirSync(job.run.out).filter(n=>n.endsWith('.log')).map(n=>join(job.run.out,n)):[];
for(const log of paths){
const text=readFileSync(log,'utf8');const bad=text.match(/^(?:\s*✖ (?!failing tests:)|not ok \d+ - |FAIL\s+)(.+)$/m);if(!bad)continue;
const detail=text.match(/✖ failing tests:[\s\S]*$/)?.[0]??text;
const file=detail.match(/^test at (.+?\.test\.(?:mjs|js|cjs))(?::\d+)+/m)?.[1]??detail.match(/file:\/\/([^\s)]+\.test\.(?:mjs|js|cjs))/)?.[1];
return {lane:job.run.lane,gate:job.run.label,step:log.split('/').at(-1).slice(0,-4),file:file??'unresolved; inspect preserved log',test:bad[1],log};
}
const failed=steps.find(e=>e.rc||e.requiredSkips?.length);
return {lane:job.run.lane,gate:job.run.label,step:failed?.name??'preflight or cleanup',file:'see gate.log',test:failed?.requiredSkips?.join('; ')??'wrapper failed',log:join(job.run.dir,'gate.log')};
}
function abortActive(){
for(const job of active)job.child.kill('SIGTERM');
if(!abortTimer){abortTimer=setTimeout(()=>{for(const job of active)job.child.kill('SIGKILL');},30000);abortTimer.unref();}
}
abortOnSignal=abortActive;
function newFailure(job){
if(!existsSync(job.run.out))return null;job.logs??=new Map();
for(const name of readdirSync(job.run.out).filter(n=>n.endsWith('.log'))){
const log=join(job.run.out,name),st=statSync(log),cursor=job.logs.get(log)??{size:0,tail:''};if(st.size===cursor.size)continue;
const fd=openSync(log,'r'),bytes=Buffer.alloc(st.size-cursor.size);try{readSync(fd,bytes,0,bytes.length,cursor.size);}finally{closeSync(fd);}
const text=cursor.tail+bytes.toString('utf8'),match=text.match(/^(?:\s*✖ (?!failing tests:)|not ok \d+ - |FAIL\s+)(.+)$/m);
job.logs.set(log,{size:st.size,tail:text.slice(-8192)});
if(match)return {lane:job.run.lane,gate:job.run.label,step:name.slice(0,-4),file:'awaiting runner failure details',test:match[1],log};
}
return null;
}
function monitor(){
try{
ensureWatchers();
for(const job of active){const detail=newFailure(job);if(detail&&!firstFailure){firstFailure=detail;process.stderr.write('FIRST_FAILURE '+JSON.stringify(detail)+'\n');abortActive();}}
if(eventError.length&&!firstFailure){firstFailure={kind:'Docker evidence',errors:[...eventError]};abortActive();}
}catch(error){eventError.push(error.message);abortActive();}
}
function launch(run){
const fd=openSync(join(run.dir,'gate.log'),'wx',0o600);
const child=spawn('bash',[join(launcher,'scripts/testing/gate-lane.sh'),'run','--lane',run.lane,'--candidate',run.candidate,'--checkout',run.checkout,'--out',run.out,'--state-root',state,'--image-receipt',receipt,'--grant',grant],{stdio:['ignore',fd,fd]});closeSync(fd);
const job={child,run};active.add(job);
job.done=new Promise(resolve=>{child.once('error',error=>{active.delete(job);resolve({rc:127,error:error.message});});child.once('exit',(code,signal)=>{active.delete(job);resolve({rc:code??128,signal});});});
return job;
}
function gateReceipt(run){
const item=runRecords(state).find(r=>r.records.some(e=>e.kind==='start'&&e.checkout===run.checkout));
if(!item)fail('missing start receipt: '+run.label);
const terminal=item.records.filter(e=>e.kind==='terminal').at(-1);
if(!terminal||terminal.rc!==0||!terminal.clean||terminal.imageBuilds!==0)fail('gate failed or unclean: '+run.label);
if(item.records.some(e=>e.requiredSkips?.length))fail('required prerequisite skipped: '+run.label);
return {...terminal,label:run.label,lane:run.lane,candidate:run.candidate,receiptDir:item.dir};
}
async function stop(){
const jobs=[...active];for(const j of jobs)j.child.kill('SIGTERM');
const timer=setTimeout(()=>{for(const j of jobs)j.child.kill('SIGKILL');},30000);timer.unref();
await Promise.all(jobs.map(j=>j.done));clearTimeout(timer);
}
const measured=[],start=proofStart;
const monitorTimer=setInterval(monitor,100);
let summary;
try{
// Preparation is inside the cleanup path. Yield between synchronous clones
// so a pending signal stops preparation before another invocation is made.
for(const run of plan.flat()){
await new Promise(resolve=>setImmediate(resolve));
if(interrupted)fail('proof interrupted during preparation');
mkdirSync(run.dir,{mode:0o700});
execFileSync('git',['clone','--quiet','--no-hardlinks','--local',repo,run.checkout],{stdio:'pipe'});
execFileSync('git',['-C',run.checkout,'checkout','--quiet','--detach',run.candidate],{stdio:'pipe'});
symlinkSync('/mnt/storage/src/mosaic-stack/node_modules',join(run.checkout,'node_modules'));
validateTooling(run.checkout);verifyImage(image,run.checkout);
}
for(const wave of plan){
if(interrupted)fail('proof interrupted');
const observed=status(state);
if(observed.legacy.length||observed.a.state!=='CLEAR'||observed.b.state!=='CLEAR')fail('proof requires both lanes clear and no legacy gate');
const jobs=wave.map(launch);
for(const j of jobs)j.done.then(result=>{if(result.rc){firstFailure??=failureDetails(j);abortActive();}}).catch(error=>{eventError.push(error.message);abortActive();});
const results=await Promise.all(jobs.map(j=>j.done));
if(firstFailure?.gate){const failedJob=jobs.find(j=>j.run.label===firstFailure.gate);if(failedJob)firstFailure=failureDetails(failedJob);}
if(interrupted||firstFailure||eventError.length||results.some(r=>r.rc))fail('failed wave: '+wave.map(r=>r.label).join(',')+' '+JSON.stringify(firstFailure));
measured.push(wave.map(gateReceipt));
const end=status(state);
if(end.legacy.length||end.a.state!=='CLEAR'||end.b.state!=='CLEAR')fail('unclean wave status');
}
const solos=measured.slice(0,2).flat(),pairs=measured.slice(2);summary={grant,solos,pairs,summary:summarizePairs(solos,pairs),wallMs:performance.now()-start,finished:new Date().toISOString()};
}catch(error){
await stop();put('failed.json',{grant,message:error.message,firstFailure,measured,at:new Date().toISOString()});process.stderr.write(error.message+'\n');process.exitCode=1;
}finally{
clearInterval(monitorTimer);clearTimeout(abortTimer);await stop();const cleanupErrors=[];
try{ensureWatchers(true);}catch(error){cleanupErrors.push({message:error.message});}
for(const run of knownRuns.values())try{inventory(run.project);cleanupProject({project:run.project},docker);if(inventory(run.project).length)fail('Docker resources remain');}catch(error){cleanupErrors.push({project:run.project,message:error.message});}
for(const watcher of watchers.values()){watcher.stopping=true;watcher.child.kill('SIGTERM');}
await Promise.all([...watchers.values()].map(w=>w.done));
let finalStatus;try{finalStatus=status(state);if(finalStatus.legacy.length||finalStatus.a.state!=='CLEAR'||finalStatus.b.state!=='CLEAR')cleanupErrors.push({message:'lane status not clear',status:finalStatus});}catch(error){cleanupErrors.push({message:error.message});}
// Start receipts establish which short roots belong to this proof. Never
// discover roots by scanning scratch or follow a substituted root symlink.
const tempRoots=[];let procs;
try{procs=processInventory();}catch(error){cleanupErrors.push({message:'process inventory failed: '+error.message});}
for(const run of knownRuns.values()){
const record={gate:plan.flat().find(p=>p.checkout===run.checkout)?.label,lane:run.lane,tempRoot:run.tempRoot};
try{
if(!/^\/mnt\/storage\/scratch\/g[ab]-[A-Za-z0-9]{6}$/.test(run.tempRoot??'')||!run.tempRoot.startsWith('/mnt/storage/scratch/g'+run.lane+'-'))fail('invalid recorded temp root');
const st=lstatSync(run.tempRoot);
if(!st.isDirectory()||st.isSymbolicLink()||st.uid!==process.getuid()||realpathSync(run.tempRoot)!==run.tempRoot)fail('temp root is not an owned real directory');
const receipt=runRecords(state).find(r=>r.records.some(e=>e.kind==='start'&&e.checkout===run.checkout));
const groups=new Set(receipt?.records.filter(e=>e.kind==='child').map(e=>e.group));
if(!procs||sameProcess(run.owner)||procs.some(p=>p.state!=='Z'&&p.pid!==process.pid&&(groups.has(p.group)||p.argv.some(a=>a.includes(run.tempRoot)||a.includes(run.checkout)))))fail('run process still present or inventory unavailable; temp root retained');
const terminal=receipt?.records.filter(e=>e.kind==='terminal').at(-1);
if(!terminal?.clean){record.action='kept';record.reason='run cleanup unverified';cleanupErrors.push({...record});tempRoots.push(record);continue;}
if(terminal.rc!==0||firstFailure?.gate===record.gate){record.action='kept';record.reason='failed gate diagnosis';}
else{rmSync(run.tempRoot,{recursive:true});record.action='removed';}
}catch(error){record.action='error';record.message=error.message;cleanupErrors.push({...record});}
tempRoots.push(record);
}
put('docker-summary.json',{objects:[...objects.values()],eventError,cleanupErrors,finalStatus,tempRoots});
if(cleanupErrors.length||eventError.length||[...objects.values()].some(o=>!o.name))process.exitCode=1;
if(summary&&!process.exitCode){
for(const run of knownRuns.values())for(const kind of ['container','network'])if(![...objects.values()].some(o=>o.project===run.project&&o.kind===kind&&o.created)){process.exitCode=1;process.stderr.write('missing Docker creation evidence: '+run.project+' '+kind+'\n');}
if(!process.exitCode){put('summary.json',summary);process.stdout.write('PROOF_PASSED '+join(root,'summary.json')+'\n');}
}
for(const name of ['SIGINT','SIGTERM','SIGHUP'])process.off(name,signal);
}
+207
View File
@@ -0,0 +1,207 @@
{
"r1Base": "aa1bd209e",
"r1Candidate": "6318f0d0bba9ee90ccaa3503cac8c2e04c6397f2ef6c3564f687fbc2092989e3",
"r2Base": "57f59e0c399ce29b1bb85ca6af686fdc10eb195d",
"rows": [
{
"path": "packages/cli/tests/fixtures/gate-lane-child.mjs",
"r1Sha256": "209682b968e929da4b46dc97c64fcd7aa68c490f3c6be44595f5d85f8d2e1483",
"r2Sha256": "209682b968e929da4b46dc97c64fcd7aa68c490f3c6be44595f5d85f8d2e1483",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "packages/cli/tests/gate-lane-fixtures.test.mjs",
"r1Sha256": "29afa181804a724c695678eb9459c156e00f1084647c704e814c5bee156c0c25",
"r2Sha256": "29afa181804a724c695678eb9459c156e00f1084647c704e814c5bee156c0c25",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "packages/cli/tests/gate-lane-image.test.mjs",
"r1Sha256": "0253621a49c2e77ffcdbf32beee7e1f773ec2c52c8487dcc5aa254e61807af7c",
"r2Sha256": "0253621a49c2e77ffcdbf32beee7e1f773ec2c52c8487dcc5aa254e61807af7c",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "packages/cli/tests/gate-lane-state.test.mjs",
"r1Sha256": "948d62c76447d8269f03518562a261564c202ec9207461fb21d2b1c11c9bb9fc",
"r2Sha256": "948d62c76447d8269f03518562a261564c202ec9207461fb21d2b1c11c9bb9fc",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "packages/cli/tests/gate-lane.test.mjs",
"r1Sha256": "240bf0c15025ad6f88b251917a3c8d72bd21c2e5df6771e1ceb83a1d90b24a51",
"r2Sha256": "240bf0c15025ad6f88b251917a3c8d72bd21c2e5df6771e1ceb83a1d90b24a51",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "packages/cli/tests/stack-release.test.mjs",
"r1Sha256": "8ca733a600d5f5534d7e14d8533e1b16d710ebd43080ef8421238b5dd145fade",
"r2Sha256": "b95c0c125b6a283961e24adc07d43150bcbd05d2f11005180b836043015c52a6",
"mode": "0o644",
"landedPathChanged": true,
"r1BlobEqual": false
},
{
"path": "packages/control-board/tests/serve.test.mjs",
"r1Sha256": "4838cb4609da28fc921370c5542a82402fbc9ceb469692a36f0eafc713bb56c8",
"r2Sha256": "4838cb4609da28fc921370c5542a82402fbc9ceb469692a36f0eafc713bb56c8",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "packages/harness/tests/claude-session.test.mjs",
"r1Sha256": "0a09b0255e9144a039428eb8e9f56181f94ccb2d187e474ea13392b485fdd008",
"r2Sha256": "0a09b0255e9144a039428eb8e9f56181f94ccb2d187e474ea13392b485fdd008",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "packages/harness/tests/pi-session.test.mjs",
"r1Sha256": "6db5c3b7063baf933061e91e8bdf66740717dbb3066be3b70ac779fb17022094",
"r2Sha256": "6db5c3b7063baf933061e91e8bdf66740717dbb3066be3b70ac779fb17022094",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "packages/webui/tests/preview.test.mjs",
"r1Sha256": "ada49ec100d75b3e739263593164b2798c378f3df0f38d73d80cf011ef16e234",
"r2Sha256": "ada49ec100d75b3e739263593164b2798c378f3df0f38d73d80cf011ef16e234",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "packages/webui/tests/proof-cli.test.mjs",
"r1Sha256": "d3cec62f6eb8e2c1e0747f2a1bf1fc3f1a2122e45870d5a416d1830d3b6182cb",
"r2Sha256": "d3cec62f6eb8e2c1e0747f2a1bf1fc3f1a2122e45870d5a416d1830d3b6182cb",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "packages/webui/tests/serve.test.mjs",
"r1Sha256": "36ecdfcb3d0d9a2411eec751421f770b5bb41ec2fbf9236c5794fb241fd3196b",
"r2Sha256": "36ecdfcb3d0d9a2411eec751421f770b5bb41ec2fbf9236c5794fb241fd3196b",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "packages/webui/tests/service.test.mjs",
"r1Sha256": "c915239f0c223209baf84c96ec9f5ae2845fce7dd98ad46b073a96033aa7d6df",
"r2Sha256": "845313239547ddbc478090b29f96385f0b8853e5087c572393be328e7075f578",
"mode": "0o644",
"landedPathChanged": true,
"r1BlobEqual": false
},
{
"path": "packages/webui/tests/tester-cli.test.mjs",
"r1Sha256": "8bf4491f7982399b4162cfbe36d0b68888e99ceb2cbaefefc2361732dd95e0c8",
"r2Sha256": "8bf4491f7982399b4162cfbe36d0b68888e99ceb2cbaefefc2361732dd95e0c8",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "scripts/test-discord.sh",
"r1Sha256": "c10af53b0717344026cb22462964f253c7401c5082ac6ef3722a89d9dfa5c661",
"r2Sha256": "a5dcf783c51a5d5d184a357f91ebf9fa1c4e054f0cb84a3d1d7668f6156e025e",
"mode": "0o755",
"landedPathChanged": true,
"r1BlobEqual": false
},
{
"path": "scripts/test-foundation.sh",
"r1Sha256": "2e11038e7f784ec4f539eb768f63bab0fab9c9940334388a1fe2d47ab9e2b008",
"r2Sha256": "2e11038e7f784ec4f539eb768f63bab0fab9c9940334388a1fe2d47ab9e2b008",
"mode": "0o755",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "scripts/test-queue.sh",
"r1Sha256": "c270f181d1fa8b6d42ff4131c339beba786a8fd620bb66f8afdcec05e8b10868",
"r2Sha256": "c270f181d1fa8b6d42ff4131c339beba786a8fd620bb66f8afdcec05e8b10868",
"mode": "0o755",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "scripts/testing/README.md",
"r1Sha256": "845db994e648f29f10efb5e6a4ddcf4be4e28798fc27f1eddf9d150f0e8b5319",
"r2Sha256": "845db994e648f29f10efb5e6a4ddcf4be4e28798fc27f1eddf9d150f0e8b5319",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "scripts/testing/gate-lane.mjs",
"r1Sha256": "2e5decf42a546b1441c199337c9a3162693051f2299878f617dd039d6231f226",
"r2Sha256": "2e5decf42a546b1441c199337c9a3162693051f2299878f617dd039d6231f226",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "scripts/testing/gate-lane.sh",
"r1Sha256": "41804d964c55dd07b6224855aae9dc44037f4e80c515c10f98fe04d6d3fa0084",
"r2Sha256": "41804d964c55dd07b6224855aae9dc44037f4e80c515c10f98fe04d6d3fa0084",
"mode": "0o755",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "scripts/testing/lane-docker.mjs",
"r1Sha256": "fbe54165439923a852f64d69a6188ef29d237755289e9e21ddbfdab00e9ef933",
"r2Sha256": "fbe54165439923a852f64d69a6188ef29d237755289e9e21ddbfdab00e9ef933",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "scripts/testing/lane-evidence.mjs",
"r1Sha256": "51d4dcff4ab01730b923e0a59c218f9a3d33e6ba729be1def03703bc457fb37e",
"r2Sha256": "51d4dcff4ab01730b923e0a59c218f9a3d33e6ba729be1def03703bc457fb37e",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "scripts/testing/lane-fixtures.mjs",
"r1Sha256": "8634070d68bca694c8a58911eda9576991ed0eca1eb055a2c1bbf10261305939",
"r2Sha256": "8634070d68bca694c8a58911eda9576991ed0eca1eb055a2c1bbf10261305939",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "scripts/testing/lane-image.mjs",
"r1Sha256": "2259045913b4e3042b7afd3ff6a0fb9c2cc9b4cf359f943e4dce9b8658b91982",
"r2Sha256": "2259045913b4e3042b7afd3ff6a0fb9c2cc9b4cf359f943e4dce9b8658b91982",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
},
{
"path": "scripts/testing/lane-state.mjs",
"r1Sha256": "c37b3d5ec3025948dd8150d18d4ccc4eeb3539739ca37e4d86edede5bcafd1a8",
"r2Sha256": "c37b3d5ec3025948dd8150d18d4ccc4eeb3539739ca37e4d86edede5bcafd1a8",
"mode": "0o644",
"landedPathChanged": false,
"r1BlobEqual": true
}
]
}
@@ -0,0 +1,19 @@
packages/cli/tests/gate-lane-fixtures.test.mjs
packages/cli/tests/gate-lane-image.test.mjs
packages/cli/tests/gate-lane-state.test.mjs
packages/cli/tests/gate-lane.test.mjs
packages/cli/tests/stack-instances.test.mjs
packages/cli/tests/stack-release-external.test.mjs
packages/cli/tests/stack-release.test.mjs
packages/cli/tests/stack-upgrade.test.mjs
packages/control-board/tests/serve.test.mjs
packages/harness/tests/claude-session.test.mjs
packages/harness/tests/pi-session.test.mjs
packages/webui/tests/preview.test.mjs
packages/webui/tests/proof-cli.test.mjs
packages/webui/tests/serve.test.mjs
packages/webui/tests/service.test.mjs
packages/webui/tests/tester-cli.test.mjs
scripts/test-discord.sh
scripts/test-foundation.sh
scripts/test-queue.sh
@@ -0,0 +1,45 @@
diff --git a/packages/cli/tests/stack-release.test.mjs b/packages/cli/tests/stack-release.test.mjs
index 8788158c3..a78f52062 100644
--- a/packages/cli/tests/stack-release.test.mjs
+++ b/packages/cli/tests/stack-release.test.mjs
@@ -0,0 +1,2 @@
+import { leasePort } from '../../../scripts/testing/lane-fixtures.mjs';
+import { after as laneAfter } from 'node:test';
@@ -1777 +1779 @@ const TESTER='7499';
-const freePort=async()=>{const s=createServer();s.listen(0,'127.0.0.1');await once(s,'listening');const p=String(s.address().port);await new Promise((r)=>s.close(r));return p};
+const freePort=async()=>String(await leasePort({after:laneAfter},{closed:true}));
diff --git a/packages/webui/tests/service.test.mjs b/packages/webui/tests/service.test.mjs
index a6e66608a..adb460d07 100644
--- a/packages/webui/tests/service.test.mjs
+++ b/packages/webui/tests/service.test.mjs
@@ -0,0 +1,2 @@
+import { leasePort } from '../../../scripts/testing/lane-fixtures.mjs';
+import { after as laneAfter } from 'node:test';
@@ -48,9 +50 @@ function filled(flags, root = REPO) {
-async function closedPort() {
- const server = createServer();
- server.listen(0, '127.0.0.1');
- await once(server, 'listening');
- const { port } = server.address();
- server.close();
- await once(server, 'close');
- return port;
-}
+async function closedPort() { return leasePort({after:laneAfter}, {closed:true}); }
diff --git a/scripts/test-discord.sh b/scripts/test-discord.sh
index 114994936..4daf0d143 100755
--- a/scripts/test-discord.sh
+++ b/scripts/test-discord.sh
@@ -73,0 +74,2 @@ TOOLS_JSON="{\"roots\":[{\"name\":\"docs\",\"path\":\"$SANDBOX/toolroot\"}],\"ma
+export PI_CODING_AGENT_DIR="$SANDBOX/pi-agent"
+mkdir -p "$PI_CODING_AGENT_DIR"
@@ -145 +147,8 @@ fi
-node_tests() { env -u NODE_TEST_CONTEXT node --test "$@"; }
+node_tests() {
+ local lane_concurrency=()
+ if [ -n "${MOSAIC_TEST_CONCURRENCY:-}" ]; then
+ [ "$MOSAIC_TEST_CONCURRENCY" = 2 ] || return 2
+ lane_concurrency=(--test-concurrency=2)
+ fi
+ env -u NODE_TEST_CONTEXT node --test "${lane_concurrency[@]}" "$@"
+}
@@ -0,0 +1,235 @@
{
"base": "c771a36f4bd0b6c359282425c668996a90072f93",
"priorCandidate": "a3b7cbde8ef51c206a83769a7c272e4f9a1de5bc",
"originalCandidate": "6318f0d0bba9ee90ccaa3503cac8c2e04c6397f2ef6c3564f687fbc2092989e3",
"overlap": [
"packages/cli/tests/stack-release.test.mjs"
],
"rows": [
{
"path": "packages/cli/tests/fixtures/gate-lane-child.mjs",
"r1Sha256": "209682b968e929da4b46dc97c64fcd7aa68c490f3c6be44595f5d85f8d2e1483",
"priorCandidateSha256": "209682b968e929da4b46dc97c64fcd7aa68c490f3c6be44595f5d85f8d2e1483",
"newSha256": "209682b968e929da4b46dc97c64fcd7aa68c490f3c6be44595f5d85f8d2e1483",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "packages/cli/tests/gate-lane-fixtures.test.mjs",
"r1Sha256": "29afa181804a724c695678eb9459c156e00f1084647c704e814c5bee156c0c25",
"priorCandidateSha256": "29afa181804a724c695678eb9459c156e00f1084647c704e814c5bee156c0c25",
"newSha256": "29afa181804a724c695678eb9459c156e00f1084647c704e814c5bee156c0c25",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "packages/cli/tests/gate-lane-image.test.mjs",
"r1Sha256": "0253621a49c2e77ffcdbf32beee7e1f773ec2c52c8487dcc5aa254e61807af7c",
"priorCandidateSha256": "0253621a49c2e77ffcdbf32beee7e1f773ec2c52c8487dcc5aa254e61807af7c",
"newSha256": "0253621a49c2e77ffcdbf32beee7e1f773ec2c52c8487dcc5aa254e61807af7c",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "packages/cli/tests/gate-lane-state.test.mjs",
"r1Sha256": "948d62c76447d8269f03518562a261564c202ec9207461fb21d2b1c11c9bb9fc",
"priorCandidateSha256": "948d62c76447d8269f03518562a261564c202ec9207461fb21d2b1c11c9bb9fc",
"newSha256": "948d62c76447d8269f03518562a261564c202ec9207461fb21d2b1c11c9bb9fc",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "packages/cli/tests/gate-lane.test.mjs",
"r1Sha256": "240bf0c15025ad6f88b251917a3c8d72bd21c2e5df6771e1ceb83a1d90b24a51",
"priorCandidateSha256": "240bf0c15025ad6f88b251917a3c8d72bd21c2e5df6771e1ceb83a1d90b24a51",
"newSha256": "240bf0c15025ad6f88b251917a3c8d72bd21c2e5df6771e1ceb83a1d90b24a51",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "packages/cli/tests/stack-release.test.mjs",
"r1Sha256": "8ca733a600d5f5534d7e14d8533e1b16d710ebd43080ef8421238b5dd145fade",
"priorCandidateSha256": "b95c0c125b6a283961e24adc07d43150bcbd05d2f11005180b836043015c52a6",
"newSha256": "071ea2803b8c7e0450bb4c86ac9b49ae7c180af1599dca52bfce6d0274ca205b",
"unchangedFromPrior": false,
"overlaps234": true,
"mode": "0o644"
},
{
"path": "packages/control-board/tests/serve.test.mjs",
"r1Sha256": "4838cb4609da28fc921370c5542a82402fbc9ceb469692a36f0eafc713bb56c8",
"priorCandidateSha256": "4838cb4609da28fc921370c5542a82402fbc9ceb469692a36f0eafc713bb56c8",
"newSha256": "4838cb4609da28fc921370c5542a82402fbc9ceb469692a36f0eafc713bb56c8",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "packages/harness/tests/claude-session.test.mjs",
"r1Sha256": "0a09b0255e9144a039428eb8e9f56181f94ccb2d187e474ea13392b485fdd008",
"priorCandidateSha256": "0a09b0255e9144a039428eb8e9f56181f94ccb2d187e474ea13392b485fdd008",
"newSha256": "0a09b0255e9144a039428eb8e9f56181f94ccb2d187e474ea13392b485fdd008",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "packages/harness/tests/pi-session.test.mjs",
"r1Sha256": "6db5c3b7063baf933061e91e8bdf66740717dbb3066be3b70ac779fb17022094",
"priorCandidateSha256": "6db5c3b7063baf933061e91e8bdf66740717dbb3066be3b70ac779fb17022094",
"newSha256": "6db5c3b7063baf933061e91e8bdf66740717dbb3066be3b70ac779fb17022094",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "packages/webui/tests/preview.test.mjs",
"r1Sha256": "ada49ec100d75b3e739263593164b2798c378f3df0f38d73d80cf011ef16e234",
"priorCandidateSha256": "ada49ec100d75b3e739263593164b2798c378f3df0f38d73d80cf011ef16e234",
"newSha256": "ada49ec100d75b3e739263593164b2798c378f3df0f38d73d80cf011ef16e234",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "packages/webui/tests/proof-cli.test.mjs",
"r1Sha256": "d3cec62f6eb8e2c1e0747f2a1bf1fc3f1a2122e45870d5a416d1830d3b6182cb",
"priorCandidateSha256": "d3cec62f6eb8e2c1e0747f2a1bf1fc3f1a2122e45870d5a416d1830d3b6182cb",
"newSha256": "d3cec62f6eb8e2c1e0747f2a1bf1fc3f1a2122e45870d5a416d1830d3b6182cb",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "packages/webui/tests/serve.test.mjs",
"r1Sha256": "36ecdfcb3d0d9a2411eec751421f770b5bb41ec2fbf9236c5794fb241fd3196b",
"priorCandidateSha256": "36ecdfcb3d0d9a2411eec751421f770b5bb41ec2fbf9236c5794fb241fd3196b",
"newSha256": "36ecdfcb3d0d9a2411eec751421f770b5bb41ec2fbf9236c5794fb241fd3196b",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "packages/webui/tests/service.test.mjs",
"r1Sha256": "c915239f0c223209baf84c96ec9f5ae2845fce7dd98ad46b073a96033aa7d6df",
"priorCandidateSha256": "845313239547ddbc478090b29f96385f0b8853e5087c572393be328e7075f578",
"newSha256": "845313239547ddbc478090b29f96385f0b8853e5087c572393be328e7075f578",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "packages/webui/tests/tester-cli.test.mjs",
"r1Sha256": "8bf4491f7982399b4162cfbe36d0b68888e99ceb2cbaefefc2361732dd95e0c8",
"priorCandidateSha256": "8bf4491f7982399b4162cfbe36d0b68888e99ceb2cbaefefc2361732dd95e0c8",
"newSha256": "8bf4491f7982399b4162cfbe36d0b68888e99ceb2cbaefefc2361732dd95e0c8",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "scripts/test-discord.sh",
"r1Sha256": "c10af53b0717344026cb22462964f253c7401c5082ac6ef3722a89d9dfa5c661",
"priorCandidateSha256": "a5dcf783c51a5d5d184a357f91ebf9fa1c4e054f0cb84a3d1d7668f6156e025e",
"newSha256": "a5dcf783c51a5d5d184a357f91ebf9fa1c4e054f0cb84a3d1d7668f6156e025e",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o755"
},
{
"path": "scripts/test-foundation.sh",
"r1Sha256": "2e11038e7f784ec4f539eb768f63bab0fab9c9940334388a1fe2d47ab9e2b008",
"priorCandidateSha256": "2e11038e7f784ec4f539eb768f63bab0fab9c9940334388a1fe2d47ab9e2b008",
"newSha256": "2e11038e7f784ec4f539eb768f63bab0fab9c9940334388a1fe2d47ab9e2b008",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o755"
},
{
"path": "scripts/test-queue.sh",
"r1Sha256": "c270f181d1fa8b6d42ff4131c339beba786a8fd620bb66f8afdcec05e8b10868",
"priorCandidateSha256": "c270f181d1fa8b6d42ff4131c339beba786a8fd620bb66f8afdcec05e8b10868",
"newSha256": "c270f181d1fa8b6d42ff4131c339beba786a8fd620bb66f8afdcec05e8b10868",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o755"
},
{
"path": "scripts/testing/README.md",
"r1Sha256": "845db994e648f29f10efb5e6a4ddcf4be4e28798fc27f1eddf9d150f0e8b5319",
"priorCandidateSha256": "845db994e648f29f10efb5e6a4ddcf4be4e28798fc27f1eddf9d150f0e8b5319",
"newSha256": "845db994e648f29f10efb5e6a4ddcf4be4e28798fc27f1eddf9d150f0e8b5319",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "scripts/testing/gate-lane.mjs",
"r1Sha256": "2e5decf42a546b1441c199337c9a3162693051f2299878f617dd039d6231f226",
"priorCandidateSha256": "2e5decf42a546b1441c199337c9a3162693051f2299878f617dd039d6231f226",
"newSha256": "2e5decf42a546b1441c199337c9a3162693051f2299878f617dd039d6231f226",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "scripts/testing/gate-lane.sh",
"r1Sha256": "41804d964c55dd07b6224855aae9dc44037f4e80c515c10f98fe04d6d3fa0084",
"priorCandidateSha256": "41804d964c55dd07b6224855aae9dc44037f4e80c515c10f98fe04d6d3fa0084",
"newSha256": "41804d964c55dd07b6224855aae9dc44037f4e80c515c10f98fe04d6d3fa0084",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o755"
},
{
"path": "scripts/testing/lane-docker.mjs",
"r1Sha256": "fbe54165439923a852f64d69a6188ef29d237755289e9e21ddbfdab00e9ef933",
"priorCandidateSha256": "fbe54165439923a852f64d69a6188ef29d237755289e9e21ddbfdab00e9ef933",
"newSha256": "fbe54165439923a852f64d69a6188ef29d237755289e9e21ddbfdab00e9ef933",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "scripts/testing/lane-evidence.mjs",
"r1Sha256": "51d4dcff4ab01730b923e0a59c218f9a3d33e6ba729be1def03703bc457fb37e",
"priorCandidateSha256": "51d4dcff4ab01730b923e0a59c218f9a3d33e6ba729be1def03703bc457fb37e",
"newSha256": "51d4dcff4ab01730b923e0a59c218f9a3d33e6ba729be1def03703bc457fb37e",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "scripts/testing/lane-fixtures.mjs",
"r1Sha256": "8634070d68bca694c8a58911eda9576991ed0eca1eb055a2c1bbf10261305939",
"priorCandidateSha256": "8634070d68bca694c8a58911eda9576991ed0eca1eb055a2c1bbf10261305939",
"newSha256": "8634070d68bca694c8a58911eda9576991ed0eca1eb055a2c1bbf10261305939",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "scripts/testing/lane-image.mjs",
"r1Sha256": "2259045913b4e3042b7afd3ff6a0fb9c2cc9b4cf359f943e4dce9b8658b91982",
"priorCandidateSha256": "2259045913b4e3042b7afd3ff6a0fb9c2cc9b4cf359f943e4dce9b8658b91982",
"newSha256": "2259045913b4e3042b7afd3ff6a0fb9c2cc9b4cf359f943e4dce9b8658b91982",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
},
{
"path": "scripts/testing/lane-state.mjs",
"r1Sha256": "c37b3d5ec3025948dd8150d18d4ccc4eeb3539739ca37e4d86edede5bcafd1a8",
"priorCandidateSha256": "c37b3d5ec3025948dd8150d18d4ccc4eeb3539739ca37e4d86edede5bcafd1a8",
"newSha256": "c37b3d5ec3025948dd8150d18d4ccc4eeb3539739ca37e4d86edede5bcafd1a8",
"unchangedFromPrior": true,
"overlaps234": false,
"mode": "0o644"
}
]
}
@@ -0,0 +1,47 @@
Compared 57f59e0c3 to c771a36f4bd0b6c359282425c668996a90072f93
agents/gizmoduck/work/queue-234/BUILD.md
docs/SESSIONS.md
packages/bus/README.md
packages/bus/schema.sql
packages/bus/src/broker.mjs
packages/bus/src/credentials.mjs
packages/bus/src/host-grant.mjs
packages/bus/src/host-run.mjs
packages/bus/src/migrations/3i-3j.sql
packages/bus/src/runtime.mjs
packages/bus/src/server.mjs
packages/bus/src/store.mjs
packages/bus/tests/account-migration.test.mjs
packages/bus/tests/chief-of-staff.test.mjs
packages/bus/tests/comment-migration.test.mjs
packages/bus/tests/fixtures/schema-3i.sql
packages/bus/tests/grant-enroll.test.mjs
packages/bus/tests/grant-migration.test.mjs
packages/bus/tests/hard-waits.test.mjs
packages/bus/tests/host-fixtures.mjs
packages/bus/tests/host-run.test.mjs
packages/bus/tests/host-tools.test.mjs
packages/bus/tests/name-migration.test.mjs
packages/bus/tests/project-runtime.test.mjs
packages/bus/tests/schema.test.mjs
packages/bus/tests/spec-fixtures.mjs
packages/bus/tests/spec-migration.test.mjs
packages/bus/tests/thread-migration.test.mjs
packages/bus/tests/usage-migration.test.mjs
packages/business/src/business.mjs
packages/business/src/hosts.mjs
packages/business/tests/hosts.test.mjs
packages/cli/README.md
packages/cli/src/cli.mjs
packages/cli/src/hosts.mjs
packages/cli/tests/hosts.test.mjs
packages/cli/tests/stack-release.test.mjs
scripts/mosaic
tools/host/mosaic-keys
tools/host/mosaic-keys-write
tools/host/mosaic-keys.sshd.conf
tools/host/mosaic-keys.sudoers
Overlap with row 197 source manifest:
packages/cli/tests/stack-release.test.mjs
Other 24 source paths: no overlap.
@@ -0,0 +1,10 @@
diff --git a/packages/cli/tests/stack-release.test.mjs b/packages/cli/tests/stack-release.test.mjs
index daae6efb7..910785bc4 100644
--- a/packages/cli/tests/stack-release.test.mjs
+++ b/packages/cli/tests/stack-release.test.mjs
@@ -0,0 +1,2 @@
+import { leasePort } from '../../../scripts/testing/lane-fixtures.mjs';
+import { after as laneAfter } from 'node:test';
@@ -1779 +1781 @@ const TESTER='7499';
-const freePort=async()=>{const s=createServer();s.listen(0,'127.0.0.1');await once(s,'listening');const p=String(s.address().port);await new Promise((r)=>s.close(r));return p};
+const freePort=async()=>String(await leasePort({after:laneAfter},{closed:true}));
+71
View File
@@ -0,0 +1,71 @@
# Row 241: one decision lands once (#1747, P19 441)
Megavolt, 2026-10-21 (from `date -u`). Brief: `docs/plans/2026-10-20_agent-credentials.md`
§ "Brief: row 237 follow-up, one decision lands once, with tests at the broker", c25785d3d. Source:
row 237 residual 3, megavolt-0068, megavolt-0069 and sage-1651. Base: origin/refactor f5753c0e8, row 210's landing.
Reviewer Duckworth, gate owner Sage. Tests only; no source changes.
## Why landing has no row 237 race
Row 237 fixed publish: two tickets on one decision both pushed, because the decision is spent only
after the push, and a push to a remote fork can't be taken back. Landing has the same ticket shape
(`checkLand` opens a ticket, `recordLand` spends the decision), but three guards keep one decision
from landing twice:
1. **land-busy, the outer guard.** `doLand` in `packages/cli/src/git-socket.mjs:351` refuses
`land-busy` while another landing runs. The slot is taken before the first await, and freed only
when the landing ends. There is one git socket per launcher, and one host per dataRoot (host.lock).
So through the socket two landings never overlap, and a later one stops at `checkLand` with
`decision-consumed`. Even past that, applying the same reviewed patch to the new main fails
`base-conflict` in `prepareLand`.
2. **The CAS on main.** `finishLand` (`packages/seat/src/working.mjs:391`) moves main with
`update-ref main <commit> <old>`, so a landing prepared on a tip that has since moved refuses
`base-moved` and writes nothing.
3. **The recordLand recheck.** `finishLand` calls `record()`, which is `recordLand`, under the same
repository lock. `recordLand` runs `#landFacts` again (`packages/bus/src/broker.mjs:963`), which
refuses `decision-consumed` once the decision is spent. When `record()` throws, `finishLand` moves
main back.
Guards 2 and 3 can't be reached end to end, because guard 1 stops a second landing first. So they are
pinned below the socket, and no test bypasses land-busy.
### The unrecorded-commit window
Between the `update-ref` and the end of `record()`, main names the squash, which isn't recorded yet.
That window is under the repository's lock, so no other push, review or landing in this host sees
it. If `record()` refuses, main moves back. If moving back fails, `finishLand` refuses
`rollback-failed` and main keeps naming an unrecorded commit (tested at `working.test.mjs:555`, the
main.lock case). That is existing behaviour, unchanged here.
## The test
`packages/bus/tests/land.test.mjs:312`. Two tickets pass `checkLand` on one decision before either
records. `recordLand` on the first lands. `recordLand` on the second refuses `decision-consumed`
(:319), and leaves one merge `action.allowed`, one RESULT and no `action.refused`. `landRefused`, as
the socket would call it, then files the refusal and ends the ticket.
**The assert that pins the recordLand recheck is :319.** The `#spent` line is shared with
`checkLand`, so dropping it also fails :144 (a `checkLand` after the decision is spent). Only :319
shows the recheck at record time.
Single file on this branch: land.test.mjs 18/18.
Whole suites on this branch before in-review: bus 500/500, seat 164/164, cli 565/565 (02:52 to 03:01Z).
## Mutants
| Mutant | Change | Killed by |
|---|---|---|
| M1 | drop the `#spent` line in `#landFacts` (broker.mjs:963) | land.test.mjs: :312 at :319, and :94 at :144 (16/2) |
| M2 | drop `land.old` from the `update-ref` in `finishLand` (working.mjs:391) | working.test.mjs:555 at :567, the `base-moved` assert (25/1) |
| M3 | drop the `land-busy` throw (git-socket.mjs:351) | git-socket.test.mjs:812 at :825, the `land-busy` assert (23/1) |
Each run was one test file on base f5753c0e8 (row 210's landing, which moved the `#spent` line from
:945 to :963), 2026-10-21 02:51Z. Each assert failed with "Missing expected exception" or "rejection".
`cmp` confirmed each source matched its saved copy after the restore. M2 is killed by the existing
working.test:555, so per the brief working.test.mjs isn't changed. Logs: `/mnt/storage/scratch/megavolt-r241/`
(m1.log to m3.log, with each diff).
## Gate
Per the brief: Duckworth approves the manifest; the bus, seat and cli suites and every
`scripts/test-*.sh` are green on Sage's gate; the three mutants are killed.
+108
View File
@@ -0,0 +1,108 @@
# Row 242 build: a retry after publish-timeout pushes once
Row 242, #1748, P19 440. Brief: docs/plans/2026-10-20_agent-credentials.md, "Brief: row 237
follow-up N1, a retry after publish-timeout pushes once" (9d165be8c). Base: origin/refactor
8fede5fc6 (row 210 landed f5753c0e8).
## The defect
The git socket polls publishResult until publishDeadlineMs, then refuses publish-timeout
(packages/cli/src/git-socket.mjs:433) while the bus process's push (runtime publishPush) can
still be running. The socket then calls publishRefused, which went straight to
broker.publishRefused: it filed the refusal and deleted the ticket. checkPublish no longer saw an
open ticket on the decision, so a retry got a new ticket and pushed a second ref while the first
push was still on its way to the fork.
## Fix chosen: hold the decision until the push ends, then record a push that reached the fork
packages/bus/src/runtime.mjs, `publishRefused` and `settle`:
- If the bus has no push for the ticket, publishRefused goes to the broker as before.
- If the push is still running, the refusal code is held on the entry and the ticket stays open.
checkPublish refuses another publish on the decision with publish-busy (row 237's check),
the same as when the socket is still polling.
- When the push ends, `settle` runs:
- A push that reached the fork is recorded with broker.recordPublish, which spends the
decision. The bus logs `pushed after the git socket refused <code>; recorded`.
- A push that failed is filed as publishRefused with the socket's code.
- If recordPublish refuses with a BusError, that code is filed instead.
- If the push has already ended when the refusal arrives, it's settled at once. The sage-1440 test
relies on this: its push fails fast, so publish-timeout is filed once, immediately, as before.
Why hold rather than kill. Killing the git client doesn't undo a push the fork has already
received: receive-pack can update the ref after the client dies. After a kill the bus would have
to ls-remote the fork to learn what landed, and the result could still race the server. Holding
waits for the push's own outcome. Each git run in publishRemote already has its own limit
(publishRemote defaults to checkout.mjs GIT_LIMITS), so a hold is bounded. Holding also reuses the existing publish-busy
state, and the refusal text is unchanged.
Why record and not only hold. If the hold just filed publish-timeout once the push ended, the
ticket would close and the decision would be free again with a ref on the fork. A retry would then
push a second ref. Recording the late push spends the decision, so the retry gets
decision-consumed.
The publish-timeout refusal text is unchanged (brief condition).
## Test
packages/cli/tests/publish-socket.test.mjs:302, "row 242: a retry after publish-timeout refuses
publish-busy while the bus process's push runs; the late push is recorded and the decision pushes
once".
- publishDeadlineMs 150 and a store token. The fork's pre-receive hook writes `held`, then loops
until the test writes `release`.
- The first publish gives publish-timeout. The test waits for `held`.
- A retry to branch `other` gives publish-busy (:322). Refused events: ["publish-busy"]. No
git.push.remote allowed event.
- The test writes `release` and waits until there is exactly one allowed event, with ref
refs/heads/coder/feat and the work commit.
- The next retry gives decision-consumed (:330).
- Exactly one git-receive-pack request reached the fake Gitea, and the fork has only
refs/heads/coder/feat. Refused events: ["publish-busy", "decision-consumed"]. No TOKEN appears in
events or logs.
Fail fast (sage-1681): the body is wrapped in try/finally, and the finally writes `release`. Under
the first mutant run, the assert at the retry failed and the hook was never released. setup's
teardown waits for the fake Gitea, which waited for the held push, so the file hung until I stopped
it. Now a failed assert ends the hook and the run. Each wait is bounded at 10 s with an assert.fail.
## Mutants
Script: /mnt/storage/scratch/megavolt-r242/mutants.sh. Each mutant: save, one-line sed, run
publish-socket.test.mjs whole, restore, cmp.
| Mutant | Change | Expected kill |
|---|---|---|
| M1 | export `publishRefused: (ticket, args) => broker.publishRefused(ticket, args)` again (removes the fix) | :302 at :322: the retry gets a new ticket and its own push, so publish-timeout, not publish-busy (seen in the stopped 03:06Z run) |
| M2 | `settle` files publishRefused without trying recordPublish (hold only) | :302 at the wait for one allowed event, then :330 |
Results (Sage's slot, sage-1686, 2026-10-21T03:35:00Z to 03:35:15Z): both killed, both restored
(cmp ok).
- M1: publish-socket 8/1, the row 242 test fails at :322 in 0.5 s.
- M2: publish-socket 8/1, the row 242 test fails at :327, the wait for one allowed event, through
the assert.fail inside `until` (:311) after its 10 s bound, in 10.3 s.
## Suites
On Sage's slot (sage-1686), on base 8fede5fc6 plus this row:
- bus `node --test tests/*.test.mjs`: 499/499, 03:35:20Z to 03:35:34Z. This is one fewer than
Sage's 241 gate (500) because the base predates row 241's land.test case.
- cli `node --test tests/*.test.mjs`: 566/566, 03:35:34Z to 03:43:47Z. That's 565 plus the
row 242 test.
Singles before the slot: publish-socket.test.mjs 9/9, bus publish.test.mjs 13/13. The seat suite
and scripts/test-*.sh are left for Sage's landing gate.
## Residuals
- The agent was told publish-timeout, but the push is recorded later and the run isn't
notified. The allowed event and the bus log line are the only trace. A later publish on the
decision gets decision-consumed, which tells the agent the decision was used. Whether the run
should get a RESULT for a late record is a follow-up question for Sage.
- A late push whose recordPublish refuses (Duckworth N2) is filed with that code: the ref stays on
the fork and the decision is not spent, the same residual as an in-band push whose record
refuses.
- The hold lives in the bus process's memory, like the broker's open tickets. A bus restart during
a hold drops both, the same as today for any publish in flight.
@@ -20,12 +20,12 @@ so Mos doesn't check this draft. Draft 1 names NEW and goes to Mos for the P0 ch
- OLD, live: `4429a104f6d65ccf165f6c83b5d1dce3147dfde0`, schema 3i, pointer `pm: true` (record 99a406ed3).
P1 re-reads the activation id and the Console flags from `status`. I don't copy them from the last window.
- NEW: open. It's the row 234 landing, or a later code landing if Mos rules. Candidates in landing order:
194, 234 (210 and 237 have landed).
- NEW: `c771a36f4bd0b6c359282425c668996a90072f93`, the row 234 landing (2026-10-21 06:16Z), unless Mos rules a
later code landing. 194 is the open candidate. 210, 237, 232, 241, 242, 233 and 234 have landed.
- REPO, `P`, `WIN=agents/sage/work/window-repin-<NEW12>/`, `SNAP`: as in `repin-4429a104f.md`, with the
sha checks at P0 and before S1, S2 and any D3.
## What NEW carries over OLD (so far, refactor 8fede5fc6)
## What NEW carries over OLD (refactor c771a36f4)
`git log --first-parent 4429a104f..HEAD`, path-limited to packages, scripts, roles, contracts, adapters, skills
and the package files:
@@ -41,7 +41,11 @@ and the package files:
| 230 | c8bc5cf4c | Case 2 scripts, tools only. Not run live. |
| 210 | f5753c0e8 | Host access. The live business has no `hosts`, so nothing is granted. Bus start runs the host-run sweep (M34d). P3 must show it removes nothing. |
| 237 | d7d943199 | Publish busy check (199 N1). |
| 234 | open | Machine grant row a and the 3i to 3j schema step. `--plan` must say `migration: 3i -> 3j`. |
| 232 | e65a3ab4d | Installer I1: `stack-release.sh --units external` and `stack-copy.sh copy --bundle`, `wrapper --create`. Both are opt-in, and this window uses neither. The default units path must render the same units, so the P2 unit diff covers it with row 190. P0 checks that no window command line carries `--units external`. |
| 241 | 5d4a61776 | Test only (bus land.test.mjs). No live effect. |
| 242 | 5fb736b92 | Bus publish hold: after publish-timeout the bus holds the decision until its push ends, then records a late push. The live business has no publish in flight at W1, and P3 shows none open. Its follow-up fc1cb856b is comments only (git-socket.mjs). |
| 233 | 57f59e0c3 | Installer I3: new tool `scripts/stack-upgrade.sh` and a `stack-release.sh` change. This window doesn't run `stack-upgrade.sh`. The P2 unit diff covers the `stack-release.sh` change with rows 190 and 232. |
| 234 | c771a36f4 | Machine grant row a and the 3i to 3j schema step (new tables `hosts` and `host_grants`). `--plan` must say `migration: 3i -> 3j`. |
| PRD 1.7+1.8 | 0adf8ddfe | `docs/prd/mosaic-stack.md` and `packages/tasks/requirements.json` move to version 1.8 (mos-1154, mos-1156, DECISIONS Q192). The live pm cites 1.6 until this window. Check after: the pm's requirements read answers version 1.8. |
Service script drift OLD to HEAD is 415+/91- over the four scripts, mostly row 190. P5 holds them at NEW.
@@ -53,7 +57,14 @@ Service script drift OLD to HEAD is 415+/91- over the four scripts, mostly row 1
- W0b: the business file copy, as in the last window.
- Rollback after a migration isn't the plain rollback. OLD is 3i and the store is 3j, so going back needs
`--restore <W5 activation>`, and that loses the events written after W5. The row 181 restore guard counts
11 tables, and the untimed ones need the D3 snapshot. Mos decides any restore past W5, as before.
11 tables, and the untimed ones need the D3 snapshot. Row 234 adds `hosts` and `host_grants` to the untimed
list (T46), so at 3j the D3 snapshot covers them too. Draft 1 recounts the list from NEW. Mos decides any restore past W5, as before.
- Grant look-alikes (row 234 A22b, gizmoduck-0107). `3i-3j.sql` adds `events_grant_actor` but doesn't check
rows already in the store, so after the migration only the actor filter in `GRANT_EVIDENCE` (actor_role NULL,
actor_run 'grant') ignores an old session-written row. P3, and again just before `--migrate`: on the
backup copy, count the `action.allowed` events whose body.action or body.operation starts with `grant.`, or
whose body has `hostLineLeft`. Expect 0, since no grant code has run live. Any nonzero count stops the window
until I've read those rows.
- P3 is the rehearsal with `--migrate` on a copy of the live backup. It must show the 3j step and the row
counts before and after.
@@ -65,10 +76,15 @@ history unless the window fails.
## Open before draft 1
- Q1: NEW, once 234 lands. Do 194 and 237 go in if they land first? I recommend yes for 237 (small, bus
only) and asking Mos about 194, since 194 changes the lifecycle verbs and roles/chief-of-staff.*.
- Q1, ruled by mos-1161: draft 1 doesn't wait for 194. NEW is the latest code landing when I cut draft 1. If
194 has landed with a green full gate (15 packages, ten script suites), NEW is the 194 landing and P3 also runs
the lifecycle verbs once on the backup copy and gets one pm reply under the new roles/chief-of-staff files. A
P3 failure that traces to 194 drops NEW back to c771a36f4 before anything touches live, and the window goes on.
Otherwise NEW is c771a36f4 and 194 rides a later window.
- Q2, settled by mos-1148: phase 2 is its own window on 10-23. Draft 1 takes OLD and the Console flags,
`--sso` included, from `status` after that window.
- Q3: who operates. If Darkwing is free, Darkwing operates and I check. Otherwise I operate as in the last
window.
- The P-step evidence list for rows 199 and 210 at bus start, written once both have landed.
- The P-step evidence list for rows 199 and 210 at bus start. Both have landed (cb5fdb064, f5753c0e8), so
it's due in draft 1. I take each log line and exit code from a P3 rehearsal run, not from the code.
- Rows 241 and 242 (bus): P3 shows no publish or land decision open before W5.
+8
View File
@@ -1395,3 +1395,11 @@ are never rewritten or removed; corrections are new entries.
2026-10-21T01:49:55Z | Sage (Opus 5.5, 1ef1e4f8) | row 237 landed d7d943199 (#1743, gate green, M1 killed); PRD 1.7+1.8 landed 0adf8ddfe (Honker 2884ee496, Duckworth approve bf52628f9, tasks 97/0 bus 478/0 on the tip, DECISIONS Q192); rows 239 (#1745) and 240 (#1746) added; 233 briefed; queue revs 2540-2603 committed 584e2bd82 | landed, push next
2026-10-21T02:51:46Z | Sage (Opus 5.5, 1ef1e4f8) | row 210 landing (r5 0485384e9, Duckworth r3 29851) | landed f5753c0e8, patch-id 723b1c83 = r5 diff, manifest 688d38b9 OK; gate 2796/0 packages, 429/0 scripts; mutants 52/53 (M22b equivalent); #1707 closed (29853), Vikunja 424 comment 429; queue revs 2604-2617 c084d8d42; row 234 briefed; Megavolt has the slot for 241 (sage-1668)
2026-10-21T03:01:43Z | Filbert (Opus 5.5, 048f3c63) | row 232 installer I1 review r2, rebase scope (#1734, manifest 7af3c94308e0, da2da2f6c on 8fede5fc6) | APPROVE, Gitea 29865, queue rev 2622 (left for Sage's queue-commit); README = predicted 3-way blob 89a95b32, 7 blobs same as r1, BUILD.md +31/-0; corrected my r1 reader claim (host.test.mjs:855 reads cli README)
2026-10-21T04:14:49Z | Sage (Opus 5.5, 1ef1e4f8) | lead: rows 241 and 242 landed, slot order | row 241 landed 5d4a61776 (#1747 closed 29883, M1-M3 killed, gate 2797/0 + 429/0); row 242 landed 5fb736b92 (#1748 closed 29894, Duckworth approve 29889, gate 2826/0 + 429/0); late-record notice ruled a follow-up (Vikunja 440 c432); row 233 brief re-pinned rev 2631; carry items on #1744 (29885) and #1734 (29886); queue revs 2628-2638 committed 96bac588f
2026-10-21T04:50:44Z | Darkwing (Opus 5.5, 7525c8e4) | row 233 I3 gate, mutants and review request (sage-1704, sage-1707, sage-1708) | gate green on d99166ade (1201/0, cli 616/0) and m1-m8 8/8 killed; N1 flock-holder orphan fixed test-only in 08db6ac3f, rerun c0 20/0 + m1 killed, sweep 0 left; in review round 1, manifest 464ce9eb at 1a1dc80ca on darkwing/row233-r1, comment 29901, revs 2644 and 2647 (lock race resolved); Filbert woken
2026-10-21T04:58:01Z | Filbert (Opus 5.5, 048f3c63) | row 233 (I3, #1735) review round 1 on manifest 464ce9eb177a (1a1dc80ca) | APPROVE, Gitea 29922, rev 2659 (left for Sage); static, no slot; 5 non-blocking notes N1-N5 (args-failed exit 1, REPO ownership, signals, run sheet npm/claude path, sections 8/10 precheck)
2026-10-21T05:47:45Z | Sage (Opus 5.5, 1ef1e4f8) | rows 233+234 landing gate, row 233 landing, 240 brief, drift sweep (sage-1742..1758) | combined gate stopped on two row 234 failures (bus schema.test.mjs:522 counts, cli stack-release T46 restore-guard lists; mos-0823); 233 landed alone 57f59e0c3 (patch-id b16ac924, verify-commit ok, cli 617/0, scripts 585/0), pushed with b00d7d94d, 5465fd52d and riders 642a9b87d, 16019b80e; #1735 closed 29931, done rev 2675, Vikunja 432 comment 436; 240 briefed rev 2676; 234 r3 c783d5e39 rebuilt on the landing in Gizmoduck's slot; 197 and 238 drifted, rebase sent (sage-1754, 1755, 1757 stash fix)
2026-10-21T06:20:30Z | Sage (Opus 5.5, 1ef1e4f8) | row 234 r3 landing gate (2890/0 packages, 585/0 scripts) and landing c771a36f4, pushed 06:16Z; #1740 closed 29938, Vikunja 436 c437; 235 briefed rev 2686, rotateBy ruling; 238/228/197 rebuilt on 234, slot order Darkwing 240 then 238, 228, 197; 10-24 draft NEW c771a36f4 (sage-1766..1787) | landed
2026-10-21T06:37:41Z | Darkwing (Opus 5.5, 7525c8e4) | row 240 I4 build, slot run and review request (sage-1759, sage-1773, sage-1788, sage-1795) | r1 on 47ec9fe2f stopped at singles (stack-upgrade 20/1: the N1 test's own after hook threw ENOENT once the fixture removed the root), b45852daa drops it; r2 green: singles 37/0, M1-M8 8/8 killed, cli 639/0, scripts 585/0; darkwing/row240-r3 pushed at ad1b23ab2 on c771a36f4, manifest 573ece95 (9 paths), in-review revs 2694 and 2695, request comment 29943 on #1746, Filbert woken | in review
2026-10-21T06:41:13Z | Filbert (Opus 5.5, 048f3c63) | row 240 (I4, #1746) round 1 review on manifest 573ece95 (ad1b23ab2) | approve, Gitea 29945, rev 2696; 233 carries N1/N4/N5 present; notes N1 boot unit Before= names one business, N2 doc-only, N3 info
2026-10-21T07:15:48Z | Darkwing (Opus 5.5, 7525c8e4) | row 195 (#1693) DESIGN r7, Case 1 install on .53 after the installer split | darkwing/row195-r7 5425ea2f3 pushed (r6 carried as 7398b7dd7 on c515273db), DESIGN.md blob 17dfbad95ba5; business id mosaic-stack named in §7, so DEFERRED 240 N1 doesn't apply (sage-1813); root steps left to ops-01, nothing run on .53 (sage-1809); Duckworth review requested 07:14Z, Gitea 29960; row 226 items (dev.args writer, adopt) sent to Gosalyn and copied to Sage per sage-1814; r7 :21 and :326 still say 226 is unclaimed, to fix next revision
+117 -3
View File
@@ -599,7 +599,8 @@ error. Before anything stops, a run whose Console flags hold `--sso` refuses
`sso-target` for a target Console without it, `sso-config` when the target's own
`checkConsole` refuses the config's `console` key (or there is none, in which
case, from another copy, it names the first step: activate the target without
`--sso`, then write `console.sso`), and `sso-slot` for each provider's slot file
`--sso`, then write `console.sso`; on a named instance both commands carry
`--instance NAME`), and `sso-slot` for each provider's slot file
that is missing, a symlink, not a regular file, not owned by this uid, has group
or other bits, resolves inside the repository or the copies, or is empty. It
uses `lstat`, `stat` and `realpath` on the slot and never reads it. `--plan`
@@ -669,7 +670,7 @@ pinned copy](../packages/cli/README.md#one-pinned-copy-row-85-1579)).
#### Rehearsal on a scratch root (D2, row 163)
```
scripts/stack-release.sh rehearse ID BUSINESS --backup ACTIVATION|PATH --port N [--timeout S]
scripts/stack-release.sh rehearse ID BUSINESS --backup ACTIVATION|PATH|last --port N [--timeout S]
```
`rehearse` starts the target copy's bus host and Console on a copy of a
@@ -680,7 +681,11 @@ runs it against live data. It stops and starts no unit and writes no pointer
or log line.
`--backup` takes an activation id under `<dataRoot>/backups/stack/` or an
absolute path. `--port` is the scratch Console's loopback port, 1024 to
absolute path. `--backup last` (row 233) takes the newest backup the
activation log records and prints `PASS backup-last` naming it. With none in
the log it fails `no-backup`, exits 3 and makes no directory. After the
backup's checks it prints `BACKUP_STORE <activation id> schema <version>`.
`--port` is the scratch Console's loopback port, 1024 to
65535. The run refuses `port-reserved` for 7332, 7333 and the pointer's
`--port` and `--tester-port`, and `port-in-use` for a port something holds;
it never picks another port. `--timeout` (default 60 seconds) bounds the
@@ -779,6 +784,115 @@ instance (the business's `launch.by`), for this business, whose run started at
or after the floor. `status` adds an
`awaiting units:` line for an open activation.
Under `--units external`, `activate --plan` also refuses `open-activation`
while any rollback set holds `AWAITING-UNITS`, and `migrate-required` when
the target migrates the store and the run lacks `--migrate` (row 233). Under
`--units user` the plan's bytes are unchanged.
#### Root's upgrade driver (`scripts/stack-upgrade.sh`, row 233)
```
scripts/stack-upgrade.sh upgrade --instance dev --bundle FILE --sha SHA [--migrate] [--no-rehearsal]
[--rehearse-port N] [--timeout S]
scripts/stack-upgrade.sh rollback --instance dev --to ID [--restore ACT [--events N]] [--timeout S]
```
Root runs it from a root-owned checkout of the reviewed commit, never from a
copy, because the stack's user can write the copies. Each run needs Mos's go,
and `--migrate` needs Jason present (decision 96). The host must be prepared
first (`docs/guides/host-run-sheet.md`). Only the default instance, `dev`, is
accepted; a named instance's units are a later row.
The unit values come from `/etc/mosaic-stack/dev.args`, which I2's install
writes: one `case2-unit.sh` argument per line, the serve flags after `--`,
owned by root with no group or other write, exactly one `--copy`, and that
copy the one both installed units run. Otherwise the run refuses
`args-unreadable` or `args-differ`. The driver writes the new copy id into
that file only after `confirm` passes.
`upgrade`, in order. Every stack user step runs as
`sudo -u USER -H env PATH=<node dir>:/usr/local/bin:/usr/bin:/bin` from the
user's clone `<home>/src/mosaic-stack`:
1. Takes `flock -n` on `/run/mosaic-stack/upgrade-dev.lock` and holds it to
the end, rollback included. A second run refuses `upgrade-running`, and
a lock it can't open refuses `lock-failed`.
2. Fetches the bundle's `refactor` into the clone and fast-forwards it to
SHA (`fetch-failed`), then `stack-copy.sh copy --bundle FILE SHA`
(`copy-failed`).
3. A1: this checkout's HEAD is SHA (`head-mismatch`), the copy exists
(`copy-missing`), `/usr/local/lib/mosaic-stack/hostcheck.sh --check`
exits 0 (`profile-drift`, row 240), both units name one copy
(`units-unreadable`, `units-disagree`) and it is the args file's
(`args-differ`), and
`activate SHA BUSINESS --units external --plan --expect-from OLD` exits 0
(`plan-refused`, which covers `open-activation`, `migrate-required` and
`pointer-differs`).
4. The rehearsal, `rehearse SHA BUSINESS --backup last --port 7339`
(`--rehearse-port`). Exit 0 goes on. Exit 3 means no backup in the log
and goes on only with `--no-rehearsal`, logging `rehearsal-skipped`;
without the flag it refuses `no-backup`. Exit 0 with `--no-rehearsal`
refuses `rehearsal-required`; any other exit refuses `rehearsal-failed`.
5. A2 stops the Console, then the bus, and waits for both to read inactive
or failed. A3 is `activate --units external`, A4 the render with the new
`--copy` (which also starts the bus, then the Console), A5 `confirm`.
The driver writes the new args file beside the old one before A2
(`args-failed`, exit 3) and moves it into place after A5. A failed move
exits 5 `args-failed`: the new copy is live and confirmed, the args file
still names the old one, and root moves the new file (the result line
names it) into place by hand.
A failure in A2 or a stage that exits 3 starts the old units again and gates
them (A2f). A later failure stops both units, runs `revert`, renders the old
copy and runs `confirm --rollback OLD --cause A3|A4|A5`. `rollback` is the
same from A1 on, with `rollback BUSINESS --to ID`, no fetch, no copy and no
rehearsal; `--restore` and `--events` pass through.
Root reads no output the stack user wrote except exit codes and the
rehearsal's `BACKUP_STORE` line, which it copies into its log after a match
against a fixed pattern (`BACKUP_STORE none` otherwise). The step log
`/var/log/mosaic-stack/upgrade-dev.log`, 0640, gets one
`<time> <step> rc=N argv: ...` line per step and a last
`<time> result exit=N <code> <text>` line.
Exit 0 ok, 1 failed with the old stack running and gated, 2 rollback-failed
with both units stopped, 3 refused with nothing stopped, 4 usage, 5
args-failed with the new stack live and confirmed. A refusal
after step 2 leaves the user's clone fast-forwarded and the new copy made. After exit
2 the activation stays open, so the next run's plan refuses until root
finishes it.
#### Root's host check (`scripts/case1-hostcheck.sh`, row 240)
```
/usr/local/lib/mosaic-stack/hostcheck.sh --check | --enforce | --upstream
```
Installed from `scripts/case1-hostcheck.sh` as `hostcheck.sh`, with
`scripts/case1-pins.sh` beside it (`docs/guides/host-run-sheet.md` section 4).
The pins are the profile's sha256, the `apparmor-profiles` version it came
from and the bubblewrap install pin; changing one is a reviewed commit.
Seven items, one `ok N` or `FAIL N` line each: the profile file matches the
pin and no package owns it; `aa-status` has `bwrap` and `unpriv_bwrap` in
enforce mode and no `mosaic-bwrap`; no other file under `/etc/apparmor.d`
names `bwrap`; a bwrap child under the units' no-new-privs reads
`bwrap//&unpriv_bwrap (enforce)`; `/usr/bin/bwrap` is `root mosaic 0750` by
the statoverride and by `stat`; the user-namespace restriction and
`fs.protected_hardlinks` are 1; `net.ipv4.ip_unprivileged_port_start` is
above the Console unit's `--port`, or above 1023 with no unit. A `note` line
says whether bubblewrap is the install pin and never fails.
`--check` exits 0 or 1; `stack-upgrade.sh` refuses `profile-drift` on 1.
`--enforce` exits 0. On a failure, or while `/etc/mosaic-stack/DRIFT` exists,
it unloads the profile if loaded, sets `/usr/bin/bwrap` to `root:root` 0700,
appends the failed items to `DRIFT` and one line to
`/var/log/mosaic-stack/hostcheck.log`. It never writes the statoverride. The
apt hook runs it after every package run, and `mosaic-hostcheck.service`
(`packages/cli/systemd/`) at boot before the bus. `--upstream` compares the
`apparmor-profiles` candidate's profile to the pin, prints any difference
and exits 0, or 1 when the download fails. Exit 2 is usage or a refusal:
root takes no `--root` (the test seam), and other users run only with it.
## Console service (`scripts/webui-service.sh`)
```bash
+281
View File
@@ -0,0 +1,281 @@
# Host run sheet: preparing a host for the stack
ops-01 runs this on a fresh host before install (I2), and once per host for enrollment. Jason places the
credentials in section 9. Brief: `docs/plans/2026-10-20_installer-rows.md`, I3 (row 233, #1735). The
host design is Darkwing's r6 (`agents/darkwing/work/queue-195/DESIGN.md`, section 2) as the brief
changes it.
The hosts are .53 (profile `factory`) and .54 (profile `library`). Both get every section unless a
section says otherwise. Every command runs as root unless it starts with `sudo -u mosaic`.
Each item names the `scripts/case2-precheck.sh` line that verifies it. Install's precheck runs as the
stack's user with `--business ID --profile factory|library`. Where no precheck line checks an item, the
item says "precheck line missing", and Gosalyn adds the line to I2. Today's precheck (C2-13a) has these
gaps; C2-13b adds the library lines (`library.sync`, persona modes, no Discord binding).
## 1. The stack's user and its home
```sh
useradd --system --user-group --home-dir /srv/mosaic --create-home --shell /usr/sbin/nologin mosaic
chmod 0700 /srv/mosaic
install -d -o root -g root -m 0755 /etc/mosaic-stack
install -d -o root -g root -m 0750 /var/log/mosaic-stack
```
`mosaic` gets no sudo rule and no supplementary group. Check with `sudo -l -U mosaic` (expect "not allowed
to run sudo") and `id -G mosaic` (one number).
Precheck lines:
- `ok HOME is /srv/mosaic`
- `ok sudo -n true fails as mosaic`
- `ok id -G prints one group (<gid>)`
- `/srv/mosaic` at 0700: precheck line missing.
## 2. node, pinned under /opt
The pin is the version and sha256 the window names. `<v>` below is that version, for example `v24.x.y`.
```sh
install -d -o root -g root -m 0755 /etc/mosaic-stack/node-keys /opt/mosaic-stack
cd "$(mktemp -d)"
curl -fsSLO https://nodejs.org/dist/<v>/node-<v>-linux-x64.tar.xz
curl -fsSLO https://nodejs.org/dist/<v>/SHASUMS256.txt
curl -fsSLO https://nodejs.org/dist/<v>/SHASUMS256.txt.asc
gpg --no-default-keyring --keyring /etc/mosaic-stack/node-keys/pubring.kbx --verify SHASUMS256.txt.asc SHASUMS256.txt
grep " node-<v>-linux-x64.tar.xz\$" SHASUMS256.txt | sha256sum -c -
echo "<pinned sha256> node-<v>-linux-x64.tar.xz" | sha256sum -c -
tar -xJf node-<v>-linux-x64.tar.xz --no-same-owner
mv node-<v>-linux-x64 /opt/mosaic-stack/node-<v>
chown -R root:root /opt/mosaic-stack/node-<v>
chmod -R go-w /opt/mosaic-stack/node-<v>
```
The keyring holds the Node release keys from the nodejs/release-keys repository, nothing else. Stop on any
failed check; don't unpack. Rollback: `rm -r /opt/mosaic-stack/node-<v>`.
Precheck lines:
- `ok node <v>` (version 24 or later only).
- The `--node` path in the args file must be a root-owned regular file outside the home. `case2-unit.sh`
refuses otherwise at render, which is not a precheck line.
- The installed version equals the pin: precheck line missing.
## 3. claude, factory only (.53)
Root installs it under `/opt`, never under the stack's home (this replaces r6's `/srv/mosaic/.local/bin`).
The pin is 2.1.304 unless the window names a newer one.
```sh
install -d -o root -g root -m 0755 /opt/mosaic-stack/claude-<pin> /etc/claude-code
PATH=/opt/mosaic-stack/node-<v>/bin:/usr/sbin:/usr/bin:/sbin:/bin \
npm install --prefix /opt/mosaic-stack/claude-<pin> @anthropic-ai/claude-code@<pin>
chown -R root:root /opt/mosaic-stack/claude-<pin>
/opt/mosaic-stack/claude-<pin>/node_modules/@anthropic-ai/claude-code/bin/claude.exe --version
printf '%s\n' '{"env":{"DISABLE_AUTOUPDATER":"1"}}' > /etc/claude-code/managed-settings.json
chown root:root /etc/claude-code/managed-settings.json
chmod 0644 /etc/claude-code/managed-settings.json
```
Rollback: `rm -r /opt/mosaic-stack/claude-<pin> /etc/claude-code/managed-settings.json`.
Root's PATH has no npm, so the install line puts section 2's pinned node first; the package's postinstall
also runs `node`. The `--version` line must print `<pin> (Claude Code)`. Anything else means the postinstall
left its 500-byte stub in `bin/claude.exe` (no native package for the platform, or scripts were off): stop and
roll back.
The args file's `--claude` is
`/opt/mosaic-stack/claude-<pin>/node_modules/@anthropic-ai/claude-code/bin/claude.exe`, the regular file the
postinstall hardlinks or copies the native binary to. Not `node_modules/.bin/claude`: that is npm's symlink
to it, and `case2-unit.sh` refuses a symlink, as it does anything that isn't a root-owned regular file outside
the home. Checked against the 2.1.304 package's `install.cjs`; a newer pin needs the same check.
Precheck lines:
- `ok claude replaces its prompt with --system-prompt-file (<n> input tokens)`. It runs the claude the bus
unit's `MOSAIC_CLAUDE_BIN` names, so it needs the model credential from section 9.
- Managed settings present with the autoupdater off: precheck line missing.
## 4. bubblewrap, AppArmor and the drift stop
Both hosts. These are r6's items, unchanged.
```sh
apt-get install bubblewrap=0.9.0-1ubuntu0.3
stat -c '%U:%G %a' /usr/bin/bwrap > /etc/mosaic-stack/bwrap.orig # expect root:root 755
cd "$(mktemp -d)"
apt-get download apparmor-profiles=4.0.1really4.0.1-0ubuntu0.24.04.9
dpkg-deb --fsys-tarfile apparmor-profiles_*.deb | tar -x ./usr/share/apparmor/extra-profiles/bwrap-userns-restrict
echo "11d39094f044f0cda0febb3ad517b830301da6b2ce929664af09ee9e4dd264f9 usr/share/apparmor/extra-profiles/bwrap-userns-restrict" | sha256sum -c -
install -o root -g root -m 0644 usr/share/apparmor/extra-profiles/bwrap-userns-restrict /etc/apparmor.d/bwrap-userns-restrict
apparmor_parser -r /etc/apparmor.d/bwrap-userns-restrict
dpkg-statoverride --update --add root mosaic 0750 /usr/bin/bwrap
```
`aa-status` then lists `bwrap` and `unpriv_bwrap` in enforce mode. Install the one profile file, never the
whole `apparmor-profiles` package, whose 20 other profiles would load at the next boot. Rollback, in
reverse order: `dpkg-statoverride --remove /usr/bin/bwrap`, then `chown` and `chmod` to the values in
`bwrap.orig`; `apparmor_parser -R /etc/apparmor.d/bwrap-userns-restrict`, then `rm` it;
`apt-get remove bubblewrap`.
The no-new-privs test must print `bwrap//&unpriv_bwrap (enforce)`:
```sh
systemd-run --wait --pipe --uid=mosaic --gid=mosaic -p NoNewPrivileges=yes -p ProtectSystem=strict \
-p ProtectHome=yes -p PrivateTmp=yes \
/usr/bin/bwrap --unshare-user --unshare-pid --ro-bind / / cat /proc/self/attr/current
```
The P9 checks: another user's `/usr/bin/bwrap --version` fails with Permission denied, `getent group mosaic`
lists no member, and `sysctl fs.protected_hardlinks` prints 1. `sysctl
kernel.apparmor_restrict_unprivileged_userns` prints 1 too (Ubuntu 24.04's default); leave it, so nothing
else on the host gains user namespaces.
The drift stop (row 240, I4, #1746) goes in last, after the profile and the statoverride. As root, from
the root-owned checkout of the reviewed commit:
```sh
install -d -o root -g root -m 0755 /usr/local/lib/mosaic-stack
install -o root -g root -m 0755 scripts/case1-hostcheck.sh /usr/local/lib/mosaic-stack/hostcheck.sh
install -o root -g root -m 0644 scripts/case1-pins.sh /usr/local/lib/mosaic-stack/case1-pins.sh
printf '%s\n' 'DPkg::Post-Invoke { "/usr/local/lib/mosaic-stack/hostcheck.sh --enforce || true"; };' \
> /etc/apt/apt.conf.d/80mosaic-hostcheck
install -o root -g root -m 0644 packages/cli/systemd/mosaic-hostcheck.service /etc/systemd/system/
systemctl daemon-reload && systemctl enable mosaic-hostcheck.service
/usr/local/lib/mosaic-stack/hostcheck.sh --check
```
`--check` prints one `ok` line for each of its seven items and exits 0. Item 7 reads 1023 as the port
until I2 installs the Console unit (section 10), then the unit's `--port`. `stack-upgrade.sh` runs `--check`
before its plan and refuses `profile-drift` on any exit but 0, a missing file included. Rollback:
`systemctl disable mosaic-hostcheck.service`, then `rm` the unit, the hook file and
`/usr/local/lib/mosaic-stack`.
On drift, `--enforce` (the apt hook after every package run, and the unit at boot) unloads the profile,
closes `/usr/bin/bwrap` to `root:root` 0700, appends the failed items to `/etc/mosaic-stack/DRIFT` and logs
a line in `/var/log/mosaic-stack/hostcheck.log`. Every role session then refuses to start. ops-01 reads the
log in each STATE report and sends any `DRIFT` to Mos. Only root removes `DRIFT`, in a reviewed step with
a new pin or a Mos go. After that, `chown root:mosaic /usr/bin/bwrap`, `chmod 0750 /usr/bin/bwrap` and
`apparmor_parser -r /etc/apparmor.d/bwrap-userns-restrict` put bwrap back, and `--check` must exit 0.
Before each re-pin window, ops-01 runs `/usr/local/lib/mosaic-stack/hostcheck.sh --upstream`. It
downloads the `apparmor-profiles` candidate and compares its `bwrap-userns-restrict` to the pin. A
`DIFFERS` line goes to Mos as a new-pin request; it refuses nothing.
Precheck lines:
- `ok bwrap --unshare-all runs`
- The AppArmor profile, the statoverride, the no-new-privs result and the P9 checks: precheck line missing.
`/usr/local/lib/mosaic-stack/hostcheck.sh --check` covers them (row 240).
## 5. The port floor
Leave `net.ipv4.ip_unprivileged_port_start` at its default (1024) or higher. The Console's port is below
it and the unit gets `CAP_NET_BIND_SERVICE`, so no other `mosaic` process can bind that port.
Precheck line: `ok net.ipv4.ip_unprivileged_port_start <n> is above the Console's port <port>`.
## 6. The release transport
A release is a git bundle of `refs/heads/refactor` plus the 40-character commit id (I1, mos-1118 choice 2).
Mos's go names both, with the bundle's sha256. `<sha>` below is that id.
First install:
```sh
# Copy the bundle in, then check it before anything reads it.
echo "<bundle sha256> /root/<sha>.bundle" | sha256sum -c -
git bundle verify /root/<sha>.bundle
# Root's checkout. stack-upgrade.sh and case2-unit.sh run from here, never from a copy.
git clone -q -b refactor /root/<sha>.bundle /root/mosaic-stack
git -C /root/mosaic-stack checkout -q --detach <sha>
git -C /root/mosaic-stack rev-parse HEAD # must print <sha>
# The stack's user's copy of the bundle, and its own clone.
install -d -o mosaic -g mosaic -m 0700 /srv/mosaic/incoming /srv/mosaic/src
install -o mosaic -g mosaic -m 0600 /root/<sha>.bundle /srv/mosaic/incoming/<sha>.bundle
sudo -u mosaic -H git clone -q -b refactor /srv/mosaic/incoming/<sha>.bundle /srv/mosaic/src/mosaic-stack
```
Each upgrade, before `stack-upgrade.sh upgrade`:
```sh
echo "<bundle sha256> /root/<sha>.bundle" | sha256sum -c -
git bundle verify /root/<sha>.bundle
git -C /root/mosaic-stack fetch -q /root/<sha>.bundle refs/heads/refactor:refs/remotes/origin/refactor
git -C /root/mosaic-stack checkout -q --detach <sha>
install -o mosaic -g mosaic -m 0600 /root/<sha>.bundle /srv/mosaic/incoming/<sha>.bundle
/root/mosaic-stack/scripts/stack-upgrade.sh upgrade --instance dev --bundle /srv/mosaic/incoming/<sha>.bundle --sha <sha>
```
The driver fetches the bundle into the stack's user's clone and makes the copy itself. It refuses
`head-mismatch` when root's checkout isn't at `<sha>`. Usage, exits and the step log are in
`docs/TOOLS.md`, under "Root's upgrade driver". Each run needs Mos's go, and `--migrate` needs Jason present
(decision 96).
When a run doesn't exit 0, send Mos the exit and the last lines of `/var/log/mosaic-stack/upgrade-dev.log`:
- Exit 3: refused before anything stopped. The last line names the code. The stack's user's clone and the
new copy may already hold the new commit; the units still run the old one.
- Exit 1: the old copy runs and passed its gate. Either the driver restarted it after A2 or a refused A3
(A2f), or it rolled back after A3, A4 or A5.
- Exit 2: both units are stopped and the activation stays open, so the next run refuses at its plan until
it is finished with `confirm` or `revert`. One way here: a unit that doesn't stop within `--timeout`
keeps its old start time, A2f's gate fails it `stale-start`, and the driver stops both (duckworth-0055
N1). That is on purpose; a unit that won't stop is a reason to stop.
- Exit 5: `args-failed`. The new copy is live and passed confirm, but `/etc/mosaic-stack/dev.args` still names
the old copy. The next run refuses `args-differ`, and a render from that file would put the old copy back.
Move the file the result line names into place with `mv -f`. The two differ only in the `--copy` value.
Precheck lines: none. Root's checkout at `<sha>` and the stack's user's clone: precheck line missing.
After install, `stack-copy.sh check --units external` reads the copy the units run.
## 7. The args file
`/etc/mosaic-stack/dev.args` holds the `case2-unit.sh` arguments, one per line, with the serve flags after a
line holding `--`. I2's install writes it; ops-01 doesn't write it by hand. It must be root's, with no group
or other write (`chmod 0644` or tighter). Before `--` it takes only `--user`, `--home`, `--business`,
`--copy` (exactly once), `--node`, `--profile`, `--claude` and `--pm`. `stack-upgrade.sh` changes only the
`--copy` line, and only after confirm passes.
Precheck line: none. `stack-upgrade.sh` refuses `args-unreadable` on any of these rules.
## 8. Enrollment, once per host
The machine-grant enrollment is Gizmoduck's design r6, commit f9039682e, section 3, steps E1-E4. ops-01's
Q-O1 is answered (mos-1125). The host side is the `tools/host/` files: `mosaic-keys`, `mosaic-keys-write`,
their sudoers line and the sshd drop-in `/etc/ssh/sshd_config.d/<nn>-mosaic-keys.conf`. The verbs that run
it (`hosts.enroll`, the `host-enroll` decision) are rows 234 and 235, not yet on refactor. This item has no
commands until they land.
Precheck line: none. The enrolled host's key and its `mosaic-keys` sudoers line: precheck line missing.
## 9. Credential slots
Jason places these, all in one sitting per host. No value appears here or in any message. Check a placed
file with `stat -c '%U %a %n' PATH`, never `cat`. `<business>` is the host's business id.
### .53, factory
| Path | File | Mode | Owner | Profile | Precheck line |
|---|---|---|---|---|---|
| `/srv/mosaic/.config/mosaic-dev/secrets/` | `claude-session.env`, one line `CLAUDE_CODE_OAUTH_TOKEN=...` (Q185) | 0600 | mosaic | factory | `claude replaces its prompt with --system-prompt-file` runs claude with it. The file's mode and owner: precheck line missing. |
| `/srv/mosaic/.config/mosaic-dev/secrets/agents/<business>/<agent>/` | one file per service | 0600, dirs 0700 | mosaic | both | none. Filled only by `mosaic credentials issue`, or Approve after #436. Never by hand. |
### .54, library
| Path | File | Mode | Owner | Profile | Precheck line |
|---|---|---|---|---|---|
| `/srv/mosaic/.config/mosaic-dev/secrets/` | `claude-session.env`, one line `ZAI_API_KEY=...` | 0600 | mosaic | library | `<path> sets ZAI_API_KEY` |
| `/srv/mosaic/.config/mosaic-dev/secrets/` | the Console's proof file, the path the args file's `--proof-file` names | 0600 or 0400 | mosaic | library | none. `case2-unit.sh` refuses at render. Precheck line missing. |
| `/srv/mosaic/.config/mosaic-dev/secrets/<business>/` | `library-gitea` | 0600 | mosaic | library | C2-13b's `library.sync` (not landed) |
| `/srv/mosaic/.config/mosaic-dev/secrets/agents/<business>/<agent>/` | one file per service | 0600, dirs 0700 | mosaic | both | none. Filled only by `mosaic credentials issue`, or Approve after #436. Never by hand. |
The installer key, `<configDir>/secrets/hosts/<business>/<hostId>/installer`, is not a slot anyone places.
Enrollment (section 8) writes it under the enrolling stack's config directory and deletes it when the host
leaves.
## 10. Console start
None by hand. I2's `case2-unit.sh` enables both units, starts the bus, then the Console (Q189, mos-1122).
`stack-upgrade.sh` stops and starts both units itself.
Precheck line: none. Both units active after the render: precheck line missing.
+15
View File
@@ -256,6 +256,21 @@ r6 B2 as designed:
The I3 run sheet's hostcheck item points at this row. I3's A1 and its fake don't change.
### Carried from the I3 review (Filbert 29922 on #1735, sage-1727)
Row 233 lands with three non-blocking notes that must be fixed before ops-01. This row takes them, and that
widens its paths to `scripts/stack-upgrade.sh` and `packages/cli/tests/stack-upgrade.test.mjs`:
- N1: after A5 confirms, a failed rename of the new args file calls `finish 1 args-failed` while the new copy
is live (`stack-upgrade.sh:280-282` at 1a1dc80ca). TOOLS.md and the run sheet define exit 1 as "the old copy
runs". Give this case its own exit code, add it to the exit table in TOOLS.md and the run sheet, and add a
test that makes the rename fail after A5 and expects the new code.
- N4: run sheet section 3 doesn't put the pinned node's npm on root's PATH and doesn't name the `--claude`
path. The npm symlink `node_modules/.bin/claude` is what `case2-unit.sh:140` refuses. The sheet must name
the path the check accepts.
- N5: run sheet sections 8 and 10 get their precheck lines.
Filbert's N2 and N3 (no `$REPO` check, no signal handling between A2 and A5) are in DEFERRED.md, not here.
### Gate
Filbert approves the manifest. The test file, the cli suite and every `scripts/test-*.sh` are green on
+54
View File
@@ -216,6 +216,60 @@ at every gate. Started 2026-09-12 during the control board MVP.
READMEs, `docs/TOOLS.md` and Sage's suite recipe should use the glob
form wherever Node 24 is a target. Darkwing, S1 packet. (2026-10-04,
#1518)
- **`stack-upgrade.sh` doesn't check `$REPO`.** Nothing confirms the
source checkout is root-owned or that its tree is clean before the
upgrade builds from it. Filbert N2 on row 233 (29922, #1735).
(2026-10-21)
- **`stack-upgrade.sh` has no signal handling between A2 and A5.** A HUP,
INT or TERM in that stretch leaves the run failed closed, but recovery
is by hand. A trap that writes the stop point and the recovery step to
the log would make it a sheet step. Filbert N3 on row 233 (29922,
#1735). (2026-10-21)
- **`checkCreative` doesn't refuse `credentials`.** Row 213's
`checkCreative` (packages/business/src/role.mjs:143 at 2d2ad2760)
limits a creative role's tools, authority and messaging, but a role
file that gave companion or research a credential would still load.
The business file could then map a credential to that instance
(business.mjs:157). Nobody has traced whether a mapped credential
reaches the run's environment. The decision paths stay closed (M6),
but a creative role with `web` holding a live token is a wider channel
than design 4 means. The fix: refuse any `credentials` in
`checkCreative`, with a role.test.mjs case. It goes after row 213
lands, as a follow-up row or in the next row that edits role.mjs.
Webby N1 on row 213 r3 (REVIEW-r3.md, 3ab3c010b), mos-1160.
(2026-10-21)
- **The boot host check orders only one business's bus.**
`mosaic-hostcheck.service` (row 240) has
`[email protected]`. `case2-unit.sh` enables the
bus as `mosaic-bus@<business>.service`, so on a host whose business id
isn't `mosaic-stack` nothing orders the two. The bus can start before
the boot stop runs, and after a boot-time drift role sessions can use
bwrap until it does. The fix: `After=mosaic-hostcheck.service` in
`packages/cli/systemd/mosaic-bus.service.in`, which orders every
instance and does nothing where the unit isn't installed. It goes in
before the first install whose business id isn't `mosaic-stack`.
Filbert N1 on row 240 (29945, #1746). (2026-10-21)
- **The run sheet writes the apt hook before the first `--check`.**
Section 4 installs `80mosaic-hostcheck`, then runs
`hostcheck.sh --check`. If that check fails, the next package run
closes bwrap before anyone has read the failure. It fails closed, but
it surprises an installer. The fix: run `--check` first, or say what a
failure at that step means. Doc-only, for the next row that edits
`docs/guides/host-run-sheet.md`. Filbert N2 on row 240 (29945, #1746).
Filbert's N3, that a closed bwrap appends to `DRIFT` on every later
apt run and boot, is by design; ops-01's STATE read should expect the
repeats. (2026-10-21)
- **The sso-config first-step hint has no business.** When the target
copy has no `console.sso`, `stack-release.sh` (`:938`) refuses with
"activate $TO without --sso first, then ... activate $TO --sso". The
usage line is `activate ID BUSINESS` (`:47`), so pasting either command
as printed gives a usage error, not the first step. It fails safe,
since the usage error changes nothing. The fix: add `$BUSINESS`
(set at `:2138`) to both commands, and the business (`acme`)
to the three expected strings in `stack-release.test.mjs`
(`:1538`, `:1689`, `:1691`). For a later row that edits
`stack-release.sh`. Webby N1 on row 238 r1 (REVIEW-r1.md, 433a95292,
#1744). (2026-10-21)
## Queue
+5 -5
View File
@@ -33,7 +33,7 @@ Gaps found while working go to `docs/plans/DEFERRED.md`, not here.
<!-- mosaic-queue:begin -->
Generated from `docs/plans/queue.json` revision 2627 by `queue render`. Do not edit between the markers; change the queue with `scripts/mosaic queue`.
Generated from `docs/plans/queue.json` revision 2641 by `queue render`. Do not edit between the markers; change the queue with `scripts/mosaic queue`.
| # | Piece | Owner | Issues | State | After | Gate | Brief | Note |
|---|---|---|---|---|---|---|---|---|
@@ -268,8 +268,8 @@ Generated from `docs/plans/queue.json` revision 2627 by `queue render`. Do not e
| 229 | Row B: per-instance pi agent dir for Chet's Codex login, design only, starts after 10-27 (#428) | gosalyn; reviewers webby | #1730 | queued | — | A live check, with Jason's or Mos's go, that two clients on one ChatGPT login don't revoke each other's refresh tokens. No build row starts before it passes. Held until after 10-27. (sage) | `docs/plans/2026-10-20_case2-rows.md` § Brief: row B, a per-instance pi agent dir for Chet's Codex login (#428) | — |
| 230 | Case 2 C2-13a unit render and host precheck (case2-unit.sh, case2-precheck.sh) | gosalyn; reviewers webby | #1732 | done | — | Webby approves the manifest. The test file and the cli suite are green on Sage's gate, along with every scripts/test-*.sh. A run on a host is a separate window with Jason's go. (sage) | `docs/plans/2026-10-20_case2-rows.md` § C2-13a unit render and host precheck | Landed c8bc5cf4c on refactor (base 390ea5261), pushed. Patch-id f4b49eb68997 matches git diff 722136772..96768766c on the 5 manifest paths (manifest fbb0d7e42c67, Webby approve r2 rev 2563). packages/cli/README.md 3-way with row 199, hunks apart. Gosalyn gate on 96768766c: cli 541/0, scripts 429/0, mutants 10/10. Merged tree check: case2-unit 30/0, host 42/0. |
| 231 | Case 2 C2-01b survey escapes, list item lines and attachment embeds (row 212 N1-N3) | gosalyn; reviewers webby | #1733 | briefed | 212 done | Webby approves the manifest. The library tests are green on Sage's gate, along with every scripts/test-*.sh. (sage) | `docs/plans/2026-10-20_case2-rows.md` § C2-01b survey escapes, list item lines and attachment embeds | — |
| 232 | Installer I1: stack-release external units and stack-copy bundle copies (MVP, Q187, Vikunja #432) | darkwing; reviewers filbert | #1734 | in-review, round 2, request posted | — | Filbert approves the manifest. The cli suite and every scripts/test-*.sh are green on Sage's gate, with the mutants killed. (sage) | `docs/plans/2026-10-20_installer-rows.md` § I1 stack-release external units and bundle copies | — |
| 233 | Installer I3: stack-upgrade.sh root driver and the ops-01 host run sheet (MVP, Q187, Vikunja #432) | darkwing; reviewers filbert | #1735 | briefed | 232 done | Filbert approves the manifest. The test file, the cli suite and every scripts/test-*.sh are green on Sage's gate, with the mutants killed. (sage) | `docs/plans/2026-10-20_installer-rows.md` § I3 the upgrade command and the host run sheet | — |
| 232 | Installer I1: stack-release external units and stack-copy bundle copies (MVP, Q187, Vikunja #432) | darkwing; reviewers filbert | #1734 | done | — | Filbert approves the manifest. The cli suite and every scripts/test-*.sh are green on Sage's gate, with the mutants killed. (sage) | `docs/plans/2026-10-20_installer-rows.md` § I1 stack-release external units and bundle copies | Landed e65a3ab4d on refactor (base 0444acd08), pushed. Patch-id 5adfbab9 matches git diff 8fede5fc6..da2da2f6c on the 9 manifest paths; tree entries (modes too) equal da2da2f6c, files pass manifest 7af3c943. Filbert approve r1 29829, r2 29865. Darkwing's landing gate on da2da2f6c: cli 593/0, scripts 429/0 (+156 queue node tests). Mutants 17/17. |
| 233 | Installer I3: stack-upgrade.sh root driver and the ops-01 host run sheet (MVP, Q187, Vikunja #432) | darkwing; reviewers filbert | #1735 | in-progress | 232 done | Filbert approves the manifest. The test file, the cli suite and every scripts/test-*.sh are green on Sage's gate, with the mutants killed. (sage) | `docs/plans/2026-10-20_installer-rows.md` § I3 the upgrade command and the host run sheet | — |
| 234 | Machine grant row a: host enrollment, store records and migration 3i-3j (#436a, design r6 f9039682e, lands by 10-25 before the .53/.54 installs) | gizmoduck; reviewers duckworth | #1740 | in-progress | 210 done | Duckworth approves the manifest. The bus, business and cli suites and every scripts/test-*.sh are green on Sage's gate. The design's section 10 row a tests and mutants pass, the migration test included. (sage) (sage) | `docs/plans/2026-10-20_host-credentials.md` § Machine grant row a: enrollment, store records and migration 3i-3j | — |
| 235 | Machine grant row b: raise binding, executor, reconcile and revoke (#436, design r6 f9039682e, C1, N1, G27, G28) | gizmoduck; reviewers duckworth | #1741 | queued | 234 done | Duckworth approves the manifest. The bus, business and cli suites and every scripts/test-*.sh are green on Sage's gate. Section 10's row b tests and mutants pass, G27, G28 and the upsert mutant included. (sage) (sage) | `docs/plans/2026-10-20_host-credentials.md` § Machine grant row b: raise binding, executor, reconcile and revoke | — |
| 236 | #436b Gitea grant path build: seat accounts and tokens through an approved grant, schema 3j-3k (design r5 1be535aca, due 10-27 at risk; live waits for Jason's section 8 approval and bootstrap step) | megavolt; reviewers dewey | #1742 | queued | 210 done, 234 done | Dewey approves the manifest. The bus, business and cli suites and every scripts/test-*.sh are green on Sage's gate. Section 9's mutants are killed, N9's restart test included. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: #436b Gitea grant path, build (schema 3j-3k) | — |
@@ -277,8 +277,8 @@ Generated from `docs/plans/queue.json` revision 2627 by `queue render`. Do not e
| 238 | row 200 follow-up N1: the sso-config hint carries --instance NAME (#1744) | launchpad; reviewers webby | #1744 | in-review, round 1, request posted | 200 done | Webby approves the manifest; stack-release.test.mjs whole, cli suite and every scripts/test-*.sh green on Sage's gate; the mutant killed (sage) | `docs/plans/2026-10-20_stack-release-sso.md` § Brief: row 200 follow-up N1, the sso-config hint names the instance | — |
| 239 | Row 194 follow-up: the credential revoke runs under the writer's lock, refusals after a revoke say so, and a stale-revision fire test (duckworth-0074, mos-1158) | honker; reviewers duckworth | #1745 | queued | 194 done | Duckworth approves the manifest. The bus, business, cli and webui suites and every scripts/test-*.sh are green on Sage's gate, and the mutants are killed. (sage) (sage) | `docs/plans/2026-10-20_agent-lifecycle.md` § Brief: row 194 follow-up, the revoke under the writer's lock and a fire test | — |
| 240 | Installer I4: hostcheck, the bwrap profile check, pins and boot unit, from r6 B2 (MVP, Q187, Vikunja #432; split from I3) | darkwing; reviewers filbert | #1746 | queued | 233 done | Filbert approves the manifest. The test file, the cli suite and every scripts/test-*.sh are green on Sage's gate, with the mutants killed. (sage) (sage) | `docs/plans/2026-10-20_installer-rows.md` § I4 hostcheck: the bwrap profile check, pins and boot unit | — |
| 241 | Row 237 follow-up: one decision lands once, a two-ticket broker land test and three mutants on the landing guards (megavolt-0069, sage-1651, P19 441) | megavolt; reviewers duckworth | #1747 | in-review, round 1, request posted | 237 done | Duckworth approves the manifest. The bus, seat and cli suites and every scripts/test-*.sh are green on Sage's gate, and the three mutants are killed. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: row 237 follow-up, one decision lands once, with tests at the broker | — |
| 242 | Row 237 follow-up N1: a retry after publish-timeout pushes once; hold the decision or kill and wait for the push, a slow-push retry test and a mutant (duckworth-0069, P19 440) | megavolt; reviewers duckworth | #1748 | in-progress | 237 done | Duckworth approves the manifest. The bus and cli suites and every scripts/test-*.sh are green on Sage's gate, and the mutant is killed. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: row 237 follow-up N1, a retry after publish-timeout pushes once | — |
| 241 | Row 237 follow-up: one decision lands once, a two-ticket broker land test and three mutants on the landing guards (megavolt-0069, sage-1651, P19 441) | megavolt; reviewers duckworth | #1747 | done | 237 done | Duckworth approves the manifest. The bus, seat and cli suites and every scripts/test-*.sh are green on Sage's gate, and the three mutants are killed. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: row 237 follow-up, one decision lands once, with tests at the broker | Landed 5d4a61776 on refactor, pushed (base e65a3ab4d). Blobs from 3e3ed303e on 2 paths; patch-id a1649f80 equals git diff f5753c0e8..3e3ed303e on those paths; ls-tree modes equal; manifest e8266864 verifies on the landed files. Sage slot on bbf3b1ee8: M1-M3 killed; gate 15 packages 2797/0, 10 script suites 429/0. Duckworth approve 29870 rev 2628. |
| 242 | Row 237 follow-up N1: a retry after publish-timeout pushes once; hold the decision or kill and wait for the push, a slow-push retry test and a mutant (duckworth-0069, P19 440) | megavolt; reviewers duckworth | #1748 | done | 237 done | Duckworth approves the manifest. The bus and cli suites and every scripts/test-*.sh are green on Sage's gate, and the mutant is killed. (sage) (sage) | `docs/plans/2026-10-20_agent-credentials.md` § Brief: row 237 follow-up N1, a retry after publish-timeout pushes once | Landed 5fb736b92 on refactor, pushed (base 96bac588f). Blobs from 635c10924 on 3 paths; patch-id ceb741ee equals git diff 8fede5fc6..635c10924 on those paths; ls-tree modes equal; manifest 1fbfab6c verifies on the landed files. Megavolt M1/M2 killed on Sage's slot; Sage gate 15 packages 2826/0, 10 script suites 429/0. Duckworth approve 29889 rev 2639. |
<!-- mosaic-queue:end -->
+417 -30
View File
@@ -1,7 +1,7 @@
{
"version": 1,
"canonicalRoot": "/mnt/storage/src/mosaic-stack",
"revision": 2627,
"revision": 2641,
"rows": [
{
"id": 1,
@@ -22631,7 +22631,7 @@
"closes": [
1734
],
"state": "in-review",
"state": "done",
"previousState": null,
"required": false,
"requiredSince": null,
@@ -22730,15 +22730,12 @@
}
]
},
"claim": {
"seat": "darkwing",
"op": "darkwing-232-inprogress-r1"
},
"note": null,
"claim": null,
"note": "Landed e65a3ab4d on refactor (base 0444acd08), pushed. Patch-id 5adfbab9 matches git diff 8fede5fc6..da2da2f6c on the 9 manifest paths; tree entries (modes too) equal da2da2f6c, files pass manifest 7af3c943. Filbert approve r1 29829, r2 29865. Darkwing's landing gate on da2da2f6c: cli 593/0, scripts 429/0 (+156 queue node tests). Mutants 17/17.",
"blockedReason": null,
"createdAt": "2026-10-20T22:56:29.929Z",
"updatedAt": "2026-10-21T03:01:22.609Z",
"updatedBy": "filbert"
"updatedAt": "2026-10-21T03:20:36.830Z",
"updatedBy": "sage"
},
{
"id": 233,
@@ -22750,7 +22747,7 @@
"closes": [
1735
],
"state": "briefed",
"state": "in-progress",
"previousState": null,
"required": false,
"requiredSince": null,
@@ -22759,7 +22756,7 @@
"brief": {
"path": "docs/plans/2026-10-20_installer-rows.md",
"anchor": "I3 the upgrade command and the host run sheet",
"blob": "f35e7e57daee39a760159e069994f213d733b167"
"blob": "e61291bcf1c6e9f43f599ca94fe9c6eb0dce1383"
},
"after": [
{
@@ -22771,12 +22768,15 @@
"filbert"
],
"review": null,
"claim": null,
"claim": {
"seat": "darkwing",
"op": "darkwing-233-inprogress-r1"
},
"note": null,
"blockedReason": null,
"createdAt": "2026-10-20T22:56:59.719Z",
"updatedAt": "2026-10-21T01:30:41.511Z",
"updatedBy": "sage"
"updatedAt": "2026-10-21T03:25:49.757Z",
"updatedBy": "darkwing"
},
{
"id": 234,
@@ -23178,7 +23178,7 @@
"closes": [
1747
],
"state": "in-review",
"state": "done",
"previousState": null,
"required": false,
"requiredSince": null,
@@ -23247,18 +23247,24 @@
}
],
"duplicateRisk": false,
"receipts": []
"receipts": [
{
"reviewer": "duckworth",
"op": "duckworth-r241-record-1",
"at": "2026-10-21T03:06:16.441Z",
"verdict": "approve",
"comment": 29870,
"candidate": "e82668642589eefca8b22433f1b0c7a24432f1993c0050f0a4b74c1146c93c2f"
}
]
}
]
},
"claim": {
"seat": "megavolt",
"op": "megavolt-241-start-1"
},
"note": null,
"claim": null,
"note": "Landed 5d4a61776 on refactor, pushed (base e65a3ab4d). Blobs from 3e3ed303e on 2 paths; patch-id a1649f80 equals git diff f5753c0e8..3e3ed303e on those paths; ls-tree modes equal; manifest e8266864 verifies on the landed files. Sage slot on bbf3b1ee8: M1-M3 killed; gate 15 packages 2797/0, 10 script suites 429/0. Duckworth approve 29870 rev 2628.",
"blockedReason": null,
"createdAt": "2026-10-21T01:52:55.772Z",
"updatedAt": "2026-10-21T03:04:33.826Z",
"updatedAt": "2026-10-21T03:35:52.657Z",
"updatedBy": "sage"
},
{
@@ -23271,7 +23277,7 @@
"closes": [
1748
],
"state": "in-progress",
"state": "done",
"previousState": null,
"required": false,
"requiredSince": null,
@@ -23291,16 +23297,70 @@
"reviewers": [
"duckworth"
],
"review": null,
"claim": {
"seat": "megavolt",
"op": "megavolt-242-start-1"
"review": {
"rounds": [
{
"n": 1,
"op": "megavolt-242-inreview-r1",
"by": "megavolt",
"at": "2026-10-21T03:44:17.331Z",
"issue": 1748,
"candidate": {
"kind": "manifest",
"digest": "1fbfab6c68e8d76d2d69b13c415257280d4fa8c5584e24d5dfd13dca1420a3da",
"text": "d245f805bfc9ad8a0064a8898effa05148e9c3fb7deb22b85f7af499e0ded369 agents/megavolt/work/queue-242/BUILD.md\ncfecc67698ad18a94288ba4e852b372d92c8d1cb984494256f929b03a467baad packages/bus/src/runtime.mjs\na0783575b7da4e3b8e1ae161d7031fdf62a8a1a402a48ac898ef1b5c2d378d9b packages/cli/tests/publish-socket.test.mjs\n"
},
"request": "comment",
"attempts": [
{
"op": "megavolt-242-inreview-r1",
"by": "megavolt",
"at": "2026-10-21T03:44:17.331Z",
"state": "failed",
"comment": null,
"transport": {
"outcome": "failed",
"status": null,
"comment": null,
"detail": "pre-send: the credential or account check failed"
},
"resolutions": []
},
{
"op": "sage-242-request-1",
"by": "sage",
"at": "2026-10-21T03:45:41.063Z",
"state": "posted",
"comment": 29888,
"transport": {
"outcome": "posted",
"status": 201,
"comment": 29888,
"detail": "created"
},
"resolutions": []
}
],
"duplicateRisk": false,
"receipts": [
{
"reviewer": "duckworth",
"op": "duckworth-r242-record-1",
"at": "2026-10-21T03:48:36.391Z",
"verdict": "approve",
"comment": 29889,
"candidate": "1fbfab6c68e8d76d2d69b13c415257280d4fa8c5584e24d5dfd13dca1420a3da"
}
]
}
]
},
"note": null,
"claim": null,
"note": "Landed 5fb736b92 on refactor, pushed (base 96bac588f). Blobs from 635c10924 on 3 paths; patch-id ceb741ee equals git diff 8fede5fc6..635c10924 on those paths; ls-tree modes equal; manifest 1fbfab6c verifies on the landed files. Megavolt M1/M2 killed on Sage's slot; Sage gate 15 packages 2826/0, 10 script suites 429/0. Duckworth approve 29889 rev 2639.",
"blockedReason": null,
"createdAt": "2026-10-21T01:54:53.103Z",
"updatedAt": "2026-10-21T03:03:20.738Z",
"updatedBy": "megavolt"
"updatedAt": "2026-10-21T04:13:55.780Z",
"updatedBy": "sage"
}
],
"log": [
@@ -92957,6 +93017,333 @@
"receipt": "ok sage-241-request-1-resolve rev 2627 row 241 round 1 request sage-241-request-1 requesting→posted comment 29868"
},
"viewSha": "ec48b0f767195fb8ddaa8e67e9526a533ef4ddc789d6b11c0998befbf32433de"
},
{
"rev": 2628,
"op": "duckworth-r241-record-1",
"verb": "review-record",
"args": {
"id": 241,
"verdict": "approve",
"comment": 29870,
"candidate": "e82668642589eefca8b22433f1b0c7a24432f1993c0050f0a4b74c1146c93c2f"
},
"by": "duckworth",
"at": "2026-10-21T03:06:16.441Z",
"semantics": 2,
"result": {
"row": 241,
"round": 1,
"verdict": "approve",
"receipt": "ok duckworth-r241-record-1 rev 2628 row 241 round 1 approve by duckworth (comment 29870)"
},
"viewSha": "094f036ccb430caef42be43e00643081c1ed2f7ec8253b865faba8dd28ad7298"
},
{
"rev": 2629,
"op": "sage-232-landing-note",
"verb": "note",
"args": {
"id": 232,
"text": "Landed e65a3ab4d on refactor (base 0444acd08), pushed. Patch-id 5adfbab9 matches git diff 8fede5fc6..da2da2f6c on the 9 manifest paths; tree entries (modes too) equal da2da2f6c, files pass manifest 7af3c943. Filbert approve r1 29829, r2 29865. Darkwing's landing gate on da2da2f6c: cli 593/0, scripts 429/0 (+156 queue node tests). Mutants 17/17."
},
"by": "sage",
"at": "2026-10-21T03:20:12.732Z",
"semantics": 2,
"result": {
"row": 232,
"field": "note",
"receipt": "ok sage-232-landing-note rev 2629 row 232 note"
},
"viewSha": "e0f5ff1d8b8c1592a39c64a2df6b32d4d8f2a65e4ee095566a01989d491250ea"
},
{
"rev": 2630,
"op": "sage-232-done",
"verb": "move",
"args": {
"id": 232,
"to": "done",
"reason": null,
"candidate": null,
"evidence": null,
"issue": null
},
"by": "sage",
"at": "2026-10-21T03:20:36.830Z",
"semantics": 2,
"result": {
"row": 232,
"from": "in-review",
"to": "done",
"round": 2,
"issue": null,
"candidate": null,
"receipt": "ok sage-232-done rev 2630 row 232 in-review→done round 2"
},
"viewSha": "8d3708d10c659ec2d626c28be1cb87089cd606574278e9b51b99fae515928050"
},
{
"rev": 2631,
"op": "sage-233-repin-brief-1",
"verb": "set",
"args": {
"id": 233,
"field": "brief",
"value": "docs/plans/2026-10-20_installer-rows.md#I3 the upgrade command and the host run sheet",
"reason": null
},
"by": "sage",
"at": "2026-10-21T03:25:06.360Z",
"semantics": 2,
"result": {
"row": 233,
"field": "brief",
"from": {
"path": "docs/plans/2026-10-20_installer-rows.md",
"anchor": "I3 the upgrade command and the host run sheet",
"blob": "f35e7e57daee39a760159e069994f213d733b167"
},
"to": {
"path": "docs/plans/2026-10-20_installer-rows.md",
"anchor": "I3 the upgrade command and the host run sheet",
"blob": "e61291bcf1c6e9f43f599ca94fe9c6eb0dce1383"
},
"receipt": "ok sage-233-repin-brief-1 rev 2631 row 233 brief: docs/plans/2026-10-20_installer-rows.md § I3 the upgrade command and the host run sheet @f35e7e57daee→docs/plans/2026-10-20_installer-rows.md § I3 the upgrade command and the host run sheet @e61291bcf1c6"
},
"viewSha": "da749491f801e3ec47f908872aa4453fc3caa30b23e4194a205ddfd9d076241e"
},
{
"rev": 2632,
"op": "darkwing-233-inprogress-r1",
"verb": "move",
"args": {
"id": 233,
"to": "in-progress",
"reason": null,
"candidate": null,
"evidence": null,
"issue": null
},
"by": "darkwing",
"at": "2026-10-21T03:25:49.757Z",
"semantics": 2,
"result": {
"row": 233,
"from": "briefed",
"to": "in-progress",
"round": null,
"issue": null,
"candidate": null,
"receipt": "ok darkwing-233-inprogress-r1 rev 2632 row 233 briefed→in-progress"
},
"viewSha": "f77ca73f963f35d77733bfe6fa5e8ad3a4144d3ae6b7aa2fcba1d62eb3da4e12"
},
{
"rev": 2633,
"op": "sage-241-landed-note",
"verb": "note",
"args": {
"id": 241,
"text": "Landed 5d4a61776 on refactor, pushed (base e65a3ab4d). Blobs from 3e3ed303e on 2 paths; patch-id a1649f80 equals git diff f5753c0e8..3e3ed303e on those paths; ls-tree modes equal; manifest e8266864 verifies on the landed files. Sage slot on bbf3b1ee8: M1-M3 killed; gate 15 packages 2797/0, 10 script suites 429/0. Duckworth approve 29870 rev 2628."
},
"by": "sage",
"at": "2026-10-21T03:35:10.676Z",
"semantics": 2,
"result": {
"row": 241,
"field": "note",
"receipt": "ok sage-241-landed-note rev 2633 row 241 note"
},
"viewSha": "60a6381f3fa3ff4ca5774baed90a9b383fd69b09bf88afc6c5aa123fa0379208"
},
{
"rev": 2634,
"op": "sage-241-done",
"verb": "move",
"args": {
"id": 241,
"to": "done",
"reason": null,
"candidate": null,
"evidence": null,
"issue": null
},
"by": "sage",
"at": "2026-10-21T03:35:52.657Z",
"semantics": 2,
"result": {
"row": 241,
"from": "in-review",
"to": "done",
"round": 1,
"issue": null,
"candidate": null,
"receipt": "ok sage-241-done rev 2634 row 241 in-review→done round 1"
},
"viewSha": "369663313667edd50358fd170a5e652a06332ec394918089a02cc79b19eed63f"
},
{
"rev": 2635,
"op": "megavolt-242-inreview-r1",
"verb": "move",
"args": {
"id": 242,
"to": "in-review",
"reason": null,
"candidate": "/mnt/storage/scratch/megavolt-r242/manifest-r1-635c10924.txt",
"evidence": null,
"issue": null
},
"by": "megavolt",
"at": "2026-10-21T03:44:17.331Z",
"semantics": 2,
"result": {
"row": 242,
"from": "in-progress",
"to": "in-review",
"round": 1,
"issue": 1748,
"candidate": {
"kind": "manifest",
"digest": "1fbfab6c68e8d76d2d69b13c415257280d4fa8c5584e24d5dfd13dca1420a3da",
"text": "d245f805bfc9ad8a0064a8898effa05148e9c3fb7deb22b85f7af499e0ded369 agents/megavolt/work/queue-242/BUILD.md\ncfecc67698ad18a94288ba4e852b372d92c8d1cb984494256f929b03a467baad packages/bus/src/runtime.mjs\na0783575b7da4e3b8e1ae161d7031fdf62a8a1a402a48ac898ef1b5c2d378d9b packages/cli/tests/publish-socket.test.mjs\n"
},
"receipt": "ok megavolt-242-inreview-r1 rev 2635 row 242 in-progress→in-review round 1 on #1748; request megavolt-242-inreview-r1 requesting"
},
"viewSha": "432a911c27ffb0374b1ca49cb60215cb4211ce4c8a7f02bdfff91054f6a54e69"
},
{
"rev": 2636,
"op": "megavolt-242-inreview-r1.outcome",
"verb": "review-outcome",
"args": {
"id": 242,
"attempt": "megavolt-242-inreview-r1",
"outcome": "failed",
"status": null,
"comment": null,
"detail": "pre-send: the credential or account check failed"
},
"by": "megavolt",
"at": "2026-10-21T03:44:42.695Z",
"semantics": 2,
"result": {
"row": 242,
"round": 1,
"attempt": "megavolt-242-inreview-r1",
"state": "failed",
"receipt": "ok megavolt-242-inreview-r1.outcome rev 2636 row 242 round 1 request megavolt-242-inreview-r1 failed"
},
"viewSha": "42cd5e62d4b725838d45d409f48444e769c173db4825dcdc22e33c86a297ecda"
},
{
"rev": 2637,
"op": "sage-242-request-1",
"verb": "review-request",
"args": {
"id": 242
},
"by": "sage",
"at": "2026-10-21T03:45:41.063Z",
"semantics": 2,
"result": {
"row": 242,
"round": 1,
"attempt": "sage-242-request-1",
"state": "requesting",
"receipt": "ok sage-242-request-1 rev 2637 row 242 round 1 request sage-242-request-1 requesting"
},
"viewSha": "bc6018e64090bd29a7770b4f03511edc1314428a99d9114b3b274f2d248ce228"
},
{
"rev": 2638,
"op": "sage-242-request-1.outcome",
"verb": "review-outcome",
"args": {
"id": 242,
"attempt": "sage-242-request-1",
"outcome": "posted",
"status": 201,
"comment": 29888,
"detail": "created"
},
"by": "sage",
"at": "2026-10-21T03:46:05.721Z",
"semantics": 2,
"result": {
"row": 242,
"round": 1,
"attempt": "sage-242-request-1",
"state": "posted",
"receipt": "ok sage-242-request-1.outcome rev 2638 row 242 round 1 request sage-242-request-1 posted comment 29888"
},
"viewSha": "d7bf764deb95140e4b2748863b838f5598f57162a0388adcfb24b4ca6cb90449"
},
{
"rev": 2639,
"op": "duckworth-r242-record-1",
"verb": "review-record",
"args": {
"id": 242,
"verdict": "approve",
"comment": 29889,
"candidate": "1fbfab6c68e8d76d2d69b13c415257280d4fa8c5584e24d5dfd13dca1420a3da"
},
"by": "duckworth",
"at": "2026-10-21T03:48:36.391Z",
"semantics": 2,
"result": {
"row": 242,
"round": 1,
"verdict": "approve",
"receipt": "ok duckworth-r242-record-1 rev 2639 row 242 round 1 approve by duckworth (comment 29889)"
},
"viewSha": "80ea9e552b2edb1937fce5ff23bc6cf9b6b9e2381cbb2e5808d7c87acf3d4b45"
},
{
"rev": 2640,
"op": "sage-242-landed-note",
"verb": "note",
"args": {
"id": 242,
"text": "Landed 5fb736b92 on refactor, pushed (base 96bac588f). Blobs from 635c10924 on 3 paths; patch-id ceb741ee equals git diff 8fede5fc6..635c10924 on those paths; ls-tree modes equal; manifest 1fbfab6c verifies on the landed files. Megavolt M1/M2 killed on Sage's slot; Sage gate 15 packages 2826/0, 10 script suites 429/0. Duckworth approve 29889 rev 2639."
},
"by": "sage",
"at": "2026-10-21T04:13:30.071Z",
"semantics": 2,
"result": {
"row": 242,
"field": "note",
"receipt": "ok sage-242-landed-note rev 2640 row 242 note"
},
"viewSha": "b0997e13a14bd17120aef27778c84deacdb08149688d0399e035fb8bc365bc37"
},
{
"rev": 2641,
"op": "sage-242-done",
"verb": "move",
"args": {
"id": 242,
"to": "done",
"reason": null,
"candidate": null,
"evidence": null,
"issue": null
},
"by": "sage",
"at": "2026-10-21T04:13:55.780Z",
"semantics": 2,
"result": {
"row": 242,
"from": "in-review",
"to": "done",
"round": 1,
"issue": null,
"candidate": null,
"receipt": "ok sage-242-done rev 2641 row 242 in-review→done round 1"
},
"viewSha": "418b83170fd6c8ba7aa577b935f802b3a5dfead59be2260b11115206f324abbc"
}
]
}
+15 -4
View File
@@ -218,6 +218,9 @@ an operator inspect content and delivery evidence after an uncertain result.
| `credentials.issue` | `{agent,service,token,date}`; human only (row 199, #1697). Writes the token to `<configDir>/secrets/agents/<business>/<agent>/<service>` (0600, dirs 0700: tmp, fsync, rename), rewrites the agent's reference in the business file to `{"store":true,<rotateBy or expires>:date}`, then holds the token in memory, with no restart. Over an issued entry it is a rotate: the old value is gone from disk and memory when it returns. `service` is `gitea` or `vikunja` and the business file must already name it for the agent (`credential-unknown`). A token the bus already holds is `credential-leak`; a bad token `credential-format`; a bad date `credential-date`. Returns `{agent,verb,service,date,state}`, `verb` `issue` or `rotate`. The tester gets `tester-refused`. |
| `credentials.revoke` | `{agent,service?,reason}`; human, or the tester (Q173). Drops the entry from memory, then unlinks a store file; a `file` or `env` reference keeps its file and becomes `store`, so a restart loads nothing. Without `service`, every kind the agent has. Returns `{agent,verb,services:[{service,state}]}`. |
| `credentials.delete` | `{agent,reason}`; human only. `credentials.revoke` for every kind, then removes the agent's store dir; a file there other than a token is `credential-file`. While the agent or one of its threads holds a claim, `agent-running`. |
| `hosts.status` | No args; the human, the tester or the business's chief (row 234, #1740). `[{host,address,port,user,sudo,state,hostKeys,installerFingerprint,checkedAt,hostLineLeft}]` from the `hosts` records, with host keys as fingerprints. Never a key. |
| `hosts.check` | `{host}`; same callers. Runs the host's `list <user>` as `mosaic-installer` with the installer key and the pinned host keys, then sets `state` `ready` and `checkedAt` and writes `grant.host.checked`. A nonzero exit is `host-failed`, a record that changed during the run `host-changed`, a record with no installer key `host-not-enrolled`. |
| `hosts.remove` | `{host}`; the human or the tester. Deletes the record, writes `grant.host.removed` and removes the installer key. A `host_grants` record for the host is `host-in-use`, a guarded event naming a host line left on the host `host-line-left`. |
Each of the three leaves one receipt in the organization's `changes.jsonl`: the actor and a
`detail` of `{verb,agent,kind,date or reason}`, never a value. A change to the store alone writes
@@ -704,10 +707,10 @@ scope probes and rotation/expiry notifications; S2 never probes real services.
## Schema and verification
`schema.sql` is v3i, SHA-256
`6cde5232c57ac04dc6073112c0830c89cfc5e953d7a5727e7716263747fe8737`.
It has 16 tables, 71 triggers and five views. Source digest and a trusted full
sqlite_master digest are checked at open, alongside schema version `3i`, WAL,
`schema.sql` is v3j, SHA-256
`0a7b633ce7eec9d0bb37c3255f4244d97bd0c8eac38db91a11b4ef6cc982ec8c`.
It has 18 tables, 72 triggers and five views. Source digest and a trusted full
sqlite_master digest are checked at open, alongside schema version `3j`, WAL,
quick_check and foreign-key integrity. Missing metadata refuses; it never
reinitializes an existing file. Full synchronous WAL transactions either commit
or roll back. SQL UPDATE, DELETE and INSERT OR REPLACE refuse on evidence tables.
@@ -867,6 +870,14 @@ one transaction; a failure leaves the store at 3h. It runs only through
refuses a 3i store. The 3h fixture, `tests/fixtures/schema-3h.sql`, is pinned from
81f26551b's tree; no live store is used.
Schema 3j (row 234, #1740) adds only new objects: the `hosts` and `host_grants` tables, which
hold enrolled hosts and host grants, and the `events_grant_actor` trigger. The trigger refuses an
`action.allowed` event whose `body.action` or `body.operation` matches `grant.*` unless its actor
is `actor_role` NULL with `actor_run` 'grant', the broker's grant writer (#436b section 7). The
step runs in one transaction, and a failure leaves the store at 3i. The 3i fixture,
`tests/fixtures/schema-3i.sql`, is pinned from 8fede5fc6's tree, and `tests/grant-migration.test.mjs`
covers the step and the trigger.
The managed runner calls usage.begin before an actual harness turn and usage.end
after it, including a failed/stopped turn. Each takes only {turn: deliveredMessageId}.
The broker requires the bound role holder and its delivery, binds run identity,
+31
View File
@@ -363,3 +363,34 @@ CREATE TABLE usage_records (
CREATE TRIGGER usage_records_no_update BEFORE UPDATE ON usage_records BEGIN SELECT RAISE(ABORT, 'usage_records is append-only'); END;
CREATE TRIGGER usage_records_no_delete BEFORE DELETE ON usage_records BEGIN SELECT RAISE(ABORT, 'usage_records is append-only'); END;
CREATE TRIGGER usage_records_no_replace BEFORE INSERT ON usage_records WHEN EXISTS (SELECT 1 FROM usage_records WHERE seq=NEW.seq OR (business=NEW.business AND run=NEW.run AND turn=NEW.turn AND source=NEW.source AND phase=NEW.phase)) BEGIN SELECT RAISE(ABORT, 'usage_records is append-only'); END;
CREATE TABLE hosts (
business TEXT NOT NULL,
host TEXT NOT NULL,
address TEXT NOT NULL,
port INTEGER NOT NULL CHECK (port BETWEEN 1 AND 65535),
user TEXT NOT NULL,
sudo INTEGER NOT NULL CHECK (sudo IN (0,1)),
host_keys TEXT NOT NULL CHECK (json_valid(host_keys) AND json_type(host_keys) = 'array' AND json_array_length(host_keys) > 0),
installer_fingerprint TEXT,
state TEXT NOT NULL CHECK (state IN ('pending-install','ready')),
decision TEXT NOT NULL REFERENCES decisions(id),
checked_at TEXT,
PRIMARY KEY (business, host),
CHECK (state = 'pending-install' OR (installer_fingerprint IS NOT NULL AND checked_at IS NOT NULL))
) STRICT;
CREATE TABLE host_grants (
business TEXT NOT NULL,
agent TEXT NOT NULL,
host TEXT NOT NULL,
commands TEXT NOT NULL CHECK (json_valid(commands) AND json_type(commands) = 'array' AND json_array_length(commands) > 0),
rotate_by TEXT NOT NULL,
tag TEXT NOT NULL CHECK (length(tag) = 39 AND tag GLOB 'mosaic-*' AND NOT substr(tag, 8) GLOB '*[^0-9a-f]*'),
decision TEXT NOT NULL REFERENCES decisions(id),
PRIMARY KEY (business, agent, host),
FOREIGN KEY (business, host) REFERENCES hosts(business, host)
) STRICT;
CREATE TRIGGER events_grant_actor BEFORE INSERT ON events
WHEN NEW.kind = 'action.allowed'
AND (json_extract(NEW.body, '$.action') GLOB 'grant.*' OR json_extract(NEW.body, '$.operation') GLOB 'grant.*')
AND NOT (NEW.actor_role IS NULL AND NEW.actor_run IS 'grant')
BEGIN SELECT RAISE(ABORT, 'only the grant writer records grant events'); END;
+400 -17
View File
@@ -3,7 +3,7 @@ import { isAbsolute } from 'node:path';
import { NAME as HARNESS_NAME } from '../../plugins/src/kinds.mjs';
import { loadRequirements } from '../../tasks/src/requirements.mjs';
import { MERGE, LandingRefusal, readLanding } from './landing.mjs';
import { ID_PATTERN, THREAD_DEFINITION, END_PROTECTED, forkProblem, forkUrl, credentialKind, validArgv, commandAllowed } from '../../business/src/vocabulary.mjs';
import { ID_PATTERN, THREAD_DEFINITION, END_PROTECTED, forkProblem, forkUrl, credentialKind, validArgv, commandAllowed, HOST_ID } from '../../business/src/vocabulary.mjs';
// Kept closed while S1 lands; its vocabulary is the integration authority.
export const ACTIONS = Object.freeze([
@@ -182,6 +182,52 @@ const STAFF_COUNT = 20;
// Row 93 (#1587): the review loop's evidence, written only by the verbs below. A push is an
// action.allowed for git.push.working that names its commit, written only by recordPush.
const REVIEW_KINDS = ['review.requested', 'review.verdict'];
// Row 234 (#1740, #436b section 7): grant evidence is an action.allowed whose body.action or
// body.operation starts with grant., written only by #grantEvent under this actor. No session has a
// NULL role with a non-NULL run, and the 3j trigger refuses any other actor.
const GRANT_RUN = 'grant';
const grantName = (body) => [body?.action, body?.operation].some(n => typeof n === 'string' && n.startsWith('grant.'));
// Every grant-evidence read filters on this (duckworth-0067 N1): a look-alike under another kind or
// actor is not evidence.
const GRANT_EVIDENCE = "kind='action.allowed' AND actor_role IS NULL AND actor_run='grant'";
// r6 R4: the host record's identity, hashed at raise and at each later check.
export const registrySha256 = ({ address, port, user, sudo, hostKeys }) =>
createHash('sha256').update(JSON.stringify({ address, port, user, sudo: !!sudo, hostKeys })).digest('hex');
// Row 234 (#1740, #436b section 7): the one writer of grant evidence, session-less, as actor (NULL,
// 'grant'). The broker's #grantEvent calls it after its secret check, and the tests write their fixtures
// through it (duckworth-0079 F1). No verb reaches it. Only the consume record {operation, decision}
// names the decision as `decision`, so #spent counts it and nothing else. Every other body has a
// grant.* action, names the decision grantDecision, and body.by names the caller the broker checked.
export function writeGrantEvent(store, at, business, body, subject = null) {
const consume = body?.operation !== undefined;
if (
!grantName(body) ||
(consume
? Object.keys(body).length !== 2 || typeof body.decision !== 'string' || !body.operation.startsWith('grant.')
: typeof body.action !== 'string' || !body.action.startsWith('grant.') || 'decision' in body)
)
throw new Error('grant event shape');
const eid = randomUUID();
store.run(
'INSERT INTO events(id,at,business,kind,actor_role,actor_run,subject,body) VALUES(?,?,?,?,?,?,?,?)',
eid,
at,
business,
'action.allowed',
null,
GRANT_RUN,
subject,
JSON.stringify(body),
);
return eid;
}
export const HOSTS_VERBS = Object.freeze(['hosts.status', 'hosts.check', 'hosts.remove']);
// The OpenSSH fingerprint of a "<type> <base64>" host key line. host-grant.mjs has the same; the broker
// doesn't import it, so the sandbox's RUNNER_FILES stay as they are.
const fingerprintOf = (line) =>
'SHA256:' + createHash('sha256').update(Buffer.from(line.split(' ')[1], 'base64')).digest('base64').replace(/=+$/, '');
// r6 E3: the installer line's comment, then the tag's 32 hex digits.
const INSTALLER_COMMENT = 'mosaic-installer-';
// Row 121 (#1615): an instance of THREAD_DEFINITION (vocabulary.mjs since row 155) talks with the
// user in threads. Its runs claim a thread, not the role, and every message to or from it names its
// thread (REQ-MSG-1).
@@ -224,11 +270,13 @@ export class Broker {
#usage;
#limits;
#harnessOn;
// Row 234 (#1740): the trusted enrollment half (host-grant.mjs), or null where hosts are off.
#hostGrant;
// Row 125 (#1619, REQ-LIMIT-4): a run a limit stops, by business:run, until its next receive or state.
#stops = new Map();
#clock = 0;
// credentialStatus is the runtime's metadata-only Credentials.status(); the broker never gets use().
constructor({ store, businesses, secretCheck = () => {}, credentialStatus = () => [], organization = null, project = null, usage = null, limits = null, harnessOn = null, credentialVerbs = null }) {
constructor({ store, businesses, secretCheck = () => {}, credentialStatus = () => [], organization = null, project = null, usage = null, limits = null, harnessOn = null, credentialVerbs = null, hostGrant = null }) {
if (!object(businesses) || !Object.keys(businesses).length) fail('invalid-business');
this.#businesses = structuredClone(businesses);
this.#store = store;
@@ -237,6 +285,7 @@ export class Broker {
this.#organization = organization;
// Row 199: the runtime's credential lifecycle (credential-verbs.mjs); it holds the store, not the broker.
this.#credentialVerbs = credentialVerbs;
this.#hostGrant = hostGrant;
this.#project = project;
this.#usage = usage;
this.#limits = limits;
@@ -604,9 +653,323 @@ export class Broker {
);
return eid;
}
// Row 234 (#1740): grant evidence, through writeGrantEvent. Caller owns the transaction.
#grantEvent(business, body, subject = null) {
this.#secretCheck({ kind: 'action.allowed', body, subject });
return writeGrantEvent(this.#store, this.#now(), business, body, subject);
}
// PRD 1.6 REQ-WAIT-1: a hard wait goes to the human at every class, cited or not. Jason's ruling
// Q169 makes a role.revoke whose target's definition is in END_PROTECTED one, so no arbiter
// resolves a revoke of the assistant.
// Row 234 (#1740, r6 X1): the raise's grant binding, if the human's cli resolve chose its approval
// option on a credential.mint routed to the human. Null otherwise: an arbiter, the tester or an agent
// closes with via 'broker'.
#approvedGrant(business, decision) {
const d = this.#store.get('SELECT action,route_to FROM decisions WHERE business=? AND id=?', business, decision);
const raise = this.#raised(business, decision);
if (d?.action !== 'credential.mint' || d.route_to !== 'human' || !raise?.grant) return null;
const r = this.#closed(decision);
if (r?.op !== 'resolved' || r.choice !== raise.approvalChoice || r.via !== 'cli' || r.by !== this.#business(business).human)
return null;
return raise.grant;
}
// r6 X1: the executor's own spend. The caller owns the transaction. The consume record is the only
// guarded body with `decision` (#436b section 7), so #spent counts it.
#consumeGrant(business, decision, method) {
const grant = this.#approvedGrant(business, decision);
if (grant?.method !== method) fail('not-authorized');
if (this.#spent(business, decision)) fail('decision-consumed');
this.#grantEvent(business, { operation: 'grant.' + method, decision }, decision);
return grant;
}
#hostRecord(business, host) {
const r = this.#store.get('SELECT * FROM hosts WHERE business=? AND host=?', business, host);
return r && { ...r, sudo: r.sudo === 1, hostKeys: JSON.parse(r.host_keys) };
}
// r6 X4 first bullet, the Approve record check: the host record and the session's grant record, whose
// decision is a credential.mint with a `host` grant, resolved yes by the human via cli and spent by
// #consumeGrant as grant.host. Read at each call, so a grant, a revoke and a hosts.remove need no reload.
#hostAccess(s, host) {
const r = this.#hostRecord(s.business, host);
if (!r) fail('host-unknown');
const g = this.#store.get('SELECT * FROM host_grants WHERE business=? AND agent=? AND host=?', s.business, s.role, host);
const spent = g && this.#store.get(
`SELECT 1 FROM events WHERE business=? AND ${GRANT_EVIDENCE} AND json_extract(body,'$.operation')='grant.host' AND json_extract(body,'$.decision')=? LIMIT 1`,
s.business,
g.decision,
);
// The grant must name this agent and host too, so one approval can't stand behind another record.
const grant = spent && this.#approvedGrant(s.business, g.decision);
if (grant?.method !== 'host' || grant.agent !== s.role || grant.host !== host) fail('host-not-granted');
return { entry: { address: r.address, port: r.port, user: r.user, hostKeys: r.hostKeys }, commands: JSON.parse(g.commands) };
}
// A grant outcome, from the bus, to the human's inbox and to the raising role, threaded as a
// close is.
#grantNotice(business, decision, body) {
const d = this.#store.get('SELECT raised_by_role,raised_by_run FROM decisions WHERE id=?', decision);
const thread = this.#threaded(business, d.raised_by_role)
? (this.#store.get(
"SELECT thread FROM thread_claims WHERE business=? AND holder_run=? AND op='claim' ORDER BY seq DESC LIMIT 1",
business,
d.raised_by_run,
)?.thread ?? null)
: null;
for (const [to, t] of [['human', null], [d.raised_by_role, thread]]) {
const mid = randomUUID();
this.#store.run(
'INSERT INTO messages(id,at,business,from_role,from_run,to_role,class,in_reply_to,decision,corrects,body) VALUES(?,?,?,?,?,?,?,?,?,?,?)',
mid,
this.#now(),
business,
NOTICE_ROLE,
NOTICE_RUN,
to,
'RESULT',
null,
decision,
null,
body,
);
this.#route(business, mid, to, t);
}
}
#enrollFailed(business, decision, host, code, extra = {}) {
this.#store.run('DELETE FROM hosts WHERE business=? AND host=? AND decision=?', business, host, decision);
this.#grantEvent(business, { action: 'grant.failed', grantDecision: decision, host, code, ...extra }, host);
this.#grantNotice(business, decision, `Enrollment of host ${host} (decision ${decision}) failed: ${code}.`);
}
// Trusted host API, as reconcileGrants is, and no verb reaches it: the executor's entry for one
// decision. Step 1 checks everything again, so a second start for a spent decision ends
// decision-consumed, and an unresolved or agent-resolved one ends not-authorized, with nothing
// written (r6 G10).
runEnrollment(business, decision) {
return this.#runEnrollment(business, decision);
}
// r6 E2, started by the human's cli yes. Never throws: the outcome is in the store and the notices,
// and the return value is its code, 'enrolled' or the refusal.
async #runEnrollment(business, decision) {
let grant;
try {
// Step 1, one transaction: spend, the not-enrolled check, the registry step and the record.
grant = this.#store.transaction(() => {
const g = this.#consumeGrant(business, decision, 'host-enroll');
if (this.#hostRecord(business, g.host)) {
this.#grantEvent(business, { action: 'grant.failed', grantDecision: decision, host: g.host, code: 'host-enrolled' }, g.host);
this.#grantNotice(business, decision, `Enrollment of host ${g.host} (decision ${decision}) failed: host-enrolled.`);
return null;
}
this.#grantEvent(business, { action: 'grant.host.enroll.step', grantDecision: decision, host: g.host, step: 'registry' }, g.host);
this.#store.run(
'INSERT INTO hosts(business,host,address,port,user,sudo,host_keys,installer_fingerprint,state,decision,checked_at) VALUES(?,?,?,?,?,?,?,?,?,?,?)',
business, g.host, g.address, g.port, g.user, Number(g.sudo), JSON.stringify(g.hostKeys), null, 'pending-install', decision, null,
);
return g;
});
} catch (e) {
// Spent, not approved or no store: nothing was written. The start reconcile ends a resolved,
// unspent decision.
return e instanceof BusError ? e.code : 'host-failed';
}
if (!grant) return 'host-enrolled';
const host = grant.host;
try {
if (!this.#hostGrant) fail('host-not-configured');
// Steps 2 and 3: the key pair, and the private half to its file.
const key = await this.#hostGrant.enroll(business, host);
// Step 4.
this.#store.transaction(() => {
const r = this.#hostRecord(business, host);
if (r?.decision !== decision) fail('host-changed');
this.#store.run('UPDATE hosts SET installer_fingerprint=? WHERE business=? AND host=?', key.fingerprint, business, host);
this.#grantEvent(
business,
{
action: 'grant.host.enrolled',
grantDecision: decision,
host,
registrySha256: registrySha256(r),
installerFingerprint: key.fingerprint,
user: r.user,
sudo: r.sudo,
},
host,
);
this.#grantNotice(
business,
decision,
`Host ${host} is enrolled (decision ${decision}) and pending install. ops-01 writes this line to ` +
`/etc/ssh/mosaic-keys/mosaic-installer, with <from> the value in /etc/mosaic-keys/from:\n` +
`restrict,command="/usr/local/sbin/mosaic-keys",from="<from>" ${key.publicLine} ` +
`${INSTALLER_COMMENT}${key.tag.slice('mosaic-'.length)}\n` +
`/etc/mosaic-keys/users lists ${r.user}.` +
(r.sudo ? ` /etc/mosaic-keys/privileged lists ${r.user}.` : ` /etc/mosaic-keys/privileged doesn't list ${r.user}.`) +
`\nThen run: mosaic hosts check ${host}`,
);
});
return 'enrolled';
} catch (e) {
const code = e instanceof BusError ? e.code : 'host-failed';
try {
this.#hostGrant?.unenroll(business, host);
} catch {}
try {
this.#store.transaction(() => this.#enrollFailed(business, decision, host, code));
} catch {}
return code;
}
}
// r6 X3 and duckworth-0067 N1: credential.mint decisions for `method`, resolved yes by the human via
// cli and never spent.
#openGrantDecisions(business, method) {
return this.#store
.all("SELECT id FROM decisions WHERE business=? AND action='credential.mint' AND route_to='human' ORDER BY at,id", business)
.map((d) => d.id)
.filter((id) => this.#approvedGrant(business, id)?.method === method && !this.#spent(business, id));
}
// Trusted host API, at start before the socket opens (r6 X3). An enrollment spent with no
// grant.host.enrolled or grant.failed died after its registry step: the key file and the record
// go, and it ends grant.failed interrupted. A resolved, unspent one is spent and ends the same way.
// Neither runs late.
reconcileGrants() {
const ended = [];
for (const business of Object.keys(this.#businesses)) {
const spent = this.#store.all(
`SELECT json_extract(body,'$.decision') AS decision FROM events WHERE business=? AND ${GRANT_EVIDENCE} AND json_extract(body,'$.operation')='grant.host-enroll' ORDER BY seq`,
business,
);
for (const { decision } of spent) {
const end = this.#store.get(
`SELECT 1 FROM events WHERE business=? AND ${GRANT_EVIDENCE} AND json_extract(body,'$.grantDecision')=? AND json_extract(body,'$.action') IN ('grant.host.enrolled','grant.failed') LIMIT 1`,
business,
decision,
);
if (end) continue;
const host = this.#raised(business, decision).grant.host;
this.#hostGrant?.unenroll(business, host);
this.#store.transaction(() =>
this.#enrollFailed(business, decision, host, 'interrupted', { tag: this.#hostGrant?.installerTag(business, host) ?? null }),
);
ended.push(decision);
}
for (const decision of this.#openGrantDecisions(business, 'host-enroll')) {
this.#store.transaction(() => {
const g = this.#consumeGrant(business, decision, 'host-enroll');
this.#grantEvent(business, { action: 'grant.failed', grantDecision: decision, host: g.host, code: 'interrupted', tag: this.#hostGrant?.installerTag(business, g.host) ?? null }, g.host);
this.#grantNotice(business, decision, `Enrollment of host ${g.host} (decision ${decision}) failed: interrupted.`);
});
ended.push(decision);
}
}
return ended;
}
// r6 E1, R3: the raise checks and the keyscan, then the raise transaction.
async #raiseGrant(s, a) {
try {
this.#store.transaction(() => this.#agent(s));
if (a.action !== 'credential.mint' || 'spec_path' in a || 'spec_commit' in a) fail('invalid-request');
if (this.#business(s.business).roles[s.role]?.definition !== STAFF_TO) fail('grant-chief-only');
if (!this.#hostGrant) fail('host-not-configured');
const enroll = this.#hostGrant.enrollCheck(a.grant);
if (this.#hostRecord(s.business, enroll.host)) fail('host-enrolled');
const hostKeys = await this.#hostGrant.keyscan(enroll.address, enroll.port);
if (!this.#hostGrant.sameFingerprints(hostKeys, enroll.fingerprints)) fail('host-key-mismatch');
return this.#store.transaction(() => {
if (this.#hostRecord(s.business, enroll.host)) fail('host-enrolled');
const result = this.#dispatch(s, 'decision.raise', a, null, { ...enroll, hostKeys });
this.#secretCheck(result);
return result;
});
} catch (e) {
throw this.#refused(s, e, { verb: 'decision.raise', args: a });
}
}
#hostsCaller(s, verb) {
if (s.reader) fail('read-only');
if (s.tester) return 'tester';
if (s.human) {
this.#human(s);
return s.human;
}
// hosts.remove takes access away, as a revoke does: the human or the tester (r6 E4).
if (verb === 'hosts.remove') fail('human-required');
this.#store.transaction(() => this.#agent(s));
if (this.#business(s.business).roles[s.role]?.definition !== STAFF_TO) fail('grant-chief-only');
return s.role;
}
// r6 E4: hosts.status, hosts.check and hosts.remove. Async, because check runs ssh.
async #requestHosts(s, verb, a) {
try {
const by = this.#hostsCaller(s, verb);
if (verb === 'hosts.status') {
keys(a, []);
return this.#store
.all('SELECT host FROM hosts WHERE business=? ORDER BY host', s.business)
.map(({ host }) => {
const r = this.#hostRecord(s.business, host);
return {
host,
address: r.address,
port: r.port,
user: r.user,
sudo: r.sudo,
state: r.state,
hostKeys: r.hostKeys.map(fingerprintOf),
installerFingerprint: r.installer_fingerprint,
checkedAt: r.checked_at,
hostLineLeft: this.#hostLinesLeft(s.business, host),
};
});
}
keys(a, ['host'], ['host']);
if (typeof a.host !== 'string' || !HOST_ID.test(a.host)) fail('invalid-request');
if (!this.#hostGrant) fail('host-not-configured');
if (verb === 'hosts.remove') {
return this.#store.transaction(() => {
const r = this.#hostRecord(s.business, a.host);
if (!r) fail('host-unknown');
if (this.#store.get('SELECT 1 FROM host_grants WHERE business=? AND host=? LIMIT 1', s.business, a.host)) fail('host-in-use');
if (this.#hostLinesLeft(s.business, a.host).length) fail('host-line-left');
this.#store.run('DELETE FROM hosts WHERE business=? AND host=?', s.business, a.host);
this.#grantEvent(s.business, { action: 'grant.host.removed', host: a.host, registrySha256: registrySha256(r), by }, a.host);
// Last, so a key file that won't go rolls the record and the event back.
this.#hostGrant.unenroll(s.business, a.host);
return { host: a.host, removed: true };
});
}
const before = this.#hostRecord(s.business, a.host);
if (!before) fail('host-unknown');
if (!before.installer_fingerprint) fail('host-not-enrolled');
const out = await this.#hostGrant.check(s.business, a.host, before);
if (out.exit !== 0) fail('host-failed');
return this.#store.transaction(() => {
const r = this.#hostRecord(s.business, a.host);
const sha = r && registrySha256(r);
if (!r || r.decision !== before.decision || sha !== registrySha256(before) || out.installerFingerprint !== r.installer_fingerprint)
fail('host-changed');
const at = this.#now();
this.#store.run("UPDATE hosts SET state='ready',checked_at=? WHERE business=? AND host=?", at, s.business, a.host);
this.#grantEvent(
s.business,
{ action: 'grant.host.checked', host: a.host, registrySha256: sha, installerFingerprint: r.installer_fingerprint, by },
a.host,
);
return { host: a.host, state: 'ready', checkedAt: at };
});
} catch (e) {
throw this.#refused(s, e, { verb, args: a });
}
}
// r6 E4, duckworth-0079 F2 and F3: every guarded body that left a line on the host names the host.
// Row b adds the sweep clear (F4).
#hostLinesLeft(business, host) {
return this.#store
.all(
`SELECT json_extract(body,'$.hostLineLeft') AS tag FROM events WHERE business=? AND ${GRANT_EVIDENCE} AND json_extract(body,'$.host')=? AND json_extract(body,'$.hostLineLeft') IS NOT NULL ORDER BY seq`,
business,
host,
)
.map((e) => e.tag);
}
#hardWait(s, action, target) {
if (GATED.has(action)) return true;
return action === 'role.revoke' && END_PROTECTED.includes(this.#business(s.business).roles[target]?.definition);
@@ -1246,6 +1609,7 @@ export class Broker {
this.#agent(s);
if (['human.input', 'launch.revoked', 'launch.restored', ...REVIEW_KINDS].includes(kind)) fail('reserved-event');
if (kind === 'action.allowed' && ['git.push.working', MERGE].includes(body?.action)) fail('reserved-event');
if (kind === 'action.allowed' && grantName(body)) fail('reserved-event');
return this.#event(s, kind, body, subject);
});
}
@@ -1371,6 +1735,9 @@ export class Broker {
return this.#requestUsage(s, request.verb, structuredClone(args));
if ((request.verb === 'message.receive' || request.verb === 'state') && this.#stops.has(s.business + ':' + s.run))
this.#limitStop(s);
// Row 234 (#1740, r6 E1): a raise with `grant` runs its keyscan before the raise transaction.
if (request.verb === 'decision.raise' && 'grant' in args) return this.#raiseGrant(s, structuredClone(args));
if (HOSTS_VERBS.includes(request.verb)) return this.#requestHosts(s, request.verb, structuredClone(args));
if (request.verb === 'spec.read' || request.verb === 'record.write' ||
(request.verb === 'decision.raise' && ('spec_path' in args || 'spec_commit' in args)))
return this.#requestProject(s, { verb: request.verb, args: structuredClone(args) });
@@ -1727,21 +2094,18 @@ export class Broker {
a = request.args ?? {};
keys(a, ['host', 'argv', 'stdin', 'timeout', 'decision'], ['host', 'argv']);
this.#secretCheck(a);
this.#store.transaction(() => this.#agent(s));
const b = this.#business(s.business);
identifier(a.host);
if (!Object.hasOwn(b.hosts ?? {}, a.host)) fail('host-unknown');
const grant = Object.hasOwn(b.roles[s.role]?.hosts ?? {}, a.host) ? b.roles[s.role].hosts[a.host] : null;
if (!grant) fail('host-not-granted');
// Row 234 (#1740, r6 X4): the store records, not the business file.
const access = this.#store.transaction(() => (this.#agent(s), this.#hostAccess(s, a.host)));
if (!validArgv(a.argv) || !validStdin(a.stdin)) fail('invalid-request');
if (!commandAllowed(a.argv, grant.commands)) fail('command-not-granted');
if (!commandAllowed(a.argv, access.commands)) fail('command-not-granted');
timeout = a.timeout ?? HOST_TIMEOUT_DEFAULT;
if (!Number.isInteger(timeout) || timeout < 1 || timeout > HOST_TIMEOUT_MAX) fail('invalid-request');
if (a.decision !== undefined) identifier(a.decision);
if (!executor) fail('host-not-configured');
// A busy host refuses before the decision is spent; the executor checks again under its lock.
if (executor.busy(s, a.host)) fail('host-busy');
entry = b.hosts[a.host];
entry = access.entry;
bound = hostRun(a.host, a.argv, a.stdin);
this.#store.transaction(() =>
this.#consumeAuthority(
@@ -1793,7 +2157,7 @@ export class Broker {
});
return out;
}
#dispatch(s, verb, a, spec = null) {
#dispatch(s, verb, a, spec = null, enroll = null) {
const b = this.#business(s.business),
by = s.human || s.run;
switch (verb) {
@@ -1898,9 +2262,12 @@ export class Broker {
'spec_commit',
'argv',
'stdin',
'grant',
],
['action', 'question', 'options', 'recommendation', 'blocking'],
);
// Only #raiseGrant passes `enroll`, after its checks and keyscan.
if (('grant' in a) !== (enroll !== null)) fail('invalid-request');
this.#agent(s);
if ((a.spec_path !== undefined || a.spec_commit !== undefined) && !spec) fail('invalid-spec-citation');
const cls = this.#classification(s, a.action, a.target);
@@ -1961,18 +2328,29 @@ export class Broker {
if (!sudo && (a.argv !== undefined || a.stdin !== undefined)) fail('invalid-request');
let question = pushCommit ? `${a.question} (commit ${pushCommit})` : a.question;
if (sudo) {
if (a.target === undefined || !Object.hasOwn(b.hosts ?? {}, a.target)) fail('host-unknown');
if (!validArgv(a.argv) || !validStdin(a.stdin)) fail('invalid-request');
if (a.target === undefined) fail('host-unknown');
// The human is asked only about a call host.run would take: the raiser's own grant and prefix (N1).
const grant = Object.hasOwn(b.roles[s.role]?.hosts ?? {}, a.target) ? b.roles[s.role].hosts[a.target] : null;
if (!grant) fail('host-not-granted');
if (!commandAllowed(a.argv, grant.commands)) fail('command-not-granted');
const h = b.hosts[a.target];
// Row 234 (r6 X4): read from the store records, with the Approve record check.
const access = this.#hostAccess(s, a.target);
if (!validArgv(a.argv) || !validStdin(a.stdin)) fail('invalid-request');
if (!commandAllowed(a.argv, access.commands)) fail('command-not-granted');
const h = access.entry;
question +=
`\n\nHost ${a.target}, ${h.user} at ${h.address} port ${h.port}, runs as root:` +
`\nargv: ${a.argv.map((x) => JSON.stringify(x)).join(' ')}` +
`\nstdin: ${JSON.stringify(a.stdin ?? '')}`;
}
// Row 234 (#1740, r6 E1): the human sees each scanned key with its fingerprint, and the sudo
// mark in plain words. The chief can't shape this block.
if (enroll) {
question +=
`\n\nEnroll host ${enroll.host}: user ${enroll.user} at ${enroll.address} port ${enroll.port}.` +
`\nHost keys from ssh-keyscan, equal to the requested fingerprints:` +
enroll.hostKeys.map((k) => `\n${fingerprintOf(k)} ${k}`).join('') +
(enroll.sudo
? `\n${enroll.user} has sudo on this host. Approving lets later grants for ${enroll.user} be installed, and section 9's first limit applies to ${enroll.user}.`
: `\n${enroll.user} has no sudo on this host.`);
}
string(question);
const did = randomUUID();
this.#store.run(
@@ -2010,6 +2388,7 @@ export class Broker {
a.action === 'role.revoke' ? (this.#holder(s.business, a.target)?.holder_run ?? null) : null,
...(pushCommit ? { pushCommit } : {}),
...(sudo ? { hostRun: hostRun(a.target, a.argv, a.stdin) } : {}),
...(enroll ? { grant: enroll } : {}),
},
a.task_ref ?? (taskRef(a.target) ? a.target : did),
);
@@ -2060,6 +2439,10 @@ export class Broker {
a.note ?? null,
s.human ? 'cli' : 'broker',
);
// Row 234 (#1740, r6 X1): the human's cli approve of a grant mint starts the executor once this
// transaction commits. It rechecks all of this itself, so a rollback starts nothing.
const grant = op === 'resolved' ? this.#approvedGrant(s.business, d.id) : null;
if (grant?.method === 'host-enroll') setImmediate(() => this.#runEnrollment(s.business, d.id));
if (s.human)
this.#event(
s,
@@ -2103,7 +2486,7 @@ export class Broker {
);
this.#route(s.business, mid, d.raised_by_role, thread);
}
return { id: d.id, op };
return grant ? { id: d.id, op, grant: 'started' } : { id: d.id, op };
}
case 'message.send': {
keys(a, ['to', 'body', 'class', 'in_reply_to', 'corrects', 'decision'], ['to', 'body']);
+1 -1
View File
@@ -98,7 +98,7 @@ export function assertCleanKey(value, key) {
if (key.needles.some((n) => encoded.includes(n))) deny('credential-leak');
}
// Row 199 (#1697, design D1, D4): a store dir is the stack instance's user's own and nobody else's.
const privateDir = (path, { missing = false } = {}) => {
export const privateDir = (path, { missing = false } = {}) => {
let st;
try {
st = lstatSync(path);
+217
View File
@@ -0,0 +1,217 @@
// Row 234 (#1740, #436a row a): the trusted half of host enrollment, run in the bus child. The broker
// gets it as `hostGrant` and holds every store write. This module holds what touches the network or
// a key file: the raise's ssh-keyscan (E1), the installer key pair (E2 steps 2 and 3) and the installer's
// `list` for hosts.check (E4).
//
// The installer key lives at <configDir>/secrets/hosts/<business>/<hostId>/installer, 0600 in 0700 dirs,
// written as 210's issue writes (tmp, fsync, rename). It is never read into an event or a message.
import { spawn } from 'node:child_process';
import { createHash } from 'node:crypto';
import {
closeSync, constants, fsyncSync, mkdirSync, mkdtempSync, openSync, readFileSync, renameSync, rmSync, rmdirSync,
unlinkSync, writeSync,
} from 'node:fs';
import { dirname, join } from 'node:path';
import { BusError } from './broker.mjs';
import { parseHostKey, privateDir } from './credentials.mjs';
import { runHost, HOST_LIMITS } from './host-run.mjs';
import { checkAddress, HOST_KEY, USER } from '../../business/src/hosts.mjs';
import { HOST_ID } from '../../business/src/vocabulary.mjs';
const deny = (code) => {
throw new BusError(code);
};
const PATH = '/usr/bin:/bin';
const KILL_WITH_PARENT = ['/usr/bin/setpriv', '--pdeathsig', 'KILL', '--'];
// E1: the users no enrollment may name. The host writer refuses them again (tools/host/mosaic-keys-write).
export const RESERVED_USERS = Object.freeze(['root', 'localadmin', 'mosaic-installer']);
const FINGERPRINT = /^SHA256:[A-Za-z0-9+/]{43}$/;
// E1: the three key types the keyscan asks for, so any other fingerprint can't match.
const SCAN_TYPES = 'ed25519,ecdsa,rsa';
export const INSTALLER_USER = 'mosaic-installer';
const INSTALLER_AGENT = '@installer';
const SCAN_MS = 10000;
// X2 step 2: `mosaic-` and the first 32 hex digits of sha256 over business, agent and host id joined
// with NUL. Names match ID_PATTERN or HOST_ID, neither holds NUL, so the join is one to one.
export function grantTag(business, agent, hostId) {
return 'mosaic-' + createHash('sha256').update([business, agent, hostId].join('\0')).digest('hex').slice(0, 32);
}
export const installerTag = (business, hostId) => grantTag(business, INSTALLER_AGENT, hostId);
// The OpenSSH fingerprint of a "<type> <base64>" line: sha256 over the decoded blob.
export function fingerprint(line) {
const blob = Buffer.from(line.split(' ')[1], 'base64');
return 'SHA256:' + createHash('sha256').update(blob).digest('base64').replace(/=+$/, '');
}
// E1's binding. Anything off refuses invalid-request; the broker adds the chief, action and enrolled
// checks. Returns a fresh object with only the bound keys.
export function enrollCheck(grant) {
const allowed = ['method', 'host', 'address', 'port', 'user', 'sudo', 'fingerprints'];
if (!grant || typeof grant !== 'object' || Array.isArray(grant)) deny('invalid-request');
const own = Object.keys(grant);
if (own.some((k) => !allowed.includes(k)) || allowed.some((k) => !own.includes(k))) deny('invalid-request');
if (grant.method !== 'host-enroll') deny('invalid-request');
if (typeof grant.host !== 'string' || !HOST_ID.test(grant.host)) deny('invalid-request');
try {
checkAddress(grant.address, 'grant.address');
} catch {
deny('invalid-request');
}
if (!Number.isInteger(grant.port) || grant.port < 1 || grant.port > 65535) deny('invalid-request');
if (typeof grant.user !== 'string' || !USER.test(grant.user) || RESERVED_USERS.includes(grant.user)) deny('invalid-request');
if (typeof grant.sudo !== 'boolean') deny('invalid-request');
const prints = grant.fingerprints;
if (!Array.isArray(prints) || prints.length < 1 || prints.length > 3) deny('invalid-request');
if (prints.some((f) => typeof f !== 'string' || !FINGERPRINT.test(f)) || new Set(prints).size !== prints.length) deny('invalid-request');
return { method: 'host-enroll', host: grant.host, address: grant.address, port: grant.port, user: grant.user, sudo: grant.sudo, fingerprints: [...prints] };
}
// The keyscan's output lines, "<host> <type> <base64>", as "<type> <base64>". Comment lines and any
// other type drop out. A key listed twice counts once.
export function scanLines(stdout) {
const keys = [];
for (const line of stdout.split('\n')) {
if (!line || line.startsWith('#')) continue;
const parts = line.trim().split(/\s+/);
if (parts.length !== 3) continue;
const key = parts[1] + ' ' + parts[2];
if (HOST_KEY.test(key) && !keys.includes(key)) keys.push(key);
}
return keys;
}
// E1: `ssh-keyscan -T 5 -p <port> -t ed25519,ecdsa,rsa <address>` from the stack host. It reads public
// keys and logs in to nothing. A scan that can't start refuses host-failed. One that runs and gets no key,
// as from a host that doesn't answer, returns no lines, so the raise refuses host-key-mismatch
// (duckworth-0081 N3).
export function keyscan(address, port, { bin = 'ssh-keyscan', limitMs = SCAN_MS } = {}) {
return new Promise((resolve, reject) => {
const child = spawn(KILL_WITH_PARENT[0], [...KILL_WITH_PARENT.slice(1), bin, '-T', '5', '-p', String(port), '-t', SCAN_TYPES, '--', address], {
env: { PATH },
detached: true,
stdio: ['ignore', 'pipe', 'ignore'],
});
const out = [];
let size = 0;
child.stdout.on('data', (chunk) => {
size += chunk.length;
if (size <= 65536) out.push(chunk);
});
const timer = setTimeout(() => {
try {
process.kill(-child.pid, 'SIGKILL');
} catch {}
}, limitMs);
child.on('error', () => (clearTimeout(timer), reject(new BusError('host-failed'))));
child.on('close', () => (clearTimeout(timer), resolve(scanLines(Buffer.concat(out).toString('utf8')))));
});
}
// E1: the scanned set equals the requested set, or host-key-mismatch.
export function sameFingerprints(lines, prints) {
const scanned = new Set(lines.map(fingerprint));
return scanned.size === prints.length && prints.every((f) => scanned.has(f));
}
const run = (bin, args) =>
new Promise((resolve, reject) => {
const child = spawn(bin, args, { env: { PATH }, stdio: ['ignore', 'ignore', 'ignore'] });
child.on('error', () => reject(new BusError('host-failed')));
child.on('close', (code) => (code === 0 ? resolve() : reject(new BusError('host-failed'))));
});
export function hostGrant({ configDir, runRoot, limits = HOST_LIMITS, scan = keyscan, keygen = 'ssh-keygen' }) {
const keyRoot = join(configDir, 'secrets', 'hosts');
const keyPath = (business, hostId) => join(keyRoot, business, hostId, 'installer');
const readKey = (business, hostId) => {
const file = keyPath(business, hostId);
for (const dir of [keyRoot, dirname(dirname(file)), dirname(file)]) privateDir(dir);
let fd;
try {
fd = openSync(file, constants.O_RDONLY | constants.O_NOFOLLOW);
return parseHostKey(readFileSync(fd, 'utf8'));
} catch (e) {
if (e instanceof BusError) throw e;
deny('credential-file');
} finally {
if (fd !== undefined) closeSync(fd);
}
};
return {
enrollCheck,
sameFingerprints,
fingerprint,
installerTag,
keyscan: (address, port) => scan(address, port),
// E2 steps 2 and 3. Returns the public line and fingerprint; the private half only reaches the file.
async enroll(business, hostId) {
mkdirSync(runRoot, { recursive: true, mode: 0o700 });
privateDir(runRoot);
const dir = mkdtempSync(join(runRoot, 'h-'));
try {
const tag = installerTag(business, hostId);
await run(keygen, ['-q', '-t', 'ed25519', '-N', '', '-C', INSTALLER_USER + '-' + tag.slice('mosaic-'.length), '-f', join(dir, 'key')]);
const key = parseHostKey(readFileSync(join(dir, 'key'), 'utf8'));
const file = keyPath(business, hostId);
mkdirSync(keyRoot, { recursive: true, mode: 0o700 });
privateDir(keyRoot);
for (const d of [dirname(dirname(file)), dirname(file)]) {
if (!privateDir(d, { missing: true })) mkdirSync(d, { mode: 0o700 });
privateDir(d);
}
const tmp = file + '.tmp';
try {
unlinkSync(tmp);
} catch (e) {
if (e.code !== 'ENOENT') deny('credential-file');
}
let fd;
try {
fd = openSync(tmp, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW, 0o600);
writeSync(fd, key.text);
fsyncSync(fd);
} finally {
if (fd !== undefined) closeSync(fd);
}
renameSync(tmp, file);
const dirFd = openSync(dirname(file), constants.O_RDONLY);
try {
fsyncSync(dirFd);
} finally {
closeSync(dirFd);
}
return { publicLine: key.publicLine, fingerprint: key.fingerprint, tag };
} finally {
rmSync(dir, { recursive: true, force: true });
}
},
// The X3 undo and hosts.remove: the installer key and its two dirs, if present.
unenroll(business, hostId) {
const file = keyPath(business, hostId);
for (const f of [file + '.tmp', file]) {
try {
unlinkSync(f);
} catch (e) {
if (e.code !== 'ENOENT') deny('credential-file');
}
}
for (const d of [dirname(file), dirname(dirname(file))]) {
try {
rmdirSync(d);
} catch (e) {
if (!['ENOENT', 'ENOTEMPTY'].includes(e.code)) deny('credential-file');
}
}
},
// E4: `list <user>` as mosaic-installer with the installer key and the pinned host keys. A wrong
// pin refuses host-key-mismatch (runHost). Returns the installer fingerprint and the exit.
async check(business, hostId, record) {
const key = readKey(business, hostId);
const entry = { address: record.address, port: record.port, user: INSTALLER_USER, hostKeys: record.hostKeys };
const out = await runHost({ root: runRoot, hostId, entry, key, timeout: 30, limits, command: `list ${record.user}` });
return { installerFingerprint: key.fingerprint, exit: out.exit };
},
};
}
+4 -2
View File
@@ -159,7 +159,9 @@ function runSsh(args, stdin, timeoutMs, signal, limits, onSpawn) {
// D7: one call. `key` is the parsed entry from credentials.mjs (text and publicLine); `entry` is
// the registry's { address, port, user, hostKeys }. `root` is <configDir>/secrets/run.
export async function runHost({ root, hostId, entry, key, argv, stdin, timeout, signal = null, limits = HOST_LIMITS }) {
// Row 234 (#1740): `command`, when set, is sent as is in place of the sudo argv. The installer key's
// forced command reads it (tools/host/mosaic-keys).
export async function runHost({ root, hostId, entry, key, argv, stdin, timeout, signal = null, limits = HOST_LIMITS, command = null }) {
if (signal?.aborted) deny('credential-unavailable');
const dir = mkdtempSync(join(runRoot(root), 'h-'));
const sock = join(dir, 'a.sock');
@@ -178,7 +180,7 @@ export async function runHost({ root, hostId, entry, key, argv, stdin, timeout,
await addKey(sock, key.text, timeout + 30, { limitMs: limits.agentWaitMs, signal });
if (signal?.aborted) deny('credential-unavailable');
const out = await runSsh(
[...sshArgs({ hostId, ...entry, dir }), remoteCommand(argv)],
[...sshArgs({ hostId, ...entry, dir }), command ?? remoteCommand(argv)],
stdin,
timeout * 1000,
signal,
+35
View File
@@ -0,0 +1,35 @@
-- Row 234 (#436a row a, #1740): enrolled hosts and host grants live in the store, written only by
-- the grant executor, hosts.check, hosts.remove and revoke, each in the transaction of its event.
-- A grant event is an action.allowed whose body.action or body.operation is grant.*; the trigger
-- refuses one unless the actor is (NULL, 'grant'), the broker's own grant writer (#436b section 7).
CREATE TABLE hosts (
business TEXT NOT NULL,
host TEXT NOT NULL,
address TEXT NOT NULL,
port INTEGER NOT NULL CHECK (port BETWEEN 1 AND 65535),
user TEXT NOT NULL,
sudo INTEGER NOT NULL CHECK (sudo IN (0,1)),
host_keys TEXT NOT NULL CHECK (json_valid(host_keys) AND json_type(host_keys) = 'array' AND json_array_length(host_keys) > 0),
installer_fingerprint TEXT,
state TEXT NOT NULL CHECK (state IN ('pending-install','ready')),
decision TEXT NOT NULL REFERENCES decisions(id),
checked_at TEXT,
PRIMARY KEY (business, host),
CHECK (state = 'pending-install' OR (installer_fingerprint IS NOT NULL AND checked_at IS NOT NULL))
) STRICT;
CREATE TABLE host_grants (
business TEXT NOT NULL,
agent TEXT NOT NULL,
host TEXT NOT NULL,
commands TEXT NOT NULL CHECK (json_valid(commands) AND json_type(commands) = 'array' AND json_array_length(commands) > 0),
rotate_by TEXT NOT NULL,
tag TEXT NOT NULL CHECK (length(tag) = 39 AND tag GLOB 'mosaic-*' AND NOT substr(tag, 8) GLOB '*[^0-9a-f]*'),
decision TEXT NOT NULL REFERENCES decisions(id),
PRIMARY KEY (business, agent, host),
FOREIGN KEY (business, host) REFERENCES hosts(business, host)
) STRICT;
CREATE TRIGGER events_grant_actor BEFORE INSERT ON events
WHEN NEW.kind = 'action.allowed'
AND (json_extract(NEW.body, '$.action') GLOB 'grant.*' OR json_extract(NEW.body, '$.operation') GLOB 'grant.*')
AND NOT (NEW.actor_role IS NULL AND NEW.actor_run IS 'grant')
BEGIN SELECT RAISE(ABORT, 'only the grant writer records grant events'); END;
+50 -4
View File
@@ -16,6 +16,7 @@ import { organizationWriter } from '../../business/src/organization.mjs';
import { credentialVerbs } from './credential-verbs.mjs';
import { webService } from './web.mjs';
import { hostExecutor, sweepRunDir } from './host-run.mjs';
import { hostGrant } from './host-grant.mjs';
import { verifyHuman } from './human.mjs';
import { createUsage } from './usage.mjs';
import { createLimits } from './limits.mjs';
@@ -95,8 +96,16 @@ export async function startBroker({
];
// Row 199 (#1697, design D1): the per-agent store is <configDir>/secrets/agents.
const storeRoot = organization ? join(organization.dir, 'secrets', 'agents') : null;
credentials = new Credentials({ references, repoRoots: protectedRoots, dataRoot, storeRoot });
store = new Store(dataRoot);
// Row 234 (#1740, r6 X4 third bullet): each host grant record gives its agent a store reference to
// host.<id>, so a restart finds the keys the grant executor wrote. The loader refuses host.<id> in a
// role's credentials, so these never meet a business-file reference.
for (const g of store.all('SELECT business, agent, host, rotate_by FROM host_grants ORDER BY business, agent, host')) {
if (!Object.hasOwn(businesses, g.business)) continue;
const refs = (references[g.business + '/' + g.agent] = { ...(references[g.business + '/' + g.agent] ?? {}) });
refs['host.' + g.host] = { store: true, file: storeRoot ? join(storeRoot, g.business, g.agent, 'host.' + g.host) : null, rotateBy: g.rotate_by };
}
credentials = new Credentials({ references, repoRoots: protectedRoots, dataRoot, storeRoot });
// vikunja:19/305: the plug-in switch through the same human-only path as the limits: the verbs
// plugins and plugins.set, logged in changes.jsonl with the actor, a missing module refused.
const plugins=organization?pluginSettings({dir:organization.dir,rolesDir:organization.rolesDir,exists:hasModule}):null;
@@ -112,7 +121,10 @@ export async function startBroker({
const limits = createLimits({ store, read: id => settings ? settings.current(id) : businesses[id], log: limitLog });
for (const id of Object.keys(businesses)) await limits.refresh(id);
const withTokens = (id, result) => ({ ...result, tokens: limits.report(id) });
// Row 210 (#1707, design D7 step 6) and row 234 (#1740, r6 E2): the run dir and the host enrollment half.
const runRoot = organization ? join(organization.dir, 'secrets', 'run') : null;
const broker = new Broker({
hostGrant: runRoot ? hostGrant({ configDir: organization.dir, runRoot }) : null,
usage,
limits,
store,
@@ -181,8 +193,9 @@ export async function startBroker({
}
const web = await webServices(businesses);
// Row 210 (#1707, design D7 step 6): clear what a killed bus child left, before anyone can call.
const runRoot = organization ? join(organization.dir, 'secrets', 'run') : null;
if (runRoot) sweepRunDir(runRoot);
// Row 234 (r6 X3): end the enrollments a killed bus child left, before anyone can call.
broker.reconcileGrants();
const path = join(store.directory, 'broker.sock');
server = await serve({
web: (business) => web.get(business) ?? null,
@@ -257,9 +270,42 @@ export async function startBroker({
if (!(e instanceof BusError)) process.stderr.write(`bus: publish git-failed: ${JSON.stringify({ name: e?.name, code: e?.code })}\n`);
Object.assign(entry, { state: 'refused', code: e instanceof BusError ? e.code : 'git-failed' });
},
);
).then(() => {
if (entry.held) settle(ticket, entry, entry.held);
});
return { state: 'running' };
}
// Row 242 (#1748): the git socket gave up on a publish (publish-timeout) whose push it never
// collected. Its publishRefused waits for the push: until that ends the ticket stays open, so
// checkPublish refuses another publish on the decision with publish-busy (row 237). A push that
// reached the fork is then recorded, which spends the decision; one that didn't is filed with
// the socket's code. Either way the decision can't push a second ref.
function settle(ticket, entry, code) {
remotes.delete(ticket);
try {
if (entry.state === 'pushed') {
try {
broker.recordPublish(ticket, { ref: entry.ref, commit: entry.commit, previous: entry.previous });
process.stderr.write(`bus: publish ${ticket}: pushed after the git socket refused ${code}; recorded\n`);
return;
} catch (e) {
if (!(e instanceof BusError)) throw e;
code = e.code;
}
}
broker.publishRefused(ticket, { code });
} catch (e) {
process.stderr.write(`bus: publish ${ticket}: not settled: ${JSON.stringify({ name: e?.name, code: e?.code })}\n`);
}
}
function publishRefused(ticket, args) {
const entry = remotes.get(ticket);
if (!entry) return broker.publishRefused(ticket, args);
if (!/^[a-z-]{1,64}$/.test(args?.code ?? '')) throw new BusError('invalid-request');
if (entry.state === 'running') entry.held = args.code;
else settle(ticket, entry, args.code);
return args.code;
}
function publishResult(ticket) {
const entry = remotes.get(ticket);
if (!entry) throw new BusError('publish-unknown');
@@ -296,7 +342,7 @@ export async function startBroker({
publishPush,
publishResult,
recordPublish: (ticket, pushed) => broker.recordPublish(ticket, pushed),
publishRefused: (ticket, args) => broker.publishRefused(ticket, args),
publishRefused,
landHeld: (record) => broker.landHeld(record),
// Row 156 (#1651): the launcher's landing pass.
landHolders: (business) => broker.landHolders(business),
+5 -2
View File
@@ -1,6 +1,6 @@
import { createServer } from 'node:net';
import { lstatSync, chmodSync, unlinkSync } from 'node:fs';
import { BusError, TASK_VERBS, WEB_VERBS, HOST_VERBS, PROJECT_FILE_VERBS, PROJECT_SAVE_VERB, PROJECT_SAVE_REQUEST_MAX } from './broker.mjs';
import { BusError, TASK_VERBS, WEB_VERBS, HOST_VERBS, HOSTS_VERBS, PROJECT_FILE_VERBS, PROJECT_SAVE_VERB, PROJECT_SAVE_REQUEST_MAX } from './broker.mjs';
const LIMIT = 65536;
// Row 148: a document save may carry up to 1 MiB of text. Only a request that names that verb near
// its start (the proof object comes first) may pass LIMIT; the parsed verb is checked again below.
@@ -123,7 +123,10 @@ export async function serve({ broker, path, authenticateHuman = null, authentica
// Row138: only project Git effects need an await; preserve every other sync path below.
if (r.verb === 'usage.begin' || r.verb === 'usage.end' ||
r.verb === 'spec.read' || r.verb === 'record.write' || PROJECT_FILE_VERBS.includes(r.verb) ||
(r.verb === 'decision.raise' && (Object.hasOwn(r.args ?? {}, 'spec_path') || Object.hasOwn(r.args ?? {}, 'spec_commit')))) {
// Row 234 (#1740): a grant raise awaits its keyscan, and hosts.check its ssh.
HOSTS_VERBS.includes(r.verb) ||
(r.verb === 'decision.raise' && (Object.hasOwn(r.args ?? {}, 'spec_path') || Object.hasOwn(r.args ?? {}, 'spec_commit') ||
Object.hasOwn(r.args ?? {}, 'grant')))) {
const own = transient;
transient = null;
// Local Git may take 100 seconds; keep its result beyond the ordinary idle deadline.
+7 -2
View File
@@ -13,8 +13,8 @@ import { join, isAbsolute } from 'node:path';
import { createHash } from 'node:crypto';
const ddl = readFileSync(new URL('../schema.sql', import.meta.url), 'utf8');
export const SCHEMA_SHA256 = '6cde5232c57ac04dc6073112c0830c89cfc5e953d7a5727e7716263747fe8737';
export const SCHEMA_VERSION = '3i';
export const SCHEMA_SHA256 = '0a7b633ce7eec9d0bb37c3255f4244d97bd0c8eac38db91a11b4ef6cc982ec8c';
export const SCHEMA_VERSION = '3j';
// Ordered, named offline steps. New schema changes append a step and keep prior DDL pinned.
const legacyDDL = readFileSync(new URL('./migrations/3b.sql', import.meta.url), 'utf8');
const LEGACY_SHA256 = '179ffe356d4ff19a49b5ebad39b6c6bfd7771deb8e1b7c55b5e746f039d69e65';
@@ -56,6 +56,11 @@ const migrations = [{
from: '3h',
to: '3i',
sql: readFileSync(new URL('./migrations/3h-3i.sql', import.meta.url), 'utf8'),
}, {
// Row 234 (#1740): enrolled hosts, host grants and the grant event guard. Only new objects.
from: '3i',
to: '3j',
sql: readFileSync(new URL('./migrations/3i-3j.sql', import.meta.url), 'utf8'),
}];
const timedTables = [
'events',
@@ -69,6 +69,7 @@ async function engine(root, name, { store, schema, step = (s) => s }) {
for (const file of ['3b.sql', '3b-3c.sql', '3c-3d.sql', '3d-3e.sql', '3e-3f.sql', '3f-3g.sql', '3g-3h.sql'])
copyFileSync(here('../src/migrations/' + file), join(moduleRoot, 'src/migrations', file));
writeFileSync(join(moduleRoot, 'src/migrations/3h-3i.sql'), step(readFileSync(here('../src/migrations/3h-3i.sql'), 'utf8')));
copyFileSync(here('../src/migrations/3i-3j.sql'), join(moduleRoot, 'src/migrations/3i-3j.sql'));
return (await import(pathToFileURL(join(moduleRoot, 'src/store.mjs')))).Store;
}
@@ -76,7 +77,7 @@ test('REQ-LIMIT-4 3h copy migrates to 3i: every row keeps its seq, the sequence
const { root, path } = source3h(t), before = snapshot(path), was = master(path);
assert.throws(() => new Store(root), /schema-mismatch/);
new Store(root, { migrateSchema: true }).close();
assert.equal(version(path), '3i');
assert.equal(version(path), '3j');
// Every table's rows as they were, usage_records with its seq values and sqlite_sequence with 50.
assert.deepEqual(snapshot(path, before.columns).rows, before.rows);
assert.deepEqual(before.rows.usage_records.map(r => r.seq), [1, 2, 3]);
@@ -84,7 +85,9 @@ test('REQ-LIMIT-4 3h copy migrates to 3i: every row keeps its seq, the sequence
// sqlite_master as at 3h, the account line aside: same tables, indexes and triggers, same text.
const old = ' account TEXT CHECK (account IS NULL),\n';
assert.equal(was.filter(r => r.sql?.includes(old)).map(r => r.name).join(), 'usage_records');
assert.deepEqual(master(path), was.map(r => r.name === 'usage_records' ? { ...r, sql: r.sql.replace(old, ' account TEXT,\n') } : r));
// Row 234 (#1740) adds 3i→3j's own objects after this step. Its test is grant-migration.test.mjs.
const later = ['hosts', 'host_grants', 'events_grant_actor', 'sqlite_autoindex_hosts_1', 'sqlite_autoindex_host_grants_1'];
assert.deepEqual(master(path).filter(r => !later.includes(r.name)), was.map(r => r.name === 'usage_records' ? { ...r, sql: r.sql.replace(old, ' account TEXT,\n') } : r));
assert.equal(was.filter(r => r.type === 'trigger' && r.tbl_name === 'usage_records').length, 3);
const store = new Store(root);
try {
@@ -122,11 +125,11 @@ test('REQ-LIMIT-4 a 3h→3i step that fails partway leaves the store at 3h with
assert.deepEqual(snapshot(path, before.columns).rows, before.rows);
// Still a whole 3h store: the real step migrates it.
new Store(root, { migrateSchema: true }).close();
assert.equal(version(path), '3i');
assert.equal(version(path), '3j');
assert.deepEqual(snapshot(path, before.columns).rows, before.rows);
});
test('REQ-LIMIT-4 the 3h engine refuses a 3i store, migrating or not, and leaves it at 3i', async t => {
test('REQ-LIMIT-4 the 3h engine refuses a migrated store, migrating or not, and leaves it as it was', async t => {
const { root, path } = source3h(t);
new Store(root, { migrateSchema: true }).close();
const after = snapshot(path), was = master(path);
@@ -134,14 +137,14 @@ test('REQ-LIMIT-4 the 3h engine refuses a 3i store, migrating or not, and leaves
schema: ddl3h,
store: (source) => source
.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3h).digest('hex')}';`)
.replace("export const SCHEMA_VERSION = '3i';", "export const SCHEMA_VERSION = '3h';")
.replace("export const SCHEMA_VERSION = '3j';", "export const SCHEMA_VERSION = '3h';")
.replace(/\}, \{\n {2}\/\/ Row 126 \(#1620\)[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables'),
});
const pinned = readFileSync(join(root, 'engine-3h/src/store.mjs'), 'utf8');
assert.ok(pinned.includes("export const SCHEMA_VERSION = '3h';") && !pinned.includes('3h-3i'), 'the 3h engine has no 3h→3i step');
assert.throws(() => new Store3h(root), /schema-mismatch/);
assert.throws(() => new Store3h(root, { migrateSchema: true }), /schema-mismatch/);
assert.equal(version(path), '3i');
assert.equal(version(path), '3j');
assert.deepEqual(master(path), was);
assert.deepEqual(snapshot(path, after.columns).rows, after.rows);
});
+2 -2
View File
@@ -606,7 +606,7 @@ async function database3c(t) {
let source = readFileSync(here('../src/store.mjs'), 'utf8');
const pinned = source
.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3c).digest('hex')}';`)
.replace("export const SCHEMA_VERSION = '3i';", "export const SCHEMA_VERSION = '3c';")
.replace("export const SCHEMA_VERSION = '3j';", "export const SCHEMA_VERSION = '3c';")
.replace(/\}, \{\n {2}from: '3c',[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables');
assert.notEqual(pinned, source);
assert.ok(!pinned.includes('3c-3d'), 'the 3c engine has no 3c→3d step');
@@ -630,7 +630,7 @@ test('REQ-ROLE-2, REQ-COS-1: the 3c→3d migration lets the chief of staff end a
assert.throws(() => new Store(root), /schema-mismatch/);
const store = new Store(root, { migrateSchema: true });
t.after(() => store.close());
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3i');
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3j');
assert.equal(store.get("SELECT body FROM events WHERE id='preserved'").body, '{"fixture":true}');
assert.equal(store.get("SELECT holder_run FROM role_claims WHERE role='coder'").holder_run, 'coder-run');
const b = new Broker({ store, businesses: demo(), organization: null });
@@ -33,7 +33,7 @@ test('explicit 3b copy migration preserves evidence and opens as 3g; normal star
const original = readFileSync(source);
assert.throws(() => new Store(root), /schema-mismatch/);
const store = new Store(root, { migrateSchema: true });
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3i');
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3j');
assert.equal(store.get("SELECT body FROM events WHERE id='preserved'").body, '{"fixture":true}');
assert.equal(store.get('SELECT count(*) n FROM task_comment_cursors').n, 0);
const epoch = store.get("SELECT value FROM meta WHERE key='comment_cursor_epoch'").value;
@@ -75,15 +75,15 @@ test('named migration registry chains 3b through 3g to a fixture-only future ver
const sha = createHash('sha256').update(futureDDL).digest('hex');
let source = readFileSync(new URL('../src/store.mjs', import.meta.url), 'utf8');
source = source.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${sha}';`)
.replace("export const SCHEMA_VERSION = '3i';", "export const SCHEMA_VERSION = '3j';")
.replace('}];\nconst timedTables', `}, { from: '3i', to: '3j', sql: ${JSON.stringify(futureSQL)} }];\nconst timedTables`);
.replace("export const SCHEMA_VERSION = '3j';", "export const SCHEMA_VERSION = '3k';")
.replace('}];\nconst timedTables', `}, { from: '3j', to: '3k', sql: ${JSON.stringify(futureSQL)} }];\nconst timedTables`);
writeFileSync(join(moduleRoot, 'src/store.mjs'), source);
writeFileSync(join(moduleRoot, 'schema.sql'), futureDDL);
for (const name of ['3b.sql', '3b-3c.sql', '3c-3d.sql', '3d-3e.sql', '3e-3f.sql', '3f-3g.sql', '3g-3h.sql', '3h-3i.sql'])
for (const name of ['3b.sql', '3b-3c.sql', '3c-3d.sql', '3d-3e.sql', '3e-3f.sql', '3f-3g.sql', '3g-3h.sql', '3h-3i.sql', '3i-3j.sql'])
copyFileSync(new URL('../src/migrations/' + name, import.meta.url), join(moduleRoot, 'src/migrations', name));
const { Store: FutureStore } = await import(pathToFileURL(join(moduleRoot, 'src/store.mjs')));
const future = new FutureStore(root, { migrateSchema: true });
assert.equal(future.get("SELECT value FROM meta WHERE key='schema_version'").value, '3j');
assert.equal(future.get("SELECT value FROM meta WHERE key='schema_version'").value, '3k');
assert.equal(future.get("SELECT name FROM sqlite_master WHERE name='future_fixture'").name, 'future_fixture');
assert.equal(future.get("SELECT count(*) n FROM task_comment_cursors").n, 0);
assert.equal(future.get("SELECT id FROM events").id, 'preserved');
@@ -109,7 +109,7 @@ test('offline command migrates only the supplied fixture root and reports usage
const command = new URL('../src/migrate-schema.mjs', import.meta.url).pathname;
const migrated = spawnSync(process.execPath, [command, root], { encoding: 'utf8' });
assert.equal(migrated.status, 0, migrated.stderr);
assert.match(migrated.stdout, /bus schema 3i ready/);
assert.match(migrated.stdout, /bus schema 3j ready/);
new Store(root).close();
const usage = spawnSync(process.execPath, [command], { encoding: 'utf8' });
assert.equal(usage.status, 2);
+365
View File
@@ -0,0 +1,365 @@
PRAGMA journal_mode = WAL;
PRAGMA foreign_keys = ON;
CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL) STRICT;
CREATE TABLE events (
seq INTEGER PRIMARY KEY AUTOINCREMENT,
id TEXT NOT NULL UNIQUE,
at TEXT NOT NULL,
business TEXT NOT NULL,
kind TEXT NOT NULL CHECK (kind IN (
'session.launched',
'session.ended',
'action.allowed',
'action.refused',
'task.created',
'task.assigned',
'task.state',
'task.closed',
'task.changed.external',
'task.conflict',
'task.missing',
'review.requested',
'review.verdict',
'human.input',
'config.refused',
'credential.expiring',
'credential.expired',
'credential.changed',
'launch.revoked',
'launch.restored',
'digest.sent')),
actor_role TEXT, actor_run TEXT,
subject TEXT,
corrects TEXT REFERENCES events(id),
body TEXT NOT NULL CHECK (json_valid(body))
) STRICT;
CREATE TABLE role_claims (
seq INTEGER PRIMARY KEY AUTOINCREMENT,
at TEXT NOT NULL,
business TEXT NOT NULL, role TEXT NOT NULL,
op TEXT NOT NULL CHECK (op IN ('claim','release','revoke')),
holder_run TEXT NOT NULL,
harness TEXT NOT NULL, address TEXT,
by TEXT NOT NULL, reason TEXT,
decision TEXT
) STRICT;
CREATE TABLE decisions (
seq INTEGER PRIMARY KEY AUTOINCREMENT,
id TEXT NOT NULL UNIQUE,
at TEXT NOT NULL,
business TEXT NOT NULL, project TEXT,
raised_by_role TEXT NOT NULL, raised_by_run TEXT NOT NULL,
class TEXT NOT NULL CHECK (class IN ('routine','within-role','cross-role','gated')),
action TEXT NOT NULL,
route_to TEXT NOT NULL,
question TEXT NOT NULL,
options TEXT NOT NULL CHECK (json_valid(options) AND json_array_length(options) BETWEEN 2 AND 9),
recommendation TEXT NOT NULL,
task_ref TEXT CHECK (task_ref IS NULL OR (task_ref GLOB 'vikunja:[1-9]*/[1-9]*' AND NOT substr(task_ref, 9) GLOB '*[^0-9/]*' AND NOT substr(task_ref, 9) GLOB '*/*/*')), requirement_ref TEXT,
blocking INTEGER NOT NULL CHECK (blocking IN (0,1)),
supersedes TEXT REFERENCES decisions(id)
, spec_path TEXT, spec_commit TEXT) STRICT;
CREATE TABLE decision_events (
seq INTEGER PRIMARY KEY AUTOINCREMENT,
decision TEXT NOT NULL REFERENCES decisions(id),
at TEXT NOT NULL,
op TEXT NOT NULL CHECK (op IN ('seen','resolved','withdrawn','expired')),
by TEXT NOT NULL,
choice TEXT, note TEXT, via TEXT
) STRICT;
CREATE TABLE messages (
seq INTEGER PRIMARY KEY AUTOINCREMENT,
id TEXT NOT NULL UNIQUE,
at TEXT NOT NULL,
business TEXT NOT NULL,
from_role TEXT NOT NULL, from_run TEXT NOT NULL,
to_role TEXT NOT NULL,
class TEXT NOT NULL,
in_reply_to TEXT REFERENCES messages(id),
decision TEXT REFERENCES decisions(id),
corrects TEXT REFERENCES messages(id),
body TEXT NOT NULL
, to_name TEXT) STRICT;
CREATE TABLE deliveries (
seq INTEGER PRIMARY KEY AUTOINCREMENT,
message TEXT NOT NULL REFERENCES messages(id),
at TEXT NOT NULL,
op TEXT NOT NULL CHECK (op IN ('routed','delivered','failed','read')),
holder_run TEXT, transport TEXT, address TEXT, detail TEXT
) STRICT;
CREATE TABLE task_snapshots (
seq INTEGER PRIMARY KEY AUTOINCREMENT,
at TEXT NOT NULL,
business TEXT NOT NULL,
task_ref TEXT NOT NULL CHECK (task_ref GLOB 'vikunja:[1-9]*/[1-9]*' AND NOT substr(task_ref, 9) GLOB '*[^0-9/]*' AND NOT substr(task_ref, 9) GLOB '*/*/*'),
updated TEXT NOT NULL,
etag TEXT,
digest TEXT NOT NULL CHECK (length(digest) = 64 AND NOT digest GLOB '*[^0-9a-f]*'),
fields TEXT NOT NULL CHECK (json_valid(fields)),
source TEXT NOT NULL CHECK (source IN ('self','poll')),
via TEXT CHECK (via IN ('board','cursor','task','reconcile')),
read_at TEXT,
role TEXT, run TEXT,
CHECK ((source = 'self') = (role IS NOT NULL AND run IS NOT NULL)),
CHECK ((source = 'poll') = (via IS NOT NULL AND read_at IS NOT NULL)),
CHECK (json_type(fields, '$.bucket') IS 'integer'
OR (source IS 'poll' AND via IS 'task' AND json_type(fields, '$.gone') IS 'text'))
) STRICT;
CREATE INDEX task_snapshots_ref ON task_snapshots (business, task_ref, seq);
CREATE TRIGGER decisions_resolve_once BEFORE INSERT ON decision_events
WHEN NEW.op IN ('resolved','withdrawn','expired') AND EXISTS (
SELECT 1 FROM decision_events WHERE decision = NEW.decision AND op IN ('resolved','withdrawn','expired'))
BEGIN SELECT RAISE(ABORT, 'decision already closed'); END;
CREATE TRIGGER decisions_resolved_choice BEFORE INSERT ON decision_events
WHEN NEW.op = 'resolved' AND (NEW.choice IS NULL OR NOT EXISTS (
SELECT 1 FROM decisions d, json_each(d.options) o WHERE d.id = NEW.decision AND json_extract(o.value,'$.key') = NEW.choice))
BEGIN SELECT RAISE(ABORT, 'resolution must name one of the options'); END;
CREATE TRIGGER role_one_holder BEFORE INSERT ON role_claims
WHEN NEW.op = 'claim' AND (SELECT op FROM role_claims WHERE business = NEW.business AND role = NEW.role ORDER BY seq DESC LIMIT 1) = 'claim'
BEGIN SELECT RAISE(ABORT, 'role already held'); END;
CREATE TRIGGER role_release_by_holder BEFORE INSERT ON role_claims
WHEN NEW.op IN ('release','revoke') AND COALESCE((SELECT op FROM role_claims WHERE business = NEW.business AND role = NEW.role ORDER BY seq DESC LIMIT 1),'') <> 'claim'
BEGIN SELECT RAISE(ABORT, 'role is not held'); END;
CREATE TRIGGER role_release_same_run BEFORE INSERT ON role_claims
WHEN NEW.op = 'release' AND (SELECT holder_run FROM role_claims WHERE business = NEW.business AND role = NEW.role ORDER BY seq DESC LIMIT 1) <> NEW.holder_run
BEGIN SELECT RAISE(ABORT, 'only the holder releases; others revoke'); END;
CREATE TRIGGER role_revoke_needs_decision BEFORE INSERT ON role_claims
WHEN NEW.op = 'revoke' AND NEW.decision IS NULL AND NEW.reason IS NULL
BEGIN SELECT RAISE(ABORT, 'revoke needs a resolved decision or a reason'); END;
CREATE TRIGGER meta_no_update BEFORE UPDATE ON meta BEGIN SELECT RAISE(ABORT, 'meta is append-only'); END;
CREATE TRIGGER meta_no_delete BEFORE DELETE ON meta BEGIN SELECT RAISE(ABORT, 'meta is append-only'); END;
CREATE TRIGGER events_no_update BEFORE UPDATE ON events BEGIN SELECT RAISE(ABORT, 'events is append-only'); END;
CREATE TRIGGER events_no_delete BEFORE DELETE ON events BEGIN SELECT RAISE(ABORT, 'events is append-only'); END;
CREATE TRIGGER role_claims_no_update BEFORE UPDATE ON role_claims BEGIN SELECT RAISE(ABORT, 'role_claims is append-only'); END;
CREATE TRIGGER role_claims_no_delete BEFORE DELETE ON role_claims BEGIN SELECT RAISE(ABORT, 'role_claims is append-only'); END;
CREATE TRIGGER decisions_no_update BEFORE UPDATE ON decisions BEGIN SELECT RAISE(ABORT, 'decisions is append-only'); END;
CREATE TRIGGER decisions_no_delete BEFORE DELETE ON decisions BEGIN SELECT RAISE(ABORT, 'decisions is append-only'); END;
CREATE TRIGGER decision_events_no_update BEFORE UPDATE ON decision_events BEGIN SELECT RAISE(ABORT, 'decision_events is append-only'); END;
CREATE TRIGGER decision_events_no_delete BEFORE DELETE ON decision_events BEGIN SELECT RAISE(ABORT, 'decision_events is append-only'); END;
CREATE TRIGGER messages_no_update BEFORE UPDATE ON messages BEGIN SELECT RAISE(ABORT, 'messages is append-only'); END;
CREATE TRIGGER messages_no_delete BEFORE DELETE ON messages BEGIN SELECT RAISE(ABORT, 'messages is append-only'); END;
CREATE TRIGGER deliveries_no_update BEFORE UPDATE ON deliveries BEGIN SELECT RAISE(ABORT, 'deliveries is append-only'); END;
CREATE TRIGGER deliveries_no_delete BEFORE DELETE ON deliveries BEGIN SELECT RAISE(ABORT, 'deliveries is append-only'); END;
CREATE TRIGGER meta_no_replace BEFORE INSERT ON meta WHEN EXISTS (SELECT 1 FROM meta WHERE key = NEW.key) BEGIN SELECT RAISE(ABORT, 'meta is append-only'); END;
CREATE TRIGGER events_no_replace BEFORE INSERT ON events WHEN EXISTS (SELECT 1 FROM events WHERE seq = NEW.seq OR id = NEW.id) BEGIN SELECT RAISE(ABORT, 'events is append-only'); END;
CREATE TRIGGER role_claims_no_replace BEFORE INSERT ON role_claims WHEN EXISTS (SELECT 1 FROM role_claims WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'role_claims is append-only'); END;
CREATE TRIGGER decisions_no_replace BEFORE INSERT ON decisions WHEN EXISTS (SELECT 1 FROM decisions WHERE seq = NEW.seq OR id = NEW.id) BEGIN SELECT RAISE(ABORT, 'decisions is append-only'); END;
CREATE TRIGGER decision_events_no_replace BEFORE INSERT ON decision_events WHEN EXISTS (SELECT 1 FROM decision_events WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'decision_events is append-only'); END;
CREATE TRIGGER messages_no_replace BEFORE INSERT ON messages WHEN EXISTS (SELECT 1 FROM messages WHERE seq = NEW.seq OR id = NEW.id) BEGIN SELECT RAISE(ABORT, 'messages is append-only'); END;
CREATE TRIGGER deliveries_no_replace BEFORE INSERT ON deliveries WHEN EXISTS (SELECT 1 FROM deliveries WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'deliveries is append-only'); END;
CREATE TRIGGER task_snapshots_no_update BEFORE UPDATE ON task_snapshots BEGIN SELECT RAISE(ABORT, 'task_snapshots is append-only'); END;
CREATE TRIGGER task_snapshots_no_delete BEFORE DELETE ON task_snapshots BEGIN SELECT RAISE(ABORT, 'task_snapshots is append-only'); END;
CREATE TRIGGER task_snapshots_no_replace BEFORE INSERT ON task_snapshots WHEN EXISTS (SELECT 1 FROM task_snapshots WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'task_snapshots is append-only'); END;
CREATE TRIGGER decisions_blocking_needs_task BEFORE INSERT ON decisions
WHEN NEW.blocking = 1 AND NEW.task_ref IS NULL
BEGIN SELECT RAISE(ABORT, 'a blocking decision cites the task it blocks'); END;
CREATE TRIGGER events_launch_by_human BEFORE INSERT ON events
WHEN NEW.kind IN ('launch.revoked','launch.restored') AND (NEW.actor_role IS NOT NULL OR NEW.actor_run IS NOT NULL)
BEGIN SELECT RAISE(ABORT, 'only the human revokes or restores launching'); END;
CREATE TRIGGER events_credential_body BEFORE INSERT ON events
WHEN NEW.kind GLOB 'credential.*' AND (
json_extract(NEW.body, '$.service') IS NULL OR json_extract(NEW.body, '$.service') NOT IN ('gitea','vikunja')
OR json_extract(NEW.body, '$.instance') IS NULL)
BEGIN SELECT RAISE(ABORT, 'credential events name a service and a role instance'); END;
CREATE TRIGGER events_task_missing_body BEFORE INSERT ON events
WHEN NEW.kind = 'task.missing' AND (
json_extract(NEW.body, '$.reason') IS NULL OR json_extract(NEW.body, '$.reason') NOT IN ('moved','not-found','no-access')
OR (json_extract(NEW.body, '$.reason') = 'moved' AND json_type(NEW.body, '$.project') IS NOT 'integer'))
BEGIN SELECT RAISE(ABORT, 'task.missing carries a reason, and the new project when moved'); END;
CREATE TRIGGER events_task_subject BEFORE INSERT ON events
WHEN NEW.kind GLOB 'task.*' AND (NEW.subject IS NULL OR NOT (NEW.subject GLOB 'vikunja:[1-9]*/[1-9]*' AND NOT substr(NEW.subject, 9) GLOB '*[^0-9/]*' AND NOT substr(NEW.subject, 9) GLOB '*/*/*'))
BEGIN SELECT RAISE(ABORT, 'a task event names its task in subject'); END;
CREATE TRIGGER events_task_created_body BEFORE INSERT ON events
WHEN NEW.kind = 'task.created' AND (
json_type(NEW.body, '$.request') IS NOT 'text'
OR NOT EXISTS (SELECT 1 FROM events WHERE id = json_extract(NEW.body, '$.request')
AND kind = 'human.input' AND business = NEW.business)
OR json_type(NEW.body, '$.requirement') IS NOT 'text'
OR NOT json_extract(NEW.body, '$.requirement') GLOB 'REQ-[A-Z]*-[1-9]*'
OR json_extract(NEW.body, '$.requirement') GLOB 'REQ-*[^A-Z0-9-]*')
BEGIN SELECT RAISE(ABORT, 'task.created cites the human.input that asked for it and a requirement id'); END;
CREATE VIEW launch_state AS
SELECT business, CASE kind WHEN 'launch.revoked' THEN 'revoked' ELSE 'allowed' END AS state, at
FROM events e WHERE kind IN ('launch.revoked','launch.restored')
AND seq = (SELECT max(seq) FROM events WHERE business = e.business AND kind IN ('launch.revoked','launch.restored'));
CREATE VIEW task_external_changes AS
SELECT p.business, p.task_ref, p.seq, p.via, p.updated, p.digest, ls.digest AS self_digest
FROM task_snapshots p
LEFT JOIN task_snapshots ls ON ls.seq = (SELECT max(seq) FROM task_snapshots
WHERE business = p.business AND task_ref = p.task_ref AND source = 'self')
WHERE p.source = 'poll'
AND p.seq = (SELECT max(seq) FROM task_snapshots WHERE business = p.business AND task_ref = p.task_ref)
AND (ls.seq IS NULL OR (p.updated >= ls.updated AND p.read_at > ls.at AND p.digest <> ls.digest));
CREATE VIEW task_current AS
SELECT s.seq, s.at, s.business, s.task_ref, s.updated, s.etag, s.digest, s.fields, s.source, s.via, s.read_at, s.role, s.run
FROM task_snapshots s
WHERE s.seq = (SELECT max(c.seq) FROM task_snapshots c
WHERE c.business = s.business AND c.task_ref = s.task_ref
AND (c.source = 'self'
OR c.read_at > coalesce((SELECT ls.at FROM task_snapshots ls
WHERE ls.business = c.business AND ls.task_ref = c.task_ref AND ls.source = 'self'
ORDER BY ls.seq DESC LIMIT 1), '')));
CREATE VIEW tasks_open AS
SELECT s.business, s.task_ref, json_extract(s.fields, '$.bucket') AS bucket, s.seq
FROM task_current s
WHERE json_type(s.fields, '$.gone') IS NULL
AND json_extract(s.fields, '$.done') = 0;
CREATE VIEW urgent_inbox AS
SELECT d.id, d.business, d.task_ref, d.question, d.at
FROM decisions d
WHERE d.class = 'gated' AND d.blocking = 1
AND NOT EXISTS (SELECT 1 FROM decision_events x WHERE x.decision = d.id AND x.op IN ('resolved','withdrawn','expired'));
CREATE TABLE task_comment_cursors (
seq INTEGER PRIMARY KEY AUTOINCREMENT,
business TEXT NOT NULL,
task_id INTEGER NOT NULL CHECK (task_id > 0),
last_comment INTEGER NOT NULL CHECK (last_comment > 0)
) STRICT;
CREATE INDEX task_comment_cursors_latest ON task_comment_cursors (business, task_id, seq);
CREATE TRIGGER task_comment_cursors_no_update BEFORE UPDATE ON task_comment_cursors
BEGIN SELECT RAISE(ABORT, 'task_comment_cursors is append-only'); END;
CREATE TRIGGER task_comment_cursors_no_delete BEFORE DELETE ON task_comment_cursors
BEGIN SELECT RAISE(ABORT, 'task_comment_cursors is append-only'); END;
CREATE TRIGGER task_comment_cursors_no_replace BEFORE INSERT ON task_comment_cursors
WHEN EXISTS (SELECT 1 FROM task_comment_cursors WHERE seq = NEW.seq)
BEGIN SELECT RAISE(ABORT, 'task_comment_cursors is append-only'); END;
CREATE TRIGGER task_comment_cursors_forward BEFORE INSERT ON task_comment_cursors
WHEN NEW.last_comment <= COALESCE((SELECT last_comment FROM task_comment_cursors
WHERE business = NEW.business AND task_id = NEW.task_id ORDER BY seq DESC LIMIT 1), 0)
BEGIN SELECT RAISE(ABORT, 'comment cursor must advance'); END;
CREATE TABLE projects (
seq INTEGER PRIMARY KEY AUTOINCREMENT,
id TEXT NOT NULL UNIQUE,
at TEXT NOT NULL,
business TEXT NOT NULL,
name TEXT NOT NULL,
repo TEXT,
created_by TEXT NOT NULL,
UNIQUE (business, name)
) STRICT;
CREATE TABLE threads (
seq INTEGER PRIMARY KEY AUTOINCREMENT,
id TEXT NOT NULL UNIQUE,
at TEXT NOT NULL,
business TEXT NOT NULL,
project TEXT NOT NULL REFERENCES projects(id),
instance TEXT NOT NULL,
owner TEXT NOT NULL,
name TEXT NOT NULL,
UNIQUE (project, name)
) STRICT;
CREATE TABLE thread_claims (
seq INTEGER PRIMARY KEY AUTOINCREMENT,
at TEXT NOT NULL,
business TEXT NOT NULL,
thread TEXT NOT NULL REFERENCES threads(id),
op TEXT NOT NULL CHECK (op IN ('claim','release','revoke')),
holder_run TEXT NOT NULL,
harness TEXT NOT NULL, address TEXT,
by TEXT NOT NULL, reason TEXT,
decision TEXT
) STRICT;
CREATE TABLE message_threads (
message TEXT PRIMARY KEY REFERENCES messages(id),
thread TEXT NOT NULL REFERENCES threads(id)
) STRICT;
CREATE TABLE thread_opens (
seq INTEGER PRIMARY KEY AUTOINCREMENT,
at TEXT NOT NULL,
thread TEXT NOT NULL REFERENCES threads(id),
by TEXT NOT NULL
) STRICT;
CREATE INDEX thread_claims_thread ON thread_claims (thread, seq);
CREATE INDEX message_threads_thread ON message_threads (thread);
CREATE TRIGGER threads_project_business BEFORE INSERT ON threads
WHEN (SELECT business FROM projects WHERE id = NEW.project) IS NOT NEW.business
BEGIN SELECT RAISE(ABORT, 'a thread belongs to its project''s business'); END;
CREATE TRIGGER thread_one_holder BEFORE INSERT ON thread_claims
WHEN NEW.op = 'claim' AND (SELECT op FROM thread_claims WHERE thread = NEW.thread ORDER BY seq DESC LIMIT 1) = 'claim'
BEGIN SELECT RAISE(ABORT, 'thread already held'); END;
CREATE TRIGGER thread_one_per_run BEFORE INSERT ON thread_claims
WHEN NEW.op = 'claim' AND EXISTS (SELECT 1 FROM thread_claims c WHERE c.holder_run = NEW.holder_run AND c.business = NEW.business
AND c.seq = (SELECT max(seq) FROM thread_claims WHERE thread = c.thread) AND c.op = 'claim')
BEGIN SELECT RAISE(ABORT, 'run already holds a thread'); END;
CREATE TRIGGER thread_release_by_holder BEFORE INSERT ON thread_claims
WHEN NEW.op IN ('release','revoke') AND COALESCE((SELECT op FROM thread_claims WHERE thread = NEW.thread ORDER BY seq DESC LIMIT 1),'') <> 'claim'
BEGIN SELECT RAISE(ABORT, 'thread is not held'); END;
CREATE TRIGGER thread_release_same_run BEFORE INSERT ON thread_claims
WHEN NEW.op = 'release' AND (SELECT holder_run FROM thread_claims WHERE thread = NEW.thread ORDER BY seq DESC LIMIT 1) <> NEW.holder_run
BEGIN SELECT RAISE(ABORT, 'only the holder releases; others revoke'); END;
CREATE TRIGGER thread_revoke_needs_decision BEFORE INSERT ON thread_claims
WHEN NEW.op = 'revoke' AND NEW.decision IS NULL AND NEW.reason IS NULL
BEGIN SELECT RAISE(ABORT, 'revoke needs a resolved decision or a reason'); END;
CREATE TRIGGER thread_claims_business BEFORE INSERT ON thread_claims
WHEN (SELECT business FROM threads WHERE id = NEW.thread) IS NOT NEW.business
BEGIN SELECT RAISE(ABORT, 'a thread claim names the thread''s business'); END;
CREATE TRIGGER message_threads_match BEFORE INSERT ON message_threads
WHEN NOT EXISTS (SELECT 1 FROM messages m JOIN threads t ON t.id = NEW.thread
WHERE m.id = NEW.message AND m.business = t.business AND t.instance IN (m.to_role, m.from_role))
BEGIN SELECT RAISE(ABORT, 'a threaded message is to or from the thread''s instance'); END;
CREATE TRIGGER delivery_same_thread BEFORE INSERT ON deliveries
WHEN NEW.op IN ('routed','delivered') AND (NEW.op = 'delivered' OR NEW.holder_run IS NOT NULL)
AND EXISTS (SELECT 1 FROM message_threads mt JOIN messages m ON m.id = mt.message JOIN threads t ON t.id = mt.thread
WHERE mt.message = NEW.message AND m.to_role = t.instance
AND NOT EXISTS (SELECT 1 FROM thread_claims c WHERE c.seq = (SELECT max(seq) FROM thread_claims WHERE thread = t.id)
AND c.op = 'claim' AND c.holder_run IS NEW.holder_run))
BEGIN SELECT RAISE(ABORT, 'a threaded message goes only to its thread''s holder'); END;
CREATE TRIGGER projects_no_update BEFORE UPDATE ON projects BEGIN SELECT RAISE(ABORT, 'projects is append-only'); END;
CREATE TRIGGER projects_no_delete BEFORE DELETE ON projects BEGIN SELECT RAISE(ABORT, 'projects is append-only'); END;
CREATE TRIGGER projects_no_replace BEFORE INSERT ON projects WHEN EXISTS (SELECT 1 FROM projects WHERE seq = NEW.seq OR id = NEW.id) BEGIN SELECT RAISE(ABORT, 'projects is append-only'); END;
CREATE TRIGGER threads_no_update BEFORE UPDATE ON threads BEGIN SELECT RAISE(ABORT, 'threads is append-only'); END;
CREATE TRIGGER threads_no_delete BEFORE DELETE ON threads BEGIN SELECT RAISE(ABORT, 'threads is append-only'); END;
CREATE TRIGGER threads_no_replace BEFORE INSERT ON threads WHEN EXISTS (SELECT 1 FROM threads WHERE seq = NEW.seq OR id = NEW.id) BEGIN SELECT RAISE(ABORT, 'threads is append-only'); END;
CREATE TRIGGER thread_claims_no_update BEFORE UPDATE ON thread_claims BEGIN SELECT RAISE(ABORT, 'thread_claims is append-only'); END;
CREATE TRIGGER thread_claims_no_delete BEFORE DELETE ON thread_claims BEGIN SELECT RAISE(ABORT, 'thread_claims is append-only'); END;
CREATE TRIGGER thread_claims_no_replace BEFORE INSERT ON thread_claims WHEN EXISTS (SELECT 1 FROM thread_claims WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'thread_claims is append-only'); END;
CREATE TRIGGER message_threads_no_update BEFORE UPDATE ON message_threads BEGIN SELECT RAISE(ABORT, 'message_threads is append-only'); END;
CREATE TRIGGER message_threads_no_delete BEFORE DELETE ON message_threads BEGIN SELECT RAISE(ABORT, 'message_threads is append-only'); END;
CREATE TRIGGER message_threads_no_replace BEFORE INSERT ON message_threads WHEN EXISTS (SELECT 1 FROM message_threads WHERE message = NEW.message) BEGIN SELECT RAISE(ABORT, 'message_threads is append-only'); END;
CREATE TRIGGER thread_opens_no_update BEFORE UPDATE ON thread_opens BEGIN SELECT RAISE(ABORT, 'thread_opens is append-only'); END;
CREATE TRIGGER thread_opens_no_delete BEFORE DELETE ON thread_opens BEGIN SELECT RAISE(ABORT, 'thread_opens is append-only'); END;
CREATE TRIGGER thread_opens_no_replace BEFORE INSERT ON thread_opens WHEN EXISTS (SELECT 1 FROM thread_opens WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'thread_opens is append-only'); END;
CREATE TABLE instance_names (
seq INTEGER PRIMARY KEY AUTOINCREMENT,
at TEXT NOT NULL,
business TEXT NOT NULL,
instance TEXT NOT NULL,
name TEXT NOT NULL,
by TEXT NOT NULL
) STRICT;
CREATE TRIGGER instance_names_unique BEFORE INSERT ON instance_names
WHEN EXISTS (SELECT 1 FROM instance_names n WHERE n.business = NEW.business AND n.name = NEW.name
AND n.instance <> NEW.instance
AND n.seq = (SELECT max(seq) FROM instance_names WHERE business = n.business AND instance = n.instance))
BEGIN SELECT RAISE(ABORT, 'an instance name is unique in its business'); END;
CREATE TRIGGER instance_names_no_update BEFORE UPDATE ON instance_names BEGIN SELECT RAISE(ABORT, 'instance_names is append-only'); END;
CREATE TRIGGER instance_names_no_delete BEFORE DELETE ON instance_names BEGIN SELECT RAISE(ABORT, 'instance_names is append-only'); END;
CREATE TRIGGER instance_names_no_replace BEFORE INSERT ON instance_names WHEN EXISTS (SELECT 1 FROM instance_names WHERE seq = NEW.seq) BEGIN SELECT RAISE(ABORT, 'instance_names is append-only'); END;
-- Row 124 (#1618): immutable source snapshots and turn outcomes, never transcript text.
CREATE TABLE usage_records (
seq INTEGER PRIMARY KEY AUTOINCREMENT,
at TEXT NOT NULL,
business TEXT NOT NULL,
run TEXT NOT NULL,
instance TEXT NOT NULL,
provider TEXT,
model TEXT,
account TEXT,
turn TEXT NOT NULL,
source TEXT NOT NULL,
phase TEXT NOT NULL CHECK (phase IN ('begin','end')),
status TEXT NOT NULL CHECK (status IN ('known','unknown')),
code TEXT,
cursor TEXT CHECK (cursor IS NULL OR json_valid(cursor)),
input INTEGER,
output INTEGER,
cache_write INTEGER,
cache_read INTEGER,
UNIQUE (business,run,turn,source,phase),
CHECK ((status='unknown' AND input IS NULL AND output IS NULL AND cache_write IS NULL AND cache_read IS NULL)
OR (status='known' AND input >= 0 AND output >= 0 AND cache_write >= 0 AND cache_read >= 0
AND input IS NOT NULL AND output IS NOT NULL AND cache_write IS NOT NULL AND cache_read IS NOT NULL))
) STRICT;
CREATE TRIGGER usage_records_no_update BEFORE UPDATE ON usage_records BEGIN SELECT RAISE(ABORT, 'usage_records is append-only'); END;
CREATE TRIGGER usage_records_no_delete BEFORE DELETE ON usage_records BEGIN SELECT RAISE(ABORT, 'usage_records is append-only'); END;
CREATE TRIGGER usage_records_no_replace BEFORE INSERT ON usage_records WHEN EXISTS (SELECT 1 FROM usage_records WHERE seq=NEW.seq OR (business=NEW.business AND run=NEW.run AND turn=NEW.turn AND source=NEW.source AND phase=NEW.phase)) BEGIN SELECT RAISE(ABORT, 'usage_records is append-only'); END;
+489
View File
@@ -0,0 +1,489 @@
// Row 234 (#1740, #436a row a): host enrollment through a credential.mint raise with a `grant` binding
// (r6 E1, E2, E4, X1, X3). The keyscan is a fake that returns key lines this test makes, ssh-keygen is
// real, and the key files go under a scratch config dir. Nothing connects to any host.
import test from 'node:test';
import assert from 'node:assert/strict';
import { execFileSync } from 'node:child_process';
import { existsSync, mkdtempSync, readFileSync, readdirSync, rmSync, statSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { Broker, BusError, writeGrantEvent, registrySha256 } from '../src/broker.mjs';
import { Store } from '../src/store.mjs';
import { hostGrant, fingerprint, grantTag, installerTag } from '../src/host-grant.mjs';
import { grantDecision } from './host-fixtures.mjs';
const options = [{ key: 'yes', text: 'Yes' }, { key: 'no', text: 'No' }];
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const keyLine = (dir, name) => {
execFileSync('ssh-keygen', ['-q', '-t', 'ed25519', '-N', '', '-C', 'x', '-f', join(dir, name)]);
return readFileSync(join(dir, name + '.pub'), 'utf8').trim().split(' ').slice(0, 2).join(' ');
};
const refusesWith = async (p, code) => assert.rejects(Promise.resolve().then(() => p), (e) => (assert.equal(e.code, code), true));
const refusesSync = (fn, code) => assert.throws(fn, (e) => (assert.equal(e.code, code), true));
function setup(t, { keygen, check, hang = false } = {}) {
const root = mkdtempSync(join(tmpdir(), 'g234-'));
const store = new Store(root);
t.after(() => {
store.close();
rmSync(root, { recursive: true, force: true });
});
const hostKeys = [keyLine(root, 'hk1'), keyLine(root, 'hk2')];
const scans = [];
const scan = { lines: hostKeys };
const configDir = join(root, 'config');
const real = hostGrant({
configDir,
runRoot: join(configDir, 'secrets', 'run'),
scan: async (address, port) => (scans.push([address, port]), scan.lines),
...(keygen ? { keygen } : {}),
});
const checks = [];
// `hang`: steps 2 and 3 run, then the call never returns, as when the child dies before step 4.
const enroll = hang ? async (...args) => (await real.enroll(...args), new Promise(() => {})) : real.enroll;
const hg = { ...real, enroll, check: async (...args) => (checks.push(args), check ? check(...args) : { installerFingerprint: args[2].installer_fingerprint, exit: 0 }) };
const business = {
id: 'demo',
human: 'jason',
arbiters: { technical: 'pm', delivery: 'pm' },
roles: {
chief: { definition: 'chief-of-staff', authority: { withinRole: ['message.send'], crossRole: [] } },
pm: { definition: 'pm', authority: { withinRole: ['message.send'], crossRole: [] } },
},
vars: { 'bus.tester': true },
};
const make = (h = hg) => new Broker({ store, businesses: { demo: business }, hostGrant: h });
const broker = make();
const caps = {};
const bind = (b) => {
for (const role of ['chief', 'pm']) {
caps[role] = b.bindLaunch({ business: 'demo', role, run: role + '-run-' + Math.random(), harness: 'pi', address: role });
b.request(caps[role], { verb: 'role.claim' });
}
caps.human = b.bindHuman({ business: 'demo', human: 'jason', via: 'cli', outsideAgent: true });
caps.tester = b.bindTester({ business: 'demo' });
caps.reader = b.bindReader({ business: 'demo' });
};
bind(broker);
const grant = (over = {}) => ({
method: 'host-enroll',
host: 'web1',
address: '192.0.2.10',
port: 22,
user: 'jarvis',
sudo: true,
fingerprints: hostKeys.map(fingerprint),
...over,
});
const call = (who, verb, args = {}, b = broker) => b.request(caps[who], { verb, args });
const raise = (over = {}, who = 'chief', extra = {}) =>
call(who, 'decision.raise', {
action: 'credential.mint',
question: 'Enroll web1?',
options,
recommendation: 'yes',
blocking: false,
grant: grant(over),
...extra,
});
const events = () =>
store
.all("SELECT kind, actor_role, actor_run, body FROM events WHERE business='demo' ORDER BY seq")
.map((e) => ({ kind: e.kind, actor_role: e.actor_role, actor_run: e.actor_run, ...JSON.parse(e.body) }));
const grantEvents = () => events().filter((e) => e.actor_run === 'grant');
const settle = async (decision) => {
for (let i = 0; i < 250; i++) {
if (grantEvents().some((e) => e.grantDecision === decision && ['grant.host.enrolled', 'grant.failed'].includes(e.action))) return;
await sleep(20);
}
throw new Error('enrollment did not end');
};
const keyFile = (host = 'web1') => join(configDir, 'secrets', 'hosts', 'demo', host, 'installer');
const hostRow = (host = 'web1') => store.get("SELECT * FROM hosts WHERE business='demo' AND host=?", host);
const messages = () => store.all("SELECT from_role, to_role, class, decision, body FROM messages WHERE business='demo' ORDER BY rowid");
return { root, store, broker, make, bind, caps, call, raise, grant, hostKeys, scan, scans, checks, events, grantEvents, settle, keyFile, hostRow, messages, configDir };
}
const approve = async (f, over) => {
const d = await f.raise(over);
const r = f.call('human', 'decision.resolve', { id: d.id, choice: 'yes' });
await f.settle(d.id);
return { d, r };
};
test('G1, A1: only the chief raises a grant, and a refused raise stores nothing and runs no keyscan', async (t) => {
const f = setup(t);
await refusesWith(f.raise({}, 'pm'), 'grant-chief-only');
await refusesWith(f.call('human', 'decision.raise', { action: 'credential.mint', question: 'q', options, recommendation: 'yes', blocking: false, grant: f.grant() }), 'agent-required');
await refusesWith(f.call('tester', 'decision.raise', { action: 'credential.mint', question: 'q', options, recommendation: 'yes', blocking: false, grant: f.grant() }), 'tester-refused');
assert.equal(f.scans.length, 0);
assert.equal(f.store.get('SELECT count(*) AS n FROM decisions').n, 0);
assert.deepEqual(f.events().filter((e) => e.kind === 'action.refused').map((e) => e.code), ['grant-chief-only', 'agent-required', 'tester-refused']);
});
test('G1: grant goes only with credential.mint, row a takes only host-enroll, and E1 refuses each bad field', async (t) => {
const f = setup(t);
await refusesWith(f.raise({}, 'chief', { action: 'deploy' }), 'invalid-request');
for (const over of [
{ method: 'host' },
{ user: 'root' },
{ user: 'localadmin' },
{ user: 'mosaic-installer' },
{ user: 'Bad' },
{ sudo: 'yes' },
{ port: 0 },
{ port: 65536 },
{ host: 'Web_1' },
{ address: '127.0.0.1' },
{ address: 'localhost' },
{ fingerprints: [] },
{ fingerprints: ['SHA256:short'] },
{ fingerprints: [f.grant().fingerprints[0], f.grant().fingerprints[0]] },
{ extra: 1 },
])
await refusesWith(f.raise(over), 'invalid-request');
const { method, ...noMethod } = f.grant();
await refusesWith(
f.call('chief', 'decision.raise', { action: 'credential.mint', question: 'q', options, recommendation: 'yes', blocking: false, grant: noMethod }),
'invalid-request',
);
assert.equal(f.scans.length, 0);
assert.equal(f.store.get('SELECT count(*) AS n FROM decisions').n, 0);
});
test('G5, A5, A6: the scanned key set must equal the requested set, or host-key-mismatch and nothing stored', async (t) => {
const f = setup(t);
const other = keyLine(f.root, 'hk3');
for (const lines of [[f.hostKeys[0]], [...f.hostKeys, other], [f.hostKeys[0], other], []]) {
f.scan.lines = lines;
await refusesWith(f.raise(), 'host-key-mismatch');
}
f.scan.lines = f.hostKeys;
await refusesWith(f.raise({ fingerprints: [fingerprint(f.hostKeys[0])] }), 'host-key-mismatch');
assert.deepEqual(f.scans.at(-1), ['192.0.2.10', 22]);
assert.equal(f.store.get('SELECT count(*) AS n FROM decisions').n, 0);
});
test('G4 (host-enroll block, M1): the broker appends the keys, fingerprints and sudo line, and the cap counts the block', async (t) => {
const f = setup(t);
const d = await f.raise();
assert.equal(d.route_to ?? f.store.get('SELECT route_to FROM decisions WHERE id=?', d.id).route_to, 'human');
const q = f.store.get('SELECT question FROM decisions WHERE id=?', d.id).question;
assert.ok(q.startsWith('Enroll web1?\n\nEnroll host web1: user jarvis at 192.0.2.10 port 22.\n'));
for (const k of f.hostKeys) assert.ok(q.includes(`\n${fingerprint(k)} ${k}`));
assert.ok(q.endsWith('\njarvis has sudo on this host. Approving lets later grants for jarvis be installed, and section 9\'s first limit applies to jarvis.'));
const raiseBody = f.events().find((e) => e.operation === 'decision.raise');
assert.deepEqual(raiseBody.grant, { ...f.grant(), hostKeys: f.hostKeys });
// A question that carries a forged block still gets the broker's block after it.
const forged = 'x\n\nEnroll host web1: user other at 192.0.2.99 port 22.\nother has no sudo on this host.';
const d2 = await f.raise({ host: 'web2', sudo: false }, 'chief', { question: forged });
const q2 = f.store.get('SELECT question FROM decisions WHERE id=?', d2.id).question;
assert.ok(q2.startsWith(forged + '\n\nEnroll host web2: user jarvis at 192.0.2.10 port 22.\n'));
assert.ok(q2.endsWith('\njarvis has no sudo on this host.'));
// The 4096 cap holds with the block appended.
await refusesWith(f.raise({ host: 'web3' }, 'chief', { question: 'q'.repeat(4000) }), 'invalid-request');
assert.equal(f.store.get("SELECT count(*) AS n FROM decisions").n, 2);
});
test('G6, X1: the human\'s cli yes starts enrollment, which writes the record, the key file and the events in order', async (t) => {
const f = setup(t);
const { d, r } = await approve(f);
assert.deepEqual(r, { id: d.id, op: 'resolved', grant: 'started' });
const row = f.hostRow();
assert.equal(row.state, 'pending-install');
assert.equal(row.decision, d.id);
assert.equal(row.user, 'jarvis');
assert.equal(row.sudo, 1);
assert.deepEqual(JSON.parse(row.host_keys), f.hostKeys);
assert.match(row.installer_fingerprint, /^SHA256:/);
// E2 step 3: 0600 in 0700 dirs.
assert.equal(statSync(f.keyFile()).mode & 0o777, 0o600);
for (const dir of [join(f.keyFile(), '..'), join(f.keyFile(), '..', '..')]) assert.equal(statSync(dir).mode & 0o777, 0o700);
const g = f.grantEvents();
assert.deepEqual(g.map((e) => e.operation ?? e.action), ['grant.host-enroll', 'grant.host.enroll.step', 'grant.host.enrolled']);
for (const e of g) assert.deepEqual([e.kind, e.actor_role, e.actor_run], ['action.allowed', null, 'grant']);
assert.deepEqual(g[0], { kind: 'action.allowed', actor_role: null, actor_run: 'grant', operation: 'grant.host-enroll', decision: d.id });
assert.equal(g[1].grantDecision, d.id);
assert.equal(g[1].step, 'registry');
const enrolled = g[2];
assert.equal('decision' in enrolled, false);
assert.equal(enrolled.registrySha256, registrySha256({ address: '192.0.2.10', port: 22, user: 'jarvis', sudo: true, hostKeys: f.hostKeys }));
assert.equal(enrolled.installerFingerprint, row.installer_fingerprint);
assert.deepEqual([enrolled.user, enrolled.sudo], ['jarvis', true]);
// E2 step 4's message to the human and the chief: the installer line, public half only.
const tag = installerTag('demo', 'web1');
const pub = execFileSync('ssh-keygen', ['-y', '-f', f.keyFile()], { encoding: 'utf8' }).trim().split(' ').slice(0, 2).join(' ');
const notes = f.messages().filter((m) => m.from_role === '_bus' && m.decision === d.id);
assert.deepEqual(notes.map((m) => [m.to_role, m.class]), [['human', 'RESULT'], ['chief', 'RESULT']]);
for (const m of notes)
assert.ok(m.body.includes(`\nrestrict,command="/usr/local/sbin/mosaic-keys",from="<from>" ${pub} mosaic-installer-${tag.slice(7)}\n`));
// A7: no part of the private key reaches the store.
const secret = readFileSync(f.keyFile(), 'utf8').split('\n').filter((l) => l.length >= 24 && !l.startsWith('-----'));
const dump = JSON.stringify([f.store.all('SELECT body FROM events'), f.store.all('SELECT body FROM messages'), f.store.all('SELECT * FROM hosts')]);
for (const l of secret) assert.equal(dump.includes(l), false);
// The tmp dir under the run root is gone.
assert.deepEqual(readdirSync(join(f.configDir, 'secrets', 'run')), []);
});
test('A2, A4: a credential.mint without grant, and a no, start nothing', async (t) => {
const f = setup(t);
const plain = f.call('chief', 'decision.raise', { action: 'credential.mint', question: 'Hand-made key?', options, recommendation: 'yes', blocking: false });
assert.deepEqual(f.call('human', 'decision.resolve', { id: plain.id, choice: 'yes' }), { id: plain.id, op: 'resolved' });
const d = await f.raise();
assert.deepEqual(f.call('human', 'decision.resolve', { id: d.id, choice: 'no' }), { id: d.id, op: 'resolved' });
await sleep(200);
assert.deepEqual(f.grantEvents(), []);
assert.equal(f.hostRow(), undefined);
});
test('A3, X1: a resolve by the tester or an agent starts nothing', async (t) => {
const f = setup(t);
const d = await f.raise();
for (const who of ['tester', 'pm', 'chief']) {
try {
f.call(who, 'decision.resolve', { id: d.id, choice: 'yes' });
} catch {}
}
await sleep(200);
assert.deepEqual(f.grantEvents(), []);
assert.equal(f.hostRow(), undefined);
});
test('G6 (duckworth-0081 B3): a withdraw by the chief or the human, and an arbiter resolve, start nothing', async (t) => {
const f = setup(t);
for (const who of ['chief', 'human']) {
const d = await f.raise();
assert.deepEqual(f.call(who, 'decision.withdraw', { id: d.id }), { id: d.id, op: 'withdrawn' });
}
// A grant raise always routes to the human, so the fixture writes an open one routed to the pm
// arbiter, which then resolves it yes through the verb.
const d = { id: grantDecision(f.store, { route: 'pm', grant: f.grant(), close: null }) };
assert.deepEqual(f.call('pm', 'decision.resolve', { id: d.id, choice: 'yes' }), { id: d.id, op: 'resolved' });
await sleep(200);
assert.deepEqual(f.grantEvents(), []);
assert.equal(f.hostRow(), undefined);
assert.deepEqual(f.broker.reconcileGrants(), []);
assert.equal(await f.broker.runEnrollment('demo', d.id), 'not-authorized');
assert.deepEqual(f.grantEvents(), []);
});
test('G10 (duckworth-0081 B1): a second start for a spent decision refuses decision-consumed, and an unresolved or agent-resolved one not-authorized', async (t) => {
const f = setup(t);
const { d } = await approve(f);
const before = { events: f.grantEvents(), row: f.hostRow(), messages: f.messages().length };
assert.equal(await f.broker.runEnrollment('demo', d.id), 'decision-consumed');
assert.deepEqual(f.grantEvents(), before.events);
assert.deepEqual(f.hostRow(), before.row);
assert.equal(f.messages().length, before.messages);
// Unresolved, then resolved by an agent: written by SQL, since the verb refuses the agent first.
const open = await f.raise({ host: 'web2' });
assert.equal(await f.broker.runEnrollment('demo', open.id), 'not-authorized');
f.store.run('INSERT INTO decision_events(decision,at,op,by,choice,via) VALUES(?,?,?,?,?,?)', open.id, new Date().toISOString(), 'resolved', 'chief', 'yes', 'broker');
assert.equal(await f.broker.runEnrollment('demo', open.id), 'not-authorized');
// A cli yes by someone other than the human, so the by check alone refuses it (duckworth-0083 N8, A3b).
const other = await f.raise({ host: 'web3' });
f.store.run('INSERT INTO decision_events(decision,at,op,by,choice,via) VALUES(?,?,?,?,?,?)', other.id, new Date().toISOString(), 'resolved', 'chief', 'yes', 'cli');
assert.equal(await f.broker.runEnrollment('demo', other.id), 'not-authorized');
assert.deepEqual(f.broker.reconcileGrants(), []);
assert.deepEqual(f.grantEvents(), before.events);
assert.equal(f.hostRow('web2'), undefined);
assert.equal(existsSync(f.keyFile('web2')), false);
assert.equal(f.hostRow('web3'), undefined);
assert.equal(existsSync(f.keyFile('web3')), false);
});
test('A8, S1: two approved enrollments for one host: the second ends host-enrolled and the pin stays', async (t) => {
const f = setup(t);
const d1 = await f.raise();
const d2 = await f.raise({ user: 'melanie', sudo: false });
f.call('human', 'decision.resolve', { id: d1.id, choice: 'yes' });
await f.settle(d1.id);
const before = f.hostRow();
f.call('human', 'decision.resolve', { id: d2.id, choice: 'yes' });
await f.settle(d2.id);
assert.deepEqual(f.hostRow(), before);
const failed = f.grantEvents().find((e) => e.grantDecision === d2.id && e.action === 'grant.failed');
assert.equal(failed.code, 'host-enrolled');
assert.ok(f.grantEvents().some((e) => e.operation === 'grant.host-enroll' && e.decision === d2.id));
assert.equal(f.grantEvents().filter((e) => e.grantDecision === d2.id && e.action === 'grant.host.enroll.step').length, 0);
// A raise for an enrolled host refuses before the keyscan.
const scans = f.scans.length;
await refusesWith(f.raise(), 'host-enrolled');
assert.equal(f.scans.length, scans);
});
test('A9: the hosts primary key refuses a second record for (business, host)', (t) => {
const f = setup(t);
const d = f.call('chief', 'decision.raise', { action: 'credential.mint', question: 'q', options, recommendation: 'yes', blocking: false });
const insert = () =>
f.store.run(
"INSERT INTO hosts(business,host,address,port,user,sudo,host_keys,state,decision) VALUES('demo','web1','192.0.2.1',22,'u',0,'[\"k\"]','pending-install',?)",
d.id,
);
insert();
assert.throws(insert, /UNIQUE|PRIMARY/);
});
test('X3: a failure at step 2 deletes the record again and ends grant.failed, with no key file', async (t) => {
const f = setup(t, { keygen: '/nonexistent/ssh-keygen' });
const { d } = await approve(f);
assert.equal(f.hostRow(), undefined);
assert.equal(existsSync(f.keyFile()), false);
const failed = f.grantEvents().find((e) => e.action === 'grant.failed');
assert.deepEqual([failed.grantDecision, failed.host, failed.code], [d.id, 'web1', 'host-failed']);
const notes = f.messages().filter((m) => m.from_role === '_bus' && m.decision === d.id);
assert.deepEqual(notes.map((m) => m.to_role), ['human', 'chief']);
assert.ok(notes[0].body.includes('failed: host-failed'));
});
test('A10, X3: at start a resolved, unspent enrollment is spent and ends interrupted, and never runs late', async (t) => {
const f = setup(t);
const d = await f.raise();
// The resolve commits, and the child dies before step 1: the resolve row alone, written here.
f.store.run("INSERT INTO decision_events(decision,at,op,by,choice,via) VALUES(?,?,?,?,?,?)", d.id, new Date().toISOString(), 'resolved', 'jason', 'yes', 'cli');
const restarted = f.make();
assert.deepEqual(restarted.reconcileGrants(), [d.id]);
const g = f.grantEvents();
assert.deepEqual(g.map((e) => e.operation ?? e.action), ['grant.host-enroll', 'grant.failed']);
assert.deepEqual([g[1].code, g[1].host, g[1].tag], ['interrupted', 'web1', installerTag('demo', 'web1')]);
assert.equal(f.hostRow(), undefined);
assert.deepEqual(restarted.reconcileGrants(), []);
});
test('X3: at start an enrollment that stopped after its registry step loses its record and key file', async (t) => {
const f = setup(t, { hang: true });
const d = await f.raise();
f.call('human', 'decision.resolve', { id: d.id, choice: 'yes' });
for (let i = 0; i < 250 && !existsSync(f.keyFile()); i++) await sleep(20);
assert.equal(f.hostRow().state, 'pending-install');
assert.ok(existsSync(f.keyFile()));
const restarted = f.make();
assert.deepEqual(restarted.reconcileGrants(), [d.id]);
assert.equal(f.hostRow(), undefined);
assert.equal(existsSync(f.keyFile()), false);
const failed = f.grantEvents().find((e) => e.grantDecision === d.id && e.action === 'grant.failed');
assert.deepEqual([failed.code, failed.host, failed.tag], ['interrupted', 'web1', installerTag('demo', 'web1')]);
assert.deepEqual(f.messages().filter((m) => m.decision === d.id && m.from_role === '_bus').map((m) => m.to_role), ['human', 'chief']);
assert.deepEqual(restarted.reconcileGrants(), []);
});
test('G24 first sentence, M2, A12: tags are mosaic- and 32 hex, and differ by business, agent and host', () => {
const long = 'a'.repeat(30) + '.' + '_'.repeat(2) + 'b'.repeat(31);
assert.equal(long.length, 64);
const tags = [grantTag('demo', long, 'web1'), grantTag('demo', long + 'x', 'web1'), grantTag('other', long, 'web1'), grantTag('demo', long, 'web2'), installerTag('demo', 'web1'), installerTag('other', 'web1')];
for (const t of tags) assert.match(t, /^mosaic-[0-9a-f]{32}$/);
assert.equal(new Set(tags).size, tags.length);
});
test('E4: hosts.check sets the record ready and records grant.host.checked; a non-zero list leaves it pending', async (t) => {
const exits = [1, 0];
const f = setup(t, { check: (b, h, r) => ({ installerFingerprint: r.installer_fingerprint, exit: exits.shift() }) });
await approve(f);
await refusesWith(f.call('chief', 'hosts.check', { host: 'web1' }), 'host-failed');
assert.equal(f.hostRow().state, 'pending-install');
await refusesWith(f.call('pm', 'hosts.check', { host: 'web1' }), 'grant-chief-only');
await refusesWith(f.call('chief', 'hosts.check', { host: 'nope' }), 'host-unknown');
const r = await f.call('chief', 'hosts.check', { host: 'web1' });
assert.equal(r.state, 'ready');
assert.equal(f.hostRow().state, 'ready');
assert.equal(f.hostRow().checked_at, r.checkedAt);
const checked = f.grantEvents().find((e) => e.action === 'grant.host.checked');
assert.deepEqual([checked.host, checked.by, checked.installerFingerprint], ['web1', 'chief', f.hostRow().installer_fingerprint]);
assert.equal(checked.registrySha256, registrySha256({ address: '192.0.2.10', port: 22, user: 'jarvis', sudo: true, hostKeys: f.hostKeys }));
const status = await f.call('human', 'hosts.status');
assert.deepEqual(status.map((h) => [h.host, h.state, h.hostKeys, h.hostLineLeft]), [['web1', 'ready', f.hostKeys.map(fingerprint), []]]);
});
test('G20 (duckworth-0081 B3): the human\'s hosts.check marks the host ready, and the reader refuses', async (t) => {
const f = setup(t);
await approve(f);
await refusesWith(f.call('reader', 'hosts.check', { host: 'web1' }), 'read-only');
const r = await f.call('human', 'hosts.check', { host: 'web1' });
assert.equal(r.state, 'ready');
assert.equal(f.hostRow().state, 'ready');
const checked = f.grantEvents().filter((e) => e.action === 'grant.host.checked');
assert.deepEqual(checked.map((e) => e.by), ['jason']);
});
test('G18 (duckworth-0081 B2), broker half: a check refused host-key-mismatch leaves the host pending-install with no grant.host.checked', async (t) => {
const f = setup(t, { check: () => { throw new BusError('host-key-mismatch'); } });
await approve(f);
await refusesWith(f.call('chief', 'hosts.check', { host: 'web1' }), 'host-key-mismatch');
assert.equal(f.checks.length, 1);
assert.equal(f.hostRow().state, 'pending-install');
assert.equal(f.hostRow().checked_at, null);
assert.deepEqual(f.grantEvents().filter((e) => e.action === 'grant.host.checked'), []);
});
test('E4: a check whose installer fingerprint differs from the record refuses host-changed', async (t) => {
const f = setup(t, { check: () => ({ installerFingerprint: 'SHA256:' + 'A'.repeat(43), exit: 0 }) });
await approve(f);
await refusesWith(f.call('tester', 'hosts.check', { host: 'web1' }), 'host-changed');
assert.equal(f.hostRow().state, 'pending-install');
});
test('E4, G21: hosts.remove is the human\'s or the tester\'s, deletes the key and the record, and records grant.host.removed', async (t) => {
const f = setup(t);
await approve(f);
await refusesWith(f.call('chief', 'hosts.remove', { host: 'web1' }), 'human-required');
await refusesWith(f.call('reader', 'hosts.remove', { host: 'web1' }), 'read-only');
assert.deepEqual(await f.call('human', 'hosts.remove', { host: 'web1' }), { host: 'web1', removed: true });
assert.equal(f.hostRow(), undefined);
assert.equal(existsSync(f.keyFile()), false);
assert.equal(existsSync(join(f.keyFile(), '..')), false);
const removed = f.grantEvents().find((e) => e.action === 'grant.host.removed');
assert.deepEqual([removed.host, removed.by], ['web1', 'jason']);
await refusesWith(f.call('tester', 'hosts.remove', { host: 'web1' }), 'host-unknown');
});
test('G21, F1, F2, M4, A22: hosts.remove refuses host-line-left on a guarded hostLineLeft, and ignores look-alikes', async (t) => {
const f = setup(t);
const { d } = await approve(f);
const at = new Date().toISOString();
// Look-alikes: the same body under action.refused, and under another actor of another kind.
const lookAlike = JSON.stringify({ action: 'grant.failed', grantDecision: d.id, host: 'web1', hostLineLeft: installerTag('demo', 'web1') });
f.store.run("INSERT INTO events(id,at,business,kind,actor_role,actor_run,subject,body) VALUES('la1',?,'demo','action.refused',NULL,'grant','web1',?)", at, lookAlike);
f.store.run("INSERT INTO events(id,at,business,kind,actor_role,actor_run,subject,body) VALUES('la2',?,'demo','action.refused','chief','chief-run','web1',?)", at, lookAlike);
// The trigger refuses the look-alike as an action.allowed from a session.
assert.throws(() =>
f.store.run("INSERT INTO events(id,at,business,kind,actor_role,actor_run,subject,body) VALUES('la3',?,'demo','action.allowed','chief','chief-run','web1',?)", at, lookAlike),
);
assert.equal((await f.call('human', 'hosts.status'))[0].hostLineLeft.length, 0);
// F1: the real shape, through the grant writer. F2: any guarded body, not only grant.failed.
writeGrantEvent(f.store, at, 'demo', { action: 'grant.revoke.closed', tag: 'mosaic-' + '0'.repeat(32), host: 'web1', hostLineLeft: 'mosaic-' + '0'.repeat(32) }, 'web1');
assert.deepEqual((await f.call('human', 'hosts.status'))[0].hostLineLeft, ['mosaic-' + '0'.repeat(32)]);
await refusesWith(f.call('human', 'hosts.remove', { host: 'web1' }), 'host-line-left');
assert.ok(f.hostRow());
assert.ok(existsSync(f.keyFile()));
});
test('hosts.remove refuses host-in-use while a grant record names the host', async (t) => {
const f = setup(t);
const { d } = await approve(f);
f.store.run(
"INSERT INTO host_grants(business,agent,host,commands,rotate_by,tag,decision) VALUES('demo','pm','web1','[[\"true\"]]','2099-01-01',?,?)",
grantTag('demo', 'pm', 'web1'),
d.id,
);
await refusesWith(f.call('human', 'hosts.remove', { host: 'web1' }), 'host-in-use');
assert.ok(f.hostRow());
});
test('the grant writer refuses a body outside section 7\'s shape', (t) => {
const f = setup(t);
const at = new Date().toISOString();
for (const body of [
{ action: 'host.sudo' },
{ action: 'grant.failed', decision: 'd' },
{ operation: 'grant.host-enroll', decision: 'd', host: 'web1' },
{ operation: 'grant.host-enroll' },
{ operation: 'decision.raise', decision: 'd' },
])
assert.throws(() => writeGrantEvent(f.store, at, 'demo', body), /grant event shape/);
});
test('recordEvent refuses reserved-event for a grant.* action or operation', (t) => {
const f = setup(t);
for (const body of [{ action: 'grant.failed' }, { operation: 'grant.host-enroll', decision: 'x' }])
refusesSync(() => f.broker.recordEvent(f.caps.chief, { kind: 'action.allowed', body }), 'reserved-event');
assert.equal(f.store.get("SELECT count(*) AS n FROM events WHERE json_extract(body,'$.action') GLOB 'grant.*'").n, 0);
});
+212
View File
@@ -0,0 +1,212 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { chmodSync, copyFileSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { pathToFileURL } from 'node:url';
import { createHash } from 'node:crypto';
import { DatabaseSync } from 'node:sqlite';
import { Store, schemaDigest } from '../src/store.mjs';
import { Broker } from '../src/broker.mjs';
// Row 234 (#1740, #436a row a): the 3i→3j step adds the hosts and host_grants tables and the grant
// event guard (#436b section 7 at fa5658941). It runs on a 3i store built here from the pinned 3i
// schema (8fede5fc6's schema.sql) and fixture rows. No live store is read or copied.
const AT = '2026-10-21T00:00:00.000Z';
const TAG = 'mosaic-' + 'a'.repeat(32);
const ddl3i = readFileSync(new URL('./fixtures/schema-3i.sql', import.meta.url), 'utf8');
const here = (p) => new URL(p, import.meta.url);
const NEW = ['hosts', 'host_grants', 'events_grant_actor', 'sqlite_autoindex_hosts_1', 'sqlite_autoindex_host_grants_1'];
function source3i(t) {
const root = mkdtempSync(join(tmpdir(), 'grant-migrate-'));
t.after(() => rmSync(root, { recursive: true, force: true }));
mkdirSync(join(root, 'bus'), { mode: 0o700 });
const path = join(root, 'bus/bus.sqlite'), db = new DatabaseSync(path);
assert.equal(createHash('sha256').update(ddl3i).digest('hex'), '6cde5232c57ac04dc6073112c0830c89cfc5e953d7a5727e7716263747fe8737');
db.exec(ddl3i);
const run = (sql, ...args) => db.prepare(sql).run(...args);
run('INSERT INTO meta VALUES (?,?)', 'schema_version', '3i');
run('INSERT INTO meta VALUES (?,?)', 'schema_digest', schemaDigest(db));
run('INSERT INTO meta VALUES (?,?)', 'comment_cursor_epoch', AT);
run('INSERT INTO events(id,at,business,kind,actor_role,actor_run,body) VALUES(?,?,?,?,?,?,?)', 'e1', AT, 'demo', 'action.allowed', 'coder', 'run1', '{"action":"host.sudo"}');
run('INSERT INTO events(id,at,business,kind,body) VALUES(?,?,?,?,?)', 'e2', AT, 'demo', 'action.allowed', '{}');
db.close(); chmodSync(path, 0o600);
return { root, path };
}
function rows(path) {
const db = new DatabaseSync(path, { readOnly: true });
try {
return Object.fromEntries(db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name NOT IN ('hosts','host_grants')").all()
.map(({ name }) => [name, db.prepare(`SELECT * FROM ${name} ORDER BY rowid`).all()
.filter(r => name !== 'meta' || !['schema_version', 'schema_digest'].includes(r.key)).map(r => ({ ...r }))]));
} finally { db.close(); }
}
function master(path) {
const db = new DatabaseSync(path, { readOnly: true });
try {
return db.prepare('SELECT type,name,tbl_name,sql FROM sqlite_master ORDER BY type,name').all().map(r => ({ ...r }));
} finally { db.close(); }
}
const version = (path) => {
const db = new DatabaseSync(path, { readOnly: true });
try { return db.prepare("SELECT value FROM meta WHERE key='schema_version'").get().value; } finally { db.close(); }
};
// A copy of the bus module whose store.mjs, schema.sql and 3i→3j step the caller edits.
async function engine(root, name, { store, schema, step = (s) => s }) {
const moduleRoot = join(root, name);
mkdirSync(join(moduleRoot, 'src/migrations'), { recursive: true });
const source = readFileSync(here('../src/store.mjs'), 'utf8'), edited = store ? store(source) : source;
if (store) assert.notEqual(edited, source);
writeFileSync(join(moduleRoot, 'src/store.mjs'), edited);
writeFileSync(join(moduleRoot, 'schema.sql'), schema ?? readFileSync(here('../schema.sql'), 'utf8'));
for (const file of ['3b.sql', '3b-3c.sql', '3c-3d.sql', '3d-3e.sql', '3e-3f.sql', '3f-3g.sql', '3g-3h.sql', '3h-3i.sql'])
copyFileSync(here('../src/migrations/' + file), join(moduleRoot, 'src/migrations', file));
writeFileSync(join(moduleRoot, 'src/migrations/3i-3j.sql'), step(readFileSync(here('../src/migrations/3i-3j.sql'), 'utf8')));
return (await import(pathToFileURL(join(moduleRoot, 'src/store.mjs')))).Store;
}
function fresh(t) {
const root = mkdtempSync(join(tmpdir(), 'grant-store-'));
const store = new Store(root);
t.after(() => { store.close(); rmSync(root, { recursive: true, force: true }); });
store.run(`INSERT INTO decisions(id,at,business,raised_by_role,raised_by_run,class,action,route_to,question,options,recommendation,blocking)
VALUES('d1',?,'demo','chief','run1','gated','credential.mint','human','q','["yes","no"]','yes',0)`, AT);
return store;
}
const event = (store, actor, body, kind = 'action.allowed') => store.run(
'INSERT INTO events(id,at,business,kind,actor_role,actor_run,body) VALUES(?,?,?,?,?,?,?)',
'e' + Math.random().toString(16).slice(2), AT, 'demo', kind, actor[0], actor[1], JSON.stringify(body));
const host = (store, values = {}) => store.run(
`INSERT INTO hosts(business,host,address,port,user,sudo,host_keys,installer_fingerprint,state,decision,checked_at)
VALUES(:business,:host,:address,:port,:user,:sudo,:host_keys,:fp,:state,:decision,:checked)`,
{ business: 'demo', host: 'web1', address: '192.0.2.10', port: 22, user: 'deploy', sudo: 0,
host_keys: '["ssh-ed25519 AAAA"]', fp: null, state: 'pending-install', decision: 'd1', checked: null, ...values });
test('3i copy migrates to 3j: every row kept, and sqlite_master gains only the two tables, their keys and the guard', t => {
const { root, path } = source3i(t), before = rows(path), was = master(path);
assert.throws(() => new Store(root), /schema-mismatch/);
new Store(root, { migrateSchema: true }).close();
assert.equal(version(path), '3j');
assert.deepEqual(rows(path), before);
const now = master(path);
assert.deepEqual(now.filter(r => !NEW.includes(r.name)), was);
assert.deepEqual(now.filter(r => NEW.includes(r.name)).map(r => r.name).sort(), [...NEW].sort());
const store = new Store(root);
try {
assert.equal(store.get('PRAGMA quick_check').quick_check, 'ok');
assert.deepEqual(store.all('PRAGMA foreign_key_check'), []);
assert.equal(store.get('SELECT count(*) AS n FROM hosts').n, 0);
// An event row 210 wrote before the step stays, and a session can still write host.sudo.
event(store, ['coder', 'run2'], { action: 'host.sudo', phase: 'requested' });
} finally { store.close(); }
});
test('a 3i→3j step that fails partway leaves the store at 3i with every row and the schema intact', async t => {
const { root, path } = source3i(t), before = rows(path), was = master(path);
// The real step plus a statement that fails only when events has a row, so the empty replay at
// import passes and the store's run fails after both tables and the trigger exist.
const Failing = await engine(root, 'engine-failing', { step: (sql) => sql + 'INSERT INTO events SELECT * FROM events LIMIT 1;\n' });
assert.throws(() => new Failing(root, { migrateSchema: true }), /events is append-only/);
assert.equal(version(path), '3i');
assert.deepEqual(master(path), was);
assert.deepEqual(rows(path), before);
new Store(root, { migrateSchema: true }).close();
assert.equal(version(path), '3j');
});
test('the 3i engine refuses a 3j store, migrating or not, and leaves it at 3j', async t => {
const { root, path } = source3i(t);
new Store(root, { migrateSchema: true }).close();
const after = rows(path), was = master(path);
const Store3i = await engine(root, 'engine-3i', {
schema: ddl3i,
store: (source) => source
.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3i).digest('hex')}';`)
.replace("export const SCHEMA_VERSION = '3j';", "export const SCHEMA_VERSION = '3i';")
.replace(/\}, \{\n {2}\/\/ Row 234 \(#1740\)[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables'),
});
const pinned = readFileSync(join(root, 'engine-3i/src/store.mjs'), 'utf8');
assert.ok(pinned.includes("export const SCHEMA_VERSION = '3i';") && !pinned.includes('3i-3j'), 'the 3i engine has no 3i→3j step');
assert.throws(() => new Store3i(root), /schema-mismatch/);
assert.throws(() => new Store3i(root, { migrateSchema: true }), /schema-mismatch/);
assert.equal(version(path), '3j');
assert.deepEqual(master(path), was);
assert.deepEqual(rows(path), after);
});
test('A22b, sage-1703: a session-written grant.* row from before the step survives it, and the grant readers ignore it', async t => {
// The step adds the guard but leaves old rows. At 3i recordEvent had no reserved-event refusal, so a
// session could write these, and after the step only GRANT_EVIDENCE's actor filter keeps them out.
const { root, path } = source3i(t);
const db = new DatabaseSync(path);
const run = (sql, ...args) => db.prepare(sql).run(...args);
run(`INSERT INTO decisions(id,at,business,raised_by_role,raised_by_run,class,action,route_to,question,options,recommendation,blocking)
VALUES('d1',?,'demo','chief','run1','gated','credential.mint','human','q','["yes","no"]','yes',0)`, AT);
const legacy = [{ action: 'grant.host.removed', host: 'web1', hostLineLeft: TAG }, { operation: 'grant.host-enroll', decision: 'd1' }];
for (const [i, body] of legacy.entries())
run('INSERT INTO events(id,at,business,kind,actor_role,actor_run,body) VALUES(?,?,?,?,?,?,?)', 'old' + i, AT, 'demo', 'action.allowed', 'chief', 'run1', JSON.stringify(body));
db.close();
new Store(root, { migrateSchema: true }).close();
const store = new Store(root);
t.after(() => store.close());
assert.equal(store.get("SELECT count(*) AS n FROM events WHERE id IN ('old0','old1')").n, 2);
host(store);
const business = {
id: 'demo',
human: 'jason',
arbiters: { technical: 'pm', delivery: 'pm' },
roles: {
chief: { definition: 'chief-of-staff', authority: { withinRole: ['message.send'], crossRole: [] } },
pm: { definition: 'pm', authority: { withinRole: ['message.send'], crossRole: [] } },
},
vars: {},
};
const unenrolled = [];
const broker = new Broker({ store, businesses: { demo: business }, hostGrant: { unenroll: (b, h) => unenrolled.push([b, h]), installerTag: () => null } });
// The enrollment look-alike is not a spent decision, so nothing ends interrupted.
assert.deepEqual(broker.reconcileGrants(), []);
assert.equal(store.get("SELECT count(*) AS n FROM events WHERE actor_run='grant'").n, 0);
// The hostLineLeft look-alike is not a line left on the host.
const human = broker.bindHuman({ business: 'demo', human: 'jason', via: 'cli', outsideAgent: true });
assert.deepEqual(await broker.request(human, { verb: 'hosts.remove', args: { host: 'web1' } }), { host: 'web1', removed: true });
assert.deepEqual(unenrolled, [['demo', 'web1']]);
});
test('the guard: a grant.* action or operation on action.allowed needs the (NULL, grant) actor', t => {
const store = fresh(t);
const refused = /only the grant writer records grant events/;
for (const body of [{ action: 'grant.step' }, { action: 'grant.host.enrolled' }, { operation: 'grant.host-enroll', decision: 'd1' }]) {
assert.throws(() => event(store, ['chief', 'run1'], body), refused, JSON.stringify(body));
assert.throws(() => event(store, ['tester', 'run1'], body), refused);
assert.throws(() => event(store, [null, null], body), refused, 'the human and session-less pair is not the grant actor');
assert.throws(() => event(store, [null, 'run1'], body), refused);
assert.throws(() => event(store, ['grant', 'grant'], body), refused);
event(store, [null, 'grant'], body);
}
// Row 210's host.sudo and host.run stay outside the namespace, and so does any other kind.
event(store, ['coder', 'run1'], { action: 'host.sudo', phase: 'completed' });
event(store, ['coder', 'run1'], { action: 'grant.step' }, 'action.refused');
event(store, ['coder', 'run1'], { action: 'granted.step', operation: 'grantx' });
assert.equal(store.get("SELECT count(*) AS n FROM events WHERE actor_run='grant'").n, 3);
});
test('hosts: one record per business and host, a ready host has its installer and check, grants need the host', t => {
const store = fresh(t);
host(store);
assert.throws(() => host(store, { address: '192.0.2.11' }), /UNIQUE constraint failed: hosts\.business, hosts\.host/);
host(store, { business: 'other' });
assert.throws(() => host(store, { host: 'web2', state: 'ready' }), /CHECK constraint failed/);
assert.throws(() => host(store, { host: 'web2', state: 'ready', fp: 'SHA256:x' }), /CHECK constraint failed/);
host(store, { host: 'web2', state: 'ready', fp: 'SHA256:x', checked: AT });
assert.throws(() => host(store, { host: 'web3', state: 'live' }), /CHECK constraint failed/);
assert.throws(() => host(store, { host: 'web3', host_keys: '[]' }), /CHECK constraint failed/);
assert.throws(() => host(store, { host: 'web3', decision: 'nope' }), /FOREIGN KEY constraint failed/);
const grant = (values = {}) => store.run(
'INSERT INTO host_grants(business,agent,host,commands,rotate_by,tag,decision) VALUES(:business,:agent,:host,:commands,:rotate,:tag,:decision)',
{ business: 'demo', agent: 'coder', host: 'web2', commands: '[["true"]]', rotate: '2026-12-01', tag: TAG, decision: 'd1', ...values });
assert.throws(() => grant({ host: 'web9' }), /FOREIGN KEY constraint failed/);
for (const tag of ['mosaic-' + 'A'.repeat(32), 'mosaic-' + 'a'.repeat(31), 'mosaix-' + 'a'.repeat(32), 'mosaic-' + 'a'.repeat(31) + 'g'])
assert.throws(() => grant({ tag }), /CHECK constraint failed/, tag);
assert.throws(() => grant({ commands: '[]' }), /CHECK constraint failed/);
grant();
assert.throws(() => grant({ tag: 'mosaic-' + 'b'.repeat(32) }), /UNIQUE constraint failed/);
assert.throws(() => store.run("DELETE FROM hosts WHERE business='demo' AND host='web2'"), /FOREIGN KEY constraint failed/);
});
+6 -4
View File
@@ -7,6 +7,7 @@ import { Store } from '../src/store.mjs';
import { Broker, ACTIONS, HARD_WAITS } from '../src/broker.mjs';
import { ACTIONS as VOCABULARY, GATED_ONLY } from '../../business/src/vocabulary.mjs';
import { fixture } from './spec-fixtures.mjs';
import { hostGrantRecord, hostRecord } from './host-fixtures.mjs';
// PRD 1.5 REQ-WAIT-1, REQ-WAIT-2 and REQ-DEC-2 (row 139, #1633).
const WAITS = ['spend', 'data.delete', 'network.expose', 'account.create', 'credential.mint', 'message.external',
@@ -37,10 +38,11 @@ test('REQ-WAIT-2: a business granting a hard wait at either class is refused; th
test('REQ-DEC-2: every hard wait goes to the human, cited or not, and the arbiter cannot resolve it', async t => {
const f = await fixture(t, { roles: { coder: { definition: 'coder',
authority: { withinRole: ['message.send', 'record.write', 'git.push.working'], crossRole: [] },
hosts: { web1: { commands: [['systemctl', 'status']] } } } },
// Row 210: a host.sudo raise names a registered host the raiser is granted and an argv its grant allows.
hosts: { web1: { address: '10.0.0.5', port: 22, user: 'admin', hostKeys: ['ssh-ed25519 AAAA'] } } });
authority: { withinRole: ['message.send', 'record.write', 'git.push.working'], crossRole: [] } } } });
// Row 210: a host.sudo raise names a registered host the raiser is granted and an argv its grant allows.
// Row 234 (#1740): both are store records.
hostRecord(f.store, { host: 'web1', entry: { address: '10.0.0.5', port: 22, user: 'admin', hostKeys: ['ssh-ed25519 AAAA'] } });
hostGrantRecord(f.store, { agent: 'coder', host: 'web1', commands: [['systemctl', 'status']] });
// Row 199: a git.push.remote raise binds the task's latest push, so the task has one.
f.broker.recordPush(f.coder, { task_ref: 'vikunja:1/2', project: 'stack', branch: 'feat', ref: 'work/vikunja-stack-2',
commit: 'a'.repeat(40), previous: null });
+62
View File
@@ -0,0 +1,62 @@
// Row 234 (#1740, r6 X4): host and grant store records for tests. Row a has no way to make a host_grants
// record outside a fixture, because row b's grant executor writes them. The decision, its raise event and
// its close go in by SQL. The grant evidence goes through writeGrantEvent (F1), so the trigger sees the
// real shape.
import { randomUUID } from 'node:crypto';
import { registrySha256, writeGrantEvent } from '../src/broker.mjs';
import { grantTag } from '../src/host-grant.mjs';
const AT = '2026-10-21T00:00:00.000Z';
const OPTIONS = JSON.stringify([{ key: 'yes', text: 'Yes' }, { key: 'no', text: 'No' }]);
export const HUMAN_YES = { op: 'resolved', by: 'jason', choice: 'yes', via: 'cli' };
// A credential.mint decision with `grant` in its raise body. `close` is the decision_events row, or null
// for an open decision.
export function grantDecision(store, { business = 'demo', action = 'credential.mint', route = 'human', grant, close = HUMAN_YES }) {
const id = randomUUID();
store.run(
'INSERT INTO decisions(id,at,business,raised_by_role,raised_by_run,class,action,route_to,question,options,recommendation,blocking) VALUES(?,?,?,?,?,?,?,?,?,?,?,?)',
id, AT, business, 'chief', 'chief-run', 'gated', action, route, 'Grant?', OPTIONS, 'yes', 0,
);
store.run(
"INSERT INTO events(id,at,business,kind,actor_role,actor_run,subject,body) VALUES(?,?,?,'action.allowed',?,?,?,?)",
randomUUID(), AT, business, 'chief', 'chief-run', id,
JSON.stringify({ decision: id, action, operation: 'decision.raise', approvalChoice: 'yes', ...(grant ? { grant } : {}) }),
);
if (close)
store.run('INSERT INTO decision_events(decision,at,op,by,choice,via) VALUES(?,?,?,?,?,?)', id, AT, close.op, close.by, close.choice ?? null, close.via ?? null);
return id;
}
// A ready host record from an approved, spent and finished enrollment. grant.host.enrolled keeps the
// start reconcile from reading the enrollment as interrupted.
export function hostRecord(store, { business = 'demo', host = 'web1', entry, sudo = true }) {
const grant = { method: 'host-enroll', host, address: entry.address, port: entry.port, user: entry.user, sudo, fingerprints: [] };
const decision = grantDecision(store, { business, grant });
writeGrantEvent(store, AT, business, { operation: 'grant.host-enroll', decision }, decision);
const installerFingerprint = 'SHA256:' + 'A'.repeat(43);
writeGrantEvent(store, AT, business, {
action: 'grant.host.enrolled', grantDecision: decision, host, installerFingerprint, user: entry.user, sudo,
registrySha256: registrySha256({ ...entry, sudo }),
}, decision);
store.run(
'INSERT INTO hosts(business,host,address,port,user,sudo,host_keys,installer_fingerprint,state,decision,checked_at) VALUES(?,?,?,?,?,?,?,?,?,?,?)',
business, host, entry.address, entry.port, entry.user, sudo ? 1 : 0, JSON.stringify(entry.hostKeys), installerFingerprint, 'ready', decision, AT,
);
return decision;
}
// One agent's grant record on a host. By default its decision is a `host` grant for the same agent and
// host, resolved yes by the human via cli and spent as grant.host. Each option breaks one part.
export function hostGrantRecord(store, { business = 'demo', agent, host = 'web1', commands, rotateBy = '2099-01-01', grant = {}, close = HUMAN_YES, spend = 'grant.host', action }) {
const decision = grantDecision(store, {
business, close, action,
grant: grant === null ? undefined : { method: 'host', agent, host, commands, rotateBy, ...grant },
});
if (spend) writeGrantEvent(store, AT, business, { operation: spend, decision }, decision);
store.run(
'INSERT INTO host_grants(business,agent,host,commands,rotate_by,tag,decision) VALUES(?,?,?,?,?,?,?)',
business, agent, host, JSON.stringify(commands), rotateBy, grantTag(business, agent, host), decision,
);
return decision;
}
+115 -7
View File
@@ -12,8 +12,10 @@ import { fileURLToPath } from 'node:url';
import { Credentials, parseHostKey } from '../src/credentials.mjs';
import { addKey, hostExecutor, runHost, sshArgs, startTime, sweepRunDir, quote, remoteCommand } from '../src/host-run.mjs';
import { fixture } from './spec-fixtures.mjs';
import { HUMAN_YES, hostGrantRecord, hostRecord } from './host-fixtures.mjs';
import { Broker, ORGANIZATION_KEYS } from '../src/broker.mjs';
import { Store } from '../src/store.mjs';
import { hostGrant } from '../src/host-grant.mjs';
import { startBroker } from '../src/runtime.mjs';
import { businessDoc, rolesCopy, systemFor, writeJson } from '../../business/tests/helpers.mjs';
@@ -108,13 +110,12 @@ async function hostFixture(t, { commands = GRANT, reviewerHosts, wrongHostKey =
const executor = hostExecutor({ credentials, root });
const entry = { address: '127.0.0.1', port, user: USER, hostKeys: [pubLine(join(dir, wrongHostKey ? 'otherhost' : 'hostkey'))] };
const authority = { withinRole: ['message.send', 'record.write'], crossRole: [] };
const f = await fixture(t, {
roles: {
coder: { definition: 'coder', authority, hosts: { web1: { commands }, ...(coderWeb2 ? { web2: { commands } } : {}) } },
reviewer: { definition: 'reviewer', authority: { withinRole: ['message.send'], crossRole: [] }, ...(reviewerHosts ? { hosts: reviewerHosts } : {}) },
},
hosts: { web1: entry, web2: { ...entry } },
});
const f = await fixture(t, { roles: { coder: { definition: 'coder', authority } } });
// Row 234 (#1740, r6 X4): host.run reads store records, so the hosts and grants are fixtures there.
for (const host of ['web1', 'web2']) hostRecord(f.store, { host, entry });
hostGrantRecord(f.store, { agent: 'coder', host: 'web1', commands });
if (coderWeb2) hostGrantRecord(f.store, { agent: 'coder', host: 'web2', commands });
for (const [host, g] of Object.entries(reviewerHosts ?? {})) hostGrantRecord(f.store, { agent: 'reviewer', host, commands: g.commands });
const approve = async (argv, stdin, host = 'web1') => {
const d = await f.raise({ action: 'host.sudo', target: host, argv, ...(stdin !== undefined ? { stdin } : {}) });
f.call(f.human, 'decision.resolve', { id: d.id, choice: 'yes' });
@@ -291,6 +292,26 @@ test('T14: a host key not in the registry refuses and nothing runs', async (t) =
assert.deepEqual(f.argvSeen(), []);
});
// Row 234 (#1740, r6 G18, duckworth-0081 B2): hosts.check's `list` goes through runHost with the record's
// pinned keys. A stub `list` on the scratch sshd's PATH records each run.
test('G18: hosts.check with a wrong pin refuses host-key-mismatch and runs nothing on the host', async (t) => {
const f = await hostFixture(t);
writeFileSync(join(f.dir, 'fake', 'list'), `#!/bin/sh\necho "$@" >> '${f.dir}/list.log'\n`, { mode: 0o755 });
const configDir = join(f.dir, 'cfg');
const keyDir = join(configDir, 'secrets', 'hosts', 'demo', 'web1');
mkdirSync(keyDir, { recursive: true, mode: 0o700 });
for (const d of [join(configDir, 'secrets'), join(configDir, 'secrets', 'hosts'), join(configDir, 'secrets', 'hosts', 'demo'), keyDir]) chmodSync(d, 0o700);
writeFileSync(join(keyDir, 'installer'), readFileSync(f.keyFile), { mode: 0o600 });
const hg = hostGrant({ configDir, runRoot: f.root });
const record = { address: '127.0.0.1', port: f.port, user: 'jarvis', hostKeys: [pubLine(join(f.dir, 'otherhost'))] };
await refusesWith(hg.check('demo', 'web1', record), 'host-key-mismatch');
assert.equal(existsSync(join(f.dir, 'list.log')), false);
// Control: with the right pin the same command reaches the stub.
const out = await runHost({ root: f.root, hostId: 'web1', entry: f.entry, key: f.key, timeout: 10, command: 'list jarvis' });
assert.equal(out.exit, 0);
assert.equal(readFileSync(join(f.dir, 'list.log'), 'utf8'), 'jarvis\n');
});
test('N3: a remote exit 255 that prints the pin refusal text is the command result, and an ssh failure keeps its message', async (t) => {
const f = await hostFixture(t, { commands: [['fail255']] });
const out = await f.call(['fail255']);
@@ -532,3 +553,90 @@ test('T27: the bus start sweeps <configDir>/secrets/run before it listens, and a
// A bus that starts anyway is closed, so a failure here ends the file instead of holding it open.
await assert.rejects(async () => { const again = await start(); await again.close(); }, /credential-file/);
});
// Row 234 (#1740, duckworth-0079 M3, r6 X4 first bullet): host.run takes a grant record only behind the
// Approve record check. Each case breaks one part of it. The refusals come before the executor, so none
// needs sshd. The control passes the check and stops at host-not-configured.
test('M3: host.run refuses host-not-granted unless the grant record has an approved, spent host grant', async (t) => {
const f = await fixture(t, { roles: { coder: { definition: 'coder', authority: { withinRole: ['message.send'], crossRole: [] } } } });
const entry = { address: '10.0.0.5', port: 22, user: 'admin', hostKeys: ['ssh-ed25519 AAAA'] };
for (const host of ['web1', 'web2']) hostRecord(f.store, { host, entry });
const run = (cap = f.coder, host = 'web1') => f.broker.requestHost(cap, { verb: 'host.run', args: { host, argv: ['true'] } }, null);
const commands = [['true']];
const cases = {
'no record': null,
unresolved: { close: null },
'resolved no': { close: { ...HUMAN_YES, choice: 'no' } },
withdrawn: { close: { op: 'withdrawn', by: 'jason', via: 'cli' } },
'an arbiter': { close: { ...HUMAN_YES, by: 'reviewer', via: 'broker' } },
'the tester': { close: { ...HUMAN_YES, by: 'tester' } },
'an agent': { close: { ...HUMAN_YES, by: 'coder', via: 'broker' } },
'not via cli': { close: { ...HUMAN_YES, via: 'web' } },
'consumed as grant.host-enroll': { spend: 'grant.host-enroll' },
unspent: { spend: null },
'not a credential.mint': { action: 'deploy' },
'no grant in the raise': { grant: null },
'a host-enroll grant': { grant: { method: 'host-enroll' } },
'a grant for another agent': { grant: { agent: 'reviewer' } },
'a grant for another host': { grant: { host: 'web2' } },
};
for (const [name, opts] of Object.entries(cases)) {
f.store.run('DELETE FROM host_grants');
if (opts) hostGrantRecord(f.store, { agent: 'coder', host: 'web1', commands, ...opts });
await refusesWith(run(), 'host-not-granted').catch((e) => assert.fail(`${name}: ${e.message}`));
}
f.store.run('DELETE FROM host_grants');
hostGrantRecord(f.store, { agent: 'coder', host: 'web1', commands });
await refusesWith(run(), 'host-not-configured');
// The record is per agent: the reviewer holds none on web1, and web9 has no host record.
await refusesWith(run(f.reviewer), 'host-not-granted');
await refusesWith(run(f.coder, 'web9'), 'host-unknown');
// The host.sudo raise takes the same check (N1).
f.store.run('DELETE FROM host_grants');
hostGrantRecord(f.store, { agent: 'coder', host: 'web1', commands, spend: null });
assert.throws(() => f.raise({ action: 'host.sudo', target: 'web1', argv: ['true'] }), /host-not-granted/);
});
// Row 234 (#1740, M3, r6 X4 third bullet): at start each host_grants record gives Credentials a store
// reference to host.<id>, so a restart finds the key the grant executor wrote.
test('M3: a host_grants record gives Credentials a store reference at start, and a restart reads its key', async (t) => {
const root = mkdtempSync(join(tmpdir(), 'h234-'));
t.after(() => rmSync(root, { recursive: true, force: true }));
const rolesDir = rolesCopy(root);
mkdirSync(join(root, 'data'), { mode: 0o700 });
mkdirSync(join(root, 'secrets', 'agents'), { recursive: true, mode: 0o700 });
chmodSync(join(root, 'secrets'), 0o700);
const demo = {
id: 'demo', human: 'jason', arbiters: { technical: 'coder', delivery: 'coder' },
roles: { coder: { authority: { withinRole: ['message.send'], crossRole: [] }, credentials: {} } },
};
const start = () => startBroker({ dataRoot: join(root, 'data'), businesses: { demo }, organization: { dir: root, rolesDir, system: systemFor(root) } });
const hostKeys = (bus) => bus.credentials.status().filter((c) => c.service.startsWith('host.'));
let bus = await start();
assert.deepEqual(hostKeys(bus), []);
await bus.close();
const store = new Store(join(root, 'data'));
hostRecord(store, { host: 'web1', entry: { address: '10.0.0.5', port: 22, user: 'admin', hostKeys: ['ssh-ed25519 AAAA'] } });
hostGrantRecord(store, { agent: 'coder', host: 'web1', commands: [['true']], rotateBy: '2099-02-03' });
store.close();
bus = await start();
try {
assert.deepEqual(hostKeys(bus).map(({ instance, service, date, state }) => ({ instance, service, date, state })), [
{ instance: 'demo/coder', service: 'host.web1', date: '2099-02-03', state: 'not-issued' },
]);
} finally {
await bus.close();
}
const dir = join(root, 'secrets', 'agents', 'demo', 'coder');
mkdirSync(dir, { recursive: true, mode: 0o700 });
chmodSync(join(root, 'secrets', 'agents', 'demo'), 0o700);
keygen(join(dir, 'host.web1'));
bus = await start();
try {
const [key] = hostKeys(bus);
assert.equal(key.publicKey, pubLine(join(dir, 'host.web1')));
assert.notEqual(key.state, 'not-issued');
} finally {
await bus.close();
}
});
+358
View File
@@ -0,0 +1,358 @@
// Row 234 (#1740, #436a row a), design r6 E3: the four host files in tools/host, as G16 and G25.
// The writer runs as the test user through a harness that loads it by path and passes its own dirs,
// owner uid and user, group and sudo lookups, so nothing runs as root and nothing under /etc is read.
// mosaic-keys runs from a copy whose sudo and logger paths point at stubs in a scratch dir.
import test from 'node:test';
import assert from 'node:assert/strict';
import { execFileSync, spawnSync } from 'node:child_process';
import { createHash, randomBytes } from 'node:crypto';
import {
chmodSync, existsSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, statSync, symlinkSync,
writeFileSync,
} from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { fileURLToPath } from 'node:url';
const REPO = fileURLToPath(new URL('../../../', import.meta.url));
const TOOLS = join(REPO, 'tools', 'host');
const WRITER = join(TOOLS, 'mosaic-keys-write');
const REFUSED_GROUPS = ['sudo', 'admin', 'wheel', 'docker', 'lxd', 'disk', 'libvirt', 'incus-admin'];
const tag = (n) => 'mosaic-' + createHash('sha256').update(String(n)).digest('hex').slice(0, 32);
const u32 = (n) => {
const b = Buffer.alloc(4);
b.writeUInt32BE(n);
return b;
};
const keyBlob = () => Buffer.concat([u32(11), Buffer.from('ssh-ed25519'), u32(32), randomBytes(32)]).toString('base64');
const fingerprint = (b64) => 'SHA256:' + createHash('sha256').update(Buffer.from(b64, 'base64')).digest('base64').replace(/=+$/, '');
const notAllowed = (u) => `User ${u} is not allowed to run sudo on testhost.\n`;
const HARNESS = `import importlib.machinery, importlib.util, json, os, sys
sys.dont_write_bytecode = True
cfg = json.loads(sys.argv[1])
loader = importlib.machinery.SourceFileLoader('mosaic_keys_write', cfg['writer'])
m = importlib.util.module_from_spec(importlib.util.spec_from_loader('mosaic_keys_write', loader))
loader.exec_module(m)
users = cfg['users']
def log(msg):
with open(cfg['log'], 'a') as f:
f.write(msg + '\\n')
def sudo(u):
with open(cfg['sudoLog'], 'a') as f:
f.write(u + '\\n')
return users[u]['sudo'] if 'sudo' in users[u] else 'User %s is not allowed to run sudo on testhost.\\n' % u
host = m.Host(key_dir=cfg['keyDir'], conf_dir=cfg['confDir'], owner=os.getuid(),
passwd=lambda u: users[u]['uid'] if u in users else None,
groups=lambda u: users[u].get('groups', []), sudo=sudo, hostname='testhost', log=log)
sys.exit(m.main(sys.argv[2:], host))
`;
// The users the harness's lookups know. users lists all of them but dave and eve. bob and dave are
// privileged. carol has a sudo line. Each gX is in one refused group.
function userTable() {
const users = {
alice: { uid: 1500 },
bob: { uid: 1501, groups: ['sudo', 'docker'], sudo: 'User bob may run the following commands on testhost:\n (ALL) ALL\n' },
carol: { uid: 1502, sudo: 'User carol may run the following commands on testhost:\n (root) /usr/bin/true\n' },
dave: { uid: 1503 },
eve: { uid: 1504 },
frank: { uid: 1505, groups: ['users', 'adm'] },
'../x': { uid: 1506 },
root: { uid: 0 },
toor: { uid: 0 },
daemon2: { uid: 999 },
localadmin: { uid: 1000 },
'mosaic-installer': { uid: 1001 },
};
for (const [i, g] of REFUSED_GROUPS.entries()) users['g' + g] = { uid: 1600 + i, groups: ['users', g] };
return users;
}
function hostDirs() {
const root = mkdtempSync(join(tmpdir(), 'hk234-'));
const keyDir = join(root, 'keys');
const confDir = join(root, 'conf');
mkdirSync(keyDir, { mode: 0o755 });
mkdirSync(confDir, { mode: 0o755 });
chmodSync(keyDir, 0o755);
const users = userTable();
const conf = (name, text) => {
writeFileSync(join(confDir, name), text, { mode: 0o644 });
chmodSync(join(confDir, name), 0o644);
};
conf('users', '# users the writer may touch\n' + Object.keys(users).filter((u) => !['dave', 'eve'].includes(u)).join('\n') + '\n');
conf('privileged', 'bob\ndave\n');
conf('from', '10.1.2.3/32\n');
const harness = join(root, 'harness.py');
writeFileSync(harness, HARNESS);
const cfg = { writer: WRITER, keyDir, confDir, users, log: join(root, 'syslog'), sudoLog: join(root, 'sudo.log') };
const run = (...args) => {
const r = spawnSync('python3', ['-B', '-I', harness, JSON.stringify(cfg), ...args], { encoding: 'utf8' });
return { status: r.status, stdout: r.stdout, stderr: r.stderr };
};
const read = (file) => readFileSync(join(keyDir, file), 'utf8');
const lastLog = () => readFileSync(cfg.log, 'utf8').trim().split('\n').at(-1);
const sudoCalls = () => (existsSync(cfg.sudoLog) ? readFileSync(cfg.sudoLog, 'utf8').trim().split('\n') : []);
return { root, keyDir, confDir, conf, run, read, lastLog, sudoCalls, done: () => rmSync(root, { recursive: true, force: true }) };
}
const refused = (r, code) => {
assert.equal(r.status, 1, r.stderr);
assert.equal(r.stderr, `mosaic-keys-write: refused ${code}\n`);
};
test('G16: the writer refuses each user, tag and key the design names, before any key file', () => {
const h = hostDirs();
try {
const key = keyBlob();
const t = tag(1);
const add = (user) => h.run('add', user, 'ssh-ed25519', key, t);
// The user pattern stops `../x` before any path, though the lookups and users know it.
for (const user of ['../x', 'Alice', 'a'.repeat(33), '1alice', 'al ice', '']) {
refused(add(user), 'user-pattern');
refused(h.run('list', user), 'user-pattern');
}
// users is always required: privileged alone allows nothing.
refused(add('dave'), 'user-not-listed');
refused(add('eve'), 'user-not-listed');
for (const user of ['root', 'localadmin', 'mosaic-installer']) refused(add(user), 'user-not-listed');
refused(add('toor'), 'user-system');
refused(add('daemon2'), 'user-system');
refused(add('ghost'), 'user-not-listed');
for (const g of REFUSED_GROUPS) {
refused(add('g' + g), 'user-group');
refused(h.run('remove', 'g' + g, t), 'user-group');
}
refused(add('carol'), 'user-sudo');
refused(h.run('list', 'carol'), 'user-sudo');
assert.equal(h.lastLog(), 'result=refused user-sudo user=carol');
for (const bad of ['mosaic-' + 'a'.repeat(31), 'mosaic-' + 'A'.repeat(32), 'mosaic-installer-' + 'a'.repeat(32), 'mosaic-' + 'g'.repeat(32)]) {
refused(h.run('add', 'alice', 'ssh-ed25519', key, bad), 'tag-pattern');
refused(h.run('remove', 'alice', bad), 'tag-pattern');
}
refused(h.run('add', 'alice', 'ssh-dss', key, t), 'key-format');
refused(h.run('add', 'alice', 'ssh-ed25519', key.slice(0, -2) + '!!', t), 'key-format');
refused(h.run('add', 'alice', 'ssh-ed25519', 'A', t), 'key-format');
refused(h.run('add', 'alice', 'ssh-ed25519', key + ' x', t), 'key-format');
for (const args of [[], ['list'], ['list', 'alice', 'bob'], ['add', 'alice', 'ssh-ed25519', key], ['remove', 'alice'], ['rm', 'alice', t]])
refused(h.run(...args), 'request');
// No refusal reached the key dir, so not even the lock exists.
assert.deepEqual(readdirSync(h.keyDir), []);
// Only users past the groups check reach sudo -l, and the privileged never do.
assert.deepEqual([...new Set(h.sudoCalls())], ['carol']);
} finally {
h.done();
}
});
test('G16: from= comes from the host file, a privileged user skips the group and sudo checks, and the line is exact', () => {
const h = hostDirs();
try {
const k1 = keyBlob();
const k2 = keyBlob();
assert.equal(h.run('add', 'alice', 'ssh-ed25519', k1, tag(1)).status, 0);
assert.equal(h.lastLog(), `result=added user=alice tag=${tag(1)} key=${fingerprint(k1)}`);
h.conf('from', '# the stack host\n10.9.9.9,192.168.1.0/24\n');
assert.equal(h.run('add', 'alice', 'ssh-ed25519', k2, tag(2)).status, 0);
assert.equal(
h.read('alice'),
`from="10.1.2.3/32",restrict ssh-ed25519 ${k1} ${tag(1)}\nfrom="10.9.9.9,192.168.1.0/24",restrict ssh-ed25519 ${k2} ${tag(2)}\n`,
);
// A tag already there changes nothing and exits 0.
assert.equal(h.run('add', 'alice', 'ssh-ed25519', k2, tag(1)).status, 0);
assert.equal(h.lastLog(), `result=present user=alice tag=${tag(1)} key=${fingerprint(k2)}`);
assert.equal(h.read('alice').split('\n').length, 3);
// bob is in sudo and docker with a sudo line, and privileged.
assert.equal(h.run('add', 'bob', 'ssh-ed25519', k1, tag(3)).status, 0);
// frank's groups are none of the refused ones.
assert.equal(h.run('add', 'frank', 'ssh-ed25519', k1, tag(4)).status, 0);
assert.deepEqual(h.sudoCalls(), ['alice', 'alice', 'alice', 'frank']);
// A from value that could close the quote, two values, or none, refuses and writes nothing.
for (const text of ['10.0.0.1",command="/bin/sh\n', '10.0.0.1\n10.0.0.2\n', '# none\n', '10.0.0.1 10.0.0.2\n']) {
h.conf('from', text);
refused(h.run('add', 'frank', 'ssh-ed25519', k2, tag(5)), 'config-from');
}
rmSync(join(h.confDir, 'from'));
refused(h.run('add', 'frank', 'ssh-ed25519', k2, tag(5)), 'config-from');
assert.ok(!h.read('frank').includes(tag(5)));
// list and remove don't need from.
assert.equal(h.run('list', 'frank').stdout, `from="10.9.9.9,192.168.1.0/24",restrict ssh-ed25519 ${k1} ${tag(4)}\n`);
for (const name of ['users', 'privileged']) {
chmodSync(join(h.confDir, name), 0o664);
refused(h.run('list', 'alice'), 'not-root-owned');
chmodSync(join(h.confDir, name), 0o644);
}
rmSync(join(h.confDir, 'users'));
refused(h.run('list', 'alice'), 'config-users');
} finally {
h.done();
}
});
test('G16: list shows tagged lines only, and add and remove never touch a line outside the tag pattern', () => {
const h = hostDirs();
try {
const k = keyBlob();
const hand = [
'# written by hand',
`ssh-ed25519 ${k} mosaic-installer-${'a'.repeat(32)}`,
'',
`ssh-ed25519 ${k} ${tag(9).toUpperCase()}`,
`ssh-ed25519 ${k} ${tag(1)} trailing`,
];
const tagged = `from="10.1.2.3/32",restrict ssh-ed25519 ${k} ${tag(7)}`;
writeFileSync(join(h.keyDir, 'alice'), [hand[0], tagged, ...hand.slice(1)].join('\n') + '\n', { mode: 0o644 });
assert.equal(h.run('list', 'alice').stdout, tagged + '\n');
// The last field is the tag, so `... mosaic-<1> trailing` isn't tag 1.
assert.equal(h.run('remove', 'alice', tag(1)).status, 0);
assert.equal(h.lastLog(), `result=absent user=alice tag=${tag(1)}`);
assert.equal(h.run('add', 'alice', 'ssh-ed25519', k, tag(8)).status, 0);
assert.equal(h.run('remove', 'alice', tag(7)).status, 0);
assert.equal(h.lastLog(), `result=removed user=alice tag=${tag(7)}`);
assert.equal(h.read('alice'), [...hand, `from="10.1.2.3/32",restrict ssh-ed25519 ${k} ${tag(8)}`].join('\n') + '\n');
assert.equal(h.run('remove', 'alice', tag(8)).status, 0);
assert.equal(h.read('alice'), hand.join('\n') + '\n');
// A missing file lists nothing and a remove there changes nothing.
assert.equal(h.run('list', 'frank').stdout, '');
assert.equal(h.run('remove', 'frank', tag(1)).status, 0);
assert.ok(!existsSync(join(h.keyDir, 'frank')));
} finally {
h.done();
}
});
test('G25: the writer refuses a symlinked dir or user file, a group-writable dir and an odd tmp, and writes 0644', () => {
const h = hostDirs();
try {
const k = keyBlob();
const add = (n, user = 'alice') => h.run('add', user, 'ssh-ed25519', k, tag(n));
// A first add creates the file, regular, 0644, owned by the owner uid, with no tmp left.
assert.equal(add(1).status, 0);
const st = statSync(join(h.keyDir, 'alice'));
assert.equal(st.mode & 0o777, 0o644);
assert.equal(st.uid, process.getuid());
assert.deepEqual(readdirSync(h.keyDir).sort(), ['.lock', 'alice']);
// A leftover regular tmp is unlinked and the write goes ahead.
writeFileSync(join(h.keyDir, '.alice.tmp'), 'half a write');
assert.equal(add(2).status, 0);
assert.ok(h.read('alice').includes(tag(2)));
assert.ok(!existsSync(join(h.keyDir, '.alice.tmp')));
// A tmp that is a symlink, a dir or a fifo refuses, and nothing is written.
const decoy = join(h.root, 'decoy');
writeFileSync(decoy, 'decoy\n');
const before = h.read('alice');
for (const make of [
(p) => symlinkSync(decoy, p),
(p) => mkdirSync(p),
(p) => execFileSync('mkfifo', [p]),
]) {
make(join(h.keyDir, '.alice.tmp'));
refused(add(3), 'tmp-file');
assert.equal(h.read('alice'), before);
assert.equal(readFileSync(decoy, 'utf8'), 'decoy\n');
rmSync(join(h.keyDir, '.alice.tmp'), { recursive: true });
}
// A symlinked user file refuses, though its target is a regular file the owner owns.
symlinkSync(decoy, join(h.keyDir, 'frank'));
refused(add(4, 'frank'), 'user-file');
refused(h.run('list', 'frank'), 'user-file');
assert.equal(readFileSync(decoy, 'utf8'), 'decoy\n');
assert.ok(lstatSync(join(h.keyDir, 'frank')).isSymbolicLink());
// A user "file" that is a dir refuses.
mkdirSync(join(h.keyDir, 'bob'));
refused(add(5, 'bob'), 'not-root-owned');
// A group-writable dir refuses.
chmodSync(h.keyDir, 0o775);
refused(add(6), 'not-root-owned');
chmodSync(h.keyDir, 0o755);
// A symlinked key dir refuses, and its target is untouched.
const real = join(h.root, 'real-keys');
mkdirSync(real, { mode: 0o755 });
rmSync(h.keyDir, { recursive: true });
symlinkSync(real, h.keyDir);
refused(add(7), 'key-dir');
assert.deepEqual(readdirSync(real), []);
} finally {
h.done();
}
});
// mosaic-keys from a copy with its sudo and logger paths on stubs. The stub sudo records its argv,
// prints a line and exits 3, so the test sees the exit and output pass through.
function forcedCommand() {
const root = mkdtempSync(join(tmpdir(), 'hk234-'));
const sudo = join(root, 'sudo');
const logger = join(root, 'logger');
writeFileSync(sudo, `#!/bin/bash\nprintf '%s\\n' "$@" > ${root}/sudo.argv\necho from-writer\nexit 3\n`, { mode: 0o755 });
writeFileSync(logger, `#!/bin/bash\nprintf '%s\\n' "$*" >> ${root}/logger.log\n`, { mode: 0o755 });
const source = readFileSync(join(TOOLS, 'mosaic-keys'), 'utf8');
assert.equal(source.split('/usr/bin/sudo').length, 2);
assert.equal(source.split('/usr/bin/logger').length, 2);
const copy = join(root, 'mosaic-keys');
writeFileSync(copy, source.replace('/usr/bin/sudo', sudo).replace('/usr/bin/logger', logger), { mode: 0o755 });
const run = (request) => {
rmSync(join(root, 'sudo.argv'), { force: true });
const env = { PATH: '/usr/bin:/bin', SSH_CONNECTION: '10.1.2.3 50000 10.1.2.9 22' };
if (request !== undefined) env.SSH_ORIGINAL_COMMAND = request;
const r = spawnSync(copy, [], { encoding: 'utf8', env });
const argv = existsSync(join(root, 'sudo.argv')) ? readFileSync(join(root, 'sudo.argv'), 'utf8').trim().split('\n') : null;
return { status: r.status, stdout: r.stdout, stderr: r.stderr, argv };
};
const logged = () => readFileSync(join(root, 'logger.log'), 'utf8').trim().split('\n');
return { run, logged, done: () => rmSync(root, { recursive: true, force: true }) };
}
test('G16: mosaic-keys passes the three requests to the writer through sudo and refuses anything else without it', () => {
const f = forcedCommand();
try {
const k = keyBlob();
const t = tag(1);
const writer = ['-n', '/usr/local/sbin/mosaic-keys-write'];
for (const words of [['list', 'alice'], ['add', 'alice', 'ssh-ed25519', k, t], ['add', 'alice', 'ecdsa-sha2-nistp256', 'AAAA', t], ['remove', 'alice', t]]) {
const r = f.run(words.join(' '));
assert.deepEqual(r.argv, [...writer, ...words]);
assert.equal(r.status, 3);
assert.equal(r.stdout, 'from-writer\n');
}
const bad = [
undefined, '', 'list', 'list ../x', 'list Alice', 'list alice;id', 'list alice ', ' list alice', 'list alice', 'list\talice',
'list alice\nlist bob', 'list $(id)', 'list alice`id`', 'list *', 'rm alice', 'list ' + 'a'.repeat(33),
`add alice ssh-dss ${k} ${t}`, `add alice ssh-ed25519 ${k}! ${t}`, `add alice ssh-ed25519 ${k}`, `add alice ssh-ed25519 ${k} ${t} x`,
`remove alice mosaic-${'a'.repeat(31)}`, `remove alice ${t.toUpperCase()}`, `remove alice ${t}\n`, `remove alice ${t}; rm -rf /`,
];
for (const request of bad) {
const r = f.run(request);
assert.equal(r.argv, null, JSON.stringify(request));
assert.equal(r.status, 1);
assert.equal(r.stderr, 'mosaic-keys: refused\n');
}
// Two bad requests hold a newline, so count the logger calls, not the lines.
const logged = f.logged().filter((l) => l.startsWith('-t mosaic-keys -- '));
assert.equal(logged.length, 4 + bad.length);
assert.equal(logged[0], '-t mosaic-keys -- connection=10.1.2.3 50000 10.1.2.9 22 request=list alice');
} finally {
f.done();
}
});
test('E3: the sudoers line and the sshd drop-in are the design lines and parse', () => {
const lines = (file) => readFileSync(join(TOOLS, file), 'utf8').split('\n').filter((l) => l && !l.startsWith('#'));
assert.deepEqual(lines('mosaic-keys.sudoers'), ['mosaic-installer ALL=(root) NOPASSWD: /usr/local/sbin/mosaic-keys-write']);
assert.deepEqual(lines('mosaic-keys.sshd.conf'), ['AuthorizedKeysFile .ssh/authorized_keys .ssh/authorized_keys2 /etc/ssh/mosaic-keys/%u']);
assert.ok(existsSync('/usr/bin/visudo'), 'visudo on the test host');
execFileSync('/usr/bin/visudo', ['-c', '-q', '-f', join(TOOLS, 'mosaic-keys.sudoers')]);
const dir = mkdtempSync(join(tmpdir(), 'hk234-'));
try {
execFileSync('ssh-keygen', ['-q', '-t', 'ed25519', '-N', '', '-f', join(dir, 'hk')]);
writeFileSync(join(dir, 'cfg'), `HostKey ${join(dir, 'hk')}\nInclude ${join(TOOLS, 'mosaic-keys.sshd.conf')}\n`);
const out = execFileSync('/usr/bin/sshd', ['-T', '-f', join(dir, 'cfg'), '-C', 'user=alice,host=x,addr=127.0.0.1'], { encoding: 'utf8' });
// sshd -T prints this key as AuthorizedKeysFile, unlike most.
const line = out.split('\n').filter((l) => /^authorizedkeysfile /i.test(l));
assert.deepEqual(line.map((l) => l.split(' ').slice(1).join(' ')), ['.ssh/authorized_keys .ssh/authorized_keys2 /etc/ssh/mosaic-keys/%u']);
} finally {
rmSync(dir, { recursive: true, force: true });
}
for (const file of ['mosaic-keys', 'mosaic-keys-write']) assert.equal(statSync(join(TOOLS, file)).mode & 0o111, 0o111, file);
assert.equal(readFileSync(WRITER, 'utf8').split('\n')[0], '#!/usr/bin/python3 -I');
});
+22
View File
@@ -306,6 +306,28 @@ test('a ticket outlives the run that asked, and recordLand checks the task again
assert.throws(() => w.b.landRefused(ticket, { code: 'gate-failed' }), /land-unknown/);
});
// Row 241 (#1747): one decision lands once. Two tickets on one decision can't both reach
// recordLand through the git socket (land-busy, git-socket.mjs), so this pins the broker's own
// guard: recordLand runs #landFacts again, and a decision spent by the first refuses the second.
test('two tickets on one decision: the first recordLand lands, the second refuses decision-consumed and records nothing', (t) => {
const w = setup(t);
const { d } = w.ready();
const first = w.check({ task_ref: TASK, decision: d.id }).ticket;
const second = w.check({ task_ref: TASK, decision: d.id }).ticket;
assert.notEqual(first, second);
assert.deepEqual(w.b.recordLand(first, w.land), { task_ref: TASK, commit: B });
assert.throws(() => w.b.recordLand(second, { ...w.land, commit: C, previous: B }), /decision-consumed/);
assert.equal(w.merges().length, 1);
assert.equal(w.merges()[0].body.commit, B);
assert.equal(w.results().length, 1);
assert.deepEqual(w.events('action.refused'), []);
// The git socket then ends the second ticket with the broker's code, as it does any record refusal.
assert.equal(w.b.landRefused(second, { code: 'decision-consumed', notify: true }), 'decision-consumed');
assert.deepEqual(w.events('action.refused').map((e) => e.body), [{ code: 'decision-consumed' }]);
assert.equal(w.merges().length, 1);
assert.throws(() => w.b.recordLand(second, w.land), /land-unknown/);
});
test('landRefused files the code on the task, tells the role only with notify, and ends the ticket', (t) => {
const w = setup(t);
const { d } = w.ready();
+1 -1
View File
@@ -55,7 +55,7 @@ test('3f copy migrates to 3g preserving every old column/row, with an empty inst
assert.throws(() => new Store(root), /schema-mismatch/);
const store = new Store(root, { migrateSchema: true });
try {
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3i');
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3j');
assert.deepEqual({ ...store.get('SELECT to_name FROM messages') }, { to_name: null });
assert.equal(store.get('SELECT count(*) AS n FROM instance_names').n, 0);
assert.deepEqual({ ...store.get('SELECT spec_path,spec_commit FROM decisions') }, { spec_path: 'docs/spec.md', spec_commit: 'c'.repeat(40) });
+2 -1
View File
@@ -25,7 +25,8 @@ test('Broker keeps the sandbox client import closure: only the listed imports, n
// RUNNER_FILES and imports nothing. Row 155 (#1649) reads THREAD_DEFINITION and END_PROTECTED
// from it too. Row 199 (#1697) reads the D9c fork parse, forkProblem and forkUrl, from it.
// Row 210 (#1707) reads credentialKind and the D6 argv check, validArgv and commandAllowed, from it.
"import { ID_PATTERN, THREAD_DEFINITION, END_PROTECTED, forkProblem, forkUrl, credentialKind, validArgv, commandAllowed } from '../../business/src/vocabulary.mjs';",
// Row 234 (#1740) reads HOST_ID for the hosts verbs.
"import { ID_PATTERN, THREAD_DEFINITION, END_PROTECTED, forkProblem, forkUrl, credentialKind, validArgv, commandAllowed, HOST_ID } from '../../business/src/vocabulary.mjs';",
]);
assert.doesNotMatch(source, /\bimport\s*\(/);
assert.doesNotMatch(readFileSync(new URL('../src/landing.mjs', import.meta.url), 'utf8'), /\bimport\b/);
+3 -2
View File
@@ -519,7 +519,8 @@ test('v3b prototype refusals, views and append-only mutations', (t) => {
// Row 121 (#1615) adds 5 tables and 24 triggers for projects and threads; threads.test.mjs tests them.
// Row 141 (#1635) adds instance_names and its 4 triggers; names.test.mjs tests them.
// Row 124 adds usage_records and its three append-only triggers.
assert.equal(count('table') - 1, 16);
assert.equal(count('trigger'), 70);
// Row 234 (#1740) adds hosts, host_grants and the events_grant_actor guard; grant-migration.test.mjs tests them.
assert.equal(count('table') - 1, 18);
assert.equal(count('trigger'), 71);
assert.equal(count('view'), 5);
});
+2 -2
View File
@@ -7,7 +7,7 @@ import { Broker } from '../src/broker.mjs';
import { initProject, readSpec, writeRecord, listProjectFiles, readProjectFile, writeHumanFile, ProjectError } from '../../project/src/index.mjs';
export const git = (repo, ...args) => execFileSync('git', ['-c', 'core.hooksPath=/dev/null', '-C', repo, ...args], { encoding: 'utf8' }).trim();
const options = [{ key: 'yes', text: 'Yes' }, { key: 'no', text: 'No' }];
export async function fixture(t, { adapter = true, roles = {}, arbiters = { technical: 'reviewer', delivery: 'reviewer' }, hosts } = {}) {
export async function fixture(t, { adapter = true, roles = {}, arbiters = { technical: 'reviewer', delivery: 'reviewer' } } = {}) {
const root = mkdtempSync(join(tmpdir(), 'bus-spec-'));
const store = new Store(root);
t.after(() => { store.close(); rmSync(root, { recursive: true, force: true }); });
@@ -17,7 +17,7 @@ export async function fixture(t, { adapter = true, roles = {}, arbiters = { tech
coder: { definition: 'coder', authority: authority(['message.send', 'record.write']) },
reviewer: { definition: 'reviewer', authority: authority(['message.send']) },
...roles,
}, ...(hosts ? { hosts } : {}) };
} };
const { repo, commit } = await initProject({ business, project: 'stack', dataRoot: root });
const broker = new Broker({ store, businesses: { demo: business }, project: adapter ? {
readSpec: context => readSpec({ ...context, dataRoot: root }),
+1 -1
View File
@@ -53,7 +53,7 @@ test('3e copy migrates through 3f to 3g preserving every old column/row and addi
assert.throws(() => new Store(root), /schema-mismatch/);
const store = new Store(root, { migrateSchema: true });
try {
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3i');
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3j');
assert.deepEqual({ ...store.get('SELECT spec_path,spec_commit FROM decisions') }, { spec_path: null, spec_commit: null });
assert.equal(store.get('PRAGMA quick_check').quick_check, 'ok');
assert.deepEqual(store.all('PRAGMA foreign_key_check'), []);
+3 -3
View File
@@ -49,7 +49,7 @@ async function database3d(t) {
const source = readFileSync(here('../src/store.mjs'), 'utf8');
const pinned = source
.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3d).digest('hex')}';`)
.replace("export const SCHEMA_VERSION = '3i';", "export const SCHEMA_VERSION = '3d';")
.replace("export const SCHEMA_VERSION = '3j';", "export const SCHEMA_VERSION = '3d';")
.replace(/\}, \{\n {2}from: '3d',[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables');
assert.ok(!pinned.includes('3d-3e'), 'the 3d engine has no 3d→3e step');
writeFileSync(join(moduleRoot, 'src/store.mjs'), pinned);
@@ -101,7 +101,7 @@ test('REQ-THREAD-1 REQ-ORG-2: the 3d→3e migration adds the thread tables and l
assert.throws(() => new Store(root), /schema-mismatch/);
const store = new Store(root, { migrateSchema: true });
t.after(() => store.close());
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3i');
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3j');
for (const table of ['projects', 'threads', 'thread_claims', 'message_threads', 'thread_opens'])
assert.equal(store.get(`SELECT count(*) n FROM ${table}`).n, 0, table);
// Parked: the undelivered ones are counted per business and role, and none is changed or given a thread.
@@ -161,5 +161,5 @@ test('REQ-THREAD-1: the offline command prints the kept-message counts before th
const command = new URL('../src/migrate-schema.mjs', import.meta.url).pathname;
const migrated = spawnSync(process.execPath, [command, root], { encoding: 'utf8' });
assert.equal(migrated.status, 0, migrated.stderr);
assert.equal(migrated.stdout, [...REPORT, 'bus schema 3i ready', ''].join('\n'));
assert.equal(migrated.stdout, [...REPORT, 'bus schema 3j ready', ''].join('\n'));
});
+2 -2
View File
@@ -51,12 +51,12 @@ function snapshot(path, columns) {
return { columns, rows };
} finally { db.close(); }
}
test('REQ-LIMIT-4 3g copy migrates through 3h to 3i preserving every old column, row and sequence; new usage table is empty', t => {
test('REQ-LIMIT-4 3g copy migrates through 3h and 3i to 3j preserving every old column, row and sequence; new usage table is empty', t => {
const { root, path, copy } = source3g(t), before = snapshot(path), untouched = readFileSync(copy);
assert.throws(() => new Store(root), /schema-mismatch/);
const store = new Store(root, { migrateSchema: true });
try {
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3i');
assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3j');
assert.deepEqual({ ...store.get('SELECT to_name FROM messages') }, { to_name: null });
assert.equal(store.get('SELECT count(*) AS n FROM usage_records').n, 0);
assert.equal(store.get('SELECT name FROM instance_names').name, 'Builder');
+11 -15
View File
@@ -10,8 +10,7 @@ import { checkVars } from "./vars.mjs";
import { parseCredentialRef } from "./credentials.mjs";
import { loadRole } from "./role.mjs";
import { checkPlugins } from "../../plugins/src/kinds.mjs";
import { forkProblem, giteaName, parseGiteaUrl, HOST_SERVICE, HOST_PREFIX } from "./vocabulary.mjs";
import { checkHosts, checkHostGrants } from "./hosts.mjs";
import { forkProblem, giteaName, parseGiteaUrl, HOST_SERVICE } from "./vocabulary.mjs";
import {
requireObject, rejectUnknownKeys, requireId, requirePositiveInt, requireString, requireDistinctList, readJsonFile, deepFreeze,
} from "./util.mjs";
@@ -103,7 +102,11 @@ function checkTracker(tracker, file) {
return { sync, labels };
}
function checkRoles(roles, file, rolesDir, business, hosts) {
// Row 234 (#1740, r6 S1): hosts and host grants are bus store records made by a human-approved grant,
// so the business file can't name them. The key stays known so the refusal says where they went.
const HOSTS_MOVED = "lives in the bus store now: enroll a host with a host-enroll grant (mosaic hosts status)";
function checkRoles(roles, file, rolesDir, business) {
const where = `${file} roles`;
requireObject(roles, where);
if (Object.keys(roles).length === 0) refuse(`${where} must declare at least one role instance`);
@@ -121,14 +124,9 @@ function checkRoles(roles, file, rolesDir, business, hosts) {
definitions[entry.definition] = definition;
if (definition.roleVersion !== 2) refuse(`${at}: role definition ${entry.definition} is version 1 and can't back a business role instance`);
const services = definition.credentials.map((c) => c.service);
// Row 210 (#1707, design D3 steps 1 and 3): a host grant needs a definition that declares the
// host credential, and each granted host needs its own host.<id> key reference.
let hostGrants = null;
if (entry.hosts !== undefined) {
if (!services.includes(HOST_SERVICE)) refuse(`${at}.hosts needs a role definition that declares the host credential`);
hostGrants = checkHostGrants(entry.hosts, hosts, `${at}.hosts`);
}
const kinds = [...services.filter((s) => s !== HOST_SERVICE), ...Object.keys(hostGrants ?? {}).map((id) => HOST_PREFIX + id)];
if (entry.hosts !== undefined) refuse(`${at}.hosts ${HOSTS_MOVED}`);
// A host.<id> key reference comes from the agent's grant record at bus start (r6 X4), never from here.
const kinds = services.filter((s) => s !== HOST_SERVICE);
let tracker = null;
if (services.includes("vikunja")) {
if (entry.tracker === undefined) refuse(`${at} needs "tracker" with its Vikunja bot, because ${entry.definition} uses vikunja`);
@@ -151,7 +149,6 @@ function checkRoles(roles, file, rolesDir, business, hosts) {
// Row 199 (#1697, design D8, D9 step 2): identity and publish are optional, so today's files
// load unchanged. publish needs both an identity and a gitea credential; the project ids it
// names are checked against projects once those are read.
if (hostGrants) out[instance].hosts = hostGrants;
if (entry.identity !== undefined) out[instance].identity = checkIdentity(entry.identity, `${at}.identity`);
if (entry.publish !== undefined) {
// N10: one message for both, naming the grant.
@@ -207,8 +204,8 @@ export function validateBusinessDocument(document, file, { rolesDir }) {
}
const human = requireId(document.human, "business human");
const vars = checkVars(document.vars ?? {}, "business", `${file} vars`);
const hosts = checkHosts(document.hosts, `${file} hosts`);
const { roles, definitions } = checkRoles(document.roles, file, rolesDir, document.id, hosts);
if (document.hosts !== undefined) refuse(`${file} hosts ${HOSTS_MOVED}`);
const { roles, definitions } = checkRoles(document.roles, file, rolesDir, document.id);
requireObject(document.arbiters, `${file} arbiters`);
rejectUnknownKeys(document.arbiters, ["delivery", "technical"], `${file} arbiters`);
@@ -268,7 +265,6 @@ export function validateBusinessDocument(document, file, { rolesDir }) {
return deepFreeze({
businessVersion: 1, id: document.id, file, human, arbiters, projects, vars, tracker, roles, launch, plugins, definitions,
...(document.hosts === undefined ? {} : { hosts }),
});
}
+8 -49
View File
@@ -1,19 +1,19 @@
// Row 210 (#1707, design D3 and D6): the host registry in the business file, an agent's host
// grants, and the argv allowlist the broker matches each host.run against. Hand edits only: the
// chief's organization.set and the human's instances.set take neither key.
// Row 210 (#1707, design D3 and D6): the address, user and host key checks, and the argv allowlist the
// broker matches each host.run against. Row 234 (#1740, r6 S1) moved the host registry and the grants
// out of the business file into bus store records. The enrollment raise checks its fields with these,
// and row b's host grant raise checks its commands with checkCommands.
import { isIP } from "node:net";
import { refuse } from "./errors.mjs";
import { HOST_ID, ARGV_MAX } from "./vocabulary.mjs";
import { ARGV_MAX } from "./vocabulary.mjs";
// Row 210 r4 (#1707): the argv check lives in vocabulary.mjs, which the worker sandbox mounts
// and which imports nothing, so the broker can read it. Re-exported so the D6 checks keep one
// business home here.
export { ARGV_MAX, ARG_BYTES, ARGV_BYTES, validArgv, commandAllowed } from "./vocabulary.mjs";
import { requireObject, rejectUnknownKeys, isPlainObject } from "./util.mjs";
const DNS = /^[a-z0-9]([a-z0-9.-]{0,251}[a-z0-9])?$/;
const USER = /^[a-z_][a-z0-9_-]{0,31}$/;
const HOST_KEY = /^(ssh-ed25519|ecdsa-sha2-nistp256|ssh-rsa) [A-Za-z0-9+/]+={0,2}$/;
export const USER = /^[a-z_][a-z0-9_-]{0,31}$/;
export const HOST_KEY = /^(ssh-ed25519|ecdsa-sha2-nistp256|ssh-rsa) [A-Za-z0-9+/]+={0,2}$/;
// D6: a prefix starting with one of these, by basename, would turn the grant back into "any".
// The list can't be complete. Under PRD 1.7 the control is the human's decision on each call (D4).
@@ -49,7 +49,7 @@ export function ownMachine(address) {
return address === "localhost" || address.endsWith(".localhost");
}
function checkAddress(address, where) {
export function checkAddress(address, where) {
if (typeof address !== "string") refuse(`${where} must be an IP address or a DNS name`);
const version = isIP(address);
if (version === 6 && address.includes("%")) refuse(`${where} must not carry an IPv6 zone`);
@@ -63,32 +63,6 @@ function checkAddress(address, where) {
return address;
}
// The top-level "hosts" key: { <hostId>: { address, port, user, hostKeys } }.
export function checkHosts(hosts, where) {
if (hosts === undefined) return {};
requireObject(hosts, where);
const out = {};
for (const [id, entry] of Object.entries(hosts)) {
if (!HOST_ID.test(id)) refuse(`${where} host id ${JSON.stringify(id)} must match ${HOST_ID}`);
const at = `${where}.${id}`;
requireObject(entry, at);
rejectUnknownKeys(entry, ["address", "port", "user", "hostKeys"], at);
const address = checkAddress(entry.address, `${at}.address`);
if (!Number.isInteger(entry.port) || entry.port < 1 || entry.port > 65535) refuse(`${at}.port must be 1 to 65535`);
if (typeof entry.user !== "string" || !USER.test(entry.user)) refuse(`${at}.user must match ${USER}`);
if (!Array.isArray(entry.hostKeys) || entry.hostKeys.length === 0) refuse(`${at}.hostKeys must be a non-empty list`);
const keys = entry.hostKeys.map((key, i) => {
if (typeof key !== "string" || !HOST_KEY.test(key)) {
refuse(`${at}.hostKeys[${i}] must be "<type> <base64>" with type ssh-ed25519, ecdsa-sha2-nistp256 or ssh-rsa, and no comment`);
}
return key;
});
if (new Set(keys).size !== keys.length) refuse(`${at}.hostKeys lists a key twice`);
out[id] = { address, port: entry.port, user: entry.user, hostKeys: keys };
}
return out;
}
// D6: an argv allowlist and nothing else. No wildcard: "any", "*", {} and [] all refuse.
export function checkCommands(commands, where) {
if (!Array.isArray(commands)) refuse(`${where} must be a list of argv prefixes; there is no wildcard grant`);
@@ -110,18 +84,3 @@ export function checkCommands(commands, where) {
return [...prefix];
});
}
// A role entry's "hosts": { <hostId>: { commands } }, each id in the registry.
export function checkHostGrants(grants, registry, where) {
requireObject(grants, where);
if (Object.keys(grants).length === 0) refuse(`${where} must name at least one host`);
const out = {};
for (const [id, grant] of Object.entries(grants)) {
const at = `${where}.${id}`;
if (!Object.hasOwn(registry, id)) refuse(`${at}: host ${id} isn't in the business file's hosts`);
if (!isPlainObject(grant)) refuse(`${at} must be an object`);
rejectUnknownKeys(grant, ["commands"], at);
out[id] = { commands: checkCommands(grant.commands, `${at}.commands`) };
}
return out;
}
+29 -57
View File
@@ -1,13 +1,12 @@
// Row 210 (#1707, design D3 and D6): the host registry, an agent's host grant and its argv
// allowlist, checked at business load. Tests T1, T2 and T26 of the design.
// Row 210 (#1707, design D3 and D6): the address checks and the argv allowlist. Row 234 (#1740, r6 S1)
// moved the registry and the grants to bus store records, so the business file refuses them.
import { test } from "node:test";
import assert from "node:assert/strict";
import { readFileSync, writeFileSync } from "node:fs";
import { join } from "node:path";
import { BusinessError, validateBusinessDocument } from "../src/index.mjs";
import { resolveInstance } from "../src/resolve.mjs";
import { REFUSED_COMMANDS, commandAllowed, ownMachine, validArgv } from "../src/hosts.mjs";
import { businessDoc, rolesCopy, systemFor, tmp, tokenFile, FAR_DATE } from "./helpers.mjs";
import { HOST_KEY as HOST_KEY_PATTERN, REFUSED_COMMANDS, USER, checkAddress, checkCommands, commandAllowed, ownMachine, validArgv } from "../src/hosts.mjs";
import { businessDoc, rolesCopy, tmp, tokenFile, FAR_DATE } from "./helpers.mjs";
const HOST_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOv6Rq3Mvo9w2j3eYvYbqX0J1pJ0x9m8Zr0w1kJ8p0Qn";
@@ -19,8 +18,7 @@ function refuses(fn, pattern) {
});
}
// The fixture with coder's definition declaring the host credential, one registered host, and
// coder granted it with one command.
// The fixture with coder's definition declaring the host credential and no hosts.
function check(mutate, { declare = true } = {}) {
const root = tmp();
const rolesDir = rolesCopy(root);
@@ -31,62 +29,44 @@ function check(mutate, { declare = true } = {}) {
writeFileSync(file, JSON.stringify(coder));
}
const doc = businessDoc(root);
doc.hosts = { web1: { address: "10.1.1.53", port: 22, user: "localadmin", hostKeys: [HOST_KEY] } };
doc.roles.coder.hosts = { web1: { commands: [["systemctl", "status"]] } };
doc.roles.coder.credentials["host.web1"] = { file: tokenFile(join(root, "secrets"), "coder-host.key"), rotateBy: FAR_DATE };
mutate?.(doc, root);
return validateBusinessDocument(doc, join(root, "businesses", "acme.json"), { rolesDir });
}
test("T1: a registry and a grant load, and the resolved session credentials leave the host key out", () => {
const business = check();
assert.deepEqual(business.hosts.web1, { address: "10.1.1.53", port: 22, user: "localadmin", hostKeys: [HOST_KEY] });
assert.deepEqual(business.roles.coder.hosts, { web1: { commands: [["systemctl", "status"]] } });
assert.ok(business.roles.coder.credentials["host.web1"]);
assert.equal(check((d) => { d.hosts.web1.address = "host.example"; }).hosts.web1.address, "host.example");
// A file without hosts loads with no hosts key at all.
assert.equal("hosts" in check((d) => { delete d.hosts; delete d.roles.coder.hosts; delete d.roles.coder.credentials["host.web1"]; }), false);
// Row 234 (#1740, r6 S1): hosts and grants are bus store records, so the business file refuses both keys
// and any host.<id> key reference. The control loads.
test("S1: the business file refuses hosts at the top level and in a role, and host.<id> references", () => {
assert.equal("hosts" in check(), false);
assert.equal("hosts" in check().roles.coder, false);
const entry = { address: "10.1.1.53", port: 22, user: "localadmin", hostKeys: [HOST_KEY] };
refuses(() => check((d) => { d.hosts = { web1: entry }; }), /acme\.json hosts lives in the bus store/);
refuses(() => check((d) => { d.hosts = {}; }), /hosts lives in the bus store/);
refuses(() => check((d) => { d.roles.coder.hosts = { web1: { commands: [["systemctl", "status"]] } }; }), /roles\.coder\.hosts lives in the bus store/);
refuses(() => check((d) => { d.roles.coder.hosts = {}; }, { declare: false }), /roles\.coder\.hosts lives in the bus store/);
const key = (d, root) => { d.roles.coder.credentials["host.web1"] = { file: tokenFile(join(root, "secrets"), "coder-host.key"), rotateBy: FAR_DATE }; };
refuses(() => check(key), /host\.web1/);
});
test("T1: registry field checks name the key path", () => {
refuses(() => check((d) => { d.hosts = []; }), /hosts/);
refuses(() => check((d) => { d.hosts = { "Web 1": d.hosts.web1 }; }), /host id/);
refuses(() => check((d) => { d.hosts.web1.extra = 1; }), /hosts\.web1/);
for (const port of [0, 65536, "22", 1.5]) refuses(() => check((d) => { d.hosts.web1.port = port; }), /hosts\.web1\.port/);
for (const user of ["Root", "a b", "", 7]) refuses(() => check((d) => { d.hosts.web1.user = user; }), /hosts\.web1\.user/);
refuses(() => check((d) => { d.hosts.web1.hostKeys = []; }), /hosts\.web1\.hostKeys/);
refuses(() => check((d) => { d.hosts.web1.hostKeys = [`${HOST_KEY} root@web1`]; }), /hosts\.web1\.hostKeys\[0\]/);
refuses(() => check((d) => { d.hosts.web1.hostKeys = ["ssh-dss AAAA"]; }), /hosts\.web1\.hostKeys\[0\]/);
refuses(() => check((d) => { d.hosts.web1.hostKeys = [HOST_KEY, HOST_KEY]; }), /twice/);
for (const address of ["bad_name", "a..b", "-a.b", "fe80::1%eth0", 7])
refuses(() => check((d) => { d.hosts.web1.address = address; }), /hosts\.web1\.address/);
});
test("T1: every own-machine form refuses, and the controls load", () => {
test("T1: address, user and host key checks, which the enrollment raise uses", () => {
const own = ["127.0.0.2", "0.0.0.0", "::1", "::", "0:0:0:0:0:0:0:1", "::ffff:127.0.0.1", "::ffff:0.0.0.0", "localhost", "a.localhost",
"127.1", "2130706433", "0177.0.0.1", "0x7f.1"];
for (const address of own) refuses(() => check((d) => { d.hosts.web1.address = address; }), /hosts\.web1\.address/);
for (const address of ["10.1.1.53", "host.example", "2001:db8::1"]) assert.equal(check((d) => { d.hosts.web1.address = address; }).hosts.web1.address, address);
"127.1", "2130706433", "0177.0.0.1", "0x7f.1", "bad_name", "a..b", "-a.b", "fe80::1%eth0", 7];
for (const address of own) refuses(() => checkAddress(address, "grant.address"), /grant\.address/);
for (const address of ["10.1.1.53", "host.example", "2001:db8::1"]) assert.equal(checkAddress(address, "grant.address"), address);
assert.equal(ownMachine("::ffff:7f00:1"), true);
assert.equal(ownMachine("::ffff:10.0.0.1"), false);
});
test("T2: a grant naming an unknown host, or on a definition without host, refuses at roles.<agent>.hosts", () => {
refuses(() => check((d) => { d.roles.coder.hosts = { web2: { commands: [["ls"]] } }; }), /roles\.coder\.hosts\.web2: host web2 isn't/);
refuses(() => check(null, { declare: false }), /roles\.coder\.hosts needs a role definition that declares the host credential/);
refuses(() => check((d) => { d.roles.coder.hosts = {}; }), /roles\.coder\.hosts must name at least one host/);
refuses(() => check((d) => { d.roles.coder.hosts.web1.extra = 1; }), /roles\.coder\.hosts\.web1/);
// The key reference is for a granted host only.
refuses(() => check((d) => { d.roles.coder.credentials["host.web2"] = d.roles.coder.credentials["host.web1"]; }), /host\.web2/);
for (const user of ["Root", "a b", ""]) assert.equal(USER.test(user), false);
assert.equal(USER.test("localadmin"), true);
for (const key of [`${HOST_KEY} root@web1`, "ssh-dss AAAA"]) assert.equal(HOST_KEY_PATTERN.test(key), false);
assert.equal(HOST_KEY_PATTERN.test(HOST_KEY), true);
});
test("T26: no wildcard and no shell, runner or interpreter as a prefix", () => {
const at = /roles\.coder\.hosts\.web1\.commands/;
const at = /grant\.commands/;
for (const commands of ["any", "*", {}, [], [[]], [[7]], [["ls", ""]], [["ls"], ["ls"]], [["ls", "*"]], [["ls", "a?"]], [["ls", "[a]"]], [["bin/ls"]], [["ls\0"]]])
refuses(() => check((d) => { d.roles.coder.hosts.web1.commands = commands; }), at);
refuses(() => checkCommands(commands, "grant.commands"), at);
const names = [...REFUSED_COMMANDS, "python", "python3", "python3.12", "awk", "gawk", "mawk", "nawk", "php", "php8.2", "/bin/bash", "/usr/bin/env"];
for (const name of names) refuses(() => check((d) => { d.roles.coder.hosts.web1.commands = [[name, "x"]]; }), at);
for (const name of ["systemctl", "/usr/bin/journalctl", "tee"]) assert.ok(check((d) => { d.roles.coder.hosts.web1.commands = [[name]]; }));
for (const name of names) refuses(() => checkCommands([[name, "x"]], "grant.commands"), at);
for (const name of ["systemctl", "/usr/bin/journalctl", "tee"]) assert.deepEqual(checkCommands([[name]], "grant.commands"), [[name]]);
});
test("D6: argv shape and the literal prefix match", () => {
@@ -101,11 +81,3 @@ test("D6: argv shape and the literal prefix match", () => {
assert.equal(commandAllowed(["systemctl", "restart", "nginx"], commands), false);
assert.equal(commandAllowed(["systemctl", "status*"], commands), false);
});
test("D5: resolveInstance leaves host.<id> out of the session's credentials", () => {
let root;
const business = check((d, r) => { root = r; });
const resolved = resolveInstance({ system: systemFor(root), business, instance: "coder" });
assert.equal("host.web1" in resolved.credentials, false);
assert.ok(resolved.credentials.gitea);
});
+19 -7
View File
@@ -432,9 +432,11 @@ A Claude Code session logs in with `CLAUDE_CODE_OAUTH_TOKEN`,
on. The bus unit loads it ("Unit" below). Nothing from the host's
`~/.claude` reaches a session, whose `HOME` is its own. With none of them
set, a Claude Code launch refuses `no-session-credential` before anything
starts. A Pi session gets the same variables through the allowlist. A
shell the harness opens, Claude Code's `Bash` or Pi's `bash`, can read
them.
starts. A Pi session gets the same variables through the allowlist. A Pi
session on provider `zai` logs in with `ZAI_API_KEY`, which the allowlist
passes. Like every allowlist variable, it reaches every session the host
starts, Claude Code ones included. A shell the harness opens, Claude Code's
`Bash` or Pi's `bash`, can read all of these variables.
A Pi session logs in only through the variables `ENV_ALLOW` passes. Its
`HOME` is the run's own, so Pi never reads the operator's `~/.pi/agent`
@@ -706,13 +708,14 @@ instead and sessions run `claude` from the unit's `PATH`, which in the
sandbox finds only a `claude` under `/usr`. A running host
keeps the environment it started with until it restarts.
The session credential comes from
The session credentials come from
`EnvironmentFile=-%h/.config/mosaic-dev/secrets/claude-session.env`, the
one place the path is set. The file holds one line, for example
`CLAUDE_CODE_OAUTH_TOKEN=<token>`, mode 0600. The operator writes it;
one place the path is set. The file holds one line per variable, for
example `CLAUDE_CODE_OAUTH_TOKEN=<token>`, mode 0600. It holds every session
provider key, not only Claude's. The operator writes it;
nothing in the repository creates it, and only the Case 2 precheck reads
it, printing variable names, never values. No session sees the file
itself, only the variable. The leading `-` lets the unit start
itself, only the variables. The leading `-` lets the unit start
without it, so Pi sessions still launch and a Claude Code launch refuses
`no-session-credential`. To change the credential, edit or swap the file
(or this line and this paragraph), then restart the unit; a running host
@@ -1046,6 +1049,15 @@ entries; `delete` refuses `agent-running` while the agent holds a claim. With
`--tester` only `status` and `revoke` run. After a rotate or revoke, delete
the old token in Gitea or Vikunja yourself; the bus only forgets it.
## hosts (row 234)
`mosaic hosts [status] [--json] [--tester] [--business ID]` prints one tab-separated line per
enrolled host: id, state, `user@address:port`, `sudo` or `no-sudo`, the last check time or
`unchecked`, and any host lines a failed step left. `--json` prints the bus rows. `check HOST`
runs the bus's `hosts.check` and `remove HOST` its `hosts.remove`, each printing the JSON result.
Enrollment itself is a `host-enroll` grant the chief raises and the human approves, not a CLI
command. A usage error exits 4 and a bus refusal 2.
## Token usage
`mosaic usage [--business ID] [--json] [--tester]` reads the bus's accounting, without
+4
View File
@@ -18,6 +18,7 @@ import { resolveThread, threadCommand, threadMembers, threadsCommand, THREAD_TAL
import { limitsCommand, LIMITS_USAGE } from "./limits.mjs";
import { pluginsCommand, PLUGINS_USAGE } from "./plugins.mjs";
import { credentialsCommand, CREDENTIALS_USAGE } from "./credentials.mjs";
import { hostsCommand, HOSTS_USAGE } from "./hosts.mjs";
import { instancesCommand, INSTANCES_USAGE } from "./instances.mjs";
import { createChannelCaller } from "../../plugins/src/channel.mjs";
import { PluginError } from "../../plugins/src/kinds.mjs";
@@ -40,6 +41,7 @@ export const USAGE = `usage:
${LIMITS_USAGE}
${PLUGINS_USAGE}
${CREDENTIALS_USAGE}
${HOSTS_USAGE}
${INSTANCES_USAGE}
${PROJECT_USAGE}
${PROJECT_BUS_USAGE}
@@ -580,6 +582,8 @@ export async function main(argv, io = { env: process.env, stdin: process.stdin,
if (verb === "plugins") return pluginsCommand(rest, io, (p) => callFor(p, io, deps));
// Row 199 (#1697): issue and rotate read the token on stdin.
if (verb === "credentials") return credentialsCommand(rest, io, (p) => callFor(p, io, deps));
// Row 234 (#1740, r6 E4): the host records.
if (verb === "hosts") return hostsCommand(rest, io, (p) => callFor(p, io, deps));
// Row 165 (#1662): scripts/mosaic sends only `business instances` and `business instance` here.
if (verb === "business") return instancesCommand(rest, io, (p) => callFor(p, io, deps));
if (verb === "talk") return talk(rest, io, deps);
+7 -3
View File
@@ -38,7 +38,10 @@
// one op at a time within 10 s. The poll ends at publishDeadlineMs
// (publish-timeout), past git's own limit on each of the four git
// runs. recordPublish files the push; any refusal after checkPublish
// is filed once, by publishRefused.
// is filed once, by publishRefused. After publish-timeout the bus
// holds the decision until its push ends (a retry gets publish-busy),
// then records a push that reached the fork or files the refusal
// (row 242, #1748).
//
// A refusal is {"ok": false, "error": "<code>"}, and the connection ends.
// The host checks, in order: identity (a role of this business), the request
@@ -427,8 +430,9 @@ export function createGitSocket({ host, business: b, dataRoot, env = process.env
await op({ op: "publishPush", ticket });
const until = Date.now() + publishDeadlineMs;
for (;;) {
// The bus's git may still finish after this; the fork could then hold the commit with
// no record, as when recordPublish refuses after a push.
// The bus's git may still finish after this. The bus holds the decision until it does,
// then records a push that reached the fork (row 242); the fork can still hold the
// commit with no record if recordPublish refuses, as after an in-band push.
if (Date.now() >= until) {
log(`git socket: publish ${task_ref}: publish-timeout; the bus process's push may still reach the fork`);
throw new Refusal("publish-timeout");
+44
View File
@@ -0,0 +1,44 @@
// `mosaic hosts` (row 234, #1740, r6 E4): the broker's hosts.status, hosts.check and hosts.remove. The
// human runs all three. With --tester the tester's session runs them. check logs in as mosaic-installer
// and runs `list` on the host, then marks the host ready. remove drops a host no grant uses. Nothing here
// reads a key: the answers carry fingerprints only.
import { CliError } from "./errors.mjs";
export const HOSTS_USAGE = "mosaic hosts [status] [--json] [--tester] [--business ID] | check|remove HOST [--tester] [--business ID]";
function parse(args) {
const out = { positional: [], flags: new Set(), values: {} };
for (let i = 0; i < args.length; i++) {
const arg = args[i];
if (arg === "--json" || arg === "--tester") out.flags.add(arg);
else if (arg === "--business") {
if (!args[i + 1] || Object.hasOwn(out.values, arg)) throw new CliError(HOSTS_USAGE, 4);
out.values[arg] = args[++i];
} else if (arg.startsWith("--")) throw new CliError(HOSTS_USAGE, 4);
else out.positional.push(arg);
}
return out;
}
const line = (h) =>
[h.host, h.state, `${h.user}@${h.address}:${h.port}`, h.sudo ? "sudo" : "no-sudo", h.checkedAt ?? "unchecked",
...(h.hostLineLeft.length ? [`line left: ${h.hostLineLeft.join(" ")}`] : [])].join("\t");
export async function hostsCommand(args, io, callFor) {
const parsed = parse(args);
const [verb = "status", host, ...extra] = parsed.positional;
const shape = { status: host === undefined, check: host !== undefined, remove: host !== undefined };
if (extra.length || !shape[verb] || (parsed.flags.has("--json") && verb !== "status")) throw new CliError(HOSTS_USAGE, 4);
try {
const call = callFor(parsed);
if (verb === "status") {
const rows = await call("hosts.status", {});
io.stdout.write(parsed.flags.has("--json") ? JSON.stringify(rows, null, 2) + "\n" : rows.map(line).join("\n") + (rows.length ? "\n" : ""));
return;
}
io.stdout.write(JSON.stringify(await call(`hosts.${verb}`, { host }), null, 2) + "\n");
} catch (error) {
if (error instanceof CliError) throw error;
throw new CliError(error.message, 2);
}
}
@@ -0,0 +1,16 @@
# Mosaic Case 1 host check at boot (row 240, #1746; r6 B2). Root installs it as
# /etc/systemd/system/mosaic-hostcheck.service, root:root 0644, and enables it
# (docs/guides/host-run-sheet.md). It runs the drift stop before the bus starts, which covers a kernel
# or parser change that takes effect at boot. --enforce always exits 0; on drift it closes bwrap to
# root, and every role session then refuses to start.
[Unit]
Description=Mosaic Case 1 host check: the bwrap profile, the statoverride and the sysctls
After=apparmor.service
Before=[email protected]
[Service]
Type=oneshot
ExecStart=/usr/local/lib/mosaic-stack/hostcheck.sh --enforce
[Install]
WantedBy=multi-user.target
+335
View File
@@ -0,0 +1,335 @@
// Row 240 (I4, #1746): scripts/case1-hostcheck.sh, the Case 1 host's drift stop for bwrap, and its
// pins and boot unit. Brief: docs/plans/2026-10-20_installer-rows.md "I4 hostcheck"; design r6 B2
// (agents/darkwing/work/queue-195/DESIGN.md at b7239bb66). Every case copies the script into a scratch
// lib directory with a fixture case1-pins.sh beside it and runs it with --root on a scratch host tree.
// Fake aa-status, dpkg, dpkg-statoverride, apparmor_parser, systemd-run, stat, chown, chmod, apt-get
// and dpkg-deb on PATH record their argv. Nothing here reads or writes the real /etc, /proc or /usr.
import { test } from "node:test";
import assert from "node:assert/strict";
import { spawnSync } from "node:child_process";
import { createHash } from "node:crypto";
import { chmodSync, copyFileSync, existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import { tmp } from "./helpers.mjs";
const REPO = fileURLToPath(new URL("../../../", import.meta.url));
const SCRIPT = join(REPO, "scripts", "case1-hostcheck.sh");
const PINS = join(REPO, "scripts", "case1-pins.sh");
const UNIT = join(REPO, "packages", "cli", "systemd", "mosaic-hostcheck.service");
const PROFILE = "/etc/apparmor.d/bwrap-userns-restrict";
const LABEL = "bwrap//&unpriv_bwrap (enforce)";
// A made-up profile; the fixture pins carry its sha256.
const PROFILE_TEXT = "abi <abi/4.0>,\ninclude <tunables/global>\nprofile bwrap /usr/bin/bwrap flags=(attach_disconnected,mediate_deleted) {\n allow capability,\n change_profile /** -> &unpriv_bwrap,\n}\nprofile unpriv_bwrap flags=(attach_disconnected,mediate_deleted) {\n audit deny capability,\n}\n";
const sha256 = (s) => createHash("sha256").update(s).digest("hex");
const AA_OK = { version: "2", profiles: { "/usr/bin/man": "enforce", bwrap: "enforce", unpriv_bwrap: "enforce", "lsb_release": "enforce" }, processes: {} };
const SYSTEMD_RUN = ["--wait", "--pipe", "--uid=mosaic", "--gid=mosaic", "-p", "NoNewPrivileges=yes", "-p", "ProtectSystem=strict",
"-p", "ProtectHome=yes", "-p", "PrivateTmp=yes", "/usr/bin/bwrap", "--unshare-user", "--unshare-pid", "--ro-bind", "/", "/",
"cat", "/proc/self/attr/current"];
function put(file, text, mode) {
mkdirSync(dirname(file), { recursive: true });
writeFileSync(file, text);
if (mode) chmodSync(file, mode);
}
// Each fake appends `name argv...` to calls.log and answers from a file in the state directory.
const FAKES = {
"aa-status": `[ -f "$HC/aa-rc" ] && exit "$(cat "$HC/aa-rc")"; cat "$HC/aa-status.json"`,
dpkg: `case "$1" in
-S) [ -f "$HC/owner" ] && cat "$HC/owner"; exit "$(cat "$HC/dpkg-S-rc")" ;;
-s) [ "$2" = bubblewrap ] && printf 'Package: bubblewrap\\nStatus: install ok installed\\nVersion: %s\\n' "$(cat "$HC/bwrap-version")" ;;
*) exit 2 ;;
esac`,
"dpkg-statoverride": `[ "$1" = --list ] && cat "$HC/statoverride"`,
apparmor_parser: `exit 0`,
"systemd-run": `cat "$HC/label"`,
stat: `cat "$HC/stat"`,
chown: `exit 0`,
chmod: `exit 0`,
"apt-get": `[ -f "$HC/apt-rc" ] && exit "$(cat "$HC/apt-rc")"; [ "$1 $2" = "download apparmor-profiles" ] && cp "$HC/deb" "./apparmor-profiles_9.9_all.deb"`,
"dpkg-deb": `case "$1" in
-f) [ "$3" = Version ] && echo 9.9-upstream ;;
-x) mkdir -p "$3/usr/share/apparmor/extra-profiles"; [ ! -f "$HC/upstream-profile" ] || cp "$HC/upstream-profile" "$3/usr/share/apparmor/extra-profiles/bwrap-userns-restrict" ;;
esac`,
};
// A scratch host where all seven items hold.
function host(t) {
const dir = tmp(t, "case1-hostcheck-");
const root = join(dir, "root"), hc = join(dir, "hc"), bin = join(dir, "bin"), lib = join(dir, "lib");
put(join(root, PROFILE), PROFILE_TEXT);
put(join(root, "etc", "apparmor.d", "abstractions", "base"), " /etc/ld.so.cache r,\n");
put(join(root, "etc", "apparmor.d", "tunables", "global"), "@{HOME}=/home/*/\n");
put(join(root, "proc", "sys", "kernel", "apparmor_restrict_unprivileged_userns"), "1\n");
put(join(root, "proc", "sys", "fs", "protected_hardlinks"), "1\n");
put(join(root, "proc", "sys", "net", "ipv4", "ip_unprivileged_port_start"), "1024\n");
put(join(root, "etc", "systemd", "system", "mosaic-webui.service"),
"[Service]\nExecStart=/opt/mosaic-stack/node/bin/node /srv/mosaic/copies/x/packages/webui/src/cli.mjs serve --port 732 --business mosaic-stack --tester-port 7333\n");
put(join(root, "usr", "bin", "bwrap"), "", 0o755);
put(join(hc, "aa-status.json"), JSON.stringify(AA_OK));
put(join(hc, "dpkg-S-rc"), "1");
put(join(hc, "bwrap-version"), "0.9.0-1ubuntu0.3");
put(join(hc, "statoverride"), "root mosaic 0750 /usr/bin/bwrap\n");
put(join(hc, "label"), `${LABEL}\n`);
put(join(hc, "stat"), "root mosaic 750\n");
put(join(hc, "deb"), "fake deb\n");
put(join(hc, "upstream-profile"), PROFILE_TEXT);
for (const [name, body] of Object.entries(FAKES)) {
put(join(bin, name), `#!/usr/bin/env bash\nprintf '%s' ${JSON.stringify(name)} >> "$HC/calls.log"; printf ' %s' "$@" >> "$HC/calls.log"; echo >> "$HC/calls.log"\n${body}\n`, 0o755);
}
mkdirSync(lib);
copyFileSync(SCRIPT, join(lib, "hostcheck.sh"));
chmodSync(join(lib, "hostcheck.sh"), 0o755);
put(join(lib, "case1-pins.sh"), `PIN_PROFILES_VERSION=4.0.1-fixture\nPIN_PROFILE_SHA256=${sha256(PROFILE_TEXT)}\nPIN_BUBBLEWRAP_VERSION=0.9.0-1ubuntu0.3\n`);
return { dir, root, hc, bin, lib, script: join(lib, "hostcheck.sh") };
}
function run(h, ...args) {
const r = spawnSync(h.script, [...args, "--root", h.root], {
encoding: "utf8",
env: { ...process.env, PATH: `${h.bin}:${process.env.PATH}`, HC: h.hc, TMPDIR: h.dir },
});
return { status: r.status, out: r.stdout, err: r.stderr, lines: r.stdout.split("\n").filter(Boolean) };
}
const calls = (h, name) => (existsSync(join(h.hc, "calls.log")) ? readFileSync(join(h.hc, "calls.log"), "utf8") : "")
.split("\n").filter((l) => l === name || l.startsWith(`${name} `)).map((l) => l.slice(name.length + 1));
const fails = (r) => r.lines.filter((l) => l.startsWith("FAIL "));
const drift = (h) => join(h.root, "etc", "mosaic-stack", "DRIFT");
const hlog = (h) => join(h.root, "var", "log", "mosaic-stack", "hostcheck.log");
const aa = (h, profiles) => put(join(h.hc, "aa-status.json"), JSON.stringify({ version: "2", profiles, processes: {} }));
// One fault per item; each makes that item, and only that one, fail.
const FAULTS = {
1: (h) => put(join(h.root, PROFILE), `${PROFILE_TEXT}# drift\n`),
2: (h) => aa(h, { ...AA_OK.profiles, bwrap: "complain" }),
3: (h) => put(join(h.root, "etc", "apparmor.d", "usr.bin.other"), "profile other /usr/bin/other {\n /usr/bin/bwrap ix,\n}\n"),
4: (h) => put(join(h.hc, "label"), "unconfined\n"),
5: (h) => put(join(h.hc, "statoverride"), "root root 0755 /usr/bin/bwrap\n"),
6: (h) => put(join(h.root, "proc", "sys", "kernel", "apparmor_restrict_unprivileged_userns"), "0\n"),
7: (h) => put(join(h.root, "proc", "sys", "net", "ipv4", "ip_unprivileged_port_start"), "700\n"),
};
test("row 240 check: exits 0 with all seven items and a note that bubblewrap is the install pin", (t) => {
const h = host(t);
const r = run(h, "--check");
assert.equal(r.status, 0, r.out + r.err);
assert.deepEqual(r.lines.map((l) => l.split(" ").slice(0, 2).join(" ")), ["ok 1", "ok 2", "ok 3", "ok 4", "ok 5", "ok 6", "ok 7", "note bubblewrap"]);
assert.match(r.lines[6], /is 1024, above the Console's --port 732$/);
assert.equal(r.lines[7], "note bubblewrap 0.9.0-1ubuntu0.3 is the install pin");
});
test("row 240 check: exits 1 with each item failing alone, and its line names it", (t) => {
for (const [n, fault] of Object.entries(FAULTS)) {
const h = host(t);
fault(h);
const r = run(h, "--check");
assert.equal(r.status, 1, `item ${n}: ${r.out}`);
assert.deepEqual(fails(r).map((l) => l.split(" ")[1]), [n], `item ${n}: ${r.out}`);
}
});
test("row 240 check: item 1 fails when a package owns the profile, when dpkg -S errs, or with no pin", (t) => {
let h = host(t);
put(join(h.hc, "dpkg-S-rc"), "0");
put(join(h.hc, "owner"), `apparmor-profiles: ${PROFILE}\n`);
let r = run(h, "--check");
assert.equal(r.status, 1);
assert.deepEqual(fails(r), [`FAIL 1 ${PROFILE} is owned by a package: apparmor-profiles: ${PROFILE}`]);
assert.deepEqual(calls(h, "dpkg").filter((a) => a.startsWith("-S")), [`-S ${PROFILE}`]);
h = host(t);
put(join(h.hc, "dpkg-S-rc"), "2");
r = run(h, "--check");
assert.deepEqual(fails(r), [`FAIL 1 dpkg -S ${PROFILE} exited 2`]);
h = host(t);
rmSync(join(h.lib, "case1-pins.sh"));
r = run(h, "--check");
assert.equal(r.status, 1);
assert.deepEqual(fails(r), [`FAIL 1 no profile pin in ${h.lib}/case1-pins.sh`]);
});
test("row 240 check: item 2 fails on a loaded mosaic-bwrap, a missing unpriv_bwrap or a failed aa-status", (t) => {
for (const [fault, text] of [
[(h) => aa(h, { ...AA_OK.profiles, "mosaic-bwrap": "enforce" }), "a mosaic-bwrap profile"],
[(h) => aa(h, { bwrap: "enforce" }), "no unpriv_bwrap profile in enforce mode"],
[(h) => put(join(h.hc, "aa-rc"), "1"), "aa-status --json failed"],
]) {
const h = host(t);
fault(h);
const r = run(h, "--check");
assert.equal(r.status, 1);
assert.equal(fails(r).length, 1, r.out);
assert.ok(fails(r)[0].startsWith("FAIL 2 ") && fails(r)[0].includes(text), r.out);
}
});
test("row 240 check: item 3 fails on a second file naming bwrap as a word, /{usr/,}bin/bwrap included", (t) => {
for (const text of ["profile x /{usr/,}bin/bwrap {\n}\n", "# runs bwrap\n", "/usr/bin/bwrap ix,\n"]) {
const h = host(t);
put(join(h.root, "etc", "apparmor.d", "local", "extra"), text);
const r = run(h, "--check");
assert.equal(r.status, 1, text);
assert.deepEqual(fails(r).map((l) => l.split(" ")[1]), ["3"], text);
assert.ok(fails(r)[0].includes(join(h.root, "etc", "apparmor.d", "local", "extra")), r.out);
}
const h = host(t);
put(join(h.root, "etc", "apparmor.d", "local", "extra"), "# bwrapper and unpriv-bwrapx are other words\n");
assert.equal(run(h, "--check").status, 0);
});
test("row 240 check: item 4 reads the label under the units' no-new-privs and matches it exactly", (t) => {
let h = host(t);
assert.equal(run(h, "--check").status, 0);
assert.deepEqual(calls(h, "systemd-run"), [SYSTEMD_RUN.join(" ")]);
for (const label of ["unpriv_bwrap (enforce)", `${LABEL} `, `x${LABEL}`]) {
h = host(t);
put(join(h.hc, "label"), `${label}\n`);
const r = run(h, "--check");
assert.equal(r.status, 1, JSON.stringify(label));
assert.deepEqual(fails(r), [`FAIL 4 a bwrap child under no-new-privs has the label '${label}', not '${LABEL}'`]);
}
});
test("row 240 check: item 5 needs the statoverride and stat to agree", (t) => {
const h = host(t);
put(join(h.hc, "stat"), "root root 700\n");
const r = run(h, "--check");
assert.deepEqual(fails(r), ["FAIL 5 /usr/bin/bwrap is 'root root 700', not 'root mosaic 750'"]);
assert.deepEqual(calls(h, "dpkg-statoverride"), ["--list /usr/bin/bwrap"]);
assert.deepEqual(calls(h, "stat"), [`-c %U %G %a ${h.root}/usr/bin/bwrap`]);
});
test("row 240 check: item 6 fails on protected_hardlinks 0; item 7 fails with the floor at 732 and reads 1023 with no unit", (t) => {
let h = host(t);
put(join(h.root, "proc", "sys", "fs", "protected_hardlinks"), "0\n");
assert.deepEqual(fails(run(h, "--check")).map((l) => l.split(" ")[1]), ["6"]);
h = host(t);
put(join(h.root, "proc", "sys", "net", "ipv4", "ip_unprivileged_port_start"), "732\n");
assert.deepEqual(fails(run(h, "--check")), ["FAIL 7 net.ipv4.ip_unprivileged_port_start is 732, not above the Console's --port 732"]);
put(join(h.root, "proc", "sys", "net", "ipv4", "ip_unprivileged_port_start"), "733\n");
assert.equal(run(h, "--check").status, 0);
rmSync(join(h.root, "etc", "systemd", "system", "mosaic-webui.service"));
assert.deepEqual(fails(run(h, "--check")), ["FAIL 7 net.ipv4.ip_unprivileged_port_start is 733, not above 1023, because no Console unit is installed"]);
put(join(h.root, "proc", "sys", "net", "ipv4", "ip_unprivileged_port_start"), "1024\n");
assert.equal(run(h, "--check").status, 0);
put(join(h.root, "etc", "systemd", "system", "mosaic-webui.service"), "[Service]\nExecStart=/x serve --business mosaic-stack --tester-port 7333\n");
assert.deepEqual(fails(run(h, "--check")), ["FAIL 7 /etc/systemd/system/mosaic-webui.service has no --port"]);
});
test("row 240 check: a bubblewrap version other than the install pin is a note, not a failure", (t) => {
const h = host(t);
put(join(h.hc, "bwrap-version"), "0.9.0-1ubuntu0.4");
const r = run(h, "--check");
assert.equal(r.status, 0);
assert.equal(r.lines.at(-1), "note bubblewrap 0.9.0-1ubuntu0.4 is not the install pin 0.9.0-1ubuntu0.3, which is not a drift item");
});
test("row 240 enforce: a pass with no DRIFT changes nothing and exits 0", (t) => {
const h = host(t);
const r = run(h, "--enforce");
assert.equal(r.status, 0);
assert.equal(r.lines.at(-1), "hostcheck: all seven hold");
assert.deepEqual([calls(h, "apparmor_parser"), calls(h, "chown"), calls(h, "chmod")], [[], [], []]);
assert.equal(existsSync(drift(h)) || existsSync(hlog(h)), false);
});
test("row 240 enforce: each failure exits 0, unloads the profile, closes bwrap, writes DRIFT and one log line", (t) => {
for (const [n, fault] of Object.entries(FAULTS)) {
const h = host(t);
fault(h);
const r = run(h, "--enforce");
assert.equal(r.status, 0, `item ${n}: ${r.out}${r.err}`);
assert.deepEqual(calls(h, "apparmor_parser"), [`-R ${h.root}${PROFILE}`], `item ${n}`);
assert.deepEqual(calls(h, "chown"), [`root:root ${h.root}/usr/bin/bwrap`], `item ${n}`);
assert.deepEqual(calls(h, "chmod"), [`0700 ${h.root}/usr/bin/bwrap`], `item ${n}`);
const d = readFileSync(drift(h), "utf8").split("\n").filter(Boolean);
assert.equal(d.length, 1, `item ${n}`);
assert.match(d[0], new RegExp(`^\\d{4}-\\d\\d-\\d\\dT\\d\\d:\\d\\d:\\d\\dZ ${n} `), `item ${n}`);
const l = readFileSync(hlog(h), "utf8").split("\n").filter(Boolean);
assert.equal(l.length, 1, `item ${n}`);
assert.match(l[0], new RegExp(`^\\d{4}-\\d\\d-\\d\\dT\\d\\d:\\d\\d:\\d\\dZ enforce stop items=${n} drift-before=0 unload=rc=0 chown=0 chmod=0$`), `item ${n}`);
}
});
test("row 240 enforce: with DRIFT present and every item passing, it still closes bwrap", (t) => {
const h = host(t);
put(drift(h), "2026-10-20T00:00:00Z 1 an earlier drift\n");
const r = run(h, "--enforce");
assert.equal(r.status, 0);
assert.equal(fails(r).length, 0);
assert.deepEqual(calls(h, "chown"), [`root:root ${h.root}/usr/bin/bwrap`]);
assert.deepEqual(calls(h, "chmod"), [`0700 ${h.root}/usr/bin/bwrap`]);
assert.equal(readFileSync(drift(h), "utf8"), "2026-10-20T00:00:00Z 1 an earlier drift\n");
assert.match(readFileSync(hlog(h), "utf8"), /Z enforce stop items=none drift-before=1 unload=rc=0 chown=0 chmod=0\n$/);
});
test("row 240 enforce: no unload when aa-status lists neither bwrap profile; never writes the statoverride", (t) => {
const h = host(t);
aa(h, { "/usr/bin/man": "enforce" });
const r = run(h, "--enforce");
assert.equal(r.status, 0);
assert.deepEqual(calls(h, "apparmor_parser"), []);
assert.deepEqual(calls(h, "chmod"), [`0700 ${h.root}/usr/bin/bwrap`]);
assert.match(readFileSync(hlog(h), "utf8"), /Z enforce stop items=2 drift-before=0 unload=skipped chown=0 chmod=0\n$/);
for (const [n, fault] of Object.entries(FAULTS)) {
const g = host(t);
fault(g);
run(g, "--enforce");
for (const a of calls(g, "dpkg-statoverride")) assert.doesNotMatch(a, /--(update|add|remove)/, `item ${n}`);
assert.deepEqual(calls(g, "dpkg-statoverride"), ["--list /usr/bin/bwrap"], `item ${n}`);
}
});
test("row 240 upstream: a candidate whose profile differs prints the difference and exits 0", (t) => {
let h = host(t);
put(join(h.hc, "upstream-profile"), `${PROFILE_TEXT} # upstream change\n`);
let r = run(h, "--upstream");
assert.equal(r.status, 0, r.err);
assert.match(r.out, new RegExp(`^upstream: DIFFERS: apparmor-profiles 9.9-upstream's bwrap-userns-restrict has sha256 ${sha256(`${PROFILE_TEXT} # upstream change\n`)}; the pin is ${sha256(PROFILE_TEXT)} from 4.0.1-fixture\\.`));
assert.match(r.out, /^\+ {2}# upstream change$/m);
assert.deepEqual(calls(h, "apt-get"), ["download apparmor-profiles"]);
assert.deepEqual([calls(h, "chown"), calls(h, "chmod"), existsSync(drift(h))], [[], [], false]);
h = host(t);
r = run(h, "--upstream");
assert.equal(r.status, 0);
assert.equal(r.out, "upstream: apparmor-profiles 9.9-upstream's bwrap-userns-restrict matches the pin from 4.0.1-fixture\n");
h = host(t);
rmSync(join(h.hc, "upstream-profile"));
r = run(h, "--upstream");
assert.equal(r.status, 0);
assert.match(r.out, /^upstream: DIFFERS: apparmor-profiles 9.9-upstream has no bwrap-userns-restrict;/);
h = host(t);
put(join(h.hc, "apt-rc"), "100");
r = run(h, "--upstream");
assert.equal(r.status, 1);
assert.equal(r.err, "hostcheck: upstream: apt-get download apparmor-profiles failed\n");
});
test("row 240 boot unit: a oneshot after apparmor and before the bus that runs --enforce", () => {
const text = readFileSync(UNIT, "utf8");
const lines = text.split("\n").filter((l) => l && !l.startsWith("#"));
assert.deepEqual(lines.filter((l) => /^(Type|After|Before|ExecStart|WantedBy)=/.test(l)), [
"After=apparmor.service",
"[email protected]",
"Type=oneshot",
"ExecStart=/usr/local/lib/mosaic-stack/hostcheck.sh --enforce",
"WantedBy=multi-user.target",
]);
});
test("row 240 pins and usage: the shipped pins are r6's, the script reads the pins beside it, and bad usage exits 2", (t) => {
assert.equal(readFileSync(PINS, "utf8").split("\n").filter((l) => /^PIN_/.test(l)).join("\n"),
"PIN_PROFILES_VERSION=4.0.1really4.0.1-0ubuntu0.24.04.9\nPIN_PROFILE_SHA256=11d39094f044f0cda0febb3ad517b830301da6b2ce929664af09ee9e4dd264f9\nPIN_BUBBLEWRAP_VERSION=0.9.0-1ubuntu0.3");
const h = host(t);
const r = run({ ...h, script: SCRIPT }, "--check");
assert.equal(r.status, 1);
assert.deepEqual(fails(r), [`FAIL 1 ${PROFILE} sha256 is ${sha256(PROFILE_TEXT)}, not the pin 11d39094f044f0cda0febb3ad517b830301da6b2ce929664af09ee9e4dd264f9`]);
for (const args of [[], ["--check", "--enforce"], ["--check", "--root", "relative"], ["--status"]]) {
const u = spawnSync(h.script, args, { encoding: "utf8", env: { ...process.env, PATH: `${h.bin}:${process.env.PATH}`, HC: h.hc } });
assert.equal(u.status, 2, args.join(" "));
}
if (process.getuid() !== 0) {
const u = spawnSync(h.script, ["--check"], { encoding: "utf8", env: { ...process.env, PATH: `${h.bin}:${process.env.PATH}`, HC: h.hc } });
assert.deepEqual([u.status, u.stderr], [2, "hostcheck: refusing: run it as root\n"]);
}
});
+20
View File
@@ -0,0 +1,20 @@
// Controlled child for lane crash tests; no live service, network or model.
import { writeFileSync } from 'node:fs';
const [mode,marker]=process.argv.slice(2);
if(marker&&mode!=='lock-probe')writeFileSync(marker,String(process.pid));
if(mode!=='lock-probe')process.stdout.write('READY\n');
if(mode==='ignore-term')process.on('SIGTERM',()=>{});
else process.on('SIGTERM',()=>process.exit(0));
setInterval(()=>{},1000);
if(mode==='lock-probe'){
const stateRoot=marker;
const {verifyLocks,executeSteps}=await import('../../../../scripts/testing/gate-lane.mjs');
verifyLocks({lane:'a',stateRoot,exclusive:false});
const {appendRecord,identity}=await import('../../../../scripts/testing/lane-state.mjs');
const {mkdirSync}=await import('node:fs');const {join}=await import('node:path');
const out=join(stateRoot,'out');mkdirSync(out);
appendRecord(join(stateRoot,'runs','abcdef123456'),{kind:'start',lane:'a',owner:identity()});
const result=await executeSteps([{name:'fds',cwd:stateRoot,argv:[process.execPath,'-e',"const fs=require('fs'); for(const fd of fs.readdirSync('/proc/self/fd')){let p;try{p=fs.readlinkSync('/proc/self/fd/'+fd)}catch{continue}if(p.endsWith('/a.lock')||p.endsWith('/image.lock'))process.exit(7)}"]}],{out,env:process.env,record:()=>{}});
if(result.rc)process.exit(result.rc);
writeFileSync(join(stateRoot,'ready'),'verified');
}
@@ -0,0 +1,94 @@
import test from 'node:test';import assert from 'node:assert/strict';
import { mkdtempSync,rmSync,mkdirSync,writeFileSync } from 'node:fs';import { join } from 'node:path';import { createServer } from 'node:net';import { once } from 'node:events';
const api=await import('../../../scripts/testing/lane-fixtures.mjs').catch(e=>{if(e.code!=='ERR_MODULE_NOT_FOUND')throw e;return {};});
const evidence=await import('../../../scripts/testing/lane-evidence.mjs').catch(e=>{if(e.code!=='ERR_MODULE_NOT_FOUND')throw e;return {};});
test('parallel lease holders use only the active lane and its shared state',async t=>{
const ctx=leaseContext(t),lane=ctx.lane;
const [a,b]=await Promise.all([api.leasePort(ctx.fixture,ctx),api.leasePort(ctx.fixture,ctx)]);
const bands=BANDS,[lo,hi]=bands[lane];
assert.ok(a>=lo&&a<=hi);assert.ok(b>=lo&&b<=hi);assert.notEqual(a,b);
assert.ok(bands.a[1]<bands.b[0]);assert.equal(bands.a[1]-bands.a[0],127);assert.equal(bands.b[1]-bands.b[0],127);
});
test('unrelated home changes do not become lane writes, and own-root references do',()=>{
assert.equal(typeof evidence.attributeEntries,'function','attribution is required');
const result=evidence.attributeEntries([{path:'/home/person/.claude/projects/other-session',link:null},{path:'/home/person/.cache/lane-abcdef123456',link:null},{path:'/home/person/.pi/item',link:'/mnt/storage/scratch/gt-ABCDEF/work'}],{nonce:'abcdef123456',tempRoot:'/mnt/storage/scratch/gt-ABCDEF',fixturePaths:[]});
assert.equal(result.owned.length,2);assert.equal(result.unrelated.length,1);
});
test('proof summaries refuse missing timings, non-overlap and a pair without a lane swap',()=>{
const solos=[{candidate:'one',wallMs:100},{candidate:'two',wallMs:100}];
const make=(lane,candidate,startMs=0,endMs=120)=>({lane,candidate,startMs,endMs,rc:0,clean:true});
const pairs=[[make('a','one'),make('b','two',5,125)],[make('b','one'),make('a','two',5,125)],[make('a','one'),make('b','two',5,125)]];
const result=evidence.summarizePairs(solos,pairs);assert.equal(result.length,3);assert.equal(result[0].makespanMs,125);assert.equal(result[0].runs[0].slowdown,1.2);assert.equal(result[0].throughput,1.6);
const bad=structuredClone(pairs);delete bad[0][0].endMs;assert.throws(()=>evidence.summarizePairs(solos,bad),/timing/);
const noSwap=structuredClone(pairs);noSwap[1]=structuredClone(pairs[0]);assert.throws(()=>evidence.summarizePairs(solos,noSwap),/swap/);
const serial=structuredClone(pairs);serial[0][1].startMs=130;serial[0][1].endMs=250;assert.throws(()=>evidence.summarizePairs(solos,serial),/overlap/);
});
import { BANDS } from '../../../scripts/testing/lane-state.mjs';
import { spawn,spawnSync } from 'node:child_process';
import { readFileSync,existsSync } from 'node:fs';
function leaseContext(t){
const lane=process.env.MOSAIC_GATE_LANE??'a',stateRoot=process.env.MOSAIC_GATE_STATE??'/mnt/storage/scratch/mosaic-gate-lanes',callbacks=[];
mkdirSync(stateRoot,{recursive:true,mode:0o700});
t.after(async()=>{for(const f of callbacks.reverse())await f();});
return {lane,stateRoot,fixture:{after:f=>callbacks.push(f)},cleanup:f=>callbacks.push(f)};
}
const delay=ms=>new Promise(r=>setTimeout(r,ms));
async function waitFor(check){for(let i=0;i<100;i++){if(check())return;await delay(20);}throw Error('fixture not ready');}
test('an exhausted single-port band refuses, and an occupied foreign listener survives the probe',async t=>{
const ctx=leaseContext(t),callbacks=[];
const port=await api.leasePort({after:f=>callbacks.push(f)},ctx),original=BANDS[ctx.lane];
const server=createServer();
try{
BANDS[ctx.lane]=[port,port];
await assert.rejects(api.leasePort(t,ctx),/exhausted/);
await new Promise((resolve,reject)=>{server.once('error',reject);server.listen(port,'127.0.0.1',resolve);});
// Release only our lock. The sentinel listener remains foreign to the next lease.
await callbacks.pop()();
await assert.rejects(api.leasePort(t,ctx),/exhausted/);assert.equal(server.listening,true);
}finally{BANDS[ctx.lane]=original;if(server.listening)await new Promise(r=>server.close(r));for(const f of callbacks)await f();}
});
test('parallel child allocators share real lane locks; a lease remains held through child shutdown',async t=>{
const ctx=leaseContext(t),root=mkdtempSync('/mnt/storage/scratch/gf-');
const modulePath=new URL('../../../scripts/testing/lane-fixtures.mjs',import.meta.url).href;
const code=`import test from 'node:test';import fs from 'node:fs';const {leasePort}=await import(${JSON.stringify(modulePath)});test('lease held until fixture child finishes',async t=>{const port=await leasePort(t,JSON.parse(process.env.LEASE_CONTEXT));fs.writeFileSync(process.env.LEASE_PORT_FILE,String(port));await new Promise(r=>{const timer=setInterval(()=>{if(fs.existsSync(process.env.LEASE_STOP_FILE)){clearInterval(timer);r()}},5)});});`;
const start=number=>{const file=join(root,`lease-${number}.test.mjs`),portFile=join(root,`port-${number}`),stopFile=join(root,`stop-${number}`);writeFileSync(file,code);const env={...process.env,LEASE_CONTEXT:JSON.stringify(ctx),LEASE_PORT_FILE:portFile,LEASE_STOP_FILE:stopFile};delete env.NODE_TEST_CONTEXT;const child=spawn(process.execPath,['--test','--test-concurrency=2','--test-reporter=tap',file],{env,stdio:['ignore','pipe','pipe']});child.output='';child.stderr.on('data',b=>child.output+=b);child.stdout.on('data',b=>child.output+=b);child.done=new Promise(r=>child.once('exit',code=>{child.exitedCode=code;r(code);}));child.portFile=portFile;child.stopFile=stopFile;return child;};
const children=[start(1),start(2)];ctx.cleanup(async()=>{try{for(const c of children)writeFileSync(c.stopFile,'stop');await Promise.all(children.map(c=>c.done));}finally{rmSync(root,{recursive:true,force:true});}});
await waitFor(()=>{for(const c of children)if(c.exitedCode!==undefined)throw Error(`allocator exited ${c.exitedCode}: ${c.output}`);return children.every(c=>existsSync(c.portFile));});
const ports=children.map(c=>Number(readFileSync(c.portFile,'utf8')));assert.notEqual(...ports);
const lock=join(ctx.stateRoot,'ports',ctx.lane,`${ports[0]}.lock`);
assert.equal(spawnSync('flock',['-n',lock,'true']).status,1);
writeFileSync(children[0].stopFile,'stop');assert.equal(await children[0].done,0,children[0].output);
assert.equal(spawnSync('flock',['-n',lock,'true']).status,0);
assert.equal(spawnSync('flock',['-n',join(ctx.stateRoot,'ports',ctx.lane,`${ports[1]}.lock`),'true']).status,1);
});
test('abrupt allocator death releases its holder without touching a foreign lease',async t=>{
const ctx=leaseContext(t),foreign=await api.leasePort(ctx.fixture,ctx),modulePath=new URL('../../../scripts/testing/lane-fixtures.mjs',import.meta.url).href;
const code=`const {leasePort}=await import(${JSON.stringify(modulePath)});const port=await leasePort({after:()=>{}},JSON.parse(process.argv[1]));console.log(port);setInterval(()=>{},1000);`;
const child=spawn(process.execPath,['--input-type=module','-e',code,JSON.stringify(ctx)],{stdio:['ignore','pipe','pipe']});const done=new Promise(r=>child.once('exit',r));ctx.cleanup(async()=>{child.kill('SIGKILL');await done;});
const port=await new Promise(r=>child.stdout.once('data',b=>r(Number(String(b).trim()))));assert.notEqual(port,foreign);
child.kill('SIGKILL');await done;
const lock=join(ctx.stateRoot,'ports',ctx.lane,`${port}.lock`);
await waitFor(()=>spawnSync('flock',['-n',lock,'true']).status===0);
assert.equal(spawnSync('flock',['-n',join(ctx.stateRoot,'ports',ctx.lane,`${foreign}.lock`),'true']).status,1);
});
test('fixture caches survive a replacement environment with a separate tmp subdirectory',async t=>{
const root=mkdtempSync('/mnt/storage/scratch/gf-');t.after(()=>rmSync(root,{recursive:true,force:true}));
const env=api.fixtureCacheEnv(root);assert.equal(env.TMPDIR,join(root,'tmp'));assert.equal(env.JITI_FS_CACHE,join(root,'jiti'));assert.equal(env.XDG_CACHE_HOME,join(root,'cache'));
const code="const fs=require('fs'),p=require('path');for(const k of ['TMPDIR','JITI_FS_CACHE','XDG_CACHE_HOME'])fs.writeFileSync(p.join(process.env[k],k),'fixture');if(process.env.HOME)process.exit(7)";
const result=spawnSync(process.execPath,['-e',code],{env,encoding:'utf8'});assert.equal(result.status,0,result.stderr);
for(const key of Object.keys(env))assert.equal(readFileSync(join(env[key],key),'utf8'),'fixture');
});
test('an unrelated session can append while a supervised step runs without becoming an attributed write',async t=>{
const root=mkdtempSync('/mnt/storage/scratch/gf-');t.after(()=>rmSync(root,{recursive:true,force:true}));
const home=join(root,'home'),out=join(root,'out'),file=join(home,'.claude','other-session');mkdirSync(join(home,'.claude'),{recursive:true});mkdirSync(out);writeFileSync(file,'before');
const before=evidence.homeMetadata(home),prior=new Map(before.map(e=>[e.path,JSON.stringify(e)]));
const {executeSteps}=await import('../../../scripts/testing/gate-lane.mjs');
const barrier=join(root,'step-running');
const append=spawn(process.execPath,['-e',"const fs=require('fs');const deadline=setTimeout(()=>{console.error('append barrier timed out');process.exit(8)},5000);const timer=setInterval(()=>{if(fs.existsSync(process.argv[2])){clearInterval(timer);clearTimeout(deadline);fs.appendFileSync(process.argv[1],'unrelated append')}},5)",file,barrier],{stdio:'ignore'});const appended=new Promise(r=>append.once('exit',r));t.after(()=>append.kill('SIGKILL'));
let appendedDuringStep=false;
const stepCode="const fs=require('fs');fs.writeFileSync(process.argv[1],'running');const deadline=setTimeout(()=>{console.error('session append timed out');process.exit(8)},5000);const timer=setInterval(()=>{if(fs.readFileSync(process.argv[2],'utf8').includes('unrelated append')){clearInterval(timer);clearTimeout(deadline)}},5)";
const result=await executeSteps([{name:'wait',cwd:root,argv:[process.execPath,'-e',stepCode,barrier,file]}],{out,env:process.env,record:r=>{if(r.kind==='step')appendedDuringStep=readFileSync(file,'utf8').includes('unrelated append');}});
assert.equal(await appended,0);assert.equal(result.rc,0);assert.equal(appendedDuringStep,true);
const changed=evidence.homeMetadata(home).filter(e=>prior.get(e.path)!==JSON.stringify(e));assert.ok(changed.some(e=>e.path===file));
const attributed=evidence.attributeEntries(changed,{nonce:'abcdef123456',tempRoot:join(root,'temp'),fixturePaths:[]});assert.equal(attributed.owned.length,0);assert.ok(attributed.unrelated.length>0);
});
@@ -0,0 +1,80 @@
import test from 'node:test';import assert from 'node:assert/strict';
import { mkdtempSync,rmSync,writeFileSync,mkdirSync,cpSync } from 'node:fs';import { join } from 'node:path';
const image=await import('../../../scripts/testing/lane-image.mjs').catch(e=>{if(e.code!=='ERR_MODULE_NOT_FOUND')throw e;return {};});
const docker=await import('../../../scripts/testing/lane-docker.mjs').catch(e=>{if(e.code!=='ERR_MODULE_NOT_FOUND')throw e;return {};});
test('a changed COPY input or actual image ID refuses compatible image sharing',t=>{
assert.equal(typeof image.imageFingerprint,'function','image fingerprint required');
const root=mkdtempSync('/mnt/storage/scratch/gi-');t.after(()=>rmSync(root,{recursive:true,force:true}));
for(const f of ['Containerfile','package.json','package-lock.json'])cpSync(new URL('../../../'+f,import.meta.url),join(root,f));
writeFileSync(join(root,'RELEASE'),'0.0.1\n');
for(const d of ['contracts','src','adapters']){mkdirSync(join(root,d));writeFileSync(join(root,d,'one'),'same');}
const digest=image.imageFingerprint(root),id='sha256:'+'a'.repeat(64);const receipt={inputDigest:digest,imageId:id,tag:'mosaic-poc-agent:0.85.1-r0.0.1',baseDigest:'node@sha256:'+'b'.repeat(64),platform:'linux/amd64'};
const inspect=()=>({Id:id,Os:'linux',Architecture:'amd64'});
assert.equal(image.verifyImage(receipt,root,inspect).imageId,id);
assert.throws(()=>image.verifyImage(receipt,root,()=>({...inspect(),Id:'sha256:'+'c'.repeat(64)})),/image ID/);
writeFileSync(join(root,'adapters','one'),'different');assert.throws(()=>image.verifyImage(receipt,root,inspect),/inputs/);
});
test('warm Docker guard disables implicit build and pull and refuses image mutation',()=>{
assert.equal(typeof docker.guardDocker,'function','Docker guard required');
const run={project:'mg197-a-abc',tag:'mosaic-poc-agent:fixture'};
assert.deepEqual(docker.guardDocker(['compose','run','--rm','-T','mosaic-agent','prompt'],run),['compose','run','--no-build','--pull','never','--rm','-T','mosaic-agent','prompt']);
for(const args of [['build','.'],['image','prune'],['compose','build'],['tag','a','b'],['network','rm','foreign']])assert.throws(()=>docker.guardDocker(args,run),/refused/);
});
test('exclusive image preparation records actual image ID and build wall time and requires a grant',t=>{
assert.equal(typeof image.prepareImage,'function','prepareImage is required');
const root=mkdtempSync('/mnt/storage/scratch/gi-');t.after(()=>rmSync(root,{recursive:true,force:true}));
for(const f of ['Containerfile','package.json','package-lock.json'])cpSync(new URL('../../../'+f,import.meta.url),join(root,f));
writeFileSync(join(root,'RELEASE'),'0.0.1\n');for(const d of ['contracts','src','adapters'])mkdirSync(join(root,d));
const calls=[];const invoke=(args)=>{calls.push(args);if(args[0]==='compose')return '';return JSON.stringify(args[2]==='node:24-bookworm-slim'?{RepoDigests:['node@sha256:'+'b'.repeat(64)]}:{Id:'sha256:'+'a'.repeat(64),Os:'linux',Architecture:'amd64'});};
assert.throws(()=>image.prepareImage(root,{grant:'',tempRoot:root},invoke),/grant/);
const result=image.prepareImage(root,{grant:'fixture',tempRoot:root},invoke);assert.equal(result.imageId,'sha256:'+'a'.repeat(64));assert.equal(result.builds,1);assert.ok(result.buildWallMs>=0);assert.ok(calls.some(x=>x.join(' ')==='compose build'));
});
test('cleanup refuses a foreign Docker label and removes only an exact owned project',()=>{
const project='mg197-a-abcdef123456',id='123456789abc';let removed=false,label='foreign';
const fake=(args)=>{if(args[0]==='ps')return removed?'':id;if(args[0]==='network')return '';if(args[0]==='container'&&args[1]==='inspect')return label;if(args[0]==='container'&&args[1]==='rm'){removed=true;return '';}throw Error('unexpected command');};
// The injectable executor exercises ownership checks without touching Docker.
assert.throws(()=>docker.cleanupProject({project},fake),/foreign/);assert.equal(removed,false);
label=project;assert.deepEqual(docker.cleanupProject({project},fake).removed,[{kind:'container',id}]);assert.equal(removed,true);
});
function imageFixture(t){
const root=mkdtempSync('/mnt/storage/scratch/gi-');t.after(()=>rmSync(root,{recursive:true,force:true}));
for(const f of ['Containerfile','package.json','package-lock.json'])cpSync(new URL('../../../'+f,import.meta.url),join(root,f));
writeFileSync(join(root,'RELEASE'),'0.0.1\n');for(const d of ['contracts','src','adapters'])mkdirSync(join(root,d));
const receipt={inputDigest:image.imageFingerprint(root),imageId:'sha256:'+'a'.repeat(64),tag:'mosaic-poc-agent:0.85.1-r0.0.1',baseDigest:'node@sha256:'+'b'.repeat(64),platform:'linux/amd64'};
return {root,receipt};
}
test('missing receipt and missing image refuse without permitting a gate',t=>{
const {root,receipt}=imageFixture(t);assert.throws(()=>image.verifyImage(null,root),/provenance receipt/);
assert.throws(()=>image.verifyImage(receipt,root,()=>{throw Error('No such image');}),/No such image/);
assert.throws(()=>image.verifyImage({...receipt,tag:'mosaic-poc-agent:wrong'},root,()=>({Id:receipt.imageId,Os:'linux',Architecture:'amd64'})),/release identity/);
});
test('a tag changed between supervised steps fails the next boundary and suppresses gate success',async t=>{
const {root,receipt}=imageFixture(t),out=join(root,'out');mkdirSync(out);
let actual=receipt.imageId;const inspect=()=>({Id:actual,Os:'linux',Architecture:'amd64'}),events=[];
const {executePinnedSteps}=await import('../../../scripts/testing/gate-lane.mjs');
await assert.rejects(executePinnedSteps([
{name:'first',cwd:root,argv:[process.execPath,'-e','process.exit(0)']},
{name:'second',cwd:root,argv:[process.execPath,'-e','process.exit(0)']}
],{out,env:process.env,record:r=>events.push(r)}, {receipt,checkout:root,inspect,afterStep:()=>actual='sha256:'+'c'.repeat(64)}),/image ID/);
assert.equal(events.filter(e=>e.kind==='step').length,1);
assert.equal(events.some(e=>e.kind==='terminal'&&e.rc===0),false);
});
test('mutable-base rebuilds with identical files have distinct image IDs; cold and warm counters stay separate',t=>{
const {root}=imageFixture(t);let builds=0,id='sha256:'+'a'.repeat(64);
const fake=args=>{if(args[0]==='compose'){builds++;return 'fixture build';}return JSON.stringify(args[2]==='node:24-bookworm-slim'?{RepoDigests:['node@sha256:'+'b'.repeat(64)]}:{Id:id,Os:'linux',Architecture:'amd64'});};
const cold=image.prepareImage(root,{grant:'fixture-cold',tempRoot:root},fake);
assert.equal(cold.builds,1);assert.ok(cold.buildWallMs>=0);assert.equal(builds,1);
for(let i=0;i<3;i++)image.verifyImage(cold,root,()=>({Id:id,Os:'linux',Architecture:'amd64'}));
assert.equal(builds,1,'warm verification builds zero images');
id='sha256:'+'c'.repeat(64);const rebuilt=image.prepareImage(root,{grant:'fixture-rebuild',tempRoot:root},fake);
assert.equal(rebuilt.inputDigest,cold.inputDigest);assert.equal(rebuilt.builds,1);assert.equal(builds,2);
assert.throws(()=>image.verifyImage(cold,root,()=>({Id:rebuilt.imageId,Os:'linux',Architecture:'amd64'})),/image ID/);
});
test('warm pinned execution records zero builds and checks the tag again after its final step',async t=>{
const {root,receipt}=imageFixture(t),out=join(root,'out');mkdirSync(out);let id=receipt.imageId;
const {executePinnedSteps}=await import('../../../scripts/testing/gate-lane.mjs');
const options={out,env:process.env,record:()=>{}},inspect=()=>({Id:id,Os:'linux',Architecture:'amd64'});
const result=await executePinnedSteps([{name:'warm',cwd:root,argv:[process.execPath,'-e','process.exit(0)']}],options,{receipt,checkout:root,inspect});
assert.equal(result.rc,0);assert.equal(result.imageBuilds,0);
await assert.rejects(executePinnedSteps([{name:'last',cwd:root,argv:[process.execPath,'-e','process.exit(0)']}],options,{receipt,checkout:root,inspect,afterStep:()=>id='sha256:'+'c'.repeat(64)}),/image ID/);
});
+124
View File
@@ -0,0 +1,124 @@
import * as fsExtra from 'node:fs';
import * as childModule from 'node:child_process';
import test from 'node:test';
import assert from 'node:assert/strict';
import { mkdtempSync, rmSync, writeFileSync, readFileSync } from 'node:fs';
import { join } from 'node:path';
const api=await import('../../../scripts/testing/lane-state.mjs');
function fixture(t){const root=mkdtempSync('/mnt/storage/scratch/gs-');t.after(()=>rmSync(root,{recursive:true,force:true}));return root;}
test('unfinished dead-owner runs quarantine their lane and a recovery record clears only that run',t=>{
assert.equal(typeof api.appendRecord,'function','appendRecord is required');
const root=fixture(t),dir=join(root,'runs','one');
api.appendRecord(dir,{kind:'start',lane:'a',owner:{pid:123456789,start:'1',boot:'fixture'}});
assert.equal(api.status(root,{processes:[]}).a.state,'QUARANTINED');
api.appendRecord(dir,{kind:'recovered',clean:true});
assert.equal(api.status(root,{processes:[]}).a.state,'CLEAR');
});
test('a legacy landing gate beside a lane stays legacy even with the same parent',t=>{
assert.equal(typeof api.appendRecord,'function');
const root=fixture(t);const owner={pid:123,start:'5',boot:'b',state:'S',ppid:1,argv:['node','gate-lane.mjs']};
api.appendRecord(join(root,'runs','one'),{kind:'start',lane:'a',owner});
const procs=[owner,{pid:124,start:'6',boot:'b',state:'S',ppid:1,argv:['node','/fixture/landing.mjs','gate','/fixture/wt','/fixture/logs']}];
const result=api.status(root,{processes:procs});assert.equal(result.a.state,'BUSY');assert.equal(result.b.state,'CLEAR');assert.deepEqual(result.legacy,[{pid:124,start:'6'}]);
});
test('the shell status entry reads scratch receipts and reports legacy processes without mutating old slots',t=>{
const root=fixture(t);const {spawnSync}=requireChild();
const r=spawnSync('bash',[new URL('../../../scripts/testing/gate-lane.sh',import.meta.url).pathname,'status','--state-root',root],{encoding:'utf8'});
assert.equal(r.status,0,r.stderr);const state=JSON.parse(r.stdout);assert.equal(state.a.state,'CLEAR');assert.equal(state.b.state,'CLEAR');assert.ok(Array.isArray(state.legacy));
});
function requireChild(){return childModule;}
test('recovery supersedes an unclean terminal without rewriting either receipt',t=>{
const root=fixture(t),dir=join(root,'runs','one');api.appendRecord(dir,{kind:'start',lane:'a',owner:{pid:99999999,start:'1',boot:'fixture'}});api.appendRecord(dir,{kind:'terminal',clean:false});api.appendRecord(dir,{kind:'recovered',clean:true});assert.equal(api.status(root,{processes:[]}).a.state,'CLEAR');assert.equal(api.records(dir).length,3);
});
test('the wrapper refuses a held lane lock and preserves its inode',async t=>{
const root=fixture(t),file=join(root,'a.lock');const {spawn,spawnSync}=childModule;
const holder=spawn('flock',['-x',file,'bash','-c','echo ready; read -r done'],{stdio:['pipe','pipe','pipe']});const exited=new Promise(r=>holder.once('exit',r));await new Promise(r=>holder.stdout.once('data',r));
const {statSync}=await import('node:fs');const ino=statSync(file).ino;
t.after(async()=>{holder.stdin.end('\n');await exited;});
const r=spawnSync('bash',[new URL('../../../scripts/testing/gate-lane.sh',import.meta.url).pathname,'run','--state-root',root,'--lane','a','--grant','fixture'],{encoding:'utf8'});
assert.equal(r.status,2);assert.match(r.stderr,/lock busy/);assert.equal(statSync(file).ino,ino);
});
test('a held kernel lock without a start receipt is BUSY rather than CLEAR',async t=>{
const root=fixture(t),file=join(root,'a.lock');const holder=childModule.spawn('flock',['-x',file,'bash','-c','echo ready; read -r done'],{stdio:['pipe','pipe','pipe']});const exited=new Promise(r=>holder.once('exit',r));await new Promise(r=>holder.stdout.once('data',r));t.after(async()=>{holder.stdin.end('\n');await exited;});
assert.equal(api.status(root,{processes:[]}).a.state,'BUSY');
});
test('SIGKILL leaves a quarantined run and locked recovery publishes a new clean receipt',async t=>{
const root=fixture(t),temp=mkdtempSync('/mnt/storage/scratch/gt-');t.after(()=>rmSync(temp,{recursive:true,force:true}));
const child=childModule.spawn(process.execPath,[new URL('./fixtures/gate-lane-child.mjs',import.meta.url).pathname,'wait',join(temp,'started')],{stdio:['ignore','pipe','pipe']});
const exited=new Promise(r=>child.once('exit',r));await new Promise(r=>child.stdout.once('data',r));
const id='abcdef123456',dir=join(root,'runs',id),docker=join(root,'docker-fixture');writeFileSync(docker,'#!/bin/sh\nexit 0\n',{mode:0o700});
api.appendRecord(dir,{kind:'start',lane:'a',owner:api.identity(child.pid),tempRoot:temp,project:`mg197-a-${id}`,docker});
child.kill('SIGKILL');await exited;assert.equal(api.status(root,{processes:[]}).a.state,'QUARANTINED');
const r=childModule.spawnSync('bash',[new URL('../../../scripts/testing/gate-lane.sh',import.meta.url).pathname,'recover','--state-root',root,'--run-id',id,'--grant','fixture-recover'],{encoding:'utf8'});
assert.equal(r.status,0,r.stderr);assert.equal(api.status(root,{processes:[]}).a.state,'CLEAR');assert.equal(api.records(dir).filter(e=>e.kind==='start').length,1);assert.equal(api.records(dir).filter(e=>e.kind==='recovered').length,1);
});
test('a malformed receipt refuses status and an identity mismatch cannot look live',t=>{
const root=fixture(t),dir=join(root,'runs','one');api.appendRecord(dir,{kind:'start',lane:'a',owner:{pid:123,start:'old',boot:'b'}});
assert.equal(api.status(root,{processes:[{pid:123,start:'new',boot:'b',state:'S',ppid:1,argv:['node','owned'] }]}).a.state,'QUARANTINED');
writeFileSync(join(dir,'bad.json'),'{');assert.throws(()=>api.status(root,{processes:[]}),/invalid run receipt/);
});
test('a symlink lock is refused before acquiring or replacing its target',t=>{
const root=fixture(t),target=join(root,'foreign');writeFileSync(target,'untouched');const {symlinkSync}=fsExtra;symlinkSync(target,join(root,'a.lock'));
const r=childModule.spawnSync('bash',[new URL('../../../scripts/testing/gate-lane.sh',import.meta.url).pathname,'run','--state-root',root,'--lane','a','--grant','fixture'],{encoding:'utf8'});
assert.equal(r.status,2);assert.match(r.stderr,/ordinary file/);assert.equal(readFileSync(target,'utf8'),'untouched');
});
test('failed recovery keeps quarantine and never publishes a clean recovery receipt',async t=>{
const root=fixture(t),temp=mkdtempSync('/mnt/storage/scratch/gt-');t.after(()=>rmSync(temp,{recursive:true,force:true}));
const id='abcdef123456',dir=join(root,'runs',id),docker=join(root,'docker-fixture');writeFileSync(docker,'#!/bin/sh\nexit 1\n',{mode:0o700});
api.appendRecord(dir,{kind:'start',lane:'a',owner:{pid:99999999,start:'1',boot:'fixture'},tempRoot:temp,project:`mg197-a-${id}`,docker});
const r=childModule.spawnSync('bash',[new URL('../../../scripts/testing/gate-lane.sh',import.meta.url).pathname,'recover','--state-root',root,'--run-id',id,'--grant','fixture-recover'],{encoding:'utf8'});
assert.equal(r.status,2);assert.equal(api.status(root,{processes:[]}).a.state,'QUARANTINED');assert.equal(api.records(dir).some(e=>e.kind==='recovered'),false);
});
test('two shared image holders coexist, but exclusive preparation refuses their lock',async t=>{
const root=fixture(t),file=join(root,'image.lock');const holder=childModule.spawn('flock',['-s',file,'bash','-c','echo ready; read -r done'],{stdio:['pipe','pipe','pipe']});const exited=new Promise(r=>holder.once('exit',r));await new Promise(r=>holder.stdout.once('data',r));t.after(async()=>{holder.stdin.end('\n');await exited;});
const entry=new URL('../../../scripts/testing/gate-lane.sh',import.meta.url).pathname;
const shared=childModule.spawnSync('bash',[entry,'run','--state-root',root,'--lane','a','--grant','fixture'],{encoding:'utf8'});
assert.equal(shared.status,2);assert.doesNotMatch(shared.stderr,/lock busy|descriptor|not locked/);
const exclusive=childModule.spawnSync('bash',[entry,'prepare-image','--state-root',root,'--grant','fixture'],{encoding:'utf8'});
assert.equal(exclusive.status,2);assert.match(exclusive.stderr,/lock busy/);
});
const entry=new URL('../../../scripts/testing/gate-lane.sh',import.meta.url).pathname;
const childFixture=new URL('./fixtures/gate-lane-child.mjs',import.meta.url).pathname;
const sleep=ms=>new Promise(r=>setTimeout(r,ms));
async function until(check){for(let n=0;n<100;n++){if(check())return;await sleep(20);}throw Error('fixture barrier did not become ready');}
test('two simultaneous shell claims have one winner; supervisor retains lock FDs and ordinary children do not',async t=>{
const root=fixture(t),bin=join(root,'bin');fsExtra.mkdirSync(bin);const shim=join(bin,'node');
writeFileSync(shim,`#!/bin/sh\nif [ "$2" = --locked ]; then exec '${process.execPath}' '${childFixture}' lock-probe '${root}'; fi\nexec '${process.execPath}' "$@"\n`,{mode:0o700});
const env={...process.env,PATH:bin+':'+process.env.PATH};
const start=()=>{const p=childModule.spawn('bash',[entry,'run','--state-root',root,'--lane','a','--grant','fixture'],{env,stdio:['ignore','pipe','pipe']});p.done=new Promise(r=>p.once('exit',code=>r(code)));return p;};
const first=start(),second=start();t.after(()=>{first.kill('SIGKILL');second.kill('SIGKILL');});
await until(()=>fsExtra.existsSync(join(root,'ready')));
const all=api.runRecords(root);assert.equal(all.length,1);assert.equal(readFileSync(join(root,'out/fds.log'),'utf8'),'');
const winner=all[0].records.find(r=>r.kind==='start').owner.pid;assert.ok([first.pid,second.pid].includes(winner));
const loser=first.pid===winner?second:first;assert.equal(await loser.done,2);
const ino=fsExtra.statSync(join(root,'a.lock')).ino;
assert.equal(childModule.spawnSync('flock',['-n',join(root,'a.lock'),'true']).status,1);
const won=first.pid===winner?first:second;won.kill('SIGTERM');await won.done;
assert.equal(fsExtra.statSync(join(root,'a.lock')).ino,ino);
});
test('same checkout and a legacy land gate each refuse a claim on an otherwise free lane',async t=>{
const {activeConflicts}=await import('../../../scripts/testing/gate-lane.mjs');
const root=fixture(t),owner={pid:321,start:'1',boot:'test',state:'S',ppid:1,argv:['node','owned']};
api.appendRecord(join(root,'runs','one'),{kind:'start',lane:'a',checkout:'/fixture/candidate',owner});
const o={stateRoot:root,lane:'b',checkout:'/fixture/candidate'};
assert.throws(()=>activeConflicts(o,{processes:[owner]}),/checkout already owned/);
assert.throws(()=>activeConflicts({...o,checkout:'/fixture/other'},{processes:[owner,{pid:322,start:'2',boot:'test',state:'S',ppid:1,argv:['node','/fixture/landing.mjs','gate']}]}),/LEGACY BUSY/);
});
test('recovery kills a proven surviving child, preserves a foreign child and keeps quarantine if interrupted',async t=>{
const root=fixture(t),temp=mkdtempSync('/mnt/storage/scratch/gt-');t.after(()=>rmSync(temp,{recursive:true,force:true}));
const owned=childModule.spawn(process.execPath,[childFixture,'wait',join(temp,'child')],{cwd:temp,stdio:'ignore'});
const foreign=childModule.spawn(process.execPath,[childFixture,'wait',join(root,'foreign')],{stdio:'ignore'});
const ownedDone=new Promise(r=>owned.once('exit',r)),foreignDone=new Promise(r=>foreign.once('exit',r));t.after(async()=>{owned.kill('SIGKILL');foreign.kill('SIGKILL');await Promise.all([ownedDone,foreignDone]);});
await until(()=>fsExtra.existsSync(join(temp,'child'))&&fsExtra.existsSync(join(root,'foreign')));
const docker=join(root,'docker'),pidFile=join(root,'docker-pid'),id='abcdef123456',dir=join(root,'runs',id);
writeFileSync(docker,`#!/bin/sh\necho $$ > '${pidFile}'\nexec sleep 60\n`,{mode:0o700});
api.appendRecord(dir,{kind:'start',id,stateRoot:root,lane:'a',owner:{pid:99999999,start:'1',boot:'test'},tempRoot:temp,project:`mg197-a-${id}`,docker});
const recovery=childModule.spawn('bash',[entry,'recover','--state-root',root,'--run-id',id,'--grant','fixture'],{stdio:'ignore'});const done=new Promise(r=>recovery.once('exit',r));
await until(()=>fsExtra.existsSync(pidFile));const dockerOwner=api.identity(Number(readFileSync(pidFile,'utf8')));recovery.kill('SIGKILL');await done;
if(api.sameProcess(dockerOwner))process.kill(dockerOwner.pid,'SIGKILL');
await ownedDone;assert.equal(api.sameProcess(api.identity(foreign.pid)),true);
assert.equal(api.records(dir).some(r=>r.kind==='recovered'),false);assert.equal(api.status(root,{processes:[]}).a.state,'QUARANTINED');
writeFileSync(docker,'#!/bin/sh\nexit 0\n',{mode:0o700});
const retried=childModule.spawnSync('bash',[entry,'recover','--state-root',root,'--run-id',id,'--grant','fixture-resume'],{encoding:'utf8'});
assert.equal(retried.status,0,retried.stderr);assert.equal(api.status(root,{processes:[]}).a.state,'CLEAR');assert.equal(api.sameProcess(api.identity(foreign.pid)),true);
});
+105
View File
@@ -0,0 +1,105 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { mkdtempSync, mkdirSync, writeFileSync, symlinkSync, rmSync } from 'node:fs';
import { execFileSync } from 'node:child_process';
import { join } from 'node:path';
const api = await import('../../../scripts/testing/gate-lane.mjs').catch(e => {
if (e.code !== 'ERR_MODULE_NOT_FOUND') throw e;
return {};
});
function fixture(t) {
const root = mkdtempSync('/mnt/storage/scratch/gt-');
// Node appends six characters, keeping this root at 30 bytes.
t.after(() => rmSync(root, { recursive: true, force: true }));
const checkout = join(root, 'checkout'); mkdirSync(checkout);
for (const [f,text] of [['package.json','{}'],['RELEASE','0.0.1\n']]) writeFileSync(join(checkout,f),text);
execFileSync('git',['init','-q',checkout]);
execFileSync('git',['-C',checkout,'add','.']);
execFileSync('git',['-C',checkout,'-c','user.name=Fixture','-c','user.email=f@invalid','commit','-qm','fixture']);
symlinkSync('/mnt/storage/src/mosaic-stack/node_modules',join(checkout,'node_modules'));
return {lane:'a',candidate:execFileSync('git',['-C',checkout,'rev-parse','HEAD'],{encoding:'utf8'}).trim(),checkout,out:join(root,'out'),tempRoot:root,stateRoot:join(root,'state'),grant:'fixture-grant',hostFacts:{unprivilegedPortStart:1024,ephemeral:[32768,60999],listeners:[]}};
}
test('preflight refuses unsafe lane roots, missing grants and live port 9 before side effects', t => {
assert.equal(typeof api.validateRun,'function','validateRun is required');
const f=fixture(t);
assert.equal(api.validateRun(f).lane,'a');
for(const [patch,reason] of [[{lane:'c'},/lane/],[{grant:''},/grant/],[{tempRoot:'/tmp'},/tmp/],[{tempRoot:join(f.tempRoot,'deeper')},/30|missing/],[{candidate:'a'.repeat(40)},/candidate/],[{hostFacts:{...f.hostFacts,listeners:[{address:'0.0.0.0',port:9}]}},/port.9/],[{hostFacts:{...f.hostFacts,unprivilegedPortStart:0}},/privileged/]]) assert.throws(()=>api.validateRun({...f,...patch}),reason);
const link=join(f.tempRoot,'link');symlinkSync('/tmp',link);assert.throws(()=>api.validateRun({...f,tempRoot:link}),/tmp/);
writeFileSync(join(f.checkout,'RELEASE'),'changed');assert.throws(()=>api.validateRun(f),/dirty/);
});
test('discovery preserves all package and script steps while ignoring non-file shell names', t => {
const f=fixture(t);mkdirSync(join(f.checkout,'packages'));
for(const p of ['z','queue','a']){mkdirSync(join(f.checkout,'packages',p,'tests'),{recursive:true});writeFileSync(join(f.checkout,'packages',p,'tests','one.test.mjs'),'');}
mkdirSync(join(f.checkout,'scripts'));writeFileSync(join(f.checkout,'scripts','test-z.sh'),'');writeFileSync(join(f.checkout,'scripts','test-a.sh'),'');mkdirSync(join(f.checkout,'scripts','test-directory.sh'));
const steps=api.discoverSteps(f.checkout);
assert.deepEqual(steps.map(s=>s.name),['pkg-a','pkg-z','script-a','script-z']);
assert.deepEqual(steps[0].argv,['node','--test','--test-concurrency=2','tests/one.test.mjs']);
});
test('a failed step stops the gate and records its nonzero exit without running the next step',async t=>{
assert.equal(typeof api.executeSteps,'function','executeSteps is required');
const f=fixture(t),events=[],out=f.out;mkdirSync(out);const marker=join(f.tempRoot,'should-not-exist');
const result=await api.executeSteps([{name:'first',cwd:f.checkout,argv:['bash','-c','exit 7']},{name:'second',cwd:f.checkout,argv:['touch',marker]}],{out,env:process.env,record:r=>events.push(r)});
assert.equal(result.rc,7);assert.equal(events.filter(e=>e.kind==='step').length,1);assert.ok(events.find(e=>e.kind==='step').wallMs>=0);
const {existsSync}=await import('node:fs');assert.equal(existsSync(marker),false);
});
test('a cancelled step stops its process group and records failure',async t=>{
const f=fixture(t);mkdirSync(f.out);const controller=new AbortController(),events=[];
const child=new URL('./fixtures/gate-lane-child.mjs',import.meta.url).pathname;
const result=await api.executeSteps([{name:'child',cwd:f.checkout,argv:[process.execPath,child,'wait',join(f.tempRoot,'started')]}],{out:f.out,env:process.env,record:e=>{events.push(e);if(e.kind==='child')setTimeout(()=>controller.abort(),100);},signal:controller.signal});
assert.notEqual(result.rc,0);const owned=events.find(e=>e.kind==='child').owner;const {sameProcess}=await import('../../../scripts/testing/lane-state.mjs');assert.equal(sameProcess(owned),false);
});
test('step receipts retain pass fail skipped counts and hashes of their actual logs',async t=>{
const f=fixture(t);mkdirSync(f.out);const events=[];
await api.executeSteps([{name:'counts',cwd:f.checkout,argv:[process.execPath,'-e',"console.log('ℹ pass 2\\nℹ fail 0\\nℹ skipped 1')"]}],{out:f.out,env:process.env,record:r=>events.push(r)});
const step=events.find(e=>e.kind==='step');assert.deepEqual(step.counts,{pass:2,fail:0,skipped:1});assert.match(step.logSha256,/^[a-f0-9]{64}$/);
});
test('preflight rejects a real 31-byte root, missing dependency tree, occupied band and missing inventory',t=>{
const f=fixture(t),long=f.tempRoot+'x';mkdirSync(long,{mode:0o700});t.after(()=>rmSync(long,{recursive:true,force:true}));assert.equal(Buffer.byteLength(long),31);
assert.throws(()=>api.validateRun({...f,tempRoot:long}),/30 bytes/);
assert.throws(()=>api.validateRun({...f,hostFacts:{...f.hostFacts,ephemeral:[60000,62000]}}),/overlaps/);
assert.throws(()=>api.validateRun({...f,hostFacts:{...f.hostFacts,listeners:[{address:'127.0.0.1',port:61005}]}}),/live listener/);
assert.throws(()=>api.validateRun({...f,hostFacts:{}}),/inventory/);
rmSync(join(f.checkout,'node_modules'));assert.throws(()=>api.validateRun(f),/node_modules/);
});
test('preflight refuses canonical and builder checkouts and an existing output',t=>{
const f=fixture(t);
for(const checkout of ['/mnt/storage/src/mosaic-stack',new URL('../../../',import.meta.url).pathname])assert.throws(()=>api.validateRun({...f,checkout}),/canonical or builder/);
mkdirSync(f.out);assert.throws(()=>api.validateRun(f),/output already exists/);
});
test('both candidates must carry every approved launcher file',t=>{
const f=fixture(t),other=join(f.tempRoot,'second');mkdirSync(other);
for(const checkout of [f.checkout,other]){
const target=join(checkout,'scripts','testing');fsExtra.cpSync(new URL('../../../scripts/testing',import.meta.url),target,{recursive:true});
api.validateTooling(checkout);
}
fsExtra.appendFileSync(join(other,'scripts/testing/lane-state.mjs'),'\n// changed\n');
assert.throws(()=>api.validateTooling(other),/tooling differs/);
fsExtra.rmSync(join(f.checkout,'scripts/testing/lane-fixtures.mjs'));
assert.throws(()=>api.validateTooling(f.checkout),/tooling differs/);
});
test('the 92-byte socket example passes and an over-107-byte pathname refuses',()=>{
const root='/mnt/storage/scratch/gt-ABCDEF';assert.equal(Buffer.byteLength(root),30);
const socket=root+'/mosaic-cli-ABCDEF/data/launches/acme/r0123456789ab/tools.sock';
assert.equal(Buffer.byteLength(socket),92);assert.equal(stateApi.socketPath(socket),socket);
assert.throws(()=>stateApi.socketPath(socket+'x'.repeat(16)),/107 bytes/);
});
test('package/script duplicates stay and every supervised step clears NODE_TEST_CONTEXT',async t=>{
const f=fixture(t);mkdirSync(join(f.checkout,'packages/cli/tests'),{recursive:true});mkdirSync(join(f.checkout,'scripts'));
writeFileSync(join(f.checkout,'packages/cli/tests/repeated.test.mjs'),'');writeFileSync(join(f.checkout,'scripts/test-cli.sh'),'node --test packages/cli/tests/repeated.test.mjs\n');
const steps=api.discoverSteps(f.checkout);assert.deepEqual(steps.map(s=>s.name),['pkg-cli','script-cli']);
const original=process.env.NODE_TEST_CONTEXT;process.env.NODE_TEST_CONTEXT='parent-context';
let env;try{env=api.cleanEnv(f,{project:'fixture'},'/fake/docker');}finally{if(original===undefined)delete process.env.NODE_TEST_CONTEXT;else process.env.NODE_TEST_CONTEXT=original;}
assert.equal(env.NODE_TEST_CONTEXT,undefined);assert.equal(env.HOME,process.env.HOME);
mkdirSync(f.out);const events=[];
for(const s of steps)s.argv=[process.execPath,'-e',"if(process.env.NODE_TEST_CONTEXT)process.exit(9);console.log('cleared')"];
const result=await api.executeSteps(steps,{out:f.out,env,record:e=>events.push(e)});
assert.equal(result.rc,0);assert.equal(events.filter(e=>e.kind==='step').length,2);
});
test('scoped ss loopback and wildcard port 9 listeners refuse',t=>{
const f=fixture(t);
for(const address of ['127.0.0.1%lo','0.0.0.0%lo','::%lo','::ffff:127.0.0.1%lo'])assert.throws(()=>api.validateRun({...f,hostFacts:{...f.hostFacts,listeners:[{address,port:9}]}}),/port 9/);
});
import * as fsExtra from 'node:fs';
import * as stateApi from '../../../scripts/testing/lane-state.mjs';
+74
View File
@@ -0,0 +1,74 @@
// Row 234 (#1740, r6 E4): `mosaic hosts status|check|remove`. A fake transport records the calls; no bus
// runs here. The bus side of each verb is in packages/bus/tests/grant-enroll.test.mjs.
import test from "node:test";
import assert from "node:assert/strict";
import { spawnSync } from "node:child_process";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { main } from "../src/cli.mjs";
import { io, tmp } from "./helpers.mjs";
const MOSAIC = fileURLToPath(new URL("../../../scripts/mosaic", import.meta.url));
const ROWS = [
{ host: "web1", address: "10.1.1.53", port: 22, user: "deploy", sudo: true, state: "ready", hostKeys: ["SHA256:a"],
installerFingerprint: "SHA256:b", checkedAt: "2026-10-21T00:00:00.000Z", hostLineLeft: [] },
{ host: "web2", address: "10.1.1.54", port: 2222, user: "app", sudo: false, state: "pending-install", hostKeys: ["SHA256:c"],
installerFingerprint: "SHA256:d", checkedAt: null, hostLineLeft: ["mosaic-" + "0".repeat(32)] },
];
function deps(calls) {
const answer = (who) => (business) => async (verb, args) => {
calls.push({ who, business, verb, args });
if (verb === "hosts.status") return ROWS;
if (verb === "hosts.check") return { host: args.host, state: "ready", checkedAt: "2026-10-21T00:00:00.000Z" };
return { host: args.host, removed: true };
};
return { system: { dataRoot: "/nonexistent" }, transport: answer("human"), testerTransport: answer("tester") };
}
test("row 234: hosts status prints one line per host, and --json the rows", async () => {
const calls = [];
const x = io();
await main(["hosts", "--business", "acme"], x, deps(calls));
assert.deepEqual(calls, [{ who: "human", business: "acme", verb: "hosts.status", args: {} }]);
assert.equal(x.out.text,
"web1\tready\[email protected]:22\tsudo\t2026-10-21T00:00:00.000Z\n" +
`web2\tpending-install\t[email protected]:2222\tno-sudo\tunchecked\tline left: mosaic-${"0".repeat(32)}\n`);
const y = io();
await main(["hosts", "status", "--json", "--tester", "--business", "acme"], y, deps(calls));
assert.deepEqual(JSON.parse(y.out.text), ROWS);
assert.equal(calls[1].who, "tester");
});
test("row 234: hosts check and remove send the host id, as the human or the tester", async () => {
const calls = [];
const x = io();
await main(["hosts", "check", "web1", "--business", "acme"], x, deps(calls));
await main(["hosts", "remove", "web2", "--tester", "--business", "acme"], x, deps(calls));
assert.deepEqual(calls.map(({ who, verb, args }) => ({ who, verb, args })), [
{ who: "human", verb: "hosts.check", args: { host: "web1" } },
{ who: "tester", verb: "hosts.remove", args: { host: "web2" } },
]);
});
test("row 234: a malformed hosts command is a usage error and calls nothing", async () => {
const calls = [];
for (const argv of [["hosts", "check"], ["hosts", "remove"], ["hosts", "status", "web1"], ["hosts", "check", "web1", "web2"],
["hosts", "purge", "web1"], ["hosts", "check", "web1", "--json"], ["hosts", "--business"], ["hosts", "--force"]])
await assert.rejects(main([...argv], io(), deps(calls)), (e) => (assert.equal(e.exitCode, 4, argv.join(" ")), assert.match(e.message, /mosaic hosts/), true));
assert.equal(calls.length, 0);
});
test("row 234: a bus refusal exits 2 with its code", async () => {
const refusing = { system: { dataRoot: "/nonexistent" }, transport: () => async () => { throw new Error("host-in-use"); } };
await assert.rejects(main(["hosts", "remove", "web1", "--business", "acme"], io(), refusing), (e) => (assert.equal(e.exitCode, 2), assert.match(e.message, /host-in-use/), true));
});
test("row 234: scripts/mosaic sends hosts to packages/cli, not the seat CLI", (t) => {
const root = tmp(t, "mosaic-cli-hosts-route-");
const env = { PATH: process.env.PATH, HOME: join(root, "home"), MOSAIC_CONFIG: join(root, "none.json") };
const r = spawnSync(MOSAIC, ["hosts", "purge", "web1"], { env, encoding: "utf8" });
assert.equal(r.status, 4);
assert.match(r.stderr, /^mosaic: mosaic hosts \[status\]/);
assert.doesNotMatch(r.stderr, /mosaic launch </);
});
+39 -1
View File
@@ -5,7 +5,7 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { spawnSync } from "node:child_process";
import { chmodSync, lstatSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { chmodSync, existsSync, lstatSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { connect } from "node:net";
import { tmpdir } from "node:os";
import { join } from "node:path";
@@ -298,3 +298,41 @@ test("sage-1440: the publishResult poll is bounded: publish-timeout, filed once,
assert.equal(w.events("action.allowed").filter((e) => e.body.action === "git.push.remote" && e.body.operation !== "decision.raise").length, 0);
assert.ok(w.logs.some((l) => l.includes("publish-timeout")));
});
test("row 242: a retry after publish-timeout refuses publish-busy while the bus process's push runs; the late push is recorded and the decision pushes once", async (t) => {
const w = await setup(t, { publishDeadlineMs: 150 });
w.bus.credentials.issue("demo/coder", "gitea", TOKEN, "2099-01-01");
// The fork holds the push in pre-receive until the test writes `release`; `held` says it's there.
const held = join(w.root, "held");
const release = join(w.root, "release");
writeFileSync(join(w.forkRepo, "hooks", "pre-receive"), `#!/bin/sh\n: > '${held}'\nwhile [ ! -e '${release}' ]; do sleep 0.05; done\ncat > /dev/null\n`, { mode: 0o755 });
const until = async (what, ok) => {
for (const end = Date.now() + 10_000; !ok(); ) {
if (Date.now() > end) assert.fail(`timed out waiting for ${what}`);
await new Promise((r) => setTimeout(r, 20));
}
};
const allowed = () => w.events("action.allowed").filter((e) => e.body.action === "git.push.remote" && e.body.operation !== "decision.raise");
// A failed assert must still let the hook end: teardown waits for the fake Gitea, which waits for the push.
try {
const d = w.approve();
assert.deepEqual(await w.publish(d), { ok: false, error: "publish-timeout" });
await until("the push to reach pre-receive", () => existsSync(held));
// The socket gave up, but the push still runs: the decision is held, and nothing is filed yet.
assert.deepEqual(await w.publish(d, { branch: "other" }), { ok: false, error: "publish-busy" });
assert.deepEqual(w.events("action.refused").map((e) => e.body.code), ["publish-busy"]);
assert.equal(allowed().length, 0);
writeFileSync(release, "");
// The push reached the fork, so it's recorded, which spends the decision.
await until("the late push to be recorded", () => allowed().length === 1);
assert.equal(allowed()[0].body.ref, "refs/heads/coder/feat");
assert.equal(allowed()[0].body.commit, w.work);
assert.deepEqual(await w.publish(d, { branch: "other" }), { ok: false, error: "decision-consumed" });
assert.equal(w.gitea.requests.filter((r) => r.path.endsWith("/git-receive-pack")).length, 1);
assert.equal(git(w.root, `--git-dir=${w.forkRepo}`, "for-each-ref", "--format=%(refname)", "refs/heads/coder/"), "refs/heads/coder/feat");
assert.deepEqual(w.events("action.refused").map((e) => e.body.code), ["publish-busy", "decision-consumed"]);
} finally {
writeFileSync(release, "");
}
assert.equal(JSON.stringify([w.events("action.allowed"), w.events("action.refused"), w.logs]).includes(TOKEN), false);
});
+56 -1
View File
@@ -1,3 +1,5 @@
import { leasePort } from '../../../scripts/testing/lane-fixtures.mjs';
import { after as laneAfter } from 'node:test';
// scripts/stack-release.sh (D1, #1659). Design, the T table and the mutants:
// agents/darkwing/work/queue-162/BUILD.md. Every run uses a temp data root, temp units and stand-in
// systemctl, journalctl and systemd-analyze on PATH; nothing here reads ~/.mosaic-dev or a live unit.
@@ -916,6 +918,8 @@ test('T46 row 181 item 4: every table in the bus schema is in exactly one of fou
const hasAt=(n)=>db.prepare(`SELECT count(*) AS n FROM pragma_table_info(?) WHERE name='at'`).get(n).n===1;
const always={decisions:'decision.raise writes action.allowed in the same transaction'};
const untimed={
host_grants:'no time column; row 234: each write goes in one transaction with a grant.* events row, which is counted',
hosts:'no time column; row 234: each write goes in one transaction with a grant.* events row, which is counted',
meta:'key and value only; its one UPDATE is the migration step',
message_threads:'no time column; each row goes with a messages row, which is counted',
sqlite_sequence:'SQLite\'s own AUTOINCREMENT table',
@@ -1681,6 +1685,14 @@ test('T57 row 200: activate --sso, its refusals before any stop, and the rollbac
assert.ok(stepText(a).includes(`Z drop --sso; ${B}'s Console does not support it, so the Console flags come from ${set}/${show}\n`));
await noLeak(a);
});
await t.test('row 238 (#1744): on a named instance the sso-config first step names --instance NAME in both commands',async(t)=>{
const e=await mk(t),a=await named(e,'alpha');await seed(a);
assert.equal(JSON.parse(readFileSync(a.cfg,'utf8')).console,undefined);
await refused(a,['activate',C1,'acme','--sso'],'sso-config',`${C1}'s Console would refuse its --sso start: there is no console.sso in the system config; activate ${C1} --instance alpha without --sso first, then write console.sso, then run activate ${C1} --instance alpha --sso`);
const out=await planOnly(a,3,'activate',C1,'acme','--sso');
assert.ok(out.includes(`refuses sso-config: ${C1}'s Console would refuse its --sso start: there is no console.sso in the system config; activate ${C1} --instance alpha without --sso first, then write console.sso, then run activate ${C1} --instance alpha --sso`),out.join('\n'));
await noLeak(a);
});
});
// --- rehearse (D2, row 163; the brief's "Stack rehearsal on a scratch root") ---
@@ -1774,7 +1786,7 @@ for(const [id,v,froms] of [[B,'3g',['3f']],[C,'3h',['3f','3g']],[A,'3g',['3f']],
}
const TESTER='7499';
const freePort=async()=>{const s=createServer();s.listen(0,'127.0.0.1');await once(s,'listening');const p=String(s.address().port);await new Promise((r)=>s.close(r));return p};
const freePort=async()=>String(await leasePort({after:laneAfter},{closed:true}));
const rh=(e,...p)=>join(e.env.HOME,...p);
const rfault=(e,name)=>{mkdirSync(rh(e,'faults'),{recursive:true});writeFileSync(rh(e,'faults',name),'')};
@@ -2062,6 +2074,49 @@ test('R15 a rehearsal stopped by TERM, INT or HUP fails signal and still stops t
}
});
// --- Row 233 (#1735): --backup last, the newest backup the log records (sage-1650, sage-1653) ---
// The log's lines, as the runs write them, with only the fields from_log reads set.
function rlog(e,...ls){
mkdirSync(e.state,{recursive:true});
writeFileSync(e.log,ls.map(([op,to,backup],i)=>JSON.stringify({at:`2026-10-20T00:00:0${i}Z`,op,from:null,to,backup,activation:`20261020T00000${i}Z-${to.slice(0,12)}`,result:'ok'})+'\n').join(''));
}
test('row 233: --backup last takes the newest backup the log records; a later line with backup null does not hide it, and the store line names it',async(t)=>{
const e=await rmk(t);
// An older backup that isn't on disk: a run that took it would fail backup-missing.
rlog(e,['activate',C1,`20261019T000000Z-${C1.slice(0,12)}`],['activate',B,e.act],['rollback',B,null]);
const l0=readFileSync(e.log,'utf8'),before=untouched(e);
const r=await rehearse(e,B,'--backup','last','--port',await freePort(),'--timeout','10');
assert.equal(r.code,0,r.out+r.err);
assert.deepEqual(r.names,['PASS backup-last',...PRE,'PASS migration','PASS gate','PASS no-session',...POST]);
assert.ok(r.out.startsWith(`PASS backup-last: the log's newest backup is ${e.act}\n`),r.out);
assert.ok(r.out.includes(`\nBACKUP_STORE ${e.act} schema 3g\n`),r.out);
assert.equal(readFileSync(e.log,'utf8'),l0);assert.equal(untouched(e),before);
// The rehearsal's own steps.log has the PASS and FAIL lines only.
assert.deepEqual(lines(join(r.dir,'steps.log')).map((l)=>l.replace(/^\S+ /,'')),r.out.split('\n').filter((l)=>/^(PASS|FAIL) |^REHEARSAL_/.test(l)));
});
test('row 233: --backup last with no backup in the log, or no log, exits 3 with no-backup and makes no rehearsal directory',async(t)=>{
const e=await rmk(t);
for(const how of ['no log','adopt only']){
if(how==='adopt only')rlog(e,['adopt',B,null]);
const r=await rehearse(e,B,'--backup','last','--port',await freePort(),'--timeout','3');
assert.equal(r.code,3,how+r.out+r.err);
assert.equal(r.out,'FAIL no-backup: the log records no backup\nREHEARSAL_FAIL\n',how);
assert.equal(r.dir,null);assert.deepEqual(rDirs(e),[],how);assert.ok(!existsSync(join(e.data,'rehearsals')),how);
}
});
test('row 233: --backup last refuses a newest backup that is not an activation id, with exit 1',async(t)=>{
const e=await rmk(t);
rlog(e,['activate',B,'../../etc']);
const r=await rehearse(e,B,'--backup','last','--port',await freePort(),'--timeout','3');
assert.equal(r.code,1,r.out+r.err);
assert.equal(r.out,'FAIL backup-unreadable: the log\'s newest backup is not an activation id\nREHEARSAL_FAIL\n');
assert.deepEqual(rDirs(e),[]);
});
// --- Row 190 (#1688): stack instances on one host. Design and the T-I table:
// agents/darkwing/work/queue-190/DESIGN.md. T-I1, T-I2, T-I5, T-I10 and the loader half of T-I7 are in
// stack-instances.test.mjs; these need the units, the stand-ins and stack-release.sh. Every existing
+464
View File
@@ -0,0 +1,464 @@
// scripts/stack-upgrade.sh (row 233, I3, #1735): root's upgrade and rollback of a stack whose units are
// root's, driving stack-release.sh --units external (row 232). Design: r6 sections 4 and 7,
// agents/darkwing/work/queue-195/DESIGN.md, as the I3 brief changes them; deviations:
// agents/darkwing/work/queue-233/BUILD.md. The stand-ins and the module fixture are
// stack-release-fixture.mjs's. Root's sudo, case2-unit.sh render and row 240 hostcheck are stand-ins
// here; the stack's user's steps are the real stack-release.sh from the user's clone, except the
// rehearsal, which the fixture's stub copies can't pass (its own tests are in stack-release.test.mjs).
import test from 'node:test';
import assert from 'node:assert/strict';
import {execFileSync,spawn} from 'node:child_process';
import {mkdirSync,writeFileSync,readFileSync,existsSync,readdirSync,rmSync,chmodSync,copyFileSync,symlinkSync,statSync} from 'node:fs';
import {DatabaseSync} from 'node:sqlite';
import {join,dirname} from 'node:path';
import {once} from 'node:events';
import {B,C,C1,U,COPIES,NODE,SBIN,base,fault,git,lines,logLines,mk,pointer,repo,sha,storeVersion,sysctl,units} from './stack-release-fixture.mjs';
// --- Root's stand-ins. Each is serialized into its own file with K, its baked paths. ---
// sudo -u USER -H env PATH=P CMD...: the form the driver must use, logged to sudo.log, then CMD with
// SBIN before P. A one-shot hook arm/<key>.sh runs first and arm/<key>.id replaces CMD's ID. The
// rehearsal prints a fixed BACKUP_STORE line and exits faults/rehearse-rc unless faults/real-rehearse.
function fakeSudo(K){
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),cp=process.getBuiltinModule('node:child_process');
const a=process.argv.slice(2),sd=process.env.SD_STATE,f=(n)=>path.join(sd,'faults',n);
if(a[0]!=='-u'||!a[1]||a[2]!=='-H'||a[3]!=='env'||!/^PATH=/.test(a[4]||'')||a.length<6){console.error(`fake sudo: not the driver's form: ${a.join(' ')}`);process.exit(99)}
const P=a[4].slice(5),cmd=a.slice(5);
let key=path.basename(cmd[0]);
if(cmd[0].endsWith('/scripts/stack-release.sh')){
key=cmd[1];
if(['activate','rollback'].includes(key)&&cmd.includes('--plan'))key+='-plan';
if(key==='confirm'&&cmd.includes('--rollback'))key='confirm-rollback';
}else if(cmd[0]==='git')key=`git-${cmd[3]}`;
fs.appendFileSync(path.join(sd,'sudo.log'),JSON.stringify({user:a[1],path:P,key,cmd})+'\n');
const env={...process.env,PATH:`${K.sbin}:${P}`},arm=(n)=>path.join(sd,'arm',`${key}.${n}`);
if(fs.existsSync(arm('sh'))){const run=arm('sh')+'.ran';fs.renameSync(arm('sh'),run);cp.spawnSync('bash',[run],{env,stdio:'inherit'})}
if(fs.existsSync(arm('id'))){cmd[2]=fs.readFileSync(arm('id'),'utf8').trim();fs.rmSync(arm('id'))}
if(key==='rehearse'&&!fs.existsSync(f('real-rehearse'))){
console.log(`fixture rehearse: ${cmd.slice(1).join(' ')}`);
let line='BACKUP_STORE 20261020T000000Z-0123456789ab schema 3g';try{line=fs.readFileSync(f('rehearse-store-line'),'utf8').replace(/\n$/,'')}catch{}
if(line)console.log(line);
let rc=0;try{rc=Number(fs.readFileSync(f('rehearse-rc'),'utf8'))}catch{}
process.exit(rc);
}
const r=cp.spawnSync(cmd[0],cmd.slice(1),{env,stdio:'inherit'});process.exit(r.status??1);
}
// case2-unit.sh: logs its argv, then installs the copy's units as the fixture's user units (its
// WorkingDirectory under the stack's copies) and starts the bus, then the Console, as case2-unit.sh
// does. faults/render-fails-<copy> exits 1 first; faults/render-probe-<copy> (one-shot) first runs
// probe.sh, a second driver, and records its exit and stderr.
function fakeRender(K){
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),cp=process.getBuiltinModule('node:child_process');
const a=process.argv.slice(2),sd=process.env.SD_STATE,f=(n)=>path.join(sd,'faults',n);
fs.appendFileSync(path.join(sd,'render.log'),JSON.stringify(a)+'\n');
const cut=a.indexOf('--'),pre=a.slice(0,cut),serve=a.slice(cut+1),get=(k)=>pre[pre.indexOf(k)+1];
const copy=get('--copy'),copies=path.join(get('--home'),'.local','share','mosaic-stack','copies');
if(fs.existsSync(f(`render-probe-${copy}`))){
fs.rmSync(f(`render-probe-${copy}`));
const p=cp.spawnSync('bash',[path.join(sd,'probe.sh')],{encoding:'utf8'});
fs.writeFileSync(path.join(sd,'probe.json'),JSON.stringify({code:p.status,err:p.stderr}));
}
if(fs.existsSync(f(`render-fails-${copy}`))){console.error(`fixture render: ${copy} fails`);process.exit(1)}
const sh=(n,...x)=>cp.execFileSync('bash',[path.join(K.repo,'scripts',n),...x],{stdio:['ignore','ignore','inherit']});
sh('bus-service.sh','install','--copy',copy,'--copies',copies,...(pre.includes('--pm')?['--pm']:[]),'--claude',get('--claude'));
sh('webui-service.sh','install','--copy',copy,'--copies',copies,...serve);
for(const u of [`mosaic-bus@${get('--business')}.service`,'mosaic-webui.service'])cp.execFileSync(path.join(K.sbin,'systemctl'),['start',u],{stdio:'inherit'});
}
// Root's systemctl: the fixture's, with systemd's no-op start of an active unit, and a one-shot
// faults/stop-noop-<unit> that leaves the unit running, as a stop past its timeout would.
function fakeSystemctl(K){
const fs=process.getBuiltinModule('node:fs'),path=process.getBuiltinModule('node:path'),cp=process.getBuiltinModule('node:child_process');
const a=process.argv.slice(2),sd=process.env.SD_STATE;
fs.appendFileSync(path.join(sd,'root-systemctl.log'),a.join(' ')+'\n');
const noop=path.join(sd,'faults',`stop-noop-${a[1]}`);
if(a[0]==='stop'&&fs.existsSync(noop)){fs.rmSync(noop);process.exit(0)}
if(a[0]==='start'){let s={};try{s=JSON.parse(fs.readFileSync(path.join(sd,'units.json'),'utf8'))}catch{}if((s[a[1]]||{}).state==='active')process.exit(0)}
const r=cp.spawnSync(path.join(K.sbin,'systemctl'),a,{stdio:'inherit'});process.exit(r.status??1);
}
const FAKES=join(base,'upgrade-fakes');mkdirSync(FAKES);
const K={sbin:SBIN,repo};
for(const [n,fn] of [['sudo',fakeSudo],['render',fakeRender],['systemctl',fakeSystemctl]])writeFileSync(join(FAKES,n),`#!${NODE}\n(${fn})(${JSON.stringify(K)});\n`,{mode:0o755});
writeFileSync(join(FAKES,'hostcheck'),'#!/bin/sh\necho "$*" >> "$SD_STATE/hostcheck.log"\nexit "$(cat "$SD_STATE/faults/hostcheck-rc" 2>/dev/null || echo 0)"\n',{mode:0o755});
const BUNDLE=join(base,'fixture.bundle');git(repo,'bundle','create','-q',BUNDLE,'refactor');
// Root's checkout of a reviewed commit: a clone of the fixture repo, detached at it.
const checkouts=new Map();
const checkout=(id)=>{
if(!checkouts.has(id)){const d=join(base,`root-${id.slice(0,12)}`);git(base,'clone','-q','--no-checkout',repo,d);git(d,'checkout','-q','--detach',id);checkouts.set(id,d)}
return checkouts.get(id);
};
const TO=['--timeout','3'];
// --- Per-test stack ---
// The stack's user's home: its copies (the fixture's, through a link), its clone, its bin. Both units
// installed on `copy` and active, adopted under --units external, and the I2 args file naming it.
async function useed(t,{copy=B}={}){
const e=await mk(t),home=e.env.HOME;
e.hc=join(home,'.local','share','mosaic-stack','copies');mkdirSync(dirname(e.hc),{recursive:true});symlinkSync(COPIES,e.hc);
const bin=join(home,'.npm-global','bin');mkdirSync(bin,{recursive:true});
writeFileSync(join(bin,'mosaic'),readFileSync(join(e.bin,'mosaic'),'utf8').replaceAll(`${e.bin}/`,`${bin}/`),{mode:0o755});
copyFileSync(join(e.bin,'mosaic-old'),join(bin,'mosaic-old'));chmodSync(join(bin,'mosaic-old'),0o755);
e.bin=bin;e.clone=join(home,'src','mosaic-stack');git(base,'clone','-q',repo,e.clone);
const db=new DatabaseSync(e.store);
db.exec('PRAGMA journal_mode=WAL');
db.exec('CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL)');
db.exec('CREATE TABLE events (id INTEGER PRIMARY KEY, at TEXT NOT NULL)');
db.prepare("INSERT INTO meta VALUES ('schema_version', '3g')").run();
for(let i=0;i<3;i++)db.prepare('INSERT INTO events (at) VALUES (?)').run(`2000-01-01T00:00:0${i}.000Z`);
db.close();
const sh=(n,...a)=>execFileSync('bash',[join(e.clone,'scripts',n),...a],{env:e.env,encoding:'utf8',stdio:['ignore','pipe','pipe']});
sh('bus-service.sh','install','--copy',copy,'--copies',e.hc,'--claude',e.claude);
sh('webui-service.sh','install','--copy',copy,'--copies',e.hc,'--port',e.port,'--business','acme');
sh('stack-copy.sh','wrapper',copy,'--copies',e.hc,'--bin',e.bin);
execFileSync(join(SBIN,'systemctl'),['start',e.bus],{env:{...e.env,SD_CLOCK_OFFSET:'120'}});
sysctl(e,'start',e.con);
sh('stack-release.sh','adopt','acme','--units','external','--copies',e.hc,'--bin',e.bin);
e.etc=join(e.root,'etc');mkdirSync(e.etc);e.args=join(e.etc,'dev.args');
e.argsText=`--user\nfixture\n--home\n${home}\n--business\nacme\n--copy\n${copy}\n--node\n${NODE}\n--claude\n${e.claude}\n--\n--port\n${e.port}\n--business\nacme\n`;
writeFileSync(e.args,e.argsText);chmodSync(e.args,0o640);
e.run=join(e.root,'run');e.slog=join(e.root,'log','upgrade-dev.log');e.lockf=join(e.run,'upgrade-dev.lock');
mkdirSync(join(e.sd,'arm'));
e.denv={...e.env,STACK_UPGRADE_ETC:e.etc,STACK_UPGRADE_RUN:e.run,STACK_UPGRADE_LOG_DIR:dirname(e.slog),STACK_UPGRADE_UNIT_DIR:e.units,STACK_UPGRADE_HOSTCHECK:join(FAKES,'hostcheck'),STACK_UPGRADE_RENDER:join(FAKES,'render'),STACK_UPGRADE_SUDO:join(FAKES,'sudo'),STACK_UPGRADE_SYSTEMCTL:join(FAKES,'systemctl')};
writeFileSync(join(e.sd,'probe.sh'),`exec bash ${join(checkout(C1),'scripts','stack-upgrade.sh')} upgrade --instance dev --bundle ${BUNDLE} --sha ${C1} --timeout 3\n`);
writeFileSync(join(e.sd,'snap.jsonl'),'');
return e;
}
// The driver from root's checkout of `at`.
async function drive(e,at,...args){
const ch=spawn('bash',[join(checkout(at),'scripts','stack-upgrade.sh'),...args],{env:e.denv,stdio:['ignore','pipe','pipe']});
let out='',err='';ch.stdout.on('data',(d)=>{out+=d});ch.stderr.on('data',(d)=>{err+=d});
const timer=setTimeout(()=>ch.kill('SIGKILL'),180000);
const [code]=await once(ch,'close');clearTimeout(timer);
return {code,out,err,all:out+err};
}
const upgrade=(e,id,...a)=>drive(e,id,'upgrade','--instance','dev','--bundle',BUNDLE,'--sha',id,...TO,...a);
const arm=(e,key,text)=>writeFileSync(join(e.sd,'arm',key),text);
// The step log: every line `<time> <step> rc=N ...` or the result line, root:root 0640 (here the
// test user's), and nothing the stack's user printed. Returns the lines without the time.
const STEP=/^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ (start|fetch|ff|copy|hostcheck|units|plan|rehearse|rehearse-store|rehearsal-skipped|stop-console|stop-bus|wait-down|stage|render|confirm|args|revert|render-old|confirm-rollback|gate|start-bus|start-console|stop-units) rc=\d+ |^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ result exit=[0-35] [a-z-]+ /;
function slog(e){
const l=lines(e.slog);
for(const x of l){assert.match(x,STEP);assert.ok(!/stack-release:|fixture rehearse|PASS |FAIL |REHEARSAL/.test(x),x)}
assert.equal(statSync(e.slog).mode&0o777,0o640);
return l.map((x)=>x.slice(21));
}
const names=(e)=>slog(e).map((x)=>x.split(' ')[0]);
const result=(e)=>slog(e).at(-1);
const sudoKeys=(e)=>lines(join(e.sd,'sudo.log')).map((l)=>JSON.parse(l).key);
const lastAct=(e)=>logLines(e).filter((o)=>o.result==='started').at(-1).activation;
const mine=(e,a)=>logLines(e).filter((o)=>o.activation===a);
const open=(e)=>{const m=new Set();for(const o of logLines(e)){if(!o.activation)continue;if(o.result==='started')m.add(o.activation);else m.delete(o.activation)}return [...m]};
const final=(e,a)=>{const [s,f,...more]=mine(e,a);assert.equal(s.result,'started');assert.equal(more.length,0);return f};
const live=(e)=>[units(e)[e.bus].state,units(e)[e.con].state,units(e)[e.bus].wd.split('/').at(-1),units(e)[e.con].wd.split('/').at(-1)];
// Nothing the run did reached the args file: the same bytes and mode, and no temporary file left.
const argsKept=(e)=>{assert.equal(readFileSync(e.args,'utf8'),e.argsText);assert.equal(statSync(e.args).mode&0o777,0o640);assert.deepEqual(readdirSync(e.etc),['dev.args'])};
const argsCopy=(e)=>{const l=readFileSync(e.args,'utf8').split('\n');return l[l.indexOf('--copy')+1]};
const PRE=['start','fetch','ff','copy','hostcheck','units','plan','rehearse','rehearse-store'];
const A2=['stop-console','stop-bus','wait-down','wait-down'];
const R=[...A2,'revert','render-old','confirm-rollback'];
// --- r6 sequences 1-8, through the driver ---
test('row 233: sequence 1: confirm fails healthz, the driver reverts, renders the old copy and confirm --rollback --cause A5 passes: exit 1',async(t)=>{
const e=await useed(t);
fault(e,`healthz-${C1}`);
const r=await upgrade(e,C1);assert.equal(r.code,1,r.all);
assert.equal(r.err.split('\n').at(-2),`stack-upgrade: failed: A5 failed; rolled back to ${B}, which passed the gate`);
assert.deepEqual(names(e),[...PRE,...A2,'stage','render','confirm',...R,'result']);
assert.equal(result(e),`result exit=1 failed A5 failed; rolled back to ${B}, which passed the gate`);
const a=lastAct(e),l=final(e,a);assert.deepEqual([l.op,l.result,l.code,l.rollback,l.from,l.to],['activate','gate-failed','healthz','ok',B,C1]);
assert.deepEqual(open(e),[]);assert.deepEqual(live(e),['active','active',B,B]);
argsKept(e);
// The stack's user's steps went through sudo in the driver's form, as fixture, the pinned node first.
for(const l of lines(join(e.sd,'sudo.log')).map((x)=>JSON.parse(x)))assert.deepEqual([l.user,l.path],['fixture',`${dirname(NODE)}:/usr/local/bin:/usr/bin:/bin`]);
assert.deepEqual(sudoKeys(e),['git-fetch','git-merge','stack-copy.sh','activate-plan','rehearse','activate','confirm','revert','confirm-rollback']);
});
test('row 233: sequence 2: activate --migrate fails at the wrapper, revert restores the store, confirm --rollback --cause A3: exit 1',async(t)=>{
const e=await useed(t);
const s0=sha(e.store);
arm(e,'activate.sh',`chmod 0555 ${e.bin}\n`);arm(e,'revert.sh',`chmod 0755 ${e.bin}\n`);
const r=await upgrade(e,C,'--migrate');assert.equal(r.code,1,r.all);
assert.deepEqual(names(e),[...PRE,...A2,'stage',...R,'result']);
assert.equal(result(e),`result exit=1 failed A3 failed; rolled back to ${B}, which passed the gate`);
const l=final(e,lastAct(e));assert.deepEqual([l.result,l.code,l.rollback],['failed','wrapper','ok']);
assert.equal(storeVersion(e.store,true),'3g');assert.deepEqual(live(e),['active','active',B,B]);
// The bus holds the store open again; its bytes are the backup's once it stops.
sysctl(e,'stop',e.bus);assert.equal(sha(e.store),s0);
argsKept(e);
});
test('row 233: sequence 3: root\'s render of the new copy fails, the driver rolls back with --cause A4, and holds the lock to the end: exit 1',async(t)=>{
const e=await useed(t);
fault(e,`render-fails-${C1}`);fault(e,`render-probe-${B}`);
const r=await upgrade(e,C1);assert.equal(r.code,1,r.all);
// The probe ran inside render-old, during the rollback, and its one line is its refusal.
assert.deepEqual(names(e),[...PRE,...A2,'stage','render',...A2,'revert','result','render-old','confirm-rollback','result']);
const p=JSON.parse(readFileSync(join(e.sd,'probe.json'),'utf8'));assert.equal(p.code,3,p.err);
assert.equal(p.err,`stack-upgrade: upgrade-running: another stack-upgrade.sh run holds ${e.lockf} (nothing was stopped)\n`);
assert.match(slog(e).at(-4),/^result exit=3 upgrade-running /);
assert.match(slog(e).find((x)=>x.startsWith('render ')),/^render rc=1 /);
const l=final(e,lastAct(e));assert.deepEqual([l.result,l.code,l.rollback],['failed','units-start','ok']);
assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
});
test('row 233: sequence 4: a capture failure is exit 3 from the stage, and A2f restarts the old units and gates them: exit 1',async(t)=>{
const e=await useed(t);
arm(e,'activate.sh',`: > ${join(e.sd,'faults','no-fragment')}\n`);arm(e,'gate.sh',`rm ${join(e.sd,'faults','no-fragment')}\n`);
const r=await upgrade(e,C1);assert.equal(r.code,1,r.all);
assert.deepEqual(names(e),[...PRE,...A2,'stage','start-bus','start-console','gate','result']);
assert.match(slog(e).find((x)=>x.startsWith('stage ')),/^stage rc=3 /);
assert.equal(result(e),`result exit=1 failed A3 failed; ${B} is running again and passed the gate`);
const a=lastAct(e);assert.deepEqual(mine(e,a).map((o)=>[o.result,o.code]),[['started',undefined],['refused','capture-failed']]);
assert.deepEqual(open(e),[]);assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
});
test('row 233: N1: a bus that outlives A2\'s timeout keeps its old start, so A2f\'s gate fails stale-start: exit 2, both units stopped',async(t)=>{
const e=await useed(t);
fault(e,`stop-noop-${e.bus}`);
const r=await upgrade(e,C1);assert.equal(r.code,2,r.all);
assert.deepEqual(names(e),[...PRE,...A2,'start-bus','start-console','gate','stop-units','result']);
assert.ok(slog(e).includes(`wait-down rc=1 unit: ${e.bus} state: active`),slog(e).join('\n'));
assert.ok(r.out.includes(`gate failed on ${B}: stale-start`),r.out);
assert.equal(result(e),`result exit=2 rollback-failed A2 failed and ${B} failed its gate after the restart; both units are stopped`);
assert.deepEqual(live(e).slice(0,2),['inactive','inactive']);
assert.ok(!sudoKeys(e).includes('activate'));argsKept(e);
});
test('row 233: sequence 5: confirm of the wrong copy is refused, and the driver rolls back with --cause A5: exit 1',async(t)=>{
const e=await useed(t);
arm(e,'confirm.id',B);
const r=await upgrade(e,C1);assert.equal(r.code,1,r.all);
assert.ok(r.out.includes('refusing open-activation')||r.err.includes('refusing open-activation'),r.all);
assert.match(slog(e).find((x)=>x.startsWith('confirm ')),/^confirm rc=3 /);
assert.deepEqual(names(e),[...PRE,...A2,'stage','render','confirm',...R,'result']);
const l=final(e,lastAct(e));assert.deepEqual([l.result,l.code,l.rollback],['gate-failed','unrecorded','ok']);
assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
});
test('row 233: sequence 6: a revert that fails stops both units at exit 2, and the next run refuses at its --plan with nothing stopped',async(t)=>{
const e=await useed(t);
fault(e,`render-fails-${C1}`);
arm(e,'revert.sh',`for d in ${join(e.state,'stack-rollback')}/*/; do [ -f "$d/AWAITING-UNITS" ] && rm "$d/wrapper"; done\n`);
const r=await upgrade(e,C1);assert.equal(r.code,2,r.all);
assert.deepEqual(names(e),[...PRE,...A2,'stage','render',...A2,'revert','stop-units','result']);
assert.equal(result(e),`result exit=2 rollback-failed A4 failed and revert failed; both units are stopped`);
const a=lastAct(e);assert.deepEqual(open(e),[a]);assert.deepEqual(live(e).slice(0,2),['inactive','inactive']);
argsKept(e);
const n=slog(e).length,sc=lines(join(e.sd,'root-systemctl.log')).length;
rmSync(join(e.sd,'faults',`render-fails-${C1}`));
const again=await upgrade(e,C1);assert.equal(again.code,3,again.all);
assert.ok(again.out.includes(`refuses interrupted-run: activation ${a} started and has no final line`),again.out);
assert.deepEqual(names(e).slice(n),PRE.slice(0,7).concat('result'));
assert.match(result(e),/^result exit=3 plan-refused /);
assert.equal(lines(join(e.sd,'root-systemctl.log')).length,sc);argsKept(e);
});
test('row 233: sequence 7: two upgrades and a rollback --to the previous copy with --restore, the args file following each confirm',async(t)=>{
const e=await useed(t);
// The args file at confirm still names the old copy; the new one moves in after it.
arm(e,'confirm.sh',`cp ${e.args} ${join(e.sd,'args-at-confirm')}\n`);
const one=await upgrade(e,C1);assert.equal(one.code,0,one.all);
assert.equal(one.out.split('\n').at(-2),`stack-upgrade: upgrade ok: ${C1} is live for acme`);
assert.equal(readFileSync(join(e.sd,'args-at-confirm'),'utf8'),e.argsText);
assert.equal(readFileSync(e.args,'utf8'),e.argsText.replace(`--copy\n${B}\n`,`--copy\n${C1}\n`));
assert.equal(statSync(e.args).mode&0o777,0o640);assert.deepEqual(readdirSync(e.etc),['dev.args']);
assert.deepEqual(names(e),[...PRE,...A2,'stage','render','confirm','args','result']);
assert.ok(slog(e).includes(`args rc=0 copy: ${C1}`));
const a1=lastAct(e);
const two=await upgrade(e,C,'--migrate');assert.equal(two.code,0,two.all);
const a2=lastAct(e);assert.equal(argsCopy(e),C);
const l1=final(e,a1),l2=final(e,a2);
assert.deepEqual([l1.op,l1.result,l1.from,l1.to],['activate','ok',B,C1]);
assert.deepEqual([l2.op,l2.result,l2.from,l2.to,l2.backup],['activate','ok',C1,C,a2]);
assert.equal(storeVersion(e.store,true),'3h');
const n=slog(e).length;
const rb=await drive(e,C,'rollback','--instance','dev','--to',C1,'--restore',a2,...TO);assert.equal(rb.code,0,rb.all);
assert.equal(rb.out.split('\n').at(-2),`stack-upgrade: rollback ok: ${C1} is live for acme`);
// A rollback fetches, copies and rehearses nothing.
assert.deepEqual(names(e).slice(n),['start','hostcheck','units','plan',...A2,'stage','render','confirm','args','result']);
const ra=lastAct(e),l3=final(e,ra);assert.deepEqual([l3.op,l3.result,l3.from,l3.to,l3.backup],['rollback','ok',C,C1,a2]);
assert.deepEqual([pointer(e).copy,pointer(e).schema],[C1,'3g']);assert.equal(storeVersion(e.store,true),'3g');
assert.equal(argsCopy(e),C1);assert.deepEqual(live(e),['active','active',C1,C1]);
const m=slog(e).length,sc=lines(join(e.sd,'root-systemctl.log')).length;
const no=await drive(e,C,'rollback','--instance','dev','--to',B,...TO);assert.equal(no.code,3,no.all);
assert.ok(no.all.includes(`refusing to-not-previous: --to ${B} is not the previous copy; the log names ${C}`),no.all);
assert.deepEqual(names(e).slice(m),['start','hostcheck','units','plan','result']);
assert.equal(lines(join(e.sd,'root-systemctl.log')).length,sc);assert.equal(argsCopy(e),C1);
});
test('row 240: I3 N1: a failed args move after A5 exits 5 args-failed, the new copy live and confirmed and both args files kept',async(t)=>{
const e=await useed(t);
// A read-only /etc/mosaic-stack at confirm: the stage and the render are done, and the move fails.
// The fixture's cleanup runs chmod -R u+w before it removes the root, so the 0555 dir needs no hook here.
arm(e,'confirm.sh',`chmod 0555 ${e.etc}\n`);
const l=await upgrade(e,C1);assert.equal(l.code,5,l.all);
assert.deepEqual(names(e),[...PRE,...A2,'stage','render','confirm','args','result']);
assert.ok(slog(e).includes(`args rc=1 copy: ${C1}`));
assert.match(result(e),new RegExp(`^result exit=5 args-failed ${C1} is live and confirmed, but ${e.args} still names ${B}; put ${e.etc}/\\.dev\\.args\\.\\w{6} in its place$`));
assert.deepEqual(open(e),[]);assert.deepEqual(live(e),['active','active',C1,C1]);
assert.equal(readFileSync(e.args,'utf8'),e.argsText);
const left=readdirSync(e.etc).filter((n)=>n!=='dev.args');assert.equal(left.length,1);
assert.equal(readFileSync(join(e.etc,left[0]),'utf8'),e.argsText.replace(`--copy\n${B}\n`,`--copy\n${C1}\n`));
});
test('row 233: sequence 8: a migration that writes and fails: the driver reverts to the backup byte for byte, --cause A3: exit 1',async(t)=>{
const e=await useed(t);
const s0=sha(e.store);
fault(e,'migrate-fails');
const r=await upgrade(e,C,'--migrate');assert.equal(r.code,1,r.all);
assert.match(slog(e).find((x)=>x.startsWith('stage ')),/^stage rc=1 /);
assert.deepEqual(names(e),[...PRE,...A2,'stage',...R,'result']);
const a=lastAct(e),l=final(e,a);assert.deepEqual([l.result,l.code,l.rollback],['failed','migrate-failed','ok']);
sysctl(e,'stop',e.bus);assert.equal(sha(e.store),s0);assert.equal(storeVersion(e.store,false),'3g');
argsKept(e);
});
// --- The lock, A1's refusals and the rehearsal ---
test('row 233: a second run while the lock is held refuses upgrade-running with one log line and no call',async(t)=>{
const e=await useed(t);
mkdirSync(e.run,{recursive:true});
const holder=spawn('flock',[e.lockf,'sleep','60'],{detached:true,stdio:'ignore'});
t.after(()=>{try{process.kill(-holder.pid,'SIGKILL')}catch{}});
for(let i=0;i<100&&execFileSync('bash',['-c','flock -n "$1" true && echo free || echo held','_',e.lockf],{encoding:'utf8'}).trim()==='free';i++)await new Promise((r)=>setTimeout(r,50));
const r=await upgrade(e,C1);assert.equal(r.code,3,r.all);
assert.equal(r.err,`stack-upgrade: upgrade-running: another stack-upgrade.sh run holds ${e.lockf} (nothing was stopped)\n`);
assert.deepEqual(names(e),['result']);
assert.ok(!existsSync(join(e.sd,'sudo.log')));assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));
assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
});
test('row 233: A1 refuses open-activation on any AWAITING-UNITS file, through the stack\'s user\'s --plan, with nothing stopped',async(t)=>{
const e=await useed(t);
const fake=join(e.state,'stack-rollback',`20000101T000000Z-${C1.slice(0,12)}`);mkdirSync(fake,{recursive:true});writeFileSync(join(fake,'AWAITING-UNITS'),`${C1}\n`);
const logs=logLines(e).length;
const r=await upgrade(e,C1);assert.equal(r.code,3,r.all);
assert.ok(r.out.includes('refuses open-activation'),r.out);
assert.deepEqual(names(e),[...PRE.slice(0,7),'result']);assert.match(result(e),/^result exit=3 plan-refused /);
assert.equal(logLines(e).length,logs);assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));
assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
});
test('row 233: a target that migrates refuses migrate-required without --migrate, with nothing stopped',async(t)=>{
const e=await useed(t);
const r=await upgrade(e,C);assert.equal(r.code,3,r.all);
assert.ok(r.out.includes('refuses migrate-required'),r.out);
assert.deepEqual(names(e),[...PRE.slice(0,7),'result']);assert.match(result(e),/^result exit=3 plan-refused /);
assert.ok(!sudoKeys(e).includes('rehearse'));assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));
assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
});
test('row 233: the rehearsal runs after the plan and before A2; a failed one refuses rehearsal-failed with nothing stopped',async(t)=>{
const e=await useed(t);
fault(e,'rehearse-rc','1');
const r=await upgrade(e,C1);assert.equal(r.code,3,r.all);
assert.ok(r.out.includes('fixture rehearse: '),r.out);
assert.deepEqual(names(e),[...PRE,'result']);
assert.equal(result(e),'result exit=3 rehearsal-failed the rehearsal exited 1 (nothing was stopped)');
const rh=lines(join(e.sd,'sudo.log')).map((l)=>JSON.parse(l)).find((l)=>l.key==='rehearse');
assert.deepEqual(rh.cmd.slice(1),['rehearse',C1,'acme','--copies',e.hc,'--backup','last','--port','7339','--timeout','3']);
assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));assert.deepEqual(live(e),['active','active',B,B]);argsKept(e);
});
test('row 233: the rehearsal\'s BACKUP_STORE line goes to the step log after a match, a malformed one as none, the rest of its output never',async(t)=>{
const e=await useed(t);
fault(e,'rehearse-rc','1');
const r=await upgrade(e,C1,'--rehearse-port','7449');assert.equal(r.code,3,r.all);
assert.ok(slog(e).includes('rehearse-store rc=0 BACKUP_STORE 20261020T000000Z-0123456789ab schema 3g'),slog(e).join('\n'));
assert.ok(slog(e).find((x)=>x.startsWith('rehearse ')).includes(' --port 7449 '));
for(const bad of ['BACKUP_STORE ../../etc schema 3g','BACKUP_STORE 20261020T000000Z-0123456789ab schema 3g; rm -rf /','stack-release: BACKUP_STORE 20261020T000000Z-0123456789ab schema 3g','']){
fault(e,'rehearse-store-line',bad);
const n=slog(e).length;
const b=await upgrade(e,C1);assert.equal(b.code,3,b.all);
assert.deepEqual(slog(e).slice(n).filter((x)=>x.startsWith('rehearse-store ')),['rehearse-store rc=0 BACKUP_STORE none'],bad);
}
});
test('row 233: a real rehearsal with no backup in the log refuses no-backup, and goes on only with --no-rehearsal, which logs the skip',async(t)=>{
const e=await useed(t);
fault(e,'real-rehearse');
const r=await upgrade(e,C1);assert.equal(r.code,3,r.all);
assert.ok(r.out.includes('FAIL no-backup: the log records no backup\nREHEARSAL_FAIL\n'),r.out);
assert.equal(result(e),'result exit=3 no-backup the log records no backup to rehearse against; only a go that names --no-rehearsal goes on (nothing was stopped)');
assert.ok(slog(e).includes('rehearse-store rc=0 BACKUP_STORE none'));
assert.ok(!existsSync(join(e.data,'rehearsals')));assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));
const n=slog(e).length;
const s=await upgrade(e,C1,'--no-rehearsal');assert.equal(s.code,0,s.all);
assert.deepEqual(names(e).slice(n),[...PRE,'rehearsal-skipped',...A2,'stage','render','confirm','args','result']);
assert.equal(argsCopy(e),C1);assert.deepEqual(live(e),['active','active',C1,C1]);
});
test('row 233: a real rehearsal after an upgrade names the log\'s newest backup; --no-rehearsal with a backup refuses rehearsal-required',async(t)=>{
const e=await useed(t);
const one=await upgrade(e,C1);assert.equal(one.code,0,one.all);
const a1=lastAct(e);
fault(e,'real-rehearse');
const n=slog(e).length;
// The stub copies fail the rehearsal right after it names the backup (copy-without-no-launch).
const r=await upgrade(e,C,'--migrate');assert.equal(r.code,3,r.all);
assert.ok(r.out.includes(`PASS backup-last: the log's newest backup is ${a1}\n`),r.out);
assert.deepEqual(names(e).slice(n),[...PRE,'result']);assert.match(result(e),/^result exit=3 rehearsal-failed the rehearsal exited 1 /);
rmSync(join(e.sd,'faults','real-rehearse'));
const m=slog(e).length;
const q=await upgrade(e,C,'--migrate','--no-rehearsal');assert.equal(q.code,3,q.all);
assert.deepEqual(names(e).slice(m),[...PRE,'result']);
assert.equal(result(e),'result exit=3 rehearsal-required the log records a backup and the rehearsal ran; --no-rehearsal is only for a stack with none (nothing was stopped)');
assert.equal(argsCopy(e),C1);assert.deepEqual(live(e),['active','active',C1,C1]);
});
// --- Before A1 and the args file ---
test('row 233: the args file must be the running user\'s, with no group or other write, one --copy, and the copy the units run',async(t)=>{
const e=await useed(t);
const tryArgs=async(text,mode,code,why)=>{
writeFileSync(e.args,text);chmodSync(e.args,mode);
const n=existsSync(e.slog)?slog(e).length:0;
const r=await upgrade(e,C1);assert.equal(r.code,3,`${why}: ${r.all}`);
assert.match(slog(e).at(-1),new RegExp(`^result exit=3 ${code} `),why);
assert.ok(!slog(e).slice(n).some((x)=>/^(stop|stage|render|confirm)/.test(x)),why);
};
await tryArgs(e.argsText,0o660,'args-unreadable','group write');
await tryArgs(e.argsText,0o642,'args-unreadable','other write');
await tryArgs(e.argsText.replace('--\n',`--copy\n${C1}\n--\n`),0o640,'args-unreadable','two --copy');
await tryArgs(e.argsText.replace('--\n','--tester\n--\n'),0o640,'args-unreadable','an unknown flag');
await tryArgs(e.argsText.replace(`--copy\n${B}\n`,`--copy\n${C1}\n`),0o640,'args-differ','the units run B');
rmSync(e.args);symlinkSync(join(e.sd,'probe.sh'),e.args);
const l=await upgrade(e,C1);assert.equal(l.code,3);assert.match(result(e),/^result exit=3 args-unreadable /);
assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));assert.deepEqual(live(e),['active','active',B,B]);
});
test('row 233: hostcheck (row 240): a nonzero --check or a missing file refuses profile-drift before the plan',async(t)=>{
const e=await useed(t);
fault(e,'hostcheck-rc','1');
const r=await upgrade(e,C1);assert.equal(r.code,3,r.all);
assert.match(result(e),/^result exit=3 profile-drift /);
assert.deepEqual(lines(join(e.sd,'hostcheck.log')),['--check']);
assert.deepEqual(sudoKeys(e),['git-fetch','git-merge','stack-copy.sh']);
e.denv.STACK_UPGRADE_HOSTCHECK=join(e.root,'no-hostcheck');
const m=await upgrade(e,C1);assert.equal(m.code,3,m.all);
assert.equal(result(e),`result exit=3 profile-drift no executable ${join(e.root,'no-hostcheck')} (row 240) (nothing was stopped)`);
assert.ok(!sudoKeys(e).includes('activate-plan'));assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));argsKept(e);
});
test('row 233: the checkout must be at --sha (head-mismatch), and a commit the clone can\'t fast-forward to refuses fetch-failed',async(t)=>{
const e=await useed(t);
const r=await drive(e,C1,'upgrade','--instance','dev','--bundle',BUNDLE,'--sha',C,'--migrate',...TO);assert.equal(r.code,3,r.all);
assert.equal(result(e),`result exit=3 head-mismatch this checkout, ${checkout(C1)}, is at ${C1}, not ${C} (nothing was stopped)`);
const u=await drive(e,U,'upgrade','--instance','dev','--bundle',BUNDLE,'--sha',U,...TO);assert.equal(u.code,3,u.all);
assert.match(result(e),/^result exit=3 fetch-failed .* fast-forward /);
assert.ok(!sudoKeys(e).includes('activate-plan'));assert.ok(!existsSync(join(e.sd,'root-systemctl.log')));argsKept(e);
});
test('row 233: usage exits 4 before any log or lock',async(t)=>{
const e=await useed(t);
for(const a of [[],['status'],['upgrade','--instance','dev','--bundle',BUNDLE],['upgrade','--instance','other','--bundle',BUNDLE,'--sha',C1],['upgrade','--instance','dev','--bundle','rel/x','--sha',C1],['upgrade','--instance','dev','--bundle',BUNDLE,'--sha',C1.slice(0,12)],['upgrade','--instance','dev','--bundle',BUNDLE,'--sha',C1,'--to',B],['upgrade','--instance','dev','--bundle',BUNDLE,'--sha',C1,'--rehearse-port','80'],['rollback','--instance','dev','--to',B,'--migrate'],['rollback','--instance','dev','--to',B,'--events','3'],['rollback','--instance','dev'],['rollback','--instance','dev','--to',B,'--restore','x']]){
const r=await drive(e,C1,...a);assert.equal(r.code,4,a.join(' '));assert.match(r.err,/^usage: stack-upgrade.sh upgrade /,a.join(' '));
}
assert.ok(!existsSync(dirname(e.slog)));assert.ok(!existsSync(e.run));
});
+3 -10
View File
@@ -1,3 +1,5 @@
import { leasePort } from '../../../scripts/testing/lane-fixtures.mjs';
import { after as laneAfter } from 'node:test';
import { test, after } from "node:test";
import assert from "node:assert/strict";
import {
@@ -73,16 +75,7 @@ function closeServer(server) {
// Grab an ephemeral free port, then hand it back immediately so a caller can
// try to bind it themselves (used to prove startServer never opened a socket).
function getFreePort() {
return new Promise((resolvePromise, reject) => {
const probe = createNetServer();
probe.once("error", reject);
probe.listen(0, "127.0.0.1", () => {
const port = probe.address().port;
probe.close(() => resolvePromise(port));
});
});
}
function getFreePort() { return leasePort({after:laneAfter}, {closed:true}); }
function runCli(args) {
return spawnSync(process.execPath, [cli, ...args], { encoding: "utf8", timeout: 15000 });
@@ -1,3 +1,4 @@
import { fixtureCacheEnv } from '../../../scripts/testing/lane-fixtures.mjs';
// The host's claude CLI through adapters/claude with the bundle's hook and
// MCP server, against the scripted Messages API. Scratch CLAUDE_CONFIG_DIR
// and HOME, a dummy key, nonessential traffic off: nothing reaches a real
@@ -61,7 +62,7 @@ async function session(t, script, tools = ["read", "bash"]) {
function turn(env, request, cwd, adapter = ADAPTER) {
return new Promise((resolve) => {
const child = spawn("/bin/sh", [adapter], { cwd, env: { ...env, MOSAIC_REQUEST: request }, stdio: ["ignore", "pipe", "pipe"], detached: true });
const child = spawn("/bin/sh", [adapter], { cwd, env: { ...env, ...fixtureCacheEnv(env.CLAUDE_CONFIG_DIR), MOSAIC_REQUEST: request }, stdio: ["ignore", "pipe", "pipe"], detached: true });
let stdout = "";
let stderr = "";
child.stdout.on("data", (b) => (stdout += b));
+2 -1
View File
@@ -1,3 +1,4 @@
import { fixtureCacheEnv } from '../../../scripts/testing/lane-fixtures.mjs';
// Real pi (the pinned CLI) through adapters/pi with the extension, against
// the scripted Messages API. No real model, no network beyond 127.0.0.1.
@@ -57,7 +58,7 @@ async function session(t, script, place = (dir) => join(dir, "ws"), instance = "
function turn(env, request, cwd) {
return new Promise((resolve) => {
const child = spawn("/bin/sh", [ADAPTER], { cwd, env: { ...env, MOSAIC_REQUEST: request }, stdio: ["ignore", "pipe", "pipe"] });
const child = spawn("/bin/sh", [ADAPTER], { cwd, env: { ...env, ...fixtureCacheEnv(env.PI_CODING_AGENT_DIR), MOSAIC_REQUEST: request }, stdio: ["ignore", "pipe", "pipe"] });
let stdout = "";
let stderr = "";
child.stdout.on("data", (b) => (stdout += b));
+1 -1
View File
@@ -83,7 +83,7 @@ export function threadStore(dataRoot, business, thread) {
export const ENV_ALLOW = Object.freeze([
"HOME", "USER", "LOGNAME", "LANG", "LC_ALL", "TZ", "TERM",
"ANTHROPIC_API_KEY", "ANTHROPIC_AUTH_TOKEN", "ANTHROPIC_BASE_URL", "CLAUDE_CODE_OAUTH_TOKEN",
"OPENAI_API_KEY", "OPENROUTER_API_KEY",
"OPENAI_API_KEY", "OPENROUTER_API_KEY", "ZAI_API_KEY",
"MOSAIC_CLAUDE_BIN",
]);
+7
View File
@@ -58,6 +58,13 @@ test("sessionEnv passes only the allowlist, the repo's bin on PATH, and the run
assert.equal(sessionEnv({ MOSAIC_CLAUDE_BIN: "/opt/claude" }, "r0123456789ab").MOSAIC_CLAUDE_BIN, "/opt/claude");
});
test("ZAI_API_KEY passes the session allowlist, and ZAI_TOKEN does not", () => {
assert.ok(ENV_ALLOW.includes("ZAI_API_KEY"));
const env = sessionEnv({ ZAI_API_KEY: "fixture-zai", ZAI_TOKEN: "t" }, "r0123456789ab");
assert.equal(env.ZAI_API_KEY, "fixture-zai");
assert.equal(Object.hasOwn(env, "ZAI_TOKEN"), false);
});
test("sessionExit gives a runner's signal death back as the signal; a session credential is one whole set of nonempty variables", () => {
assert.deepEqual(sessionExit(0, null), { code: 0, signal: null });
assert.deepEqual(sessionExit(23, null), { code: 23, signal: null });
+3 -2
View File
@@ -1,3 +1,5 @@
import { leasePort } from '../../../scripts/testing/lane-fixtures.mjs';
import { after as laneAfter } from 'node:test';
// Row 74 (#1568, lead decision 93): the preview mode. A request for the public name needs the
// proxy's header equal to the one user id, else 401 with nothing read; in preview mode nothing
// but GET and HEAD gets through, whatever the Host; loopback requests work as they did. Round 2:
@@ -205,8 +207,7 @@ test('in preview mode every write answers 403 read-only, from the public name an
});
test('loopback requests are unchanged in preview mode, and the bus views work with the board down', async t => {
const probe = createServer(); await new Promise(r => probe.listen(0, '127.0.0.1', r));
const dead = `http://127.0.0.1:${probe.address().port}`; await close(probe);
const dead = `http://127.0.0.1:${await leasePort({after:laneAfter}, {closed:true})}`;
const { port, busCalls } = await setup(t, { board: dead, timeout: 200 });
const loopback = { host: `127.0.0.1:${port}` };
const inbox = await send(port, { headers: loopback });
+3 -2
View File
@@ -1,3 +1,5 @@
import { leasePort } from '../../../scripts/testing/lane-fixtures.mjs';
import { after as laneAfter } from 'node:test';
import test from 'node:test';
import assert from 'node:assert/strict';
import { spawn, spawnSync } from 'node:child_process';
@@ -62,8 +64,7 @@ test('serve argv carries only fixture proof path and header; startup explicitly
test('CLI applies proof on the main listener and rejects that header on its tester listener', async t => {
const f = await testerFixture(t), file = join(f.root, 'proof.fixture');
writeFileSync(file, VALUE, { mode: 0o600 });
const free = createServer(); free.listen(0, '127.0.0.1'); await once(free, 'listening');
const testerPort = free.address().port; await new Promise(r => free.close(r));
const testerPort = await leasePort({after:laneAfter}, {closed:true});
const child = spawn(process.execPath, [CLI, 'serve', '--port', '0', '--board', 'http://127.0.0.1:9', '--tester-port', String(testerPort), '--business', 'acme', ...PREVIEW, '--proof-header', 'x-console-proof', '--proof-file', file], { env: f.f.env, stdio: ['ignore', 'pipe', 'pipe'] });
let output = ''; child.stdout.on('data', c => output += c); child.stderr.resume();
t.after(async () => { if (child.exitCode === null && child.signalCode === null) { child.kill('SIGTERM'); await once(child, 'close'); } });
+3 -2
View File
@@ -1,3 +1,5 @@
import { leasePort } from '../../../scripts/testing/lane-fixtures.mjs';
import { after as laneAfter } from 'node:test';
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { createServer, get } from 'node:http';
@@ -97,8 +99,7 @@ test('proxy preserves exact request bytes, status and receipt, rejects forms and
});
test('unreachable board reports URL; CLI rejects unsupported options', async () => {
const probe = createServer(); await new Promise(r => probe.listen(0, '127.0.0.1', r));
const url = `http://127.0.0.1:${probe.address().port}`; await close(probe);
const url = `http://127.0.0.1:${await leasePort({after:laneAfter}, {closed:true})}`;
const web = await startServer({ port: 0, board: url, timeout: 100 });
try {
const res = await fetch(`http://127.0.0.1:${web.address().port}/api/board`);
+3 -9
View File
@@ -1,3 +1,5 @@
import { leasePort } from '../../../scripts/testing/lane-fixtures.mjs';
import { after as laneAfter } from 'node:test';
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { spawn } from 'node:child_process';
@@ -45,15 +47,7 @@ function filled(flags, root = REPO) {
// A loopback port with nothing listening, so status's /healthz request
// reaches no real Console.
async function closedPort() {
const server = createServer();
server.listen(0, '127.0.0.1');
await once(server, 'listening');
const { port } = server.address();
server.close();
await once(server, 'close');
return port;
}
async function closedPort() { return leasePort({after:laneAfter}, {closed:true}); }
function tmp(t) {
const dir = mkdtempSync(join(tmpdir(), 'webui-service-'));
+3 -4
View File
@@ -1,3 +1,5 @@
import { leasePort } from '../../../scripts/testing/lane-fixtures.mjs';
import { after as laneAfter } from 'node:test';
import test from 'node:test';
import assert from 'node:assert/strict';
import { createServer } from 'node:net';
@@ -15,10 +17,7 @@ const get = (port, path) => new Promise((resolve, reject) => {
let text = ''; r.on('data', c => text += c); r.on('end', () => resolve({ status: r.statusCode, body: text }));
}); q.on('error', reject); q.end();
});
async function freePort() {
const s = createServer(); s.listen(0, '127.0.0.1'); await once(s, 'listening');
const port = s.address().port; await new Promise(r => s.close(r)); return port;
}
async function freePort() { return leasePort({after:laneAfter}, {closed:true}); }
test('tester-port validates range tunnel main port and requires configured bus before listening', t => {
const root = tmp(t, 'tester-cli-invalid-'), f = fixture(root);
+216
View File
@@ -0,0 +1,216 @@
#!/usr/bin/env bash
# Row 240 (I4, #1746): the Case 1 host's drift stop for bwrap, r6 B2
# (agents/darkwing/work/queue-195/DESIGN.md at b7239bb66, as docs/plans/2026-10-20_installer-rows.md
# "I4 hostcheck" ships it). Deviations and decisions: agents/darkwing/work/queue-240/BUILD.md.
#
# case1-hostcheck.sh --check | --enforce | --upstream
#
# Installed as /usr/local/lib/mosaic-stack/hostcheck.sh, root:root 0755, with case1-pins.sh beside it,
# root:root 0644. Three callers: stack-upgrade.sh A1 runs --check and refuses profile-drift on any
# exit but 0; the apt hook /etc/apt/apt.conf.d/80mosaic-hostcheck runs --enforce after every dpkg run;
# mosaic-hostcheck.service runs --enforce at boot, before the bus.
#
# Seven items, one line each, `ok N ...` or `FAIL N ...`:
# 1. /etc/apparmor.d/bwrap-userns-restrict has the pinned sha256, and `dpkg -S` finds no owner.
# 2. `aa-status --json` lists bwrap and unpriv_bwrap in enforce mode, and no mosaic-bwrap.
# 3. `grep -rlw bwrap /etc/apparmor.d` lists that one file and nothing else.
# 4. A bwrap child started as mosaic under the units' no-new-privs reads the label
# `bwrap//&unpriv_bwrap (enforce)`, exactly.
# 5. The statoverride on /usr/bin/bwrap reads `root mosaic 0750`, and stat agrees.
# 6. kernel.apparmor_restrict_unprivileged_userns and fs.protected_hardlinks are 1.
# 7. net.ipv4.ip_unprivileged_port_start is above the Console's --port in the installed unit, or
# above 1023, the highest --port case2-unit.sh takes, while no unit is installed.
# A `note` line says whether bubblewrap is the install pin; it is never a failure.
#
# --check exits 0 when all seven hold and 1 otherwise. --enforce runs the same items and exits 0. On
# a failure, or while /etc/mosaic-stack/DRIFT exists, it stops bwrap: `apparmor_parser -R` on the
# profile if it is loaded, then root:root 0700 on /usr/bin/bwrap. It appends the failed items to DRIFT
# and one line to /var/log/mosaic-stack/hostcheck.log. Only root removes DRIFT, in a reviewed step.
# It never writes the statoverride, because dpkg's lock may be held under apt's (r6).
# --upstream downloads the apparmor-profiles candidate into a temp dir and compares its
# bwrap-userns-restrict to the pin. A difference is printed and exits 0, because it refuses nothing;
# it exits 1 when the download or the unpack fails.
#
# Exit 2 on usage or a refusal. Root takes no --root; without root it runs only with --root.
#
# Test seam, for packages/cli/tests/case1-hostcheck.test.mjs only:
# --root DIR prefixes /etc, /proc/sys, /var/log and /usr/bin/bwrap
set -uo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
usage() { echo "usage: case1-hostcheck.sh --check | --enforce | --upstream" >&2; exit 2; }
MODE="" ROOT=""
while [ $# -gt 0 ]; do
case "$1" in
--check|--enforce|--upstream) [ -z "$MODE" ] || usage; MODE="${1#--}"; shift ;;
--root) [ $# -ge 2 ] && [[ "$2" =~ ^/[A-Za-z0-9/._+-]+$ ]] && [ -d "$2" ] || usage; ROOT="${2%/}"; shift 2 ;;
*) usage ;;
esac
done
[ -n "$MODE" ] || usage
if [ "$(id -u)" = 0 ]; then
[ -z "$ROOT" ] || { echo "hostcheck: refusing: --root is a test seam, and root takes none" >&2; exit 2; }
PATH=/usr/sbin:/usr/bin:/sbin:/bin
else
[ -n "$ROOT" ] || { echo "hostcheck: refusing: run it as root" >&2; exit 2; }
fi
PIN_PROFILES_VERSION="" PIN_PROFILE_SHA256="" PIN_BUBBLEWRAP_VERSION=""
# shellcheck source=case1-pins.sh
[ -f "$HERE/case1-pins.sh" ] && . "$HERE/case1-pins.sh"
PROFILE=/etc/apparmor.d/bwrap-userns-restrict
BWRAP=/usr/bin/bwrap
DRIFT="$ROOT/etc/mosaic-stack/DRIFT"
LOGF="$ROOT/var/log/mosaic-stack/hostcheck.log"
LABEL='bwrap//&unpriv_bwrap (enforce)'
stamp() { date -u +%Y-%m-%dT%H:%M:%SZ; }
FAILED=() LOADED=1
pass() { echo "ok $*"; }
fail() { echo "FAIL $*"; FAILED+=("$*"); }
item1() {
local sum owner rc
[[ "$PIN_PROFILE_SHA256" =~ ^[0-9a-f]{64}$ ]] || { fail "1 no profile pin in $HERE/case1-pins.sh"; return; }
sum="$(sha256sum < "$ROOT$PROFILE" 2>/dev/null | cut -d' ' -f1)"
[ "$sum" = "$PIN_PROFILE_SHA256" ] || { fail "1 $PROFILE sha256 is ${sum:-unreadable}, not the pin $PIN_PROFILE_SHA256"; return; }
owner="$(dpkg -S "$PROFILE" 2>/dev/null)"; rc=$?
case "$rc" in
1) pass "1 $PROFILE matches the pin from apparmor-profiles $PIN_PROFILES_VERSION, and no package owns it" ;;
0) fail "1 $PROFILE is owned by a package: $owner" ;;
*) fail "1 dpkg -S $PROFILE exited $rc" ;;
esac
}
item2() {
local json list
if ! json="$(aa-status --json 2>/dev/null)"; then fail "2 aa-status --json failed"; return; fi
# The profiles object holds name-to-mode pairs and nothing nested; one pair per line.
list="$(sed -n 's/.*"profiles": *{\([^}]*\)}.*/\1/p' <<< "$json" | tr ',' '\n' | sed 's/^ *//; s/ *$//')"
grep -Eq '^"(bwrap|unpriv_bwrap)":' <<< "$list" || LOADED=0
grep -qxE '"bwrap": *"enforce"' <<< "$list" || { fail "2 aa-status lists no bwrap profile in enforce mode"; return; }
grep -qxE '"unpriv_bwrap": *"enforce"' <<< "$list" || { fail "2 aa-status lists no unpriv_bwrap profile in enforce mode"; return; }
! grep -qE '^"mosaic-bwrap":' <<< "$list" || { fail "2 aa-status lists a mosaic-bwrap profile"; return; }
pass "2 bwrap and unpriv_bwrap are in enforce mode, and no mosaic-bwrap is loaded"
}
item3() {
local found
found="$(grep -rlw bwrap "$ROOT/etc/apparmor.d" 2>/dev/null | sort)"
[ "$found" = "$ROOT$PROFILE" ] || { fail "3 the files under /etc/apparmor.d that name bwrap are not $PROFILE alone: $(tr '\n' ' ' <<< "$found")"; return; }
pass "3 $PROFILE is the only file under /etc/apparmor.d that names bwrap"
}
item4() {
local label
label="$(timeout 60 systemd-run --wait --pipe --uid=mosaic --gid=mosaic -p NoNewPrivileges=yes \
-p ProtectSystem=strict -p ProtectHome=yes -p PrivateTmp=yes \
"$BWRAP" --unshare-user --unshare-pid --ro-bind / / cat /proc/self/attr/current 2>/dev/null)"
[ "$label" = "$LABEL" ] || { fail "4 a bwrap child under no-new-privs has the label '$label', not '$LABEL'"; return; }
pass "4 a bwrap child under no-new-privs has the label '$LABEL'"
}
item5() {
local so st
so="$(dpkg-statoverride --list "$BWRAP" 2>/dev/null)"
st="$(stat -c '%U %G %a' "$ROOT$BWRAP" 2>/dev/null)"
[ "$so" = "root mosaic 0750 $BWRAP" ] || { fail "5 the statoverride on $BWRAP reads '$so', not 'root mosaic 0750'"; return; }
[ "$st" = "root mosaic 750" ] || { fail "5 $BWRAP is '$st', not 'root mosaic 750'"; return; }
pass "5 $BWRAP is root mosaic 0750, by the statoverride and by stat"
}
item6() {
local userns links
userns="$(cat "$ROOT/proc/sys/kernel/apparmor_restrict_unprivileged_userns" 2>/dev/null)"
links="$(cat "$ROOT/proc/sys/fs/protected_hardlinks" 2>/dev/null)"
[ "$userns" = 1 ] && [ "$links" = 1 ] ||
{ fail "6 kernel.apparmor_restrict_unprivileged_userns is '$userns' and fs.protected_hardlinks is '$links', not 1 and 1"; return; }
pass "6 kernel.apparmor_restrict_unprivileged_userns and fs.protected_hardlinks are 1"
}
item7() {
local floor unit="$ROOT/etc/systemd/system/mosaic-webui.service" ports port=0 p from
floor="$(cat "$ROOT/proc/sys/net/ipv4/ip_unprivileged_port_start" 2>/dev/null)"
[[ "$floor" =~ ^[0-9]{1,5}$ ]] || { fail "7 net.ipv4.ip_unprivileged_port_start is '$floor', not a number"; return; }
if [ -e "$unit" ]; then
ports="$(sed -n 's/^ExecStart=//p' "$unit" 2>/dev/null | grep -oE ' --port [0-9]{1,5}( |$)' | tr -dc '0-9\n')"
[ -n "$ports" ] || { fail "7 /etc/systemd/system/mosaic-webui.service has no --port"; return; }
for p in $ports; do [ "$((10#$p))" -le "$port" ] || port="$((10#$p))"; done
from="the Console's --port $port"
else
port=1023 from="1023, because no Console unit is installed"
fi
[ "$((10#$floor))" -gt "$port" ] || { fail "7 net.ipv4.ip_unprivileged_port_start is $floor, not above $from"; return; }
pass "7 net.ipv4.ip_unprivileged_port_start is $floor, above $from"
}
items() {
local v
item1; item2; item3; item4; item5; item6; item7
v="$(dpkg -s bubblewrap 2>/dev/null | sed -n 's/^Version: //p')"
if [ "$v" = "$PIN_BUBBLEWRAP_VERSION" ]; then
echo "note bubblewrap $v is the install pin"
else
echo "note bubblewrap ${v:-unknown} is not the install pin ${PIN_BUBBLEWRAP_VERSION:-unknown}, which is not a drift item"
fi
}
# The stop: unload the profile if it is loaded, then close bwrap to root. Prints the results.
stop_bwrap() {
local unload=skipped chown_rc=0 chmod_rc=0
if [ "$LOADED" = 1 ]; then
if [ -f "$ROOT$PROFILE" ]; then apparmor_parser -R "$ROOT$PROFILE" >/dev/null 2>&1; unload="rc=$?"; else unload=no-file; fi
fi
chown root:root "$ROOT$BWRAP" || chown_rc=$?
chmod 0700 "$ROOT$BWRAP" || chmod_rc=$?
echo "unload=$unload chown=$chown_rc chmod=$chmod_rc"
}
enforce() {
local had=0 now result f ids=""
[ ! -e "$DRIFT" ] || had=1
items
[ ${#FAILED[@]} -gt 0 ] || [ "$had" = 1 ] || { echo "hostcheck: all seven hold"; exit 0; }
umask 027
now="$(stamp)"
if [ ${#FAILED[@]} -gt 0 ]; then
mkdir -p "$(dirname "$DRIFT")" && for f in "${FAILED[@]}"; do echo "$now $f"; done >> "$DRIFT" ||
echo "hostcheck: cannot write $DRIFT" >&2
fi
for f in "${FAILED[@]}"; do ids+="${ids:+,}${f%% *}"; done
result="$(stop_bwrap)"
mkdir -p "$(dirname "$LOGF")" && echo "$now enforce stop items=${ids:-none} drift-before=$had $result" >> "$LOGF" ||
echo "hostcheck: cannot write $LOGF" >&2
echo "hostcheck: bwrap stopped ($result); only root removes $DRIFT"
exit 0
}
upstream() {
local tmp deb ver f sum
tmp="$(mktemp -d)" || exit 1
trap 'rm -rf "$tmp"' EXIT
(cd "$tmp" && apt-get download apparmor-profiles) >/dev/null 2>&1 || { echo "hostcheck: upstream: apt-get download apparmor-profiles failed" >&2; exit 1; }
deb=("$tmp"/apparmor-profiles_*.deb)
[ ${#deb[@]} = 1 ] && [ -f "${deb[0]}" ] || { echo "hostcheck: upstream: no single apparmor-profiles deb in the download" >&2; exit 1; }
ver="$(dpkg-deb -f "${deb[0]}" Version 2>/dev/null)" && dpkg-deb -x "${deb[0]}" "$tmp/x" >/dev/null 2>&1 ||
{ echo "hostcheck: upstream: cannot unpack ${deb[0]##*/}" >&2; exit 1; }
f="$tmp/x/usr/share/apparmor/extra-profiles/bwrap-userns-restrict"
if [ ! -f "$f" ]; then
echo "upstream: DIFFERS: apparmor-profiles $ver has no bwrap-userns-restrict; the pin is $PIN_PROFILE_SHA256 from $PIN_PROFILES_VERSION. Send it to Mos as a new-pin request."
exit 0
fi
sum="$(sha256sum < "$f" | cut -d' ' -f1)"
if [ "$sum" = "$PIN_PROFILE_SHA256" ]; then
echo "upstream: apparmor-profiles $ver's bwrap-userns-restrict matches the pin from $PIN_PROFILES_VERSION"
else
echo "upstream: DIFFERS: apparmor-profiles $ver's bwrap-userns-restrict has sha256 $sum; the pin is $PIN_PROFILE_SHA256 from $PIN_PROFILES_VERSION. Send it to Mos as a new-pin request."
diff -u --label "installed $PROFILE" --label "apparmor-profiles $ver" "$ROOT$PROFILE" "$f" || true
fi
exit 0
}
case "$MODE" in
check) items; [ ${#FAILED[@]} -eq 0 ] || exit 1; exit 0 ;;
enforce) enforce ;;
upstream) upstream ;;
esac
+12
View File
@@ -0,0 +1,12 @@
# Row 240 (I4, #1746): the Case 1 host pins, sourced by case1-hostcheck.sh, which is installed as
# /usr/local/lib/mosaic-stack/hostcheck.sh with this file beside it. r6 B2
# (agents/darkwing/work/queue-195/DESIGN.md at b7239bb66). Changing a pin is a reviewed commit and
# its own round.
#
# The AppArmor profile 2a' installs as /etc/apparmor.d/bwrap-userns-restrict, taken from
# /usr/share/apparmor/extra-profiles/bwrap-userns-restrict in this apparmor-profiles version.
PIN_PROFILES_VERSION=4.0.1really4.0.1-0ubuntu0.24.04.9
PIN_PROFILE_SHA256=11d39094f044f0cda0febb3ad517b830301da6b2ce929664af09ee9e4dd264f9
# The bubblewrap version 2a' installs. An install pin, not a drift item: a security update keeps the
# statoverride, and the profile attaches by path. hostcheck prints a line when they differ.
PIN_BUBBLEWRAP_VERSION=0.9.0-1ubuntu0.3
+1 -1
View File
@@ -35,6 +35,6 @@ if [ "${1:-}" = business ]; then
exec node "$REPO/packages/business/src/cli.mjs" "$@"
fi
case "${1:-}" in
usage|project|limits|inbox|decide|tasks|agents|trail|talk|stop|launches|bus|plugins|credentials|name|names|follow|threads|thread) exec node "$REPO/packages/cli/src/cli.mjs" "$@" ;;
usage|project|limits|inbox|decide|tasks|agents|trail|talk|stop|launches|bus|plugins|credentials|hosts|name|names|follow|threads|thread) exec node "$REPO/packages/cli/src/cli.mjs" "$@" ;;
esac
exec node "$REPO/packages/seat/src/cli.mjs" "$@"
+27 -5
View File
@@ -130,7 +130,7 @@ parse_opts() {
--sso) [ "$OP" = activate ] || usage; SSO=1; shift ;;
--migrate) [ "$OP" = activate ] || usage; MIGRATE=1; shift ;;
--restore) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])?$ ]] || usage; RESTORE="$2"; shift 2 ;;
--backup) [ "$OP" = rehearse ] && [ $# -ge 2 ] && { [[ "$2" =~ ^[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])?$ ]] || [[ "$2" =~ ^/[A-Za-z0-9/._+-]*$ ]]; } || usage; R_BACKUP="$2"; shift 2 ;;
--backup) [ "$OP" = rehearse ] && [ $# -ge 2 ] && { [ "$2" = last ] || [[ "$2" =~ ^[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])?$ ]] || [[ "$2" =~ ^/[A-Za-z0-9/._+-]*$ ]]; } || usage; R_BACKUP="$2"; shift 2 ;;
--port) [ "$OP" = rehearse ] && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9]{1,5}$ ]] && [ "$((10#$2))" -ge 1024 ] && [ "$((10#$2))" -le 65535 ] || usage; R_PORT="$((10#$2))"; shift 2 ;;
--events) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ ^(0|[1-9][0-9]*)$ ]] || usage; EVENTS="$2"; shift 2 ;;
--units) [ "$OP" != rehearse ] && [ $# -ge 2 ] && { [ "$2" = user ] || [ "$2" = external ]; } || usage; UNITS="$2"; shift 2 ;;
@@ -220,7 +220,8 @@ for(const k of open.keys())process.stdout.write(k+"\n");' -- "$LOG"
# Reads the log's ok lines, newest first. `previous` prints rollback's target: the first `to`
# that is not the pointer's copy (release.sh's rule). `backup` prints the backup field of the
# line that made the pointer's copy live, the backup a rollback would restore. `arrival` prints the
# line that made the pointer's copy live, the backup a rollback would restore. `newest` prints the
# backup field of the newest line that has one, rehearse's --backup last (row 233). `arrival` prints the
# activation and `from` of the line that moved the stack onto the pointer's copy from another one,
# past any same-copy re-pin after it (row 200, sage-1389).
from_log() {
@@ -233,6 +234,7 @@ for(const l of lines.reverse()){
if(o.result!=="ok"||!["adopt","activate","rollback"].includes(o.op)||!o.to)continue;
if(mode==="previous"&&o.to!==current){process.stdout.write(o.to+"\n");process.exit(0)}
if(mode==="backup"&&o.to===current){if(o.backup)process.stdout.write(o.backup+"\n");process.exit(o.backup?0:1)}
if(mode==="newest"&&o.backup){process.stdout.write(o.backup+"\n");process.exit(0)}
if(mode==="arrival"&&o.to===current&&o.from!==current){process.stdout.write(`${o.activation??""} ${o.from??""}\n`);process.exit(0)}
}
process.exit(1);' -- "$LOG" "$1" "$P_COPY"
@@ -927,12 +929,13 @@ sso_checks() {
sso_supported || { check_refuse sso-target "$TO's Console does not support --sso"; return 0; }
local cfg="${MOSAIC_CONFIG:-$HOME/.config/mosaic-dev/config.json}" out id slot path why
# N4 (webby-0019, sage-1403): a pre-phase-1 loader refuses a console key, so with the key not yet
# written and the stack on another copy, the first step is the re-pin without --sso.
local first=""
# written and the stack on another copy, the first step is the re-pin without --sso. On a named
# instance both commands carry --instance NAME (row 238).
local first="" inst="${INSTANCE_FLAGS[*]:+ ${INSTANCE_FLAGS[*]}}"
if ! out="$(node -e "$SSO_JS" -- "$COPIES/$TO" "$cfg" 2>&1)"; then
out="${out%%$'\n'*}"
[ "$out" != "there is no console.sso in the system config" ] || [ "$P_COPY" = "$TO" ] ||
first="; activate $TO without --sso first, then write console.sso, then run activate $TO --sso"
first="; activate $TO$inst without --sso first, then write console.sso, then run activate $TO$inst --sso"
check_refuse sso-config "$TO's Console would refuse its --sso start: $out$first"
return 0
fi
@@ -1262,6 +1265,8 @@ plan() {
echo "needs: --migrate$([ "$MIGRATE" = 1 ] && echo ' (given)' || echo ' (not given; the run would roll back with migration-needs-flag)')"
echo "needs: Jason present for this run (decision 96 condition 2)"
[ "$MIGRATE" = 1 ] || early=1
# Row 233: root reads only the plan's exit, so under external a missing --migrate refuses here.
[ "$UNITS" = user ] || [ "$MIGRATE" = 1 ] || PLAN_REFUSE+=("migrate-required: the store is $S_SCHEMA and $TO migrates it to $T_SCHEMA; the go names --migrate")
elif [ "$OP" = activate ]; then
echo "migration: none possible; $S_SCHEMA is not in $T_SCHEMA's migration chain, so the run would roll back with store-newer"
early=1
@@ -1452,6 +1457,13 @@ ext_pointer_checks() {
local open
open="$(interrupted | paste -sd' ' -)"
[ -z "$open" ] || check_refuse interrupted-run "activation $open started and has no final line; Jason or Mos settles it (status shows it)"
# Row 233: root can't read the stack's state, so the plan lists any rollback set still waiting
# for units, whatever the log says.
local m sets=""
local -a markers
mapfile -t markers < <(compgen -G "$STATE/stack-rollback/*/AWAITING-UNITS" || true)
for m in "${markers[@]}"; do sets+=" $(basename "$(dirname "$m")")"; done
[ -z "$sets" ] || check_refuse open-activation "rollback set(s)$sets hold AWAITING-UNITS; root finishes them with confirm or revert first"
}
# A3's checks before `started`. Root has stopped the units and installs them after this run, so
@@ -1973,6 +1985,9 @@ r_env() { env -i HOME="$HOME" PATH="$PATH" MOSAIC_CONFIG="$R/config/config.json"
r_migration() {
local copy="$COPIES/$TO" old="$COPIES/$P_COPY" v out rc
v="$(node -e "$STORE_JS" -- version "$R/data/bus/bus.sqlite" 2>&1)" || r_abort store-unreadable "$v"
# Row 233 (sage-1653): which store the rehearsal ran against, for root's step log. Output only:
# the rehearsal's own steps.log has the backup's path on its backup line.
echo "BACKUP_STORE $R_BACKUP schema $v"
if [ "$v" = "$T_SCHEMA" ]; then r_pass migration "not needed: the backup is already $v"; return 0; fi
in_chain "$v" || r_abort store-newer "the backup is $v, which $TO's migration chain does not start from"
(umask 077; mkdir "$R/refuse" "$R/refuse/bus" && cp "$R_BK"/bus/* "$R/refuse/bus/") || r_abort scratch-build "cannot copy the backup to $R/refuse"
@@ -2043,6 +2058,13 @@ do_rehearse() {
trap 'r_fail signal "the rehearsal was interrupted"; r_end' INT TERM HUP
read_pointer || r_abort no-pointer "no $POINTER; the pointer's copy is the old copy the checks use"
[ "$P_BUSINESS" = "$BUSINESS" ] || r_abort business-mismatch "the pointer is for $P_BUSINESS"
# Row 233 (sage-1650, sage-1653): last is the newest backup the log records. With none, exit 3
# and no directory, the one rehearse refusal stack-upgrade.sh reads as no-backup.
if [ "$R_BACKUP" = last ]; then
R_BACKUP="$(from_log newest)" || { r_fail no-backup "the log records no backup"; R_RC=3; r_end; }
[[ "$R_BACKUP" =~ ^[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])?$ ]] || r_abort backup-unreadable "the log's newest backup is not an activation id"
r_pass backup-last "the log's newest backup is $R_BACKUP"
fi
copy_exists "$TO" 2>/dev/null || r_abort copy-missing "no copy $TO in $COPIES"
copy_ok "$TO" 2>/dev/null || r_abort copy-unreachable "$TO is not reachable from refactor and origin/refactor"
target_schema "$TO" || r_abort copy-schema-unknown "cannot read SCHEMA_VERSION and its chain from $COPIES/$TO/packages/bus/src/store.mjs"
+348
View File
@@ -0,0 +1,348 @@
#!/usr/bin/env bash
# Row 233 (I3, #1735): root's upgrade and rollback for a stack whose units are root's system units
# (stack-release.sh --units external, row 232). r6's activate A1-A5, A2f, R1-R4 and rollback
# (agents/darkwing/work/queue-195/DESIGN.md), as the I3 brief changes them
# (docs/plans/2026-10-20_installer-rows.md). Deviations and decisions:
# agents/darkwing/work/queue-233/BUILD.md.
#
# stack-upgrade.sh upgrade --instance dev --bundle FILE --sha SHA [--migrate] [--no-rehearsal]
# [--rehearse-port N] [--timeout S]
# stack-upgrade.sh rollback --instance dev --to ID [--restore ACT [--events N]] [--timeout S]
#
# Root runs it from a root-owned checkout of the reviewed commit, never from a copy, because the
# stack's user can write the copies. The unit values come from /etc/mosaic-stack/<instance>.args,
# which I2's install writes: one case2-unit.sh argument per line, the serve flags after `--`. Only
# --copy changes, and the file changes only after confirm passes. The stack's user runs its steps
# from its own clone, <home>/src/mosaic-stack, through sudo. Root reads none of that user's output
# but exit codes and the rehearsal's BACKUP_STORE line (sage-1653), which it copies into its step
# log after a match against a fixed pattern and decides nothing on.
#
# --no-rehearsal is for a stack whose log records no backup. The rehearsal runs either way: exit 3
# from it (no-backup) goes on only with the flag, and exit 0 with the flag refuses
# rehearsal-required (sage-1650, sage-1652).
#
# The lock /run/mosaic-stack/upgrade-<instance>.lock is held from before A1 to the end of the run,
# rollback included; every child runs with the lock's descriptor closed. The step log
# /var/log/mosaic-stack/upgrade-<instance>.log gets one line per step, `<time> <step> rc=N argv:
# <argv>`, and a last `<time> result exit=N <code> <text>` line.
#
# Exit 0 ok, 1 failed with the old stack running, 2 rollback-failed with the units stopped,
# 3 refused with nothing stopped, 4 usage, 5 args-failed: the new stack is live and confirmed, but
# the args file still names the old copy.
#
# Test seams, for packages/cli/tests/stack-upgrade.test.mjs only. Run as root, it refuses any:
# STACK_UPGRADE_ETC default /etc/mosaic-stack
# STACK_UPGRADE_RUN default /run/mosaic-stack
# STACK_UPGRADE_LOG_DIR default /var/log/mosaic-stack
# STACK_UPGRADE_UNIT_DIR default /etc/systemd/system
# STACK_UPGRADE_HOSTCHECK default /usr/local/lib/mosaic-stack/hostcheck.sh (row 240)
# STACK_UPGRADE_RENDER default scripts/case2-unit.sh in this checkout
# STACK_UPGRADE_SUDO default sudo
# STACK_UPGRADE_SYSTEMCTL default systemctl
set -uo pipefail
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
usage() {
cat >&2 <<'USAGE'
usage: stack-upgrade.sh upgrade --instance dev --bundle FILE --sha SHA [--migrate] [--no-rehearsal] [--rehearse-port N] [--timeout S]
stack-upgrade.sh rollback --instance dev --to ID [--restore ACT [--events N]] [--timeout S]
USAGE
exit 4
}
if [ "$(id -u)" = 0 ]; then
for v in ${!STACK_UPGRADE_@}; do echo "stack-upgrade: refusing: $v is a test seam, and root takes none" >&2; exit 3; done
fi
ETC="${STACK_UPGRADE_ETC:-/etc/mosaic-stack}"
RUN_DIR="${STACK_UPGRADE_RUN:-/run/mosaic-stack}"
LOG_DIR="${STACK_UPGRADE_LOG_DIR:-/var/log/mosaic-stack}"
UNIT_DIR="${STACK_UPGRADE_UNIT_DIR:-/etc/systemd/system}"
HOSTCHECK="${STACK_UPGRADE_HOSTCHECK:-/usr/local/lib/mosaic-stack/hostcheck.sh}"
RENDER="${STACK_UPGRADE_RENDER:-$REPO/scripts/case2-unit.sh}"
SUDO="${STACK_UPGRADE_SUDO:-sudo}"
SYSTEMCTL="${STACK_UPGRADE_SYSTEMCTL:-systemctl}"
ID_RE='^[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])?$'
ARGV=("$@")
[ $# -gt 0 ] || usage
OP="$1"; shift
case "$OP" in upgrade|rollback) ;; *) usage ;; esac
INSTANCE="" BUNDLE="" SHA="" MIGRATE=0 NO_REHEARSAL=0 R_PORT=7339 TIMEOUT=60 TO="" RESTORE="" EVENTS=""
while [ $# -gt 0 ]; do
case "$1" in
--instance) [ $# -ge 2 ] || usage; INSTANCE="$2"; shift 2 ;;
--bundle) [ "$OP" = upgrade ] && [ $# -ge 2 ] && [[ "$2" =~ ^/[A-Za-z0-9/._+-]+$ ]] || usage; BUNDLE="$2"; shift 2 ;;
--sha) [ "$OP" = upgrade ] && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9a-f]{40}$ ]] || usage; SHA="$2"; shift 2 ;;
--migrate) [ "$OP" = upgrade ] || usage; MIGRATE=1; shift ;;
--no-rehearsal) [ "$OP" = upgrade ] || usage; NO_REHEARSAL=1; shift ;;
--rehearse-port) [ "$OP" = upgrade ] && [ $# -ge 2 ] && [[ "$2" =~ ^[1-9][0-9]{3,4}$ ]] && [ "$2" -le 65535 ] || usage; R_PORT="$2"; shift 2 ;;
--timeout) [ $# -ge 2 ] && [[ "$2" =~ ^[1-9][0-9]{0,4}$ ]] || usage; TIMEOUT="$2"; shift 2 ;;
--to) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ ^[0-9a-f]{40}$ ]] || usage; TO="$2"; shift 2 ;;
--restore) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ $ID_RE ]] || usage; RESTORE="$2"; shift 2 ;;
--events) [ "$OP" = rollback ] && [ $# -ge 2 ] && [[ "$2" =~ ^(0|[1-9][0-9]*)$ ]] || usage; EVENTS="$2"; shift 2 ;;
*) usage ;;
esac
done
# The default instance only, whose name is dev (row 190); a named instance's units are a later row.
[ "$INSTANCE" = dev ] || usage
if [ "$OP" = upgrade ]; then [ -n "$BUNDLE" ] && [ -n "$SHA" ] || usage; else [ -n "$TO" ] || usage; fi
[ -z "$EVENTS" ] || [ -n "$RESTORE" ] || usage
# --- The step log and the lock ---
umask 027
LOG="$LOG_DIR/upgrade-$INSTANCE.log"
LOCK="$RUN_DIR/upgrade-$INSTANCE.lock"
{ mkdir -p "$LOG_DIR" "$RUN_DIR" && touch "$LOG" && chmod 0640 "$LOG"; } 2>/dev/null ||
{ echo "stack-upgrade: refusing log-failed: cannot write $LOG or make $RUN_DIR; nothing was changed" >&2; exit 3; }
logl() { printf '%s %s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$*" >> "$LOG" || true; }
# The run's last line, in the log and on the terminal, then the exit.
finish() {
local rc="$1" code="$2"; shift 2
logl "result exit=$rc $code $*"
if [ "$rc" = 0 ]; then echo "stack-upgrade: $*"; else echo "stack-upgrade: $code: $*" >&2; fi
exit "$rc"
}
refuse() { local code="$1"; shift; finish 3 "$code" "$* (nothing was stopped)"; }
# One step: $1 its name, the rest the command, run with the lock's descriptor closed. One log line
# with its exit and argv; the command's output goes to the terminal only.
step() {
local name="$1" rc=0; shift
"$@" 9>&- || rc=$?
logl "$name rc=$rc argv:$(printf ' %q' "$@")"
return "$rc"
}
exec 9>> "$LOCK" || refuse lock-failed "cannot open $LOCK"
flock -n 9 || refuse upgrade-running "another stack-upgrade.sh run holds $LOCK"
logl "start rc=0 argv:$(printf ' %q' "$0" "${ARGV[@]}")"
# --- The args file and the stack's user ---
# I2's file: one case2-unit.sh argument per line, owned by root with no group or other write. Sets
# ARGS (the lines), S_USER, S_HOME, BUSINESS, NODE and ARGS_COPY, and the paths derived from them.
read_args() {
local f="$ETC/$INSTANCE.args" st i copies=0 v
[ -f "$f" ] && [ ! -L "$f" ] || refuse args-unreadable "no regular file $f"
st="$(stat -c '%u %a' "$f")" || refuse args-unreadable "cannot stat $f"
[ "${st% *}" = "$(id -u)" ] && (( (8#${st#* } & 8#022) == 0 )) || refuse args-unreadable "$f must be $(id -un)'s with no group or other write (it is uid ${st% *}, mode ${st#* })"
mapfile -t ARGS < "$f" || refuse args-unreadable "cannot read $f"
S_USER="" S_HOME="" BUSINESS="" NODE="" ARGS_COPY=""
for ((i = 0; i < ${#ARGS[@]}; i++)); do
case "${ARGS[i]}" in
--) break ;;
--pm) ;;
--user|--home|--business|--copy|--node|--profile|--claude)
v="${ARGS[i + 1]:-}"
case "${ARGS[i]}" in
--user) S_USER="$v" ;; --home) S_HOME="$v" ;; --business) BUSINESS="$v" ;; --node) NODE="$v" ;;
--copy) ARGS_COPY="$v"; copies=$((copies + 1)) ;;
esac
i=$((i + 1)) ;;
*) refuse args-unreadable "$f has ${ARGS[i]} before --, which case2-unit.sh does not take" ;;
esac
done
[ "$copies" = 1 ] || refuse args-unreadable "$f needs exactly one --copy before --, not $copies"
[[ "$S_USER" =~ ^[a-z_][a-z0-9_-]*$ ]] && [[ "$S_HOME" =~ ^/[A-Za-z0-9/._+-]+$ ]] && [[ "$BUSINESS" =~ ^[a-z0-9][a-z0-9-]*$ ]] &&
[[ "$NODE" =~ ^/[A-Za-z0-9/._+-]+$ ]] && [[ "$ARGS_COPY" =~ ^[0-9a-f]{40}$ ]] ||
refuse args-unreadable "$f lacks a plain --user, --home, --business, --node or --copy"
COPIES_DIR="$S_HOME/.local/share/mosaic-stack/copies"
CLONE="$S_HOME/src/mosaic-stack"
SR="$CLONE/scripts/stack-release.sh"
BUS_UNIT="mosaic-bus@$BUSINESS.service" CON_UNIT=mosaic-webui.service
# The stack's user's steps: its own HOME and default config, the pinned node first on PATH.
AS_USER=("$SUDO" -u "$S_USER" -H env "PATH=$(dirname "$NODE"):/usr/local/bin:/usr/bin:/bin")
}
# RENDER_ARGS: the args file's lines with copy $1 in place of --copy's value.
args_with() {
local i past=""
RENDER_ARGS=()
for ((i = 0; i < ${#ARGS[@]}; i++)); do
RENDER_ARGS+=("${ARGS[i]}")
[ "${ARGS[i]}" != -- ] || past=1
if [ -z "$past" ] && [ "${ARGS[i]}" = --copy ]; then RENDER_ARGS+=("$1"); i=$((i + 1)); fi
done
}
# The new args file, written beside the old one before anything stops, moved into place only after
# confirm passes. $1 is the copy.
prepare_args() {
args_with "$1"
NEW_ARGS="$(mktemp "$ETC/.$INSTANCE.args.XXXXXX")" &&
printf '%s\n' "${RENDER_ARGS[@]}" > "$NEW_ARGS" && chmod --reference="$ETC/$INSTANCE.args" "$NEW_ARGS" ||
{ rm -f "${NEW_ARGS:-}"; refuse args-failed "cannot write the new args file in $ETC"; }
}
drop_args() { rm -f "${NEW_ARGS:-}"; }
# --- A1 ---
# OLD: the copy both installed units run, from their WorkingDirectory under the stack's copies.
read_old() {
local f ids=() id n
for f in "$UNIT_DIR/[email protected]" "$UNIT_DIR/$CON_UNIT"; do
[ -f "$f" ] && [ ! -L "$f" ] || refuse units-unreadable "no regular file $f"
n="$(grep -c '^WorkingDirectory=' "$f")"
id="$(awk -v c="WorkingDirectory=$COPIES_DIR/" 'index($0, c) == 1 { print substr($0, length(c) + 1) }' "$f")"
[ "$n" = 1 ] && [[ "$id" =~ ^[0-9a-f]{40}$ ]] || refuse units-unreadable "$f names no copy under $COPIES_DIR"
ids+=("$id")
done
[ "${ids[0]}" = "${ids[1]}" ] || refuse units-disagree "$UNIT_DIR's units name ${ids[0]} and ${ids[1]}"
OLD="${ids[0]}"
logl "units rc=0 old: $OLD"
[ "$ARGS_COPY" = "$OLD" ] || refuse args-differ "$ETC/$INSTANCE.args names $ARGS_COPY and the units name $OLD"
}
# r6 A1 and row 240: the host's profile still matches, on every host.
hostcheck() {
[ -f "$HOSTCHECK" ] && [ -x "$HOSTCHECK" ] || refuse profile-drift "no executable $HOSTCHECK (row 240)"
step hostcheck "$HOSTCHECK" --check || refuse profile-drift "$HOSTCHECK --check failed"
}
# The stack's user's --plan for the run, $@ its op and arguments; any refusal stops here.
plan() {
local rc=0
step plan "${AS_USER[@]}" "$SR" "$@" --units external --copies "$COPIES_DIR" --plan --expect-from "$OLD" || rc=$?
[ "$rc" = 0 ] || refuse plan-refused "the stack's user's $1 --plan exited $rc; its output above names the refusal"
}
# --- The units ---
stop_wait() {
local u s deadline=$((SECONDS + TIMEOUT)) rc=0
step stop-console "$SYSTEMCTL" stop "$CON_UNIT" || rc=1
step stop-bus "$SYSTEMCTL" stop "$BUS_UNIT" || rc=1
for u in "$CON_UNIT" "$BUS_UNIT"; do
while :; do
s="$("$SYSTEMCTL" is-active "$u" 9>&- 2>/dev/null)"
case "$s" in inactive|failed) break ;; esac
[ "$SECONDS" -lt "$deadline" ] || { rc=1; break; }
sleep 1
done
case "$s" in inactive|failed) logl "wait-down rc=0 unit: $u state: $s" ;; *) logl "wait-down rc=1 unit: $u state: ${s:-unknown}" ;; esac
done
return "$rc"
}
stop_units() { "$SYSTEMCTL" stop "$CON_UNIT" 9>&- || true; "$SYSTEMCTL" stop "$BUS_UNIT" 9>&- || true; logl "stop-units rc=0 units: $CON_UNIT $BUS_UNIT"; }
# A2f: the old units back on the copy they never left, gated on a start at or after this second.
# An A2 that timed out leaves a unit on its old start, which the gate refuses as stale-start.
a2f() {
local cause="$1" since
drop_args
since="$(date -u +%s)"
step start-bus "$SYSTEMCTL" start "$BUS_UNIT"
step start-console "$SYSTEMCTL" start "$CON_UNIT"
if step gate "${AS_USER[@]}" "$SR" gate "$OLD" "$BUSINESS" --units external --copies "$COPIES_DIR" --since "$since" --timeout "$TIMEOUT"; then
finish 1 failed "$cause failed; $OLD is running again and passed the gate"
fi
stop_units
finish 2 rollback-failed "$cause failed and $OLD failed its gate after the restart; both units are stopped"
}
# R1-R4: back to OLD after the stack's user staged $1. $2 is the cause: A3, A4 or A5.
rollback_to_old() {
local id="$1" cause="$2"
drop_args
stop_wait || { stop_units; finish 2 rollback-failed "$cause failed and the units did not stop for the rollback; both units are stopped"; }
step revert "${AS_USER[@]}" "$SR" revert "$id" "$BUSINESS" --units external --copies "$COPIES_DIR" ||
{ stop_units; finish 2 rollback-failed "$cause failed and revert failed; both units are stopped"; }
args_with "$OLD"
step render-old "$RENDER" "${RENDER_ARGS[@]}" ||
{ stop_units; finish 2 rollback-failed "$cause failed and the render of $OLD failed; both units are stopped"; }
# confirm --rollback exits 1 when OLD passed its gate and 2 when it failed; anything else is a
# refusal, with OLD's gate never run.
local rc=0
step confirm-rollback "${AS_USER[@]}" "$SR" confirm "$id" "$BUSINESS" --units external --copies "$COPIES_DIR" --rollback "$OLD" --cause "$cause" --timeout "$TIMEOUT" || rc=$?
[ "$rc" != 1 ] || finish 1 failed "$cause failed; rolled back to $OLD, which passed the gate"
stop_units
finish 2 rollback-failed "$cause failed and the rollback to $OLD did not pass (confirm --rollback exited $rc); both units are stopped"
}
# A2 to A5 for copy $1, once A1 and any rehearsal passed. $@ after it is the stack's user's op.
run_steps() {
local id="$1" rc=0; shift
prepare_args "$id"
# A2
stop_wait || a2f A2
# A3
step stage "${AS_USER[@]}" "$SR" "$@" --units external --copies "$COPIES_DIR" || rc=$?
case "$rc" in
0) ;;
3) a2f A3 ;;
*) rollback_to_old "$id" A3 ;;
esac
# A4
args_with "$id"
step render "$RENDER" "${RENDER_ARGS[@]}" || rollback_to_old "$id" A4
# A5
step confirm "${AS_USER[@]}" "$SR" confirm "$id" "$BUSINESS" --units external --copies "$COPIES_DIR" --timeout "$TIMEOUT" ||
rollback_to_old "$id" A5
if mv -f "$NEW_ARGS" "$ETC/$INSTANCE.args"; then logl "args rc=0 copy: $id"
else logl "args rc=1 copy: $id"; finish 5 args-failed "$id is live and confirmed, but $ETC/$INSTANCE.args still names $OLD; put $NEW_ARGS in its place"
fi
}
do_upgrade() {
read_args
# Before A1: the bundle into the stack's user's clone, then the copy from it (I1).
step fetch "${AS_USER[@]}" git -C "$CLONE" fetch -q "$BUNDLE" refs/heads/refactor:refs/remotes/origin/refactor ||
refuse fetch-failed "the stack's user could not fetch $BUNDLE into $CLONE"
step ff "${AS_USER[@]}" git -C "$CLONE" merge -q --ff-only "$SHA" ||
refuse fetch-failed "the stack's user could not fast-forward $CLONE's refactor to $SHA"
step copy "${AS_USER[@]}" "$CLONE/scripts/stack-copy.sh" copy --bundle "$BUNDLE" "$SHA" --copies "$COPIES_DIR" ||
refuse copy-failed "the stack's user could not make copy $SHA from $BUNDLE"
# A1
local head
head="$(git -C "$REPO" rev-parse HEAD 2>/dev/null)"
[ "$head" = "$SHA" ] || refuse head-mismatch "this checkout, $REPO, is at ${head:-unknown}, not $SHA"
[ -x "$COPIES_DIR/$SHA/scripts/mosaic" ] || refuse copy-missing "no $COPIES_DIR/$SHA/scripts/mosaic"
hostcheck
read_old
local -a migrate=()
[ "$MIGRATE" = 0 ] || migrate=(--migrate)
plan activate "$SHA" "$BUSINESS" "${migrate[@]}"
rehearse
run_steps "$SHA" activate "$SHA" "$BUSINESS" "${migrate[@]}"
finish 0 ok "upgrade ok: $SHA is live for $BUSINESS"
}
# The rehearsal against the log's newest backup, before anything stops. Its output goes to the
# terminal; root keeps only the BACKUP_STORE line, so the go and the log show which store it used.
rehearse() {
local cap rc bs
cap="$(mktemp "$RUN_DIR/rehearse-$INSTANCE.XXXXXX")" || refuse rehearsal-failed "cannot make a capture file in $RUN_DIR"
local -a cmd=("${AS_USER[@]}" "$SR" rehearse "$SHA" "$BUSINESS" --copies "$COPIES_DIR" --backup last --port "$R_PORT" --timeout "$TIMEOUT")
"${cmd[@]}" 9>&- 2>&1 | tee "$cap" 9>&-
rc="${PIPESTATUS[0]}"
logl "rehearse rc=$rc argv:$(printf ' %q' "${cmd[@]}")"
bs="$(grep -E '^BACKUP_STORE [0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}(-[1-9])? schema [0-9a-z]{1,16}$' "$cap" | tail -n 1)"
rm -f "$cap"
logl "rehearse-store rc=0 ${bs:-BACKUP_STORE none}"
case "$rc" in
0) [ "$NO_REHEARSAL" = 0 ] || refuse rehearsal-required "the log records a backup and the rehearsal ran; --no-rehearsal is only for a stack with none" ;;
3) [ "$NO_REHEARSAL" = 1 ] || refuse no-backup "the log records no backup to rehearse against; only a go that names --no-rehearsal goes on"
logl "rehearsal-skipped rc=0 the log records no backup and the go names --no-rehearsal" ;;
*) refuse rehearsal-failed "the rehearsal exited $rc" ;;
esac
}
# r6 rollback: to the log's previous copy, which the stack's user checks against --to. No HEAD
# check: the target is an older copy than this checkout, and no rehearsal.
do_rollback() {
read_args
[ -x "$COPIES_DIR/$TO/scripts/mosaic" ] || refuse copy-missing "no $COPIES_DIR/$TO/scripts/mosaic"
hostcheck
read_old
local -a restore=()
[ -z "$RESTORE" ] || restore=(--restore "$RESTORE")
[ -z "$EVENTS" ] || restore+=(--events "$EVENTS")
plan rollback "$BUSINESS" --to "$TO" "${restore[@]}"
run_steps "$TO" rollback "$BUSINESS" --to "$TO" "${restore[@]}"
finish 0 ok "rollback ok: $TO is live for $BUSINESS"
}
case "$OP" in
upgrade) do_upgrade ;;
rollback) do_rollback ;;
esac
+10 -1
View File
@@ -71,6 +71,8 @@ EOF_PROBE
mkdir -p "$SANDBOX/toolroot/plans"
echo "row one" >"$SANDBOX/toolroot/plans/QUEUE.md"
TOOLS_JSON="{\"roots\":[{\"name\":\"docs\",\"path\":\"$SANDBOX/toolroot\"}],\"maxFileBytes\":4096,\"maxCallsPerTurn\":8}"
export PI_CODING_AGENT_DIR="$SANDBOX/pi-agent"
mkdir -p "$PI_CODING_AGENT_DIR"
PI_COMMON="--mode rpc --no-extensions --no-context-files --no-skills --no-prompt-templates --no-themes --offline --no-session --provider zai --model glm-5.3"
if [ -x "$PI_BIN" ]; then
printf '{"type":"get_state","id":"a"}\n' | MOSAIC_DISCORD_TOOLS="$TOOLS_JSON" timeout 60 "$PI_BIN" $PI_COMMON --no-builtin-tools \
@@ -142,7 +144,14 @@ fi
# A nested `node --test` inherits a parent runner's NODE_TEST_CONTEXT, reports
# to that runner and exits 0 whatever its tests do, so the suite clears it
# (#1508 N13). The planted failing test proves a failure still fails here.
node_tests() { env -u NODE_TEST_CONTEXT node --test "$@"; }
node_tests() {
local lane_concurrency=()
if [ -n "${MOSAIC_TEST_CONCURRENCY:-}" ]; then
[ "$MOSAIC_TEST_CONCURRENCY" = 2 ] || return 2
lane_concurrency=(--test-concurrency=2)
fi
env -u NODE_TEST_CONTEXT node --test "${lane_concurrency[@]}" "$@"
}
mkdir -p "$SANDBOX/planted"
printf '%s\n' 'import { test } from "node:test";' 'import assert from "node:assert/strict";' 'test("planted failure", () => assert.equal(1, 2));' >"$SANDBOX/planted/planted.test.mjs"
NODE_TEST_CONTEXT=child-v8 node_tests "$SANDBOX/planted/" >"$SANDBOX/planted.log" 2>&1
+8 -1
View File
@@ -76,7 +76,14 @@ check "checked-in demo bundles equal a fresh generation" $DEMO_OK
# A nested `node --test` inherits a parent runner's NODE_TEST_CONTEXT, reports
# to that runner and exits 0 whatever its tests do, so the suite clears it
# (#1508 N13). The planted failing test proves a failure still fails here.
node_tests() { env -u NODE_TEST_CONTEXT node --test "$@"; }
node_tests() {
local lane_concurrency=()
if [ -n "${MOSAIC_TEST_CONCURRENCY:-}" ]; then
[ "$MOSAIC_TEST_CONCURRENCY" = 2 ] || return 2
lane_concurrency=(--test-concurrency=2)
fi
env -u NODE_TEST_CONTEXT node --test "${lane_concurrency[@]}" "$@"
}
mkdir -p "$SANDBOX/planted"
printf '%s\n' 'import { test } from "node:test";' 'import assert from "node:assert/strict";' 'test("planted failure", () => assert.equal(1, 2));' >"$SANDBOX/planted/planted.test.mjs"
NODE_TEST_CONTEXT=child-v8 node_tests "$SANDBOX/planted/" >"$SANDBOX/planted.log" 2>&1
+6 -1
View File
@@ -47,7 +47,12 @@ node -e 'const p=require("./packages/queue/package.json"); process.exit(p.depend
check "packages/queue declares no dependencies" $?
# --- the package tests ---
env -u NODE_TEST_CONTEXT node --test --test-reporter=spec packages/queue/tests/ > "$SANDBOX/tests.log" 2>&1
lane_concurrency=()
if [ -n "${MOSAIC_TEST_CONCURRENCY:-}" ]; then
[ "$MOSAIC_TEST_CONCURRENCY" = 2 ] || exit 2
lane_concurrency=(--test-concurrency=2)
fi
env -u NODE_TEST_CONTEXT node --test "${lane_concurrency[@]}" --test-reporter=spec packages/queue/tests/ > "$SANDBOX/tests.log" 2>&1
rc=$?
grep -E '^ℹ (tests|pass|fail) ' "$SANDBOX/tests.log"
if [ "$rc" != 0 ]; then grep -E '^✖|Error' "$SANDBOX/tests.log" | head -40; fi
+97
View File
@@ -0,0 +1,97 @@
# Gate lanes
This tooling is for Sage's granted test gates on dragon-lin. It does not grant
itself a slot. A grant reference is an audit field, not an authentication token.
Until Sage adopts these lanes, the existing shared-slot procedure still applies.
Ask Sage with the lane, candidate SHA and exact command. Get a separate grant
for image preparation, suites, and the whole-slot paired proof. New state is
`/mnt/storage/scratch/mosaic-gate-lanes`. Old slot files are untouched.
Invoke the entry from the builder or canonical repository. The --checkout target
must be a separate clean scratch clone containing identical tooling source:
```bash
scripts/testing/gate-lane.sh run --lane a --candidate FULL_SHA \
--checkout /mnt/storage/scratch/ROW-RUN/wt \
--out /mnt/storage/scratch/ROW-RUN/out --grant SAGE_MESSAGE_ID \
--image-receipt /mnt/storage/scratch/PREP/image-receipt.json
```
Each run creates a private root like `/mnt/storage/scratch/ga-ABCDEF` (30 bytes).
An explicit `--temp-root` must already be private, outside `/tmp`, and no longer
than 30 UTF-8 bytes after resolving symlinks. The sandbox's own `/tmp` and HOME
are unchanged. Host HOME stays real. A listener covering loopback port 9, an
unsafe privileged-port threshold or unreadable host inventory refuses startup.
Lane a leases ports 61000..61127, lane b 61128..61255. Refuse any band overlapping
the host's current ephemeral range or a live listener. Port-0 fixtures with a
held server keep using port 0. Never replace the literal dead Board :9 URLs.
Two Node test files run per lane, including nested shell-suite Node runners.
```bash
scripts/testing/gate-lane.sh status
scripts/testing/gate-lane.sh recover --run-id RUN_NONCE --grant SAGE_MESSAGE_ID
```
Status reports each lane, the image lock, unfinished receipts and unmanaged
test/landing/build processes as LEGACY BUSY. It prints no command lines or
environment values. A dead owner with an unfinished receipt is quarantined even
when flock is free. Recovery takes both lanes and the exclusive image lock.
Unknown process-group, scope or Docker ownership refuses recovery. Do not remove
lock files, old receipts, or another run's resources to clear a refusal.
An image receipt records exact COPY input hashes/modes, image ID, release tag,
base digest and platform. Both concurrent candidates must match those files and
that same image ID. Rebuilding identical files is not equivalent. Warm gates
refuse implicit Docker builds/pulls and recheck the ID at each step and the end.
External same-uid commands can ignore these locks. Sage must exclude other
builders; this is test coordination, not a security boundary against that uid.
Only under an explicit image-build grant:
```bash
scripts/testing/gate-lane.sh prepare-image --lane a --candidate FULL_SHA \
--checkout /mnt/storage/scratch/ROW-PREP/wt \
--out /mnt/storage/scratch/ROW-PREP/out --grant SAGE_MESSAGE_ID
```
Preparation holds both lanes and the exclusive image lock. Its build log,
receipt, count and wall time are separate from warm gate timings. The current
Containerfile uses a mutable FROM tag and apt indexes. A missing base digest
refuses provenance publication. No credential file is copied or printed.
Run records are immutable JSON files published by link after fsync. Every step
has a log, UTC/monotonic times, exit code, pass/fail/skip counts and log hash.
The first failed step stops the gate. Cleanup touches only exact project labels,
recorded process identities and fixture paths. A process leak fails the gate
even when cleanup succeeds. Failed logs and temp roots remain for diagnosis.
A cleanup failure leaves the lane quarantined. GATE_PASSED requires all steps
and cleanup. Required prerequisite skips fail the run; optional skips remain
visible and require acceptance review.
Home metadata is evidence only. Assert only on entries attributable to this
run's nonce or fixture paths. Other sessions normally change ~/.claude, ~/.pi
and ~/.cache, so there is no global unchanged-home assertion.
The row 197 acceptance packet is `agents/herb/work/queue-197/`: two candidate
solo baselines followed by three overlapping pairs, lane assignment swapped in
pair two. Keep later paired results after adoption. Six green gates establish
the initial acceptance bar, not a reliable estimate of rare flakes.
The preflight unit fixture uses the canonical checkout's existing node_modules
at `/mnt/storage/src/mosaic-stack/node_modules` through the required plain
symlink. It does not install dependencies. Unit run roots use the `gt-` prefix;
real lane roots use `ga-` or `gb-`. Resource helper tests inside a lane lease only
that lane's band through MOSAIC_GATE_STATE, including their child test files.
The exhaustion probe temporarily narrows its process-local band to a port it
already leased, so it never reserves the whole shared band.
`closed` call-site options are accepted for compatibility; every dynamic lease
reserves an unbound endpoint. Listener fixtures decide when to bind and close.
Fixture child caches put TMPDIR under `tmp/` next to `jiti/` and `cache/`.
Outside a lane, the resource helper file uses lane a with the shared default
state `/mnt/storage/scratch/mosaic-gate-lanes`. It leaves stable lock files in
place. A single-file run therefore respects a live lane-a gate's closed leases.
Its unrelated-session probe bounds both polling children at five seconds.
+226
View File
@@ -0,0 +1,226 @@
import { createHash } from 'node:crypto';
import { realpathSync, existsSync, statSync, readdirSync } from 'node:fs';
import { join, resolve, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
import { execFileSync } from 'node:child_process';
import { shortRoot, checkHost, hostFacts, refuse, within, SCRATCH, identity } from './lane-state.mjs';
const BUILDER=resolve(dirname(fileURLToPath(import.meta.url)),'../..');
const git=(root,...args)=>execFileSync('git',['-C',root,...args],{encoding:'utf8',stdio:['ignore','pipe','pipe']}).trim();
export function validateRun(o){
if(!['a','b'].includes(o.lane))refuse('invalid lane');
if(typeof o.grant!=='string'||!o.grant.trim())refuse('missing grant reference');
for(const key of ['checkout','out','stateRoot'])if(typeof o[key]!=='string'||!o[key])refuse(`missing ${key}`);
const tempRoot=shortRoot(o.tempRoot);
const checkout=realpathSync(o.checkout);
if(checkout===BUILDER||checkout==='/mnt/storage/src/mosaic-stack'||!within(SCRATCH,checkout))refuse('canonical or builder checkout');
if(!/^[a-f0-9]{40}$/.test(o.candidate)||git(checkout,'rev-parse','HEAD')!==o.candidate)refuse('wrong candidate SHA');
const changed=git(checkout,'status','--porcelain','--untracked-files=normal').split('\n').filter(x=>x&&x!=='?? node_modules');
if(changed.length)refuse('dirty checkout');
if(!existsSync(join(checkout,'node_modules'))||!statSync(join(checkout,'node_modules')).isDirectory())refuse('missing node_modules');
const out=resolve(o.out);if(existsSync(out))refuse('output already exists');
if(!within(dirname(checkout),out)||!within(SCRATCH,realpathSync(dirname(out))))refuse('output escapes run scratch');
const stateRoot=resolve(o.stateRoot);if(!within(SCRATCH,stateRoot))refuse('state outside scratch');
checkHost(o.lane,o.hostFacts??hostFacts());
return {...o,tempRoot,checkout,out,stateRoot};
}
export function validateTooling(checkout){
for(const name of readdirSync(join(BUILDER,'scripts/testing')).filter(n=>/\.(mjs|sh)$/.test(n))){
const candidate=join(checkout,'scripts/testing',name),launcher=join(BUILDER,'scripts/testing',name);
if(!existsSync(candidate)||!readFileSync(candidate).equals(readFileSync(launcher)))refuse('candidate tooling differs from launcher');
}
}
export function discoverSteps(checkout,concurrency=2){
if(concurrency!==2)refuse('test concurrency must be 2');
const steps=[];
for(const p of readdirSync(join(checkout,'packages')).sort())if(p!=='queue'&&existsSync(join(checkout,'packages',p,'tests'))&&statSync(join(checkout,'packages',p,'tests')).isDirectory())steps.push({name:`pkg-${p}`,cwd:join(checkout,'packages',p),argv:['node','--test','--test-concurrency=2',...readdirSync(join(checkout,'packages',p,'tests')).filter(n=>n.endsWith('.test.mjs')).sort().map(n=>`tests/${n}`)]});
for(const f of readdirSync(join(checkout,'scripts')).sort())if(/^test-.+\.sh$/.test(f)&&statSync(join(checkout,'scripts',f)).isFile())steps.push({name:`script-${f.slice(5,-3)}`,cwd:checkout,argv:['bash',`scripts/${f}`]});
return steps;
}
export async function executeSteps(steps,{out,env,record,onChild=()=>{},signal}){
const {openSync,closeSync}=await import('node:fs');const {spawn}=await import('node:child_process');
for(const step of steps){
if(signal?.aborted)return {rc:130};
const begin=performance.now(),startMs=Date.now();const fd=openSync(join(out,`${step.name}.log`),'wx',0o600);
const child=spawn(step.argv[0],step.argv.slice(1),{cwd:step.cwd,env,detached:true,stdio:['ignore',fd,fd]});closeSync(fd);
onChild(child);record({kind:'child',group:child.pid,owner:identity(child.pid)});
let escalation;
const abort=()=>{try{process.kill(-child.pid,'SIGTERM');}catch{};escalation=setTimeout(()=>{try{process.kill(-child.pid,'SIGKILL');}catch{}},3000);escalation.unref();};
signal?.addEventListener('abort',abort,{once:true});
const rc=await new Promise(resolve=>{child.once('error',()=>resolve(127));child.once('close',(code,sig)=>resolve(signal?.aborted?130:(code??(sig?128:1))));});
clearTimeout(escalation);signal?.removeEventListener('abort',abort);onChild(null);
const bytes=readFileSync(join(out,`${step.name}.log`)),text=bytes.toString('utf8');
const counts={};for(const name of ['pass','fail','skipped']){const m=text.match(new RegExp('^(?:ℹ |# )'+name+' (\\d+)','m'));if(m)counts[name]=Number(m[1]);}
if(counts.pass===undefined){const summaries=[...text.matchAll(/(\d+) passed, (\d+) failed/g)];const last=summaries.at(-1);if(last){counts.pass=Number(last[1]);counts.fail=Number(last[2]);}}
const requiredSkips=text.split('\n').filter(line=>/^skip\b/i.test(line)&&/(docker|daemon|sandbox|systemd|credential)/i.test(line));
record({kind:'step',name:step.name,startMs,endMs:Date.now(),wallMs:performance.now()-begin,rc,counts,requiredSkips,logSha256:createHash('sha256').update(bytes).digest('hex')});
if(rc!==0||requiredSkips.length)return {rc:rc||2};
}
return {rc:0};
}
export async function executePinnedSteps(steps,options,{receipt,checkout,inspect,afterStep=()=>{}}){
for(const step of steps){
verifyImage(receipt,checkout,inspect);
const result=await executeSteps([step],options);afterStep(step);
if(result.rc){verifyImage(receipt,checkout,inspect);return {...result,imageBuilds:0};}
}
verifyImage(receipt,checkout,inspect);return {rc:0,imageBuilds:0};
}
import { mkdirSync, lstatSync, fstatSync, mkdtempSync, writeFileSync, readFileSync, openSync, readlinkSync, rmdirSync } from 'node:fs';
import { pathToFileURL } from 'node:url';
import { spawnSync } from 'node:child_process';
import { status, appendRecord, runRecords, processInventory, sameProcess, nonce } from './lane-state.mjs';
import { verifyImage, imageFingerprint, inspectImage, prepareImage } from './lane-image.mjs';
import { cleanupProject } from './lane-docker.mjs';
import { homeMetadata, attributeEntries, socketInventory } from './lane-evidence.mjs';
function options(argv){
const result={verb:'run',stateRoot:'/mnt/storage/scratch/mosaic-gate-lanes',lane:'a',exclusive:false};
const names={'--state-root':'stateRoot','--lane':'lane','--candidate':'candidate','--checkout':'checkout','--out':'out','--temp-root':'tempRoot','--grant':'grant','--image-receipt':'imageReceipt','--run-id':'runId'};
const args=[...argv];if(args[0]&&!args[0].startsWith('--'))result.verb=args.shift();
if(!['run','status','recover','prepare-image','lock-dir'].includes(result.verb))refuse('unknown verb');
while(args.length){const key=args.shift();if(key==='--exclusive'){result.exclusive=true;continue;}if(!names[key]||!args.length)refuse('invalid or missing option');result[names[key]]=args.shift();}
return result;
}
function stateDirectory(path,create=false){
const root=resolve(path);if(!within(SCRATCH,root)||root===SCRATCH)refuse('state must be private scratch');
if(create){const parent=realpathSync(dirname(root));if(!within(SCRATCH,parent))refuse('state parent escapes scratch');mkdirSync(root,{recursive:true,mode:0o700});}
if(existsSync(root)){const st=lstatSync(root);if(st.isSymbolicLink()||!st.isDirectory()||st.uid!==process.getuid()||(st.mode&0o077))refuse('state is not a private owned directory');
for(const n of ['a.lock','b.lock','image.lock'])if(existsSync(join(root,n))&&!lstatSync(join(root,n)).isFile())refuse('lock is not an ordinary file');
}
return root;
}
export function verifyLocks(o){
for(const [fd,name] of o.exclusive?[[3,'a.lock'],[4,'b.lock'],[5,'image.lock']]:[[3,`${o.lane}.lock`],[5,'image.lock']]){
let opened,path;try{opened=fstatSync(fd);path=lstatSync(join(o.stateRoot,name));}catch{refuse('lock descriptors missing');}
if(!opened.isFile()||!path.isFile()||opened.ino!==path.ino||opened.dev!==path.dev||path.uid!==process.getuid())refuse('lock file replaced');
// A separate open must fail while our shared/exclusive lock is held.
const r=spawnSync('flock',['-n','-x',join(o.stateRoot,name),'true']);if(r.status!==1)refuse('lock descriptor is not locked');
}
}
export function activeConflicts(o,observations={}){
const current=status(o.stateRoot,{...observations,ignoreLocks:o.exclusive?['a','b','image']:[o.lane,'image']});
if(current.legacy.length)refuse('LEGACY BUSY: unmanaged gate, test or Docker process');
for(const lane of o.exclusive?['a','b']:[o.lane])if(current[lane].state!=='CLEAR')refuse(`${lane} ${current[lane].state}`);
for(const r of runRecords(o.stateRoot)){const start=r.records.find(e=>e.kind==='start');if(!r.records.some(e=>['terminal','recovered'].includes(e.kind)&&e.clean)&&start?.checkout===o.checkout)refuse('checkout already owned');}
}
export function cleanEnv(o,run,docker){
const kept=['PATH','HOME','USER','LOGNAME','SHELL','LANG','LC_ALL','LC_CTYPE','TZ','TERM','XDG_RUNTIME_DIR','DBUS_SESSION_BUS_ADDRESS','DOCKER_HOST','CHROMIUM'];
const env=Object.fromEntries(kept.filter(k=>typeof process.env[k]==='string').map(k=>[k,process.env[k]]));
Object.assign(env,{TMPDIR:o.tempRoot,TMP:o.tempRoot,TEMP:o.tempRoot,MOSAIC_GATE_LANE:o.lane,MOSAIC_GATE_STATE:o.stateRoot,MOSAIC_TEST_CONCURRENCY:'2',COMPOSE_PROJECT_NAME:run.project,JITI_FS_CACHE:join(o.tempRoot,'jiti'),XDG_CACHE_HOME:join(o.tempRoot,'cache'),MOSAIC_CONFIG:join(o.tempRoot,'missing-config.json'),MOSAIC_GATE_DOCKER_RECORD:join(o.out,'docker.json')});
env.PATH=`${join(o.out,'bin')}:${env.PATH??'/usr/bin:/bin'}`;
return env;
}
function fixtureProcesses(run){
const all=processInventory(),ancestors=new Set([process.pid]);let parent=process.pid;
while(parent){const p=all.find(p=>p.pid===parent);if(!p||ancestors.has(p.ppid))break;ancestors.add(p.ppid);parent=p.ppid;}
return all.filter(p=>{
if(ancestors.has(p.pid)||p.state==='Z')return false;
let cwd;try{cwd=readlinkSync(`/proc/${p.pid}/cwd`);}catch{}
return p.argv.some(a=>typeof a==='string'&&(a===run.tempRoot||a.startsWith(run.tempRoot+'/')))||cwd&&(within(run.tempRoot,cwd)||run.checkout&&within(run.checkout,cwd));
});
}
async function cleanProcesses(run,record){
const before=fixtureProcesses(run);
if(run.stateRoot&&run.id){
const procs=processInventory(),known=new Set(before.map(p=>p.pid));
const {records}=await import('./lane-state.mjs');
for(const child of records(join(run.stateRoot,'runs',run.id)).filter(e=>e.kind==='child')){
const members=procs.filter(p=>p.group===child.group&&p.state!=='Z');
if(members.length&&sameProcess(child.owner,procs.find(p=>p.pid===child.owner?.pid))){
try{process.kill(-child.group,'SIGKILL');}catch(e){if(e.code!=='ESRCH')throw e;}
}else if(members.some(p=>!known.has(p.pid)))refuse('unproven orphan process-group ownership');
}
}
record({kind:'leftover-processes',objects:before.map(({pid,start,boot})=>({pid,start,boot}))});
for(const p of before){if(!sameProcess(p))continue;let raw;try{raw=readFileSync(`/proc/${p.pid}/cgroup`,'utf8');}catch(e){if(e.code==='ENOENT')continue;throw e;}const scope=raw.trim().split('/').find(x=>/^mosaic-chat-[a-z0-9]+\.scope$/.test(x));
if(scope){const shown=execFileSync('systemctl',['--user','show',scope,'-p','InvocationID','--value'],{encoding:'utf8'}).trim();if(!/^[a-f0-9]{32}$/.test(shown))refuse('scope identity unknown');record({kind:'scope-cleanup',unit:scope,invocation:shown});execFileSync('systemctl',['--user','kill','--signal=SIGKILL',scope],{stdio:'pipe'});}
if(sameProcess(p))try{process.kill(p.pid,'SIGKILL');}catch(e){if(e.code!=='ESRCH')throw e;}
}
for(let n=0;n<30;n++){if(!fixtureProcesses(run).some(p=>p.state!=='Z'))return before.length===0;await new Promise(r=>setTimeout(r,100));}
refuse('owned fixture process remains');
}
async function recoverRun(o){
if(!o.grant)refuse('missing recovery grant');
if(!/^[a-f0-9]{12}$/.test(o.runId??''))refuse('invalid run ID');
const r=runRecords(o.stateRoot).find(r=>r.id===o.runId);if(!r)refuse('run not found');
const start=r.records.find(x=>x.kind==='start');if(start?.mode==='prepare-image'&&status(o.stateRoot).legacy.length)refuse('image builder still active or unknown');if(!start||sameProcess(start.owner))refuse('run owner live or unknown');
if(r.records.some(x=>['terminal','recovered'].includes(x.kind)&&x.clean))refuse('run already clean');
shortRoot(start.tempRoot);const record=e=>appendRecord(r.dir,e);
const procs=processInventory(),known=new Set(fixtureProcesses(start).map(p=>p.pid));
for(const event of r.records.filter(e=>e.kind==='child')){
const members=procs.filter(p=>p.group===event.group&&p.state!=='Z');
if(members.length&&!sameProcess(event.owner,procs.find(p=>p.pid===event.owner?.pid))&&members.some(p=>!known.has(p.pid)))refuse('orphan process-group ownership unknown');
if(members.length&&sameProcess(event.owner)){process.kill(-event.group,'SIGKILL');}
}
await cleanProcesses(start,record);cleanupProject(start,start.docker);
appendRecord(r.dir,{kind:'recovered',clean:true,grant:o.grant,owner:identity()});return {recovered:r.id};
}
async function runGate(input){
for(const key of ['checkout','out','candidate','grant','imageReceipt'])if(!input[key])refuse(`missing ${key}`);
const automatic=!input.tempRoot,tempRoot=input.tempRoot??mkdtempSync(`/mnt/storage/scratch/g${input.lane}-`);
let o,receipt,docker;
try{
o=validateRun({...input,tempRoot});
validateTooling(o.checkout);
activeConflicts(o);
receipt=JSON.parse(readFileSync(o.imageReceipt,'utf8'));verifyImage(receipt,o.checkout);
docker=execFileSync('which',['docker'],{encoding:'utf8'}).trim();
}catch(e){if(automatic)rmdirSync(tempRoot);throw e;}
const id=nonce(),run={kind:'start',id,stateRoot:o.stateRoot,lane:o.lane,exclusive:o.exclusive,owner:identity(),grant:o.grant,candidate:o.candidate,checkout:o.checkout,tempRoot:o.tempRoot,out:o.out,project:`mg197-${o.lane}-${id}`,tag:receipt.tag,imageId:receipt.imageId,docker};
const dir=join(o.stateRoot,'runs',id);appendRecord(dir,run);const record=e=>{appendRecord(dir,e);if(e.kind==='step')process.stdout.write(`${e.name} rc=${e.rc} wall_ms=${Math.round(e.wallMs)} pass=${e.counts.pass??'-'} fail=${e.counts.fail??'-'} skipped=${e.counts.skipped??'-'}\n`);};
const begin=performance.now(),startMs=Date.now();let rc=1,clean=false,current=null;
const aborter=new AbortController();const stop=()=>aborter.abort();for(const signal of ['SIGINT','SIGTERM','SIGHUP'])process.on(signal,stop);
try{
mkdirSync(o.out,{mode:0o700});mkdirSync(join(o.out,'bin'),{mode:0o700});
writeFileSync(join(o.out,'docker.json'),JSON.stringify({...run,docker}),{mode:0o600,flag:'wx'});
const guard=join(BUILDER,'scripts/testing/lane-docker.mjs');
// Launch the guard directly so it validates every real Docker invocation.
writeFileSync(join(o.out,'bin','docker'),`#!/bin/sh\nexec '${process.execPath.replaceAll("'","'\\''")}' '${guard.replaceAll("'","'\\''")}' "$@"\n`,{mode:0o700,flag:'wx'});
const steps=discoverSteps(o.checkout);record({kind:'plan',steps});
const before=homeMetadata(process.env.HOME);record({kind:'home-before',entries:before});
const env=cleanEnv(o,run,docker);
const result=await executePinnedSteps(steps,{out:o.out,env,record,onChild:c=>current=c,signal:aborter.signal},{receipt,checkout:o.checkout,afterStep:()=>record({kind:'sockets',entries:socketInventory(o.tempRoot)})});
rc=result.rc;
const after=homeMetadata(process.env.HOME);record({kind:'home-after',entries:after});
const prior=new Map(before.map(e=>[e.path,JSON.stringify(e)]));const changed=after.filter(e=>prior.get(e.path)!==JSON.stringify(e));
if(attributeEntries(changed,{...run,nonce:id,fixturePaths:[]}).owned.length)refuse('lane-attributed real HOME write');
if(git(o.checkout,'diff','--name-only'))refuse('tracked checkout changed during gate');
const noLeaks=await cleanProcesses(run,record);cleanupProject(run,docker);clean=true;if(!noLeaks&&rc===0)rc=1;
}catch(e){record({kind:'failure',message:e.message});process.stderr.write(`${e.message}\n`);rc=rc||1;
try{await cleanProcesses(run,record);cleanupProject(run,docker);clean=true;}catch(error){record({kind:'cleanup-failed',message:error.message});}
}finally{
if(current){try{process.kill(-current.pid,'SIGKILL');}catch{}}
for(const signal of ['SIGINT','SIGTERM','SIGHUP'])process.off(signal,stop);
record({kind:'terminal',clean,rc,startMs,endMs:Date.now(),wallMs:performance.now()-begin,imageBuilds:0});
}
if(rc===0&&clean)process.stdout.write('GATE_PASSED\n');return rc;
}
async function main(argv){
const locked=argv[0]==='--locked';if(locked)argv=argv.slice(1);
if(argv[0]==='lock-dir'){
const o=options(argv.slice(1));if(!o.grant)refuse('missing grant reference');if(!['a','b'].includes(o.lane))refuse('invalid lane');process.stdout.write(stateDirectory(o.stateRoot,true)+'\n');return 0;
}
const o=options(argv);o.stateRoot=stateDirectory(o.stateRoot);
if(o.verb==='status'){process.stdout.write(JSON.stringify(status(o.stateRoot))+'\n');return 0;}
if(!locked)refuse('use gate-lane.sh for locked operations');
if(['recover','prepare-image'].includes(o.verb))o.exclusive=true;
verifyLocks(o);
if(o.verb==='recover'){process.stdout.write(JSON.stringify(await recoverRun(o))+'\n');return 0;}
if(o.verb==='prepare-image'){
for(const key of ['checkout','out','candidate','grant'])if(!o[key])refuse(`missing ${key}`);
const checked=validateRun({...o,tempRoot:o.tempRoot??mkdtempSync('/mnt/storage/scratch/ga-')});activeConflicts(checked);
const id=nonce(),dir=join(checked.stateRoot,'runs',id),docker=execFileSync('which',['docker'],{encoding:'utf8'}).trim();
appendRecord(dir,{kind:'start',mode:'prepare-image',lane:checked.lane,exclusive:true,owner:identity(),grant:checked.grant,candidate:checked.candidate,checkout:checked.checkout,tempRoot:checked.tempRoot,out:checked.out,project:`mg197-${checked.lane}-${id}`,docker});
mkdirSync(checked.out,{mode:0o700});
try{
const receipt=prepareImage(checked.checkout,checked);
writeFileSync(join(checked.out,'image-receipt.json'),JSON.stringify(receipt,null,2)+'\n',{mode:0o600,flag:'wx'});
appendRecord(dir,{kind:'terminal',clean:true,rc:0,imageId:receipt.imageId,builds:1,wallMs:receipt.buildWallMs});
process.stdout.write(JSON.stringify({imageReceipt:join(checked.out,'image-receipt.json'),imageId:receipt.imageId,buildWallMs:receipt.buildWallMs})+'\n');return 0;
}catch(e){appendRecord(dir,{kind:'terminal',clean:false,rc:1,message:e.message});throw e;}
}
return runGate(o);
}
if(process.argv[1]&&import.meta.url===pathToFileURL(process.argv[1]).href)main(process.argv.slice(2)).then(rc=>process.exitCode=rc).catch(e=>{process.stderr.write(`${e.message}\n`);process.exitCode=2;});
+26
View File
@@ -0,0 +1,26 @@
#!/usr/bin/env bash
set -euo pipefail
umask 077
entry="$(cd "$(dirname "$0")" && pwd -P)/gate-lane.mjs"
verb="${1:-run}"
if [ "$verb" = status ]; then exec node "$entry" "$@"; fi
state="$(node "$entry" lock-dir "$@")"
lane=a
exclusive=0
args=("$@")
for ((i=0;i<${#args[@]};i++)); do
case "${args[i]}" in
--lane) lane="${args[i+1]:-}" ;;
--exclusive) exclusive=1 ;;
esac
done
case "$verb" in recover|prepare-image) exclusive=1;; esac
case "$lane" in a|b) ;; *) echo 'gate-lane: refused: invalid lane' >&2; exit 2;; esac
if [ "$exclusive" = 1 ]; then
exec 3>>"$state/a.lock" 4>>"$state/b.lock" 5>>"$state/image.lock"
flock -n 3 && flock -n 4 && flock -n -x 5 || { echo 'gate-lane: refused: lane or image lock busy' >&2; exit 2; }
else
exec 3>>"$state/$lane.lock" 5>>"$state/image.lock"
flock -n 3 && flock -n -s 5 || { echo 'gate-lane: refused: lane or image lock busy' >&2; exit 2; }
fi
exec node "$entry" --locked "$@"
+34
View File
@@ -0,0 +1,34 @@
import { execFileSync, spawnSync } from 'node:child_process';
import { readFileSync } from 'node:fs';
import { pathToFileURL } from 'node:url';
import { refuse } from './lane-state.mjs';
export function guardDocker(argv,run){
if(argv[0]==='info'&&argv.length===1)return argv;
if(argv[0]==='image'&&argv[1]==='inspect'&&argv[2]===run.tag&&argv.length===3)return argv;
if(argv[0]==='compose'&&argv[1]==='run'){
const service=argv.indexOf('mosaic-agent',2);
if(service<0||argv.slice(2,service).some(a=>!['--rm','-T','--no-deps','--interactive','-i'].includes(a)))refuse('Docker Compose run options not allowed');
return ['compose','run','--no-build','--pull','never',...argv.slice(2)];
}
refuse('Docker mutation or unowned command');
}
export function cleanupProject(run,docker='docker'){
if(!/^mg197-[ab]-[a-f0-9]{12}$/.test(run.project))refuse('invalid Docker ownership project');
const invoke=typeof docker==='function'?docker:(args)=>execFileSync(docker,args,{encoding:'utf8',stdio:['ignore','pipe','pipe']});
const removed=[];
for(const [kind,list,filter] of [['container',['ps','-aq'],'com.docker.compose.project'],['network',['network','ls','-q'],'com.docker.compose.project']]){
const ids=invoke([...list,'--filter',`label=${filter}=${run.project}`]).trim().split(/\s+/).filter(Boolean);
for(const id of ids){if(!/^[a-f0-9]{12,64}$/.test(id))refuse('invalid Docker resource ID');
const label=invoke([kind,'inspect',id,'--format',`{{index .${kind==='container'?'Config.':''}Labels "${filter}"}}`]).trim();
if(label!==run.project)refuse('foreign Docker resource');
invoke(kind==='container'?['container','rm','-f',id]:['network','rm',id]);removed.push({kind,id});
}
const left=invoke([...list,'--filter',`label=${filter}=${run.project}`]).trim();if(left)refuse('Docker cleanup incomplete');
}
return {clean:true,removed};
}
if(process.argv[1]&&import.meta.url===pathToFileURL(process.argv[1]).href){
try{const run=JSON.parse(readFileSync(process.env.MOSAIC_GATE_DOCKER_RECORD,'utf8'));if(process.env.COMPOSE_PROJECT_NAME!==run.project)refuse('Compose project mismatch');
const argv=guardDocker(process.argv.slice(2),run);const result=spawnSync(run.docker,argv,{stdio:'inherit'});process.exit(result.status??1);
}catch(e){process.stderr.write(`${e.message}\n`);process.exit(2);}
}
+37
View File
@@ -0,0 +1,37 @@
import { readdirSync, lstatSync, readlinkSync, existsSync } from 'node:fs';
import { join } from 'node:path';
import { within, refuse } from './lane-state.mjs';
export function attributeEntries(metadata,run){
const result={owned:[],unrelated:[],unknown:[]};
for(const e of metadata){const owns=e.path.includes(run.nonce)||e.link&&within(run.tempRoot,e.link)||(run.fixturePaths??[]).some(path=>within(path,e.path)||e.link&&within(path,e.link));result[owns?'owned':'unrelated'].push(e);}
return result;
}
export function homeMetadata(home){
const out=[];
for(const name of ['.claude','.pi','.cache']){
const root=join(home,name);if(!existsSync(root))continue;
// Two levels are evidence only. Never open or hash file contents.
const walk=(dir,depth)=>{for(const d of readdirSync(dir,{withFileTypes:true})){const path=join(dir,d.name);try{const st=lstatSync(path);out.push({path,size:st.size,mtime:st.mtimeMs,mode:st.mode,link:st.isSymbolicLink()?readlinkSync(path):null});if(d.isDirectory()&&depth)walk(path,depth-1);}catch{}}};
walk(root,1);
}
return out;
}
export function socketInventory(root){const out=[];const walk=dir=>{for(const d of readdirSync(dir,{withFileTypes:true})){const p=join(dir,d.name);if(d.isDirectory())walk(p);else if(d.isSocket()){const bytes=Buffer.byteLength(p);out.push({path:p,bytes});if(bytes>107)refuse('socket path exceeds 107 bytes');}}};walk(root);return out;}
export function summarizePairs(solos,pairs){
const byCandidate=new Map(solos.map(r=>[r.candidate,r]));
if(solos.length!==2||byCandidate.size!==2||solos.some(r=>!Number.isFinite(r.wallMs)||r.wallMs<=0))refuse('solo timing invalid');
if(pairs.length!==3)refuse('proof needs three pairs');
let first;
return pairs.map((pair,index)=>{
if(pair.length!==2||pair[0].candidate===pair[1].candidate||new Set(pair.map(r=>r.lane)).size!==2||pair.some(r=>!['a','b'].includes(r.lane)))refuse('pair needs different candidates and lanes');
for(const run of pair){
if(!run.clean||run.rc!==0||!byCandidate.has(run.candidate))refuse('proof gate failed');
if(!Number.isFinite(run.startMs)||!Number.isFinite(run.endMs)||run.endMs<=run.startMs)refuse('paired timing invalid');
}
if(index===0)first=new Map(pair.map(r=>[r.candidate,r.lane]));
if(index===1&&pair.some(r=>first.get(r.candidate)===r.lane))refuse('second pair must swap lanes');
if(Math.max(...pair.map(r=>r.startMs))>=Math.min(...pair.map(r=>r.endMs)))refuse('paired gates did not overlap');
const makespan=Math.max(...pair.map(r=>r.endMs))-Math.min(...pair.map(r=>r.startMs));
return {makespanMs:makespan,runs:pair.map(r=>({candidate:r.candidate,wallMs:r.endMs-r.startMs,slowdown:(r.endMs-r.startMs)/byCandidate.get(r.candidate).wallMs})),throughput:pair.reduce((n,r)=>n+byCandidate.get(r.candidate).wallMs,0)/makespan};
});
}
+33
View File
@@ -0,0 +1,33 @@
// Imported only by tests. The port holder lives until the fixture closes.
import { spawn } from 'node:child_process';
import { mkdirSync, realpathSync } from 'node:fs';
import { join } from 'node:path';
import { createServer } from 'node:net';
import { once } from 'node:events';
import { BANDS, within, SCRATCH, refuse } from './lane-state.mjs';
async function hold(file){
const child=spawn('flock',['-n','-E','73',file,'bash','-c','printf "held\\n"; IFS= read -r release'],{stdio:['pipe','pipe','pipe']});
let err='';child.stderr.on('data',c=>err+=c);const exited=once(child,'exit');
const ok=await new Promise((resolve,reject)=>{child.stdout.once('data',()=>resolve(true));child.once('exit',code=>code===73?resolve(false):reject(Error(`port lock holder exited ${code}: ${err}`)));child.once('error',reject);});
if(!ok)return null;
return async()=>{child.stdin.end('\n');await exited;};
}
async function probe(port){const s=createServer();try{await new Promise((resolve,reject)=>{s.once('error',reject);s.listen(port,'127.0.0.1',resolve);});const found=s.address().port;await new Promise(r=>s.close(r));return found;}catch(e){if(s.listening)await new Promise(r=>s.close(r));if(e.code==='EADDRINUSE')return null;throw e;}}
export async function leasePort(t,{lane=process.env.MOSAIC_GATE_LANE,stateRoot=process.env.MOSAIC_GATE_STATE}={}){
if(!lane){const port=await probe(0);return port;}
if(!BANDS[lane]||!stateRoot)refuse('invalid port lane state');
const state=realpathSync(stateRoot);if(!within(SCRATCH,state))refuse('port state outside scratch');
const dir=join(state,'ports',lane);mkdirSync(dir,{recursive:true,mode:0o700});
const [lo,hi]=BANDS[lane];
for(let port=lo;port<=hi;port++){
const release=await hold(join(dir,`${port}.lock`));if(!release)continue;
let actual;try{actual=await probe(port);}catch(e){await release();throw e;}
if(!actual){await release();continue;}
t.after(release);return port;
}
refuse('lane port band exhausted');
}
export function fixtureCacheEnv(root){
mkdirSync(root,{recursive:true,mode:0o700});const dir=realpathSync(root);for(const n of ['tmp','jiti','cache'])mkdirSync(join(dir,n),{recursive:true,mode:0o700});
return {TMPDIR:join(dir,'tmp'),JITI_FS_CACHE:join(dir,'jiti'),XDG_CACHE_HOME:join(dir,'cache')};
}
+44
View File
@@ -0,0 +1,44 @@
import { createHash } from 'node:crypto';
import { readFileSync, readdirSync, lstatSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { execFileSync } from 'node:child_process';
import { refuse } from './lane-state.mjs';
export function imageFingerprint(checkout){
const dockerfile=readFileSync(join(checkout,'Containerfile'),'utf8');
const copies=dockerfile.split('\n').filter(l=>/^\s*(COPY|ADD)\s/i.test(l)).map(l=>l.trim());
const expected=['COPY package.json package-lock.json ./','COPY contracts /opt/mosaic/contracts','COPY src /opt/mosaic/src','COPY adapters /opt/mosaic/adapters'];
if(JSON.stringify(copies)!==JSON.stringify(expected))refuse('image COPY inputs changed; manifest review required');
const hash=createHash('sha256');
const visit=rel=>{const path=join(checkout,rel),st=lstatSync(path);if(st.isSymbolicLink())refuse('image input is a symlink');hash.update(JSON.stringify([rel,st.mode&0o777,st.isDirectory()?'d':'f'])+'\n');if(st.isDirectory())for(const n of readdirSync(path).sort())visit(`${rel}/${n}`);else if(st.isFile())hash.update(readFileSync(path));else refuse('unsupported image input');};
for(const rel of ['Containerfile','package.json','package-lock.json','contracts','src','adapters'])visit(rel);
return hash.digest('hex');
}
export function inspectImage(tag,docker=process.env.MOSAIC_GATE_REAL_DOCKER??'docker'){
return JSON.parse(execFileSync(docker,['image','inspect',tag,'--format','{{json .}}'],{encoding:'utf8',stdio:['ignore','pipe','pipe']}));
}
export function verifyImage(receipt,checkout,inspect=inspectImage){
if(!receipt||!/^sha256:[a-f0-9]{64}$/.test(receipt.imageId??'')||!/^.+@sha256:[a-f0-9]{64}$/.test(receipt.baseDigest??'')||!/^linux\/[a-z0-9_]+$/.test(receipt.platform??'')||!/^mosaic-poc-agent:[A-Za-z0-9_.-]+$/.test(receipt.tag??''))refuse('image provenance receipt invalid');
if(receipt.inputDigest!==imageFingerprint(checkout))refuse('image inputs differ');
const actual=inspect(receipt.tag);
if(actual.Id!==receipt.imageId)refuse('pinned image ID differs');
if(`${actual.Os}/${actual.Architecture}`!==receipt.platform)refuse('image platform differs');
const release=readFileSync(join(checkout,'RELEASE'),'utf8').trim();const pkg=JSON.parse(readFileSync(join(checkout,'package.json'),'utf8'));
if(pkg.dependencies?.['@earendil-works/pi-coding-agent']&&receipt.tag!==`mosaic-poc-agent:${pkg.dependencies['@earendil-works/pi-coding-agent']}-r${release}`)refuse('image tag differs from release identity');
return receipt;
}
// Called by the CLI only while both lanes and the image lock are exclusive.
export function prepareImage(checkout,{grant,tempRoot,out},invoke){
if(!grant)refuse('missing image preparation grant');
const inputDigest=imageFingerprint(checkout);
const pkg=JSON.parse(readFileSync(join(checkout,'package.json'),'utf8'));
const release=readFileSync(join(checkout,'RELEASE'),'utf8').trim();
if(!/^\d+\.\d+\.\d+$/.test(release))refuse('invalid image release');
const tag=`mosaic-poc-agent:${pkg.dependencies['@earendil-works/pi-coding-agent']}-r${release}`;
const run=invoke??((args)=>execFileSync('docker',args,{cwd:checkout,encoding:'utf8',stdio:['ignore','pipe','pipe'],env:{PATH:process.env.PATH,HOME:process.env.HOME,...(process.env.DOCKER_HOST?{DOCKER_HOST:process.env.DOCKER_HOST}:{}),COMPOSE_PROJECT_NAME:'mg197-image-prepare',MOSAIC_IMAGE_TAG:tag,MOSAIC_DATA_ROOT:join(tempRoot,'data'),MOSAIC_PROVIDER:'fixture',MOSAIC_MODEL:'fixture',PI_AUTH_FILE:'/dev/null'}}));
const start=performance.now();let buildOutput;try{buildOutput=run(['compose','build']);}catch(e){if(out)writeFileSync(join(out,'image-build.log'),String(e.stdout??'')+String(e.stderr??''),{mode:0o600,flag:'wx'});throw e;}if(out)writeFileSync(join(out,'image-build.log'),buildOutput,{mode:0o600,flag:'wx'});const buildWallMs=performance.now()-start;
if(inputDigest!==imageFingerprint(checkout))refuse('image inputs changed while building');
const actual=JSON.parse(run(['image','inspect',tag,'--format','{{json .}}']));
const base=JSON.parse(run(['image','inspect','node:24-bookworm-slim','--format','{{json .}}']));
const receipt={inputDigest,imageId:actual.Id,tag,baseDigest:base.RepoDigests?.find(x=>/@sha256:[a-f0-9]{64}$/.test(x)),platform:`${actual.Os}/${actual.Architecture}`,grant,builds:1,buildWallMs,at:new Date().toISOString()};
verifyImage(receipt,checkout,()=>actual);return receipt;
}
+77
View File
@@ -0,0 +1,77 @@
// Scratch-only lane ownership. No live Mosaic state is read or written.
import { realpathSync, statSync, lstatSync, existsSync, readdirSync, readFileSync, mkdirSync, openSync, writeSync, fsyncSync, closeSync, linkSync, unlinkSync, renameSync } from 'node:fs';
import { resolve, join, dirname, relative, isAbsolute } from 'node:path';
import { execFileSync, spawnSync } from 'node:child_process';
import { randomBytes } from 'node:crypto';
export const SCRATCH='/mnt/storage/scratch';
export const BANDS={a:[61000,61127],b:[61128,61255]};
export function refuse(message){throw new Error(`gate-lane: refused: ${message}`);}
export function within(root,path){const r=relative(root,path);return r===''||(!r.startsWith('..')&&!isAbsolute(r));}
export function shortRoot(path){
let real;try{real=realpathSync(path);}catch{refuse('temporary root missing');}
if(real==='/tmp'||real.startsWith('/tmp/'))refuse('temporary root under /tmp');
if(Buffer.byteLength(real)>30)refuse('temporary root exceeds 30 bytes');
if(!within(SCRATCH,real)||!statSync(real).isDirectory()||statSync(real).uid!==process.getuid()||(statSync(real).mode&0o077))refuse('temporary root must be private owned scratch');
return real;
}
export function socketPath(path){if(Buffer.byteLength(path)>107)refuse('socket path exceeds 107 bytes');return path;}
export function hostFacts(){
const range=readFileSync('/proc/sys/net/ipv4/ip_local_port_range','utf8').trim().split(/\s+/).map(Number);
const unprivilegedPortStart=Number(readFileSync('/proc/sys/net/ipv4/ip_unprivileged_port_start','utf8').trim());
const text=execFileSync('ss',['-Hltn'],{encoding:'utf8'});
const listeners=text.trim().split('\n').filter(Boolean).map(line=>{const field=line.trim().split(/\s+/)[3];const m=field?.match(/^(.*):(\d+)$/);if(!m)refuse('unreadable listener inventory');return {address:m[1].replace(/^\[|\]$/g,''),port:Number(m[2])};});
return {ephemeral:range,unprivilegedPortStart,listeners};
}
export function checkHost(lane,facts){
if(!facts||!Array.isArray(facts.listeners)||facts.ephemeral?.length!==2||!facts.ephemeral.every(Number.isInteger))refuse('host inventory unavailable');
if(!Number.isInteger(facts.unprivilegedPortStart)||facts.unprivilegedPortStart<=9)refuse('unprivileged port range includes port 9');
if(facts.listeners.some(x=>x.port===9&&['127.0.0.1','0.0.0.0','*','::','::ffff:127.0.0.1'].includes(x.address.replace(/%[^%]+$/, ''))))refuse('live port 9 listener');
const [lo,hi]=BANDS[lane];const [first,last]=facts.ephemeral;
if(lo<=last&&hi>=first)refuse('lane port band overlaps ephemeral range');
if(facts.listeners.some(x=>x.port>=lo&&x.port<=hi))refuse('lane port band has a live listener');
}
export function identity(pid=process.pid){try{const s=readFileSync(`/proc/${pid}/stat`,'utf8');const f=s.slice(s.lastIndexOf(')')+2).split(' ');return {pid:Number(pid),start:f[19],boot:readFileSync('/proc/sys/kernel/random/boot_id','utf8').trim(),state:f[0]};}catch{return null;}}
export function sameProcess(a,b=identity(a?.pid)){return !!a&&!!b&&a.pid===b.pid&&a.start===b.start&&a.boot===b.boot&&b.state!=='Z';}
export function nonce(){return randomBytes(6).toString('hex');}
export function appendRecord(dir,record){
const publishedDir=dir;if(record.kind==='start'){if(existsSync(dir))refuse('run already exists');dir=dir+'.starting';}
mkdirSync(dir,{recursive:true,mode:0o700});
const file=join(dir,`${Date.now()}-${nonce()}-${record.kind}.json`);
const pending=file+'.pending',fd=openSync(pending,'wx',0o600);try{writeSync(fd,JSON.stringify({...record,at:new Date().toISOString()})+'\n');fsyncSync(fd);}finally{closeSync(fd);}
linkSync(pending,file);unlinkSync(pending);if(record.kind==='start')renameSync(dir,publishedDir);return join(publishedDir,file.split('/').at(-1));
}
export function records(dir){if(!existsSync(dir))return [];return readdirSync(dir).filter(n=>n.endsWith('.json')).sort().map(n=>{try{return JSON.parse(readFileSync(join(dir,n),'utf8'));}catch{refuse(`invalid run receipt ${n}`);}});}
export function runRecords(stateRoot){
const runs=join(stateRoot,'runs');if(!existsSync(runs))return [];
return readdirSync(runs).filter(id=>!id.endsWith('.starting')).map(id=>({id,dir:join(runs,id),records:records(join(runs,id))}));
}
export function processInventory(){
const out=[];
for(const pid of readdirSync('/proc').filter(n=>/^\d+$/.test(n))){try{const argv=readFileSync(`/proc/${pid}/cmdline`,'utf8').split('\0').filter(Boolean);const raw=readFileSync(`/proc/${pid}/stat`,'utf8');const f=raw.slice(raw.lastIndexOf(')')+2).split(' ');out.push({...identity(Number(pid)),ppid:Number(f[1]),group:Number(f[2]),argv});}catch{}}
return out;
}
export function status(stateRoot,observations={}){
const procs=observations.processes??processInventory();const runs=runRecords(stateRoot);
const answer={a:{state:'CLEAR'},b:{state:'CLEAR'},image:'free',legacy:[],at:new Date().toISOString()};
const owned=new Set();
for(const run of runs){const start=run.records.find(x=>x.kind==='start');if(!start)refuse('run receipt has no start');const clean=run.records.some(x=>['terminal','recovered'].includes(x.kind)&&x.clean===true);
if(clean)continue;
const live=procs.find(p=>sameProcess(start.owner,p));const state=live?'BUSY':'QUARANTINED';
for(const lane of start.exclusive?['a','b']:[start.lane]){if(!BANDS[lane])refuse('invalid lane receipt');answer[lane]={state,run:run.id};}
answer.image=start.exclusive?'exclusive':'shared';
if(live){owned.add(live.pid);let changed=true;while(changed){changed=false;for(const p of procs)if(owned.has(p.ppid)&&!owned.has(p.pid)){owned.add(p.pid);changed=true;}}}
}
for(const p of procs){if(p.state==='Z'||p.pid===process.pid||owned.has(p.pid))continue;
const args=p.argv??[];const executable=args[0]?.split('/').at(-1);
const test=(executable==='node'&&args.includes('--test'))||args.some(a=>/^(?:.*\/)?(?:gate\.sh|test-[a-z-]+\.sh)$/.test(a))||(executable==='docker'&&(args.includes('compose')||args.includes('build')))||args.some(a=>a.endsWith('/landing.mjs')&&args.includes('gate'))||args.some(a=>/(?:^|\/)[^/]*mutant[^/]*\.(?:mjs|py|sh)$/.test(a));
if(test)answer.legacy.push({pid:p.pid,start:p.start});
}
for(const lane of ['a','b','image']){
if(observations.ignoreLocks?.includes(lane)||!existsSync(join(stateRoot,`${lane}.lock`)))continue;
const lock=lstatSync(join(stateRoot,`${lane}.lock`));if(!lock.isFile())refuse('lock is not an ordinary file');
const probe=spawnSync('flock',['-n','-x',join(stateRoot,`${lane}.lock`),'true']);
if(probe.status!==0&&probe.status!==1)refuse('cannot inspect lane lock');
if(probe.status===1){if(lane==='image')answer.image=answer.image==='free'?'held':answer.image;else if(answer[lane].state==='CLEAR')answer[lane]={state:'BUSY',run:null};}
}
return answer;
}
+30
View File
@@ -0,0 +1,30 @@
#!/bin/bash
# Row 234 (#1740, #436a row a), design r6 E3: the installer key's forced command.
#
# Installed as /usr/local/sbin/mosaic-keys, root, 0755, and named by mosaic-installer's own line in
# /etc/ssh/mosaic-keys/mosaic-installer:
# restrict,command="/usr/local/sbin/mosaic-keys",from="<from>" ssh-ed25519 <base64> mosaic-installer-<tag>
#
# It matches SSH_ORIGINAL_COMMAND against three fixed patterns and hands the words to
# mosaic-keys-write through sudo. Nothing else runs. sudo's env_reset drops SSH_CONNECTION, so the
# connection and the request go to syslog here, before sudo. The writer checks everything again.
set -u
export PATH=/usr/bin:/bin LC_ALL=C
request=${SSH_ORIGINAL_COMMAND-}
/usr/bin/logger -t mosaic-keys -- "connection=${SSH_CONNECTION-} request=${request:0:1024}"
user='[a-z_][a-z0-9_-]{0,31}'
tag='mosaic-[0-9a-f]{32}'
key='(ssh-ed25519|ecdsa-sha2-nistp256|ssh-rsa) [A-Za-z0-9+/]+={0,2}'
list_re="^list $user\$"
add_re="^add $user $key $tag\$"
remove_re="^remove $user $tag\$"
if [[ $request =~ $list_re || $request =~ $add_re || $request =~ $remove_re ]]; then
# The patterns hold no space, glob or quote characters, so a split on spaces gives the words back.
read -r -a words <<<"$request"
exec /usr/bin/sudo -n /usr/local/sbin/mosaic-keys-write "${words[@]}"
fi
echo 'mosaic-keys: refused' >&2
exit 1
+285
View File
@@ -0,0 +1,285 @@
#!/usr/bin/python3 -I
# Row 234 (#1740, #436a row a), design r6 E3: the root half of host enrollment.
#
# Installed as /usr/local/sbin/mosaic-keys-write, root, 0755. mosaic-keys (the installer key's forced
# command) runs it through `sudo -n` with one request:
#
# list <user> each line of /etc/ssh/mosaic-keys/<user> with a mosaic- tag
# add <user> <type> <base64> <tag> appends from="<from>",restrict <type> <base64> <tag>, unless the tag exists
# remove <user> <tag> drops the line with that exact tag; a missing tag changes nothing
#
# It checks the request again, whatever mosaic-keys did. Host config is root-owned and written by ops:
# /etc/mosaic-keys/users (the users it may touch), /etc/mosaic-keys/privileged (users with sudo that
# Jason approved by name) and /etc/mosaic-keys/from (the from= value). Standard library only. Exit 0
# on success, 1 on any refusal, with the reason on stderr and in syslog.
import base64
import fcntl
import grp
import hashlib
import os
import pwd
import re
import socket
import stat
import subprocess
import sys
import syslog
KEY_DIR = '/etc/ssh/mosaic-keys'
CONF_DIR = '/etc/mosaic-keys'
USER = re.compile(r'[a-z_][a-z0-9_-]{0,31}')
TAG = re.compile(r'mosaic-[0-9a-f]{32}')
KEY_TYPE = re.compile(r'ssh-ed25519|ecdsa-sha2-nistp256|ssh-rsa')
BASE64 = re.compile(r'[A-Za-z0-9+/]+={0,2}')
FROM = re.compile(r'[0-9A-Za-z.:/*?!_-]+(,[0-9A-Za-z.:/*?!_-]+)*')
# In the code, not in a file: no request may name these, whatever users says.
RESERVED_USERS = ('root', 'localadmin', 'mosaic-installer')
MIN_UID = 1000
# Each is root by another route. disk reads and writes raw block devices, libvirt and incus-admin start
# guests with host mounts (r6 N5). privileged lifts this check and nothing else.
REFUSED_GROUPS = ('sudo', 'admin', 'wheel', 'docker', 'lxd', 'disk', 'libvirt', 'incus-admin')
LOCK = '.lock'
class Refused(Exception):
pass
def _passwd(user):
try:
return pwd.getpwnam(user).pw_uid
except KeyError:
return None
def _groups(user):
entry = pwd.getpwnam(user)
names = set()
for gid in os.getgrouplist(user, entry.pw_gid):
try:
names.add(grp.getgrgid(gid).gr_name)
except KeyError:
pass
return names
def _sudo(user):
try:
out = subprocess.run(
['/usr/bin/sudo', '-l', '-U', user],
env={'LC_ALL': 'C', 'PATH': '/usr/bin:/bin'},
stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
timeout=10,
)
except subprocess.TimeoutExpired:
# No answer is not the expected line, so the caller refuses user-sudo and logs it (duckworth-0081 N2).
return ''
return out.stdout.decode('utf-8', 'replace')
def _log(message):
syslog.openlog('mosaic-keys-write', 0, syslog.LOG_AUTH)
syslog.syslog(syslog.LOG_INFO, message)
class Host:
# What the writer reads from the machine. Tests pass their own dirs, owner and lookups.
def __init__(self, key_dir=KEY_DIR, conf_dir=CONF_DIR, owner=0, passwd=_passwd, groups=_groups,
sudo=_sudo, hostname=None, log=_log):
self.key_dir = key_dir
self.conf_dir = conf_dir
self.owner = owner
self.passwd = passwd
self.groups = groups
self.sudo = sudo
self.hostname = hostname if hostname is not None else socket.gethostname().split('.')[0]
self.log = log
def fingerprint(blob64):
digest = hashlib.sha256(base64.b64decode(blob64, validate=True)).digest()
return 'SHA256:' + base64.b64encode(digest).decode().rstrip('=')
def parse(argv):
# The request's shape and every pattern, before any name reaches a path.
if not argv:
raise Refused('request')
verb, args = argv[0], argv[1:]
shapes = {'list': 1, 'add': 4, 'remove': 2}
if verb not in shapes or len(args) != shapes[verb]:
raise Refused('request')
if not USER.fullmatch(args[0]):
raise Refused('user-pattern')
req = {'verb': verb, 'user': args[0]}
if verb == 'add':
if not KEY_TYPE.fullmatch(args[1]) or not BASE64.fullmatch(args[2]):
raise Refused('key-format')
try:
req['fingerprint'] = fingerprint(args[2])
except ValueError:
raise Refused('key-format')
req['type'], req['base64'] = args[1], args[2]
if verb in ('add', 'remove'):
if not TAG.fullmatch(args[-1]):
raise Refused('tag-pattern')
req['tag'] = args[-1]
return req
def _owned(st, host, kind):
if st.st_uid != host.owner or st.st_mode & 0o022 or not kind(st.st_mode):
raise Refused('not-root-owned')
def read_conf(host, name):
# A root-owned regular file, not group or world writable, opened without following a symlink.
try:
fd = os.open(os.path.join(host.conf_dir, name), os.O_RDONLY | os.O_NOFOLLOW)
except OSError:
raise Refused('config-' + name)
try:
_owned(os.fstat(fd), host, stat.S_ISREG)
with os.fdopen(os.dup(fd), 'r', encoding='utf-8') as f:
text = f.read()
finally:
os.close(fd)
return [line.strip() for line in text.splitlines() if line.strip() and not line.strip().startswith('#')]
def check_user(req, host):
user = req['user']
users = read_conf(host, 'users')
privileged = read_conf(host, 'privileged')
if user in RESERVED_USERS or user not in users:
raise Refused('user-not-listed')
uid = host.passwd(user)
if uid is None:
raise Refused('user-unknown')
if uid == 0 or uid < MIN_UID:
raise Refused('user-system')
if user in privileged:
return
if set(REFUSED_GROUPS) & set(host.groups(user)):
raise Refused('user-group')
expected = 'User %s is not allowed to run sudo on %s.' % (user, host.hostname)
out = host.sudo(user)
if out not in (expected, expected + '\n'):
raise Refused('user-sudo')
def read_from(host):
lines = read_conf(host, 'from')
if len(lines) != 1 or not FROM.fullmatch(lines[0]):
raise Refused('config-from')
return lines[0]
def tag_of(line):
parts = line.split()
return parts[-1] if parts and TAG.fullmatch(parts[-1]) else None
def read_lines(dir_fd, user, host):
# ENOENT means an empty list: a first add finds no file (r6 N6).
try:
fd = os.open(user, os.O_RDONLY | os.O_NOFOLLOW, dir_fd=dir_fd)
except FileNotFoundError:
return []
except OSError:
raise Refused('user-file')
try:
_owned(os.fstat(fd), host, stat.S_ISREG)
with os.fdopen(os.dup(fd), 'r', encoding='utf-8') as f:
return f.read().splitlines()
finally:
os.close(fd)
def write_lines(dir_fd, user, lines, host):
tmp = '.' + user + '.tmp'
try:
st = os.stat(tmp, dir_fd=dir_fd, follow_symlinks=False)
except FileNotFoundError:
st = None
if st is not None:
# Under the flock a leftover tmp can only come from a writer that crashed.
if not stat.S_ISREG(st.st_mode) or st.st_uid != host.owner:
raise Refused('tmp-file')
os.unlink(tmp, dir_fd=dir_fd)
fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o644, dir_fd=dir_fd)
try:
os.fchmod(fd, 0o644)
os.write(fd, ''.join(line + '\n' for line in lines).encode('utf-8'))
os.fsync(fd)
finally:
os.close(fd)
os.rename(tmp, user, src_dir_fd=dir_fd, dst_dir_fd=dir_fd)
os.fsync(dir_fd)
def run(req, host, out):
check_user(req, host)
line_from = read_from(host) if req['verb'] == 'add' else None
try:
dir_fd = os.open(host.key_dir, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
except OSError:
raise Refused('key-dir')
try:
_owned(os.fstat(dir_fd), host, stat.S_ISDIR)
lock = os.open(LOCK, os.O_RDWR | os.O_CREAT | os.O_NOFOLLOW, 0o600, dir_fd=dir_fd)
try:
fcntl.flock(lock, fcntl.LOCK_EX)
user = req['user']
lines = read_lines(dir_fd, user, host)
if req['verb'] == 'list':
for line in lines:
if tag_of(line):
out.write(line + '\n')
return 'listed'
if req['verb'] == 'add':
if any(tag_of(line) == req['tag'] for line in lines):
return 'present'
entry = 'from="%s",restrict %s %s %s' % (line_from, req['type'], req['base64'], req['tag'])
write_lines(dir_fd, user, lines + [entry], host)
return 'added'
kept = [line for line in lines if tag_of(line) != req['tag']]
if len(kept) == len(lines):
return 'absent'
write_lines(dir_fd, user, kept, host)
return 'removed'
finally:
os.close(lock)
finally:
os.close(dir_fd)
def main(argv, host, out=sys.stdout, err=sys.stderr):
req = None
try:
req = parse(argv)
result = run(req, host, out)
code = 0
except Refused as e:
result = 'refused ' + str(e)
code = 1
except OSError as e:
result = 'refused os-' + (e.__class__.__name__)
code = 1
fields = ['result=' + result]
if req:
fields.append('user=' + req['user'])
if 'tag' in req:
fields.append('tag=' + req['tag'])
if 'fingerprint' in req:
fields.append('key=' + req['fingerprint'])
host.log(' '.join(fields))
if code:
err.write('mosaic-keys-write: ' + result + '\n')
return code
if __name__ == '__main__':
sys.exit(main(sys.argv[1:], Host()))
+3
View File
@@ -0,0 +1,3 @@
# Row 234 (#1740), design r6 E3. Installed as /etc/ssh/sshd_config.d/<nn>-mosaic-keys.conf, root, 0644.
# Keeps both paths the hosts use today and adds the root-owned per-user files mosaic-keys-write keeps.
AuthorizedKeysFile .ssh/authorized_keys .ssh/authorized_keys2 /etc/ssh/mosaic-keys/%u
+2
View File
@@ -0,0 +1,2 @@
# Row 234 (#1740), design r6 E3. Installed as /etc/sudoers.d/mosaic-keys, root, 0440.
mosaic-installer ALL=(root) NOPASSWD: /usr/local/sbin/mosaic-keys-write