mosaicstack/stack
25
0
Fork 1
Code Issues 418 Pull Requests 13 Actions Packages Projects Releases 15 Wiki Activity

agents: row 195 Case 1 install design r6 (#1693): Duckworth r5 B1 (confirm's result lines carry the activation's op, from, to and the rollback set's BACKUP, so from_log reads .53's log; sequence 7), B2 (MIGRATING recorded after backup_store and before migrate-schema.mjs, as TAKEN today; sequence 8), N1 marker step makes $RS, N2 gate --since, N3 apparmor.d word grep baseline, N4 item 4 under systemd-run with NoNewPrivileges, N5 plan --expect-from OLD

Browse Source
This commit is contained in:
jason.woltje
2026-10-20 16:53:53 -05:00
parent 0b6d29e909
commit b7239bb66e
1 changed files with 87 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
agents/darkwing/work/queue-195/DESIGN.md
+87 -20
View File
@@ -1,4 +1,4 @@
# Row 195 design: the Case 1 install on 10.1.1.53 (r5)
# Row 195 design: the Case 1 install on 10.1.1.53 (r6)
Darkwing, 2026-10-20. Brief: `docs/plans/2026-10-20_hub-own-uid.md`, blob 45be7f09afc1 (f24f2cd40,
sage-1439). Gitea #1693. Reviewer Duckworth, gate owner Sage. Chained on row 190's candidate (round 3,
@@ -9,6 +9,30 @@ brief is `docs/plans/2026-10-20_case2-rows.md` "C2-13a" at 5b8def2f1, from Gosal
brief as the interface and expect a rebase. Section 2 lists what Case 1 needs from it that the brief
doesn't give.
Changes from r5 (Duckworth's REVIEW-r4.md at 62200362b on `duckworth/reviews`, duckworth-0054). This
round changes only B1's and B2's lines, their section 7 cases and the notes:
- B1. The result line `confirm` writes said op `confirm`, because `log_line` (:58) writes `$OP` and
`OP="$1"` (:1493). `from_log` (:169) reads only `adopt`, `activate` and `rollback`, so it would have
skipped every .53 activation: after adopt C0 and re-pins to C1 and C2, `from_log previous` returns
C0. Now A5's `ok` line and R4's line carry the op, from and to of the activation's `started` line,
and the backup that A3 records in the rollback set. Real sequence 7 and a mutant that writes op
`confirm`. Section 4.
- B2. r5 recorded the migration after the migrate, while today's run sets TAKEN before
`migrate-schema.mjs` (:1004-1005). A migrate that changed the store and then exited 1 would have
left no record, so R2 would skip the restore and R4 would gate OLD on a half-migrated store. A3 now
writes `MIGRATING` after `backup_store` returns and before the migrate. Real sequence 8 and a mutant
that records after the migrate. Section 4.
- N1. The marker step makes `$RS` with capture's exclusive `mkdir` (:416), and capture under
`--units external` makes none. N2. `gate` takes `--since T`, root's clock before its A2f start, as
its start floor. N3. Section 0 now has a read-only grep of `/etc/apparmor.d` for the word `bwrap`,
and item 3 matches the word against a list of exactly one file. N4. Item 4 runs under
`systemd-run` with the units' `NoNewPrivileges=yes`, as 2a' does. N5. A1's plan takes
`--expect-from OLD` and refuses `pointer-differs`.
- Not in this round: claude's path. gosalyn-0058 proposes that `--claude`, like `--node`, takes only a
root-owned path outside the home, and `/srv/mosaic/.local/bin/claude` fails that. I asked Sage to
rule and recommended moving claude under `/opt/mosaic-stack`. That change gets its own round.
Changes from r4 (Duckworth's REVIEW-r3.md at 9f4d5556b on `duckworth/reviews`; mos-1098; mos-1101;
gosalyn-0049):
@@ -72,7 +96,7 @@ last seven rows (the brief allows this):
| unattended-upgrades | on: `APT::Periodic::Unattended-Upgrade "1"`, `apt-daily-upgrade.timer` enabled. Allowed origins `${distro_id}:${distro_codename}`, `-security`, ESM apps-security and ESM infra-security. No `Package-Blacklist` | B2: packages change daily without a window |
| bubblewrap's pocket | `0.9.0-1ubuntu0.3` is in both `noble-updates` and `noble-security`, so its later security updates install unattended | B2 |
| apparmor | installed `4.0.1really4.0.1-0ubuntu0.24.04.8`. The `.9` candidate is in `noble-updates` only, which unattended-upgrades doesn't take. `aa-status --json` works | B2 |
| profiles that name bwrap | no file under `/etc/apparmor.d` mentions `/usr/bin/bwrap` | N1, P9 item 7 |
| profiles that name bwrap | at 21:51:46Z `grep -rlw bwrap /etc/apparmor.d` and `grep -rlF /usr/bin/bwrap /etc/apparmor.d` both find nothing (exit 1). After 2a' the word grep lists only `bwrap-userns-restrict` | N1, P9 item 7, hostcheck item 3 |
| `net.ipv4.ip_unprivileged_port_start` | 1024 | the port squat |
| listeners below 1024 | 22 (sshd) and 53 (the resolver) only; 732 is free | the Console's port (section 1) |
| other listeners | 4003, 6052, 6055, 8123, 8765, 14242, 18555, and on loopback 1933, 5905, 6085, 6379, 6380, 9119, 9225, 18554, 18789, 38147 | none in 7330-7333 |
@@ -246,10 +270,13 @@ floor. So:
that owns it.
2. `aa-status --json` lists `bwrap` and `unpriv_bwrap` in enforce mode, and no loaded profile named
`mosaic-bwrap`.
3. No other file under `/etc/apparmor.d` names `bwrap`.
4. The live label: as `mosaic`, `/usr/bin/bwrap --unshare-user --ro-bind / / cat
/proc/self/attr/current` prints exactly `bwrap//&unpriv_bwrap (enforce)`. This catches a profile
that attaches by a glob that item 3's search misses.
3. `grep -rlw bwrap /etc/apparmor.d` lists exactly `/etc/apparmor.d/bwrap-userns-restrict`. The word
match also catches a path written as `/{usr/,}bin/bwrap`. Section 0's read found no file today.
4. The live label: 2a' line `systemd-run --wait --pipe --uid=mosaic --gid=mosaic -p NoNewPrivileges=yes
-p ProtectSystem=strict -p ProtectHome=yes -p PrivateTmp=yes /usr/bin/bwrap --unshare-user
--unshare-pid --ro-bind / / cat /proc/self/attr/current` prints exactly `bwrap//&unpriv_bwrap
(enforce)`. It runs under the units' no-new-privs, so it reads the label a role session gets.
This catches a profile that attaches by a glob that item 3's search misses.
5. `dpkg-statoverride --list /usr/bin/bwrap` reads `root mosaic 0750`, and `stat` agrees.
6. `kernel.apparmor_restrict_unprivileged_userns` is 1 and `fs.protected_hardlinks` is 1.
7. `net.ipv4.ip_unprivileged_port_start` is above the Console's `--port` in the installed unit.
@@ -398,6 +425,15 @@ never run as designed, and nothing tested the steps in sequence. r5 keeps an act
| the rollback fails at R4's gate | `rollback-failed code <c> rollbackCode <r>` | R4 |
| R1, R2 or R3 fails, or the run dies | none. The activation stays open, and the next activate refuses `interrupted-run` (:603-606), which Jason or Mos settles as on dragon-lin | |
What the result line carries (r5's B1). `log_line` (:58) writes `op "$OP"`, and `from_log` (:169)
reads only lines whose op is `adopt`, `activate` or `rollback`. So `confirm` doesn't log under its own
op. Before A5's `ok` line or any line R4 writes, it sets OP, FROM and TO from the open activation's
`started` line, ACT to that activation, and BACKUP from the rollback set's `BACKUP` file, which A3
writes when `backup_store` returns. A re-pin's lines then read `op activate`, a rollback's `op
rollback`, each with the from, to and backup a dragon-lin run would log. `from_log previous` and
`from_log backup` read .53's log as they read dragon-lin's. `confirm` reads only the line A3 wrote for
that activation, which `interrupted` already names.
A failing `mosaic` step after `started` writes no log line. It writes its code to `FAILED` beside
`AWAITING-UNITS` and exits 1. R4 reads that code for the result line. Root passes R4 `--cause
A3|A4|A5`, the step it saw fail, from its own exit codes. A4 is root's own step and has no `FAILED`
@@ -428,9 +464,9 @@ so it stays readable after a run that dies before `mosaic` logs anything.
| # | Step | Runner | On failure |
|---|---|---|---|
| A1 | `/root/mosaic-stack` HEAD is the commit the go names. ID is 40 lowercase hex and names a copy under `/srv/mosaic/.local/share/mosaic-stack/copies` that holds `scripts/mosaic` (row 230's copy-id rule). `case1-hostcheck.sh --check` exits 0, else `profile-drift` (B2). Root reads OLD from the unit files. `stack-release.sh activate ID mosaic-stack --units external --plan` exits 0 | root; the plan as `mosaic` | refuse, exit 3. Nothing has changed |
| A1 | `/root/mosaic-stack` HEAD is the commit the go names. ID is 40 lowercase hex and names a copy under `/srv/mosaic/.local/share/mosaic-stack/copies` that holds `scripts/mosaic` (row 230's copy-id rule). `case1-hostcheck.sh --check` exits 0, else `profile-drift` (B2). Root reads OLD from the unit files. `stack-release.sh activate ID mosaic-stack --units external --plan --expect-from OLD` exits 0. The plan refuses `pointer-differs` when the pointer's copy isn't OLD (N5), so a pointer and units that disagree stop before any stop | root; the plan as `mosaic` | refuse, exit 3. Nothing has changed |
| A2 | `systemctl stop` UNITS in reverse order, then wait until each is inactive, within `--timeout` | root | A2f below |
| A3 | `stack-release.sh activate ID mosaic-stack --units external [--migrate]`. Before it logs `started` (N3), it refuses `units-running` unless each of UNITS is inactive (system-manager reads, which need no privilege) and the store is stopped (`stopped`, :359), and `interrupted-run` while any activation is open. Then, in this order: logs `result started`; writes `AWAITING-UNITS` at once; captures the rollback set (pointer, wrapper, and the sha256 and `ExecStart=` of each installed unit, read only); takes `backup_store` (:372) into `$DATA/backups/stack/<ACT>`; migrates under `--migrate` (sets TAKEN, as today) and records in the rollback set that it did; installs the wrapper for ID. It writes no pointer | `mosaic` | a refusal before `started` exits 3 with nothing changed. A marker or capture failure removes the marker, logs its `refused` line and exits 3. Root runs A2f for both. Any later failure writes `FAILED` with its code, logs nothing, cleans up nothing and exits 1. Root runs R1-R4 with `--cause A3` |
| A3 | `stack-release.sh activate ID mosaic-stack --units external [--migrate]`. Before it logs `started` (N3), it refuses `units-running` unless each of UNITS is inactive (system-manager reads, which need no privilege) and the store is stopped (`stopped`, :359), and `interrupted-run` while any activation is open. Then, in this order: logs `result started`; makes `$RS` with capture's `mkdir -p` of `stack-rollback` and exclusive `mkdir -m 0700` (:415-416), and writes `AWAITING-UNITS` in it at once (N1); captures the rollback set into that directory, making none (pointer, wrapper, and the sha256 and `ExecStart=` of each installed unit, read only); takes `backup_store` (:372) into `$DATA/backups/stack/<ACT>` and writes its name to `BACKUP` in the rollback set; under `--migrate`, writes `MIGRATING` in the rollback set and sets TAKEN, both before `migrate-schema.mjs` runs, where today's run sets TAKEN (:1004-1005, r5's B2), then migrates; installs the wrapper for ID. It writes no pointer | `mosaic` | a refusal before `started` exits 3 with nothing changed. A marker or capture failure, an existing `$RS` included, removes the marker if it wrote one, logs its `refused` line and exits 3. Root runs A2f for both. Any later failure writes `FAILED` with its code, logs nothing, cleans up nothing and exits 1. Root runs R1-R4 with `--cause A3` |
| A4 | `case2-unit.sh` with `case1-unit.args` and `--copy ID`, `systemctl daemon-reload`, `systemctl start` UNITS in order | root | R1-R4 with `--cause A4` |
| A5 | `stack-release.sh confirm ID mosaic-stack --units external --timeout S`. This is `gate` (:522), which polls `gate_once` (:471) every second until `--timeout`, with the external checks below. On a pass it writes the pointer, logs `result ok`, removes `AWAITING-UNITS`, exit 0. When the timeout runs out it writes `FAILED` with the last gate code, logs nothing and exits 1 | `mosaic` | R1-R4 with `--cause A5` |
@@ -442,9 +478,13 @@ and A5 alike. Two cleanup paths, A3's own and R2, would each need their own test
A2f, a stop that fails or times out, or an A3 that exits 3. This run has no open activation: A2 logs nothing, and
A3's exit 3 comes either before `started` or after its own `refused` line. Nothing of `mosaic`'s has
changed. Root runs `systemctl start` UNITS in order, then `stack-release.sh gate OLD mosaic-stack
--units external --timeout S`, a new verb that runs `gate` on OLD and writes no log line, no pointer and
no file. If that passes, root logs `A2 failed, OLD running` (or `A3 refused`) in its step log and exits
changed. Root reads `date -u +%s` as T, runs `systemctl start` UNITS in order, then `stack-release.sh gate
OLD mosaic-stack --units external --since T --timeout S`, a new verb that runs `gate` on OLD and writes no
log line, no pointer and no file. A2f has no `started` line to take a start floor from (the first
re-pin after adopt has none at all), so `--since` is the floor, and `gate` refuses `usage` without it
(r5's N2). T is root's own clock, so root still parses nothing `mosaic` wrote. A unit that never
stopped keeps its old `ActiveEnterTimestamp`, fails `stale-start` and takes the exit 2 path below.
If that passes, root logs `A2 failed, OLD running` (or `A3 refused`) in its step log and exits
1. If it fails, root stops UNITS, logs `OLD gate failed <code>` and exits 2. The next run's
`interrupted` check finds nothing open either way.
@@ -490,7 +530,7 @@ serve flag in a root-owned unit, so changing it is a re-install with its own go.
| # | Step | Runner | On failure |
|---|---|---|---|
| R1 | `systemctl stop` UNITS, then wait as in A2 | root | rollback-failed |
| R2 | `stack-release.sh revert ID mosaic-stack --units external`. It finds the open activation as above, puts the wrapper back from the rollback set and checks its sha256, and restores TAKEN (`restore_store`, :396) only if the rollback set records a migration. Each part is idempotent, so it works for an A3 that stopped partway: no recorded migration means no restore, and copying the saved wrapper over an unchanged one changes nothing. It writes no log line | `mosaic` | rollback-failed, activation left open |
| R2 | `stack-release.sh revert ID mosaic-stack --units external`. It finds the open activation as above, puts the wrapper back from the rollback set and checks its sha256, and restores TAKEN (`restore_store`, :396) only if the rollback set holds `MIGRATING`. Each part is idempotent, so it works for an A3 that stopped partway: no `MIGRATING` means no restore, a migrate that failed partway is restored from the backup taken before it, and copying the saved wrapper over an unchanged one changes nothing. It writes no log line | `mosaic` | rollback-failed, activation left open |
| R3 | `case2-unit.sh` with `case1-unit.args` and `--copy OLD`, `systemctl daemon-reload`, `systemctl start` UNITS | root | rollback-failed, activation left open |
| R4 | `stack-release.sh confirm ID mosaic-stack --units external --rollback OLD --cause A3\|A4\|A5 --timeout S`. It finds the open activation, checks OLD is its `from`, gates OLD with `gate`, and writes the activation's one result line from the cause and the `FAILED` code: `failed code <c> rollback ok` (A3, A4), `gate-failed code <c> rollback ok` (A5), or `rollback-failed code <c> rollbackCode <gate>` when OLD's gate fails | `mosaic` | rollback-failed, logged |
@@ -508,7 +548,8 @@ changed.
The same list with TO in place of ID. The go names TO, from Mos's or Sage's read of `stack-release.sh
status` as `mosaic`. Root passes it. A1's plan, `stack-release.sh rollback mosaic-stack --units external
--to ID --plan`, exits 3 unless ID is `from_log previous` (:162), the previous distinct copy, so a
--to ID --plan --expect-from OLD`, refuses `pointer-differs` as in activate. It also exits 3 unless ID
is `from_log previous` (:162), the previous distinct copy, so a
stale or mistyped go refuses before anything stops. With `--restore`, the plan also refuses unless
`$DATA/backups/stack/ACT` exists. `--to` is a new flag, accepted only with `--units external`;
dragon-lin's rollback keeps reading TO from the log.
@@ -537,7 +578,8 @@ re-pin's A3 takes the first backup.
Every `mosaic` line goes to `$DATA/state/stack-activation-log.jsonl`, as today, with `units external`
added. So `from_log previous` reads .53's history the same way it reads dragon-lin's. A3 writes
`started`. A5 or R4 writes the one result line, or A3 its `refused` line before any change. Refusals of
`started`. A5 or R4 writes the one result line, with the op, from, to and backup of that activation,
or A3 its `refused` line before any change. Refusals of
`confirm`, `revert` and `gate` log no activation id. If a run dies between `started` and the result,
the log keeps `started` with no result, the next run's `interrupted` check reports it, and root's step
log shows the last step that ran.
@@ -683,6 +725,8 @@ the proof pair. The managed-settings route needs no Case 2 change.
dir from `case1-unit.args`;
- a HEAD mismatch, an ID that fails the copy-id rule, or unit files that name two copies refuses with
exit 3 before any stop;
- A1's plan argv carries `--expect-from` with OLD from the unit files;
- A2f passes `gate OLD --since T`, with T from a fake `date` that root calls before its start;
- A2f: a fake `systemctl` that never reports inactive. The script starts UNITS, runs `gate OLD`, exits
1 with OLD's gate passing and 2 with it failing, and the stack log is byte-identical before and
after;
@@ -713,8 +757,16 @@ the proof pair. The managed-settings route needs no Case 2 change.
writes `FAILED` with the gate code, leaves the pointer, the log and the marker as they were, and
exits 1;
- `confirm` without `--rollback` refuses `activation-failed` when `FAILED` exists;
- every line `confirm` writes has op `activate` or `rollback`, never `confirm`, and the from, to and
activation of its `started` line, and backup equal to the rollback set's `BACKUP`;
- `gate` (N2): with `--since 1792540800`, `@1792540800` passes and `@1792540799` fails `stale-start`;
without `--since` it refuses `usage`;
- `--plan --expect-from` (N5) refuses `pointer-differs` when the pointer names another copy, for
activate and for `rollback --to`;
- A3's order: a fake that fails the capture leaves the log with `started` and `refused code
capture-failed` for that activation, no marker and no `FAILED`, and exit 3. A fake that fails the
capture-failed` for that activation, no marker and no `FAILED`, and exit 3. An existing `$RS`
gives `refused code marker-failed` and exit 3, and a capture under `--units external` makes no
directory of its own. A fake that fails the
wrapper install after a migration leaves `started` only, the marker, `FAILED`, TAKEN as migrated,
and exit 1;
- refusals by `confirm`, `revert` and `gate` log with no activation id, and the `interrupted` set
@@ -746,6 +798,14 @@ the proof pair. The managed-settings route needs no Case 2 change.
--cause A5` still close the activation, because the refusal didn't;
6. activate, revert fails (a wrapper whose saved copy is missing): the activation stays open, and
the next `activate --plan` refuses `interrupted-run`;
7. adopt C0, activate C1 and confirm, activate C2 and confirm (r5's B1): both `ok` lines read
`op activate` with their `started` line's from and to and the backup each took. `rollback --to C1
--plan` exits 0 and `--to C0 --plan` refuses. A rollback to C1, confirmed, logs `op rollback`, and
`from_log previous` then names C2;
8. activate with `--migrate`, where the copy's `migrate-schema.mjs` is a fake that writes to the
store and exits 1 (r5's B2): `FAILED` holds `migrate-failed` and `MIGRATING` exists. revert
restores the store byte-identical to the backup, and `confirm --rollback --cause A3` gates OLD on
it: `failed code migrate-failed rollback ok`;
- `rollback --to` refuses when ID isn't `from_log previous`, and `rollback --restore` finds a backup
that an external activate took;
- `gate` writes nothing: the log, the pointer and the state dir are byte-identical before and after;
@@ -771,15 +831,21 @@ the proof pair. The managed-settings route needs no Case 2 change.
- an external verb's refusal logs with the open activation id (ACT set before the checks; killed by
sequence 5 and the refusal test);
- `stale-start` is a refusal again (killed by the polling case);
- `revert` restores TAKEN without a recorded migration (killed by the revert case).
- `revert` restores TAKEN without `MIGRATING` (killed by the revert case);
- `confirm` logs with its own op, `confirm` (killed by sequence 7 and the op case);
- A3 writes `MIGRATING` after `migrate-schema.mjs` (killed by sequence 8);
- `gate` takes its floor from the newest `started` line when `--since` is missing (killed by the
`gate` case).
- `scripts/case1-pins.sh` and `scripts/case1-hostcheck.sh` (new),
`packages/cli/systemd/mosaic-hostcheck.service` (new) and `packages/cli/tests/case1-hostcheck.test.mjs`
(new). The script takes `--root DIR` for `/etc`, `/proc/sys` and `/usr/bin/bwrap`'s stat, and fake
`aa-status`, `dpkg`, `dpkg-statoverride`, `apparmor_parser`, `bwrap`, `chown`, `chmod` and `setpriv`
`aa-status`, `dpkg`, `dpkg-statoverride`, `apparmor_parser`, `systemd-run`, `chown` and `chmod`
on PATH record their argv. Cases:
- `--check` exits 0 with all seven items and 1 with each one failing alone, its line naming it;
item 3 fails on another profile file naming `bwrap`, item 4 on a label of `unpriv_bwrap (enforce)`
or with a trailing space, item 7 with the floor at 732;
item 3 fails on a second file naming `bwrap` as a word, one that writes `/{usr/,}bin/bwrap` included,
and passes with `bwrap-userns-restrict` alone. Item 4 fails on a label of `unpriv_bwrap (enforce)`
or one with a trailing space, and its `systemd-run` argv carries `-p NoNewPrivileges=yes` and the
other 2a' properties. Item 7 fails with the floor at 732;
- `--enforce` exits 0 on a pass and on every failure. On a failure it unloads the profile, closes
bwrap to `root:root` 0700, writes `DRIFT` and one log line. With `DRIFT` present and every item
passing, it still closes bwrap;
@@ -790,7 +856,8 @@ the proof pair. The managed-settings route needs no Case 2 change.
`Before=mosaic-bus@mosaic-stack.service` and runs `--enforce`.
Mutants: `--enforce` exits 1 on a drift; `--enforce` ignores `DRIFT` when the items pass; item 1
drops the `dpkg -S` owner check; item 4 matches a substring; item 7 compares with `>=`.
drops the `dpkg -S` owner check; item 3 matches only the `/usr/bin/bwrap` path; item 4 matches a
substring; item 4 drops `-p NoNewPrivileges=yes`; item 7 compares with `>=`.
- `scripts/case1-accept.sh` (new) and `packages/cli/tests/case1-accept.test.mjs` (new), section 5.
- Row 230 (C2-13a). `--profile`, the node, bwrap and port-floor precheck lines, the copy-id rule, the
two units and the Console's capability lines are in its brief at 5b8def2f1. Section 2's four items