Compare commits
5
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b7239bb66e | ||
|
|
0b6d29e909 | ||
|
|
0075be3581 | ||
|
|
9a16d7795d | ||
|
|
818b79696d |
@@ -0,0 +1,872 @@
|
||||
# Row 195 design: the Case 1 install on 10.1.1.53 (r6)
|
||||
|
||||
Darkwing, 2026-10-20. Brief: `docs/plans/2026-10-20_hub-own-uid.md`, blob 45be7f09afc1 (f24f2cd40,
|
||||
sage-1439). Gitea #1693. Reviewer Duckworth, gate owner Sage. Chained on row 190's candidate (round 3,
|
||||
df65a3d4c on `darkwing/row190-r3`, rebased on row 192; the config canon) and on row 230's candidate
|
||||
(C2-13a, #1732, Gosalyn: `scripts/case2-unit.sh` and `case2-precheck.sh`), Q177, sage-1499. Row 230's
|
||||
brief is `docs/plans/2026-10-20_case2-rows.md` "C2-13a" at 5b8def2f1, from Gosalyn's design r11
|
||||
(eac92d956 on `gosalyn/case2-design`, approved webby-0038). It has no candidate yet, so I read that
|
||||
brief as the interface and expect a rebase. Section 2 lists what Case 1 needs from it that the brief
|
||||
doesn't give.
|
||||
|
||||
Changes from r5 (Duckworth's REVIEW-r4.md at 62200362b on `duckworth/reviews`, duckworth-0054). This
|
||||
round changes only B1's and B2's lines, their section 7 cases and the notes:
|
||||
|
||||
- B1. The result line `confirm` writes said op `confirm`, because `log_line` (:58) writes `$OP` and
|
||||
`OP="$1"` (:1493). `from_log` (:169) reads only `adopt`, `activate` and `rollback`, so it would have
|
||||
skipped every .53 activation: after adopt C0 and re-pins to C1 and C2, `from_log previous` returns
|
||||
C0. Now A5's `ok` line and R4's line carry the op, from and to of the activation's `started` line,
|
||||
and the backup that A3 records in the rollback set. Real sequence 7 and a mutant that writes op
|
||||
`confirm`. Section 4.
|
||||
- B2. r5 recorded the migration after the migrate, while today's run sets TAKEN before
|
||||
`migrate-schema.mjs` (:1004-1005). A migrate that changed the store and then exited 1 would have
|
||||
left no record, so R2 would skip the restore and R4 would gate OLD on a half-migrated store. A3 now
|
||||
writes `MIGRATING` after `backup_store` returns and before the migrate. Real sequence 8 and a mutant
|
||||
that records after the migrate. Section 4.
|
||||
- N1. The marker step makes `$RS` with capture's exclusive `mkdir` (:416), and capture under
|
||||
`--units external` makes none. N2. `gate` takes `--since T`, root's clock before its A2f start, as
|
||||
its start floor. N3. Section 0 now has a read-only grep of `/etc/apparmor.d` for the word `bwrap`,
|
||||
and item 3 matches the word against a list of exactly one file. N4. Item 4 runs under
|
||||
`systemd-run` with the units' `NoNewPrivileges=yes`, as 2a' does. N5. A1's plan takes
|
||||
`--expect-from OLD` and refuses `pointer-differs`.
|
||||
- Not in this round: claude's path. gosalyn-0058 proposes that `--claude`, like `--node`, takes only a
|
||||
root-owned path outside the home, and `/srv/mosaic/.local/bin/claude` fails that. I asked Sage to
|
||||
rule and recommended moving claude under `/opt/mosaic-stack`. That change gets its own round.
|
||||
|
||||
Changes from r4 (Duckworth's REVIEW-r3.md at 9f4d5556b on `duckworth/reviews`; mos-1098; mos-1101;
|
||||
gosalyn-0049):
|
||||
|
||||
- B1. An activation now has exactly one result line, written at the end. A5's gate failure and A3's
|
||||
failure after its capture write no log line. Each writes a `FAILED` file next to `AWAITING-UNITS` and
|
||||
exits 1, so the activation stays open for R2 and R4, and R4 writes the result. A3 writes
|
||||
`AWAITING-UNITS` right after `started`. Refusals by `confirm`, `revert` and `gate` log no activation
|
||||
id, so a refused `confirm` can't close one either. There are real-sequence tests on one log, and
|
||||
mutants that put back r4's interim line, r4's marker order and a refusal that names the activation.
|
||||
Section 4 and section 7.
|
||||
- B2. The bwrap profile has an owner and a stop. `scripts/case1-hostcheck.sh` checks the profile file
|
||||
against the pin in `scripts/case1-pins.sh`, the two loaded profiles, the live label of a process
|
||||
bwrap runs, the statoverride and mode, that no other profile names bwrap, two sysctls and the port
|
||||
floor. It has three callers:
|
||||
1. A1 of every re-pin refuses `profile-drift`.
|
||||
2. A root apt hook runs it after every package run, unattended ones included.
|
||||
3. A boot unit runs it before the bus starts.
|
||||
|
||||
On drift, callers 2 and 3 unload the profile and close `/usr/bin/bwrap` to `mosaic`, so every role
|
||||
session refuses to start. They write `DRIFT` and log it, and only root clears it. ops-01 owns the host
|
||||
side under a Mos go: it reads the log in each STATE report and, before each re-pin window, compares
|
||||
the upstream profile to the pin. A changed upstream profile becomes a new pin, which is its own review
|
||||
round. Section 1 has the details. The facts behind it are in section 0: unattended-upgrades runs daily
|
||||
on .53 with the security pocket, and the pinned bubblewrap is in that pocket.
|
||||
- N1 and mos-1098 point 2. P9 has a seventh item: no `mosaic-bwrap` profile is loaded, and no profile
|
||||
other than Ubuntu's attaches to `/usr/bin/bwrap`. r4 never installed `mosaic-bwrap`. F5 now compares
|
||||
exact strings. P9 reads `accepted mos-1096` only while all seven items hold.
|
||||
- N2: a role session gets no capabilities, so a nested bwrap fails. Section 1. N3: A3 refuses "before
|
||||
it logs `started`". N4: the stale-start rule and its fixtures handle `ActiveEnterTimestamp`'s whole
|
||||
seconds.
|
||||
- The port squat (mos-1101 item 5, gosalyn-0049). It applies: role sessions share the host's network
|
||||
namespace (`packages/seat/src/sandbox.mjs:67` unshares no net), so while the Console is down a
|
||||
session can bind its loopback port. The Console moves to 127.0.0.1:732 with
|
||||
`CAP_NET_BIND_SERVICE` on its unit (row 230's line), so no session can take it. The tester port stays
|
||||
7333, because serve refuses a tester port below 1024. That residue is P10, accepted under Mos's
|
||||
reading. Case 1 can't meet the preview trio or the proof pair, because .53 has no proxy. Section 2
|
||||
names the factory serve set for row 230.
|
||||
- Jason's Q9 (Vikunja #429) is still open. If install and upgrade become product commands, these
|
||||
steps become that installer's configuration and row 195 rebases on its row (sage-1491). Section 6.
|
||||
|
||||
## 0. What I checked on the host
|
||||
|
||||
Read only, as `localadmin` over ssh, 2026-10-20 between 20:40Z and 21:00Z, and again at 21:27Z for the
|
||||
last seven rows (the brief allows this):
|
||||
|
||||
| Item | 10.1.1.53 today | Needed |
|
||||
|---|---|---|
|
||||
| OS | Ubuntu 24.04.5, kernel 6.8.0-124, x86_64, 12 cores, 94 GiB, 217 GiB free on `/` | fits |
|
||||
| `mosaic` user, group, `/srv/mosaic` | absent | step 2a creates them |
|
||||
| node | `/usr/local/bin/node` v22.22.2, root-owned | 24 or later (every `packages/*/package.json` engines line); dragon-lin runs 26.8.1 |
|
||||
| bwrap | absent, no statoverride; apt candidate `bubblewrap 0.9.0-1ubuntu0.3` | every role session runs under `/usr/bin/bwrap` (`packages/seat/src/sandbox.mjs:43`) |
|
||||
| user namespaces | `kernel.apparmor_restrict_unprivileged_userns = 1`, apparmor 4.0.1, no bwrap profile loaded | bwrap needs a user namespace, which this setting refuses unless a profile allows it |
|
||||
| `apparmor-profiles` | not installed; candidate `4.0.1really4.0.1-0ubuntu0.24.04.9`, whose `/usr/share/apparmor/extra-profiles/bwrap-userns-restrict` has sha256 `11d39094f044f0cda0febb3ad517b830301da6b2ce929664af09ee9e4dd264f9` (read from the deb on dragon-lin) | section 1 |
|
||||
| `fs.protected_hardlinks`, `fs.protected_symlinks` | 1, 1 | mos-1096 item 2 |
|
||||
| journald | `SplitMode=uid` (the default) | section 4, B2 |
|
||||
| `useradd` | `USERGROUPS_ENAB yes` | 2a passes `--user-group` anyway |
|
||||
| claude | not on localadmin's PATH | the factory's harness; dragon-lin runs 2.1.304 |
|
||||
| ports 7330-7333 | free (no listener) | tester |
|
||||
| `localadmin` | NOPASSWD sudo (#423 step 1, applied 20:05:47Z, mos-1066, mos-1077) | the root steps run through it |
|
||||
| `jarvis` | groups sudo, docker, lxd, adm | #423 step 4 removes all four (section 5, P5) |
|
||||
| unattended-upgrades | on: `APT::Periodic::Unattended-Upgrade "1"`, `apt-daily-upgrade.timer` enabled. Allowed origins `${distro_id}:${distro_codename}`, `-security`, ESM apps-security and ESM infra-security. No `Package-Blacklist` | B2: packages change daily without a window |
|
||||
| bubblewrap's pocket | `0.9.0-1ubuntu0.3` is in both `noble-updates` and `noble-security`, so its later security updates install unattended | B2 |
|
||||
| apparmor | installed `4.0.1really4.0.1-0ubuntu0.24.04.8`. The `.9` candidate is in `noble-updates` only, which unattended-upgrades doesn't take. `aa-status --json` works | B2 |
|
||||
| profiles that name bwrap | at 21:51:46Z `grep -rlw bwrap /etc/apparmor.d` and `grep -rlF /usr/bin/bwrap /etc/apparmor.d` both find nothing (exit 1). After 2a' the word grep lists only `bwrap-userns-restrict` | N1, P9 item 7, hostcheck item 3 |
|
||||
| `net.ipv4.ip_unprivileged_port_start` | 1024 | the port squat |
|
||||
| listeners below 1024 | 22 (sshd) and 53 (the resolver) only; 732 is free | the Console's port (section 1) |
|
||||
| other listeners | 4003, 6052, 6055, 8123, 8765, 14242, 18555, and on loopback 1933, 5905, 6085, 6379, 6380, 9119, 9225, 18554, 18789, 38147 | none in 7330-7333 |
|
||||
|
||||
## 1. The install as inputs to the Case 2 scripts
|
||||
|
||||
Instance. Row 190's canon with the default name, `dev`, as Case 2 uses it (Gosalyn design 9). `~` is
|
||||
`mosaic`'s home, `/srv/mosaic`, so:
|
||||
|
||||
| Input | Value |
|
||||
|---|---|
|
||||
| user | `mosaic`, primary group `mosaic` with no other member |
|
||||
| home | `/srv/mosaic`, 0700, owner `mosaic` |
|
||||
| config dir | `/srv/mosaic/.config/mosaic-dev` |
|
||||
| data root | `/srv/mosaic/.mosaic-dev`, marker `instance.json` `{"instanceVersion":1,"name":"dev"}` |
|
||||
| business | `mosaic-stack`, file `<configDir>/businesses/mosaic-stack.json` (section 2) |
|
||||
| profile | `--profile factory` (row 230, C2-13b) |
|
||||
| roles | the factory set in section 2 |
|
||||
| human id | Jason's id, given in the window's go. Not `tester` (`broker.mjs:245`) |
|
||||
| model credential | `<configDir>/secrets/claude-session.env`, 0600 (row 190's `SESSION_ENV` for the default instance) |
|
||||
| token slot path | none for this row (section 2) |
|
||||
| Console port | 127.0.0.1:732, below the port floor, bound through the Console unit's `CAP_NET_BIND_SERVICE` (row 230). The port squat below says why |
|
||||
| tester port | 127.0.0.1:7333 (P10) |
|
||||
| board | none. UNITS has no board unit, and the Console gets no `--board` |
|
||||
| harness | claude at `/srv/mosaic/.local/bin/claude`, pinned through `MOSAIC_CLAUDE_BIN` (decision 91), which the bus unit's `@CLAUDE_ENV@` line sets (section 2) |
|
||||
| node | `/opt/mosaic-stack/node-<version>/bin/node`, a pinned 26.x release, `root:root`. Row 230 refuses a `--node` under the home or not owned by root |
|
||||
| PM | on: the bus unit's `@PM_FLAGS@` renders ` --pm` (section 2, section 4) |
|
||||
| source | `/srv/mosaic/src/mosaic-stack` (`mosaic`'s checkout, section 4) and `/root/mosaic-stack` (root's) |
|
||||
|
||||
Why `dev` and not a named instance: one instance per OS user, and Case 2's rows already assume
|
||||
`/srv/mosaic/.config/mosaic-dev`. A second instance under `mosaic` on .53 would need a name, and row 190
|
||||
makes that a flag, not a move.
|
||||
|
||||
Why node lives under `/opt/mosaic-stack`: changing `/usr/local/bin/node` would change what Hermes and
|
||||
anything else on the host runs, and Q181 keeps Hermes running. r4 put node under `/srv/mosaic`, but row
|
||||
230 requires a root-owned `--node` outside the home. Nothing else on the host puts `/opt/mosaic-stack`
|
||||
on its PATH.
|
||||
|
||||
PATH for `mosaic`'s steps. `sudo -u mosaic -H` resets PATH to sudo's `secure_path`, which finds
|
||||
`/usr/local/bin/node`, v22. The precheck would then refuse, and `scripts/mosaic` would run v22 (it runs
|
||||
`exec node` from PATH, `scripts/mosaic:22-40`). So every `mosaic` step in the run sheet and in
|
||||
`case1-repin.sh` runs as `sudo -u mosaic -H env PATH=<node dir>:/usr/local/bin:/usr/bin:/bin ...`, the
|
||||
list row 230 renders as the units' `@PATH@`. `case1-repin.sh` builds it from the `--node` in
|
||||
`case1-unit.args`.
|
||||
|
||||
The units (row 230, design r11). Two system units, from the existing templates, in start order:
|
||||
`[email protected]` from `packages/cli/systemd/mosaic-bus.service.in`, then
|
||||
`mosaic-webui.service` from `packages/webui/systemd/mosaic-webui.service.in`. Root stops them in the
|
||||
reverse order. `@REPO@` is the copy dir, and `%h` becomes the literal home. Row 230 adds
|
||||
`ProtectSystem=strict`, `ReadWritePaths=/srv/mosaic`, `ProtectHome=yes` and `PrivateTmp=yes` to both, and
|
||||
`AmbientCapabilities=CAP_NET_BIND_SERVICE` with `CapabilityBoundingSet=CAP_NET_BIND_SERVICE` to the
|
||||
Console only. The Console unit has no `[Install]`, so root starts it by hand after a reboot until Jason
|
||||
rules (mos-1097). The install call is:
|
||||
|
||||
```
|
||||
case2-unit.sh --profile factory --user mosaic --home /srv/mosaic --business mosaic-stack --copy ID \
|
||||
--node /opt/mosaic-stack/node-<version>/bin/node --pm --claude /srv/mosaic/.local/bin/claude \
|
||||
-- --port 732 --business mosaic-stack --tester-port 7333
|
||||
```
|
||||
|
||||
`--pm` and `--claude` are the two inputs row 230's brief doesn't have yet (section 2). A re-install
|
||||
must pass the same `--node`, flags and serve flags, and only the copy id may change (row 230's
|
||||
re-install check). So 2c writes the whole argument list, without `--copy`, to
|
||||
`/etc/mosaic-stack/case1-unit.args`, `root:root` 0644, one argument per line. Every later
|
||||
`case2-unit.sh` call in section 4 reads that file and adds `--copy`. `mosaic` can't write it. Changing
|
||||
a serve flag is a re-install with its own go, not a re-pin.
|
||||
|
||||
The port squat (mos-1101 item 5). Role sessions share the host's network namespace:
|
||||
`packages/seat/src/sandbox.mjs:67` unshares the user, PID and IPC namespaces and not the network. So
|
||||
while the Console is down, which A2 to A4 of every re-pin and any crash before `Restart=on-failure`
|
||||
brings it back both cause, a session can bind 127.0.0.1:7332. It would then receive Jason's `ssh -L`
|
||||
traffic, and it would answer the gate's `/healthz` (`stack-release.sh:485-487`) in the Console's place.
|
||||
At 732 that can't happen. The floor is 1024 (section 0), a session has no capabilities (below), and
|
||||
`mosaic`'s own processes outside the sandbox have none either. Only the Console unit holds
|
||||
`CAP_NET_BIND_SERVICE`. Its children, the human-cli and read scripts it spawns
|
||||
(`packages/webui/src/bus.mjs:31`, `reads.mjs:79`), inherit the ambient capability. No role session
|
||||
descends from the Console: sessions are children of the bus unit, which has no capability line. The
|
||||
gate needs no change, because `console_port` (`stack-release.sh:213-215`) reads the Console's `--port`
|
||||
from the pointer. Row 230's precheck refuses when the floor is at or below the Console's port, and
|
||||
`case1-hostcheck.sh` checks it after every package run (B2). The tester port can't go below 1024,
|
||||
because serve refuses that (`packages/webui/src/cli.mjs:33`), so P10 records it.
|
||||
|
||||
Node download (N4). The run sheet pins one release, its tarball name and its sha256. As root, in a
|
||||
root-owned temp dir: fetch the tarball, `SHASUMS256.txt` and `SHASUMS256.txt.asc`; `gpg --verify` the
|
||||
`.asc` against the Node release keys listed in the nodejs/release-keys repository, imported into a
|
||||
keyring under `/etc/mosaic-stack/node-keys`; check that the tarball's line in the verified file is the
|
||||
pinned sha256; then `sha256sum -c` the tarball and unpack it to `/opt/mosaic-stack/node-<version>`,
|
||||
`root:root`, no file writable by group or other. Any mismatch stops the step. Rollback: `rm -r` that
|
||||
dir.
|
||||
|
||||
Claude (N5). The run sheet pins the version, 2.1.304 unless the window's plan names a newer one. As
|
||||
`mosaic`: `npm install --prefix /srv/mosaic/.local @anthropic-ai/claude-code@<pin>`, with `@<pin>`
|
||||
exact. `/srv/mosaic/.local/bin/claude` is the path `MOSAIC_CLAUDE_BIN` names. The autoupdater stays off
|
||||
through `/etc/claude-code/managed-settings.json`, `root:root` 0644, holding
|
||||
`{"env":{"DISABLE_AUTOUPDATER":"1"}}`. Root writes it in 2a'. Why there:
|
||||
|
||||
- `ENV_ALLOW` (`packages/seat/src/session.mjs:83-88`) doesn't pass `DISABLE_AUTOUPDATER`, so a unit
|
||||
`Environment=` line never reaches a role session (gosalyn-0046 item 3).
|
||||
- A role session's HOME is the run's own `home/`, and the sandbox doesn't show `~/.claude`
|
||||
(`session.mjs:92-93`, `sandbox.mjs:16`). So `mosaic`'s claude settings file doesn't reach one either.
|
||||
- The sandbox mounts `/etc` read-only (`sandbox.mjs:85`), and Claude Code reads managed settings from
|
||||
`/etc/claude-code/` whatever HOME is. `mosaic`'s own CLI runs, like the precheck's marker prompt
|
||||
and F6, read it too.
|
||||
|
||||
The pinned binary is a read-only bind inside the sandbox (`sandbox.mjs:9-10`), so a session can't
|
||||
rewrite it anyway. The managed file covers `mosaic`'s runs outside the sandbox. If the installed
|
||||
version doesn't honour `env` there, F6 catches the drift. Rollback: `rm` the file.
|
||||
|
||||
bwrap and the user namespace (mos-1091 4a, 4b; mos-1096 item 2; N1). Three root steps in 2a', in this
|
||||
order, each with its rollback. The `mosaic` group must exist first (2a).
|
||||
|
||||
1. `apt-get install bubblewrap=0.9.0-1ubuntu0.3`, the version the run sheet pins. Then record
|
||||
`stat -c '%U:%G %a' /usr/bin/bwrap` to `/etc/mosaic-stack/bwrap.orig` (I expect `root:root 755`).
|
||||
Rollback: `apt-get remove bubblewrap`.
|
||||
2. Ubuntu's own `bwrap-userns-restrict`, one file, not the package. `apt-get download
|
||||
apparmor-profiles=4.0.1really4.0.1-0ubuntu0.24.04.9` into a root-owned temp dir, extract
|
||||
`./usr/share/apparmor/extra-profiles/bwrap-userns-restrict` with `dpkg-deb --fsys-tarfile`, check
|
||||
its sha256 is `11d39094f044f0cda0febb3ad517b830301da6b2ce929664af09ee9e4dd264f9`, and install it as
|
||||
`/etc/apparmor.d/bwrap-userns-restrict`, `root:root` 0644. Load it with `apparmor_parser -r
|
||||
/etc/apparmor.d/bwrap-userns-restrict`. `aa-status` then lists `bwrap` and `unpriv_bwrap` in
|
||||
enforce mode. Why one file: the package also installs 20 profiles under `/etc/apparmor.d`
|
||||
(`usr.sbin.dnsmasq`, `bin.ping`, `usr.sbin.smbd` and more), which would load at the next boot and
|
||||
could change services on the host that have nothing to do with Mosaic.
|
||||
What the profile does: `bwrap` gets `userns`, `capability` and `mount` for its own setup, and
|
||||
every program it runs is stacked under `unpriv_bwrap`, which has the same file and network rules
|
||||
with `audit deny capability`. So a role session gets a user namespace and no capabilities in it.
|
||||
The sysctl stays at 1, so nothing else on the host gains user namespaces.
|
||||
One consequence (N2): a bwrap that a session starts inside its own sandbox fails, because it needs
|
||||
capabilities in the namespace. Claude Code's own sandbox is such a bwrap, and it's off by default,
|
||||
so a role session doesn't use it. F3 shows a session still answers under the profile.
|
||||
Rollback: `apparmor_parser -R /etc/apparmor.d/bwrap-userns-restrict`, then `rm` it.
|
||||
3. `dpkg-statoverride --update --add root mosaic 0750 /usr/bin/bwrap` (Mos). Only root and the
|
||||
`mosaic` group can read or run bwrap, and package upgrades keep the mode. Rollback:
|
||||
`dpkg-statoverride --remove /usr/bin/bwrap`, then `chown` and `chmod` to the values in
|
||||
`bwrap.orig`.
|
||||
4. The drift stop (B2, below): `case1-hostcheck.sh` and `case1-pins.sh` from `/root/mosaic-stack` to
|
||||
`/usr/local/lib/mosaic-stack/`, `root:root` 0755 and 0644; the apt hook
|
||||
`/etc/apt/apt.conf.d/80mosaic-hostcheck`; the boot unit `mosaic-hostcheck.service`, enabled. Then
|
||||
`case1-hostcheck.sh --check` exits 0. Rollback: disable and remove the unit, `rm` the hook file and
|
||||
the dir.
|
||||
|
||||
Full rollback runs 4, 3, 2, 1. Tests after 2a' and again in section 5: as `mosaic`, `bwrap --unshare-all
|
||||
--ro-bind / / true` exits 0 (row 230's precheck line, F4); as `jarvis`, `/usr/bin/bwrap --version`
|
||||
fails with "Permission denied" (P9); `getent group mosaic` lists no member and no other account has
|
||||
gid `mosaic` (P9); `fs.protected_hardlinks` is 1 (P9). A role session running under the profile is F5.
|
||||
|
||||
One more test runs in 2a', because I can't settle it from here. Row 230's units keep the templates'
|
||||
`NoNewPrivileges=yes`, and AppArmor limits profile changes for a task with no-new-privs set. The
|
||||
`bwrap` attach happens on exec from an unconfined task, and the child's change is a stack
|
||||
(`bwrap//&unpriv_bwrap`), which only narrows. I expect both to pass under no-new-privs, since bwrap
|
||||
sets no-new-privs on itself before it runs the child, and Ubuntu's profile works with that. But I
|
||||
haven't seen it run, and .53 is read-only to me. So as root, after step 4: `systemd-run --wait --pipe
|
||||
--uid=mosaic --gid=mosaic -p NoNewPrivileges=yes -p ProtectSystem=strict -p ProtectHome=yes -p
|
||||
PrivateTmp=yes /usr/bin/bwrap --unshare-user --unshare-pid --ro-bind / / cat /proc/self/attr/current`
|
||||
prints `bwrap//&unpriv_bwrap (enforce)`. If it fails, the window stops at 2a' and rolls back 4 to 1,
|
||||
before any `mosaic` step. That beats finding it at F5.
|
||||
|
||||
The drift stop (B2, Duckworth's REVIEW-r3 B2, mos-1098 point 1). r4 installed a profile file that no
|
||||
package owns, from a pinned deb, onto a host where unattended-upgrades changes packages every day
|
||||
(section 0). Three things can move under it: bubblewrap (its security updates install unattended), the
|
||||
`apparmor` package and its parser, and any new profile a package ships that attaches to
|
||||
`/usr/bin/bwrap`. Two more host values matter to this design: the user-namespace sysctl and the port
|
||||
floor. So:
|
||||
|
||||
- `scripts/case1-pins.sh` holds the pins, sourced by the scripts below: the `apparmor-profiles` version
|
||||
the file came from, the file's sha256, and the bubblewrap version 2a' installs. Changing a pin is a
|
||||
reviewed commit and its own round. The bubblewrap version is an install pin, not a drift item:
|
||||
a security update to bubblewrap keeps the statoverride, and the profile attaches by path.
|
||||
- `scripts/case1-hostcheck.sh` checks seven items and prints one line each:
|
||||
1. `/etc/apparmor.d/bwrap-userns-restrict` has the pinned sha256, and `dpkg -S` finds no package
|
||||
that owns it.
|
||||
2. `aa-status --json` lists `bwrap` and `unpriv_bwrap` in enforce mode, and no loaded profile named
|
||||
`mosaic-bwrap`.
|
||||
3. `grep -rlw bwrap /etc/apparmor.d` lists exactly `/etc/apparmor.d/bwrap-userns-restrict`. The word
|
||||
match also catches a path written as `/{usr/,}bin/bwrap`. Section 0's read found no file today.
|
||||
4. The live label: 2a' line `systemd-run --wait --pipe --uid=mosaic --gid=mosaic -p NoNewPrivileges=yes
|
||||
-p ProtectSystem=strict -p ProtectHome=yes -p PrivateTmp=yes /usr/bin/bwrap --unshare-user
|
||||
--unshare-pid --ro-bind / / cat /proc/self/attr/current` prints exactly `bwrap//&unpriv_bwrap
|
||||
(enforce)`. It runs under the units' no-new-privs, so it reads the label a role session gets.
|
||||
This catches a profile that attaches by a glob that item 3's search misses.
|
||||
5. `dpkg-statoverride --list /usr/bin/bwrap` reads `root mosaic 0750`, and `stat` agrees.
|
||||
6. `kernel.apparmor_restrict_unprivileged_userns` is 1 and `fs.protected_hardlinks` is 1.
|
||||
7. `net.ipv4.ip_unprivileged_port_start` is above the Console's `--port` in the installed unit.
|
||||
|
||||
`--check` exits 0 when all hold and 1 otherwise. `--enforce` runs the same check and always exits 0.
|
||||
On a failure, or while `/etc/mosaic-stack/DRIFT` exists, it applies the stop:
|
||||
`apparmor_parser -R` on the profile if loaded, then `chown root:root` and `chmod 0700` on
|
||||
`/usr/bin/bwrap`. It writes the failed items to `DRIFT` and appends a line to
|
||||
`/var/log/mosaic-stack/hostcheck.log`. With bwrap closed to `mosaic`, every role session refuses
|
||||
to start. That's the fail-closed state, and the stack needs no change for it. The chmod is undone by
|
||||
the next bubblewrap upgrade, which reapplies the statoverride. But the apt hook runs again right
|
||||
after that upgrade and finds `DRIFT`, so it closes bwrap again. Only root removes `DRIFT`, in a
|
||||
reviewed step with a new pin or a Mos go. I don't call `dpkg-statoverride` from inside the hook,
|
||||
because I haven't confirmed it can take dpkg's lock while apt holds the frontend lock.
|
||||
- Three callers. A1 of every re-pin runs `--check` and refuses `profile-drift` on exit 1. The apt hook
|
||||
is `DPkg::Post-Invoke { "/usr/local/lib/mosaic-stack/case1-hostcheck.sh --enforce || true"; };`, so it
|
||||
runs after every package run, unattended ones included, and can't fail one. The boot unit,
|
||||
`mosaic-hostcheck.service` (`Type=oneshot`, `After=apparmor.service`,
|
||||
`Before=mosaic-bus@mosaic-stack.service`, `WantedBy=multi-user.target`), runs `--enforce` before the
|
||||
bus starts, which covers a kernel or parser change that takes effect at boot.
|
||||
- Owner. ops-01 owns the host side under a Mos go. It reads `hostcheck.log` in each STATE report and
|
||||
sends any `DRIFT` to Mos. Before each re-pin window it also runs `--upstream`, which downloads the
|
||||
current `apparmor-profiles` candidate into a temp dir and compares its `bwrap-userns-restrict` to the
|
||||
pin. A difference goes to Mos as a new-pin request and refuses nothing. The pins and the scripts are
|
||||
this row's, and later changes go through review like any other.
|
||||
|
||||
Who runs what:
|
||||
|
||||
| Step | Runs as | Through |
|
||||
|---|---|---|
|
||||
| 2a: `useradd --system --user-group --home-dir /srv/mosaic --shell /usr/sbin/nologin mosaic`, `/srv/mosaic` 0700, `/etc/mosaic-stack` `root:root` 0755, `/var/log/mosaic-stack` `root:root` 0750 | root | ops-01, `ssh localadmin@10.1.1.53`, `sudo` (#423 step 1) |
|
||||
| 2a': node under `/opt/mosaic-stack`; bubblewrap, the AppArmor profile, the statoverride, the drift stop and the managed settings file, each with the test and rollback above; the no-new-privs test | root | ops-01, same path, R27 (mos-1091) |
|
||||
| 2a'': a `git bundle` of the reviewed commit copied to .53; root clones it to `/root/mosaic-stack` and checks HEAD is the commit the go names; it installs the bundle as `/srv/mosaic/incoming/<sha>.bundle`, owner `mosaic`, 0600 | root | ops-01, same path |
|
||||
| 2b: claude under `~/.local` (above); `git clone` of the bundle to `/srv/mosaic/src/mosaic-stack`; `scripts/bootstrap.sh`; `stack-copy.sh copy`; the install with the inputs above (C2-13b's `case2-install.sh --profile factory` if it has landed, else the run sheet's steps) | `mosaic` | `sudo -u mosaic -H env PATH=...` from `localadmin` (PATH above). `mosaic` gets no key and no login shell |
|
||||
| 2b': place `claude-session.env` | Jason, or who he names (Q173 secrets rule) | section 3 |
|
||||
| 2c: write `case1-unit.args`; `case2-unit.sh` with those arguments and `--copy <id>`, from `/root/mosaic-stack`; then `case2-precheck.sh --profile factory` as `mosaic`, which reads the Console's `--port` from the installed unit for its port-floor line, so it runs after the render; `systemctl enable` the bus unit; the first start of UNITS. The Console unit has no `[Install]` until Jason rules (C2-13a), so it isn't enabled, and after a host reboot root starts it by hand until then. The run sheet says so | root | ops-01, same path |
|
||||
| 2c': `stack-release.sh adopt mosaic-stack --units external`, which records the running copy as the pointer (`do_adopt`, `scripts/stack-release.sh:856`) | `mosaic` | `sudo -u mosaic -H` |
|
||||
| every later re-pin or rollback | root, with its `mosaic` steps through `sudo -u mosaic -H` | ops-01 runs `case1-repin.sh` (section 4) |
|
||||
| acceptance (section 5) | `localadmin`, with `sudo -u mosaic -H` for the in-instance checks | the agent the window names |
|
||||
|
||||
No git credential lands on .53. Both checkouts come from bundles that ops-01 copies in, and a re-pin
|
||||
brings a new bundle that `mosaic` fetches into its checkout.
|
||||
|
||||
## 2. Where Case 1 differs from Case 2
|
||||
|
||||
| Case 2 item (row 230 brief) | Case 1 | Reason |
|
||||
|---|---|---|
|
||||
| business file with the library mapping | business `mosaic-stack` with no `library` key; `--profile factory` refuses a `kind: "library"` project (`profile-library-business`) | the factory has no books repo |
|
||||
| `bean`, `chet`, persona vars and files, specialists | factory roles from `roles/`, with a chief that has a harness var | Jason names the set in the go. I haven't read his live business file and won't. The acceptance needs only the chief (F3), since a chief with no harness var answers `no-route`. Others are added later by the human with `instances.set`, no new row |
|
||||
| token slot `library-gitea` | none | the factory's per-agent git identity is row 199's credential store (Megavolt, #1697). Until it lands, git work stays on dragon-lin (Q182) |
|
||||
| precheck: `bus.tester` is not on | `bus.tester` is on; `factory` requires it | the acceptance and the Q173 test system need the tester (agents act as tester, never Jason) |
|
||||
| precheck: persona file modes and digests | skipped under `factory` | no persona files |
|
||||
| precheck: `library.sync`, `library.status` | skipped under `factory` | no library |
|
||||
| precheck: `ZAI_API_KEY` in the key file | doesn't run | no instance has provider `zai` |
|
||||
| precheck: no Discord binding | kept | the factory's Discord connector stays on dragon-lin. Two connectors on one bot token would split its events |
|
||||
| precheck: claude marker prompt with `--system-prompt-file` | runs, since the chief's harness is claude-code | |
|
||||
| precheck: `sudo -n true` fails, `id -G` one entry, run root 0700, config validates, node 24 or later, `bwrap` exits 0, free memory, HOME | kept | node and bwrap are in row 230 (gosalyn-0034) |
|
||||
| Console serve flags: `--port` 1..1023, `--business`, the preview trio, `--send`, the proof pair, no `--tester-port` | `--port 732`, `--business mosaic-stack`, `--tester-port 7333`. No preview trio, no `--send`, no proof pair | below |
|
||||
| bus: `@PM_FLAGS@` and `@CLAUDE_ENV@` render empty | ` --pm`, and `Environment=MOSAIC_CLAUDE_BIN=/srv/mosaic/.local/bin/claude` | below |
|
||||
|
||||
All of these are inputs. Nothing in the Case 2 scripts hardcodes a Case 1 value (gosalyn-0034 item 4).
|
||||
Row 230's brief says the factory profile takes the library serve set plus an optional `--tester-port`,
|
||||
and that row 195 names any other difference (gosalyn-0049). .53 can't meet the library set, and the bus
|
||||
needs two lines the brief renders empty. The factory profile I need from row 230:
|
||||
|
||||
1. Serve flags. `--port N` with N in 1..1023 and `--business ID` are required, and `--tester-port N` in
|
||||
1024..65535, not equal to `--port`, is optional. The preview trio (`--public-host`, `--auth-header`,
|
||||
`--allow-user`) and the proof pair (`--proof-header`, `--proof-file`) are refused. Both exist for a
|
||||
proxy in front of the Console: the proxy signs the user in and adds the header, and the proof shows
|
||||
the request came through it. .53 has no proxy. Jason reaches the Console with `ssh -L` (sage-1451
|
||||
Q3), so there is no header to trust and no proxy to hold the proof. `--send` is refused too, since
|
||||
serve takes it only with the trio or `--sso` (`packages/webui/src/cli.mjs:15`). When Q175 SSO reaches
|
||||
.53, `--sso` with an optional `--send` (row 192) would be the factory's sign-in, and adding it is a
|
||||
re-install with its own go. Row 230 can accept `--sso [--send]` under factory now or later. Either
|
||||
way it needs a `console.sso` block in `mosaic`'s config, which this row doesn't write. `--checkout`
|
||||
and `--board` stay refused, as in the library set.
|
||||
2. `--pm` (factory only) renders the bus unit's `@PM_FLAGS@` as ` --pm`. The factory's PM dispatches its
|
||||
work on .53 as on dragon-lin (section 4). With the token empty the bus never launches the PM, and
|
||||
A5's PM check fails every re-pin.
|
||||
3. `--claude PATH` (factory only) renders `@CLAUDE_ENV@` as `Environment=MOSAIC_CLAUDE_BIN=PATH`, the
|
||||
line `scripts/bus-service.sh:79` renders today. PATH must be absolute. Without it the bus runs
|
||||
`claude` from the unit's PATH, which is the node dir, `/usr/local/bin`, `/usr/bin` and `/bin`, and
|
||||
holds no claude on .53. The chief's first launch would then fail. The precheck's claude marker
|
||||
prompt should run the same binary, so it reads the path from the installed bus unit.
|
||||
4. Row 190's two template tokens, if row 190 lands before row 230. Row 190 adds `@CONFIG_ENV@` to both
|
||||
templates and `@SESSION_ENV@` to the bus template's `EnvironmentFile=` line. For the default
|
||||
instance, `@CONFIG_ENV@` is deleted (`scripts/instance.sh`) and `@SESSION_ENV@` is
|
||||
`<home>/.config/mosaic-dev/secrets/claude-session.env`. Row 230's "every token has one value" list
|
||||
and its `EnvironmentFile=` check need both. This applies to the library profile too.
|
||||
|
||||
I've sent these to Gosalyn. If row 230's candidate lacks any of them, row 195's manifest names the
|
||||
change to `case2-unit.sh` or `case2-precheck.sh`, reviewed by Duckworth with Webby told.
|
||||
|
||||
## 3. Model access
|
||||
|
||||
Q185: Case 1 uses the account the factory uses today, the Claude subscription behind dragon-lin's
|
||||
`claude-session.env`. The install never copies that file from dragon-lin. In the window, Jason (or the
|
||||
person he names under the Q173 secrets rule) makes a credential for .53 from that account and writes it
|
||||
to `/srv/mosaic/.config/mosaic-dev/secrets/claude-session.env` as `mosaic`, mode 0600. Nobody prints it,
|
||||
and no script reads its value. The precheck checks only its mode, owner and that the variable name is
|
||||
present (a `grep -c` on the name, output a count).
|
||||
|
||||
Quota: the factory on dragon-lin and on .53 share one subscription. Q185's "use existing" accepts that.
|
||||
Case 2 shares nothing with it (Mos's reading of Q185).
|
||||
|
||||
The pi harness stays off on .53 until P19 #383 settles (mos-0865). The business file names claude only.
|
||||
|
||||
## 4. Re-pin, rollback and backup on .53
|
||||
|
||||
Unit form (ruled, sage-1451 Q1): system units that root writes, with `User=mosaic`. `mosaic` can't
|
||||
start or stop them, and root never runs code that `mosaic` can write. `stack-release.sh activate` today
|
||||
does everything in one run under `systemctl --user` and puts the old stack back inside that run
|
||||
(`rollback_in_run`, `scripts/stack-release.sh:561`). On .53 the same run needs two users. So one
|
||||
re-pin is one ordered list, and one root command drives all of it: `scripts/case1-repin.sh`, run by
|
||||
ops-01 from `/root/mosaic-stack`. Each `mosaic` step is `sudo -u mosaic -H` running
|
||||
`/srv/mosaic/src/mosaic-stack/scripts/stack-release.sh` with `--units external`. Copies carry no
|
||||
`stack-release.sh` (`COPY_PATHS`, `scripts/stack-copy.sh:23-30`), so the `mosaic` steps run from that
|
||||
checkout, as dragon-lin's run from the canonical one.
|
||||
|
||||
What root holds, and where each value comes from (r4's B1):
|
||||
|
||||
| Value | Source |
|
||||
|---|---|
|
||||
| ID, the target copy | the go |
|
||||
| TO, a rollback's target | the go, as `--to ID` |
|
||||
| `--restore ACT --events N` | the go, when the rollback restores |
|
||||
| OLD, the running copy | the `ExecStart=` of the two unit files root installed. If the bus and Console units name different copies, A1 refuses `units-disagree` |
|
||||
| the `case2-unit.sh` arguments | `/etc/mosaic-stack/case1-unit.args` (section 1) |
|
||||
| each `mosaic` step's result | its exit code |
|
||||
|
||||
Root parses nothing `mosaic` wrote and runs nothing under `/srv/mosaic` except through `sudo -u mosaic
|
||||
-H`. It never needs the activation id: the `mosaic` steps find it themselves.
|
||||
|
||||
One result line per activation (B1). The stack log closes an activation on the first line after
|
||||
`started` that names it, whatever the result (`interrupted`, :146-156). r4 wrote an interim
|
||||
`... rollback pending` line at A3's and A5's failures. That line closed the activation, so R2 and R4
|
||||
then refused `open-activation` against an activation the log no longer held open. The rollback could
|
||||
never run as designed, and nothing tested the steps in sequence. r5 keeps an activation open from A3's
|
||||
`started` line until its single result line, and A5 or R4 writes that line:
|
||||
|
||||
| Activation ends at | Result line | Written by |
|
||||
|---|---|---|
|
||||
| A3 fails before its rollback set exists (marker or capture) | `refused code marker-failed` or `refused code capture-failed`, as `capture || refuse` does today (:987). Nothing has changed, so there is nothing to roll back | A3 itself, exit 3, after it removes the marker. Root runs A2f |
|
||||
| A5 passes | `ok` | A5 |
|
||||
| A3 (after capture), A4 or A5 fails, and the rollback passes | `failed code <c> rollback ok` for A3 and A4, `gate-failed code <c> rollback ok` for A5 | R4 |
|
||||
| the rollback fails at R4's gate | `rollback-failed code <c> rollbackCode <r>` | R4 |
|
||||
| R1, R2 or R3 fails, or the run dies | none. The activation stays open, and the next activate refuses `interrupted-run` (:603-606), which Jason or Mos settles as on dragon-lin | |
|
||||
|
||||
What the result line carries (r5's B1). `log_line` (:58) writes `op "$OP"`, and `from_log` (:169)
|
||||
reads only lines whose op is `adopt`, `activate` or `rollback`. So `confirm` doesn't log under its own
|
||||
op. Before A5's `ok` line or any line R4 writes, it sets OP, FROM and TO from the open activation's
|
||||
`started` line, ACT to that activation, and BACKUP from the rollback set's `BACKUP` file, which A3
|
||||
writes when `backup_store` returns. A re-pin's lines then read `op activate`, a rollback's `op
|
||||
rollback`, each with the from, to and backup a dragon-lin run would log. `from_log previous` and
|
||||
`from_log backup` read .53's log as they read dragon-lin's. `confirm` reads only the line A3 wrote for
|
||||
that activation, which `interrupted` already names.
|
||||
|
||||
A failing `mosaic` step after `started` writes no log line. It writes its code to `FAILED` beside
|
||||
`AWAITING-UNITS` and exits 1. R4 reads that code for the result line. Root passes R4 `--cause
|
||||
A3|A4|A5`, the step it saw fail, from its own exit codes. A4 is root's own step and has no `FAILED`
|
||||
file, so R4 writes `failed code units-start` for it. A cause with no `FAILED` file where one should be
|
||||
writes code `unrecorded`.
|
||||
|
||||
The open activation. A3 writes `$STATE/stack-rollback/<ACT>/AWAITING-UNITS` holding the target copy
|
||||
(renamed from r3's `PENDING`, which read like the pointer's `pmPending`, `P_PENDING` at :128; N3).
|
||||
`confirm` and `revert` take the copy root passed, not an activation id. Each lists the
|
||||
`AWAITING-UNITS` files and the `interrupted` set (:146), and goes on only when there is exactly one
|
||||
file, its activation is the one `interrupted` names, and the copy in it is the copy root passed.
|
||||
Anything else refuses `open-activation` with exit 3 and the list it found, and changes nothing.
|
||||
`confirm` without `--rollback` also refuses `activation-failed` when `FAILED` exists, so a second
|
||||
confirm can't pass an activation that already failed. Whichever `confirm` writes the result line
|
||||
removes `AWAITING-UNITS` and `FAILED` after it.
|
||||
|
||||
Refusals of the three external verbs. `confirm`, `revert` and `gate` act on an open activation by
|
||||
design, so they skip the `interrupted-run` refusal (:605). Their refusals log `result refused` with no
|
||||
activation id: they set ACT only after every check has passed. So a refused `confirm`, a wrong copy
|
||||
say, leaves the open activation open. Today's `refuse` (:62-67) adds the activation id whenever ACT is
|
||||
set, and the external verbs keep that order. A test and a mutant hold it (section 7).
|
||||
|
||||
Root's step log. Every step `case1-repin.sh` runs appends one line to `/var/log/mosaic-stack/case1-repin.log`,
|
||||
`root:root` 0640: the time from `date -u`, the step, its argv, its exit code. `mosaic` can't write it,
|
||||
so it stays readable after a run that dies before `mosaic` logs anything.
|
||||
|
||||
### Activate: `case1-repin.sh activate ID [--migrate] [--timeout S]`
|
||||
|
||||
| # | Step | Runner | On failure |
|
||||
|---|---|---|---|
|
||||
| A1 | `/root/mosaic-stack` HEAD is the commit the go names. ID is 40 lowercase hex and names a copy under `/srv/mosaic/.local/share/mosaic-stack/copies` that holds `scripts/mosaic` (row 230's copy-id rule). `case1-hostcheck.sh --check` exits 0, else `profile-drift` (B2). Root reads OLD from the unit files. `stack-release.sh activate ID mosaic-stack --units external --plan --expect-from OLD` exits 0. The plan refuses `pointer-differs` when the pointer's copy isn't OLD (N5), so a pointer and units that disagree stop before any stop | root; the plan as `mosaic` | refuse, exit 3. Nothing has changed |
|
||||
| A2 | `systemctl stop` UNITS in reverse order, then wait until each is inactive, within `--timeout` | root | A2f below |
|
||||
| A3 | `stack-release.sh activate ID mosaic-stack --units external [--migrate]`. Before it logs `started` (N3), it refuses `units-running` unless each of UNITS is inactive (system-manager reads, which need no privilege) and the store is stopped (`stopped`, :359), and `interrupted-run` while any activation is open. Then, in this order: logs `result started`; makes `$RS` with capture's `mkdir -p` of `stack-rollback` and exclusive `mkdir -m 0700` (:415-416), and writes `AWAITING-UNITS` in it at once (N1); captures the rollback set into that directory, making none (pointer, wrapper, and the sha256 and `ExecStart=` of each installed unit, read only); takes `backup_store` (:372) into `$DATA/backups/stack/<ACT>` and writes its name to `BACKUP` in the rollback set; under `--migrate`, writes `MIGRATING` in the rollback set and sets TAKEN, both before `migrate-schema.mjs` runs, where today's run sets TAKEN (:1004-1005, r5's B2), then migrates; installs the wrapper for ID. It writes no pointer | `mosaic` | a refusal before `started` exits 3 with nothing changed. A marker or capture failure, an existing `$RS` included, removes the marker if it wrote one, logs its `refused` line and exits 3. Root runs A2f for both. Any later failure writes `FAILED` with its code, logs nothing, cleans up nothing and exits 1. Root runs R1-R4 with `--cause A3` |
|
||||
| A4 | `case2-unit.sh` with `case1-unit.args` and `--copy ID`, `systemctl daemon-reload`, `systemctl start` UNITS in order | root | R1-R4 with `--cause A4` |
|
||||
| A5 | `stack-release.sh confirm ID mosaic-stack --units external --timeout S`. This is `gate` (:522), which polls `gate_once` (:471) every second until `--timeout`, with the external checks below. On a pass it writes the pointer, logs `result ok`, removes `AWAITING-UNITS`, exit 0. When the timeout runs out it writes `FAILED` with the last gate code, logs nothing and exits 1 | `mosaic` | R1-R4 with `--cause A5` |
|
||||
|
||||
Why A3 writes the marker first and does no cleanup of its own. With the marker written last, as in r4,
|
||||
an A3 failure after `started` left an open activation with no marker, and R2 and R4 refused it. With
|
||||
the marker first, every A3 failure after the marker leaves exactly what R2 needs: the open activation,
|
||||
its marker and whatever part of the rollback set exists. R2 is then the one cleanup path, for A3, A4
|
||||
and A5 alike. Two cleanup paths, A3's own and R2, would each need their own tests for each failure point.
|
||||
|
||||
A2f, a stop that fails or times out, or an A3 that exits 3. This run has no open activation: A2 logs nothing, and
|
||||
A3's exit 3 comes either before `started` or after its own `refused` line. Nothing of `mosaic`'s has
|
||||
changed. Root reads `date -u +%s` as T, runs `systemctl start` UNITS in order, then `stack-release.sh gate
|
||||
OLD mosaic-stack --units external --since T --timeout S`, a new verb that runs `gate` on OLD and writes no
|
||||
log line, no pointer and no file. A2f has no `started` line to take a start floor from (the first
|
||||
re-pin after adopt has none at all), so `--since` is the floor, and `gate` refuses `usage` without it
|
||||
(r5's N2). T is root's own clock, so root still parses nothing `mosaic` wrote. A unit that never
|
||||
stopped keeps its old `ActiveEnterTimestamp`, fails `stale-start` and takes the exit 2 path below.
|
||||
If that passes, root logs `A2 failed, OLD running` (or `A3 refused`) in its step log and exits
|
||||
1. If it fails, root stops UNITS, logs `OLD gate failed <code>` and exits 2. The next run's
|
||||
`interrupted` check finds nothing open either way.
|
||||
|
||||
The external gate. `gate_once` keeps its order and codes, with these changes under `--units external`:
|
||||
|
||||
- `systemctl` without `--user` for `is-active` and `show`. Both work unprivileged against the system
|
||||
manager.
|
||||
- This start's time is the bus unit's `ActiveEnterTimestamp`, read with `systemctl show --timestamp=unix
|
||||
-p ActiveEnterTimestamp`, in place of the `START_MS` that `start_both` (:451) sets on dragon-lin.
|
||||
It is whole seconds (`@1792540800`), and the `started` line has milliseconds (N4). So the rule
|
||||
compares seconds: the start passes when its second is at or after the floor of `started`'s second.
|
||||
A unit that was active across A3 can't reach this check, because A3 refuses `units-running`, so the
|
||||
equal second can only be a start after A3. An earlier second fails `gate_once` with the gate code
|
||||
`stale-start`. It's a gate failure like any other, not a refusal (r4 had it as a refusal, which
|
||||
would have logged against the open activation): `gate` keeps polling, since `Restart=on-failure`
|
||||
can bring a fresh start, and at the timeout `FAILED` holds `stale-start`. The PM check's
|
||||
`startedAt` bound uses the same second.
|
||||
- `copy-check` runs `stack-copy.sh check --units external`, which reads the system units.
|
||||
- The PM check is below.
|
||||
|
||||
The PM (r4's B2). .53 runs the PM: the factory's PM dispatches its work, as it does on dragon-lin, and the
|
||||
serve flags carry `--pm`. The pointer's `pm` comes from the installed bus unit's `ExecStart=`, as the
|
||||
unit parse at :195 already reads it. Today's `pm_launched` (:505) takes the run id from the
|
||||
`launched ...` line in `journalctl --user -u BUS_UNIT`. On .53 that read fails twice over: the unit is
|
||||
a system unit, and under `SplitMode=uid` the logs of a system-range uid land in the system journal,
|
||||
which `mosaic` (no `adm` or `systemd-journal` group) can't read. Under `--units external` the check
|
||||
reads `launches list --json` alone. It passes when exactly one entry has business `mosaic-stack`,
|
||||
instance equal to the business file's `launch.by` (the PM's instance, the one `bus start --pm` launches
|
||||
at `packages/cli/src/launcher.mjs:675`), `live` true, and `startedAt` at or after this start's time.
|
||||
Zero entries, two, a stale one or one for another business fail with `pm-launch-unconfirmed` and what
|
||||
it saw, as today. A run the PM starts for another role has another instance, so it doesn't count.
|
||||
|
||||
Why not the other reads. A regular uid for `mosaic` would put its logs in a per-user journal it can
|
||||
read, but it changes the account Case 2's install creates. Root reading the journal and passing the run id in
|
||||
would make root parse output from `mosaic`'s process, which section 4's rule forbids.
|
||||
|
||||
Under `--units external`, `pm-on` (:1137) and `activate --pm-after-harness` refuse `external-pm`. Both
|
||||
flip the PM on dragon-lin by restarting the user bus with new arguments. On .53 the PM's state is a
|
||||
serve flag in a root-owned unit, so changing it is a re-install with its own go.
|
||||
|
||||
### Rollback inside the run (R1-R4)
|
||||
|
||||
| # | Step | Runner | On failure |
|
||||
|---|---|---|---|
|
||||
| R1 | `systemctl stop` UNITS, then wait as in A2 | root | rollback-failed |
|
||||
| R2 | `stack-release.sh revert ID mosaic-stack --units external`. It finds the open activation as above, puts the wrapper back from the rollback set and checks its sha256, and restores TAKEN (`restore_store`, :396) only if the rollback set holds `MIGRATING`. Each part is idempotent, so it works for an A3 that stopped partway: no `MIGRATING` means no restore, a migrate that failed partway is restored from the backup taken before it, and copying the saved wrapper over an unchanged one changes nothing. It writes no log line | `mosaic` | rollback-failed, activation left open |
|
||||
| R3 | `case2-unit.sh` with `case1-unit.args` and `--copy OLD`, `systemctl daemon-reload`, `systemctl start` UNITS | root | rollback-failed, activation left open |
|
||||
| R4 | `stack-release.sh confirm ID mosaic-stack --units external --rollback OLD --cause A3\|A4\|A5 --timeout S`. It finds the open activation, checks OLD is its `from`, gates OLD with `gate`, and writes the activation's one result line from the cause and the `FAILED` code: `failed code <c> rollback ok` (A3, A4), `gate-failed code <c> rollback ok` (A5), or `rollback-failed code <c> rollbackCode <gate>` when OLD's gate fails | `mosaic` | rollback-failed, logged |
|
||||
|
||||
rollback-failed: root stops UNITS and exits 2, as `rollback_failed` (:587) does. When R4's gate fails,
|
||||
the log has the `rollback-failed` line and the activation is closed. When R1, R2 or R3 fails, no
|
||||
`mosaic` step writes a result, so the activation stays open with its marker. Root's step log has the
|
||||
codes, the next activate refuses `interrupted-run` until Jason or Mos settles it (:605), and status
|
||||
shows it. I chose that over a log-only verb for root to call after an R2 failure: nothing could gate
|
||||
the old stack at that point, and a closed activation would let the next activate start against a
|
||||
wrapper and store in an unknown state. Exit codes match `stack-release.sh`: 0 ok, 1 failed with the old
|
||||
stack running (rolled back, or A2f), 2 rollback-failed with UNITS stopped, 3 refused with nothing
|
||||
changed.
|
||||
|
||||
### Rollback to the previous copy: `case1-repin.sh rollback --to ID [--restore ACT --events N]`
|
||||
|
||||
The same list with TO in place of ID. The go names TO, from Mos's or Sage's read of `stack-release.sh
|
||||
status` as `mosaic`. Root passes it. A1's plan, `stack-release.sh rollback mosaic-stack --units external
|
||||
--to ID --plan --expect-from OLD`, refuses `pointer-differs` as in activate. It also exits 3 unless ID
|
||||
is `from_log previous` (:162), the previous distinct copy, so a
|
||||
stale or mistyped go refuses before anything stops. With `--restore`, the plan also refuses unless
|
||||
`$DATA/backups/stack/ACT` exists. `--to` is a new flag, accepted only with `--units external`;
|
||||
dragon-lin's rollback keeps reading TO from the log.
|
||||
|
||||
The post-backup event count runs in A1's plan as `mosaic` while the bus still holds the store (the
|
||||
`heldcount` read, :1074), and again in A3 on the stopped store. A difference refuses
|
||||
`post-backup-events`, as `run_steps` does (:1013).
|
||||
|
||||
### Migrations (N4)
|
||||
|
||||
`--migrate` under `--units external` runs only inside A3, and A3 refuses `units-running` before it logs
|
||||
or migrates anything while any of UNITS is active. So a migration never runs against a store a live bus
|
||||
holds. The test is in section 7.
|
||||
|
||||
### Backup
|
||||
|
||||
A3's `backup_store` is today's. It copies the stopped store into `$DATA/backups/stack/<ACT>`, so
|
||||
`rollback --restore <ACT>` and row 181's guard find it. As on dragon-lin, the config dir isn't in the
|
||||
backup, and the credential isn't either. PBS backups of VM 1008 cover the disk (P4). A restore swaps
|
||||
only `bus/` and leaves `secrets/claude-session.env` in place (N6, in the run sheet).
|
||||
|
||||
The install's adopt (2c') takes no backup. The store holds only what the acceptance wrote, and the first
|
||||
re-pin's A3 takes the first backup.
|
||||
|
||||
### The log
|
||||
|
||||
Every `mosaic` line goes to `$DATA/state/stack-activation-log.jsonl`, as today, with `units external`
|
||||
added. So `from_log previous` reads .53's history the same way it reads dragon-lin's. A3 writes
|
||||
`started`. A5 or R4 writes the one result line, with the op, from, to and backup of that activation,
|
||||
or A3 its `refused` line before any change. Refusals of
|
||||
`confirm`, `revert` and `gate` log no activation id. If a run dies between `started` and the result,
|
||||
the log keeps `started` with no result, the next run's `interrupted` check reports it, and root's step
|
||||
log shows the last step that ran.
|
||||
|
||||
### Who
|
||||
|
||||
Mos or Sage plans each re-pin. ops-01 runs `case1-repin.sh` on Jason's or Mos's go, and a STATE report
|
||||
follows. The gate and the in-run rollback have one owner, the root command, as they do on dragon-lin.
|
||||
|
||||
### dragon-lin's path
|
||||
|
||||
`--units external` is a new value. Without it, `stack-release.sh` and `stack-copy.sh check` run as they
|
||||
do today, `pm_launched` still reads the user journal, and rollback still takes TO from the log.
|
||||
`confirm`, `revert`, `gate` and `--to` refuse without it. Section 7 names the tests that show this.
|
||||
|
||||
### Not taken
|
||||
|
||||
- Row 190's user units under `mosaic`, with root running `loginctl enable-linger mosaic` once. Every
|
||||
existing re-pin tool would work unchanged and root would be needed only once. But the unit form
|
||||
would differ from Case 2, and a user manager would run as `mosaic`.
|
||||
- A sudoers rule that lets `mosaic` start and stop the units. P1 and row 230's precheck require that
|
||||
`mosaic` has no sudo.
|
||||
- `confirm` and `revert` taking ACT from root. Root would have to read it from `mosaic`'s output or
|
||||
log, which the rule above forbids (r4's B1).
|
||||
|
||||
## 5. Acceptance checks
|
||||
|
||||
A new script, `scripts/case1-accept.sh`, in this row's manifest. It runs on .53 as `localadmin` and prints
|
||||
one line per check: `<id> <path> <mark> <evidence>`. The marks are `closed`, `accepted Q182`,
|
||||
`accepted Q183`, `accepted mos-1096`, and `unnamed`, which means "not named in Q182, Q183 or a Mos
|
||||
ruling". The script ends with a count of each mark. It exits non-zero only when a functional check (F)
|
||||
fails. A P check never changes the exit status. Each `unnamed` line goes to Mos, who takes it to Jason
|
||||
(R8 N1).
|
||||
|
||||
Functional checks. Any failure fails the row:
|
||||
|
||||
| Id | Check | How |
|
||||
|---|---|---|
|
||||
| F1 | the units run as `mosaic` | `systemctl show <unit> -p User -p MainPID` for each of UNITS; each MainPID's uid in `/proc/<pid>/status` is `mosaic`'s, and each unit is active |
|
||||
| F2 | the Console listens on 127.0.0.1:732 on .53 and answers there | `curl -fsS http://127.0.0.1:732/healthz` on .53 succeeds, and the listener table shows 732 bound to 127.0.0.1 only, by the Console unit's MainPID. Jason reaches it with `ssh -L 7332:127.0.0.1:732 localadmin@10.1.1.53` and his browser at 127.0.0.1:7332 as today. A LAN listener waits for Q175 SSO (sage-1451 Q3) |
|
||||
| F3 | one tester message gets a reply | as `mosaic`: `mosaic talk chief <marker> --tester`, then `mosaic follow --tester`, with a 300 s limit. A stored reply to the marked message passes. The inbox alone shows decisions only, so it isn't used (N3) |
|
||||
| F4 | the precheck passes | `case2-precheck.sh --profile factory` exits 0 as `mosaic` |
|
||||
| F5 | a role session runs under bwrap with the profile | while F3's reply is pending: a `/usr/bin/bwrap` process owned by `mosaic` has a claude process under it. `/proc/<bwrap pid>/attr/current` equals `bwrap (enforce)` and the claude process's equals `bwrap//&unpriv_bwrap (enforce)`, each compared as an exact string after the trailing newline is cut (N1). If `launches list --json` records the sandbox, the build reads that for the first part, since a short session can end before `ps` sees it |
|
||||
| F6 | claude is the pinned version | `/srv/mosaic/.local/bin/claude --version` as `mosaic`, after F3, equals the run sheet's pin |
|
||||
| F7 | the PM run is live | `launches list --json` as `mosaic` has exactly one live entry for the PM's instance, the check `confirm` runs (section 4) |
|
||||
|
||||
Privilege and host paths. These report and never fail the row:
|
||||
|
||||
| Id | Path | Mark | Evidence the script prints |
|
||||
|---|---|---|---|
|
||||
| P1 | `mosaic` has sudo | closed only when `sudo -l -U mosaic` prints "is not allowed to run sudo" (N2); any other output is unnamed | the `sudo -l` result |
|
||||
| P2 | `mosaic` has supplementary groups or a key | closed when `id -G mosaic` is one entry and `~mosaic/.ssh/authorized_keys` is absent or empty | `id`, file state |
|
||||
| P3 | `localadmin` NOPASSWD sudo, reached with the dragon-lin key | accepted Q183 (agents keep ssh to .53; localadmin from #423 step 1) | `sudo -l -U localadmin` |
|
||||
| P4 | root on pve over VM 1008, and PBS backups of VM 1008, which hold `/srv/mosaic` including the credential | accepted Q182 and Q183 (agents keep root on pve and PBS) | a fixed line; the script can't see pve from inside |
|
||||
| P5 | `jarvis`, which runs Hermes, can reach `/srv/mosaic` through sudo, or through the docker or lxd groups, which are root-equivalent | closed when `jarvis` is in none of sudo, docker and lxd; unnamed otherwise. #423 step 4 lands before the install (mos-1091 4c), and its `step4.sh` removes `jarvis` from sudo, docker, lxd and adm, so after it this reads closed (Duckworth N2) | `id jarvis` |
|
||||
| P6 | other members of sudo, docker, lxd | closed when sudo holds only `localadmin` and docker and lxd are empty. Every other member is its own unnamed line, `jarvis` included (N1) | `getent group` lines |
|
||||
| P7 | a file under `/etc/sudoers.d` naming `mosaic` | closed when none | the file names |
|
||||
| P8 | the Console or tester reachable from the LAN | closed when 732 and 7333 are bound to 127.0.0.1 only, each by the Console's MainPID, and nothing listens on 7331 | listener table |
|
||||
| P9 | a user namespace through `/usr/bin/bwrap` | accepted mos-1096 only while all seven hold, unnamed otherwise: (1) `dpkg-statoverride --list /usr/bin/bwrap` reads `root mosaic 0750`; (2) `aa-status --json` lists `bwrap` and `unpriv_bwrap` in enforce mode; (3) `/etc/apparmor.d/bwrap-userns-restrict` has the pinned sha256 and no package owns it; (4) `sudo -u jarvis /usr/bin/bwrap --version` fails with "Permission denied"; (5) `getent group mosaic` lists no member and no other account in `/etc/passwd` has its gid; (6) `fs.protected_hardlinks` is 1; (7) no `mosaic-bwrap` profile is loaded, no other file under `/etc/apparmor.d` names `bwrap`, and the live label of a process bwrap runs is exactly `bwrap//&unpriv_bwrap (enforce)` (N1, mos-1098 point 2). Items 1-3, 6 and 7 are `case1-hostcheck.sh --check`'s, so P9 reuses its lines | each item's line, and Mos's reason: "Case 1's agents are the factory's agents. In Q182 and Q183 Jason accepted that the factory reaches root on pve and, through it, every VM. So a user-namespace path to root on .53 gives that class no new reach." With the profile, what's left is bwrap's own setup, run by `mosaic` |
|
||||
| P10 | a session binds the tester port, 7333, while the Console is down | accepted mos-1101 while `ip_unprivileged_port_start` is above 732 and 7333 is held by the Console's MainPID at the check. Serve refuses a tester port below 1024 (`cli.mjs:33`), so the Console-port fix can't cover it. A squatter would get the tester's writes and could answer them. The tester's users are the factory's agents, the class Mos's reading covers (mos-1101 item 5). Mos can object, and then this line reads unnamed | the floor, and the owner pid of 7333 |
|
||||
|
||||
Tests, in `packages/cli/tests/case1-accept.test.mjs`. The script takes a fake root (`--root DIR`) for
|
||||
`/proc`, `/etc` and `getent` output, and fake `systemctl`, `curl`, `ss`, `sudo`, `bwrap`, `aa-status`,
|
||||
`dpkg-statoverride`, `sysctl` and `claude` on PATH. Cases:
|
||||
- each F check failing gives a non-zero exit with its line;
|
||||
- each P check in each of its marks, including P1 with output that grants nothing but lacks the refusal
|
||||
text (unnamed), P9 unnamed once for each of its seven items missing, and P10 unnamed with a floor of
|
||||
700 and with 7333 held by another pid;
|
||||
- F5 fails on a label of `unpriv_bwrap (enforce)`, `bwrap//&unpriv_bwrap (complain)` and
|
||||
`bwrap (enforce)` with a trailing space;
|
||||
- a run with every P check `unnamed` still exits 0;
|
||||
- the summary counts match the lines.
|
||||
|
||||
Mutants: one per F check (the check made to pass on failure), one that maps P5 to `closed`, one that
|
||||
closes P1 on any output without a rule line, one that drops P9's jarvis item, one that drops P9's item
|
||||
7, and one that makes F5 a substring match.
|
||||
|
||||
Nothing in the tests reaches .53 or dragon-lin's live state.
|
||||
|
||||
## 6. Rulings
|
||||
|
||||
sage-1451:
|
||||
|
||||
- Q1, unit form: the brief's system unit. C2-13's `ExecStart=` check accepts a varying copy id, and
|
||||
C2-13 took that (gosalyn-0034 item 3).
|
||||
- Q3, the Console: it listens on 127.0.0.1 on .53 (`packages/webui/src/serve.mjs:466`; `cli.mjs` passes no
|
||||
host), and Jason reaches it with `ssh -L`. F2 says so. A LAN listener waits for Q175 SSO.
|
||||
- Q4, the date: if C2-13's candidate is late, I build on C2-13's latest commit and rebase (Q177). Under
|
||||
the #426(a) push rules (mos-1086) a rebase goes to a new branch name.
|
||||
|
||||
mos-1091 (none goes to Jason):
|
||||
|
||||
- Q2 4a: installing bubblewrap is an install step, run by ops-01 as root under R27. Rollback is
|
||||
`apt-get remove`.
|
||||
- Q2 4b: a per-binary AppArmor profile for `/usr/bin/bwrap`, the sysctl left at 1. r4 takes Ubuntu's
|
||||
`bwrap-userns-restrict` for it (mos-1096). Section 1 has the steps.
|
||||
- 4c: P5 ends at #423 step 4, which Mos sent to Duckworth so it lands in Jason's VM 1008 boot session,
|
||||
before the install. The P5 check stays.
|
||||
|
||||
mos-1096:
|
||||
|
||||
- B1 and B2 are mine, and so is the PM answer: .53 runs the PM (section 4).
|
||||
- The `dpkg-statoverride` on `/usr/bin/bwrap`, the hardlink and group checks, the jarvis refusal and F5
|
||||
under the profile (sections 1 and 5).
|
||||
- P9 is Mos's acceptance, with the reason quoted in section 5.
|
||||
- Jason's Q9 on install and upgrade is open. r4 goes ahead, since B1's re-pin and rollback are needed
|
||||
whatever he answers. If his answer changes how upgrades reach .53, the change lands in section 4 and
|
||||
the run sheet as a new round.
|
||||
|
||||
mos-1098 and mos-1101 (on Duckworth's REVIEW-r3):
|
||||
|
||||
- Duckworth's CHANGES stand. B1 and B2 are fixed in this round, with B1's real-sequence test, and all
|
||||
three of B2's asks: an owner across upgrades, what fails closed, and the pins as reviewed values.
|
||||
- r4's PM, statoverride and P9 stand. mos-1098's two questions are answered in section 1: the profile's
|
||||
owner across upgrades is ops-01 with `case1-hostcheck.sh`, and the stop is bwrap closed to `mosaic`.
|
||||
Both profiles attach to `/usr/bin/bwrap`: `bwrap` to the binary and `unpriv_bwrap` stacked on what it
|
||||
runs. P9 item 7 checks that no third one does.
|
||||
- P9 has seven items and F5 compares exact strings (N1).
|
||||
- The port squat applies (section 1). The Console takes Case 2's fix, a port below 1024 with
|
||||
`CAP_NET_BIND_SERVICE` and the floor checked. Only the tester port can't take it, and P10 records it
|
||||
under Mos's reading that Case 1's sessions are the factory class. I've told Mos, who can object.
|
||||
- Work on r5 goes ahead while Jason's Q9 (Vikunja #429) is open. If install and upgrade become
|
||||
product commands, these steps become that installer's configuration and row 195 rebases on its row
|
||||
(sage-1481, sage-1491).
|
||||
|
||||
sage-1499: C2-13a is row 230 (#1732, Gosalyn), and row 195 chains on row 230's candidate.
|
||||
|
||||
gosalyn-0049: the factory profile is the library set plus an optional `--tester-port`, and row 195 names
|
||||
any other difference. Section 2 names four, with the reason .53 can't meet the preview trio, `--send` or
|
||||
the proof pair. The managed-settings route needs no Case 2 change.
|
||||
|
||||
## 7. Manifest (planned)
|
||||
|
||||
- `scripts/case1-repin.sh` (new) and `packages/cli/tests/case1-repin.test.mjs` (new). Fake `systemctl`,
|
||||
`sudo` and `case2-unit.sh` on PATH record their argv, with a scratch home standing in for `/srv/mosaic`,
|
||||
a scratch dir for `/etc/mosaic-stack` and one for `/var/log/mosaic-stack`. Cases:
|
||||
- a pass runs A1-A5 in order;
|
||||
- a failure at A3, A4 or A5 runs R1-R4 with `--cause` set to that step, and exits 1;
|
||||
- an A3 exit 3 runs A2f, not R1-R4;
|
||||
- a failure at R2 or R4 exits 2 with UNITS stopped;
|
||||
- A1 refuses `profile-drift` when a fake `case1-hostcheck.sh --check` exits 1, before any stop;
|
||||
- every `mosaic` step's argv carries `env PATH=<node dir>:/usr/local/bin:/usr/bin:/bin`, with the node
|
||||
dir from `case1-unit.args`;
|
||||
- a HEAD mismatch, an ID that fails the copy-id rule, or unit files that name two copies refuses with
|
||||
exit 3 before any stop;
|
||||
- A1's plan argv carries `--expect-from` with OLD from the unit files;
|
||||
- A2f passes `gate OLD --since T`, with T from a fake `date` that root calls before its start;
|
||||
- A2f: a fake `systemctl` that never reports inactive. The script starts UNITS, runs `gate OLD`, exits
|
||||
1 with OLD's gate passing and 2 with it failing, and the stack log is byte-identical before and
|
||||
after;
|
||||
- `rollback --to` with an ID that isn't `from_log previous` exits 3 before any stop;
|
||||
- every exec of a path under the home goes through `sudo -u mosaic -H` (from the recorded argv);
|
||||
- no argv root passes holds an activation id, and OLD comes from the unit files, not from anything
|
||||
the `mosaic` steps print;
|
||||
- every `case2-unit.sh` call carries the arguments in `case1-unit.args` and differs only in `--copy`;
|
||||
- the step log has one line per step with its exit code.
|
||||
- `scripts/stack-release.sh`: `--units external` for `adopt`, `activate`, `rollback` and `status`, the
|
||||
`--to` flag for an external rollback, and the new `confirm`, `revert` and `gate` verbs, which refuse
|
||||
without it. `scripts/stack-copy.sh`: `check --units external`, which reads the system units. Tests in
|
||||
`packages/cli/tests/stack-release-external.test.mjs` (new), against a fake system-scope `systemctl`
|
||||
and a fake `mosaic` for `bus status` and `launches list`:
|
||||
- `units-running` refuses while any unit is active, and `activate --migrate` with a unit active
|
||||
leaves the store, the log and TAKEN as they were (N4);
|
||||
- activate leaves the pointer as it was, writes `AWAITING-UNITS`, and writes a backup of the stopped
|
||||
store under `$DATA/backups/stack/<ACT>`;
|
||||
- `confirm` and `revert` refuse `open-activation` with zero `AWAITING-UNITS` files, with two, with
|
||||
one whose copy isn't the one passed, and with one whose activation isn't in the `interrupted` set;
|
||||
- `confirm` polls: a fake bus whose status turns good after 3 s passes with `--timeout 10` and fails
|
||||
with `--timeout 1`;
|
||||
- `stale-start` (N4): with `started` at 1792540800.900, an `ActiveEnterTimestamp` of `@1792540800`
|
||||
passes and `@1792540799` fails `gate_once` with `stale-start`. A fake that turns to `@1792540805`
|
||||
after 2 s passes with `--timeout 10`, which shows it's a gate code that polling can clear, not a
|
||||
refusal;
|
||||
- `confirm` on a pass writes the pointer, logs `ok` and removes `AWAITING-UNITS`. On a timeout it
|
||||
writes `FAILED` with the gate code, leaves the pointer, the log and the marker as they were, and
|
||||
exits 1;
|
||||
- `confirm` without `--rollback` refuses `activation-failed` when `FAILED` exists;
|
||||
- every line `confirm` writes has op `activate` or `rollback`, never `confirm`, and the from, to and
|
||||
activation of its `started` line, and backup equal to the rollback set's `BACKUP`;
|
||||
- `gate` (N2): with `--since 1792540800`, `@1792540800` passes and `@1792540799` fails `stale-start`;
|
||||
without `--since` it refuses `usage`;
|
||||
- `--plan --expect-from` (N5) refuses `pointer-differs` when the pointer names another copy, for
|
||||
activate and for `rollback --to`;
|
||||
- A3's order: a fake that fails the capture leaves the log with `started` and `refused code
|
||||
capture-failed` for that activation, no marker and no `FAILED`, and exit 3. An existing `$RS`
|
||||
gives `refused code marker-failed` and exit 3, and a capture under `--units external` makes no
|
||||
directory of its own. A fake that fails the
|
||||
wrapper install after a migration leaves `started` only, the marker, `FAILED`, TAKEN as migrated,
|
||||
and exit 1;
|
||||
- refusals by `confirm`, `revert` and `gate` log with no activation id, and the `interrupted` set
|
||||
is the same before and after each one;
|
||||
- the PM check passes with one live entry for the PM's instance since the start, and fails
|
||||
`pm-launch-unconfirmed` with none, with two, with one whose `startedAt` is before the start, with one
|
||||
not live, and with one for another business. A second live entry for another instance doesn't
|
||||
count. A fake `journalctl` on PATH records any call, and no external run makes one;
|
||||
- `pm-on` and `activate --pm-after-harness` refuse `external-pm`;
|
||||
- `revert` puts the wrapper back, and restores TAKEN only after a migration;
|
||||
- `confirm --rollback OLD` writes the line for each `--cause` (A3 and A4 `failed`, A5 `gate-failed`,
|
||||
with the `FAILED` code, `units-start` for A4, `unrecorded` when the file is missing), or
|
||||
`rollback-failed` when OLD's gate fails. It removes the marker and `FAILED` after the line, and
|
||||
refuses when OLD isn't the open activation's `from`;
|
||||
- real sequences on one log (B1). The fake `sudo` execs the real `stack-release.sh` as the test's
|
||||
user, so each step reads the log and state the last one wrote. Fakes stand in only for
|
||||
`systemctl`, `mosaic` and `case2-unit.sh`. Each sequence ends with the log holding exactly one
|
||||
`started` and one result line for its activation, an empty `interrupted` set, no marker or
|
||||
`FAILED`, and a following `activate --plan` that doesn't refuse `interrupted-run`:
|
||||
1. activate, confirm times out, revert, `confirm --rollback --cause A5` passes:
|
||||
`gate-failed code <c> rollback ok`;
|
||||
2. activate with `--migrate`, the wrapper install fails, revert restores TAKEN, `confirm --rollback
|
||||
--cause A3`: `failed code <c> rollback ok`;
|
||||
3. activate, root's A4 fails, revert, `confirm --rollback --cause A4`: `failed code units-start
|
||||
rollback ok`;
|
||||
4. activate whose capture fails: `refused code capture-failed` and nothing open, then A2f's
|
||||
`gate OLD` writes nothing;
|
||||
5. activate, then `confirm` with the wrong copy is refused, then revert and `confirm --rollback
|
||||
--cause A5` still close the activation, because the refusal didn't;
|
||||
6. activate, revert fails (a wrapper whose saved copy is missing): the activation stays open, and
|
||||
the next `activate --plan` refuses `interrupted-run`;
|
||||
7. adopt C0, activate C1 and confirm, activate C2 and confirm (r5's B1): both `ok` lines read
|
||||
`op activate` with their `started` line's from and to and the backup each took. `rollback --to C1
|
||||
--plan` exits 0 and `--to C0 --plan` refuses. A rollback to C1, confirmed, logs `op rollback`, and
|
||||
`from_log previous` then names C2;
|
||||
8. activate with `--migrate`, where the copy's `migrate-schema.mjs` is a fake that writes to the
|
||||
store and exits 1 (r5's B2): `FAILED` holds `migrate-failed` and `MIGRATING` exists. revert
|
||||
restores the store byte-identical to the backup, and `confirm --rollback --cause A3` gates OLD on
|
||||
it: `failed code migrate-failed rollback ok`;
|
||||
- `rollback --to` refuses when ID isn't `from_log previous`, and `rollback --restore` finds a backup
|
||||
that an external activate took;
|
||||
- `gate` writes nothing: the log, the pointer and the state dir are byte-identical before and after;
|
||||
- `confirm`, `revert`, `gate` and `--to` refuse without `--units external`.
|
||||
- dragon-lin unchanged. `stack-release.test.mjs` and `stack-copy.test.mjs` run unchanged and green. A
|
||||
new case in `stack-release-external.test.mjs` runs a default activate, rollback and check against a
|
||||
fake `systemctl` that records argv, and every call carries `--user`. A default `activate --plan`
|
||||
prints the same bytes on the candidate as on its base, with the base output as a fixture.
|
||||
- Mutants for the re-pin:
|
||||
- `confirm` writes the pointer on a gate failure;
|
||||
- `confirm` calls `gate_once` once in place of `gate`;
|
||||
- the external PM check counts entries of any instance;
|
||||
- the external PM check drops the `startedAt` bound;
|
||||
- `confirm` takes the newest `AWAITING-UNITS` instead of refusing two;
|
||||
- `rollback --to` skips the `from_log previous` comparison;
|
||||
- `case1-repin.sh` skips R1-R4;
|
||||
- `case1-repin.sh` exits 0 after A2f;
|
||||
- `case1-repin.sh` runs `mosaic`'s `stack-release.sh` without `sudo -u mosaic`;
|
||||
- the default path drops `--user` from one `systemctl` call;
|
||||
- `confirm`'s timeout writes r4's interim `gate-failed ... rollback pending` line (B1; killed by
|
||||
sequences 1 and 5);
|
||||
- A3 writes `AWAITING-UNITS` last, after the wrapper (killed by sequence 2);
|
||||
- an external verb's refusal logs with the open activation id (ACT set before the checks; killed by
|
||||
sequence 5 and the refusal test);
|
||||
- `stale-start` is a refusal again (killed by the polling case);
|
||||
- `revert` restores TAKEN without `MIGRATING` (killed by the revert case);
|
||||
- `confirm` logs with its own op, `confirm` (killed by sequence 7 and the op case);
|
||||
- A3 writes `MIGRATING` after `migrate-schema.mjs` (killed by sequence 8);
|
||||
- `gate` takes its floor from the newest `started` line when `--since` is missing (killed by the
|
||||
`gate` case).
|
||||
- `scripts/case1-pins.sh` and `scripts/case1-hostcheck.sh` (new),
|
||||
`packages/cli/systemd/mosaic-hostcheck.service` (new) and `packages/cli/tests/case1-hostcheck.test.mjs`
|
||||
(new). The script takes `--root DIR` for `/etc`, `/proc/sys` and `/usr/bin/bwrap`'s stat, and fake
|
||||
`aa-status`, `dpkg`, `dpkg-statoverride`, `apparmor_parser`, `systemd-run`, `chown` and `chmod`
|
||||
on PATH record their argv. Cases:
|
||||
- `--check` exits 0 with all seven items and 1 with each one failing alone, its line naming it;
|
||||
item 3 fails on a second file naming `bwrap` as a word, one that writes `/{usr/,}bin/bwrap` included,
|
||||
and passes with `bwrap-userns-restrict` alone. Item 4 fails on a label of `unpriv_bwrap (enforce)`
|
||||
or one with a trailing space, and its `systemd-run` argv carries `-p NoNewPrivileges=yes` and the
|
||||
other 2a' properties. Item 7 fails with the floor at 732;
|
||||
- `--enforce` exits 0 on a pass and on every failure. On a failure it unloads the profile, closes
|
||||
bwrap to `root:root` 0700, writes `DRIFT` and one log line. With `DRIFT` present and every item
|
||||
passing, it still closes bwrap;
|
||||
- `--enforce` never calls `dpkg-statoverride` with `--update`, `--add` or `--remove`;
|
||||
- a bubblewrap version other than the install pin is a printed line, not a failure;
|
||||
- `--upstream` against a fake deb whose file differs prints the difference and exits 0;
|
||||
- the service unit has `Type=oneshot`, `After=apparmor.service`,
|
||||
`Before=mosaic-bus@mosaic-stack.service` and runs `--enforce`.
|
||||
|
||||
Mutants: `--enforce` exits 1 on a drift; `--enforce` ignores `DRIFT` when the items pass; item 1
|
||||
drops the `dpkg -S` owner check; item 3 matches only the `/usr/bin/bwrap` path; item 4 matches a
|
||||
substring; item 4 drops `-p NoNewPrivileges=yes`; item 7 compares with `>=`.
|
||||
- `scripts/case1-accept.sh` (new) and `packages/cli/tests/case1-accept.test.mjs` (new), section 5.
|
||||
- Row 230 (C2-13a). `--profile`, the node, bwrap and port-floor precheck lines, the copy-id rule, the
|
||||
two units and the Console's capability lines are in its brief at 5b8def2f1. Section 2's four items
|
||||
are not. Any of them that row 230's candidate lacks becomes a named change here, reviewed by
|
||||
Duckworth with Webby told. I told Gosalyn earlier that the settings-file route for
|
||||
`DISABLE_AUTOUPDATER` doesn't reach a role session, and that Case 1 uses the managed settings file.
|
||||
- Docs: the Case 1 run sheet `docs/guides/case1-install.md` (new). It lists the window's steps in order,
|
||||
each with who runs it and its go, the pinned node sha256 and claude version, a pointer to
|
||||
`case1-pins.sh` for the bubblewrap and profile pins, the statoverride, the drift stop and the managed
|
||||
settings file, each with its rollback, how ops-01 clears a `DRIFT`, the `ssh -L` line for port 732,
|
||||
and a note that a restore leaves the credential in place.
|
||||
- No change to `~/.mosaic-dev`, `~/.config/mosaic-dev`, dragon-lin's units or `~/.mosaic`.
|
||||
@@ -1,11 +0,0 @@
|
||||
# Gosalyn
|
||||
|
||||
Worker seat for Mosaic Stack, MVP track T2 (Case 2, the creative writing hub, Vikunja #416).
|
||||
|
||||
- T3 thread: 7604438a-0ded-41c8-bd3f-26f57fdb5cd2
|
||||
- Model: Opus 5.5 (Claude Code, T3)
|
||||
- Launched by Mos (Jason's main agent, T3 38bc4c81-6270-4021-b00c-6a15c43b9b0e), who briefs the seat
|
||||
- Repository lead and queue: Sage (T3 1ef1e4f8-3ead-4208-beca-38f9f1add079)
|
||||
- Branch: gosalyn/case2-design, pushed only by this seat, landed on refactor by Sage
|
||||
|
||||
Work lives in `work/`. The Case 2 design is `work/case2/DESIGN.md`.
|
||||
@@ -1363,4 +1363,3 @@ are never rewritten or removed; corrections are new entries.
|
||||
2026-10-20T18:41:20Z | Filbert (Opus 5.5, 048f3c63) | Q175 SSO design D1/D2 check (DESIGN.md blob eba861f5d2bb at d55d86bb0), static git diff 817e8f51b..d55d86bb0 | ack: D1 and D2 match; design approved; Mos C1 sequence confirm still open; no Gitea/queue record
|
||||
2026-10-20T18:57:10Z | Filbert (Opus 5.5, 048f3c63) | row 190 design review r2 (#1688, DESIGN.md sha256 5493c9df3ec0) | CHANGES to Darkwing, cc Sage: B1 claim check on MOSAIC_CONFIG-only selection, B2 shared prefix drop-in dirs; N1-N7 notes; static read, no slot, no Gitea or queue record
|
||||
2026-10-20T19:00:04Z | Filbert (Opus 5.5, 048f3c63) | row 190 design review r3 (#1688, DESIGN.md sha256 98236a1aff98) | APPROVE to Darkwing, cc Sage: B1 and B2 closed with M13 and M14, N7 settled by Sage's 48ebee0ff; static read, no slot; manifest round gets a queue review record (sage-1337)
|
||||
2026-10-20T20:40:15Z | Filbert (Opus 5.5, 048f3c63) | row 190 (#1688) manifest round 1 review, candidate 8862feccc (static) | CHANGES: B1 base drift, row 189 ace27dc5e conflicts in stack-release.sh, rebase + full gate rerun; N1-N7 notes; Gitea 29634, queue rev 2362 (uncommitted, left for Sage)
|
||||
|
||||
@@ -1,955 +0,0 @@
|
||||
# Case 2 slice rows: the creative writing hub (#416, T2)
|
||||
|
||||
Gosalyn, 2026-10-20 (from `date -u`), for sage-1329. Every section follows `docs/plans/BRIEF-TEMPLATE.md`.
|
||||
The design they build is `agents/gosalyn/work/case2/DESIGN.md` on `gosalyn/case2-design`. Webby
|
||||
approved r5 at e1635308b in webby-0021, with notes at dba33aaf6. r6 and r7 change only sections 9 and 10
|
||||
and C2-13's row (mos-1057, Q183, webby-0023), and that diff is with Webby. Below, "the design" means that
|
||||
file, with its section number. A builder follows the design and doesn't reopen a decision in it. A
|
||||
departure goes to Sage before the build continues.
|
||||
|
||||
These are the slice rows from design 9.2, plus C2-11a, which the design starts in the first wave. The
|
||||
other rows (C2-02, C2-03b, C2-09b, C2-10c, C2-11b, C2-15 to C2-19) get briefs when a seat is free for
|
||||
them. Owners and models come from design section 8 (Q177, Q178). Where the design names only a model,
|
||||
Sage picks the seat. The reviewers are my proposals, and Sage can change any of them with `queue set`.
|
||||
Sage owns every gate.
|
||||
|
||||
Base: `refactor` at ae3c36f24. The design cites 9ae8c73ac. `git diff --stat 9ae8c73ac ae3c36f24 --
|
||||
packages adapters roles scripts` touches only `scripts/stack-release.sh` and its test, and no cited line
|
||||
is in either.
|
||||
|
||||
Order (design 8, "Parallel waves"):
|
||||
1. Start now: C2-01, C2-08, C2-11a.
|
||||
2. On C2-01's candidate: C2-04, C2-05a, C2-06, C2-12. On C2-08's: C2-09a, C2-10a, C2-20.
|
||||
3. Then C2-03a and C2-07. Then C2-05b, C2-10b and C2-13.
|
||||
4. Last: C2-14.
|
||||
|
||||
A row whose parent hasn't landed builds on the parent's candidate and rebases onto its landing (Q177).
|
||||
|
||||
Rules every row keeps:
|
||||
- No test touches `~/.mosaic-dev`, `~/.config/mosaic-dev`, a live unit, a slot file, Melanie's books
|
||||
repo, git.mosaicstack.dev or any Discord server. Tests use scratch roots, scratch config dirs and a
|
||||
local bare repo the test builds in her layout's shape: spaces and capitals in folder names, a `.canvas`
|
||||
file and an unmapped top-level note.
|
||||
- No row reads her text. Fixtures are made up. A marker string in a fixture body stands in for her
|
||||
text wherever a test checks that output holds none.
|
||||
- No credential value in any output, test, log or message (CONSTITUTION gate 13). Tests use a fixture
|
||||
token file with a made-up value.
|
||||
- No live stack change. Nothing here re-pins, restarts or activates anything. C2-13's run on the host
|
||||
needs Jason's go (design 10).
|
||||
- Before in-review, grep `packages/*/tests` and `scripts/test-*.sh` for each changed path and for any
|
||||
64-hex pin of it, and run every changed or touched test file on its own. Suites need Sage's slot.
|
||||
- A refusal code named in the design is the code the row ships. A new code goes to Sage first.
|
||||
|
||||
---
|
||||
|
||||
## C2-01 packages/library: mapping, scope, reader, resolver, R-rules, renderItems, survey
|
||||
|
||||
### Problem
|
||||
|
||||
Nothing in the stack reads a writing repo by a mapping of folders to books, types, canon and playground.
|
||||
`packages/project/src/read.mjs:19-26` reads blobs from a record repo but accepts executables (`:21`) and
|
||||
knows no scope. Design 1.1 to 1.3 and 7.4 set out what Case 2 needs, and every later row builds on it.
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: Gosalyn, Opus 5.5.
|
||||
- Reviewer: Webby, who reviewed the design.
|
||||
- After: none.
|
||||
|
||||
### Files owned
|
||||
|
||||
- `packages/library/` (new): `package.json`, `src/mapping.mjs`, `src/scope.mjs`, `src/reader.mjs`,
|
||||
`src/links.mjs`, `src/problems.mjs`, `src/render.mjs`, `src/survey.mjs`, `src/index.mjs`, `tests/*.test.mjs`,
|
||||
`tests/fixtures/`, `README.md`.
|
||||
- `scripts/stack-copy.sh`: add `packages/library/src` to `COPY_PATHS`. Without it the live copy has no
|
||||
library code, and no test catches the gap today.
|
||||
- `packages/cli/tests/stack-copy.test.mjs`: add `packages/library/src/index.mjs` to the list at `:37`.
|
||||
- `scripts/mosaic` or `packages/cli/src`: only the dispatch line for `library survey`.
|
||||
|
||||
### What ships
|
||||
|
||||
- The mapping schema of design 1.1, including `library-playground-outside-book` and the rule that no
|
||||
playground prefix holds another. Business file loading wires it in C2-04, so here it is a pure
|
||||
validator.
|
||||
- The scope rule of design 1.2. Only `.md` files are playground or canon.
|
||||
- The commit reader of design 1.3: `git ls-tree -r -z` and `cat-file` on a commit, never a working tree,
|
||||
through the same `git()` and `gitEnv()` as `packages/project`. R1 to R7 as problems with path and line.
|
||||
- The link resolver, in the order 1.3 gives: repo root, `vaultPrefix/` stripped, relative to the linking
|
||||
file, then shortest path. `appears_in` and `relationships` wikilinks are read as links.
|
||||
- `renderItems(body)` (7.4), one pure function for the item kinds and the three notices.
|
||||
- `library survey <repo> [--mapping <file>]`, which prints folder names, file counts, front matter key
|
||||
sets and link-form counts, and never text.
|
||||
|
||||
Tests: one case per R-rule and per resolver step. A link into another repo resolves to nothing. An
|
||||
`appears_in` wikilink is indexed. `renderItems` has a snapshot for each kind and each notice. The survey
|
||||
prints names, counts, key sets and link-form counts, and a marker string in a fixture body never
|
||||
appears in its output. The stack-copy test lists the new path. The library tests and
|
||||
`packages/cli/tests/stack-copy.test.mjs` pass.
|
||||
|
||||
### Out of scope
|
||||
|
||||
Business file loading and consumer skips (C2-04), sync (C2-03a), the index and queries (C2-05a), the
|
||||
guard (C2-06). The survey run on the host is design 10 step 7, after the token slot exists.
|
||||
|
||||
### Gate
|
||||
|
||||
Webby approves the manifest. The library tests and the cli suite are green on Sage's gate, along with
|
||||
every `scripts/test-*.sh`.
|
||||
|
||||
---
|
||||
|
||||
## C2-08 creative roles, persona file and block, M1, M2, M6, M10, prompt passed by file
|
||||
|
||||
### Problem
|
||||
|
||||
The stack has no role that may only read and answer in cited items, no way to carry Bean's and Chet's
|
||||
personas, and several paths by which a non-human actor could change a creative instance (design 4, B3).
|
||||
Both adapters pass the whole prompt on the command line (`adapters/claude/adapter.sh:79`, `:82`,
|
||||
`adapters/pi/adapter.sh:106`, `:121`), so any local user can read it in `/proc/<pid>/cmdline`.
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: an Opus 5.5 seat that Sage picks.
|
||||
- Reviewer: Webby.
|
||||
- After: none.
|
||||
|
||||
### Files owned
|
||||
|
||||
- `roles/companion.json`, `.md`, `roles/ideas.*`, `roles/research.*`, `roles/proofreader.*`,
|
||||
`roles/fact-checker.*`, `roles/continuity.*` (new).
|
||||
- `packages/business/src/vocabulary.mjs`: `THREAD_DEFINITIONS`, `END_PROTECTED` as `THREAD_DEFINITIONS`,
|
||||
`CREATIVE_DEFINITIONS`, `CREATIVE_KINDS`, `CREATIVE_TOOLS`.
|
||||
- `packages/business/src/role.mjs` (M1), `vars.mjs` (`persona.name`, `persona.sha256` in `REGISTRY`),
|
||||
`launch-settings.mjs` (the `instances` and `models` checks and the `setRoutes` branch),
|
||||
`instance-settings.mjs` (the creative branch).
|
||||
- `packages/bus/src/broker.mjs`: M6 `decision-not-allowed` only.
|
||||
- `packages/harness/src/bundle.mjs`: the persona file read and checks, the fenced block in `prompt()`, and
|
||||
the `webPlan` refusal.
|
||||
- `packages/harness/src/tools.mjs`: `raise_decision` left out for creative definitions.
|
||||
- `adapters/claude/adapter.sh`, `adapters/pi/adapter.sh`, `adapters/README.md` (`:89`, `:107`).
|
||||
- Tests in `packages/business/tests`, `packages/bus/tests`, `packages/harness/tests`.
|
||||
|
||||
### What ships
|
||||
|
||||
Design 4, 5.1 (M1, M2, M6, M10) and 5.3:
|
||||
- The six role files, version 2, with the tools, authority, messaging and network of the design 4 table.
|
||||
- The constants. `THREAD_DEFINITIONS` replaces `THREAD_DEFINITION` only where it means "may hold a
|
||||
thread", and that use is C2-09a's. The `assistants` lines at `launch-settings.mjs:140`, `:186` and
|
||||
`:191`, and `tools.mjs:186`, stay `THREAD_DEFINITION`.
|
||||
- The persona file at `<configDir>/personas/<business>/<instance>.md`, read with `O_NOFOLLOW`. The
|
||||
bundle refuses a non-regular file, another uid's, any group or other bit, more than 8,000 characters,
|
||||
a digest off the var (`persona-mismatch`), and `</persona-file>` or a line starting `# ` or `## `
|
||||
(`persona-invalid`). The block goes between the contract and "## This session", fenced as in design 4.
|
||||
- One actor rule on all five write paths (design 4): a write that touches a `CREATIVE_DEFINITIONS`
|
||||
instance is accepted only when its actor is the business file's `human`, otherwise
|
||||
`organization-creative-human-only`. The `instances` key of `limits.set` and `organization.set` refuses
|
||||
a creative instance for every actor with `organization-creative-instance`.
|
||||
- Both adapters pass the prompt file, not its text. Claude uses `--system-prompt-file`. Pi gets the path
|
||||
as `--system-prompt`, and its adapter checks `[ -r ]` first and refuses otherwise, because pi 0.85.1
|
||||
uses the path string itself as the prompt when the file is missing or unreadable
|
||||
(`resource-loader.js:21`, `:24-27`, `:30`).
|
||||
|
||||
Tests, all named in the design 8 C2-08 row:
|
||||
- Role, gate and messaging tests. A creative raise is refused and no decision row appears.
|
||||
`harness-builtin` is refused for `research` and `companion`. `scripts/test-config.sh` passes.
|
||||
- Persona vars are refused at other layers, when malformed and on a non-creative definition. Each
|
||||
persona file refusal above has a test. A prompt snapshot pins contract, persona, session, in that
|
||||
order.
|
||||
- M10:
|
||||
- Each path refuses with the file unchanged, and the human's `instances.set` and `routes.set` on
|
||||
`bean` work.
|
||||
- The human's `limits.set` change to Bean's `launch.models` key works, next to the chief of staff's
|
||||
`organization.set` refusal of the same change.
|
||||
- Tester cases run through the broker with a bound tester session, and chief of staff cases with a
|
||||
role session.
|
||||
- A mutant without the `models` check fails.
|
||||
- No `instances` read, `changes.jsonl` line or adapter argv holds a marker string from a fixture
|
||||
persona.
|
||||
- Adapter tests in `claude-session.test.mjs` and `pi-session.test.mjs`: argv holds the prompt path and no
|
||||
marker from the prompt. Pi's adapter refuses a missing prompt file and an unreadable one.
|
||||
- Business file, `.history` and `changes.jsonl` modes after an `instances.set` write.
|
||||
|
||||
### Out of scope
|
||||
|
||||
Thread creation per instance and the launcher (C2-09a), the lookup tools (C2-05b), the cited-reply guard
|
||||
(C2-06). Checking the host's claude for `--system-prompt-file` is C2-13's precheck. If the C2-20 dry run
|
||||
finds heading lines in a real persona, the fence rule changes in a follow-up to this row, reviewed,
|
||||
before the import (design 9.1).
|
||||
|
||||
### Gate
|
||||
|
||||
Webby approves the manifest. The business, bus and harness suites, `scripts/test-config.sh` and every
|
||||
`scripts/test-*.sh` are green on Sage's gate, and the `models` mutant is killed.
|
||||
|
||||
---
|
||||
|
||||
## C2-11a editor wikilink node, after measuring whether [[ round-trips
|
||||
|
||||
### Problem
|
||||
|
||||
The Console's editor opens a file in visual mode only if `roundTrip(text) === text`
|
||||
(`packages/webui/src/public/bus.js:919-920`). Its commonmark rules (`packages/webui/editor/model.mjs:119-120`)
|
||||
have no wikilink rule, and nobody has measured whether the serializer escapes `[[`. If it does, every
|
||||
note with a wikilink opens in Source.
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: Rocko, gpt-6.1-sol.
|
||||
- Reviewer: Filbert.
|
||||
- After: none.
|
||||
|
||||
### Files owned
|
||||
|
||||
`packages/webui/editor/model.mjs`, the editor's bundle output if the build commits one, and
|
||||
`packages/webui/tests/` editor test files.
|
||||
|
||||
### What ships
|
||||
|
||||
First a test that measures today's behaviour on `[[a]]`, `[[a|b]]`, `[[a#h]]`, in a list item, and next to
|
||||
front matter, with the result recorded in the manifest. Then a wikilink node that keeps each form byte
|
||||
for byte. `roundTrip` tests cover plain, labelled, heading, in a list, and next to front matter.
|
||||
|
||||
### Out of scope
|
||||
|
||||
Library pages and saving (C2-11b), link clicking to open a target (C2-11b), any `documentPath` change.
|
||||
|
||||
### Gate
|
||||
|
||||
Filbert approves the manifest. The webui suite is green on Sage's gate, along with every
|
||||
`scripts/test-*.sh`.
|
||||
|
||||
---
|
||||
|
||||
## C2-04 the library entry in the business file, consumer skips, M9, mirror, library.file and library.status
|
||||
|
||||
### Problem
|
||||
|
||||
The business file accepts only `root` and `repo` per project (`packages/business/src/business.mjs:186-187`),
|
||||
and several consumers assume every project has a root (design 2.1). Nothing stops an organization write
|
||||
from changing `projects`, and nothing creates a mirror.
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: Rocko, gpt-6.1-sol.
|
||||
- Reviewer: Filbert.
|
||||
- After: C2-01.
|
||||
|
||||
### Files owned
|
||||
|
||||
- `packages/business/src/business.mjs`, `organization.mjs` (M9 in `organizationWriter.write`, `:177`),
|
||||
`project.mjs`, `cli.mjs`, `resolve.mjs`, `instance-settings.mjs` (only the skip at `:142`).
|
||||
- `packages/seat/src/checkout.mjs` (`:310`, `:319`), `packages/cli/src/launcher.mjs` (`:353`).
|
||||
- `packages/project/src/paths.mjs` (the overlap loop at `:41-44`, and the mirror path).
|
||||
- `packages/bus/src/broker.mjs`: `library.file` and `library.status` only.
|
||||
- `packages/webui/src/cli.mjs`: dispatch lines for the two verbs.
|
||||
- Tests in the matching `tests/` directories.
|
||||
|
||||
### What ships
|
||||
|
||||
The `kind: "library"` entry of design 1.1, validated by C2-01's schema. Each consumer in design 2.1
|
||||
skips it, with its own test. M9: `organization-projects-protected` for any organization write whose
|
||||
`projects` differ from the file's in canonical JSON. The bare mirror is created at the `projectRepo` path
|
||||
with no remote in its config. `library.file` (read at the served commit) and `library.status` (reads the
|
||||
sync state, empty until C2-03a) are human only, refused from an agent session with `human-required`
|
||||
(M4).
|
||||
|
||||
Tests: business file tests. One test per consumer skip. A launcher test that the workspace has no clone.
|
||||
The M9 test calls `write` directly, once with a changed playground prefix and once with a changed hybrid
|
||||
entry, and the file and `changes.jsonl` are unchanged. Each verb from an agent session refuses.
|
||||
|
||||
### Out of scope
|
||||
|
||||
Fetching (C2-03a). Bus writes (C2-02). The Console's library pages (C2-11b).
|
||||
|
||||
### Gate
|
||||
|
||||
Filbert approves the manifest. The business, seat, cli, project, bus and webui suites are green on Sage's
|
||||
gate, along with every `scripts/test-*.sh`.
|
||||
|
||||
---
|
||||
|
||||
## C2-05a lookup index and queries
|
||||
|
||||
### Problem
|
||||
|
||||
Agents need typed lookups over her repo that keep universes apart and default to canon (design 3). The
|
||||
reader from C2-01 has no index or query layer.
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: a Haiku 5.5 subagent at high effort, dispatched by Sage (lead decision 129).
|
||||
- Reviewer: Filbert.
|
||||
- After: C2-01.
|
||||
|
||||
### Files owned
|
||||
|
||||
`packages/library/src/index-build.mjs`, `packages/library/src/query.mjs`, their exports in
|
||||
`packages/library/src/index.mjs`, and `packages/library/tests/` query test files.
|
||||
|
||||
### What ships
|
||||
|
||||
An index per commit (types, names, links, backlinks), cached in memory by commit id. Queries for find, get,
|
||||
search, manuscript and universe, with the arguments and result shapes in the design 3 table. `universe`
|
||||
is required. Results never leave it, except a hybrid's declared sources minus exclusions. Every result
|
||||
carries `scope`. Unmapped files never appear. `links_out` and `links_in` drop other-universe and unmapped
|
||||
targets. Results page under `REPLY_CAP` (`packages/harness/src/tools.mjs:271-280`) and are never cut.
|
||||
|
||||
Tests: isolation, scope labels on every query, unmapped files never returned, paging under `REPLY_CAP`,
|
||||
and a rebuild time on a 3,000-file fixture recorded in the manifest.
|
||||
|
||||
### Out of scope
|
||||
|
||||
Broker verbs and typed tools (C2-05b). Hybrid entries in the business file (C2-15). The query code takes
|
||||
hybrids as a shape, and its tests use a hand-built hybrid mapping.
|
||||
|
||||
### Gate
|
||||
|
||||
Filbert approves the manifest. The library suite is green on Sage's gate, along with every
|
||||
`scripts/test-*.sh`.
|
||||
|
||||
---
|
||||
|
||||
## C2-06 the cited-reply guard, the page store and the three notices
|
||||
|
||||
### Problem
|
||||
|
||||
Today a creative agent's free text would reach her as a stored message. Jason's ruling says agents never
|
||||
write prose for her and always refer to the source (Q176). Design 5.2 puts one guard in the broker before
|
||||
every agent-to-human message in a library business.
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: an Opus 5.5 seat that Sage picks.
|
||||
- Reviewer: Webby.
|
||||
- After: C2-05a.
|
||||
|
||||
### Files owned
|
||||
|
||||
- `packages/library/src/guard.mjs` and its export.
|
||||
- `packages/bus/src/broker.mjs`: the pre-check on `message.send` to `human`, and the page store beside the
|
||||
web verbs (`:1516`, `:1521`).
|
||||
- `packages/bus/src/web.mjs`, only if the page store needs the fetched text passed out.
|
||||
- Tests in `packages/library/tests` and `packages/bus/tests`.
|
||||
|
||||
### What ships
|
||||
|
||||
The guard as design 5.2 sets it out: the JSON body, the M5 limits, the checks and refusal codes of the
|
||||
5.2 table, and the three notices. It runs before the insert, outside the SQL transaction, as
|
||||
`#requestProject` does (`broker.mjs:1378-1430`). It doesn't look at the sender's definition to decide
|
||||
whether to run. The page store keeps the last 20 broker fetches per session, 200 KB each, in memory,
|
||||
dropped when the session ends.
|
||||
|
||||
Tests: one pass and one refusal for each check in M5 and 5.2, each limit refused one past its boundary
|
||||
and accepted at it. Each goes through the runner's RESULT path and through `send_message`, and once
|
||||
with a non-creative sender in a library business. The `messages` count is unchanged on each refusal.
|
||||
|
||||
### Out of scope
|
||||
|
||||
The runner retry and the runner's notices (C2-07). The Console's rendering (C2-10b). Any classifier of
|
||||
prose. The idea-text residual is Jason's or Melanie's to accept (design 5.1, 10 step 8).
|
||||
|
||||
### Gate
|
||||
|
||||
Webby approves the manifest. The library, bus and harness suites are green on Sage's gate, along with
|
||||
every `scripts/test-*.sh`.
|
||||
|
||||
---
|
||||
|
||||
## C2-12 the setup sheet for Melanie and the survey run sheet
|
||||
|
||||
### Problem
|
||||
|
||||
Melanie keeps writing in Obsidian, and she needs to know what the agents read, how a playground folder
|
||||
works, what changes for Bean and Chet, and two surprises (design 7.1, N2, N4). Whoever runs the survey
|
||||
on the host needs exact steps.
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: gpt-6-luna.
|
||||
- Reviewer: Webby.
|
||||
- After: C2-01.
|
||||
|
||||
### Files owned
|
||||
|
||||
`docs/case2/SETUP-SHEET.md` and `docs/case2/SURVEY-RUN.md` (new).
|
||||
|
||||
### What ships
|
||||
|
||||
The setup sheet in plain words:
|
||||
- Which folders the agents read, and that a folder named in `playground` is her playground.
|
||||
- How to add a graph colour group for that path in her own vault.
|
||||
- That moving a note into a book folder makes it canon, so agents may cite it as fact.
|
||||
- That a short link to a name that also exists in another of her repos can point somewhere else for the
|
||||
agents, and that a full-path link avoids this.
|
||||
- What changes for Bean and Chet: short cited items, no passages (design 4).
|
||||
|
||||
The survey run sheet gives the exact `library survey` command with C2-01's real flags, and states what
|
||||
the output holds: names and counts only.
|
||||
|
||||
### Out of scope
|
||||
|
||||
Sending it to her. Design 10 step 7 makes that Jason's call.
|
||||
|
||||
### Gate
|
||||
|
||||
Doc only. Webby approves it against C2-01's real flags. Before landing, grep `packages/*/tests` and
|
||||
`scripts/test-*.sh` to confirm no test reads either file.
|
||||
|
||||
---
|
||||
|
||||
## C2-09a one thread per creative instance
|
||||
|
||||
### Problem
|
||||
|
||||
`thread.create` picks the single `assistant` instance and refuses more than one
|
||||
(`packages/bus/src/broker.mjs:2331-2334`). The same rule sits in `#threaded` (`:290-291`) and the launcher
|
||||
(`packages/cli/src/launcher.mjs:135`, `:236-238`). So she can't open a thread with Bean beside one with
|
||||
Chet.
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: Rocko, gpt-6.1-sol.
|
||||
- Reviewer: Dewey.
|
||||
- After: C2-08.
|
||||
|
||||
### Files owned
|
||||
|
||||
`packages/bus/src/broker.mjs` (`#threaded`, `thread.create`), `packages/cli/src/launcher.mjs`,
|
||||
`packages/webui/src/public/bus.js` and `serve.mjs` (the new-thread control only), and tests in
|
||||
`packages/bus/tests`, `packages/cli/tests` and `packages/webui/tests`.
|
||||
|
||||
### What ships
|
||||
|
||||
Design 6, C2-09a: `#threaded` and the launcher read `THREAD_DEFINITIONS`. `thread.create` takes
|
||||
`instance`, which must be a `THREAD_DEFINITIONS` instance. With no `instance` and exactly one such
|
||||
instance, today's behaviour holds. With more than one, it refuses `thread-instance-ambiguous`. The
|
||||
Console's new-thread control lists thread instances by `persona.name` or instance name and passes the
|
||||
one she picks. No schema change.
|
||||
|
||||
Tests: a creative instance's thread starts a run, `thread.create` with `instance` binds it, and two thread
|
||||
instances with no `instance` still refuse.
|
||||
|
||||
### Out of scope
|
||||
|
||||
Rooms and @tags (C2-09b, C2-10c).
|
||||
|
||||
### Gate
|
||||
|
||||
Dewey approves the manifest. The bus, cli and webui suites are green on Sage's gate, along with every
|
||||
`scripts/test-*.sh`. Watch for hash pins on `bus.js` and `broker.mjs`
|
||||
(`packages/bus/tests/channel-proof.test.mjs` pins a webui file).
|
||||
|
||||
---
|
||||
|
||||
## C2-10a no raw session view (M7) and no raw Discord engine (M8)
|
||||
|
||||
### Problem
|
||||
|
||||
The Console's conversation view shows a model's raw text from session files
|
||||
(`packages/conversation/src/reader.mjs:225-240`, `events.mjs:58`, `:82`), so a refused answer would still
|
||||
show. The Discord connector posts its engine's output and the broker never sees it
|
||||
(`packages/discord/src/connector.mjs:381-387`).
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: a Haiku 5.5 subagent at high effort, dispatched by Sage.
|
||||
- Reviewer: Filbert.
|
||||
- After: C2-08.
|
||||
|
||||
### Files owned
|
||||
|
||||
`packages/conversation/src/reader.mjs`, `packages/discord/src/cli.mjs` (`check` and `run` only), and
|
||||
tests in `packages/conversation/tests` and `packages/discord/tests`.
|
||||
|
||||
### What ships
|
||||
|
||||
M7 and M8 as design 5.1 states them. On a business with a library project, `catalogue()` leaves creative
|
||||
sessions out and `open()` refuses them with `conversation-not-listed`. `discord check` and `discord run`
|
||||
refuse a binding whose engine isn't `mosaic-bus` with `discord-engine-not-allowed`, and also refuse a
|
||||
`mosaic-bus` binding with `tools`.
|
||||
|
||||
Tests: a creative session file under the root isn't listed and won't open by id, and an assistant session
|
||||
still lists. Both Discord verbs refuse on a library business, and a pi binding on a business without a
|
||||
library still checks clean.
|
||||
|
||||
### Out of scope
|
||||
|
||||
The relay engine itself (C2-18).
|
||||
|
||||
### Gate
|
||||
|
||||
Filbert approves the manifest. The conversation and discord suites and `scripts/test-discord.sh` are
|
||||
green on Sage's gate, along with every `scripts/test-*.sh`. No test reaches a Discord server.
|
||||
|
||||
---
|
||||
|
||||
## C2-20 Bean and Chet persona import, with a count-only dry run
|
||||
|
||||
### Problem
|
||||
|
||||
Bean's and Chet's personas live in her two Hermes profiles on 10.1.1.54. The hub needs each one as a
|
||||
0600 persona file with its digest in the business file (design 4), and nothing should read her memory,
|
||||
sessions or messages to get them (design 9.1).
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: Gosalyn, Opus 5.5. The script runs only under Jason's go, by whoever he names.
|
||||
- Reviewer: Webby.
|
||||
- After: C2-08.
|
||||
|
||||
### Files owned
|
||||
|
||||
`scripts/case2-persona-import.mjs` (new), `packages/cli/tests/case2-persona-import.test.mjs` (new) and its
|
||||
fixture tree.
|
||||
|
||||
### What ships
|
||||
|
||||
A script that reads only the persona definition of each of the two Hermes profiles. It confirms profile
|
||||
to name from each profile's own configured name, prints the pairing, and stops if a profile names
|
||||
neither Bean nor Chet. Per-channel prompts are listed by name and size only. It writes each persona to
|
||||
a 0600 staging file. A second step, run as `<user>`, installs it at
|
||||
`<configDir>/personas/<business>/<instance>.md` and prints `persona.name` and `persona.sha256`.
|
||||
Otherwise it prints names, byte counts and sha256 only. `--dry-run` writes nothing and prints only counts
|
||||
of files, lines, and lines starting `# ` or `## `.
|
||||
|
||||
Build note (gosalyn-0030, mos-1087). The dry run also counts lines that name a Hermes tool or channel,
|
||||
so Jason knows before the import whether a persona refers to things Bean and Chet won't have in
|
||||
Mosaic. The words come from a fixed list in the script: the tool names in the hermes-agent checkout at
|
||||
df5700ebe, read from its code and not her profiles, plus `Hermes` and `Discord`. It prints one count
|
||||
per persona file and never the line, its number or the matched word. A nonzero count changes nothing
|
||||
in the import. It goes to Jason with the other counts.
|
||||
|
||||
Tests, on fixture Hermes profile trees:
|
||||
- The right fields are taken.
|
||||
- Memory and session files are never opened, shown by a read trace on the fixture.
|
||||
- A profile with no persona field refuses, and a persona that fails C2-08's fence check refuses.
|
||||
- Output holds no persona text. Each written file is 0600 and its digest matches.
|
||||
- `--dry-run` writes nothing, and prints the right counts and no marker string for a fixture with two
|
||||
heading lines.
|
||||
- `--dry-run` counts three lines that name a listed Hermes tool or channel in a fixture. Its output
|
||||
holds none of the matched words.
|
||||
|
||||
### Out of scope
|
||||
|
||||
Running it. Design 10 step 6 and 9.1 make the dry run, the import and the pairing check Jason's go. If
|
||||
the dry run counts heading lines, the C2-08 follow-up changes the fence rule before the import, and her
|
||||
text is rewritten only if she says so through Jason.
|
||||
|
||||
### Gate
|
||||
|
||||
Webby approves the manifest. The test file runs green on its own and the cli suite is green on Sage's
|
||||
gate, along with every `scripts/test-*.sh`.
|
||||
|
||||
---
|
||||
|
||||
## C2-03a fetch-only sync
|
||||
|
||||
### Problem
|
||||
|
||||
The mirror from C2-04 holds nothing until something fetches her repo, safely and read-only, without the
|
||||
token ever reaching argv, config, logs or replies (design 7.2, CONSTITUTION gate 13).
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: an Opus 5.5 seat that Sage picks.
|
||||
- Reviewer: Dewey.
|
||||
- After: C2-04.
|
||||
|
||||
### Files owned
|
||||
|
||||
`packages/library/src/sync.mjs`, `packages/library/bin/askpass` (new), `packages/bus/src/broker.mjs`
|
||||
(`library.sync`, the 60-second timer and the library lock), `packages/webui/src/cli.mjs` (dispatch line),
|
||||
and tests in `packages/library/tests` and `packages/bus/tests`. If the helper sits outside
|
||||
`packages/library/src`, add its path to `COPY_PATHS` in `scripts/stack-copy.sh`.
|
||||
|
||||
### What ships
|
||||
|
||||
Design 7.2's slice:
|
||||
- `syncCommand(kind, url)`: its argv is `SAFE`, exactly one `-c protocol.https.allow=always`, `-C
|
||||
<mirror>`, then the command. Its env is `gitEnv()` plus `GIT_ASKPASS` naming a helper outside the
|
||||
mirror, which reads the `library-gitea` slot.
|
||||
- An `https://` URL only.
|
||||
- The served ref `refs/heads/served`, and `library-diverged`, held until a human `library.sync --accept
|
||||
<commit>`.
|
||||
- The sync state file, written atomically inside the mirror, and the 60-second timer.
|
||||
- `library.sync`, human only. A test reaches a local bare repo only through the module parameter that
|
||||
replaces `syncCommand`.
|
||||
|
||||
Tests: the C2-03a tests in design 7.2. A mutant that sets `protocol.allow=always` fails the argv test.
|
||||
|
||||
### Out of scope
|
||||
|
||||
Push and every bus write (C2-03b, C2-02). Creating the token slot, which is design 10 step 4.
|
||||
|
||||
### Gate
|
||||
|
||||
Dewey approves the manifest. The library and bus suites are green on Sage's gate, along with every
|
||||
`scripts/test-*.sh`, and the mutant is killed. A search of the test run's output for the fixture token
|
||||
finds nothing.
|
||||
|
||||
---
|
||||
|
||||
## C2-07 runner retry on a guard refusal, and the runner's notices
|
||||
|
||||
### Problem
|
||||
|
||||
When `message.send` refuses, the runner logs it and marks the message read, so the answer is lost
|
||||
(`packages/harness/src/runner.mjs:451-456`). Its empty and failed turn texts (`:449`) wouldn't pass the
|
||||
guard.
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: a Haiku 5.5 subagent at high effort, dispatched by Sage.
|
||||
- Reviewer: Filbert.
|
||||
- After: C2-06.
|
||||
|
||||
### Files owned
|
||||
|
||||
`packages/harness/src/runner.mjs` and `packages/harness/tests/runner.test.mjs`.
|
||||
|
||||
### What ships
|
||||
|
||||
On a refusal with a guard code, a new turn in the same session with the code and reason, at most twice,
|
||||
then `{"items": [], "notice": "no-cited-answer"}`. In a library business, `no-answer` and `turn-failed`
|
||||
replace the two texts at `:449`. The failure reason stays in the run log.
|
||||
|
||||
Tests: a fake harness answers free text, then a valid reply, and exactly one message is stored. An empty
|
||||
turn and a failed turn store the fixed notices.
|
||||
|
||||
### Out of scope
|
||||
|
||||
The guard itself (C2-06).
|
||||
|
||||
### Gate
|
||||
|
||||
Filbert approves the manifest. The harness suite is green on Sage's gate, along with every
|
||||
`scripts/test-*.sh`.
|
||||
|
||||
---
|
||||
|
||||
## C2-05b broker lookup verbs and typed tools
|
||||
|
||||
### Problem
|
||||
|
||||
The queries from C2-05a aren't reachable from an agent session. Design 3 and M3 call for typed tools
|
||||
that the broker re-checks on every call.
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: Rocko, gpt-6.1-sol.
|
||||
- Reviewer: Dewey.
|
||||
- After: C2-05a, C2-03a.
|
||||
|
||||
### Files owned
|
||||
|
||||
`packages/bus/src/broker.mjs` (the lookup verbs), `packages/harness/src/tools.mjs` (the five typed tools),
|
||||
and tests in `packages/bus/tests` and `packages/harness/tests`.
|
||||
|
||||
### What ships
|
||||
|
||||
The five lookup tools of design 3, for every agent session in a business with a library project. Each
|
||||
read is at the served commit, and a read whose last fetch is older than 60 seconds syncs first. The
|
||||
verbs refuse `library-not-configured` in a business without a library and refuse a missing `universe`.
|
||||
The creative tool set is exact: the lookups and `send_message`, with no write tool and no
|
||||
`raise_decision`.
|
||||
|
||||
Tests: broker verb tests and a snapshot of typed tool names per creative role (M3).
|
||||
|
||||
### Out of scope
|
||||
|
||||
The guard (C2-06). Write verbs (C2-02, C2-19).
|
||||
|
||||
### Gate
|
||||
|
||||
Dewey approves the manifest. The bus and harness suites are green on Sage's gate, along with every
|
||||
`scripts/test-*.sh`.
|
||||
|
||||
---
|
||||
|
||||
## C2-10b Console chat: cited items and the status line
|
||||
|
||||
### Problem
|
||||
|
||||
The Console would show a cited reply as raw JSON, and she can't see when the hub falls behind her repo
|
||||
(design 7.4).
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: a Haiku 5.5 subagent at high effort, dispatched by Sage.
|
||||
- Reviewer: Dewey.
|
||||
- After: C2-06, C2-09a, C2-03a.
|
||||
|
||||
### Files owned
|
||||
|
||||
`packages/webui/src/public/bus.js`, its stylesheet, `packages/webui/src/serve.mjs` (a `library.status`
|
||||
route only), and tests in `packages/webui/tests`.
|
||||
|
||||
### What ships
|
||||
|
||||
Design 7.4. The chat view renders a cited reply through `renderItems`: each item's kind, text and cites,
|
||||
with a scope badge (canon, playground or web) and "changed since" when the file's last commit is newer
|
||||
than the cited one. Each notice shows its fixed line. A status line reads `library.status`: the served
|
||||
commit and its age, `library-diverged`, a fetch error, and the count of R-problems with a link to the
|
||||
list.
|
||||
|
||||
Tests: webui tests for items, scope badges, notices, "changed since" and the status line.
|
||||
|
||||
### Out of scope
|
||||
|
||||
The room view (C2-10c), library pages (C2-11b). Her sign-in from her machine waits on SSO (#412) or
|
||||
Jason's interim ruling (design 10 step 5).
|
||||
|
||||
### Gate
|
||||
|
||||
Dewey approves the manifest. The webui suite is green on Sage's gate, along with every
|
||||
`scripts/test-*.sh`. Grep for hash pins on `bus.js` and `serve.mjs` before in-review.
|
||||
|
||||
---
|
||||
|
||||
## C2-13 install script and precheck for the host
|
||||
|
||||
### Problem
|
||||
|
||||
The hub runs on 10.1.1.54 as `mosaic` under `/srv/mosaic`, as `mosaic-stack.service` with
|
||||
`ProtectHome` (design 9, Q181). Nothing installs a library business there, or checks the host before it
|
||||
starts.
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: Gosalyn, Opus 5.5. The script runs only under Jason's go, by him or his delegate (design 10).
|
||||
- Reviewer: Webby.
|
||||
- After: C2-04, C2-03a, C2-20.
|
||||
|
||||
### Files owned
|
||||
|
||||
`scripts/case2-install.sh`, `scripts/case2-precheck.sh` and `scripts/case2-unit.sh` (new), the template
|
||||
`packages/cli/systemd/mosaic-stack.service.in` (new), `packages/cli/tests/case2-install.test.mjs`
|
||||
(new), and one line of `packages/cli/README.md`: at `:713`, "nothing in the repository creates or reads
|
||||
it" changes to say that nothing creates it, that only the Case 2 precheck reads it, and that the
|
||||
precheck prints only variable names, never values (webby-0034 N2, webby-0035).
|
||||
|
||||
### What ships
|
||||
|
||||
The install does: bootstrap, release package and activate, the business file with the mapping, `bean`
|
||||
and `chet` with their persona vars and files, any specialists she picks, her human id, the token slot
|
||||
path. It runs as `<user>` through `sudo -u mosaic -H` from `localadmin`, and `mosaic` gets no key. It
|
||||
doesn't create `<user>` and writes no unit file. Root does those in design 10 steps 2a and 2c. Every
|
||||
host value is an input, so a later host move changes only inputs.
|
||||
|
||||
Row 195 runs these scripts by input for Case 1 on 10.1.1.53 (darkwing to gosalyn, design at
|
||||
`agents/darkwing/work/queue-195/DESIGN.md` section 2). They don't copy them. So the install and the
|
||||
precheck take `--profile factory|library`, default `library`:
|
||||
- `library` is everything this brief lists.
|
||||
- `factory` skips the library mapping, the persona files and digests, the token slot, and
|
||||
`library.sync` and `library.status`. It requires `bus.tester` on, because Case 1's acceptance uses the
|
||||
tester.
|
||||
- `factory` refuses a business file that has a `kind: "library"` project, with
|
||||
`profile-library-business`. So Case 2 can't reach a tester-on check through a wrong flag.
|
||||
- `library` refuses with `bus.tester` on, as before.
|
||||
|
||||
`case2-unit.sh` is the 2c step. Root runs it from a root-owned checkout of the reviewed commit outside
|
||||
`/srv/mosaic`, never from the release, because `mosaic` can write the release. It does the following:
|
||||
- It renders the unit from the checkout's template, with the user, the home and the copy id as inputs.
|
||||
- It refuses unless `User=`, `Group=`, `ExecStart=`, `EnvironmentFile=` and the hardening lines match
|
||||
fixed values. `EnvironmentFile=` is the line in `packages/cli/systemd/mosaic-bus.service.in:24`. The
|
||||
copy id is the one part of `ExecStart=` that varies, so a re-pin to a new copy can re-render the
|
||||
unit (row 195). It must be exactly 40 lowercase hex characters naming a copy that holds `scripts/mosaic`,
|
||||
directly under `<home>/.local/share/mosaic-stack/copies`. That is the rule `copy_root` uses
|
||||
(`scripts/stack-copy.sh:6`, `:20`). Anything else refuses before the render.
|
||||
- It refuses any `PermissionsStartOnly=`, and any `Exec*=` value whose leading run of prefix
|
||||
characters (`@`, `-`, `:`, `+`, `!`) holds a `+` or `!`. systemd takes those prefixes in any order,
|
||||
so `-+/srv/mosaic/x` runs as root even with `User=mosaic`. The named-line check doesn't read these
|
||||
lines (webby-0026 N1, webby-0027, defence in depth).
|
||||
- It writes `/etc/systemd/system/mosaic-stack.service` as `root:root` 0644. It never uses
|
||||
`systemctl link`, and it refuses a symlink at the unit path.
|
||||
- On a re-install, it refuses an installed unit that differs from the render.
|
||||
- It reads nothing from under `<user>`'s home.
|
||||
|
||||
The precheck refuses unless all of these hold:
|
||||
- The config validates.
|
||||
- `sudo -n true` fails as `<user>`, and `id -G` prints exactly one entry, its primary group. A system
|
||||
user always has one, so the check is for supplementary groups. `bus.tester` is not on.
|
||||
- The business file, `.history`, `changes.jsonl` and persona files have the modes in design 4, and
|
||||
each persona digest matches.
|
||||
- The run root is 0700.
|
||||
- `XDG_DATA_HOME` is unset or is `<home>/.local/share`. `copy_root` reads the copies dir from it
|
||||
(`scripts/stack-copy.sh:6`), and the unit fixes `<home>/.local/share`, so any other value would
|
||||
put the copy where the unit refuses it (webby-0032 N2). A test sets it elsewhere and the precheck
|
||||
refuses.
|
||||
- `node --version` is 24 or later, because every package's `engines` field says `>=24` (row 195: .53
|
||||
has v22.22.2).
|
||||
- `bwrap --unshare-all --ro-bind / / true` exits 0 as `<user>`. Without it no session starts. Row 195
|
||||
found no bwrap on .53, and `kernel.apparmor_restrict_unprivileged_userns=1`, and .54 may be the same.
|
||||
On a refusal, the precheck prints the sysctl's value and whether `bwrap` is on PATH.
|
||||
- When some creative instance's harness is claude-code, one marker prompt run through the host's claude
|
||||
with `--system-prompt-file` shows that it replaces the default prompt. The claude version is printed.
|
||||
With design r9's model path, Bean and Chet run pi, so this check is skipped unless she adds a
|
||||
claude-code specialist.
|
||||
- When an instance's provider is `zai`, `<configDir>/secrets/claude-session.env` exists, is a regular
|
||||
file owned by `<user>` at 0600, and sets a nonempty `ZAI_API_KEY` (design 10 step 2d). It parses the
|
||||
file the way systemd does, so the last assignment of a name wins (webby-0032 N3). A pi launch
|
||||
has no credential check, so without this the first turn is where a missing key shows. The precheck
|
||||
prints the name and never the value.
|
||||
- `library.sync` fetches her repo read-only, and `library.status` lists R-problems.
|
||||
- No Discord binding exists.
|
||||
|
||||
It also prints free memory and verifies HOME is `/srv/mosaic`.
|
||||
|
||||
Tests run against a scratch data root and a local bare repo. Each precheck refusal has a case. The
|
||||
claude check runs against a fake claude that appends rather than replaces, and that case refuses. With
|
||||
no claude-code instance the claude check doesn't run, and a test shows a missing claude passes. The key
|
||||
file refuses when missing, at 0644, a symlink, with `ZAI_API_KEY=` empty, and with `ZAI_API_KEY=x`
|
||||
followed by `ZAI_API_KEY=`. A search of the test
|
||||
output for the fixture key value finds nothing.
|
||||
`case2-unit.sh` runs against a scratch unit directory. These cases refuse:
|
||||
- an installed unit edited to `User=root`
|
||||
- an edited `ExecStart=`
|
||||
- a template with a hardening line removed
|
||||
- a template with the `EnvironmentFile=` line removed or pointing elsewhere
|
||||
- a symlink at the unit path
|
||||
- a template with `ExecStartPre=+/srv/mosaic/x`
|
||||
- a template with `ExecStartPre=-+/srv/mosaic/x`
|
||||
- a copy id that is short, uppercase, holds `/` or `..`, or names no directory
|
||||
|
||||
Precheck cases for the new lines: a fake `node` printing v22 refuses, and a fake `bwrap` that exits 1
|
||||
refuses. `--profile factory` passes with the tester on and no library items, and refuses a business
|
||||
file with a library project. `--profile library` refuses with the tester on.
|
||||
A clean render writes a 0644 file. A test checks the script for `systemctl link`.
|
||||
|
||||
### Out of scope
|
||||
|
||||
Running it on the host. Design 10 steps 2 to 7 are Jason's. Step 1 no longer waits on #421 (Q183), and
|
||||
Mos lifted its hold (mos-1074). The root steps 2a and 2c are ops-01's, over ssh as `localadmin`,
|
||||
so the install waits on #423 step 1 and on nothing else in #423. The Gitea bot account and the token slot are steps 3 and 4. Stopping Hermes
|
||||
is step 9 and her choice.
|
||||
|
||||
### Gate
|
||||
|
||||
Webby approves the manifest. The test file and the cli suite are green on Sage's gate, along with every
|
||||
`scripts/test-*.sh`. The run on the host is a separate window with Jason's go.
|
||||
|
||||
---
|
||||
|
||||
## C2-14 end-to-end acceptance on a scratch root
|
||||
|
||||
### Problem
|
||||
|
||||
Each slice row tests its own part. Nothing yet shows the whole path from her push to a checked reply in
|
||||
the Console.
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: an Opus 5.5 seat that Sage picks.
|
||||
- Reviewer: Dewey, who also runs the script.
|
||||
- After: every other slice row above.
|
||||
|
||||
### Files owned
|
||||
|
||||
`scripts/test-case2.sh` (new) and its fixtures.
|
||||
|
||||
### What ships
|
||||
|
||||
One script on a scratch data root and a bare repo in her layout. It syncs, asks a creative agent in its
|
||||
thread through a fake harness, and stores a reply whose cites verify. It refuses a playground cite on a
|
||||
`fact`, holds a force-push as `library-diverged`, and checks that the Console renders the reply as
|
||||
items.
|
||||
|
||||
### Out of scope
|
||||
|
||||
Any live host or real model call.
|
||||
|
||||
### Gate
|
||||
|
||||
Dewey runs the script and approves. It is green on Sage's gate with every `scripts/test-*.sh`.
|
||||
|
||||
---
|
||||
|
||||
## Brief: row A, ZAI_API_KEY in the session env allowlist
|
||||
|
||||
### Problem
|
||||
|
||||
For 10-27, Bean and Chet both run pi with provider `zai` on `glm-5.3` (Q186, mos-1090, sage-1469). Pi
|
||||
reads the zai key from `ZAI_API_KEY` (pi-ai `env-api-keys.js:89`, with `glm-5.3` in `dist/providers/data/zai.json`).
|
||||
`ENV_ALLOW` (`packages/seat/src/session.mjs:83-88`) doesn't hold that name, so `sessionEnv` (`:97-99`)
|
||||
drops it. A pi launch has no credential check (`packages/plugins/harness/pi/index.mjs:42`
|
||||
`countsAccount: false`, `packages/cli/src/launcher.mjs:299`), so today a zai role launches and fails at
|
||||
its first turn, as `packages/cli/README.md:439-444` says.
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Owner: a Haiku 5.5 subagent at high effort, dispatched by Sage (lead decision 129). Size XS.
|
||||
- Reviewer: Webby.
|
||||
- After: nothing. It doesn't wait on any Case 2 row.
|
||||
|
||||
### Files owned
|
||||
|
||||
`packages/seat/src/session.mjs`, `packages/seat/tests/session.test.mjs` and `packages/cli/README.md`.
|
||||
|
||||
### What ships
|
||||
|
||||
- `"ZAI_API_KEY"` joins the provider line of `ENV_ALLOW` (`session.mjs:86`), after `OPENROUTER_API_KEY`.
|
||||
Nothing else in the list changes. `SESSION_CREDENTIALS` and `hasSessionCredential` (`:94-95`) don't
|
||||
change. They are Claude Code's login sets, and a zai key isn't one.
|
||||
- `packages/cli/README.md`, after "A Pi session gets the same variables through the allowlist." (`:435`),
|
||||
one sentence: a Pi session on provider `zai` logs in with `ZAI_API_KEY`, which the allowlist passes. Like
|
||||
every allowlist variable, it reaches every session the host starts, Claude Code ones included, and a
|
||||
shell inside any of them can read it. Where the README names `claude-session.env` ("Unit", `:711`),
|
||||
"The file holds one line" becomes "The file holds one line per variable", followed by a sentence
|
||||
saying it holds every session provider key, not only Claude's (webby-0032, webby-0034 N1). Leave
|
||||
`:713`, "nothing in the repository creates or reads it", alone. C2-13 owns that line.
|
||||
- One test in `session.test.mjs`: `ENV_ALLOW` includes `ZAI_API_KEY`, and
|
||||
`sessionEnv({ ZAI_API_KEY: "fixture-zai", ZAI_TOKEN: "t" }, run)` returns `ZAI_API_KEY` with the fixture
|
||||
value and no `ZAI_TOKEN`. The existing allowlist test (`:51-59`) stays as it is.
|
||||
- One mutant: the same file with `"ZAI_API_KEY"` removed from `ENV_ALLOW`. The new test fails on it.
|
||||
|
||||
### Out of scope
|
||||
|
||||
Placing a key anywhere. The hub's own key on the Z.ai account is design 10 step 2d, and it is Jason's.
|
||||
Any pi launch or live check: running a pi role waits on #383. Chet's Codex login (row B, #428). A
|
||||
credential check for pi launches.
|
||||
|
||||
### Gate
|
||||
|
||||
Webby approves the manifest. The mutant is killed. The seat suite and the cli suite are green on
|
||||
Sage's gate with every `scripts/test-*.sh`.
|
||||
|
||||
---
|
||||
|
||||
## Brief: row B, a per-instance pi agent dir for Chet's Codex login (#428)
|
||||
|
||||
### Problem
|
||||
|
||||
After 10-27, Chet moves from Bean's Z.ai account to his own ChatGPT/Codex login (Q186). Pi's
|
||||
`openai-codex` provider logs in with OAuth and a refresh token, and keeps both in `auth.json` under
|
||||
`PI_CODING_AGENT_DIR` (pi `config.js:405-438`). A role session's `HOME` is the run's own, so pi never
|
||||
sees an agent dir from the host (`packages/cli/README.md:439-441`), and `ENV_ALLOW` doesn't pass
|
||||
`PI_CODING_AGENT_DIR`. So no env variable can carry this login. Each instance needs its own agent dir
|
||||
that survives runs, because a refresh rewrites the file.
|
||||
|
||||
Chet's Hermes profile uses the same login today. Until Hermes stops, two clients would hold one
|
||||
refresh token. If a refresh by one revokes the other's, one of them drops out mid-session.
|
||||
|
||||
### Owner and reviewer
|
||||
|
||||
- Design: Gosalyn owns it (sage-1469). The build owner and reviewer are named when the design is
|
||||
approved.
|
||||
- Starts after 10-27. Nothing in this row is built or tested before then.
|
||||
|
||||
### Files owned
|
||||
|
||||
None yet. The design names them.
|
||||
|
||||
### What ships
|
||||
|
||||
The design first. It settles where the dir lives and who can read it, how the launcher mounts it into
|
||||
the sandbox writable for one instance only, how a refresh write survives the run, what a shell in the
|
||||
session can read, and how a launch with no login refuses. Build rows follow from it.
|
||||
|
||||
### Out of scope
|
||||
|
||||
Bean's model path. Any change to row A. Any change to Hermes or her server.
|
||||
|
||||
### Gate
|
||||
|
||||
A live check, run with Jason's or Mos's go, showing that two clients on one ChatGPT login don't revoke
|
||||
each other's refresh tokens. Until that check passes, no build row for this design starts.
|
||||
@@ -34,7 +34,7 @@ export function setup(t, realPin=false){
|
||||
test('shared copy: npm ci, pushed reachability, readonly/existing copy, webui delegation',t=>{
|
||||
const s=setup(t),r=s.run('stack-copy.sh',['copy',s.id,'--copies',s.copies]);assert.equal(r.status,0,r.stderr);
|
||||
const copy=join(s.copies,s.id);assert.equal(statSync(copy).mode&0o777,0o555);
|
||||
for(const f of ['scripts/mosaic','adapters/pi/adapter.sh','packages/queue/src/cli.mjs','package.json','package-lock.json','packages/tasks/requirements.json','packages/project/src/index.mjs','packages/library/src/index.mjs','packages/project/schemas/project.schema.json','packages/project/templates/north-star.md','packages/plugins/src/kinds.mjs','packages/plugins/src/loader.mjs'])assert.ok(existsSync(join(copy,f)),f);
|
||||
for(const f of ['scripts/mosaic','adapters/pi/adapter.sh','packages/queue/src/cli.mjs','package.json','package-lock.json','packages/tasks/requirements.json','packages/project/src/index.mjs','packages/project/schemas/project.schema.json','packages/project/templates/north-star.md','packages/plugins/src/kinds.mjs','packages/plugins/src/loader.mjs'])assert.ok(existsSync(join(copy,f)),f);
|
||||
const npm=join(s.bin,'npm');writeFileSync(npm,'#!/bin/sh\nexit 99\n');chmodSync(npm,0o755);
|
||||
const existing=s.run('webui-service.sh',['copy',s.id,'--copies',s.copies]);assert.equal(existing.status,0);assert.equal(existing.stdout,`exists: ${copy}\n`);unlinkSync(npm);
|
||||
git(s.repo,'update-ref','refs/remotes/origin/refactor',git(s.repo,'hash-object','-w','package.json'));
|
||||
|
||||
@@ -1,135 +0,0 @@
|
||||
# packages/library
|
||||
|
||||
Reads a writing repo by a mapping of folders to books, types, canon and playground. This is Case 2
|
||||
row C2-01 (`docs/plans/2026-10-20_case2-rows.md`), built from design sections 1.1 to 1.3 and 7.4 in
|
||||
`agents/gosalyn/work/case2/DESIGN.md` on `gosalyn/case2-design`.
|
||||
It is pure code with no state. Business file loading (C2-04), sync (C2-03a), the index (C2-05a) and the
|
||||
guard (C2-06) build on it.
|
||||
|
||||
## Mapping
|
||||
|
||||
`checkMapping(entry, where, { fail })` checks a `kind: "library"` project entry and returns a
|
||||
deep-frozen copy. `where` names the entry in messages. `fail(code, message)` defaults to throwing a
|
||||
`LibraryError`. C2-04 passes its own so a business file refusal carries the file name.
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": "library",
|
||||
"remote": "https://git.example.invalid/her/books.git",
|
||||
"branch": "main",
|
||||
"vaultPrefix": "books",
|
||||
"types": { "Characters": "character", "Places": "place", "Research": "research" },
|
||||
"manuscript": ["Chapters"],
|
||||
"universes": {
|
||||
"harbor": { "name": "Harbor", "books": ["Book One"], "playground": ["Book One/Playground/"] }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- `remote` is https with no credentials, query or fragment. `branch` is a plain ref name.
|
||||
- `vaultPrefix` is optional, one folder name.
|
||||
- `types` maps a folder name to `character`, `place`, `research`, `tracker` or `summary`.
|
||||
`manuscript` lists folder names and shares none with `types`.
|
||||
- A universe has `name`, `books` and `playground`. A hybrid adds `kind: "hybrid"`, two or more
|
||||
non-hybrid `sources`, `constraints` (each one `{ "exclude": ... }` or
|
||||
`{ "pin": { "entity", "field", "value" } }`) and `notes`.
|
||||
|
||||
Refusals:
|
||||
|
||||
| Code | When |
|
||||
|---|---|
|
||||
| `library-playground-outside-book` | a playground prefix is not strictly inside a book of its own universe |
|
||||
| `library-mapping-invalid` | any other shape fault: unknown key, bad remote or branch, a book folder in two universes, a book folder that holds another, a playground prefix that holds another, an unknown hybrid source |
|
||||
|
||||
## Scope
|
||||
|
||||
`classify(mapping, path)` decides scope from the path alone. Only paths ending in lowercase `.md` are
|
||||
mapped. A path under a playground prefix is `playground`. Any other path under a book folder is
|
||||
`canon`. Everything else is `unmapped`. The type is the first folder after the book, or after the
|
||||
playground prefix: a `types` name, `manuscript`, or `null`.
|
||||
|
||||
## Reader
|
||||
|
||||
`readLibrary({ repo, commit, mapping })` reads one commit, never a working tree. `repo` may be a
|
||||
clone or a bare mirror. `commit` must be a full 40-hex id that names a commit, else `library-commit`.
|
||||
Git runs through `git()` and `gitBytes()` from `packages/project/src/git.mjs`, so the same safe config,
|
||||
environment and ceiling directory apply. A failed git call refuses `library-git`.
|
||||
|
||||
It returns `{ commit, files, problems }`, deep-frozen and sorted. Each file has its path, scope,
|
||||
universe, book, type, numbered lines, `bodyLine`, parsed `frontmatter` or `null`, `frontmatterRaw`
|
||||
and `links`. Front matter `appears_in` and `relationships` wikilinks count as links. A link in a block
|
||||
list carries its list item's line. One in a scalar or flow value carries its key's line.
|
||||
|
||||
Problems carry `{ code, path, line, message }`:
|
||||
|
||||
| Rule | Meaning | File indexed |
|
||||
|---|---|---|
|
||||
| R1 | not a regular `100644` blob of at most 1 MiB, or not UTF-8, or holds a NUL | no |
|
||||
| R2 | path is unmapped | no |
|
||||
| R3 | front matter not fully parsed, `line` is where parsing stopped | yes, front matter kept raw |
|
||||
| R4 | link resolves to no file or to more than one | yes, link dropped |
|
||||
| R5 | canon file links into the playground | yes |
|
||||
| R6 | canon file links into another universe, a hybrid's sources excepted | yes |
|
||||
| R7 | two paths differ only in case | no, neither |
|
||||
|
||||
The front matter parser takes scalars, quoted strings, flow lists, block lists and wikilinks. Nested
|
||||
maps, anchors, block scalars and duplicate keys stop it with R3.
|
||||
|
||||
## Links
|
||||
|
||||
`resolver(paths, { vaultPrefix })` tries four steps in order, each with the target as written and
|
||||
then with `.md` added:
|
||||
|
||||
1. `root`: the path from the repo root.
|
||||
2. `vault-prefix`: the same path with `vaultPrefix/` stripped.
|
||||
3. `relative`: relative to the linking file, never above the repo root.
|
||||
4. `shortest`: a target with no `/` that matches the base name of exactly one indexed file.
|
||||
|
||||
It resolves only against indexed files. A link or `![[...]]` embed whose target ends in an extension
|
||||
other than `.md`, in any case, is an attachment: an extension is a dot and up to 8 letters and digits,
|
||||
at least one a letter. The reader neither resolves nor indexes an attachment, and it is never R4. A
|
||||
note embed, with no extension or with `.md`, resolves like any link, so an unresolved one is R4. A
|
||||
link to a `.canvas` file is an attachment too. A link into another repo resolves to nothing.
|
||||
`extractLinks` skips fenced and inline code.
|
||||
|
||||
## renderItems
|
||||
|
||||
`renderItems(body, { scopeOf, changed })` turns a reply body into the plain text both channels show.
|
||||
It is pure. Item kinds are `fact`, `finding`, `correction` (with `from` and `to`), `question` and
|
||||
`idea`. Each cite is a path with commit, line range and quote, or a url with a quote.
|
||||
|
||||
```
|
||||
1. Correction: Chapter 1 says grey.
|
||||
From: "green eyes"
|
||||
To: "grey eyes"
|
||||
[canon] Book One/Characters/Mara Vell.md:9 @bbbbbbb "green eyes" (changed since)
|
||||
```
|
||||
|
||||
A body of `{ "items": [], "notice": ... }` gives `No cited answer`, `No answer` or `The turn failed`.
|
||||
Any other shape gives the fixed line `This reply could not be shown.` and none of the body's text.
|
||||
|
||||
## Survey
|
||||
|
||||
```sh
|
||||
mosaic library survey <repo> [--mapping <file>]
|
||||
```
|
||||
|
||||
It reads the repo's HEAD commit and prints folder names, file counts by extension, front matter key
|
||||
sets, link-form counts and the count of body links to attachments, which still count under their
|
||||
form. Folder names print as JSON strings, with DEL, C1 and bidi controls also as `\uXXXX`. It never
|
||||
prints file text, front matter values, file names or link targets.
|
||||
A key that is not a short identifier prints as `(other key)`, since it could be her words. With
|
||||
`--mapping` it adds scope counts, resolver step counts and problem counts by rule.
|
||||
|
||||
Exit codes follow `packages/project`: 0 done, 2 the repo can't be read (`library-git`,
|
||||
`library-commit`), 3 a bad mapping, 4 usage. `scripts/mosaic` sends `library survey` here. Other
|
||||
library verbs will be bus verbs.
|
||||
|
||||
## Tests
|
||||
|
||||
```sh
|
||||
node --test packages/library/tests/
|
||||
```
|
||||
|
||||
Fixtures are made-up repos built per test in a temp dir. A marker string stands in for her text, and
|
||||
the survey tests check that it never appears in output.
|
||||
@@ -1,11 +0,0 @@
|
||||
{
|
||||
"name": "@mosaic/library",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"license": "UNLICENSED",
|
||||
"exports": "./src/index.mjs",
|
||||
"scripts": {
|
||||
"test": "node --test tests/*.test.mjs"
|
||||
}
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
// packages/library: her writing repo read by a mapping (Case 2, design 1 and 7.4). See README.md.
|
||||
export { LibraryError, RULES, problem, sortProblems } from './problems.mjs';
|
||||
export { checkMapping, TYPES } from './mapping.mjs';
|
||||
export { classify } from './scope.mjs';
|
||||
export { extractLinks, parseWikilink, resolver, formOf, isAttachment } from './links.mjs';
|
||||
export { readLibrary, listTree, readTexts, parseFile, parseFrontMatter, headCommit, MAX_BLOB, LINK_FIELDS } from './reader.mjs';
|
||||
export { renderItems, KINDS, NOTICES, UNREADABLE } from './render.mjs';
|
||||
export { survey, formatSurvey, surveyMain } from './survey.mjs';
|
||||
@@ -1,86 +0,0 @@
|
||||
// Wikilinks and the link resolver of design 1.3. Links are [[target]], [[target|label]] and
|
||||
// [[target#heading]], with ![[...]] embeds read the same way. Links inside fenced code blocks and
|
||||
// inline code spans are not links, as in Obsidian. A link or embed whose target ends in an extension
|
||||
// other than `.md`, in any case, refers to an attachment (design 1.3): the reader neither resolves
|
||||
// nor indexes it, and it raises no problem.
|
||||
import { posix } from 'node:path';
|
||||
|
||||
const WIKILINK = /(!?)\[\[([^\[\]\n]+?)\]\]/g;
|
||||
const MARKDOWN_LINK = /(?<!!)\[[^\[\]\n]*\]\(([^()\s]+)\)/g;
|
||||
const FENCE = /^\s{0,3}(`{3,}|~{3,})/;
|
||||
|
||||
// One wikilink's parts, or null for text that isn't one.
|
||||
export function parseWikilink(text) {
|
||||
const match = /^(!?)\[\[([^\[\]\n]+?)\]\]$/.exec(text);
|
||||
return match ? parts(match[2], match[1] === '!') : null;
|
||||
}
|
||||
function parts(inner, embed) {
|
||||
const bar = inner.indexOf('|');
|
||||
const target = bar < 0 ? inner : inner.slice(0, bar);
|
||||
const hash = target.indexOf('#');
|
||||
return {
|
||||
file: (hash < 0 ? target : target.slice(0, hash)).trim(),
|
||||
heading: hash < 0 ? null : target.slice(hash + 1).trim(),
|
||||
label: bar < 0 ? null : inner.slice(bar + 1),
|
||||
embed,
|
||||
};
|
||||
}
|
||||
// An extension is a dot and up to 8 letters and digits, at least one a letter, at the end of the
|
||||
// target's last segment, so `Chapter 1.5` and `Mr. Smith` are notes and `cover.png` is not.
|
||||
export const isAttachment = link => !link.markdown
|
||||
&& /\.(?=[a-z0-9]*[a-z])[a-z0-9]{1,8}$/i.test(link.file.slice(link.file.lastIndexOf('/') + 1))
|
||||
&& !/\.md$/i.test(link.file);
|
||||
// The form a link is counted under in the survey.
|
||||
export const formOf = link => link.markdown ? 'markdown' : link.embed ? 'embed' : !link.file ? 'self'
|
||||
: link.label !== null ? 'label' : link.heading !== null ? 'heading' : 'plain';
|
||||
|
||||
// Links in body lines. `lines` is [[number, text], ...]. Markdown links are returned only with
|
||||
// `markdown: true`, for the survey's counts. The resolver follows wikilinks only.
|
||||
export function extractLinks(lines, { markdown = false } = {}) {
|
||||
const found = [];
|
||||
let fence = null;
|
||||
for (const [line, text] of lines) {
|
||||
const open = FENCE.exec(text);
|
||||
if (fence) { if (open && open[1][0] === fence[0] && open[1].length >= fence.length) fence = null; continue; }
|
||||
if (open) { fence = open[1]; continue; }
|
||||
const plain = text.replace(/(`+)[^`]*?\1/g, match => ' '.repeat(match.length));
|
||||
for (const m of plain.matchAll(WIKILINK)) found.push({ ...parts(m[2], m[1] === '!'), line });
|
||||
if (markdown) {
|
||||
for (const m of plain.matchAll(MARKDOWN_LINK)) {
|
||||
if (!/^[a-z][a-z0-9+.-]*:/i.test(m[1]) && !m[1].startsWith('#'))
|
||||
found.push({ file: m[1], heading: null, label: null, embed: false, markdown: true, line });
|
||||
}
|
||||
}
|
||||
}
|
||||
return found;
|
||||
}
|
||||
|
||||
// A resolver over the indexed paths. resolve(from, file) tries, in order: a path from the repo
|
||||
// root, the same path with `vaultPrefix/` stripped, a path relative to the linking file, and
|
||||
// Obsidian's shortest-path form, a base name that matches exactly one indexed file. Each step tries
|
||||
// the target as written and then with `.md` added. A target in another of her repos is not in the
|
||||
// index, so it resolves to nothing.
|
||||
export function resolver(paths, { vaultPrefix } = {}) {
|
||||
const known = new Set(paths), byBase = new Map();
|
||||
for (const path of known) {
|
||||
const base = posix.basename(path);
|
||||
for (const key of new Set([base, base.replace(/\.md$/, '')])) byBase.set(key, [...(byBase.get(key) ?? []), path]);
|
||||
}
|
||||
const hit = candidate => [candidate, `${candidate}.md`].find(c => known.has(c));
|
||||
return function resolve(from, file) {
|
||||
if (typeof file !== 'string' || !file) return { path: null, step: 'missing' };
|
||||
let path = hit(file);
|
||||
if (path) return { path, step: 'root' };
|
||||
if (vaultPrefix && file.startsWith(`${vaultPrefix}/`) && (path = hit(file.slice(vaultPrefix.length + 1))))
|
||||
return { path, step: 'vault-prefix' };
|
||||
const relative = posix.normalize(posix.join(posix.dirname(from), file));
|
||||
if (relative !== '..' && !relative.startsWith('../') && !relative.startsWith('/') && (path = hit(relative)))
|
||||
return { path, step: 'relative' };
|
||||
if (!file.includes('/')) {
|
||||
const matches = byBase.get(file) ?? [];
|
||||
if (matches.length === 1) return { path: matches[0], step: 'shortest' };
|
||||
if (matches.length > 1) return { path: null, step: 'ambiguous', count: matches.length };
|
||||
}
|
||||
return { path: null, step: 'missing' };
|
||||
};
|
||||
}
|
||||
@@ -1,132 +0,0 @@
|
||||
// The library project entry of a business file (design 1.1): her folders mapped onto universes,
|
||||
// books, type folders, manuscript folders and playground prefixes. A pure validator. Business file
|
||||
// loading wires it in C2-04, which passes its own `fail` so the refusal reads as a business file
|
||||
// problem. Agents can't write this entry (M9, C2-04).
|
||||
import { raise } from './problems.mjs';
|
||||
|
||||
export const TYPES = Object.freeze(['character', 'place', 'research', 'tracker', 'summary']);
|
||||
const ID = /^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$/;
|
||||
const BRANCH = /^[A-Za-z0-9][A-Za-z0-9._/-]{0,127}$/;
|
||||
const CONSTRAINT_KEYS = ['exclude', 'pin'];
|
||||
|
||||
// A folder segment from her vault: spaces and capitals are fine, separators, control characters
|
||||
// and dot names are not.
|
||||
function segment(value) {
|
||||
return typeof value === 'string' && value.length > 0 && value.length <= 255 && value !== '.' && value !== '..'
|
||||
&& !/[\/\\\u0000-\u001f\u007f]/.test(value) && value.trim() === value;
|
||||
}
|
||||
function folder(value) {
|
||||
return typeof value === 'string' && value.length <= 1024 && value.split('/').every(segment);
|
||||
}
|
||||
const holds = (outer, inner) => inner.startsWith(outer.endsWith('/') ? outer : `${outer}/`);
|
||||
|
||||
export function checkMapping(entry, where = 'library', { fail = raise } = {}) {
|
||||
const bad = message => fail('library-mapping-invalid', `${where} ${message}`);
|
||||
const object = (value, name) => {
|
||||
if (value === null || typeof value !== 'object' || Array.isArray(value)) bad(`${name} must be an object`);
|
||||
};
|
||||
const only = (value, keys, name) => {
|
||||
for (const key of Object.keys(value)) if (!keys.includes(key)) bad(`${name} has unknown key ${JSON.stringify(key)}`);
|
||||
};
|
||||
const list = (value, name, check, what) => {
|
||||
if (!Array.isArray(value)) bad(`${name} must be a list`);
|
||||
for (const item of value) if (!check(item)) bad(`${name} holds ${JSON.stringify(item)}, which is not ${what}`);
|
||||
if (new Set(value).size !== value.length) bad(`${name} names an entry twice`);
|
||||
return [...value];
|
||||
};
|
||||
|
||||
object(entry, '');
|
||||
only(entry, ['kind', 'remote', 'branch', 'vaultPrefix', 'universes', 'types', 'manuscript'], 'entry');
|
||||
if (entry.kind !== 'library') bad('kind must be "library"');
|
||||
let url;
|
||||
try { url = new URL(entry.remote); } catch { bad('remote must be an https:// URL'); }
|
||||
if (url.protocol !== 'https:' || url.username || url.password || url.search || url.hash || entry.remote.includes('\0'))
|
||||
bad('remote must be an https:// URL with no credentials, query or fragment');
|
||||
if (typeof entry.branch !== 'string' || !BRANCH.test(entry.branch) || entry.branch.includes('..')
|
||||
|| entry.branch.includes('//') || entry.branch.endsWith('/') || entry.branch.endsWith('.lock'))
|
||||
bad('branch must be a plain branch name');
|
||||
if (Object.hasOwn(entry, 'vaultPrefix') && !segment(entry.vaultPrefix)) bad('vaultPrefix must be one folder name');
|
||||
|
||||
object(entry.types, 'types');
|
||||
for (const [name, type] of Object.entries(entry.types)) {
|
||||
if (!segment(name)) bad(`types has ${JSON.stringify(name)}, which is not one folder name`);
|
||||
if (!TYPES.includes(type)) bad(`types.${name} must be one of ${TYPES.join(', ')}`);
|
||||
}
|
||||
const manuscript = list(entry.manuscript, 'manuscript', segment, 'one folder name');
|
||||
for (const name of manuscript) if (Object.hasOwn(entry.types, name)) bad(`manuscript folder ${JSON.stringify(name)} is also a type folder`);
|
||||
|
||||
object(entry.universes, 'universes');
|
||||
if (!Object.keys(entry.universes).length) bad('universes must name at least one universe');
|
||||
const universes = {}, bookOwner = new Map(), prefixes = [];
|
||||
for (const [id, universe] of Object.entries(entry.universes)) {
|
||||
const name = `universes.${id}`;
|
||||
if (!ID.test(id)) bad(`universe id ${JSON.stringify(id)} is not a plain id`);
|
||||
object(universe, name);
|
||||
const hybrid = universe.kind === 'hybrid';
|
||||
only(universe, hybrid ? ['kind', 'name', 'sources', 'books', 'playground', 'constraints', 'notes'] : ['name', 'books', 'playground'], name);
|
||||
if (typeof universe.name !== 'string' || !universe.name.trim() || universe.name.length > 128 || /[\u0000-\u001f]/.test(universe.name))
|
||||
bad(`${name}.name must be a short display name`);
|
||||
const books = list(universe.books, `${name}.books`, folder, 'a folder path');
|
||||
if (!hybrid && !books.length) bad(`${name}.books must name at least one book folder`);
|
||||
for (const book of books) {
|
||||
if (bookOwner.has(book)) bad(`book folder ${JSON.stringify(book)} is in two universes`);
|
||||
bookOwner.set(book, id);
|
||||
}
|
||||
const playground = list(universe.playground ?? [], `${name}.playground`,
|
||||
value => typeof value === 'string' && value.endsWith('/') && folder(value.slice(0, -1)), 'a folder path ending in /');
|
||||
for (const prefix of playground) {
|
||||
// Every playground file must have a universe and a book (1.1, B1).
|
||||
if (!books.some(book => holds(book, prefix) && prefix !== `${book}/`))
|
||||
fail('library-playground-outside-book', `${where} ${name}.playground ${JSON.stringify(prefix)} is not inside one of its books`);
|
||||
prefixes.push(prefix);
|
||||
}
|
||||
const out = { name: universe.name, books, playground };
|
||||
if (hybrid) {
|
||||
out.kind = 'hybrid';
|
||||
out.sources = list(universe.sources, `${name}.sources`, value => typeof value === 'string' && ID.test(value), 'a universe id');
|
||||
if (out.sources.length < 2) bad(`${name}.sources must name two or more universes`);
|
||||
if (!Array.isArray(universe.constraints ?? [])) bad(`${name}.constraints must be a list`);
|
||||
out.constraints = (universe.constraints ?? []).map((constraint, i) => {
|
||||
const at = `${name}.constraints[${i}]`;
|
||||
object(constraint, at);
|
||||
only(constraint, CONSTRAINT_KEYS, at);
|
||||
if (Object.keys(constraint).length !== 1) bad(`${at} must hold exactly one of ${CONSTRAINT_KEYS.join(', ')}`);
|
||||
if (Object.hasOwn(constraint, 'exclude')) {
|
||||
if (!folder(constraint.exclude)) bad(`${at}.exclude must be a source path`);
|
||||
return { exclude: constraint.exclude };
|
||||
}
|
||||
object(constraint.pin, `${at}.pin`);
|
||||
only(constraint.pin, ['entity', 'field', 'value'], `${at}.pin`);
|
||||
if (!folder(constraint.pin.entity) || typeof constraint.pin.field !== 'string' || !constraint.pin.field
|
||||
|| !['string', 'number', 'boolean'].includes(typeof constraint.pin.value))
|
||||
bad(`${at}.pin needs an entity path, a field name and a scalar value`);
|
||||
return { pin: { ...constraint.pin } };
|
||||
});
|
||||
if (Object.hasOwn(universe, 'notes') && (typeof universe.notes !== 'string' || universe.notes.length > 4000))
|
||||
bad(`${name}.notes must be text of at most 4000 characters`);
|
||||
if (Object.hasOwn(universe, 'notes')) out.notes = universe.notes;
|
||||
}
|
||||
universes[id] = out;
|
||||
}
|
||||
for (const [id, universe] of Object.entries(universes)) {
|
||||
for (const source of universe.sources ?? []) {
|
||||
if (!Object.hasOwn(universes, source) || universes[source].kind === 'hybrid')
|
||||
bad(`universes.${id}.sources names ${JSON.stringify(source)}, which is not a non-hybrid universe`);
|
||||
}
|
||||
}
|
||||
const books = [...bookOwner.keys()];
|
||||
for (const a of books) for (const b of books) if (a !== b && holds(a, b)) bad(`book folder ${JSON.stringify(a)} holds book folder ${JSON.stringify(b)}`);
|
||||
for (const a of prefixes) for (const b of prefixes) if (a !== b && holds(a, b)) bad(`playground ${JSON.stringify(a)} holds playground ${JSON.stringify(b)}`);
|
||||
if (new Set(prefixes).size !== prefixes.length) bad('a playground prefix is listed twice');
|
||||
|
||||
return deepFreeze({
|
||||
kind: 'library', remote: entry.remote, branch: entry.branch,
|
||||
...(Object.hasOwn(entry, 'vaultPrefix') ? { vaultPrefix: entry.vaultPrefix } : {}),
|
||||
universes, types: { ...entry.types }, manuscript,
|
||||
});
|
||||
}
|
||||
|
||||
function deepFreeze(value) {
|
||||
if (value && typeof value === 'object') { for (const item of Object.values(value)) deepFreeze(item); Object.freeze(value); }
|
||||
return value;
|
||||
}
|
||||
@@ -1,27 +0,0 @@
|
||||
// Errors and the R-rule problems of design 1.3. A problem never holds back a commit: it names a
|
||||
// path and, where there is one, a line, and the index leaves the file or link out.
|
||||
export class LibraryError extends Error {
|
||||
constructor(code, message) { super(`${code}: ${message}`); this.code = code; }
|
||||
}
|
||||
export const raise = (code, message) => { throw new LibraryError(code, message); };
|
||||
|
||||
export const RULES = Object.freeze({
|
||||
R1: 'not a regular blob up to 1 MiB of UTF-8 text',
|
||||
R2: 'path is unmapped',
|
||||
R3: 'front matter not fully parsed',
|
||||
R4: 'link resolves to no file or to more than one',
|
||||
R5: 'canon file refers into the playground',
|
||||
R6: 'canon file refers into another universe',
|
||||
R7: 'paths differ only in case',
|
||||
});
|
||||
|
||||
export function problem(code, path, line, message = RULES[code]) {
|
||||
if (!Object.hasOwn(RULES, code)) raise('library-problem-code', `unknown problem code ${code}`);
|
||||
return Object.freeze({ code, path, ...(Number.isSafeInteger(line) ? { line } : {}), message });
|
||||
}
|
||||
|
||||
// Stable order for status output and tests: path, then line, then code.
|
||||
export function sortProblems(list) {
|
||||
return [...list].sort((a, b) => a.path < b.path ? -1 : a.path > b.path ? 1
|
||||
: (a.line ?? 0) - (b.line ?? 0) || (a.code < b.code ? -1 : a.code > b.code ? 1 : 0));
|
||||
}
|
||||
@@ -1,223 +0,0 @@
|
||||
// The commit reader of design 1.3. It reads a commit of her repo, never a working tree: it lists the
|
||||
// tree with `git ls-tree -r -z` and reads blobs with `cat-file`, through the same bounded git()
|
||||
// and gitBytes() as packages/project, so hooks, protocols and the environment are fixed the same
|
||||
// way. Nothing in the repo is checked out or run (D7). Each R-rule is a problem, never a refusal
|
||||
// of the commit; the affected file or link is left out of the index.
|
||||
import { git, gitBytes } from '../../project/src/git.mjs';
|
||||
import { classify } from './scope.mjs';
|
||||
import { extractLinks, isAttachment, parseWikilink, resolver } from './links.mjs';
|
||||
import { LibraryError, problem, raise, sortProblems } from './problems.mjs';
|
||||
|
||||
export const MAX_BLOB = 1024 * 1024;
|
||||
const MAX_LISTING = 32 * 1024 * 1024;
|
||||
const SHA = /^[0-9a-f]{40}$/;
|
||||
const ENTRY = /^([0-7]{6}) (blob|tree|commit) ([0-9a-f]{40}) +(-|\d+)\t(.+)$/s;
|
||||
export const LINK_FIELDS = Object.freeze(['appears_in', 'relationships']);
|
||||
const READERS = 8;
|
||||
|
||||
async function local(promise, message) {
|
||||
try { return await promise; } catch (e) { raise('library-git', `${message} (${e.code ?? 'error'})`); }
|
||||
}
|
||||
|
||||
export async function headCommit(repo) {
|
||||
return local(git(repo, ['rev-parse', '--verify', '--end-of-options', 'HEAD^{commit}']), 'repository has no readable HEAD commit');
|
||||
}
|
||||
|
||||
// Every entry of the commit's tree: { mode, type, oid, size, path }. Sizes come from `-l`, so no
|
||||
// blob is opened to list it.
|
||||
export async function listTree(repo, commit) {
|
||||
if (typeof commit !== 'string' || !SHA.test(commit)) raise('library-commit', 'expected a full commit id');
|
||||
const kind = await local(git(repo, ['cat-file', '-t', commit]), 'commit is not in the repository');
|
||||
if (kind !== 'commit') raise('library-commit', 'not a commit');
|
||||
const bytes = await local(gitBytes(repo, ['ls-tree', '-r', '-z', '-l', '--full-tree', commit], MAX_LISTING), 'tree listing refused');
|
||||
let text;
|
||||
try { text = new TextDecoder('utf-8', { fatal: true }).decode(bytes); }
|
||||
catch { raise('library-git', 'tree listing is not UTF-8'); }
|
||||
const entries = text.split('\0');
|
||||
if (entries.pop() !== '') raise('library-git', 'invalid tree listing');
|
||||
return entries.map(entry => {
|
||||
const m = ENTRY.exec(entry);
|
||||
if (!m) raise('library-git', 'invalid tree listing');
|
||||
return { mode: m[1], type: m[2], oid: m[3], size: m[4] === '-' ? null : Number(m[4]), path: m[5] };
|
||||
});
|
||||
}
|
||||
|
||||
// R1: only a 100644 blob up to 1 MiB of UTF-8 text without NUL is read.
|
||||
export const readable = entry => entry.mode === '100644' && entry.type === 'blob'
|
||||
&& Number.isSafeInteger(entry.size) && entry.size <= MAX_BLOB;
|
||||
|
||||
// Text of each readable entry, by path. An entry whose bytes aren't UTF-8 text is missing from the
|
||||
// map, and the caller reports it under R1.
|
||||
export async function readTexts(repo, entries) {
|
||||
const texts = new Map(), queue = [...entries];
|
||||
const decoder = new TextDecoder('utf-8', { fatal: true, ignoreBOM: false });
|
||||
const worker = async () => {
|
||||
for (let entry = queue.shift(); entry; entry = queue.shift()) {
|
||||
const bytes = await local(gitBytes(repo, ['cat-file', 'blob', entry.oid], MAX_BLOB), 'blob read refused');
|
||||
let value;
|
||||
try { value = decoder.decode(bytes); } catch { continue; }
|
||||
if (!value.includes('\0')) texts.set(entry.path, value);
|
||||
}
|
||||
};
|
||||
await Promise.all(Array.from({ length: READERS }, worker));
|
||||
return texts;
|
||||
}
|
||||
|
||||
const QUOTED = /^"(?:[^"\\]|\\.)*"$|^'(?:[^']|'')*'$/;
|
||||
const PLAIN_START = /^[^\s\[\]{}&*!|>%@`"'#,]/;
|
||||
function scalar(raw) {
|
||||
const value = raw.trim();
|
||||
if (parseWikilink(value)) return { ok: true, value };
|
||||
if (QUOTED.test(value)) {
|
||||
if (value[0] === "'") return { ok: true, value: value.slice(1, -1).replace(/''/g, "'") };
|
||||
try { return { ok: true, value: JSON.parse(value) }; } catch { return { ok: false }; }
|
||||
}
|
||||
if (value === '' || value === '~' || value === 'null') return { ok: true, value: null };
|
||||
if (!PLAIN_START.test(value) || /: |\s#/.test(value) || value.endsWith(':')) return { ok: false };
|
||||
if (value === 'true' || value === 'false') return { ok: true, value: value === 'true' };
|
||||
if (/^-?(0|[1-9]\d*)(\.\d+)?$/.test(value)) return { ok: true, value: Number(value) };
|
||||
return { ok: true, value };
|
||||
}
|
||||
function flow(raw) {
|
||||
const inner = raw.trim().slice(1, -1), items = [];
|
||||
let current = '', quote = null;
|
||||
for (const ch of inner) {
|
||||
if (quote) { current += ch; if (ch === quote) quote = null; continue; }
|
||||
if (ch === '"' || ch === "'") { quote = ch; current += ch; continue; }
|
||||
if (ch === '[' || ch === ']' || ch === '{' || ch === '}') return { ok: false };
|
||||
if (ch === ',') { items.push(current); current = ''; continue; }
|
||||
current += ch;
|
||||
}
|
||||
if (quote) return { ok: false };
|
||||
if (current.trim() || items.length) items.push(current);
|
||||
const values = [];
|
||||
for (const item of items) { const s = scalar(item); if (!s.ok || !item.trim()) return { ok: false }; values.push(s.value); }
|
||||
return { ok: true, value: values };
|
||||
}
|
||||
function value(raw) {
|
||||
const text = raw.trim();
|
||||
if (text.startsWith('[') && !parseWikilink(text)) return text.endsWith(']') ? flow(text) : { ok: false };
|
||||
return scalar(text);
|
||||
}
|
||||
|
||||
// Front matter of design 1.3 (R3): scalars, quoted strings, flow lists and block lists of scalars or
|
||||
// wikilinks. Anything else, a nested map, a block scalar or an anchor for example, leaves the whole
|
||||
// front matter as raw text with one R3 problem at the line where parsing stopped.
|
||||
// Returns { frontmatter, raw, bodyLine, stop } with 1-based line numbers. Parsed front matter also
|
||||
// returns `itemLines`, each block list key's item lines in order.
|
||||
export function parseFrontMatter(lines) {
|
||||
if (lines[0]?.replace(/^/, '').trimEnd() !== '---') return { frontmatter: null, raw: null, bodyLine: 1, stop: null };
|
||||
const close = lines.findIndex((text, i) => i > 0 && /^(---|\.\.\.)\s*$/.test(text));
|
||||
if (close < 0) return { frontmatter: null, raw: null, bodyLine: 1, stop: 1 };
|
||||
const raw = lines.slice(1, close).join('\n'), data = {}, itemLines = {};
|
||||
let list = null;
|
||||
for (let i = 1; i < close; i++) {
|
||||
const text = lines[i], line = i + 1;
|
||||
if (!text.trim() || /^\s*#/.test(text)) continue;
|
||||
const item = /^(\s*)- (.*)$|^(\s*)-$/.exec(text);
|
||||
if (item && list) {
|
||||
const parsed = scalar(item[2] ?? '');
|
||||
if (!parsed.ok) return { frontmatter: null, raw, bodyLine: close + 2, stop: line };
|
||||
data[list].push(parsed.value);
|
||||
itemLines[list].push(line);
|
||||
continue;
|
||||
}
|
||||
const pair = /^([^\s:#\-][^:]*?|"[^"]+"|'[^']+')\s*:(?:\s+(.*))?$/.exec(text);
|
||||
if (!pair) return { frontmatter: null, raw, bodyLine: close + 2, stop: line };
|
||||
const key = pair[1].replace(/^(["'])(.*)\1$/, '$2');
|
||||
if (Object.hasOwn(data, key) || key === '__proto__') return { frontmatter: null, raw, bodyLine: close + 2, stop: line };
|
||||
if (pair[2] === undefined || !pair[2].trim()) {
|
||||
const next = lines[i + 1];
|
||||
if (i + 1 < close && /^\s*-( |$)/.test(next)) { data[key] = []; itemLines[key] = []; list = key; }
|
||||
else { data[key] = null; list = null; }
|
||||
continue;
|
||||
}
|
||||
list = null;
|
||||
const parsed = value(pair[2]);
|
||||
if (!parsed.ok) return { frontmatter: null, raw, bodyLine: close + 2, stop: line };
|
||||
data[key] = parsed.value;
|
||||
}
|
||||
return { frontmatter: data, raw: null, bodyLine: close + 2, stop: null, itemLines };
|
||||
}
|
||||
|
||||
export const splitLines = text => text.replace(/^/, '').split('\n').map(l => l.endsWith('\r') ? l.slice(0, -1) : l);
|
||||
|
||||
// One parsed file: its lines (all of them, numbered from 1), front matter and the links it holds.
|
||||
// Front matter links come from LINK_FIELDS values that are wikilinks; other values are names. A
|
||||
// link in a block list carries its item's line, one in a scalar or flow value its key's line.
|
||||
export function parseFile(text, { markdown = false } = {}) {
|
||||
const lines = splitLines(text);
|
||||
const fm = parseFrontMatter(lines);
|
||||
const numbered = lines.map((t, i) => [i + 1, t]);
|
||||
const links = extractLinks(numbered.slice(fm.bodyLine - 1), { markdown });
|
||||
if (fm.frontmatter) {
|
||||
for (const field of LINK_FIELDS) {
|
||||
const raw = fm.frontmatter[field];
|
||||
const values = Array.isArray(raw) ? raw : raw === undefined || raw === null ? [] : [raw];
|
||||
const items = fm.itemLines[field];
|
||||
values.forEach((v, i) => {
|
||||
const link = typeof v === 'string' ? parseWikilink(v.trim()) : null;
|
||||
if (link) links.push({ ...link, field, line: items ? items[i] : frontMatterLine(lines, fm.bodyLine, field) });
|
||||
});
|
||||
}
|
||||
}
|
||||
return { lines: numbered, bodyLine: fm.bodyLine, frontmatter: fm.frontmatter, frontmatterRaw: fm.raw, stop: fm.stop, links };
|
||||
}
|
||||
function frontMatterLine(lines, bodyLine, field) {
|
||||
const i = lines.slice(0, bodyLine - 1).findIndex(t => new RegExp(`^["']?${field}["']?\\s*:`).test(t));
|
||||
return i < 0 ? undefined : i + 1;
|
||||
}
|
||||
|
||||
// Read a commit under a checked mapping (mapping.mjs). Returns the indexed files and every
|
||||
// R1-R7 problem: { commit, files: [...], problems: [...] }, frozen, sorted by path.
|
||||
export async function readLibrary({ repo, commit, mapping }) {
|
||||
if (!mapping || mapping.kind !== 'library') throw new LibraryError('library-mapping-invalid', 'expected a checked library mapping');
|
||||
const entries = await listTree(repo, commit);
|
||||
const problems = [], twins = new Set(), byLower = new Map();
|
||||
for (const e of entries) byLower.set(e.path.toLowerCase(), [...(byLower.get(e.path.toLowerCase()) ?? []), e.path]);
|
||||
for (const group of byLower.values()) if (group.length > 1) for (const path of group) twins.add(path);
|
||||
|
||||
const candidates = [];
|
||||
for (const entry of entries) {
|
||||
const where = classify(mapping, entry.path);
|
||||
if (where.scope === 'unmapped') { problems.push(problem('R2', entry.path)); continue; }
|
||||
if (twins.has(entry.path)) { problems.push(problem('R7', entry.path)); continue; }
|
||||
if (!readable(entry)) { problems.push(problem('R1', entry.path)); continue; }
|
||||
candidates.push({ entry, where });
|
||||
}
|
||||
const texts = await readTexts(repo, candidates.map(c => c.entry));
|
||||
const files = [];
|
||||
for (const { entry, where } of candidates) {
|
||||
if (!texts.has(entry.path)) { problems.push(problem('R1', entry.path)); continue; }
|
||||
const parsed = parseFile(texts.get(entry.path));
|
||||
if (parsed.stop !== null) problems.push(problem('R3', entry.path, parsed.stop));
|
||||
files.push({ path: entry.path, ...where, name: entry.path.split('/').pop().replace(/\.md$/, ''), ...parsed });
|
||||
}
|
||||
|
||||
const resolve = resolver(files.map(f => f.path), { vaultPrefix: mapping.vaultPrefix });
|
||||
const byPath = new Map(files.map(f => [f.path, f]));
|
||||
for (const file of files) {
|
||||
const kept = [];
|
||||
for (const link of file.links) {
|
||||
if (!link.file || isAttachment(link)) continue;
|
||||
const target = resolve(file.path, link.file);
|
||||
if (!target.path) { problems.push(problem('R4', file.path, link.line)); continue; }
|
||||
const to = byPath.get(target.path);
|
||||
if (file.scope === 'canon') {
|
||||
if (to.scope === 'playground') problems.push(problem('R5', file.path, link.line));
|
||||
const sources = mapping.universes[file.universe].sources ?? [];
|
||||
if (to.universe !== file.universe && !sources.includes(to.universe)) problems.push(problem('R6', file.path, link.line));
|
||||
}
|
||||
kept.push({ ...link, path: target.path, step: target.step });
|
||||
}
|
||||
file.links = kept;
|
||||
delete file.stop;
|
||||
}
|
||||
files.sort((a, b) => a.path < b.path ? -1 : 1);
|
||||
return deepFreeze({ commit, files, problems: sortProblems(problems) });
|
||||
}
|
||||
|
||||
function deepFreeze(value) {
|
||||
if (value && typeof value === 'object' && !Object.isFrozen(value)) { Object.freeze(value); for (const v of Object.values(value)) deepFreeze(v); }
|
||||
return value;
|
||||
}
|
||||
@@ -1,59 +0,0 @@
|
||||
// renderItems(body): how a stored cited reply looks (design 7.4), shared by the Console chat (C2-10b)
|
||||
// and the Discord relay (C2-18). Pure: no git, no clock. The caller passes what it knows about cites.
|
||||
// The guard (C2-06) checks limits before a reply is stored. This only refuses to show a body that
|
||||
// doesn't have the reply shape, and then it shows a fixed line, never the body's text.
|
||||
export const KINDS = Object.freeze({ fact: 'Fact', finding: 'Finding', correction: 'Correction', question: 'Question', idea: 'Idea' });
|
||||
export const NOTICES = Object.freeze({ 'no-cited-answer': 'No cited answer', 'no-answer': 'No answer', 'turn-failed': 'The turn failed' });
|
||||
export const UNREADABLE = 'This reply could not be shown.';
|
||||
const SHA = /^[0-9a-f]{40}$/;
|
||||
|
||||
const plain = value => value !== null && typeof value === 'object' && !Array.isArray(value);
|
||||
const line = n => Number.isSafeInteger(n) && n > 0;
|
||||
function cite(value, { scopeOf, changed }) {
|
||||
if (!plain(value) || typeof value.quote !== 'string') return null;
|
||||
if (Object.hasOwn(value, 'url')) {
|
||||
if (typeof value.url !== 'string' || Object.hasOwn(value, 'path')) return null;
|
||||
return { badge: 'web', url: value.url, ref: value.url, quote: value.quote, changed: false };
|
||||
}
|
||||
const { path, commit, lines } = value;
|
||||
if (typeof path !== 'string' || !SHA.test(commit ?? '') || !Array.isArray(lines) || lines.length !== 2
|
||||
|| !line(lines[0]) || !line(lines[1]) || lines[1] < lines[0]) return null;
|
||||
const badge = scopeOf?.(path);
|
||||
const ref = `${path}:${lines[0] === lines[1] ? lines[0] : `${lines[0]}-${lines[1]}`} @${commit.slice(0, 7)}`;
|
||||
return { badge: badge === 'canon' || badge === 'playground' ? badge : null, path, commit, lines: [...lines], ref,
|
||||
quote: value.quote, changed: changed?.({ path, commit }) === true };
|
||||
}
|
||||
const quoted = text => JSON.stringify(text);
|
||||
|
||||
// body: the stored message body (a JSON string or the parsed object).
|
||||
// options.scopeOf(path) -> 'canon' | 'playground' | undefined, for the scope badge.
|
||||
// options.changed({path, commit}) -> true when the file's last commit is newer than the cited one.
|
||||
// Returns { kind: 'items', items, text } | { kind: 'notice', notice, text } | { kind: 'unreadable', text }.
|
||||
export function renderItems(body, options = {}) {
|
||||
const unreadable = { kind: 'unreadable', text: UNREADABLE };
|
||||
let value = body;
|
||||
if (typeof body === 'string') { try { value = JSON.parse(body); } catch { return unreadable; } }
|
||||
if (!plain(value) || !Array.isArray(value.items)) return unreadable;
|
||||
const keys = Object.keys(value);
|
||||
if (value.items.length === 0) {
|
||||
if (keys.length !== 2 || !Object.hasOwn(NOTICES, value.notice)) return unreadable;
|
||||
return { kind: 'notice', notice: value.notice, text: NOTICES[value.notice] };
|
||||
}
|
||||
if (keys.length !== 1) return unreadable;
|
||||
const items = [];
|
||||
for (const item of value.items) {
|
||||
if (!plain(item) || !Object.hasOwn(KINDS, item.kind) || typeof item.text !== 'string' || !Array.isArray(item.cites)) return unreadable;
|
||||
const correction = item.kind === 'correction';
|
||||
if (correction && (typeof item.from !== 'string' || typeof item.to !== 'string')) return unreadable;
|
||||
const cites = item.cites.map(c => cite(c, options));
|
||||
if (cites.some(c => c === null)) return unreadable;
|
||||
items.push({ kind: item.kind, label: KINDS[item.kind], text: item.text,
|
||||
...(correction ? { from: item.from, to: item.to } : {}), cites });
|
||||
}
|
||||
const text = items.map((item, i) => [
|
||||
`${i + 1}. ${item.label}: ${item.text}`,
|
||||
...(item.kind === 'correction' ? [` From: ${quoted(item.from)}`, ` To: ${quoted(item.to)}`] : []),
|
||||
...item.cites.map(c => ` ${c.badge ? `[${c.badge}] ` : ''}${c.ref} ${quoted(c.quote)}${c.changed ? ' (changed since)' : ''}`),
|
||||
].join('\n')).join('\n');
|
||||
return { kind: 'items', items, text };
|
||||
}
|
||||
@@ -1,21 +0,0 @@
|
||||
// The scope rule of design 1.2. Scope comes from the path alone (D2): no file content can change it.
|
||||
// Paths are repo-relative and compared exactly, case included (R7 handles case-only twins).
|
||||
const inside = (folder, path) => path.startsWith(folder.endsWith('/') ? folder : `${folder}/`);
|
||||
|
||||
// { scope: 'playground' | 'canon', universe, book, type, folder } or { scope: 'unmapped' }.
|
||||
// `type` is the mapped type of the first folder under the book (canon) or under the playground
|
||||
// prefix (playground), 'manuscript' for a manuscript folder, else null. `folder` is that folder name.
|
||||
export function classify(mapping, path) {
|
||||
if (typeof path !== 'string' || !path.endsWith('.md')) return { scope: 'unmapped' };
|
||||
for (const [universe, entry] of Object.entries(mapping.universes)) {
|
||||
const book = entry.books.find(b => inside(b, path));
|
||||
if (!book) continue;
|
||||
const prefix = entry.playground.find(p => inside(p, path));
|
||||
const rest = path.slice((prefix ?? `${book}/`).length).split('/');
|
||||
const folder = rest.length > 1 ? rest[0] : null;
|
||||
const type = folder === null ? null : Object.hasOwn(mapping.types, folder) ? mapping.types[folder]
|
||||
: mapping.manuscript.includes(folder) ? 'manuscript' : null;
|
||||
return { scope: prefix ? 'playground' : 'canon', universe, book, type, folder };
|
||||
}
|
||||
return { scope: 'unmapped' };
|
||||
}
|
||||
@@ -1,131 +0,0 @@
|
||||
// `mosaic library survey <repo> [--mapping <file>]` (design 1.1, 10 step 7). It prints what a mapping
|
||||
// needs: folder names, file counts by extension, front matter key sets, link-form counts and the
|
||||
// count of body links to attachments. It never prints file text, front matter values, file names or
|
||||
// link targets. With a mapping it adds
|
||||
// scope counts, resolver step counts and R-problem counts by code. It reads the repo's HEAD commit
|
||||
// through the commit reader, never the working tree.
|
||||
import { readFileSync, realpathSync, statSync } from 'node:fs';
|
||||
import { resolve as resolvePath } from 'node:path';
|
||||
import { pathToFileURL } from 'node:url';
|
||||
import { checkMapping } from './mapping.mjs';
|
||||
import { classify } from './scope.mjs';
|
||||
import { formOf, isAttachment } from './links.mjs';
|
||||
import { headCommit, listTree, parseFile, readable, readLibrary, readTexts } from './reader.mjs';
|
||||
import { LibraryError, RULES } from './problems.mjs';
|
||||
|
||||
export const USAGE = 'usage: mosaic library survey <repo> [--mapping <file>]';
|
||||
// Keys are printed only in the short identifier forms property names take. A key with spaces or of
|
||||
// any other shape could be a phrase of hers, so it is counted as one "(other key)".
|
||||
const KEY = /^[A-Za-z_][A-Za-z0-9_-]{0,31}$/;
|
||||
const OTHER_KEY = '(other key)';
|
||||
const FORMS = ['plain', 'label', 'heading', 'embed', 'self', 'markdown'];
|
||||
const STEPS = ['root', 'vault-prefix', 'relative', 'shortest'];
|
||||
const extension = path => /\.md$/.test(path) ? 'md' : /\.canvas$/.test(path) ? 'canvas' : /\.base$/.test(path) ? 'base' : 'other';
|
||||
|
||||
// The survey as data. `mapping` is a checked mapping or undefined.
|
||||
export async function survey({ repo, commit, mapping }) {
|
||||
const head = commit ?? await headCommit(repo);
|
||||
const entries = await listTree(repo, head);
|
||||
const folders = new Map();
|
||||
for (const entry of entries) {
|
||||
const slash = entry.path.lastIndexOf('/');
|
||||
const name = slash < 0 ? '' : entry.path.slice(0, slash);
|
||||
const counts = folders.get(name) ?? { md: 0, canvas: 0, base: 0, other: 0 };
|
||||
counts[entry.type === 'blob' ? extension(entry.path) : 'other']++;
|
||||
folders.set(name, counts);
|
||||
}
|
||||
const markdown = entries.filter(e => e.path.endsWith('.md'));
|
||||
const texts = await readTexts(repo, markdown.filter(readable));
|
||||
const keySets = new Map(), forms = Object.fromEntries(FORMS.map(f => [f, 0]));
|
||||
const fields = { appears_in: { wikilink: 0, name: 0 }, relationships: { wikilink: 0, name: 0 } };
|
||||
let noFrontMatter = 0, unparsed = 0, attachments = 0;
|
||||
for (const text of texts.values()) {
|
||||
const file = parseFile(text, { markdown: true });
|
||||
if (file.stop !== null) unparsed++;
|
||||
else if (!file.frontmatter) noFrontMatter++;
|
||||
else {
|
||||
const set = [...new Set(Object.keys(file.frontmatter).map(k => KEY.test(k) ? k : OTHER_KEY))].sort().join(', ');
|
||||
keySets.set(set, (keySets.get(set) ?? 0) + 1);
|
||||
for (const field of Object.keys(fields)) {
|
||||
const raw = file.frontmatter[field];
|
||||
for (const v of Array.isArray(raw) ? raw : raw === undefined || raw === null ? [] : [raw])
|
||||
fields[field][typeof v === 'string' && /^\s*!?\[\[[^\]]+\]\]\s*$/.test(v) ? 'wikilink' : 'name']++;
|
||||
}
|
||||
}
|
||||
for (const link of file.links) if (!link.field) { forms[formOf(link)]++; if (isAttachment(link)) attachments++; }
|
||||
}
|
||||
const result = {
|
||||
commit: head, entries: entries.length,
|
||||
folders: [...folders].sort(([a], [b]) => a < b ? -1 : 1).map(([name, counts]) => ({ name, ...counts })),
|
||||
markdown: { files: markdown.length, read: texts.size, noFrontMatter, unparsed },
|
||||
keySets: [...keySets].sort(([a, x], [b, y]) => y - x || (a < b ? -1 : 1)).map(([keys, count]) => ({ keys, count })),
|
||||
forms, attachments, fields,
|
||||
};
|
||||
if (mapping) {
|
||||
const scopes = { canon: 0, playground: 0, unmapped: 0 };
|
||||
for (const entry of entries) scopes[classify(mapping, entry.path).scope]++;
|
||||
const library = await readLibrary({ repo, commit: head, mapping });
|
||||
const steps = Object.fromEntries(STEPS.map(s => [s, 0]));
|
||||
for (const file of library.files) for (const link of file.links) steps[link.step]++;
|
||||
const problems = Object.fromEntries(Object.keys(RULES).map(code => [code, 0]));
|
||||
for (const p of library.problems) problems[p.code]++;
|
||||
steps.unresolved = problems.R4;
|
||||
Object.assign(result, { scopes, indexed: library.files.length, steps, problems });
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
// Folder names are quoted as JSON strings, which escape C0 controls but leave DEL, C1 and bidi
|
||||
// controls raw. quote() escapes those too, so no name can carry a control sequence to the terminal
|
||||
// or reorder the line around it.
|
||||
const RAW_CONTROLS = /[\u007f-\u009f\u200e\u200f\u202a-\u202e\u2066-\u2069]/g;
|
||||
export const quote = name => JSON.stringify(name).replace(RAW_CONTROLS, c => `\\u${c.charCodeAt(0).toString(16).padStart(4, '0')}`);
|
||||
export function formatSurvey(s) {
|
||||
const out = [`commit ${s.commit}`, `entries ${s.entries}`, 'folders (direct children: md canvas base other)'];
|
||||
for (const f of s.folders) out.push(` ${f.name === '' ? '(root)' : quote(f.name)} ${f.md} ${f.canvas} ${f.base} ${f.other}`);
|
||||
out.push(`markdown ${s.markdown.files} files, ${s.markdown.read} read, ${s.markdown.noFrontMatter} without front matter, ${s.markdown.unparsed} with front matter not fully parsed`);
|
||||
out.push('front matter key sets');
|
||||
for (const k of s.keySets) out.push(` ${k.count} [${k.keys}]`);
|
||||
out.push(`body links ${FORMS.map(f => `${f} ${s.forms[f]}`).join(', ')}; attachment ${s.attachments}`);
|
||||
for (const [field, c] of Object.entries(s.fields)) out.push(`field ${field} wikilink ${c.wikilink}, name ${c.name}`);
|
||||
if (s.scopes) {
|
||||
out.push(`scopes canon ${s.scopes.canon}, playground ${s.scopes.playground}, unmapped ${s.scopes.unmapped}; indexed ${s.indexed}`);
|
||||
out.push(`resolved ${STEPS.map(st => `${st} ${s.steps[st]}`).join(', ')}, unresolved ${s.steps.unresolved}`);
|
||||
out.push(`problems ${Object.entries(s.problems).map(([code, n]) => `${code} ${n}`).join(', ')}`);
|
||||
}
|
||||
return `${out.join('\n')}\n`;
|
||||
}
|
||||
|
||||
// Exit codes as packages/project's CLI: 0 done, 2 the repo can't be read, 3 a bad mapping, 4 usage.
|
||||
export async function surveyMain(argv, io = { stdout: process.stdout, stderr: process.stderr }) {
|
||||
const fail = (code, message) => { io.stderr.write(`library survey: ${message}\n`); return code; };
|
||||
const [verb, ...rest] = argv;
|
||||
if (verb !== 'survey') return fail(4, USAGE);
|
||||
let repo, mappingFile;
|
||||
for (let i = 0; i < rest.length; i++) {
|
||||
if (rest[i] === '--mapping' && i + 1 < rest.length && mappingFile === undefined) mappingFile = rest[++i];
|
||||
else if (!rest[i].startsWith('-') && repo === undefined) repo = rest[i];
|
||||
else return fail(4, USAGE);
|
||||
}
|
||||
if (repo === undefined) return fail(4, USAGE);
|
||||
let mapping;
|
||||
if (mappingFile !== undefined) {
|
||||
let entry;
|
||||
try { entry = JSON.parse(readFileSync(mappingFile, 'utf8')); }
|
||||
catch { return fail(3, `library-mapping-invalid: ${mappingFile} is not a readable JSON file`); }
|
||||
try { mapping = checkMapping(entry, mappingFile); }
|
||||
catch (e) { if (e instanceof LibraryError) return fail(3, e.message); throw e; }
|
||||
}
|
||||
const path = resolvePath(repo);
|
||||
try { if (!statSync(path).isDirectory()) return fail(2, `library-git: ${repo} is not a directory`); }
|
||||
catch { return fail(2, `library-git: ${repo} is not a directory`); }
|
||||
try { io.stdout.write(formatSurvey(await survey({ repo: path, mapping }))); }
|
||||
catch (e) { if (e instanceof LibraryError) return fail(2, e.message); throw e; }
|
||||
return 0;
|
||||
}
|
||||
|
||||
const entry = () => { try { return import.meta.url === pathToFileURL(realpathSync(process.argv[1])).href; } catch { return false; } };
|
||||
if (process.argv[1] && entry()) {
|
||||
const code = await surveyMain(process.argv.slice(2));
|
||||
if (code !== 0) process.exitCode = code;
|
||||
}
|
||||
-12
@@ -1,12 +0,0 @@
|
||||
{
|
||||
"kind": "library",
|
||||
"remote": "https://git.example.invalid/writer/books.git",
|
||||
"branch": "main",
|
||||
"vaultPrefix": "books",
|
||||
"universes": {
|
||||
"harbor": {"name": "The Harbor", "books": ["Book One"], "playground": ["Book One/Playground/"]},
|
||||
"second": {"name": "Second World", "books": ["Second World"]}
|
||||
},
|
||||
"types": {"Characters": "character", "Places": "place", "Research": "research", "Tracker": "tracker", "Summaries": "summary"},
|
||||
"manuscript": ["Chapters"]
|
||||
}
|
||||
@@ -1,54 +0,0 @@
|
||||
// A repo the test builds in her layout's shape: spaces and capitals in folder names, a .canvas file
|
||||
// and an unmapped top-level note. Bodies are made up; MARKER stands in for her text.
|
||||
import { mkdtempSync, rmSync, mkdirSync, writeFileSync, readFileSync, symlinkSync, chmodSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { checkMapping } from '../src/mapping.mjs';
|
||||
|
||||
export const MARKER = 'MARKER-7f3a-HER-TEXT';
|
||||
export const FIXTURES = fileURLToPath(new URL('./fixtures/', import.meta.url));
|
||||
export const mappingEntry = () => JSON.parse(readFileSync(join(FIXTURES, 'mapping.json'), 'utf8'));
|
||||
export const mapping = (change = e => e) => checkMapping(change(mappingEntry()));
|
||||
|
||||
export const git = (repo, ...args) => execFileSync('git', ['-c', 'user.name=fixture', '-c', '[email protected]',
|
||||
'-c', 'commit.gpgsign=false', '-c', 'core.hooksPath=/dev/null', '-C', repo, ...args], { encoding: 'utf8' }).trim();
|
||||
|
||||
export const BASE = {
|
||||
'Book One/Characters/Mara Vell.md': [
|
||||
'---', 'type: character', 'name: Mara Vell', 'appears_in: "[[Book One/Chapters/Chapter 01]]"',
|
||||
'relationships:', ' - "[[Tomas]]"', ' - The Ferryman', '---', `Grey eyes. ${MARKER}`, ''].join('\n'),
|
||||
'Book One/Characters/Tomas.md': ['---', 'type: character', 'name: Tomas', 'appears_in: ["[[Chapter 01]]"]', '---', MARKER, ''].join('\n'),
|
||||
'Book One/Chapters/Chapter 01.md': [
|
||||
'# One', `${MARKER} [[Mara Vell]] met [[Book One/Places/Harbor Town|the harbor]].`,
|
||||
'See [[../Places/Harbor Town#Docks]] and [[books/Book One/Characters/Tomas]].',
|
||||
'```', '[[Not A Link]]', '```', 'Inline `[[Not Either]]` and [a note](Notes.md).', ''].join('\n'),
|
||||
'Book One/Places/Harbor Town.md': ['---', 'type: place', '---', `Salt. ${MARKER}`, ''].join('\n'),
|
||||
'Book One/Playground/What If.md': ['---', 'name: What If', '---', `${MARKER} [[Mara Vell]]`, ''].join('\n'),
|
||||
'Book One/Board.canvas': '{"nodes":[]}\n',
|
||||
'Book One/Playground/Sketch.canvas': '{"nodes":[]}\n',
|
||||
'Inbox Note.md': `${MARKER}\n`,
|
||||
'Second World/Chapters/01.md': `# Elsewhere ${MARKER}\n`,
|
||||
};
|
||||
|
||||
// repo({files, extra}) writes files (path -> text, or { link } / { mode, text }), commits, and
|
||||
// returns { repo, commit }.
|
||||
export function repo(t, files = BASE) {
|
||||
const root = mkdtempSync(join(tmpdir(), 'library-'));
|
||||
t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
const dir = join(root, 'books');
|
||||
mkdirSync(dir);
|
||||
git(dir, 'init', '-q', '-b', 'main');
|
||||
for (const [path, value] of Object.entries(files)) {
|
||||
const file = join(dir, path);
|
||||
mkdirSync(dirname(file), { recursive: true });
|
||||
if (typeof value === 'object' && value.link) symlinkSync(value.link, file);
|
||||
else if (typeof value === 'object' && value.bytes) writeFileSync(file, value.bytes);
|
||||
else writeFileSync(file, typeof value === 'object' ? value.text : value);
|
||||
if (typeof value === 'object' && value.mode) chmodSync(file, value.mode);
|
||||
}
|
||||
git(dir, 'add', '-A');
|
||||
git(dir, 'commit', '-qm', 'fixture');
|
||||
return { root, repo: dir, commit: git(dir, 'rev-parse', 'HEAD') };
|
||||
}
|
||||
@@ -1,59 +0,0 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { extractLinks, isAttachment, parseWikilink, resolver } from '../src/index.mjs';
|
||||
|
||||
const PATHS = ['Book One/Characters/Mara Vell.md', 'Book One/Characters/Tomas.md', 'Book One/Places/Harbor Town.md',
|
||||
'Book One/Chapters/Chapter 01.md', 'Book One/Places/Notes.md', 'Book One/Characters/Notes.md', 'Mara Vell.md'];
|
||||
const resolve = resolver(PATHS, { vaultPrefix: 'books' });
|
||||
const FROM = 'Book One/Chapters/Chapter 01.md';
|
||||
|
||||
test('resolver step 1: a path from the repo root, with or without .md', () => {
|
||||
assert.deepEqual(resolve(FROM, 'Book One/Places/Harbor Town'), { path: 'Book One/Places/Harbor Town.md', step: 'root' });
|
||||
assert.deepEqual(resolve(FROM, 'Book One/Places/Harbor Town.md'), { path: 'Book One/Places/Harbor Town.md', step: 'root' });
|
||||
// Root wins over shortest: Mara Vell.md at the root, not the character page.
|
||||
assert.deepEqual(resolve(FROM, 'Mara Vell'), { path: 'Mara Vell.md', step: 'root' });
|
||||
});
|
||||
|
||||
test('resolver step 2: the same path with vaultPrefix/ stripped', () => {
|
||||
assert.deepEqual(resolve(FROM, 'books/Book One/Characters/Tomas'), { path: 'Book One/Characters/Tomas.md', step: 'vault-prefix' });
|
||||
assert.deepEqual(resolver(PATHS)(FROM, 'books/Book One/Characters/Tomas'), { path: null, step: 'missing' });
|
||||
});
|
||||
|
||||
test('resolver step 3: a path relative to the linking file, never above the repo root', () => {
|
||||
assert.deepEqual(resolve(FROM, '../Places/Harbor Town'), { path: 'Book One/Places/Harbor Town.md', step: 'relative' });
|
||||
assert.deepEqual(resolve('Book One/Places/Harbor Town.md', './Notes'), { path: 'Book One/Places/Notes.md', step: 'relative' });
|
||||
assert.deepEqual(resolve(FROM, '../../../Book One/Places/Harbor Town'), { path: null, step: 'missing' });
|
||||
});
|
||||
|
||||
test('resolver step 4: a base name that matches exactly one indexed file', () => {
|
||||
assert.deepEqual(resolve(FROM, 'Tomas'), { path: 'Book One/Characters/Tomas.md', step: 'shortest' });
|
||||
assert.deepEqual(resolve(FROM, 'Harbor Town.md'), { path: 'Book One/Places/Harbor Town.md', step: 'shortest' });
|
||||
assert.deepEqual(resolve(FROM, 'Notes'), { path: null, step: 'ambiguous', count: 2 });
|
||||
assert.deepEqual(resolve(FROM, 'Nobody'), { path: null, step: 'missing' });
|
||||
// A partial path is not a base name.
|
||||
assert.deepEqual(resolve(FROM, 'Places/Harbor Town'), { path: null, step: 'missing' });
|
||||
});
|
||||
|
||||
test('a link into another of her repos resolves to nothing', () => {
|
||||
assert.deepEqual(resolve(FROM, 'bean-brain/Journal/Today'), { path: null, step: 'missing' });
|
||||
assert.deepEqual(resolve(FROM, '../bean-brain/Journal/Today'), { path: null, step: 'missing' });
|
||||
});
|
||||
|
||||
test('wikilink forms, embeds, and links skipped inside code', () => {
|
||||
assert.deepEqual(parseWikilink('[[A/B#Head|Label]]'), { file: 'A/B', heading: 'Head', label: 'Label', embed: false });
|
||||
assert.deepEqual(parseWikilink('![[pic.png]]'), { file: 'pic.png', heading: null, label: null, embed: true });
|
||||
assert.equal(parseWikilink('[[a]] and more'), null);
|
||||
const lines = [[1, '[[One]] [[Two|2]] [[Three#H]] ![[Four]] [[#Self]]'], [2, '```js'], [3, '[[Fenced]]'], [4, '```'],
|
||||
[5, 'Inline `[[Code]]` and [md](Five.md) and [web](https://example.invalid/x)']];
|
||||
assert.deepEqual(extractLinks(lines).map(l => [l.file, l.line]), [['One', 1], ['Two', 1], ['Three', 1], ['Four', 1], ['', 1]]);
|
||||
assert.deepEqual(extractLinks(lines, { markdown: true }).filter(l => l.markdown).map(l => l.file), ['Five.md']);
|
||||
});
|
||||
|
||||
test('a target with an extension other than .md, in any case, is an attachment', () => {
|
||||
const att = text => isAttachment(parseWikilink(text));
|
||||
for (const s of ['![[cover.png]]', '[[cover.PNG]]', '![[Art/map.pdf|Map]]', '[[Board.canvas]]', '![[clip.mp4#t=5]]'])
|
||||
assert.equal(att(s), true, s);
|
||||
for (const s of ['![[missing-note]]', '![[note.md]]', '[[Note.MD]]', '[[Chapter 1.5]]', '[[Mr. Smith]]', '[[A.b/Note]]', '[[#Self]]'])
|
||||
assert.equal(att(s), false, s);
|
||||
assert.equal(isAttachment({ file: 'pic.png', markdown: true }), false, 'markdown links are never followed anyway');
|
||||
});
|
||||
@@ -1,82 +0,0 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { checkMapping, classify, LibraryError } from '../src/index.mjs';
|
||||
import { mapping, mappingEntry } from './helpers.mjs';
|
||||
|
||||
const refuses = (change, code, pattern) => assert.throws(() => checkMapping(change(mappingEntry())),
|
||||
e => e instanceof LibraryError && e.code === code && (!pattern || pattern.test(e.message)));
|
||||
|
||||
test('the fixture mapping checks and comes back frozen', () => {
|
||||
const m = mapping();
|
||||
assert.equal(m.universes.harbor.playground[0], 'Book One/Playground/');
|
||||
assert.ok(Object.isFrozen(m.universes.harbor.books));
|
||||
assert.throws(() => { m.types.Extra = 'place'; });
|
||||
});
|
||||
|
||||
test('a playground prefix outside its universe books refuses library-playground-outside-book', () => {
|
||||
refuses(e => { e.universes.harbor.playground = ['Second World/Playground/']; return e; }, 'library-playground-outside-book');
|
||||
refuses(e => { e.universes.harbor.playground = ['Elsewhere/']; return e; }, 'library-playground-outside-book');
|
||||
// The whole book as its playground would leave no canon: not inside the book.
|
||||
refuses(e => { e.universes.harbor.playground = ['Book One/']; return e; }, 'library-playground-outside-book');
|
||||
});
|
||||
|
||||
test('no playground prefix holds another, and no book folder holds another', () => {
|
||||
refuses(e => { e.universes.harbor.playground = ['Book One/Playground/', 'Book One/Playground/Deep/']; return e; },
|
||||
'library-mapping-invalid', /holds playground/);
|
||||
refuses(e => { e.universes.second.books = ['Second World', 'Book One/Part Two']; return e; }, 'library-mapping-invalid', /holds book folder/);
|
||||
refuses(e => { e.universes.second.books = ['Book One']; return e; }, 'library-mapping-invalid', /in two universes/);
|
||||
});
|
||||
|
||||
test('shape refusals: kind, remote, branch, unknown keys, types, manuscript, folder names', () => {
|
||||
refuses(e => ({ ...e, kind: 'code' }), 'library-mapping-invalid', /kind/);
|
||||
refuses(e => ({ ...e, remote: 'http://git.example.invalid/x.git' }), 'library-mapping-invalid', /https/);
|
||||
refuses(e => ({ ...e, remote: 'https://user:[email protected]/x.git' }), 'library-mapping-invalid', /credentials/);
|
||||
refuses(e => ({ ...e, remote: '[email protected]:x.git' }), 'library-mapping-invalid', /https/);
|
||||
refuses(e => ({ ...e, branch: 'main..x' }), 'library-mapping-invalid', /branch/);
|
||||
refuses(e => ({ ...e, root: '/tmp' }), 'library-mapping-invalid', /unknown key "root"/);
|
||||
refuses(e => { e.universes.harbor.extra = 1; return e; }, 'library-mapping-invalid', /unknown key/);
|
||||
refuses(e => { e.types.Characters = 'villain'; return e; }, 'library-mapping-invalid', /types.Characters/);
|
||||
refuses(e => { e.types['A/B'] = 'place'; return e; }, 'library-mapping-invalid', /one folder name/);
|
||||
refuses(e => ({ ...e, manuscript: ['Characters'] }), 'library-mapping-invalid', /also a type folder/);
|
||||
refuses(e => { e.universes.harbor.books = ['../Book One']; return e; }, 'library-mapping-invalid', /folder path/);
|
||||
refuses(e => ({ ...e, vaultPrefix: 'a/b' }), 'library-mapping-invalid', /vaultPrefix/);
|
||||
refuses(e => ({ ...e, universes: {} }), 'library-mapping-invalid', /at least one universe/);
|
||||
refuses(e => { e.universes['bad id'] = e.universes.second; delete e.universes.second; return e; }, 'library-mapping-invalid', /plain id/);
|
||||
});
|
||||
|
||||
test('a hybrid universe is a shape the query code can take (1.4), with sources that must exist', () => {
|
||||
const hybrid = e => { e.universes.cross = { kind: 'hybrid', name: 'Crossover', sources: ['harbor', 'second'], books: ['Crossover'],
|
||||
constraints: [{ exclude: 'second/Characters' }, { pin: { entity: 'Book One/Characters/Mara Vell.md', field: 'eyes', value: 'grey' } }],
|
||||
notes: 'Set after book one.' }; return e; };
|
||||
const m = checkMapping(hybrid(mappingEntry()));
|
||||
assert.deepEqual(m.universes.cross.sources, ['harbor', 'second']);
|
||||
refuses(e => { hybrid(e); e.universes.cross.sources = ['harbor', 'nowhere']; return e; }, 'library-mapping-invalid', /non-hybrid universe/);
|
||||
refuses(e => { hybrid(e); e.universes.cross.sources = ['harbor']; return e; }, 'library-mapping-invalid', /two or more/);
|
||||
refuses(e => { hybrid(e); e.universes.cross.constraints = [{ exclude: 'x', pin: {} }]; return e; }, 'library-mapping-invalid', /exactly one/);
|
||||
});
|
||||
|
||||
test('a caller-supplied fail receives the code, as business file loading will pass it (C2-04)', () => {
|
||||
const seen = [];
|
||||
const fail = (code, message) => { seen.push(code); throw new Error(message); };
|
||||
const entry = mappingEntry();
|
||||
entry.universes.harbor.playground = ['Elsewhere/'];
|
||||
assert.throws(() => checkMapping(entry, 'businesses/acme.json projects.books', { fail }), /businesses\/acme.json projects.books/);
|
||||
assert.deepEqual(seen, ['library-playground-outside-book']);
|
||||
});
|
||||
|
||||
test('scope comes from the path alone (design 1.2)', () => {
|
||||
const m = mapping();
|
||||
assert.deepEqual(classify(m, 'Book One/Characters/Mara Vell.md'),
|
||||
{ scope: 'canon', universe: 'harbor', book: 'Book One', type: 'character', folder: 'Characters' });
|
||||
assert.deepEqual(classify(m, 'Book One/Chapters/Chapter 01.md'),
|
||||
{ scope: 'canon', universe: 'harbor', book: 'Book One', type: 'manuscript', folder: 'Chapters' });
|
||||
assert.deepEqual(classify(m, 'Book One/Playground/What If.md'),
|
||||
{ scope: 'playground', universe: 'harbor', book: 'Book One', type: null, folder: null });
|
||||
assert.deepEqual(classify(m, 'Book One/Playground/Characters/New.md'),
|
||||
{ scope: 'playground', universe: 'harbor', book: 'Book One', type: 'character', folder: 'Characters' });
|
||||
assert.deepEqual(classify(m, 'Book One/Notes.md'), { scope: 'canon', universe: 'harbor', book: 'Book One', type: null, folder: null });
|
||||
assert.equal(classify(m, 'Second World/Chapters/01.md').universe, 'second');
|
||||
for (const path of ['Inbox Note.md', 'Book One/Board.canvas', 'Book One/Playground/Sketch.canvas', 'Book One/Board.base',
|
||||
'Book One/cover.png', 'Book One/Upper.MD', 'Book Onex/a.md', 'book one/a.md', 'Elsewhere/Book One/a.md'])
|
||||
assert.deepEqual(classify(m, path), { scope: 'unmapped' }, path);
|
||||
});
|
||||
@@ -1,171 +0,0 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { writeFileSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { readLibrary, parseFrontMatter, LibraryError, MAX_BLOB } from '../src/index.mjs';
|
||||
import { BASE, git, mapping, repo } from './helpers.mjs';
|
||||
|
||||
const codes = (result, path) => result.problems.filter(p => p.path === path).map(p => p.code);
|
||||
const file = (result, path) => result.files.find(f => f.path === path);
|
||||
|
||||
test('the fixture reads clean: every resolver step, appears_in and relationships wikilinks indexed', async t => {
|
||||
const r = repo(t);
|
||||
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
|
||||
assert.equal(result.commit, r.commit);
|
||||
assert.deepEqual(result.files.map(f => [f.path, f.scope]), [
|
||||
['Book One/Chapters/Chapter 01.md', 'canon'], ['Book One/Characters/Mara Vell.md', 'canon'],
|
||||
['Book One/Characters/Tomas.md', 'canon'], ['Book One/Places/Harbor Town.md', 'canon'],
|
||||
['Book One/Playground/What If.md', 'playground'], ['Second World/Chapters/01.md', 'canon']]);
|
||||
const chapter = file(result, 'Book One/Chapters/Chapter 01.md');
|
||||
assert.deepEqual(chapter.links.map(l => [l.path, l.step, l.line]), [
|
||||
['Book One/Characters/Mara Vell.md', 'shortest', 2], ['Book One/Places/Harbor Town.md', 'root', 2],
|
||||
['Book One/Places/Harbor Town.md', 'relative', 3], ['Book One/Characters/Tomas.md', 'vault-prefix', 3]]);
|
||||
const mara = file(result, 'Book One/Characters/Mara Vell.md');
|
||||
assert.deepEqual(mara.frontmatter, { type: 'character', name: 'Mara Vell', appears_in: '[[Book One/Chapters/Chapter 01]]',
|
||||
relationships: ['[[Tomas]]', 'The Ferryman'] });
|
||||
assert.deepEqual(mara.links.map(l => [l.field, l.path, l.line]), [
|
||||
['appears_in', 'Book One/Chapters/Chapter 01.md', 4], ['relationships', 'Book One/Characters/Tomas.md', 6]]);
|
||||
assert.deepEqual(file(result, 'Book One/Characters/Tomas.md').links.map(l => [l.field, l.path]),
|
||||
[['appears_in', 'Book One/Chapters/Chapter 01.md']]);
|
||||
assert.equal(mara.bodyLine, 9);
|
||||
assert.equal(mara.lines[8][0], 9);
|
||||
assert.equal(mara.type, 'character');
|
||||
// Only the unmapped paths are problems: the top-level note and both .canvas files.
|
||||
assert.deepEqual(result.problems.map(p => [p.code, p.path]), [
|
||||
['R2', 'Book One/Board.canvas'], ['R2', 'Book One/Playground/Sketch.canvas'], ['R2', 'Inbox Note.md']]);
|
||||
assert.ok(Object.isFrozen(result.files[0].links));
|
||||
});
|
||||
|
||||
test('R1: executables, symlinks, gitlinks, blobs over 1 MiB and non-UTF-8 text are not indexed', async t => {
|
||||
const r = repo(t, { ...BASE,
|
||||
'Book One/Places/Run.md': { text: 'x\n', mode: 0o755 },
|
||||
'Book One/Places/Link.md': { link: 'Harbor Town.md' },
|
||||
'Book One/Places/Big.md': 'x'.repeat(MAX_BLOB + 1),
|
||||
'Book One/Places/Edge.md': 'y'.repeat(MAX_BLOB),
|
||||
'Book One/Places/Latin.md': { bytes: Buffer.from([0x63, 0x61, 0x66, 0xe9, 0x0a]) },
|
||||
'Book One/Places/Nul.md': 'a\0b\n' });
|
||||
git(r.repo, 'update-index', '--add', '--cacheinfo', `160000,${r.commit},Book One/Places/Sub.md`);
|
||||
git(r.repo, 'commit', '-qm', 'gitlink');
|
||||
const result = await readLibrary({ repo: r.repo, commit: git(r.repo, 'rev-parse', 'HEAD'), mapping: mapping() });
|
||||
for (const name of ['Run', 'Link', 'Big', 'Latin', 'Nul', 'Sub']) {
|
||||
assert.deepEqual(codes(result, `Book One/Places/${name}.md`), ['R1'], name);
|
||||
assert.equal(file(result, `Book One/Places/${name}.md`), undefined, name);
|
||||
}
|
||||
assert.ok(file(result, 'Book One/Places/Edge.md'), 'exactly 1 MiB is read');
|
||||
});
|
||||
|
||||
test('R2: unmapped paths are reported and not indexed', async t => {
|
||||
const r = repo(t);
|
||||
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
|
||||
for (const path of ['Inbox Note.md', 'Book One/Board.canvas', 'Book One/Playground/Sketch.canvas']) {
|
||||
assert.deepEqual(codes(result, path), ['R2']);
|
||||
assert.equal(file(result, path), undefined);
|
||||
}
|
||||
});
|
||||
|
||||
test('R3: front matter not fully parsed stays raw, with the line where parsing stopped', async t => {
|
||||
const r = repo(t, { ...BASE,
|
||||
'Book One/Places/Nested.md': ['---', 'type: place', 'coords:', ' lat: 1', '---', 'Body [[Tomas]]', ''].join('\n'),
|
||||
'Book One/Places/Block.md': ['---', 'summary: |', ' text', '---', ''].join('\n'),
|
||||
'Book One/Places/Open.md': ['---', 'type: place', 'never closed', ''].join('\n') });
|
||||
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
|
||||
const nested = file(result, 'Book One/Places/Nested.md');
|
||||
assert.deepEqual(result.problems.filter(p => p.path === 'Book One/Places/Nested.md'), [{ code: 'R3', path: 'Book One/Places/Nested.md', line: 4, message: 'front matter not fully parsed' }]);
|
||||
assert.equal(nested.frontmatter, null);
|
||||
assert.equal(nested.frontmatterRaw, 'type: place\ncoords:\n lat: 1');
|
||||
assert.deepEqual(nested.links.map(l => l.path), ['Book One/Characters/Tomas.md'], 'still indexed by body');
|
||||
assert.equal(result.problems.find(p => p.path === 'Book One/Places/Block.md').line, 2);
|
||||
assert.equal(result.problems.find(p => p.path === 'Book One/Places/Open.md').line, 1);
|
||||
assert.equal(file(result, 'Book One/Places/Open.md').bodyLine, 1);
|
||||
});
|
||||
|
||||
test('front matter forms the reader accepts and refuses', () => {
|
||||
const parse = text => parseFrontMatter(text.split('\n'));
|
||||
assert.deepEqual(parse('---\na: plain words\nb: "quoted: yes"\nc: \'it\'\'s\'\nd: [x, "y z", 3]\ne:\n- one\n- "[[Two]]"\nf: [[Wiki]]\ng: true\nh:\n---').frontmatter,
|
||||
{ a: 'plain words', b: 'quoted: yes', c: "it's", d: ['x', 'y z', 3], e: ['one', '[[Two]]'], f: '[[Wiki]]', g: true, h: null });
|
||||
for (const bad of ['a: {x: 1}', 'a: &anchor x', 'a: [x, [y]]', 'a: x: y', 'a: 1\na: 2', ' a: 1', 'a: "open', '- loose'])
|
||||
assert.notEqual(parse(`---\n${bad}\n---`).stop, null, bad);
|
||||
assert.deepEqual(parse('no front matter'), { frontmatter: null, raw: null, bodyLine: 1, stop: null });
|
||||
});
|
||||
|
||||
test('R4: a link to no file, to more than one, or into another repo is reported and not followed', async t => {
|
||||
const r = repo(t, { ...BASE,
|
||||
'Book One/Research/Notes.md': 'a\n', 'Book One/Places/Notes.md': 'b\n',
|
||||
'Book One/Chapters/Links.md': ['[[Nobody]]', '[[Notes]]', '[[bean-brain/Journal/Today]]', '[[Tomas]]', ''].join('\n') });
|
||||
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
|
||||
assert.deepEqual(result.problems.filter(p => p.path === 'Book One/Chapters/Links.md').map(p => [p.code, p.line]), [['R4', 1], ['R4', 2], ['R4', 3]]);
|
||||
assert.deepEqual(file(result, 'Book One/Chapters/Links.md').links.map(l => l.path), ['Book One/Characters/Tomas.md']);
|
||||
});
|
||||
|
||||
test('a block list link carries its item line, a scalar or flow link its key line, and problems follow', async t => {
|
||||
const r = repo(t, { ...BASE, 'Book One/Characters/Ana.md': ['---', 'appears_in: ["[[Nobody]]"]', 'relationships:', ' - "[[Tomas]]"',
|
||||
' - Plain Name', ' - "[[Ghost]]"', ' - "[[What If]]"', '---', ''].join('\n') });
|
||||
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
|
||||
assert.deepEqual(file(result, 'Book One/Characters/Ana.md').links.map(l => [l.field, l.path, l.line]), [
|
||||
['relationships', 'Book One/Characters/Tomas.md', 4], ['relationships', 'Book One/Playground/What If.md', 7]]);
|
||||
assert.deepEqual(result.problems.filter(p => p.path === 'Book One/Characters/Ana.md').map(p => [p.code, p.line]),
|
||||
[['R4', 2], ['R4', 6], ['R5', 7]]);
|
||||
});
|
||||
|
||||
test('an attachment link or embed is not followed and gives no problem; a note embed resolves', async t => {
|
||||
const r = repo(t, { ...BASE, 'Book One/Places/note.md': 'n\n', 'Book One/Places/Shore.md': ['---',
|
||||
'relationships:', ' - "![[portrait.JPG]]"', '---', '![[cover.png]] [[Board.canvas]] ![[map.pdf|Map]]',
|
||||
'![[missing-note]] ![[note.md]] ![[Harbor Town]]', ''].join('\n') });
|
||||
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
|
||||
assert.deepEqual(result.problems.filter(p => p.path === 'Book One/Places/Shore.md').map(p => [p.code, p.line]), [['R4', 6]]);
|
||||
assert.deepEqual(file(result, 'Book One/Places/Shore.md').links.map(l => [l.path, l.embed]),
|
||||
[['Book One/Places/note.md', true], ['Book One/Places/Harbor Town.md', true]]);
|
||||
});
|
||||
|
||||
test('R5: a canon file that refers into the playground is reported, and the link still goes by path', async t => {
|
||||
const r = repo(t, { ...BASE, 'Book One/Places/Dock.md': 'See [[What If]].\n' });
|
||||
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
|
||||
assert.deepEqual(codes(result, 'Book One/Places/Dock.md'), ['R5']);
|
||||
assert.deepEqual(file(result, 'Book One/Places/Dock.md').links.map(l => l.path), ['Book One/Playground/What If.md']);
|
||||
// The playground file links to canon: no problem, R5 is only about canon.
|
||||
assert.deepEqual(codes(result, 'Book One/Playground/What If.md'), []);
|
||||
});
|
||||
|
||||
test('R6: a canon file that refers into another universe is reported; a hybrid may refer into its sources', async t => {
|
||||
const r = repo(t, { ...BASE, 'Book One/Places/Far.md': 'Like [[Second World/Chapters/01]].\n',
|
||||
'Crossover/Chapters/A.md': 'From [[Mara Vell]] and [[Second World/Chapters/01]].\n' });
|
||||
const hybrid = mapping(e => { e.universes.cross = { kind: 'hybrid', name: 'Cross', sources: ['harbor', 'second'], books: ['Crossover'] }; return e; });
|
||||
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: hybrid });
|
||||
assert.deepEqual(codes(result, 'Book One/Places/Far.md'), ['R6']);
|
||||
assert.deepEqual(file(result, 'Book One/Places/Far.md').links.map(l => l.path), ['Second World/Chapters/01.md']);
|
||||
assert.deepEqual(codes(result, 'Crossover/Chapters/A.md'), []);
|
||||
});
|
||||
|
||||
test('R7: two paths that differ only in case are both left out', async t => {
|
||||
const r = repo(t, BASE);
|
||||
writeFileSync(join(r.root, 'twin'), 'twin\n');
|
||||
const blob = git(r.repo, 'hash-object', '-w', join(r.root, 'twin'));
|
||||
git(r.repo, 'update-index', '--add', '--cacheinfo', `100644,${blob},Book One/Characters/tomas.md`);
|
||||
git(r.repo, 'commit', '-qm', 'case twin');
|
||||
const result = await readLibrary({ repo: r.repo, commit: git(r.repo, 'rev-parse', 'HEAD'), mapping: mapping() });
|
||||
for (const path of ['Book One/Characters/Tomas.md', 'Book One/Characters/tomas.md']) {
|
||||
assert.deepEqual(codes(result, path), ['R7'], path);
|
||||
assert.equal(file(result, path), undefined);
|
||||
}
|
||||
});
|
||||
|
||||
test('the reader reads the commit, never the working tree, and refuses a bad commit id', async t => {
|
||||
const r = repo(t);
|
||||
writeFileSync(join(r.repo, 'Book One/Places/Harbor Town.md'), 'unsaved [[Nobody]]\n');
|
||||
writeFileSync(join(r.repo, 'Book One/Places/Untracked.md'), 'x\n');
|
||||
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
|
||||
assert.equal(file(result, 'Book One/Places/Harbor Town.md').lines[0][1], '---');
|
||||
assert.equal(file(result, 'Book One/Places/Untracked.md'), undefined);
|
||||
for (const commit of ['HEAD', r.commit.slice(0, 12), '0'.repeat(40), git(r.repo, 'rev-parse', 'HEAD^{tree}')])
|
||||
await assert.rejects(readLibrary({ repo: r.repo, commit, mapping: mapping() }), e => e instanceof LibraryError, commit);
|
||||
await assert.rejects(readLibrary({ repo: r.repo, commit: r.commit, mapping: {} }), /library-mapping-invalid/);
|
||||
});
|
||||
|
||||
test('a bare mirror reads the same as the clone', async t => {
|
||||
const r = repo(t);
|
||||
const bare = join(r.root, 'mirror.git');
|
||||
git(r.root, 'clone', '-q', '--bare', r.repo, bare);
|
||||
const a = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
|
||||
const b = await readLibrary({ repo: bare, commit: r.commit, mapping: mapping() });
|
||||
assert.deepEqual(b, a);
|
||||
});
|
||||
@@ -1,80 +0,0 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { renderItems, UNREADABLE } from '../src/index.mjs';
|
||||
|
||||
const A = 'a'.repeat(40), B = 'b'.repeat(40);
|
||||
const cite = (path, lines, quote, commit = A) => ({ path, commit, lines, quote });
|
||||
const options = {
|
||||
scopeOf: path => path.includes('/Playground/') ? 'playground' : path.endsWith('.md') ? 'canon' : undefined,
|
||||
changed: ({ path, commit }) => path === 'Book One/Characters/Mara Vell.md' && commit === A,
|
||||
};
|
||||
const render = items => renderItems(JSON.stringify({ items }), options);
|
||||
|
||||
// One snapshot per item kind: the plain text both channels show.
|
||||
const SNAPSHOTS = {
|
||||
fact: [{ kind: 'fact', text: 'Tomas is the ferryman.', cites: [cite('Book One/Characters/Tomas.md', [3, 3], 'the ferryman', B)] }],
|
||||
finding: [{ kind: 'finding', text: 'Eye colour differs.', cites: [cite('Book One/Chapters/Chapter 01.md', [41, 41], 'her grey eyes'),
|
||||
cite('Book One/Characters/Mara Vell.md', [9, 10], 'green eyes')] }],
|
||||
correction: [{ kind: 'correction', text: 'Chapter 1 says grey.', from: 'green eyes', to: 'grey eyes',
|
||||
cites: [cite('Book One/Characters/Mara Vell.md', [9, 9], 'green eyes', B)] }],
|
||||
question: [{ kind: 'question', text: 'Is the harbor north of the town?', cites: [cite('Book One/Places/Harbor Town.md', [2, 4], 'the harbor', B)] }],
|
||||
idea: [{ kind: 'idea', text: 'A storm could close the harbor.', cites: [cite('Book One/Playground/What If.md', [5, 5], 'a storm', B),
|
||||
{ url: 'https://example.invalid/tides', quote: 'spring tides' }] }],
|
||||
};
|
||||
const EXPECTED = {
|
||||
fact: '1. Fact: Tomas is the ferryman.\n [canon] Book One/Characters/Tomas.md:3 @bbbbbbb "the ferryman"',
|
||||
finding: '1. Finding: Eye colour differs.\n [canon] Book One/Chapters/Chapter 01.md:41 @aaaaaaa "her grey eyes"\n'
|
||||
+ ' [canon] Book One/Characters/Mara Vell.md:9-10 @aaaaaaa "green eyes" (changed since)',
|
||||
correction: '1. Correction: Chapter 1 says grey.\n From: "green eyes"\n To: "grey eyes"\n'
|
||||
+ ' [canon] Book One/Characters/Mara Vell.md:9 @bbbbbbb "green eyes"',
|
||||
question: '1. Question: Is the harbor north of the town?\n [canon] Book One/Places/Harbor Town.md:2-4 @bbbbbbb "the harbor"',
|
||||
idea: '1. Idea: A storm could close the harbor.\n [playground] Book One/Playground/What If.md:5 @bbbbbbb "a storm"\n'
|
||||
+ ' [web] https://example.invalid/tides "spring tides"',
|
||||
};
|
||||
|
||||
for (const kind of Object.keys(SNAPSHOTS)) {
|
||||
test(`renderItems snapshot: ${kind}`, () => {
|
||||
const out = render(SNAPSHOTS[kind]);
|
||||
assert.equal(out.kind, 'items');
|
||||
assert.equal(out.text, EXPECTED[kind]);
|
||||
assert.equal(out.items[0].kind, kind);
|
||||
});
|
||||
}
|
||||
|
||||
test('renderItems numbers several items and takes a parsed body as well as a string', () => {
|
||||
const items = [...SNAPSHOTS.fact, ...SNAPSHOTS.question];
|
||||
const out = renderItems({ items }, options);
|
||||
assert.equal(out.text, `${EXPECTED.fact}\n${EXPECTED.question.replace(/^1\./, '2.')}`);
|
||||
assert.deepEqual(out.items[1].cites[0], { badge: 'canon', path: 'Book One/Places/Harbor Town.md', commit: B, lines: [2, 4],
|
||||
ref: 'Book One/Places/Harbor Town.md:2-4 @bbbbbbb', quote: 'the harbor', changed: false });
|
||||
});
|
||||
|
||||
test('without options a path cite has no badge and is never marked changed', () => {
|
||||
assert.equal(renderItems({ items: SNAPSHOTS.finding }).text,
|
||||
'1. Finding: Eye colour differs.\n Book One/Chapters/Chapter 01.md:41 @aaaaaaa "her grey eyes"\n'
|
||||
+ ' Book One/Characters/Mara Vell.md:9-10 @aaaaaaa "green eyes"');
|
||||
});
|
||||
|
||||
const NOTICE_SNAPSHOTS = { 'no-cited-answer': 'No cited answer', 'no-answer': 'No answer', 'turn-failed': 'The turn failed' };
|
||||
for (const [notice, text] of Object.entries(NOTICE_SNAPSHOTS)) {
|
||||
test(`renderItems snapshot: notice ${notice}`, () => {
|
||||
assert.deepEqual(renderItems(JSON.stringify({ items: [], notice }), options), { kind: 'notice', notice, text });
|
||||
});
|
||||
}
|
||||
|
||||
test('a body without the reply shape shows a fixed line and none of its text', () => {
|
||||
const secret = 'free prose the guard would refuse';
|
||||
const bodies = [secret, JSON.stringify(secret), '{"items": [], "notice": "other"}', JSON.stringify({ items: [], notice: 'no-answer', extra: secret }),
|
||||
JSON.stringify({ items: [] }), JSON.stringify({ items: [{ kind: 'story', text: secret, cites: [] }] }),
|
||||
JSON.stringify({ items: [{ kind: 'fact', text: secret }] }), JSON.stringify({ items: SNAPSHOTS.fact, prose: secret }),
|
||||
JSON.stringify({ items: [{ kind: 'correction', text: secret, cites: [] }] }),
|
||||
JSON.stringify({ items: [{ kind: 'fact', text: secret, cites: [cite('a.md', [2, 1], 'q')] }] }),
|
||||
JSON.stringify({ items: [{ kind: 'fact', text: secret, cites: [{ path: 'a.md', commit: 'abc', lines: [1, 1], quote: 'q' }] }] }),
|
||||
JSON.stringify({ items: [{ kind: 'fact', text: secret, cites: [{ url: 'https://x.invalid', path: 'a.md', quote: 'q' }] }] }),
|
||||
null, 42, '[]'];
|
||||
for (const body of bodies) {
|
||||
const out = renderItems(body, options);
|
||||
assert.deepEqual(out, { kind: 'unreadable', text: UNREADABLE }, String(body));
|
||||
assert.ok(!JSON.stringify(out).includes(secret));
|
||||
}
|
||||
});
|
||||
@@ -1,100 +0,0 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { writeFileSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { surveyMain } from '../src/index.mjs';
|
||||
import { BASE, FIXTURES, MARKER, repo } from './helpers.mjs';
|
||||
|
||||
const ROOT = fileURLToPath(new URL('../../..', import.meta.url));
|
||||
async function run(args) {
|
||||
let stdout = '', stderr = '';
|
||||
const code = await surveyMain(args, { stdout: { write: s => { stdout += s; } }, stderr: { write: s => { stderr += s; } } });
|
||||
return { code, stdout, stderr };
|
||||
}
|
||||
// Strings from the fixture that are text, values, file names or link targets: none may be printed.
|
||||
const NEVER = [MARKER, 'Grey eyes', 'The Ferryman', 'Mara Vell', 'Tomas', 'Harbor Town', 'What If', 'Inbox Note', 'Chapter 01',
|
||||
'Sketch', 'Board', 'Not A Link', 'the harbor', 'Docks', 'character'];
|
||||
|
||||
test('the survey prints folder names, counts, key sets and link-form counts, and never text', async t => {
|
||||
const r = repo(t, { ...BASE, 'Book One/Places/Odd.md': ['---', `"${MARKER} key": 1`, '---', ''].join('\n') });
|
||||
const out = await run(['survey', r.repo]);
|
||||
assert.equal(out.code, 0, out.stderr);
|
||||
assert.equal(out.stdout, [
|
||||
`commit ${r.commit}`,
|
||||
'entries 10',
|
||||
'folders (direct children: md canvas base other)',
|
||||
' (root) 1 0 0 0',
|
||||
' "Book One" 0 1 0 0',
|
||||
' "Book One/Chapters" 1 0 0 0',
|
||||
' "Book One/Characters" 2 0 0 0',
|
||||
' "Book One/Places" 2 0 0 0',
|
||||
' "Book One/Playground" 1 1 0 0',
|
||||
' "Second World/Chapters" 1 0 0 0',
|
||||
'markdown 8 files, 8 read, 3 without front matter, 0 with front matter not fully parsed',
|
||||
'front matter key sets',
|
||||
' 1 [(other key)]',
|
||||
' 1 [appears_in, name, relationships, type]',
|
||||
' 1 [appears_in, name, type]',
|
||||
' 1 [name]',
|
||||
' 1 [type]',
|
||||
'body links plain 3, label 1, heading 1, embed 0, self 0, markdown 1; attachment 0',
|
||||
'field appears_in wikilink 2, name 0',
|
||||
'field relationships wikilink 1, name 1',
|
||||
''].join('\n'));
|
||||
for (const s of NEVER) assert.ok(!out.stdout.includes(s), s);
|
||||
});
|
||||
|
||||
test('folder names print DEL, C1 and bidi controls as escapes', async t => {
|
||||
// One name per escaped range, with each range's first and last code point.
|
||||
const names = ['A\u007fB', 'C\u0080D\u009fE', 'F\u200eG\u200fH', 'I\u202aJ\u202eK', 'L\u2066M\u2069N'];
|
||||
const r = repo(t, Object.fromEntries(names.map(n => [`${n}/a.md`, `${MARKER}\n`])));
|
||||
const out = await run(['survey', r.repo]);
|
||||
assert.equal(out.code, 0, out.stderr);
|
||||
assert.deepEqual(out.stdout.split('\n').slice(3, 8), [
|
||||
' "A\\u007fB" 1 0 0 0', ' "C\\u0080D\\u009fE" 1 0 0 0', ' "F\\u200eG\\u200fH" 1 0 0 0',
|
||||
' "I\\u202aJ\\u202eK" 1 0 0 0', ' "L\\u2066M\\u2069N" 1 0 0 0']);
|
||||
assert.doesNotMatch(out.stdout, /[\u007f-\u009f\u200e\u200f\u202a-\u202e\u2066-\u2069]/);
|
||||
});
|
||||
|
||||
test('an embed of an attachment counts as an embed and as an attachment', async t => {
|
||||
const r = repo(t, { 'Book One/Chapters/A.md': ['![[cover.png]] ![[missing-note]] [[map.PDF]] [[Chapter 1.5]]', ''].join('\n') });
|
||||
const out = await run(['survey', r.repo]);
|
||||
assert.match(out.stdout, /^body links plain 2, label 0, heading 0, embed 2, self 0, markdown 0; attachment 2$/m);
|
||||
});
|
||||
|
||||
test('with --mapping the survey adds scope, resolver step and problem counts', async t => {
|
||||
const r = repo(t);
|
||||
const out = await run(['survey', r.repo, '--mapping', join(FIXTURES, 'mapping.json')]);
|
||||
assert.equal(out.code, 0, out.stderr);
|
||||
const tail = out.stdout.trimEnd().split('\n').slice(-3);
|
||||
assert.deepEqual(tail, [
|
||||
'scopes canon 5, playground 1, unmapped 3; indexed 6',
|
||||
'resolved root 2, vault-prefix 1, relative 2, shortest 3, unresolved 0',
|
||||
'problems R1 0, R2 3, R3 0, R4 0, R5 0, R6 0, R7 0']);
|
||||
for (const s of NEVER) assert.ok(!out.stdout.includes(s), s);
|
||||
});
|
||||
|
||||
test('survey exit codes: usage 4, bad mapping 3, unreadable repo 2', async t => {
|
||||
const r = repo(t);
|
||||
for (const args of [[], ['scan', r.repo], ['survey'], ['survey', r.repo, '--mapping'], ['survey', r.repo, '--other'], ['survey', r.repo, r.repo]])
|
||||
assert.equal((await run(args)).code, 4, args.join(' '));
|
||||
const bad = join(r.root, 'bad.json');
|
||||
writeFileSync(bad, JSON.stringify({ kind: 'library' }));
|
||||
const refused = await run(['survey', r.repo, '--mapping', bad]);
|
||||
assert.equal(refused.code, 3);
|
||||
assert.match(refused.stderr, /library-mapping-invalid/);
|
||||
writeFileSync(bad, 'not json');
|
||||
assert.equal((await run(['survey', r.repo, '--mapping', bad])).code, 3);
|
||||
assert.equal((await run(['survey', join(r.root, 'missing')])).code, 2);
|
||||
assert.equal((await run(['survey', r.root])).code, 2, 'a directory that is not a repo');
|
||||
});
|
||||
|
||||
test('scripts/mosaic library survey reaches the survey', t => {
|
||||
const r = repo(t);
|
||||
const out = spawnSync(join(ROOT, 'scripts/mosaic'), ['library', 'survey', r.repo], { encoding: 'utf8', timeout: 30000 });
|
||||
assert.equal(out.status, 0, out.stderr);
|
||||
assert.match(out.stdout, new RegExp(`^commit ${r.commit}\n`));
|
||||
assert.ok(!out.stdout.includes(MARKER));
|
||||
});
|
||||
@@ -34,12 +34,6 @@ if [ "${1:-}" = business ]; then
|
||||
shift
|
||||
exec node "$REPO/packages/business/src/cli.mjs" "$@"
|
||||
fi
|
||||
# `mosaic library survey`: names and counts from a writing repo, never its text
|
||||
# (packages/library/README.md). Other library verbs are bus verbs.
|
||||
if [ "${1:-}" = library ] && [ "${2:-}" = survey ]; then
|
||||
shift
|
||||
exec node "$REPO/packages/library/src/survey.mjs" "$@"
|
||||
fi
|
||||
case "${1:-}" in
|
||||
usage|project|limits|inbox|decide|tasks|agents|trail|talk|stop|launches|bus|plugins|name|names|follow|threads|thread) exec node "$REPO/packages/cli/src/cli.mjs" "$@" ;;
|
||||
esac
|
||||
|
||||
@@ -27,7 +27,7 @@ COPY_PATHS=(
|
||||
packages/conversation/src packages/discord/src packages/discord/bin packages/discord/extension packages/harness/src
|
||||
packages/seat/src packages/tasks/src packages/tasks/fields.json packages/tasks/requirements.json
|
||||
packages/webui/src packages/webui/README.md packages/cli/README.md
|
||||
packages/queue/src packages/library/src
|
||||
packages/queue/src
|
||||
# Whole directory: every in-repo plug-in module ships with the copy (row 131).
|
||||
packages/plugins
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user