Files
jason.woltje f0e11cb342 feat(proxy): Console proxy setup page and CLI parity (#1873)
One Proxy setup panel on the Model proxy settings page takes an owner from
nothing to a running proxy: status with one next-step button per state, the
pinned fetch with hash checks, progress and a receipt, init with a
bus-generated key (a pasted key only over loopback or HTTPS, with a
confirmation), unit render/install/start/stop/restart with health, a fence
clear that the bus re-proves against a confirmed restart, and a stale init
lock clear that names its blockers. Own proxy per organization shows as not
built.

New Console routes under /api/proxy/admin/ use the existing owner proxy
checks; agents, testers and grant holders get human-required. New bus verbs
proxy.init {generate:true} and proxy.fence.clear. Receipts go to
<dataRoot>/state/proxy-admin.jsonl with labels and key ids only. Every new
refusal code has a refusal-table entry.

CLI: mosaic proxy status, unit render|status|install|start|stop|restart,
init --generate-key, fence clear --restart, init-lock status|clear; proxy
fetch keeps its stdout and adds progress and the receipt.

Page fix found in the end-to-end run: a stopped proxy hides the account
panels until the service answers. (The catalogue switch label fix found in
the same run landed upstream as D3 P0-4.)
2026-11-11 16:32:31 -06:00

1562 lines
100 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# TOOLS.md — command and tool reference
On-demand reference for agent sessions (conductors, bootstrapping agents,
reviewers). `AGENTS.md` routes here; this file carries the depth: usage,
inputs/outputs, exit codes, and safety notes for every entry point.
Reading guide: system entry points are `scripts/*.sh` (bash) or invoked via
`node scripts/mosaic-task.mjs` (node). Host-side helpers under `tools/`
(tmux messaging, watchers, prose checker) are covered under Tools
(host-side) below. Every script fails closed — missing
or invalid configuration/policy refuses the operation with a nonzero exit
and changes nothing.
## Lifecycle
| Command | Purpose | Notes |
|---|---|---|
| `scripts/bootstrap.sh [--instance NAME]` | Create `~/.config/mosaic-dev/config.json` if absent; with `--instance NAME`, `~/.config/mosaic-NAME/config.json` and the claimed root `~/.mosaic-NAME` | Idempotent; existing config validated, never rewritten. See [Stack instances](#stack-instances---instance-name-row-190) |
| `scripts/build.sh` | Build the release image | Tag derived from `RELEASE` + pinned pi version |
| `scripts/hello.sh` | One-shot startup request | Prints model response on stdout |
| `scripts/verify.sh` | Full gated test | Exit 0 only on exact `MOSAIC_HELLO_OK`; `EXPECTED_MARKER` overrides for negative drills |
## Tasks (missions, runs, evidence)
| Command | Purpose | Notes |
|---|---|---|
| `scripts/run-task.sh run <task.json>` | Execute a task | Immutable run record under `<dataRoot>/runs/` |
| `scripts/run-task.sh validate <task.json>` | Strict validation | Writes nothing |
| `node scripts/mosaic-task.mjs show <runId>` | Inspect a run | Full record + snapshots + artifacts |
| `node scripts/mosaic-task.mjs list` | List runs | task/workspace/session columns |
| `node scripts/mosaic-task.mjs retry <runId>` | Re-execute a run's snapshot | New run dir; `retriedFrom` lineage recorded |
| `node scripts/mosaic-task.mjs prune [--keep=N] [--yes]` | Retention | Dry-run default; receipt in `runs/.pruned.log` |
| `node scripts/mosaic-task.mjs resolve-role <roleFile>` | Validate a role file, version 1 or 2 | Prints `MOSAIC_ROLE_TOOLS` / `MOSAIC_ROLE_NETWORK`, and `MOSAIC_ROLE_CONTRACT` for version 2; config-free |
Task fields: `prompt` (required), `mission` (path), `expectExact`,
`timeoutSeconds` (5–600), `workspace` (`:run` or named), `capabilities.tools`
(allowlist: read write edit bash grep find ls), `session`,
`sessionForkFrom` (requires `session`). Mission fields: `objective`,
`directives[]`, optional governing `capabilities.tools`. Policy: a task may
narrow a mission's tools, never widen; empty intersection = tool-free run.
Legacy worker containers use a typed per-launch mount contract. Task, mission,
user and session-fork snapshots are read-only. A fork source is opened without
following its final symlink, limited to 16 MiB, checked for owner/type and
concurrent changes, then copied write-once into the current run before launch.
The host-owned `control/` parent is read-only so atomic processing-state
replacement remains visible; it contains only processing and runtime/stop
metadata, not a socket. Only the selected workspace and target session are
read-write. The host data-root root, evidence outside the current run,
unselected run/workspace/session trees and broker/control sockets are absent,
and the generated prompt exists only in the disposable container filesystem.
Legacy selected names still share namespaces with managed workers: a workspace
name can select a managed business parent and `discord-*` sessions are not
reserved. A dedicated legacy prefix or explicit collision refusal is a separate
migration follow-up. Every launch kind, including `hello.sh` and `verify.sh`
probes, refuses a symlinked `user/`, a symlinked or non-file `user/*.md`, or any
mount source path containing `:`. Missing or unsupported launch inputs also
refuse before Docker dispatch. These source properties do not by themselves
establish native container confinement.
The admitted #660 compatibility gate must run this exact legacy-worker list in
addition to its full repository gate; it was not run by this source repair:
```bash
node --test --test-concurrency=1 packages/processing-control/tests/containers.test.mjs packages/processing-control/tests/coordinator.test.mjs packages/processing-control/tests/effects.test.mjs packages/processing-control/tests/mount.test.mjs packages/processing-control/tests/runtimes.test.mjs packages/processing-control/tests/state.test.mjs packages/processing-control/tests/stop-run.test.mjs packages/processing-control/tests/task.test.mjs
node --test scripts/test-darkwing-launch.mjs
scripts/test-task.sh
scripts/build.sh
```
## Agent (interactive TUI)
```bash
scripts/agent.sh <name> [--mission <file>] [--workspace <ws>] [--session <s>] [--tools <list>]
```
Launches an interactive pi TUI inside the container with the four immutable
contracts + optional mission + agent identity as its system prompt,
persistent named session, optional workspace. Exit with `/quit`.
The interactive container uses the same narrow launcher: its selected SOUL,
mission, user context and enabled skills are read-only mounts, while only its
workspace and named session are writable. Role authority is resolved on the
host from the reviewed `roles/*.json`; the data-root root is not mounted.
A seat role (`agent.json` `role`) binds to `roles/<role>.json` (M18): the
contract's tools are a ceiling the seat definition or `--tools` may narrow,
never escalate past. Missing/invalid contract refuses the launch; empty
intersection = loud tool-free seat. An explicit `MOSAIC_AGENTS_DIR` override
that cannot resolve the named seat also refuses (#46) — unset the override
for the M13 plain governed TUI. `--auth <account>` injects
`auth.<account>.json` (beside the active credential file) as the launch's
`PI_AUTH_FILE`; a missing/invalid account refuses (M19).
For native repository development, opt in with a **leading** `--host-dev`:
```bash
scripts/agent.sh --host-dev darkwing [--fresh] [--check] [--soul FILE] [--constitution FILE] [--user FILE] [--provider NAME] [--model ID] [--thinking LEVEL]
```
This mode delegates to `scripts/agent-host-dev.sh`, uses host Pi and repository
tools/skills plus the development goal extension, and keeps its own sessions
under `.pi/state/<name>/`. It uses native Pi authentication and does not run
container release alignment or apply managed seat role ceilings. It is a host
development session, not a sandboxed worker. Container-only flags such as
`--auth`, `--mission`, and `--tools` are rejected in this mode. Omitting
`--host-dev` retains the existing container lifecycle and policy checks;
container failures never trigger a host fallback. Darkwing's agent-local shim
selects host development explicitly. See `agents/darkwing/README.md`.
## Auth (credentials)
Credential checkpoint over pi's auth model (one `auth.json` keyed by
provider; resolution order `--api-key` → `auth.json` → env → models.json).
No credential material is ever printed — provider names, credential types,
and env var NAMES only.
Ownership rule (#48): `~/.pi` is read-only to the stack, permanently. The
only interaction is the existing read-only container mount of the default
credential (`PI_AUTH_FILE`, default `~/.pi/agent/auth.json`). Mosaic-managed
accounts live under the data root: `<dataRoot>/auth/<account>.json`, perms
0600 (mirroring `scripts/gitea-api.sh` hygiene — loose perms are flagged in
listings and refused by `--auth`).
| Command | Purpose | Notes |
|---|---|---|
| `scripts/auth.sh status` | Report both credential sources | Default harness credential (read-only) + mosaic-managed accounts; never prints material |
| `scripts/auth.sh accounts` | List mosaic-managed accounts | Under the data root; marks the active one; flags non-0600 |
`agent.sh --auth <account>` injects `<dataRoot>/auth/<account>.json` as the
launch's `PI_AUTH_FILE`; missing/symlinked/non-0600 accounts refuse.
Headless task runs keep the default credential.
## Release
| Command | Purpose | Notes |
|---|---|---|
| `scripts/release.sh package` | Build + tag the release image | Tag: `mosaic-poc-agent:<pi>-r<release>` |
| `scripts/release.sh activate` | Health gate → atomic pointer swap | `--fault-injection` proves the refusal path |
| `scripts/release.sh rollback` | Health-gated return to previous | Refuses if image missing |
| `scripts/release.sh status` | Release, tag, active pointer, log | Safe on empty state |
| `scripts/release.sh ensure` | Self-determination: align active pointer to `RELEASE` | Fast path restores a missing/mismatched pointer without a gate; slow path packages + health-gates first. Invoked automatically at launch |
## Conductor (worker patches)
```bash
scripts/conductor-apply.sh <runId> [--dry-run]
```
Auto-applies a worker's patch under `roles/conductor-policy.json`:
succeeded run → clean target tree → path allowlist → syntax gates →
apply → policy suites → attribution commit. Any failure reverts.
Push is never automatic.
## Conductor (landing approved work)
```bash
scripts/land-work.sh <decision-id> [--base refactor] [--dry-run] [--subject TEXT]
[--attribution LINE] [--business ID] [--project ID]
```
Turns a merge decision the human resolved into one gated local commit on
`refactor` (#1610). It reads `<dataRoot>/bus/bus.sqlite` read-only through
`scripts/land-work/read.mjs` and checks that the human picked the raise's
approval option, that the question names one `work/vikunja-<p>-<t>`, and
that the task's latest `review.verdict` approves a commit C that the work
ref in the working repository still points at. It then applies
`git diff $(merge-base refactor C) C` in a detached worktree, checks the
patch-id, runs every package suite except `queue` and every
`scripts/test-*.sh`, and commits. The subject is the approval option's
text unless `--subject` overrides it, and `--attribution` replaces the last
line, `Applied-under: scripts/land-work.sh (conductor)`. `--business` and
`--project` are needed only when the config has more than one business file
or the business more than one project. The tool prints the line for the PM
and doesn't send it. `--dry-run` runs every check and the gate and writes
no ref. A refusal names its reason and exits 2; the reasons are listed at
the top of the script. The worktree, its `node_modules` link and its Docker
network are removed on every exit. The gate log directory stays and the
tool names it. Keep `TMPDIR` short: the gate inherits it, the cli suite
puts unix sockets under it, and a 46-byte `TMPDIR` failed 8 of those tests.
Push is a separate act.
Since #1631 the gate, its steps and the root-file rule live in
`packages/seat/src/landing.mjs`, and the decision checks in
`packages/bus/src/landing.mjs`. The bus host's `land` op uses both to land
an agent's task on its working repository's `main`, never on `refactor`
(`packages/cli/README.md`, "Git socket"). Since #1651 the host starts the
holder's landing run itself ("Landing runs"). A decision raised for one target
can't land the other. land-work still accepts only a decision the human
resolved through the CLI.
## Maintenance
| Command | Purpose | Notes |
|---|---|---|
| `scripts/reset.sh` | Delete the data root | Triple-safety-checked (path, symlink, ownership marker) |
| `scripts/test-config.sh` | Config selftests (no Docker) | 24 cases |
| `scripts/test-task.sh` | Task selftests + live cases | 90 cases |
| `scripts/test-release.sh` | Release selftests | 14 cases |
| `scripts/test-conductor.sh` | Auto-apply selftests (sandboxed) | 17 cases |
| `scripts/test-auth.sh` | Auth checkpoint selftests (no Docker) | 13 cases |
| `scripts/test-land-work.sh` | Land-work selftests (sandboxed, fake Docker) | 84 cases |
| `scripts/gitea-api.sh <METHOD> <path> [body]` | Gitea API helper; reads the JSON credential file or a raw per-seat token file | Token never on argv/stdout |
## Seat launch with registration (`scripts/mosaic`)
```bash
scripts/mosaic launch <seat|seat-dir> [--task TEXT] [--project NAME] [--workspace PATH] [--harness NAME] [--repo PATH] [--config PATH] [-- args...]
scripts/mosaic seat task <seat> <text> [--by NAME] [--layout repo|fleet|unknown] [--config PATH]
```
Runs a seat's `launch.sh` unchanged (arguments after `--` pass through) and
leaves one registration record at `<dataRoot>/seats/<layout>/<seat>/registration.json`
that the control board reads: seat, project, task, workspace, tmux session,
harness, start time, pid. Rewritten on every launch. `seat task` changes the
task field only. The four repository seats (`agents/*/launch.sh`) register
themselves through this path unless called with `--check`. Run it by
path from the checkout. On a pinned host, `mosaic` on PATH is a wrapper
around a pinned copy's `scripts/mosaic`
([Stack copies](#stack-copies-scriptsstack-copysh)) and carries the human
commands; on Jason's host the estate CLI it replaced is `mosaic-old`.
Queue commands stay on `scripts/mosaic queue` in the canonical checkout,
because a copy refuses them ([One pinned
copy](../packages/cli/README.md#one-pinned-copy-row-85-1579)). Exit
codes: the launch script's own once it runs; before that 1 could not
start, 2 invalid config or seat, 4 usage. Details and the record's
fields: `packages/seat/README.md`.
## Work queue (`scripts/mosaic queue`)
```bash
scripts/mosaic queue list | show ID | next [SEAT]
scripts/mosaic queue add|move|release|assign|note|set ... --op ID [--by NAME]
scripts/mosaic queue verify [--current] | render [--check] | sync | unlock [--check-gate]
scripts/mosaic queue review request|resolve|abandon|record ... --op ID [--by NAME]
scripts/mosaic queue review verify-commit ID REF
scripts/queue-commit.sh -m MSG
```
`docs/plans/queue.json` holds the rows and an append-only log; the table in
`docs/plans/QUEUE.md` between the `mosaic-queue` markers is rendered from it.
Canonical checkout only. Every change needs an `--op ID` chosen before the
first attempt and reused on retry; only an op whose `ok <op> rev N` receipt
printed is done. The lead commits queue changes with `scripts/queue-commit.sh`;
the pre-commit guard refuses any other commit that stages the two queue files.
Moving a row with reviewers to in-review posts one review request comment
on its issue as the acting seat, with the seat's own token file in
`MOSAIC_GITEA_CREDENTIAL_FILE`; reviewers then run `review record`.
Exit codes: `0` ok · `1` failed · `2` invalid or refused · `3` uncertain,
retry the same op (for a review request, the retry sends nothing; resolve or
abandon it) · `4` usage. Details: `packages/queue/README.md`.
## Organization files and role instances (`scripts/mosaic org config`, `scripts/mosaic org instances`)
```bash
scripts/mosaic org config validate <org>
scripts/mosaic org config resolve <org> <instance> [--project <id>]
scripts/mosaic org instances <org> [--json]
scripts/mosaic org instance add <org> <instance> --definition <def> [--vars <file>] [--launch] --revision <rev>
scripts/mosaic org instance set <org> <instance> [--definition <def>] [--vars <file>] [--launch | --no-launch] --revision <rev>
```
`scripts/mosaic business validate|resolve|instances|instance|veto` is the old
spelling of these commands, with the same arguments, and it still works. Both
spellings run the same code and give the same status, stdout and stderr.
Only a validate/resolve usage error differs: it prints the usage of the
spelling you used. Every other validate/resolve stderr line, refusals and
warnings included, starts `mosaic business: ` in both spellings. `scripts/mosaic org validate FILE` is a different command: it
checks a draft file by path and prints `valid`.
Reads `businesses/<business>.json` next to the system config
(`$MOSAIC_CONFIG` or `~/.config/mosaic-dev/`), the version 2 role files in
`roles/` (or `$MOSAIC_ROLES_DIR`) and each project's `.mosaic/project.json`.
Writes nothing. `validate` checks the whole business, stat-checks every
credential reference without opening it and prints one digest per role
instance. `resolve` prints one instance's merged variables, their sources,
its narrowed tools, network and authority, and its credential references.
The business file must belong to you and not be group or other writable.
Exit codes: `0` ok · `2` invalid file or credential reference · `3` system
config problem · `4` usage or a required file missing. Details:
`packages/organization/README.md`.
`mosaic credentials status|issue|rotate|revoke|delete` (row 199) goes to
`packages/cli` and the bus's human-only credential verbs: an agent's own
token in `<configDir>/secrets/agents/<business>/<agent>/<kind>`, read on
stdin, never printed. The tester may run `status` and `revoke`. Details:
`packages/cli/README.md`.
`instances` and `instance add|set` (row 165) go to `packages/cli` and the
bus's human-only `instances` and `instances.set` verbs, so they need a live
bus host and refuse inside an agent session. `instances` prints the revision,
each instance and the definitions `add` and `set` accept. `add` and `set`
write the business file under that revision and log the change with you as
the actor. They bind no credentials: a definition that declares credentials,
or a definition change on an instance that holds them, is refused with
`organization-credentials-out-of-scope` and stays a hand edit. After an add
or a definition change the output says a bus restart is needed; the restart
needs Mos's go or Jason's hand, and the command doesn't run it. Exit codes:
`0` ok · `2` refused, nothing written · `3` human proof refused · `4` usage.
The Console's Settings page has the same two forms. Details:
`packages/cli/README.md`.
## Inbox, decisions and the bus host (`scripts/mosaic`)
```bash
scripts/mosaic # no argument: the full-screen app
scripts/mosaic inbox | tasks | agents [--business <id>] [--json]
scripts/mosaic inbox --read <id>... | --read-all [--business <id>]
scripts/mosaic decide <decision> <option> [--note <text>] [--yes]
scripts/mosaic trail <task|decision> [--json]
scripts/mosaic bus start <business> [--pm | --no-launch] [--dev-local-owner] [--detach] | stop | restart | recover | status [--json]
scripts/mosaic console start [--dev-local-owner [--dev-insecure-bind ADDR --dev-allow-host NAME[,NAME...]]] [--detach] [--port auto|N]
scripts/mosaic console stop | status | restart
scripts/mosaic org reload <org>
scripts/mosaic talk <role> <text> [--wait SECONDS] [--business <id>]
scripts/mosaic project create <name> [--repo <repo>] | list [--json]
scripts/mosaic threads [--project <project>] [--json] [--tester]
scripts/mosaic thread create <project> <name> | open <thread> | read <thread> [--json] [--tester]
scripts/mosaic talk --thread <thread> <text> [--wait SECONDS | --follow] [--tester]
scripts/mosaic follow --thread <thread> [--tester]
scripts/mosaic stop <run>
scripts/mosaic launches off | on [--business <id>]
scripts/mosaic launches list [--json]
scripts/mosaic halt | resume [--business <id>]
scripts/mosaic processing status [--business <id>] [--json]
scripts/bus-service.sh [--instance NAME] render [--copy ID [--copies DIR]] [--pm] [--claude PATH]
scripts/bus-service.sh [--instance NAME] install [--copy ID [--copies DIR]] [--pm] [--dir DIR] [--no-reload] [--claude PATH]
scripts/bus-service.sh [--instance NAME] uninstall | status <business>
```
For local development on a named instance only, `mosaic bus start BUSINESS --dev-local-owner` admits the fixed local-dev Console owner and refuses the default instance. Launches are enabled unless `--no-launch` is supplied; `--pm` can start the business's PM. Launched agents run as this OS user and can reach the loopback Console as owner; use this instance for UI testing only. Start the matching Console with `--dev-local-owner`; the usual Console wrapper, process proof and nonce checks still apply. This mode has no sign-in and is for local development only.
`mosaic console start --dev-local-owner --dev-insecure-bind ADDR --dev-allow-host NAME[,NAME...]` opens that dev owner Console to other machines, for UI testing from another device on a trusted LAN. It has no sign-in: anyone who can reach ADDR:PORT acts as the owner. Start, and `mosaic console status` while that Console runs, print `WARNING: dev owner Console listens on ADDR:PORT with no sign-in; anyone who can reach it acts as the owner.` on stderr. ADDR is `0.0.0.0` or an IPv4 address, and a specific address must also be one of the names. NAME is a host name or IPv4 address without a port; give several as one comma-separated value. The Console answers only `NAME:<bound port>` and the loopback names, and takes a write only when Origin is `http://` plus that same Host. A browser on such an http origin sends no Fetch Metadata, so for an allowed NAME a write with that exact Origin and no `Sec-Fetch-Site` counts as same-origin; a `Sec-Fetch-Site` the browser did send is still checked. Both flags are refused without `--dev-local-owner`, and `--dev-insecure-bind` is refused without a `--dev-allow-host`. SSO, preview and tester Consoles stay on loopback. The Console state file records `bind` and `allowHosts`. `mosaic console status` prints the state as JSON (`bind` is `127.0.0.1` for a loopback Console) and never moves a stale state file.
`mosaic console restart` and `mosaic org reload ORG` are the Console's Settings controls (F3) from a terminal (U1, #1873). Each sends one request to the running Console named in the instance's console state, as the page sends it (the Console's own Host and Origin, `Sec-Fetch-Site: same-origin`, a JSON `{}` body), so the Console applies the same owner, `--send` and origin checks and writes the same action trail, `<dataRoot>/console/actions.jsonl`. Both refuse inside an agent run (exit 3) before reading anything, and refuse with exit 3 when no Console runs. `console restart` works only for a Console started in the foreground by `mosaic console start`, or by `mosaic console start --dev-local-owner --detach`, which leaves a detached supervisor (`packages/cli/src/console-supervisor.mjs`, its own session) running the Console as the foreground start does; its receipt adds the supervisor's `supervisor` pid, and its later warnings go to `console/supervisor-*.log` under the data root. `mosaic console stop` stops that Console, and its supervisor then exits. `mosaic console start` refuses inside an agent run (exit 3), with or without `--dev-local-owner` or `--detach`, checking the starting process's own environment as well as the one given for the Console (agent-guard, #1873). A detached start whose supervisor does not reply within 60 seconds signals the supervisor's process group, then kills it, and clears `console/start.lock` only when the lock names that supervisor (pid and start time) and it is gone. Any other Console answers `not-supervised`. `console restart` waits up to 30 seconds for the relaunched Console to answer and prints `{"outcome":"restarted","from":OLD,"boot":NEW}`. `org reload ORG` restarts a running bus or starts a stopped one and prints the Console's done receipt. A request that left but got no answer is reported as outcome unknown (exit 1) and is never retried; check `mosaic console status`, `mosaic bus status ORG` and the trail. `mosaic bus restart ORG` remains the trail-less host verb.
The bus and Console lifecycle verbs refuse inside an agent run (agent-guard, #1873): `mosaic bus start`, `bus stop`, `bus restart` and `bus recover`, and `mosaic console start`, `console stop` and `console restart`, and `mosaic org archive` and `org delete`, which stop the organization's bus. Each exits 3 with `refused: this runs only from a human shell, outside any agent run (found MARKER in the environment)` before it reads anything. It checks the environment it is given and its own process environment, so a sanitized environment does not hide a marker. A usage error (a missing or extra argument, `--pm` with `--no-launch`, or a bad `--dev-insecure-bind`/`--dev-allow-host`) still comes first (exit 4). `mosaic bus status` and `mosaic console status` stay open to agents. The Console's Restart bus and Restart Console controls keep working: the Console starts its bus host with its own environment, which carries no marker, and Restart Console relaunches the Console under its supervisor without going through the CLI. The markers are the ones `packages/cli/src/transport.mjs` lists. The check is cooperative: a process that drops the markers from its environment is not stopped by it.
The instance processing switch is separate from per-business launches off/on.
`mosaic halt` durably stops managed processing across every business in the
selected configuration's data root. The bus stays up for reads and human
recovery. `mosaic processing status --json` reports settled, pending or unknown
stops; `mosaic resume` checks the current generation and refuses pending stops.
Only the existing human proof can write it, independently of channel settings.
A missing/invalid state is never off: use halt, then resume; an unsafe control
parent needs operator repair. Unknown task/remote outcomes are not retried.
Native diagnostic seats remain available. First activation must stop old
processing before trusted initialization and align the image/read-only control
mount. This is a separate approved window, never a build step.
The human commands read and resolve through the broker's human transport
(`packages/bus/src/human-cli.mjs`), never the SQLite file. They refuse inside
an agent run, except the forms that take `--tester` (rows 110, 175 and 184), which
run as the business's tester; `packages/cli/README.md` lists them. `bus start` is the host the systemd user unit
`mosaic-bus@<business>` runs. It boots the broker, starts the Discord
notifier, and needs `<dataRoot>/notify/<business>/notify.json`. That
directory must be 0700: it also holds the notifier's journal `sent.jsonl`
(0600, never a symlink). On open the notifier copies a torn final line to
`torn-<UTC stamp>.bin` and truncates the journal to its last newline; a
malformed complete line refuses with exit 3.
`bus-service.sh` pins the Claude Code binary managed sessions run as the
unit's `MOSAIC_CLAUDE_BIN`, from `--claude PATH` or else `command -v claude`
(`packages/cli/README.md`, "Unit").
`bus start` also serves the session launcher (slice 1 S6): a PM session's
`launch` tool starts role sessions through it, each a runner under its own
PID namespace with a generated bundle, and `--pm` launches the business's
`launch.by` instance at start. A host that starts ends the launches a
crashed host left as `host-lost` and refuses with exit 3 on a
`sessions.json` it can't read. Every 2 s it also starts each launch-block
instance that holds an approved, unused merge decision, once per decision,
with a landing turn that calls `land_work`; a holder run that started
before the approval is stopped between turns first (`land-relaunch` in the
launch log; row 156, cli README "Landing runs"). `talk` sends a REQUEST and prints only the
reply to it, marking only that read (polls every 2 s, `--wait` default 900;
`0` only sends); other messages stay unread for `mosaic inbox`, and
`--follow` and `follow` print and read everything. `inbox --read <id>...`
marks unread messages read by the ids `inbox` prints, and `--read-all` marks
every one it lists (row 150); the cli README, "Human commands", says how
refusals print. When no run holds the
role, a line after `sent` says whether `launch.by` was told, or that the role
isn't in the launch block and `launch.by` can't launch it (row 152); the cli
README, "Human commands", lists the lines. `stop` sends SIGTERM to
a run's runner after checking it in
`sessions.json`. `launches off|on` revokes and restores `role.launch` for
the business; running sessions keep running. `launches list` reads
`sessions.json` and needs no broker. `talk`, `stop` and `launches off|on`
refuse inside an agent run.
With no argument on a terminal, `scripts/mosaic` opens the full-screen app:
Inbox, Tasks, Agents, Trail and Help, deciding through the same flow and
strings as `decide`. In an agent run it exits 3; without a terminal on
stdin and stdout it exits 4 with the usage.
Exit codes: `0` ok · `1` failed or outcome unknown · `2` invalid input ·
`3` refused or config problem · `4` usage. Details: `packages/cli/README.md`.
## Stack instances (`--instance NAME`, row 190)
Several stacks can run on one host, each with its own config, data root,
units, Console port and CLI wrapper. The default instance is today's stack
and keeps every name it had. A named instance is created once:
```
scripts/bootstrap.sh --instance NAME
```
NAME matches `^[a-z][a-z0-9]{1,15}$`; anything else exits 4
(`instance-name-invalid`), and `--instance dev` is the default. Bootstrap
writes `~/.config/mosaic-NAME/config.json` (0644) and claims `~/.mosaic-NAME`
with the marker `instance.json`, `{"instanceVersion":1,"name":"NAME"}`. It
creates either directory 0700; one that already exists keeps its mode. It refuses with exit 3 and writes
nothing when the root exists, is not empty and holds no marker naming NAME.
A second run changes nothing. `~` is the home of the OS user that runs the
instance.
`--instance NAME` is taken, once and in any position, by `bootstrap.sh`,
`bus-service.sh`, `webui-service.sh` (all but `copy`), `discord-service.sh`,
`stack-copy.sh` (`wrapper` and `check`) and `stack-release.sh`. Copies serve
every instance, so both `copy` verbs refuse it with exit 4. Inside a process the config
travels as `MOSAIC_CONFIG`. Selection, checked before any read or write:
| Given | Config | Names |
|---|---|---|
| neither | `~/.config/mosaic-dev/config.json` | the default's |
| `MOSAIC_CONFIG` only | that file | the default's |
| `--instance NAME` only | `~/.config/mosaic-NAME/config.json` | NAME's |
| both | must be one file (a symlink to it agrees), else exit 4 `instance-config-mismatch` | NAME's |
Every selection checks the data root's marker. A named instance whose root
lacks its own marker refuses `instance-root-unclaimed`; a root whose marker
names another instance refuses `instance-root-mismatch`, also when a config
reaches it with no flag. Both exit 3 before anything is read or changed. A
missing config exits 3 with the bootstrap command that creates it.
| | Default | Instance NAME |
|---|---|---|
| bus unit | `[email protected]` | `[email protected]` |
| Console unit | `mosaic-webui.service` | `mosaic-webui-NAME.service` |
| Discord unit | `[email protected]` | `[email protected]` |
| CLI wrapper | `$BIN/mosaic` (Q81 marker) | `$BIN/mosaic-NAME`, marker `# mosaic-cli wrapper, instance NAME (row 190)`, exports `MOSAIC_CONFIG` |
| bus session env | `%h/.config/mosaic-dev/secrets/claude-session.env` | `~/.config/mosaic-NAME/secrets/claude-session.env` |
| Console port | `--port`, 7332 when absent | `--port` required; 7330, 7332 and 7333 exit 4 `port-reserved` |
A named instance's units add `Environment=MOSAIC_CONFIG=<its config>` and
` [NAME]` after the Description, and the Console unit's comment on the
variables it sets names `MOSAIC_CONFIG`. The default's renders are
byte-identical to the templates' renders before row 190. `stack-copy.sh
wrapper --instance NAME` creates `$BIN/mosaic-NAME` or replaces one that
carries NAME's marker, and refuses any other file. `check --instance NAME`
needs `--business ID`, and fails with `CLI: <copy> (MOSAIC_CONFIG mismatch)`
when the wrapper does not export NAME's config.
For every instance, a Console `--port` or `--tester-port` that another
installed Console unit in the unit directory already takes refuses
`port-claimed`, naming that unit: `webui-service.sh install` exits 1, and
`stack-release.sh activate` and `rollback` exit 3 before anything stops
(`--plan` lists it). A unit with no `--port` holds 7332.
`stack-release.sh --instance NAME` reads only NAME's pointer, log, store and
backups. Its busy check sees only NAME's bus units. Its drop-in scan adds
`mosaic-bus-NAME@*.service.d` and the shared `mosaic-bus-.service.d`. Its
rollback set holds NAME's two unit files, their own `.d` directories and
`$BIN/mosaic-NAME`; it never captures or restores the shared prefix
directories (`mosaic-bus-.service.d`, `mosaic-webui-.service.d`,
`mosaic-discord-.service.d`). The live-session guard refuses a path inside
any named instance's config directory or `~/.mosaic-NAME`, even when that
instance's config is unreadable.
All instances run under one OS user for now. That user's processes can
read and control every instance; a separate user per instance is row 195.
A real second instance on this host is a Q173 window, and tunnel or Traefik
exposure of a named Console waits for Jason after Q175 SSO.
## Stack copies (`scripts/stack-copy.sh`)
```
scripts/stack-copy.sh copy COMMIT [--copies DIR] [--allow-worktree]
scripts/stack-copy.sh copy --bundle FILE SHA [--copies DIR]
scripts/stack-copy.sh wrapper ID [--instance NAME] [--copies DIR] [--bin DIR] [--create]
scripts/stack-copy.sh check [--instance NAME] [--copies DIR] [--bin DIR] [--business ID] [--units user|external]
```
One read-only tree supplies the bus, Console and human CLI (row 85,
#1579). Copies default to `${XDG_DATA_HOME:-$HOME/.local/share}/mosaic-stack/copies/<full id>`.
`copy` requires a canonical checkout (.git directory); --allow-worktree
is a test override. COMMIT must be reachable from both local refactor and
origin/refactor. It archives the runtime/import closure, adapters, roles,
`tools/unslop-hook` (the unslop check both harness modules mount, row 66),
RELEASE, package.json and package-lock.json; refuses missing manifests;
runs npm ci inside the temporary copy; removes write permissions; and
renames beside the target. A published existing copy stays unchanged.
Dependencies use the exact lock and carry Pi's installed lock record.
Failure publishes nothing and removes the temporary tree.
`copy --bundle FILE SHA` (row 232) is for a host with no refactor checkout.
SHA is a full 40-character id. The bundle must pass `git bundle verify`,
carry `refs/heads/refactor`, and reach SHA from it; each miss refuses and
writes nothing. The copy is then made from SHA as `copy` makes it, and the
script prints `release:` with the RELEASE value at SHA and `bundle sha256:`
with the bundle's digest. That pair is the release identity on such a host.
Without `--bundle`, `copy`'s reachability rule is unchanged.
`wrapper` is a live switching step, reserved for Jason or Mos. It accepts
only an existing copy and a regular non-symlink mosaic wrapper with the
Q81 marker on line 2 and its exact lease line to mosaic-old. It preserves
both lines, adds comments naming the copy root and Q81's mosaic-old lease
probe, and atomically replaces the wrapper's final exec with the
copy's scripts/mosaic. --bin is a fixture override; the default is
~/.npm-global/bin. Neither the Q81 worktree nor mosaic-old is removed.
`wrapper ID --create` (row 232) writes the first wrapper on a new host, the
same file `wrapper` writes. It refuses when anything is at the wrapper path,
a symlink included, and never replaces it. Without `--create`, a missing
wrapper still refuses.
`check` changes nothing: it reads effective systemd WorkingDirectory and
ExecStart (including drop-ins) for the bus and Console, plus the wrapper
exec. It prints all three paths and succeeds only on the same existing
copy; absent/failed reads and a mismatched ExecStart fail. Business defaults
to mosaic-stack. `check --units external` (row 232) reads root's system units
with `systemctl show`, without `--user`. Queue work continues on the canonical scripts/mosaic.
Bus render/install accept --copy ID, --copies DIR, --pm and the unchanged
--claude pin. Without --copy the rendered unit stays the checkout unit.
--pm adds the PM startup; a refusal exits 3, and RestartPreventExitStatus
prevents a retry. Install refuses any busy bus instance, even with
--no-reload; a failed state/drop-in read also refuses. Any template or
instance drop-in, or inherited mosaic-.service.d/ or service.d/ drop-in,
assigning ExecStart or WorkingDirectory refuses,
naming the file. Status reads effective roots, --pm and DropInPaths,
reports checkout or copy identity and refactor drift, and writes nothing.
### Pinned stack update
```
scripts/stack-release.sh adopt BUSINESS
scripts/stack-release.sh activate ID BUSINESS [--pm | --pm-after-harness] [--migrate] [--tester-socket | --tester-port N (legacy copies)]
[--proof-header NAME --proof-file PATH | --no-proof | --sso | --no-sso] [--send | --no-send] [--timeout S] [--plan]
scripts/stack-release.sh pm-on BUSINESS [--timeout S]
scripts/stack-release.sh rollback BUSINESS [--restore ACTIVATION [--events N]] [--timeout S] [--plan]
scripts/stack-release.sh status BUSINESS
scripts/stack-release.sh confirm ID BUSINESS --units external [--rollback OLD --cause A3|A4|A5] [--timeout S]
scripts/stack-release.sh revert ID BUSINESS --units external
scripts/stack-release.sh gate ID BUSINESS --units external --since EPOCH [--timeout S]
common options: --copies DIR, --bin DIR, --instance NAME, --units user|external
--events N is the total of the per-table counts the post-backup-events refusal prints: events and
every table in COUNTED.
--send needs --sso in the run's Console flags, given or carried. --no-sso needs --sso in the
pointer's, and takes the Console flags from the rollback set of the activation that turned it on.
--no-sso drops the pointer's --send, and so does a rollback to a Console without --sso.
An inherited legacy --send beside header-preview flags is dropped during planning because preview is
read-only; it never reaches the service installer. The drop alone is not a change: an activate of the
pointer's own copy refuses already-active unless another flag changes the Console (check_target's
list), so on a legacy pointer pass --no-send there.
```
This is an operator plan, not an action agents run. Both reviews and the
landing come before it. Mos grants each live run first. A bus restart ends
running sessions, so run it only when no PM turn is in flight. `--plan` and
`status` change nothing and need no go.
The script keeps `<dataRoot>/state/stack-active.json` (the live copy, business,
PM flag, claude pin, Console flags, schema and activation) and appends every
run to `<dataRoot>/state/stack-activation-log.jsonl`: a `started` line, then
one final line with `result` `ok`, `refused`, `gate-failed`,
`rollback-failed`, `pm-on-failed` or `pm-on-down`. A refusal exits 3, a gate
failure that rolled back exits 1, a failed rollback exits 2, usage exits 4. Only one run at a time holds
`state/stack-release.lock` (`run-locked`).
An activation id is the UTC second and the target copy's first 12 characters,
`20261020T013708Z-89af38a83afa`. A second run in the same second adds `-1`,
then `-2` up to `-9`, the first one with no rollback set, backup or
post-attempt store yet, so ids still sort by time. With all ten taken the run
refuses `capture-failed` before it stops anything. Adopt uses the same rule.
1. Once per host, `adopt BUSINESS` records the running stack as the pointer:
the copy `stack-copy.sh check` reports, the bus unit's `--pm` and
`MOSAIC_CLAUDE_BIN`, the Console's flags without `--checkout`, and the
copy's schema version. It refuses when the check fails, the bus unit has no
`MOSAIC_CLAUDE_BIN` or a pointer exists.
2. Make the pushed target copy with `stack-copy.sh copy COMMIT` and note its
full id.
3. Run `activate ID BUSINESS --plan` and take the output to Mos. It names the
store's schema and the target's, any migration with `--migrate` and the
decision 96 condition 2 line (Jason present), the backup, the commits the
run adds, and the three install commands. The `backup:` line gives the
directory the run backs the stopped store up to, or why it takes none:
there is no store file, or the run rolls back with `store-newer` or
`migration-needs-flag` before the backup. It refuses `copies-unrelated`
when neither copy is an ancestor of the other.
4. Mos removes any bus drop-in that overrides ExecStart or WorkingDirectory;
`activate` refuses `drop-in-override` and names the file.
5. With Mos's go, `activate ID BUSINESS`, plus `--migrate` when the plan
says so. Before it stops anything, it refuses a missing, unpushed or
unreadable copy, a busy `mosaic-bus@` instance, unit flags that differ
from the pointer, and a business file that names a plug-in kind the
target's `KINDS` lacks (`unknown-kind-for-target`, naming each kind). It
refuses `business-unreadable` when the target's code can't read the
business file: a copy older than eaeb5ebf2, which has no
`plugins/src/kinds.mjs`, or a file the target's loader refuses. Then it
saves the unit files, drop-ins and wrapper under
`state/stack-rollback/<activation>/`, stops the Console then the bus, and
reads the stopped store. A `-wal` or `-shm` beside it refuses
`store-unclean` and leaves the store alone. Every activate then takes an
owner-only backup of the stopped store under
`<dataRoot>/backups/stack/<activation>/`, before any migration (row 185),
and the log line names it. A missing store file takes no backup. A store
outside the target's chain (`store-newer`) or a migration without
`--migrate` rolls back before the backup.
It installs the bus, the Console and the wrapper from the pointer's flags,
starts the bus then the Console, and runs the gate: both units active, a
live bus host for this start, Console `/healthz`, `stack-copy.sh check`
and, with `--pm`, exactly one `launched` journal line for a live run of
this start. A pass writes the pointer.
6. A gate failure puts back the saved units and wrapper. The backup is
taken on every activate, but it is restored only when a migration ran.
A restore keeps the store it replaced as `bus.post-attempt-<stamp>`, where
the stamp is the activation's second and its suffix if it has one. After
a same-schema activate the store stays as the failed run left it. Then it
starts the old copy and gates it. If that gate
fails too, both units stay stopped, the log says `rollback-failed`, and
Jason or Mos settles it from the rollback set the message names.
7. `rollback BUSINESS` returns to the copy before the pointer's, taken from
the log. When the schemas differ it needs `--restore ACTIVATION`, with or
without a `-1` to `-9` suffix. A same-schema rollback can take
`--restore` too, with the backup the activate took. Rows written after that backup need
`--events N`, and Jason decides first. The count covers events and every
table in `COUNTED` (row 181), the tables a bus writer fills without an
event. The `post-backup-events` refusal prints one count per table with
rows, in that order, for example `1 event(s), 1 instance_names row(s)`.
N is the total of those printed counts, read off the refusal. It refuses
`business-unreadable` and `unknown-kind-for-target` like `activate`. For the second, set each kind
it names to `null` through the live bus first, with the `plugins set`
command in step 2 of the next section.
8. Jason's plain `mosaic talk pm "..."` must print sent and a reply. Keep the
`status` output with the receipts.
The Console's row 178 proof pair gets into the pointer only through `activate`
(row 187). `--proof-header NAME --proof-file PATH` puts the pair in the
pointer's Console flags and the unit after the preview flags and before `--send`
and the tester flag (`--tester-socket`, or `--tester-port` for legacy copies),
where `webui-service.sh` renders it, replacing any pair
already there. `--no-proof` drops it. Either may name the live copy, which a
plain activate refuses with `already-active`, so the proof goes on or off
without changing copies; `--no-proof` with no pair in the pointer still refuses
there. Later activates, `pm-on` and rollbacks carry the pair unchanged. A
rollback to a copy whose Console lacks `--proof-header` drops the pair and says
so in the steps log. An activate to such a copy refuses `proof-target` unless
`--no-proof` is given, so the proof goes off only when asked. Before anything
stops, the run also refuses `proof-needs-preview` when the flags it would
install lack `--public-host`, `--auth-header` or `--allow-user`,
`proof-header-invalid` on the Console's own header rules, and
`proof-file-invalid` for a path that isn't absolute, holds a character outside
`[A-Za-z0-9/._+-]`, is a symlink or not a regular file, isn't mode 600 or 400,
isn't owned by the uid the script runs as, or sits inside the repository or the
copies. It checks the file with `stat` and `realpath` and never reads the value.
`--plan` prints one `proof:` line: `on`, `carried`, `dropped` with the reason,
or `off`. On the live copy it also notes that the run re-pins the pointer's copy
and only the Console flags change.
`activate ... --no-send` (row 189, mos-0972) drops `--send` from the pointer's
Console flags and the unit, so the Console serves reads and no human sends. It
may name the live copy while the pointer holds `--send`; once `--send` is gone a
same-copy `--no-send` refuses `already-active`. Independently, an activate or
rollback drops an inherited legacy `--send` when the run has no `--sso`, before
the plan reaches `webui-service.sh`; verified `--sso --send` remains carried.
Later activates, `pm-on` and rollbacks keep send off. Only an explicit `--send`
beside given or carried `--sso` puts it back.
`--plan` prints one `send:` line: `dropped`, `off` with or without the
`--no-send` note, or `carried`.
`activate ... --sso` (row 200, #1698) starts the Console with Q175 sign-in
(row 192). It drops `--public-host`, `--auth-header`, `--allow-user` and any
proof pair from the pointer's Console flags and the unit, and puts `--sso` after
the other flags, before `--send` and `--tester-port`. It may name the live copy
once, so a copy already at row 192 or later turns sign-in on without a copy
change. Later activates, `pm-on` and rollbacks to a copy with `--sso` carry it.
`--sso` with a proof pair or `--no-proof`, or on any other verb, is a usage
error. Before anything stops, a run whose Console flags hold `--sso` refuses
`sso-target` for a target Console without it, `sso-config` when the target's own
`checkConsole` refuses the config's `console` key (or there is none, in which
case, from another copy, it names the first step: activate the target without
`--sso`, then write `console.sso`; on a named instance both commands carry
`--instance NAME`), and `sso-slot` for each provider's slot file
that is missing, a symlink, not a regular file, not owned by this uid, has group
or other bits, resolves inside the repository or the copies, or is empty. It
uses `lstat`, `stat` and `realpath` on the slot and never reads it. `--plan`
prints one `sso:` line: `on` with what it drops and carries, `carried`, or
`dropped` on a rollback.
A rollback from an `--sso` pointer to a copy whose Console lacks `--sso` takes
the Console flags from the rollback set of the activation that moved the stack
onto the pointer's copy from the target, past any same-copy re-pin, checked
against the set's `SHA256SUMS`. `--send` and `--tester-port` follow the pointer.
When that set is missing, fails its sums, holds no readable argv or already ran
`--sso`, the run refuses `sso-rollback-flags`. Any activate or rollback also
refuses `config-console-key` before anything stops when the target's
`scripts/mosaic-config.mjs validate` refuses the config, including a non-console
key. Every existing parseable config is checked; a missing file passes. A target
without a loader accepts only the oldest top-level keys: `configVersion`,
`environment`, `dataRoot`, and `execution`; any later key refuses. The refusal
names the rejected key and says to restore the config.json backup first. A
`console` or `federation` key keeps row 203's message, which says the config
holds a console or federation key. So a
window re-pins to row 192 or later before it writes the key, and a rollback past
row 192 removes the key first.
Two more optional system config keys bound the host's compaction collections
(row 270, #1775, DECISIONS Q222). Only a human writes them, like `console`.
`compactionReadBytes` is the stream bytes one collection reads, a whole number
from 1048576 to 1073741824, default 67108864 (64 MiB). `compactionStreams` is the
stream files one collection opens, a whole number from 32 to 65536, default 1024.
The loader passes either on as written. The launcher takes the default for a
missing or bad value and logs one `system config:` line. A collection that hits
either bound counts as `not-collected`, and the next one reads on. A copy before
row 270 refuses both keys, and the release's config check doesn't catch that
(it runs the target's loader only for a `console` key). So any re-pin or
rollback plan whose OLD copy is before row 270 removes both keys first.
#### A target past c216f47db: the PM after the harness kind
From c216f47db on, the bus launches the PM only with a `harness` kind in the
business file, and no copy before it can set one. So `activate --pm` to such a
copy fails its gate and rolls back. Use two phases instead (row 169, Gitea
#1666, mos-0848):
1. `activate ID BUSINESS --pm-after-harness`, in place of `--pm`. It needs a
pointer with `pm` true. Before anything stops, it refuses
`pm-after-harness-target` when the target's `KINDS` has no `harness`, and
`pm-after-harness-set` when the business file already sets `harness`;
use `--pm` in both cases. It installs the bus without `--pm`, runs every
gate check but the PM's, and writes the pointer with `pm` false and
`pmPending` true. The log's final line is `result ok` with `pm pending`.
`status` and `--plan` print the pending state and the next step.
2. Jason sets `harness` with `plugins set` through the live bus. Copies from
before 98d29769e use `node <copy>/packages/cli/src/cli.mjs plugins set --file P
--business ID`; later copies dispatch `mosaic plugins set --file P --business ID`.
On the Q173 test system, the tester can run the latter with `--tester`.
`<copy>` is the pointer's copy, the one the bus runs from; human commands from
any other tree refuse `human-required`. P holds one patch,
`{"revision":"<64 hex>","plugins":{"<kind>":<value>|null}}`. Read the revision
with the same copy's `plugins --business ID` command (`--tester` for the tester).
A value sets the kind, null removes it, and an unnamed kind keeps its setting.
3. `pm-on BUSINESS` refuses `not-pending` without a pending pointer and
`harness-unset` unless `harness` is a non-empty list the pointer's copy
loads. It restarts the bus alone, with `--pm` on the same copy, and runs
the whole gate. A pass writes `pm` true and drops `pmPending`. A failure
puts the phase-1 bus unit back and reruns the phase-1 gate. If that
passes, the log says `pm-on-failed`, the pointer stays pending and the run
exits 1; fix `harness` and run `pm-on` again. If it fails too, both units
stop, the log says `pm-on-down`, and the run exits 2. `pm-on` and a
`pm-on-failed` run leave the Console running; `pm-on-down` stops it. None
of them touches the copy or the store.
While the pointer is pending, `activate` refuses `pm-pending`. Run `pm-on`
or `rollback`; a rollback from pending installs the old copy with `--pm` and
gates its launch.
A `started` line with no final line is an interrupted run. `status` shows
it, and `adopt`, `activate` and `rollback` refuse until Jason or Mos settles
it. Nothing prunes copies, rollback sets, backups or post-attempt stores.
After a switch, the human commands and `--tester` work only through `mosaic`
on PATH; the checkout's `scripts/mosaic` exits 3 with `human-required` or
`tester-required`, and queue writes stay on it ([packages/cli/README.md, One
pinned copy](../packages/cli/README.md#one-pinned-copy-row-85-1579)).
#### Rehearsal on a scratch root (D2, row 163)
```
scripts/stack-release.sh rehearse ID BUSINESS --backup ACTIVATION|PATH|last --port N [--timeout S]
```
`rehearse` starts the target copy's bus host and Console on a copy of a
backup and runs `activate`'s gate checks on them, all but the PM's, while the
live stack keeps running. It reads the live config and a backup of the live
store, so Jason or Mos runs it, as a window's pre-step. No development seat
runs it against live data. It stops and starts no unit and writes no pointer
or log line.
`--backup` takes an activation id under `<dataRoot>/backups/stack/` or an
absolute path. `--backup last` (row 233) takes the newest backup the
activation log records and prints `PASS backup-last` naming it. With none in
the log it fails `no-backup`, exits 3 and makes no directory. After the
backup's checks it prints `BACKUP_STORE <activation id> schema <version>`.
`--port` is the scratch Console's loopback port, 1024 to
65535. The run refuses `port-reserved` for 7332, 7333 and the pointer's
`--port` and `--tester-port`, and `port-in-use` for a port something holds;
it never picks another port. `--timeout` (default 60 seconds) bounds the
gate and each stop.
A copy marked for the private tester receives `--tester-socket` during rehearsal;
a legacy copy receives no tester flag. The socket is under the rehearsal's
scratch dataRoot, at `data/console/tester.sock`. That path is longer than the live
one. A long dataRoot can pass activation's pathname preflight and still make the
rehearsal Console refuse its socket path; the rehearsal then fails closed.
Before it makes anything, it checks the pointer, the copy (present,
reachable from refactor, a readable schema, a bus host with `--no-launch`),
the port and the backup's `SHA256SUMS`. Then it makes
`<dataRoot>/rehearsals/<UTC second>-<copy's first 12>/`, mode 0700:
- `data/bus/` holds a copy of the backup's store files, and
`data/notify/BUSINESS/notify.json` is `{"notifyVersion":1,"binding":null}`.
- `config/config.json` is the live config with `dataRoot` set to `data/`.
- `config/businesses/BUSINESS.json` is the live business file with every
`tracker.*` variable removed and every credential reference, the roles'
and `tracker.sync`'s, pointed at its own 0600 file under `placeholders/`
holding `rehearsal-placeholder-token`. `business.diff` shows the changes.
Roles come from the target copy's `roles/`, as the live bus reads them.
- `steps.log` holds the same PASS and FAIL lines the run prints, with times;
`host.log`, `console.log` and, after a migration, `migrate.log` sit beside it.
- When the live config has `console.sso` (M3, #1873), the scratch copy keeps
its provider ids, client ids, algorithms and subjects and moves the public
origin to `rehearsal.invalid`, a name that never resolves. For a stamped
target each issuer becomes `ORIGIN/ID/` at the rehearsal provider below;
otherwise it moves to `rehearsal.invalid` too. Each client secret slot is a 0600 file under `config/secrets/mosaic-stack/`
holding `rehearsal-placeholder-secret`; the live secret is never read.
A stamped target copy (one with `.stack-build.json`) serves only with `--sso`,
so its rehearsal needs the live instance's human-made SSO steps. Without a live
`console.sso` it refuses `sso-not-prepared`; without the live `accounts.json`
beside the config it refuses `accounts-not-initialized`. Both name the run-sheet
step. Otherwise it copies the live `accounts.json` into `config/`, writes
`data/.deployed` with the target copy's own `writeDeployment`, prints
`PASS deployment`, and starts the Console with `--sso`.
Before that, a stamped run starts a rehearsal provider and prints
`PASS provider-start`. It is a node process on `127.0.0.1` port 0 that serves
HTTPS with a certificate made for this run, under `provider/` (mode 0700). For
each live provider id it serves discovery and a JWKS whose public key it made
in memory for the first algorithm it can make from that provider's list. Its
authorize and token routes answer 403 to any method and query: it issues no
code and no token, and
it reads no live secret. It logs each request to `provider/requests.log`,
deletes its TLS key file once it has read it, and exits on SIGTERM or when
the rehearsal's process is gone. `provider-start` fails when it exits early,
writes no origin within the timeout, writes an origin that is not
`https://127.0.0.1:PORT`, or leaves its key file. Only the scratch Console gets
`NODE_EXTRA_CA_CERTS` naming `provider/cert.pem`; the host does not.
The scratch Console runs every startup check against that provider. The gate
adds `sso-ready`: `/healthz` must report each provider `ready`. After the gate,
`provider-discovery` requires a 200 discovery and JWKS request per provider and
no authorize or token request. The end stops the provider with the Console and
host and prints `provider-stopped`. The live `accounts.json` joins the end check
`live-config-unchanged`. The rehearsal adds no subject and no account.
A guard then loads the scratch config with the target's code and refuses
with a named check: `data-root-live` when the scratch data root resolves to
the live one, `sessions-live` when a `sessions.json` there names a running
pid (the run never deletes the file), `data-root-scratch`, `notify-off`,
`tracker-configured` when `bootConfig` still sets a tracker,
`credential-ref-live` for an env reference or a file outside `placeholders/`,
and `sso-isolated` when a `console.sso` origin is not on `rehearsal.invalid`,
an issuer is not `ORIGIN/ID/` at the rehearsal provider (not on
`rehearsal.invalid` for an unstamped target), or a slot is not its scratch
placeholder.
When the backup's schema differs from the target's, the run checks that the
target refuses the store, migrates it, opens it, migrates it again as a
no-op, and that the pointer's copy refuses the migrated store.
It starts `scripts/mosaic bus start BUSINESS --no-launch` and the Console's
`serve --port N --business BUSINESS` from the target copy, without `--send`.
The gate needs the host's `launches disabled` line, a live `bus status` host
with this run's pid and a boot from this start, the Console's address line,
`/healthz` 200 (and `sso-ready` for a stamped target), and both command lines
on the target copy. Then it checks
that the host wrote no `sessions.json` and no `launches`, stops the Console
and the host (SIGTERM, then SIGKILL and a FAIL after the timeout), checks that
the host released `writer.lock`, and compares the backup and the live config
and business files with sums it took before. The last line is
`REHEARSAL_PASS` (exit 0) or `REHEARSAL_FAIL` (exit 1), then the directory.
Usage exits 4. Nothing prunes rehearsal directories.
`bus start --no-launch` refuses every launch with `launches-disabled`,
starts no thread ticker, and leaves an earlier host's `sessions.json` and
its runs alone. No unit passes it, and `--pm` with it is a usage error.
An accounts instance runs system units (M3, #1873). Without `--units external`,
`activate`, `rollback` and `pm-on` refuse `accounts-user-unit`, before
anything stops and before `--plan` prints, when the target copy holds a
`.stack-build.json` (a stamped copy), when `accounts.json` sits beside the
config, when the config holds `console.sso`, or when `<dataRoot>/.deployed`
exists. A config that cannot be read as JSON refuses as well. The message
names the first sign found and says root renders the units with
`case2-unit.sh`. For `activate` and `rollback` it adds that the run takes
`--units external`. For `pm-on` it says the PM is a flag in root's bus unit.
So a user-unit run never turns `--sso` on any more. The `--sso`, `sso-slot`
and `--no-sso` text above describes the pointers that pre-M3 hosts left, and
the Console's own start checks the slot (`readSlot` in
`packages/webui/src/sso.mjs`). A rollback or `--no-sso` from such a pointer
runs once `console.sso` is gone and no other sign is present.
#### Root-owned units (`--units external`, row 232)
On a host whose bus and Console are root's system units, the stack user runs
`stack-release.sh` with `--units external` from its own clone of the release
bundle (sage-1561). The script then calls `systemctl show`, `is-active` and
`list-units` without `--user` and reads no journal. It starts, stops and
installs nothing. Root does those steps, and the I3 driver calls the verbs
below in order. Without `--units`, every command line and every byte of
output is what it was before row 232. `confirm`, `revert`, `gate`, `--to`
and `--expect-from` are usage errors without `--units external`.
- `adopt`, `status`, `activate ID` and `rollback --to ID` take the flag.
`rollback` then needs `--to`, and `--to` must be the copy `rollback` alone
would pick from the log, or it refuses `to-not-previous`.
- `--plan --expect-from OLD` refuses `pointer-differs` when the pointer
names a copy other than OLD.
- `pm-on` and `activate --pm-after-harness` refuse `external-pm`; the PM
flag is in root's bus unit. `--pm`, `--tester-port`, the proof flags,
`--no-proof`, `--no-send` and `--sso` are usage errors; root's Console
unit carries the Console flags.
- `activate` and `rollback` refuse `units-running` unless both units read
inactive or failed and the store is closed. They make the rollback set
with an exclusive `mkdir`, write `AWAITING-UNITS` (the target copy) first,
capture the pointer, the wrapper and both units' show output and unit file
sums, back up and migrate or restore the store, and write the wrapper.
They write no pointer and start nothing. The last line says the run is
staged and names `confirm`. A failure after the capture writes `FAILED`
with its code, logs nothing more and exits 1; the activation stays open.
- `confirm ID BUSINESS` needs exactly one `AWAITING-UNITS` for copy ID whose
activation the log holds open, or it refuses `open-activation`. It gates
the start root made and writes the pointer and an `ok` line with the
started line's op, from, to and backup.
`confirm --rollback OLD --cause A3|A4|A5` gates OLD after root put the old
units back and logs `failed` or `gate-failed` with `rollback ok` (exit 1)
or `rollback-failed` (exit 2). Either removes `AWAITING-UNITS` and `FAILED`.
- `revert ID BUSINESS` puts the saved wrapper back and, when the run had
started a migration, restores the backup taken before it. It writes no log
line and can run again.
- `gate ID BUSINESS --since EPOCH` gates a copy with no open activation,
takes no lock and writes nothing: 0 on a pass, 1 otherwise. EPOCH is root's
clock before root started the units.
The gate under `--units external` replaces the journal read. The bus unit's
`ActiveEnterTimestamp`, in whole seconds, must be at or after the started
line's second (`confirm`) or `--since` (`gate`); an older start fails
`stale-start`, so a unit that never stopped can't pass. With the PM on, the
gate reads `launches list --json` for exactly one live launch of the PM's
instance (the business's `launch.by`), for this business, whose run started at
or after the floor. `status` adds an
`awaiting units:` line for an open activation.
Under `--units external`, `activate --plan` also refuses `open-activation`
while any rollback set holds `AWAITING-UNITS`, and `migrate-required` when
the target migrates the store and the run lacks `--migrate` (row 233). Under
`--units user` the plan's bytes are unchanged.
#### Root's upgrade driver (`scripts/stack-upgrade.sh`, row 233)
```
scripts/stack-upgrade.sh upgrade --instance dev --bundle FILE --sha SHA [--migrate] [--no-rehearsal]
[--rehearse-port N] [--next-args] [--timeout S]
scripts/stack-upgrade.sh rollback --instance dev --to ID [--restore ACT [--events N]] [--next-args]
[--timeout S]
```
Root runs it from a root-owned checkout of the reviewed commit, never from a
copy, because the stack's user can write the copies. Each run needs Mos's go,
and `--migrate` needs Jason present (decision 96). The host must be prepared
first (`docs/guides/host-run-sheet.md`). Only the default instance, `dev`, is
accepted; a named instance's units are a later row.
The unit values come from `/etc/mosaic-stack/dev.args`, which I2's install
writes: one `case2-unit.sh` argument per line, the serve flags after `--`,
owned by root with no group or other write, exactly one `--copy`, and that
copy the one both installed units run. Otherwise the run refuses
`args-unreadable` or `args-differ`. The driver writes the new copy id into
that file only after `confirm` passes. For the `library` profile, an installed
Console unit whose `ExecStart` has a `--send` token after `serve` was rendered
before #385, and the current `case2-unit.sh` refuses it. `upgrade` and
`rollback` refuse `console-unit-legacy` right after reading the args file,
before any step runs, because no migration exists yet
(`docs/guides/host-run-sheet.md` section 7). A missing Console unit is not
this refusal; the later unit read refuses it `units-unreadable`, also before
anything stops.
`upgrade`, in order. Every stack user step runs as
`sudo -u USER -H env PATH=<node dir>:/usr/local/bin:/usr/bin:/bin` from the
user's clone `<home>/src/mosaic-stack`:
1. Takes `flock -n` on `/run/mosaic-stack/upgrade-dev.lock` and holds it to
the end, rollback included. A second run refuses `upgrade-running`, and
a lock it can't open refuses `lock-failed`.
2. Fetches the bundle's `refactor` into the clone and fast-forwards it to
SHA (`fetch-failed`), then `stack-copy.sh copy --bundle FILE SHA`
(`copy-failed`).
3. A1: this checkout's HEAD is SHA (`head-mismatch`), the copy exists
(`copy-missing`), `/usr/local/lib/mosaic-stack/hostcheck.sh --check`
exits 0 (`profile-drift`, row 240), both units name one copy
(`units-unreadable`, `units-disagree`) and it is the args file's
(`args-differ`), and
`activate SHA BUSINESS --units external --plan --expect-from OLD` exits 0
(`plan-refused`, which covers `open-activation`, `migrate-required` and
`pointer-differs`). Then the render checks (#1873 M3):
`case2-unit.sh --check` renders OLD from the args file and compares the
installed units with it, so a rollback could render them, then checks
SHA's render the same way. Either refusal is `render-refused`. A re-pin
that changes the serve flags fails the second check, because an
installed unit must equal the render but for the copy id (row 195); it
takes `--next-args`, below.
4. The rehearsal, `rehearse SHA BUSINESS --backup last --port 7339`
(`--rehearse-port`). Exit 0 goes on. Exit 3 means no backup in the log
and goes on only with `--no-rehearsal`, logging `rehearsal-skipped`;
without the flag it refuses `no-backup`. Exit 0 with `--no-rehearsal`
refuses `rehearsal-required`; any other exit refuses `rehearsal-failed`.
5. A2 stops the Console, then the bus, and waits for both to read inactive
or failed. A3 is `activate --units external`, A4 the render with the new
`--copy` (which also starts the bus, then the Console), A5 `confirm`.
The driver writes the new args file beside the old one before A2
(`args-failed`, exit 3) and moves it into place after A5. A failed move
exits 5 `args-failed`: the new copy is live and confirmed, the args file
still names the old one, and root moves the new file (the result line
names it) into place by hand.
`--next-args` changes the serve flags, as the SSO migration does
(`docs/guides/host-run-sheet.md` section 7). The run reads
`/etc/mosaic-stack/dev.args.next` as it reads the args file
(`args-unreadable`). It must equal the args file before `--` but for
`--copy`'s value (`next-args-differ`) and differ from it after `--`
(`next-args-same`). The target's check is `case2-unit.sh --check-new`, a
first install's checks from the next args, without the installed units'.
After A3, root moves the installed unit files into a new 0700 directory
`/etc/mosaic-stack/dev.replaced.XXXXXX` (step `units-aside`), so A4 renders
as a first install. A failure from there moves them back unchanged
(`units-back`), runs `daemon-reload` and starts the old units with no render.
After A5 the old args file is copied into the replaced directory, the next
args file is removed (step `next-args`), and the new args file, made from
the next args, moves into place. The result line names the replaced
directory. Nothing removes it.
A failure in A2 or a stage that exits 3 starts the old units again and gates
them (A2f). A later failure stops both units, runs `revert`, renders the old
copy and runs `confirm --rollback OLD --cause A3|A4|A5`. `rollback` is the
same from A1 on, with `rollback BUSINESS --to ID`, no fetch, no copy and no
rehearsal; `--restore` and `--events` pass through.
Root reads no output the stack user wrote except exit codes and the
rehearsal's `BACKUP_STORE` line, which it copies into its log after a match
against a fixed pattern (`BACKUP_STORE none` otherwise). The step log
`/var/log/mosaic-stack/upgrade-dev.log`, 0640, gets one
`<time> <step> rc=N argv: ...` line per step and a last
`<time> result exit=N <code> <text>` line.
Exit 0 ok, 1 failed with the old stack running and gated, 2 rollback-failed
with both units stopped, 3 refused with nothing stopped, 4 usage, 5
args-failed with the new stack live and confirmed. A refusal
after step 2 leaves the user's clone fast-forwarded and the new copy made. After exit
2 the activation stays open, so the next run's plan refuses until root
finishes it.
#### Root's host check (`scripts/case1-hostcheck.sh`, row 240)
```
/usr/local/lib/mosaic-stack/hostcheck.sh --check | --enforce | --upstream
```
Installed from `scripts/case1-hostcheck.sh` as `hostcheck.sh`, with
`scripts/case1-pins.sh` beside it (`docs/guides/host-run-sheet.md` section 4).
The pins are the profile's sha256, the `apparmor-profiles` version it came
from and the bubblewrap install pin; changing one is a reviewed commit.
Seven items, one `ok N` or `FAIL N` line each: the profile file matches the
pin and no package owns it; `aa-status` has `bwrap` and `unpriv_bwrap` in
enforce mode and no `mosaic-bwrap`; no other file under `/etc/apparmor.d`
names `bwrap`; a bwrap child under the units' no-new-privs reads
`bwrap//&unpriv_bwrap (enforce)`; `/usr/bin/bwrap` is `root mosaic 0750` by
the statoverride and by `stat`; the user-namespace restriction and
`fs.protected_hardlinks` are 1; `net.ipv4.ip_unprivileged_port_start` is
above the Console unit's `--port`, or above 1023 with no unit. A `note` line
says whether bubblewrap is the install pin and never fails.
`--check` exits 0 or 1; `stack-upgrade.sh` refuses `profile-drift` on 1.
`--enforce` exits 0. On a failure, or while `/etc/mosaic-stack/DRIFT` exists,
it unloads the profile if loaded, sets `/usr/bin/bwrap` to `root:root` 0700,
appends the failed items to `DRIFT` and one line to
`/var/log/mosaic-stack/hostcheck.log`. It never writes the statoverride. The
apt hook runs it after every package run, and `mosaic-hostcheck.service`
(`packages/cli/systemd/`) at boot before the bus. `--upstream` compares the
`apparmor-profiles` candidate's profile to the pin, prints any difference
and exits 0, or 1 when the download fails. Exit 2 is usage or a refusal:
root takes no `--root` (the test seam), and other users run only with it.
#### Root's acceptance run (`scripts/case1-accept.sh`, row 195)
```
case1-accept.sh --business ID --args FILE --claude-pin PIN --node-pin vN.N.N
```
Root runs it on the Case 1 host after the install window, from a root-owned
checkout. It reads the user, home, copy, node, claude and both ports from the
args file (`/etc/mosaic-stack/dev.args` on .53), which must pass
`stack-upgrade.sh`'s rules, say `--profile factory` and name `--business`.
Every command that runs the stack's code runs as
`sudo -u USER -H env PATH=<node dir>:/usr/local/bin:/usr/bin:/bin` from `/`.
Root itself reads `ss`, `/proc`, the account database and file metadata. It
runs `systemctl show` and `is-enabled`, `curl` on the Console's loopback
`/healthz`, `sudo -l`, `sudo -u jarvis bwrap --version`, `hostcheck.sh
--check`, and node to parse `managed-settings.json` for F9. F9 runs node only
when `--node` is the `--node-pin` path, and the file and every directory above
it are root's with no group or other write. The one write is F3's message and
its reply, and it never opens `claude-session.env`.
It prints one line per check, `<id> <path> <mark> <evidence>`, then
`count <mark> N` for each mark and a result line. The functional checks
F1-F11 are `pass` or `FAIL`: both units active as the user, the Console on
127.0.0.1 only, a tester message to chief answered, `case2-precheck.sh
--profile factory`, the bwrap labels of a live role session, the claude and
node pins, one live PM launch, the credential file's owner and mode, the
autoupdater off, no Discord binding. The host paths P1-P11 are `closed`,
`accepted <ruling>` (Q182, Q183, mos-1096, mos-1101) or `unnamed`, which means
no ruling names the path; each `unnamed` line goes to Mos. P9 runs
`hostcheck.sh --check`, never `--enforce`.
Exit 0 when every F check passes, 1 when one fails, 2 on usage or a refusal:
a bad args file, or `--root` misused. A P check never changes the exit.
`--root DIR` is the test seam for `packages/cli/tests/case1-accept.test.mjs`;
root takes none, and other users run only with it. Design: row 195 r7
section 5 (`agents/darkwing/work/queue-195/DESIGN.md`).
## Console service (`scripts/webui-service.sh`)
```bash
scripts/webui-service.sh copy COMMIT [--copies DIR] # make a pinned copy
# render, install, uninstall and status also take --instance NAME (row 190); copy refuses it (exit 4)
scripts/webui-service.sh render [--copy ID [--copies DIR]] [FLAGS] # print the unit
scripts/webui-service.sh install --copy ID [--copies DIR] [--dir DIR] [--no-reload] [FLAGS]
scripts/webui-service.sh uninstall [--dir DIR] [--no-reload]
scripts/webui-service.sh status [--dir DIR]
# FLAGS: [--port N] [--business ID] [--board URL]
# [--sso [--send] | --public-host NAME --auth-header HEADER --allow-user ID]
```
The systemd user unit `mosaic-webui.service` runs the Console,
`node packages/webui/src/cli.mjs serve`, with serve's flags as given:
`--port` defaults to 7332 (7330 is the operator's own Console), and the
preview set goes all three or none and stays read-only. `--send` (row 83)
requires `--sso` and is refused otherwise (exit 1, nothing written); ExecStart
then ends with `--send`, and the Console needs the system config at start
including `console.sso` (exit 2 if missing). Without `--send` the unit is
read-only. The template is
`packages/webui/systemd/mosaic-webui.service.in`.
For direct local development (not the service renderer), `node packages/webui/src/cli.mjs serve --dev-local-owner --send` uses the owner without sign-in. It requires `MOSAIC_CONFIG` naming a non-default `~/.config/mosaic-NAME/config.json`, listens on 127.0.0.1 unless `--dev-insecure-bind ADDR --dev-allow-host NAME[,NAME...]` opens it to the LAN as described for `mosaic console start` above, and requires an exact local or allowed Host and same-origin writes. It refuses SSO, preview, proof, and tester flags. Use an absolute path to the Console CLI so the bus can verify its process path. Start the named instance bus with `--dev-local-owner` as above for decision and request operations. Anyone able to reach the local listener acts as the owner.
Under `--send` the Console's Docs page (row 148) can also save an existing
spec/north-star.md or docs/NAME.md of a project: one local commit authored by the
business's human, refused `stale-base` if the file changed since the editor opened it.
It is the human bus verb `project.file.save` through the human transport; an agent
refuses it. Nothing is pushed. Without `--send` the page reads and previews only. See
`packages/webui/README.md` ("Spec and docs editor"). The page opens in a visual editor
built from reviewable source (row 154): `third_party/editor/` is unmodified upstream
source with its licenses and `PROVENANCE.json`, and `npm run build:editor` in
`packages/webui` rebuilds `src/public/editor.js` and `editor.css` with esbuild 0.28.2;
the save, the verb and the routes are unchanged. See "Visual editor" there. A save of spec/north-star.md also
sends the business's pm and cto one notice naming the commit (row 143); a BLOCK from
either goes to the chief of staff and on to the assistant. See `packages/bus/README.md`
("Spec review notices").
The unit runs a pinned copy, not the working checkout (row 77, #1571).
`copy COMMIT` resolves COMMIT to its full id, refuses one not reachable
from both `refs/heads/refactor` and `refs/remotes/origin/refactor`, and
delegates to stack-copy.sh for the archive and pinned npm install into `DIR/<id>/`, by default
`${XDG_DATA_HOME:-$HOME/.local/share}/mosaic-stack/copies/<id>/`. It writes a
temp directory beside the target, makes every file and directory read-only
(so a writer that replaces by rename can't change it either), then renames
it, so the copy's path is always whole. A copy that exists is left alone
(`exists:`). `install --copy ID` and `render --copy ID` point
`WorkingDirectory` and ExecStart at the copy and pass serve
`--checkout <this checkout>`, where the queue read still runs; `RELEASE`,
the roles, the config check and the bus child come from the copy. The bus
child is the human transport, `packages/bus/src/human-cli.mjs`, and the bus
checks that path against its own, so the bus must run from the same copy
(below). `install` refuses without `--copy`, or when the copy is missing.
`render` without `--copy` prints the unit for the checkout, as before.
`install --copy ID` also refuses unless the bus runs from the same copy
(round 2, #225). The bus accepts a human proof only from its own
`packages/bus/src/human-cli.mjs`, so a Console on a copy whose bus runs
from the checkout would fail every bus call with `human-required`. With
`--business ID`, install reads `[email protected]`; without it, every
loaded `mosaic-bus@` unit. It reads effective WorkingDirectory and ExecStart from
`systemctl --user show -p LoadState -p WorkingDirectory -p ExecStart`, so a drop-in counts,
and every ExecStart path must lie inside the copy. It refuses, naming both
paths, when one doesn't; when the unit is not installed or none is loaded;
when ExecStart has no path; and when the systemctl call fails. The check
runs before the unit state read and changes nothing.
`install` writes a temp file, renames it and runs
`systemctl --user daemon-reload`. `install` and `uninstall` refuse while the
unit is in any state but inactive or failed (`systemctl --user list-units`),
and say Jason or Mos stops it first; `--no-reload` skips only the
daemon-reload, never that check. If the state read itself fails they refuse
with one line naming the call, and change nothing. `copy` and `install`
refuse outside the canonical checkout (a `.git` directory, not a worktree's
`.git` file); `--allow-worktree` overrides that for the tests only. `status`
prints the unit state, the copy id it runs, whether `refactor` points at
that commit or another, whether the installed unit sends, and
`GET /healthz` on the installed port. It changes nothing.
Each value is held to a plain character set the unit can carry (no space,
`%`, `$`, quote, backslash, `&` or `|`), and the preview values must start
with a letter or digit; anything else is refused. With `--auth-header
X-authentik-uid`, in any letter case, `--allow-user` must be the Authentik
uid, `^[0-9a-f]{64}$` (lead decision 101). The default's unit sets no `MOSAIC_*`,
`CLAUDE*`, `CODEX*` or `PI_*` variable; a named instance's sets only
`MOSAIC_CONFIG` (row 190). Serve's exit 2 (usage, config, or a
port already in use) is not restarted.
The update/rollback sequence is [Pinned stack update](#pinned-stack-update)
above. The queue view reads the checkout and stays current. Copies remain
until an operator removes them; nothing here prunes or migrates copies.
There is no start, stop, enable or restart verb. The Console reads the bus
through the human transport, so whoever starts it holds the human's
capability. Jason starts, stops and restarts it
(`systemctl --user enable --now mosaic-webui`), and so may Mos under lead
decision 102, which accepts that sends from a Console Mos started enter the
trail as Jason (`human-cli`). Any other agent starting it, even through
systemd, works around the human proof (lead decisions 62 and 94).
Exit codes: `0` ok · `1` failed or refused · `4` usage.
## Discord connector (`scripts/discord.sh`)
```bash
scripts/discord.sh check <binding>
scripts/discord.sh run <binding> [--supervised]
scripts/discord.sh stop <binding>
scripts/discord.sh unlock <binding>
scripts/discord.sh recover <binding>
scripts/discord.sh reload <binding>
scripts/discord-service.sh [--instance NAME] render | install [--dir DIR] [--no-reload] | uninstall [--dir DIR] [--no-reload] | status <binding>
```
One seat's conversation reachable from listed channels of one Discord
server, chat only. `<binding>` names `<dataRoot>/discord/<binding>.json`
(0600, never committed; shape in `packages/discord/fixtures/binding.example.json`).
`check` validates the binding, token file mode, context files and pi, reads
the bot, guild and channels over REST, opens and closes one gateway
connection; it sends nothing. `run` serves turns until `stop`, SIGTERM or
SIGINT and refuses to start when `STOP` exists or a delivery cannot be
reconciled. `stop` writes `STOP` and signals the running process. `unlock`
writes `STOP` and removes a `run.lock` left by a dead process; it refuses
while the owner is alive with an unverifiable identity or its record cannot be
parsed. `run` refuses a stale lock rather than reclaiming it, and releases a
claim that meets `STOP`. `recover` is the supervised pre-start, and
`run --supervised` does it first in the same process: it refuses (exit 3)
while `STOP` is present or the binding is held, clears a lock whose owner
is gone and removes only the `STOP` it wrote for that. `reload` validates
the binding file and sends SIGHUP to the running connector, which applies
`guildName`, `channels`, `users` and `limits` in place and refuses anything
else, keeping the old binding; attempts are journaled in `reloads.jsonl`. A
user entry may carry `channels`, an allowlist of listed channel ids.
`scripts/discord-service.sh install` renders and writes the systemd user
unit `[email protected]` (one instance per binding, restart on
failure, exit 3 never retried, SIGTERM on `systemctl --user stop`, SIGHUP on
`systemctl --user reload`). With
`install --copy ID [--copies DIR]`, a full id selects an existing shared
copy holding the Discord CLI and pi; ExecStart runs Node on that CLI with
--repo and WorkingDirectory in the copy. Without --copy the checkout unit
is unchanged. Status names its effective cwd/copy id, including drop-ins.
`check --offline` validates local binding/context/tool roots/pi/token mode
and reports journal/STOP without REST or gateway calls; run refuses this
flag. For a Mos-approved re-pin after the SetSpark freeze: make the copy,
run `node <copy>/packages/discord/src/cli.mjs check <binding> --repo <copy> --offline`,
save the installed unit, install --copy, then restart on Mos's go. Context
files resolve inside the copy and missing files refuse. Rollback restores
the saved unit and daemon-reloads/restarts on Mos's go; keep both copies.
The connector's pin is independent of stack-copy.sh check. An optional `tools` key in the binding declares
file roots; pi then runs with its own tools off and the package's
extension providing `list_dir`, `read_file` and `search` confined to those
roots, plus `write_file` and `edit_file` for roots marked `"write": true`
(temp file and rename, no folder creation, no dot-prefixed path, no
credential shapes), and `web_fetch` and `web_search` when `tools.web`
names a SearXNG instance (https only, public addresses only, three
redirects, capped body, html to text), with each call in the turn record.
A writable root that is a git work tree may carry `git` (`branch`,
`identity`, `tokenFile`, `author`, optional `protocol: "vault"`) and
gains `git_status`, `git_commit` (explicit paths, seat author,
`Requested-by:` trailer, push at once), `git_pull` (ff-only) and
`git_push` (one branch, never force), with `reserve_id` under the vault
protocol; git runs with no host config and the package's own credential
helper reading a 0600 token file, never printed. `tools.setspark` names the
SetSpark record service and adds `record_list`, `record_get`,
`record_create`, `record_update`, `resolve_id`, `open_approval_request`,
`get_approval_request` and `create_document`, one fixed path each, writes
keyed per turn and call; approvals of its proposals are an Approve button
or an exact `approve` reply, checked by the connector against the
request's approvers, never by the model, with one Discord message per
approval as evidence. `tools` is a fixed key: changing
it needs a stop and start. Records
under `<dataRoot>/discord/<binding>/`: `inbox.jsonl`, `outbox.jsonl`,
`drops.jsonl`, `admissions.jsonl`, `notices.jsonl`, `reloads.jsonl`, `approvals.jsonl`, write-once `turns/<id>.json`. Suite:
`scripts/test-discord.sh`.
Exit codes: 0 ok, 1 operation failed, 2 invalid data or configuration, 3
refused by a brake (a supervisor must not retry), 4 usage. Details:
`packages/discord/README.md`.
## Tools (host-side)
Host-side helpers under `tools/`, outside the `scripts/` command surface.
Per-tool READMEs: `tools/tmux/README.md` and `tools/unslop-hook/README.md`.
| Command | Purpose | Notes |
|---|---|---|
| `tools/tmux/agent-send.sh` | Inter-agent tmux message with addressing preamble | Reliable submit (bracketed paste, Enter flush, draft detection); ships `send-message.sh` over ssh for remote panes (remote needs only bash + tmux + base64) |
| `tools/agent-watch/agent-watch.sh` | Condition watcher per agent seat | One transient systemd `--user` timer + service per watch; fires `agent-send.sh` when the condition command exits 0 |
| `node tools/unslop-hook/unslop-check.js <file>` | Mechanical AI-tell prose check | Dependency-free node CLI + module driven by `lists.json`; `extension.ts` is the pi extension wrapper |
`agent-send.sh` prepends the preamble
`[<src_host>:<src_session> -> <dst_host>:<dst_session>]`; `-C`/`--class` adds
a ` class=<CLASS>` token (`terminal-log`, `actionable`, `human`, `reaction`,
`digest`; consumers treat an absent class as `actionable`). Flags: `-s` dst
session (required) · `-H` ssh target for a remote pane · `-L` named tmux
socket · `-n` dst hostname for the preamble · `-m`/`-f`/stdin message body ·
`-S` source-label override · `-r N` Enter-flush attempts (default 2) · `-v`
verbose · `-h` help. Exit codes: `0` delivered/queued · `1` target not found ·
`2` still draft · `3` usage error · `4` ambiguous socket (the session exists
on more than one tmux server; disambiguate with `-L` or `MOSAIC_TMUX_SOCKET`).
`agent-watch.sh` subcommands: `start --name <id> --session <session> --when
'<shell command; exit 0 = met>' --message <text>` with `--class`,
`--interval` (default 30), `--timeout` (default 3600), `--repeat`,
`--quiet-timeout`, `--socket` · `list` · `status [--json]` · `stop <name>` ·
`log <name>` · `meta-install [--interval 300] [--unit-name <unit>]` ·
`meta-remove [--unit-name <unit>]`. Interval floor is 10s (a watcher is a
fallback cadence, never a tight poll); hidden `_tick`/`_scan` subcommands run
inside the systemd services. `status` exit codes: `0` clean · `3` any stale
watch or dead meta-watch · `6` systemd user bus unreachable. Delivery goes
through `agent-send.sh`; rc `2` means the text reached the pane as an
unsubmitted draft, which counts as delivered and is not retried (other
failures retry twice, then the watch gives up). Watches are one-shot by
default; `--repeat` re-arms. Notices carry a `[watch:<name>]` prefix.
`unslop-check.js` checks a file (or stdin) against the word, phrase,
punctuation-density, and pattern lists in `lists.json`, stripping fenced and
inline code first so a quoted mention never flags. Invocation:
`node tools/unslop-hook/unslop-check.js <file>`; `UNSLOP_LISTS=<path>`
overrides the lists location. Exit codes: `0` clean · `1` violations (findings
printed as JSON on stdout) · `2` gate broken (invalid lists or unreadable
input; error on stderr, never a clean verdict).
## Exit-code convention
`0` success · `1` operation failed · `2` invalid data/configuration ·
`3` configuration missing for a read operation · `4` usage/file/environment
problem. Scripts print diagnostics on stderr; model responses (and only
model responses) on stdout.
## Model proxy setup (`scripts/mosaic proxy`, #1873)
Human-shell verbs that match the Console's Proxy setup panel; each refuses
`human-required` in an agent shell or with `--tester`, and prints no key or
absolute path. `proxy fetch` (existing output unchanged) now prints the
page's progress steps on stderr and writes the same fetch receipt to
`<dataRoot>/state/proxy-admin.jsonl`. `proxy status [--json]` shows release, init, unit, fence, init
lock, own proxy and the next step. `proxy init --generate-key [--port N]
[--rotate-grace-minutes N] [--first-output-seconds N] --business ID` has the
bus generate the management key (no prompt, no TTY). `proxy unit
render|status|install|start|stop|restart` manages the proxy user unit.
`proxy fence clear --restart --business ID` restarts the unit, requires a
confirmed new process listening on the port, then clears the fence with that
evidence (`proxy-restart-unconfirmed` otherwise). `proxy init-lock
status|clear` clears a stale init lock only when every bus is stopped and no
proxy state exists. Details: `packages/model-proxy/README.md`.
## Agent limits (`scripts/mosaic limits`)
View: `scripts/mosaic limits --business ID --json`. Set: `scripts/mosaic limits
set --business ID --file PATCH`, with the current revision and replacement
`limits` and/or `models`, or `instances` mapping role names to model-table keys,
or `assistants` mapping an assistant instance to any of its `vars`, `tracker`
(`null` removes it) and `definition` (row 155, #1649, Jason's ruling Q169). The
definition stays an assistant one; `holder` and `credentials` stay hand edits.
Both use the human bus proof. Settings offers the same edits in signed-in send
mode. Unset means unlimited, zero blocks new launches, and running sessions
remain alive when a limit is lowered. Total, provider and exact model limits
are checked for each launch; counts have no hard-coded policy ceiling.
Token limits (row 125) are `limits.tokens.instances.NAME`, `providers.NAME`,
`models.PROVIDER.MODEL` and `keys.NAME`, each `{ "limit": N }`. The writer adds
and keeps `since`. `scripts/mosaic limits reset WORD... [--business ID]` (for
example `reset tokens providers anthropic`) moves one limit's `since` to now,
logged with its actor. A token limit is enforced only by a switched-on
`limit-check` plug-in (`token-limit` ships): it refuses a launch at the limit
and stops the session at the end of the turn that reaches it. Otherwise the
read shows it as `not enforced`; a failing check refuses and stops (fail
closed). Without `--json` the read prints JSON without `tokens` followed by
`token limits:` text lines, so scripts must pass `--json`.
The Console's Assistant section (signed-in send mode) also sets an instance's
seat name, as `scripts/mosaic name` does, and stops the assistant's runs in the
human's threads. The chief of staff makes the same changes with its
`organization_revision`, `change_organization`, `set_name` and `end_run` tools;
`organization_revision` reads the revision that `change_organization` sends.
Convert an old organization file with `scripts/mosaic limits migrate --file
ORGANIZATION --models MAPPING --roles-dir roles > CONVERTED`. It needs no bus,
never edits the input or system config and has no write flag. Inherited models
require a valid system config and an exact matching provider/model mapping.
Mapping JSON contains `models`,
`families` and optional per-role `instances` selections. Jason checks and installs
the output in Mos's window with the PM run idle; re-pin in that same window.
Never migrate the live file from an agent session.
### Local project records (`mosaic project`)
`mosaic project create` and `mosaic project list` work on the bus record of a
business's projects; `mosaic project init` and `mosaic project check` work on
the repository on disk. Threads (`mosaic threads`, `mosaic thread create|open|read`,
`mosaic talk --thread`, `mosaic follow --thread`) are described in
`packages/cli/README.md`; a plain `mosaic talk` to the threaded instance exits 2
and asks for `--thread`.
`mosaic project init <business> <project>` creates the declared project's
separate local record repository, with layout version 1, package-validated JSON
templates, Markdown spec/docs and its first commit authored as the business's
human. Init refuses an agent environment, an undeclared project or a non-empty
directory. Failed init cleans up its own layout and allows retry, preserving a
pre-existing empty target and unrelated files. A changed owned path refuses
`project-init-cleanup` for inspection. Records may not overlap any declared
project's code root. No bus or remote is involved.
`mosaic project check <business> <project>` validates project metadata, every
record, filename/ID agreement and the fixed layout, naming each problem. It
refuses `project-no-commit` if the layout has no readable HEAD commit. It
exits 0 when valid, 2 for project/record problems (including an undeclared
project), 3 for configuration loading problems and 4 for usage. The optional business project `repo` is a normalized absolute
record-repository path; the default is
`<dataRoot>/projects/<business>/<project>/`. For a registered organization,
`<dataRoot>` is that organization's data root for both the CLI and its bus.
If an instance-root records repository already exists, initialization and reads
refuse and name both locations for inspection; they never move or copy it.
`root` remains the code checkout.
The package README describes schemas and the shared agent record writer; agent
writes never enter `spec/`, never absorb pending edits and refuse remotes.
### Token usage (row 124)
`mosaic usage [--business ID] [--json]` reads token accounting from the bus.
Each total, run, instance, model, provider and account group shows input,
output, cache write, cache read and unknown turns. A malformed or unavailable
transcript contributes an unknown turn, never a zero-token reading. Accounts
are unassigned until account tracking ships. Exact totals larger than the
safe integer range are decimal strings in JSON. Settings has the same
read-only Token usage section. Counts are a best reading, not proof: the
session can edit its own transcript. This command changes no setting and
performs no token-limit enforcement.
### Console Chat (row 129, #1623)
The Console's **Chat** section uses `#/chat`, `#/chat/projects/UUID` and
`#/chat/threads/UUID`. Create and name a project and its threads, then send a
message in that thread's conversation. Names are creation-only. The project
catalog is business-wide; thread lists and histories belong to the current
human. This is separate from the Board's seat reply route.
`GET/POST /api/chat/projects` lists/creates projects; `GET/POST /api/chat/threads`
lists/creates threads (`project=UUID` optionally filters the GET).
`GET /api/chat/threads/UUID/messages?limit=100&after=UUID` pages history forward;
`POST` on that messages route sends `{body}` as a REQUEST.
`POST /api/chat/threads/UUID/open` takes `{}` and explicitly resumes the assistant.
GET never writes, opens a thread or marks messages delivered/read. No browser
owner, tenant, proof or arbitrary bus verb is accepted. A same-business thread
owned by someone else is 403 `thread-not-owned`; foreign/missing ids are 404.
The four chat POSTs use the existing signed-in `--send` boundary: named Host,
one exact proxy user header, exact HTTPS Origin, same-origin Fetch Metadata,
strict JSON and 16 KiB. Read-only mode disables them. Bodies are inert text;
drafts are memory-only per thread and cleared on sign-out. History starts
oldest-first in pages of 100, with a 200-message display window. Sending does
not claim an assistant answer; an uncertain receipt is never retried. Read the
current list/history before trying again. Activation still needs Mos's separate
window; changing these files does not authorize a live Console action.
### Console tester listener (`serve --tester-socket`, row 196)
`node packages/webui/src/cli.mjs serve --tester-socket` adds private HTTP at
`<dataRoot>/console/tester.sock` (0600) under console/ (0700). The path must fit
107 bytes. Unsafe ownership, modes, symlinks and pre-existing entries refuse.
The main Console remains on its configured TCP port. New copies refuse --tester-port.
The tester's fixed Host is mosaic-tester.invalid and its Origin is
http://mosaic-tester.invalid. No DNS is used. POST also needs same-origin Fetch
Metadata and bounded JSON. The endpoint uses only TESTER_CLI. It admits the
existing instance, route, name and tester-chat writes, never owner settings,
document saves, assistant stop or role messages. bus.tester stays Jason's switch.
Authorization, login/session cookies and configured proxy headers refuse.
```sh
node packages/cli/src/tester-http.mjs GET /api/bus/agents
printf '%s' '{"name":"fixture"}' | node packages/cli/src/tester-http.mjs POST /api/chat/projects
```
The helper loads the selected instance config and reports HTTP status/text/body
as JSON, exiting 2 for refusal. It never retries writes. Direct browser access
is removed with Mos's acceptance (mos-1015). A future authenticated browser relay
behind SSO is a separate row. No TCP compatibility bridge is provided.
`webui-service.sh render|install --tester-socket` checks target support and carries
the flag into ExecStart. `stack-release.sh activate ID BUSINESS --tester-socket`
explicitly replaces an inherited --tester-port pair in both unit and pointer.
Without explicit conversion a new target refuses tester-transport-migration before
stopping anything. Unsupported targets refuse tester-socket-target. Rollback drops
unsupported socket enablement, visibly in plan/steps, without enabling TCP. Legacy
flags remain supported by the release tool for legacy target copies only. Failed
activation recovery can restore the previous TCP unit and its old limits.
Activation of a new copy preflights the pinned root-owned system Python ELF,
records its resolved path/version/identity/hash and checks the bus's fresh startup
receipt. Restarts revalidate and append actual interpreter identity to
`<dataRoot>/bus/peercred-starts.jsonl`, without comparing an old activation hash.
After an OS interpreter update, the next authorized re-pin uses a different reviewed
copy and refreshes activation evidence. Do not bypass already-active or use a PATH shim.
All live operations keep their existing window authority.
Console proxy proof (row 178, #1676): `serve`, `webui-service.sh render` and
`install` accept `--proof-header NAME --proof-file PATH` together, under preview
mode only. The header must differ from Host, Origin, the auth header and
x-mosaic-tester. Serve exits 2 unless PATH is a non-symlink regular file owned by
its uid, mode 0600 or 0400, containing at least 32 base64url characters in one
line (an optional final LF is allowed). Render checks metadata only and refuses
paths inside the checkout or copies; it never opens the file. Keep the value
out of argv, logs and messages. Serve reads it once, and prints only `proxy
proof on` or `proxy proof off`. The public Host needs exactly one matching proof
before sign-in is checked; bad or absent proof gets 401 and no bus or board
call. The proof header on another Host or the tester listener gets 403.
`stack-release.sh` preserves both flags unchanged on re-pin and rollback.
This row installs no proof file, middleware or live flags. Those belong to a
separate Mos plan-checked window: middleware first, Console flags second, then a
public sign-in smoke and a forged-loopback 401 check; rollback drops flags
before middleware. Same-uid callers can still read the file; separate service
uid is Jason's question on #385. The Authentik allowed-uid file remains an
identity input and is never reused for proof.
## `scripts/push-check.sh` — verify a prospective push
`scripts/push-check.sh SHA EXPECTED_LOG [--row N]... [--base-ref REF] [--fetch]`
checks the repository at the caller's cwd; it never pushes or writes the queue.
Git checks cover the full repository even from a subdirectory; EXPECTED_LOG is
resolved relative to the original caller. Non-UTF-8 git paths refuse with exit 2.
EXPECTED_LOG must contain exactly `git log --oneline BASE..SHA` (including its
newlines). BASE defaults to `refs/remotes/origin/refactor`; `--base-ref` overrides
it. Only `--fetch` fetches origin, before resolving BASE.
Success prints `LOG_OK`, `FF_OK`, `NO_BINARY` or `BINARY_REVIEWED n`,
`SECRETS_CLEAN`, `NO_ATTRIBUTION`, then `PUSH_READY BASE..SHA` as the last line.
A changed present blob counts as binary when numstat marks it binary or its
first 8000 committed bytes contain NUL, regardless of git attributes. Each
present binary needs its committed blob sha256 and exact path in an approved
latest-round candidate from a supplied row. A `changes` receipt blocks that round.
Commit candidates authorize only changed paths with matching bytes and mode.
Deleted binaries need no review. Queue reads retain canonical-checkout checks
and refuse unconfirmed durable tails with exit 2.
Exit 0 means ready; 1 means refusal (`STOP reason`, always the last line); 2 means
usage or environment error. A refusal never prints later success tokens.
`timeout -k 10 240 bash scripts/test-push-check.sh` exercises isolated scratch
repositories and the six required mutants; set TMPDIR to a scratch filesystem.