researcher: I1 container survey report

Packet commit, one path. Secrets scan clean. Not pushed. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-12 17:55:28 -05:00
parent 01618e930d
commit 85aa676f32
1 changed files with 187 additions and 0 deletions
@@ -0,0 +1,187 @@
 # I1 survey: what a containerized Mosaic Stack looks like
 Researcher seat, read-only survey for Sage. Written 2026-10-11T22:39Z. Due 2026-10-12T18:00Z.
 Nothing here moves or lands anything. It feeds the draft brief that waits on Jason (Q97).
 Anything I could not confirm is marked "unverified".
 ## Recommendation
 Containerize by role, and replace the `/proc` ancestry proof with reachability. In the containerized install the broker is the only container that holds service credentials and the dataRoot. It exposes two sockets. An agent socket accepts capability verbs only. A human socket accepts human verbs, is owned by its own uid, and its directory is mounted into the Console container and nothing else. The Console sits on a network that agent containers cannot reach, and no container gets an engine socket. That closes decision 62's gap for agents that run inside containers. The Authentik header alone closes nothing, because anything that reaches the Console port can set it. It only says who is typing. Add WebAuthn for a short list of actions (decisions, role changes) as a second step, so a compromised Console is not human authority. Use rootless Docker. It is still `docker`, so `execution.backend` needs no allowlist change. This host runs a rootful daemon and Jason is in the `docker` group, so the switch is a host change for Jason to make. The local install stays as it is, labelled as the ancestry proof with decision 95's accepted limit. Both modes share one broker protocol, and `authenticateHuman` at `packages/bus/src/runtime.mjs:61-69` is the single swap point. The closure holds only for agents inside containers. A host-side agent still passes the old check.
 ## 1. Host components today
 All of these run as Jason's uid. The store refuses a dataRoot that another uid owns or that has group or other bits (`packages/bus/src/store.mjs:48-58`).
 ### Bus host, broker, notifier, launcher
 - The host process is `mosaic bus start <business>`. It forks the broker and the notifier (`packages/cli/src/host.mjs:31-32`) and writes `bus-host/host.json`, mode 0600 (`host.mjs:36-37`).
 - The broker runs only when forked with an IPC channel (`packages/bus/src/process.mjs:30-33`). It gets its boot config over IPC, so capabilities never touch argv or env.
 - Broker writes: the SQLite store under `bus/` and `bus/writer.lock`, one broker per data root (`store.mjs:75-80`). The schema is pinned by SHA256 (`store.mjs:16`).
 - Broker socket: `<dataRoot>/bus/broker.sock` (`runtime.mjs:56`), chmod 0600 (`packages/bus/src/server.mjs:93`). One JSON request per connection, 64 KiB cap, envelope `{cap | human, verb, args}` (`server.mjs:43-51`).
 - Broker credentials: Gitea and Vikunja tokens, loaded from business-file references (`runtime.mjs:21-32`). A `file` reference must be absolute, outside the repo and dataRoot, owned by the process uid, mode 0600, at most 16 KiB (`packages/bus/src/credentials.mjs:35-79`). `use()` runs the call and `assertClean` rejects any leak. The tokens stay in memory.
 - Broker outbound: HTTPS `fetch` to Vikunja (`packages/tasks/src/vikunja.mjs`) and to Gitea. I did not locate the Gitea adapter file.
 - Notifier: gets the reader capability over IPC (`packages/cli/src/notifier-process.mjs:1-6`). It uses the Discord bot token file and REST client (`packages/discord/src/notify.mjs:2`). Outbound is Discord REST.
 - Launcher: serves `bus-host/launch.sock`, mode 0600 (`packages/cli/src/launcher.mjs:333-378`). It writes `launches/<business>/<run>/` and `launches/<business>.jsonl` (`packages/seat/src/session.mjs:42-46`). It builds the bundle, spawns the runner, binds the launch with the outer unshare pid, and writes the capability to the runner's stdin (`launcher.mjs:156-238`).
 - Env: the host needs the system config at `~/.config/mosaic-dev/config.json`, read by `loadSystem` (`packages/cli/src/config.mjs`). I did not read that file.
 ### Console (packages/webui)
 - An HTTP server on 127.0.0.1:7330 (`packages/webui/src/cli.mjs:13`, `serve.mjs:81-83`). It refuses a non-loopback bind and a non-loopback Host (`serve.mjs:82,91-92`), and sets a CSP (`serve.mjs:88`).
 - It proxies `/api/board`, `/api/seen`, `/api/reply` and the conversation routes to the board at 127.0.0.1:7331 (`serve.mjs:99-117`).
 - It reads the bus through `humanCall`, which spawns `human-cli.mjs` as a child with `process.env` (`packages/webui/src/bus.mjs:25-29`). Routes are GET only (`serve.mjs:50,119-124`). It reads the live business from `bus-host/host.json` (`bus.mjs:63`).
 - `reads.mjs` supplies queue, business and settings reads. Which dataRoot paths it opens, I did not trace. Unverified.
 - It holds no service credentials. Its authority is that it is a same-uid process that can spawn the human CLI.
 - No forward-auth header check exists in `packages/webui/src` today. Row 74 plans `--public-host`, `--auth-header` and `--allow-user` (`docs/plans/2026-10-11_webui-preview.md:56-72`). Row 76 adds the human `message.send` route (`docs/plans/2026-10-11_dogfood-cutover.md:112-134`).
 ### Board (packages/control-board)
 - An HTTP server on 127.0.0.1:7331, no auth (`packages/control-board/src/serve.mjs:1-41`, bind at `312-322`).
 - `/api/reply` calls `tools/tmux/agent-send.sh`, which pastes into a tmux pane (`serve.mjs:74-111`). `/api/seen` writes board files.
 - It rescans on each `/api/board` and defaults to the old fleet directory under `~/.mosaic` (`packages/control-board/src/cli.mjs`, line not recorded). I did not read that directory.
 - Credentials: none. Env: none that I found.
 ### Managed sessions
 - Start: `unshare --user --map-current-user --pid --fork --kill-child --mount-proc node runner.mjs <runDir>` (`session.mjs:39,110-118`). A PID namespace and a same-uid user namespace. No network, filesystem or IPC isolation.
 - Reads and writes: its run dir (0700, `session.mjs:78-94`), `workspaces/<business>/<instance>/` (0700, kept across launches, `session.mjs:46`). Nothing stops it from reading the rest of the home directory.
 - Sockets: `tools.sock` in the run dir, which stays under 107 bytes (`session.mjs:90`). It also reaches `broker.sock` and `launch.sock` through the shared filesystem. The runner talks to the broker with the capability it got on stdin (`packages/harness/src/runner.mjs:85-111`).
 - Env: an allowlist (`session.mjs:61-74`). It passes `ANTHROPIC_API_KEY`, `ANTHROPIC_AUTH_TOKEN`, `ANTHROPIC_BASE_URL`, `CLAUDE_CODE_OAUTH_TOKEN`, `OPENAI_API_KEY`, `OPENROUTER_API_KEY`, `MOSAIC_CLAUDE_BIN`, plus `HOME` and locale. `FOUNDER_ENV` makes the runner exit 20 if it sees `GITEA_TOKEN`, `GH_TOKEN`, `SSH_AUTH_SOCK` and similar (`runner.mjs:39-43`).
 - Outbound: made by the provider CLI under the adapter (`adapters/*/adapter.sh` contain no URLs). Where the Claude CLI keeps login state under `HOME`: unverified.
 - Each turn spawns `/bin/sh adapter.sh` in its own process group under a wall clock (`runner.mjs:153-201`).
 ### Pi worker
 - Image `node:24-bookworm-slim`, `USER node`, entrypoint `src/run-agent.sh` (`Containerfile:4,35,43`). Compose service `mosaic-agent` runs as `1000:1000` (`compose.yaml:7`).
 - Mounts: the whole dataRoot read-write at `/var/lib/mosaic` (`compose.yaml:40`), and the pi `auth.json` read-only (`compose.yaml:43`).
 - Writes inside the dataRoot mount: `system-prompt.md` (`src/run-agent.sh:37-39`), the copied mission file and the workspace directory (`scripts/agent.sh:192-203`).
 - Env: `ZAI_API_KEY` and `ANTHROPIC_API_KEY` (`compose.yaml:36-37`), plus `PI_*` and `MOSAIC_*`.
 - Sockets and ports: none. Launched by `scripts/run-task.sh`, `scripts/agent.sh:208` (`docker compose run --rm mosaic-agent`) and `scripts/release.sh`.
 ## 2. How far the worker image and compose get us
 The worker is a sound start on filesystem layout and a poor start on everything the brief cares about. It covers one component of six.
 | Gap | Today | What to change |
 |---|---|---|
 | Network | No `networks` key, so the default bridge with open egress (`compose.yaml`, whole file). | A network with `internal: true` plus an egress proxy. See section 5. |
 | Mounts | Whole dataRoot read-write (`compose.yaml:40`). That includes `bus/broker.sock`, `bus-host/`, every `runs/` record and every other workspace. | Mount only the instance workspace read-write, the run dir, and contracts read-only. |
 | Env keys | Provider keys as env (`compose.yaml:36-37`). They show in `docker inspect` and the process environment. | File secrets or a key-holding gateway. See section 5. |
 | User | `1000:1000`, the host user (`compose.yaml:7`, `Containerfile:11-13`). Same uid as the broker socket owner. | A distinct uid per role. See section 4. |
 | Caps | None dropped, none added. Docker's default set. | `cap_drop: [ALL]`, `security_opt: [no-new-privileges:true]`. Both attributes exist in the compose spec. |
 | Read-only root | Not set. `Containerfile:23-28` chmods `/opt/mosaic` to 0555, but the root is writable. | `read_only: true` plus `tmpfs` for `/tmp` and `HOME`, and the workspace mount. |
 | Limits | No `pids_limit`, memory or CPU limits. | `pids_limit`, `mem_limit`, CPU limit. |
 | Seccomp | Docker's default profile applies (`docker info`: `seccomp,profile=builtin`). | Keep the default. A custom profile is optional. |
 Two consequences of `compose.yaml:40`:
 - A worker sees `broker.sock` and runs as the broker's uid, so it can open the socket. Agent verbs need a capability and the human path needs a proof the worker cannot produce from another PID namespace. I did not test what the broker answers to a hostile client over a mounted socket. Unverified.
 - `scripts/agent.sh:192-203` copies the mission file and creates the workspace directory on the host before the container starts, so host scripts write into the same tree the container mounts.
 The Docker docs describe `read_only`, `cap_drop`, `security_opt`, `pids_limit`, `mem_limit`, `tmpfs`, `secrets`, `user` and `userns_mode` as service attributes: https://docs.docker.com/reference/compose-file/services/. That page does not mention seccomp options. Unverified from the docs.
 Host facts from this machine: Docker 29.7.2. `docker info` lists `seccomp` and `cgroupns` and no rootless entry, so the daemon is rootful. Jason's uid is in the `docker` group. That is root-equivalent on the host, which matters for the launcher in section 4. `/etc/subuid` and `/etc/subgid` each give `jwoltje` a 65536-ID range, which meets the rootless Docker minimum (https://docs.docker.com/engine/security/rootless/). Podman is not installed.
 Reference style already in the tree: `packages/tasks/deploy/vikunja/compose.yaml` pins the image by digest (line 10), runs as the operator's uid (14), and publishes on 127.0.0.1 only (16). It sets no `read_only`, `cap_drop` or limits. The Vikunja probe report in `agents/researcher/work/2026-10-04_vikunja-probes.md` covers how it behaves under such bounds.
 ## 3. Human proof that holds inside containers
 Why ancestry cannot carry over: `verifyHuman` reads `/proc` for the CLI's uid, start time, `MOSAIC_BUS_CLI_NONCE`, argv and parent chain (`packages/bus/src/human.mjs:52-101`). Its own comment says it is "not peer credentials or a hostile-process wall" (line 50). From another container the broker cannot read the caller's `/proc`, and a PID inside a container names a different process on the host. The check has to be replaced in containers, not hardened.
 Decision 62's gap, restated: any process that can reach the human path can pass. On the host, `systemd-run` or `setsid -f env -i` detaches from the agent's ancestry (`docs/plans/2026-09-26_lead-decisions.md:1037-1060`; `docs/plans/2026-10-11_dogfood-cutover.md:19`). A fix has to remove the agent's ability to reach the endpoint, or to produce the credential.
 ### Options
 **A. Credential from the forward-auth header.** Authentik sets `X-authentik-username`, `X-authentik-uid` and others on requests it forwards (https://docs.goauthentik.io/add-secure-apps/providers/proxy/). The Console checks the header and then acts for the human.
 - Tradeoff: cheap, and it matches rows 74 and 76. It is a claim, not a proof. Anything that can reach the Console port can send the header. The Authentik page I read does not say the upstream must be unreachable except through the outpost, so that rule is ours to enforce.
 - Closes 62: only together with network isolation. If agent containers cannot route to the Console port, the header cannot be forged by them. Then reachability does the work and the header names the user.
 - The Console becomes a standing human-authority holder. Its CSP and the read-only route list matter more than they do now.
 **B. Unix socket with peer credentials and a uid per identity.** The kernel reports the connecting process's uid through `SO_PEERCRED`, as it was at `connect(2)` (https://manpages.debian.org/bookworm/manpages/unix.7.en.html). The human socket is owned by a human-role uid, mode 0600.
 - Tradeoff: strong against agents that have no route to the socket. It needs no secret to leak.
 - Node's `net` module does not expose `SO_PEERCRED`, as far as I know. I did not check Node's docs. If so, rely on the file mode instead: the kernel enforces it at `connect`, and a separate socket per role removes the need to tell callers apart on one socket.
 - How uids translate across user namespaces: unverified. The man page I read does not say.
 - Closes 62: yes for agents in containers, if the socket is not mounted into theirs and their uid differs. It is a second wall even if a mount leaks.
 **C. Hardware key, WebAuthn.** An assertion is a signature over a relying-party challenge, with user presence or verification flags in the authenticator data, and credentials are scoped to the origin (https://www.w3.org/TR/webauthn-3/; I read the first 100,000 of 503,000 characters).
 - Tradeoff: survives a compromised Console and a leaked socket, because the broker verifies a signature against a stored public key. Cost: a touch per action, a verifier in the broker, and a dependency, since Node has no built-in WebAuthn verification. The repo pins versions strictly, so adding one is a decision for Sage and Jason.
 - It signs a challenge, not text. A compromised Console could show one instruction and ask for a signature over a challenge built from another. Bind the challenge to a hash of the action and keep the action text on a path the Console does not compose. That reduces the risk and does not remove it.
 - Closes 62: yes, by itself. It is also the only option here that does not depend on network layout.
 **D. Socket per role, mounted only into that role's container.** This is B's file-mode logic with the mount as the access control. Agent containers never get the human socket, so there is no endpoint to forge against.
 - Tradeoff: cheap and legible. Mount the socket's directory, not the socket file, so a broker restart does not leave a stale inode. That is general Docker behavior I did not re-check in the docs.
 - Closes 62: yes for agents in containers, provided no container has an engine socket or `--privileged`.
 ### Which closes it
 D plus A closes the gap for agents in containers. B adds depth. C is the one that closes it against a Console compromise. My ranking for the brief: D, with A for identity, as the minimum. C for decisions and approvals as the second step. None of them helps an agent that still runs on the host as Jason's uid. Decision 95 carries that limit and says the containerized install is meant to close it (`lead-decisions.md:2058-2062`).
 ### Work this implies
 - `server.mjs:53-56` handles `human` and capability envelopes on the same socket. Containerized mode needs two listeners, with the agent listener refusing human envelopes. This is a code change in `server.mjs` and `runtime.mjs`.
 - `authenticateHuman` (`runtime.mjs:61-69`) calls `verifyHuman` and binds `{via:'cli', outsideAgent:true}` (line 68). A containerized mode binds with a new `via` value instead. The ancestry mode stays for the local install.
 ## 4. Separating identities
 - One uid per role. Compose `user:` sets it (spec link above). Bind-mounted directories need matching ownership, and Compose cannot chown host paths. For file-sourced secrets, Compose ignores `uid`, `gid` and `mode`, because it bind-mounts the host file. Plan an init step or named volumes.
 - Rootless Docker. Container uids map into the user's subuid range, so container root is an unprivileged host uid. Needs `newuidmap`, `newgidmap`, and 65,536 subordinate IDs. The docs list no other prerequisite on that page (https://docs.docker.com/engine/security/rootless/). They do not cover networking limits or cgroup requirements there. Unverified.
 - `userns-remap`. Daemon-wide: it turns user namespaces on for every container, hides existing images and containers, and complicates bind-mount ownership (https://docs.docker.com/engine/security/userns-remap/). Worse fit than rootless for a host that already runs other Docker workloads.
 - Podman. Not installed. The config allowlist is `docker` only (`scripts/mosaic-config.mjs:32,131-133`), and `scripts/test-config.sh:107-109` asserts that Podman is refused. Rootless Podman would need that allowlist widened by a reviewed commit and the `docker compose` calls in `scripts/` made backend-aware. Its `--secret`, `--read-only`, `--cap-drop`, `--network none` and `--pids-limit` match the Docker set (https://docs.podman.io/en/latest/markdown/podman-run.1.html). I could not confirm `--userns=auto` or `keep-id` from the page text I got.
 - What the docker-only backend needs: nothing for rootless Docker. It is still `docker`. The scripts use the ambient Docker context (`scripts/agent.sh:208`), so a rootless daemon's `DOCKER_HOST` has to be set where they run. Whether they set it: unverified.
 - Who starts session containers. `session.mjs` and the launcher use host pids, `/proc` start times and `process.kill` (`session.mjs:123,195,202-216`; `launcher.mjs:309-331`). A container backend replaces those with container ids, `docker inspect` and `docker stop`. The runner already assumes it is pid 1 of its namespace (`session.mjs:9-12`), so it fits as a container entrypoint.
 - Keep the engine socket out of every container. A launcher with the engine socket mounted is root on a rootful daemon. v1 made the same call: no runtime socket in a seat container (`v1/docs/requirements/cli-capability-migration.md:50`, T78-D4). The launcher belongs outside the agent-facing containers, as a host-side process or as the only holder of a rootless engine socket.
 - Socket per role: broker (agent socket, human socket, internal store), launcher, Console. Each role's container mounts only its own socket directories.
 ## 5. Network and secrets
 ### Egress
 - Put agent containers on a network with `internal: true`. The compose page says it "lets you create an externally isolated network" (https://docs.docker.com/reference/compose-file/networks/). Whether published ports still work on such a network: unverified.
 - Add one proxy container on both the internal network and an external one. It allows only the provider hostnames. Agents get `HTTPS_PROXY` or, for the Anthropic API, `ANTHROPIC_BASE_URL`, which is already in `ENV_ALLOW` (`session.mjs:63`). Whether pi and the Claude CLI honor `HTTPS_PROXY` through CONNECT: unverified.
 - Docker's own pages did not describe the `DOCKER-USER` chain, so I do not recommend host iptables rules on the strength of the docs I read.
 - Broker and notifier need their own egress: Vikunja, Gitea, Discord. They sit on a different network from the agents.
 ### Keys without env or a whole-dataRoot mount
 - Provider API keys: a gateway container holds the key and injects it. The agent gets only `ANTHROPIC_BASE_URL` pointing at the gateway. The key never enters an agent container. This does not cover the subscription path (`CLAUDE_CODE_OAUTH_TOKEN`, `session.mjs:63`). Whether a header-injecting gateway works with that path: unverified.
 - File secrets: Compose mounts `secrets:` read-only at `/run/secrets/<name>` (spec link above), and Podman's `--secret` defaults to a file mount. Env delivery exists in both. Avoid it.
 - The broker's `file` credential rule wants an absolute path outside the repo and dataRoot, owned by the process uid, mode 0600 (`credentials.mjs:35-79`). `/run/secrets/<name>` fits if the container uid matches the host file owner. Compose's file-sourced secrets keep host ownership, so the two have to match.
 - Per-run narrow mount: mount `launches/<business>/<run>/` (0700 today, `session.mjs:78-94`), the instance workspace, and read-only contracts. Do not mount `bus/`, `bus-host/` or `runs/`.
 - Capability delivery: the host writes it to the runner's stdin (`launcher.mjs:156-238`). Doing that into a container with `docker run -i` should work. Unverified.
 ## 6. What the local install keeps
 - Systemd user units: `packages/cli/systemd/mosaic-bus.service.in` and `packages/discord/systemd/[email protected]` set `Restart=on-failure`, `KillMode=mixed` and `NoNewPrivileges=yes`. No `ProtectHome`, `PrivateTmp`, `ProtectSystem` or address-family limits. The services run as Jason with the whole home directory visible.
 - The Console and the board have no unit templates yet. Rows 74 and 75 add `scripts/webui-service.sh` and `packages/webui/systemd/mosaic-webui.service.in` (`docs/plans/2026-10-11_dogfood-cutover.md:39-66`). They are being designed now, so give the flags and env names the container entrypoints will use.
 - Unit hardening can narrow what a service sees. It cannot stop a same-uid process from reaching a same-uid socket, so the local install cannot close decision 62. Say so in the docs.
 - Identical in both modes: the broker protocol and envelope (`server.mjs:43-51`), the store schema pin (`store.mjs:16`), contracts, mission and role files, run-record rules, the credential file rules, `config.json`, and the fail-closed behavior.
 - Allowed to differ: how a session starts (unshare or container), how a human is proven (ancestry or socket), how secrets arrive, and egress.
 - Test both modes against one suite of bus verbs. Only the human-proof and session-start layers get mode-specific tests.
 ## 7. Prior art and stale material
 ### v1 (archived)
 - `v1/docs/requirements/cli-capability-migration.md:50,444,489`: T78-D4 and EXE-REQ-005 want one rootless container per seat, a read-only root with bounded writable mounts, no engine socket, no host credential tree. Status "active ... implementation held by the M0 gates". It depends on `mosaicd`, and I found no `mosaicd` outside `v1/` and `agents/`. Treat as intent, not as design.
 - `v1/docker-compose.dogfood.yml:1-37`: a single-seat dogfood with a dedicated worktree bind, the seat home read-only, and the base stack credential-free. Lines 7-8 say the credential helper decides ownership from process ancestry and needs `init: true`. v1 hit the same ancestry problem in a container. Worth noting for the brief.
 - `v1/docker-compose.yml`, `v1/docker-compose.federated.yml`, `v1/docker/gateway.Dockerfile`, `v1/docker/appservice.Dockerfile`, `v1/infra/*`, `v1/docs/archive/planning/legacy/chroot-sandboxing.md`: I listed them and read only parts of the dogfood file. I did not read the federated compose or the chroot document. Unverified.
 ### Current docs
 - `docs/plans/2026-09-02_atomic-mosaic-foundation.md:46` lists bare-metal versus containerized installation as an open question. Lines 170 and 216 say no state lives only in a container's writable layer and the image and container hold no credentials. Both still hold, and both are the invariants to keep.
 - `docs/plans/ROADMAP.md:64` says containerization removes ambiguity about what agents get. `ROADMAP.md:178` defers network egress policy to role contracts at M18. If the containerized install lands before M18, the egress work in section 5 moves up with it.
 - `docs/plans/CONDUCTOR.md:17,40`: the worker table says a container is the boundary and workers get no docker socket or host filesystem. True for the pi worker. Stale for managed sessions, which run on the host.
 - `LAYERS.md`: L0 only implemented, and L4 describes Claude in the same container. Stale. Managed Claude sessions run on the host (`session.mjs:35-39`).
 - `docs/plans/2026-10-11_dogfood-cutover.md:19`: the same gap, in the cutover plan's own words.
 ## Findings the brief should carry
 1. The forward-auth check is planned (rows 74, 76), not shipped. Decision 95's wording depends on it.
 2. The board has no auth and `/api/reply` pastes into tmux. tmux sockets do not cross containers, so the board needs a different delivery path before it containerizes.
 3. `compose.yaml:40` gives the worker the broker socket and the full run history today.
 4. The session registry and `stopSession` rely on host pids. A container backend rewrites them.
 5. `ENV_ALLOW` and `compose.yaml:36-37` pass keys as env in both execution paths.
 6. A rootful daemon with Jason in the `docker` group means any container launcher is root-equivalent until the switch to rootless Docker.
 ## Unverified list
 Gitea adapter location; `reads.mjs` dataRoot paths; hostile-client behavior over a mounted `broker.sock`; Node access to `SO_PEERCRED`; uid translation across user namespaces; published ports on internal networks; proxy support in pi and the Claude CLI; gateway injection for the OAuth path; `DOCKER_HOST` handling in `scripts/`; `docker run -i` for the capability line; Podman `--userns` modes; `v1/docker-compose.federated.yml` and the chroot document.