Build, patch and manifest for the bwrap sandbox, the credential check and the unit's EnvironmentFile= line. Gate 14 suites rc=0; 26 mutants killed. Co-Authored-By: Claude Opus 5.5 <[email protected]>
302 lines
17 KiB
Markdown
302 lines
17 KiB
Markdown
# Row 92, role sessions confined to their own files, build (Darkwing)
|
|
|
|
Issue #1586. Brief: `docs/plans/2026-10-12_session-confinement.md`
|
|
(blob `21ee4458`, queue rev 741, decision 118), with Sage's direction on
|
|
decision 119 (Q126, `b66bfe71`, queue rev 754): any provider credential
|
|
that `ENV_ALLOW` carries passes the launch check, the credential file's
|
|
path appears once in the unit template and once in the README paragraph,
|
|
and tests use stand-ins. Base `af01465c`. No commit from the base to the
|
|
packet's commit touches the 13 paths.
|
|
|
|
`build.patch` is `git diff --cached --binary af01465c` of the 13 paths.
|
|
Apply it with `git apply --index` to a detached worktree of the base, and
|
|
`sha256sum -c candidate-manifest.sha256` then gives 13/13 OK.
|
|
|
|
| File | What changed |
|
|
|---|---|
|
|
| `packages/seat/src/sandbox.mjs` | new: the bind table, the bwrap command line, the probe |
|
|
| `packages/seat/src/session.mjs` | the runner spawns in the sandbox; `SESSION_CREDENTIALS`; `sessionExit` |
|
|
| `packages/seat/tests/sandbox.test.mjs` | new: real confined sessions against stand-ins, binds broken on purpose |
|
|
| `packages/seat/tests/session.test.mjs` | the session's HOME; the exit mapping; the credential sets |
|
|
| `packages/seat/README.md` | "The sandbox (#1586)" in place of the PID namespace limits |
|
|
| `packages/cli/src/launcher.mjs` | two new refusal steps; the paths a session needs; recovery matches bwrap |
|
|
| `packages/cli/tests/launcher.test.mjs` | both refusals; the real Claude Code coder; credential stand-ins |
|
|
| `packages/cli/tests/fixtures/launch-host.mjs` | passes the fake adapter as a read-only file |
|
|
| `packages/cli/tests/host.test.mjs` | the unit has one `EnvironmentFile=`, the README names it |
|
|
| `packages/cli/systemd/mosaic-bus.service.in` | the `EnvironmentFile=` line and its comment |
|
|
| `packages/cli/README.md` | the two refusals, the credential, the unit paragraph |
|
|
| `packages/harness/src/gate.mjs` | header comment only |
|
|
| `packages/harness/README.md` | "Limits" rewritten for the sandbox |
|
|
|
|
## The sandbox
|
|
|
|
`spawnSession` now runs
|
|
`bwrap <namespaces> <binds> --remount-ro / --setenv HOME <run>/home --setenv TMPDIR /tmp --chdir <run> -- setpriv --pdeathsig KILL -- node runner.mjs <run>`.
|
|
The root is an empty tmpfs; each bind lands at its host path, so the paths
|
|
in `session.json` mean the same inside and out. The namespaces are
|
|
`--unshare-user --unshare-pid --unshare-ipc --as-pid-1`, so the runner is
|
|
still pid 1 of its own PID namespace.
|
|
|
|
### Bind table
|
|
|
|
In mount order, as `binds()` returns them (each entry carries its `why`
|
|
in the code too):
|
|
|
|
| Source | Inside | Mode | Why |
|
|
|---|---|---|---|
|
|
| `/usr` | same | ro | the system's programs and libraries: sh, node, git, coreutils |
|
|
| `/bin`, `/lib`, `/lib64`, `/sbin` | same | symlink (or ro if a real dir) | the host's links into `/usr` |
|
|
| `/etc` | same | ro | name resolution, CA certificates, passwd for the user's name |
|
|
| (none) | `/dev` | private | null, zero, urandom and a pty |
|
|
| (none) | `/proc` | private | the session's own PID namespace |
|
|
| node's real path, if outside `/usr` | node's path | ro | node runs the runner and the gate |
|
|
| the 10 `RUNNER_FILES` | same | ro | the repository files the runner loads |
|
|
| the harness adapter | same | ro | the runner starts it each turn |
|
|
| `<repo>/node_modules` (real path), pi only | `<repo>/node_modules` | ro | pi and its packages, on the session's PATH |
|
|
| `MOSAIC_CLAUDE_BIN`'s real path, Claude Code only, if outside `/usr` | `MOSAIC_CLAUDE_BIN` | ro | the pinned Claude Code binary |
|
|
| the broker socket | same | ro | the runner claims and calls through it |
|
|
| the launch socket, the launching role only | same | ro | the `launch` tool calls it |
|
|
| the instance's workspace | same | rw | where the harness works |
|
|
| the run directory | same | rw | turns/, session/, tools.sock, turn.done, home/, tmp/ |
|
|
| `<run>/bundle` | same | ro | stays as the host wrote it |
|
|
| `<run>/session.json` | same | ro | stays as the host wrote it |
|
|
| `<run>/tmp` | `/tmp` | rw | a private /tmp on disk in the run directory |
|
|
|
|
HOME is `<run>/home`, made empty (0700) before the spawn. Nothing else
|
|
exists: no data root beyond the run directory, no other workspace or run,
|
|
no `~/.config`, `~/.mosaic`, `~/.ssh` or `~/.claude`, no checkout beyond
|
|
the runner's files. A bind to a socket is read-only as a mount, and
|
|
`connect()` on a socket in a read-only mount still works; that is what the
|
|
broker and launch sockets need.
|
|
|
|
## Refusals
|
|
|
|
Two steps join the launcher's checks, both before family and capacity and
|
|
both through `refuse()`, so each is an `action.refused` event and a launch
|
|
log line:
|
|
|
|
- `no-session-credential`: a Claude Code launch when none of the
|
|
`SESSION_CREDENTIALS` sets is fully set in the bus host's environment:
|
|
`CLAUDE_CODE_OAUTH_TOKEN`, `ANTHROPIC_API_KEY`, or
|
|
`ANTHROPIC_AUTH_TOKEN` with `ANTHROPIC_BASE_URL`. A pi launch doesn't
|
|
check.
|
|
- `sandbox-unavailable`: `sandboxProblem()` runs bwrap with the namespaces
|
|
and the system binds and `true`. A missing bwrap or a failing one refuses
|
|
with the first line of bwrap's stderr. There is no unconfined path left:
|
|
`namespace: false` and `UNSHARE` are gone.
|
|
|
|
## Choices to review
|
|
|
|
1. **setpriv's parent-death signal, not bwrap's `--die-with-parent`.**
|
|
`--die-with-parent` sets the death signal on bwrap itself as well as on
|
|
the child. bwrap's parent is the bus host, so a host that dies hard
|
|
would take every session with it, and the launcher's `recover()` (the
|
|
next host finds the sessions and ends their runs) would have nothing to
|
|
recover. That breaks the two "host that died hard" tests. So the
|
|
runner gets `setpriv --pdeathsig KILL`: it dies when the outer bwrap
|
|
does, as it did with unshare's `--kill-child`, and bwrap outlives the
|
|
host. The race: if the outer bwrap dies between the fork and setpriv's
|
|
`prctl`, the runner keeps running. The host ends the launch when the
|
|
outer bwrap exits, and `endLaunch` voids every capability of the run,
|
|
so the runner's next poll gets `unauthenticated` and it exits
|
|
(`runner.mjs`, `SESSION_GONE`). A turn already running finishes first,
|
|
up to the turn timeout. The window to hit is the few instructions
|
|
between the fork and the `prctl`.
|
|
2. **The refusal names the variables, not the file.** The brief said a
|
|
Claude Code launch with no credential refuses "naming the missing
|
|
file". Sage's one-path rule puts the path only in the unit template and
|
|
the README paragraph, so the message names the variables and "the bus
|
|
unit's EnvironmentFile". The README paragraph names the file.
|
|
3. **Any provider credential passes, per decision 119.** The check accepts
|
|
the OAuth token, an API key, or a proxy's token with its base URL, so
|
|
the proxy-issued key (Vikunja #237) swaps in by editing the file. A
|
|
proxy token without its URL doesn't count.
|
|
4. **Pi sessions see the credential variables too.** `ENV_ALLOW` passes
|
|
them to every session, so a pi session's bash can print them. That was
|
|
so before this row for whatever the shell had; it is now so for the
|
|
file's credential. The cli README says so. Narrowing `ENV_ALLOW` per
|
|
harness would be a change to `sessionEnv`, outside this brief.
|
|
5. **`claude` from `PATH` finds only `/usr`.** With `MOSAIC_CLAUDE_BIN`
|
|
unset, the adapter runs `claude` from `PATH`, and inside the sandbox
|
|
that is only a `claude` under `/usr`; a `~/.local/bin/claude` isn't
|
|
there. The installed unit pins `MOSAIC_CLAUDE_BIN` (decision 91), so
|
|
only a hand-started bus meets this. The cli README says so.
|
|
6. **Edits past the brief's file list.** `host.test.mjs` gains the unit's
|
|
one-`EnvironmentFile=` check, and `launch-host.mjs` passes the fake
|
|
adapter as a read-only bind (the sandbox otherwise hides it). The cli
|
|
README's refusal table gains its two rows, beyond the unit paragraph
|
|
the brief names.
|
|
7. **The test fixture strips the shell's credentials.** The cli fixture's
|
|
env started from `process.env`, so a credential in the shell running
|
|
the tests reached test sessions. `prepare()` now deletes every
|
|
`SESSION_CREDENTIALS` variable and sets a stand-in.
|
|
8. **NoNewPrivileges.** The unit sets `NoNewPrivileges=yes`. bwrap here
|
|
is not setuid and uses user namespaces, which work under no-new-privs:
|
|
`setpriv --no-new-privs` around `sandboxProblem()` returns null.
|
|
|
|
## Probes
|
|
|
|
Scratch probes, not part of the candidate:
|
|
|
|
- bwrap under `setpriv --no-new-privs`: works (choice 8).
|
|
- The host's `claude` inside the Claude Code binds with an empty
|
|
environment: `claude --version` gives 2.1.296; DNS resolves
|
|
`api.anthropic.com` and a request reaches it (405); HOME is
|
|
`<run>/home`; `claude -p` with no credential prints "Not logged in", exit
|
|
1. The founder's login in `~/.claude` is out of reach. Afterwards HOME
|
|
holds the `.claude` and `.claude.json` claude made itself.
|
|
|
|
## Tests
|
|
|
|
`packages/seat/tests/sandbox.test.mjs` (new):
|
|
|
|
1. `RUNNER_FILES` is exactly the runner's import graph plus the three
|
|
files the harness starts: the test walks the imports, so a new import
|
|
that isn't bound fails here, not in a live session.
|
|
2. `sandboxProblem` names a missing bwrap and one that fails.
|
|
3. Every bind has a `why`; `bundle/` and `session.json` are mounted
|
|
read-only after the read-write run directory; `/tmp` is last; no
|
|
launch socket for a role that may not launch; no `node_modules` for
|
|
Claude Code.
|
|
4. A real confined session (the real runner, a real broker, an adapter
|
|
that runs each message's body in `sh`) answers a table of 20 checks
|
|
against stand-ins under a scratch root: the stand-in
|
|
`~/.config/mosaic-dev/secrets/stand-in.env` and `~/.mosaic`, another
|
|
workspace, another run directory, the data root's bus store, the
|
|
checkout (`AGENTS.md`, the seat's own source), writes to `/`, beside the
|
|
workspace, in the data root, to a runner file, the bundle and the
|
|
session file: all refused. Positive: HOME is the run's empty `home/`,
|
|
TMPDIR is `/tmp`, the data root shows only this run, the workspace, the
|
|
run directory and `/tmp` take a write and a read, node runs.
|
|
5. Breaking it on purpose: the same session with the two stand-ins bound
|
|
in fails exactly their two checks; a sandbox from `binds()` passes all
|
|
20; without the workspace bind, exactly the workspace check fails.
|
|
|
|
`packages/seat/tests/session.test.mjs`: the existing real-session test now
|
|
runs in the sandbox and checks the runner's HOME is `<run>/home`. A new
|
|
test covers `sessionExit` (0, 23 and 128 stay codes; 137 and 143 become
|
|
SIGKILL and SIGTERM, ending `killed`) and `hasSessionCredential` (each of
|
|
the three sets alone passes; a proxy token or URL alone, empty strings and
|
|
nothing fail; every name is in `ENV_ALLOW`).
|
|
|
|
`packages/cli/tests/launcher.test.mjs`:
|
|
|
|
- A Claude Code PM with only a proxy token refuses `no-session-credential`
|
|
with the exact message, one log line, no session, no run directory.
|
|
- A bwrap that fails, then a missing one, refuse `sandbox-unavailable`
|
|
with bwrap's line and "is missing"; a session's launch request gets the
|
|
same code through `refuse()`; no session, no run.
|
|
- **The real Claude Code launch.** The PM (fake adapter) launches a coder
|
|
whose harness is Claude Code, with the real `adapters/claude` and the
|
|
host's `claude` pinned by `MOSAIC_CLAUDE_BIN`. Its credential is a
|
|
stand-in proxy key, `ANTHROPIC_AUTH_TOKEN` with `ANTHROPIC_BASE_URL`
|
|
pointing at the scripted Messages API from the harness tests, so no real
|
|
credential or model is involved. The script makes one `Bash` call that
|
|
tries nine things; the coder's reply carries the tool result. Refused:
|
|
the stand-in secret (`FAKE_TOKEN=stand-in-value`), the stand-in
|
|
`~/.mosaic`, the PM's run directory, the data root's `sessions.json`,
|
|
the checkout, a write outside. Allowed: a write and read in its
|
|
workspace and in `/tmp`, a writable HOME ending `/home`. The reply never
|
|
contains the secret's value. The test passes in about 3 seconds; a rerun
|
|
that printed the reply body gave
|
|
`ANSWER [[false,"0 no\n1 no\n2 no\n3 no\n4 no\n5 no\n6 yes\n7 yes\n8 yes"]]`.
|
|
With the stand-in secret added as a read-only bind, the test fails on
|
|
exactly `'read the stand-in secret': true` (`real-exposed.txt`).
|
|
- The test where the PM launches a coder now also reads both sessions'
|
|
bwrap command lines: the PM's binds the launch socket, the coder's
|
|
doesn't. This was the one mutant survivor (below).
|
|
- `prepare()` strips the shell's credential variables (choice 7).
|
|
|
|
`packages/cli/tests/host.test.mjs`: the rendered unit has exactly one
|
|
`EnvironmentFile=`, of the form `-%h/...`, and the cli README contains
|
|
that line.
|
|
|
|
The unit renders, and `systemd-analyze --user verify` on it prints
|
|
nothing.
|
|
|
|
## Mutants
|
|
|
|
`mutants.sh` with `agents/darkwing/work/queue-59-review/r1/mut.mjs`. Each
|
|
mutant edits one site (exactly one match), runs the test files it names
|
|
and restores the file. The run went 2026-10-12 20:43:33Z to 20:47:09Z,
|
|
after Dewey's row 89 run had ended (`mutants.txt`).
|
|
|
|
26 mutants. 23 were killed on the first run. Three needed a second look:
|
|
|
|
- `bind-checkout` and `probe-always-ok` were killed by a SyntaxError, not
|
|
by a test. Their replacement text was single-quoted, so it carried a
|
|
literal `\n`. I fixed the quoting and reran both. Each is now killed by
|
|
an assertion (the confined-shell table and the bwrap refusal test).
|
|
- `launch-socket-always` survived: a coder session given the launch
|
|
socket passed every test. The coder's sandbox couldn't reach the socket
|
|
in any test, because no test asked. The launcher test above now reads
|
|
the command lines, and the rerun kills it.
|
|
|
|
The table after the fixes, all 26 killed:
|
|
|
|
| Mutant | What it undoes | Killed by |
|
|
|---|---|---|
|
|
| no-workspace-bind | the workspace bind | confined shell table |
|
|
| runDir-read-only | the run directory is read-write | bind order test |
|
|
| bind-all-runs | the run bind is the run directory, not its parent | bind order test |
|
|
| bundle-writable | `bundle/` read-only over the run directory | bind order test |
|
|
| session-json-writable | `session.json` read-only | bind order test |
|
|
| no-private-tmp | the private `/tmp` | bind order test |
|
|
| bind-checkout | the checkout stays out | confined shell table |
|
|
| node-modules-always | `node_modules` only for pi | bind order test |
|
|
| runner-file-missing | a runner import left out of `RUNNER_FILES` | import walk |
|
|
| no-remount-ro | the read-only root | confined shell table |
|
|
| host-home | HOME in the run directory | confined shell table |
|
|
| no-pid-namespace | the PID namespace | the PM launch test |
|
|
| die-with-parent | bwrap's own `--die-with-parent` added | the host-died-hard test |
|
|
| no-pdeathsig | `setpriv --pdeathsig KILL` removed | the host-died-hard test: the runner outlives bwrap |
|
|
| exit-code-not-mapped | 128+n back to the signal | `sessionExit` test |
|
|
| recover-matches-unshare | `recover()` matching bwrap's command line | the host-died-hard test |
|
|
| no-sandbox-check | the `sandbox-unavailable` refusal | the bwrap refusal test |
|
|
| probe-always-ok | the probe | the bwrap refusal test |
|
|
| no-credential-check | the `no-session-credential` refusal | the credential refusal test |
|
|
| proxy-key-alone | the proxy set needs its URL | credential tests |
|
|
| proxy-key-ignored | the proxy set counts | credential tests |
|
|
| any-variable-will-do | every variable of a set | credential tests |
|
|
| claude-bin-not-bound | the pinned `claude` bind | the real Claude Code launch |
|
|
| launch-socket-always | the launch socket only for the launching role | the PM launches a coder test |
|
|
| unit-no-envfile | the unit's `EnvironmentFile=` | `host.test.mjs` |
|
|
| unit-envfile-required | its leading `-` | `host.test.mjs` |
|
|
|
|
My three-mutant rerun overlapped Filbert's row 95 run for about a minute:
|
|
I chained the busy check and the run in one command, so the run didn't
|
|
stop when the check found Filbert's processes.
|
|
|
|
## Gate
|
|
|
|
My run on the candidate, in a worktree of `af01465c` holding the 13 paths,
|
|
`TMPDIR` in my scratch (`gate.sh`, `gate-summary.txt`). Sequential, each
|
|
package's own test command from its directory, then every
|
|
`scripts/test-*.sh`, with `test-task.sh` under
|
|
`DOCKER_HOST=unix:///nonexistent`. Nothing of another seat's was running:
|
|
Filbert called the machine clear at 21:30:30Z and my check found no
|
|
process. 2026-10-12T21:30:49Z to 21:34:10Z, all rc=0.
|
|
|
|
| Suite | Result |
|
|
|---|---|
|
|
| seat | 33 pass |
|
|
| cli | 118 pass |
|
|
| harness | 59 pass |
|
|
| bus | 102 pass |
|
|
| webui | 84 pass |
|
|
| test-auth | 15 passed |
|
|
| test-conductor | 17 passed |
|
|
| test-config | 24 passed |
|
|
| test-discord | 66 passed |
|
|
| test-extension-package | 18 passed |
|
|
| test-foundation | 44 passed |
|
|
| test-queue | 27 passed |
|
|
| test-release | 14 passed |
|
|
| test-task | 26 passed; four Docker groups skip (adapter seam, workspace/capability, live task, live recall pair) |
|
|
|
|
No node suite skipped a test. The cli count includes the launch socket
|
|
check that the third mutant rerun needed; this is its first unmutated run.
|
|
One test made a compose network, `wt_default`, at 21:33:58Z; it was empty
|
|
afterwards and I removed it. The worktree still checks 13/13 against the
|
|
manifest after the run.
|