jason.woltjeandClaude Opus 5.5
a4fa775a51
Row 89 landed on packages/harness/README.md. The patch applies cleanly; gate 14/14 green and 34/34 mutants killed on the new base. Co-Authored-By: Claude Opus 5.5 <[email protected]>
1358 lines
76 KiB
Diff
1358 lines
76 KiB
Diff
diff --git a/packages/cli/README.md b/packages/cli/README.md
|
|
index c90fc3d7..1abec3d0 100644
|
|
--- a/packages/cli/README.md
|
|
+++ b/packages/cli/README.md
|
|
@@ -220,6 +220,8 @@ is `{"ok": true, "result": {run, instance, harness, model}}` or
|
|
| the instance isn't running | `instance-running` |
|
|
| the instance resolves to `pi` or `claude-code` and a model | `unsupported-harness` |
|
|
| its role has a contract file | `no-role-contract` |
|
|
+| a Claude Code session has `CLAUDE_CODE_OAUTH_TOKEN`, `ANTHROPIC_API_KEY`, or `ANTHROPIC_AUTH_TOKEN` with `ANTHROPIC_BASE_URL` to log in with | `no-session-credential` |
|
|
+| bwrap runs here | `sandbox-unavailable` |
|
|
| its model names exactly one `launch.max` family | `unknown-model-family` |
|
|
| that family has room; every running session counts, the PM's too | `capacity-full` |
|
|
| the broker authorizes `role.launch` for the caller | the broker's code (`launch-revoked`, `decision-required`, …) |
|
|
@@ -229,13 +231,31 @@ Every refusal is an `action.refused` event (the broker's `refuse`, or its
|
|
own refusal for the last broker check) and a line in the launch log. Then
|
|
the launcher reads the instance's credential status from the broker
|
|
(metadata only), builds the bundle in the run directory, spawns the runner
|
|
-under `unshare`, binds the outer process's pid and start time, adds the
|
|
+in its bwrap sandbox (`packages/seat/README.md`, "The sandbox"), binds the
|
|
+outer bwrap process's pid and start time, adds the
|
|
session to `sessions.json`, and writes the capability as one line on the
|
|
runner's stdin. The capability is never on disk, in argv or in the
|
|
environment. When the runner exits, the launcher sends `endLaunch` with the
|
|
end reason from its exit code (`packages/harness/README.md`), which writes
|
|
`session.ended`, releases the role and kills the run's capabilities.
|
|
|
|
+A Claude Code session logs in with `CLAUDE_CODE_OAUTH_TOKEN`,
|
|
+`ANTHROPIC_API_KEY`, or `ANTHROPIC_AUTH_TOKEN` with `ANTHROPIC_BASE_URL`
|
|
+(a proxy's key), from the host's environment, which the allowlist passes
|
|
+on. The bus unit loads it ("Unit" below). Nothing from the host's
|
|
+`~/.claude` reaches a session, whose `HOME` is its own. With none of them
|
|
+set, a Claude Code launch refuses `no-session-credential` before anything
|
|
+starts. A Pi session gets the same variables through the allowlist. A
|
|
+shell the harness opens, Claude Code's `Bash` or Pi's `bash`, can read
|
|
+them.
|
|
+
|
|
+A Pi session logs in only through the variables `ENV_ALLOW` passes. Its
|
|
+`HOME` is the run's own, so Pi never reads the operator's `~/.pi/agent`
|
|
+(`auth.json`, `models.json`, `settings.json`). A Pi launch has no
|
|
+credential check: a Pi role on a provider whose login lives only in those
|
|
+files, or whose key isn't an `ENV_ALLOW` variable, launches and then fails
|
|
+at its first turn.
|
|
+
|
|
The manifest records the Pi version from the repository's pin (the
|
|
`@earendil-works/pi-coding-agent` dependency in the root `package.json`)
|
|
and, for Claude Code, `--version` from the binary sessions run:
|
|
@@ -257,7 +277,7 @@ A host that starts reads the `sessions.json` the last host left. An entry
|
|
for this business means that host died without ending the launch, and its
|
|
run would keep holding the role. Before the launch socket opens, the
|
|
launcher SIGKILLs each such session that still runs (same start time,
|
|
-`unshare` on its command line), binds the run again with the recorded
|
|
+bwrap on its command line), binds the run again with the recorded
|
|
identity and ends it as `host-lost`, which releases the role. A
|
|
`sessions.json` that can't be read refuses the host start with exit 3; move
|
|
it aside after checking which sessions still run.
|
|
@@ -308,6 +328,7 @@ The unit settings:
|
|
first when it isn't one of those
|
|
- `Environment=MOSAIC_CLAUDE_BIN=`: the Claude Code binary that managed
|
|
sessions and the manifest's version run
|
|
+- `EnvironmentFile=`: the Claude Code session credential (below)
|
|
|
|
`MOSAIC_CLAUDE_BIN` comes from `--claude PATH`, or else `command -v claude`
|
|
in the installing shell. The path is kept as given, so the native install's
|
|
@@ -316,9 +337,21 @@ following its updates; reinstall to change it. It must be absolute, an
|
|
executable file, and only `A-Z a-z 0-9 / . _ + -`, because systemd expands
|
|
`%` and splits `Environment=` on spaces; anything else refuses with exit 1
|
|
and writes nothing. With no claude found, the unit gets a comment line
|
|
-instead and sessions run `claude` from the unit's `PATH`. A running host
|
|
+instead and sessions run `claude` from the unit's `PATH`, which in the
|
|
+sandbox finds only a `claude` under `/usr`. A running host
|
|
keeps the environment it started with until it restarts.
|
|
|
|
+The session credential comes from
|
|
+`EnvironmentFile=-%h/.config/mosaic-dev/secrets/claude-session.env`, the
|
|
+one place the path is set. The file holds one line, for example
|
|
+`CLAUDE_CODE_OAUTH_TOKEN=<token>`, mode 0600. The operator writes it;
|
|
+nothing in the repository creates or reads it, and no session sees the
|
|
+file itself, only the variable. The leading `-` lets the unit start
|
|
+without it, so Pi sessions still launch and a Claude Code launch refuses
|
|
+`no-session-credential`. To change the credential, edit or swap the file
|
|
+(or this line and this paragraph), then restart the unit; a running host
|
|
+keeps the environment it started with.
|
|
+
|
|
The unit has no `network-online.target` ordering, since a user unit cannot
|
|
order on that system target. The notifier needs no network at start: a
|
|
send that fails is journaled and retried.
|
|
diff --git a/packages/cli/src/launcher.mjs b/packages/cli/src/launcher.mjs
|
|
index 9ef08fcd..084264f5 100644
|
|
--- a/packages/cli/src/launcher.mjs
|
|
+++ b/packages/cli/src/launcher.mjs
|
|
@@ -13,17 +13,21 @@
|
|
// 3. the instance isn't already running (instance-running)
|
|
// 4. it resolves, has a role contract (no-role-contract) and a supported
|
|
// harness and model (unsupported-harness)
|
|
-// 5. its model is in exactly one launch.max family (unknown-model-family)
|
|
-// 6. that family has room; every running session counts, the PM's too
|
|
+// 5. a Claude Code session has a credential to log in with
|
|
+// (no-session-credential)
|
|
+// 6. bwrap runs here (sandbox-unavailable)
|
|
+// 7. its model is in exactly one launch.max family (unknown-model-family)
|
|
+// 8. that family has room; every running session counts, the PM's too
|
|
// (capacity-full)
|
|
-// 7. the broker authorizes role.launch for the caller (its own codes:
|
|
+// 9. the broker authorizes role.launch for the caller (its own codes:
|
|
// launch-revoked, gated, ...)
|
|
//
|
|
-// Steps 2 to 6 refuse through the broker's refuse(), so every refusal is an
|
|
-// action.refused event; step 7's refusals the broker records itself. Then
|
|
-// the host builds the bundle, spawns the runner, binds the run's process
|
|
-// identity (the outer unshare pid) and writes the new capability on the
|
|
-// runner's stdin. When the runner exits, the host ends the launch.
|
|
+// Steps 2 to 8 refuse through the broker's refuse(), so every refusal is an
|
|
+// action.refused event; step 9's refusals the broker records itself. Then
|
|
+// the host builds the bundle, spawns the runner in its sandbox
|
|
+// (packages/seat/src/sandbox.mjs) with the paths it may see, binds the run's
|
|
+// process identity (the outer bwrap pid) and writes the new capability on
|
|
+// the runner's stdin. When the runner exits, the host ends the launch.
|
|
//
|
|
// `bus start <business> --pm` launches launch.by the same way with
|
|
// launchedBy = the business's human and no caller capability.
|
|
@@ -39,17 +43,20 @@ import {
|
|
appendLaunchLog,
|
|
endReason,
|
|
family,
|
|
+ hasSessionCredential,
|
|
launchSocketPath,
|
|
newRun,
|
|
readSessions,
|
|
Registry,
|
|
runnerOf,
|
|
+ SESSION_CREDENTIALS,
|
|
sessionEnv,
|
|
+ sessionExit,
|
|
spawnSession,
|
|
- UNSHARE,
|
|
workspaceDir,
|
|
writeSessionFile,
|
|
} from "../../seat/src/session.mjs";
|
|
+import { BWRAP, sandboxProblem } from "../../seat/src/sandbox.mjs";
|
|
import { cmdlineOf, startTimeOf } from "../../seat/src/proc.mjs";
|
|
import { REPO, rolesDir } from "./config.mjs";
|
|
import { CliError } from "./errors.mjs";
|
|
@@ -84,7 +91,7 @@ export function claudeVersion(env = process.env) {
|
|
}
|
|
|
|
// host: startHost()'s handle. tracker: true when the broker has task verbs for
|
|
-// the business. adapters, namespace, sessionDefaults, versions and
|
|
+// the business. adapters, bwrap, readOnly, sessionDefaults, versions and
|
|
// stopTimeoutMs exist for the tests; the command line never sets them.
|
|
export function createLauncher({
|
|
host,
|
|
@@ -92,7 +99,8 @@ export function createLauncher({
|
|
env = process.env,
|
|
tracker = false,
|
|
adapters = ADAPTERS,
|
|
- namespace = true,
|
|
+ bwrap = BWRAP,
|
|
+ readOnly = [],
|
|
sessionDefaults = {},
|
|
versions = null,
|
|
stopTimeoutMs = STOP_TIMEOUT_MS,
|
|
@@ -132,7 +140,7 @@ export function createLauncher({
|
|
return run;
|
|
}
|
|
|
|
- // Steps 2 to 6. Returns what the launch needs. Only `--pm` may name
|
|
+ // Steps 2 to 8. Returns what the launch needs. Only `--pm` may name
|
|
// launch.by itself.
|
|
function check(instance, { pm = false } = {}) {
|
|
if (!b.launch) throw new Refusal("launch-not-configured", `business ${b.id} has no launch block`);
|
|
@@ -146,6 +154,12 @@ export function createLauncher({
|
|
throw new Refusal("unsupported-harness", e.message);
|
|
}
|
|
if (!resolved.contract) throw new Refusal("no-role-contract", `role ${resolved.definition} has no contract file`);
|
|
+ if (model.harness === "claude-code" && !hasSessionCredential(env)) {
|
|
+ const names = SESSION_CREDENTIALS.map((set) => set.join(" with ")).join(", ");
|
|
+ throw new Refusal("no-session-credential", `the bus host has none of ${names}; the bus unit's EnvironmentFile sets one when the bus starts`);
|
|
+ }
|
|
+ const unavailable = sandboxProblem({ bwrap });
|
|
+ if (unavailable) throw new Refusal("sandbox-unavailable", unavailable);
|
|
const fam = family(model.model, b.launch.max);
|
|
if (!fam) throw new Refusal("unknown-model-family", `model ${model.model} is in none, or more than one, of ${Object.keys(b.launch.max).join(", ")}`);
|
|
if (registry.count(b.id, fam) >= b.launch.max[fam]) throw new Refusal("capacity-full", `${fam} sessions: ${registry.count(b.id, fam)} of ${b.launch.max[fam]}`);
|
|
@@ -194,16 +208,25 @@ export function createLauncher({
|
|
turnMarker,
|
|
});
|
|
|
|
- const child = spawnSession({ runDir, env: sessionEnv(env, run), namespace });
|
|
+ const paths = {
|
|
+ workspace,
|
|
+ adapter: adapters[model.harness],
|
|
+ harness: model.harness,
|
|
+ brokerSocket: host.path,
|
|
+ launchSocket: resolved.launch ? launchSocketPath(dataRoot) : null,
|
|
+ claudeBin: env.MOSAIC_CLAUDE_BIN || null,
|
|
+ readOnly,
|
|
+ };
|
|
+ const child = spawnSession({ runDir, env: sessionEnv(env, run), paths, bwrap });
|
|
let bound = false;
|
|
let entry = null;
|
|
- const exited = new Promise((resolve) => child.once("close", (code, signal) => resolve({ code, signal })));
|
|
+ const exited = new Promise((resolve) => child.once("close", (code, signal) => resolve(sessionExit(code, signal))));
|
|
child.on("error", (e) => log(`run ${run}: ${e.message}`));
|
|
child.stdin.on("error", () => {});
|
|
try {
|
|
if (child.pid === undefined) throw new Error("the session process did not start");
|
|
const outerStart = startTimeOf(child.pid);
|
|
- const runner = await runnerOf(child, { namespace });
|
|
+ const runner = await runnerOf(child);
|
|
if (!outerStart) throw new Error("the session process exited at once");
|
|
const launch = await host.bindLaunch({ business: b.id, role: instance, run, harness: model.harness, pid: child.pid, startTime: outerStart });
|
|
bound = true;
|
|
@@ -308,7 +331,7 @@ export function createLauncher({
|
|
// its identical launch record, which the broker accepts.
|
|
async function recover() {
|
|
for (const s of leftover) {
|
|
- if (startTimeOf(s.pid) === s.startTime && (cmdlineOf(s.pid) ?? [])[0] === UNSHARE[0]) {
|
|
+ if (startTimeOf(s.pid) === s.startTime && (cmdlineOf(s.pid) ?? [])[0] === bwrap) {
|
|
try {
|
|
process.kill(s.pid, "SIGKILL");
|
|
} catch {}
|
|
diff --git a/packages/cli/systemd/mosaic-bus.service.in b/packages/cli/systemd/mosaic-bus.service.in
|
|
index 8f82f933..77be6912 100644
|
|
--- a/packages/cli/systemd/mosaic-bus.service.in
|
|
+++ b/packages/cli/systemd/mosaic-bus.service.in
|
|
@@ -17,6 +17,11 @@ Environment=PATH=@PATH@
|
|
# claudeVersion run, pinned at install (`--claude PATH`, default
|
|
# `command -v claude`). Without it they run `claude` from PATH above.
|
|
@CLAUDE_ENV@
|
|
+# Claude Code sessions log in with what this 0600 file sets, which the
|
|
+# operator writes: CLAUDE_CODE_OAUTH_TOKEN, ANTHROPIC_API_KEY, or a proxy's
|
|
+# ANTHROPIC_AUTH_TOKEN with ANTHROPIC_BASE_URL. The `-` lets the unit start
|
|
+# without it, and a Claude Code launch then refuses.
|
|
+EnvironmentFile=-%h/.config/mosaic-dev/secrets/claude-session.env
|
|
# The host reads the system config and the business file, boots the broker
|
|
# (packages/bus/src/process.mjs) and starts the notifier. Config problems and
|
|
# refusals exit 3, usage 4, invalid input 2: none of those is retried. A
|
|
diff --git a/packages/cli/tests/fixtures/launch-host.mjs b/packages/cli/tests/fixtures/launch-host.mjs
|
|
index 0acc8424..e4cbd21b 100644
|
|
--- a/packages/cli/tests/fixtures/launch-host.mjs
|
|
+++ b/packages/cli/tests/fixtures/launch-host.mjs
|
|
@@ -1,12 +1,13 @@
|
|
// A bus host with its launcher in a process of its own, so a test can kill
|
|
// it hard. It launches the PM, prints the launch result as one JSON line and
|
|
-// runs until it is killed. argv: the adapter to use for both harnesses.
|
|
+// runs until it is killed. argv: the adapter to use for both harnesses, then
|
|
+// the files the sandbox shows the session read-only (the fake adapter).
|
|
|
|
import { bootConfig, loadSystem } from "../../src/config.mjs";
|
|
import { startHost } from "../../src/host.mjs";
|
|
import { createLauncher } from "../../src/launcher.mjs";
|
|
|
|
-const [adapter] = process.argv.slice(2);
|
|
+const [adapter, ...readOnly] = process.argv.slice(2);
|
|
const system = loadSystem({ env: process.env });
|
|
const boot = bootConfig({ system, businessId: "acme", env: process.env });
|
|
const host = await startHost({ boot, business: "acme", log: () => {} });
|
|
@@ -17,6 +18,7 @@ const launcher = createLauncher({
|
|
adapters: { pi: adapter, "claude-code": adapter },
|
|
sessionDefaults: { pollInterval: 100 },
|
|
versions: { pi: "0.85.1", claude: null },
|
|
+ readOnly,
|
|
log: () => {},
|
|
});
|
|
await launcher.listen();
|
|
diff --git a/packages/cli/tests/host.test.mjs b/packages/cli/tests/host.test.mjs
|
|
index 6f0961af..b6644cf3 100644
|
|
--- a/packages/cli/tests/host.test.mjs
|
|
+++ b/packages/cli/tests/host.test.mjs
|
|
@@ -482,6 +482,12 @@ test("bus-service.sh renders the unit and installs it into a given directory", (
|
|
assert.equal(render.status, 0, render.stderr);
|
|
assert.match(render.stdout, new RegExp(`ExecStart=${REPO.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}/scripts/mosaic bus start %i`));
|
|
assert.match(render.stdout, /RestartPreventExitStatus=2 3 4/);
|
|
+ // One session credential file, optional so Pi sessions start without it.
|
|
+ // The README's paragraph names the same file; nothing else does.
|
|
+ const files = render.stdout.match(/^EnvironmentFile=.*$/gm);
|
|
+ assert.equal(files.length, 1, files.join("\n"));
|
|
+ assert.match(files[0], /^EnvironmentFile=-%h\/\S+$/);
|
|
+ assert.ok(readFileSync(join(REPO, "packages", "cli", "README.md"), "utf8").includes(`\`${files[0]}\``), "the README names the unit's file");
|
|
assert.doesNotMatch(render.stdout, /@REPO@|@PATH@|@CLAUDE_ENV@/);
|
|
// A user unit cannot order on a system target (Darkwing F3).
|
|
assert.doesNotMatch(render.stdout, /network-online/);
|
|
diff --git a/packages/cli/tests/launcher.test.mjs b/packages/cli/tests/launcher.test.mjs
|
|
index 15b36fe0..f8013726 100644
|
|
--- a/packages/cli/tests/launcher.test.mjs
|
|
+++ b/packages/cli/tests/launcher.test.mjs
|
|
@@ -1,5 +1,5 @@
|
|
// The S6 launcher inside a real bus host: real broker child, real runners
|
|
-// under unshare, the fake adapter from packages/harness standing in for the
|
|
+// in the bwrap sandbox, the fake adapter from packages/harness standing in for the
|
|
// harness. The PM launches a coder through its `launch` tool; every host
|
|
// refusal comes back over the launch socket and lands in the launch log.
|
|
|
|
@@ -9,20 +9,21 @@ import { spawn, spawnSync } from "node:child_process";
|
|
import { once } from "node:events";
|
|
import { createInterface } from "node:readline";
|
|
import { connect } from "node:net";
|
|
-import { chmodSync, existsSync, mkdirSync, readFileSync, readdirSync, statSync, writeFileSync } from "node:fs";
|
|
+import { chmodSync, existsSync, mkdirSync, readFileSync, readdirSync, rmSync, statSync, writeFileSync } from "node:fs";
|
|
import { join } from "node:path";
|
|
import { fileURLToPath } from "node:url";
|
|
import { Client } from "../../bus/src/client.mjs";
|
|
-import { launchLogFile, launchSocketPath, readSessions, Registry, sessionsFile, UNSHARE } from "../../seat/src/session.mjs";
|
|
+import { launchLogFile, launchSocketPath, readSessions, Registry, SESSION_CREDENTIALS, sessionsFile } from "../../seat/src/session.mjs";
|
|
+import { sandboxProblem } from "../../seat/src/sandbox.mjs";
|
|
import { bootConfig, loadSystem } from "../src/config.mjs";
|
|
import { startHost, startTimeOf } from "../src/host.mjs";
|
|
import { claudeVersion, createLauncher } from "../src/launcher.mjs";
|
|
+import { mockAnthropic, REPO } from "../../harness/tests/helpers.mjs";
|
|
import { fixture, tmp } from "./helpers.mjs";
|
|
|
|
const FAKE = fileURLToPath(new URL("../../harness/tests/fixtures/fake-adapter.mjs", import.meta.url));
|
|
const HOST = fileURLToPath(new URL("./fixtures/launch-host.mjs", import.meta.url));
|
|
-const NAMESPACES = spawnSync(UNSHARE[0], [...UNSHARE.slice(1), "true"]).status === 0;
|
|
-const skip = NAMESPACES ? false : "unprivileged user and PID namespaces are unavailable here";
|
|
+const skip = sandboxProblem() ?? false;
|
|
|
|
// pm opus, coder and reviewer sonnet, one of each family; cto is listed but
|
|
// runs the system model, which is in no family.
|
|
@@ -44,9 +45,13 @@ async function until(fn, ms = 15000) {
|
|
}
|
|
|
|
// A config and data root with the fake adapter; setup() starts a host on it.
|
|
+// The session credential is a stand-in, and the fake adapter never uses it.
|
|
+// Every credential variable of the shell running the tests is dropped.
|
|
function prepare(t, more = (doc) => doc) {
|
|
const root = tmp(t, "mosaic-launch-");
|
|
const f = fixture(root, "acme", (doc) => more(edit(doc)));
|
|
+ for (const k of SESSION_CREDENTIALS.flat()) delete f.env[k];
|
|
+ f.env.CLAUDE_CODE_OAUTH_TOKEN = "stand-in";
|
|
mkdirSync(f.dataRoot, { mode: 0o700 });
|
|
const adapter = join(root, "adapter.sh");
|
|
writeFileSync(adapter, `exec '${process.execPath}' '${FAKE}'\n`, { mode: 0o644 });
|
|
@@ -67,6 +72,7 @@ async function setup(t, more = (doc) => doc, { root, f, adapter } = prepare(t, m
|
|
adapters: { pi: adapter, "claude-code": adapter },
|
|
sessionDefaults: { pollInterval: 100 },
|
|
versions: { pi: "0.85.1", claude: null },
|
|
+ readOnly: [FAKE],
|
|
log: (l) => logs.push(l),
|
|
...options,
|
|
});
|
|
@@ -117,7 +123,7 @@ test("bus start --pm: the PM runs under its own PID namespace, registered, with
|
|
assert.equal(s.live, true);
|
|
assert.equal(s.family, "opus");
|
|
assert.equal(s.launchedBy, "jason");
|
|
- assert.notEqual(s.pid, s.runnerPid, "the runner is the unshare process's child");
|
|
+ assert.notEqual(s.pid, s.runnerPid, "the runner is the bwrap process's child");
|
|
assert.equal(statSync(sessionsFile(f.dataRoot)).mode & 0o777, 0o600);
|
|
|
|
const runDir = s.runDir;
|
|
@@ -190,6 +196,10 @@ test("the PM launches a coder through its launch tool; the coder answers; refusa
|
|
assert.equal(coder.run, result.run);
|
|
assert.equal(coder.launchedBy, "pm");
|
|
assert.equal(coder.live, true);
|
|
+ // Only the role that may launch has the launch socket in its sandbox.
|
|
+ const bound = (pid) => readFileSync(`/proc/${pid}/cmdline`, "utf8").split("\0").includes(sock);
|
|
+ assert.equal(bound(readSessions(f.dataRoot).find((s) => s.instance === "pm").pid), true);
|
|
+ assert.equal(bound(coder.pid), false);
|
|
|
|
// The coder claimed its role and answers its own messages.
|
|
const hello = await client.call("message.send", { to: "coder", class: "REQUEST", body: "hello" });
|
|
@@ -197,7 +207,7 @@ test("the PM launches a coder through its launch tool; the coder answers; refusa
|
|
assert.equal(echo.from_role, "coder");
|
|
assert.match(echo.body, /^echo: Message .* from cto, class REQUEST:\n\nhello/);
|
|
|
|
- // Host refusals, asked with the cto capability (steps 2 to 6 come before
|
|
+ // Host refusals, asked with the cto capability (steps 2 to 8 come before
|
|
// the broker's role.launch check, which cto would fail).
|
|
const refusals = [
|
|
["pm", "instance-not-listed"],
|
|
@@ -249,11 +259,131 @@ test("a runner that stops at once ends its launch with the runner's reason", { s
|
|
assert.match(readFileSync(join(f.dataRoot, "launches", "acme", pm.run, "runner.log"), "utf8"), /vikunja/);
|
|
});
|
|
|
|
+test("a Claude Code launch with no session credential refuses before it starts", { timeout: 30000 }, async (t) => {
|
|
+ const given = prepare(t, (doc) => {
|
|
+ doc.roles.pm.vars.harness = "claude-code";
|
|
+ return doc;
|
|
+ });
|
|
+ delete given.f.env.CLAUDE_CODE_OAUTH_TOKEN;
|
|
+ given.f.env.ANTHROPIC_AUTH_TOKEN = "stand-in"; // a proxy key with no proxy
|
|
+ const { f, launcher } = await setup(t, undefined, given);
|
|
+ const names = "CLAUDE_CODE_OAUTH_TOKEN, ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN with ANTHROPIC_BASE_URL";
|
|
+ await assert.rejects(launcher.launchPm(), (e) => e.exitCode === 3 && e.message === `PM launch refused: no-session-credential (the bus host has none of ${names}; the bus unit's EnvironmentFile sets one when the bus starts)`);
|
|
+ assert.deepEqual(log(f).map((e) => [e.event, e.code]), [["refused", "no-session-credential"]]);
|
|
+ assert.deepEqual(readSessions(f.dataRoot), []);
|
|
+ assert.deepEqual(readdirSync(join(f.dataRoot, "launches")), ["acme.jsonl"], "no run was made");
|
|
+});
|
|
+
|
|
+test("a pi launch needs no session credential: pi logs in through its own ENV_ALLOW variables", { skip, timeout: 30000 }, async (t) => {
|
|
+ const given = prepare(t);
|
|
+ delete given.f.env.CLAUDE_CODE_OAUTH_TOKEN;
|
|
+ assert.deepEqual(SESSION_CREDENTIALS.flat().filter((k) => k in given.f.env), []);
|
|
+ const { f, launcher } = await setup(t, undefined, given);
|
|
+ const pm = await launcher.launchPm();
|
|
+ assert.equal(pm.harness, "pi");
|
|
+ assert.deepEqual(log(f).map((e) => [e.event, e.instance]), [["launch", "pm"]]);
|
|
+});
|
|
+
|
|
+// The runner is pid 1 in its namespace and ignores a SIGKILL sent from
|
|
+// inside it, so the host kills it by its host pid. bwrap then exits 137,
|
|
+// which the launcher reads back as the signal.
|
|
+test("a runner killed from the host ends its launch as killed, not failed", { skip, timeout: 30000 }, async (t) => {
|
|
+ const { f, launcher } = await setup(t);
|
|
+ const pm = await launcher.launchPm();
|
|
+ const runnerLog = join(f.dataRoot, "launches", "acme", pm.run, "runner.log");
|
|
+ await until(() => existsSync(runnerLog) && readFileSync(runnerLog, "utf8").includes("claimed by run"));
|
|
+ const [s] = readSessions(f.dataRoot);
|
|
+ process.kill(s.runnerPid, "SIGKILL");
|
|
+ const end = await until(() => log(f).find((e) => e.event === "end" && e.run === pm.run));
|
|
+ assert.deepEqual([end.reason, end.exitCode, end.signal], ["killed", null, "SIGKILL"]);
|
|
+ assert.deepEqual(readSessions(f.dataRoot), []);
|
|
+});
|
|
+
|
|
+test("a missing or failing bwrap refuses sandbox-unavailable before anything starts; there is no unconfined fallback",{ timeout: 30000 }, async (t) => {
|
|
+ const given = prepare(t);
|
|
+ const bad = join(given.root, "bwrap");
|
|
+ writeFileSync(bad, "#!/bin/sh\necho 'bwrap: setting up uid map: Permission denied' >&2\nexit 1\n", { mode: 0o755 });
|
|
+ const { f, launcher, cto } = await setup(t, undefined, given, { bwrap: bad });
|
|
+ await assert.rejects(launcher.launchPm(), (e) => e.message === `PM launch refused: sandbox-unavailable (${bad} failed: bwrap: setting up uid map: Permission denied)`);
|
|
+ rmSync(bad);
|
|
+ await assert.rejects(launcher.launchPm(), (e) => e.message === `PM launch refused: sandbox-unavailable (${bad} is missing)`);
|
|
+ // A session's launch request gets the same code, through refuse().
|
|
+ assert.deepEqual(await launcher.fromSession(cto.cap, "coder"), { ok: false, error: "sandbox-unavailable" });
|
|
+ assert.deepEqual(log(f).map((e) => e.code), ["sandbox-unavailable", "sandbox-unavailable", "sandbox-unavailable"]);
|
|
+ assert.deepEqual(readSessions(f.dataRoot), []);
|
|
+ assert.deepEqual(readdirSync(join(f.dataRoot, "launches")), ["acme.jsonl"], "no run was made");
|
|
+});
|
|
+
|
|
+// The brief's one real Claude Code launch: the host's claude in a coder's
|
|
+// sandbox, logged in with a stand-in proxy key against the scripted Messages
|
|
+// API, runs one Bash call that tries the founder's stand-ins, another run
|
|
+// and the checkout. No real credential and no real model.
|
|
+const which = spawnSync("sh", ["-c", "command -v claude"], { encoding: "utf8" });
|
|
+const CLAUDE = which.status === 0 ? which.stdout.trim() : null;
|
|
+
|
|
+test("a real Claude Code coder's shell can't read a stand-in secret or anything else its sandbox doesn't hold", { skip: skip || (CLAUDE ? false : "claude is not on PATH"), timeout: 180000 }, async (t) => {
|
|
+ const script = [];
|
|
+ const api = await mockAnthropic(t, { script, done: (r) => `ANSWER ${JSON.stringify(r.map((x) => [x.isError, x.content.slice(0, 1500)]))}` });
|
|
+ const given = prepare(t, (doc) => {
|
|
+ doc.roles.coder.vars.harness = "claude-code";
|
|
+ doc.roles.coder.vars["limits.tools"] = ["bash"];
|
|
+ return doc;
|
|
+ });
|
|
+ delete given.f.env.CLAUDE_CODE_OAUTH_TOKEN;
|
|
+ Object.assign(given.f.env, { ANTHROPIC_AUTH_TOKEN: "stand-in", ANTHROPIC_BASE_URL: api.url, MOSAIC_CLAUDE_BIN: CLAUDE });
|
|
+ const secret = join(given.root, "home", ".config", "mosaic-dev", "secrets", "stand-in.env");
|
|
+ const mosaic = join(given.root, "home", ".mosaic", "fleet", "token");
|
|
+ for (const p of [secret, mosaic]) {
|
|
+ mkdirSync(join(p, ".."), { recursive: true, mode: 0o700 });
|
|
+ writeFileSync(p, "FAKE_TOKEN=stand-in-value\n", { mode: 0o600 });
|
|
+ }
|
|
+ const claudeAdapter = join(REPO, "adapters", "claude", "adapter.sh");
|
|
+ const { f, launcher, client } = await setup(t, undefined, given, { adapters: { pi: given.adapter, "claude-code": claudeAdapter } });
|
|
+ const pm = await launcher.launchPm();
|
|
+ const pmRun = readSessions(f.dataRoot).find((s) => s.run === pm.run).runDir;
|
|
+
|
|
+ // [what, shell command, may the session do it]
|
|
+ const probes = [
|
|
+ ["read the stand-in secret", `cat '${secret}'`, false],
|
|
+ ["read the stand-in ~/.mosaic", `cat '${mosaic}'`, false],
|
|
+ ["read the PM's run directory", `cat '${pmRun}/session.json'`, false],
|
|
+ ["read the data root's sessions file", `cat '${sessionsFile(f.dataRoot)}'`, false],
|
|
+ ["read the checkout", `cat '${join(REPO, "AGENTS.md")}'`, false],
|
|
+ ["write outside", `touch '${given.root}/escaped'`, false],
|
|
+ ["write and read its workspace", "echo ws > probe && cat probe", true],
|
|
+ ["write and read /tmp", "echo tmp > /tmp/probe && cat /tmp/probe", true],
|
|
+ ["HOME is the run's home/", `case "$HOME" in */home) test -w "$HOME";; *) false;; esac`, true],
|
|
+ ];
|
|
+ const command = probes.map(([what, cmd], i) => `if { ${cmd}; } >/dev/null 2>&1; then echo "${i} yes"; else echo "${i} no"; fi`).join("\n");
|
|
+ script.push({ name: "Bash", input: { command, description: "probe the sandbox" } });
|
|
+
|
|
+ const sent = await client.call("message.send", { to: "pm", class: "REQUEST", body: 'TOOL launch {"instance":"coder"}' });
|
|
+ const launched = await until(async () => (await client.call("message.receive")).find((m) => m.in_reply_to === sent.id));
|
|
+ assert.equal(JSON.parse(launched.body).instance, "coder", launched.body);
|
|
+ const coder = readSessions(f.dataRoot).find((s) => s.instance === "coder");
|
|
+ assert.equal(coder.harness, "claude-code");
|
|
+
|
|
+ const hello = await client.call("message.send", { to: "coder", class: "REQUEST", body: "probe" });
|
|
+ const reply = await until(async () => (await client.call("message.receive")).find((m) => m.in_reply_to === hello.id), 150000);
|
|
+ assert.equal(reply.from_role, "coder");
|
|
+ const answer = reply.body.match(/ANSWER (\[.*\])/s);
|
|
+ assert.ok(answer, reply.body);
|
|
+ const [[isError, out]] = JSON.parse(answer[1]);
|
|
+ assert.equal(isError, false, out);
|
|
+ const got = Object.fromEntries(out.trim().split("\n").map((l) => l.split(" ")).map(([i, v]) => [probes[i][0], v === "yes"]));
|
|
+ assert.deepEqual(got, Object.fromEntries(probes.map(([what, , may]) => [what, may])));
|
|
+ assert.ok(!existsSync(join(given.root, "escaped")));
|
|
+ assert.ok(!reply.body.includes("stand-in-value"), "the secret's value never comes back");
|
|
+ // The session logged in with the stand-in proxy key, against the mock.
|
|
+ assert.ok(api.requests.length > 0);
|
|
+ assert.ok(api.requests.some((r) => r.tools.includes("Bash")), api.requests.map((r) => r.tools.join(",")).join(" | "));
|
|
+});
|
|
+
|
|
for (const exited of [false, true]) {
|
|
test(`a host that died hard leaves its sessions to the next host, which ${exited ? "finds one already exited (23)" : "kills them"} and ends their runs so the role can be launched again`, { skip, timeout: 30000 }, async (t) => {
|
|
const given = prepare(t);
|
|
const { f } = given;
|
|
- const old = spawn(process.execPath, [HOST, given.adapter], { env: f.env, stdio: ["ignore", "pipe", "inherit"] });
|
|
+ const old = spawn(process.execPath, [HOST, given.adapter, FAKE], { env: f.env, stdio: ["ignore", "pipe", "inherit"] });
|
|
t.after(() => old.kill("SIGKILL"));
|
|
const [line] = await once(createInterface({ input: old.stdout }), "line");
|
|
const first = JSON.parse(line);
|
|
diff --git a/packages/harness/README.md b/packages/harness/README.md
|
|
index 1d1d7c5b..7c040c66 100644
|
|
--- a/packages/harness/README.md
|
|
+++ b/packages/harness/README.md
|
|
@@ -176,28 +176,41 @@ Each turn's stderr goes to `turns/NNNN.stderr` (0600) in the run directory.
|
|
|
|
These are limits, not bugs, and nothing in this package claims otherwise.
|
|
|
|
-- **The gate is not a sandbox.** It gates the harness's own tool calls.
|
|
- `bash`, where the role's tool limit grants it, can reach anything the OS
|
|
- user can (S0 line 5).
|
|
-- **Same UID.** Every session runs as the host's user. Founder files in
|
|
- `HOME` (`~/.git-credentials`, the tea config, `~/.ssh`) are readable
|
|
- through `bash`; only the environment is filtered. `/tmp` is shared. The
|
|
- PID namespace (below) hides other processes, so a session can't signal
|
|
- or `ptrace` another one by pid.
|
|
-- **The broker socket is shared.** A session that can run `bash` can
|
|
+- **The gate is not the sandbox.** It gates the harness's own tool calls.
|
|
+ `bash`, where the role's tool limit grants it (S0 line 5), reaches what
|
|
+ the session's sandbox holds (`packages/seat/README.md`, "The sandbox"):
|
|
+ the system directories, the runner's own repository files, its
|
|
+ workspace and its run directory. The data root, other workspaces and
|
|
+ runs, `~/.config`, `~/.mosaic`, `~/.ssh`, `~/.claude` and the rest of
|
|
+ the checkout aren't there. `HOME` and `/tmp` are the run's own. The
|
|
+ session's credential variable is in its environment, so `bash` (or
|
|
+ Claude Code's `Bash`) can read it (`packages/cli/README.md`,
|
|
+ "Sessions").
|
|
+- **Same UID.** The sandbox's user namespace maps the session to the
|
|
+ host's user. It can't reach the host's other files, but anything outside the
|
|
+ sandbox that it can talk to (below) acts with that user's full rights.
|
|
+- **Network and abstract sockets aren't confined.** The session shares
|
|
+ the host's network namespace: it reaches the network, every service on
|
|
+ `127.0.0.1`, and every abstract Unix socket (a name starting with a NUL
|
|
+ byte, which no mount hides). A service listening there, an X server or a
|
|
+ session bus that offers one, acts for the session outside the sandbox.
|
|
+ The shared namespace also shows up in `/proc/net/unix`, which lists the
|
|
+ host's Unix sockets by name, path ones included; a name gives no access
|
|
+ to a path socket the mounts hide.
|
|
+ The policy's `network` field is recorded, not enforced.
|
|
+- **The broker socket is bound in.** A session that can run `bash` can
|
|
connect to it; it still needs a capability to do anything there. The
|
|
- tool socket is reachable through `bash` as well, and acts with the
|
|
- session's own capability only.
|
|
-- **Network.** Abstract Unix sockets and the network are not confined; the
|
|
- policy's `network` field is recorded, not enforced.
|
|
+ tool socket is in the run directory and acts with the session's own
|
|
+ capability only. The launch socket is bound in only for the role whose
|
|
+ launch tool calls it.
|
|
- **PID namespace.** Each session's runner is pid 1 of its own namespace
|
|
- (`unshare --user --map-current-user --pid --fork --kill-child
|
|
- --mount-proc`). It doesn't reap zombies; turns are reaped by the runner's
|
|
- own waits, and the namespace dies with it. A same-UID process can still
|
|
- ask something outside its tree (a systemd user manager, an existing tmux
|
|
- server) to run a command.
|
|
-- **`mosaic stop`** is same-UID process control: it checks the registry's
|
|
- start time and that the process is a runner before sending SIGTERM.
|
|
+ (bwrap `--unshare-pid --as-pid-1`), with its own IPC namespace. A
|
|
+ session sees no other process, so it can't signal or `ptrace` one by
|
|
+ pid. The runner doesn't reap zombies; turns are reaped by the runner's
|
|
+ own waits, and the namespace dies with it.
|
|
+- **`mosaic stop`** is same-UID process control from outside the sandbox:
|
|
+ it checks the registry's start time and that the process is a runner
|
|
+ before sending SIGTERM.
|
|
- **If the host dies,** sessions keep running until their broker polls fail
|
|
`brokerRetries` times, then exit 23. The next host kills what still runs
|
|
and ends those launches as `host-lost` (`packages/cli/README.md`,
|
|
@@ -224,7 +237,7 @@ These are limits, not bugs, and nothing in this package claims otherwise.
|
|
another process of the same user) isn't seen. That is the same reach as
|
|
`bash` itself.
|
|
- **The system prompt is in argv** for both adapters, so the same UID can
|
|
- read it in `/proc/<pid>/cmdline`; the PID namespace hides it from other
|
|
- sessions. It holds no secret.
|
|
+ read it in `/proc/<pid>/cmdline` outside the sandbox; the PID namespace
|
|
+ hides it from other sessions. It holds no secret.
|
|
- **Pi adapter paths** with spaces break its unquoted `-e` and skill
|
|
splitting; the launcher's paths don't contain spaces.
|
|
diff --git a/packages/harness/src/gate.mjs b/packages/harness/src/gate.mjs
|
|
index e68e8f92..4bb3240c 100644
|
|
--- a/packages/harness/src/gate.mjs
|
|
+++ b/packages/harness/src/gate.mjs
|
|
@@ -3,10 +3,10 @@
|
|
// everything else is blocked, and every path argument of a file tool must
|
|
// resolve inside the workspace.
|
|
//
|
|
-// This is a gate on the harness's own tool calls, not a sandbox. `bash`
|
|
-// isn't confined (the role's tool limit decides whether it exists at all),
|
|
-// and anything bash can reach, the session can reach. See the README's
|
|
-// limits.
|
|
+// This is a gate on the harness's own tool calls, not the sandbox. `bash`
|
|
+// isn't gated (the role's tool limit decides whether it exists at all):
|
|
+// what it can reach is what the session's bubblewrap sandbox holds
|
|
+// (packages/seat/src/sandbox.mjs). See the README's limits.
|
|
|
|
import { lstatSync, realpathSync } from "node:fs";
|
|
import { homedir } from "node:os";
|
|
diff --git a/packages/seat/README.md b/packages/seat/README.md
|
|
index 64700dbd..836a356f 100644
|
|
--- a/packages/seat/README.md
|
|
+++ b/packages/seat/README.md
|
|
@@ -165,10 +165,10 @@ launch flow and the on-disk layout.
|
|
(`ENV_ALLOW`), the repository's `node_modules/.bin` on `PATH`, and
|
|
`MOSAIC_RUN_ID`. Founder credential variables never pass.
|
|
`MOSAIC_CLAUDE_BIN`, the host's pinned Claude Code binary, does.
|
|
-- `spawnSession` starts `packages/harness/src/runner.mjs` under
|
|
- `unshare --user --map-current-user --pid --fork --kill-child --mount-proc`,
|
|
- detached, with stdin open for the capability line and output to
|
|
- `runner.log` (0600). `runnerOf` finds the runner inside it.
|
|
+- `spawnSession` starts `packages/harness/src/runner.mjs` in its sandbox
|
|
+ (`src/sandbox.mjs`, below), detached, with stdin open for the capability
|
|
+ line and output to `runner.log` (0600). `runnerOf` finds the runner, the
|
|
+ outer bwrap process's only child.
|
|
- `Registry` is the host's list of running sessions, mirrored to
|
|
`bus-host/sessions.json` (0600); `readSessions` reads it and marks each
|
|
entry live or stale by the runner's start time, and refuses a file it
|
|
@@ -176,22 +176,63 @@ launch flow and the on-disk layout.
|
|
- `stopSession` is `mosaic stop`: SIGTERM to a live runner, resent every
|
|
second until it exits.
|
|
|
|
-Limits of the PID namespace, which are limits and not a sandbox:
|
|
-
|
|
-- The runner is pid 1 of its namespace. A detached child is reparented to
|
|
- it, not to the user's init, so it stays under the launch's pid. Pid 1
|
|
- ignores a signal it has no handler for; the runner installs its handlers
|
|
- first, and the senders resend SIGTERM to cover Node's startup.
|
|
-- The namespace isolates pids only. With its own `/proc`, a session sees
|
|
- no process outside it, so it can't signal or `ptrace` another session or
|
|
- the host by pid (checked on this host, Yama `ptrace_scope` 1). It still
|
|
- runs as the host's user, with that user's files, `/tmp`, network and
|
|
- sockets.
|
|
-- A session can still ask a process outside its namespace (a systemd user
|
|
- manager, an existing tmux server) to run a command for it.
|
|
+### The sandbox (#1586)
|
|
+
|
|
+`src/sandbox.mjs` builds the bubblewrap command line a session runs under.
|
|
+The mount namespace starts from an empty root and gets only these binds,
|
|
+each at its own host path; `binds()` returns them with the reason for each,
|
|
+and the file's header comment has the table.
|
|
+
|
|
+- Read-only: `/usr`, `/etc` and the host's `/bin`, `/lib`, `/lib64`,
|
|
+ `/sbin` links into `/usr`; the ten repository files the runner loads
|
|
+ (`RUNNER_FILES`); the adapter; for pi the repository's `node_modules`,
|
|
+ for Claude Code the `MOSAIC_CLAUDE_BIN` binary when it lives outside
|
|
+ `/usr`; node when it lives outside `/usr`; the broker socket, and the
|
|
+ launch socket for the role whose launch tool calls it.
|
|
+- Read-write: the instance's workspace and the run directory. The run's
|
|
+ `bundle/` and `session.json` are bound read-only again on top.
|
|
+- Private: `/tmp` is the run's `tmp/`, `HOME` the run's empty `home/`, and
|
|
+ `/dev` and `/proc` are the namespace's own.
|
|
+
|
|
+The root is remounted read-only, so a write anywhere else fails. Nothing
|
|
+else of the host's filesystem exists inside: no data root, no other
|
|
+workspace or run, no `~/.config`, `~/.mosaic`, `~/.ssh` or `~/.claude`, no
|
|
+checkout beyond the runner's files. Claude Code logs in with a
|
|
+credential variable the environment passes (`SESSION_CREDENTIALS`;
|
|
+`packages/cli/README.md`, "Sessions"), never with a file from the host's
|
|
+`~/.claude`. Pi likewise logs in only through `ENV_ALLOW`'s variables,
|
|
+never with the host's `~/.pi/agent`.
|
|
+
|
|
+There is no unconfined fallback. `sandboxProblem` runs bwrap once with the
|
|
+system binds; when bwrap is missing or fails, the launcher refuses the
|
|
+launch with `sandbox-unavailable` and the reason.
|
|
+
|
|
+The namespaces and how a session dies:
|
|
+
|
|
+- bwrap unshares the user, PID and IPC namespaces. The runner is pid 1 of
|
|
+ its PID namespace (`--as-pid-1`). A detached child is reparented to it,
|
|
+ so it stays under the launch's pid. Pid 1 ignores a signal it has no
|
|
+ handler for; the runner installs its handlers first, and the senders
|
|
+ resend SIGTERM to cover Node's startup.
|
|
+- The runner dies when the outer bwrap process does: bwrap starts it
|
|
+ through `setpriv --pdeathsig KILL`. bwrap's own `--die-with-parent`
|
|
+ isn't used, because it also ties bwrap to the host, and a session has to
|
|
+ outlive a host that died hard so the next host can recover it. The
|
|
+ parent-death signal is set inside the namespace just before the runner's
|
|
+ exec; if bwrap died in that instant, the runner would keep going until
|
|
+ its broker polls fail.
|
|
+- With its own `/proc`, a session sees no process outside its namespace,
|
|
+ so it can't signal or `ptrace` another session or the host by pid.
|
|
+- The outer bwrap exits with 128 plus the signal number when the runner is
|
|
+ killed; `sessionExit` turns that back into the signal for the launch
|
|
+ log.
|
|
- The runner doesn't reap zombies other than its own turns; they go when
|
|
the namespace does.
|
|
|
|
+The sandbox doesn't confine the network, abstract Unix sockets, or the
|
|
+user: a session runs as the host's UID. `packages/harness/README.md`
|
|
+("Limits") says what that leaves open.
|
|
+
|
|
## Exit codes
|
|
|
|
`launch` exits with the launch script's own code once the script runs.
|
|
diff --git a/packages/seat/src/sandbox.mjs b/packages/seat/src/sandbox.mjs
|
|
new file mode 100644
|
|
index 00000000..21c938ce
|
|
--- /dev/null
|
|
+++ b/packages/seat/src/sandbox.mjs
|
|
@@ -0,0 +1,143 @@
|
|
+// The confinement a managed role session runs in (#1586). The runner and
|
|
+// everything it starts, the harness and every shell the harness opens, run
|
|
+// inside a bubblewrap mount namespace built from an empty root. The session
|
|
+// sees only the binds below, each at its own host path so session.json's
|
|
+// paths mean the same inside and out:
|
|
+//
|
|
+// read-only /usr, /etc and the host's /bin, /lib, /lib64, /sbin links;
|
|
+// the repository files the runner loads; the adapter; for pi
|
|
+// the repository's node_modules, for Claude Code its pinned
|
|
+// binary; node when it lives outside /usr; the broker socket,
|
|
+// and the launch socket for the role that may launch
|
|
+// read-write the instance's workspace and the run directory, except the
|
|
+// run's bundle/ and session.json, which stay read-only
|
|
+// private /tmp (the run's tmp/), HOME (the run's home/), /dev, /proc
|
|
+//
|
|
+// Nothing else exists inside: no data root, no other workspace or run
|
|
+// directory, no ~/.config, ~/.mosaic or ~/.ssh, no checkout beyond the
|
|
+// runner's files. The root is remounted read-only, so a write anywhere but
|
|
+// a read-write bind fails. The session keeps a user and PID namespace (the
|
|
+// runner is pid 1, as before) and an IPC namespace. The runner dies with the
|
|
+// bwrap process outside, as it did with unshare's --kill-child: setpriv
|
|
+// gives it SIGKILL as its parent-death signal. bwrap's own --die-with-parent
|
|
+// isn't used, because it also kills bwrap when the host dies, and a session
|
|
+// outlives a host that died hard (the next host recovers it). The network,
|
|
+// abstract Unix sockets and the host UID are not confined
|
|
+// (packages/harness/README.md, "Limits").
|
|
+//
|
|
+// There is no unconfined fallback: a bwrap that is missing or fails refuses
|
|
+// the launch (sandbox-unavailable).
|
|
+
|
|
+import { spawnSync } from "node:child_process";
|
|
+import { existsSync, lstatSync, mkdirSync, readlinkSync, realpathSync } from "node:fs";
|
|
+import { dirname, join, resolve } from "node:path";
|
|
+import { fileURLToPath } from "node:url";
|
|
+
|
|
+export const REPO = resolve(dirname(fileURLToPath(import.meta.url)), "..", "..", "..");
|
|
+export const BWRAP = "/usr/bin/bwrap";
|
|
+const KILL_WITH_PARENT = ["/usr/bin/setpriv", "--pdeathsig", "KILL", "--"];
|
|
+
|
|
+// What the runner imports, and the three files the harness starts from the
|
|
+// bundle's settings (claude-gate, mcp-server, pi-extension) with theirs.
|
|
+// packages/seat/tests/sandbox.test.mjs walks the imports and fails when one
|
|
+// is missing here.
|
|
+export const RUNNER_FILES = Object.freeze([
|
|
+ "packages/harness/src/runner.mjs",
|
|
+ "packages/harness/src/bundle.mjs",
|
|
+ "packages/harness/src/gate.mjs",
|
|
+ "packages/harness/src/tools.mjs",
|
|
+ "packages/harness/src/claude-gate.mjs",
|
|
+ "packages/harness/src/mcp-server.mjs",
|
|
+ "packages/harness/src/pi-extension.mjs",
|
|
+ "packages/bus/src/client.mjs",
|
|
+ "packages/bus/src/broker.mjs",
|
|
+ "packages/business/src/vocabulary.mjs",
|
|
+].map((p) => join(REPO, p)));
|
|
+
|
|
+const SYSTEM_LINKS = ["/bin", "/lib", "/lib64", "/sbin"];
|
|
+const NAMESPACES = ["--unshare-user", "--unshare-pid", "--unshare-ipc", "--as-pid-1"];
|
|
+
|
|
+// The system part every session gets, and the probe runs: /usr, the host's
|
|
+// links into it, /etc, a private /dev and /proc, and node when it lives
|
|
+// outside /usr.
|
|
+function systemBinds(node = process.execPath) {
|
|
+ const out = [{ kind: "ro", src: "/usr", why: "the system's programs and libraries: sh, node, git, coreutils" }];
|
|
+ for (const p of SYSTEM_LINKS) {
|
|
+ let st;
|
|
+ try {
|
|
+ st = lstatSync(p);
|
|
+ } catch {
|
|
+ continue;
|
|
+ }
|
|
+ if (st.isSymbolicLink()) out.push({ kind: "symlink", src: readlinkSync(p), dest: p, why: `the host's ${p} link into /usr` });
|
|
+ else out.push({ kind: "ro", src: p, why: `the host's ${p}, which isn't a link into /usr` });
|
|
+ }
|
|
+ out.push(
|
|
+ { kind: "ro", src: "/etc", why: "name resolution, CA certificates, passwd for the user's name" },
|
|
+ { kind: "dev", dest: "/dev", why: "a private /dev with null, zero, urandom and a pty" },
|
|
+ { kind: "proc", dest: "/proc", why: "the session's own PID namespace; node and the shell read /proc/self" },
|
|
+ );
|
|
+ const real = realpathSync(node);
|
|
+ if (!real.startsWith("/usr/")) out.push({ kind: "ro", src: real, dest: node, why: "node, which runs the runner and the gate, lives outside /usr" });
|
|
+ return out;
|
|
+}
|
|
+
|
|
+// The binds for one session, in mount order; each says why the session
|
|
+// needs it. paths: the run directory, the instance's workspace, the adapter,
|
|
+// the harness ("pi" or "claude-code"), the broker socket, the launch socket
|
|
+// (null for a role that may not launch), the Claude Code binary
|
|
+// (MOSAIC_CLAUDE_BIN, or null) and node. `readOnly` adds files, for tests only.
|
|
+export function binds({ runDir, workspace, adapter, harness, brokerSocket, launchSocket = null, claudeBin = null, node = process.execPath, readOnly = [] }) {
|
|
+ const out = systemBinds(node);
|
|
+ for (const f of RUNNER_FILES) out.push({ kind: "ro", src: f, why: "a repository file the runner loads" });
|
|
+ out.push({ kind: "ro", src: adapter, why: "the harness adapter the runner starts each turn" });
|
|
+ if (harness === "pi") out.push({ kind: "ro", src: realpathSync(join(REPO, "node_modules")), dest: join(REPO, "node_modules"), why: "pi and its packages, on the session's PATH" });
|
|
+ if (harness === "claude-code" && claudeBin) {
|
|
+ const bin = realpathSync(claudeBin);
|
|
+ if (!bin.startsWith("/usr/")) out.push({ kind: "ro", src: bin, dest: claudeBin, why: "the pinned Claude Code binary (MOSAIC_CLAUDE_BIN)" });
|
|
+ }
|
|
+ out.push({ kind: "ro", src: brokerSocket, why: "the broker socket the runner claims and calls through" });
|
|
+ if (launchSocket) out.push({ kind: "ro", src: launchSocket, why: "the launch socket, for the role whose launch tool calls it" });
|
|
+ for (const f of readOnly) out.push({ kind: "ro", src: f, why: "a test fixture" });
|
|
+ out.push(
|
|
+ { kind: "rw", src: workspace, why: "the instance's workspace, where the harness works" },
|
|
+ { kind: "rw", src: runDir, why: "the run directory: turns/, session/, tools.sock, turn.done, home/, tmp/" },
|
|
+ { kind: "ro", src: join(runDir, "bundle"), why: "the run's bundle stays as the host wrote it" },
|
|
+ { kind: "ro", src: join(runDir, "session.json"), why: "the session file stays as the host wrote it" },
|
|
+ { kind: "rw", src: join(runDir, "tmp"), dest: "/tmp", why: "a private /tmp, on disk in the run directory" },
|
|
+ );
|
|
+ return out;
|
|
+}
|
|
+
|
|
+function argsOf(list) {
|
|
+ const args = [];
|
|
+ for (const b of list) {
|
|
+ if (b.kind === "ro") args.push("--ro-bind", b.src, b.dest ?? b.src);
|
|
+ else if (b.kind === "rw") args.push("--bind", b.src, b.dest ?? b.src);
|
|
+ else if (b.kind === "symlink") args.push("--symlink", b.src, b.dest);
|
|
+ else if (b.kind === "dev") args.push("--dev", b.dest);
|
|
+ else if (b.kind === "proc") args.push("--proc", b.dest);
|
|
+ }
|
|
+ return args;
|
|
+}
|
|
+
|
|
+// The run's home/ and tmp/, made before the spawn: binds need their sources.
|
|
+export function prepareRun(runDir) {
|
|
+ for (const d of ["home", "tmp"]) mkdirSync(join(runDir, d), { recursive: true, mode: 0o700 });
|
|
+}
|
|
+
|
|
+// The full command line for `command` inside the sandbox. HOME is the run's
|
|
+// home/, empty at the start; the harness makes what it needs there.
|
|
+export function sandboxArgv(list, runDir, command, { bwrap = BWRAP } = {}) {
|
|
+ return [bwrap, ...NAMESPACES, ...argsOf(list), "--remount-ro", "/", "--setenv", "HOME", join(runDir, "home"), "--setenv", "TMPDIR", "/tmp", "--chdir", runDir, "--", ...KILL_WITH_PARENT, ...command];
|
|
+}
|
|
+
|
|
+// Whether bwrap runs here at all: the namespaces and the system binds, then
|
|
+// setpriv and `true`. Returns null, or why not.
|
|
+export function sandboxProblem({ bwrap = BWRAP } = {}) {
|
|
+ if (!existsSync(bwrap)) return `${bwrap} is missing`;
|
|
+ const r = spawnSync(bwrap, [...NAMESPACES, ...argsOf(systemBinds()), "--remount-ro", "/", "--", ...KILL_WITH_PARENT, "/usr/bin/true"], { encoding: "utf8", timeout: 10000 });
|
|
+ if (r.status === 0) return null;
|
|
+ const why = (r.stderr ?? "").trim().split("\n")[0] || (r.error ? r.error.code : `exit ${r.status ?? r.signal}`);
|
|
+ return `${bwrap} failed: ${why}`;
|
|
+}
|
|
diff --git a/packages/seat/src/session.mjs b/packages/seat/src/session.mjs
|
|
index a116f578..eb5fa18b 100644
|
|
--- a/packages/seat/src/session.mjs
|
|
+++ b/packages/seat/src/session.mjs
|
|
@@ -4,12 +4,13 @@
|
|
// decides authority and capacity first, and this module only does what it
|
|
// is told.
|
|
//
|
|
-// A session is `node packages/harness/src/runner.mjs <run dir>` inside
|
|
-// `unshare --user --map-current-user --pid --fork --kill-child --mount-proc`.
|
|
-// The runner is pid 1 of its own PID namespace, so a child that detaches
|
|
-// (`setsid -f`) is reparented to the runner and stays under the launch's
|
|
-// pid, which the human CLI's ancestry check refuses. unshare itself ignores
|
|
-// SIGTERM and passes the runner's exit status on; `stop` signals the runner.
|
|
+// A session is `node packages/harness/src/runner.mjs <run dir>` inside the
|
|
+// bubblewrap sandbox of sandbox.mjs, which sees only its own workspace, its
|
|
+// run directory and the files it runs from. The runner is pid 1 of its own
|
|
+// PID namespace, so a child that detaches (`setsid -f`) is reparented to the
|
|
+// runner and stays under the launch's pid, which the human CLI's ancestry
|
|
+// check refuses. bwrap passes the runner's exit status on, a death by signal
|
|
+// as 128 + the signal; `stop` signals the runner.
|
|
//
|
|
// On disk, under the data root:
|
|
// launches/<business>/<run>/ the run directory (0700): session.json,
|
|
@@ -25,18 +26,18 @@
|
|
import { spawn } from "node:child_process";
|
|
import { randomBytes } from "node:crypto";
|
|
import { appendFileSync, closeSync, existsSync, mkdirSync, openSync, readFileSync, renameSync, writeFileSync } from "node:fs";
|
|
-import { dirname, join, resolve } from "node:path";
|
|
-import { fileURLToPath } from "node:url";
|
|
+import { constants } from "node:os";
|
|
+import { dirname, join } from "node:path";
|
|
import { childrenOf, cmdlineOf, startTimeOf } from "./proc.mjs";
|
|
+import { binds, prepareRun, REPO, sandboxArgv } from "./sandbox.mjs";
|
|
import { SeatError } from "./seat.mjs";
|
|
|
|
-export const REPO = resolve(dirname(fileURLToPath(import.meta.url)), "..", "..", "..");
|
|
+export { REPO };
|
|
export const RUNNER = join(REPO, "packages", "harness", "src", "runner.mjs");
|
|
export const ADAPTERS = Object.freeze({
|
|
pi: join(REPO, "adapters", "pi", "adapter.sh"),
|
|
"claude-code": join(REPO, "adapters", "claude", "adapter.sh"),
|
|
});
|
|
-export const UNSHARE = Object.freeze(["unshare", "--user", "--map-current-user", "--pid", "--fork", "--kill-child", "--mount-proc"]);
|
|
export const SESSIONS_VERSION = 1;
|
|
|
|
export const sessionsFile = (dataRoot) => join(dataRoot, "bus-host", "sessions.json");
|
|
@@ -57,7 +58,8 @@ export function family(model, max) {
|
|
// environment passes, so a founder credential variable (GITEA_TOKEN,
|
|
// GH_TOKEN, SSH_AUTH_SOCK, ...) never reaches a session (REQ-CRED-2).
|
|
// Provider variables pass because the harness needs them to reach a model.
|
|
-// MOSAIC_CLAUDE_BIN, the host's pinned Claude Code binary, is a path.
|
|
+// MOSAIC_CLAUDE_BIN, the host's pinned Claude Code binary, is a path. HOME
|
|
+// passes to bwrap only; inside, HOME is the run's own home/.
|
|
export const ENV_ALLOW = Object.freeze([
|
|
"HOME", "USER", "LOGNAME", "LANG", "LC_ALL", "TZ", "TERM",
|
|
"ANTHROPIC_API_KEY", "ANTHROPIC_AUTH_TOKEN", "ANTHROPIC_BASE_URL", "CLAUDE_CODE_OAUTH_TOKEN",
|
|
@@ -65,6 +67,13 @@ export const ENV_ALLOW = Object.freeze([
|
|
"MOSAIC_CLAUDE_BIN",
|
|
]);
|
|
|
|
+// The ways a Claude Code session can log in, each a set of variables that
|
|
+// must all be set; ENV_ALLOW passes them on. The bus unit loads them
|
|
+// (packages/cli/README.md, "Unit"). The session gets nothing from ~/.claude,
|
|
+// which the sandbox doesn't show.
|
|
+export const SESSION_CREDENTIALS = Object.freeze([["CLAUDE_CODE_OAUTH_TOKEN"], ["ANTHROPIC_API_KEY"], ["ANTHROPIC_AUTH_TOKEN", "ANTHROPIC_BASE_URL"]].map((set) => Object.freeze(set)));
|
|
+export const hasSessionCredential = (env) => SESSION_CREDENTIALS.some((set) => set.every((k) => Boolean(env[k])));
|
|
+
|
|
export function sessionEnv(env, run) {
|
|
const out = {};
|
|
for (const k of ENV_ALLOW) if (env[k] !== undefined) out[k] = env[k];
|
|
@@ -103,13 +112,15 @@ export function appendLaunchLog(dataRoot, business, entry) {
|
|
appendFileSync(file, `${JSON.stringify({ at: new Date().toISOString(), ...entry })}\n`, { mode: 0o600 });
|
|
}
|
|
|
|
-// Start the runner. stdin stays open for the capability line the host writes
|
|
-// once the broker has bound the launch; stdout and stderr go to runner.log.
|
|
-// `namespace: false` exists for hosts without unprivileged user namespaces
|
|
-// in the tests only; the host always passes true.
|
|
-export function spawnSession({ runDir, env, namespace = true, node = process.execPath }) {
|
|
+// Start the runner in its sandbox. stdin stays open for the capability line
|
|
+// the host writes once the broker has bound the launch; stdout and stderr go
|
|
+// to runner.log. paths: what sandbox.mjs's binds() takes besides the run
|
|
+// directory (workspace, adapter, harness, brokerSocket, launchSocket,
|
|
+// claudeBin; readOnly for tests). `bwrap` is for the tests.
|
|
+export function spawnSession({ runDir, env, paths, node = process.execPath, bwrap }) {
|
|
+ prepareRun(runDir);
|
|
+ const argv = sandboxArgv(binds({ ...paths, runDir, node }), runDir, [node, RUNNER, runDir], { bwrap });
|
|
const out = openSync(join(runDir, "runner.log"), "a", 0o600);
|
|
- const argv = [...(namespace ? UNSHARE : []), node, RUNNER, runDir];
|
|
try {
|
|
return spawn(argv[0], argv.slice(1), { cwd: runDir, env, stdio: ["pipe", out, out], detached: true });
|
|
} finally {
|
|
@@ -118,18 +129,17 @@ export function spawnSession({ runDir, env, namespace = true, node = process.exe
|
|
}
|
|
|
|
// A runner's command line is `<node> <RUNNER> <run dir>`. Matching argv[1]
|
|
-// rather than any argument skips unshare's forked child before its exec,
|
|
-// whose argv still names the runner.
|
|
+// rather than any argument skips bwrap's forked child before its exec, whose
|
|
+// argv still names the runner.
|
|
export const isRunner = (pid) => (cmdlineOf(pid) ?? [])[1] === RUNNER;
|
|
|
|
-// The runner's pid and start time as seen from here. Under unshare the
|
|
-// runner is the outer process's only child; without a namespace it is the
|
|
-// process itself.
|
|
-export async function runnerOf(child, { namespace = true, timeoutMs = 5000 } = {}) {
|
|
+// The runner's pid and start time as seen from here: the outer bwrap
|
|
+// process's only child.
|
|
+export async function runnerOf(child, { timeoutMs = 5000 } = {}) {
|
|
const deadline = Date.now() + timeoutMs;
|
|
while (Date.now() < deadline) {
|
|
if (child.exitCode !== null || child.signalCode !== null) break;
|
|
- const pid = namespace ? childrenOf(child.pid)[0] : child.pid;
|
|
+ const pid = childrenOf(child.pid)[0];
|
|
const start = pid ? startTimeOf(pid) : null;
|
|
if (start && isRunner(pid)) return { pid, startTime: start };
|
|
await new Promise((r) => setTimeout(r, 20));
|
|
@@ -137,6 +147,14 @@ export async function runnerOf(child, { namespace = true, timeoutMs = 5000 } = {
|
|
throw new SeatError(`runner for ${child.pid} did not start`, 1);
|
|
}
|
|
|
|
+// The outer bwrap's exit as the runner's: bwrap exits 128 + n when the
|
|
+// runner dies of signal n, and the runner's own codes are all below 128.
|
|
+const SIGNALS = Object.fromEntries(Object.entries(constants.signals).map(([name, n]) => [n, name]));
|
|
+export function sessionExit(code, signal) {
|
|
+ if (!signal && code > 128 && SIGNALS[code - 128]) return { code: null, signal: SIGNALS[code - 128] };
|
|
+ return { code, signal };
|
|
+}
|
|
+
|
|
// The launch end reason the broker records for a runner's exit
|
|
// (packages/harness/README.md, "The runner").
|
|
const REASONS = { 0: "stopped", 2: "failed", 20: "founder-credentials", 21: "claim-refused", 22: "session-invalid", 23: "broker-unreachable" };
|
|
diff --git a/packages/seat/tests/sandbox.test.mjs b/packages/seat/tests/sandbox.test.mjs
|
|
new file mode 100644
|
|
index 00000000..704d2032
|
|
--- /dev/null
|
|
+++ b/packages/seat/tests/sandbox.test.mjs
|
|
@@ -0,0 +1,259 @@
|
|
+// The sandbox a role session runs in (src/sandbox.mjs, #1586). A real
|
|
+// session, the runner in its bwrap sandbox against an in-process broker,
|
|
+// answers each message by running the body in /bin/sh; the tests read the
|
|
+// answers. Stand-ins for the founder's files sit in the test's own
|
|
+// directory, plus a file in the host's /dev/shm and a SysV shared memory
|
|
+// segment, each checked to exist on the host first, so a refusal means the
|
|
+// sandbox hid it.
|
|
+
|
|
+import { test } from "node:test";
|
|
+import assert from "node:assert/strict";
|
|
+import { spawnSync } from "node:child_process";
|
|
+import { chmodSync, existsSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync } from "node:fs";
|
|
+import { tmpdir } from "node:os";
|
|
+import { basename, dirname, join, resolve } from "node:path";
|
|
+import { Broker } from "../../bus/src/broker.mjs";
|
|
+import { serve } from "../../bus/src/server.mjs";
|
|
+import { Store } from "../../bus/src/store.mjs";
|
|
+import { SOURCES } from "../../harness/src/bundle.mjs";
|
|
+import { startTimeOf } from "../src/proc.mjs";
|
|
+import { newRun, RUNNER, runnerOf, sessionEnv, spawnSession, writeSessionFile } from "../src/session.mjs";
|
|
+import { binds, prepareRun, REPO, RUNNER_FILES, sandboxArgv, sandboxProblem } from "../src/sandbox.mjs";
|
|
+
|
|
+const FAKE = new URL("../../harness/tests/fixtures/fake-adapter.mjs", import.meta.url).pathname;
|
|
+const skip = sandboxProblem() ?? false;
|
|
+
|
|
+function scratch(t) {
|
|
+ const root = realpathSync(mkdtempSync(join(tmpdir(), "mosaic-sandbox-")));
|
|
+ t.after(() => rmSync(root, { recursive: true, force: true }));
|
|
+ return root;
|
|
+}
|
|
+
|
|
+test("RUNNER_FILES is exactly what the runner and the harness sources it starts import", () => {
|
|
+ const seen = new Set();
|
|
+ const walk = (file) => {
|
|
+ if (seen.has(file)) return;
|
|
+ seen.add(file);
|
|
+ for (const [, rel] of readFileSync(file, "utf8").matchAll(/(?:from|import\()\s*["'](\.\.?\/[^"']+)["']/g)) walk(resolve(dirname(file), rel));
|
|
+ };
|
|
+ for (const file of [RUNNER, ...Object.values(SOURCES)]) walk(file);
|
|
+ assert.deepEqual([...seen].sort(), [...RUNNER_FILES].sort());
|
|
+});
|
|
+
|
|
+test("sandboxProblem names a missing bwrap and a bwrap that fails", (t) => {
|
|
+ const root = scratch(t);
|
|
+ assert.equal(sandboxProblem({ bwrap: join(root, "nope") }), `${join(root, "nope")} is missing`);
|
|
+ const bad = join(root, "bwrap");
|
|
+ writeFileSync(bad, "#!/bin/sh\necho 'bwrap: No permissions to create new namespace' >&2\nexit 1\n");
|
|
+ chmodSync(bad, 0o755);
|
|
+ assert.equal(sandboxProblem({ bwrap: bad }), `${bad} failed: bwrap: No permissions to create new namespace`);
|
|
+ if (!skip) assert.equal(sandboxProblem(), null);
|
|
+});
|
|
+
|
|
+test("every bind says why; the bundle and session file stay read-only over the run directory", (t) => {
|
|
+ const root = scratch(t);
|
|
+ const runDir = join(root, "run");
|
|
+ const list = binds({ runDir, workspace: join(root, "ws"), adapter: join(root, "a.sh"), harness: "claude-code", brokerSocket: join(root, "bus.sock") });
|
|
+ assert.ok(list.every((b) => b.why));
|
|
+ const at = (src) => list.findIndex((b) => b.src === src);
|
|
+ assert.equal(list[at(runDir)].kind, "rw");
|
|
+ for (const f of ["bundle", "session.json"]) {
|
|
+ assert.equal(list[at(join(runDir, f))].kind, "ro");
|
|
+ assert.ok(at(join(runDir, f)) > at(runDir), `${f} is mounted over the run directory`);
|
|
+ }
|
|
+ assert.deepEqual(list.at(-1), { kind: "rw", src: join(runDir, "tmp"), dest: "/tmp", why: list.at(-1).why });
|
|
+ assert.equal(at(join(root, "launch.sock")), -1, "no launch socket for a role that may not launch");
|
|
+ assert.ok(!list.some((b) => b.src === realpathSync(join(REPO, "node_modules"))), "Claude Code gets no node_modules");
|
|
+});
|
|
+
|
|
+// The founder's files and the other sessions' state, as stand-ins. The
|
|
+// /dev/shm file and the SysV segment are the host's, made for this test and
|
|
+// removed after it.
|
|
+function standIns(t, root, dataRoot) {
|
|
+ const s = {
|
|
+ secret: join(root, "home", ".config", "mosaic-dev", "secrets", "stand-in.env"),
|
|
+ mosaic: join(root, "home", ".mosaic", "fleet", "token"),
|
|
+ otherWs: join(root, "ws-other"),
|
|
+ shm: join("/dev/shm", basename(root)),
|
|
+ };
|
|
+ writeFileSync(s.shm, "stand-in\n", { mode: 0o600 });
|
|
+ t.after(() => rmSync(s.shm, { force: true }));
|
|
+ const ipc = spawnSync("/usr/bin/ipcmk", ["-M", "4096"], { encoding: "utf8" });
|
|
+ assert.equal(ipc.status, 0, ipc.stderr);
|
|
+ s.segment = Number(ipc.stdout.trim().split(" ").at(-1));
|
|
+ t.after(() => spawnSync("/usr/bin/ipcrm", ["-m", String(s.segment)]));
|
|
+ for (const f of [s.secret, s.mosaic]) {
|
|
+ mkdirSync(dirname(f), { recursive: true, mode: 0o700 });
|
|
+ writeFileSync(f, "stand-in\n", { mode: 0o600 });
|
|
+ }
|
|
+ mkdirSync(s.otherWs);
|
|
+ writeFileSync(join(s.otherWs, "notes.md"), "another instance's work\n");
|
|
+ const other = newRun(dataRoot, "acme");
|
|
+ writeFileSync(join(other.runDir, "session.json"), "{}\n", { mode: 0o600 });
|
|
+ s.otherRun = other.runDir;
|
|
+ return s;
|
|
+}
|
|
+
|
|
+// [what, shell command, may the session do it, the output it must give].
|
|
+function checks({ s, ws, runDir, run, dataRoot, store }) {
|
|
+ return [
|
|
+ ["HOME is the run's home/, empty", `printf %s "$HOME:"; ls -A "$HOME"`, true, `${join(runDir, "home")}:`],
|
|
+ ["TMPDIR is the private /tmp", `printf %s "$TMPDIR"`, true, "/tmp"],
|
|
+ ["read a stand-in ~/.config/mosaic-dev/secrets", `cat '${s.secret}'`, false],
|
|
+ ["read a stand-in ~/.mosaic", `cat '${s.mosaic}'`, false],
|
|
+ ["read another instance's workspace", `cat '${s.otherWs}/notes.md'`, false],
|
|
+ ["read another run directory", `cat '${s.otherRun}/session.json'`, false],
|
|
+ ["read the data root's store", `cat '${store}' >/dev/null`, false],
|
|
+ ["the data root holds only this run", `ls -A '${dataRoot}'; ls -A '${dataRoot}/launches/acme'`, true, `launches\n${run}`],
|
|
+ ["read the checkout outside the runner's files", `cat '${join(REPO, "AGENTS.md")}' >/dev/null`, false],
|
|
+ ["read the seat's own source", `cat '${join(REPO, "packages", "seat", "src", "session.mjs")}' >/dev/null`, false],
|
|
+ ["read a stand-in in the host's /dev/shm", `cat '${s.shm}'`, false],
|
|
+ ["see the test's own pid in /proc", `test -e /proc/${process.pid}`, false],
|
|
+ ["read its own IPC namespace's segment list", "test -r /proc/sysvipc/shm", true],
|
|
+ ["see a SysV segment the host made", `awk '$2 == ${s.segment} { f = 1 } END { exit !f }' /proc/sysvipc/shm`, false],
|
|
+ ["write to /", "touch /probe", false],
|
|
+ ["write beside the workspace", `touch '${dirname(ws)}/probe'`, false],
|
|
+ ["write in the data root", `touch '${dataRoot}/probe'`, false],
|
|
+ ["write a runner file", `test -w '${RUNNER}'`, false],
|
|
+ ["write the run's bundle", `test -w '${runDir}/bundle/prompt.md' || touch '${runDir}/bundle/probe'`, false],
|
|
+ ["write the session file", `test -w '${runDir}/session.json'`, false],
|
|
+ ["write and read its workspace", `echo ws > '${ws}/probe' && cat '${ws}/probe'`, true, "ws"],
|
|
+ ["write and read its run directory", `echo run > '${runDir}/probe' && cat '${runDir}/probe'`, true, "run"],
|
|
+ ["write and read /tmp", "echo tmp > /tmp/probe && cat /tmp/probe", true, "tmp"],
|
|
+ ["run node", `'${process.execPath}' -e 'console.log(6 * 7)'`, true, "42"],
|
|
+ ["run pi, the session's harness", `'${join(REPO, "node_modules", ".bin", "pi")}' --version >/dev/null`, true],
|
|
+ ];
|
|
+}
|
|
+
|
|
+// The checks whose answer isn't the expected one, by name.
|
|
+async function wrong(table, sh) {
|
|
+ const out = [];
|
|
+ for (const [what, cmd, may, expect] of table) {
|
|
+ const r = await sh(cmd);
|
|
+ if ((r.rc === 0) !== may || (may && expect !== undefined && r.out !== expect)) out.push(what);
|
|
+ }
|
|
+ return out;
|
|
+}
|
|
+
|
|
+// A real session for coder, its adapter a shell that runs each message's
|
|
+// body, or the fake adapter for a TOOL message.
|
|
+// expose(standIns) names extra read-only binds, for the test that breaks
|
|
+// the sandbox on purpose.
|
|
+async function session(t, { expose = () => [] } = {}) {
|
|
+ const root = scratch(t);
|
|
+ const dataRoot = join(root, "data");
|
|
+ mkdirSync(dataRoot, { mode: 0o700 });
|
|
+ const store = new Store(dataRoot);
|
|
+ const broker = new Broker({
|
|
+ store,
|
|
+ businesses: {
|
|
+ acme: {
|
|
+ id: "acme",
|
|
+ human: "jason",
|
|
+ arbiters: { technical: "pm", delivery: "pm" },
|
|
+ roles: {
|
|
+ pm: { authority: { withinRole: ["message.send"], crossRole: [] } },
|
|
+ coder: { authority: { withinRole: ["message.send"], crossRole: [] } },
|
|
+ },
|
|
+ },
|
|
+ },
|
|
+ });
|
|
+ const brokerSocket = join(root, "bus.sock");
|
|
+ const server = await serve({ broker, path: brokerSocket });
|
|
+ t.after(async () => {
|
|
+ await server.close().catch(() => {});
|
|
+ store.close();
|
|
+ });
|
|
+
|
|
+ const { run, runDir } = newRun(dataRoot, "acme");
|
|
+ const s = standIns(t, root, dataRoot);
|
|
+ const bundle = join(runDir, "bundle");
|
|
+ const ws = join(root, "ws");
|
|
+ mkdirSync(bundle, { mode: 0o700 });
|
|
+ mkdirSync(ws);
|
|
+ writeFileSync(join(bundle, "prompt.md"), "# test\n");
|
|
+ writeFileSync(join(bundle, "policy.json"), JSON.stringify({ harness: "pi", workspace: ws, tools: [], typed: [], network: "none", needs: [], credentials: [] }));
|
|
+ writeFileSync(join(bundle, "tools.json"), "[]");
|
|
+ const adapter = join(root, "adapter.sh");
|
|
+ writeFileSync(adapter, [
|
|
+ `body=$(printf '%s\\n' "$MOSAIC_REQUEST" | sed '1,/^$/d')`,
|
|
+ `case $body in TOOL\\ *) exec '${process.execPath}' '${FAKE}' ;; esac`,
|
|
+ `out=$(cd "$MOSAIC_WORKSPACE" && /bin/sh -c "$body" 2>&1); rc=$?`,
|
|
+ `printf 'rc=%s\\n%s' "$rc" "$out"`,
|
|
+ `echo done > "$MOSAIC_TURN_MARKER"`,
|
|
+ "",
|
|
+ ].join("\n"), { mode: 0o644 });
|
|
+ writeSessionFile(runDir, {
|
|
+ business: "acme", instance: "coder", run, launchedBy: "pm", harness: "pi", provider: "fake", model: "fake-model",
|
|
+ brokerSocket, launchSocket: join(root, "launch.sock"), workspace: ws, sessionDir: join(runDir, "session"), adapter,
|
|
+ bundle: { prompt: join(bundle, "prompt.md"), policy: join(bundle, "policy.json"), tools: join(bundle, "tools.json") },
|
|
+ toolSocket: join(runDir, "tools.sock"), turnMarker: join(runDir, "turn.done"), pollInterval: 50,
|
|
+ });
|
|
+
|
|
+ const paths = { workspace: ws, adapter, harness: "pi", brokerSocket, readOnly: [FAKE, ...expose(s)] };
|
|
+ const child = spawnSession({ runDir, env: sessionEnv({ HOME: process.env.HOME, PATH: process.env.PATH }, run), paths });
|
|
+ const exited = new Promise((r) => child.once("close", (code, signal) => r({ code, signal })));
|
|
+ t.after(async () => {
|
|
+ try {
|
|
+ process.kill(child.pid, "SIGTERM");
|
|
+ } catch {}
|
|
+ await exited;
|
|
+ });
|
|
+ await runnerOf(child);
|
|
+ const cap = broker.bindLaunch({ business: "acme", role: "coder", run, harness: "pi", pid: child.pid, startTime: startTimeOf(child.pid) });
|
|
+ const pm = broker.bindLaunch({ business: "acme", role: "pm", run: "pm-run", harness: "pi" });
|
|
+ broker.request(pm, { verb: "role.claim" });
|
|
+ child.stdin.end(`${JSON.stringify({ cap })}\n`);
|
|
+
|
|
+ const ask = async (body) => {
|
|
+ const sent = broker.request(pm, { verb: "message.send", args: { to: "coder", body, class: "REQUEST" } });
|
|
+ const end = Date.now() + 15000;
|
|
+ while (Date.now() < end) {
|
|
+ const reply = broker.request(pm, { verb: "message.receive" }).find((m) => m.in_reply_to === sent.id);
|
|
+ if (reply) return reply.body;
|
|
+ await new Promise((r) => setTimeout(r, 25));
|
|
+ }
|
|
+ throw new Error(`no answer to ${body}`);
|
|
+ };
|
|
+ const sh = async (cmd) => {
|
|
+ const m = (await ask(cmd)).match(/^rc=(\d+)(?:\n([\s\S]*))?$/);
|
|
+ assert.ok(m, `the session's shell answered ${cmd}`);
|
|
+ return { rc: Number(m[1]), out: m[2] ?? "" };
|
|
+ };
|
|
+ const facts = { s, ws, runDir, run, dataRoot, store: store.path, root, adapter, brokerSocket };
|
|
+ for (const f of [s.secret, s.mosaic, s.shm, join(s.otherWs, "notes.md"), join(s.otherRun, "session.json"), store.path, join(REPO, "AGENTS.md"), `/proc/${process.pid}`]) {
|
|
+ assert.ok(existsSync(f), `${f} exists on the host`);
|
|
+ }
|
|
+ const listed = spawnSync("awk", [`$2 == ${s.segment} { f = 1 } END { exit !f }`, "/proc/sysvipc/shm"]);
|
|
+ assert.equal(listed.status, 0, `segment ${s.segment} is listed on the host`);
|
|
+ return { ...facts, ask, sh };
|
|
+}
|
|
+
|
|
+test("a confined session's shell reaches its own workspace and run directory and nothing of the founder's or another session's", { skip, timeout: 60000 }, async (t) => {
|
|
+ const x = await session(t);
|
|
+ assert.deepEqual(await wrong(checks(x), x.sh), []);
|
|
+ // The private /tmp is the run's tmp/ on the host, not the host's /tmp.
|
|
+ assert.equal(readFileSync(join(x.runDir, "tmp", "probe"), "utf8"), "tmp\n");
|
|
+ assert.equal(readFileSync(join(x.ws, "probe"), "utf8"), "ws\n");
|
|
+ // The harness reaches its tool socket: the refusal is the tool server's,
|
|
+ // not a failed connect.
|
|
+ assert.equal(await x.ask("TOOL no_such_tool {}"), "refused: unknown-tool");
|
|
+});
|
|
+
|
|
+test("a bind that exposes a stand-in, or a missing workspace bind, fails the matching check", { skip, timeout: 60000 }, async (t) => {
|
|
+ const x = await session(t, { expose: (s) => [s.secret, s.mosaic] });
|
|
+ assert.deepEqual(await wrong(checks(x), x.sh), ["read a stand-in ~/.config/mosaic-dev/secrets", "read a stand-in ~/.mosaic"]);
|
|
+
|
|
+ // The same checks in a sandbox started directly from binds(), without the
|
|
+ // session: all pass with every bind, and the workspace check fails
|
|
+ // without the workspace's.
|
|
+ const all = binds({ runDir: x.runDir, workspace: x.ws, adapter: x.adapter, harness: "pi", brokerSocket: x.brokerSocket });
|
|
+ const direct = (list) => async (cmd) => {
|
|
+ const argv = sandboxArgv(list, x.runDir, ["/bin/sh", "-c", cmd]);
|
|
+ const r = spawnSync(argv[0], argv.slice(1), { encoding: "utf8", timeout: 10000 });
|
|
+ return { rc: r.status, out: (r.stdout + r.stderr).replace(/\n$/, "") };
|
|
+ };
|
|
+ prepareRun(x.runDir);
|
|
+ assert.deepEqual(await wrong(checks(x), direct(all)), []);
|
|
+ assert.deepEqual(await wrong(checks(x), direct(all.filter((b) => b.src !== x.ws))), ["write and read its workspace"]);
|
|
+});
|
|
diff --git a/packages/seat/tests/session.test.mjs b/packages/seat/tests/session.test.mjs
|
|
index f815245d..0e3b893b 100644
|
|
--- a/packages/seat/tests/session.test.mjs
|
|
+++ b/packages/seat/tests/session.test.mjs
|
|
@@ -1,11 +1,10 @@
|
|
// Managed role sessions on disk and as processes (src/session.mjs). The
|
|
-// runner is the real one, under the real unshare, against an in-process
|
|
+// runner is the real one, in the real bwrap sandbox, against an in-process
|
|
// broker on a real socket; the harness turn is packages/harness's fake
|
|
// adapter. The launcher's checks are tested in packages/cli.
|
|
|
|
import { test } from "node:test";
|
|
import assert from "node:assert/strict";
|
|
-import { spawnSync } from "node:child_process";
|
|
import { existsSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, statSync, writeFileSync } from "node:fs";
|
|
import { tmpdir } from "node:os";
|
|
import { join } from "node:path";
|
|
@@ -19,6 +18,8 @@ import {
|
|
endReason,
|
|
ENV_ALLOW,
|
|
family,
|
|
+ hasSessionCredential,
|
|
+ SESSION_CREDENTIALS,
|
|
isRunner,
|
|
launchLogFile,
|
|
newRun,
|
|
@@ -28,14 +29,14 @@ import {
|
|
sessionEnv,
|
|
sessionsFile,
|
|
spawnSession,
|
|
+ sessionExit,
|
|
stopSession,
|
|
- UNSHARE,
|
|
writeSessionFile,
|
|
} from "../src/session.mjs";
|
|
+import { BWRAP, sandboxProblem } from "../src/sandbox.mjs";
|
|
|
|
const FAKE = new URL("../../harness/tests/fixtures/fake-adapter.mjs", import.meta.url).pathname;
|
|
-const NAMESPACES = spawnSync(UNSHARE[0], [...UNSHARE.slice(1), "true"]).status === 0;
|
|
-const skip = NAMESPACES ? false : "unprivileged user and PID namespaces are unavailable here";
|
|
+const skip = sandboxProblem() ?? false;
|
|
|
|
function scratch(t) {
|
|
const dir = realpathSync(mkdtempSync(join(tmpdir(), "mosaic-session-")));
|
|
@@ -63,6 +64,24 @@ test("sessionEnv passes only the allowlist, the repo's bin on PATH, and the run
|
|
assert.equal(sessionEnv({ MOSAIC_CLAUDE_BIN: "/opt/claude" }, "r0123456789ab").MOSAIC_CLAUDE_BIN, "/opt/claude");
|
|
});
|
|
|
|
+test("sessionExit gives a runner's signal death back as the signal; a session credential is one whole set of nonempty variables", () => {
|
|
+ assert.deepEqual(sessionExit(0, null), { code: 0, signal: null });
|
|
+ assert.deepEqual(sessionExit(23, null), { code: 23, signal: null });
|
|
+ assert.deepEqual(sessionExit(128, null), { code: 128, signal: null });
|
|
+ assert.deepEqual(sessionExit(137, null), { code: null, signal: "SIGKILL" });
|
|
+ assert.deepEqual(sessionExit(143, null), { code: null, signal: "SIGTERM" });
|
|
+ assert.deepEqual(sessionExit(null, "SIGTERM"), { code: null, signal: "SIGTERM" }, "bwrap itself killed");
|
|
+ assert.equal(endReason(...Object.values(sessionExit(137, null))), "killed");
|
|
+ assert.equal(hasSessionCredential({ CLAUDE_CODE_OAUTH_TOKEN: "t" }), true);
|
|
+ assert.equal(hasSessionCredential({ ANTHROPIC_API_KEY: "k" }), true);
|
|
+ assert.equal(hasSessionCredential({ ANTHROPIC_AUTH_TOKEN: "t", ANTHROPIC_BASE_URL: "http://127.0.0.1:1" }), true, "a proxy's key");
|
|
+ assert.equal(hasSessionCredential({ ANTHROPIC_AUTH_TOKEN: "t" }), false, "a key with no proxy to send it to");
|
|
+ assert.equal(hasSessionCredential({ ANTHROPIC_BASE_URL: "http://127.0.0.1:1" }), false);
|
|
+ assert.equal(hasSessionCredential({ CLAUDE_CODE_OAUTH_TOKEN: "", ANTHROPIC_API_KEY: "" }), false);
|
|
+ assert.equal(hasSessionCredential({}), false);
|
|
+ for (const k of SESSION_CREDENTIALS.flat()) assert.ok(ENV_ALLOW.includes(k), k);
|
|
+});
|
|
+
|
|
test("newRun: short ids, 0700 directories, and a refusal when the socket path won't fit", (t) => {
|
|
const root = scratch(t);
|
|
const a = newRun(join(root, "data"), "acme");
|
|
@@ -130,7 +149,7 @@ test("stopSession refuses an unknown run, reports a stale one, and won't signal
|
|
await assert.rejects(stopSession(root, "me"), /is not a session runner; refusing to signal it/);
|
|
});
|
|
|
|
-test("a session runs under unshare as pid 1 of its namespace, claims, answers, and stops on mosaic stop", { skip }, async (t) => {
|
|
+test("a session runs in its sandbox as pid 1 of its namespace, claims, answers, and stops on mosaic stop", { skip }, async (t) => {
|
|
const root = scratch(t);
|
|
const store = new Store(root);
|
|
const broker = new Broker({
|
|
@@ -172,7 +191,8 @@ test("a session runs under unshare as pid 1 of its namespace, claims, answers, a
|
|
toolSocket: join(runDir, "tools.sock"), turnMarker: join(runDir, "turn.done"), pollInterval: 50,
|
|
});
|
|
|
|
- const child = spawnSession({ runDir, env: sessionEnv({ HOME: process.env.HOME, PATH: process.env.PATH, GITEA_TOKEN: "x" }, run) });
|
|
+ const paths = { workspace: ws, adapter, harness: "pi", brokerSocket, readOnly: [FAKE] };
|
|
+ const child = spawnSession({ runDir, env: sessionEnv({ HOME: process.env.HOME, PATH: process.env.PATH, GITEA_TOKEN: "x" }, run), paths });
|
|
t.after(() => {
|
|
try {
|
|
process.kill(child.pid, "SIGKILL");
|
|
@@ -181,15 +201,16 @@ test("a session runs under unshare as pid 1 of its namespace, claims, answers, a
|
|
const exited = new Promise((r) => child.once("close", (code, signal) => r({ code, signal })));
|
|
const runner = await runnerOf(child);
|
|
assert.notEqual(runner.pid, child.pid);
|
|
- assert.deepEqual(cmdlineOf(child.pid).slice(0, UNSHARE.length), UNSHARE);
|
|
+ assert.equal(cmdlineOf(child.pid)[0], BWRAP);
|
|
assert.ok(isRunner(runner.pid));
|
|
- assert.equal(isRunner(child.pid), false, "unshare's argv names the runner too");
|
|
+ assert.equal(isRunner(child.pid), false, "bwrap's argv names the runner too");
|
|
// Inside its namespace the runner is pid 1.
|
|
const nspid = readFileSync(`/proc/${runner.pid}/status`, "utf8").match(/^NSpid:\s*(.*)$/m)[1].trim().split(/\s+/);
|
|
assert.equal(nspid.at(-1), "1");
|
|
const environ = readFileSync(`/proc/${runner.pid}/environ`, "utf8").split("\0");
|
|
assert.ok(environ.includes(`MOSAIC_RUN_ID=${run}`));
|
|
assert.ok(!environ.some((v) => v.startsWith("GITEA_TOKEN=")));
|
|
+ assert.ok(environ.includes(`HOME=${join(runDir, "home")}`), "HOME is the run's own");
|
|
|
|
const cap = broker.bindLaunch({ business: "acme", role: "coder", run, harness: "pi", pid: child.pid, startTime: startTimeOf(child.pid) });
|
|
const pm = broker.bindLaunch({ business: "acme", role: "pm", run: "pm-run", harness: "pi" });
|
|
@@ -209,7 +230,7 @@ test("a session runs under unshare as pid 1 of its namespace, claims, answers, a
|
|
registry.add({ business: "acme", instance: "coder", run, pid: child.pid, startTime: startTimeOf(child.pid), runnerPid: runner.pid, runnerStartTime: runner.startTime });
|
|
const stopped = await stopSession(dataRoot, run, { timeoutMs: 10000 });
|
|
assert.deepEqual(stopped, { stopped: true, business: "acme", instance: "coder", run });
|
|
- assert.deepEqual(await exited, { code: 0, signal: null }, "unshare passes the runner's exit 0 on");
|
|
+ assert.deepEqual(await exited, { code: 0, signal: null }, "bwrap passes the runner's exit 0 on");
|
|
assert.equal(store.get("SELECT op FROM role_claims WHERE role='coder' ORDER BY seq DESC LIMIT 1").op, "release");
|
|
assert.equal(existsSync(join(runDir, "tools.sock")), false);
|
|
assert.equal(mode(join(runDir, "runner.log")), 0o600);
|