fix(pi): pi-unpin S3 re-check 5: five -p runs in a bus session's shape pass every pair of the adapter's session branches (tool list or --no-tools, -c over a planted session, --thinking, --mode json), a -c run must resume the planted session, each loads two -e, the -e extension's command list shows skills and prompt templates in -p, the drift test drives the adapter through all sixteen branch sets and fails on a pair no run passes, and a test holds the --no-mcp run on its own (#1873)

This commit is contained in:
jason.woltje committed 2026-11-20 01:17:33 -06:00
1 parent 749d6d98f5
commit b41bc00b26
4 files changed
+264 -123

No files matched your search

+25 -21
View File
@@ -34,21 +34,23 @@ empty environment and a scratch home, and calls no model. It needs
(with rpc mode and each thinking level), and a missing `-e` file to stop
the start in rpc and in `-p`. The planted ambient sources must stay
unloaded with the seal flags on (skills and prompt templates are read
from `get_commands`, so in rpc only; the rest in rpc and in `-p`): extensions, MCP
from `get_commands` in rpc and from the `-e` extension's command list in
`-p`): extensions, MCP
servers, skills and prompt templates (found in their directories, listed
in a `settings.json`, or in a local package a `settings.json` lists),
context files (`AGENTS.md`, `CLAUDE.md`, `SYSTEM.md`) and a trusted
project's `.pi/SYSTEM.md` and `.pi/APPEND_SYSTEM.md`. The project has a
saved trust, so only `--no-approve` keeps those two out. An `-e`
extension writes the system prompt it sees at `session_start`; the
sealed runs must show the `--system-prompt` file and none of the planted
files. A control run passes the seal flags less every loader flag
extension writes the system prompt, command names and session id it sees
at `session_start`; the sealed runs must show the `--system-prompt` file
and none of the planted sources. A control run passes the seal flags less every loader flag
(`--no-approve`, `--no-extensions`, `--no-skills`,
`--no-prompt-templates`, `--no-context-files`, and `--no-mcp` when
listed), so it keeps `--offline`, `--no-themes`, `--no-tools` and
`--no-session`, and the same `-e` and `--system-prompt`. It must show
each kind of source (an MCP server only when `--help` lists `--no-mcp`)
and answer `get_state` and `get_commands`, or the check can't tell and
and answer `get_state` and `get_commands`, and its `-e` extension must
list the planted skill and prompt template, or the check can't tell and
refuses. For every kind but the trusted project files, only a source
outside the project counts there, since a project source can wait on
trust and `--no-approve` would then hide a seal flag the binary ignores.
@@ -56,22 +58,24 @@ Then one sealed rpc run per loader flag leaves out only that flag: its
kind must show, and no other kind (an MCP server may show beside an
extension when `--help` lists no `--no-mcp`, since pi then ties MCP to
`--no-extensions`). So each loader flag is shown to keep its own kind
out, and no other seal flag, nor their order, can stand in for it. One
more sealed rpc run has a bus session's shape: the arguments the adapter
passes for a session with tools and a fresh persistent session, in its
order, from `sessionShape` in `src/pi-fit.mjs`, which a test holds to
the adapter's real argv. That is `--tools read` in place of
`--no-tools`, `--session-dir` in place of `--no-session`, a fixed
`--provider` and `--model`, and `-e` right after `--no-extensions`. It
must show no planted source. Each run gets a fresh scratch directory, so
no run sees what another left. What a session can pass that no fit run
does: `--mode json` (the model event monitor) or `-p` with a request in
the session's shape (that run is rpc), `--thinking`, `-c` (a session
directory that already holds a session), `--fork`, `--skill` with
`--no-skills` dropped (worker runs only), more than one `-e`, a tool
list other than `read`, its own provider and model, and a provider key.
The runs are offline with no key, so nothing past the start is tried. A
pi built to act one way in the fit runs and another way with those is
out, and no other seal flag, nor their order, can stand in for it. Five
sealed `-p` runs have a bus session's shape: the arguments the adapter
passes for a session, in its order, from `sessionShape` in
`src/pi-fit.mjs`, with two `-e` and `--session-dir`. The adapter's four
session branches (a tool list or `--no-tools`, `-c` over a stored
session or none, `--thinking` or none, `--mode json` or none) are spread
over the five runs so every pair of them is passed together. A `-c` run
gets a stored session and must resume it. Each must load both `-e`
files and show no planted source. A test drives the adapter through all
sixteen branch sets and fails on an argv `sessionShape` doesn't build,
or a pair no run passes. A last sealed `-p` run uses `--no-session`, as
worker runs do. Each run gets a fresh scratch directory, so no run sees
what another left. Not run: `--fork`, `--skill` and the interactive TUI
(worker containers and `scripts/agent.sh` only), the branches three or
four at a time beyond the five runs, and a session's own values (tool
list, thinking level, provider, model, key, `-e` files, request). The
runs are offline with no key, so nothing past the start is tried. A pi
built to act one way in the fit runs and another way with those is
outside what a start-time check can catch. A run killed for taking
over 20 s or printing over 4 MB can't be checked and says which. A
`--version` that fails or hangs past 3 s only leaves no label. Pi
+114 -58
View File
@@ -10,8 +10,10 @@
// every loader flag (the ones that each keep one kind of source out: --no-approve,
// --no-extensions, --no-skills, --no-prompt-templates, --no-context-files, and
// --no-mcp when --help lists it), the --system-prompt file the sealed runs pass, and
// one -e extension that writes the system prompt it sees at session_start. The
// binary answers get_state and get_commands, and shows each kind of planted source:
// one -e extension that writes what it sees at session_start: the system prompt, the
// command names and the session id. The binary answers get_state and get_commands,
// its -e extension lists the planted skill and prompt template (the -p runs see
// those kinds only that way), and it shows each kind of planted source:
// an extension, a skill, a prompt template, a context file, a trusted project file
// (the project's .pi/APPEND_SYSTEM.md, which only the trust --no-approve must undo
// lets in), and an MCP server when --help lists --no-mcp. Only a source outside the
@@ -25,16 +27,18 @@
// --no-extensions, so an MCP server may show beside an extension). So each loader flag
// is shown to keep its own kind out, and no other seal flag, nor the order they come
// in, can stand in for it.
// session one more sealed rpc run in a bus session's shape, with the arguments
// adapters/pi/adapter.sh passes for a session with tools and a fresh persistent
// session, in its order (sessionShape below; a test holds it to the adapter's argv):
// --tools read in place of --no-tools, --session-dir in place of --no-session, a fixed
// --provider and --model, and -e right after --no-extensions. It answers both, loads the
// -e extension, uses the --system-prompt file, and shows no planted source.
// sealed -p the same in -p, the mode the adapter runs. Its -e extension must reach
// session_start, and no planted extension, MCP server, context file or trusted
// project file may show. -p answers no get_commands, so skills and prompt templates
// are checked in rpc only.
// session five -p runs in a bus session's shape, with the arguments adapters/pi/adapter.sh
// passes for a session, in its order (sessionShape below): two -e, --session-dir in
// place of --no-session, a fixed --provider and --model, -p and a message. The
// adapter's four session branches (a tool list or --no-tools, -c over a stored
// session or none, --thinking or none, --mode json or none) are spread over the five
// runs so each pair of them is passed together (SESSION_RUNS). A -c run gets a
// stored session for the project and must resume it. Each run loads both -e
// extensions, uses the --system-prompt file, and shows no planted source. A test
// drives the adapter through all sixteen branch sets and fails on an argv
// sessionShape doesn't build or a pair no run passes.
// sealed -p the seal flags in -p with --no-session, as a worker run passes them. Its -e
// extension must reach session_start, and no planted source may show.
// missing -e a missing -e file, in rpc and in -p, stops the start: exit non-zero, no answer,
// and no session_start for the -e extension given after it
//
@@ -53,13 +57,12 @@
// repository's pi), never a host path.
//
// Known limit: pi 0.85.1 appends the agent directory's APPEND_SYSTEM.md whatever the flags. The
// check doesn't plant one; a session's agent directory is the run's own. What a session can pass
// that no run does: --mode json (the model event monitor) or -p with a request in the session's
// shape (that run is rpc), --thinking, -c (a session directory that already holds a session),
// --fork, --skill with --no-skills dropped (worker runs only), more than one -e, a tool list
// other than read, its own provider and model, and a provider key. The runs are offline with no
// key, so nothing past the start is tried. A pi built to act differently with those is outside
// what a start-time check can catch.
// check doesn't plant one; a session's agent directory is the run's own. Not run: --fork,
// --skill with --no-skills dropped, and the interactive TUI, which only worker containers and
// scripts/agent.sh pass; the session branches three or four at a time beyond the five runs; and
// a session's own values: its tool list, thinking level, provider, model, key, -e files and
// request. The runs are offline with no key, so nothing past the start is tried. A pi built to
// act differently with those is outside what a start-time check can catch.
import { spawnSync } from "node:child_process";
import { createHash } from "node:crypto";
import { lstatSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, realpathSync, rmSync, writeFileSync } from "node:fs";
@@ -73,22 +76,38 @@ export const FIT_FLAGS = Object.freeze(["--mode", "--session", "--no-session", "
const SEAL = ["--offline", "--no-approve", "--no-extensions", "--no-skills", "--no-prompt-templates", "--no-themes", "--no-context-files", "--no-tools", "--no-session"];
// The loader flags: each keeps one kind of planted source out. --no-mcp counts when --help lists it.
const LOADERS = [["--no-approve", "trusted project file"], ["--no-extensions", "extension"], ["--no-skills", "skill"], ["--no-prompt-templates", "prompt template"], ["--no-context-files", "context file"]];
// The session-shaped run's fixed tools, provider and model. Offline with no key, pi stops before
// any model call.
export const SESSION_SHAPE = Object.freeze({ tools: "read", provider: "anthropic", model: "claude-sonnet-4-5" });
// The arguments adapters/pi/adapter.sh builds for a bus session with tools (SESSION_SHAPE.tools),
// a fresh persistent session directory, the given -e files and --system-prompt file, no --thinking
// and no --skill, less the -p and request it appends. Keep its order: pi-fit.test.mjs runs the
// adapter and fails when the two differ.
export function sessionShape({ noMcp, extensions, prompt, sessionDir }) {
// The session-shaped runs' fixed values: a two-tool list, a thinking level, a provider and model.
// Offline with no key, pi stops before any model call.
export const SESSION_SHAPE = Object.freeze({ tools: "read,bash", thinking: "low", provider: "anthropic", model: "claude-sonnet-4-5" });
// The branches adapters/pi/adapter.sh takes for a bus session, each run once or not: a tool list or
// --no-tools, -c over a stored session or a fresh one, --thinking or none, --mode json (the model
// event monitor) or none. The five runs pass every pair of those together; pi-fit.test.mjs drives
// the adapter through all sixteen and fails on a pair no run passes.
const T = SESSION_SHAPE.tools;
const L = SESSION_SHAPE.thinking;
export const SESSION_RUNS = Object.freeze([
{ tools: "", cont: false, thinking: "", json: false },
{ tools: "", cont: true, thinking: L, json: true },
{ tools: T, cont: false, thinking: L, json: true },
{ tools: T, cont: true, thinking: "", json: true },
{ tools: T, cont: true, thinking: L, json: false },
].map((r) => Object.freeze(r)));
// The arguments adapters/pi/adapter.sh builds for a bus session: the given -e files, a tool list
// or none, a persistent session directory with -c when it holds a session, a thinking level or
// none, the --system-prompt file, -p and the request, and --mode json when the model event monitor
// runs pi. No --skill and no --fork: a bus session never passes them. Keep its order:
// pi-fit.test.mjs runs the adapter through every branch and fails when the two differ.
export function sessionShape({ noMcp, extensions, prompt, sessionDir, tools, cont, thinking, json, message }) {
return [
"--offline", "--no-approve", "--no-extensions", ...(noMcp ? ["--no-mcp"] : []), ...extensions.flatMap((e) => ["-e", e]),
"--no-skills", "--no-prompt-templates", "--no-themes", "--no-context-files",
"--tools", SESSION_SHAPE.tools, "--session-dir", sessionDir,
"--provider", SESSION_SHAPE.provider, "--model", SESSION_SHAPE.model,
"--system-prompt", prompt,
...(tools ? ["--tools", tools] : ["--no-tools"]), "--session-dir", sessionDir, ...(cont ? ["-c"] : []),
"--provider", SESSION_SHAPE.provider, "--model", SESSION_SHAPE.model, ...(thinking ? ["--thinking", thinking] : []),
"--system-prompt", prompt, "-p", message, ...(json ? ["--mode", "json"] : []),
];
}
// What a session-shaped run is named by in a refusal line.
const shapeName = ({ tools, cont, thinking, json }) => `in a session's shape with ${[tools ? "--tools" : "--no-tools", ...(cont ? ["-c"] : []), ...(thinking ? ["--thinking"] : []), ...(json ? ["--mode json"] : [])].join(", ")}`;
const RUN_TIMEOUT = 20000;
// --version gives only a label, so a hang costs this much and leaves no label.
const VERSION_TIMEOUT = 3000;
@@ -118,17 +137,20 @@ export function piFit(found, { levels = [], bwrap = BWRAP, node = process.execPa
}
function check(found, { levels, bwrap, node, who, digest }) {
// args: the arguments, or a function of the run's scratch directory that returns them. The
// answer is read before the scratch directory goes: the rpc answers, whether the -e extension
// loaded, the system prompt it saw, and the planted sources that showed (commands: false leaves
// get_commands out, for -p).
const run = (args, input = "", { timeout = RUN_TIMEOUT, commands = true } = {}) => {
// args: the arguments, or a function of the run's scratch directory that returns them (it may
// plant more first). The answer is read before the scratch directory goes: the rpc answers,
// whether each -e extension loaded, what the first one saw at session_start (the system prompt,
// the command names, the session id), and the planted sources that showed, through the rpc
// get_commands answer or through the -e extension's command list (listed: through that alone).
const run = (args, input = "", { timeout = RUN_TIMEOUT } = {}) => {
const scratch = mkdtempSync(join(tmpdir(), "mosaic-pi-fit-"));
try {
plant(scratch);
const r = runIn(found, { bwrap, node, scratch, args: typeof args === "function" ? args(scratch) : args, input, timeout });
const answer = answered(r.stdout);
return { ...r, answer, explicit: marked(scratch, EXPLICIT), prompt: prompted(scratch), seen: seen(scratch, commands ? answer : { commands: [] }) };
const start = started(scratch);
const listed = start?.commands ?? [];
return { ...r, answer, explicit: marked(scratch, EXPLICIT), second: marked(scratch, SECOND), prompt: start?.prompt ?? null, session: start?.session ?? null, listed: named(listed), seen: seen(scratch, [...(answer.commands ?? []), ...listed], start) };
} finally {
rmSync(scratch, { recursive: true, force: true });
}
@@ -168,6 +190,11 @@ function check(found, { levels, bwrap, node, who, digest }) {
const shown = new Set(control.seen.filter((s) => s.kind === "trusted project file" || !s.source.startsWith("project ")).map((s) => s.kind));
const unshown = KINDS.filter((k) => (k !== "MCP server" || noMcp) && !shown.has(k));
if (unshown.length) return refuse(`can't be checked: it didn't load a planted ${unshown.join(", ")} without the seal flags`);
// -p answers no get_commands, so the -p runs see skills and prompt templates only through the -e
// extension's command list, which must show them here.
const listed = new Set(control.listed.filter((s) => !s.source.startsWith("project ")).map((s) => s.kind));
const unlisted = ["skill", "prompt template"].filter((k) => !listed.has(k));
if (unlisted.length) return refuse(`can't be checked: its -e extension didn't list a planted ${unlisted.join(", ")} without the seal flags`);
const rpc = rpcRun(seal);
if (rpc.stopped) return refuse(`can't be checked: the start with the seal flags ${rpc.stopped}`);
@@ -190,22 +217,28 @@ function check(found, { levels, bwrap, node, who, digest }) {
if (others.length) return refuse(`loaded a planted ${others.map((s) => s.source).join(", ")} ${but}`);
}
// A bus session's shape: every session passes --tools, and a persistent one --session-dir, so a
// seal that holds only with --no-tools or --no-session, or only in the order above, shows here.
const shaped = run((s) => [...sessionShape({ noMcp, extensions: [join(s, "explicit", "fit.ts")], prompt: join(s, "prompt.md"), sessionDir: join(s, "sessions") }), "--mode", "rpc"], ask);
const shape = "in a session's shape";
if (shaped.stopped) return refuse(`can't be checked: the start ${shape} ${shaped.stopped}`);
if (!shaped.answer.state || !shaped.answer.commands) return refuse(`can't be checked: it didn't answer get_state and get_commands ${shape}`);
if (!shaped.explicit) return refuse(`didn't load an -e extension ${shape}`);
if (shaped.prompt === null) return refuse(`can't be checked: its -e extension didn't see the system prompt ${shape}`);
if (!shaped.prompt.includes(GIVEN)) return refuse(`didn't use the --system-prompt file ${shape}`);
if (shaped.seen.length) return refuse(`loaded a planted ${shaped.seen.map((s) => s.source).join(", ")} with the seal flags ${shape}`);
// A bus session's shapes, in -p as its turns run: --session-dir in place of --no-session, two -e
// as a session passes, and each pair of the adapter's session branches in one run. A -c run gets
// a stored session for the project first and must resume it, since -c with none starts afresh.
for (const shape of SESSION_RUNS) {
const r = run((s) => {
if (shape.cont) plantSession(s);
return sessionShape({ noMcp, extensions: [join(s, "explicit", "fit.ts"), join(s, "explicit", "second.ts")], prompt: join(s, "prompt.md"), sessionDir: join(s, "sessions"), message: "fit check", ...shape });
});
const name = shapeName(shape);
if (r.stopped) return refuse(`can't be checked: the start ${name} ${r.stopped}`);
if (r.prompt === null) return refuse(`can't be checked: its -e extension didn't see the system prompt ${name}`);
if (!r.explicit || !r.second) return refuse(`didn't load every -e extension ${name}`);
if (!r.prompt.includes(GIVEN)) return refuse(`didn't use the --system-prompt file ${name}`);
if (shape.cont && r.session !== PLANTED_SESSION) return refuse(`can't be checked: it didn't resume the planted session with -c ${name}`);
if (r.seen.length) return refuse(`loaded a planted ${r.seen.map((x) => x.source).join(", ")} with the seal flags ${name}`);
}
// -p takes a message; offline with no key, pi stops before any model call.
const p = run((s) => ["-p", ...seal, ...given(s), "fit check"], "", { commands: false });
const p = run((s) => ["-p", ...seal, ...given(s), "fit check"]);
if (p.stopped) return refuse(`can't be checked: a -p start with the seal flags ${p.stopped}`);
if (p.prompt === null) return refuse("can't be checked: a -p start with the seal flags didn't run its -e extension");
if (p.seen.length) return refuse(`loaded a planted ${p.seen.map((s) => s.source).join(", ")} with the seal flags on in -p`);
if (p.seen.length) return refuse(`loaded a planted ${p.seen.map((x) => x.source).join(", ")} with the seal flags on in -p`);
// A missing -e file stops the start. pi 0.85.1 loads the -e files in order and still loads the
// one after the missing one, then exits 1 before session_start, so only session_start tells.
@@ -304,12 +337,24 @@ const TEXTS = [
];
// The --system-prompt file's word, which the sealed runs must show.
const GIVEN = "FITGIVENPROMPT";
// The -e extension the control and sealed runs pass, which must load.
// The -e extension every run passes, which must load, and the second one the session-shaped runs
// pass after it.
const EXPLICIT = "explicit extension";
const SECOND = "second explicit extension";
// The id of the session a -c run must resume.
const PLANTED_SESSION = "fit-planted-session";
const markerName = (source) => source.replace(/[^a-z]/g, "-");
const marked = (scratch, source) => { try { return readdirSync(join(scratch, "marks")).includes(markerName(source)); } catch { return false; } };
// The system prompt the -e extension saw at session_start, or null when it wrote none.
const prompted = (scratch) => { try { return readFileSync(join(scratch, "marks", "system-prompt"), "utf8"); } catch { return null; } };
// What the -e extension saw at session_start ({ prompt, commands, session }), or null when it wrote
// nothing that parses into that shape.
function started(scratch) {
let m;
try { m = JSON.parse(readFileSync(join(scratch, "marks", "start.json"), "utf8")); } catch { return null; }
if (!m || typeof m.prompt !== "string" || !Array.isArray(m.commands)) return null;
return { prompt: m.prompt, commands: m.commands.filter((c) => typeof c === "string"), session: typeof m.session === "string" ? m.session : null };
}
// The planted skills and prompt templates among command names, as { source, kind }.
const named = (names) => Object.entries(NAMED).filter(([name]) => names.some((c) => c === name || c.endsWith(`:${name}`))).map(([, [source, kind]]) => ({ source, kind }));
function plant(scratch) {
const marks = join(scratch, "marks");
@@ -323,7 +368,7 @@ function plant(scratch) {
mkdirSync(marks, { recursive: true });
mkdirSync(join(scratch, "tmp"), { recursive: true });
mkdirSync(project, { recursive: true });
// The session-shaped run's --session-dir, made as the adapter makes it.
// The session-shaped runs' --session-dir, made as the adapter makes it.
mkdirSync(join(scratch, "sessions"), { recursive: true });
// Each settings.json lists an extension, a skill directory, a prompt template and a local
// package holding one of each.
@@ -360,18 +405,29 @@ function plant(scratch) {
put(join(fallback, "skills", "fit-default-skill", "SKILL.md"), skill("fit-default-skill"));
put(join(fallback, "prompts", "fit-default-prompt.md"), prompt);
for (const [file, text] of TEXTS) put(join(scratch, file), `${text}\n`);
put(join(scratch, "explicit", "fit.ts"), `import { writeFileSync } from "node:fs";\nexport default function (pi: any): void {\n writeFileSync(${JSON.stringify(join(marks, markerName(EXPLICIT)))}, "");\n pi.on("session_start", (_event: any, ctx: any) => { writeFileSync(${JSON.stringify(join(marks, "system-prompt"))}, ctx.getSystemPrompt()); });\n}\n`);
put(join(scratch, "explicit", "fit.ts"), `import { writeFileSync } from "node:fs";\nexport default function (pi: any): void {\n writeFileSync(${JSON.stringify(join(marks, markerName(EXPLICIT)))}, "");\n pi.on("session_start", (_event: any, ctx: any) => { writeFileSync(${JSON.stringify(join(marks, "start.json"))}, JSON.stringify({ prompt: ctx.getSystemPrompt(), commands: pi.getCommands().map((c: any) => c.name), session: ctx.sessionManager.getSessionId() })); });\n}\n`);
put(join(scratch, "explicit", "second.ts"), ext(SECOND));
put(join(scratch, "prompt.md"), `${GIVEN}\n`);
}
// The planted sources a run loaded, as { source, kind }: markers present, planted names
// get_commands listed, and planted words in the system prompt the -e extension saw.
function seen(scratch, answer) {
// A stored session for the project, as pi 0.85.1 writes one: a header whose cwd is the project's
// real path (pi's -c takes only a session for its own cwd) and one user message.
function plantSession(scratch) {
const at = new Date().toISOString();
writeFileSync(join(scratch, "sessions", `${PLANTED_SESSION}.jsonl`), [
{ type: "session", version: 3, id: PLANTED_SESSION, timestamp: at, cwd: realpathSync(join(scratch, "project")) },
{ type: "message", id: "f17e0001", parentId: null, timestamp: at, message: { role: "user", content: [{ type: "text", text: "an earlier turn" }], timestamp: Date.now() } },
].map((e) => `${JSON.stringify(e)}\n`).join(""));
}
// The planted sources a run loaded, as { source, kind }: markers present, planted names among the
// command names, and planted words in the system prompt the -e extension saw.
function seen(scratch, commands, start) {
let marks = [];
try { marks = readdirSync(join(scratch, "marks")); } catch { marks = []; }
const out = Object.entries(MARKED).filter(([s]) => marks.includes(markerName(s))).map(([source, kind]) => ({ source, kind }));
for (const [name, [source, kind]] of Object.entries(NAMED)) if ((answer.commands ?? []).some((c) => c === name || c.endsWith(`:${name}`))) out.push({ source, kind });
const text = prompted(scratch) ?? "";
out.push(...named(commands));
const text = start?.prompt ?? "";
for (const [, word, source, kind] of TEXTS) if (text.includes(word)) out.push({ source, kind });
return out;
}
+92 -32
View File
@@ -7,13 +7,15 @@ import assert from "node:assert/strict";
import { mkdirSync, mkdtempSync, readFileSync, readdirSync, realpathSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
import { spawnSync } from "node:child_process";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import * as fitModule from "../src/pi-fit.mjs";
import { piStandIn } from "./pi-stand-in.mjs";
import { locate } from "../../plugins/harness/pi/binary.mjs";
import * as pi from "../../plugins/harness/pi/index.mjs";
import { BWRAP, sandboxProblem } from "../../seat/src/sandbox.mjs";
import { ENV_ALLOW, sessionEnv } from "../../seat/src/session.mjs";
import { adapterEnv } from "../src/runner.mjs";
const REPO = resolve(fileURLToPath(new URL("../../../", import.meta.url)));
const NAME = "@earendil-works/pi-coding-agent";
@@ -145,16 +147,18 @@ test("pi-unpin S3 re-check 3: every run has its own scratch directory, and the r
});
const seal = [...SEAL, ...extraFlags];
const loaders = [...LOADERS, ...extraFlags];
// --version, --help, the control, the sealed rpc, one per loader flag, the session-shaped run,
// --version, --help, the control, the sealed rpc, one per loader flag, the session-shaped runs,
// -p, two missing -e.
assert.equal(runs.length, 2 + 2 + loaders.length + 1 + 1 + 2);
const shapes = fitModule.SESSION_RUNS;
assert.equal(runs.length, 2 + 2 + loaders.length + shapes.length + 1 + 2);
assert.equal(new Set(runs.map((x) => x.scratch)).size, runs.length, "each run has its own scratch directory");
// Re-check 4 item 1: the session-shaped run comes after the one-out runs, built by the same
// function the adapter test below holds to the adapter's argv, with rpc at the end.
// Re-check 5 item 1: the session-shaped runs come after the one-out runs, in -p, each built by
// the function the adapter test below holds to every argv the adapter builds for a session,
// with two -e as a session passes.
const at = 2 + 2 + loaders.length;
assert.deepEqual(runs[at].args, [...fitModule.sessionShape({ noMcp: extraFlags.length > 0, extensions: ["<s>/explicit/fit.ts"], prompt: "<s>/prompt.md", sessionDir: "<s>/sessions" }), "--mode", "rpc"]);
shapes.forEach((shape, i) => assert.deepEqual(runs[at + i].args, fitModule.sessionShape({ noMcp: extraFlags.length > 0, extensions: ["<s>/explicit/fit.ts", "<s>/explicit/second.ts"], prompt: "<s>/prompt.md", sessionDir: "<s>/sessions", message: "fit check", ...shape }), JSON.stringify(shape)));
const flagLists = [];
for (const { args } of runs.slice(2).filter((_, i) => i + 2 !== at)) {
for (const { args } of runs.slice(2).filter((_, i) => i + 2 < at || i + 2 >= at + shapes.length)) {
const flags = args.filter((a) => seal.includes(a));
assert.deepEqual(flags, seal.filter((f) => flags.includes(f)), "the seal flags keep their order");
const rest = [];
@@ -209,14 +213,47 @@ test("pi-unpin S3 re-check 4: a pi that loads other kinds when one loader flag i
}
});
// Re-check 4 item 1: every bus session runs pi with --tools and most with --session-dir, so one
// sealed run has a session's shape: no --no-tools, no --no-session.
// Re-check 4 item 1: every bus session runs pi with --session-dir, and most with --tools, so the
// session-shaped runs have no --no-session, and most no --no-tools.
test("pi-unpin S3 re-check 4: a pi whose loader flags work only with --no-tools or --no-session is refused in a session's shape", SKIP, t => {
for (const extraFlags of [[], ["--no-mcp"]]) for (const needs of ["--no-tools", "--no-session"]) {
assert.match(fit(standIn(t, { needs, extraFlags }).env).message ?? "passed", /^pi 2\.0\.0 \(MOSAIC_PI_BIN\) loaded a planted user extension, .* with the seal flags in a session's shape$/, `${needs} ${extraFlags}`);
for (const extraFlags of [[], ["--no-mcp"]]) for (const [needs, shape] of [["--no-tools", "--tools, --thinking, --mode json"], ["--no-session", "--no-tools"]]) {
assert.match(fit(standIn(t, { needs, extraFlags }).env).message ?? "passed", new RegExp(`^pi 2\\.0\\.0 \\(MOSAIC_PI_BIN\\) loaded a planted user extension, .* with the seal flags in a session's shape with ${shape}$`), `${needs} ${extraFlags}`);
}
});
// Re-check 5 item 1: the reviewer's fakes, each keyed on what only a session passes, and the
// branch pairs no single-flag fake reaches. Each is refused in the first session-shaped run that
// passes what it is keyed on, and the -p runs see skills and prompt templates through the -e
// extension, since -p answers no get_commands.
test("pi-unpin S3 re-check 5: a pi that leaks only with a session's -c, --thinking, -p with --tools, --mode json, or a pair of them, is refused in that shape", SKIP, t => {
const line = (shape) => new RegExp(`^pi 2\\.0\\.0 \\(MOSAIC_PI_BIN\\) loaded a planted user extension, .*, user skill, .*, user prompt template, .* with the seal flags in a session's shape with ${shape}$`);
for (const [knobs, shape] of [
[{ leakWith: ["-c"] }, "--no-tools, -c, --thinking, --mode json"],
[{ leakWith: ["--thinking"] }, "--no-tools, -c, --thinking, --mode json"],
[{ leakWith: ["-p", "--tools"] }, "--tools, --thinking, --mode json"],
[{ leakWith: ["--mode", "json"] }, "--no-tools, -c, --thinking, --mode json"],
[{ leakWith: ["--no-tools", "--session-dir"] }, "--no-tools"],
[{ leakWith: ["-c"], leakWithout: ["--thinking"] }, "--tools, -c, --mode json"],
[{ leakWith: ["--tools"], leakWithout: ["-c"] }, "--tools, --thinking, --mode json"],
[{ leakWith: ["--thinking"], leakWithout: ["--mode"] }, "--tools, -c, --thinking"],
[{ leakWith: ["-c", "--tools"], leakWithout: ["--thinking"] }, "--tools, -c, --mode json"],
]) for (const extraFlags of [[], ["--no-mcp"]]) assert.match(fit(standIn(t, { ...knobs, extraFlags }).env).message ?? "passed", line(shape), `${JSON.stringify(knobs)} ${extraFlags}`);
});
test("pi-unpin S3 re-check 5: a -c run must resume the planted session, a session run must load both -e extensions, and the control's -e extension must list skills and prompt templates", SKIP, t => {
assert.equal(fit(standIn(t, { noResume: true }).env).message, "pi 2.0.0 (MOSAIC_PI_BIN) can't be checked: it didn't resume the planted session with -c in a session's shape with --no-tools, -c, --thinking, --mode json");
assert.equal(fit(standIn(t, { oneExt: true }).env).message, "pi 2.0.0 (MOSAIC_PI_BIN) didn't load every -e extension in a session's shape with --no-tools");
assert.equal(fit(standIn(t, { extNoCommands: true }).env).message, "pi 2.0.0 (MOSAIC_PI_BIN) can't be checked: its -e extension didn't list a planted skill, prompt template without the seal flags");
});
// Re-check 5 item 2: the --no-mcp run is the only one that shows --no-mcp alone keeps MCP out.
// This pi keeps MCP out with --no-mcp or with --no-context-files, so every other run hides that
// --no-mcp does nothing on its own.
test("pi-unpin S3 re-check 5: a pi whose --no-mcp works only beside another seal flag is can't tell at the --no-mcp run", SKIP, t => {
assert.equal(fit(standIn(t, { extraFlags: ["--no-mcp"], mcpAlso: "--no-context-files" }).env).message, "pi 2.0.0 (MOSAIC_PI_BIN) can't be checked: it didn't load a planted MCP server with every seal flag but --no-mcp");
assert.equal(fit(standIn(t, { extraFlags: ["--no-mcp"] }).env).ok, true);
});
test("pi-unpin S3 re-check 3: a pi that remembers the control in its scratch directory and hides later is refused", SKIP, t => {
assert.match(fit(standIn(t, { remember: true }).env).message, /^pi 2\.0\.0 \(MOSAIC_PI_BIN\) loaded a planted user extension, .* with the seal flags on$/);
});
@@ -241,7 +278,8 @@ test("pi-unpin S3 re-check 2: a --version that hangs costs a few seconds, not th
test("pi-unpin S3 review: a stand-in that drops the --system-prompt file, or runs no session_start in -p, is refused", SKIP, t => {
assert.equal(fit(standIn(t, { ignoreSystemPrompt: true }).env).message, "pi 2.0.0 (MOSAIC_PI_BIN) didn't use the --system-prompt file with the seal flags on");
assert.equal(fit(standIn(t, { printNoStart: true }).env).message, "pi 2.0.0 (MOSAIC_PI_BIN) can't be checked: a -p start with the seal flags didn't run its -e extension");
// The session-shaped runs are -p, so the first of them is where that shows.
assert.equal(fit(standIn(t, { printNoStart: true }).env).message, "pi 2.0.0 (MOSAIC_PI_BIN) can't be checked: its -e extension didn't see the system prompt in a session's shape with --no-tools");
});
// Review finding 3: only the option column counts, not a flag named in a sentence.
@@ -367,29 +405,51 @@ test("pi-unpin S3: the adapter passes --no-mcp only when the module asks for it"
assert.equal(run(standIn(t, {})).includes("--no-mcp"), false);
});
// Re-check 4 item 1: the session-shaped fit run's arguments are the adapter's for a bus session
// with tools and a fresh persistent session, so the two can't drift. The adapter runs as a session
// runs it, from pi.env(), with the fit run's tools, provider, model and files put in.
test("pi-unpin S3 re-check 4: the session-shaped fit run passes exactly what the adapter passes", SKIP, t => {
assert.equal(typeof fitModule.sessionShape, "function", "pi-fit.mjs exports sessionShape");
const session = { provider: "fixture", model: "m", thinking: "low", toolSocket: "/run/tools.sock" };
// Re-check 5 item 1: the adapter runs through every branch a bus session can reach, with the
// environment the runner hands it (sessionEnv's allowlist, adapterEnv, and pi.env() as harnessEnv).
// Each argv must be one sessionShape() builds, and each pair of its branches must be passed
// together by one session-shaped fit run. Every variable the adapter reads must come from that
// environment or be one no bus session gets, so a new adapter branch can't go unseen.
test("pi-unpin S3 re-check 5: every argv the adapter builds for a bus session is one sessionShape builds, and each pair of its branches is one a fit run passes", SKIP, t => {
const adapter = join(REPO, "adapters", "pi", "adapter.sh");
// Set only by worker containers (packages/processing-control) and the TUI (scripts/agent.sh).
const NOT_BUS = ["MOSAIC_SKILLS", "MOSAIC_SESSION_FORK", "MOSAIC_INTERACTIVE"];
for (const name of NOT_BUS) assert.ok(!ENV_ALLOW.includes(name), name);
const reads = new Set([...readFileSync(adapter, "utf8").matchAll(/\$\{?((?:MOSAIC|PI)_[A-Z_]+)/g)].map((m) => m[1]));
const set = new Set(["MOSAIC_REQUEST"]); // the runner adds it for each turn
const branchKeys = ["tools", "cont", "thinking", "json"];
const covered = (b) => fitModule.SESSION_RUNS.some((r) => branchKeys.every((k) => !(k in b) || Boolean(r[k]) === b[k]));
let runs = 0;
for (const extraFlags of [[], ["--no-mcp"]]) {
const s = standIn(t, { extraFlags });
const out = join(s.root, "argv");
const prompt = join(s.root, "prompt.md");
const ext = join(s.root, "fit.ts");
const sessions = join(s.root, "sessions");
writeFileSync(prompt, "p");
writeFileSync(ext, "");
const given = pi.env({ policy: { tools: ["read"] }, tools: [], session, env: s.env });
assert.ok(given.MOSAIC_TOOLS, "a bus session always has tools");
const shape = fitModule.SESSION_SHAPE;
const r = spawnSync("/bin/sh", [join(REPO, "adapters", "pi", "adapter.sh")], {
encoding: "utf8",
env: { ...given, MOSAIC_TOOLS: shape.tools, PI_PROVIDER: shape.provider, PI_MODEL: shape.model, MOSAIC_THINKING: "", MOSAIC_EXTENSIONS: ext, PATH: "/usr/bin:/bin", ARGV_OUT: out, MOSAIC_SYSTEM_PROMPT_FILE: prompt, MOSAIC_REQUEST: "x", MOSAIC_WORKSPACE: join(s.root, "ws"), MOSAIC_SESSION_DIR: sessions },
});
assert.equal(r.status, 0, r.stderr);
const argv = readFileSync(out, "utf8").split("\n").slice(0, -1);
assert.deepEqual(argv, [...fitModule.sessionShape({ noMcp: extraFlags.length > 0, extensions: [ext], prompt, sessionDir: sessions }), "-p", "x"]);
for (const tools of [[], ["read", "bash"]]) for (const cont of [false, true]) for (const thinking of ["", "low"]) for (const json of [false, true]) {
const dir = mkdtempSync(join(s.root, "run-"));
const sessionDir = join(dir, "session");
mkdirSync(sessionDir);
if (cont) writeFileSync(join(sessionDir, "earlier.jsonl"), "{}\n");
const shape = fitModule.SESSION_SHAPE;
const session = { provider: shape.provider, model: shape.model, thinking, toolSocket: join(dir, "tools.sock"), proxyTransport: { piProvider: shape.provider } };
const harnessEnv = pi.env({ policy: { tools }, tools: [], session, proxy: json ? { key: "fixture-key" } : null, env: s.env });
const env = adapterEnv({ ...session, bundle: { prompt, policy: join(dir, "policy.json"), tools: join(dir, "tools.json") }, workspace: join(dir, "ws"), sessionDir, turnMarker: join(dir, "turn"), harnessEnv, runDir: dir, proxyFirstTurn: json },
sessionEnv({ ...s.env, PATH: `${dirname(process.execPath)}:/usr/bin:/bin` }, "fit1", json ? {} : null));
for (const name of NOT_BUS) assert.equal(env[name], undefined, name);
for (const name of Object.keys(env)) set.add(name);
const out = join(dir, "argv");
const r = spawnSync("/bin/sh", [adapter], { encoding: "utf8", env: { ...env, ARGV_OUT: out, MOSAIC_REQUEST: "x" } });
const label = JSON.stringify({ extraFlags, tools, cont, thinking, json });
// The monitor reads the stand-in's empty output as no answer; the argv is written first.
if (!json) assert.equal(r.status, 0, r.stderr);
const argv = readFileSync(out, "utf8").split("\n").slice(0, -1);
const extensions = env.MOSAIC_EXTENSIONS.split(",");
assert.equal(extensions.length, 2, "the session-shaped fit runs pass two -e, as a session does");
assert.deepEqual(argv, fitModule.sessionShape({ noMcp: extraFlags.length > 0, extensions, prompt, sessionDir, tools: tools.join(","), cont, thinking, json, message: "x" }), label);
const branch = { tools: tools.length > 0, cont, thinking: thinking !== "", json };
for (const a of branchKeys) for (const b of branchKeys) if (a < b) assert.ok(covered({ [a]: branch[a], [b]: branch[b] }), `${label}: no fit run passes ${a}=${branch[a]} with ${b}=${branch[b]}`);
runs++;
}
}
assert.equal(runs, 32);
for (const name of reads) assert.ok(set.has(name) || NOT_BUS.includes(name), `the adapter reads ${name}, which no bus session environment here sets`);
});
+33 -12
View File
@@ -13,8 +13,9 @@
// -e <file> a file that doesn't exist stops it with exit 1, unless `missingExtOk` is true or
// names the mode ("rpc" or "print") it starts in anyway
// --mode rpc loads the planted sources the way pi 0.85.1 finds them, honouring the seal flags,
// or -p runs each -e extension's session_start (one that writes ctx.getSystemPrompt()
// gets the prompt this stand-in builds), then answers get_state and get_commands
// or -p runs each -e extension's session_start (one that writes JSON.stringify(...) gets
// the prompt this stand-in builds, the command names and the session id), then
// answers get_state and get_commands
// (rpc), or prints `sessionOut` and exits 0 (-p).
// An -e file that exists loads even with --no-extensions, unless `dropExplicit`
// (then it loads only without --no-extensions). `printNoStart`: -p runs no
@@ -35,6 +36,15 @@
// -e file's directory) loads every kind whatever the seal flags; a later run loads
// none. `needs`: a flag the loader flags work only with; without it they do
// nothing. `leakWithout`: flags; when any of them is absent, every kind loads.
// `leakWith`: flags; when all of them are given, every kind loads (with
// `leakWithout` too, both must hold). `mcpAlso`: a flag that keeps MCP servers out
// as well as their own flag does.
// An -e extension that writes JSON.stringify({...}) at session_start gets the
// prompt, the command names, the session id (with -c, the id in the header of a
// .jsonl in --session-dir whose cwd is the working directory, else a new one) and
// the --thinking level (else "medium"). `noResume`: -c always starts a new
// session. `oneExt`: only the first -e loads. `extNoCommands`: the extension sees
// no commands.
// `commands` "fail" answers get_commands with success false; "list" answers
// with a bare list.
// otherwise a session with no $ARGV_OUT: prints `sessionOut` and exits 0.
@@ -94,7 +104,7 @@ if (rpc || print) {
if (fs.existsSync(marker)) ignoreAll = true;
else { fs.writeFileSync(marker, ""); leak = new Set(ALL); }
}
if ((K.leakWithout || []).some((f) => !has(f))) leak = new Set(ALL);
if ((K.leakWith || K.leakWithout) && (K.leakWith || []).every(has) && (!K.leakWithout || K.leakWithout.some((f) => !has(f)))) leak = new Set(ALL);
// Whether a seal flag is on: with standIn, its flag reads as the other flag; with needs, only
// when that flag is given too.
const sealed = (flag) => has(K.standIn && K.standIn[0] === flag ? K.standIn[1] : flag) && (!K.needs || has(K.needs));
@@ -109,18 +119,19 @@ if (rpc || print) {
const saved = settings.some((s) => s.defaultProjectTrust === "always") || trust[cwd] === true;
const trusted = !ignoreAll && (has("--approve") || (saved && (!sealed("--no-approve") || leak.has("approve"))));
const commands = [];
// A planted extension writes one marker on load; an -e extension may also write the system
// prompt at session_start. Do what each would do.
// A planted extension writes one marker on load; an -e extension may also write what it saw at
// session_start. Do what each would do.
const starts = [];
const load = (f) => {
const text = read(f);
if (text === null) return;
for (const m of text.matchAll(/writeFileSync\(("(?:[^"\\]|\\.)*")(, ctx\.getSystemPrompt\(\))?/g)) {
for (const m of text.matchAll(/writeFileSync\(("(?:[^"\\]|\\.)*")(, JSON\.stringify\()?/g)) {
if (m[2]) starts.push(JSON.parse(m[1]));
else fs.writeFileSync(JSON.parse(m[1]), "");
}
};
if (!(K.dropExplicit && has("--no-extensions"))) for (let i = 0; i < argv.length; i++) if (argv[i] === "-e") load(argv[i + 1]);
const given = argv.flatMap((a, i) => a === "-e" ? [argv[i + 1]] : []);
if (!(K.dropExplicit && has("--no-extensions"))) for (const f of K.oneExt ? given.slice(0, 1) : given) load(f);
const packages = settings.flatMap((s) => (s.packages || []).map((p) => typeof p === "string" ? p : p.source));
if (K.trustExtOnly) {
if (trusted) for (const f of list(path.join(cwd, ".pi", "extensions"), (n) => n.endsWith(".ts"))) load(f);
@@ -130,7 +141,8 @@ if (rpc || print) {
for (const p of packages) files.push(...list(path.join(p, "extensions"), (n) => n.endsWith(".ts")));
for (const f of files) load(f);
}
if (on("mcp", (K.extraFlags || []).includes("--no-mcp") ? "--no-mcp" : "--no-extensions")) {
const mcpFlag = (K.extraFlags || []).includes("--no-mcp") ? "--no-mcp" : "--no-extensions";
if (!ignoreAll && (!(sealed(mcpFlag) || (K.mcpAlso && has(K.mcpAlso))) || leak.has("mcp"))) {
for (const f of [path.join(agent, "mcp.json"), path.join(cwd, ".pi", "mcp.json"), path.join(cwd, ".mcp.json")]) {
for (const s of Object.values(json(f)?.mcpServers || {})) cp.spawnSync(s.command, s.args || [], { stdio: "ignore" });
}
@@ -148,10 +160,10 @@ if (rpc || print) {
}
// The system prompt: the --system-prompt file, else SYSTEM.md; then APPEND_SYSTEM.md; then the
// context files, from the agent directory and from / down to the working directory.
const given = K.ignoreSystemPrompt ? null : val("--system-prompt");
const sys = K.ignoreSystemPrompt ? null : val("--system-prompt");
const own = (name) => (trusted && read(path.join(cwd, ".pi", name))) || (!ignoreAll && read(path.join(agent, name))) || "";
const hide = K.sysReplacesAll && given;
const parts = [given ? (read(given) || "") : own("SYSTEM.md"), K.ignoreAppend || hide ? "" : own("APPEND_SYSTEM.md")];
const hide = K.sysReplacesAll && sys;
const parts = [sys ? (read(sys) || "") : own("SYSTEM.md"), K.ignoreAppend || hide ? "" : own("APPEND_SYSTEM.md")];
if (K.trustCtxOnly) {
if (trusted) parts.push(read(path.join(cwd, "AGENTS.md")) || "");
} else if (!hide && on("context", "--no-context-files")) {
@@ -160,7 +172,16 @@ if (rpc || print) {
parts.push(read(path.join(agent, "AGENTS.md")) || "");
for (const d of dirs) parts.push(read(path.join(d, "AGENTS.md")) || read(path.join(d, "CLAUDE.md")) || "");
}
if (!(print && K.printNoStart)) for (const f of starts) fs.writeFileSync(f, parts.join("\n"));
// The session -c resumes: the newest .jsonl in --session-dir whose header names this cwd.
let session = "stand-in-" + process.pid;
const dir = val("--session-dir");
if (has("-c") && dir && !K.noResume) for (const f of list(dir, (n) => n.endsWith(".jsonl"))) {
let h = null;
try { h = JSON.parse((read(f) || "").split("\n")[0]); } catch { h = null; }
if (h && h.type === "session" && h.cwd === cwd) session = h.id;
}
const start = { prompt: parts.join("\n"), commands: K.extNoCommands ? [] : commands.map((c) => c.name), session };
if (!(print && K.printNoStart)) for (const f of starts) fs.writeFileSync(f, JSON.stringify(start));
if (print) {
if (K.sessionOut) process.stdout.write(K.sessionOut);
process.exit(0);