fix(adapters,harness): pi-skills re-check: the docs no longer call the fit check's last -p run a headless worker's argv, since task launches refuse skill directories and set MOSAIC_SKILLS empty (packages/processing-control containers.mjs) and only the interactive launch (scripts/agent.sh) sets it, in the TUI with --tools and --session-dir; the run is now named the nearest headless form of the adapter's --skill argv, and the Not run notes name the TUI launch's flags and --skill with no -e (R2-P2-1, R2-P3-1); the untested --system-prompt check added to that run is dropped, as at BASE (R2-P3-2) (#1873, lane pi-skills)

This commit was merged in pull request #2014.
This commit is contained in:
jason.woltje committed 2026-11-20 04:30:23 -06:00
1 parent 59af86a5ee
commit ca7388743b
5 files changed
+56 -43

No files matched your search

+8 -5
View File
@@ -108,16 +108,19 @@ activity through its existing extension. Mock remains a test-only echo.
### A worker's skills (M17, #1873) ### A worker's skills (M17, #1873)
Worker runs and `scripts/agent.sh` may set `MOSAIC_SKILLS`, a comma list of `MOSAIC_SKILLS` is a comma list of skill directories. Today only the
skill directories. The pi adapter passes `--no-skills` and then interactive container launch (`scripts/agent.sh`) sets it; task launches
refuse skill directories and set it empty
(`packages/processing-control/src/containers.mjs`). The pi adapter passes `--no-skills` and then
`--skill <dir>` for each one. On pi 0.85.1 `--skill` adds to discovery and `--skill <dir>` for each one. On pi 0.85.1 `--skill` adds to discovery and
does not replace it. With `--skill` alone, a probe with a scratch HOME also does not replace it. With `--skill` alone, a probe with a scratch HOME also
loaded the user's skills, the skills a `settings.json` or a listed package loaded the user's skills, the skills a `settings.json` or a listed package
names, and `~/.agents/skills` (and the project's, without `--no-approve`). names, and `~/.agents/skills` (and the project's, without `--no-approve`).
With `--no-skills` first, only the given directories load. A directory that With `--no-skills` first, only the given directories load. A directory that
doesn't exist stops the adapter with exit 2. The fit check runs this doesn't exist stops the adapter with exit 2. The fit check runs the
argv, with two skill directories, one `-e` file and no tools, as its last headless form of this argv, with two skill directories, one `-e` file, no
`-p` run (`workerShape`, `packages/harness/README.md`). The fit check runs tools and no session, as its last `-p` run (`workerShape`,
`packages/harness/README.md`); the TUI shape the launch uses is not run. The fit check runs
before bus sessions only; worker containers run the pi pinned in the image. before bus sessions only; worker containers run the pi pinned in the image.
### A role's skills (row 64) ### A role's skills (row 64)
+2 -1
View File
@@ -62,7 +62,8 @@ TOOLS_FLAG="--no-tools"
# with --skill alone pi also loads the user's skills, settings and package # with --skill alone pi also loads the user's skills, settings and package
# skills and ~/.agents/skills (a planted-skill probe showed it, offline, with a # skills and ~/.agents/skills (a planted-skill probe showed it, offline, with a
# scratch HOME). With --no-skills first only the given dirs load. The fit check # scratch HOME). With --no-skills first only the given dirs load. The fit check
# (packages/harness/src/pi-fit.mjs) runs this argv as its last -p run (workerShape). # (packages/harness/src/pi-fit.mjs) runs its -p form as its last run (workerShape).
# Only the interactive launch (scripts/agent.sh) sets MOSAIC_SKILLS today.
# Role sessions never get MOSAIC_SKILLS (ENV_ALLOW in packages/seat/src/ # Role sessions never get MOSAIC_SKILLS (ENV_ALLOW in packages/seat/src/
# session.mjs leaves it out), so they run with --no-skills alone; their skills # session.mjs leaves it out), so they run with --no-skills alone; their skills
# are text in the system prompt (row 64). # are text in the system prompt (row 64).
+13 -8
View File
@@ -92,11 +92,15 @@ that no session sets. So a branch on `TZ` or `LC_ALL` being unset, or on
`/bin/sh` sets an unset `TERM` to `dumb` for the shell without exporting `/bin/sh` sets an unset `TERM` to `dumb` for the shell without exporting
it, so `${TERM:-}` in the adapter is never empty, in a session or here.) it, so `${TERM:-}` in the adapter is never empty, in a session or here.)
It can't see a branch keyed on a It can't see a branch keyed on a
value it doesn't try (a model, a path, another tool or provider). A last `-p` run is a value it doesn't try (a model, a path, another tool or provider). A last `-p` run is the
headless worker's: the argv the adapter builds with `MOSAIC_SKILLS`, one argv the adapter builds in `-p` with `MOSAIC_SKILLS`, one `-e` file, no
`-e` file and no tools (`workerShape`), so `--no-skills` and then two tools and no session (`workerShape`), so `--no-skills` and then two
`--skill` directories, `--no-tools`, `--no-session`, `--provider` and `--skill` directories, `--no-tools`, `--no-session`, `--provider` and
`--model`. A test runs the adapter and holds its argv to `workerShape`. `--model`. A test runs the adapter and holds its argv to `workerShape`.
No caller makes this exact run today: task launches refuse skill
directories, and only the interactive launch (`scripts/agent.sh`) sets
`MOSAIC_SKILLS`, in the TUI, which no fit run can drive. It is the
nearest headless form of the adapter's `--skill` argv.
The run's `-e` extension must list both skills and no planted one. pi The run's `-e` extension must list both skills and no planted one. pi
0.85.1 adds a `--skill` directory to discovery, so a pi that let 0.85.1 adds a `--skill` directory to discovery, so a pi that let
`--skill` bring the planted skills back is refused there, and so is one `--skill` bring the planted skills back is refused there, and so is one
@@ -105,11 +109,12 @@ replaces the plain sealed `-p` run with `--no-session` and no `--skill`,
so it adds no run. Each run gets a fresh scratch directory, so no run sees so it adds no run. Each run gets a fresh scratch directory, so no run sees
what another left. Not run: `--fork` and the interactive TUI what another left. Not run: `--fork` and the interactive TUI
(worker containers and `scripts/agent.sh` only), `-p` with (worker containers and `scripts/agent.sh` only), `-p` with
`--no-session` and no `--skill` (a worker without `MOSAIC_SKILLS`; every `--no-session` and no `--skill` (a task launch's shape; every bus session
bus session passes `--session-dir`), `--skill` beside `--tools`, passes `--session-dir`), `--skill` beside `--tools`, `--session-dir`,
`--session-dir`, `-c`, `--thinking` or `--mode json`, or in rpc (a worker `-c`, `--thinking` or `--mode json`, or in rpc (the TUI launch passes the
may pass the first two; no bus session passes `--skill`), one or three or first three; no bus session passes `--skill`), `--skill` with no `-e`
more `--skill` directories, the branches three or (every fit run needs its `-e` extension to see what loaded), one or three
or more `--skill` directories, the branches three or
four at a time beyond the five runs, `-c` over a session directory that four at a time beyond the five runs, `-c` over a session directory that
holds nothing for the project (one more run costs about 340 ms cold), holds nothing for the project (one more run costs about 340 ms cold),
and a session's own values (tools beyond `read` and `bash`, thinking and a session's own values (tools beyond `read` and `bash`, thinking
+25 -21
View File
@@ -39,16 +39,21 @@
// planted source. A test drives the adapter through every branch set, tool list // planted source. A test drives the adapter through every branch set, tool list
// length and thinking level, and fails on an argv sessionShape doesn't build or a // length and thinking level, and fails on an argv sessionShape doesn't build or a
// pair no run passes. // pair no run passes.
// worker -p the arguments adapters/pi/adapter.sh passes for a headless worker run with // worker -p the arguments adapters/pi/adapter.sh builds in -p with MOSAIC_SKILLS, one -e
// MOSAIC_SKILLS and no tools, in its order (workerShape below): the -e, --no-skills // file, no tools and no session, in its order (workerShape below): the -e,
// then two --skill directories, --no-tools, --no-session, the fixed --provider and // --no-skills then two --skill directories, --no-tools, --no-session, the fixed
// --model, --system-prompt, -p and a message. Its -e extension must reach // --provider and --model, --system-prompt, -p and a message. Its -e extension must
// session_start and list both given skills, and no planted source may show. pi // reach session_start and list both given skills, and no planted source may show.
// 0.85.1 adds a --skill directory to discovery, so a pi that let --skill turn // pi 0.85.1 adds a --skill directory to discovery, so a pi that let --skill turn
// discovery back on would show the planted skills here, and one that drops a --skill // discovery back on would show the planted skills here, and one that drops a --skill
// directory once --no-skills is on would not list it. This run replaces the sealed // directory once --no-skills is on would not list it. No caller makes this exact
// -p run the check made before (the seal flags in -p with --no-session and no // run today: task launches refuse skill directories (packages/processing-control
// --skill), so it adds no run; that plain shape is not run on its own any more. // containers.mjs), and only the interactive container launch (scripts/agent.sh)
// sets MOSAIC_SKILLS, in the TUI with --tools and --session-dir, which no fit run
// can drive. This is the nearest headless form of the adapter's --skill argv. It
// replaces the sealed -p run the check made before (the seal flags in -p with
// --no-session and no --skill), so it adds no run; that plain shape is not run on
// its own any more.
// missing -e a missing -e file, in rpc and in -p, stops the start: exit non-zero, no answer, // missing -e a missing -e file, in rpc and in -p, stops the start: exit non-zero, no answer,
// and no session_start for the -e extension given after it // and no session_start for the -e extension given after it
// //
@@ -86,10 +91,11 @@
// Known limit: pi 0.85.1 appends the agent directory's APPEND_SYSTEM.md whatever the flags. The // Known limit: pi 0.85.1 appends the agent directory's APPEND_SYSTEM.md whatever the flags. The
// check doesn't plant one; a session's agent directory is the run's own. Not run: --fork and // check doesn't plant one; a session's agent directory is the run's own. Not run: --fork and
// the interactive TUI, which only worker containers and scripts/agent.sh pass; -p with // the interactive TUI, which only worker containers and scripts/agent.sh pass; -p with
// --no-session and no --skill (a worker with no MOSAIC_SKILLS; every bus session passes // --no-session and no --skill (a task launch's shape; every bus session passes --session-dir);
// --session-dir); --skill beside --tools, --session-dir, -c, --thinking or --mode json, or in rpc // --skill beside --tools, --session-dir, -c, --thinking or --mode json, or in rpc (the TUI
// (a worker may pass the first two; no bus session passes --skill), and one or three or more // launch passes the first three; no bus session passes --skill); --skill with no -e (every fit
// --skill directories; the session branches three or four at a time beyond the session-shaped // run needs its -e extension to see what loaded); one or three or more --skill directories; the
// session branches three or four at a time beyond the session-shaped
// runs; -c over a session directory that holds nothing for the project (the adapter passes -c // runs; -c over a session directory that holds nothing for the project (the adapter passes -c
// whenever the directory isn't empty), since one more run costs about 340 ms cold; and a // whenever the directory isn't empty), since one more run costs about 340 ms cold; and a
// session's own values: its tool names beyond read and bash, thinking levels beyond low, // session's own values: its tool names beyond read and bash, thinking levels beyond low,
@@ -143,10 +149,10 @@ export function sessionShape({ noMcp, extensions, prompt, sessionDir, tools, con
"--system-prompt", prompt, "-p", message, ...(json ? ["--mode", "json"] : []), "--system-prompt", prompt, "-p", message, ...(json ? ["--mode", "json"] : []),
]; ];
} }
// pi-skills: the arguments adapters/pi/adapter.sh builds for a headless worker run (run-task.sh in // pi-skills: the arguments adapters/pi/adapter.sh builds in -p with MOSAIC_SKILLS, no tools and no
// a container) with MOSAIC_SKILLS and no tools: the given -e files, --no-skills and then a --skill // session: the given -e files, --no-skills and then a --skill per directory, --no-tools,
// per directory, --no-tools, --no-session, the --provider and --model, the --system-prompt file, // --no-session, the --provider and --model, the --system-prompt file, -p and the request. Only
// -p and the request. Keep its order: pi-fit.test.mjs runs the adapter with MOSAIC_SKILLS and // the interactive launch sets MOSAIC_SKILLS today, in the TUI; this is the nearest headless form. Keep its order: pi-fit.test.mjs runs the adapter with MOSAIC_SKILLS and
// fails when the two differ. // fails when the two differ.
export function workerShape({ noMcp, extensions, skills, prompt, message }) { export function workerShape({ noMcp, extensions, skills, prompt, message }) {
return [ return [
@@ -403,9 +409,8 @@ async function judge(found, { levels, who, digest, run }) {
if (shape.cont) plantSession(s); if (shape.cont) plantSession(s);
return sessionShape({ noMcp, extensions: [join(s, "explicit", "fit.ts"), join(s, "explicit", "second.ts")], prompt: join(s, "prompt.md"), sessionDir: join(s, "sessions"), message: "fit check", ...shape }); return sessionShape({ noMcp, extensions: [join(s, "explicit", "fit.ts"), join(s, "explicit", "second.ts")], prompt: join(s, "prompt.md"), sessionDir: join(s, "sessions"), message: "fit check", ...shape });
}, "", { noMcp, shape })]); }, "", { noMcp, shape })]);
// -p takes a message; offline with no key, pi stops before any model call. A headless worker's // -p takes a message; offline with no key, pi stops before any model call. The adapter's -p argv
// shape with MOSAIC_SKILLS: two --skill directories right after --no-skills, as the adapter puts // with MOSAIC_SKILLS: two --skill directories right after --no-skills, as the adapter puts them.
// them.
const pRun = run((s) => workerShape({ noMcp, extensions: [join(s, "explicit", "fit.ts")], skills: GIVEN_SKILLS.map((n) => join(s, "given", n)), prompt: join(s, "prompt.md"), message: "fit check" }), "", { noMcp }); const pRun = run((s) => workerShape({ noMcp, extensions: [join(s, "explicit", "fit.ts")], skills: GIVEN_SKILLS.map((n) => join(s, "given", n)), prompt: join(s, "prompt.md"), message: "fit check" }), "", { noMcp });
// A missing -e file stops the start. pi 0.85.1 loads the -e files in order and still loads the // A missing -e file stops the start. pi 0.85.1 loads the -e files in order and still loads the
// one after the missing one, then exits 1 before session_start, so only session_start tells. // one after the missing one, then exits 1 before session_start, so only session_start tells.
@@ -464,7 +469,6 @@ async function judge(found, { levels, who, digest, run }) {
const p = await pRun; const p = await pRun;
if (p.stopped) return refuse(`can't be checked: the start in a worker's shape ${p.stopped}`); if (p.stopped) return refuse(`can't be checked: the start in a worker's shape ${p.stopped}`);
if (p.prompt === null) return refuse("can't be checked: its -e extension didn't see the system prompt in a worker's shape"); if (p.prompt === null) return refuse("can't be checked: its -e extension didn't see the system prompt in a worker's shape");
if (!p.prompt.includes(GIVEN)) return refuse("didn't use the --system-prompt file in a worker's shape");
if (p.seen.length) return refuse(`loaded a planted ${p.seen.map((x) => x.source).join(", ")} in a worker's shape with --skill`); if (p.seen.length) return refuse(`loaded a planted ${p.seen.map((x) => x.source).join(", ")} in a worker's shape with --skill`);
if (!p.skills) return refuse("didn't load every --skill directory with --no-skills on in a worker's shape"); if (!p.skills) return refuse("didn't load every --skill directory with --no-skills on in a worker's shape");
+8 -8
View File
@@ -124,8 +124,8 @@ test("pi-unpin S3: a stand-in that touches the planted MCP marker under the seal
}); });
// pi-skills: the worker's shape. pi 0.85.1 adds a --skill directory to discovery, so the adapter // pi-skills: the worker's shape. pi 0.85.1 adds a --skill directory to discovery, so the adapter
// keeps --no-skills in front of each --skill; the fit check's -p run passes the adapter's worker // keeps --no-skills in front of each --skill; the fit check's last -p run passes the adapter's -p
// argv (workerShape) with two --skill directories. // argv with MOSAIC_SKILLS (workerShape) and two --skill directories.
test("pi-skills: a stand-in that loads the planted skills once a --skill directory is given is refused", SKIP, async t => { test("pi-skills: a stand-in that loads the planted skills once a --skill directory is given is refused", SKIP, async t => {
const s = standIn(t, { skillLeak: true }); const s = standIn(t, { skillLeak: true });
const problem = await pi.preflight({ env: { ...CREDENTIAL, ...s.env } }); const problem = await pi.preflight({ env: { ...CREDENTIAL, ...s.env } });
@@ -140,7 +140,7 @@ test("pi-skills: a stand-in that drops a --skill directory once --no-skills is o
noPath(problem.message, s.paths); noPath(problem.message, s.paths);
}); });
// Review P3-1: the run is the adapter's worker argv, --provider and --model included, so a pi that // Review P3-1: the run is the adapter's -p argv, --provider and --model included, so a pi that
// leaks only once --skill and --provider are both given is seen. // leaks only once --skill and --provider are both given is seen.
test("pi-skills review: a stand-in that loads the planted sources only with --skill and --provider is refused", SKIP, async t => { test("pi-skills review: a stand-in that loads the planted sources only with --skill and --provider is refused", SKIP, async t => {
const s = standIn(t, { leakWith: ["--skill", "--provider"] }); const s = standIn(t, { leakWith: ["--skill", "--provider"] });
@@ -215,7 +215,7 @@ test("pi-unpin S3 re-check 3: every run has its own scratch directory, and the r
const shaped = (args) => args.includes("--session-dir"); const shaped = (args) => args.includes("--session-dir");
assert.deepEqual(runs.filter((x) => shaped(x.args)).map((x) => JSON.stringify(x.args)).sort(), shapes.map((shape) => JSON.stringify(fitModule.sessionShape({ noMcp: extraFlags.length > 0, extensions: ["<s>/explicit/fit.ts", "<s>/explicit/second.ts"], prompt: "<s>/prompt.md", sessionDir: "<s>/sessions", message: "fit check", ...shape }))).sort()); assert.deepEqual(runs.filter((x) => shaped(x.args)).map((x) => JSON.stringify(x.args)).sort(), shapes.map((shape) => JSON.stringify(fitModule.sessionShape({ noMcp: extraFlags.length > 0, extensions: ["<s>/explicit/fit.ts", "<s>/explicit/second.ts"], prompt: "<s>/prompt.md", sessionDir: "<s>/sessions", message: "fit check", ...shape }))).sort());
const flagLists = []; const flagLists = [];
// pi-skills: the -p run is the adapter's worker argv with two --skill directories, built by // pi-skills: the last -p run is the adapter's -p argv with two --skill directories, built by
// workerShape, which the adapter test below holds to the adapter. No other run passes --skill. // workerShape, which the adapter test below holds to the adapter. No other run passes --skill.
const skilled = runs.filter((x) => x.args.includes("--skill")); const skilled = runs.filter((x) => x.args.includes("--skill"));
assert.equal(skilled.length, 1); assert.equal(skilled.length, 1);
@@ -1041,9 +1041,9 @@ test("pi-skills: with MOSAIC_SKILLS the adapter passes --no-skills, then --skill
assert.ok(!bare.argv().includes("--skill")); assert.ok(!bare.argv().includes("--skill"));
}); });
// Review P3-1: the fit check's worker-shaped run is the argv the adapter builds for a headless // Review P3-1: the fit check's worker-shaped run is the argv the adapter builds in -p with
// worker with MOSAIC_SKILLS, an -e file and no tools, flag for flag and in order. // MOSAIC_SKILLS, an -e file, no tools and no session, flag for flag and in order.
test("pi-skills review: the adapter's worker argv with MOSAIC_SKILLS is workerShape's", t => { test("pi-skills review: the adapter's -p argv with MOSAIC_SKILLS is workerShape's", t => {
const w = workerAdapter(t, (root) => { const w = workerAdapter(t, (root) => {
for (const n of ["one", "two"]) mkdirSync(join(root, "skills", n), { recursive: true }); for (const n of ["one", "two"]) mkdirSync(join(root, "skills", n), { recursive: true });
writeFileSync(join(root, "fit.ts"), ""); writeFileSync(join(root, "fit.ts"), "");
@@ -1063,7 +1063,7 @@ test("pi-skills: a missing MOSAIC_SKILLS directory still exits 2 before pi runs"
assert.equal(existsSync(join(w.root, "argv")), false, "pi did not run"); assert.equal(existsSync(join(w.root, "argv")), false, "pi did not run");
}); });
// pi-skills: the worker's shape end to end. The adapter runs real pi 0.85.1 with MOSAIC_SKILLS, an // pi-skills: the adapter's -p shape end to end. It runs real pi 0.85.1 with MOSAIC_SKILLS, an
// -e extension that writes the command names at session_start, a HOME and workspace holding planted // -e extension that writes the command names at session_start, a HOME and workspace holding planted
// skills in every place pi 0.85.1 looks, the network unshared and no key: pi stops before any model // skills in every place pi 0.85.1 looks, the network unshared and no key: pi stops before any model
// call. Only the given skills may load. On the adapter that dropped --no-skills, the user, settings // call. Only the given skills may load. On the adapter that dropped --no-skills, the user, settings