fix(adapters,harness): pi-skills re-check: the docs no longer call the fit check's last -p run a headless worker's argv, since task launches refuse skill directories and set MOSAIC_SKILLS empty (packages/processing-control containers.mjs) and only the interactive launch (scripts/agent.sh) sets it, in the TUI with --tools and --session-dir; the run is now named the nearest headless form of the adapter's --skill argv, and the Not run notes name the TUI launch's flags and --skill with no -e (R2-P2-1, R2-P3-1); the untested --system-prompt check added to that run is dropped, as at BASE (R2-P3-2) (#1873, lane pi-skills)
This commit was merged in pull request #2014.
This commit is contained in:
1 parent
59af86a5ee
commit
ca7388743b
5 files changed
+56
-43
No files matched your search
+8
-5
@@ -108,16 +108,19 @@ activity through its existing extension. Mock remains a test-only echo.
|
|||||||
|
|
||||||
### A worker's skills (M17, #1873)
|
### A worker's skills (M17, #1873)
|
||||||
|
|
||||||
Worker runs and `scripts/agent.sh` may set `MOSAIC_SKILLS`, a comma list of
|
`MOSAIC_SKILLS` is a comma list of skill directories. Today only the
|
||||||
skill directories. The pi adapter passes `--no-skills` and then
|
interactive container launch (`scripts/agent.sh`) sets it; task launches
|
||||||
|
refuse skill directories and set it empty
|
||||||
|
(`packages/processing-control/src/containers.mjs`). The pi adapter passes `--no-skills` and then
|
||||||
`--skill <dir>` for each one. On pi 0.85.1 `--skill` adds to discovery and
|
`--skill <dir>` for each one. On pi 0.85.1 `--skill` adds to discovery and
|
||||||
does not replace it. With `--skill` alone, a probe with a scratch HOME also
|
does not replace it. With `--skill` alone, a probe with a scratch HOME also
|
||||||
loaded the user's skills, the skills a `settings.json` or a listed package
|
loaded the user's skills, the skills a `settings.json` or a listed package
|
||||||
names, and `~/.agents/skills` (and the project's, without `--no-approve`).
|
names, and `~/.agents/skills` (and the project's, without `--no-approve`).
|
||||||
With `--no-skills` first, only the given directories load. A directory that
|
With `--no-skills` first, only the given directories load. A directory that
|
||||||
doesn't exist stops the adapter with exit 2. The fit check runs this
|
doesn't exist stops the adapter with exit 2. The fit check runs the
|
||||||
argv, with two skill directories, one `-e` file and no tools, as its last
|
headless form of this argv, with two skill directories, one `-e` file, no
|
||||||
`-p` run (`workerShape`, `packages/harness/README.md`). The fit check runs
|
tools and no session, as its last `-p` run (`workerShape`,
|
||||||
|
`packages/harness/README.md`); the TUI shape the launch uses is not run. The fit check runs
|
||||||
before bus sessions only; worker containers run the pi pinned in the image.
|
before bus sessions only; worker containers run the pi pinned in the image.
|
||||||
|
|
||||||
### A role's skills (row 64)
|
### A role's skills (row 64)
|
||||||
|
|||||||
@@ -62,7 +62,8 @@ TOOLS_FLAG="--no-tools"
|
|||||||
# with --skill alone pi also loads the user's skills, settings and package
|
# with --skill alone pi also loads the user's skills, settings and package
|
||||||
# skills and ~/.agents/skills (a planted-skill probe showed it, offline, with a
|
# skills and ~/.agents/skills (a planted-skill probe showed it, offline, with a
|
||||||
# scratch HOME). With --no-skills first only the given dirs load. The fit check
|
# scratch HOME). With --no-skills first only the given dirs load. The fit check
|
||||||
# (packages/harness/src/pi-fit.mjs) runs this argv as its last -p run (workerShape).
|
# (packages/harness/src/pi-fit.mjs) runs its -p form as its last run (workerShape).
|
||||||
|
# Only the interactive launch (scripts/agent.sh) sets MOSAIC_SKILLS today.
|
||||||
# Role sessions never get MOSAIC_SKILLS (ENV_ALLOW in packages/seat/src/
|
# Role sessions never get MOSAIC_SKILLS (ENV_ALLOW in packages/seat/src/
|
||||||
# session.mjs leaves it out), so they run with --no-skills alone; their skills
|
# session.mjs leaves it out), so they run with --no-skills alone; their skills
|
||||||
# are text in the system prompt (row 64).
|
# are text in the system prompt (row 64).
|
||||||
|
|||||||
@@ -92,11 +92,15 @@ that no session sets. So a branch on `TZ` or `LC_ALL` being unset, or on
|
|||||||
`/bin/sh` sets an unset `TERM` to `dumb` for the shell without exporting
|
`/bin/sh` sets an unset `TERM` to `dumb` for the shell without exporting
|
||||||
it, so `${TERM:-}` in the adapter is never empty, in a session or here.)
|
it, so `${TERM:-}` in the adapter is never empty, in a session or here.)
|
||||||
It can't see a branch keyed on a
|
It can't see a branch keyed on a
|
||||||
value it doesn't try (a model, a path, another tool or provider). A last `-p` run is a
|
value it doesn't try (a model, a path, another tool or provider). A last `-p` run is the
|
||||||
headless worker's: the argv the adapter builds with `MOSAIC_SKILLS`, one
|
argv the adapter builds in `-p` with `MOSAIC_SKILLS`, one `-e` file, no
|
||||||
`-e` file and no tools (`workerShape`), so `--no-skills` and then two
|
tools and no session (`workerShape`), so `--no-skills` and then two
|
||||||
`--skill` directories, `--no-tools`, `--no-session`, `--provider` and
|
`--skill` directories, `--no-tools`, `--no-session`, `--provider` and
|
||||||
`--model`. A test runs the adapter and holds its argv to `workerShape`.
|
`--model`. A test runs the adapter and holds its argv to `workerShape`.
|
||||||
|
No caller makes this exact run today: task launches refuse skill
|
||||||
|
directories, and only the interactive launch (`scripts/agent.sh`) sets
|
||||||
|
`MOSAIC_SKILLS`, in the TUI, which no fit run can drive. It is the
|
||||||
|
nearest headless form of the adapter's `--skill` argv.
|
||||||
The run's `-e` extension must list both skills and no planted one. pi
|
The run's `-e` extension must list both skills and no planted one. pi
|
||||||
0.85.1 adds a `--skill` directory to discovery, so a pi that let
|
0.85.1 adds a `--skill` directory to discovery, so a pi that let
|
||||||
`--skill` bring the planted skills back is refused there, and so is one
|
`--skill` bring the planted skills back is refused there, and so is one
|
||||||
@@ -105,11 +109,12 @@ replaces the plain sealed `-p` run with `--no-session` and no `--skill`,
|
|||||||
so it adds no run. Each run gets a fresh scratch directory, so no run sees
|
so it adds no run. Each run gets a fresh scratch directory, so no run sees
|
||||||
what another left. Not run: `--fork` and the interactive TUI
|
what another left. Not run: `--fork` and the interactive TUI
|
||||||
(worker containers and `scripts/agent.sh` only), `-p` with
|
(worker containers and `scripts/agent.sh` only), `-p` with
|
||||||
`--no-session` and no `--skill` (a worker without `MOSAIC_SKILLS`; every
|
`--no-session` and no `--skill` (a task launch's shape; every bus session
|
||||||
bus session passes `--session-dir`), `--skill` beside `--tools`,
|
passes `--session-dir`), `--skill` beside `--tools`, `--session-dir`,
|
||||||
`--session-dir`, `-c`, `--thinking` or `--mode json`, or in rpc (a worker
|
`-c`, `--thinking` or `--mode json`, or in rpc (the TUI launch passes the
|
||||||
may pass the first two; no bus session passes `--skill`), one or three or
|
first three; no bus session passes `--skill`), `--skill` with no `-e`
|
||||||
more `--skill` directories, the branches three or
|
(every fit run needs its `-e` extension to see what loaded), one or three
|
||||||
|
or more `--skill` directories, the branches three or
|
||||||
four at a time beyond the five runs, `-c` over a session directory that
|
four at a time beyond the five runs, `-c` over a session directory that
|
||||||
holds nothing for the project (one more run costs about 340 ms cold),
|
holds nothing for the project (one more run costs about 340 ms cold),
|
||||||
and a session's own values (tools beyond `read` and `bash`, thinking
|
and a session's own values (tools beyond `read` and `bash`, thinking
|
||||||
|
|||||||
@@ -39,16 +39,21 @@
|
|||||||
// planted source. A test drives the adapter through every branch set, tool list
|
// planted source. A test drives the adapter through every branch set, tool list
|
||||||
// length and thinking level, and fails on an argv sessionShape doesn't build or a
|
// length and thinking level, and fails on an argv sessionShape doesn't build or a
|
||||||
// pair no run passes.
|
// pair no run passes.
|
||||||
// worker -p the arguments adapters/pi/adapter.sh passes for a headless worker run with
|
// worker -p the arguments adapters/pi/adapter.sh builds in -p with MOSAIC_SKILLS, one -e
|
||||||
// MOSAIC_SKILLS and no tools, in its order (workerShape below): the -e, --no-skills
|
// file, no tools and no session, in its order (workerShape below): the -e,
|
||||||
// then two --skill directories, --no-tools, --no-session, the fixed --provider and
|
// --no-skills then two --skill directories, --no-tools, --no-session, the fixed
|
||||||
// --model, --system-prompt, -p and a message. Its -e extension must reach
|
// --provider and --model, --system-prompt, -p and a message. Its -e extension must
|
||||||
// session_start and list both given skills, and no planted source may show. pi
|
// reach session_start and list both given skills, and no planted source may show.
|
||||||
// 0.85.1 adds a --skill directory to discovery, so a pi that let --skill turn
|
// pi 0.85.1 adds a --skill directory to discovery, so a pi that let --skill turn
|
||||||
// discovery back on would show the planted skills here, and one that drops a --skill
|
// discovery back on would show the planted skills here, and one that drops a --skill
|
||||||
// directory once --no-skills is on would not list it. This run replaces the sealed
|
// directory once --no-skills is on would not list it. No caller makes this exact
|
||||||
// -p run the check made before (the seal flags in -p with --no-session and no
|
// run today: task launches refuse skill directories (packages/processing-control
|
||||||
// --skill), so it adds no run; that plain shape is not run on its own any more.
|
// containers.mjs), and only the interactive container launch (scripts/agent.sh)
|
||||||
|
// sets MOSAIC_SKILLS, in the TUI with --tools and --session-dir, which no fit run
|
||||||
|
// can drive. This is the nearest headless form of the adapter's --skill argv. It
|
||||||
|
// replaces the sealed -p run the check made before (the seal flags in -p with
|
||||||
|
// --no-session and no --skill), so it adds no run; that plain shape is not run on
|
||||||
|
// its own any more.
|
||||||
// missing -e a missing -e file, in rpc and in -p, stops the start: exit non-zero, no answer,
|
// missing -e a missing -e file, in rpc and in -p, stops the start: exit non-zero, no answer,
|
||||||
// and no session_start for the -e extension given after it
|
// and no session_start for the -e extension given after it
|
||||||
//
|
//
|
||||||
@@ -86,10 +91,11 @@
|
|||||||
// Known limit: pi 0.85.1 appends the agent directory's APPEND_SYSTEM.md whatever the flags. The
|
// Known limit: pi 0.85.1 appends the agent directory's APPEND_SYSTEM.md whatever the flags. The
|
||||||
// check doesn't plant one; a session's agent directory is the run's own. Not run: --fork and
|
// check doesn't plant one; a session's agent directory is the run's own. Not run: --fork and
|
||||||
// the interactive TUI, which only worker containers and scripts/agent.sh pass; -p with
|
// the interactive TUI, which only worker containers and scripts/agent.sh pass; -p with
|
||||||
// --no-session and no --skill (a worker with no MOSAIC_SKILLS; every bus session passes
|
// --no-session and no --skill (a task launch's shape; every bus session passes --session-dir);
|
||||||
// --session-dir); --skill beside --tools, --session-dir, -c, --thinking or --mode json, or in rpc
|
// --skill beside --tools, --session-dir, -c, --thinking or --mode json, or in rpc (the TUI
|
||||||
// (a worker may pass the first two; no bus session passes --skill), and one or three or more
|
// launch passes the first three; no bus session passes --skill); --skill with no -e (every fit
|
||||||
// --skill directories; the session branches three or four at a time beyond the session-shaped
|
// run needs its -e extension to see what loaded); one or three or more --skill directories; the
|
||||||
|
// session branches three or four at a time beyond the session-shaped
|
||||||
// runs; -c over a session directory that holds nothing for the project (the adapter passes -c
|
// runs; -c over a session directory that holds nothing for the project (the adapter passes -c
|
||||||
// whenever the directory isn't empty), since one more run costs about 340 ms cold; and a
|
// whenever the directory isn't empty), since one more run costs about 340 ms cold; and a
|
||||||
// session's own values: its tool names beyond read and bash, thinking levels beyond low,
|
// session's own values: its tool names beyond read and bash, thinking levels beyond low,
|
||||||
@@ -143,10 +149,10 @@ export function sessionShape({ noMcp, extensions, prompt, sessionDir, tools, con
|
|||||||
"--system-prompt", prompt, "-p", message, ...(json ? ["--mode", "json"] : []),
|
"--system-prompt", prompt, "-p", message, ...(json ? ["--mode", "json"] : []),
|
||||||
];
|
];
|
||||||
}
|
}
|
||||||
// pi-skills: the arguments adapters/pi/adapter.sh builds for a headless worker run (run-task.sh in
|
// pi-skills: the arguments adapters/pi/adapter.sh builds in -p with MOSAIC_SKILLS, no tools and no
|
||||||
// a container) with MOSAIC_SKILLS and no tools: the given -e files, --no-skills and then a --skill
|
// session: the given -e files, --no-skills and then a --skill per directory, --no-tools,
|
||||||
// per directory, --no-tools, --no-session, the --provider and --model, the --system-prompt file,
|
// --no-session, the --provider and --model, the --system-prompt file, -p and the request. Only
|
||||||
// -p and the request. Keep its order: pi-fit.test.mjs runs the adapter with MOSAIC_SKILLS and
|
// the interactive launch sets MOSAIC_SKILLS today, in the TUI; this is the nearest headless form. Keep its order: pi-fit.test.mjs runs the adapter with MOSAIC_SKILLS and
|
||||||
// fails when the two differ.
|
// fails when the two differ.
|
||||||
export function workerShape({ noMcp, extensions, skills, prompt, message }) {
|
export function workerShape({ noMcp, extensions, skills, prompt, message }) {
|
||||||
return [
|
return [
|
||||||
@@ -403,9 +409,8 @@ async function judge(found, { levels, who, digest, run }) {
|
|||||||
if (shape.cont) plantSession(s);
|
if (shape.cont) plantSession(s);
|
||||||
return sessionShape({ noMcp, extensions: [join(s, "explicit", "fit.ts"), join(s, "explicit", "second.ts")], prompt: join(s, "prompt.md"), sessionDir: join(s, "sessions"), message: "fit check", ...shape });
|
return sessionShape({ noMcp, extensions: [join(s, "explicit", "fit.ts"), join(s, "explicit", "second.ts")], prompt: join(s, "prompt.md"), sessionDir: join(s, "sessions"), message: "fit check", ...shape });
|
||||||
}, "", { noMcp, shape })]);
|
}, "", { noMcp, shape })]);
|
||||||
// -p takes a message; offline with no key, pi stops before any model call. A headless worker's
|
// -p takes a message; offline with no key, pi stops before any model call. The adapter's -p argv
|
||||||
// shape with MOSAIC_SKILLS: two --skill directories right after --no-skills, as the adapter puts
|
// with MOSAIC_SKILLS: two --skill directories right after --no-skills, as the adapter puts them.
|
||||||
// them.
|
|
||||||
const pRun = run((s) => workerShape({ noMcp, extensions: [join(s, "explicit", "fit.ts")], skills: GIVEN_SKILLS.map((n) => join(s, "given", n)), prompt: join(s, "prompt.md"), message: "fit check" }), "", { noMcp });
|
const pRun = run((s) => workerShape({ noMcp, extensions: [join(s, "explicit", "fit.ts")], skills: GIVEN_SKILLS.map((n) => join(s, "given", n)), prompt: join(s, "prompt.md"), message: "fit check" }), "", { noMcp });
|
||||||
// A missing -e file stops the start. pi 0.85.1 loads the -e files in order and still loads the
|
// A missing -e file stops the start. pi 0.85.1 loads the -e files in order and still loads the
|
||||||
// one after the missing one, then exits 1 before session_start, so only session_start tells.
|
// one after the missing one, then exits 1 before session_start, so only session_start tells.
|
||||||
@@ -464,7 +469,6 @@ async function judge(found, { levels, who, digest, run }) {
|
|||||||
const p = await pRun;
|
const p = await pRun;
|
||||||
if (p.stopped) return refuse(`can't be checked: the start in a worker's shape ${p.stopped}`);
|
if (p.stopped) return refuse(`can't be checked: the start in a worker's shape ${p.stopped}`);
|
||||||
if (p.prompt === null) return refuse("can't be checked: its -e extension didn't see the system prompt in a worker's shape");
|
if (p.prompt === null) return refuse("can't be checked: its -e extension didn't see the system prompt in a worker's shape");
|
||||||
if (!p.prompt.includes(GIVEN)) return refuse("didn't use the --system-prompt file in a worker's shape");
|
|
||||||
if (p.seen.length) return refuse(`loaded a planted ${p.seen.map((x) => x.source).join(", ")} in a worker's shape with --skill`);
|
if (p.seen.length) return refuse(`loaded a planted ${p.seen.map((x) => x.source).join(", ")} in a worker's shape with --skill`);
|
||||||
if (!p.skills) return refuse("didn't load every --skill directory with --no-skills on in a worker's shape");
|
if (!p.skills) return refuse("didn't load every --skill directory with --no-skills on in a worker's shape");
|
||||||
|
|
||||||
|
|||||||
@@ -124,8 +124,8 @@ test("pi-unpin S3: a stand-in that touches the planted MCP marker under the seal
|
|||||||
});
|
});
|
||||||
|
|
||||||
// pi-skills: the worker's shape. pi 0.85.1 adds a --skill directory to discovery, so the adapter
|
// pi-skills: the worker's shape. pi 0.85.1 adds a --skill directory to discovery, so the adapter
|
||||||
// keeps --no-skills in front of each --skill; the fit check's -p run passes the adapter's worker
|
// keeps --no-skills in front of each --skill; the fit check's last -p run passes the adapter's -p
|
||||||
// argv (workerShape) with two --skill directories.
|
// argv with MOSAIC_SKILLS (workerShape) and two --skill directories.
|
||||||
test("pi-skills: a stand-in that loads the planted skills once a --skill directory is given is refused", SKIP, async t => {
|
test("pi-skills: a stand-in that loads the planted skills once a --skill directory is given is refused", SKIP, async t => {
|
||||||
const s = standIn(t, { skillLeak: true });
|
const s = standIn(t, { skillLeak: true });
|
||||||
const problem = await pi.preflight({ env: { ...CREDENTIAL, ...s.env } });
|
const problem = await pi.preflight({ env: { ...CREDENTIAL, ...s.env } });
|
||||||
@@ -140,7 +140,7 @@ test("pi-skills: a stand-in that drops a --skill directory once --no-skills is o
|
|||||||
noPath(problem.message, s.paths);
|
noPath(problem.message, s.paths);
|
||||||
});
|
});
|
||||||
|
|
||||||
// Review P3-1: the run is the adapter's worker argv, --provider and --model included, so a pi that
|
// Review P3-1: the run is the adapter's -p argv, --provider and --model included, so a pi that
|
||||||
// leaks only once --skill and --provider are both given is seen.
|
// leaks only once --skill and --provider are both given is seen.
|
||||||
test("pi-skills review: a stand-in that loads the planted sources only with --skill and --provider is refused", SKIP, async t => {
|
test("pi-skills review: a stand-in that loads the planted sources only with --skill and --provider is refused", SKIP, async t => {
|
||||||
const s = standIn(t, { leakWith: ["--skill", "--provider"] });
|
const s = standIn(t, { leakWith: ["--skill", "--provider"] });
|
||||||
@@ -215,7 +215,7 @@ test("pi-unpin S3 re-check 3: every run has its own scratch directory, and the r
|
|||||||
const shaped = (args) => args.includes("--session-dir");
|
const shaped = (args) => args.includes("--session-dir");
|
||||||
assert.deepEqual(runs.filter((x) => shaped(x.args)).map((x) => JSON.stringify(x.args)).sort(), shapes.map((shape) => JSON.stringify(fitModule.sessionShape({ noMcp: extraFlags.length > 0, extensions: ["<s>/explicit/fit.ts", "<s>/explicit/second.ts"], prompt: "<s>/prompt.md", sessionDir: "<s>/sessions", message: "fit check", ...shape }))).sort());
|
assert.deepEqual(runs.filter((x) => shaped(x.args)).map((x) => JSON.stringify(x.args)).sort(), shapes.map((shape) => JSON.stringify(fitModule.sessionShape({ noMcp: extraFlags.length > 0, extensions: ["<s>/explicit/fit.ts", "<s>/explicit/second.ts"], prompt: "<s>/prompt.md", sessionDir: "<s>/sessions", message: "fit check", ...shape }))).sort());
|
||||||
const flagLists = [];
|
const flagLists = [];
|
||||||
// pi-skills: the -p run is the adapter's worker argv with two --skill directories, built by
|
// pi-skills: the last -p run is the adapter's -p argv with two --skill directories, built by
|
||||||
// workerShape, which the adapter test below holds to the adapter. No other run passes --skill.
|
// workerShape, which the adapter test below holds to the adapter. No other run passes --skill.
|
||||||
const skilled = runs.filter((x) => x.args.includes("--skill"));
|
const skilled = runs.filter((x) => x.args.includes("--skill"));
|
||||||
assert.equal(skilled.length, 1);
|
assert.equal(skilled.length, 1);
|
||||||
@@ -1041,9 +1041,9 @@ test("pi-skills: with MOSAIC_SKILLS the adapter passes --no-skills, then --skill
|
|||||||
assert.ok(!bare.argv().includes("--skill"));
|
assert.ok(!bare.argv().includes("--skill"));
|
||||||
});
|
});
|
||||||
|
|
||||||
// Review P3-1: the fit check's worker-shaped run is the argv the adapter builds for a headless
|
// Review P3-1: the fit check's worker-shaped run is the argv the adapter builds in -p with
|
||||||
// worker with MOSAIC_SKILLS, an -e file and no tools, flag for flag and in order.
|
// MOSAIC_SKILLS, an -e file, no tools and no session, flag for flag and in order.
|
||||||
test("pi-skills review: the adapter's worker argv with MOSAIC_SKILLS is workerShape's", t => {
|
test("pi-skills review: the adapter's -p argv with MOSAIC_SKILLS is workerShape's", t => {
|
||||||
const w = workerAdapter(t, (root) => {
|
const w = workerAdapter(t, (root) => {
|
||||||
for (const n of ["one", "two"]) mkdirSync(join(root, "skills", n), { recursive: true });
|
for (const n of ["one", "two"]) mkdirSync(join(root, "skills", n), { recursive: true });
|
||||||
writeFileSync(join(root, "fit.ts"), "");
|
writeFileSync(join(root, "fit.ts"), "");
|
||||||
@@ -1063,7 +1063,7 @@ test("pi-skills: a missing MOSAIC_SKILLS directory still exits 2 before pi runs"
|
|||||||
assert.equal(existsSync(join(w.root, "argv")), false, "pi did not run");
|
assert.equal(existsSync(join(w.root, "argv")), false, "pi did not run");
|
||||||
});
|
});
|
||||||
|
|
||||||
// pi-skills: the worker's shape end to end. The adapter runs real pi 0.85.1 with MOSAIC_SKILLS, an
|
// pi-skills: the adapter's -p shape end to end. It runs real pi 0.85.1 with MOSAIC_SKILLS, an
|
||||||
// -e extension that writes the command names at session_start, a HOME and workspace holding planted
|
// -e extension that writes the command names at session_start, a HOME and workspace holding planted
|
||||||
// skills in every place pi 0.85.1 looks, the network unshared and no key: pi stops before any model
|
// skills in every place pi 0.85.1 looks, the network unshared and no key: pi stops before any model
|
||||||
// call. Only the given skills may load. On the adapter that dropped --no-skills, the user, settings
|
// call. Only the given skills may load. On the adapter that dropped --no-skills, the user, settings
|
||||||
|
|||||||
Reference in new issue
Block a user