feat(plugins): harness watch W1: one pass over every installed harness gives a record and its transitions, with inspect for pi and claude-code (#1873) #2013
No files matched your search
@@ -367,6 +367,87 @@ the kind's whole setting, so it names every harness that stays on:
|
||||
{ "revision": "<from mosaic plugins>", "plugins": { "harness": ["pi", "claude-code"] } }
|
||||
```
|
||||
|
||||
### Harness watch and `inspect` (#1873, slice W1)
|
||||
|
||||
Jason's ruling (2026-11-19): no pins on any harness, a periodic check
|
||||
instead. `src/harness-watch.mjs` is the function that check calls.
|
||||
`watchHarnesses({ last, env, scratch, dataRoot, timeoutMs })` makes one
|
||||
pass over every harness `loadInstalled('harness', { dataRoot })` gives,
|
||||
tree or installed. It holds no name list, so a harness added later is
|
||||
watched at once. It writes nothing: it takes the records the last pass
|
||||
returned and gives back `{ records, transitions, refused, gone }`.
|
||||
|
||||
`inspect({ env, scratch, signal })` is an optional harness export, outside
|
||||
`kinds.mjs`. It returns a promise of:
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| `found` | `true` when the binary sessions run was found, `false` when not |
|
||||
| `source` | the lookup step that won, such as `MOSAIC_PI_BIN`, `PATH` or `repo`; never a path |
|
||||
| `version` | the binary's version label, or null |
|
||||
| `digest` | a sha256 hex digest of what runs, or null |
|
||||
| `fit` | `"pass"`, `"fail"` or `"unknown"`: the module's own fit check |
|
||||
| `definite` | for a `"fail"`, true when the check decided; a failure that could not tell is false |
|
||||
| `failed` | rule ids the check names, when it names any |
|
||||
| `line` | one plain line for the owner; it names a variable or a lookup step, never a path |
|
||||
| `surfaces` | name to text, such as `help` |
|
||||
|
||||
`env` is the environment of the process sessions launch from. `scratch` is
|
||||
a directory the module may use for a scratch HOME; it removes what it makes
|
||||
there. `signal` aborts when the watch stops waiting. `pi` returns its fit
|
||||
check (`packages/harness/src/pi-fit.mjs`), called exactly as `preflight`
|
||||
calls it, so the two share one cache and one running check. Its digest is
|
||||
that check's digest of the binary's files; a failed check gives none. A
|
||||
failure whose line, after pi's lookup label, starts with "can't be checked"
|
||||
is `fit: "unknown"`.
|
||||
`claude-code` returns the sha256 of the binary's real file and the first
|
||||
line of `--version`, and captures `--help` as the `help` surface, both run
|
||||
from that same real file with only PATH and a scratch HOME. Its lookup for
|
||||
the watch skips a relative PATH entry, as pi's does. Its fit stays
|
||||
`"unknown"` until slice W2 adds the claude fit rule (and runs these
|
||||
captures with no network).
|
||||
|
||||
A module with no `inspect` is watched through `version({ env })`, with fit
|
||||
`"unknown"`. A record's `state` is:
|
||||
|
||||
- `fit`: found, the fit check passed, the last known digest (or none known);
|
||||
- `changed`: found, the fit check passed, a digest that is not the last known
|
||||
one. The last known digest is the latest any earlier record saw, kept in
|
||||
each record's `lastDigest` across unknown, red and missing records, so a
|
||||
new binary first seen through a timeout still reads `changed`;
|
||||
- `red`: found, and the fit check failed with `definite: true`;
|
||||
- `unknown`: anything else, including no `inspect`, an `inspect` that throws
|
||||
or runs past `timeoutMs` (default 300 s), and an answer that isn't this
|
||||
shape;
|
||||
- `missing`: `inspect` says `found: false`.
|
||||
|
||||
Only `found: true` with `fit: "pass"` reads as `fit` or `changed`. The watch
|
||||
copies the answer's own data fields only: a field that is a getter, or an
|
||||
answer that throws when read (a Proxy), is a failed `inspect`, and an
|
||||
inherited field is not read. It checks each field's shape and drops what
|
||||
doesn't fit (a `source` that isn't a plain name, a digest that isn't 64 hex
|
||||
characters, a `source` or rule id that is an env value). In every text
|
||||
field it replaces the scratch directory, the home directory, every env value
|
||||
of six characters or more and every PATH entry, each also as its real path
|
||||
and without a trailing slash, with a name for it, the user name (USER,
|
||||
LOGNAME, the process's account) with `<user>`, then any other absolute path
|
||||
of two segments or more with `<path>`. The loader's refusals in `refused`
|
||||
get the same treatment. A record also carries `via` (`inspect` or
|
||||
`version`), `lastDigest`, `checkedAt` and `changedAt`. A transition
|
||||
(`{ harness, at, from, to, failed, line }`, with `from` null on a first
|
||||
sight) is given only when the state or the binary is not the last record's.
|
||||
The digest names the binary; when neither record has one (pi gives none for
|
||||
a failed check), the version label stands in for it, and only when both
|
||||
records have a version: a timeout or a throw knows none, and is not a new
|
||||
binary. So unknown 2.0, a timeout, then unknown 2.1, all with no digest,
|
||||
gives no transition for the version; W1 keeps no last known version.
|
||||
|
||||
The redaction applies to the stored surfaces too, so a surface is for
|
||||
display and comparison, not for fit rules: with USER set to `model`, the
|
||||
stored `help` reads `--<user>` where claude printed `--model`. A fit rule
|
||||
(slice W2's flag check, for one) runs inside the module on the text the
|
||||
binary printed and puts its answer in `fit`, `failed` and `line`.
|
||||
|
||||
## Transcript usage source (row 124)
|
||||
|
||||
`usage-source/transcript` now ships interface 1's `readTurn({harness, root})`.
|
||||
|
||||
@@ -4,8 +4,9 @@
|
||||
// both stay in packages/harness (a module can't change that boundary). The launcher, the bundle
|
||||
// and the usage reader call these on the host; the runner gets the answers as data in
|
||||
// session.json. packages/plugins/README.md lists the interface.
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { accessSync, constants, realpathSync, statSync } from "node:fs";
|
||||
import { spawn, spawnSync } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { accessSync, constants, createReadStream, mkdtempSync, realpathSync, rmSync, statSync } from "node:fs";
|
||||
import { delimiter, dirname, isAbsolute, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { claudeBuiltins, MCP_PREFIX } from "../../../harness/src/gate.mjs";
|
||||
@@ -89,6 +90,61 @@ export function version({ env }) {
|
||||
return r.status === 0 ? r.stdout.trim().split("\n")[0] || null : null;
|
||||
}
|
||||
|
||||
// #1873 harness watch W1: what the watch records for the binary sessions run: the lookup step that
|
||||
// found it (MOSAIC_CLAUDE_BIN or PATH, never a path), the sha256 of its real file, the first line
|
||||
// of --version and the whole --help as a surface. Both run with an empty environment but PATH and
|
||||
// a fresh HOME under scratch, which is removed after. Without scratch they don't run. The real path
|
||||
// is resolved once, then hashed and run, so the digest and the captures are of one file; a relative
|
||||
// PATH entry is skipped, as pi's lookup skips it. Fit stays "unknown": the claude fit rule (every
|
||||
// flag the adapter passes is in --help) is slice W2, with the no-network wrapper for these runs.
|
||||
const CAPTURE_TIMEOUT = 10000;
|
||||
const CAPTURE_MAX = 1024 * 1024;
|
||||
export async function inspect({ env = {}, scratch = null, signal } = {}) {
|
||||
const bin = binary({ ...env, PATH: (env.PATH ?? "").split(delimiter).filter((dir) => isAbsolute(dir)).join(delimiter) });
|
||||
const source = env.MOSAIC_CLAUDE_BIN ? "MOSAIC_CLAUDE_BIN" : bin ? "PATH" : null;
|
||||
const base = { source, fit: "unknown", definite: false, failed: [], surfaces: {} };
|
||||
if (!bin) return { ...base, found: false, version: null, digest: null, line: env.MOSAIC_CLAUDE_BIN ? "MOSAIC_CLAUDE_BIN isn't an executable file at an absolute path" : "no claude on PATH" };
|
||||
let real;
|
||||
try { real = realpathSync(bin); } catch { return { ...base, found: false, version: null, digest: null, line: `the claude from ${source} can't be resolved` }; }
|
||||
const digest = await sha256(real);
|
||||
if (typeof scratch !== "string" || !isAbsolute(scratch)) return { ...base, found: true, version: null, digest, line: "no scratch directory, so --version and --help were not run; fit not checked" };
|
||||
const home = mkdtempSync(join(scratch, "claude-home-"));
|
||||
try {
|
||||
const runEnv = { PATH: "/usr/bin:/bin", HOME: home };
|
||||
const [v, help] = [await capture(real, ["--version"], runEnv, signal), await capture(real, ["--help"], runEnv, signal)];
|
||||
const line = v?.split("\n")[0].trim() ?? "";
|
||||
const version = line && line.length <= 128 ? line : null;
|
||||
return { ...base, found: true, version, digest, line: help === null ? "--help gave no answer; fit not checked" : "fit not checked", surfaces: help === null ? {} : { help } };
|
||||
} finally {
|
||||
rmSync(home, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
function sha256(file) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const h = createHash("sha256");
|
||||
createReadStream(file).on("error", reject).on("data", (c) => h.update(c)).on("end", () => resolve(h.digest("hex")));
|
||||
});
|
||||
}
|
||||
|
||||
// stdout of bin args when it exits 0 within CAPTURE_TIMEOUT and prints at most CAPTURE_MAX, else null.
|
||||
function capture(bin, args, runEnv, signal) {
|
||||
return new Promise((resolve) => {
|
||||
let child;
|
||||
try { child = spawn(bin, args, { env: runEnv, cwd: runEnv.HOME, stdio: ["ignore", "pipe", "ignore"] }); } catch { resolve(null); return; }
|
||||
const out = [];
|
||||
let size = 0, done = false;
|
||||
const finish = (v) => { if (done) return; done = true; clearTimeout(timer); signal?.removeEventListener?.("abort", kill); resolve(v); };
|
||||
const kill = () => { try { child.kill("SIGKILL"); } catch {} finish(null); };
|
||||
const timer = setTimeout(kill, CAPTURE_TIMEOUT);
|
||||
signal?.addEventListener?.("abort", kill, { once: true });
|
||||
if (signal?.aborted) kill();
|
||||
child.stdout.on("data", (c) => { size += c.length; if (size > CAPTURE_MAX) kill(); else out.push(c); });
|
||||
child.on("error", () => finish(null));
|
||||
child.on("close", (status) => finish(status === 0 ? Buffer.concat(out).toString("utf8") : null));
|
||||
});
|
||||
}
|
||||
|
||||
// Its launches spend the business's launch.account (row 126).
|
||||
export const countsAccount = () => true;
|
||||
|
||||
|
||||
@@ -55,6 +55,28 @@ export async function version({ env = {} } = {}) {
|
||||
return printedVersion(locate(env).bin);
|
||||
}
|
||||
|
||||
// #1873 harness watch W1: what the watch records for the pi binary sessions run. It calls the fit
|
||||
// check exactly as preflight does (same found binary, same levels, this process's environment), so
|
||||
// it shares preflight's cache and its running check: a binary that passed is not checked again.
|
||||
// digest is the fit check's digest of the binary's files; a check that fails gives none, since
|
||||
// pi-fit.mjs exports no digest of its own. A failure whose reason starts "can't be checked" could
|
||||
// not decide, so its fit is "unknown"; any other failure is definite. source names the lookup step,
|
||||
// never a path. scratch is not used: the fit check makes its own scratch directories.
|
||||
const SOURCES = Object.freeze({ MOSAIC_PI_BIN: "MOSAIC_PI_BIN", "the pi on PATH": "PATH", "the repository's pi": "repo" });
|
||||
export async function inspect({ env = {} } = {}) {
|
||||
const at = locate(env);
|
||||
if (!at.bin) return { found: false, source: env.MOSAIC_PI_BIN ? "MOSAIC_PI_BIN" : null, version: null, digest: null, fit: "unknown", definite: false, failed: [], line: at.problem, surfaces: {} };
|
||||
const source = SOURCES[at.label] ?? null;
|
||||
const fit = await piFit(at, { levels: LEVELS });
|
||||
if (fit.ok) return { found: true, source, version: fit.version ?? null, digest: fit.digest, fit: "pass", definite: true, failed: [], line: `pi${fit.version ? ` ${fit.version}` : ""} (${at.label}) passed the fit check`, surfaces: {} };
|
||||
// Read after pi-fit's "pi <version> (<label>) " prefix, so a version label cannot decide it. A line
|
||||
// without that prefix keeps the whole-line match: an unknown line must not read as definite.
|
||||
const mark = `(${at.label}) `;
|
||||
const cut = fit.message.indexOf(mark);
|
||||
const undecided = cut === -1 ? /can't be checked/.test(fit.message) : fit.message.slice(cut + mark.length).startsWith("can't be checked");
|
||||
return { found: true, source, version: await printedVersion(at.bin), digest: null, fit: undecided ? "unknown" : "fail", definite: !undecided, failed: [], line: fit.message, surfaces: {} };
|
||||
}
|
||||
|
||||
// #1873 S3: runs the fit check, or shares the one running, and throws its refusal line when the
|
||||
// binary fails it. A host that calls env() without preflight (Triage) awaits this first.
|
||||
export async function ready({ env = process.env } = {}) {
|
||||
|
||||
@@ -0,0 +1,254 @@
|
||||
// #1873 harness watch, slice W1 (reports/harness-watch-design.md sections 1 and 5): one pass over
|
||||
// every harness the loader gives, tree or installed. No name list: a harness added later is
|
||||
// watched at once. For each module it calls the optional inspect({ env, scratch, signal }), or
|
||||
// falls back to version({ env }) with fit "unknown". It compares the answer with the last record
|
||||
// the caller passes in and returns the new records and the transitions. It writes nothing: no
|
||||
// state file, no log, no timer. The caller (W2's host timer) keeps the records.
|
||||
//
|
||||
// A record's state:
|
||||
// fit found, the module's fit check passed, and the digest is the last known one (or there
|
||||
// is no known digest to compare with)
|
||||
// changed found, the fit check passed, and the digest differs from the last known one: the
|
||||
// latest digest any earlier record saw, kept in lastDigest across unknown, red and
|
||||
// missing records, so a new binary first seen through a timeout still reads changed
|
||||
// red found, and the fit check failed with a definite answer
|
||||
// unknown anything else: the check could not decide, the module has no inspect, inspect threw,
|
||||
// timed out or gave an answer that isn't the contract's
|
||||
// missing inspect says the binary is not found
|
||||
// Only fit === "pass" with found === true reads as fit or changed. Nothing else can.
|
||||
//
|
||||
// No record field holds a host path or an env value: source must be a lookup step's name
|
||||
// (MOSAIC_PI_BIN, PATH, repo), never a path, and every text field has the scratch directory,
|
||||
// the home directory, each env value and each PATH entry (and their real paths) replaced by a
|
||||
// name for it, then every other absolute path replaced by <path>.
|
||||
import { realpathSync } from "node:fs";
|
||||
import { homedir, userInfo } from "node:os";
|
||||
import { isAbsolute } from "node:path";
|
||||
import { loadInstalled } from "./loader.mjs";
|
||||
|
||||
export const STATES = Object.freeze(["fit", "changed", "red", "unknown", "missing"]);
|
||||
const FITS = new Set(["pass", "fail", "unknown"]);
|
||||
// An inspect that takes longer than this gives unknown. pi's fit check runs many sandboxed starts.
|
||||
export const INSPECT_TIMEOUT_MS = 300000;
|
||||
const SOURCE = /^[A-Za-z][A-Za-z0-9_]{0,63}$/;
|
||||
const RULE = /^[A-Za-z0-9][A-Za-z0-9_.:-]{0,63}$/;
|
||||
const SURFACE = /^[a-z][a-z0-9-]{0,31}$/;
|
||||
const DIGEST = /^[a-f0-9]{64}$/;
|
||||
const LABEL_MAX = 128;
|
||||
const LINE_MAX = 500;
|
||||
const SURFACE_MAX = 256 * 1024;
|
||||
const SURFACES_MAX = 16;
|
||||
// An env value shorter than this is too common a word to mean anything ("1", "true", "C").
|
||||
const VALUE_MIN = 6;
|
||||
const CONTROL = /[\u0000-\u0008\u000b-\u001f\u007f-\u009f\u2028\u2029]/g;
|
||||
|
||||
// An absolute path of two segments or more that no name replaced. Not one that follows a word,
|
||||
// a colon, a slash, a dot, a dash, a tilde or a name's closing ">" (URLs, ~/x, <home>/x, a/b/c).
|
||||
const LOOSE_PATH = /(?<![\w:/.~>-])\/[^\s/'"`<>()[\]{},;]+(?:\/[^\s'"`<>()[\]{},;]*)+/g;
|
||||
|
||||
// One pass. last: the records a previous pass returned (an array; others are ignored). env: the
|
||||
// environment of the process sessions launch from, which the modules look up their binary in.
|
||||
// scratch: a directory the modules may use for a scratch HOME. Returns { records, transitions,
|
||||
// refused, gone }: one record per harness the loader gives, a transition for each harness whose
|
||||
// state or digest is not the last record's (or that has no last record), the loader's refusals
|
||||
// as it reports them, and the names of last records whose harness the loader no longer gives.
|
||||
export async function watchHarnesses({ last = [], env = process.env, scratch = null, dataRoot = null, timeoutMs = INSPECT_TIMEOUT_MS, now = () => new Date() } = {}) {
|
||||
const before = new Map();
|
||||
for (const r of Array.isArray(last) ? last : []) if (r && typeof r.harness === "string") before.set(r.harness, r);
|
||||
const loaded = await loadInstalled("harness", { dataRoot });
|
||||
const redact = redactor(env, scratch);
|
||||
// The loader's refusals name the module's directory, a host path, so their text is redacted too.
|
||||
const refused = loaded.refused.map((r) => ({ ...r, message: typeof r.message === "string" ? redact(r.message) : null }));
|
||||
const modules = loaded.modules;
|
||||
const records = [];
|
||||
const transitions = [];
|
||||
for (const { name, module } of modules) {
|
||||
const answer = await ask(module, { env, scratch, timeoutMs });
|
||||
const at = now().toISOString();
|
||||
const prior = before.get(name) ?? null;
|
||||
const record = { harness: name, ...settle(clean(answer, redact, env), prior), checkedAt: at };
|
||||
record.lastDigest = record.digest ?? known(prior);
|
||||
const moved = !prior || prior.state !== record.state || !sameBinary(prior, record);
|
||||
record.changedAt = moved ? at : (typeof prior.changedAt === "string" ? prior.changedAt : at);
|
||||
records.push(record);
|
||||
if (moved) transitions.push({ harness: name, at, from: prior ? brief(prior) : null, to: brief(record), failed: record.failed, line: record.line });
|
||||
}
|
||||
const seen = new Set(records.map((r) => r.harness));
|
||||
return { records, transitions, refused, gone: [...before.keys()].filter((n) => !seen.has(n)).sort() };
|
||||
}
|
||||
|
||||
// The latest digest the last record knew: its own, or the one it carried forward.
|
||||
const known = (prior) => [prior?.digest, prior?.lastDigest].find((d) => typeof d === "string" && DIGEST.test(d)) ?? null;
|
||||
// Whether two records are of the same binary. The digest decides; when neither record has one
|
||||
// (pi gives none for a failed check), the version label stands in for it, but only when both
|
||||
// records have one: a pass that timed out or threw knows no version, which is not a new binary.
|
||||
function sameBinary(prior, record) {
|
||||
const was = typeof prior.digest === "string" ? prior.digest : null;
|
||||
if (was !== null || record.digest !== null) return was === record.digest;
|
||||
const before = typeof prior.version === "string" ? prior.version : null;
|
||||
return before === null || record.version === null || before === record.version;
|
||||
}
|
||||
|
||||
const brief = (r) => ({ state: STATES.includes(r.state) ? r.state : "unknown", version: typeof r.version === "string" ? r.version : null, digest: typeof r.digest === "string" ? r.digest : null });
|
||||
|
||||
// The module's answer, or { error } when it threw or ran past timeoutMs. via says which export.
|
||||
async function ask(module, { env, scratch, timeoutMs }) {
|
||||
const via = typeof module.inspect === "function" ? "inspect" : "version";
|
||||
const stop = new AbortController();
|
||||
let timer = null;
|
||||
const late = new Promise((resolve) => { timer = setTimeout(() => { stop.abort(); resolve({ via, error: "timeout" }); }, timeoutMs); });
|
||||
const asked = (async () => {
|
||||
try {
|
||||
// The answer is copied here, inside the try, to plain own data: a getter, a Proxy or an
|
||||
// inherited field that throws or lies is a failed inspect, not a failed pass.
|
||||
if (via === "inspect") return { via, answer: plain(await module.inspect({ env, scratch, signal: stop.signal })) };
|
||||
const version = await module.version({ env });
|
||||
return { via, version: typeof version === "string" ? version : null };
|
||||
} catch {
|
||||
// The error's text can name a host path, so the record keeps none of it.
|
||||
return { via, error: "threw" };
|
||||
}
|
||||
})();
|
||||
try {
|
||||
return await Promise.race([asked, late]);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
}
|
||||
|
||||
// The contract's fields of an answer as own data properties, copied to a plain object whose
|
||||
// values are strings, booleans, null, an array of strings or a map of strings. A getter is not
|
||||
// called (a field that is one makes the answer a failed inspect), an inherited field is not read,
|
||||
// and failed and surfaces are cut before they are read.
|
||||
const FIELDS = ["found", "source", "version", "digest", "fit", "definite", "line"];
|
||||
function plain(answer) {
|
||||
if (!answer || typeof answer !== "object") return null;
|
||||
const own = (o, k) => {
|
||||
const d = Object.getOwnPropertyDescriptor(o, k);
|
||||
if (d && !("value" in d)) throw new Error("an accessor field");
|
||||
return d ? d.value : undefined;
|
||||
};
|
||||
const out = {};
|
||||
for (const k of FIELDS) {
|
||||
const v = own(answer, k);
|
||||
if (typeof v === "string" || typeof v === "boolean") out[k] = v;
|
||||
}
|
||||
const failed = own(answer, "failed");
|
||||
if (Array.isArray(failed)) {
|
||||
out.failed = [];
|
||||
for (let i = 0; i < Math.min(failed.length, 32); i++) {
|
||||
const id = own(failed, String(i));
|
||||
if (typeof id === "string") out.failed.push(id);
|
||||
}
|
||||
}
|
||||
const surfaces = own(answer, "surfaces");
|
||||
if (surfaces && typeof surfaces === "object" && !Array.isArray(surfaces)) {
|
||||
out.surfaces = {};
|
||||
for (const k of Object.keys(surfaces).slice(0, SURFACES_MAX)) {
|
||||
const v = own(surfaces, k);
|
||||
if (typeof v === "string") out.surfaces[k] = v.slice(0, SURFACE_MAX);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// The contract's fields from an answer, each checked and redacted. Anything else is dropped. A
|
||||
// source or rule id that is an env value is dropped too: they are not text, so not redacted.
|
||||
function clean({ via, answer, version, error }, redact, env) {
|
||||
const base = { via, found: null, source: null, version: null, digest: null, fit: "unknown", definite: false, failed: [], line: null, surfaces: {} };
|
||||
if (error === "timeout") return { ...base, line: `${via} did not answer in time` };
|
||||
if (error) return { ...base, line: `${via} failed` };
|
||||
if (via === "version") {
|
||||
const label = text(version, LABEL_MAX, redact);
|
||||
return { ...base, version: label, line: label ? "no inspect; fit not checked" : "no inspect, and version gave no answer" };
|
||||
}
|
||||
if (!answer || typeof answer !== "object") return { ...base, line: "inspect gave no answer" };
|
||||
const values = envValues(env);
|
||||
const found = answer.found === true ? true : answer.found === false ? false : null;
|
||||
const fit = FITS.has(answer.fit) ? answer.fit : "unknown";
|
||||
const surfaces = {};
|
||||
if (answer.surfaces && typeof answer.surfaces === "object") {
|
||||
for (const [k, v] of Object.entries(answer.surfaces)) if (SURFACE.test(k)) surfaces[k] = redact(v);
|
||||
}
|
||||
return {
|
||||
...base,
|
||||
found,
|
||||
source: typeof answer.source === "string" && SOURCE.test(answer.source) && !values.has(answer.source) ? answer.source : null,
|
||||
version: text(answer.version, LABEL_MAX, redact),
|
||||
digest: typeof answer.digest === "string" && DIGEST.test(answer.digest) ? answer.digest : null,
|
||||
fit,
|
||||
// A pass is an answer, so it is definite. A failure is definite only when the module says so
|
||||
// (otherwise the check could not decide), and an unknown never is.
|
||||
definite: fit === "pass" || (fit === "fail" && answer.definite === true),
|
||||
failed: Array.isArray(answer.failed) ? [...new Set(answer.failed.filter((id) => RULE.test(id) && !values.has(id)))] : [],
|
||||
line: text(answer.line, LINE_MAX, redact),
|
||||
surfaces,
|
||||
};
|
||||
}
|
||||
|
||||
// The state for a cleaned answer and the last record.
|
||||
function settle(r, prior) {
|
||||
let state = "unknown";
|
||||
if (r.via === "inspect" && r.found === false) state = "missing";
|
||||
else if (r.via === "inspect" && r.found === true && r.fit === "pass") {
|
||||
const was = known(prior);
|
||||
state = was !== null && r.digest !== null && was !== r.digest ? "changed" : "fit";
|
||||
} else if (r.via === "inspect" && r.found === true && r.fit === "fail" && r.definite) state = "red";
|
||||
return { state, ...r };
|
||||
}
|
||||
|
||||
// One printable line, cut to max, redacted; or null.
|
||||
function text(v, max, redact) {
|
||||
if (typeof v !== "string") return null;
|
||||
const line = redact(v.split("\n")[0].replace(CONTROL, " ").trim()).slice(0, max);
|
||||
return line || null;
|
||||
}
|
||||
|
||||
// Every nonempty string value in env and in this process's environment.
|
||||
function envValues(env) {
|
||||
const out = new Set();
|
||||
for (const source of [env ?? {}, process.env]) for (const v of Object.values(source)) if (typeof v === "string" && v) out.add(v);
|
||||
return out;
|
||||
}
|
||||
|
||||
const escape = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
|
||||
|
||||
// Replaces every host path and env value a record could pick up with a name for it: the scratch
|
||||
// directory, the home directory, each env value of VALUE_MIN characters or more from env and this
|
||||
// process's own environment, and each absolute PATH entry, each also as its real path and without
|
||||
// a trailing slash. Longest first, so a path inside another is not left half replaced. The user
|
||||
// name (this process's account, USER and LOGNAME), short as it may be, is replaced as a whole
|
||||
// word. Last, any absolute path left is <path>.
|
||||
function redactor(env, scratch) {
|
||||
const pairs = new Map();
|
||||
const put = (value, name) => { if (value.length >= VALUE_MIN && !pairs.has(value)) pairs.set(value, name); };
|
||||
const add = (value, name) => {
|
||||
if (typeof value !== "string") return;
|
||||
put(value, name);
|
||||
if (!value.startsWith("/")) return;
|
||||
const bare = value.replace(/\/+$/, "");
|
||||
if (bare) put(bare, name);
|
||||
try { put(realpathSync(bare || "/"), name); } catch {}
|
||||
};
|
||||
if (typeof scratch === "string" && scratch) add(scratch, "<scratch>");
|
||||
for (const source of [env ?? {}, process.env]) {
|
||||
for (const [k, v] of Object.entries(source)) {
|
||||
if (typeof v !== "string") continue;
|
||||
add(v, `<${k}>`);
|
||||
if (k === "PATH") for (const dir of v.split(":")) if (isAbsolute(dir) && dir !== "/") add(dir, "<PATH entry>");
|
||||
}
|
||||
}
|
||||
try { add(homedir(), "<home>"); } catch {}
|
||||
const users = new Set();
|
||||
try { users.add(userInfo().username); } catch {}
|
||||
for (const source of [env ?? {}, process.env]) for (const k of ["USER", "LOGNAME"]) if (typeof source[k] === "string") users.add(source[k]);
|
||||
const words = [...users].filter((u) => typeof u === "string" && u.length >= 2 && /^[A-Za-z0-9._-]+$/.test(u));
|
||||
const word = words.length ? new RegExp(`(?<![A-Za-z0-9_])(?:${words.sort((a, b) => b.length - a.length).map(escape).join("|")})(?![A-Za-z0-9_])`, "g") : null;
|
||||
const order = [...pairs.keys()].sort((a, b) => b.length - a.length);
|
||||
return (s) => {
|
||||
let out = s;
|
||||
for (const value of order) if (out.includes(value)) out = out.split(value).join(pairs.get(value));
|
||||
if (word) out = out.replace(word, "<user>");
|
||||
return out.replace(LOOSE_PATH, "<path>");
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,414 @@
|
||||
// #1873 harness watch, slice W1 (reports/harness-watch-design.md sections 1 and 5): one pass over
|
||||
// every harness the loader gives, an optional inspect per module or the version fallback, a
|
||||
// state from the answer and the last record, and transitions only on a state or digest change.
|
||||
// No record holds a host path or an env value.
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { createReadStream, mkdirSync, mkdtempSync, readFileSync, readdirSync, realpathSync, rmSync, symlinkSync, watch, writeFileSync } from "node:fs";
|
||||
import { createHash } from "node:crypto";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join, resolve } from "node:path";
|
||||
import { fileURLToPath, pathToFileURL } from "node:url";
|
||||
import { tree } from "./helpers.mjs";
|
||||
import { KINDS } from "../src/kinds.mjs";
|
||||
import { piStandIn } from "../../harness/tests/pi-stand-in.mjs";
|
||||
import { sandboxProblem } from "../../seat/src/sandbox.mjs";
|
||||
import * as pi from "../harness/pi/index.mjs";
|
||||
import * as claude from "../harness/claude-code/index.mjs";
|
||||
|
||||
const REPO = resolve(fileURLToPath(new URL("../../../", import.meta.url)));
|
||||
const WATCH_SRC = join(REPO, "packages", "plugins", "src", "harness-watch.mjs");
|
||||
const PI_NAME = "@earendil-works/pi-coding-agent";
|
||||
const NO_BWRAP = sandboxProblem() ? { skip: "no working bwrap here" } : {};
|
||||
const DIGEST_A = "a".repeat(64);
|
||||
const DIGEST_B = "b".repeat(64);
|
||||
|
||||
function scratch(t) {
|
||||
const root = mkdtempSync(join(tmpdir(), "mosaic-watch-"));
|
||||
t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
return root;
|
||||
}
|
||||
|
||||
// A stand-in harness module: every export the kind needs, a version() that answers
|
||||
// env.WATCH_VERSION, and (with inspect) an inspect() that answers its own entry of the JSON file
|
||||
// env.WATCH_ANSWER names, read at each call: "throw" throws, "hang" never answers, "hang-signal"
|
||||
// never answers but writes <answer file>.aborted when its signal aborts, and the rest build an
|
||||
// answer JSON cannot carry (a throwing getter, a Proxy, inherited fields, a huge sparse array).
|
||||
const standIn = (name, { inspect = true } = {}) => `import { readFileSync, writeFileSync } from "node:fs";
|
||||
export const interfaceVersion = ${KINDS.harness.interfaceVersion};
|
||||
${KINDS.harness.exports.filter((e) => e !== "version").map((e) => `export function ${e}() { return null; }`).join("\n")}
|
||||
export async function version({ env }) {
|
||||
if (env.WATCH_VERSION === "throw") throw new Error("version failed at " + env.WATCH_SECRET);
|
||||
return env.WATCH_VERSION ?? null;
|
||||
}
|
||||
${inspect ? `export async function inspect({ env, signal }) {
|
||||
const a = JSON.parse(readFileSync(env.WATCH_ANSWER, "utf8"))[${JSON.stringify(name)}];
|
||||
const boom = () => { throw new Error("inspect failed at " + env.WATCH_SECRET); };
|
||||
if (a === "throw") boom();
|
||||
if (a === "hang") return new Promise(() => {});
|
||||
if (a === "hang-signal") return new Promise(() => { signal.addEventListener("abort", () => writeFileSync(env.WATCH_ANSWER + ".aborted", "1")); });
|
||||
if (a === "getter") return { found: true, fit: "pass", definite: true, get digest() { return boom(); } };
|
||||
if (a === "proxy") return new Proxy({}, { get: boom, getOwnPropertyDescriptor: boom, ownKeys: boom });
|
||||
if (a === "surfaces-proxy") return { found: true, fit: "pass", definite: true, digest: "${DIGEST_A}", surfaces: new Proxy({}, { ownKeys: boom }) };
|
||||
if (a === "inherited") return Object.create({ found: true, fit: "pass", definite: true, digest: "${DIGEST_A}" });
|
||||
if (a === "sparse") { const failed = []; failed.length = 2 ** 32 - 1; failed[0] = "C1"; return { found: true, fit: "fail", definite: true, failed, line: "sparse" }; }
|
||||
return a;
|
||||
}` : ""}
|
||||
`;
|
||||
|
||||
// A scratch plugin tree whose harness directory holds only the named stand-ins, and its watch.
|
||||
async function watchTree(t, harnesses) {
|
||||
const tr = tree(t);
|
||||
for (const [name, opts] of Object.entries(harnesses)) tr.write("harness", name, standIn(name, opts));
|
||||
const { watchHarnesses } = await import(pathToFileURL(join(tr.pkg, "src", "harness-watch.mjs")).href);
|
||||
const dir = scratch(t);
|
||||
const file = join(dir, "answer.json");
|
||||
const pass = async (answers, { last = [], env = {}, timeoutMs = 2000 } = {}) => {
|
||||
writeFileSync(file, JSON.stringify(answers));
|
||||
return watchHarnesses({ last, env: { WATCH_ANSWER: file, ...env }, scratch: dir, timeoutMs });
|
||||
};
|
||||
return { pass, dir, tr, file };
|
||||
}
|
||||
|
||||
const ok = (digest, extra = {}) => ({ found: true, source: "PATH", version: "1.0.0", digest, fit: "pass", definite: true, failed: [], line: "passed", surfaces: {}, ...extra });
|
||||
const byName = (out) => Object.fromEntries(out.records.map((r) => [r.harness, r]));
|
||||
|
||||
test("W1 discovery: every harness the loader gives is watched, with no name list in the watch", async (t) => {
|
||||
// Names made for this run, so nothing in the watch could have listed them.
|
||||
const names = [`w${process.pid}-one`, `w${process.pid}-two`, `w${process.pid}-three`].sort();
|
||||
const { pass } = await watchTree(t, { [names[0]]: {}, [names[1]]: { inspect: false }, [names[2]]: {} });
|
||||
const out = await pass({ [names[0]]: ok(DIGEST_A), [names[2]]: ok(DIGEST_B) }, { env: { WATCH_VERSION: "3.1.4" } });
|
||||
assert.deepEqual(out.records.map((r) => r.harness).sort(), names);
|
||||
assert.deepEqual(out.refused, []);
|
||||
const r = byName(out);
|
||||
assert.equal(r[names[0]].via, "inspect");
|
||||
assert.equal(r[names[1]].via, "version");
|
||||
assert.equal(r[names[2]].digest, DIGEST_B);
|
||||
// The watch's source names no harness: no quoted name and no bare word for a shipped one.
|
||||
const src = readFileSync(WATCH_SRC, "utf8").replace(/^\s*\/\/.*$/gm, "");
|
||||
assert.doesNotMatch(src, /["'`](?:pi|claude|claude-code|codex)["'`]/);
|
||||
assert.doesNotMatch(src, /\b(?:claude|codex)\b|\bpi\b/i);
|
||||
});
|
||||
|
||||
test("W1 the version fallback: a harness with no inspect is watched through version() and is unknown", async (t) => {
|
||||
const { pass } = await watchTree(t, { plain: { inspect: false } });
|
||||
let rec = byName(await pass({}, { env: { WATCH_VERSION: "2.0.0" } })).plain;
|
||||
assert.deepEqual([rec.state, rec.via, rec.version, rec.fit, rec.found, rec.digest], ["unknown", "version", "2.0.0", "unknown", null, null]);
|
||||
rec = byName(await pass({}, {})).plain;
|
||||
assert.deepEqual([rec.state, rec.version], ["unknown", null]);
|
||||
rec = byName(await pass({}, { env: { WATCH_VERSION: "throw", WATCH_SECRET: "/srv/hidden-value" } })).plain;
|
||||
assert.deepEqual([rec.state, rec.line], ["unknown", "version failed"]);
|
||||
});
|
||||
|
||||
test("W1 each state from a pair of records: same digest, new digest, fail, undecided, throw, timeout, not found", async (t) => {
|
||||
const { pass } = await watchTree(t, { h: {} });
|
||||
const first = await pass({ h: ok(DIGEST_A) });
|
||||
const prior = first.records;
|
||||
assert.equal(byName(first).h.state, "fit");
|
||||
const cases = [
|
||||
["same digest, fit pass", ok(DIGEST_A), "fit"],
|
||||
["new digest, fit pass", ok(DIGEST_B), "changed"],
|
||||
["fit fail, definite", ok(DIGEST_B, { fit: "fail", definite: true, failed: ["C5"], line: "lacks --no-mcp" }), "red"],
|
||||
["fit fail, not definite", ok(DIGEST_B, { fit: "fail", definite: false }), "unknown"],
|
||||
["fit unknown", ok(DIGEST_B, { fit: "unknown" }), "unknown"],
|
||||
["inspect throws", "throw", "unknown"],
|
||||
["inspect times out", "hang", "unknown"],
|
||||
["binary not found", { found: false, source: null, version: null, digest: null, fit: "unknown", definite: false, failed: [], line: "no binary", surfaces: {} }, "missing"],
|
||||
// Answers that are not the contract's never read as fit.
|
||||
["fit spelled otherwise", ok(DIGEST_A, { fit: "PASS" }), "unknown"],
|
||||
["found not a boolean", ok(DIGEST_A, { found: "yes" }), "unknown"],
|
||||
["no answer", null, "unknown"],
|
||||
["a pass with found false", ok(DIGEST_A, { found: false }), "missing"],
|
||||
];
|
||||
for (const [what, answer, state] of cases) {
|
||||
const out = await pass({ h: answer }, { last: prior, timeoutMs: 300 });
|
||||
assert.equal(byName(out).h.state, state, what);
|
||||
}
|
||||
const red = byName(await pass({ h: ok(DIGEST_B, { fit: "fail", definite: true, failed: ["C5", "bad id with spaces"], line: "lacks --no-mcp" }) }, { last: prior })).h;
|
||||
assert.deepEqual([red.failed, red.line, red.definite], [["C5"], "lacks --no-mcp", true]);
|
||||
const late = byName(await pass({ h: "hang" }, { last: prior, timeoutMs: 300 })).h;
|
||||
assert.equal(late.line, "inspect did not answer in time");
|
||||
});
|
||||
|
||||
test("W1 a transition is reported only when the state or the digest changes", async (t) => {
|
||||
const { pass } = await watchTree(t, { h: {}, k: {} });
|
||||
const one = await pass({ h: ok(DIGEST_A), k: ok(DIGEST_A) });
|
||||
assert.deepEqual(one.transitions.map((x) => [x.harness, x.from, x.to.state]), [["h", null, "fit"], ["k", null, "fit"]]);
|
||||
// Same answers: no transition, and changedAt stays the first pass's.
|
||||
const two = await pass({ h: ok(DIGEST_A), k: ok(DIGEST_A, { version: "1.0.1", line: "another line" }) }, { last: one.records });
|
||||
assert.deepEqual(two.transitions, []);
|
||||
assert.equal(byName(two).h.changedAt, byName(one).h.changedAt);
|
||||
// A state change is a transition, with or without a new digest.
|
||||
const three = await pass({ h: ok(DIGEST_B), k: ok(DIGEST_A, { fit: "fail", definite: true }) }, { last: two.records });
|
||||
assert.deepEqual(three.transitions.map((x) => [x.harness, x.from.state, x.to.state, x.from.digest, x.to.digest]),
|
||||
[["h", "fit", "changed", DIGEST_A, DIGEST_B], ["k", "fit", "red", DIGEST_A, DIGEST_A]]);
|
||||
const four = await pass({ h: ok(DIGEST_B, { fit: "unknown" }), k: ok(DIGEST_A, { fit: "fail", definite: true }) }, { last: three.records });
|
||||
// A new digest with the state unchanged (unknown to unknown) is a transition too.
|
||||
const five = await pass({ h: ok(DIGEST_A, { fit: "unknown" }), k: ok(DIGEST_A, { fit: "fail", definite: true }) }, { last: four.records });
|
||||
assert.deepEqual(five.transitions.map((x) => [x.harness, x.from.state, x.to.state]), [["h", "unknown", "unknown"]]);
|
||||
// A harness the loader no longer gives is named in gone, and has no record.
|
||||
const six = await pass({ h: ok(DIGEST_A) }, { last: [...five.records, { harness: "removed", state: "fit", digest: DIGEST_A }] });
|
||||
assert.deepEqual(six.gone, ["removed"]);
|
||||
});
|
||||
|
||||
test("W1 a new binary first seen through an unknown, red or missing record still reads changed", async (t) => {
|
||||
const { pass } = await watchTree(t, { h: {} });
|
||||
const none = { found: true, source: "PATH", version: "2.0.0", digest: null, fit: "unknown", definite: false, failed: [], line: "timed out", surfaces: {} };
|
||||
for (const [what, between] of [["unknown", none], ["red", { ...none, fit: "fail", definite: true }], ["missing", { ...none, found: false }], ["timeout", "hang"], ["throw", "throw"]]) {
|
||||
const one = await pass({ h: ok(DIGEST_A) });
|
||||
const two = await pass({ h: between }, { last: one.records, timeoutMs: 200 });
|
||||
assert.equal(byName(two).h.lastDigest, DIGEST_A, what);
|
||||
const three = await pass({ h: ok(DIGEST_B) }, { last: two.records });
|
||||
assert.equal(byName(three).h.state, "changed", what);
|
||||
const four = await pass({ h: ok(DIGEST_B) }, { last: three.records });
|
||||
assert.equal(byName(four).h.state, "fit", what);
|
||||
}
|
||||
// With no digest on either record, a new version label is a new binary: red 2.0 to red 2.1 is a
|
||||
// transition, red 2.1 again is not.
|
||||
const red = (version, line) => ({ ...none, version, line, fit: "fail", definite: true });
|
||||
const a = await pass({ h: red("2.0.0", "lacks --no-skills") });
|
||||
const b = await pass({ h: red("2.1.0", "lacks --no-mcp") }, { last: a.records });
|
||||
assert.deepEqual(b.transitions.map((x) => [x.from.version, x.to.version, x.line]), [["2.0.0", "2.1.0", "lacks --no-mcp"]]);
|
||||
const c = await pass({ h: red("2.1.0", "lacks --no-mcp") }, { last: b.records });
|
||||
assert.deepEqual(c.transitions, []);
|
||||
// A pass that knows no version (a timeout or a throw) is not a new binary: unknown with 2.0,
|
||||
// then a timeout, then unknown with 2.0 again, then a throw gives no transition after the first.
|
||||
const d = await pass({ h: none }, { last: [] });
|
||||
const e = await pass({ h: "hang" }, { last: d.records, timeoutMs: 200 });
|
||||
const f = await pass({ h: { ...none, version: "2.0.0" } }, { last: e.records });
|
||||
const g = await pass({ h: "throw" }, { last: f.records });
|
||||
assert.deepEqual([d, e, f, g].map((x) => x.transitions.length), [1, 0, 0, 0]);
|
||||
assert.deepEqual([e, g].map((x) => byName(x).h.version), [null, null]);
|
||||
});
|
||||
|
||||
test("W1 an answer that throws when read, a Proxy or inherited fields is unknown, and the other harnesses still get records", async (t) => {
|
||||
const { pass, file } = await watchTree(t, { bad: {}, good: {} });
|
||||
for (const shape of ["getter", "proxy", "surfaces-proxy"]) {
|
||||
const out = await pass({ bad: shape, good: ok(DIGEST_A) }, { env: { WATCH_SECRET: "/srv/fixture-secret-place" } });
|
||||
const r = byName(out);
|
||||
assert.deepEqual([r.bad.state, r.bad.line, r.good.state], ["unknown", "inspect failed", "fit"], shape);
|
||||
assert.ok(!JSON.stringify(out).includes("fixture-secret"), shape);
|
||||
}
|
||||
// An inherited fit is not the module's answer.
|
||||
assert.equal(byName(await pass({ bad: "inherited", good: ok(DIGEST_A) })).bad.state, "unknown");
|
||||
// A huge sparse failed list is cut before it is read, so the pass ends at once.
|
||||
const began = Date.now();
|
||||
const sparse = byName(await pass({ bad: "sparse", good: ok(DIGEST_A) })).bad;
|
||||
assert.deepEqual([sparse.state, sparse.failed], ["red", ["C1"]]);
|
||||
assert.ok(Date.now() - began < 2000);
|
||||
// On a timeout the watch aborts the signal it passed.
|
||||
const late = byName(await pass({ bad: "hang-signal", good: ok(DIGEST_A) }, { timeoutMs: 200 })).bad;
|
||||
await new Promise((done) => setTimeout(done, 50));
|
||||
assert.equal(late.state, "unknown");
|
||||
assert.equal(readFileSync(file + ".aborted", "utf8"), "1");
|
||||
});
|
||||
|
||||
test("W1 fields are checked: a digest that is a path is dropped, version and line keep one printable line", async (t) => {
|
||||
const { pass, dir } = await watchTree(t, { h: {} });
|
||||
const r = byName(await pass({ h: ok(dir, { version: "1.0\u0007beta\nsecond line", line: "first\r\nsecond", surfaces: { help: "a\nb" } }) })).h;
|
||||
assert.deepEqual([r.digest, r.version, r.line, r.surfaces.help], [null, "1.0 beta", "first", "a\nb"]);
|
||||
});
|
||||
|
||||
test("W1 redaction covers real paths, trailing slashes, the user name, rule ids, sources, loose paths and the loader's refusals", async (t) => {
|
||||
const { pass, dir, tr } = await watchTree(t, { h: {} });
|
||||
mkdirSync(join(dir, "real", "home"), { recursive: true });
|
||||
mkdirSync(join(dir, "real", "bin"), { recursive: true });
|
||||
symlinkSync(join(dir, "real", "home"), join(dir, "linkhome"));
|
||||
symlinkSync(join(dir, "real", "bin"), join(dir, "linkbin"));
|
||||
const realHome = realpathSync(join(dir, "linkhome"));
|
||||
const realBin = realpathSync(join(dir, "linkbin"));
|
||||
// A symlinked scratch: the answer names its real path.
|
||||
const linkScratch = join(dir, "linkscratch");
|
||||
symlinkSync(dir, linkScratch);
|
||||
const env = { HOME: join(dir, "linkhome") + "/", PATH: `${join(dir, "linkbin")}:/usr/bin:/bin`, USER: "jq", WATCH_TOKEN: "sk-AbCdEf123456", WATCH_NAME: "fixturesource" };
|
||||
// Each rule on its own line, with what the line must read after. scratch: the one passed.
|
||||
const lines = {
|
||||
scratch: [`in ${dir}/x`, "in <scratch>/x", linkScratch],
|
||||
home: [`home ${realHome}`, "home <HOME>", dir],
|
||||
slash: [`home ${join(dir, "linkhome")}`, "home <HOME>", dir],
|
||||
bin: [`bin ${realBin}/claude`, "bin <PATH entry>/claude", dir],
|
||||
user: ["built by jq on host", "built by <user> on host", dir],
|
||||
loose: ["at /opt/notinenv/bin/tool and /etc/passwd", "at <path> and <path>", dir],
|
||||
kept: ["see https://example.com/a/b, ~/x/y and /model", "see https://example.com/a/b, ~/x/y and /model", dir],
|
||||
};
|
||||
const { watchHarnesses } = await import(pathToFileURL(join(tr.pkg, "src", "harness-watch.mjs")).href);
|
||||
for (const [what, [line, want, at]] of Object.entries(lines)) {
|
||||
const file = join(dir, `answer-${what}.json`);
|
||||
writeFileSync(file, JSON.stringify({ h: ok(DIGEST_A, { line }) }));
|
||||
const out = await watchHarnesses({ env: { ...env, WATCH_ANSWER: file }, scratch: at });
|
||||
assert.equal(byName(out).h.line, want, what);
|
||||
}
|
||||
const rec = byName(await pass({ h: ok(DIGEST_A, { source: "fixturesource", failed: ["sk-AbCdEf123456", "C5"], line: `in ${realpathSync(dir)}` }) }, { env })).h;
|
||||
assert.deepEqual([rec.source, rec.failed], [null, ["C5"]]);
|
||||
assert.equal(rec.line, "in <scratch>");
|
||||
// The loader's refusal of a harness directory with no index.mjs names its path; the watch's copy doesn't.
|
||||
mkdirSync(join(tr.pkg, "harness", "noindex"));
|
||||
const out = await pass({ h: ok(DIGEST_A) });
|
||||
const refusal = out.refused.find((r) => r.name === "noindex");
|
||||
assert.ok(refusal && typeof refusal.message === "string", JSON.stringify(out.refused));
|
||||
assert.ok(!refusal.message.includes(tr.pkg), refusal.message);
|
||||
});
|
||||
|
||||
test("W1 no record holds a path under the scratch directory, the home directory or an env value", async (t) => {
|
||||
const { pass, dir } = await watchTree(t, { h: {} });
|
||||
const secret = "/srv/fixture-secret-place";
|
||||
const env = { WATCH_SECRET: secret, HOME: join(dir, "home"), MOSAIC_FIXTURE_BIN: "/opt/fixture/bin/tool", PATH: "/opt/fixture-tools/bin:/usr/bin:/bin" };
|
||||
const answer = ok(DIGEST_A, {
|
||||
source: dir, version: `9.9 at ${dir}/bin`,
|
||||
line: `found at ${env.MOSAIC_FIXTURE_BIN} in ${secret}, home ${env.HOME}, ${realpathSync(dir)}`,
|
||||
surfaces: { help: `usage: ${dir}/tool\nsee /opt/fixture-tools/bin/tool and ${tmpdir()}`, "Bad Name": dir },
|
||||
});
|
||||
const out = await pass({ h: answer }, { env });
|
||||
const text = JSON.stringify(out);
|
||||
for (const p of [dir, realpathSync(dir), secret, env.MOSAIC_FIXTURE_BIN, "/opt/fixture-tools/bin", env.HOME, tmpdir()]) assert.ok(!text.includes(p), p);
|
||||
const rec = byName(out).h;
|
||||
assert.equal(rec.source, null, "a source that is a path is dropped");
|
||||
assert.deepEqual(Object.keys(rec.surfaces), ["help"]);
|
||||
assert.match(rec.line, /<MOSAIC_FIXTURE_BIN>/);
|
||||
assert.match(rec.surfaces.help, /<scratch>\/tool/);
|
||||
});
|
||||
|
||||
test("W1 real pi from node_modules gives a version, a digest and fit pass", NO_BWRAP, async (t) => {
|
||||
const dir = scratch(t);
|
||||
const env = { PATH: "/usr/bin:/bin" };
|
||||
const r = await pi.inspect({ env, scratch: dir });
|
||||
const pkg = JSON.parse(readFileSync(join(REPO, "node_modules", PI_NAME, "package.json"), "utf8"));
|
||||
assert.equal(r.fit, "pass", r.line);
|
||||
assert.deepEqual([r.found, r.source, r.version, r.definite], [true, "repo", pkg.version, true]);
|
||||
assert.match(r.digest, /^[a-f0-9]{64}$/);
|
||||
const { watchHarnesses } = await import(pathToFileURL(WATCH_SRC).href);
|
||||
const out = await watchHarnesses({ env, scratch: dir });
|
||||
const rec = out.records.find((x) => x.via === "inspect" && x.source === "repo");
|
||||
assert.deepEqual([rec.state, rec.digest, rec.version], ["fit", r.digest, pkg.version]);
|
||||
const text = JSON.stringify(out);
|
||||
for (const p of [dir, REPO, realpathSync(join(REPO, "node_modules"))]) assert.ok(!text.includes(p), p);
|
||||
});
|
||||
|
||||
// A stand-in pi package named by MOSAIC_PI_BIN, as pi-fit.test.mjs makes one.
|
||||
function piPackage(t, knobs) {
|
||||
const root = scratch(t);
|
||||
const pkg = join(root, "prefix", "lib", "node_modules", PI_NAME);
|
||||
mkdirSync(join(pkg, "dist", "bundle"), { recursive: true });
|
||||
writeFileSync(join(pkg, "package.json"), JSON.stringify({ name: PI_NAME, version: "2.0.0", bin: { pi: "dist/bundle/cli.js" } }));
|
||||
writeFileSync(join(pkg, "dist", "bundle", "cli.js"), piStandIn({ version: "2.0.0", ...knobs }), { mode: 0o755 });
|
||||
mkdirSync(join(root, "prefix", "bin"));
|
||||
symlinkSync(`../lib/node_modules/${PI_NAME}/dist/bundle/cli.js`, join(root, "prefix", "bin", "pi"));
|
||||
return { root, env: { MOSAIC_PI_BIN: join(root, "prefix", "bin", "pi"), PATH: "/usr/bin:/bin" } };
|
||||
}
|
||||
|
||||
test("W1 pi inspect shares preflight's fit check: a binary that passed is not checked again", NO_BWRAP, async (t) => {
|
||||
const s = piPackage(t, {});
|
||||
// A fit check makes its scratch directories in TMPDIR. Control: preflight's check is seen making
|
||||
// them. Then inspect, on the binary that passed, must make none.
|
||||
const made = [];
|
||||
const watcher = watch(tmpdir(), (_, name) => { if (String(name).startsWith("mosaic-pi-fit-")) made.push(name); });
|
||||
t.after(() => watcher.close());
|
||||
const settle = () => new Promise((done) => setTimeout(done, 200));
|
||||
assert.equal(await pi.preflight({ env: { ...s.env, ZAI_API_KEY: "fixture-zai" } }), null);
|
||||
await settle();
|
||||
assert.ok(made.length > 0, "the watcher sees a fit check run");
|
||||
made.length = 0;
|
||||
const r = await pi.inspect({ env: s.env });
|
||||
await settle();
|
||||
assert.deepEqual(made, []);
|
||||
assert.deepEqual([r.fit, r.source, r.version], ["pass", "MOSAIC_PI_BIN", "2.0.0"]);
|
||||
assert.ok(!JSON.stringify(r).includes(s.root));
|
||||
});
|
||||
|
||||
test("W1 pi inspect: a definite failure is fail, one that can't be checked is unknown, no binary is not found", NO_BWRAP, async (t) => {
|
||||
const lacking = piPackage(t, { dropFlags: ["--no-skills"] });
|
||||
let r = await pi.inspect({ env: lacking.env });
|
||||
assert.deepEqual([r.found, r.fit, r.definite, r.digest, r.version, r.line], [true, "fail", true, null, "2.0.0", "pi 2.0.0 (MOSAIC_PI_BIN) lacks --no-skills"]);
|
||||
const flood = piPackage(t, { flood: true });
|
||||
r = await pi.inspect({ env: flood.env });
|
||||
assert.deepEqual([r.fit, r.definite], ["unknown", false], r.line);
|
||||
assert.match(r.line, /can't be checked/);
|
||||
r = await pi.inspect({ env: { MOSAIC_PI_BIN: "relative/pi", PATH: "/usr/bin:/bin" } });
|
||||
assert.deepEqual([r.found, r.source, r.line], [false, "MOSAIC_PI_BIN", "MOSAIC_PI_BIN isn't an executable file at an absolute path"]);
|
||||
});
|
||||
|
||||
const sha256 = (file) => new Promise((done, fail) => {
|
||||
const h = createHash("sha256");
|
||||
createReadStream(file).on("error", fail).on("data", (c) => h.update(c)).on("end", () => done(h.digest("hex")));
|
||||
});
|
||||
|
||||
const CLAUDE = process.env.MOSAIC_CLAUDE_BIN;
|
||||
test("W1 the real claude binary named by MOSAIC_CLAUDE_BIN gives a version, a sha256 and its help", CLAUDE ? {} : { skip: "MOSAIC_CLAUDE_BIN is not set" }, async (t) => {
|
||||
const dir = scratch(t);
|
||||
const env = { MOSAIC_CLAUDE_BIN: CLAUDE, PATH: "/usr/bin:/bin" };
|
||||
const r = await claude.inspect({ env, scratch: dir });
|
||||
assert.deepEqual([r.found, r.source, r.fit, r.definite], [true, "MOSAIC_CLAUDE_BIN", "unknown", false]);
|
||||
assert.match(r.version, /\d+\.\d+\.\d+/);
|
||||
assert.equal(r.digest, await sha256(realpathSync(CLAUDE)));
|
||||
assert.match(r.surfaces.help, /--version/);
|
||||
const { watchHarnesses } = await import(pathToFileURL(WATCH_SRC).href);
|
||||
const out = await watchHarnesses({ env, scratch: dir });
|
||||
const rec = out.records.find((x) => x.source === "MOSAIC_CLAUDE_BIN");
|
||||
assert.deepEqual([rec.state, rec.digest, rec.version], ["unknown", r.digest, r.version]);
|
||||
assert.ok(!JSON.stringify(out).includes(CLAUDE));
|
||||
// The scratch HOME it ran in is gone.
|
||||
assert.deepEqual(readdirNames(dir), []);
|
||||
});
|
||||
|
||||
test("W1 claude-code inspect: no binary is not found, naming the variable; no scratch runs nothing", async (t) => {
|
||||
let r = await claude.inspect({ env: { MOSAIC_CLAUDE_BIN: "relative/claude" }, scratch: scratch(t) });
|
||||
assert.deepEqual([r.found, r.source, r.line], [false, "MOSAIC_CLAUDE_BIN", "MOSAIC_CLAUDE_BIN isn't an executable file at an absolute path"]);
|
||||
r = await claude.inspect({ env: { PATH: scratch(t) }, scratch: scratch(t) });
|
||||
assert.deepEqual([r.found, r.source], [false, null]);
|
||||
// A relative PATH entry is skipped, as pi's lookup skips it, though "bin" from this cwd holds a claude.
|
||||
const rel = scratch(t);
|
||||
mkdirSync(join(rel, "bin"));
|
||||
writeFileSync(join(rel, "bin", "claude"), "#!/bin/sh\necho 1.0.0\n", { mode: 0o755 });
|
||||
const cwd = process.cwd();
|
||||
process.chdir(rel);
|
||||
try {
|
||||
r = await claude.inspect({ env: { PATH: "bin" }, scratch: scratch(t) });
|
||||
} finally {
|
||||
process.chdir(cwd);
|
||||
}
|
||||
assert.deepEqual([r.found, r.source], [false, null]);
|
||||
const dir = scratch(t);
|
||||
mkdirSync(join(dir, "bin"));
|
||||
writeFileSync(join(dir, "bin", "claude"), "#!/bin/sh\necho ran >> \"$0.log\"\necho 1.2.3\n", { mode: 0o755 });
|
||||
r = await claude.inspect({ env: { PATH: join(dir, "bin") } });
|
||||
assert.deepEqual([r.found, r.source, r.version, r.surfaces], [true, "PATH", null, {}]);
|
||||
assert.equal(r.digest, await sha256(join(dir, "bin", "claude")));
|
||||
assert.deepEqual(readdirNames(join(dir, "bin")), ["claude"], "nothing ran without scratch");
|
||||
});
|
||||
|
||||
test("W1 claude-code inspect runs --version and --help with PATH and a scratch HOME only, then removes it", async (t) => {
|
||||
const dir = scratch(t);
|
||||
const work = scratch(t);
|
||||
mkdirSync(join(dir, "bin"));
|
||||
const bin = join(dir, "bin", "claude");
|
||||
writeFileSync(bin, "#!/bin/sh\nenv > \"$0.env$1\"\n[ \"$1\" = --version ] && echo '1.2.3 (stand-in)'\n[ \"$1\" = --help ] && echo 'usage: claude --version'\nexit 0\n", { mode: 0o755 });
|
||||
const r = await claude.inspect({ env: { MOSAIC_CLAUDE_BIN: bin, PATH: "/usr/bin:/bin", FIXTURE_TOKEN: "fixture-token-value" }, scratch: work });
|
||||
assert.deepEqual([r.found, r.source, r.version, r.surfaces.help], [true, "MOSAIC_CLAUDE_BIN", "1.2.3 (stand-in)", "usage: claude --version\n"]);
|
||||
for (const arg of ["--version", "--help"]) {
|
||||
const vars = Object.fromEntries(readFileSync(`${bin}.env${arg}`, "utf8").trim().split("\n").map((l) => [l.slice(0, l.indexOf("=")), l.slice(l.indexOf("=") + 1)]));
|
||||
assert.deepEqual(Object.keys(vars).filter((k) => !["PWD", "SHLVL", "_", "OLDPWD"].includes(k)).sort(), ["HOME", "PATH"], arg);
|
||||
assert.equal(vars.PATH, "/usr/bin:/bin");
|
||||
assert.ok(vars.HOME.startsWith(join(realpathSync(work), "claude-home-")), arg);
|
||||
}
|
||||
assert.deepEqual(readdirNames(work), [], "the scratch HOME is removed");
|
||||
});
|
||||
|
||||
test("W1 claude-code inspect: an aborted signal kills the capture it is running", async (t) => {
|
||||
const dir = scratch(t);
|
||||
mkdirSync(join(dir, "bin"));
|
||||
const bin = join(dir, "bin", "claude");
|
||||
writeFileSync(bin, "#!/bin/sh\necho $$ > \"$0.pid$1\"\nexec sleep 30\n", { mode: 0o755 });
|
||||
const stop = new AbortController();
|
||||
setTimeout(() => stop.abort(), 300);
|
||||
const began = Date.now();
|
||||
const r = await claude.inspect({ env: { MOSAIC_CLAUDE_BIN: bin }, scratch: scratch(t), signal: stop.signal });
|
||||
assert.ok(Date.now() - began < 5000);
|
||||
assert.deepEqual([r.found, r.version, r.surfaces], [true, null, {}]);
|
||||
const pid = Number(readFileSync(`${bin}.pid--version`, "utf8"));
|
||||
await new Promise((done) => setTimeout(done, 200));
|
||||
assert.throws(() => process.kill(pid, 0), { code: "ESRCH" });
|
||||
});
|
||||
|
||||
function readdirNames(dir) {
|
||||
return readdirSync(dir).sort();
|
||||
}
|
||||
Reference in new issue
Block a user