Compare commits
25
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
57af33630d | ||
|
|
17b972b92d | ||
|
|
d531d7181d | ||
|
|
67e1ff193d | ||
|
|
9a3004cba4 | ||
|
|
4b833d23e9 | ||
|
|
05be6c62bd | ||
|
|
c9fad00d25 | ||
|
|
4c8527fcd3 | ||
|
|
2d44750b96 | ||
|
|
9343baffff | ||
|
|
38ea639d70 | ||
|
|
fc558df31a | ||
|
|
56cc7b59dd | ||
|
|
c641dbfbc9 | ||
|
|
eb1146ebad | ||
|
|
4b5d027361 | ||
|
|
80ce06851d | ||
|
|
b39d5e2ff3 | ||
|
|
c85c768a4a | ||
|
|
b7649ecd73 | ||
|
|
0354452971 | ||
|
|
bbbefb3f28 | ||
|
|
2f91ff5bea | ||
|
|
1ca3a38f86 |
No files matched your search
@@ -1425,3 +1425,100 @@ installer writes `<dataRoot>/.deployed` as mode 0444 naming the instance.
|
||||
`webui-service.sh` refuses accounts and deployed instances because its user
|
||||
manager runs under a human or development seat uid. The receipt uid identifies
|
||||
the invoking process; it does not attribute a human or resist tampering by that uid.
|
||||
|
||||
## Triage route (#1873, held for Jason's ruling)
|
||||
|
||||
`mosaic triage route show` prints the Triage route the instance owner set, or
|
||||
`triage route: not set` (exit 0). `mosaic triage route set --model KEY
|
||||
--harness NAME --cap TOKENS --time-zone ZONE [--revision N]` writes it. Both
|
||||
refuse inside an agent run before anything is read (exit 3).
|
||||
|
||||
The file is `<dataRoot>/triage/route.json`
|
||||
(packages/webui/src/triage-route-store.mjs): a catalogue model key, a harness
|
||||
name and a daily cap leaf `{limit, since, reset, timeZone}`. It holds no env
|
||||
value and no key. The day resets at 00:00 in the zone given, which has no
|
||||
default; an unknown zone refuses. The first cap counts from the time it is
|
||||
set; a later change keeps that `since`, as an organization's token limits do,
|
||||
so lowering the cap never drops tokens already spent. The directory is 0700 and the file 0600,
|
||||
owned by the running user; a link, a non-file, a wrong mode or owner, or an
|
||||
unknown field refuses (exit 3). A write is compare-and-set on `revision`
|
||||
(`--revision 0`: only when nothing is set), atomic, and appends one line with
|
||||
the old and new values to `route-receipts.jsonl`. A write that died leaves
|
||||
`route.lock`; the next write refuses and names it, and you remove it by hand.
|
||||
If the route changed but its receipt line could not be written, the write
|
||||
refuses with `triage-route-unrecorded` (exit 3) and names the new revision.
|
||||
`set` checks the harness against packages/plugins/harness/; it does not check
|
||||
the model key against a catalogue yet.
|
||||
|
||||
The Console does not read this file: the Triage host keeps `route: null` until
|
||||
the wiring slice.
|
||||
|
||||
## Triage key (#1873)
|
||||
|
||||
`mosaic triage key issue|rotate|retire|status` manage Triage's own model
|
||||
proxy client key. Every verb refuses inside an agent run before anything is
|
||||
read (exit 3), and takes no options.
|
||||
|
||||
The key is `mst-<tag>-triage-<43 base64url>`, where `<tag>` is the first 12
|
||||
hex of sha256 of the config directory, the tag on the instance's own `msk-`
|
||||
sentinel. An org bus reconciles only its `mso-<tag>-<org>-key-` names and a
|
||||
no-org bus only its `msk-<tag>-` names, so neither removes it. The file is
|
||||
`<configDir>/secrets/triage/model`: `{version, key, keyId, issuedAt,
|
||||
rotateBy, retiring}`. The directory is 0700 and the file 0600, owned by the
|
||||
running user. A link, a non-file, a second hard link, a wrong mode or owner, a
|
||||
file over 64 KiB, or a record that does not match the shape refuses every verb
|
||||
with `triage-key-unreadable` (exit 3).
|
||||
|
||||
- `issue` adds one key and writes the file. It refuses `triage-key-issued`
|
||||
when a key is already issued; that is `rotate`.
|
||||
- `rotate` adds a new key and moves the old one to `retiring`, with a retire
|
||||
time of now plus `rotateGraceMinutes` from proxy.json, where the bus reads
|
||||
it (default 120). `rotateBy` is 30 days on, as for an org key. An old key
|
||||
the proxy no longer holds is dropped at once.
|
||||
- `retire` removes retiring keys whose grace has passed or that the proxy no
|
||||
longer holds. It changes nothing before then.
|
||||
|
||||
Nothing runs on a timer. The retire time is the earliest a retiring key may
|
||||
go; the key stays live in the proxy until someone runs `retire` (or a later
|
||||
`rotate` or `issue`) after it. `rotateBy` works the same way: `status` marks
|
||||
it due, and only `rotate` replaces the key.
|
||||
- `status` prints whether a key is issued, its key id, its age, its rotate-by
|
||||
day, each retiring key, what the proxy list holds, a set fence, any key in
|
||||
the Triage namespace the file does not hold, and the next command. It takes
|
||||
no lock and writes nothing.
|
||||
|
||||
`issue`, `rotate` and `retire` hold the instance proxy lock that org buses
|
||||
take (`<dataRoot>/proxy`). Each reads the list, refuses without this
|
||||
instance's sentinel (`triage-key-proxy-open`), reads it again and refuses if
|
||||
it moved (`triage-key-raced`), sets the fence, writes, reads back and clears
|
||||
the fence. The proxy changes first and the file second. Each run that
|
||||
changes something appends one line of key ids, never keys, to
|
||||
`<dataRoot>/proxy/mosaic-keys.receipts.jsonl`. That file is checked under the
|
||||
lock before anything changes; an unsafe one refuses
|
||||
`triage-key-receipts-unsafe`. A receipt line that still fails after the change
|
||||
refuses `triage-key-unrecorded`. No output, error or receipt holds a key.
|
||||
|
||||
The file is written to a 0600 temp file in the same directory, synced, renamed
|
||||
into place, and the directory is synced. A failed directory sync after the
|
||||
rename refuses `triage-key-unsynced`: the file is in place but may not survive
|
||||
a power loss. `proxy-lock-busy` means another proxy writer holds the lock; if
|
||||
none is running, `mosaic proxy fence status` shows the lock state.
|
||||
|
||||
After a crash:
|
||||
|
||||
- Killed after the proxy write and before the read-back: the fence stays set
|
||||
and every proxy writer on the instance refuses `proxy-uncertain`. `status`
|
||||
shows the fence. Restart the proxy, run `mosaic proxy fence clear
|
||||
--proxy-restarted`, then run the verb again.
|
||||
- Killed after the read-back and before the file write: the proxy holds a key
|
||||
the file does not. `status` names it; the next `issue`, `rotate` or `retire`
|
||||
removes it. A file that could not be written refuses
|
||||
`triage-key-unwritten` and says to run the verb again.
|
||||
- Killed after the temp file is written and before its rename: a
|
||||
`.model.<pid>.<hex>.tmp` file that may hold a key stays in the triage
|
||||
directory. `status` counts it; the next `issue`, `rotate` or `retire`
|
||||
removes it under the lock.
|
||||
- A `retire` killed after the proxy write leaves a retiring key the proxy no
|
||||
longer holds. `status` marks it; the next `retire` drops it from the file.
|
||||
|
||||
Nothing gives the key to the Triage host yet; that is the wiring slice.
|
||||
@@ -56,6 +56,7 @@ import { ORG_CONFIG, runConfigCommand } from "../../organization/src/config-comm
|
||||
import { devCommand, DEV_USAGE, fixtureLaunchEnv, fixtureLauncherOptions } from "./dev-fixture.mjs";
|
||||
import { runCommand, RUN_USAGE } from "./run-evidence.mjs";
|
||||
import { tasksWriteCommand, TASKS_WRITE_OPS, TASKS_WRITE_USAGE } from "./tasks-write.mjs";
|
||||
import { triageCommand, TRIAGE_USAGE } from "./triage.mjs";
|
||||
|
||||
export const USAGE = `usage:
|
||||
${ACCOUNTS_USAGE}
|
||||
@@ -103,6 +104,7 @@ export const USAGE = `usage:
|
||||
mosaic console start [--dev-local-owner [--dev-insecure-bind ADDR --dev-allow-host NAME[,NAME...]]] [--detach] [--port auto|N]
|
||||
mosaic console stop|status
|
||||
mosaic console restart
|
||||
${TRIAGE_USAGE}
|
||||
${DEV_USAGE}
|
||||
${RUN_USAGE}`;
|
||||
|
||||
@@ -818,6 +820,8 @@ export async function main(argv, io = { env: process.env, stdin: process.stdin,
|
||||
if (verb === "launches") return launches(rest, io, deps);
|
||||
if (verb === "bus") return bus(rest, io, deps);
|
||||
if (verb === "console") return consoleCommand(rest, io, deps);
|
||||
// triage-route-store (#1873): the owner's Triage route file, read and written as the person.
|
||||
if (verb === "triage") return triageCommand(rest, io, deps);
|
||||
if (verb === "dev") return devCommand(rest, io, { system: deps.system });
|
||||
if (verb === "run") return runCommand(rest, io, { system: deps.system, callFor: (p) => channelCallFor(p, io, deps) });
|
||||
if (verb === "-h" || verb === "--help") return io.stdout.write(`${USAGE}\n`);
|
||||
|
||||
@@ -0,0 +1,392 @@
|
||||
// #1873 triage-key: Triage's own model proxy client key, issued from the owner's terminal
|
||||
// (design report reports/triage-route-design.md slice 3, Jason's ruling 2026-11-18).
|
||||
//
|
||||
// <configDir>/secrets/triage/model (directory 0700, file 0600, the running user's)
|
||||
// { version, key, keyId, issuedAt, rotateBy, retiring: [{ key, keyId, issuedAt, retireAt }] }
|
||||
//
|
||||
// The key is `mst-<instance tag>-triage-<43 base64url>`, where the tag is the first 12 hex of
|
||||
// sha256(configDir), the tag the instance's own `msk-` sentinel carries. An org bus reconciles only
|
||||
// its own `mso-<tag>-<org>-key-` names and a no-org bus only `msk-<tag>-`, so neither touches it.
|
||||
//
|
||||
// Every proxy write runs under the instance proxy lock (packages/bus/src/proxy-lock.mjs), the lock
|
||||
// org buses take: read the list, check this instance's sentinel, fence, PUT, read back, confirm.
|
||||
// The proxy changes first and the file second, as the bus does for an agent key. A crash between
|
||||
// leaves a key in the Triage namespace that the file does not hold; the next issue, rotate or retire
|
||||
// removes it, and status names it. A crash after the PUT and before the read-back leaves the lock's
|
||||
// uncertain fence, which every proxy writer on the instance refuses until a human clears it
|
||||
// (`mosaic proxy fence clear --proxy-restarted`).
|
||||
//
|
||||
// Nothing runs on a timer. rotate records when the old key's grace ends; the key stays in the proxy
|
||||
// until a later retire, rotate or issue removes it, and status says when that is due. The same goes
|
||||
// for rotateBy: status marks it due; only rotate replaces the key.
|
||||
//
|
||||
// Nothing here prints, logs or records a key. Receipts and output carry key ids only (the first 12
|
||||
// hex of sha256(key), the id the bus uses). Messages name files and options, never values.
|
||||
// Nothing wires the key into the Triage host or the Console; that is a later slice.
|
||||
import { closeSync, constants, fchmodSync, fstatSync, fsyncSync, lstatSync, mkdirSync, openSync, readdirSync, readFileSync, renameSync, unlinkSync, writeSync } from "node:fs";
|
||||
import { createHash, randomBytes } from "node:crypto";
|
||||
import { join } from "node:path";
|
||||
import { isDeepStrictEqual } from "node:util";
|
||||
import { createProxyLock } from "../../bus/src/proxy-lock.mjs";
|
||||
import { loadManagementClient } from "../../model-proxy/src/client.mjs";
|
||||
|
||||
export const KEY_VERSION = 1;
|
||||
// The org key lifetime and the rotate grace default. The bus holds both as literals
|
||||
// (packages/bus/src/model-keys.mjs #issue: 30 days to rotateBy; packages/bus/src/runtime.mjs:
|
||||
// `proxy.rotateGraceMinutes ?? 120`) and exports neither, and this lane may not change the bus.
|
||||
// packages/cli/tests/triage-key.test.mjs drives the bus's own ModelKeys with the same clock and
|
||||
// proves these give the same rotateBy and retireAt, so a change on either side goes red there.
|
||||
// The grace itself is read from proxy.json first, where the bus reads it.
|
||||
export const KEY_LIFETIME_DAYS = 30;
|
||||
export const DEFAULT_ROTATE_GRACE_MINUTES = 120;
|
||||
const FIELDS = Object.freeze(["version", "key", "keyId", "issuedAt", "rotateBy", "retiring"]);
|
||||
const RETIRING_FIELDS = Object.freeze(["key", "keyId", "issuedAt", "retireAt"]);
|
||||
const MAX_BYTES = 64 * 1024;
|
||||
const MAX_RETIRING = 16;
|
||||
const SECRET = /^[A-Za-z0-9_-]{43}$/;
|
||||
const ISO = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$/;
|
||||
const DAY = /^\d{4}-\d{2}-\d{2}$/;
|
||||
|
||||
export class TriageKeyError extends Error {
|
||||
constructor(code, message, exitCode = 3) {
|
||||
super(message);
|
||||
this.name = "TriageKeyError";
|
||||
this.code = code;
|
||||
this.exitCode = exitCode;
|
||||
}
|
||||
}
|
||||
const unreadable = (why) => new TriageKeyError("triage-key-unreadable", `the Triage key store is unreadable: ${why}`);
|
||||
|
||||
export const keyIdOf = (key) => createHash("sha256").update(key).digest("hex").slice(0, 12);
|
||||
export const instanceTag = (configDir) => createHash("sha256").update(configDir).digest("hex").slice(0, 12);
|
||||
export const keyPrefix = (configDir) => `mst-${instanceTag(configDir)}-triage-`;
|
||||
const sentinelPrefix = (configDir) => `msk-${instanceTag(configDir)}-sentinel-`;
|
||||
const ownKey = (configDir) => (k) => typeof k === "string" && k.startsWith(keyPrefix(configDir)) && SECRET.test(k.slice(keyPrefix(configDir).length));
|
||||
// Any name under the prefix is Triage's to remove, well formed or not, as the bus treats its own
|
||||
// `msk-<tag>-` names. Only a well-formed key may be stored in the file.
|
||||
const inNamespace = (configDir) => (k) => typeof k === "string" && k.startsWith(keyPrefix(configDir));
|
||||
|
||||
export function keyPaths(configDir) {
|
||||
const secrets = join(configDir, "secrets");
|
||||
const dir = join(secrets, "triage");
|
||||
return { secrets, dir, file: join(dir, "model") };
|
||||
}
|
||||
|
||||
const plain = (v) => v !== null && typeof v === "object" && !Array.isArray(v) && Object.getPrototypeOf(v) === Object.prototype;
|
||||
const exactKeys = (v, keys) => plain(v) && Object.keys(v).length === keys.length && keys.every((k) => Object.hasOwn(v, k));
|
||||
const canonicalTime = (s) => typeof s === "string" && ISO.test(s) && !Number.isNaN(Date.parse(s)) && new Date(s).toISOString() === s;
|
||||
const canonicalDay = (s) => typeof s === "string" && DAY.test(s) && !Number.isNaN(Date.parse(`${s}T00:00:00Z`)) && new Date(`${s}T00:00:00Z`).toISOString().slice(0, 10) === s;
|
||||
|
||||
export function validRecord(doc, configDir) {
|
||||
const own = ownKey(configDir);
|
||||
return exactKeys(doc, FIELDS)
|
||||
&& doc.version === KEY_VERSION
|
||||
&& own(doc.key) && doc.keyId === keyIdOf(doc.key)
|
||||
&& canonicalTime(doc.issuedAt) && canonicalDay(doc.rotateBy)
|
||||
&& Array.isArray(doc.retiring) && doc.retiring.length <= MAX_RETIRING
|
||||
&& doc.retiring.every((r) => exactKeys(r, RETIRING_FIELDS) && own(r.key) && r.key !== doc.key && r.keyId === keyIdOf(r.key)
|
||||
&& canonicalTime(r.issuedAt) && canonicalTime(r.retireAt))
|
||||
&& new Set(doc.retiring.map((r) => r.key)).size === doc.retiring.length;
|
||||
}
|
||||
|
||||
const uidOf = (opts) => opts.uid ?? process.getuid();
|
||||
|
||||
// A real directory, the running user's, mode 0700. Missing: false, or made when `create` is on.
|
||||
function checkDir(dir, uid, what, { create }) {
|
||||
let st;
|
||||
try { st = lstatSync(dir); }
|
||||
catch (e) {
|
||||
if (e.code !== "ENOENT") throw unreadable(`${what} cannot be read`);
|
||||
if (!create) return false;
|
||||
try { mkdirSync(dir, { mode: 0o700 }); } catch { throw unreadable(`${what} cannot be made`); }
|
||||
try {
|
||||
const fd = openSync(dir, constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW);
|
||||
try { fchmodSync(fd, 0o700); } finally { closeSync(fd); }
|
||||
} catch { throw unreadable(`${what} cannot be made`); }
|
||||
st = lstatSync(dir);
|
||||
}
|
||||
if (st.isSymbolicLink()) throw unreadable(`${what} is a link`);
|
||||
if (!st.isDirectory()) throw unreadable(`${what} is not a directory`);
|
||||
if (st.uid !== uid) throw unreadable(`${what} is not owned by this user`);
|
||||
if ((st.mode & 0o777) !== 0o700) throw unreadable(`${what} mode is not 0700`);
|
||||
return true;
|
||||
}
|
||||
|
||||
function checkFile(fd, uid, what, fail = unreadable) {
|
||||
const st = fstatSync(fd);
|
||||
if (!st.isFile()) throw fail(`${what} is not a regular file`);
|
||||
if (st.nlink !== 1) throw fail(`${what} has more than one link`);
|
||||
if (st.uid !== uid) throw fail(`${what} is not owned by this user`);
|
||||
if ((st.mode & 0o777) !== 0o600) throw fail(`${what} mode is not 0600`);
|
||||
return st;
|
||||
}
|
||||
|
||||
// The stored record, or null when no key is issued. Throws triage-key-unreadable on anything unsafe.
|
||||
// opts.uid (both directories and the file) and opts.fileUid (the file only) are test seams for the
|
||||
// owner checks.
|
||||
export function readKey(configDir, opts = {}) {
|
||||
const { secrets, dir, file } = keyPaths(configDir);
|
||||
const uid = uidOf(opts);
|
||||
if (!checkDir(secrets, uid, "the secrets directory", { create: false })) return null;
|
||||
if (!checkDir(dir, uid, "the triage secrets directory", { create: false })) return null;
|
||||
let fd;
|
||||
try { fd = openSync(file, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK); }
|
||||
catch (e) {
|
||||
if (e.code === "ENOENT") return null;
|
||||
if (e.code === "ELOOP") throw unreadable("the key file is a link");
|
||||
throw unreadable("the key file cannot be opened");
|
||||
}
|
||||
try {
|
||||
const st = checkFile(fd, opts.fileUid ?? uid, "the key file");
|
||||
if (st.size > MAX_BYTES) throw unreadable("the key file is too large");
|
||||
let doc;
|
||||
try { doc = JSON.parse(readFileSync(fd, "utf8")); } catch { throw unreadable("the key file is not JSON"); }
|
||||
if (!validRecord(doc, configDir)) throw unreadable("the key file does not match the key shape");
|
||||
return doc;
|
||||
} finally { closeSync(fd); }
|
||||
}
|
||||
|
||||
// Both directories, checked (and the triage one made) before any proxy write, so a store that cannot
|
||||
// be written refuses while the proxy is still untouched.
|
||||
function prepareDirs(configDir, uid) {
|
||||
const { secrets, dir } = keyPaths(configDir);
|
||||
if (!checkDir(secrets, uid, "the secrets directory", { create: false })) throw unreadable("the secrets directory is missing");
|
||||
checkDir(dir, uid, "the triage secrets directory", { create: true });
|
||||
}
|
||||
|
||||
// A temp file writeKey left when it was killed before its rename. It may hold a key.
|
||||
const TEMP = /^\.model\.\d+\.[0-9a-f]{12}\.tmp$/;
|
||||
const tempsIn = (dir) => { try { return readdirSync(dir).filter((n) => TEMP.test(n)); } catch { return []; } };
|
||||
// Under the lock no other Triage writer runs, so every temp file left in the directory is dead.
|
||||
function removeTemps(configDir) {
|
||||
const { dir } = keyPaths(configDir);
|
||||
for (const name of tempsIn(dir)) {
|
||||
try { if (!lstatSync(join(dir, name)).isDirectory()) unlinkSync(join(dir, name)); }
|
||||
catch { throw unreadable("a leftover temp file in the triage secrets directory cannot be removed"); }
|
||||
}
|
||||
}
|
||||
|
||||
// A directory sync that failed after the rename: the file is in place but may not be durable.
|
||||
class NotSynced extends Error {}
|
||||
|
||||
// Atomic: a 0600 temp file in the same directory, fsync, rename, fsync of the directory.
|
||||
function writeKey(configDir, record, uid, hook = () => {}) {
|
||||
const { dir, file } = keyPaths(configDir);
|
||||
prepareDirs(configDir, uid);
|
||||
const temp = join(dir, `.model.${process.pid}.${randomBytes(6).toString("hex")}.tmp`);
|
||||
let fd;
|
||||
try { fd = openSync(temp, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW, 0o600); }
|
||||
catch { throw unreadable("the key file cannot be written"); }
|
||||
try {
|
||||
try { fchmodSync(fd, 0o600); writeSync(fd, `${JSON.stringify(record, null, 2)}\n`); fsyncSync(fd); }
|
||||
finally { closeSync(fd); }
|
||||
hook("rename");
|
||||
renameSync(temp, file);
|
||||
} catch { try { unlinkSync(temp); } catch {} throw unreadable("the key file cannot be written"); }
|
||||
try {
|
||||
hook("sync");
|
||||
const dfd = openSync(dir, constants.O_RDONLY | constants.O_DIRECTORY);
|
||||
try { fsyncSync(dfd); } finally { closeSync(dfd); }
|
||||
} catch { throw new NotSynced(); }
|
||||
}
|
||||
|
||||
// One line per write in the proxy lock's receipts file, ids only. `receiptsReady` runs under the
|
||||
// lock before any change, so an unsafe receipts file refuses while nothing has changed yet.
|
||||
const RECEIPTS_FLAGS = constants.O_WRONLY | constants.O_APPEND | constants.O_CREAT | constants.O_NOFOLLOW | constants.O_NONBLOCK;
|
||||
const receiptsUnsafe = (why) => new TriageKeyError("triage-key-receipts-unsafe", `${why}; nothing was changed`);
|
||||
function receiptsReady(stateDir, uid) {
|
||||
let fd;
|
||||
try { fd = openSync(join(stateDir, "mosaic-keys.receipts.jsonl"), RECEIPTS_FLAGS, 0o600); }
|
||||
catch { throw receiptsUnsafe("the proxy receipts file cannot be opened"); }
|
||||
try { checkFile(fd, uid, "the proxy receipts file", receiptsUnsafe); } finally { closeSync(fd); }
|
||||
}
|
||||
function receipt(stateDir, uid, entry) {
|
||||
const unrecorded = new TriageKeyError("triage-key-unrecorded", "the change was made but its receipt line was not written; run mosaic triage key status");
|
||||
let fd;
|
||||
try { fd = openSync(join(stateDir, "mosaic-keys.receipts.jsonl"), RECEIPTS_FLAGS, 0o600); } catch { throw unrecorded; }
|
||||
try {
|
||||
checkFile(fd, uid, "the proxy receipts file", () => unrecorded);
|
||||
writeSync(fd, `${JSON.stringify(entry)}\n`);
|
||||
fsyncSync(fd);
|
||||
} catch { throw unrecorded; }
|
||||
finally { closeSync(fd); }
|
||||
}
|
||||
|
||||
// The grace the bus uses: proxy.json's rotateGraceMinutes, or the bus default.
|
||||
export const graceMinutes = (proxy) => proxy?.rotateGraceMinutes ?? DEFAULT_ROTATE_GRACE_MINUTES;
|
||||
export const rotateByFor = (nowMs) => new Date(nowMs + KEY_LIFETIME_DAYS * 86400_000).toISOString().slice(0, 10);
|
||||
export const retireAtFor = (nowMs, proxy) => new Date(nowMs + graceMinutes(proxy) * 60_000).toISOString();
|
||||
|
||||
const proxyFailure = (code) => {
|
||||
const words = {
|
||||
"proxy-lock-busy": "another proxy writer holds the instance proxy lock; try again when it ends, and if no writer is running, see mosaic proxy fence status",
|
||||
"proxy-lock-uncertain": "the instance proxy lock is in a state this verb will not guess about; check <dataRoot>/proxy by hand",
|
||||
"proxy-lock-invalid": "the instance proxy lock directory is missing or unsafe, or names another proxy",
|
||||
"proxy-uncertain": "an earlier proxy write has an uncertain outcome; see mosaic proxy fence status",
|
||||
"proxy-management-key-invalid": "the proxy management key file is missing or unsafe",
|
||||
"proxy-client-invalid": "the proxy settings are not usable",
|
||||
"proxy-client-down": "the proxy did not answer",
|
||||
"proxy-client-refused": "the proxy refused the management request",
|
||||
"proxy-client-redirect": "the proxy answered with a redirect",
|
||||
"proxy-client-response": "the proxy gave an answer this verb cannot use",
|
||||
};
|
||||
return new TriageKeyError(code, words[code] ?? "the proxy write failed");
|
||||
};
|
||||
|
||||
// opts.uid, opts.now and opts.hook(stage) are test seams; a stage hook may throw or kill the process.
|
||||
function context({ configDir, system, uid, fileUid, now, hook }) {
|
||||
if (!system?.proxy || !Number.isInteger(system.proxy.port)) throw new TriageKeyError("triage-key-no-proxy", "the instance has no model proxy set up (proxy.json)");
|
||||
return {
|
||||
configDir, dataRoot: system.dataRoot, proxy: system.proxy, stateDir: join(system.dataRoot, "proxy"),
|
||||
uid: uid ?? process.getuid(), fileUid, now: now ?? Date.now, hook: hook ?? (() => {}),
|
||||
};
|
||||
}
|
||||
|
||||
function lockFor(c) {
|
||||
try { return createProxyLock({ stateDir: c.stateDir, identity: `${c.configDir}:${c.proxy.port}` }); }
|
||||
catch (e) { throw proxyFailure(e.code ?? "proxy-lock-invalid"); }
|
||||
}
|
||||
|
||||
async function clientFor(c) {
|
||||
try { return await loadManagementClient({ configDir: c.configDir, proxy: c.proxy }); }
|
||||
catch (e) { throw proxyFailure(e.code ?? "proxy-management-key-invalid"); }
|
||||
}
|
||||
|
||||
function checkSentinel(c, keys) {
|
||||
const prefix = sentinelPrefix(c.configDir);
|
||||
if (!keys.some((k) => k.startsWith(prefix) && SECRET.test(k.slice(prefix.length)))) {
|
||||
throw new TriageKeyError("triage-key-proxy-open", "the proxy's key list has no sentinel for this instance, so it may be another proxy");
|
||||
}
|
||||
}
|
||||
|
||||
// Under the instance proxy lock: read the list, check the sentinel, run `plan`, write the result.
|
||||
// `plan(keys, record)` returns { after, record, removed, added, done }; `record` null leaves the
|
||||
// file alone. The proxy changes first, then the file, then one receipt line.
|
||||
async function change(c, op, plan) {
|
||||
const lock = lockFor(c);
|
||||
const client = await clientFor(c);
|
||||
try {
|
||||
return await lock.run(async () => {
|
||||
const before = await client.getKeys();
|
||||
checkSentinel(c, before);
|
||||
const current = readKey(c.configDir, { uid: c.uid, fileUid: c.fileUid });
|
||||
const result = plan(before, current);
|
||||
if (result.record) prepareDirs(c.configDir, c.uid);
|
||||
removeTemps(c.configDir);
|
||||
receiptsReady(c.stateDir, c.uid);
|
||||
const after = result.after;
|
||||
const removed = before.filter((k) => !after.includes(k));
|
||||
const added = after.filter((k) => !before.includes(k));
|
||||
// A key in another namespace never leaves the list.
|
||||
if (removed.some((k) => !inNamespace(c.configDir)(k))) throw new TriageKeyError("triage-key-failed", "the change would remove a key outside the Triage namespace");
|
||||
if (added.length || removed.length) {
|
||||
// As the bus does: a list that moved since the first read is not overwritten.
|
||||
if (!isDeepStrictEqual(await client.getKeys(), before)) throw new TriageKeyError("triage-key-raced", "the proxy's key list changed while this verb ran; run it again");
|
||||
lock.fence({ org: null, op: `triage.${op}` });
|
||||
await client.putKeys(after);
|
||||
c.hook("put");
|
||||
const confirmed = await client.getKeys();
|
||||
if (!confirmed.some((k) => k.startsWith(sentinelPrefix(c.configDir))) || added.some((k) => !confirmed.includes(k)) || removed.some((k) => confirmed.includes(k))) {
|
||||
throw new TriageKeyError("triage-key-unconfirmed", "the proxy did not confirm the change; the instance proxy fence stays set, see mosaic proxy fence status");
|
||||
}
|
||||
lock.confirm();
|
||||
}
|
||||
c.hook("write");
|
||||
let unsynced = false;
|
||||
if (result.record) {
|
||||
try { writeKey(c.configDir, result.record, c.uid, c.hook); }
|
||||
catch (e) {
|
||||
if (e instanceof NotSynced) unsynced = true;
|
||||
else throw new TriageKeyError("triage-key-unwritten", `the proxy list changed but ${keyPaths(c.configDir).file} was not written (${e.message}); fix the file's directory, then run mosaic triage key ${op} again, which clears what this run left`);
|
||||
}
|
||||
}
|
||||
c.hook("receipt");
|
||||
// A run that changed neither the proxy nor the file leaves no receipt.
|
||||
if (added.length || removed.length || result.record) receipt(c.stateDir, c.uid, { op: `triage.key.${op}`, time: new Date(c.now()).toISOString(), addedKeyIds: added.map(keyIdOf), removedKeyIds: removed.map(keyIdOf) });
|
||||
// The file is in place and the receipt written; only its durability is in doubt.
|
||||
if (unsynced) throw new TriageKeyError("triage-key-unsynced", `${keyPaths(c.configDir).file} was written but its directory could not be synced, so it may not survive a power loss; run mosaic triage key status`);
|
||||
return { ...result.done, addedKeyIds: added.map(keyIdOf), removedKeyIds: removed.map(keyIdOf) };
|
||||
});
|
||||
} catch (e) {
|
||||
if (e instanceof TriageKeyError) throw e;
|
||||
throw proxyFailure(typeof e?.code === "string" && /^proxy-[a-z-]+$/.test(e.code) ? e.code : "triage-key-failed");
|
||||
}
|
||||
}
|
||||
|
||||
const newKey = (configDir) => keyPrefix(configDir) + randomBytes(32).toString("base64url");
|
||||
|
||||
// Keys in the Triage namespace that the file does not hold: left by a crash between the PUT and the
|
||||
// file write, or by a file that was removed by hand.
|
||||
const orphansOf = (configDir, keys, record) => {
|
||||
const held = record ? [record.key, ...record.retiring.map((r) => r.key)] : [];
|
||||
return keys.filter((k) => inNamespace(configDir)(k) && !held.includes(k));
|
||||
};
|
||||
|
||||
export async function issueKey(opts) {
|
||||
const c = context(opts);
|
||||
return change(c, "issue", (before, current) => {
|
||||
if (current) throw new TriageKeyError("triage-key-issued", "a Triage key is already issued; use mosaic triage key rotate");
|
||||
const orphans = orphansOf(c.configDir, before, null);
|
||||
const key = newKey(c.configDir), nowMs = c.now();
|
||||
const record = { version: KEY_VERSION, key, keyId: keyIdOf(key), issuedAt: new Date(nowMs).toISOString(), rotateBy: rotateByFor(nowMs), retiring: [] };
|
||||
return { after: [...before.filter((k) => !orphans.includes(k)), key], record, done: { keyId: record.keyId, rotateBy: record.rotateBy } };
|
||||
});
|
||||
}
|
||||
|
||||
// Retiring keys whose grace has passed, or that the proxy no longer holds.
|
||||
const dueOf = (record, keys, nowMs) => record.retiring.filter((r) => Date.parse(r.retireAt) <= nowMs || !keys.includes(r.key));
|
||||
|
||||
export async function rotateKey(opts) {
|
||||
const c = context(opts);
|
||||
return change(c, "rotate", (before, current) => {
|
||||
if (!current) throw new TriageKeyError("triage-key-not-issued", "no Triage key is issued; use mosaic triage key issue");
|
||||
const nowMs = c.now();
|
||||
const orphans = orphansOf(c.configDir, before, current);
|
||||
const due = dueOf(current, before, nowMs);
|
||||
const keep = before.includes(current.key);
|
||||
const retiring = [...current.retiring.filter((r) => !due.includes(r)),
|
||||
...(keep ? [{ key: current.key, keyId: current.keyId, issuedAt: current.issuedAt, retireAt: retireAtFor(nowMs, c.proxy) }] : [])];
|
||||
if (retiring.length > MAX_RETIRING) throw new TriageKeyError("triage-key-failed", "too many keys are retiring; run mosaic triage key retire once their grace has passed");
|
||||
const key = newKey(c.configDir);
|
||||
const record = { version: KEY_VERSION, key, keyId: keyIdOf(key), issuedAt: new Date(nowMs).toISOString(), rotateBy: rotateByFor(nowMs), retiring };
|
||||
const drop = [...orphans, ...due.map((r) => r.key)];
|
||||
return {
|
||||
after: [...before.filter((k) => !drop.includes(k)), key], record,
|
||||
done: { keyId: record.keyId, rotateBy: record.rotateBy, old: keep ? { keyId: current.keyId, retireAt: retiring.at(-1).retireAt } : { keyId: current.keyId, lost: true } },
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
export async function retireKeys(opts) {
|
||||
const c = context(opts);
|
||||
return change(c, "retire", (before, current) => {
|
||||
const nowMs = c.now();
|
||||
const orphans = orphansOf(c.configDir, before, current);
|
||||
const due = current ? dueOf(current, before, nowMs) : [];
|
||||
const record = current && due.length ? { ...current, retiring: current.retiring.filter((r) => !due.includes(r)) } : null;
|
||||
const drop = [...orphans, ...due.map((r) => r.key)];
|
||||
return { after: before.filter((k) => !drop.includes(k)), record, done: { droppedKeyIds: due.map((r) => r.keyId) } };
|
||||
});
|
||||
}
|
||||
|
||||
// Read only: the file, and when the proxy is set up, what its list holds. No lock, no write.
|
||||
// The fence is checked by name only, so status writes nothing, not even the lock's identity file.
|
||||
export async function keyStatus(opts) {
|
||||
const { configDir, system, uid, fileUid } = opts;
|
||||
const record = readKey(configDir, { uid, fileUid });
|
||||
const proxy = { state: "not-set-up", fence: false, leftoverTemps: tempsIn(keyPaths(configDir).dir).length };
|
||||
if (system?.proxy && Number.isInteger(system.proxy.port)) {
|
||||
try { lstatSync(join(system.dataRoot, "proxy", "mosaic-keys.uncertain")); proxy.fence = true; }
|
||||
catch (e) { if (e.code !== "ENOENT") proxy.fence = true; }
|
||||
try {
|
||||
const client = await loadManagementClient({ configDir, proxy: system.proxy });
|
||||
const keys = await client.getKeys();
|
||||
const prefix = sentinelPrefix(configDir);
|
||||
proxy.state = keys.some((k) => k.startsWith(prefix) && SECRET.test(k.slice(prefix.length))) ? "read" : "no-sentinel";
|
||||
proxy.holdsKey = record ? keys.includes(record.key) : null;
|
||||
proxy.retiringHeld = record ? record.retiring.map((r) => keys.includes(r.key)) : [];
|
||||
proxy.orphanKeyIds = orphansOf(configDir, keys, record).map(keyIdOf);
|
||||
} catch (e) { proxy.state = "unreadable"; proxy.code = typeof e?.code === "string" && /^proxy-[a-z-]+$/.test(e.code) ? e.code : "proxy-client-response"; }
|
||||
}
|
||||
return { record, proxy };
|
||||
}
|
||||
@@ -0,0 +1,188 @@
|
||||
// #1873 triage-route-store: the owner's terminal verbs for the Triage route file,
|
||||
// <dataRoot>/triage/route.json (packages/webui/src/triage-route-store.mjs).
|
||||
//
|
||||
// mosaic triage route show
|
||||
// mosaic triage route set --model KEY --harness NAME --cap TOKENS --time-zone ZONE [--revision N]
|
||||
//
|
||||
// Both are human verbs: an agent run is refused before anything is read (exit 3), from this
|
||||
// process's environment and the one the command was given, as the bus and Console lifecycle verbs
|
||||
// refuse it. `show` with no file prints "not set" and exits 0. Without --revision a write
|
||||
// compares with the revision read under the store's lock; pass --revision from `show` to make the
|
||||
// write conditional on what was shown (0: only when no route is set yet).
|
||||
//
|
||||
// Nothing here wires the route into the Triage host: the Console keeps `route: null` until the
|
||||
// wiring slice. `set` checks the harness against the repository's harness modules; it does not
|
||||
// check the model key against a catalogue, since none exists outside an organization's bus yet.
|
||||
//
|
||||
// mosaic triage key issue | rotate | retire | status
|
||||
//
|
||||
// Triage's own model proxy client key (packages/cli/src/triage-key.mjs). Human verbs as well,
|
||||
// refused in an agent run before anything is read. No output names a key, only its id.
|
||||
import { lstatSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { readRoute, RouteStoreError, writeRoute } from "../../webui/src/triage-route-store.mjs";
|
||||
import { loadSystem } from "./config.mjs";
|
||||
import { configDir } from "../../organization/src/organization-config.mjs";
|
||||
import { issueKey, keyPaths, keyStatus, retireKeys, rotateKey, TriageKeyError } from "./triage-key.mjs";
|
||||
import { CliError } from "./errors.mjs";
|
||||
import { refuseInsideAgent } from "./transport.mjs";
|
||||
|
||||
export const TRIAGE_USAGE = "mosaic triage route show | set --model KEY --harness NAME --cap TOKENS --time-zone ZONE [--revision N] | key issue | rotate | retire | status";
|
||||
const KEY_VERBS = ["issue", "rotate", "retire", "status"];
|
||||
const HARNESS_ROOT = fileURLToPath(new URL("../../plugins/harness/", import.meta.url));
|
||||
const VALUES = ["--model", "--harness", "--cap", "--time-zone", "--revision"];
|
||||
const WHOLE = /^(0|[1-9][0-9]*)$/;
|
||||
// The actor every terminal write records: the instance owner, the only person who runs this verb.
|
||||
const ACTOR = "owner";
|
||||
|
||||
const usage = (why) => new CliError(`${why}\nusage: ${TRIAGE_USAGE}`, 4);
|
||||
|
||||
// Option names, never values, in a refusal.
|
||||
function parse(args) {
|
||||
const values = {}, positional = [];
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
const a = args[i];
|
||||
if (VALUES.includes(a)) {
|
||||
if (i + 1 >= args.length) throw usage(`${a} needs one value`);
|
||||
if (Object.hasOwn(values, a)) throw usage(`${a} is given twice`);
|
||||
values[a] = args[++i];
|
||||
} else if (a.startsWith("--")) throw usage("unknown option");
|
||||
else positional.push(a);
|
||||
}
|
||||
return { values, positional };
|
||||
}
|
||||
|
||||
function whole(text, name) {
|
||||
if (!WHOLE.test(text)) throw new CliError(`triage-route-invalid: ${name} must be a whole number`, 2);
|
||||
return Number(text);
|
||||
}
|
||||
|
||||
function knownHarness(name) {
|
||||
if (!/^[a-z][a-z0-9-]{0,31}$/.test(name)) return false;
|
||||
try { return lstatSync(join(HARNESS_ROOT, name, "index.mjs")).isFile(); } catch { return false; }
|
||||
}
|
||||
|
||||
function storeError(error) {
|
||||
if (!(error instanceof RouteStoreError)) throw error;
|
||||
return new CliError(`${error.code}: ${error.message}`, error.code === "triage-route-invalid" ? 2 : 3);
|
||||
}
|
||||
|
||||
function show(route) {
|
||||
if (!route) return "triage route: not set\n";
|
||||
const { cap } = route;
|
||||
return [
|
||||
`triage route: revision ${route.revision}`,
|
||||
`model: ${route.model}`,
|
||||
`harness: ${route.harness}`,
|
||||
`cap: ${cap.limit} tokens a day, resets ${cap.reset} ${cap.timeZone}, counted since ${cap.since}`,
|
||||
`updated: ${route.updatedAt} by ${route.updatedBy}`,
|
||||
"",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
// A key error prints its code and fixed words. Anything else becomes one fixed line, so no value
|
||||
// that an unexpected error might carry reaches the terminal.
|
||||
function keyError(error) {
|
||||
if (error instanceof CliError) return error;
|
||||
if (error instanceof TriageKeyError) return new CliError(`${error.code}: ${error.message}`, error.exitCode);
|
||||
return new CliError("triage-key-failed: the key verb stopped on an unexpected error; run mosaic triage key status", 3);
|
||||
}
|
||||
|
||||
const age = (ms) => {
|
||||
const m = Math.max(0, Math.floor(ms / 60_000));
|
||||
return `${Math.floor(m / 1440)}d ${Math.floor((m % 1440) / 60)}h ${m % 60}m`;
|
||||
};
|
||||
const ids = (list) => (list.length ? list.join(", ") : "none");
|
||||
|
||||
function statusText({ record, proxy }, nowMs, file) {
|
||||
const lines = [`triage key: ${record ? "issued" : "not issued"}`, `file: ${file}`];
|
||||
const rotateDue = record && Date.parse(`${record.rotateBy}T00:00:00Z`) <= nowMs;
|
||||
const retireDue = record ? record.retiring.filter((r) => Date.parse(r.retireAt) <= nowMs) : [];
|
||||
if (record) {
|
||||
lines.push(`key id: ${record.keyId}`, `issued: ${record.issuedAt} (age ${age(nowMs - Date.parse(record.issuedAt))})`, `rotate by: ${record.rotateBy}${rotateDue ? " (due)" : ""}`);
|
||||
record.retiring.forEach((r, i) => {
|
||||
const held = proxy.retiringHeld?.[i];
|
||||
lines.push(`retiring: ${r.keyId}, live until a retire after ${r.retireAt}${Date.parse(r.retireAt) <= nowMs ? " (due)" : ""}${held === false ? " (not in the proxy list)" : ""}`);
|
||||
});
|
||||
}
|
||||
const state = {
|
||||
"not-set-up": "not set up for this instance (no proxy.json)",
|
||||
"no-sentinel": "its key list has no sentinel for this instance",
|
||||
unreadable: `not read (${proxy.code})`,
|
||||
read: record ? (proxy.holdsKey ? "holds the key" : "does not hold the key") : "read",
|
||||
}[proxy.state];
|
||||
lines.push(`proxy: ${state}`);
|
||||
if (proxy.fence) lines.push("proxy fence: set, an earlier proxy write has an uncertain outcome");
|
||||
if (proxy.leftoverTemps) lines.push(`leftover temp files in the triage secrets directory: ${proxy.leftoverTemps} (the next issue, rotate or retire removes them)`);
|
||||
if (proxy.orphanKeyIds?.length) lines.push(`keys in the Triage namespace the file does not hold: ${ids(proxy.orphanKeyIds)}`);
|
||||
let next;
|
||||
if (proxy.fence) next = "restart the proxy, then mosaic proxy fence clear --proxy-restarted";
|
||||
else if (proxy.state !== "read") next = proxy.state === "not-set-up" ? "set up the model proxy first (mosaic proxy init)" : "check the proxy with mosaic proxy status";
|
||||
else if (!record) next = "mosaic triage key issue";
|
||||
else if (!proxy.holdsKey || rotateDue) next = "mosaic triage key rotate";
|
||||
else if (retireDue.length || proxy.leftoverTemps || proxy.orphanKeyIds?.length || proxy.retiringHeld?.includes(false)) next = "mosaic triage key retire";
|
||||
else next = "nothing";
|
||||
lines.push(`next: ${next}`, "");
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
async function keyCommand(sub, args, io, deps) {
|
||||
if (!KEY_VERBS.includes(sub)) throw usage("unknown triage key command");
|
||||
if (args.length) throw usage(args[0].startsWith("--") ? "unknown option" : "unexpected argument");
|
||||
try {
|
||||
const system = deps.system ?? loadSystem({ env: io.env });
|
||||
const dir = configDir(io.env);
|
||||
const now = deps.now ? () => deps.now().getTime() : Date.now;
|
||||
const opts = { configDir: dir, system, now, uid: deps.triageKeyUid, fileUid: deps.triageKeyFileUid, hook: deps.triageKeyHook };
|
||||
if (sub === "status") {
|
||||
io.stdout.write(statusText(await keyStatus(opts), now(), keyPaths(dir).file));
|
||||
return;
|
||||
}
|
||||
if (sub === "issue") {
|
||||
const r = await issueKey(opts);
|
||||
io.stdout.write(`triage key issued: ${r.keyId}, rotate by ${r.rotateBy}\nfile: ${keyPaths(dir).file}\n`);
|
||||
if (r.removedKeyIds.length) io.stdout.write(`removed keys this file did not hold: ${ids(r.removedKeyIds)}\n`);
|
||||
return;
|
||||
}
|
||||
if (sub === "rotate") {
|
||||
const r = await rotateKey(opts);
|
||||
io.stdout.write(`triage key rotated: ${r.keyId}, rotate by ${r.rotateBy}\n`);
|
||||
io.stdout.write(r.old.lost
|
||||
? `old key ${r.old.keyId} was not in the proxy list; nothing to retire\n`
|
||||
: `old key ${r.old.keyId} stays live until mosaic triage key retire runs at or after ${r.old.retireAt}\n`);
|
||||
if (r.removedKeyIds.length) io.stdout.write(`removed: ${ids(r.removedKeyIds)}\n`);
|
||||
return;
|
||||
}
|
||||
const r = await retireKeys(opts);
|
||||
io.stdout.write(r.removedKeyIds.length ? `triage key retire: removed ${ids(r.removedKeyIds)}\n` : "triage key retire: nothing to remove\n");
|
||||
} catch (e) { throw keyError(e); }
|
||||
}
|
||||
|
||||
export function triageCommand(rest, io, deps = {}) {
|
||||
refuseInsideAgent(process.env);
|
||||
refuseInsideAgent(io.env);
|
||||
const [noun, sub, ...args] = rest;
|
||||
if (noun === "key" && sub !== undefined) return keyCommand(sub, args, io, deps);
|
||||
if (noun !== "route" || !["show", "set"].includes(sub)) throw usage("unknown triage command");
|
||||
const { values, positional } = parse(args);
|
||||
if (positional.length) throw usage("unexpected argument");
|
||||
const dataRoot = () => (deps.system ?? loadSystem({ env: io.env })).dataRoot;
|
||||
if (sub === "show") {
|
||||
if (Object.keys(values).length) throw usage("show takes no options");
|
||||
let route;
|
||||
try { route = readRoute(dataRoot()); } catch (e) { throw storeError(e); }
|
||||
io.stdout.write(show(route));
|
||||
return;
|
||||
}
|
||||
for (const name of ["--model", "--harness", "--cap", "--time-zone"]) if (!Object.hasOwn(values, name)) throw usage(`set needs ${name}`);
|
||||
if (!knownHarness(values["--harness"])) throw new CliError("triage-route-invalid: no harness module has that name", 2);
|
||||
const input = {
|
||||
model: values["--model"], harness: values["--harness"], limit: whole(values["--cap"], "--cap"),
|
||||
timeZone: values["--time-zone"], by: ACTOR,
|
||||
...(Object.hasOwn(values, "--revision") ? { revision: whole(values["--revision"], "--revision") } : {}),
|
||||
};
|
||||
let route;
|
||||
try { route = writeRoute(dataRoot(), input, deps.now ? { now: deps.now } : {}); } catch (e) { throw storeError(e); }
|
||||
io.stdout.write(`triage route set: revision ${route.revision}\n`);
|
||||
}
|
||||
@@ -0,0 +1,695 @@
|
||||
// #1873 triage-key: `mosaic triage key issue|rotate|retire|status`, Triage's own model proxy key.
|
||||
// A stand-in management server on loopback and a temporary config directory only; nothing here
|
||||
// reads a real config, proxy or secret. Every stdout, stderr, receipt and file the verbs leave is
|
||||
// scanned for every key the stand-in ever saw, at the end of each test and once more at the end.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { spawn, spawnSync } from "node:child_process";
|
||||
import fs from "node:fs";
|
||||
import http from "node:http";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { createHash, randomBytes } from "node:crypto";
|
||||
import { run } from "../src/cli.mjs";
|
||||
import { AGENT_MARKERS } from "../src/transport.mjs";
|
||||
import { DEFAULT_ROTATE_GRACE_MINUTES, keyPaths, retireAtFor, rotateByFor } from "../src/triage-key.mjs";
|
||||
import { ModelKeys } from "../../bus/src/model-keys.mjs";
|
||||
import { ManagementClient } from "../../model-proxy/src/client.mjs";
|
||||
import { startBroker } from "../../bus/src/runtime.mjs";
|
||||
import { busOrganization } from "../../bus/src/organization-context.mjs";
|
||||
import { Registry } from "../../seat/src/session.mjs";
|
||||
import { loadBusiness } from "../../organization/src/organization-config.mjs";
|
||||
import { resolveInstance } from "../../organization/src/resolve.mjs";
|
||||
import { businessDoc, rolesCopy, systemFor, writeJson } from "../../organization/tests/helpers.mjs";
|
||||
import { humanEnv, io } from "./helpers.mjs";
|
||||
|
||||
const REPO = new URL("../../../", import.meta.url).pathname;
|
||||
const NOW = Date.parse("2026-11-18T12:00:00.000Z");
|
||||
const MIN = 60_000, HOUR = 60 * MIN, DAY = 24 * HOUR;
|
||||
const MANAGE = "fx1873-manage";
|
||||
const KEY_PATH = "/v8/management/config/access/api-keys";
|
||||
const sha = (s) => createHash("sha256").update(s).digest("hex");
|
||||
const tag = (dir) => sha(dir).slice(0, 12);
|
||||
const id = (key) => sha(key).slice(0, 12);
|
||||
const b64 = () => randomBytes(32).toString("base64url");
|
||||
const iso = (ms) => new Date(ms).toISOString();
|
||||
|
||||
// Every output any verb produced in this file, and every Triage key the stand-in ever held.
|
||||
const OUTPUTS = [];
|
||||
const SEEN = new Set();
|
||||
const VERBS = ["issue", "rotate", "retire", "status"];
|
||||
|
||||
function leaks(text, keys = SEEN) {
|
||||
for (const key of keys) {
|
||||
const secret = key.slice(-43);
|
||||
if (text.includes(key) || text.includes(secret)) return true;
|
||||
for (let i = 0; i + 8 <= secret.length; i++) if (text.includes(secret.slice(i, i + 8))) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function walk(dir, skip, out = []) {
|
||||
for (const name of fs.readdirSync(dir)) {
|
||||
const p = join(dir, name), st = fs.lstatSync(p);
|
||||
if (st.isDirectory()) walk(p, skip, out);
|
||||
else if (st.isFile() && !skip.includes(p)) out.push(p);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
async function fixture(t, { sentinel = true } = {}) {
|
||||
const root = fs.realpathSync(fs.mkdtempSync(join(tmpdir(), "tk-")));
|
||||
const configDir = join(root, "config");
|
||||
fs.mkdirSync(join(configDir, "secrets"), { recursive: true, mode: 0o700 });
|
||||
fs.chmodSync(configDir, 0o700); fs.chmodSync(join(configDir, "secrets"), 0o700);
|
||||
fs.writeFileSync(join(configDir, "secrets", "proxy-management.key"), MANAGE, { mode: 0o600 });
|
||||
fs.mkdirSync(join(root, "proxy"), { mode: 0o700 });
|
||||
const prefix = `mst-${tag(configDir)}-triage-`;
|
||||
const state = {
|
||||
keys: [...(sentinel ? [`msk-${tag(configDir)}-sentinel-${b64()}`] : []), "foreign-client-key", `mso-${tag(configDir)}-demo-key-${b64()}`],
|
||||
gets: 0, puts: [], ignorePut: false, dropSentinelOnPut: false, onGet: null,
|
||||
};
|
||||
const server = http.createServer(async (req, res) => {
|
||||
if (req.url !== KEY_PATH) {
|
||||
const key = req.headers.authorization?.slice(7);
|
||||
res.writeHead(state.keys.includes(key) ? 200 : 401); res.end("{}"); return;
|
||||
}
|
||||
if (req.headers.authorization !== `Bearer ${MANAGE}`) { res.writeHead(401); res.end("{}"); return; }
|
||||
if (req.method === "GET") { state.gets++; const body = JSON.stringify(state.keys); state.onGet?.(state.gets); res.end(body); return; }
|
||||
const chunks = []; for await (const c of req) chunks.push(c);
|
||||
const next = JSON.parse(Buffer.concat(chunks));
|
||||
for (const k of next) if (/^mst-.*-[A-Za-z0-9_-]{43}$/.test(k)) SEEN.add(k);
|
||||
state.puts.push(next);
|
||||
if (state.dropSentinelOnPut) state.keys = next.filter((k) => !k.startsWith("msk-") || !k.includes("-sentinel-"));
|
||||
else if (!state.ignorePut) state.keys = next;
|
||||
res.end('{"status":"ok"}');
|
||||
});
|
||||
await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));
|
||||
const port = server.address().port;
|
||||
const closers = [];
|
||||
t.after(async () => {
|
||||
for (const close of closers) await close();
|
||||
server.closeAllConnections(); await new Promise((resolve) => server.close(resolve));
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
const f = {
|
||||
root, configDir, prefix, state, server, port, closers, now: NOW,
|
||||
system: { dataRoot: root, proxy: { port, binary: "/fixture-unused" } },
|
||||
file: keyPaths(configDir).file, dir: keyPaths(configDir).dir,
|
||||
env: (extra = {}) => humanEnv({ MOSAIC_CONFIG: join(configDir, "config.json"), ...extra }),
|
||||
async cli(argv, { env, deps = {}, system = f.system } = {}) {
|
||||
const s = io(env ?? f.env());
|
||||
const code = await run(argv, s, { system, now: () => new Date(f.now), ...deps });
|
||||
const out = { code, stdout: s.out.text, stderr: s.err.text };
|
||||
OUTPUTS.push(out.stdout, out.stderr);
|
||||
return out;
|
||||
},
|
||||
// The same verb in a child process that SIGKILLs itself at `stage`.
|
||||
crash(argv, stage) {
|
||||
const script = `
|
||||
import { run } from ${JSON.stringify(new URL("../src/cli.mjs", import.meta.url).href)};
|
||||
import { io } from ${JSON.stringify(new URL("./helpers.mjs", import.meta.url).href)};
|
||||
const p = JSON.parse(process.env.TK_CRASH);
|
||||
const s = io(process.env);
|
||||
const code = await run(p.argv, s, { system: p.system, now: () => new Date(p.now),
|
||||
triageKeyHook: (stage) => { if (stage === p.stage) process.kill(process.pid, "SIGKILL"); } });
|
||||
process.stdout.write(s.out.text); process.stderr.write(s.err.text); process.exitCode = code;`;
|
||||
const env = f.env({ TK_CRASH: JSON.stringify({ argv, stage, system: f.system, now: f.now }) });
|
||||
return new Promise((resolve) => {
|
||||
const child = spawn(process.execPath, ["--input-type=module", "-e", script], { env, stdio: ["ignore", "pipe", "pipe"] });
|
||||
let stdout = "", stderr = "";
|
||||
child.stdout.on("data", (d) => { stdout += d; });
|
||||
child.stderr.on("data", (d) => { stderr += d; });
|
||||
child.on("exit", (code, signal) => { OUTPUTS.push(stdout, stderr); resolve({ code, signal, stdout, stderr }); });
|
||||
});
|
||||
},
|
||||
record: () => JSON.parse(fs.readFileSync(f.file, "utf8")),
|
||||
triageKeys: () => state.keys.filter((k) => k.startsWith(prefix)),
|
||||
receipts: () => {
|
||||
const file = join(root, "proxy", "mosaic-keys.receipts.jsonl");
|
||||
return fs.existsSync(file) ? fs.readFileSync(file, "utf8").split("\n").filter(Boolean).map((l) => JSON.parse(l)) : [];
|
||||
},
|
||||
fenced: () => fs.existsSync(join(root, "proxy", "mosaic-keys.uncertain")),
|
||||
// No file under the fixture but the key file itself holds a key; neither does any output.
|
||||
scan(alsoKeyFiles = []) {
|
||||
for (const k of state.keys) if (/^mst-.*-[A-Za-z0-9_-]{43}$/.test(k)) SEEN.add(k);
|
||||
for (const p of walk(root, [f.file, join(configDir, "secrets", "proxy-management.key"), ...alsoKeyFiles])) {
|
||||
assert.equal(leaks(fs.readFileSync(p, "latin1")), false, `a key is in ${p}`);
|
||||
}
|
||||
for (const text of OUTPUTS) assert.equal(leaks(text), false, "a key is in an output");
|
||||
},
|
||||
};
|
||||
return f;
|
||||
}
|
||||
|
||||
const ok = (r) => assert.equal(r.code, 0, r.stderr);
|
||||
const refusal = (r, code, words) => {
|
||||
assert.equal(r.code, 3, r.stderr);
|
||||
assert.equal(r.stdout, "");
|
||||
assert.match(r.stderr, new RegExp(`^mosaic: ${code}: ${words ?? ""}`));
|
||||
};
|
||||
|
||||
test("issue adds one key under the lock, writes the 0600 file, and records ids only", async (t) => {
|
||||
const f = await fixture(t);
|
||||
const before = [...f.state.keys];
|
||||
const r = await f.cli(["triage", "key", "issue"]);
|
||||
ok(r);
|
||||
const rec = f.record();
|
||||
assert.deepEqual(Object.keys(rec), ["version", "key", "keyId", "issuedAt", "rotateBy", "retiring"]);
|
||||
assert.match(rec.key, new RegExp(`^${f.prefix}[A-Za-z0-9_-]{43}$`));
|
||||
assert.equal(rec.keyId, id(rec.key));
|
||||
assert.equal(rec.issuedAt, iso(NOW));
|
||||
assert.equal(rec.rotateBy, "2026-12-18");
|
||||
assert.deepEqual(rec.retiring, []);
|
||||
assert.equal(r.stdout, `triage key issued: ${rec.keyId}, rotate by 2026-12-18\nfile: ${f.file}\n`);
|
||||
assert.deepEqual(f.state.keys, [...before, rec.key]);
|
||||
assert.equal(f.state.puts.length, 1);
|
||||
const file = fs.lstatSync(f.file), dir = fs.lstatSync(f.dir);
|
||||
assert.equal(file.mode & 0o777, 0o600); assert.equal(dir.mode & 0o777, 0o700);
|
||||
assert.equal(file.uid, process.getuid()); assert.equal(dir.uid, process.getuid());
|
||||
assert.deepEqual(f.receipts(), [{ op: "triage.key.issue", time: iso(NOW), addedKeyIds: [rec.keyId], removedKeyIds: [] }]);
|
||||
assert.equal(f.fenced(), false);
|
||||
assert.equal(fs.readFileSync(join(f.root, "proxy", "keys.generation"), "utf8"), "1\n");
|
||||
// A second issue refuses and changes nothing; rotate is the verb for that.
|
||||
const again = await f.cli(["triage", "key", "issue"]);
|
||||
refusal(again, "triage-key-issued", "a Triage key is already issued; use mosaic triage key rotate\n$");
|
||||
assert.equal(f.state.puts.length, 1);
|
||||
assert.deepEqual(f.record(), rec);
|
||||
f.scan();
|
||||
});
|
||||
|
||||
test("status reads the file and the list and writes nothing", async (t) => {
|
||||
const f = await fixture(t);
|
||||
const none = await f.cli(["triage", "key", "status"]);
|
||||
ok(none);
|
||||
assert.equal(none.stdout, `triage key: not issued\nfile: ${f.file}\nproxy: read\nnext: mosaic triage key issue\n`);
|
||||
ok(await f.cli(["triage", "key", "issue"]));
|
||||
const rec = f.record(), puts = f.state.puts.length, receipts = f.receipts().length;
|
||||
f.now = NOW + 3 * DAY + 4 * HOUR + 5 * MIN;
|
||||
const shown = await f.cli(["triage", "key", "status"]);
|
||||
ok(shown);
|
||||
assert.equal(shown.stdout, [
|
||||
"triage key: issued", `file: ${f.file}`, `key id: ${rec.keyId}`, `issued: ${iso(NOW)} (age 3d 4h 5m)`,
|
||||
"rotate by: 2026-12-18", "proxy: holds the key", "next: nothing", "",
|
||||
].join("\n"));
|
||||
f.now = Date.parse("2026-12-18T00:00:00.000Z");
|
||||
assert.match((await f.cli(["triage", "key", "status"])).stdout, /rotate by: 2026-12-18 \(due\)\n[^]*next: mosaic triage key rotate\n$/);
|
||||
f.now = NOW + HOUR;
|
||||
f.state.keys = f.state.keys.filter((k) => k !== rec.key);
|
||||
assert.match((await f.cli(["triage", "key", "status"])).stdout, /proxy: does not hold the key\nnext: mosaic triage key rotate\n$/);
|
||||
assert.equal(f.state.puts.length, puts);
|
||||
assert.equal(f.receipts().length, receipts);
|
||||
assert.equal(fs.readdirSync(join(f.root, "proxy")).includes("mosaic-keys.lock"), false);
|
||||
f.scan();
|
||||
});
|
||||
|
||||
test("rotate adds a new key and keeps the old one for the bus grace; retire removes it once due", async (t) => {
|
||||
const f = await fixture(t);
|
||||
refusal(await f.cli(["triage", "key", "rotate"]), "triage-key-not-issued", "no Triage key is issued; use mosaic triage key issue\n$");
|
||||
assert.equal(f.state.puts.length, 0);
|
||||
ok(await f.cli(["triage", "key", "issue"]));
|
||||
const first = f.record();
|
||||
f.now = NOW + HOUR;
|
||||
const r = await f.cli(["triage", "key", "rotate"]);
|
||||
ok(r);
|
||||
const second = f.record();
|
||||
const retireAt = iso(NOW + HOUR + 120 * MIN);
|
||||
assert.notEqual(second.key, first.key);
|
||||
assert.equal(second.issuedAt, iso(NOW + HOUR));
|
||||
assert.deepEqual(second.retiring, [{ key: first.key, keyId: first.keyId, issuedAt: first.issuedAt, retireAt }]);
|
||||
assert.equal(r.stdout, `triage key rotated: ${second.keyId}, rotate by 2026-12-18\nold key ${first.keyId} stays live until mosaic triage key retire runs at or after ${retireAt}\n`);
|
||||
assert.deepEqual(f.triageKeys(), [first.key, second.key]);
|
||||
f.now = NOW + HOUR + 119 * MIN;
|
||||
const early = await f.cli(["triage", "key", "retire"]);
|
||||
ok(early);
|
||||
assert.equal(early.stdout, "triage key retire: nothing to remove\n");
|
||||
assert.equal(f.state.puts.length, 2);
|
||||
assert.match((await f.cli(["triage", "key", "status"])).stdout, new RegExp(`retiring: ${first.keyId}, live until a retire after ${retireAt}\\n[^]*next: nothing\\n$`));
|
||||
f.now = NOW + HOUR + 120 * MIN;
|
||||
assert.match((await f.cli(["triage", "key", "status"])).stdout, new RegExp(`retiring: ${first.keyId}, live until a retire after ${retireAt} \\(due\\)\\n[^]*next: mosaic triage key retire\\n$`));
|
||||
const done = await f.cli(["triage", "key", "retire"]);
|
||||
ok(done);
|
||||
assert.equal(done.stdout, `triage key retire: removed ${first.keyId}\n`);
|
||||
assert.deepEqual(f.triageKeys(), [second.key]);
|
||||
assert.deepEqual(f.record(), { ...second, retiring: [] });
|
||||
assert.deepEqual(f.receipts().map((e) => e.op), ["triage.key.issue", "triage.key.rotate", "triage.key.retire"]);
|
||||
assert.deepEqual(f.receipts()[2].removedKeyIds, [first.keyId]);
|
||||
f.scan();
|
||||
});
|
||||
|
||||
test("rotate takes the grace from proxy.json, where the bus reads it, and drops an old key the proxy lost", async (t) => {
|
||||
const f = await fixture(t);
|
||||
f.system = { ...f.system, proxy: { ...f.system.proxy, rotateGraceMinutes: 30 } };
|
||||
ok(await f.cli(["triage", "key", "issue"]));
|
||||
const first = f.record();
|
||||
ok(await f.cli(["triage", "key", "rotate"]));
|
||||
const second = f.record();
|
||||
assert.equal(second.retiring[0].retireAt, iso(NOW + 30 * MIN));
|
||||
// The proxy lost the current key: rotate drops it at once, as the bus does for an agent key.
|
||||
f.state.keys = f.state.keys.filter((k) => k !== second.key);
|
||||
const r = await f.cli(["triage", "key", "rotate"]);
|
||||
ok(r);
|
||||
assert.match(r.stdout, new RegExp(`old key ${second.keyId} was not in the proxy list; nothing to retire\\n$`));
|
||||
assert.deepEqual(f.record().retiring.map((e) => e.keyId), [first.keyId]);
|
||||
// A retiring key past its time, or no longer in the list, leaves on the next rotate.
|
||||
f.now = NOW + 31 * MIN;
|
||||
ok(await f.cli(["triage", "key", "rotate"]));
|
||||
const last = f.record();
|
||||
assert.deepEqual(last.retiring.map((e) => e.keyId), [id(f.triageKeys()[0])]);
|
||||
assert.equal(f.triageKeys().includes(first.key), false);
|
||||
assert.equal(f.triageKeys().length, 2);
|
||||
f.scan();
|
||||
});
|
||||
|
||||
test("lifetime and grace match the bus's own ModelKeys for the same clock and proxy settings", async (t) => {
|
||||
const runtime = fs.readFileSync(join(REPO, "packages/bus/src/runtime.mjs"), "utf8");
|
||||
const literal = runtime.match(/rotateGraceMinutes: proxy\.rotateGraceMinutes \?\? (\d+)/);
|
||||
assert.ok(literal, "the bus no longer reads rotateGraceMinutes as this test expects");
|
||||
assert.equal(Number(literal[1]), DEFAULT_ROTATE_GRACE_MINUTES);
|
||||
for (const grace of [undefined, 45]) {
|
||||
const f = await fixture(t);
|
||||
if (grace !== undefined) f.system = { ...f.system, proxy: { ...f.system.proxy, rotateGraceMinutes: grace } };
|
||||
ok(await f.cli(["triage", "key", "issue"]));
|
||||
f.now = NOW + 3 * HOUR;
|
||||
ok(await f.cli(["triage", "key", "rotate"]));
|
||||
const ours = f.record();
|
||||
// The bus, driven the way runtime.mjs drives it, on its own in-memory list.
|
||||
let keys = [`msk-${tag(f.configDir)}-sentinel-${b64()}`], record = null, now = NOW;
|
||||
const credentials = { registerModel() {}, modelRecord: () => record, writeModel: (_, r) => { record = r; }, status: () => [] };
|
||||
const client = { getKeys: async () => [...keys], putKeys: async (k) => { keys = [...k]; return { ok: true }; } };
|
||||
const bus = new ModelKeys({ credentials, client, configDir: f.configDir, baseUrl: "http://127.0.0.1:1", now: () => now,
|
||||
...(grace === undefined ? {} : { rotateGraceMinutes: f.system.proxy.rotateGraceMinutes ?? Number(literal[1]) }) });
|
||||
await bus.reconcile();
|
||||
await bus.ensure("demo/x");
|
||||
assert.equal(record.rotateBy, rotateByFor(NOW));
|
||||
now = NOW + 3 * HOUR;
|
||||
await bus.rotate("demo/x");
|
||||
assert.equal(ours.rotateBy, record.rotateBy);
|
||||
assert.equal(ours.retiring[0].retireAt, record.retiring[0].retireAt);
|
||||
assert.equal(ours.retiring[0].retireAt, retireAtFor(now, f.system.proxy));
|
||||
f.scan();
|
||||
}
|
||||
});
|
||||
|
||||
test("an agent caller is refused before anything is read, for every key verb", async (t) => {
|
||||
const f = await fixture(t);
|
||||
for (const marker of AGENT_MARKERS) {
|
||||
for (const verb of VERBS) {
|
||||
const r = await f.cli(["triage", "key", verb], { env: { ...f.env(), [marker]: "1" } });
|
||||
assert.equal(r.code, 3, marker);
|
||||
assert.equal(r.stderr, `mosaic: refused: this runs only from a human shell, outside any agent run (found ${marker} in the environment)\n`);
|
||||
}
|
||||
const saved = process.env[marker];
|
||||
process.env[marker] = "1";
|
||||
try {
|
||||
for (const verb of VERBS) {
|
||||
const r = await f.cli(["triage", "key", verb]);
|
||||
assert.equal(r.code, 3, marker);
|
||||
assert.match(r.stderr, new RegExp(`found ${marker} in the environment`));
|
||||
}
|
||||
} finally { if (saved === undefined) delete process.env[marker]; else process.env[marker] = saved; }
|
||||
}
|
||||
assert.equal(f.state.gets, 0);
|
||||
assert.equal(f.state.puts.length, 0);
|
||||
assert.equal(fs.existsSync(f.dir), false);
|
||||
assert.deepEqual(fs.readdirSync(join(f.root, "proxy")), []);
|
||||
});
|
||||
|
||||
test("usage: unknown key verbs, options and arguments exit 4 and echo no value", async (t) => {
|
||||
const f = await fixture(t);
|
||||
for (const argv of [["triage", "key"], ["triage", "key", "nope"], ["triage", "key", "issue", "extra"], ["triage", "key", "status", "--json"]]) {
|
||||
const r = await f.cli(argv);
|
||||
assert.equal(r.code, 4, argv.join(" "));
|
||||
assert.equal(r.stdout, "");
|
||||
assert.match(r.stderr, /\nusage: mosaic triage route show [^\n]*\| key issue \| rotate \| retire \| status\n$/);
|
||||
}
|
||||
const r = await f.cli(["triage", "key", "rotate", "--key", "secret-value"]);
|
||||
assert.equal(r.code, 4);
|
||||
assert.match(r.stderr, /^mosaic: unknown option\n/);
|
||||
assert.doesNotMatch(r.stderr, /secret-value/);
|
||||
assert.equal(f.state.gets, 0);
|
||||
});
|
||||
|
||||
test("an unsafe key store refuses every verb before any proxy write", async (t) => {
|
||||
const uid = process.getuid();
|
||||
const cases = [
|
||||
["file mode", (f) => fs.chmodSync(f.file, 0o644), {}, "the key file mode is not 0600"],
|
||||
["file owner", () => {}, { triageKeyFileUid: uid + 1 }, "the key file is not owned by this user"],
|
||||
["file link", (f) => { fs.renameSync(f.file, join(f.dir, "real")); fs.symlinkSync(join(f.dir, "real"), f.file); }, {}, "the key file is a link"],
|
||||
["file is a directory", (f) => { fs.rmSync(f.file); fs.mkdirSync(f.file, { mode: 0o700 }); }, {}, "the key file is not a regular file"],
|
||||
["file is a FIFO", (f) => { fs.rmSync(f.file); assert.equal(spawnSync("mkfifo", ["-m", "600", f.file]).status, 0); }, {}, "the key file is not a regular file"],
|
||||
["hard link", (f) => fs.linkSync(f.file, join(f.dir, "second")), {}, "the key file has more than one link"],
|
||||
["not JSON", (f) => fs.writeFileSync(f.file, "{"), {}, "the key file is not JSON"],
|
||||
["extra field", (f) => fs.writeFileSync(f.file, JSON.stringify({ ...f.record(), extra: 1 })), {}, "the key file does not match the key shape"],
|
||||
["another instance's key", (f) => { const r = f.record(); const key = `mst-${tag("/elsewhere")}-triage-${b64()}`; fs.writeFileSync(f.file, JSON.stringify({ ...r, key, keyId: id(key) })); }, {}, "the key file does not match the key shape"],
|
||||
["wrong key id", (f) => fs.writeFileSync(f.file, JSON.stringify({ ...f.record(), keyId: "000000000000" })), {}, "the key file does not match the key shape"],
|
||||
["too large", (f) => fs.writeFileSync(f.file, " ".repeat(64 * 1024 + 1)), {}, "the key file is too large"],
|
||||
["triage directory mode", (f) => fs.chmodSync(f.dir, 0o755), {}, "the triage secrets directory mode is not 0700"],
|
||||
["triage directory link", (f) => { fs.renameSync(f.dir, join(f.configDir, "secrets", "moved")); fs.symlinkSync(join(f.configDir, "secrets", "moved"), f.dir); }, {}, "the triage secrets directory is a link"],
|
||||
["triage directory is a file", (f) => { fs.rmSync(f.dir, { recursive: true }); fs.writeFileSync(f.dir, "", { mode: 0o600 }); }, {}, "the triage secrets directory is not a directory"],
|
||||
["secrets mode", (f) => fs.chmodSync(join(f.configDir, "secrets"), 0o750), {}, "the secrets directory mode is not 0700"],
|
||||
["owner", () => {}, { triageKeyUid: uid + 1 }, "the secrets directory is not owned by this user"],
|
||||
];
|
||||
for (const [name, spoil, deps, words] of cases) {
|
||||
const f = await fixture(t);
|
||||
ok(await f.cli(["triage", "key", "issue"]));
|
||||
spoil(f);
|
||||
const keys = [...f.state.keys], puts = f.state.puts.length;
|
||||
for (const verb of VERBS) {
|
||||
const r = await f.cli(["triage", "key", verb], { deps });
|
||||
refusal(r, "triage-key-unreadable", `the Triage key store is unreadable: ${words.replace(/[.()]/g, "\\$&")}\n$`);
|
||||
assert.equal(f.state.puts.length, puts, `${name}: ${verb}`);
|
||||
}
|
||||
assert.deepEqual(f.state.keys, keys);
|
||||
// The spoils above move or link the key file itself; those copies are the key file.
|
||||
f.scan([join(f.dir, "real"), join(f.dir, "second"), join(f.configDir, "secrets", "moved", "model")]);
|
||||
}
|
||||
// No secrets directory at all: status says not issued; issue refuses before the proxy changes.
|
||||
const f = await fixture(t);
|
||||
fs.rmSync(join(f.configDir, "secrets"), { recursive: true });
|
||||
fs.mkdirSync(join(f.configDir, "secrets"), { mode: 0o700 });
|
||||
fs.writeFileSync(join(f.configDir, "secrets", "proxy-management.key"), MANAGE, { mode: 0o600 });
|
||||
fs.chmodSync(join(f.configDir, "secrets"), 0o755);
|
||||
refusal(await f.cli(["triage", "key", "issue"]), "triage-key-unreadable", "the Triage key store is unreadable: the secrets directory mode is not 0700\n$");
|
||||
assert.equal(f.state.puts.length, 0);
|
||||
});
|
||||
|
||||
test("the instance proxy lock, the fence, the sentinel and the proxy itself each refuse plainly", async (t) => {
|
||||
// Lock held by a live process: busy after the lock's wait, and nothing changes.
|
||||
const f = await fixture(t);
|
||||
const lockFile = join(f.root, "proxy", "mosaic-keys.lock");
|
||||
ok(await f.cli(["triage", "key", "status"]));
|
||||
fs.writeFileSync(lockFile, `${JSON.stringify({ pid: process.pid, token: "a".repeat(32) })}\n`, { mode: 0o600 });
|
||||
const busy = await f.cli(["triage", "key", "issue"]);
|
||||
refusal(busy, "proxy-lock-busy", "another proxy writer holds the instance proxy lock; try again when it ends, and if no writer is running, see mosaic proxy fence status\n$");
|
||||
assert.equal(f.state.gets, 1);
|
||||
// A lock left by a process that is gone is reaped, and the verb goes ahead.
|
||||
const dead = spawnSync(process.execPath, ["-e", ""]).pid;
|
||||
fs.writeFileSync(lockFile, `${JSON.stringify({ pid: dead, token: "b".repeat(32) })}\n`, { mode: 0o600 });
|
||||
ok(await f.cli(["triage", "key", "issue"]));
|
||||
assert.equal(fs.existsSync(lockFile), false);
|
||||
assert.equal(f.triageKeys().length, 1);
|
||||
fs.rmSync(f.file);
|
||||
f.state.keys = f.state.keys.filter((k) => !k.startsWith(f.prefix));
|
||||
// A set fence: every write refuses, status names it and the way out.
|
||||
const putsBefore = f.state.puts.length;
|
||||
fs.writeFileSync(join(f.root, "proxy", "mosaic-keys.uncertain"), "{}\n", { mode: 0o600 });
|
||||
for (const verb of ["issue", "rotate", "retire"]) refusal(await f.cli(["triage", "key", verb]), "proxy-uncertain");
|
||||
assert.match((await f.cli(["triage", "key", "status"])).stdout, /proxy fence: set, an earlier proxy write has an uncertain outcome\nnext: restart the proxy, then mosaic proxy fence clear --proxy-restarted\n$/);
|
||||
assert.equal(f.state.puts.length, putsBefore);
|
||||
// No sentinel for this instance (another instance's does not count).
|
||||
const g = await fixture(t, { sentinel: false });
|
||||
g.state.keys.push(`msk-${tag("/elsewhere")}-sentinel-${b64()}`);
|
||||
refusal(await g.cli(["triage", "key", "issue"]), "triage-key-proxy-open");
|
||||
assert.match((await g.cli(["triage", "key", "status"])).stdout, /proxy: its key list has no sentinel for this instance\nnext: check the proxy with mosaic proxy status\n$/);
|
||||
assert.equal(g.state.puts.length, 0);
|
||||
// No proxy set up, no management key, proxy down.
|
||||
const h = await fixture(t);
|
||||
const bare = { dataRoot: h.root };
|
||||
refusal(await h.cli(["triage", "key", "issue"], { system: bare }), "triage-key-no-proxy");
|
||||
assert.equal((await h.cli(["triage", "key", "status"], { system: bare })).stdout,
|
||||
`triage key: not issued\nfile: ${h.file}\nproxy: not set up for this instance (no proxy.json)\nnext: set up the model proxy first (mosaic proxy init)\n`);
|
||||
fs.chmodSync(join(h.configDir, "secrets", "proxy-management.key"), 0o644);
|
||||
refusal(await h.cli(["triage", "key", "issue"]), "proxy-management-key-invalid");
|
||||
fs.chmodSync(join(h.configDir, "secrets", "proxy-management.key"), 0o600);
|
||||
h.server.closeAllConnections(); await new Promise((resolve) => h.server.close(resolve));
|
||||
h.server.close = (cb) => cb();
|
||||
refusal(await h.cli(["triage", "key", "issue"]), "proxy-client-down");
|
||||
assert.match((await h.cli(["triage", "key", "status"])).stdout, /proxy: not read \(proxy-client-down\)\n/);
|
||||
assert.equal(fs.existsSync(h.file), false);
|
||||
f.scan(); g.scan(); h.scan();
|
||||
});
|
||||
|
||||
test("a write the proxy does not confirm keeps the fence; a list that moves mid-verb is not overwritten", async (t) => {
|
||||
const f = await fixture(t);
|
||||
f.state.ignorePut = true;
|
||||
refusal(await f.cli(["triage", "key", "issue"]), "triage-key-unconfirmed");
|
||||
assert.equal(f.fenced(), true);
|
||||
assert.equal(fs.existsSync(f.file), false);
|
||||
// The store's directory was made and checked before the proxy changed, not after.
|
||||
assert.equal(fs.lstatSync(f.dir).mode & 0o777, 0o700);
|
||||
assert.deepEqual(f.receipts(), []);
|
||||
// A PUT the proxy takes but whose read-back has lost this instance's sentinel is not confirmed.
|
||||
const s = await fixture(t);
|
||||
s.state.dropSentinelOnPut = true;
|
||||
refusal(await s.cli(["triage", "key", "issue"]), "triage-key-unconfirmed");
|
||||
assert.equal(s.fenced(), true);
|
||||
assert.equal(fs.existsSync(s.file), false);
|
||||
s.scan();
|
||||
const g = await fixture(t);
|
||||
g.state.onGet = (n) => { if (n === 1) g.state.keys = [...g.state.keys, "late-foreign-key"]; };
|
||||
refusal(await g.cli(["triage", "key", "issue"]), "triage-key-raced");
|
||||
assert.equal(g.state.puts.length, 0);
|
||||
assert.equal(g.fenced(), false);
|
||||
f.scan(); g.scan();
|
||||
});
|
||||
|
||||
test("keys outside the Triage namespace are never touched; Triage keys the file does not hold are removed", async (t) => {
|
||||
const f = await fixture(t);
|
||||
const agent = `msk-${tag(f.configDir)}-${b64()}`, other = `mst-${tag("/elsewhere")}-triage-${b64()}`;
|
||||
const orphan = `${f.prefix}${b64()}`, odd = `${f.prefix}short`;
|
||||
f.state.keys.push(agent, other, orphan, odd);
|
||||
const foreign = f.state.keys.filter((k) => k !== orphan && k !== odd);
|
||||
const r = await f.cli(["triage", "key", "issue"]);
|
||||
ok(r);
|
||||
// A malformed name under the Triage prefix is still Triage's, as the bus treats its own prefix.
|
||||
assert.match(r.stdout, new RegExp(`removed keys this file did not hold: ${id(orphan)}, ${id(odd)}\\n$`));
|
||||
for (const verb of ["rotate", "retire"]) { f.now += 3 * HOUR; ok(await f.cli(["triage", "key", verb])); }
|
||||
for (const k of foreign) assert.ok(f.state.keys.includes(k), "a key outside the namespace left the list");
|
||||
assert.equal(f.state.keys.includes(orphan), false);
|
||||
assert.equal(f.state.keys.includes(odd), false);
|
||||
assert.deepEqual(f.triageKeys(), [f.record().key]);
|
||||
f.scan();
|
||||
});
|
||||
|
||||
test("crash after the PUT, before the read-back: the fence holds; status says so; clear, then issue recovers", async (t) => {
|
||||
const f = await fixture(t);
|
||||
const crashed = await f.crash(["triage", "key", "issue"], "put");
|
||||
assert.equal(crashed.signal, "SIGKILL");
|
||||
assert.equal(f.fenced(), true);
|
||||
assert.equal(f.triageKeys().length, 1);
|
||||
assert.equal(fs.existsSync(f.file), false);
|
||||
const status = await f.cli(["triage", "key", "status"]);
|
||||
ok(status);
|
||||
assert.match(status.stdout, new RegExp(`proxy fence: set[^\\n]*\\nkeys in the Triage namespace the file does not hold: ${id(f.triageKeys()[0])}\\nnext: restart the proxy, then mosaic proxy fence clear --proxy-restarted\\n$`));
|
||||
refusal(await f.cli(["triage", "key", "issue"]), "proxy-uncertain");
|
||||
ok(await f.cli(["proxy", "fence", "clear", "--proxy-restarted"]));
|
||||
assert.equal(f.fenced(), false);
|
||||
const stray = f.triageKeys()[0];
|
||||
const r = await f.cli(["triage", "key", "issue"]);
|
||||
ok(r);
|
||||
assert.match(r.stdout, new RegExp(`removed keys this file did not hold: ${id(stray)}\\n$`));
|
||||
assert.deepEqual(f.triageKeys(), [f.record().key]);
|
||||
f.scan();
|
||||
});
|
||||
|
||||
test("crash after the read-back, before the file write: no fence; status names the stray key; issue removes it", async (t) => {
|
||||
const f = await fixture(t);
|
||||
assert.equal((await f.crash(["triage", "key", "issue"], "write")).signal, "SIGKILL");
|
||||
assert.equal(f.fenced(), false);
|
||||
assert.equal(fs.existsSync(f.file), false);
|
||||
const [stray] = f.triageKeys();
|
||||
assert.match((await f.cli(["triage", "key", "status"])).stdout,
|
||||
new RegExp(`^triage key: not issued\\n[^]*keys in the Triage namespace the file does not hold: ${id(stray)}\\nnext: mosaic triage key issue\\n$`));
|
||||
ok(await f.cli(["triage", "key", "issue"]));
|
||||
assert.deepEqual(f.triageKeys(), [f.record().key]);
|
||||
// The same crash in rotate: the file keeps the old key, the stray new one is named, retire removes it.
|
||||
const held = f.record();
|
||||
assert.equal((await f.crash(["triage", "key", "rotate"], "write")).signal, "SIGKILL");
|
||||
assert.deepEqual(f.record(), held);
|
||||
const strayNew = f.triageKeys().find((k) => k !== held.key);
|
||||
assert.match((await f.cli(["triage", "key", "status"])).stdout,
|
||||
new RegExp(`proxy: holds the key\\nkeys in the Triage namespace the file does not hold: ${id(strayNew)}\\nnext: mosaic triage key retire\\n$`));
|
||||
const retired = await f.cli(["triage", "key", "retire"]);
|
||||
ok(retired);
|
||||
assert.equal(retired.stdout, `triage key retire: removed ${id(strayNew)}\n`);
|
||||
assert.deepEqual(f.triageKeys(), [held.key]);
|
||||
f.scan();
|
||||
});
|
||||
|
||||
test("file written but the old key not yet retired: status shows it due; a crash inside retire recovers on the next retire", async (t) => {
|
||||
const f = await fixture(t);
|
||||
ok(await f.cli(["triage", "key", "issue"]));
|
||||
const first = f.record();
|
||||
ok(await f.cli(["triage", "key", "rotate"]));
|
||||
f.now = NOW + 121 * MIN;
|
||||
assert.match((await f.cli(["triage", "key", "status"])).stdout, new RegExp(`retiring: ${first.keyId}, live until a retire after [^\\n]* \\(due\\)\\n[^]*next: mosaic triage key retire\\n$`));
|
||||
// retire dies after the proxy dropped the key but before the file was rewritten.
|
||||
assert.equal((await f.crash(["triage", "key", "retire"], "write")).signal, "SIGKILL");
|
||||
assert.equal(f.state.keys.includes(first.key), false);
|
||||
assert.equal(f.record().retiring.length, 1);
|
||||
assert.match((await f.cli(["triage", "key", "status"])).stdout, /\(due\) \(not in the proxy list\)\n[^]*next: mosaic triage key retire\n$/);
|
||||
const puts = f.state.puts.length;
|
||||
const r = await f.cli(["triage", "key", "retire"]);
|
||||
ok(r);
|
||||
assert.equal(f.state.puts.length, puts);
|
||||
assert.deepEqual(f.record().retiring, []);
|
||||
// And a retire killed after its PUT leaves the fence, like any other write.
|
||||
ok(await f.cli(["triage", "key", "rotate"]));
|
||||
f.now += 121 * MIN;
|
||||
assert.equal((await f.crash(["triage", "key", "retire"], "put")).signal, "SIGKILL");
|
||||
assert.equal(f.fenced(), true);
|
||||
refusal(await f.cli(["triage", "key", "retire"]), "proxy-uncertain");
|
||||
ok(await f.cli(["proxy", "fence", "clear", "--proxy-restarted"]));
|
||||
ok(await f.cli(["triage", "key", "retire"]));
|
||||
assert.deepEqual(f.record().retiring, []);
|
||||
assert.deepEqual(f.triageKeys(), [f.record().key]);
|
||||
f.scan();
|
||||
});
|
||||
|
||||
test("a file that cannot be written after the proxy changed is reported with its recovery", async (t) => {
|
||||
const f = await fixture(t);
|
||||
ok(await f.cli(["triage", "key", "status"]));
|
||||
fs.mkdirSync(f.dir, { mode: 0o700 });
|
||||
const r = await f.cli(["triage", "key", "issue"], { deps: { triageKeyHook: (stage) => { if (stage === "write") fs.chmodSync(f.dir, 0o755); } } });
|
||||
refusal(r, "triage-key-unwritten", "the proxy list changed but ");
|
||||
assert.match(r.stderr, /run mosaic triage key issue again, which clears what this run left\n$/);
|
||||
assert.equal(f.triageKeys().length, 1);
|
||||
fs.chmodSync(f.dir, 0o700);
|
||||
ok(await f.cli(["triage", "key", "issue"]));
|
||||
assert.deepEqual(f.triageKeys(), [f.record().key]);
|
||||
f.scan();
|
||||
});
|
||||
|
||||
test("crash after the temp file, before its rename: status counts it; the next verb removes it", async (t) => {
|
||||
const f = await fixture(t);
|
||||
assert.equal((await f.crash(["triage", "key", "issue"], "rename")).signal, "SIGKILL");
|
||||
assert.equal(fs.existsSync(f.file), false);
|
||||
const temps = fs.readdirSync(f.dir).filter((n) => n.endsWith(".tmp"));
|
||||
assert.equal(temps.length, 1);
|
||||
assert.match(temps[0], /^\.model\.\d+\.[0-9a-f]{12}\.tmp$/);
|
||||
// The temp file holds the key the proxy now has; that is why it must go.
|
||||
const [stray] = f.triageKeys();
|
||||
assert.ok(fs.readFileSync(join(f.dir, temps[0]), "utf8").includes(stray));
|
||||
const status = await f.cli(["triage", "key", "status"]);
|
||||
ok(status);
|
||||
assert.match(status.stdout, new RegExp(`leftover temp files in the triage secrets directory: 1 \\(the next issue, rotate or retire removes them\\)\\nkeys in the Triage namespace the file does not hold: ${id(stray)}\\nnext: mosaic triage key issue\\n$`));
|
||||
// Something else in the directory is not a leftover and stays.
|
||||
fs.writeFileSync(join(f.dir, "notes.tmp"), "", { mode: 0o600 });
|
||||
ok(await f.cli(["triage", "key", "issue"]));
|
||||
assert.deepEqual(fs.readdirSync(f.dir).sort(), ["model", "notes.tmp"]);
|
||||
assert.deepEqual(f.triageKeys(), [f.record().key]);
|
||||
// Retire, with nothing else to do, still removes a leftover.
|
||||
assert.equal((await f.crash(["triage", "key", "rotate"], "rename")).signal, "SIGKILL");
|
||||
ok(await f.cli(["triage", "key", "retire"]));
|
||||
assert.deepEqual(fs.readdirSync(f.dir).sort(), ["model", "notes.tmp"]);
|
||||
assert.doesNotMatch((await f.cli(["triage", "key", "status"])).stdout, /leftover/);
|
||||
f.scan();
|
||||
});
|
||||
|
||||
test("the receipts file is checked before the proxy changes; a receipt that fails after is named", async (t) => {
|
||||
const receipts = (f) => join(f.root, "proxy", "mosaic-keys.receipts.jsonl");
|
||||
const spoils = [
|
||||
["mode", (f) => { fs.writeFileSync(receipts(f), "", { mode: 0o644 }); fs.chmodSync(receipts(f), 0o644); }, "the proxy receipts file mode is not 0600"],
|
||||
["link", (f) => { fs.writeFileSync(join(f.root, "elsewhere"), "", { mode: 0o600 }); fs.symlinkSync(join(f.root, "elsewhere"), receipts(f)); }, "the proxy receipts file cannot be opened"],
|
||||
["FIFO", (f) => assert.equal(spawnSync("mkfifo", ["-m", "600", receipts(f)]).status, 0), "the proxy receipts file cannot be opened"],
|
||||
];
|
||||
for (const [name, spoil, words] of spoils) {
|
||||
const f = await fixture(t);
|
||||
spoil(f);
|
||||
refusal(await f.cli(["triage", "key", "issue"]), "triage-key-receipts-unsafe", `${words}; nothing was changed\n$`);
|
||||
assert.equal(f.state.puts.length, 0, name);
|
||||
assert.equal(f.fenced(), false, name);
|
||||
assert.equal(fs.existsSync(f.file), false, name);
|
||||
f.scan();
|
||||
}
|
||||
// Safe before the change, unsafe at the receipt: the change stands and the refusal says so.
|
||||
const f = await fixture(t);
|
||||
const r = await f.cli(["triage", "key", "issue"], { deps: { triageKeyHook: (stage) => { if (stage === "receipt") fs.chmodSync(receipts(f), 0o644); } } });
|
||||
refusal(r, "triage-key-unrecorded", "the change was made but its receipt line was not written; run mosaic triage key status\n$");
|
||||
assert.deepEqual(f.triageKeys(), [f.record().key]);
|
||||
assert.equal(f.fenced(), false);
|
||||
assert.equal(fs.readFileSync(receipts(f), "utf8"), "");
|
||||
f.scan();
|
||||
});
|
||||
|
||||
test("a directory sync that fails after the rename is named, and the receipt is still written", async (t) => {
|
||||
const f = await fixture(t);
|
||||
const r = await f.cli(["triage", "key", "issue"], { deps: { triageKeyHook: (stage) => { if (stage === "sync") throw new Error("EIO"); } } });
|
||||
refusal(r, "triage-key-unsynced", `${f.file.replace(/[.]/g, "\\.")} was written but its directory could not be synced, so it may not survive a power loss; run mosaic triage key status\n$`);
|
||||
const rec = f.record();
|
||||
assert.deepEqual(f.triageKeys(), [rec.key]);
|
||||
assert.deepEqual(f.receipts().map((e) => e.addedKeyIds), [[rec.keyId]]);
|
||||
assert.equal(f.fenced(), false);
|
||||
assert.match((await f.cli(["triage", "key", "status"])).stdout, /proxy: holds the key\nnext: nothing\n$/);
|
||||
f.scan();
|
||||
});
|
||||
|
||||
test("an unexpected error prints fixed words, never the error's own text", async (t) => {
|
||||
const f = await fixture(t);
|
||||
const secret = `mst-${tag(f.configDir)}-triage-${b64()}`;
|
||||
SEEN.add(secret);
|
||||
// Outside the proxy write: the verb's own fallback line.
|
||||
const broken = { dataRoot: f.root, get proxy() { throw new Error(`boom ${secret}`); } };
|
||||
for (const verb of VERBS) {
|
||||
const r = await f.cli(["triage", "key", verb], { system: broken });
|
||||
assert.equal(r.code, 3, verb);
|
||||
assert.equal(r.stderr, "mosaic: triage-key-failed: the key verb stopped on an unexpected error; run mosaic triage key status\n");
|
||||
}
|
||||
// Inside the proxy write: the write's own fixed line, and the fence stays for a human.
|
||||
const r = await f.cli(["triage", "key", "issue"], { deps: { triageKeyHook: (stage) => { if (stage === "put") throw new Error(`boom ${secret}`); } } });
|
||||
refusal(r, "triage-key-failed", "the proxy write failed\n$");
|
||||
assert.equal(f.fenced(), true);
|
||||
f.scan();
|
||||
});
|
||||
|
||||
// The org bus reconciles its own `mso-` namespace at start and a no-org bus its `msk-` one; a
|
||||
// Triage key sits in neither, so both leave it. Stray keys in each namespace show the reconcile ran.
|
||||
test("an org bus that starts and reconciles, and a no-org reconcile, both leave the Triage key alone", async (t) => {
|
||||
const f = await fixture(t);
|
||||
ok(await f.cli(["triage", "key", "issue"]));
|
||||
const triage = f.record().key;
|
||||
const strayOrg = `mso-${tag(f.configDir)}-demo-key-${b64()}`;
|
||||
f.state.keys.push(strayOrg);
|
||||
const rolesDir = rolesCopy(f.root), system = systemFor(f.root), doc = businessDoc(f.root, "demo");
|
||||
doc.plugins = { ...doc.plugins, harness: ["pi"] };
|
||||
const file = writeJson(join(f.configDir, "businesses", "demo.json"), doc);
|
||||
assert.ok(file);
|
||||
const business = loadBusiness("demo", { dir: f.configDir, rolesDir }), resolved = {};
|
||||
for (const instance of Object.keys(business.roles)) resolved[instance] = resolveInstance({ system, business, instance });
|
||||
fs.mkdirSync(join(f.root, "data"), { mode: 0o700 });
|
||||
new Registry(join(f.root, "data"));
|
||||
const bus = await startBroker({ dataRoot: join(f.root, "data"), businesses: { demo: busOrganization(business, resolved) },
|
||||
organization: { dir: f.configDir, rolesDir, system }, proxy: { port: f.port, binary: "/fixture-unused" },
|
||||
org: { id: "demo" }, instanceDataRoot: f.root, instanceConfigDir: f.configDir });
|
||||
f.closers.push(() => bus.close());
|
||||
const orgKey = (await bus.modelPreflight({ business: "demo", role: "chief" })).key;
|
||||
assert.ok(orgKey.startsWith(`mso-${tag(f.configDir)}-demo-key-`));
|
||||
assert.equal(f.state.keys.includes(strayOrg), false, "the org reconcile did not run");
|
||||
assert.ok(f.state.keys.includes(triage), "the org bus removed the Triage key");
|
||||
// A no-org reconcile on the same instance.
|
||||
const strayAgent = `msk-${tag(f.configDir)}-${b64()}`;
|
||||
f.state.keys.push(strayAgent);
|
||||
const client = new ManagementClient({ baseUrl: `http://127.0.0.1:${f.port}`, key: MANAGE });
|
||||
const keys = new ModelKeys({ credentials: { status: () => [], modelRecord: () => null }, client, configDir: f.configDir, baseUrl: `http://127.0.0.1:${f.port}` });
|
||||
await keys.reconcile();
|
||||
assert.equal(f.state.keys.includes(strayAgent), false, "the no-org reconcile did not run");
|
||||
assert.ok(f.state.keys.includes(triage), "the no-org reconcile removed the Triage key");
|
||||
// Triage still works under the shared lock while the org bus runs, and leaves the org key.
|
||||
f.now += HOUR;
|
||||
ok(await f.cli(["triage", "key", "rotate"]));
|
||||
assert.ok(f.state.keys.includes(orgKey));
|
||||
assert.match((await f.cli(["triage", "key", "status"])).stdout, /proxy: holds the key\n/);
|
||||
f.scan();
|
||||
});
|
||||
|
||||
test("scripts/mosaic routes triage key status with the instance's own config directory", (t) => {
|
||||
const home = fs.mkdtempSync(join(tmpdir(), "tkm-"));
|
||||
t.after(() => fs.rmSync(home, { recursive: true, force: true }));
|
||||
const env = humanEnv({ HOME: home, MOSAIC_CONFIG: join(home, ".config/mosaic-tktest/config.json") });
|
||||
const setup = spawnSync(join(REPO, "scripts/bootstrap.sh"), ["--instance", "tktest"], { env, encoding: "utf8" });
|
||||
assert.equal(setup.status, 0, setup.stderr);
|
||||
const r = spawnSync(join(REPO, "scripts/mosaic"), ["triage", "key", "status"], { env, encoding: "utf8", timeout: 60000 });
|
||||
OUTPUTS.push(r.stdout, r.stderr);
|
||||
assert.equal(r.status, 0, r.stderr);
|
||||
assert.equal(r.stdout, `triage key: not issued\nfile: ${join(home, ".config/mosaic-tktest/secrets/triage/model")}\nproxy: not set up for this instance (no proxy.json)\nnext: set up the model proxy first (mosaic proxy init)\n`);
|
||||
const refused = spawnSync(join(REPO, "scripts/mosaic"), ["triage", "key", "issue"], { env: { ...env, CLAUDECODE: "1" }, encoding: "utf8", timeout: 60000 });
|
||||
assert.equal(refused.status, 3);
|
||||
assert.match(refused.stderr, /found CLAUDECODE in the environment/);
|
||||
});
|
||||
|
||||
test("no output of any verb in this file holds any key the stand-in saw", () => {
|
||||
assert.ok(SEEN.size >= 20, `only ${SEEN.size} keys were seen`);
|
||||
assert.ok(OUTPUTS.length >= 100);
|
||||
for (const text of OUTPUTS) assert.equal(leaks(text), false);
|
||||
});
|
||||
@@ -0,0 +1,149 @@
|
||||
// #1873 triage-route-store: `mosaic triage route show|set`, the owner's terminal verbs for the
|
||||
// Triage route file (packages/webui/src/triage-route-store.mjs). Human verbs: an agent run is
|
||||
// refused before anything is read (exit 3), as the other human verbs refuse it. `show` with no
|
||||
// file says the route is not set and exits 0.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { chmodSync, existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { run } from "../src/cli.mjs";
|
||||
import { AGENT_MARKERS } from "../src/transport.mjs";
|
||||
import { routePaths } from "../../webui/src/triage-route-store.mjs";
|
||||
import { humanEnv, io } from "./helpers.mjs";
|
||||
|
||||
const REPO = new URL("../../../", import.meta.url).pathname;
|
||||
const MOSAIC = join(REPO, "scripts/mosaic");
|
||||
const NOW = new Date("2026-11-18T12:00:00.000Z");
|
||||
const SET = ["triage", "route", "set", "--model", "glm-5", "--harness", "claude-code", "--cap", "200000", "--time-zone", "America/Chicago"];
|
||||
|
||||
function dataRoot(t) {
|
||||
const d = mkdtempSync(join(tmpdir(), "tr-"));
|
||||
t.after(() => rmSync(d, { recursive: true, force: true }));
|
||||
return d;
|
||||
}
|
||||
async function cli(argv, d, env = humanEnv()) {
|
||||
const s = io(env);
|
||||
const code = await run(argv, s, { system: { dataRoot: d }, now: () => NOW });
|
||||
return { code, stdout: s.out.text, stderr: s.err.text };
|
||||
}
|
||||
|
||||
test("show with no route says it is not set and exits 0", async (t) => {
|
||||
const d = dataRoot(t);
|
||||
assert.deepEqual(await cli(["triage", "route", "show"], d), { code: 0, stdout: "triage route: not set\n", stderr: "" });
|
||||
assert.equal(existsSync(routePaths(d).dir), false);
|
||||
});
|
||||
|
||||
test("set writes the route; show reads it back", async (t) => {
|
||||
const d = dataRoot(t);
|
||||
const set = await cli(SET, d);
|
||||
assert.equal(set.code, 0, set.stderr);
|
||||
assert.equal(set.stdout, "triage route set: revision 1\n");
|
||||
const shown = await cli(["triage", "route", "show"], d);
|
||||
assert.equal(shown.code, 0);
|
||||
assert.equal(shown.stdout, [
|
||||
"triage route: revision 1",
|
||||
"model: glm-5",
|
||||
"harness: claude-code",
|
||||
"cap: 200000 tokens a day, resets 00:00 America/Chicago, counted since 2026-11-18T12:00:00.000Z",
|
||||
"updated: 2026-11-18T12:00:00.000Z by owner",
|
||||
"",
|
||||
].join("\n"));
|
||||
const doc = JSON.parse(readFileSync(routePaths(d).file, "utf8"));
|
||||
assert.equal(doc.updatedBy, "owner");
|
||||
// --revision makes the write conditional on what show printed.
|
||||
const five = ["triage", "route", "set", "--model", "glm-5", "--harness", "claude-code", "--cap", "5", "--time-zone", "America/Chicago", "--revision", "1"];
|
||||
assert.equal((await cli(five, d)).code, 0);
|
||||
assert.equal(JSON.parse(readFileSync(routePaths(d).file, "utf8")).cap.limit, 5);
|
||||
const stale = await cli([...SET, "--revision", "1"], d);
|
||||
assert.equal(stale.code, 3);
|
||||
assert.match(stale.stderr, /^mosaic: triage-route-stale: /);
|
||||
});
|
||||
|
||||
test("refused input exits 2 or 4 and writes nothing", async (t) => {
|
||||
const d = dataRoot(t);
|
||||
const without = (flag) => { const a = [...SET]; const i = a.indexOf(flag); a.splice(i, 2); return a; };
|
||||
const swap = (flag, value) => { const a = [...SET]; a[a.indexOf(flag) + 1] = value; return a; };
|
||||
const cases = [
|
||||
[without("--model"), 4], [without("--harness"), 4], [without("--cap"), 4], [without("--time-zone"), 4],
|
||||
[swap("--cap", "-1"), 2], [swap("--cap", "1e3"), 2], [swap("--cap", "01"), 2], [swap("--cap", "9007199254740993"), 2],
|
||||
[swap("--time-zone", "Atlantis/Lost"), 2], [swap("--model", "Has Space"), 2],
|
||||
[swap("--harness", "nonesuch"), 2], [swap("--harness", "../pi"), 2],
|
||||
[[...SET, "--revision", "x"], 2], [[...SET, "--revision", "-1"], 2],
|
||||
[[...SET, "--cap", "5"], 4], [[...SET, "--unknown", "1"], 4], [[...SET, "extra"], 4],
|
||||
[["triage", "route", "show", "--model", "x"], 4], [["triage", "route", "show", "extra"], 4],
|
||||
[["triage", "route"], 4], [["triage", "route", "remove"], 4], [["triage"], 4], [["triage", "key"], 4],
|
||||
];
|
||||
for (const [argv, want] of cases) {
|
||||
const r = await cli(argv, d);
|
||||
assert.equal(r.code, want, `${argv.join(" ")}: ${r.stderr}`);
|
||||
assert.equal(r.stdout, "");
|
||||
}
|
||||
// An unknown option is named as one, and its value never reaches the message.
|
||||
const unknown = await cli([...SET, "--unknown", "secret-value"], d);
|
||||
assert.equal(unknown.code, 4);
|
||||
assert.match(unknown.stderr, /^mosaic: unknown option\n/);
|
||||
assert.doesNotMatch(unknown.stderr, /secret-value/);
|
||||
assert.equal(existsSync(routePaths(d).file), false);
|
||||
});
|
||||
|
||||
test("an unreadable store exits 3 on show and on set", async (t) => {
|
||||
const d = dataRoot(t);
|
||||
assert.equal((await cli(SET, d)).code, 0);
|
||||
chmodSync(routePaths(d).file, 0o644);
|
||||
for (const argv of [["triage", "route", "show"], [...SET, "--revision", "1"]]) {
|
||||
const r = await cli(argv, d);
|
||||
assert.equal(r.code, 3);
|
||||
assert.match(r.stderr, /^mosaic: triage-route-unreadable: the Triage route store is unreadable: route\.json mode is not 0600\n$/);
|
||||
}
|
||||
});
|
||||
|
||||
test("an agent caller is refused before anything is read, for show and set", async (t) => {
|
||||
const d = dataRoot(t);
|
||||
for (const marker of AGENT_MARKERS) {
|
||||
for (const argv of [["triage", "route", "show"], SET]) {
|
||||
const r = await cli(argv, d, { ...humanEnv(), [marker]: "1" });
|
||||
assert.equal(r.code, 3, marker);
|
||||
assert.equal(r.stderr, `mosaic: refused: this runs only from a human shell, outside any agent run (found ${marker} in the environment)\n`);
|
||||
}
|
||||
}
|
||||
assert.equal(existsSync(routePaths(d).dir), false);
|
||||
});
|
||||
|
||||
test("scripts/mosaic routes triage to the verb; a marker in the process env refuses", (t) => {
|
||||
const home = mkdtempSync(join(tmpdir(), "trm-"));
|
||||
t.after(() => rmSync(home, { recursive: true, force: true }));
|
||||
const env = humanEnv({ HOME: home, MOSAIC_CONFIG: join(home, ".config/mosaic-trtest/config.json") });
|
||||
const setup = spawnSync(join(REPO, "scripts/bootstrap.sh"), ["--instance", "trtest"], { env, encoding: "utf8" });
|
||||
assert.equal(setup.status, 0, setup.stderr);
|
||||
const mosaic = (args, extra = {}) => spawnSync(MOSAIC, args, { env: { ...env, ...extra }, encoding: "utf8", timeout: 60000 });
|
||||
const shown = mosaic(["triage", "route", "show"]);
|
||||
assert.equal(shown.status, 0, shown.stderr);
|
||||
assert.equal(shown.stdout, "triage route: not set\n");
|
||||
const set = mosaic(SET.slice(0));
|
||||
assert.equal(set.status, 0, set.stderr);
|
||||
assert.equal(set.stdout, "triage route set: revision 1\n");
|
||||
const dataDir = join(home, ".mosaic-trtest");
|
||||
assert.equal(JSON.parse(readFileSync(routePaths(dataDir).file, "utf8")).model, "glm-5");
|
||||
const refused = mosaic(["triage", "route", "show"], { CLAUDECODE: "1" });
|
||||
assert.equal(refused.status, 3);
|
||||
assert.match(refused.stderr, /found CLAUDECODE in the environment/);
|
||||
assert.equal(refused.stdout, "");
|
||||
});
|
||||
|
||||
test("a marker in this process's own environment refuses even when the command's env is clean", async (t) => {
|
||||
const d = dataRoot(t);
|
||||
for (const marker of AGENT_MARKERS) {
|
||||
const saved = process.env[marker];
|
||||
process.env[marker] = "1";
|
||||
try {
|
||||
for (const argv of [["triage", "route", "show"], SET]) {
|
||||
const r = await cli(argv, d, humanEnv());
|
||||
assert.equal(r.code, 3, marker);
|
||||
assert.match(r.stderr, new RegExp(`found ${marker} in the environment`));
|
||||
}
|
||||
} finally { if (saved === undefined) delete process.env[marker]; else process.env[marker] = saved; }
|
||||
}
|
||||
assert.equal(existsSync(routePaths(d).dir), false);
|
||||
});
|
||||
@@ -258,6 +258,28 @@ export returns:
|
||||
`gateTimeout`, `bundle`, `env`, `turnEnd`, `outputPolicy` and `transcript` give its
|
||||
sandbox binds, web, tool names, S0 lines, bundle files, adapter variables,
|
||||
how a turn ends and where its transcripts land.
|
||||
- `envNames` and `installRoots({env})` (#1873 triage-fences) are optional and
|
||||
outside `kinds.mjs`. A session the Console hosts with no organization
|
||||
(Triage) reads them; organization launches do not. `envNames` lists the
|
||||
names `env()` returns, less the provider's. The host takes a route's
|
||||
`harnessEnv` name only when it is listed here and is not one the host
|
||||
always refuses: `MOSAIC_*`, `NODE_*`, `LD_*`, `DYLD_*`, `PYTHON*`, `BASH_*`,
|
||||
`GIT_*`, `MALLOC_*`, `NPM_CONFIG_*`, `XDG_*`, any name ending in `_DIR`,
|
||||
`_CONF`, `_CONFIG`, `_ASKPASS`, `_CA_BUNDLE` or `_BASE_URL`, PATH, HOME,
|
||||
TMPDIR, ENV, the founder's and the provider's names, and named shell,
|
||||
proxy, resolver, libc, OpenSSL and Java hooks, Perl and Ruby start
|
||||
options and library paths, less's input filters, Lua's start code,
|
||||
curl's config home and a TLS key log. The full list is
|
||||
`REFUSED_NAMES` in `packages/webui/src/triage-host.mjs`. `installRoots` lists the directories
|
||||
where the adapter and the mounts live. A hosted adapter or mount source
|
||||
must resolve inside one of them, and never in or around the data root,
|
||||
the config directory, HOME, `/srv/mosaic` or a path with a `secret`
|
||||
segment. HOME does not count against a source inside the stack's own
|
||||
code, so a copy or a worktree under HOME still passes. Every claude-code
|
||||
name is `MOSAIC_*`, so a hosted session cannot run Claude Code yet. pi's
|
||||
`MOSAIC_TOOLS`, `MOSAIC_EXTENSIONS` and `MOSAIC_THINKING` are refused the
|
||||
same way, so a hosted pi gets no tools and no extension until the host
|
||||
sets those names itself.
|
||||
- `compactions({store, cursors, budget, streams, first})` reads a thread
|
||||
session's native store for row 270's compaction count (row 307). Core keeps
|
||||
the ledger (`packages/seat/src/compactions.mjs`) and passes `store` as
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
// session.json. packages/plugins/README.md lists the interface.
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { accessSync, constants, realpathSync, statSync } from "node:fs";
|
||||
import { delimiter, isAbsolute, join } from "node:path";
|
||||
import { delimiter, dirname, isAbsolute, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { claudeBuiltins, MCP_PREFIX } from "../../../harness/src/gate.mjs";
|
||||
import { hasSessionCredential, SESSION_CREDENTIALS } from "../../../seat/src/session.mjs";
|
||||
@@ -145,6 +145,16 @@ export function env({ files, policy, session, proxy = null }) {
|
||||
};
|
||||
}
|
||||
|
||||
// #1873 triage-fences: the names env() returns, less the proxy's, and the directories its adapter
|
||||
// and mounts live in: the pinned binary's when MOSAIC_CLAUDE_BIN is set. A hosted session's host
|
||||
// (Triage) takes harnessEnv names and mount sources only from these; launches don't read them.
|
||||
export const envNames = Object.freeze(["MOSAIC_TOOLS", "MOSAIC_CLAUDE_SETTINGS", "MOSAIC_CLAUDE_MCP_CONFIG", "MOSAIC_THINKING", "MOSAIC_MODEL_ACTIVITY_SOURCE"]);
|
||||
export function installRoots({ env = {} } = {}) {
|
||||
const roots = [join(REPO, "adapters", "claude"), join(REPO, "tools", "unslop-hook")];
|
||||
if (env.MOSAIC_CLAUDE_BIN) roots.push(dirname(realpathSync(env.MOSAIC_CLAUDE_BIN)));
|
||||
return roots;
|
||||
}
|
||||
|
||||
// A turn ends when claude exits; there is no marker.
|
||||
export const turnEnd = () => "exit";
|
||||
export const transcript = ({ runDir, store = null }) => (store ? store.projects : join(runDir, "home", ".claude", "projects"));
|
||||
|
||||
@@ -118,6 +118,14 @@ export function env({ policy, tools, session, proxy = null }) {
|
||||
};
|
||||
}
|
||||
|
||||
// #1873 triage-fences: the names env() returns, less the provider key, and the directories its
|
||||
// adapter and mounts live in. A hosted session's host (Triage) takes harnessEnv names and mount
|
||||
// sources only from these; launches don't read them. The host refuses every MOSAIC_* name from a
|
||||
// route, so MOSAIC_TOOLS, MOSAIC_EXTENSIONS and MOSAIC_THINKING are listed only to say what env()
|
||||
// hands a session: a hosted pi needs the host to set them itself (a later slice).
|
||||
export const envNames = Object.freeze(["PI_OFFLINE", "MOSAIC_TOOLS", "MOSAIC_EXTENSIONS", "PI_PROVIDER", "PI_MODEL", "MOSAIC_THINKING"]);
|
||||
export const installRoots = () => [join(REPO, "adapters", "pi"), join(REPO, "packages", "plugins", "harness", "pi"), realpathSync(join(REPO, "node_modules")), join(REPO, "tools", "unslop-hook")];
|
||||
|
||||
// The extension writes the turn marker at agent_end; an exit without it is a failed turn.
|
||||
export const turnEnd = () => "marker";
|
||||
export const transcript = ({ runDir, store = null }) => (store ? store.session : join(runDir, "session"));
|
||||
|
||||
@@ -34,7 +34,7 @@
|
||||
// the launch (sandbox-unavailable).
|
||||
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { existsSync, lstatSync, mkdirSync, readlinkSync, realpathSync } from "node:fs";
|
||||
import { existsSync, lstatSync, mkdirSync, readlinkSync, realpathSync, statSync } from "node:fs";
|
||||
import { dirname, isAbsolute, join, relative, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { ID_PATTERN } from "../../organization/src/vocabulary.mjs";
|
||||
@@ -193,6 +193,117 @@ export function sandboxArgv(list, runDir, command, { business, role, bwrap = BWR
|
||||
return [bwrap, ...NAMESPACES, ...argsOf(list), "--remount-ro", "/", "--setenv", "HOME", join(runDir, "home"), "--setenv", "TMPDIR", "/tmp", ...identity, "--chdir", runDir, "--", ...KILL_WITH_PARENT, ...command];
|
||||
}
|
||||
|
||||
// #1873 triage-fences: what a model route may hand a hosted session, checked before the host
|
||||
// writes anything and again in hostedBinds. The adapter and each mount's source must be a clean
|
||||
// absolute path whose real path lies under one of the harness module's install roots, named under
|
||||
// that same root, and outside every protected path: the data root, the config directory and HOME
|
||||
// the caller passes, the run directory, ALWAYS_PROTECTED, and any path with a segment that
|
||||
// contains "secret" in any case. A protected path counts in its own spelling and its real path, and
|
||||
// a source may neither lie inside one nor contain one. The bind's source is the real path, so a
|
||||
// link swapped after the check cannot steer it. A mount's dest (its source when it names none) is
|
||||
// never /tmp, the run directory or a system bind's dest, nor inside or around one, never a parent
|
||||
// of or equal to a runner file, the adapter or node, and never in or around another mount's dest.
|
||||
// At most HOSTED_MOUNTS_MAX mounts. A refusal is a HostedRouteError naming its fence, never a path.
|
||||
// `codeRoot` is the stack's own code and `codeHolder` the one protected path that may hold it:
|
||||
// HOME, when the stack runs from a copy or a worktree under it. That path does not refuse a source
|
||||
// whose named and real paths both lie in the code root. Every other check still applies to it, and
|
||||
// no other protected path, the run directory or ALWAYS_PROTECTED is ever set aside. A code root in
|
||||
// or around the run directory or ALWAYS_PROTECTED, in either path's spelling or real path, is refused.
|
||||
export const HOSTED_MOUNTS_MAX = 8;
|
||||
const ROOTS_MAX = 16;
|
||||
const PATH_MAX = 4096;
|
||||
const ALWAYS_PROTECTED = Object.freeze(["/srv/mosaic", "/proc", "/sys", "/dev", "/run", "/root", "/boot"]);
|
||||
const MOUNT_KEYS = Object.freeze(["src", "dest", "why"]);
|
||||
|
||||
export class HostedRouteError extends Error {
|
||||
constructor(fence, message) {
|
||||
super(`hosted binds: ${message}`);
|
||||
this.code = "hosted-binds-refused";
|
||||
this.fence = fence;
|
||||
}
|
||||
}
|
||||
const refuseRoute = (fence, message) => { throw new HostedRouteError(fence, message); };
|
||||
|
||||
// Absolute, already normal (no "..", ".", "//" or trailing slash), not "/", no NUL.
|
||||
const cleanPath = (p) => typeof p === "string" && p.length > 1 && p.length <= PATH_MAX && !p.includes("\0") && isAbsolute(p) && resolve(p) === p;
|
||||
const within = (child, parent) => child === parent || child.startsWith(parent + "/");
|
||||
const overlaps = (a, b) => within(a, b) || within(b, a);
|
||||
const secretPath = (p) => p.split("/").some((seg) => /secret/i.test(seg));
|
||||
const realOr = (p) => { try { return realpathSync(p); } catch { return p; } };
|
||||
const plainObject = (o) => o !== null && typeof o === "object" && !Array.isArray(o) && [Object.prototype, null].includes(Object.getPrototypeOf(o));
|
||||
|
||||
// `alsoFixed` is a test hook: it adds stand-in fixed paths and never removes a shipped one.
|
||||
export function checkHostedRoute({ adapter, harnessMounts, installRoots, protectedPaths, codeRoot = null, codeHolder = null, runDir, node = process.execPath, alsoFixed = [] }) {
|
||||
if (!cleanPath(runDir)) refuseRoute("protected", "the run directory is not a clean absolute path");
|
||||
if (!Array.isArray(protectedPaths)) refuseRoute("protected", "no protected paths were given");
|
||||
if (!Array.isArray(alsoFixed)) refuseRoute("protected", "the added fixed paths are not a list");
|
||||
// Lists are copied once, so a Proxy cannot answer one length to a check and another to the map.
|
||||
protectedPaths = Array.from(protectedPaths);
|
||||
// Each guarded entry: its spellings, and whether a source in the code root is clear of it. The
|
||||
// fixed paths and the run dir never hold the code. Their spellings are looked up once, here, and
|
||||
// serve both the code-root check and their guards, so the two always see the same real path.
|
||||
const fixed = [...ALWAYS_PROTECTED, ...Array.from(alsoFixed), runDir].map((p) => {
|
||||
if (!cleanPath(p)) refuseRoute("protected", "a protected path is not a clean absolute path");
|
||||
return { spellings: [p, realOr(p)], holdsCode: false };
|
||||
});
|
||||
let code = null;
|
||||
if (codeRoot !== null) {
|
||||
if (!cleanPath(codeRoot)) refuseRoute("protected", "the code root is not a clean absolute path");
|
||||
if (!protectedPaths.includes(codeHolder)) refuseRoute("protected", "the code root's holder is not a protected path");
|
||||
code = [codeRoot, realOr(codeRoot)];
|
||||
if (code.some((c) => fixed.some((g) => g.spellings.some((s) => overlaps(c, s))))) refuseRoute("protected", "the code root is in or around a path that is never set aside");
|
||||
}
|
||||
const guarded = [...fixed];
|
||||
for (const p of protectedPaths) {
|
||||
if (!cleanPath(p)) refuseRoute("protected", "a protected path is not a clean absolute path");
|
||||
const spellings = [p, realOr(p)];
|
||||
const holdsCode = p === codeHolder && code !== null && spellings.some((g) => code.some((c) => c !== g && within(c, g)));
|
||||
guarded.push({ spellings, holdsCode });
|
||||
}
|
||||
if (!Array.isArray(installRoots)) refuseRoute("roots", "the harness module declares no usable install roots");
|
||||
installRoots = Array.from(installRoots);
|
||||
if (installRoots.length === 0 || installRoots.length > ROOTS_MAX) refuseRoute("roots", "the harness module declares no usable install roots");
|
||||
const roots = installRoots.map((r) => {
|
||||
if (!cleanPath(r)) refuseRoute("roots", "an install root is not a clean absolute path");
|
||||
let real;
|
||||
try { real = realpathSync(r); } catch { refuseRoute("roots", "an install root is missing"); }
|
||||
if (real === "/" || !statSync(real).isDirectory()) refuseRoute("roots", "an install root is not a directory");
|
||||
return { named: r, real };
|
||||
});
|
||||
const source = (src, fence, what) => {
|
||||
if (!cleanPath(src)) refuseRoute(fence, `${what} is not a clean absolute path`);
|
||||
let real;
|
||||
try { real = realpathSync(src); } catch { refuseRoute(fence, `${what} is missing`); }
|
||||
if (!roots.some((r) => (within(src, r.named) || within(src, r.real)) && within(real, r.real))) refuseRoute(fence, `${what} is not under an install root`);
|
||||
const inCode = code !== null && code.some((c) => within(src, c)) && within(real, code[1]);
|
||||
if (guarded.some((g) => !(g.holdsCode && inCode) && [src, real].some((p) => g.spellings.some((s) => overlaps(p, s))))) refuseRoute(fence, `${what} is in or around a protected path`);
|
||||
if ([src, real].some(secretPath)) refuseRoute(fence, `${what} is in a secrets path`);
|
||||
return real;
|
||||
};
|
||||
const adapterReal = source(adapter, "adapter", "the adapter");
|
||||
if (!statSync(adapterReal).isFile()) refuseRoute("adapter", "the adapter is not a file");
|
||||
if (!Array.isArray(harnessMounts)) refuseRoute("mounts", `a hosted session takes at most ${HOSTED_MOUNTS_MAX} harness mounts`);
|
||||
harnessMounts = Array.from(harnessMounts);
|
||||
if (harnessMounts.length > HOSTED_MOUNTS_MAX) refuseRoute("mounts", `a hosted session takes at most ${HOSTED_MOUNTS_MAX} harness mounts`);
|
||||
const apart = ["/tmp", runDir, ...systemBinds(node).map((b) => b.dest ?? b.src)];
|
||||
const kept = [...RUNNER_FILES, adapter, node];
|
||||
const dests = [];
|
||||
const mounts = harnessMounts.map((m, i) => {
|
||||
const what = `harness mount ${i + 1}`;
|
||||
if (!plainObject(m) || Object.keys(m).some((k) => !MOUNT_KEYS.includes(k))) refuseRoute("mounts", `${what} is not { src, dest, why }`);
|
||||
// Each field is read once, so a getter cannot answer one value to a check and another to the bind.
|
||||
const { src, dest: named, why } = m;
|
||||
if (typeof why !== "string" || !why || why.length > 256) refuseRoute("mounts", `${what} is not { src, dest, why }`);
|
||||
if (named !== undefined && !cleanPath(named)) refuseRoute("mount-dest", `${what}'s dest is not a clean absolute path`);
|
||||
const real = source(src, "mount-src", what);
|
||||
const dest = named ?? src;
|
||||
if (apart.some((p) => overlaps(dest, p)) || kept.some((p) => within(p, dest)) || dests.some((d) => overlaps(dest, d))) refuseRoute("mount-dest", `${what}'s dest would cover a path the session needs`);
|
||||
dests.push(dest);
|
||||
return { src: real, dest, why };
|
||||
});
|
||||
return { adapter: { src: adapterReal, dest: adapter }, mounts };
|
||||
}
|
||||
|
||||
// #1873 triage-host: the binds for a session the Console hosts in no organization (Triage). It
|
||||
// gets the system part, the runner's files, the adapter and the harness module's mounts like any
|
||||
// session, and from the data root exactly two directories: its own `runDir` read-write, and
|
||||
@@ -200,7 +311,9 @@ export function sandboxArgv(list, runDir, command, { business, role, bwrap = BWR
|
||||
// calls through stay read-only inside it, and its tmp/ is /tmp. There is no workspace, broker,
|
||||
// launch or git socket, thread store or org root. Each source inside runDir must be what the host
|
||||
// just made there, never a link the session left: bwrap resolves a bind's source on the host.
|
||||
export function hostedBinds({ runDir, healthDir, adapter, harnessMounts = [], relaySocket, node = process.execPath, readOnly = [] }) {
|
||||
// The adapter and the mounts pass checkHostedRoute against `installRoots`, `protectedPaths` and
|
||||
// `codeRoot` with its `codeHolder` here too; without install roots nothing binds.
|
||||
export function hostedBinds({ runDir, healthDir, adapter, harnessMounts = [], installRoots, protectedPaths, codeRoot = null, codeHolder = null, relaySocket, node = process.execPath, readOnly = [] }) {
|
||||
if (relaySocket !== join(runDir, "relay.sock")) throw new Error("hosted binds: the relay socket must be the run directory's relay.sock");
|
||||
const healthDest = join(runDir, "work", "health");
|
||||
for (const p of [runDir, healthDir, join(runDir, "bundle"), join(runDir, "tmp"), join(runDir, "work"), healthDest]) {
|
||||
@@ -213,10 +326,11 @@ export function hostedBinds({ runDir, healthDir, adapter, harnessMounts = [], re
|
||||
try { st = lstatSync(p); } catch { throw new Error(`hosted binds: ${relative(runDir, p)} is missing`); }
|
||||
if (!ok(st)) throw new Error(`hosted binds: ${relative(runDir, p)} is not what the host made`);
|
||||
}
|
||||
const route = checkHostedRoute({ adapter, harnessMounts, installRoots, protectedPaths, codeRoot, codeHolder, runDir, node });
|
||||
const out = systemBinds(node);
|
||||
for (const f of RUNNER_FILES) out.push({ kind: "ro", src: f, why: "a repository file the runner loads" });
|
||||
out.push({ kind: "ro", src: adapter, why: "the harness adapter the runner starts each turn" });
|
||||
for (const m of harnessMounts) out.push({ kind: "ro", src: m.src, ...(m.dest ? { dest: m.dest } : {}), why: m.why });
|
||||
out.push({ kind: "ro", src: route.adapter.src, ...(route.adapter.src !== adapter ? { dest: adapter } : {}), why: "the harness adapter the runner starts each turn" });
|
||||
for (const m of route.mounts) out.push({ kind: "ro", src: m.src, ...(m.dest !== m.src ? { dest: m.dest } : {}), why: m.why });
|
||||
for (const f of readOnly) out.push({ kind: "ro", src: f, why: "a test fixture" });
|
||||
out.push(
|
||||
{ kind: "rw", src: runDir, why: "the session's own state: turns/, session/, work/, tools.sock, turn.done, home/, tmp/" },
|
||||
|
||||
@@ -5,8 +5,8 @@ import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { createServer } from "node:net";
|
||||
import { mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { RUNNER_FILES, hostedArgv, hostedBinds } from "../src/sandbox.mjs";
|
||||
import { dirname, join } from "node:path";
|
||||
import { HOSTED_MOUNTS_MAX, RUNNER_FILES, checkHostedRoute, hostedArgv, hostedBinds } from "../src/sandbox.mjs";
|
||||
|
||||
function layout(t) {
|
||||
const root = mkdtempSync(join(process.env.TMPDIR, "hb-"));
|
||||
@@ -15,19 +15,21 @@ function layout(t) {
|
||||
const healthDir = join(dataRoot, "triage", "health");
|
||||
for (const d of [join(dataRoot, "orgs"), join(dataRoot, "triage", "host"), healthDir, join(runDir, "bundle"), join(runDir, "tmp"), join(runDir, "work", "health")]) mkdirSync(d, { recursive: true });
|
||||
writeFileSync(join(runDir, "session.json"), "{}");
|
||||
const adapter = join(root, "adapter.sh");
|
||||
const install = join(root, "install");
|
||||
mkdirSync(install);
|
||||
const adapter = join(install, "adapter.sh");
|
||||
writeFileSync(adapter, "exit 0\n");
|
||||
return new Promise((resolve) => {
|
||||
const server = createServer().listen(join(runDir, "relay.sock"), () => {
|
||||
t.after(() => server.close());
|
||||
resolve({ root, dataRoot, runDir, healthDir, adapter, relaySocket: join(runDir, "relay.sock") });
|
||||
resolve({ root, dataRoot, runDir, healthDir, adapter, install, relaySocket: join(runDir, "relay.sock"), installRoots: [install], protectedPaths: [dataRoot] });
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
test("hosted binds hold the run directory and the health directory of the data root, and nothing else of it", async (t) => {
|
||||
const p = await layout(t);
|
||||
const list = hostedBinds({ runDir: p.runDir, healthDir: p.healthDir, adapter: p.adapter, relaySocket: p.relaySocket });
|
||||
const list = hostedBinds({ runDir: p.runDir, healthDir: p.healthDir, adapter: p.adapter, relaySocket: p.relaySocket, installRoots: p.installRoots, protectedPaths: p.protectedPaths });
|
||||
const fromData = list.filter((b) => typeof b.src === "string" && b.src.startsWith(p.dataRoot + "/"));
|
||||
assert.deepEqual(fromData.map((b) => [b.kind, b.src, b.dest ?? null]), [
|
||||
["rw", p.runDir, null],
|
||||
@@ -46,7 +48,7 @@ test("hosted binds hold the run directory and the health directory of the data r
|
||||
|
||||
test("hosted binds refuse a relay socket elsewhere, and any source in the run directory that the host did not make", async (t) => {
|
||||
const p = await layout(t);
|
||||
const base = { runDir: p.runDir, healthDir: p.healthDir, adapter: p.adapter, relaySocket: p.relaySocket };
|
||||
const base = { runDir: p.runDir, healthDir: p.healthDir, adapter: p.adapter, relaySocket: p.relaySocket, installRoots: p.installRoots, protectedPaths: p.protectedPaths };
|
||||
assert.throws(() => hostedBinds({ ...base, relaySocket: join(p.root, "bus.sock") }), /relay socket must be the run directory's relay\.sock/);
|
||||
// A link in place of each directory the binds read: bwrap would follow it on the host.
|
||||
for (const d of ["bundle", "tmp", "work"]) {
|
||||
@@ -72,7 +74,7 @@ test("hosted binds refuse a relay socket elsewhere, and any source in the run di
|
||||
|
||||
test("the hosted command line sets no git identity and runs the command under the namespaces", async (t) => {
|
||||
const p = await layout(t);
|
||||
const argv = hostedArgv(hostedBinds({ runDir: p.runDir, healthDir: p.healthDir, adapter: p.adapter, relaySocket: p.relaySocket }), p.runDir, ["node", "runner.mjs", p.runDir], { bwrap: "/b/bwrap" });
|
||||
const argv = hostedArgv(hostedBinds({ runDir: p.runDir, healthDir: p.healthDir, adapter: p.adapter, relaySocket: p.relaySocket, installRoots: p.installRoots, protectedPaths: p.protectedPaths }), p.runDir, ["node", "runner.mjs", p.runDir], { bwrap: "/b/bwrap" });
|
||||
assert.equal(argv[0], "/b/bwrap");
|
||||
assert.deepEqual(argv.slice(1, 5), ["--unshare-user", "--unshare-pid", "--unshare-ipc", "--as-pid-1"]);
|
||||
assert.deepEqual(argv.slice(-3), ["node", "runner.mjs", p.runDir]);
|
||||
@@ -81,3 +83,163 @@ test("the hosted command line sets no git identity and runs the command under th
|
||||
assert.equal(argv[home + 1], join(p.runDir, "home"));
|
||||
assert.ok(argv.includes("--remount-ro"));
|
||||
});
|
||||
|
||||
// #1873 triage-fences: hostedBinds checks the adapter and the mounts again, so a caller that skips
|
||||
// the host's own check still binds nothing outside the install roots.
|
||||
test("hosted binds refuse without install roots or protected paths, and refuse an adapter or a mount source outside the roots or in a protected path", async (t) => {
|
||||
const p = await layout(t);
|
||||
const base = { runDir: p.runDir, healthDir: p.healthDir, adapter: p.adapter, relaySocket: p.relaySocket, installRoots: p.installRoots, protectedPaths: p.protectedPaths };
|
||||
const refused = (opts, fence, label) => assert.throws(() => hostedBinds({ ...base, ...opts }), (e) => e.code === "hosted-binds-refused" && e.fence === fence, label);
|
||||
refused({ installRoots: undefined }, "roots", "no roots");
|
||||
refused({ installRoots: [] }, "roots", "empty roots");
|
||||
refused({ installRoots: ["/"] }, "roots", "root /");
|
||||
refused({ installRoots: [join(p.root, "absent")] }, "roots", "missing root");
|
||||
refused({ installRoots: [p.adapter] }, "roots", "a file as root");
|
||||
refused({ protectedPaths: undefined }, "protected", "no protected paths");
|
||||
refused({ protectedPaths: ["relative"] }, "protected", "relative protected path");
|
||||
writeFileSync(join(p.root, "out.sh"), "x");
|
||||
refused({ adapter: join(p.root, "out.sh") }, "adapter", "adapter outside");
|
||||
symlinkSync(join(p.root, "out.sh"), join(p.install, "link.sh"));
|
||||
refused({ adapter: join(p.install, "link.sh") }, "adapter", "adapter link out");
|
||||
refused({ adapter: p.install }, "adapter", "adapter is a directory");
|
||||
refused({ harnessMounts: [{ src: join(p.install, "link.sh"), why: "w" }] }, "mount-src", "mount link out");
|
||||
refused({ harnessMounts: [{ src: join(p.dataRoot, "orgs"), why: "w" }], installRoots: [p.root] }, "mount-src", "mount in the data root");
|
||||
refused({ harnessMounts: [{ src: p.root, why: "w" }], installRoots: [p.root] }, "mount-src", "mount around the data root");
|
||||
mkdirSync(join(p.install, "secrets"));
|
||||
writeFileSync(join(p.install, "secrets", "k"), "x");
|
||||
refused({ harnessMounts: [{ src: join(p.install, "secrets", "k"), why: "w" }] }, "mount-src", "mount in secrets");
|
||||
writeFileSync(join(p.install, "m"), "x");
|
||||
refused({ harnessMounts: Array.from({ length: HOSTED_MOUNTS_MAX + 1 }, (_, i) => ({ src: join(p.install, "m"), dest: `/opt/m${i}`, why: "w" })) }, "mounts", "too many");
|
||||
refused({ harnessMounts: [{ src: join(p.install, "m"), dest: "/tmp/m", why: "w" }] }, "mount-dest", "dest under /tmp");
|
||||
refused({ harnessMounts: [{ src: join(p.install, "m"), dest: p.runDir, why: "w" }] }, "mount-dest", "dest is the run dir");
|
||||
refused({ harnessMounts: [{ src: join(p.install, "m"), dest: join(p.runDir, "session.json"), why: "w" }] }, "mount-dest", "dest is session.json");
|
||||
refused({ harnessMounts: [{ src: join(p.install, "m"), dest: "/usr", why: "w" }] }, "mount-dest", "dest is a system bind");
|
||||
refused({ harnessMounts: [{ src: join(p.install, "m"), dest: dirname(RUNNER_FILES[0]), why: "w" }] }, "mount-dest", "dest holds a runner file");
|
||||
refused({ harnessMounts: [{ src: join(p.install, "m"), why: "w", kind: "rw" }] }, "mounts", "a kind of its own");
|
||||
});
|
||||
|
||||
test("hosted binds bind the real file of a link that stays inside the install root, at the path the route named", async (t) => {
|
||||
const p = await layout(t);
|
||||
writeFileSync(join(p.install, "real.mjs"), "x");
|
||||
symlinkSync(join(p.install, "real.mjs"), join(p.install, "named.mjs"));
|
||||
symlinkSync(p.adapter, join(p.install, "adapter-link.sh"));
|
||||
const list = hostedBinds({
|
||||
runDir: p.runDir, healthDir: p.healthDir, adapter: join(p.install, "adapter-link.sh"), relaySocket: p.relaySocket, installRoots: p.installRoots, protectedPaths: p.protectedPaths,
|
||||
harnessMounts: [{ src: join(p.install, "named.mjs"), why: "w" }, { src: join(p.install, "real.mjs"), dest: "/opt/real.mjs", why: "w2" }],
|
||||
});
|
||||
assert.ok(list.some((b) => b.kind === "ro" && b.src === p.adapter && b.dest === join(p.install, "adapter-link.sh")));
|
||||
assert.ok(list.some((b) => b.kind === "ro" && b.src === join(p.install, "real.mjs") && b.dest === join(p.install, "named.mjs") && b.why === "w"));
|
||||
assert.ok(list.some((b) => b.kind === "ro" && b.src === join(p.install, "real.mjs") && b.dest === "/opt/real.mjs" && b.why === "w2"));
|
||||
assert.ok(!list.some((b) => b.src === join(p.install, "named.mjs")));
|
||||
});
|
||||
|
||||
test("a protected path that holds the code root (HOME over a copy or a worktree) does not refuse a source inside the code root; everything else still holds", async (t) => {
|
||||
const p = await layout(t);
|
||||
// home/ holds the code root code/, which holds the install root.
|
||||
const home = join(p.root, "home");
|
||||
const code = join(home, "code");
|
||||
const inst = join(code, "inst");
|
||||
mkdirSync(join(inst, "secrets"), { recursive: true });
|
||||
for (const f of ["adapter.sh", "m", join("secrets", "k")]) writeFileSync(join(inst, f), "x");
|
||||
writeFileSync(join(home, "outside"), "x");
|
||||
mkdirSync(join(code, "data"));
|
||||
const base = { runDir: p.runDir, healthDir: p.healthDir, adapter: join(inst, "adapter.sh"), relaySocket: p.relaySocket, installRoots: [inst], protectedPaths: [p.dataRoot, home], codeRoot: code, codeHolder: home };
|
||||
const refused = (opts, fence, label) => assert.throws(() => hostedBinds({ ...base, ...opts }), (e) => e.code === "hosted-binds-refused" && e.fence === fence, label);
|
||||
const list = hostedBinds({ ...base, harnessMounts: [{ src: join(inst, "m"), dest: "/opt/m", why: "w" }] });
|
||||
assert.ok(list.some((b) => b.src === join(inst, "adapter.sh")) && list.some((b) => b.src === join(inst, "m") && b.dest === "/opt/m"));
|
||||
// Without the code root, HOME refuses the same adapter.
|
||||
refused({ codeRoot: null }, "adapter", "no code root");
|
||||
// A code root that is HOME itself sets nothing aside.
|
||||
refused({ codeRoot: home }, "adapter", "code root is HOME");
|
||||
// A source under HOME outside the code root, even inside an install root.
|
||||
refused({ installRoots: [home], adapter: join(home, "outside") }, "adapter", "outside the code root");
|
||||
// A link inside the code root to a file outside it.
|
||||
symlinkSync(join(home, "outside"), join(inst, "out"));
|
||||
refused({ installRoots: [inst, home], harnessMounts: [{ src: join(inst, "out"), why: "w" }] }, "mount-src", "link out of the code root");
|
||||
// A protected path inside the code root still refuses.
|
||||
writeFileSync(join(code, "data", "f"), "x");
|
||||
refused({ installRoots: [code], protectedPaths: [p.dataRoot, home, join(code, "data")], harnessMounts: [{ src: join(code, "data", "f"), why: "w" }] }, "mount-src", "protected path in the code root");
|
||||
// The secrets check still holds inside the code root.
|
||||
refused({ harnessMounts: [{ src: join(inst, "secrets", "k"), why: "w" }] }, "mount-src", "secrets in the code root");
|
||||
// The run directory and the fixed list are never set aside, and a code root around them is refused.
|
||||
refused({ codeRoot: p.root, installRoots: [p.root], harnessMounts: [{ src: join(p.dataRoot, "orgs"), why: "w" }] }, "protected", "code root holds the run dir");
|
||||
refused({ codeRoot: "/srv/mosaic/x" }, "protected", "code root in a fixed path");
|
||||
refused({ codeRoot: "relative" }, "protected", "relative code root");
|
||||
// Only the named holder sets the code aside: another protected path that holds the code still refuses.
|
||||
refused({ protectedPaths: [p.dataRoot, home, p.root] }, "adapter", "a second protected path holds the code");
|
||||
refused({ codeHolder: null }, "protected", "no holder");
|
||||
refused({ codeHolder: join(p.root, "elsewhere") }, "protected", "a holder that is not a protected path");
|
||||
// A run directory named through a link: a code root in its real path is refused before any check.
|
||||
const realRun = join(home, "run");
|
||||
mkdirSync(join(realRun, "code", "inst"), { recursive: true });
|
||||
writeFileSync(join(realRun, "code", "inst", "a.sh"), "x");
|
||||
symlinkSync(realRun, join(p.root, "runlink"));
|
||||
assert.throws(() => checkHostedRoute({ adapter: join(realRun, "code", "inst", "a.sh"), harnessMounts: [], installRoots: [join(realRun, "code", "inst")], protectedPaths: [p.dataRoot, home], codeRoot: join(realRun, "code"), codeHolder: home, runDir: join(p.root, "runlink") }), (e) => e.fence === "protected", "code root in the run dir's real path");
|
||||
});
|
||||
|
||||
test("a fixed path named through a link: a code root in its real path is refused before any guard, and a source there is refused by its guard", async (t) => {
|
||||
const p = await layout(t);
|
||||
// None of the shipped fixed paths is a link on a test host, so the test hook adds a stand-in.
|
||||
const home = join(p.root, "home");
|
||||
// The real path lies outside HOME and the data root, so only the stand-in's own guard can refuse
|
||||
// a source there.
|
||||
const real = join(p.root, "fixedreal");
|
||||
const inst = join(real, "code", "inst");
|
||||
mkdirSync(inst, { recursive: true });
|
||||
writeFileSync(join(inst, "a.sh"), "x");
|
||||
const link = join(p.root, "fixedlink");
|
||||
symlinkSync(real, link);
|
||||
const base = { adapter: join(inst, "a.sh"), harnessMounts: [], installRoots: [inst], protectedPaths: [p.dataRoot, home], codeHolder: home, runDir: p.runDir, alsoFixed: [link] };
|
||||
const refused = (opts, fence, message, label) => assert.throws(() => checkHostedRoute({ ...base, ...opts }), (e) => e.code === "hosted-binds-refused" && e.fence === fence && message.test(e.message), label);
|
||||
// The code root lies in the stand-in's real path, not under the name the list gives.
|
||||
refused({ codeRoot: join(real, "code") }, "protected", /never set aside/, "code root in a fixed path's real path");
|
||||
// With the code root in HOME, the source in the stand-in's real path is refused by the stand-in's
|
||||
// guard, which holds the same real spelling. No other guard holds that path.
|
||||
mkdirSync(join(home, "code"), { recursive: true });
|
||||
refused({ codeRoot: join(home, "code") }, "adapter", /protected path/, "source in a fixed path's real path");
|
||||
// The same source passes without the stand-in: only the stand-in refused it.
|
||||
assert.equal(checkHostedRoute({ ...base, alsoFixed: [], codeRoot: join(home, "code") }).adapter.src, join(inst, "a.sh"));
|
||||
// The hook adds fixed paths and never removes one, and takes only a list of clean absolute paths.
|
||||
refused({ alsoFixed: [], codeRoot: "/srv/mosaic/x" }, "protected", /never set aside/, "shipped fixed path with an empty hook");
|
||||
refused({ alsoFixed: ["relative"], codeRoot: join(real, "code") }, "protected", /clean absolute/, "relative stand-in");
|
||||
refused({ alsoFixed: "/x", codeRoot: join(real, "code") }, "protected", /clean absolute|list/, "a hook that is not a list");
|
||||
});
|
||||
|
||||
test("a list that answers one length to the check and another later is read once", async (t) => {
|
||||
const p = await layout(t);
|
||||
const items = [];
|
||||
for (let i = 0; i <= HOSTED_MOUNTS_MAX; i++) { writeFileSync(join(p.install, `m${i}`), "x"); items.push({ src: join(p.install, `m${i}`), dest: `/opt/m${i}`, why: "w" }); }
|
||||
let reads = 0;
|
||||
const mounts = new Proxy(items, { get: (o, k) => (k === "length" ? (reads++ === 0 ? 1 : o.length) : o[k]) });
|
||||
assert.throws(() => checkHostedRoute({ adapter: p.adapter, harnessMounts: mounts, installRoots: p.installRoots, protectedPaths: p.protectedPaths, runDir: p.runDir }), (e) => e.fence === "mounts");
|
||||
// Install roots: two short answers pass the count, then a long one would add roots the count never saw.
|
||||
const roots = [];
|
||||
for (let i = 0; i <= 16; i++) { roots.push(join(p.root, `r${i}`)); mkdirSync(roots[i]); }
|
||||
writeFileSync(join(roots[16], "a.sh"), "x");
|
||||
let rootReads = 0;
|
||||
const rootList = new Proxy(roots, { get: (o, k) => (k === "length" ? (rootReads++ < 2 ? 1 : o.length) : o[k]) });
|
||||
assert.throws(() => checkHostedRoute({ adapter: join(roots[16], "a.sh"), harnessMounts: [], installRoots: rootList, protectedPaths: p.protectedPaths, runDir: p.runDir }), (e) => e.fence === "adapter");
|
||||
// Protected paths: the holder check sees HOME, then a shorter answer would leave HOME unguarded.
|
||||
const home = join(p.root, "home");
|
||||
mkdirSync(join(home, "code"), { recursive: true });
|
||||
writeFileSync(join(home, "f"), "x");
|
||||
let guardReads = 0;
|
||||
const guards = new Proxy([p.dataRoot, home], { get: (o, k) => (k === "length" ? (guardReads++ === 0 ? 2 : 1) : o[k]) });
|
||||
assert.throws(() => checkHostedRoute({ adapter: p.adapter, harnessMounts: [{ src: join(home, "f"), dest: "/opt/f", why: "w" }], installRoots: [p.install, home], protectedPaths: guards, codeRoot: join(home, "code"), codeHolder: home, runDir: p.runDir }), (e) => e.fence === "protected" || e.fence === "mount-src");
|
||||
});
|
||||
|
||||
test("each mount field is read once, and no mount's dest lies in or around another's", async (t) => {
|
||||
const p = await layout(t);
|
||||
const base = { runDir: p.runDir, healthDir: p.healthDir, adapter: p.adapter, relaySocket: p.relaySocket, installRoots: p.installRoots, protectedPaths: p.protectedPaths };
|
||||
writeFileSync(join(p.install, "m"), "x");
|
||||
writeFileSync(join(p.install, "n"), "x");
|
||||
let reads = 0;
|
||||
const list = hostedBinds({ ...base, harnessMounts: [{ src: join(p.install, "m"), why: "w", get dest() { return reads++ === 0 ? undefined : "/opt/../tmp"; } }] });
|
||||
assert.equal(reads, 1);
|
||||
assert.ok(list.some((b) => b.src === join(p.install, "m") && b.dest === undefined));
|
||||
assert.ok(!list.some((b) => b.dest === "/opt/../tmp"));
|
||||
for (const [a, b] of [["/opt/x", "/opt/x/f"], ["/opt/x/f", "/opt/x"], ["/opt/x", "/opt/x"]]) {
|
||||
assert.throws(() => hostedBinds({ ...base, harnessMounts: [{ src: join(p.install, "m"), dest: a, why: "w" }, { src: join(p.install, "n"), dest: b, why: "w" }] }), (e) => e.fence === "mount-dest", `${a} then ${b}`);
|
||||
}
|
||||
assert.equal(hostedBinds({ ...base, harnessMounts: [{ src: join(p.install, "m"), dest: "/opt/x", why: "w" }, { src: join(p.install, "n"), dest: "/opt/xy", why: "w" }] }).filter((b) => b.dest?.startsWith("/opt/")).length, 2);
|
||||
});
|
||||
@@ -39,7 +39,7 @@ import { composeSnapshot, readSystemConfig, readOrgConfig } from './triage-snaps
|
||||
import { busesStopped } from '../../cli/src/proxy-admin-host.mjs';
|
||||
import { agentLevelsWriteRoutes } from './agent-levels-write.mjs';
|
||||
import { RESTART_EXIT } from '../../cli/src/org-console.mjs';
|
||||
import { ID_PATTERN } from '../../organization/src/vocabulary.mjs';
|
||||
import { ID_PATTERN, systemOnly } from '../../organization/src/vocabulary.mjs';
|
||||
import { selectorScope } from '../../organization/src/selector.mjs';
|
||||
// The bus verifies the Console's exact same-tree script in /proc, whose argv preserves a
|
||||
// relative shell argument even though Node's process.argv resolves it. Re-exec the same
|
||||
@@ -204,8 +204,9 @@ try {
|
||||
// and the preview are reads over requestRead.
|
||||
templatesWrite: session.user === undefined && sendOn ? templatesCaller(consoleCall({ dataRoot: root, socket: paths.socket, business: id })) : null,
|
||||
// Starter packs SP6 (#1873): the owner's workflow read, the organization file and its revision as the
|
||||
// writer reads them (the read `mosaic org workflow set` takes its revision from), and the V2 template names.
|
||||
workflowsRead: session.user === undefined ? () => ({ ...organizationWriter({ dir: configDir(), rolesDir: rolesDir() }).read(id), templates: templateHashes(rolesDir()).map(t => t.template) }) : null,
|
||||
// writer reads them (the read `mosaic org workflow set` takes its revision from), and the V2 template names
|
||||
// less the system-only ones. packs-http's workflowsRead drops them again and is the tested gate.
|
||||
workflowsRead: session.user === undefined ? () => ({ ...organizationWriter({ dir: configDir(), rolesDir: rolesDir() }).read(id), templates: templateHashes(rolesDir()).map(t => t.template).filter(t => !systemOnly(t)) }) : null,
|
||||
reads: consoleReads({ root: REPO, queueRoot: options.checkout ?? REPO, system, configDir: configDir(), rolesDir: rolesDir(), business: id, orgRoot: root }),
|
||||
};
|
||||
},
|
||||
|
||||
@@ -24,6 +24,8 @@
|
||||
// console cap alone, and runs the merge, the digest and the held-credential check itself. `workflows`
|
||||
// (SP6) is the owner's read of the organization file as the writer reads it ({document, revision,
|
||||
// templates}), the same read `mosaic org workflow set` takes its revision from; no bus verb answers it.
|
||||
import { systemOnly } from '../../organization/src/vocabulary.mjs';
|
||||
|
||||
export const PACKS_ROUTE = /^\/api\/(?:packs(?:\/(preview|apply|hire))?|board|workflows(?:\/([^/]+))?)$/;
|
||||
// Which requests the pack route answers. GET and HEAD /api/board stay the Board read (the Console's own task
|
||||
// board through its upstream); every other method on /api/board is this route's (POST writes, the rest 405).
|
||||
@@ -171,7 +173,8 @@ async function workflowsRead(res, { workflows, live, session, writable }) {
|
||||
}
|
||||
const roles = {
|
||||
definitions: Object.keys(plain(doc.definitions) ? doc.definitions : {}).sort(),
|
||||
templates: current.templates.filter(n => typeof n === 'string').sort(),
|
||||
// #1873 triage-followups: a system-only template (organization vocabulary SYSTEM_ONLY) is never offered.
|
||||
templates: current.templates.filter(n => typeof n === 'string' && !systemOnly(n)).sort(),
|
||||
};
|
||||
return json(res, 200, { workflows: flowsOf(doc), projects, roles, revision: current.revision, writable });
|
||||
}
|
||||
@@ -193,7 +193,7 @@
|
||||
catch { return { unknown: true }; }
|
||||
if (res.type === 'opaqueredirect') return { signedOut: true };
|
||||
try { result = await res.json(); } catch { return { unknown: true }; }
|
||||
return { ok: res.ok, result };
|
||||
return { ok: res.ok, status: res.status, result };
|
||||
}
|
||||
// The request's closing read, or a reason the chain stops.
|
||||
async function closing(request, org) {
|
||||
@@ -215,31 +215,39 @@
|
||||
return e ? (Array.isArray(e.running) ? e.running.length : 0) : 0;
|
||||
} catch { return null; }
|
||||
}
|
||||
async function restartAgent(name, thread) {
|
||||
const org = window.MosaicOrg?.id ?? null;
|
||||
const r = restarts[name] = { busy: true, text: `Terminating ${name}…` };
|
||||
const paint = () => paintRestart(name, org);
|
||||
const stop = text => { r.busy = false; r.text = text; paint(); };
|
||||
paint();
|
||||
const asked = await send(pin(org, '/api/agent/terminate'), { instance: name, reason: 'route change' });
|
||||
if (asked.signedOut) return stop('Sign in again, then retry.');
|
||||
if (asked.unknown || !asked.ok || !REQUEST_ID.test(asked.result?.request ?? '')) return stop(asked.ok || asked.unknown ? UNKNOWN : refused(asked.result));
|
||||
// SP19 (#1873): the agent drawer's Restart runs this same chain. `org` is the agent's own organization
|
||||
// (on #/agents the page has none), `reason` goes with the terminate, `text(step, detail)` may give the
|
||||
// words for a step (nothing keeps the route-change words below), and `onStatus({ step, busy, text })`
|
||||
// takes each line in place of the Routing receipt's button. The chain never regenerates.
|
||||
async function restartAgent(name, thread, { org = window.MosaicOrg?.id ?? null, reason = 'route change', text, onStatus } = {}) {
|
||||
const r = { busy: true, text: '' };
|
||||
if (!onStatus) restarts[name] = r;
|
||||
const paint = step => onStatus ? onStatus({ step, busy: r.busy, text: r.text }) : paintRestart(name, org);
|
||||
const words = (step, detail, own) => (typeof text === 'function' ? text(step, detail) : undefined) ?? own;
|
||||
const say = (step, detail, own) => { r.text = words(step, detail, own); paint(step); };
|
||||
const stop = (step, detail, own) => { r.busy = false; say(step, detail, own); };
|
||||
say('ending', {}, `Terminating ${name}…`);
|
||||
const asked = await send(pin(org, '/api/agent/terminate'), { instance: name, reason });
|
||||
if (asked.signedOut) return stop('signedOut', {}, 'Sign in again, then retry.');
|
||||
if (asked.unknown || !asked.ok || !REQUEST_ID.test(asked.result?.request ?? '')) return asked.ok || asked.unknown ? stop('endUnknown', {}, UNKNOWN) : stop('endRefused', { status: asked.status, result: asked.result }, refused(asked.result));
|
||||
const ended = await closing(asked.result.request, org);
|
||||
if (ended.stop) return stop(ended.stop);
|
||||
if (ended.done.reason) return stop(`${name}: ${ended.done.reason}. It was not restarted.`);
|
||||
r.text = `Terminated ${name}. Waiting for its session to end…`; paint();
|
||||
if (ended.stop) return stop('endOpen', { stop: ended.stop }, ended.stop);
|
||||
if (ended.done.reason) return stop('endClosed', { reason: ended.done.reason }, `${name}: ${ended.done.reason}. It was not restarted.`);
|
||||
say('ended', {}, `Terminated ${name}. Waiting for its session to end…`);
|
||||
let left = false;
|
||||
for (let i = 0; i < RESTART_WAIT && !left; i++) { await new Promise(res => setTimeout(res, POLL_MS)); left = (await runs(name, org)) === 0; }
|
||||
if (!left) return stop(`${name} was asked to end but its session is still running. Launch it from Agents once it has ended.`);
|
||||
if (thread) return stop(`Terminated ${name} at ${new Date().toLocaleTimeString()}. Its next message starts it on the new route.`);
|
||||
r.text = `Launching ${name}…`; paint();
|
||||
if (!left) return stop('stillRunning', {}, `${name} was asked to end but its session is still running. Launch it from Agents once it has ended.`);
|
||||
const time = new Date().toLocaleTimeString();
|
||||
if (thread) return stop('threadEnded', { time }, `Terminated ${name} at ${time}. Its next message starts it on the new route.`);
|
||||
say('launching', {}, `Launching ${name}…`);
|
||||
const launch = await send(pin(org, '/api/agent/launch'), { instance: name });
|
||||
if (launch.signedOut) return stop('Sign in again. It was terminated and not launched.');
|
||||
if (launch.unknown || !launch.ok || !REQUEST_ID.test(launch.result?.request ?? '')) return stop(launch.ok || launch.unknown ? `${UNKNOWN} It was terminated.` : `${refused(launch.result)} It was terminated and not launched.`);
|
||||
if (launch.signedOut) return stop('launchSignedOut', {}, 'Sign in again. It was terminated and not launched.');
|
||||
if (launch.unknown || !launch.ok || !REQUEST_ID.test(launch.result?.request ?? '')) return launch.ok || launch.unknown ? stop('launchUnknown', {}, `${UNKNOWN} It was terminated.`) : stop('launchRefused', { status: launch.status, result: launch.result }, `${refused(launch.result)} It was terminated and not launched.`);
|
||||
const started = await closing(launch.result.request, org);
|
||||
if (started.stop) return stop(started.stop);
|
||||
if (started.done.reason) return stop(`${name}: ${started.done.reason}. It was terminated and not launched.`);
|
||||
return stop(`Restarted ${name} on the new route${started.done.done?.run ? ` (run ${started.done.done.run})` : ''}.`);
|
||||
if (started.stop) return stop('launchOpen', { stop: started.stop }, started.stop);
|
||||
if (started.done.reason) return stop('launchClosed', { reason: started.done.reason }, `${name}: ${started.done.reason}. It was terminated and not launched.`);
|
||||
const run = started.done.done?.run ?? null;
|
||||
return stop('restarted', { run }, `Restarted ${name} on the new route${run ? ` (run ${run})` : ''}.`);
|
||||
}
|
||||
document.addEventListener('click', e => {
|
||||
const b = e.target.closest?.('[data-agent-restart]');
|
||||
@@ -247,7 +255,7 @@
|
||||
restartAgent(b.dataset.agentRestart, b.dataset.agentThread === 'yes');
|
||||
});
|
||||
|
||||
window.MosaicAgents = Object.freeze({ section, restartMarkup, isThread: definition => threadDefinitions.has(definition) });
|
||||
window.MosaicAgents = Object.freeze({ section, restartMarkup, restart: restartAgent, isThread: definition => threadDefinitions.has(definition) });
|
||||
})();
|
||||
|
||||
/* #1873 A2: Settings > Agents, the root and project agent levels. bus.js awaits section() for the
|
||||
|
||||
@@ -2246,5 +2246,9 @@
|
||||
window.addEventListener('hashchange', e => render(!agentOnly(e.oldURL, e.newURL)));
|
||||
$('refresh').addEventListener('click', () => render(false));
|
||||
$('pause').addEventListener('click', () => setTimeout(schedule)); // after app.js flips aria-pressed
|
||||
render(false).then(() => { if (!$('bus-view').hidden) { const h1 = $('bus-view').querySelector('h1'); if (h1 && location.hash.startsWith('#/')) h1.focus(); } });
|
||||
// #1873 triage-followups: the first render ends by focusing the page's heading. A drawer or dialog the
|
||||
// reader opened before then keeps its focus: the Triage drawer, an agent drawer, the narrow-screen
|
||||
// menu or any open dialog.
|
||||
const holdsFocus = () => { const a = document.activeElement; return !!a && (!!a.closest('.ws-drawer:not([hidden]), [data-ws-drawer], dialog[open], [role="dialog"], [role="alertdialog"]') || (drawerOpen && nav.contains(a))); };
|
||||
render(false).then(() => { if (!$('bus-view').hidden && !holdsFocus()) { const h1 = $('bus-view').querySelector('h1'); if (h1 && location.hash.startsWith('#/')) h1.focus(); } });
|
||||
})();
|
||||
@@ -43,6 +43,7 @@ window.MosaicLocalTime = value => Number.isFinite(Date.parse(value)) ? new Date(
|
||||
x.addEventListener('click', close);
|
||||
hd.append(title, x);
|
||||
const state = el('p', 'triage-state'); state.id = 'triage-state'; state.setAttribute('role', 'status');
|
||||
const usage = el('p', 'note triage-usage'); usage.id = 'triage-usage'; usage.hidden = true;
|
||||
const healthSec = el('details', 'triage-health-sec'); healthSec.open = true;
|
||||
const healthSum = el('summary', '', 'Health');
|
||||
const health = el('ul', 'triage-health'); health.id = 'triage-health';
|
||||
@@ -51,10 +52,10 @@ window.MosaicLocalTime = value => Number.isFinite(Date.parse(value)) ? new Date(
|
||||
const transcript = el('ol', 'triage-transcript'); transcript.id = 'triage-transcript'; transcript.tabIndex = 0;
|
||||
transcript.setAttribute('aria-label', 'Triage conversation');
|
||||
const compose = el('div', 'triage-compose');
|
||||
drawer.append(hd, state, healthSec, convo, transcript, compose);
|
||||
drawer.append(hd, state, usage, healthSec, convo, transcript, compose);
|
||||
drawer.addEventListener('keydown', e => { if (e.key === 'Escape') { e.preventDefault(); close(); } });
|
||||
document.body.append(drawer);
|
||||
parts = { title, state, health, transcript, compose, box: null, send: null, status: null, off: null };
|
||||
parts = { title, state, usage, health, transcript, compose, box: null, send: null, status: null, off: null };
|
||||
}
|
||||
|
||||
// The message box is built once and never rewritten or removed by a read, so a draft survives every
|
||||
@@ -109,6 +110,19 @@ window.MosaicLocalTime = value => Number.isFinite(Date.parse(value)) ? new Date(
|
||||
};
|
||||
|
||||
const notRead = v => v === 'not read' || v === null || v === undefined || typeof v !== 'object';
|
||||
// #1873 triage-cap: one line, today's count against the stored cap and the next reset. Only what the
|
||||
// read holds: a field the read says is not set or not read is said so, never filled in.
|
||||
const usageLine = view => {
|
||||
if (!Object.hasOwn(view, 'usageToday')) return '';
|
||||
const u = view.usageToday, c = view.cap;
|
||||
if (u === 'not set' || c === 'not set') return 'Daily token cap: not set.';
|
||||
if (notRead(u) || notRead(c)) return "Today's token usage: not read.";
|
||||
const n = v => (Number.isSafeInteger(v) ? v.toLocaleString() : 'not read');
|
||||
const reset = typeof view.resetsAt === 'string' && Number.isFinite(Date.parse(view.resetsAt)) ? ` Resets at ${when(view.resetsAt)}.` : '';
|
||||
const k = Number.isSafeInteger(u.unknownTurns) && u.unknownTurns > 0 ? u.unknownTurns : 0;
|
||||
const unknown = k ? ` ${k} ${k === 1 ? 'turn has' : 'turns have'} no usage reading.` : '';
|
||||
return `Today: ${n(u.count)} of ${n(c.limit)} tokens.${reset}${unknown}`;
|
||||
};
|
||||
const configWords = c => (notRead(c) ? 'not read' : c.valid === true ? 'valid' : `not valid${c.code ? ` (${clean(c.code)})` : ''}`);
|
||||
function healthLines(s) {
|
||||
if (notRead(s)) return ['Health: not read'];
|
||||
@@ -173,6 +187,9 @@ window.MosaicLocalTime = value => Number.isFinite(Date.parse(value)) ? new Date(
|
||||
const t = parts.transcript, atBottom = t.scrollHeight - t.scrollTop - t.clientHeight < 24, top = t.scrollTop;
|
||||
parts.state.textContent = stateLine(view);
|
||||
parts.state.dataset.kind = view.route !== 'set' ? 'off' : view.overCapUntil ? 'cap' : view.running === true ? 'running' : 'stopped';
|
||||
const usage = usageLine(view);
|
||||
parts.usage.textContent = usage;
|
||||
parts.usage.hidden = !usage;
|
||||
const lines = healthLines(view.snapshot);
|
||||
const healthKey = JSON.stringify(lines);
|
||||
if (healthKey !== lastHealth) { lastHealth = healthKey; parts.health.replaceChildren(...lines.map(l => el('li', l.endsWith('not read') ? 'not-read' : '', l))); }
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
[data-packs],[data-workflows]{margin-top:16px}
|
||||
[data-packs]{margin-top:16px}
|
||||
.pack-list,.pack-hires{list-style:none;margin:8px 0;padding:0}
|
||||
.pack-row,.pack-hire{border-top:1px solid var(--border);padding:12px 0;overflow-wrap:anywhere}
|
||||
.pack-preview{border:1px solid var(--border);border-radius:var(--r);padding:12px;margin:12px 0;overflow-wrap:anywhere}
|
||||
|
||||
@@ -631,6 +631,14 @@ html[data-organizations] .ws-orgcard .at{grid-column:3;grid-row:1;font:500 11.5p
|
||||
html[data-organizations] .ws-orgcard .why{grid-column:1/-1;font-size:12px;color:var(--muted)}
|
||||
html[data-organizations] .ws-orgcard.off a{border-style:dashed}
|
||||
html[data-organizations] .ws-home-split{margin-top:4px}
|
||||
/* SP18: the Dashboard's columns line up with its tile row. Three equal tracks with the tiles' gap, the left
|
||||
column spanning two, end on a tile edge whether the row holds six tiles (after the fourth) or three
|
||||
(after the second). Below 760 px the split is one column. */
|
||||
html[data-organizations] .ws-home>.ws-tiles{gap:var(--gap)}
|
||||
@media (min-width:760px){
|
||||
html[data-organizations] .ws-home-split{grid-template-columns:repeat(3,minmax(0,1fr));gap:var(--gap)}
|
||||
html[data-organizations] .ws-home-split>:first-child{grid-column:span 2}
|
||||
}
|
||||
html[data-organizations] .ws-trail{list-style:none;margin:0;padding:0;border:1px solid var(--line);border-radius:8px;background:var(--surface)}
|
||||
html[data-organizations] .ws-trail li{display:grid;grid-template-columns:48px 20px minmax(0,1fr);gap:10px;padding:7px 12px;border-bottom:1px solid color-mix(in srgb,var(--line) 60%,transparent);font-size:13px;align-items:center}
|
||||
html[data-organizations] .ws-trail li:last-child{border-bottom:0}
|
||||
@@ -770,6 +778,11 @@ html[data-organizations] .ws-drawer-acts .note{flex-basis:100%}
|
||||
@media (min-width:1200px){html[data-organizations] body:has(.ws-drawer) .content{padding-right:calc(420px + 22px)}}
|
||||
@media (max-width:700px){
|
||||
html[data-organizations] .ws-drawer{top:0;left:0;right:0;width:auto;border-left:0;box-shadow:none}
|
||||
/* SP18: the sheet covers the screen, so the page under it is held to one screen and does not scroll; it
|
||||
kept its own height before, and the sheet showed over a long blank area. workspace.js puts the page's
|
||||
place back on Close. */
|
||||
html[data-organizations]:has(#bus-view .ws-drawer){overflow:hidden}
|
||||
html[data-organizations]:has(#bus-view .ws-drawer) body{height:100dvh;overflow:hidden}
|
||||
}
|
||||
@media (max-width:1100px){
|
||||
html[data-organizations] .ws-tiles.six{grid-template-columns:repeat(3,minmax(0,1fr))}
|
||||
@@ -819,6 +832,7 @@ html[data-organizations] .ws-drawer-acts .note{flex-basis:100%}
|
||||
#triage-drawer h2:focus{outline:none}
|
||||
#triage-drawer .x{margin-left:auto;flex:none;width:32px;height:32px;border:1px solid var(--line);border-radius:6px;background:var(--canvas);color:var(--strong);font-size:20px;line-height:1;cursor:pointer}
|
||||
#triage-drawer .triage-state{margin:0;padding:10px 14px;flex:none;font-size:14px;font-weight:600;color:var(--strong);border-bottom:1px solid var(--line);overflow-wrap:anywhere}
|
||||
#triage-drawer .triage-usage{margin:0;padding:6px 14px;flex:none;font-size:13px;color:var(--muted);border-bottom:1px solid var(--line);overflow-wrap:anywhere}
|
||||
#triage-drawer .triage-state[data-kind=off],#triage-drawer .triage-state[data-kind=cap]{color:var(--warning);background:color-mix(in srgb,var(--warning) 9%,var(--surface))}
|
||||
#triage-drawer .triage-state[data-kind=running]::before{content:"";display:inline-block;width:8px;height:8px;margin-right:7px;border-radius:50%;background:var(--success);vertical-align:1px}
|
||||
#triage-drawer .triage-health-sec{flex:none;padding:8px 14px;border-bottom:1px solid var(--line)}
|
||||
|
||||
@@ -40,8 +40,8 @@
|
||||
return value;
|
||||
}
|
||||
const plain = v => v !== null && typeof v === 'object' && !Array.isArray(v);
|
||||
// #1873: the templates no organization may hold (organization vocabulary SYSTEM_ONLY). The read may still
|
||||
// list them; a step role never resolves to one and the picker never offers one.
|
||||
// #1873: the templates no organization may hold (organization vocabulary SYSTEM_ONLY). The server leaves
|
||||
// them out of the read; this filter stays so a step role never resolves to one and the picker never offers one.
|
||||
const SYSTEM_ONLY = ['triage'];
|
||||
const offered = list => (list || []).filter(r => !SYSTEM_ONLY.includes(r));
|
||||
async function read(org, s) {
|
||||
@@ -823,7 +823,5 @@
|
||||
window.matchMedia(NARROW).addEventListener('change', () => {
|
||||
if (document.querySelector('[data-wfg-graph]')) document.dispatchEvent(new CustomEvent('mosaic-workspace-render'));
|
||||
});
|
||||
// section(): the organization page's old form editor is gone (SP17); workspace.js still asks for it.
|
||||
const section = () => '';
|
||||
window.MosaicWorkflows = { load, section, page, card, hasDraft, forget, landed };
|
||||
window.MosaicWorkflows = { load, page, card, hasDraft, forget, landed };
|
||||
})();
|
||||
@@ -367,21 +367,20 @@
|
||||
${orgTiles}
|
||||
${docrow}
|
||||
<div class="ws-split">
|
||||
<section aria-label="Sessions and workflows">
|
||||
<section aria-label="Sessions, workflows, autonomy and starter packs">
|
||||
${secH('Active agent sessions', live ? (noRuns ? 'running not read' : live.length ? `${live.length} running${tileAgents ? `, ${liveWorking} working` : ', working not read'}` : 'none running') : '', `<a href="#/agents?${oq}" data-focus-key="agents-all">This organization's agents</a>`)}
|
||||
${sessions}
|
||||
${window.MosaicWorkflows?.card?.(id, { stopped, label: label(org) }) ?? ''}
|
||||
<div class="ws-wide">
|
||||
${stopped ? stoppedPanel('data-autonomy', 'Autonomy') : window.MosaicAutonomy?.section(id) ?? ''}
|
||||
${stopped ? stoppedPanel('data-packs', 'Starter packs') : window.MosaicPacks?.section(id) ?? ''}
|
||||
</div>
|
||||
</section>
|
||||
<section aria-label="Settings">
|
||||
${secH('Settings', 'every setting, with where its value comes from')}
|
||||
${settingsList(org, { business, settings, instances, noRuns, orgCount: list ? list.length : null })}
|
||||
</section>
|
||||
</div>
|
||||
<div class="ws-wide">
|
||||
${stopped ? stoppedPanel('data-autonomy', 'Autonomy') : window.MosaicAutonomy?.section(id) ?? ''}
|
||||
${stopped ? stoppedPanel('data-packs', 'Starter packs') : window.MosaicPacks?.section(id) ?? ''}
|
||||
${stopped ? stoppedPanel('data-workflows', 'Workflows') : window.MosaicWorkflows?.section(id) ?? ''}
|
||||
</div>
|
||||
${await drawerRead}
|
||||
</article>`;
|
||||
}
|
||||
@@ -1119,8 +1118,16 @@
|
||||
const DRAWER_LIMIT = 5, STOP_REASON = 'Stopped from the agent drawer.';
|
||||
const STOP_UNKNOWN = 'Outcome unknown. Read the agents again before trying; nothing was retried.';
|
||||
// The Stop state of the drawer's agent survives repaints: `confirm` while it asks, `busy` while the
|
||||
// request is open, then the result in words beside the button.
|
||||
let stopState = null; // { id, confirm, busy, text }
|
||||
// request is open, then the result in words beside the button. SP19: `kind` says whether Stop or
|
||||
// Restart asks (no kind is Stop). While either asks or runs, both buttons wait.
|
||||
let stopState = null; // { id, kind, confirm, busy, text }
|
||||
// SP19: each drawer agent's restart, kept per agent so several can run at once and none loses its
|
||||
// line when another drawer asks or stops: `busy` while it runs, then its outcome in words. An entry
|
||||
// goes when a new question starts on that agent, or when the drawer closes after it finished.
|
||||
const restarts = new Map(); // id -> { busy, text }
|
||||
// The state a drawer draws: its own question or Stop first, else its restart.
|
||||
const actState = id => stopState?.id === id ? stopState : restarts.has(id) ? { id, kind: 'restart', ...restarts.get(id) } : null;
|
||||
const RESTART_REASON = 'restart from the agent drawer';
|
||||
let drawerFocus = false, drawerShown = null;
|
||||
// The drawer holds at most one history entry. A row opened from the page without a drawer pushes one
|
||||
// entry marked in history.state; switching rows replaces it. Close goes Back only from that marked
|
||||
@@ -1133,11 +1140,16 @@
|
||||
const sansAgent = () => { const n = queryOf(); n.delete('agent'); const s = n.toString(); return `${/^#\/orgs\/[^/?]+/.exec(location.hash)?.[0] ?? '#'}${pathOf()}${s ? `?${s}` : ''}`; };
|
||||
// The row's name as a link that opens the drawer: a native link, so Tab reaches it and Enter opens it.
|
||||
const drawerLink = (q, a) => `<a class="ws-drawer-open" href="${esc(drawerHref(q, `${a.org}:${a.key}`))}" data-ws-drawer-open="${esc(`${a.org}:${a.key}`)}" data-focus-key="${esc(`agent:${a.org}:${a.key}`)}">${esc(a.name)}</a>`;
|
||||
const stopConfirm = (name, who) => `<span class="ws-drawer-confirm" data-drawer-stop-confirm>Stop ${esc(who)}? <button type="button" class="ws-btn danger" data-drawer-stop-yes data-focus-key="drawer-stop-yes">Stop ${esc(name)}</button> <button type="button" class="ws-btn" data-drawer-stop-no data-focus-key="drawer-stop-no">Cancel</button></span>`;
|
||||
function stopActions(pick, a, orgName) {
|
||||
const st = stopState?.id === pick.id ? stopState : null;
|
||||
const who = `${a.name} in ${orgName}`;
|
||||
return `<button type="button" class="ws-btn" data-drawer-stop data-focus-key="drawer-stop"${st?.busy || st?.confirm ? ' disabled' : ''}>${st?.busy ? 'Stopping…' : 'Stop'}<span class="ws-sr"> ${esc(who)}</span></button>${st?.confirm ? stopConfirm(a.name, who) : ''}`;
|
||||
// SP19: Stop and Restart ask in the same confirm step, each under its own data attributes.
|
||||
const actConfirm = (kind, name, question) => `<span class="ws-drawer-confirm" data-drawer-${kind}-confirm>${esc(question)} <button type="button" class="ws-btn danger" data-drawer-${kind}-yes data-focus-key="drawer-${kind}-yes">${kind === 'stop' ? 'Stop' : 'Restart'} ${esc(name)}</button> <button type="button" class="ws-btn" data-drawer-${kind}-no data-focus-key="drawer-${kind}-no">Cancel</button></span>`;
|
||||
const stopConfirm = (name, who) => actConfirm('stop', name, `Stop ${who}?`);
|
||||
const restartConfirm = (name, who, thread) => actConfirm('restart', name, `Restart ${who}? ${thread ? 'It ends its session; its next message starts it again.' : 'It ends its session and launches it again.'}`);
|
||||
function stopActions(pick, a, orgName, thread) {
|
||||
const st = actState(pick.id), kind = st?.kind || 'stop';
|
||||
const who = `${a.name} in ${orgName}`, hold = st?.busy || st?.confirm ? ' disabled' : '';
|
||||
const stop = `<button type="button" class="ws-btn" data-drawer-stop data-focus-key="drawer-stop"${hold}>${st?.busy && kind === 'stop' ? 'Stopping…' : 'Stop'}<span class="ws-sr"> ${esc(who)}</span></button>${st?.confirm && kind === 'stop' ? stopConfirm(a.name, who) : ''}`;
|
||||
if (!window.MosaicAgents?.restart) return stop;
|
||||
return `${stop} <button type="button" class="ws-btn" data-drawer-restart data-drawer-thread="${thread ? 'yes' : 'no'}" data-focus-key="drawer-restart"${hold}>${st?.busy && kind === 'restart' ? 'Restarting…' : 'Restart'}<span class="ws-sr"> ${esc(who)}</span></button>${st?.confirm && kind === 'restart' ? restartConfirm(a.name, who, thread) : ''}`;
|
||||
}
|
||||
// SP13 (#1873): the drawer's instances read gives the agent's state. The drawer does not hold the
|
||||
// page for it: once the page's own reads have answered (`opts.page`), a drawer whose instances read
|
||||
@@ -1336,7 +1348,8 @@
|
||||
: a.level !== 'org' ? 'No Message: a copy is not messaged from here.'
|
||||
: 'No Message: this Console messages only conversation agents, such as the Assistant.'}</p>`);
|
||||
else if (change) acts.push(`<p class="note" data-drawer-nomessage>No Message: this Console does not send chat in ${esc(orgName)}.</p>`);
|
||||
if (change && a.level === 'org' && !a.keep && running(item) > 0) acts.push(stopActions(pick, a, orgName));
|
||||
// SP19: a restart that is running keeps its buttons even when a repaint reads no run between its end and its launch.
|
||||
if (change && a.level === 'org' && !a.keep && (running(item) > 0 || restarts.get(id)?.busy)) acts.push(stopActions(pick, a, orgName, !!window.MosaicAgents?.isThread?.(def)));
|
||||
// Why Stop is not offered, in words, where this Console could otherwise change the organization's agents.
|
||||
// SP13: "not running" is a reason only once the instances read has given the agent's runs.
|
||||
else if (change) acts.push(`<p class="note" data-drawer-nostop>${reading ? 'Its state is being read. Stop is offered once it is read and the agent is running.'
|
||||
@@ -1344,10 +1357,25 @@
|
||||
: a.level !== 'org' ? 'A copy is not stopped from here.'
|
||||
: a.keep ? `${esc(a.name)} is one of ${esc(orgName)}’s protected agents, so it is not stopped from here.`
|
||||
: 'It is not running, so there is nothing to stop.'}</p>`);
|
||||
const st = stopState?.id === id ? stopState : null;
|
||||
const st = actState(id);
|
||||
const actions = chat || change
|
||||
? `<div class="ws-drawer-acts" data-drawer-actions>${acts.join(' ')}<p class="ws-drawer-status" id="ws-drawer-status" role="status" data-drawer-status>${esc(st?.text || '')}</p></div>`
|
||||
: `<p class="note" data-drawer-readonly>This Console only reads ${esc(orgName)}. It can not message or stop its agents.</p>`;
|
||||
// SP18: the agents read lists only roles a run holds now, and thread.claims only conversations a run
|
||||
// holds now; each row carries that run and when it took the role or the conversation. A claim counts
|
||||
// only when the instances read lists its run as running, so a claim left by a run that ended does not
|
||||
// date a stopped agent. Last activity is the latest such time. No read keeps when a run ended or when
|
||||
// the agent last sent a message (a thread's messages read pages from the oldest message), so neither is shown.
|
||||
const runs = !unknown && runsRead(item) ? item.running : null;
|
||||
const claims = [a.s ? { ...a.s, what: 'the role' } : null, ...arr(body?.sessions).filter(x => x?.kind === 'thread' && x.org === org && x.instance === key).map(x => ({ ...x, what: 'the conversation' }))]
|
||||
.filter(c => c && msOf(c.claimedAt) !== null && (!runs || runs.includes(c.run))).sort((x, y) => msOf(y.claimedAt) - msOf(x.claimedAt));
|
||||
const last = claims[0], claimWhy = notRead(['agents', 'claims']);
|
||||
// With the runs not read, nothing checked the claim against a live run, so the line says "a run".
|
||||
const activity = last ? `${!runs ? 'a run' : runs.length > 1 ? `one of its ${runs.length} runs` : 'its run'} took ${last.what} <time datetime="${esc(last.claimedAt)}" title="${esc(new Date(msOf(last.claimedAt)).toLocaleString())}">${esc(ago(last.claimedAt))}</time>`
|
||||
: `<span class="muted">${claimWhy ? `not read: ${esc(claimWhy)}`
|
||||
: runs?.length ? `not recorded: ${runs.length > 1 ? 'its runs hold' : 'its run holds'} no role or conversation yet`
|
||||
: !runs ? 'not recorded: the agents and claims reads show no run holding its role or a conversation, and its runs were not read'
|
||||
: 'not recorded: no run holds it now, and no read keeps when its last run ended'}</span>`;
|
||||
const fields = [
|
||||
field('org', 'Organization', orgChip(o, org)),
|
||||
// A pack hire's pack is named once, in Hired from; the Role row says only that it was hired.
|
||||
@@ -1358,11 +1386,7 @@
|
||||
model ? field('model', 'Model', `<span class="mono">${esc(model)}</span>`) : '',
|
||||
project ? field('project', 'Project', `<a href="${esc(`${orgHref(org)}/projects/${enc(project.id)}`)}">${esc(project.name || project.key || project.id)}</a>${project.from ? ` <span class="t2">${project.from}</span>` : ''}`) : '',
|
||||
task ? field('task', 'Task', `<a href="${esc(taskHref(task.ref))}">${esc(task.title || task.ref)}</a>${task.title ? ` <span class="t2 mono">${esc(task.ref)}</span>` : ''}`) : '',
|
||||
// SP16: no read the Console makes carries the time of a role agent's last message or run. The agents
|
||||
// read's time is when it took the role and `since` is when its present state began, so neither is shown.
|
||||
// An agent holding a conversation has its messages in that thread's messages read, but that read pages
|
||||
// from the oldest message, so its last message is a walk of every page away; the drawer does not make it.
|
||||
field('activity', 'Last activity', `<span class="muted">${thread?.thread ? 'not shown: reading it means reading the whole conversation' : 'not recorded'}</span>`),
|
||||
field('activity', 'Last activity', activity),
|
||||
].join('');
|
||||
return shell(a.name, `<dl class="ws-drawer-fields">${fields}</dl>
|
||||
<section class="ws-drawer-sec" data-drawer-tasks aria-label="Open tasks assigned to ${esc(a.name)}"><h3>Open tasks</h3>${tasksBody}${tasksAll}</section>
|
||||
@@ -1373,10 +1397,21 @@
|
||||
// `agent` from the address in place. Either way the page repaints without it and focus returns to the row.
|
||||
function closeDrawer() {
|
||||
if (!stopState?.busy) stopState = null;
|
||||
for (const [k, r] of restarts) if (!r.busy) restarts.delete(k);
|
||||
if (stateOf()?.[DRAWER_MARK] && queryOf().has('agent')) { history.back(); return; }
|
||||
history.replaceState(unmarked(), '', sansAgent());
|
||||
document.dispatchEvent(new CustomEvent('mosaic-workspace-render'));
|
||||
}
|
||||
// SP18: below 701 px the open drawer is a sheet and shell.css holds the page under it to one screen, which
|
||||
// scrolls the page to the top. The page's place is kept while no drawer is open, with the page the sheet
|
||||
// opened over. It is put back only when the drawer closes onto that same page (Close, Escape or Back); a
|
||||
// link in the sheet goes to another page, which starts at its top, and the kept place is dropped. Some
|
||||
// pages a link opens (a project, the Assistant) are drawn without afterPaint, so the place is dropped as
|
||||
// soon as the address leaves the page under the sheet.
|
||||
const SHEET = window.matchMedia('(max-width:700px)');
|
||||
let pageY = 0, sheetOver = null;
|
||||
window.addEventListener('scroll', () => { if (!document.querySelector('#bus-view [data-ws-drawer]')) pageY = window.scrollY; }, { passive: true });
|
||||
window.addEventListener('hashchange', () => { if (sheetOver !== null && sansAgent() !== sheetOver) { sheetOver = null; pageY = 0; } });
|
||||
// bus.js calls this after each paint: focus moves into a drawer a row opened, and back to the row
|
||||
// when the drawer has closed and focus was left nowhere.
|
||||
function afterPaint() {
|
||||
@@ -1384,8 +1419,13 @@
|
||||
const view = document.getElementById('bus-view'), d = view?.querySelector('[data-ws-drawer]');
|
||||
const now = d?.dataset.wsDrawer || null, was = drawerShown;
|
||||
drawerShown = now;
|
||||
if (now) sheetOver = sansAgent();
|
||||
if (d && drawerFocus) { drawerFocus = false; d.querySelector('h2')?.focus({ preventScroll: true }); return; }
|
||||
drawerFocus = false;
|
||||
if (!now && sheetOver !== null) {
|
||||
if (SHEET.matches && pageY && sheetOver === sansAgent()) window.scrollTo(0, pageY);
|
||||
sheetOver = null; pageY = window.scrollY;
|
||||
}
|
||||
if (!now && was) {
|
||||
const a = document.activeElement;
|
||||
if (!a || a === document.body || !view?.contains(a)) view?.querySelector(`[data-focus-key="${CSS.escape(`agent:${was}`)}"]`)?.focus({ preventScroll: true });
|
||||
@@ -1395,7 +1435,7 @@
|
||||
const id = d.dataset.wsDrawer, org = d.dataset.drawerOrgId, name = id.slice(org.length + 1), orgName = d.dataset.drawerOrgName;
|
||||
stopState = { id, busy: true, text: `Stopping ${name}…` };
|
||||
const repaint = () => document.dispatchEvent(new CustomEvent('mosaic-workspace-render'));
|
||||
const paint = () => { const now = document.querySelector(`[data-ws-drawer="${CSS.escape(id)}"]`); if (!now) return; const s = now.querySelector('[data-drawer-status]'); if (s) s.textContent = stopState.text; const b = now.querySelector('[data-drawer-stop]'); if (b) { b.disabled = !!stopState.busy; b.firstChild.textContent = stopState.busy ? 'Stopping…' : 'Stop'; } now.querySelector('[data-drawer-stop-confirm]')?.remove(); };
|
||||
const paint = () => { const now = document.querySelector(`[data-ws-drawer="${CSS.escape(id)}"]`); if (!now) return; const s = now.querySelector('[data-drawer-status]'); if (s) s.textContent = stopState.text; const b = now.querySelector('[data-drawer-stop]'); if (b) { b.disabled = !!stopState.busy; b.firstChild.textContent = stopState.busy ? 'Stopping…' : 'Stop'; } const rb = now.querySelector('[data-drawer-restart]'); if (rb) rb.disabled = !!stopState.busy; now.querySelector('[data-drawer-stop-confirm]')?.remove(); };
|
||||
const finish = text => { stopState = { id, text }; paint(); const b = document.querySelector(`[data-ws-drawer="${CSS.escape(id)}"] [data-drawer-stop]`); b?.focus({ preventScroll: true }); };
|
||||
paint();
|
||||
let res, body = null;
|
||||
@@ -1423,6 +1463,72 @@
|
||||
}
|
||||
finish(`Request ${request} is still open. Read the agents again later.`);
|
||||
}
|
||||
// SP19: the drawer's Restart. The agents.js chain ends the agent through its own organization's routes
|
||||
// (the drawer's, which on #/agents is not the page's), waits for the request to close and its runs to
|
||||
// reach 0, then launches it; a conversation agent is not launched. It never regenerates. Each step is
|
||||
// painted in place; a refusal before the end leaves both buttons usable and focus on Restart.
|
||||
const refusedWhy = (d, orgName) => {
|
||||
const e = d?.result?.error, m = d?.result?.message;
|
||||
if (d?.status === 403 && e === 'read-only') return `this Console only reads agents in ${orgName} (read-only).`;
|
||||
if (d?.status === 403 && (e === 'unavailable' || e === 'owner-only')) return `this Console is not the owner of ${orgName}, so it does not restart its agents.`;
|
||||
if (e === 'no-bus-host') return `the bus of ${orgName} is not running. Nothing changed.`;
|
||||
if (e === 'org-archived') return `${orgName} is archived, so its agents are not changed. Nothing changed.`;
|
||||
if (e === 'org-unavailable') return `${orgName} is not available to this Console. Nothing changed.`;
|
||||
if (d?.result?.sent === false && typeof m === 'string' && m) return m;
|
||||
if (d?.result?.sent === false && typeof e === 'string') return `the bus refused (${e}).`;
|
||||
return STOP_UNKNOWN;
|
||||
};
|
||||
const restartWords = (name, orgName) => (step, d) => ({
|
||||
ending: `Restarting ${name}: asking it to end…`,
|
||||
signedOut: 'Not restarted: sign in again, then read the agents.',
|
||||
endUnknown: `Not restarted: ${STOP_UNKNOWN}`,
|
||||
endRefused: `Not restarted: ${refusedWhy(d, orgName)}`,
|
||||
endOpen: `Not restarted: ${d?.stop}`,
|
||||
endClosed: `Not restarted: ${name} was not ended (${d?.reason}).`,
|
||||
ended: `${name} was asked to end. Waiting for its session to end…`,
|
||||
stillRunning: `Not restarted: ${name} was asked to end, but its session is still running. Read the agents again before trying.`,
|
||||
threadEnded: `Ended ${name} at ${d?.time}. Its next message starts it.`,
|
||||
launching: `${name} has ended. Launching it…`,
|
||||
launchSignedOut: `${name} was ended and not launched: sign in again, then read the agents.`,
|
||||
launchUnknown: `${name} was ended. The launch outcome is unknown; read the agents again before trying. Nothing was retried.`,
|
||||
launchRefused: `${name} was ended and not launched: ${refusedWhy(d, orgName)}`,
|
||||
launchOpen: `${name} was ended. ${d?.stop}`,
|
||||
launchClosed: `${name} was ended and not launched: ${d?.reason}.`,
|
||||
restarted: `Restarted ${name}${d?.run ? ` (run ${d.run})` : ''}.`,
|
||||
})[step];
|
||||
async function restartAgent(d) {
|
||||
const id = d.dataset.wsDrawer, org = d.dataset.drawerOrgId, name = id.slice(org.length + 1), orgName = d.dataset.drawerOrgName;
|
||||
const thread = d.querySelector('[data-drawer-restart]')?.dataset.drawerThread === 'yes';
|
||||
if (restarts.get(id)?.busy) return;
|
||||
// The question is answered: from now on this agent's drawer draws its restart.
|
||||
if (stopState?.id === id) stopState = null;
|
||||
restarts.set(id, { busy: true, text: '' });
|
||||
// Paint this agent's drawer in place, when it is on screen and no question of its own holds it.
|
||||
const paint = () => {
|
||||
const now = document.querySelector(`[data-ws-drawer="${CSS.escape(id)}"]`), r = restarts.get(id); if (!now || !r || stopState?.id === id) return;
|
||||
const s = now.querySelector('[data-drawer-status]'); if (s) s.textContent = r.text;
|
||||
const rb = now.querySelector('[data-drawer-restart]'), sb = now.querySelector('[data-drawer-stop]');
|
||||
if (rb) { rb.disabled = r.busy; rb.firstChild.textContent = r.busy ? 'Restarting…' : 'Restart'; }
|
||||
if (sb) sb.disabled = r.busy;
|
||||
now.querySelector('.ws-drawer-confirm')?.remove();
|
||||
};
|
||||
// The finished line stays in the entry. A refusal before the end changed nothing, so focus goes back
|
||||
// to Restart. Once the agent was ended its runs changed: the repaint reads the instances again, so the
|
||||
// buttons follow them (after a refused launch it has none, so Stop and Restart go), and focus goes to
|
||||
// the drawer's heading, which every repaint keeps.
|
||||
const after = new Set(['restarted', 'threadEnded', 'launchSignedOut', 'launchUnknown', 'launchRefused', 'launchOpen', 'launchClosed']);
|
||||
const done = step => {
|
||||
document.querySelector(`[data-ws-drawer="${CSS.escape(id)}"]`)?.querySelector(after.has(step) ? '[data-focus-key="drawer-h"]' : '[data-drawer-restart]')?.focus({ preventScroll: true });
|
||||
if (after.has(step)) document.dispatchEvent(new CustomEvent('mosaic-workspace-render'));
|
||||
};
|
||||
const onStatus = ({ step, busy, text }) => {
|
||||
restarts.set(id, { busy: !!busy, text });
|
||||
paint();
|
||||
if (!busy) done(step);
|
||||
};
|
||||
try { await window.MosaicAgents.restart(name, thread, { org, reason: RESTART_REASON, text: restartWords(name, orgName), onStatus }); }
|
||||
catch { restarts.set(id, { busy: false, text: `Not restarted: ${STOP_UNKNOWN}` }); paint(); done('endUnknown'); }
|
||||
}
|
||||
document.addEventListener('click', e => {
|
||||
const link = e.target.closest?.('[data-ws-drawer-open]');
|
||||
if (link) {
|
||||
@@ -1441,33 +1547,41 @@
|
||||
if (!d) return;
|
||||
if (e.target.closest('[data-drawer-close]')) { closeDrawer(); return; }
|
||||
const id = d.dataset.wsDrawer;
|
||||
if (e.target.closest('[data-drawer-stop]') && !(stopState?.id === id && stopState.busy)) {
|
||||
const b = d.querySelector('[data-drawer-stop]'), name = d.dataset.drawerName;
|
||||
stopState = { id, confirm: true, text: '' };
|
||||
d.querySelector('[data-drawer-stop-confirm]')?.remove();
|
||||
b.insertAdjacentHTML('afterend', stopConfirm(name, `${name} in ${d.dataset.drawerOrgName}`));
|
||||
b.disabled = true;
|
||||
// SP19: nothing starts while this agent's Stop or Restart runs.
|
||||
const held = (stopState?.id === id && stopState.busy) || !!restarts.get(id)?.busy;
|
||||
const ask = e.target.closest('[data-drawer-stop]') ? 'stop' : e.target.closest('[data-drawer-restart]') ? 'restart' : null;
|
||||
if (ask && !held) {
|
||||
const b = d.querySelector(`[data-drawer-${ask}]`), name = d.dataset.drawerName, who = `${name} in ${d.dataset.drawerOrgName}`;
|
||||
stopState = { id, kind: ask, confirm: true, text: '' }; restarts.delete(id);
|
||||
d.querySelector('.ws-drawer-confirm')?.remove();
|
||||
b.insertAdjacentHTML('afterend', ask === 'stop' ? stopConfirm(name, who) : restartConfirm(name, who, b.dataset.drawerThread === 'yes'));
|
||||
for (const x of d.querySelectorAll('[data-drawer-stop], [data-drawer-restart]')) x.disabled = true;
|
||||
const s = d.querySelector('[data-drawer-status]'); if (s) s.textContent = '';
|
||||
d.querySelector('[data-drawer-stop-no]')?.focus({ preventScroll: true });
|
||||
d.querySelector(`[data-drawer-${ask}-no]`)?.focus({ preventScroll: true });
|
||||
return;
|
||||
}
|
||||
if (e.target.closest('[data-drawer-stop-no]')) { cancelStop(d); return; }
|
||||
if (ask) return;
|
||||
if (e.target.closest('[data-drawer-stop-no], [data-drawer-restart-no]')) { cancelStop(d); return; }
|
||||
if (held) return;
|
||||
if (e.target.closest('[data-drawer-stop-yes]')) stopAgent(d);
|
||||
else if (e.target.closest('[data-drawer-restart-yes]')) restartAgent(d);
|
||||
});
|
||||
// Cancel the Stop question: nothing sent, Stop enabled again and focused.
|
||||
// Cancel the Stop or Restart question: nothing sent, both buttons enabled again, focus on the one that asked.
|
||||
function cancelStop(d) {
|
||||
const kind = d.querySelector('[data-drawer-restart-confirm]') ? 'restart' : 'stop';
|
||||
stopState = null;
|
||||
d.querySelector('[data-drawer-stop-confirm]')?.remove();
|
||||
const b = d.querySelector('[data-drawer-stop]'); if (b) { b.disabled = false; b.focus({ preventScroll: true }); }
|
||||
d.querySelector('.ws-drawer-confirm')?.remove();
|
||||
for (const x of d.querySelectorAll('[data-drawer-stop], [data-drawer-restart]')) x.disabled = false;
|
||||
d.querySelector(`[data-drawer-${kind}]`)?.focus({ preventScroll: true });
|
||||
}
|
||||
// Escape cancels the Stop question while it asks, else closes the drawer, unless a dialog is open
|
||||
// Escape cancels the Stop or Restart question while it asks, else closes the drawer, unless a dialog is open
|
||||
// or something else took the key.
|
||||
document.addEventListener('keydown', e => {
|
||||
if (e.key !== 'Escape' || e.defaultPrevented || document.querySelector('dialog[open]')) return;
|
||||
const d = document.querySelector('#bus-view [data-ws-drawer]');
|
||||
if (!d) return;
|
||||
e.preventDefault();
|
||||
if (d.querySelector('[data-drawer-stop-confirm]')) cancelStop(d); else closeDrawer();
|
||||
if (d.querySelector('.ws-drawer-confirm')) cancelStop(d); else closeDrawer();
|
||||
});
|
||||
|
||||
// ---- controls: filters write the hash query and repaint in place ----
|
||||
|
||||
@@ -22,18 +22,46 @@
|
||||
// state says so. The route's answer names the harness: adapter, its mounts, harnessEnv, turnEnd,
|
||||
// model and provider, and the proxy variables for the spawn. Over the cap the route refuses until
|
||||
// the next day, and the message is refused with its code.
|
||||
//
|
||||
// #1873 triage-fences: the host checks a copy of the route's answer before it writes anything, and
|
||||
// uses only that copy. harnessEnv names must be in the harness module's envNames and pass
|
||||
// refusedHarnessName; values are clean strings. env holds only ROUTE_ENV_ALLOW names, with
|
||||
// ANTHROPIC_BASE_URL loopback http on a port (the host's proxyPort when it names one) and the
|
||||
// token matching the host's tokenPattern when it names one. The adapter and the mounts pass
|
||||
// checkHostedRoute (sandbox.mjs) against the module's installRoots and the data root, the config
|
||||
// directory and HOME (HOME not counted against the stack's own code), here and again in
|
||||
// hostedBinds. A refused answer reaches no spawn, answers triage-route-invalid, and writes
|
||||
// ROUTE_REFUSED to the transcript; a refusal at the binds also removes what the start wrote.
|
||||
//
|
||||
// #1873 triage-cap: the relay counts each turn. usage.begin takes a baseline of the harness's
|
||||
// transcript with the transcript usage-source the bus uses; usage.end takes the delta (the bus's
|
||||
// own rules, triage-usage.mjs) and appends one line to <dataRoot>/triage/usage.jsonl. When a cap is
|
||||
// stored (triage-route-store.mjs), the daily-quota plugin decides over the lines in that file: before
|
||||
// a start (admit), before the runner takes queued messages (message.receive) and before each
|
||||
// message's turn (the runner's state read outside a turn). Over the cap, or with the store or the
|
||||
// file unreadable, the turn in flight finishes, every message still queued gets a host line and
|
||||
// leaves the queue, and the runner is told its session is gone. Nothing is held in memory: each
|
||||
// check reads the file again, so a restart counts what the file holds. With no cap stored, nothing
|
||||
// is refused for usage and the lines are still written.
|
||||
|
||||
import { spawn } from "node:child_process";
|
||||
import { randomBytes, timingSafeEqual } from "node:crypto";
|
||||
import { appendFileSync, closeSync, lstatSync, mkdirSync, openSync, readFileSync, rmSync, chmodSync, writeFileSync } from "node:fs";
|
||||
import { createServer } from "node:net";
|
||||
import { homedir } from "node:os";
|
||||
import { dirname, join } from "node:path";
|
||||
import { BWRAP, REPO, RUNNER_FILES, hostedArgv, hostedBinds } from "../../seat/src/sandbox.mjs";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { BWRAP, REPO, RUNNER_FILES, checkHostedRoute, hostedArgv, hostedBinds } from "../../seat/src/sandbox.mjs";
|
||||
import { isProviderVariable } from "../../seat/src/session.mjs";
|
||||
import { configDir as systemConfigDir } from "../../organization/src/organization-config.mjs";
|
||||
import { typedTools } from "../../harness/src/tools.mjs";
|
||||
import { builtinTools } from "../../harness/src/bundle.mjs";
|
||||
import { FOUNDER_ENV } from "../../harness/src/runner.mjs";
|
||||
import { extractCards, plainText } from "./triage-cards.mjs";
|
||||
import { SNAPSHOT_FILE, writeSnapshot } from "./triage-snapshot.mjs";
|
||||
import { readRoute } from "./triage-route-store.mjs";
|
||||
import { ID as USAGE_ID, appendUsage, capState, delta, nextReset, readUsage, snapshot, usagePath } from "./triage-usage.mjs";
|
||||
import { readTurn } from "../../plugins/usage-source/transcript/index.mjs";
|
||||
|
||||
export const RUNNER = RUNNER_FILES[0];
|
||||
export const RELAY_REFUSED = "triage-refused";
|
||||
@@ -43,6 +71,9 @@ export const QUEUE_MAX = 20;
|
||||
export const UNANSWERED = "Triage stopped before it answered this message. Send it again.";
|
||||
export const NOT_STARTED = "Triage did not start, so this message was not delivered.";
|
||||
export const NOT_TAKEN = "Triage stopped before it read this message. Send it again.";
|
||||
export const ROUTE_REFUSED = "Triage did not start: the model route's answer did not pass the host's checks, so this message was not delivered.";
|
||||
export const overCapNotTaken = (until) => `Triage reached its daily token cap before it took this message. Send it again after ${until}.`;
|
||||
export const USAGE_NOT_READ = "Triage could not read its token usage or its cap, so it did not take this message.";
|
||||
// An over-cap answer is held for at most this long; a later time is not taken.
|
||||
export const CAP_HOLD_MAX_MS = 48 * 3600 * 1000;
|
||||
const ISO_TIME = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d{1,3})?Z$/;
|
||||
@@ -56,7 +87,36 @@ const CAP = /^[0-9a-f]{64}$/;
|
||||
export const SPAWN_ENV_ALLOW = Object.freeze(["PATH", "LANG", "LC_ALL", "TZ", "TERM"]);
|
||||
export const ROUTE_ENV_ALLOW = Object.freeze(["ANTHROPIC_BASE_URL", "ANTHROPIC_AUTH_TOKEN"]);
|
||||
const ENV_NAME = /^[A-Z][A-Z0-9_]{0,63}$/;
|
||||
const deniedHarnessName = (k) => !ENV_NAME.test(k) || k === "MOSAIC_CONFIG" || k.startsWith("MOSAIC_BUS_") || FOUNDER_ENV.includes(k) || k === "SSH_AUTH_SOCK";
|
||||
const ENV_VALUE_MAX = 4096;
|
||||
const HARNESS_ENV_MAX = 64;
|
||||
// Refused in harnessEnv even when the harness module declares them: the runner's own MOSAIC_*
|
||||
// values, loader, libc and interpreter hooks, the shell's start files, a tool's config directory
|
||||
// and the founder's, an askpass program, a CA bundle or a base URL, Lua's start code, curl's config
|
||||
// home and a TLS key log. A provider name (every ANTHROPIC_* among them) goes through route env or nowhere.
|
||||
const REFUSED_PREFIX = /^(MOSAIC_|NODE_|LD_|DYLD_|PYTHON|BASH_|GIT_|MALLOC_|NPM_CONFIG_|XDG_)/;
|
||||
const REFUSED_SUFFIX = /_(DIR|CONF|CONFIG|ASKPASS|CA_BUNDLE|BASE_URL)$/;
|
||||
const REFUSED_NAMES = new Set([
|
||||
"PATH", "HOME", "TMPDIR", "BASH_ENV", "ENV", "SSH_AUTH_SOCK", "SHELL", "SHELLOPTS", "BASHOPTS", "PS4", "IFS", "CDPATH",
|
||||
"GLOBIGNORE", "PROMPT_COMMAND", "ZDOTDIR", "PERL5OPT", "PERL5LIB", "RUBYOPT", "RUBYLIB", "PERLLIB", "LESSOPEN", "LESSCLOSE", "HOSTALIASES", "HTTP_PROXY", "HTTPS_PROXY", "ALL_PROXY", "NO_PROXY",
|
||||
"GCONV_PATH", "GLIBC_TUNABLES", "LOCPATH", "OPENSSL_CONF", "OPENSSL_ENGINES", "OPENSSL_MODULES", "SSL_CERT_FILE", "SSL_CERT_DIR",
|
||||
"JAVA_TOOL_OPTIONS", "_JAVA_OPTIONS", "JDK_JAVA_OPTIONS", "LUA_INIT", "CURL_HOME", "SSLKEYLOGFILE",
|
||||
...FOUNDER_ENV,
|
||||
]);
|
||||
export const refusedHarnessName = (k) => typeof k !== "string" || !ENV_NAME.test(k) || REFUSED_PREFIX.test(k) || REFUSED_SUFFIX.test(k) || REFUSED_NAMES.has(k) || isProviderVariable(k);
|
||||
// No control character of any kind (C0, DEL, C1) and no line or paragraph separator.
|
||||
const cleanValue = (v) => typeof v === "string" && v.length <= ENV_VALUE_MAX && !/[\u0000-\u001f\u007f-\u009f\u2028\u2029]/.test(v);
|
||||
const plainObject = (o) => o !== null && typeof o === "object" && !Array.isArray(o) && [Object.prototype, null].includes(Object.getPrototypeOf(o));
|
||||
// The proxy is on this machine: http, 127.0.0.1, a port, nothing after it.
|
||||
const LOOPBACK_URL = /^http:\/\/127\.0\.0\.1:([1-9][0-9]{0,4})$/;
|
||||
const HARNESS_NAME = /^[a-z][a-z0-9-]{0,31}$/;
|
||||
|
||||
// The harness module a route names, from the repository's own tree; null when there is none.
|
||||
async function repoHarnessModule(name) {
|
||||
if (!HARNESS_NAME.test(name)) return null;
|
||||
const file = join(REPO, "packages", "plugins", "harness", name, "index.mjs");
|
||||
try { if (!lstatSync(file).isFile()) return null; } catch { return null; }
|
||||
return import(pathToFileURL(file).href);
|
||||
}
|
||||
|
||||
export function triageEnv(source, routeEnv = {}, { node = process.execPath } = {}) {
|
||||
const env = { PATH: `${dirname(node)}:/usr/bin:/bin` };
|
||||
@@ -132,9 +192,13 @@ function sameCap(a, b) {
|
||||
return timingSafeEqual(Buffer.from(a), Buffer.from(b));
|
||||
}
|
||||
|
||||
// The relay server. `onSend(body)` keeps a human-bound send; `take()` returns the queued owner
|
||||
// messages; `read(id)` drops one that was answered.
|
||||
export function createRelay({ socket, cap, take, read, onSend, log = () => {} }) {
|
||||
// The relay server. `onSend({ body, in_reply_to, result })` keeps a human-bound send, `result` true
|
||||
// only for class RESULT, the class the runner gives its answer to a turn; `take()` returns the
|
||||
// queued owner messages; `read(id)` drops one that was answered.
|
||||
// #1873 triage-cap: `usage(phase, turn)` (a promise) counts the turn; the relay answers usage.begin
|
||||
// and usage.end only once it settles. `stopped(verb)` is asked before state and message.receive;
|
||||
// true answers "not-holder", which ends the runner's loop after the turn in flight.
|
||||
export function createRelay({ socket, cap, take, read, onSend, usage = null, stopped = () => false, log = () => {} }) {
|
||||
const answer = (conn, value) => { try { conn.end(JSON.stringify(value) + "\n"); } catch {} };
|
||||
const refuse = (conn) => answer(conn, { ok: false, error: RELAY_REFUSED, reason: "the Triage host answers only a message to the person" });
|
||||
const server = createServer((conn) => {
|
||||
@@ -155,12 +219,19 @@ export function createRelay({ socket, cap, take, read, onSend, log = () => {} })
|
||||
const args = req.args && typeof req.args === "object" && !Array.isArray(req.args) ? req.args : {};
|
||||
const verb = req.verb;
|
||||
try {
|
||||
if (usage && (verb === "usage.begin" || verb === "usage.end")) {
|
||||
return void Promise.resolve()
|
||||
.then(() => usage(verb === "usage.begin" ? "begin" : "end", args.turn))
|
||||
.catch((e) => log(`triage relay: ${verb} not counted: ${e?.code ?? "error"}`))
|
||||
.then(() => answer(conn, { ok: true, result: {} }));
|
||||
}
|
||||
if ((verb === "state" || verb === "message.receive") && stopped(verb)) return answer(conn, { ok: false, error: "not-holder" });
|
||||
if (Object.hasOwn(LIFECYCLE, verb)) return answer(conn, { ok: true, result: LIFECYCLE[verb]() });
|
||||
if (verb === "message.receive") return answer(conn, { ok: true, result: take() });
|
||||
if (verb === "message.read") { read(args.id); return answer(conn, { ok: true, result: {} }); }
|
||||
if (verb === "message.send" && args.to === "human" && typeof args.body === "string" && args.body.length <= 32768) {
|
||||
// Only an id the host issued is kept. The model writes this field, and the transcript serves it.
|
||||
onSend({ body: args.body, in_reply_to: typeof args.in_reply_to === "string" && OWNER_ID.test(args.in_reply_to) ? args.in_reply_to : null });
|
||||
onSend({ body: args.body, in_reply_to: typeof args.in_reply_to === "string" && OWNER_ID.test(args.in_reply_to) ? args.in_reply_to : null, result: args.class === "RESULT" });
|
||||
return answer(conn, { ok: true, result: { id: `triage-${randomBytes(6).toString("hex")}` } });
|
||||
}
|
||||
} catch (e) {
|
||||
@@ -185,8 +256,23 @@ export function createRelay({ socket, cap, take, read, onSend, log = () => {} })
|
||||
// provider, env } | { ok: false, code, until? } } or null for no route
|
||||
// readSnapshot async () -> the snapshot (triage-snapshot.mjs composeSnapshot), written to health/
|
||||
// pollInterval the runner's receive interval in ms, for tests; the runner's default otherwise
|
||||
export function createTriageHost({ dataRoot, route = null, readSnapshot = async () => null, env = process.env, node = process.execPath, bwrap = BWRAP, now = () => new Date(), devOwner = false, log = () => {}, roleDir = join(REPO, "roles"), readOnly = [], pollInterval = null }) {
|
||||
// proxyPort the port ANTHROPIC_BASE_URL must name, or null for any port
|
||||
// tokenPattern a RegExp the whole ANTHROPIC_AUTH_TOKEN must match, or null for no shape check
|
||||
// harnessModule async (name) -> the harness module (envNames, installRoots) or null; the
|
||||
// repository's packages/plugins/harness/<name> by default
|
||||
// home HOME, which no mount may reach outside the stack's own code; the Console user's by default
|
||||
// usageSource { readTurn({ harness, root }) } the transcript usage-source; the repository's own by default
|
||||
export function createTriageHost({ dataRoot, route = null, readSnapshot = async () => null, env = process.env, node = process.execPath, bwrap = BWRAP, now = () => new Date(), devOwner = false, log = () => {}, roleDir = join(REPO, "roles"), readOnly = [], pollInterval = null, proxyPort = null, tokenPattern = null, harnessModule = repoHarnessModule, home = homedir(), usageSource = { readTurn } }) {
|
||||
if (typeof dataRoot !== "string" || !dataRoot.startsWith("/")) throw new TriageError("triage-no-data-root", "no data root");
|
||||
if (proxyPort !== null && !(Number.isInteger(proxyPort) && proxyPort > 0 && proxyPort <= 65535)) throw new TypeError("proxyPort must be a port number or null");
|
||||
if (tokenPattern !== null && !(tokenPattern instanceof RegExp)) throw new TypeError("tokenPattern must be a RegExp or null");
|
||||
if (typeof harnessModule !== "function") throw new TypeError("harnessModule must be a function");
|
||||
if (typeof usageSource?.readTurn !== "function") throw new TypeError("usageSource must have readTurn");
|
||||
const tokenShape = tokenPattern && new RegExp(`^(?:${tokenPattern.source})$`, tokenPattern.flags.replace(/[gmy]/g, ""));
|
||||
// What no mount source may lie in or around, besides the run directory and the fixed list. HOME
|
||||
// does not refuse the stack's own code when the stack runs from under it (a copy or a worktree).
|
||||
const protectedPaths = [dataRoot, systemConfigDir(env), home];
|
||||
const codeRoot = REPO;
|
||||
const paths = triagePaths(dataRoot);
|
||||
const role = JSON.parse(readFileSync(join(roleDir, "triage.json"), "utf8"));
|
||||
const contract = readFileSync(join(roleDir, role.contract), "utf8");
|
||||
@@ -199,6 +285,10 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
|
||||
let cap = null;
|
||||
let overCapUntil = null; // the route's over-cap answer, held until its time passes
|
||||
const queue = [];
|
||||
let usage = null; // this run's counting: { run, root, harness, baselines, ended, inTurn }
|
||||
let usageChain = Promise.resolve(); // usage calls and exit records, one at a time
|
||||
let usageLost = false; // a turn's line could not be written: the count is short, so a stored cap refuses
|
||||
const badSeen = new Set();
|
||||
|
||||
const ensureDirs = () => {
|
||||
mkdirSync(dataRoot, { recursive: true, mode: 0o700 });
|
||||
@@ -262,29 +352,159 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
|
||||
writeFileSync(bundle.prompt, hostPrompt(contract, { snapshotPath: join(workspace, "health", SNAPSHOT_FILE), steps }), { mode: 0o600, flag: "wx" });
|
||||
writeFileSync(bundle.policy, JSON.stringify(policy, null, 2) + "\n", { mode: 0o600, flag: "wx" });
|
||||
writeFileSync(bundle.tools, JSON.stringify(tools, null, 2) + "\n", { mode: 0o600, flag: "wx" });
|
||||
const run = `triage-${now().toISOString().replace(/[^0-9]/g, "").slice(0, 14)}`;
|
||||
const session = {
|
||||
bundle, adapter: admitted.adapter, harness: admitted.harness, harnessEnv: admitted.harnessEnv, turnEnd: admitted.turnEnd,
|
||||
workspace, sessionDir: join(a, "session"), toolSocket: join(a, "tools.sock"), turnMarker: join(a, "turn.done"),
|
||||
brokerSocket: join(a, "relay.sock"), business: "system", instance: "triage", run: `triage-${now().toISOString().replace(/[^0-9]/g, "").slice(0, 14)}`,
|
||||
brokerSocket: join(a, "relay.sock"), business: "system", instance: "triage", run,
|
||||
model: admitted.model, provider: admitted.provider ?? "",
|
||||
...(Number.isInteger(pollInterval) && pollInterval >= 100 ? { pollInterval } : {}),
|
||||
};
|
||||
writeFileSync(join(a, "session.json"), JSON.stringify(session, null, 2) + "\n", { mode: 0o600, flag: "wx" });
|
||||
return run;
|
||||
};
|
||||
|
||||
const checkAdmitted = (a) => {
|
||||
if (!a || typeof a !== "object" || typeof a.adapter !== "string" || !a.adapter.startsWith("/")
|
||||
|| !["marker", "exit"].includes(a.turnEnd) || typeof a.model !== "string" || !a.model
|
||||
|| typeof a.harness !== "string" || !/^[a-z][a-z0-9-]{0,31}$/.test(a.harness)
|
||||
|| !a.harnessEnv || typeof a.harnessEnv !== "object" || Array.isArray(a.harnessEnv)
|
||||
|| Object.entries(a.harnessEnv).some(([k, v]) => typeof v !== "string" || deniedHarnessName(k))
|
||||
|| !Array.isArray(a.harnessMounts ?? [])) throw new TriageError("triage-route-invalid", "the model route's answer is not usable");
|
||||
// The route's answer, checked. It returns a copy taken once, so a getter or a later change to
|
||||
// the route's object cannot hand the bundle or the binds anything the checks did not see, and
|
||||
// the module's install roots, for hostedBinds. The log names the fence, never a value.
|
||||
const checkAdmitted = async (answer) => {
|
||||
const refuse = (fence) => {
|
||||
log(`triage host: route refused: ${fence}`);
|
||||
throw new TriageError("triage-route-invalid", "the model route's answer is not usable");
|
||||
};
|
||||
let a;
|
||||
try { a = structuredClone(answer); } catch { refuse("shape"); }
|
||||
if (!plainObject(a) || typeof a.adapter !== "string" || !["marker", "exit"].includes(a.turnEnd)
|
||||
|| typeof a.model !== "string" || !a.model || typeof a.harness !== "string" || !HARNESS_NAME.test(a.harness)
|
||||
|| (a.provider !== undefined && typeof a.provider !== "string")) refuse("shape");
|
||||
let mod = null;
|
||||
try { mod = await harnessModule(a.harness); } catch { mod = null; }
|
||||
const declared = mod?.envNames;
|
||||
if (!Array.isArray(declared) || typeof mod.installRoots !== "function") refuse("module");
|
||||
const he = a.harnessEnv;
|
||||
if (!plainObject(he) || Object.keys(he).length > HARNESS_ENV_MAX) refuse("harness-env");
|
||||
for (const [k, v] of Object.entries(he)) if (refusedHarnessName(k) || !declared.includes(k) || !cleanValue(v)) refuse("harness-env");
|
||||
const re = a.env ?? {};
|
||||
if (!plainObject(re) || Object.entries(re).some(([k, v]) => !ROUTE_ENV_ALLOW.includes(k) || !cleanValue(v))) refuse("route-env");
|
||||
const url = LOOPBACK_URL.exec(re.ANTHROPIC_BASE_URL ?? "");
|
||||
if (!url || Number(url[1]) > 65535 || (proxyPort !== null && Number(url[1]) !== proxyPort)) refuse("route-env");
|
||||
if (tokenShape && !(typeof re.ANTHROPIC_AUTH_TOKEN === "string" && tokenShape.test(re.ANTHROPIC_AUTH_TOKEN))) refuse("route-env");
|
||||
let installRoots;
|
||||
try { installRoots = structuredClone(mod.installRoots({ env })); } catch { refuse("module"); }
|
||||
try {
|
||||
checkHostedRoute({ adapter: a.adapter, harnessMounts: a.harnessMounts ?? [], installRoots, protectedPaths, codeRoot, codeHolder: home, runDir: paths.agent, node });
|
||||
} catch (e) {
|
||||
if (e?.code === "hosted-binds-refused") refuse(e.fence);
|
||||
throw e;
|
||||
}
|
||||
// Where the harness keeps its session files, as the bus binds it (the module's transcript()).
|
||||
// Only a path inside the run directory is read; anything else reads as no transcript.
|
||||
let transcriptRoot = null;
|
||||
try {
|
||||
const r = typeof mod.transcript === "function" ? mod.transcript({ runDir: paths.agent }) : null;
|
||||
if (typeof r === "string" && r.startsWith(paths.agent + "/") && !r.split("/").includes("..")) transcriptRoot = r;
|
||||
} catch { transcriptRoot = null; }
|
||||
return { admitted: a, installRoots, transcriptRoot };
|
||||
};
|
||||
|
||||
const start = async (admitted) => {
|
||||
checkAdmitted(admitted);
|
||||
// #1873 triage-cap. A baseline or a reading of the transcript, with the bus's codes.
|
||||
const sample = async (u) => {
|
||||
if (u.root === null) return { status: "unknown", code: "no-transcript" };
|
||||
try { return snapshot(await usageSource.readTurn({ harness: u.harness, root: u.root })); }
|
||||
catch { return { status: "unknown", code: "source-failed" }; }
|
||||
};
|
||||
// A turn id the runner names, or "unnamed". The first begin for a turn wins, as on the bus.
|
||||
const turnKey = (turn) => (typeof turn === "string" && USAGE_ID.test(turn) ? turn : null);
|
||||
const usageBegin = (u, turn) => {
|
||||
const named = turnKey(turn);
|
||||
if (!named) log("triage host: usage.begin without a usable turn id");
|
||||
const k = named ?? "unnamed";
|
||||
u.inTurn = true;
|
||||
if (!u.baselines.has(k)) u.baselines.set(k, sample(u));
|
||||
};
|
||||
// One line per turn: the delta since its baseline, known or unknown.
|
||||
const finishTurn = async (u, k) => {
|
||||
const base = u.baselines.has(k) ? await u.baselines.get(k) : null;
|
||||
u.baselines.delete(k);
|
||||
const result = delta(base, await sample(u));
|
||||
const known = result.status === "known";
|
||||
if (!known) log(`triage host: turn usage unknown: ${result.code}`);
|
||||
const line = { at: now().toISOString(), run: u.run, turn: k, status: result.status,
|
||||
input: known ? result.input : null, output: known ? result.output : null, cacheRead: known ? result.cacheRead : null, cacheWrite: known ? result.cacheWrite : null };
|
||||
try {
|
||||
appendUsage(usagePath(dataRoot), line);
|
||||
} catch (e) {
|
||||
usageLost = true;
|
||||
log(`triage host: usage line not written: ${e?.code ?? "error"}`);
|
||||
}
|
||||
};
|
||||
const usageEnd = async (u, turn) => {
|
||||
u.inTurn = false;
|
||||
const named = turnKey(turn);
|
||||
if (!named) log("triage host: usage.end without a usable turn id");
|
||||
if (named && u.ended.has(named)) { log("triage host: usage.end repeated for a turn; not counted twice"); return; }
|
||||
if (named) u.ended.add(named);
|
||||
await finishTurn(u, named ?? "unnamed");
|
||||
};
|
||||
const chainUsage = (fn) => (usageChain = usageChain.then(fn).catch((e) => log(`triage host: usage failed: ${e?.code ?? "error"}`)));
|
||||
|
||||
// The stored cap's state, read from the store and the usage file each time.
|
||||
// { state: "not set" } no cap stored
|
||||
// { state: "route-unreadable" } | { state: "usage-unreadable", cap, resetsAt? } refused
|
||||
// { state: "under" | "over", cap, usageToday, resetsAt }
|
||||
const capNow = () => {
|
||||
let stored;
|
||||
try { stored = readRoute(dataRoot); } catch { return { state: "route-unreadable" }; }
|
||||
if (!stored) return { state: "not set" };
|
||||
const at = now().getTime();
|
||||
let resetsAt;
|
||||
try { resetsAt = new Date(nextReset(stored.cap, at)).toISOString(); } catch { return { state: "usage-unreadable", cap: stored.cap }; }
|
||||
if (usageLost) return { state: "usage-unreadable", cap: stored.cap, resetsAt };
|
||||
let read;
|
||||
try { read = readUsage(usagePath(dataRoot)); } catch { return { state: "usage-unreadable", cap: stored.cap, resetsAt }; }
|
||||
for (const b of read.bad) {
|
||||
const key = `${read.ino}:${b.line}`;
|
||||
if (badSeen.has(key)) continue;
|
||||
badSeen.add(key);
|
||||
log(`triage host: usage.jsonl line ${b.line} is not a usage line; it counts as an unknown turn`);
|
||||
}
|
||||
try {
|
||||
const s = capState({ cap: stored.cap, records: read.records, now: at });
|
||||
return { state: s.allow ? "under" : "over", cap: stored.cap, usageToday: s.usageToday, resetsAt: s.resetsAt };
|
||||
} catch { return { state: "usage-unreadable", cap: stored.cap, resetsAt }; }
|
||||
};
|
||||
const unsafeRoot = () => {
|
||||
let st;
|
||||
try { st = lstatSync(paths.root); } catch { return false; }
|
||||
return st.isSymbolicLink() || !st.isDirectory() || (typeof process.getuid === "function" && st.uid !== process.getuid());
|
||||
};
|
||||
const refusedByCap = (c) => c.state === "over" || c.state === "usage-unreadable" || c.state === "route-unreadable";
|
||||
// Before the runner takes messages or starts a message's turn: over the cap, every message still
|
||||
// queued gets a host line and leaves the queue, and the runner's loop ends. That holds with nothing
|
||||
// queued too: an idle session over the cap ends at its next receive, and the next message is
|
||||
// refused by admit. A state read inside a turn (a continuation) is not stopped: the turn finishes.
|
||||
const capStop = (verb) => {
|
||||
const u = usage;
|
||||
if (verb === "state" && u?.inTurn) return false;
|
||||
const c = capNow();
|
||||
if (!refusedByCap(c)) return false;
|
||||
log(`triage host: stopping the session at ${verb}: ${c.state === "over" ? "over the daily token cap" : "usage or cap not read"}`);
|
||||
// A message whose turn began and was never answered says so; one never begun says why not.
|
||||
const begun = (id) => Boolean(u && (u.baselines.has(id) || u.ended.has(id)));
|
||||
for (const m of queue.splice(0)) {
|
||||
const text = begun(m.id) ? UNANSWERED : c.state === "over" ? overCapNotTaken(c.resetsAt) : USAGE_NOT_READ;
|
||||
try { append({ at: now().toISOString(), from: "host", in_reply_to: m.id, text }); }
|
||||
catch { log("triage host: transcript write failed"); }
|
||||
}
|
||||
return true;
|
||||
};
|
||||
|
||||
const start = async (answer) => {
|
||||
const { admitted, installRoots, transcriptRoot } = await checkAdmitted(answer);
|
||||
ensureDirs();
|
||||
writeBundle(admitted);
|
||||
const run = writeBundle(admitted);
|
||||
const u = { run, root: transcriptRoot, harness: admitted.harness, baselines: new Map(), ended: new Set(), inTurn: false };
|
||||
usage = u;
|
||||
cap = randomBytes(32).toString("hex");
|
||||
relay = await createRelay({
|
||||
socket: join(paths.agent, "relay.sock"), cap,
|
||||
@@ -297,17 +517,33 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
|
||||
const i = queue.findIndex((m) => m.id === id);
|
||||
if (i >= 0) queue.splice(i, 1);
|
||||
},
|
||||
// A reply that names a queued message answers it: the message leaves the queue here, so a
|
||||
// session that ends before its message.read leaves no "unanswered" line under an answer.
|
||||
onSend: ({ body, in_reply_to }) => {
|
||||
// An answer (class RESULT) that names a queued message answers it: the message leaves the queue
|
||||
// here, so a session that ends before its message.read leaves no "unanswered" line under an
|
||||
// answer. A note in any other class stays in the transcript but leaves the message queued, so
|
||||
// a session that ends after a progress note still gets the "unanswered" line. The model's tool
|
||||
// can also pick class RESULT; such a note settles the message as the runner's answer would.
|
||||
onSend: ({ body, in_reply_to, result }) => {
|
||||
const { text, cards, dropped } = extractCards(body, orgIds(), { devOwner: devOwner === true });
|
||||
append({ at: now().toISOString(), from: "triage", text, cards, dropped, ...(in_reply_to ? { in_reply_to } : {}) });
|
||||
const i = in_reply_to ? queue.findIndex((m) => m.id === in_reply_to) : -1;
|
||||
const i = in_reply_to && result === true ? queue.findIndex((m) => m.id === in_reply_to) : -1;
|
||||
if (i >= 0) queue.splice(i, 1);
|
||||
},
|
||||
usage: (phase, turn) => chainUsage(() => (phase === "begin" ? usageBegin(u, turn) : usageEnd(u, turn))),
|
||||
stopped: capStop,
|
||||
log,
|
||||
});
|
||||
const list = hostedBinds({ runDir: paths.agent, healthDir: paths.health, adapter: admitted.adapter, harnessMounts: admitted.harnessMounts ?? [], relaySocket: join(paths.agent, "relay.sock"), node, readOnly });
|
||||
let list;
|
||||
try {
|
||||
list = hostedBinds({ runDir: paths.agent, healthDir: paths.health, adapter: admitted.adapter, harnessMounts: admitted.harnessMounts ?? [], installRoots, protectedPaths, codeRoot, codeHolder: home, relaySocket: join(paths.agent, "relay.sock"), node, readOnly });
|
||||
} catch (e) {
|
||||
if (e?.code !== "hosted-binds-refused") throw e;
|
||||
log(`triage host: route refused at the binds: ${e.fence}`);
|
||||
// Nothing of the refused start stays: the relay, its socket, the bundle and session.json go.
|
||||
relay?.close();
|
||||
relay = null;
|
||||
for (const f of ["relay.sock", "session.json", "bundle"]) rmSync(join(paths.agent, f), { recursive: true, force: true });
|
||||
throw new TriageError("triage-route-invalid", "the model route's answer is not usable");
|
||||
}
|
||||
const argv = hostedArgv(list, paths.agent, [node, RUNNER, paths.agent], { bwrap });
|
||||
const out = openSync(paths.log, "a", 0o600);
|
||||
try {
|
||||
@@ -328,6 +564,9 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
|
||||
if (child !== mine) return;
|
||||
lastExit = { code: Number.isInteger(code) ? code : null, signal: signal ?? null, at: now().toISOString() };
|
||||
child = null;
|
||||
// A turn the session began and never ended (killed, crashed) still spent tokens: it gets its line.
|
||||
// The open turns are read when this runs, after any usage.end already in the chain.
|
||||
chainUsage(async () => { for (const k of [...u.baselines.keys()]) await finishTurn(u, k); });
|
||||
// Every message still queued leaves it, so none holds a slot with no session to answer it, and
|
||||
// the transcript says which: one the session took and never answered, or one it never read.
|
||||
for (const m of queue.splice(0)) {
|
||||
@@ -357,6 +596,15 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
|
||||
if (!route || typeof route.admit !== "function") return { ok: false, code: "triage-no-route" };
|
||||
const held = capUntil();
|
||||
if (held !== null) return { ok: false, code: "triage-over-cap", until: held };
|
||||
// A turn still being counted (a session that just ended mid-turn, or a usage.end in flight) is in
|
||||
// the file before the count is read.
|
||||
await usageChain;
|
||||
const c = capNow();
|
||||
// The store sits in the Triage directory: one that is a link or not the Console user's is the
|
||||
// unsafe directory start would refuse, not an unreadable route.
|
||||
if (c.state === "route-unreadable") return { ok: false, code: unsafeRoot() ? "triage-dir-unsafe" : "triage-route-unreadable" };
|
||||
if (c.state === "usage-unreadable") return { ok: false, code: "triage-usage-unreadable" };
|
||||
if (c.state === "over") return { ok: false, code: "triage-over-cap", until: c.resetsAt };
|
||||
let a;
|
||||
try { a = await route.admit(); } catch { return { ok: false, code: "triage-route-unreadable" }; }
|
||||
if (a?.ok === true) return a;
|
||||
@@ -375,9 +623,15 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
|
||||
get pid() { return child?.pid ?? null; },
|
||||
routeSet: () => Boolean(route && typeof route.admit === "function"),
|
||||
async state() {
|
||||
const c = capNow();
|
||||
const unset = c.state === "not set";
|
||||
return {
|
||||
route: this.routeSet() ? "set" : "not set",
|
||||
overCapUntil: capUntil(),
|
||||
overCapUntil: c.state === "over" ? c.resetsAt : capUntil(),
|
||||
// #1873 triage-cap: today's count against the stored cap. Names and numbers only.
|
||||
usageToday: unset ? "not set" : c.usageToday ?? "not read",
|
||||
cap: unset ? "not set" : c.cap ? { limit: c.cap.limit, reset: c.cap.reset, timeZone: c.cap.timeZone } : "not read",
|
||||
resetsAt: unset ? "not set" : c.resetsAt ?? "not read",
|
||||
running: child !== null,
|
||||
lastExit,
|
||||
snapshot: last,
|
||||
@@ -394,7 +648,7 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
|
||||
if (text.length > MESSAGE_MAX) throw new TriageError("triage-message-too-long", `a message holds at most ${MESSAGE_MAX} characters`);
|
||||
if (queue.length >= QUEUE_MAX) throw new TriageError("triage-busy", "Triage has too many messages waiting");
|
||||
const admitted = await admit();
|
||||
if (!admitted.ok) throw Object.assign(new TriageError(admitted.code, admitted.code === "triage-over-cap" ? "Triage is over its daily cap" : admitted.code === "triage-no-route" ? "no model route is set for Triage" : "the model route refused"), admitted.until ? { until: admitted.until } : {});
|
||||
if (!admitted.ok) throw Object.assign(new TriageError(admitted.code, admitted.code === "triage-over-cap" ? "Triage is over its daily cap" : admitted.code === "triage-no-route" ? "no model route is set for Triage" : admitted.code === "triage-usage-unreadable" ? "Triage's token usage could not be read" : admitted.code === "triage-route-unreadable" ? "the model route could not be read" : admitted.code === "triage-dir-unsafe" ? "a Triage directory is not a real directory" : "the model route refused"), admitted.until ? { until: admitted.until } : {});
|
||||
await refresh();
|
||||
const at = now().toISOString();
|
||||
const id = `owner-${randomBytes(6).toString("hex")}`;
|
||||
@@ -406,7 +660,7 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
|
||||
catch (e) {
|
||||
const i = queue.findIndex((m) => m.id === id);
|
||||
if (i >= 0) queue.splice(i, 1);
|
||||
try { append({ at: now().toISOString(), from: "host", in_reply_to: id, text: NOT_STARTED }); } catch {}
|
||||
try { append({ at: now().toISOString(), from: "host", in_reply_to: id, text: e?.code === "triage-route-invalid" ? ROUTE_REFUSED : NOT_STARTED }); } catch {}
|
||||
relay?.close();
|
||||
relay = null;
|
||||
log(`triage host: start failed: ${e.code ?? "error"}`);
|
||||
@@ -426,6 +680,7 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
|
||||
}
|
||||
relay?.close();
|
||||
relay = null;
|
||||
await usageChain;
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -31,6 +31,7 @@ const MESSAGES = Object.freeze({
|
||||
"triage-route-refused": "the model route refused to start Triage",
|
||||
"triage-route-invalid": "the model route's answer is not usable",
|
||||
"triage-route-unreadable": "the model route could not be read",
|
||||
"triage-usage-unreadable": "Triage's token usage could not be read, so it takes no message",
|
||||
"triage-over-cap": "Triage is over its daily token cap and starts again after the time shown",
|
||||
"triage-busy": "Triage has too many messages waiting",
|
||||
"triage-dir-unsafe": "a Triage directory is not safe to use",
|
||||
@@ -69,6 +70,8 @@ export async function triageView(triage, { send, now = Date.now } = {}) {
|
||||
...(s.route === "set" ? {} : { routeMessage: NO_ROUTE_MESSAGE }),
|
||||
overCapUntil: s.overCapUntil ?? null,
|
||||
...(s.overCapUntil ? { capMessage: capMessage(s.overCapUntil) } : {}),
|
||||
// #1873 triage-cap: today's count, the stored cap and the next reset, or "not set" / "not read".
|
||||
...(Object.hasOwn(s, "usageToday") ? { usageToday: s.usageToday, cap: s.cap, resetsAt: s.resetsAt } : {}),
|
||||
running: s.running,
|
||||
lastExit: s.lastExit,
|
||||
snapshot: s.snapshot,
|
||||
|
||||
@@ -0,0 +1,271 @@
|
||||
// #1873 triage-route-store: the owner-only file that holds Triage's model route and daily cap,
|
||||
// <dataRoot>/triage/route.json (design report reports/triage-route-design.md section 1, option C).
|
||||
//
|
||||
// { version, revision, model, harness, cap: { limit, since, reset, timeZone }, updatedAt, updatedBy }
|
||||
//
|
||||
// Names only: a catalogue model key and a harness name. No env value and no key ever enter this
|
||||
// file; the host derives env, mounts and the adapter itself (a later slice). Nothing here is wired
|
||||
// into the Triage host or the Console yet: the Console keeps `route: null`.
|
||||
//
|
||||
// Fail closed. The triage directory is 0700 and route.json 0600, both owned by the running user;
|
||||
// a link, a hard link, a non-file, a wrong mode or owner, an unknown field or an unknown time zone
|
||||
// refuses. A write is compare-and-set on `revision` under an exclusive lock file, atomic (a 0600
|
||||
// temp file in the same directory, fsync, rename, fsync of the directory), and appends one line
|
||||
// to route-receipts.jsonl. Files and the directory the store makes get their modes set on the open
|
||||
// descriptor, so the owner's umask cannot leave them unreadable. A writer that dies leaves its lock; the next write refuses and names
|
||||
// it, and the person removes it by hand, as with the organization lifecycle fence.
|
||||
//
|
||||
// Choices Jason may still reverse each live in one place: the file location (routePaths), the
|
||||
// field names (ROUTE_FIELDS, CAP_FIELDS) and the reset rule (capFor).
|
||||
import { closeSync, constants, fchmodSync, fstatSync, fsyncSync, lstatSync, mkdirSync, openSync, readFileSync, renameSync, unlinkSync, writeSync } from "node:fs";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { join } from "node:path";
|
||||
import { isDailyModelQuota } from "../../plugins/src/model-quota.mjs";
|
||||
import { processState } from "../../seat/src/process-state.mjs";
|
||||
|
||||
export const ROUTE_VERSION = 1;
|
||||
export const ROUTE_FIELDS = Object.freeze(["version", "revision", "model", "harness", "cap", "updatedAt", "updatedBy"]);
|
||||
export const CAP_FIELDS = Object.freeze(["limit", "since", "reset", "timeZone"]);
|
||||
// The catalogue's model key shape (packages/bus/src/model-catalogue.mjs validId) and the Triage
|
||||
// host's harness name shape (packages/webui/src/triage-host.mjs HARNESS_NAME).
|
||||
const MODEL_KEY = /^[a-z0-9][a-z0-9._-]{0,63}$/;
|
||||
const HARNESS_NAME = /^[a-z][a-z0-9-]{0,31}$/;
|
||||
const ACTOR = /^[a-z][a-z0-9-]{0,31}$/;
|
||||
const ISO = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$/;
|
||||
// isDailyModelQuota checks one leaf under a twelve-hex key; the Triage cap has no key of its own.
|
||||
const LEAF_KEY = "000000000000";
|
||||
const MAX_BYTES = 64 * 1024;
|
||||
const INPUT_FIELDS = Object.freeze(["model", "harness", "limit", "timeZone", "by", "revision"]);
|
||||
|
||||
export class RouteStoreError extends Error {
|
||||
constructor(code, message) {
|
||||
super(message);
|
||||
this.name = "RouteStoreError";
|
||||
this.code = code;
|
||||
}
|
||||
}
|
||||
const unreadable = (why) => new RouteStoreError("triage-route-unreadable", `the Triage route store is unreadable: ${why}`);
|
||||
const invalid = (why) => new RouteStoreError("triage-route-invalid", `the Triage route is invalid: ${why}`);
|
||||
|
||||
// The file location, in one place.
|
||||
export function routePaths(dataRoot) {
|
||||
const dir = join(dataRoot, "triage");
|
||||
return { dir, file: join(dir, "route.json"), receipts: join(dir, "route-receipts.jsonl"), lock: join(dir, "route.lock") };
|
||||
}
|
||||
|
||||
// The reset rule, in one place: the day resets at 00:00 in the owner's zone, which has no default.
|
||||
// A first cap counts from now; a changed cap keeps its stored `since`, as an organization's token
|
||||
// limits do (packages/organization/src/launch-settings.mjs stamp), so lowering the cap never drops
|
||||
// the tokens already spent today. Only a reset would move it, and this slice has no reset verb.
|
||||
export function capFor({ limit, timeZone, previous, now }) {
|
||||
return { limit, since: previous ? previous.since : now, reset: "00:00", timeZone };
|
||||
}
|
||||
|
||||
const plain = (v) => v !== null && typeof v === "object" && !Array.isArray(v) && Object.getPrototypeOf(v) === Object.prototype;
|
||||
const exactKeys = (v, keys) => plain(v) && Object.keys(v).length === keys.length && keys.every((k) => Object.hasOwn(v, k));
|
||||
const canonicalTime = (s) => typeof s === "string" && ISO.test(s) && !Number.isNaN(Date.parse(s)) && new Date(s).toISOString() === s;
|
||||
// The same zone rule the stored cap is held to (isDailyModelQuota: its zone shape, then Intl).
|
||||
const knownZone = (zone) => typeof zone === "string"
|
||||
&& isDailyModelQuota(LEAF_KEY, { limit: 0, since: "2000-01-01T00:00:00.000Z", reset: "00:00", timeZone: zone });
|
||||
|
||||
export function validRoute(doc) {
|
||||
return exactKeys(doc, ROUTE_FIELDS)
|
||||
&& doc.version === ROUTE_VERSION
|
||||
&& Number.isSafeInteger(doc.revision) && doc.revision >= 1
|
||||
&& typeof doc.model === "string" && MODEL_KEY.test(doc.model)
|
||||
&& typeof doc.harness === "string" && HARNESS_NAME.test(doc.harness)
|
||||
// isDailyModelQuota also checks that the zone is one Intl knows.
|
||||
&& exactKeys(doc.cap, CAP_FIELDS) && isDailyModelQuota(LEAF_KEY, doc.cap)
|
||||
&& canonicalTime(doc.updatedAt)
|
||||
&& typeof doc.updatedBy === "string" && ACTOR.test(doc.updatedBy);
|
||||
}
|
||||
|
||||
const uidOf = (opts) => opts.uid ?? process.getuid();
|
||||
// The owner the files are held to: the same user, unless a test checks the file owner alone.
|
||||
const fileUidOf = (opts) => opts.fileUid ?? uidOf(opts);
|
||||
|
||||
// The triage directory: a real directory, the running user's, mode 0700. Returns false when it is
|
||||
// missing and `create` is off; with `create` it makes it (never the data root above it).
|
||||
function checkDir(dir, uid, { create }) {
|
||||
let st;
|
||||
try { st = lstatSync(dir); }
|
||||
catch (e) {
|
||||
if (e.code !== "ENOENT") throw unreadable("the triage directory cannot be read");
|
||||
if (!create) return false;
|
||||
try { mkdirSync(dir, { mode: 0o700 }); } catch { throw unreadable("the triage directory cannot be made"); }
|
||||
// The umask may have narrowed the mode; set it on the directory just made, never through a link.
|
||||
try {
|
||||
const fd = openSync(dir, constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW);
|
||||
try { fchmodSync(fd, 0o700); } finally { closeSync(fd); }
|
||||
} catch { throw unreadable("the triage directory cannot be made"); }
|
||||
st = lstatSync(dir);
|
||||
}
|
||||
if (st.isSymbolicLink()) throw unreadable("the triage directory is a link");
|
||||
if (!st.isDirectory()) throw unreadable("the triage directory is not a directory");
|
||||
if (st.uid !== uid) throw unreadable("the triage directory is not owned by this user");
|
||||
if ((st.mode & 0o777) !== 0o700) throw unreadable("the triage directory mode is not 0700");
|
||||
return true;
|
||||
}
|
||||
|
||||
// An opened file's own checks: a regular file, one link, the running user's, mode 0600.
|
||||
function checkFile(fd, uid, what) {
|
||||
const st = fstatSync(fd);
|
||||
if (!st.isFile()) throw unreadable(`${what} is not a regular file`);
|
||||
if (st.nlink !== 1) throw unreadable(`${what} has more than one link`);
|
||||
if (st.uid !== uid) throw unreadable(`${what} is not owned by this user`);
|
||||
if ((st.mode & 0o777) !== 0o600) throw unreadable(`${what} mode is not 0600`);
|
||||
return st;
|
||||
}
|
||||
|
||||
function readFile(file, uid) {
|
||||
let fd;
|
||||
try { fd = openSync(file, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK); }
|
||||
catch (e) {
|
||||
if (e.code === "ENOENT") return null;
|
||||
if (e.code === "ELOOP") throw unreadable("route.json is a link");
|
||||
throw unreadable("route.json cannot be opened");
|
||||
}
|
||||
try {
|
||||
const st = checkFile(fd, uid, "route.json");
|
||||
if (st.size > MAX_BYTES) throw unreadable("route.json is too large");
|
||||
let doc;
|
||||
try { doc = JSON.parse(readFileSync(fd, "utf8")); } catch { throw unreadable("route.json is not JSON"); }
|
||||
if (!validRoute(doc)) throw unreadable("route.json does not match the route shape");
|
||||
return doc;
|
||||
} finally { closeSync(fd); }
|
||||
}
|
||||
|
||||
// The stored route, or null when none is set. Throws triage-route-unreadable on anything unsafe.
|
||||
export function readRoute(dataRoot, opts = {}) {
|
||||
const { dir, file } = routePaths(dataRoot);
|
||||
if (!checkDir(dir, uidOf(opts), { create: false })) return null;
|
||||
return readFile(file, fileUidOf(opts));
|
||||
}
|
||||
|
||||
function checkInput(input) {
|
||||
if (!plain(input)) throw invalid("the input is not an object");
|
||||
for (const k of Object.keys(input)) if (!INPUT_FIELDS.includes(k)) throw invalid(`unknown field ${k}`);
|
||||
if (typeof input.model !== "string" || !MODEL_KEY.test(input.model)) throw invalid("model must be a catalogue model key");
|
||||
if (typeof input.harness !== "string" || !HARNESS_NAME.test(input.harness)) throw invalid("harness must be a harness name");
|
||||
if (!Number.isSafeInteger(input.limit) || input.limit < 0) throw invalid("the cap must be a whole number of tokens, 0 or more");
|
||||
if (input.timeZone === undefined) throw invalid("a time zone is required");
|
||||
if (!knownZone(input.timeZone)) throw invalid("the time zone is unknown");
|
||||
if (typeof input.by !== "string" || !ACTOR.test(input.by)) throw invalid("the actor name is invalid");
|
||||
if (input.revision !== undefined && (!Number.isSafeInteger(input.revision) || input.revision < 0)) throw invalid("revision must be a whole number, 0 or more");
|
||||
}
|
||||
|
||||
function holderLine(lock) {
|
||||
let holder = null;
|
||||
try {
|
||||
const fd = openSync(lock, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK);
|
||||
try { holder = JSON.parse(readFileSync(fd, "utf8")); } finally { closeSync(fd); }
|
||||
} catch { holder = null; }
|
||||
if (!Number.isSafeInteger(holder?.pid) || holder.pid <= 0) return `another write holds ${lock}, or a writer left it; remove it by hand once no write is running`;
|
||||
let alive;
|
||||
try { alive = processState(holder.pid).alive; } catch { alive = null; }
|
||||
if (alive === false) return `a writer that is not running (pid ${holder.pid}) left ${lock}; remove it by hand`;
|
||||
return `another write (pid ${holder.pid}) holds ${lock}; try again when it ends`;
|
||||
}
|
||||
|
||||
function takeLock(lock) {
|
||||
let fd;
|
||||
try { fd = openSync(lock, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW, 0o600); }
|
||||
catch (e) {
|
||||
if (e.code === "EEXIST") throw new RouteStoreError("triage-route-busy", `the Triage route is busy: ${holderLine(lock)}`);
|
||||
throw unreadable("the lock cannot be made");
|
||||
}
|
||||
try {
|
||||
writeSync(fd, `${JSON.stringify({ pid: process.pid, at: new Date().toISOString() })}\n`);
|
||||
return { fd, st: fstatSync(fd) };
|
||||
} catch (e) {
|
||||
closeSync(fd);
|
||||
try { unlinkSync(lock); } catch {}
|
||||
throw unreadable("the lock cannot be written");
|
||||
}
|
||||
}
|
||||
|
||||
function dropLock(lock, held) {
|
||||
try {
|
||||
const now = lstatSync(lock);
|
||||
if (now.dev === held.st.dev && now.ino === held.st.ino) unlinkSync(lock);
|
||||
} catch {}
|
||||
closeSync(held.fd);
|
||||
}
|
||||
|
||||
// The receipts file, opened for append before the route changes, so a bad one refuses first. An
|
||||
// existing file is held to its checks as it is; only a file made here gets its mode set.
|
||||
function openReceipts(receipts, uid) {
|
||||
const flags = constants.O_WRONLY | constants.O_APPEND | constants.O_NOFOLLOW | constants.O_NONBLOCK;
|
||||
let fd;
|
||||
try { fd = openSync(receipts, flags); }
|
||||
catch (e) {
|
||||
if (e.code === "ELOOP") throw unreadable("route-receipts.jsonl is a link");
|
||||
if (e.code !== "ENOENT") throw unreadable("route-receipts.jsonl cannot be opened");
|
||||
try { fd = openSync(receipts, flags | constants.O_CREAT | constants.O_EXCL, 0o600); }
|
||||
catch { throw unreadable("route-receipts.jsonl cannot be made"); }
|
||||
try { fchmodSync(fd, 0o600); } catch { closeSync(fd); throw unreadable("route-receipts.jsonl cannot be made"); }
|
||||
}
|
||||
try { checkFile(fd, uid, "route-receipts.jsonl"); } catch (e) { closeSync(fd); throw e; }
|
||||
return fd;
|
||||
}
|
||||
|
||||
const summary = (doc) => (doc ? { model: doc.model, harness: doc.harness, cap: doc.cap } : null);
|
||||
|
||||
// Set the route. input: { model, harness, limit, timeZone, by, revision? }. With `revision` the
|
||||
// write happens only if the stored revision equals it (0: only when no route is set yet); without
|
||||
// it, the revision read under the lock. Returns the new route.
|
||||
// opts.now, opts.uid and opts.fileUid are for tests; opts.hook(stage) is a test seam at "write"
|
||||
// (inside the temp write), "crash" (after the temp file is written), "before-rename" and
|
||||
// "after-rename" (before the receipt line). A failure after the rename refuses as
|
||||
// triage-route-unrecorded: the route did change, and its receipt line is missing.
|
||||
export function writeRoute(dataRoot, input, opts = {}) {
|
||||
checkInput(input);
|
||||
const { dir, file, receipts, lock } = routePaths(dataRoot);
|
||||
const uid = uidOf(opts);
|
||||
const fileUid = fileUidOf(opts);
|
||||
const now = (opts.now ?? (() => new Date()))().toISOString();
|
||||
const hook = opts.hook ?? (() => {});
|
||||
checkDir(dir, uid, { create: true });
|
||||
const held = takeLock(lock);
|
||||
try {
|
||||
const current = readFile(file, fileUid);
|
||||
const stored = current?.revision ?? 0;
|
||||
if (input.revision !== undefined && input.revision !== stored) {
|
||||
throw new RouteStoreError("triage-route-stale", `the Triage route changed: the stored revision is ${stored}, not ${input.revision}`);
|
||||
}
|
||||
const next = {
|
||||
version: ROUTE_VERSION, revision: stored + 1, model: input.model, harness: input.harness,
|
||||
cap: capFor({ limit: input.limit, timeZone: input.timeZone, previous: current?.cap ?? null, now }),
|
||||
updatedAt: now, updatedBy: input.by,
|
||||
};
|
||||
if (!validRoute(next)) throw invalid("the new route does not match the route shape");
|
||||
const receiptFd = openReceipts(receipts, fileUid);
|
||||
try {
|
||||
const temp = join(dir, `.route.json.${process.pid}.${randomBytes(6).toString("hex")}.tmp`);
|
||||
let tfd;
|
||||
try { tfd = openSync(temp, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW, 0o600); }
|
||||
catch { throw unreadable("route.json cannot be written"); }
|
||||
try {
|
||||
try {
|
||||
fchmodSync(tfd, 0o600);
|
||||
hook("write");
|
||||
writeSync(tfd, `${JSON.stringify(next, null, 2)}\n`);
|
||||
fsyncSync(tfd);
|
||||
} finally { closeSync(tfd); }
|
||||
} catch { try { unlinkSync(temp); } catch {} throw unreadable("route.json cannot be written"); }
|
||||
hook("crash");
|
||||
hook("before-rename");
|
||||
try { renameSync(temp, file); } catch (e) { try { unlinkSync(temp); } catch {} throw unreadable("route.json cannot be replaced"); }
|
||||
try {
|
||||
hook("after-rename");
|
||||
const dfd = openSync(dir, constants.O_RDONLY | constants.O_DIRECTORY);
|
||||
try { fsyncSync(dfd); } finally { closeSync(dfd); }
|
||||
writeSync(receiptFd, `${JSON.stringify({ at: now, by: input.by, revision: next.revision, from: summary(current), to: summary(next) })}\n`);
|
||||
fsyncSync(receiptFd);
|
||||
} catch {
|
||||
throw new RouteStoreError("triage-route-unrecorded", `the Triage route changed to revision ${next.revision}, but its receipt line was not written`);
|
||||
}
|
||||
} finally { closeSync(receiptFd); }
|
||||
return next;
|
||||
} finally { dropLock(lock, held); }
|
||||
}
|
||||
@@ -39,7 +39,7 @@ export const BUS_STATES = Object.freeze(["running", "not-answering", "stopped",
|
||||
const UNIT_ACTIVE = Object.freeze(["active", "inactive", "failed", "activating", "deactivating", "reloading", "unknown"]);
|
||||
export const READ_TIMEOUT_MS = 5000;
|
||||
export const MAX_ORGS = 200;
|
||||
// At most this many organizations are read at once; each read can start a node child for its config check.
|
||||
// At most this many organizations' config checks run at once; each starts a node child.
|
||||
export const ORG_READS_AT_ONCE = 4;
|
||||
|
||||
// What a failed system config check may say. Each refusal code the validator prints as its first
|
||||
@@ -190,8 +190,11 @@ async function mapLimit(items, limit, fn) {
|
||||
}
|
||||
|
||||
// The organization stage runs under one deadline, `timeoutMs` from its start, however many
|
||||
// organizations there are: an organization whose reads have not finished by then is "not read", and
|
||||
// one not yet begun is not read at all. A whole snapshot takes at most about two timeouts.
|
||||
// organizations there are: a read that has not finished by then is "not read", and one not yet
|
||||
// begun is not read at all. The bus and processing reads are socket reads and begin for every
|
||||
// organization at once. Only the config checks, a node child each, wait for one of
|
||||
// ORG_READS_AT_ONCE slots, so a hung check holds back other checks but never another
|
||||
// organization's socket reads. A whole snapshot takes at most about two timeouts.
|
||||
export async function composeSnapshot(readers = {}, { now = () => new Date(), timeoutMs = READ_TIMEOUT_MS, clock = Date.now } = {}) {
|
||||
const at = now().toISOString();
|
||||
const read = (name, ...args) => (typeof readers[name] === "function" ? bounded(() => readers[name](...args), timeoutMs) : Promise.resolve(NOT_READ));
|
||||
@@ -200,21 +203,26 @@ export async function composeSnapshot(readers = {}, { now = () => new Date(), ti
|
||||
if (Array.isArray(orgsRaw)) {
|
||||
const active = orgsRaw.filter((o) => o && typeof o === "object" && o.state === "active" && typeof o.id === "string" && ID_PATTERN.test(o.id)).slice(0, MAX_ORGS);
|
||||
const deadline = clock() + timeoutMs;
|
||||
const list = await mapLimit(active, ORG_READS_AT_ONCE, async ({ id }) => {
|
||||
// fn bounded by what is left of the deadline, or NOT_READ once nothing is.
|
||||
const inTime = (fn) => {
|
||||
const left = deadline - clock();
|
||||
if (left <= 0) return { id, bus: NOT_READ, processing: NOT_READ, config: NOT_READ };
|
||||
const readOrg = (name) => (typeof readers[name] === "function" ? bounded(() => readers[name](id), left) : NOT_READ);
|
||||
const [bus, processing, config] = await Promise.all([
|
||||
typeof readers.busStatus === "function"
|
||||
? bounded(async () => {
|
||||
try { return busFields(await readers.busStatus(id)); }
|
||||
catch (e) { return e?.name === "CliError" || e?.dead === true ? busFields(null, true) : NOT_READ; }
|
||||
}, left)
|
||||
: NOT_READ,
|
||||
readOrg("processing"),
|
||||
readOrg("orgConfig"),
|
||||
]);
|
||||
return { id, bus: bus === NOT_READ ? NOT_READ : bus, processing: processingFields(processing), config: configFields(config, ORG_CONFIG_CODES) };
|
||||
return left > 0 ? bounded(fn, left) : Promise.resolve(NOT_READ);
|
||||
};
|
||||
const readOrg = (name, id) => (typeof readers[name] === "function" ? inTime(() => readers[name](id)) : Promise.resolve(NOT_READ));
|
||||
const sockets = Promise.all(active.map(({ id }) => Promise.all([
|
||||
typeof readers.busStatus === "function"
|
||||
? inTime(async () => {
|
||||
try { return busFields(await readers.busStatus(id)); }
|
||||
catch (e) { return e?.name === "CliError" || e?.dead === true ? busFields(null, true) : NOT_READ; }
|
||||
})
|
||||
: NOT_READ,
|
||||
readOrg("processing", id),
|
||||
])));
|
||||
const configs = await mapLimit(active, ORG_READS_AT_ONCE, ({ id }) => readOrg("orgConfig", id));
|
||||
const socketReads = await sockets;
|
||||
const list = active.map(({ id }, i) => {
|
||||
const [bus, processing] = socketReads[i];
|
||||
return { id, bus: bus === NOT_READ ? NOT_READ : bus, processing: processingFields(processing), config: configFields(configs[i], ORG_CONFIG_CODES) };
|
||||
});
|
||||
organizations = { count: list.length, list };
|
||||
}
|
||||
|
||||
@@ -0,0 +1,220 @@
|
||||
// #1873 triage-cap: Triage's token usage and its daily cap.
|
||||
//
|
||||
// The Triage host appends one line per turn to <dataRoot>/triage/usage.jsonl:
|
||||
// { at, run, turn, status, input, output, cacheRead, cacheWrite }
|
||||
// A known turn carries four non-negative safe integers; an unknown turn carries four nulls. The file
|
||||
// sits outside the run directory, so a new start never clears it. It is held to the route store's
|
||||
// file checks: a regular file, one link, the running user's, mode 0600. A link, a non-file, a wrong
|
||||
// mode or owner refuses the read and the append alike, and the append leaves the file as it was.
|
||||
// Lines are only ever appended. Nothing here prunes the file.
|
||||
//
|
||||
// The cap is the daily-quota plugin's own rule (packages/plugins/limit-check/daily-quota), called
|
||||
// with the organization limits shape and the stored cap as its one leaf, for one account named
|
||||
// "triage". No counting rule lives here: the count, the window, the DST handling and the hit all
|
||||
// come from mayLaunch and report. nextReset only asks report when the window turns over.
|
||||
import { closeSync, constants, fchmodSync, fstatSync, fsyncSync, openSync, readSync, writeSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { mayLaunch, report } from "../../plugins/limit-check/daily-quota/index.mjs";
|
||||
|
||||
export const USAGE_ACCOUNT = "triage";
|
||||
export const USAGE_FIELDS = Object.freeze(["at", "run", "turn", "status", "input", "output", "cacheRead", "cacheWrite"]);
|
||||
const KINDS = ["input", "output", "cacheWrite", "cacheRead"];
|
||||
const ISO = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$/;
|
||||
// A run id is the host's own triage-<14 digits>; a turn id is the runner's message id, maybe with
|
||||
// ":retry". Both are held to one short, plain shape so a line never carries free text.
|
||||
export const ID = /^[A-Za-z0-9:._-]{1,80}$/;
|
||||
// 32 MiB holds well over a hundred thousand lines; a larger file refuses rather than stalls a read.
|
||||
export const USAGE_MAX_BYTES = 32 * 1024 * 1024;
|
||||
const HOUR = 3600000;
|
||||
|
||||
export class UsageError extends Error {
|
||||
constructor(why) {
|
||||
super(`the Triage usage file is unreadable: ${why}`);
|
||||
this.name = "UsageError";
|
||||
this.code = "triage-usage-unreadable";
|
||||
}
|
||||
}
|
||||
const unreadable = (why) => new UsageError(why);
|
||||
|
||||
export const usagePath = (dataRoot) => join(dataRoot, "triage", "usage.jsonl");
|
||||
|
||||
const plain = (v) => v !== null && typeof v === "object" && !Array.isArray(v) && Object.getPrototypeOf(v) === Object.prototype;
|
||||
const canonicalTime = (s) => typeof s === "string" && ISO.test(s) && !Number.isNaN(Date.parse(s)) && new Date(s).toISOString() === s;
|
||||
const count = (n) => Number.isSafeInteger(n) && n >= 0;
|
||||
|
||||
// One line's shape, exactly.
|
||||
export function validLine(v) {
|
||||
if (!plain(v) || Object.keys(v).length !== USAGE_FIELDS.length || !USAGE_FIELDS.every((k) => Object.hasOwn(v, k))) return false;
|
||||
if (!canonicalTime(v.at) || typeof v.run !== "string" || !ID.test(v.run) || typeof v.turn !== "string" || !ID.test(v.turn)) return false;
|
||||
if (v.status === "known") return KINDS.every((k) => count(v[k]));
|
||||
if (v.status === "unknown") return KINDS.every((k) => v[k] === null);
|
||||
return false;
|
||||
}
|
||||
|
||||
// An opened file's checks, as the route store holds route.json to.
|
||||
function checkFile(fd, uid) {
|
||||
const st = fstatSync(fd);
|
||||
if (!st.isFile()) throw unreadable("usage.jsonl is not a regular file");
|
||||
if (st.nlink !== 1) throw unreadable("usage.jsonl has more than one link");
|
||||
if (st.uid !== uid) throw unreadable("usage.jsonl is not owned by this user");
|
||||
if ((st.mode & 0o777) !== 0o600) throw unreadable("usage.jsonl mode is not 0600");
|
||||
return st;
|
||||
}
|
||||
|
||||
const uidOf = (opts) => opts.uid ?? process.getuid();
|
||||
|
||||
// Append one line. A torn last line (a write that died) is closed off with a newline first, so the
|
||||
// new line always stands alone and the torn one stays one bad line.
|
||||
export function appendUsage(file, line, opts = {}) {
|
||||
if (!validLine(line)) throw unreadable("the line does not match the usage shape");
|
||||
const text = JSON.stringify(Object.fromEntries(USAGE_FIELDS.map((k) => [k, line[k]]))) + "\n";
|
||||
const flags = constants.O_RDWR | constants.O_APPEND | constants.O_NOFOLLOW | constants.O_NONBLOCK;
|
||||
let fd;
|
||||
try { fd = openSync(file, flags); }
|
||||
catch (e) {
|
||||
if (e.code === "ELOOP") throw unreadable("usage.jsonl is a link");
|
||||
if (e.code !== "ENOENT") throw unreadable("usage.jsonl cannot be opened");
|
||||
try { fd = openSync(file, flags | constants.O_CREAT | constants.O_EXCL, 0o600); }
|
||||
catch { throw unreadable("usage.jsonl cannot be made"); }
|
||||
// The umask may have narrowed the mode; set it on the descriptor just made.
|
||||
try { fchmodSync(fd, 0o600); } catch { closeSync(fd); throw unreadable("usage.jsonl cannot be made"); }
|
||||
}
|
||||
try {
|
||||
const st = checkFile(fd, uidOf(opts));
|
||||
let lead = "";
|
||||
if (st.size > 0) {
|
||||
const last = Buffer.alloc(1);
|
||||
if (readSync(fd, last, 0, 1, st.size - 1) !== 1) throw unreadable("usage.jsonl cannot be read");
|
||||
if (last[0] !== 0x0a) lead = "\n";
|
||||
}
|
||||
try {
|
||||
writeSync(fd, lead + text);
|
||||
fsyncSync(fd);
|
||||
} catch { throw unreadable("usage.jsonl cannot be written"); }
|
||||
} finally { closeSync(fd); }
|
||||
}
|
||||
|
||||
const nulls = { input: null, output: null, cacheRead: null, cacheWrite: null };
|
||||
|
||||
// Every line as a daily-quota record for the triage account. A line that does not parse or does
|
||||
// not match the shape (a torn last line, an edit by hand) is refused: it becomes one unknown turn,
|
||||
// keyed by its line number, dated by the next good line (or the file's change time when none
|
||||
// follows), and is listed in `bad` so the host can log it. No file reads as no turns.
|
||||
export function readUsage(file, opts = {}) {
|
||||
const maxBytes = opts.maxBytes ?? USAGE_MAX_BYTES;
|
||||
let fd;
|
||||
try { fd = openSync(file, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK); }
|
||||
catch (e) {
|
||||
if (e.code === "ENOENT") return { records: [], bad: [] };
|
||||
if (e.code === "ELOOP") throw unreadable("usage.jsonl is a link");
|
||||
throw unreadable("usage.jsonl cannot be opened");
|
||||
}
|
||||
let st;
|
||||
let buf;
|
||||
try {
|
||||
st = checkFile(fd, uidOf(opts));
|
||||
if (st.size > maxBytes) throw unreadable("usage.jsonl is too large");
|
||||
buf = Buffer.alloc(st.size);
|
||||
let got = 0;
|
||||
while (got < st.size) {
|
||||
const n = readSync(fd, buf, got, st.size - got, got);
|
||||
if (n === 0) break;
|
||||
got += n;
|
||||
}
|
||||
buf = buf.subarray(0, got);
|
||||
} finally { closeSync(fd); }
|
||||
const lines = buf.toString("utf8").split("\n");
|
||||
if (lines.at(-1) === "") lines.pop();
|
||||
const parsed = lines.map((text) => {
|
||||
try {
|
||||
const v = JSON.parse(text);
|
||||
return validLine(v) ? v : null;
|
||||
} catch { return null; }
|
||||
});
|
||||
const records = new Array(parsed.length);
|
||||
const bad = [];
|
||||
let nextAt = st.mtime.toISOString();
|
||||
for (let i = parsed.length - 1; i >= 0; i--) {
|
||||
const v = parsed[i];
|
||||
if (v) {
|
||||
records[i] = { account: USAGE_ACCOUNT, ...v };
|
||||
nextAt = v.at;
|
||||
} else {
|
||||
records[i] = { account: USAGE_ACCOUNT, at: nextAt, run: null, turn: `line-${i + 1}`, status: "unknown", ...nulls };
|
||||
bad.unshift({ line: i + 1 });
|
||||
}
|
||||
}
|
||||
return { records, bad, ino: st.ino };
|
||||
}
|
||||
|
||||
// The organization limits shape with the stored cap as its one leaf.
|
||||
export const limitsFor = (cap) => ({ tokens: { daily: { [USAGE_ACCOUNT]: cap } } });
|
||||
const SUBJECT = Object.freeze({ account: USAGE_ACCOUNT });
|
||||
const iso = (ms) => new Date(ms).toISOString();
|
||||
|
||||
// The next reset after nowMs, or after the cap's `since` when that is later. A probe record at that
|
||||
// instant counts while the window holding t still starts at or before it; the reset is the first
|
||||
// millisecond at which report stops counting it. A day is at most 25 hours, so 49 hours always
|
||||
// holds the next one.
|
||||
export function nextReset(cap, nowMs) {
|
||||
const from = Math.max(nowMs, Date.parse(cap.since));
|
||||
const limits = limitsFor(cap);
|
||||
const probe = [{ account: USAGE_ACCOUNT, at: iso(from), run: "probe", turn: "probe", status: "known", input: 1, output: 0, cacheRead: 0, cacheWrite: 0 }];
|
||||
const counts = (t) => report({ limits, records: probe, now: iso(t) })[0].count > 0;
|
||||
let lo = from;
|
||||
let hi = from + 49 * HOUR;
|
||||
if (counts(hi)) throw unreadable("the cap has no reset in the next two days");
|
||||
while (hi - lo > 1) {
|
||||
const mid = Math.floor((lo + hi) / 2);
|
||||
if (counts(mid)) lo = mid; else hi = mid;
|
||||
}
|
||||
return hi;
|
||||
}
|
||||
|
||||
// Today's count, the decision and the next reset for the stored cap.
|
||||
export function capState({ cap, records, now }) {
|
||||
const at = iso(now);
|
||||
const limits = limitsFor(cap);
|
||||
const decision = mayLaunch({ limits, subject: SUBJECT, records, now: at });
|
||||
const [r] = report({ limits, records, now: at });
|
||||
return {
|
||||
allow: decision.allow,
|
||||
hit: decision.allow ? null : decision.hit,
|
||||
usageToday: { count: r.count, ...r.kinds, unknownTurns: r.unknownTurns },
|
||||
resetsAt: iso(nextReset(cap, now)),
|
||||
};
|
||||
}
|
||||
|
||||
// The turn delta, copied from packages/bus/src/usage.mjs (snapshot, delta; not exported there and
|
||||
// that package is not this lane's to change). Same codes, same rules: the turn's usage is the sum
|
||||
// of the transcript entries that are new since the turn began; a changed or missing earlier entry
|
||||
// is "transcript-rewritten", and no new entry is "no-new-usage".
|
||||
const unknown = (code) => ({ status: "unknown", code });
|
||||
export function snapshot(value) {
|
||||
if (value?.status === "unknown") return unknown(/^transcript-(path|size|line|entries|records|changed|parse|usage|harness)$/.test(value.code) ? value.code : "source-unknown");
|
||||
if (value?.status !== "known" || !value.entries || typeof value.entries !== "object" || Array.isArray(value.entries)) return unknown("source-invalid");
|
||||
const entries = Object.entries(value.entries);
|
||||
if (entries.length > 20000) return unknown("source-invalid");
|
||||
for (const [id, v] of entries) {
|
||||
if (id.length > 512 || !v || Object.keys(v).length !== 4 || KINDS.some((k) => !Number.isSafeInteger(v[k]) || v[k] < 0)) return unknown("source-invalid");
|
||||
}
|
||||
return { status: "known", entries: value.entries };
|
||||
}
|
||||
export function delta(before, after) {
|
||||
if (!before || before.status !== "known") return unknown("baseline-unknown");
|
||||
if (after.status !== "known") return after;
|
||||
for (const [id, v] of Object.entries(before.entries)) {
|
||||
if (!Object.hasOwn(after.entries, id) || KINDS.some((k) => after.entries[id][k] !== v[k])) return unknown("transcript-rewritten");
|
||||
}
|
||||
const sum = { input: 0, output: 0, cacheWrite: 0, cacheRead: 0 };
|
||||
let added = false;
|
||||
for (const [id, v] of Object.entries(after.entries)) {
|
||||
if (Object.hasOwn(before.entries, id)) continue;
|
||||
added = true;
|
||||
for (const k of KINDS) {
|
||||
sum[k] += v[k];
|
||||
if (!Number.isSafeInteger(sum[k])) return unknown("usage-overflow");
|
||||
}
|
||||
}
|
||||
return added ? { status: "known", ...sum } : unknown("no-new-usage");
|
||||
}
|
||||
@@ -18,7 +18,7 @@ const at = minutesAgo => new Date(Date.now() - minutesAgo * 60000).toISOString()
|
||||
const HIRE = '0f3c1a2b-4d5e-4f60-8a7b-9c0d1e2f3a4b';
|
||||
const THREAD = '7b1e2c3d-4f50-4a61-8b72-9c83d4e5f6a7';
|
||||
|
||||
function fixture() {
|
||||
function fixture(tweak) {
|
||||
const LIST = [
|
||||
{ id: 'acme', label: 'Acme Studio', theme: 'cobalt' },
|
||||
{ id: 'beta', label: 'Beta Works', theme: 'grove' },
|
||||
@@ -52,6 +52,7 @@ function fixture() {
|
||||
model: {},
|
||||
},
|
||||
};
|
||||
tweak?.(DATA);
|
||||
const value = (id, name) => Promise.resolve(DATA[id]?.[name] ?? (['usage', 'limits'].includes(name) ? {} : []));
|
||||
const opts = id => ({
|
||||
bus: { inbox: () => value(id, 'inbox'), agents: () => value(id, 'agents'), 'thread.claims': () => value(id, 'claims'), tasks: () => value(id, 'tasks'), usage: () => value(id, 'usage'), limits: () => value(id, 'limits'), decisions: () => value(id, 'decisions') },
|
||||
@@ -107,16 +108,26 @@ const STUB = `(() => {
|
||||
if (window.__packsBody) return new Response(window.__packsBody, { status: 200, headers: { 'content-type': 'text/html' } });
|
||||
return answer(200, { writable: false, applied: [{ id: 'coding', origin: 'shipped', project: null, hired: ['research-2'], open: [] }], packs: packs[1] === 'beta' ? [{ id: 'coding', origin: 'shipped', pack: { id: 'coding', label: 'Coding', version: 2, hires: [{ name: 'research-2', definition: 'researcher', reason: 'Research.' }], definitions: { researcher: { template: 'researcher', sha256: '${'a'.repeat(64)}' } } } }] : [] });
|
||||
}
|
||||
// SP19: a test may set __runs[org][name] to the runs the instances read lists for that agent.
|
||||
const inst = u.match(/^\\/api\\/orgs\\/([^/]+)\\/bus\\/instances$/);
|
||||
if (inst && window.__runs?.[inst[1]]) {
|
||||
const got = await real(url, init), v = await got.json();
|
||||
for (const [n, runs] of Object.entries(window.__runs[inst[1]])) if (v?.rows?.[0]?.instances?.[n]) v.rows[0].instances[n].running = runs;
|
||||
return answer(got.status, v);
|
||||
}
|
||||
if (!/\\/agent\\//.test(u) || (init.method || 'GET') === 'GET' && !u.includes('/agent/request')) return real(url, init);
|
||||
window.__posts.push({ url: u, method: init.method || 'GET', body: init.body ?? null });
|
||||
const a = window.__answers.shift();
|
||||
if (!a) throw new TypeError('no answer queued');
|
||||
// SP19: an answer may wait on a promise (\`hold\`) and run a step once it is given (\`then\`).
|
||||
if (a.hold) await a.hold;
|
||||
a.then?.();
|
||||
return answer(a.status ?? 200, a.body);
|
||||
};
|
||||
})();`;
|
||||
|
||||
async function setup(t, { viewport = [1440, 900], hash = '#/agents', stub = true, scripts = [] } = {}) {
|
||||
const server = await startServer({ port: 0, sso: createDevOwner(), organizations: fixture(), log: () => {} });
|
||||
async function setup(t, { viewport = [1440, 900], hash = '#/agents', stub = true, scripts = [], data } = {}) {
|
||||
const server = await startServer({ port: 0, sso: createDevOwner(), organizations: fixture(data), log: () => {} });
|
||||
t.after(() => close(server));
|
||||
const b = await browser({ machine: false }); t.after(() => b.close());
|
||||
await b.viewport(...viewport);
|
||||
@@ -185,9 +196,12 @@ test('the drawer shows each field from its read: name, organization, role and so
|
||||
// Decisions waiting on a human that it raised, each into the Inbox. Chief's is not its own.
|
||||
const decisions = await b.evaluate(`[...document.querySelectorAll('${d} [data-drawer-decisions] li a')].map(a => [a.textContent.trim(), a.getAttribute("href")])`);
|
||||
assert.deepEqual(decisions, [['Ship the hero copy?', '#/inbox?d=acme%3Adec-1']]);
|
||||
// SP16: no read gives the time of an agent's last activity. The agents read's time (when it took the
|
||||
// role) and `since` (when its present state began) are not that, so the drawer says not recorded.
|
||||
assert.equal(await text(`${d} [data-drawer-activity]`), 'not recorded');
|
||||
// SP18: the agents read lists only roles a run holds now, and its time is when that run took the role,
|
||||
// so Last activity is when coder's running run took the role, 40 minutes before the read, with the time itself.
|
||||
assert.match(await text(`${d} [data-drawer-activity]`), /^its run took the role 4[01] min ago$/);
|
||||
const started = await attr(`${d} [data-drawer-activity] time`, 'datetime');
|
||||
assert.ok(Math.abs(Date.now() - Date.parse(started) - 40 * 60000) < 90000, started);
|
||||
assert.match(await attr(`${d} [data-drawer-activity] time`, 'title'), /\d/);
|
||||
assert.equal(await b.evaluate(`document.querySelector('${d} [data-drawer-activity]').closest('.ws-drawer-f').querySelector('dt').textContent`), 'Last activity');
|
||||
// Not a pack hire: no packs read, and no Hired from line.
|
||||
assert.equal(await has(`${d} [data-drawer-hired]`), false);
|
||||
@@ -209,6 +223,53 @@ test('a hired agent shows where its role came from; a not-running agent says not
|
||||
assert.match(await text(`${d} [data-drawer-source]`), new RegExp(`hire decision ${HIRE}`));
|
||||
assert.match(await text(`${d} [data-drawer-state]`), /(^|\s)not running$/);
|
||||
assert.equal(await has(`${d} [data-drawer-model]`), false);
|
||||
// SP18: no run holds it, and no read keeps the time of a run that ended: one plain line, no time.
|
||||
assert.equal(await text(`${d} [data-drawer-activity]`), 'not recorded: no run holds it now, and no read keeps when its last run ended');
|
||||
assert.equal(await has(`${d} [data-drawer-activity] time`), false);
|
||||
});
|
||||
|
||||
// SP18 review: a claim dates the agent only when the instances read lists its run as running.
|
||||
test('SP18 review: Last activity counts only a claim whose run is running', async t => {
|
||||
// Coder runs run-2, but the role is still held by run-1, whose process ended: nothing dates run-2 yet.
|
||||
{
|
||||
const { wait, text, has } = await setup(t, { hash: '#/agents?agent=acme%3Acoder', data: d => { d.acme.instances.coder.running = ['run-2']; } });
|
||||
const d = '[data-ws-drawer="acme:coder"]';
|
||||
await wait(`${open('acme:coder')} && /running$/.test(document.querySelector('${d} [data-drawer-state]')?.textContent ?? '')`);
|
||||
assert.equal(await text(`${d} [data-drawer-activity]`), 'not recorded: its run holds no role or conversation yet');
|
||||
assert.equal(await has(`${d} [data-drawer-activity] time`), false);
|
||||
}
|
||||
// Coder runs nothing, and a claim from three days ago is left: not running, and no time from that claim.
|
||||
{
|
||||
const { wait, text, has } = await setup(t, { hash: '#/agents?agent=acme%3Acoder', data: d => { d.acme.instances.coder.running = []; d.acme.agents[0].at = at(3 * 24 * 60); d.acme.agents[0].working = null; } });
|
||||
const d = '[data-ws-drawer="acme:coder"]';
|
||||
await wait(`${open('acme:coder')} && /not running$/.test(document.querySelector('${d} [data-drawer-state]')?.textContent ?? '')`);
|
||||
assert.equal(await text(`${d} [data-drawer-activity]`), 'not recorded: no run holds it now, and no read keeps when its last run ended');
|
||||
assert.equal(await has(`${d} [data-drawer-activity] time`), false);
|
||||
}
|
||||
// Two runs, one holding the role: no read says which run is newer, so the line says one of them.
|
||||
{
|
||||
const { wait, text } = await setup(t, { hash: '#/agents?agent=acme%3Acoder', data: d => { d.acme.instances.coder.running = ['run-1', 'run-2']; } });
|
||||
const d = '[data-ws-drawer="acme:coder"]';
|
||||
await wait(`${open('acme:coder')} && !!document.querySelector('${d} [data-drawer-activity] time')`);
|
||||
assert.match(await text(`${d} [data-drawer-activity]`), /^one of its 2 runs took the role 4[01] min ago$/);
|
||||
}
|
||||
// The instances read fails: nothing checked the claim against a live run, so the line says "a run".
|
||||
{
|
||||
const fail = `(() => { const f = window.fetch; window.fetch = (u, i) => /\\/api\\/orgs\\/acme\\/bus\\/instances$/.test(String(u)) ? Promise.resolve(new Response('{}', { status: 503 })) : f(u, i); })();`;
|
||||
const { wait, text } = await setup(t, { hash: '#/agents?agent=acme%3Acoder', scripts: [fail] });
|
||||
const d = '[data-ws-drawer="acme:coder"]';
|
||||
await wait(`${open('acme:coder')} && /^State not read/.test(document.querySelector('${d} [data-drawer-state]')?.textContent ?? '')`);
|
||||
assert.match(await text(`${d} [data-drawer-activity]`), /^a run took the role 4[01] min ago$/);
|
||||
}
|
||||
// The same failed read for chief, which holds no role and no conversation: the line names both gaps.
|
||||
{
|
||||
const fail = `(() => { const f = window.fetch; window.fetch = (u, i) => /\\/api\\/orgs\\/acme\\/bus\\/instances$/.test(String(u)) ? Promise.resolve(new Response('{}', { status: 503 })) : f(u, i); })();`;
|
||||
const { wait, text, has } = await setup(t, { hash: '#/agents?agent=acme%3Achief', scripts: [fail] });
|
||||
const d = '[data-ws-drawer="acme:chief"]';
|
||||
await wait(`${open('acme:chief')} && /^State not read/.test(document.querySelector('${d} [data-drawer-state]')?.textContent ?? '')`);
|
||||
assert.equal(await text(`${d} [data-drawer-activity]`), 'not recorded: the agents and claims reads show no run holding its role or a conversation, and its runs were not read');
|
||||
assert.equal(await has(`${d} [data-drawer-activity] time`), false);
|
||||
}
|
||||
});
|
||||
|
||||
test('SP16: a pack hire names its pack and the template the packs read gives; a refused read says so', async t => {
|
||||
@@ -501,8 +562,8 @@ test('Message opens the agent\'s thread in an Assistant pane; Stop asks first an
|
||||
assert.equal(await attr('[data-ws-drawer="acme:assistant"] [data-drawer-message]', 'href'), `#/assistant?add=acme%3A${THREAD}`);
|
||||
// SP16: Message is offered, so no line says why it is not.
|
||||
assert.equal(await has('[data-ws-drawer="acme:assistant"] [data-drawer-nomessage]'), false);
|
||||
// It holds a conversation, whose messages read pages from the oldest message: the drawer says it does not read it.
|
||||
assert.equal(await text('[data-ws-drawer="acme:assistant"] [data-drawer-activity]'), 'not shown: reading it means reading the whole conversation');
|
||||
// SP18: a conversation agent's run took its conversation 5 minutes before the read (thread.claims' time).
|
||||
assert.match(await text('[data-ws-drawer="acme:assistant"] [data-drawer-activity]'), /^its run took the conversation [56] min ago$/);
|
||||
// A role agent holds no thread to open, so it has no Message; it can still be stopped.
|
||||
await b.evaluate('location.hash = "#/agents?agent=acme%3Acoder"');
|
||||
await wait(`${open('acme:coder')} && document.querySelector('[data-ws-drawer="acme:coder"] [data-drawer-stop]')`);
|
||||
@@ -577,3 +638,275 @@ test('at 390 wide the drawer is a full-width sheet', async t => {
|
||||
assert.equal(box[1], box[2]);
|
||||
assert.ok(box[3] <= box[2], `no horizontal page scroll: ${box[3]} > ${box[2]}`);
|
||||
});
|
||||
|
||||
// SP18: the sheet is fixed to the screen, but the page under it kept its own height, so the page still
|
||||
// scrolled behind the sheet and a full-page shot showed the sheet stretched down a long blank area. With the
|
||||
// sheet open the page is one screen tall; Close puts the page back where the reader left it.
|
||||
test('SP18: at 390 wide the open sheet is the whole screen, the page under it does not scroll, and Close keeps the place', async t => {
|
||||
const { b, wait } = await setup(t, { viewport: [390, 844], hash: '#/orgs/acme/organization' });
|
||||
const link = JSON.stringify(sessionLink('acme:coder'));
|
||||
await wait(`document.querySelector(${link}) && document.querySelectorAll("[data-org-settings] [data-org-setting]").length > 10`);
|
||||
await b.evaluate(`document.querySelector(${link}).scrollIntoView({ block: 'center' })`);
|
||||
const at = await b.evaluate('Math.round(scrollY)');
|
||||
assert.ok(at > 100, `the row is down the page: ${at}`);
|
||||
await b.evaluate(`document.querySelector(${link}).click()`);
|
||||
await wait(open('acme:coder'));
|
||||
await b.evaluate('new Promise(r => requestAnimationFrame(() => requestAnimationFrame(r)))');
|
||||
const m = await b.evaluate(`(() => { const r = ${drawer('acme:coder')}.getBoundingClientRect(), e = document.documentElement; return { top: Math.round(r.top), bottom: Math.round(r.bottom), left: Math.round(r.left), width: Math.round(r.width), vh: innerHeight, vw: e.clientWidth, docH: e.scrollHeight, bodyH: Math.round(document.body.getBoundingClientRect().height), sw: e.scrollWidth }; })()`);
|
||||
assert.equal(m.top, 0, JSON.stringify(m));
|
||||
assert.equal(m.bottom, m.vh, `the sheet ends at the screen's foot: ${JSON.stringify(m)}`);
|
||||
assert.equal(m.left, 0); assert.equal(m.width, m.vw);
|
||||
assert.ok(m.docH <= m.vh && m.bodyH <= m.vh, `the page under the sheet is one screen tall, nothing below to scroll to: ${JSON.stringify(m)}`);
|
||||
assert.ok(m.sw <= m.vw, 'no sideways scroll');
|
||||
// A scroll on the page does nothing while the sheet is open.
|
||||
await b.evaluate('window.scrollTo(0, 2000)');
|
||||
assert.equal(await b.evaluate('Math.round(scrollY)'), 0);
|
||||
await b.evaluate(`${drawer('acme:coder')}.querySelector("[data-drawer-close]").click()`);
|
||||
await wait(noDrawer);
|
||||
await b.evaluate('new Promise(r => requestAnimationFrame(() => requestAnimationFrame(r)))');
|
||||
assert.ok(Math.abs(await b.evaluate('Math.round(scrollY)') - at) <= 1, `Close puts the page back at ${at}: ${await b.evaluate('Math.round(scrollY)')}`);
|
||||
assert.ok(await b.evaluate('document.documentElement.scrollHeight') > 844, 'the page is its full height again');
|
||||
// At 1440 the drawer is a side panel: the page keeps its height and scrolls.
|
||||
await b.viewport(1440, 900);
|
||||
await b.evaluate(`document.querySelector(${link}).click()`);
|
||||
await wait(open('acme:coder'));
|
||||
assert.ok(await b.evaluate('document.documentElement.scrollHeight') > 900, 'at 1440 the page beside the drawer keeps its height');
|
||||
});
|
||||
|
||||
// SP18 review: only Close (or Back to the same page) puts the page back. A link in the sheet goes to a new
|
||||
// page, which starts at its top, and the place kept for the page under the sheet is not used later either.
|
||||
// The Task page is drawn through afterPaint; a project address is drawn by bus.js's chat branch without it
|
||||
// (this fixture has no chat backend, so that page is the Projects list saying the route is invalid).
|
||||
for (const [label, sel, hashStart, ready] of [
|
||||
['Task', '[data-drawer-task] a', '#/orgs/acme/tasks?task=', 'document.querySelector("[data-tasks-board]")'],
|
||||
['Project', '[data-drawer-project] a', '#/orgs/acme/projects/p-site', 'document.querySelector("#bus-view h1")?.textContent.trim() === "Projects"'],
|
||||
]) test(`SP18 review: at 390 the sheet's ${label} link opens its page at the top, and a later page does not jump`, async t => {
|
||||
const { b, wait } = await setup(t, { viewport: [390, 844], hash: '#/orgs/acme/organization' });
|
||||
const link = JSON.stringify(sessionLink('acme:coder'));
|
||||
await wait(`document.querySelector(${link}) && document.querySelectorAll("[data-org-settings] [data-org-setting]").length > 10`);
|
||||
// Every page is long, so a jump to the kept place would show wherever it lands.
|
||||
await b.evaluate('document.head.insertAdjacentHTML("beforeend", "<style>#bus-view{padding-bottom:4000px}</style>")');
|
||||
await b.evaluate(`document.querySelector(${link}).scrollIntoView({ block: 'center' })`);
|
||||
const at = await b.evaluate('Math.round(scrollY)');
|
||||
assert.ok(at > 100, `the row is down the page: ${at}`);
|
||||
await b.evaluate(`document.querySelector(${link}).click()`);
|
||||
await wait(open('acme:coder'));
|
||||
await b.evaluate(`${drawer('acme:coder')}.querySelector(${JSON.stringify(sel)}).click()`);
|
||||
await wait(`${noDrawer} && location.hash.startsWith(${JSON.stringify(hashStart)}) && ${ready}`);
|
||||
await b.evaluate('new Promise(r => setTimeout(r, 500))');
|
||||
const y = await b.evaluate('Math.round(scrollY)');
|
||||
assert.ok(y <= 1, `the ${label} page is at its top, not at ${at}: ${y}`);
|
||||
// The next page with no drawer does not take the kept place either.
|
||||
await b.evaluate('location.hash = "#/orgs/acme/organization"');
|
||||
await wait('document.querySelectorAll("[data-org-settings] [data-org-setting]").length > 10');
|
||||
await b.evaluate('new Promise(r => setTimeout(r, 500))');
|
||||
const later = await b.evaluate('Math.round(scrollY)');
|
||||
assert.ok(later <= 1, `the organization page opens at its top: ${later}`);
|
||||
});
|
||||
|
||||
// #1873 triage-followups: the page focuses its heading when its first render ends. On load the page
|
||||
// starts a second render that reads GET /api/workspace again, so when the first render's read is slow
|
||||
// the second draws the rows first and a reader can open a drawer before the first render ends. Its late
|
||||
// end must not take the drawer's focus. The page holds the first GET /api/workspace until __go.
|
||||
const HOLD_FIRST = `(() => { const f = window.fetch; let first = true; window.__firstDone = false; window.fetch = (u, o) => { if (first && String(u?.url ?? u).includes('/api/workspace')) { first = false; return new Promise(r => { const t = setInterval(() => { if (window.__go) { clearInterval(t); r(f(u, o).finally(() => setTimeout(() => { window.__firstDone = true; }, 0))); } }, 20); }); } return f(u, o); }; })()`;
|
||||
|
||||
test('triage-followups: a drawer opened before the first render ends keeps focus when that render ends', async t => {
|
||||
const { b, wait } = await setup(t, { scripts: [HOLD_FIRST] });
|
||||
await wait(`document.querySelector(${JSON.stringify(rowLink('acme:coder'))})`);
|
||||
assert.notEqual(await b.evaluate('document.activeElement?.tagName'), 'H1', 'the first render has not ended');
|
||||
await b.evaluate(`document.querySelector(${JSON.stringify(rowLink('acme:coder'))}).click()`);
|
||||
await wait(`${open('acme:coder')} && ${drawer('acme:coder')}.contains(document.activeElement)`);
|
||||
await b.evaluate('window.__go = true');
|
||||
await wait('window.__firstDone');
|
||||
await new Promise(r => setTimeout(r, 400));
|
||||
assert.equal(await b.evaluate(`${drawer('acme:coder')}.contains(document.activeElement)`), true, `focus stays in the drawer, not on ${await b.evaluate('document.activeElement?.tagName')}`);
|
||||
});
|
||||
|
||||
// SP19 (#1873): Restart in the drawer, offered only where Stop is. It reuses Stop's confirm step and
|
||||
// status line, ends the agent through its own organization's /agent/terminate, waits for the request to
|
||||
// close and for its runs to reach 0, then launches it through /agent/launch. It never regenerates. A
|
||||
// conversation agent is ended and not launched. The page's 2 s poll steps run every 20 ms here.
|
||||
const FAST = `(() => { const st = window.setTimeout.bind(window); window.setTimeout = (f, ms, ...a) => st(f, ms === 2000 ? 20 : ms, ...a); })()`;
|
||||
const END = '5e000000-0000-4000-8000-0000000000e1', LAUNCH = '5e000000-0000-4000-8000-0000000000a1';
|
||||
const closed = (request, action, extra = {}) => ({ request, instance: 'coder', action, state: 'closed', done: { run: null, transaction: null, refusal: null }, cancelled: null, ...extra });
|
||||
// Every distinct state of the drawer's actions, from now on: [status, Restart's label, Restart disabled, Stop disabled].
|
||||
const WATCH = id => `(() => { window.__seen = []; const snap = () => { const d = document.querySelector('[data-ws-drawer="${id}"]'); if (!d) return; const r = d.querySelector('[data-drawer-restart]'), s = d.querySelector('[data-drawer-stop]'); const v = [d.querySelector('[data-drawer-status]')?.textContent ?? '', r ? r.firstChild.textContent : null, !!r?.disabled, !!s?.disabled]; const l = window.__seen.at(-1); if (!l || l.join('|') !== v.join('|')) window.__seen.push(v); }; snap(); new MutationObserver(snap).observe(document.getElementById('bus-view'), { subtree: true, childList: true, characterData: true, attributes: true }); return true; })()`;
|
||||
const status = d => `(document.querySelector('${d} [data-drawer-status]')?.textContent ?? "")`;
|
||||
const usable = d => `document.querySelector('${d} [data-drawer-restart]')?.disabled === false && document.querySelector('${d} [data-drawer-stop]')?.disabled === false && !document.querySelector('${d} .ws-drawer-confirm')`;
|
||||
|
||||
test('SP19: Restart asks first, then ends and launches the agent in its own organization, and never regenerates', async t => {
|
||||
// The reader was last on Beta Works, so the Console's chosen organization is beta. #/agents shows every
|
||||
// organization's agents; the drawer's own organization is used, not the chosen one.
|
||||
const { b, wait, text } = await setup(t, { hash: '#/orgs/beta/organization', scripts: [FAST] });
|
||||
await wait(`window.MosaicOrg?.id === 'beta' && document.querySelectorAll("[data-org-settings] [data-org-setting]").length > 0`);
|
||||
await b.evaluate('location.hash = "#/agents?agent=acme%3Acoder"');
|
||||
const d = '[data-ws-drawer="acme:coder"]';
|
||||
await wait(`${open('acme:coder')} && document.querySelector('${d} [data-drawer-restart]')`);
|
||||
assert.equal(await b.evaluate('window.MosaicOrg?.id ?? null'), 'beta');
|
||||
// Beside Stop, in the same action group as the status line.
|
||||
assert.equal(await b.evaluate(`document.querySelector('${d} [data-drawer-restart]').closest('[data-drawer-actions]') === document.querySelector('${d} [data-drawer-stop]').closest('[data-drawer-actions]')`), true);
|
||||
assert.equal(await b.evaluate(`document.querySelector('${d} [data-drawer-restart]').textContent`), 'Restart coder in Acme Studio');
|
||||
// It asks first, in Stop's confirm step. Cancel sends nothing and puts focus back on Restart.
|
||||
await b.evaluate(`document.querySelector('${d} [data-drawer-restart]').click()`);
|
||||
await wait(`document.querySelector('${d} [data-drawer-restart-yes]')`);
|
||||
assert.match(await text(`${d} [data-drawer-restart-confirm]`), /^Restart coder in Acme Studio\? It ends its session and launches it again\. Restart coder Cancel$/);
|
||||
assert.equal(await b.evaluate(`document.querySelector('${d} [data-drawer-restart-confirm]').classList.contains('ws-drawer-confirm')`), true);
|
||||
assert.deepEqual(await b.evaluate(`[document.querySelector('${d} [data-drawer-restart]').disabled, document.querySelector('${d} [data-drawer-stop]').disabled]`), [true, true]);
|
||||
await b.evaluate(`document.querySelector('${d} [data-drawer-restart-no]').click()`);
|
||||
await wait(`document.activeElement === document.querySelector('${d} [data-drawer-restart]') && ${usable(d)}`);
|
||||
// Escape cancels the question too, and the drawer stays.
|
||||
await b.evaluate(`document.querySelector('${d} [data-drawer-restart]').click()`);
|
||||
await wait(`document.querySelector('${d} [data-drawer-restart-no]') === document.activeElement`);
|
||||
await b.key('Escape');
|
||||
await wait(`document.activeElement === document.querySelector('${d} [data-drawer-restart]') && ${usable(d)}`);
|
||||
assert.equal(await b.evaluate(agentParam), 'acme:coder');
|
||||
assert.deepEqual(await b.evaluate('window.__posts'), []);
|
||||
// Yes: one terminate, its request followed until it closes, the runs read until none is left, one launch.
|
||||
await b.evaluate(`window.__answers.push(
|
||||
{ body: { request: '${END}', runs: ['run-1'], cancelled: [], taken: [] }, then: () => { window.__runs = { acme: { coder: [] } }; } },
|
||||
{ body: ${JSON.stringify(closed(END, 'end'))} },
|
||||
{ body: { request: '${LAUNCH}' }, then: () => { window.__runs = { acme: { coder: ['run-2'] } }; } },
|
||||
{ body: ${JSON.stringify(closed(LAUNCH, 'launch', { done: { run: 'run-2', transaction: null, refusal: null } }))} })`);
|
||||
await b.evaluate(WATCH('acme:coder'));
|
||||
await b.evaluate(`document.querySelector('${d} [data-drawer-restart]').click()`);
|
||||
await wait(`document.querySelector('${d} [data-drawer-restart-yes]')`);
|
||||
await b.evaluate(`document.querySelector('${d} [data-drawer-restart-yes]').click()`);
|
||||
await wait(`/^Restarted coder \\(run run-2\\)\\.$/.test(${status(d)}) && ${usable(d)}`);
|
||||
const posts = await b.evaluate('window.__posts');
|
||||
assert.deepEqual(posts.map(p => [p.url, p.method]), [
|
||||
['/api/orgs/acme/agent/terminate', 'POST'], [`/api/orgs/acme/agent/request?id=${END}`, 'GET'],
|
||||
['/api/orgs/acme/agent/launch', 'POST'], [`/api/orgs/acme/agent/request?id=${LAUNCH}`, 'GET'],
|
||||
]);
|
||||
assert.deepEqual(JSON.parse(posts[0].body), { instance: 'coder', reason: 'restart from the agent drawer' });
|
||||
assert.deepEqual(JSON.parse(posts[2].body), { instance: 'coder' });
|
||||
assert.equal(posts.some(p => /regenerate/.test(p.url)), false, 'Restart never regenerates');
|
||||
// The status line names each step; while it runs, Restart reads Restarting… and both buttons wait.
|
||||
const seen = await b.evaluate('window.__seen');
|
||||
const steps = seen.map(s => s[0]).filter((s, i, all) => s && s !== all[i - 1]);
|
||||
assert.deepEqual(steps, ['Restarting coder: asking it to end…', 'coder was asked to end. Waiting for its session to end…', 'coder has ended. Launching it…', 'Restarted coder (run run-2).']);
|
||||
for (const [s, label, rd, sd] of seen.filter(s => steps.slice(0, 3).includes(s[0]))) assert.deepEqual([label, rd, sd], ['Restarting…', true, true], s);
|
||||
assert.equal(await b.evaluate(`document.querySelector('${d} [data-drawer-restart]').firstChild.textContent`), 'Restart');
|
||||
});
|
||||
|
||||
// The Assistant is a protected agent, so it has no Stop and no Restart; ideas is a conversation agent that is not.
|
||||
test('SP19: a conversation agent is ended and not launched; its next message starts it', async t => {
|
||||
const data = D => { D.acme.instances.ideas = { definition: 'ideas', running: ['run-4'] }; };
|
||||
const { b, wait } = await setup(t, { hash: '#/agents?agent=acme%3Aideas', scripts: [FAST], data });
|
||||
const d = '[data-ws-drawer="acme:ideas"]';
|
||||
await wait(`${open('acme:ideas')} && document.querySelector('${d} [data-drawer-restart]')`);
|
||||
assert.equal(await b.evaluate(`document.querySelector('${d} [data-drawer-restart]').dataset.drawerThread`), 'yes');
|
||||
await b.evaluate(`document.querySelector('${d} [data-drawer-restart]').click()`);
|
||||
await wait(`document.querySelector('${d} [data-drawer-restart-yes]')`);
|
||||
assert.match(await b.evaluate(`document.querySelector('${d} [data-drawer-restart-confirm]').textContent`), /^Restart ideas in Acme Studio\? It ends its session; its next message starts it again\./);
|
||||
await b.evaluate(`window.__answers.push(
|
||||
{ body: { request: '${END}', runs: ['run-4'], cancelled: [], taken: [] }, then: () => { window.__runs = { acme: { ideas: [] } }; } },
|
||||
{ body: ${JSON.stringify({ ...closed(END, 'end'), instance: 'ideas' })} })`);
|
||||
await b.evaluate(`document.querySelector('${d} [data-drawer-restart-yes]').click()`);
|
||||
await wait(`/^Ended ideas at .+\\. Its next message starts it\\.$/.test(${status(d)})`);
|
||||
const posts = await b.evaluate('window.__posts');
|
||||
assert.deepEqual(posts.map(p => [p.url, p.method]), [['/api/orgs/acme/agent/terminate', 'POST'], [`/api/orgs/acme/agent/request?id=${END}`, 'GET']]);
|
||||
assert.deepEqual(JSON.parse(posts[0].body), { instance: 'ideas', reason: 'restart from the agent drawer' });
|
||||
});
|
||||
|
||||
test('SP19: a second click while a restart runs does nothing', async t => {
|
||||
const { b, wait } = await setup(t, { hash: '#/agents?agent=acme%3Acoder', scripts: [FAST] });
|
||||
const d = '[data-ws-drawer="acme:coder"]';
|
||||
await wait(`${open('acme:coder')} && document.querySelector('${d} [data-drawer-restart]')`);
|
||||
// The terminate's answer is held, so the restart is still running while the page is clicked again.
|
||||
await b.evaluate(`window.__answers.push({ hold: new Promise(r => { window.__release = r; }), status: 403, body: { error: 'unavailable', sent: false } })`);
|
||||
await b.evaluate(`document.querySelector('${d} [data-drawer-restart]').click()`);
|
||||
await wait(`document.querySelector('${d} [data-drawer-restart-yes]')`);
|
||||
await b.evaluate(`document.querySelector('${d} [data-drawer-restart-yes]').click()`);
|
||||
await wait(`window.__posts.length === 1 && document.querySelector('${d} [data-drawer-restart]').disabled`);
|
||||
// Even a Restart button made clickable again starts nothing: no second question, no second request.
|
||||
await b.evaluate(`(() => { const r = document.querySelector('${d} [data-drawer-restart]'), s = document.querySelector('${d} [data-drawer-stop]'); r.disabled = false; s.disabled = false; r.click(); s.click(); return true; })()`);
|
||||
await b.evaluate('new Promise(r => setTimeout(r, 300))');
|
||||
assert.equal(await b.evaluate(`!!document.querySelector('${d} .ws-drawer-confirm')`), false);
|
||||
assert.equal(await b.evaluate('window.__posts.length'), 1);
|
||||
await b.evaluate('window.__release()');
|
||||
await wait(`/^Not restarted:/.test(${status(d)}) && ${usable(d)}`);
|
||||
assert.equal(await b.evaluate('window.__posts.length'), 1);
|
||||
});
|
||||
|
||||
// SP19 review: a restart keeps its own state, so another drawer's question or Stop does not lose its
|
||||
// line, and its drawer still shows it running when the reader comes back.
|
||||
test('SP19: a restart keeps its line and its running state across another drawer\'s Stop', async t => {
|
||||
const data = D => { D.acme.instances.ideas = { definition: 'ideas', running: ['run-4'] }; };
|
||||
const { b, wait } = await setup(t, { hash: '#/agents?agent=acme%3Acoder', scripts: [FAST], data });
|
||||
const d = '[data-ws-drawer="acme:coder"]', i = '[data-ws-drawer="acme:ideas"]';
|
||||
await wait(`${open('acme:coder')} && document.querySelector('${d} [data-drawer-restart]')`);
|
||||
await b.evaluate(`window.__answers.push({ hold: new Promise(r => { window.__release = r; }), status: 403, body: { error: 'unavailable', sent: false } })`);
|
||||
await b.evaluate(`document.querySelector('${d} [data-drawer-restart]').click()`);
|
||||
await wait(`document.querySelector('${d} [data-drawer-restart-yes]')`);
|
||||
await b.evaluate(`document.querySelector('${d} [data-drawer-restart-yes]').click()`);
|
||||
await wait(`window.__posts.length === 1`);
|
||||
// Ideas asks to stop and is cancelled; back on coder, its restart still shows as running.
|
||||
await b.evaluate('location.hash = "#/agents?agent=acme%3Aideas"');
|
||||
await wait(`${open('acme:ideas')} && document.querySelector('${i} [data-drawer-stop]')`);
|
||||
await b.evaluate(`document.querySelector('${i} [data-drawer-stop]').click()`);
|
||||
await wait(`document.querySelector('${i} [data-drawer-stop-no]')`);
|
||||
await b.evaluate(`document.querySelector('${i} [data-drawer-stop-no]').click()`);
|
||||
await b.evaluate('location.hash = "#/agents?agent=acme%3Acoder"');
|
||||
await wait(`${open('acme:coder')} && document.querySelector('${d} [data-drawer-restart]')`);
|
||||
assert.deepEqual(await b.evaluate(`[${status(d)}, document.querySelector('${d} [data-drawer-restart]').firstChild.textContent, document.querySelector('${d} [data-drawer-restart]').disabled, document.querySelector('${d} [data-drawer-stop]').disabled]`),
|
||||
['Restarting coder: asking it to end…', 'Restarting…', true, true]);
|
||||
// Ideas' Stop question is open when coder's restart is refused; coder's drawer still says so after.
|
||||
await b.evaluate('location.hash = "#/agents?agent=acme%3Aideas"');
|
||||
await wait(`${open('acme:ideas')} && document.querySelector('${i} [data-drawer-stop]')`);
|
||||
await b.evaluate(`document.querySelector('${i} [data-drawer-stop]').click()`);
|
||||
await wait(`document.querySelector('${i} [data-drawer-stop-no]')`);
|
||||
await b.evaluate('window.__release()');
|
||||
await b.evaluate('new Promise(r => setTimeout(r, 300))');
|
||||
assert.equal(await b.evaluate(`!!document.querySelector('${i} [data-drawer-stop-confirm]')`), true, 'the other drawer keeps its question');
|
||||
await b.evaluate(`document.querySelector('${i} [data-drawer-stop-no]').click()`);
|
||||
await b.evaluate('location.hash = "#/agents?agent=acme%3Acoder"');
|
||||
await wait(`${open('acme:coder')} && document.querySelector('${d} [data-drawer-restart]')`);
|
||||
await wait(`${status(d)} === 'Not restarted: this Console is not the owner of Acme Studio, so it does not restart its agents.' && ${usable(d)}`);
|
||||
assert.equal(await b.evaluate('window.__posts.length'), 1);
|
||||
});
|
||||
|
||||
// Each refusal the two routes can give says so in one plain line. Where the agent still runs, both
|
||||
// buttons are usable again and focus is on Restart; after a refused launch it has no run, so the
|
||||
// repaint drops Stop and Restart, keeps the line and puts focus on the drawer's heading.
|
||||
const ENDED = [{ body: { request: END, runs: ['run-1'], cancelled: [], taken: [] }, runs: [] }, { body: closed(END, 'end') }];
|
||||
for (const [label, answers, line] of [
|
||||
['not the owner', [{ status: 403, body: { error: 'unavailable', sent: false } }],
|
||||
/^Not restarted: this Console is not the owner of Acme Studio, so it does not restart its agents\.$/],
|
||||
['the organization is archived', [{ status: 409, body: { error: 'org-archived', message: 'organization is archived' } }],
|
||||
/^Not restarted: Acme Studio is archived, so its agents are not changed\. Nothing changed\.$/],
|
||||
['the bus is stopped', [{ status: 503, body: { error: 'no-bus-host', message: 'no-bus-host: no-bus-host. Nothing changed.', sent: false } }],
|
||||
/^Not restarted: the bus of Acme Studio is not running\. Nothing changed\.$/],
|
||||
['the end is refused', [{ body: { request: END, runs: ['run-1'], cancelled: [], taken: [] } }, { body: { ...closed(END, 'end', { done: { run: null, transaction: null, refusal: 'instance-kept' } }), reason: 'the instance is kept; unkeep it first' } }],
|
||||
/^Not restarted: coder was not ended \(the instance is kept; unkeep it first\)\.$/],
|
||||
['the launch is refused', [...ENDED, { status: 409, body: { error: 'capacity-full', message: 'capacity-full: a launch limit is reached. Nothing changed.', sent: false } }],
|
||||
/^coder was ended and not launched: capacity-full: a launch limit is reached\. Nothing changed\.$/],
|
||||
['the launch request closes refused', [...ENDED, { body: { request: LAUNCH } }, { body: { ...closed(LAUNCH, 'launch', { done: { run: null, transaction: null, refusal: 'no-route' } }), reason: 'the instance has no harness, model and thinking yet; set its route first' } }],
|
||||
/^coder was ended and not launched: the instance has no harness, model and thinking yet; set its route first\.$/],
|
||||
['its runs never reach 0', [{ body: { request: END, runs: ['run-1'], cancelled: [], taken: [] } }, { body: closed(END, 'end') }],
|
||||
/^Not restarted: coder was asked to end, but its session is still running\. Read the agents again before trying\.$/],
|
||||
]) test(`SP19: a refusal says so in the drawer: ${label}`, async t => {
|
||||
const { b, wait } = await setup(t, { hash: '#/agents?agent=acme%3Acoder', scripts: [FAST] });
|
||||
const d = '[data-ws-drawer="acme:coder"]';
|
||||
await wait(`${open('acme:coder')} && document.querySelector('${d} [data-drawer-restart]')`);
|
||||
const queued = JSON.stringify(answers);
|
||||
await b.evaluate(`window.__answers.push(...${queued}.map(a => a.runs ? { ...a, then: () => { window.__runs = { acme: { coder: a.runs } }; } } : a))`);
|
||||
await b.evaluate(`document.querySelector('${d} [data-drawer-restart]').click()`);
|
||||
await wait(`document.querySelector('${d} [data-drawer-restart-yes]')`);
|
||||
await b.evaluate(`document.querySelector('${d} [data-drawer-restart-yes]').click()`);
|
||||
const launched = label.startsWith('the launch');
|
||||
await wait(launched ? `/ was ended and not launched: /.test(${status(d)}) && !document.querySelector('${d} [data-drawer-restart]')`
|
||||
: `/^Not restarted: /.test(${status(d)}) && document.querySelector('${d} [data-drawer-restart]')?.disabled === false`);
|
||||
await b.evaluate('new Promise(r => setTimeout(r, 300))');
|
||||
assert.match(await b.evaluate(status(d)), line);
|
||||
if (launched) {
|
||||
assert.deepEqual(await b.evaluate(`[!!document.querySelector('${d} [data-drawer-stop]'), document.querySelector('${d} [data-drawer-nostop]')?.textContent]`), [false, 'It is not running, so there is nothing to stop.']);
|
||||
assert.equal(await b.evaluate(`document.activeElement === document.querySelector('${d} h2')`), true, `focus is on the drawer heading, not ${await b.evaluate('document.activeElement?.outerHTML.slice(0, 80)')}`);
|
||||
} else {
|
||||
assert.equal(await b.evaluate(usable(d)), true, 'both buttons are usable again');
|
||||
assert.equal(await b.evaluate(`document.activeElement === document.querySelector('${d} [data-drawer-restart]')`), true, 'focus is back on Restart');
|
||||
}
|
||||
const posts = await b.evaluate('window.__posts');
|
||||
assert.equal(posts.some(p => /regenerate/.test(p.url)), false);
|
||||
assert.equal(posts.filter(p => p.url.endsWith('/agent/launch')).length, launched ? 1 : 0);
|
||||
assert.equal(await b.evaluate('window.__answers.length'), 0, 'every queued answer was asked for');
|
||||
});
|
||||
@@ -218,8 +218,9 @@ test('row 194 N5: agents.js is served as JavaScript and index.html loads it once
|
||||
const s = await setup(t);
|
||||
const js = await get(s, '/agents.js');
|
||||
assert.equal(js.status, 200); assert.match(js.type, /^text\/javascript/);
|
||||
// F3 adds the agent restart receipt and the thread test that Settings > Routing uses.
|
||||
assert.match(js.text, /window\.MosaicAgents = Object\.freeze\(\{ section, restartMarkup, isThread: definition => threadDefinitions\.has\(definition\) \}\)/);
|
||||
// F3 adds the agent restart receipt and the thread test that Settings > Routing uses; SP19 the
|
||||
// restart chain the agent drawer calls.
|
||||
assert.match(js.text, /window\.MosaicAgents = Object\.freeze\(\{ section, restartMarkup, restart: restartAgent, isThread: definition => threadDefinitions\.has\(definition\) \}\)/);
|
||||
assert.match(js.text, /Agent actions are unavailable here\. Use mosaic org instance from the instance CLI\./);
|
||||
const html = await get(s, '/');
|
||||
assert.equal(html.text.match(/<script src="\/agents\.js" defer><\/script>/g)?.length, 1);
|
||||
|
||||
+21
-3
@@ -6,9 +6,14 @@
|
||||
// CARDS answer a reply with a cards fence: one valid card, one unknown step
|
||||
// TOOL <to> call send_message to <to> through the tool socket and answer the raw reply
|
||||
// SLEEP hang until killed
|
||||
// REPLY-SLEEP send the person an answer to this message through the tool socket, then hang until killed
|
||||
// REPLY-SLEEP send the person an answer (class RESULT) to this message through the tool socket, then hang until killed
|
||||
// NOTE-SLEEP send the person a progress note (class INFO) on this message through the tool socket, then hang until killed
|
||||
// SPEND <n> add one usage entry of <n> input tokens to the session's spend file (session/spend.jsonl,
|
||||
// which the host tests' usage source reads as the transcript), then answer "spent <n>"
|
||||
// GATED-SPEND <n> write work/started, wait for work/gate, then spend <n> as SPEND does
|
||||
// anything else answer "echo: <body>"
|
||||
import { existsSync, readFileSync, writeFileSync } from "node:fs";
|
||||
import { appendFileSync, existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { connect } from "node:net";
|
||||
import { join } from "node:path";
|
||||
|
||||
@@ -57,8 +62,21 @@ if (body === "PROBE") {
|
||||
process.stdout.write(await toolCall("send_message", { to: body.slice(5), body: "from the tool", class: "INFO" }));
|
||||
} else if (body === "REPLY-SLEEP") {
|
||||
const id = request.match(/^Message (\S+) from /m)?.[1];
|
||||
await toolCall("send_message", { to: "human", body: "answered early", class: "INFO", in_reply_to: id });
|
||||
await toolCall("send_message", { to: "human", body: "answered early", class: "RESULT", in_reply_to: id });
|
||||
setInterval(() => {}, 1000);
|
||||
} else if (body === "NOTE-SLEEP") {
|
||||
const id = request.match(/^Message (\S+) from /m)?.[1];
|
||||
await toolCall("send_message", { to: "human", body: "still looking", class: "INFO", in_reply_to: id });
|
||||
setInterval(() => {}, 1000);
|
||||
} else if (/^(GATED-)?SPEND \d{1,9}$/.test(body)) {
|
||||
const n = Number(body.split(" ")[1]);
|
||||
if (body.startsWith("GATED-")) {
|
||||
writeFileSync(join(workspace, "started"), "x");
|
||||
while (!existsSync(join(workspace, "gate"))) await new Promise((r) => setTimeout(r, 50));
|
||||
}
|
||||
mkdirSync(join(workspace, "..", "session"), { recursive: true });
|
||||
appendFileSync(join(workspace, "..", "session", "spend.jsonl"), JSON.stringify({ id: randomBytes(6).toString("hex"), n }) + "\n");
|
||||
process.stdout.write(`spent ${n}`);
|
||||
} else if (body === "SLEEP") {
|
||||
setInterval(() => {}, 1000);
|
||||
} else {
|
||||
|
||||
@@ -279,3 +279,24 @@ test('At 390 wide the tiles sit two to a row with no sideways scroll, on the Das
|
||||
assert.equal(await cols('[data-org-tiles]'), 2);
|
||||
assert.equal(await b.evaluate('document.documentElement.scrollWidth <= document.documentElement.clientWidth'), true, 'organization page');
|
||||
});
|
||||
|
||||
test('SP18: at 1440 and 1024 the Dashboard\'s two columns line up with the tile row above them', async t => {
|
||||
const { b, wait } = await setup(t, { viewport: [1440, 1000] });
|
||||
await wait(homeReady);
|
||||
// The left column ends where a tile ends and the right column starts where the next tile starts: the fourth
|
||||
// and fifth of six at 1440, the second and third of three a row at 1024.
|
||||
const edges = () => b.evaluate(`(() => {
|
||||
const tiles = [...document.querySelectorAll('[data-ws-home] [data-ws-tile]')].map(e => e.getBoundingClientRect());
|
||||
const cols = [...document.querySelectorAll('[data-ws-home] .ws-home-split>*')].map(e => e.getBoundingClientRect());
|
||||
return { rights: tiles.map(r => Math.round(r.right)), lefts: tiles.map(r => Math.round(r.left)), left: cols[0] && [Math.round(cols[0].left), Math.round(cols[0].right)], right: cols[1] && [Math.round(cols[1].left), Math.round(cols[1].right)] };
|
||||
})()`);
|
||||
for (const [w, nth] of [[1440, 4], [1024, 2]]) {
|
||||
await b.viewport(w, 1000);
|
||||
await b.evaluate('new Promise(r => requestAnimationFrame(() => requestAnimationFrame(r)))');
|
||||
const e = await edges();
|
||||
assert.ok(e.left && e.right && e.right[0] > e.left[1], `two columns side by side at ${w}: ${JSON.stringify(e)}`);
|
||||
assert.ok(Math.abs(e.left[1] - e.rights[nth - 1]) <= 1, `at ${w} the left column ends with tile ${nth}: ${JSON.stringify(e)}`);
|
||||
assert.ok(Math.abs(e.right[0] - e.lefts[nth]) <= 1, `at ${w} the right column starts with tile ${nth + 1}: ${JSON.stringify(e)}`);
|
||||
assert.ok(Math.abs(e.right[1] - Math.max(...e.rights)) <= 1, `at ${w} the right column ends with the row`);
|
||||
}
|
||||
});
|
||||
@@ -180,6 +180,20 @@ test('the Settings card fits at 1440 with the drawer open, at 735 and 390 withou
|
||||
assert.deepEqual(cols[1].heads, ['Settings']);
|
||||
const gap = await b.evaluate(`(() => { const left = document.querySelector('#bus-view .ws-dash>.ws-split>section'), card = left.querySelector('[data-wfg-card]'), above = card.previousElementSibling; return Math.round(card.getBoundingClientRect().top - above.getBoundingClientRect().bottom); })()`);
|
||||
assert.ok(gap >= 0 && gap <= 40, `the Workflows card follows the sessions directly: ${gap}px`);
|
||||
// SP18: Autonomy and Starter packs follow the Workflows card in the left column, so the left column is not
|
||||
// empty beside the long Settings card and nothing runs full width below it.
|
||||
const left = await b.evaluate(`(() => { const s = [...document.querySelectorAll('#bus-view .ws-dash>.ws-split>section')]; return { autonomy: !!s[0].querySelector('[data-autonomy]'), packs: !!s[0].querySelector('[data-packs]'), after: !!document.querySelector('#bus-view .ws-dash>.ws-split ~ :is(div,section):not([data-ws-drawer])'), ends: s.map(x => Math.round(x.getBoundingClientRect().bottom)) }; })()`);
|
||||
assert.equal(left.autonomy && left.packs, true, `Autonomy and Starter packs are in the left column: ${JSON.stringify(left)}`);
|
||||
assert.equal(left.after, false, 'no full-width block under the split');
|
||||
// How near level the two end depends on the data: in this fixture Autonomy is a one-line refusal and no pack
|
||||
// is installed, so the Settings card is still the taller. The left column ends with Starter packs.
|
||||
assert.ok(await b.evaluate(`Math.abs(document.querySelector('#bus-view .ws-dash>.ws-split>section [data-packs]').getBoundingClientRect().bottom - document.querySelector('#bus-view .ws-dash>.ws-split>section').getBoundingClientRect().bottom) <= 1`), `the left column ends with Starter packs: ${JSON.stringify(left.ends)}`);
|
||||
// At 390 the page is one column: sessions, Workflows, Autonomy, Starter packs, then Settings.
|
||||
await b.viewport(390, 844);
|
||||
await b.evaluate('new Promise(r => requestAnimationFrame(() => requestAnimationFrame(r)))');
|
||||
const order = await b.evaluate(`[...document.querySelectorAll('#bus-view .ws-dash>.ws-split h2, #bus-view .ws-dash>.ws-split h3')].map(h => h.textContent.trim()).filter(x => ['Active agent sessions', 'Workflows', 'Autonomy', 'Starter packs', 'Settings'].includes(x))`);
|
||||
assert.deepEqual(order, ['Active agent sessions', 'Workflows', 'Autonomy', 'Starter packs', 'Settings']);
|
||||
assert.ok(await b.evaluate('document.documentElement.scrollWidth <= document.documentElement.clientWidth'), 'no sideways scroll at 390');
|
||||
});
|
||||
|
||||
test('the bare organization address with a query is the organization page, query kept; ?agent= opens the drawer', async t => {
|
||||
|
||||
@@ -262,8 +262,10 @@ test('SP6 GET /workflows returns the org and code-project workflows, the role na
|
||||
assert.deepEqual(body.projects, Object.fromEntries(Object.entries(doc.projects).map(([id, p]) => [id, p.workflows ?? {}])));
|
||||
assert.deepEqual(body.projects.stack, doc.projects.stack.workflows);
|
||||
assert.deepEqual(body.roles.definitions, ['triage']);
|
||||
assert.deepEqual(body.roles.templates, templateHashes(join(f.root, 'roles')).map(x => x.template).sort());
|
||||
assert.ok(body.roles.templates.includes('coder') && !body.roles.templates.includes('ghost'));
|
||||
// #1873 triage-followups: the shipped triage template is system-only, so the read leaves it out.
|
||||
assert.ok(templateHashes(join(f.root, 'roles')).some(x => x.template === 'triage'), 'the fixture ships the triage template');
|
||||
assert.deepEqual(body.roles.templates, templateHashes(join(f.root, 'roles')).map(x => x.template).filter(n => n !== 'triage').sort());
|
||||
assert.ok(body.roles.templates.includes('coder') && !body.roles.templates.includes('ghost') && !body.roles.templates.includes('triage'));
|
||||
assert.equal(body.revision, createHash('sha256').update(readFileSync(f.file)).digest('hex'), 'the revision is the writer\'s: the hash of the file bytes');
|
||||
assert.equal(body.writable, true);
|
||||
assert.equal(JSON.stringify(body).includes(f.root), false, 'no host path leaves');
|
||||
@@ -283,7 +285,7 @@ test('SP6 GET /workflows returns the org and code-project workflows, the role na
|
||||
|
||||
test('SP6 GET /workflows leaves out a library project and answers outcome-unknown for a read it cannot trust', async t => {
|
||||
const REV = 'a'.repeat(64);
|
||||
const read = { document: { projects: { code: { workflows: { w: FLOW('W') } }, lib: { kind: 'library', workflows: { never: FLOW('N') } }, bare: {} } }, revision: REV, templates: ['coder', 7] };
|
||||
const read = { document: { projects: { code: { workflows: { w: FLOW('W') } }, lib: { kind: 'library', workflows: { never: FLOW('N') } }, bare: {} } }, revision: REV, templates: ['coder', 7, 'triage'] };
|
||||
const f = await fixture(t, { workflows: () => read });
|
||||
const [status, body] = await call(f, `${ORG}/workflows`);
|
||||
assert.equal(status, 200, JSON.stringify(body));
|
||||
|
||||
@@ -20,7 +20,8 @@ const END = '10000000-0000-4000-8000-0000000000e1', LAUNCH = '10000000-0000-4000
|
||||
const DEV = { provider: 'local-dev', iss: 'local-dev', sub: 'local-dev-owner', name: 'Local operator' };
|
||||
const route = thinking => ({ harness: 'claude', model: 'opus', thinking, fallback: null });
|
||||
|
||||
test('F3 Settings > Routing: a save lists how the change reaches each agent, and Restart now moves a running one', { timeout: 120000 }, async t => {
|
||||
// A real server with the stub bus, on Settings > Routing. `refusal` closes coder's end request with that code.
|
||||
async function stage(t, { refusal = null } = {}) {
|
||||
const root = mkdtempSync(join(tmpdir(), 'f3r-')); t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
const entry = (definition, extra) => ({ definition, vars: {}, launched: true, launcher: false, protected: false, kept: false, running: [], alive: [], ...extra });
|
||||
const instances = {
|
||||
@@ -52,7 +53,7 @@ test('F3 Settings > Routing: a save lists how the change reaches each agent, and
|
||||
if (verb === 'instance.launch') { requests.set(LAUNCH, { instance: args.instance, action: 'launch' }); instances[args.instance].running = ['run-2']; return { request: LAUNCH }; }
|
||||
if (verb === 'instance.request') {
|
||||
const r = requests.get(args.request);
|
||||
return { request: args.request, instance: r.instance, action: r.action, state: 'closed', done: { run: r.action === 'launch' ? 'run-2' : null, transaction: null, refusal: null }, cancelled: null };
|
||||
return { request: args.request, instance: r.instance, action: r.action, state: 'closed', done: { run: r.action === 'launch' ? 'run-2' : null, transaction: null, refusal: r.action === 'end' && r.instance === 'coder' ? refusal : null }, cancelled: null };
|
||||
}
|
||||
throw Error(`unexpected ${verb}`);
|
||||
};
|
||||
@@ -72,6 +73,11 @@ test('F3 Settings > Routing: a save lists how the change reaches each agent, and
|
||||
for (const name of ['chief', 'coder', 'assistant', 'reviewer']) await b.evaluate(`(()=>{const s=${cell(name, 'thinking')};s.value='low';s.dispatchEvent(new Event('change',{bubbles:true}));return true;})()`);
|
||||
await b.evaluate(`document.getElementById('routes-form').requestSubmit();true`);
|
||||
await wait(`document.querySelector('.routes-apply')`);
|
||||
return { b, wait, asked, items, dir };
|
||||
}
|
||||
|
||||
test('F3 Settings > Routing: a save lists how the change reaches each agent, and Restart now moves a running one', { timeout: 120000 }, async t => {
|
||||
const { b, wait, asked, items, dir } = await stage(t);
|
||||
assert.deepEqual(asked, [['routes.set', { revision: REV, routes: { chief: route('low'), coder: route('low'), assistant: route('low'), reviewer: route('low') } }]]);
|
||||
assert.equal(await b.evaluate(`document.getElementById('routes-receipt').textContent`), 'Routes saved for chief, coder, assistant, reviewer. The next launch of each uses the new route.');
|
||||
assert.deepEqual(await items(), [
|
||||
@@ -103,3 +109,29 @@ test('F3 Settings > Routing: a save lists how the change reaches each agent, and
|
||||
await b.viewport(390, 844);
|
||||
assert.equal(await b.evaluate(`document.documentElement.scrollWidth<=390`), true, 'no sideways scroll at phone width');
|
||||
});
|
||||
|
||||
// SP19 (#1873): the drawer's Restart shares this chain. The route-change flow keeps its reason, its
|
||||
// unprefixed routes and every line it shows, step by step, and its refusal line.
|
||||
const WATCH = name => `(()=>{window.__seen=[];const snap=()=>{const s=document.querySelector('[data-agent-restart-status="${name}"]')?.textContent??'';if(s&&s!==window.__seen.at(-1))window.__seen.push(s);};new MutationObserver(snap).observe(document.body,{subtree:true,childList:true,characterData:true});return true;})()`;
|
||||
test('SP19: Restart now keeps its reason and its words', { timeout: 120000 }, async t => {
|
||||
const { b, wait, asked } = await stage(t);
|
||||
const posts = [];
|
||||
await b.evaluate(`(()=>{const f=window.fetch;window.__urls=[];window.fetch=(u,o)=>{window.__urls.push(String(u));return f(u,o);};return true;})()`);
|
||||
await b.evaluate(WATCH('coder'));
|
||||
await b.evaluate(`document.querySelector('[data-agent-restart="coder"]').click();true`);
|
||||
await wait(`document.querySelector('[data-agent-restart-status="coder"]')?.textContent==='Restarted coder on the new route (run run-2).'`, 30000);
|
||||
assert.deepEqual(await b.evaluate('window.__seen'), ['Terminating coder…', 'Terminated coder. Waiting for its session to end…', 'Launching coder…', 'Restarted coder on the new route (run run-2).']);
|
||||
assert.deepEqual(asked.filter(([v]) => v === 'instance.end'), [['instance.end', { instance: 'coder', reason: 'route change' }]]);
|
||||
posts.push(...(await b.evaluate('window.__urls')).filter(u => /\/api\/agent\/(terminate|launch|regenerate)/.test(u)));
|
||||
assert.deepEqual(posts, ['/api/agent/terminate', '/api/agent/launch']);
|
||||
assert.equal(await b.evaluate(`document.querySelector('[data-agent-restart="coder"]').textContent`), 'Restart coder now');
|
||||
});
|
||||
|
||||
test('SP19: Restart now says when the end is refused, in its own words', { timeout: 120000 }, async t => {
|
||||
const { b, wait, asked } = await stage(t, { refusal: 'instance-kept' });
|
||||
await b.evaluate(`document.querySelector('[data-agent-restart="coder"]').click();true`);
|
||||
await wait(`/It was not restarted/.test(document.querySelector('[data-agent-restart-status="coder"]')?.textContent??'')`, 30000);
|
||||
assert.equal(await b.evaluate(`document.querySelector('[data-agent-restart-status="coder"]').textContent`), 'coder: the instance is kept; unkeep it first. It was not restarted.');
|
||||
assert.equal(asked.filter(([v]) => v === 'instance.launch').length, 0);
|
||||
assert.equal(await b.evaluate(`document.querySelector('[data-agent-restart="coder"]').disabled`), false);
|
||||
});
|
||||
@@ -63,7 +63,7 @@ function fakeHost(over = {}) {
|
||||
const triage = {
|
||||
snapshotAt: () => Date.now(),
|
||||
refresh: async () => null,
|
||||
state: async () => { h.reads += 1; return { route: h.route, running: h.running, lastExit: h.lastExit, overCapUntil: h.overCapUntil, snapshot: h.snapshot, transcript: h.transcript }; },
|
||||
state: async () => { h.reads += 1; return { route: h.route, running: h.running, lastExit: h.lastExit, overCapUntil: h.overCapUntil, snapshot: h.snapshot, transcript: h.transcript, ...(Object.hasOwn(h, 'usageToday') ? { usageToday: h.usageToday, cap: h.cap, resetsAt: h.resetsAt } : {}) }; },
|
||||
message: async text => {
|
||||
h.sent.push(text);
|
||||
const e = h.next; h.next = null;
|
||||
@@ -409,3 +409,61 @@ test('item 5: markup and control characters in every field stay text: no element
|
||||
await sleep(500);
|
||||
assert.equal(await f.b.evaluate('window.__pwned ?? null'), null);
|
||||
});
|
||||
|
||||
// #1873 triage-followups: the page focuses its heading when its first render ends. A reader who opened
|
||||
// the drawer before then keeps focus in the drawer. The page holds GET /api/workspace, which the first
|
||||
// render waits on, until the drawer has opened and taken focus.
|
||||
const HOLD_WORKSPACE = `(() => { const f = window.fetch; window.__held = 0; window.fetch = (u, o) => { if (String(u?.url ?? u).includes('/api/workspace') && !window.__go) { window.__held += 1; return new Promise(r => { const t = setInterval(() => { if (window.__go) { clearInterval(t); r(f(u, o)); } }, 20); }); } return f(u, o); }; })()`;
|
||||
|
||||
test('triage-followups: a drawer opened before the first render ends keeps focus when the render ends', async t => {
|
||||
const f = await setup(t, { init: HOLD_WORKSPACE });
|
||||
await f.wait('document.querySelector("#triage-open") && window.__held > 0');
|
||||
const heading = () => f.b.evaluate('document.querySelector("#bus-view h1")?.textContent ?? null');
|
||||
await sleep(300);
|
||||
assert.equal(await heading(), 'Reading…', 'the first render has not ended');
|
||||
await f.click('#triage-open');
|
||||
await f.wait('document.querySelector("#triage-drawer")?.contains(document.activeElement)');
|
||||
await f.b.evaluate('window.__go = true');
|
||||
await f.wait('document.querySelector("#bus-view h1")?.textContent === "Dashboard"');
|
||||
await sleep(400);
|
||||
assert.equal(await f.b.evaluate('document.querySelector("#triage-drawer").contains(document.activeElement)'), true, `focus stays in the drawer, not on ${await f.b.evaluate('document.activeElement?.tagName')}`);
|
||||
// Escape still closes it and puts focus back on the entry.
|
||||
await f.b.call('Input.dispatchKeyEvent', { type: 'keyDown', key: 'Escape', code: 'Escape', windowsVirtualKeyCode: 27 });
|
||||
await f.wait('document.querySelector("#triage-drawer").hidden');
|
||||
assert.equal(await f.b.evaluate('document.activeElement?.id'), 'triage-open');
|
||||
});
|
||||
|
||||
test('triage-followups: with no drawer open the first render still focuses the heading', async t => {
|
||||
const f = await setup(t);
|
||||
await f.wait('document.activeElement?.tagName === "H1"');
|
||||
});
|
||||
|
||||
test('triage-cap: one usage line from the read: not set, the count against the cap with the reset, not read, and markup as text', async t => {
|
||||
const f = await setup(t, { host: fakeHost({ usageToday: 'not set', cap: 'not set', resetsAt: 'not set' }) });
|
||||
await open(f);
|
||||
assert.equal(await f.text('#triage-usage'), 'Daily token cap: not set.');
|
||||
assert.equal(await f.b.evaluate('document.querySelector("#triage-usage").hidden'), false);
|
||||
const resetsAt = new Date(Date.now() + 5 * 3600000).toISOString();
|
||||
const local = await f.b.evaluate(`window.MosaicLocalTime(${JSON.stringify(resetsAt)})`);
|
||||
Object.assign(f.h, { usageToday: { count: 1234, input: 1200, output: 34, cacheWrite: 0, cacheRead: 0, unknownTurns: 2 }, cap: { limit: 200000, reset: '00:00', timeZone: 'UTC' }, resetsAt });
|
||||
await f.wait('/^Today:/.test(document.querySelector("#triage-usage").textContent)', 400);
|
||||
const line = await f.text('#triage-usage');
|
||||
const [count, limit] = await f.b.evaluate('[(1234).toLocaleString(), (200000).toLocaleString()]');
|
||||
assert.ok(line.startsWith(`Today: ${count} of ${limit} tokens. Resets at `), line);
|
||||
assert.ok(line.includes(local), line);
|
||||
assert.ok(line.endsWith(' 2 turns have no usage reading.'), line);
|
||||
Object.assign(f.h, { usageToday: 'not read', cap: { limit: 200000, reset: '00:00', timeZone: 'UTC' }, resetsAt: 'not read' });
|
||||
await f.wait('/not read/.test(document.querySelector("#triage-usage").textContent)', 400);
|
||||
assert.equal(await f.text('#triage-usage'), "Today's token usage: not read.");
|
||||
// A count that is not a whole number is not shown as one; markup stays text.
|
||||
Object.assign(f.h, { usageToday: { count: '<img src=x onerror=alert(1)>', unknownTurns: 0 }, cap: { limit: 1.5 }, resetsAt: '<b>x</b>' });
|
||||
await f.wait('/^Today: not read/.test(document.querySelector("#triage-usage").textContent)', 400);
|
||||
assert.equal(await f.text('#triage-usage'), 'Today: not read of not read tokens.');
|
||||
assert.equal(await f.b.evaluate('document.querySelector("#triage-usage").children.length'), 0);
|
||||
});
|
||||
|
||||
test('triage-cap: a read without the usage fields shows no usage line', async t => {
|
||||
const f = await setup(t);
|
||||
await open(f);
|
||||
assert.equal(await f.b.evaluate('document.querySelector("#triage-usage").hidden'), true);
|
||||
});
|
||||
@@ -5,13 +5,18 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { connect } from "node:net";
|
||||
import { existsSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, symlinkSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { appendFileSync, chmodSync, existsSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
|
||||
import { dirname, join } from "node:path";
|
||||
import { Client } from "../../bus/src/client.mjs";
|
||||
import { sandboxProblem } from "../../seat/src/sandbox.mjs";
|
||||
import { REPO, RUNNER_FILES, checkHostedRoute, sandboxProblem } from "../../seat/src/sandbox.mjs";
|
||||
import { isProviderVariable } from "../../seat/src/session.mjs";
|
||||
import {
|
||||
NOT_STARTED, NOT_TAKEN, RELAY_REFUSED, ROUTE_ENV_ALLOW, SPAWN_ENV_ALLOW, UNANSWERED, createRelay, createTriageHost, hostPrompt, triageEnv,
|
||||
NOT_STARTED, NOT_TAKEN, RELAY_REFUSED, ROUTE_ENV_ALLOW, ROUTE_REFUSED, SPAWN_ENV_ALLOW, UNANSWERED, createRelay, createTriageHost, hostPrompt,
|
||||
USAGE_NOT_READ, overCapNotTaken, refusedHarnessName, triageEnv,
|
||||
} from "../src/triage-host.mjs";
|
||||
import { writeRoute } from "../src/triage-route-store.mjs";
|
||||
import { appendUsage, readUsage, usagePath } from "../src/triage-usage.mjs";
|
||||
import { triageView } from "../src/triage-http.mjs";
|
||||
|
||||
const FIXTURE = new URL("./fixtures/triage-adapter.mjs", import.meta.url).pathname;
|
||||
const skip = sandboxProblem() ?? false;
|
||||
@@ -63,13 +68,20 @@ test("the relay refuses every verb but a message to the person, and any call wit
|
||||
assert.deepEqual(await client.call("message.receive"), [{ id: "owner-0123456789ab", from_role: "human", class: "REQUEST", body: "hello", at: "2026-11-16T12:00:00.000Z" }]);
|
||||
assert.match((await client.call("message.send", { to: "human", body: "answer", class: "RESULT", in_reply_to: "owner-0123456789ab" })).id, /^triage-/);
|
||||
await client.call("message.read", { id: "owner-0123456789ab" });
|
||||
assert.deepEqual(sent, [{ body: "answer", in_reply_to: "owner-0123456789ab" }]);
|
||||
assert.deepEqual(sent, [{ body: "answer", in_reply_to: "owner-0123456789ab", result: true }]);
|
||||
assert.deepEqual(read, ["owner-0123456789ab"]);
|
||||
// Only the class RESULT send, the runner's answer to its turn, says the message is answered. A note
|
||||
// in any other class, or with no class, is kept but does not.
|
||||
for (const cls of ["INFO", "REQUEST", "result", undefined, 7]) {
|
||||
sent.length = 0;
|
||||
await client.call("message.send", { to: "human", body: "note", class: cls, in_reply_to: "owner-0123456789ab" });
|
||||
assert.deepEqual(sent, [{ body: "note", in_reply_to: "owner-0123456789ab", result: false }], String(cls));
|
||||
}
|
||||
// The model writes in_reply_to. Anything but an id the host issued is dropped, never stored.
|
||||
for (const bad of ["<img src=x onerror=1>", "owner-1", "owner-0123456789ab\u202e", "owner-0123456789abc", "x".repeat(60000)]) {
|
||||
sent.length = 0;
|
||||
await client.call("message.send", { to: "human", body: "b", in_reply_to: bad });
|
||||
assert.deepEqual(sent, [{ body: "b", in_reply_to: null }], bad.slice(0, 24));
|
||||
assert.deepEqual(sent, [{ body: "b", in_reply_to: null, result: false }], bad.slice(0, 24));
|
||||
}
|
||||
const refused = [
|
||||
["decision.raise", { kind: "hire", body: "x" }], ["event.emit", { kind: "action.allowed", body: {} }],
|
||||
@@ -95,13 +107,68 @@ test("the prompt names the snapshot inside the session and the card form, after
|
||||
assert.match(text, /```cards\n\[\{"step": "bus-start", "org": "alpha"\}\]\n```/);
|
||||
});
|
||||
|
||||
// Names the route may try in harnessEnv that the host refuses even when the module declares them.
|
||||
const REFUSED_TRIED = [
|
||||
"MOSAIC_POLICY_FILE", "MOSAIC_TOOLS_FILE", "MOSAIC_CONFIG", "MOSAIC_BUS_CONSOLE_SOCKET", "NODE_OPTIONS", "NODE_PATH", "LD_PRELOAD",
|
||||
"LD_LIBRARY_PATH", "DYLD_INSERT_LIBRARIES", "PYTHONPATH", "PYTHONSTARTUP", "PATH", "HOME", "TMPDIR", "BASH_ENV", "ENV", "SSH_AUTH_SOCK",
|
||||
"GITEA_TOKEN", "GH_TOKEN", "GIT_ASKPASS", "GIT_CONFIG_PARAMETERS", "VIKUNJA_TOKEN", "ANTHROPIC_BASE_URL", "ANTHROPIC_AUTH_TOKEN",
|
||||
"ANTHROPIC_API_KEY", "CLAUDE_CODE_OAUTH_TOKEN", "OPENAI_API_KEY", "GLIBC_TUNABLES", "PI_CODING_AGENT_DIR", "XDG_CONFIG_HOME",
|
||||
];
|
||||
// The fake harness module: it declares the probe names and every refused one, so only the host's
|
||||
// own refusal stops those. Its one install root is <root>/install.
|
||||
const DECLARED = Object.freeze(["TRIAGE_PROBE", "FAKE_OK", "FAKE_TWO", ...REFUSED_TRIED]);
|
||||
// Its transcript is <run>/session, where the stub adapter's SPEND writes (spendSource reads it).
|
||||
const fakeModule = (roots, names = DECLARED) => async (name) => name === "fake" ? { envNames: names, installRoots: () => roots, transcript: ({ runDir }) => join(runDir, "session") } : null;
|
||||
|
||||
function host(t, root, opts = {}) {
|
||||
const dataRoot = join(root, "data");
|
||||
const h = createTriageHost({ dataRoot, readSnapshot: async () => SNAPSHOT, env: CONSOLE_ENV, readOnly: [FIXTURE], pollInterval: 100, ...opts });
|
||||
mkdirSync(join(root, "install"), { recursive: true });
|
||||
if (!existsSync(join(root, "install", "adapter.sh"))) writeFileSync(join(root, "install", "adapter.sh"), "exit 0\n");
|
||||
const h = createTriageHost({ dataRoot, readSnapshot: async () => SNAPSHOT, env: CONSOLE_ENV, readOnly: [FIXTURE], pollInterval: 100, harnessModule: fakeModule([join(root, "install")]), ...opts });
|
||||
t.after(() => h.stop());
|
||||
return { h, dataRoot };
|
||||
}
|
||||
|
||||
// A launcher that only leaves a mark: a spawn is proved by the mark, no spawn by its absence.
|
||||
function spy(root) {
|
||||
const f = join(root, "spy");
|
||||
writeFileSync(f, `#!/bin/sh\n: > '${join(root, "spawned")}'\nexit 0\n`, { mode: 0o755 });
|
||||
return f;
|
||||
}
|
||||
const answer = (root, over = {}) => ({
|
||||
ok: true, adapter: join(root, "install", "adapter.sh"), harness: "fake", harnessEnv: {}, harnessMounts: [], turnEnd: "exit",
|
||||
model: "m", provider: "p", env: { ANTHROPIC_BASE_URL: "http://127.0.0.1:9" }, ...over,
|
||||
});
|
||||
// One route answer through a fresh host: refused before anything is written or spawned, with the
|
||||
// one code, the owner-readable line, and a log line that names the fence.
|
||||
async function refusedRoute(t, label, make, opts = {}) {
|
||||
const root = scratch();
|
||||
mkdirSync(join(root, "install"), { recursive: true });
|
||||
const logs = [];
|
||||
const over = await make(root);
|
||||
const route = { admit: async () => (typeof over === "function" ? over() : answer(root, over)) };
|
||||
const { h, dataRoot } = host(t, root, { route, bwrap: spy(root), log: (l) => logs.push(l), ...(typeof opts === "function" ? opts(root) : opts) });
|
||||
await assert.rejects(h.message("hello"), (e) => e.code === "triage-route-invalid" && e.message === "the model route's answer is not usable", label);
|
||||
assert.equal(existsSync(join(root, "spawned")), false, `${label}: nothing spawned`);
|
||||
assert.equal(h.running, false, label);
|
||||
assert.equal(existsSync(join(dataRoot, "triage", "agent", "session.json")), false, `${label}: no bundle`);
|
||||
const [owner, note, ...rest] = (await h.state()).transcript;
|
||||
assert.deepEqual([owner.from, note.from, note.in_reply_to, note.text, rest.length], ["owner", "host", owner.id, ROUTE_REFUSED, 0], label);
|
||||
assert.ok(logs.some((l) => /^triage host: route refused: [a-z-]+$/.test(l)), `${label}: ${logs.join(" | ")}`);
|
||||
return logs;
|
||||
}
|
||||
// One route answer through a fresh host that must reach the launcher; the session.json it wrote.
|
||||
async function admittedRoute(t, label, make, opts = {}) {
|
||||
const root = scratch();
|
||||
mkdirSync(join(root, "install"), { recursive: true });
|
||||
const over = await make(root);
|
||||
const route = { admit: async () => (typeof over === "function" ? over() : answer(root, over)) };
|
||||
const { h, dataRoot } = host(t, root, { route, bwrap: spy(root), ...(typeof opts === "function" ? opts(root) : opts) });
|
||||
await h.message("hello");
|
||||
await waitFor(() => existsSync(join(root, "spawned")));
|
||||
return JSON.parse(readFileSync(join(dataRoot, "triage", "agent", "session.json"), "utf8"));
|
||||
}
|
||||
|
||||
test("no model route: nothing launches, the message is refused, and the state says the route is not set", async (t) => {
|
||||
const { h, dataRoot } = host(t, scratch());
|
||||
assert.equal((await h.state()).route, "not set");
|
||||
@@ -147,20 +214,250 @@ test("over the cap: the message is refused until the next day and nothing launch
|
||||
await assert.rejects(other.message("hello"), (e) => e.code === "triage-route-refused" && e.until === undefined);
|
||||
});
|
||||
|
||||
test("a route whose answer carries a config, bus or founder variable is refused before anything starts", async (t) => {
|
||||
for (const name of ["MOSAIC_CONFIG", "MOSAIC_BUS_CONSOLE_SOCKET", "GITEA_TOKEN", "SSH_AUTH_SOCK", "lower"]) {
|
||||
const root = scratch();
|
||||
const adapter = join(root, "adapter.sh");
|
||||
writeFileSync(adapter, "exit 0\n");
|
||||
const route = { admit: async () => ({ ok: true, adapter, harness: "fake", harnessEnv: { [name]: "x" }, harnessMounts: [], turnEnd: "exit", model: "m", provider: "p", env: {} }) };
|
||||
const { h, dataRoot } = host(t, root, { route });
|
||||
await assert.rejects(h.message("hello"), (e) => e.code === "triage-route-invalid", name);
|
||||
assert.equal(h.running, false);
|
||||
assert.equal(existsSync(join(dataRoot, "triage", "agent", "session.json")), false);
|
||||
// The owner's line is written before the start; the transcript then says it was not delivered.
|
||||
const [owner, note, ...rest] = (await h.state()).transcript;
|
||||
assert.deepEqual([owner.from, owner.text, note.from, note.in_reply_to, note.text, rest.length], ["owner", "hello", "host", owner.id, NOT_STARTED, 0], name);
|
||||
// #1873 triage-fences, design section 4. Every refused answer reaches no spawn (the spy launcher
|
||||
// leaves no mark), writes no bundle, answers triage-route-invalid and leaves ROUTE_REFUSED under
|
||||
// the owner's line.
|
||||
|
||||
test("fence 1: a harnessEnv name the host refuses is refused even when the harness module declares it", async (t) => {
|
||||
for (const name of REFUSED_TRIED) {
|
||||
assert.ok(DECLARED.includes(name));
|
||||
await refusedRoute(t, name, () => ({ harnessEnv: { [name]: "x" } }));
|
||||
assert.equal(refusedHarnessName(name), true, name);
|
||||
}
|
||||
// Names that are not one plain upper-case name: lower or mixed case, an equals sign, a NUL.
|
||||
for (const name of ["lower", "Mosaic_policy_file", "mosaic_policy_file", "FAKE_OK=x", "A=B", "FAKE_OK\0", " FAKE_OK", "fake_ok"]) {
|
||||
await refusedRoute(t, JSON.stringify(name), () => ({ harnessEnv: { [name]: "x" } }), (root) => ({ harnessModule: fakeModule([join(root, "install")], [...DECLARED, name]) }));
|
||||
}
|
||||
for (const name of [
|
||||
"GIT_DIR", "BASH_FUNC_x%%", "LD_AUDIT", "SHELLOPTS", "PS4", "IFS", "HTTPS_PROXY", "NODE_EXTRA_CA_CERTS", "PYTHONHOME", "MOSAIC_X", "GCONV_PATH",
|
||||
"LOCPATH", "MALLOC_CHECK_", "OPENSSL_CONF", "SSL_CERT_FILE", "ZDOTDIR", "PROMPT_COMMAND", "JAVA_TOOL_OPTIONS", "_JAVA_OPTIONS", "NPM_CONFIG_PREFIX",
|
||||
"CLAUDE_CONFIG_DIR", "XDG_DATA_HOME", "ANY_CONF", "ANY_CONFIG", "PERLLIB", "LESSOPEN", "LESSCLOSE", "HOSTALIASES",
|
||||
"SSH_ASKPASS", "SUDO_ASKPASS", "CURL_CA_BUNDLE", "REQUESTS_CA_BUNDLE", "OPENAI_BASE_URL", "ANY_BASE_URL",
|
||||
"LUA_INIT", "CURL_HOME", "SSLKEYLOGFILE",
|
||||
]) assert.equal(refusedHarnessName(name), true, name);
|
||||
for (const name of ["FAKE_OK", "PI_MODEL", "TRIAGE_PROBE"]) assert.equal(refusedHarnessName(name), false, name);
|
||||
});
|
||||
|
||||
test("fence 1: values are strings of at most 4096 characters with no control character or line separator, and at most 64 names", async (t) => {
|
||||
for (const [label, v] of [
|
||||
["newline", "x\ny"], ["nul", "x\0"], ["cr", "x\ry"], ["tab", "x\ty"], ["escape", "x\u001by"], ["del", "x\u007f"], ["c1 next line", "x\u0085y"],
|
||||
["line separator", "x\u2028y"], ["paragraph separator", "x\u2029y"], ["4097", "a".repeat(4097)], ["number", 5], ["null", null], ["object", {}], ["array", ["x"]],
|
||||
]) {
|
||||
await refusedRoute(t, label, () => ({ harnessEnv: { FAKE_OK: v } }));
|
||||
}
|
||||
const many = Object.fromEntries(Array.from({ length: 65 }, (_, i) => [`FAKE_N${i}`, "x"]));
|
||||
await refusedRoute(t, "65 names", () => ({ harnessEnv: many }), (root) => ({ harnessModule: fakeModule([join(root, "install")], Object.keys(many)) }));
|
||||
await admittedRoute(t, "64 names", () => ({ harnessEnv: Object.fromEntries(Object.entries(many).slice(0, 64)) }), (root) => ({ harnessModule: fakeModule([join(root, "install")], Object.keys(many)) }));
|
||||
await refusedRoute(t, "array harnessEnv", () => ({ harnessEnv: ["FAKE_OK"] }));
|
||||
await refusedRoute(t, "no harnessEnv", () => ({ harnessEnv: undefined }));
|
||||
const s = await admittedRoute(t, "4096", () => ({ harnessEnv: { FAKE_OK: "a".repeat(4096), FAKE_TWO: "" } }));
|
||||
assert.deepEqual(Object.keys(s.harnessEnv), ["FAKE_OK", "FAKE_TWO"]);
|
||||
});
|
||||
|
||||
test("fence 1: a name the harness module does not declare is refused; a declared one reaches the session's bundle", async (t) => {
|
||||
await refusedRoute(t, "undeclared", () => ({ harnessEnv: { FAKE_UNDECLARED: "x" } }));
|
||||
await refusedRoute(t, "no such module", () => ({ harness: "nope" }));
|
||||
await refusedRoute(t, "module without envNames", () => ({}), (root) => ({ harnessModule: async () => ({ installRoots: () => [join(root, "install")] }) }));
|
||||
await refusedRoute(t, "module without installRoots", () => ({}), { harnessModule: async () => ({ envNames: DECLARED }) });
|
||||
await refusedRoute(t, "module that throws", () => ({}), { harnessModule: async () => { throw new Error("x"); } });
|
||||
await refusedRoute(t, "installRoots that throws", () => ({}), { harnessModule: async () => ({ envNames: DECLARED, installRoots: () => { throw new Error("x"); } }) });
|
||||
const s = await admittedRoute(t, "declared", () => ({ harnessEnv: { FAKE_OK: "yes" } }));
|
||||
assert.deepEqual(s.harnessEnv, { FAKE_OK: "yes" });
|
||||
});
|
||||
|
||||
test("fence 1: the bundle holds the copy the host checked; no MOSAIC_ name reaches session.json, even from a getter", async (t) => {
|
||||
let reads = 0;
|
||||
const s = await admittedRoute(t, "getter", (root) => () => {
|
||||
const a = answer(root);
|
||||
delete a.harnessEnv;
|
||||
Object.defineProperty(a, "harnessEnv", { enumerable: true, get: () => (reads++ === 0 ? { FAKE_OK: "checked" } : { MOSAIC_POLICY_FILE: "/evil", FAKE_OK: "swapped" }) });
|
||||
return a;
|
||||
});
|
||||
assert.equal(reads, 1, "the answer is read once");
|
||||
assert.deepEqual(s.harnessEnv, { FAKE_OK: "checked" });
|
||||
assert.ok(!Object.keys(s.harnessEnv).some((k) => k.startsWith("MOSAIC_")));
|
||||
// An answer that cannot be copied (a function in it) is refused.
|
||||
await refusedRoute(t, "function", (root) => ({ harnessEnv: { FAKE_OK: "x" }, extra: () => 1 }));
|
||||
});
|
||||
|
||||
test("fence 2: route env holds only the proxy's two names, the base URL is http on 127.0.0.1 with a port, and the host's port and token shape hold when given", async (t) => {
|
||||
const bad = [
|
||||
undefined, "", "http://localhost:9", "http://[::1]:9", "https://127.0.0.1:9", "http://127.0.0.1", "http://127.0.0.1:", "http://127.0.0.1:9/",
|
||||
"http://127.0.0.1:9/v1", "http://127.0.0.1:09", "http://127.0.0.1:0", "http://127.0.0.1:65536", "http://127.0.0.1:99999", "http://127.0.0.2:9",
|
||||
"http://[email protected]:9", "HTTP://127.0.0.1:9", "http://127.0.0.1:9\n", "http://127.0.0.1:9 ", "http://127.1:9", "http://2130706433:9",
|
||||
"http://127.0.0.1:9#x", "http://127.0.0.1:9?x", "http://10.0.0.1:9", 9,
|
||||
];
|
||||
for (const url of bad) await refusedRoute(t, `url ${JSON.stringify(url)}`, () => ({ env: url === undefined ? {} : { ANTHROPIC_BASE_URL: url } }));
|
||||
for (const extra of ["OTHER", "MOSAIC_CONFIG", "ANTHROPIC_API_KEY", "PATH", "anthropic_base_url"]) {
|
||||
await refusedRoute(t, `env ${extra}`, () => ({ env: { ANTHROPIC_BASE_URL: "http://127.0.0.1:9", [extra]: "x" } }));
|
||||
}
|
||||
for (const token of ["a\nb", "a\u2028b", "a\u0000b", "a\u007fb"]) await refusedRoute(t, `token ${JSON.stringify(token)}`, () => ({ env: { ANTHROPIC_BASE_URL: "http://127.0.0.1:9", ANTHROPIC_AUTH_TOKEN: token } }));
|
||||
await refusedRoute(t, "env array", () => ({ env: ["ANTHROPIC_BASE_URL"] }));
|
||||
await admittedRoute(t, "port 65535", () => ({ env: { ANTHROPIC_BASE_URL: "http://127.0.0.1:65535" } }));
|
||||
// The host's port.
|
||||
await refusedRoute(t, "other port", () => ({ env: { ANTHROPIC_BASE_URL: "http://127.0.0.1:9" } }), { proxyPort: 9001 });
|
||||
await admittedRoute(t, "its port", () => ({ env: { ANTHROPIC_BASE_URL: "http://127.0.0.1:9001" } }), { proxyPort: 9001 });
|
||||
// The host's token shape: the whole token, never a part of it.
|
||||
const tokenPattern = /mst-[0-9a-f]{8}/g;
|
||||
for (const token of [undefined, "xmst-deadbeef", "mst-deadbeefx", "mst-DEADBEEF", "", "mst-deadbee"]) {
|
||||
await refusedRoute(t, `token ${token}`, () => ({ env: { ANTHROPIC_BASE_URL: "http://127.0.0.1:9", ...(token === undefined ? {} : { ANTHROPIC_AUTH_TOKEN: token }) } }), { tokenPattern });
|
||||
}
|
||||
await admittedRoute(t, "token", () => ({ env: { ANTHROPIC_BASE_URL: "http://127.0.0.1:9", ANTHROPIC_AUTH_TOKEN: "mst-deadbeef" } }), { tokenPattern });
|
||||
await admittedRoute(t, "token twice", () => ({ env: { ANTHROPIC_BASE_URL: "http://127.0.0.1:9", ANTHROPIC_AUTH_TOKEN: "mst-deadbeef" } }), { tokenPattern });
|
||||
for (const p of [0, 65536, "9", 1.5]) assert.throws(() => createTriageHost({ dataRoot: "/x", proxyPort: p }), /proxyPort/);
|
||||
assert.throws(() => createTriageHost({ dataRoot: "/x", tokenPattern: "mst-" }), /tokenPattern/);
|
||||
});
|
||||
|
||||
test("fence 3: a mount source outside the install roots, through a link out of them, or in the data root, the config directory, HOME or a secrets path is refused", async (t) => {
|
||||
const file = (p) => { mkdirSync(dirname(p), { recursive: true }); writeFileSync(p, "x"); return p; };
|
||||
const mount = (src, extra = {}) => ({ harnessMounts: [{ src, why: "w", ...extra }] });
|
||||
await refusedRoute(t, "outside", (root) => mount(file(join(root, "elsewhere", "f"))));
|
||||
await refusedRoute(t, "link out", (root) => { file(join(root, "elsewhere", "f")); symlinkSync(join(root, "elsewhere", "f"), join(root, "install", "out")); return mount(join(root, "install", "out")); });
|
||||
await refusedRoute(t, "dir link out", (root) => { file(join(root, "elsewhere", "f")); symlinkSync(join(root, "elsewhere"), join(root, "install", "d")); return mount(join(root, "install", "d", "f")); });
|
||||
await refusedRoute(t, "dotdot", (root) => { file(join(root, "elsewhere", "f")); return mount(`${root}/install/../elsewhere/f`); });
|
||||
await refusedRoute(t, "dotdot in root", (root) => { file(join(root, "install", "f")); return mount(`${root}/install/x/../f`); });
|
||||
await refusedRoute(t, "trailing slash", (root) => { mkdirSync(join(root, "install", "d")); return mount(join(root, "install", "d") + "/"); });
|
||||
await refusedRoute(t, "double slash", (root) => { file(join(root, "install", "f")); return mount(root + "//install/f"); });
|
||||
await refusedRoute(t, "relative", () => mount("install/adapter.sh"));
|
||||
await refusedRoute(t, "missing", (root) => mount(join(root, "install", "absent")));
|
||||
await refusedRoute(t, "case", (root) => { file(join(root, "install", "f")); return mount(join(root, "INSTALL", "f")); });
|
||||
// With the whole scratch root as the install root, only the protected-path fence stands.
|
||||
const wide = (root) => ({ harnessModule: fakeModule([root]) });
|
||||
await refusedRoute(t, "data root", (root) => mount(file(join(root, "data", "f"))), wide);
|
||||
await refusedRoute(t, "parent of data root", (root) => { mkdirSync(join(root, "data"), { recursive: true }); return mount(join(root)); }, wide);
|
||||
await refusedRoute(t, "alias of data root", (root) => { file(join(root, "data", "f")); symlinkSync(join(root, "data"), join(root, "alias")); return mount(join(root, "alias", "f")); }, wide);
|
||||
await refusedRoute(t, "config dir", (root) => mount(file(join(root, "cfg", "f"))), (root) => ({ ...wide(root), env: { ...CONSOLE_ENV, MOSAIC_CONFIG: join(root, "cfg", "config.json") } }));
|
||||
await refusedRoute(t, "home", (root) => mount(file(join(root, "h", "f"))), (root) => ({ ...wide(root), home: join(root, "h") }));
|
||||
await refusedRoute(t, "parent of home", (root) => { file(join(root, "p", "h", "f")); return mount(join(root, "p")); }, (root) => ({ ...wide(root), home: join(root, "p", "h") }));
|
||||
for (const seg of ["secrets", "Secrets", "SECRET", "my-secret-store"]) {
|
||||
await refusedRoute(t, seg, (root) => mount(file(join(root, "install", seg, "token"))));
|
||||
}
|
||||
await refusedRoute(t, "link into secrets", (root) => { file(join(root, "install", "secrets", "token")); symlinkSync(join(root, "install", "secrets", "token"), join(root, "install", "t")); return mount(join(root, "install", "t")); });
|
||||
await refusedRoute(t, "9 mounts", (root) => ({ harnessMounts: Array.from({ length: 9 }, (_, i) => ({ src: file(join(root, "install", `m${i}`)), dest: `/opt/m${i}`, why: "w" })) }));
|
||||
await refusedRoute(t, "mounts not a list", (root) => ({ harnessMounts: { src: file(join(root, "install", "f")) } }));
|
||||
await refusedRoute(t, "extra key", (root) => mount(file(join(root, "install", "f")), { mode: "rw" }));
|
||||
await refusedRoute(t, "no why", (root) => ({ harnessMounts: [{ src: file(join(root, "install", "f")) }] }));
|
||||
await refusedRoute(t, "string mount", (root) => ({ harnessMounts: [file(join(root, "install", "f"))] }));
|
||||
await refusedRoute(t, "install root is /", () => ({}), { harnessModule: fakeModule(["/"]) });
|
||||
await refusedRoute(t, "install root not a list", (root) => ({}), (root) => ({ harnessModule: fakeModule(join(root, "install")) }));
|
||||
await refusedRoute(t, "no install roots", () => ({}), { harnessModule: fakeModule([]) });
|
||||
await admittedRoute(t, "8 mounts", (root) => ({ harnessMounts: Array.from({ length: 8 }, (_, i) => ({ src: file(join(root, "install", `m${i}`)), dest: `/opt/m${i}`, why: "w" })) }));
|
||||
});
|
||||
|
||||
test("fence 3: a mount dest that is or holds /tmp, the run directory, the relay socket, the bundle, session.json, the health dest, a system bind or a runner file is refused", async (t) => {
|
||||
const withDest = (dest) => (root) => {
|
||||
writeFileSync(join(root, "install", "f"), "x");
|
||||
return { harnessMounts: [{ src: join(root, "install", "f"), dest: typeof dest === "function" ? dest(root) : dest, why: "w" }] };
|
||||
};
|
||||
const run = (root, ...p) => join(root, "data", "triage", "agent", ...p);
|
||||
const dests = [
|
||||
["/tmp", "/tmp"], ["under /tmp", "/tmp/x"], ["/tmp/", "/tmp/"], ["root", "/"], ["/usr/bin/sh", "/usr/bin/sh"], ["/usr", "/usr"], ["/etc", "/etc"],
|
||||
["/etc/hosts", "/etc/hosts"], ["relative", "opt/x"], ["dotdot", "/opt/../tmp"], ["empty", ""], ["number", 5],
|
||||
["run dir", (r) => run(r)], ["in run dir", (r) => run(r, "x")], ["relay", (r) => run(r, "relay.sock")], ["bundle", (r) => run(r, "bundle")],
|
||||
["session.json", (r) => run(r, "session.json")], ["health dest", (r) => run(r, "work", "health")], ["parent of run dir", (r) => dirname(run(r))],
|
||||
["parent of data root", (r) => r], ["runner file", RUNNER_FILES[0]], ["holds runner files", join(REPO, "packages")], ["repo", REPO],
|
||||
["node", process.execPath], ["holds node", dirname(process.execPath)], ["adapter", (r) => join(r, "install", "adapter.sh")],
|
||||
["holds adapter", (r) => join(r, "install")],
|
||||
];
|
||||
for (const [label, dest] of dests) await refusedRoute(t, `dest ${label}`, withDest(dest));
|
||||
await refusedRoute(t, "same dest twice", (root) => {
|
||||
writeFileSync(join(root, "install", "f"), "x");
|
||||
writeFileSync(join(root, "install", "g"), "x");
|
||||
return { harnessMounts: [{ src: join(root, "install", "f"), dest: "/opt/x", why: "w" }, { src: join(root, "install", "g"), dest: "/opt/x", why: "w" }] };
|
||||
});
|
||||
for (const [label, a, b] of [["nested dest", "/opt/x", "/opt/x/f"], ["parent dest", "/opt/x/f", "/opt/x"]]) {
|
||||
await refusedRoute(t, label, (root) => {
|
||||
writeFileSync(join(root, "install", "f"), "x");
|
||||
writeFileSync(join(root, "install", "g"), "x");
|
||||
return { harnessMounts: [{ src: join(root, "install", "f"), dest: a, why: "w" }, { src: join(root, "install", "g"), dest: b, why: "w" }] };
|
||||
});
|
||||
}
|
||||
await admittedRoute(t, "dest /opt/x", withDest("/opt/x"));
|
||||
});
|
||||
|
||||
test("fence 3: the adapter takes the same checks: inside an install root, a file, not through a link out, not in a protected path", async (t) => {
|
||||
await refusedRoute(t, "outside", (root) => { writeFileSync(join(root, "a.sh"), "exit 0\n"); return { adapter: join(root, "a.sh") }; });
|
||||
await refusedRoute(t, "link out", (root) => { writeFileSync(join(root, "a.sh"), "exit 0\n"); symlinkSync(join(root, "a.sh"), join(root, "install", "a.sh")); return { adapter: join(root, "install", "a.sh") }; });
|
||||
await refusedRoute(t, "directory", (root) => { mkdirSync(join(root, "install", "d")); return { adapter: join(root, "install", "d") }; });
|
||||
await refusedRoute(t, "relative", () => ({ adapter: "install/adapter.sh" }));
|
||||
await refusedRoute(t, "dotdot", (root) => ({ adapter: `${root}/install/../install/adapter.sh` }));
|
||||
await refusedRoute(t, "missing", (root) => ({ adapter: join(root, "install", "absent.sh") }));
|
||||
await refusedRoute(t, "secrets", (root) => { mkdirSync(join(root, "install", "secrets")); writeFileSync(join(root, "install", "secrets", "a.sh"), "x"); return { adapter: join(root, "install", "secrets", "a.sh") }; });
|
||||
await refusedRoute(t, "data root", (root) => { mkdirSync(join(root, "data"), { recursive: true }); writeFileSync(join(root, "data", "a.sh"), "x"); return { adapter: join(root, "data", "a.sh") }; }, (root) => ({ harnessModule: fakeModule([root]) }));
|
||||
await refusedRoute(t, "not a string", () => ({ adapter: 5 }));
|
||||
// A link that stays inside the root is followed: the bind's source is the file itself.
|
||||
await admittedRoute(t, "link inside", (root) => { writeFileSync(join(root, "install", "real.sh"), "exit 0\n"); symlinkSync(join(root, "install", "real.sh"), join(root, "install", "a.sh")); return { adapter: join(root, "install", "a.sh") }; });
|
||||
});
|
||||
|
||||
test("the shipped harness modules declare what their own env() and mounts() hand a session, and their adapter and mounts pass the host's checks", async () => {
|
||||
const dir = scratch();
|
||||
for (const name of ["pi", "claude-code"]) {
|
||||
const mod = await import(new URL(`../../plugins/harness/${name}/index.mjs`, import.meta.url).href);
|
||||
assert.ok(Array.isArray(mod.envNames) && Object.isFrozen(mod.envNames), name);
|
||||
const keys = Object.keys(mod.env({
|
||||
files: { settings: "/s", mcp: "/m" }, policy: { tools: ["read"] }, tools: [], session: { model: "m", thinking: "off", provider: "p" },
|
||||
}));
|
||||
assert.deepEqual(keys.filter((k) => !isProviderVariable(k)).sort(), [...mod.envNames].sort(), name);
|
||||
const roots = mod.installRoots({ env: {} });
|
||||
const route = { adapter: mod.adapter(), harnessMounts: mod.mounts({ env: {} }), installRoots: roots, runDir: join(dir, "data", "triage", "agent") };
|
||||
const got = checkHostedRoute({ ...route, protectedPaths: [join(dir, "data"), join(dir, "cfg"), join(dir, "home")] });
|
||||
assert.equal(got.adapter.dest, mod.adapter(), name);
|
||||
// A stack that runs from under HOME (a copy or a worktree): HOME holds the repository, and the
|
||||
// code root keeps HOME from refusing the stack's own files. Without it, HOME refuses them.
|
||||
const underHome = { ...route, protectedPaths: [join(dir, "data"), join(dir, "cfg"), dirname(REPO)] };
|
||||
assert.equal(checkHostedRoute({ ...underHome, codeRoot: REPO, codeHolder: dirname(REPO) }).adapter.dest, mod.adapter(), name);
|
||||
assert.throws(() => checkHostedRoute(underHome), (e) => e.fence === "adapter", name);
|
||||
}
|
||||
});
|
||||
|
||||
test("a host whose HOME holds the repository starts the shipped pi harness's own route; a source under HOME outside the repository is still refused", async (t) => {
|
||||
const pi = await import(new URL("../../plugins/harness/pi/index.mjs", import.meta.url).href);
|
||||
const home = dirname(REPO);
|
||||
const opts = { home, harnessModule: async (name) => (name === "pi" ? pi : null) };
|
||||
const piRoute = (root) => answer(root, { harness: "pi", adapter: pi.adapter(), harnessMounts: pi.mounts({ env: {} }), harnessEnv: { PI_MODEL: "m", PI_OFFLINE: "1" } });
|
||||
const s = await admittedRoute(t, "pi under HOME", (root) => () => piRoute(root), opts);
|
||||
assert.equal(s.adapter, pi.adapter());
|
||||
assert.deepEqual(s.harnessEnv, { PI_MODEL: "m", PI_OFFLINE: "1" });
|
||||
// The same host and module, with HOME's own directory (which holds the repository) as an extra
|
||||
// install root and mount: it is under HOME and not in the code, so it is refused.
|
||||
const wide = { ...pi, installRoots: (o) => [...pi.installRoots(o), home] };
|
||||
const logs = await refusedRoute(t, "under HOME, outside the repository", (root) => () => ({ ...piRoute(root), harnessMounts: [...pi.mounts({ env: {} }), { src: home, dest: "/opt/home", why: "w" }] }), { home, harnessModule: async (name) => (name === "pi" ? wide : null) });
|
||||
assert.ok(logs.includes("triage host: route refused: mount-src"), logs.join(" | "));
|
||||
});
|
||||
|
||||
test("a route that changes after the first check is refused at the binds: nothing spawns, what the start wrote is removed, and the next message starts", async (t) => {
|
||||
const root = scratch();
|
||||
mkdirSync(join(root, "install"), { recursive: true });
|
||||
writeFileSync(join(root, "outside.sh"), "exit 0\n");
|
||||
const logs = [];
|
||||
let checked = false;
|
||||
let swapped = false;
|
||||
const harnessModule = async (name) => (name === "fake" ? { envNames: DECLARED, installRoots: () => { checked = true; return [join(root, "install")]; } } : null);
|
||||
// The run id in session.json reads the clock after the first check: the adapter becomes a link
|
||||
// out of the install root then, so only hostedBinds' own check can see it.
|
||||
const now = () => {
|
||||
if (checked && !swapped) {
|
||||
swapped = true;
|
||||
rmSync(join(root, "install", "adapter.sh"));
|
||||
symlinkSync(join(root, "outside.sh"), join(root, "install", "adapter.sh"));
|
||||
}
|
||||
return new Date("2026-11-16T12:00:00.000Z");
|
||||
};
|
||||
const route = { admit: async () => answer(root) };
|
||||
const { h, dataRoot } = host(t, root, { route, bwrap: spy(root), log: (l) => logs.push(l), harnessModule, now });
|
||||
const agent = join(dataRoot, "triage", "agent");
|
||||
await assert.rejects(h.message("hello"), (e) => e.code === "triage-route-invalid" && e.message === "the model route's answer is not usable");
|
||||
assert.ok(swapped, "the swap happened between the checks");
|
||||
assert.ok(logs.includes("triage host: route refused at the binds: adapter"), logs.join(" | "));
|
||||
assert.equal(existsSync(join(root, "spawned")), false);
|
||||
assert.equal(h.running, false);
|
||||
for (const f of ["session.json", "relay.sock", "bundle"]) assert.equal(existsSync(join(agent, f)), false, f);
|
||||
const [owner, note, ...rest] = (await h.state()).transcript;
|
||||
assert.deepEqual([owner.from, note.from, note.in_reply_to, note.text, rest.length], ["owner", "host", owner.id, ROUTE_REFUSED, 0]);
|
||||
rmSync(join(root, "install", "adapter.sh"));
|
||||
writeFileSync(join(root, "install", "adapter.sh"), "exit 0\n");
|
||||
await h.message("again");
|
||||
await waitFor(() => existsSync(join(root, "spawned")));
|
||||
});
|
||||
|
||||
test("a read and a message that arrive together share one snapshot read", async (t) => {
|
||||
@@ -211,7 +508,8 @@ test("end to end under the sandbox: the first message starts the session, which
|
||||
// A link left where the host makes its work directory must not steer the bind.
|
||||
mkdirSync(join(dataRoot, "triage", "agent"), { recursive: true, mode: 0o700 });
|
||||
symlinkSync(join(dataRoot, "orgs"), join(dataRoot, "triage", "agent", "work"));
|
||||
const adapter = join(root, "adapter.sh");
|
||||
mkdirSync(join(root, "install"));
|
||||
const adapter = join(root, "install", "adapter.sh");
|
||||
writeFileSync(adapter, `exec '${process.execPath}' '${FIXTURE}'\n`, { mode: 0o644 });
|
||||
const probe = [
|
||||
join(dataRoot, "orgs", "alpha", "org.json"), join(root, "config", "config.json"), join(root, "secrets", "token"),
|
||||
@@ -219,7 +517,7 @@ test("end to end under the sandbox: the first message starts the session, which
|
||||
join(dataRoot, "triage", "health", "snapshot.json"), join(dataRoot, "triage", "agent", "work", "health", "snapshot.json"),
|
||||
];
|
||||
let admits = 0;
|
||||
const route = { admit: async () => { admits += 1; return { ok: true, adapter, harness: "fake", harnessEnv: { TRIAGE_PROBE: JSON.stringify(probe) }, harnessMounts: [], turnEnd: "exit", model: "stub", provider: "stub", env: { ANTHROPIC_BASE_URL: "http://127.0.0.1:9", OTHER: "x" } }; } };
|
||||
const route = { admit: async () => { admits += 1; return { ok: true, adapter, harness: "fake", harnessEnv: { TRIAGE_PROBE: JSON.stringify(probe) }, harnessMounts: [], turnEnd: "exit", model: "stub", provider: "stub", env: { ANTHROPIC_BASE_URL: "http://127.0.0.1:9" } }; } };
|
||||
const { h } = host(t, root, { route });
|
||||
assert.equal(h.running, false, "nothing starts before the owner's first message");
|
||||
assert.equal(existsSync(join(dataRoot, "triage", "agent", "session.json")), false);
|
||||
@@ -280,9 +578,10 @@ test("end to end under the sandbox: the first message starts the session, which
|
||||
|
||||
// A route answer that runs the stub adapter under the sandbox.
|
||||
function stubRoute(root) {
|
||||
const adapter = join(root, "adapter.sh");
|
||||
mkdirSync(join(root, "install"), { recursive: true });
|
||||
const adapter = join(root, "install", "adapter.sh");
|
||||
writeFileSync(adapter, `exec '${process.execPath}' '${FIXTURE}'\n`, { mode: 0o644 });
|
||||
return { admit: async () => ({ ok: true, adapter, harness: "fake", harnessEnv: {}, harnessMounts: [], turnEnd: "exit", model: "stub", provider: "stub", env: {} }) };
|
||||
return { admit: async () => ({ ok: true, adapter, harness: "fake", harnessEnv: {}, harnessMounts: [], turnEnd: "exit", model: "stub", provider: "stub", env: { ANTHROPIC_BASE_URL: "http://127.0.0.1:9" } }) };
|
||||
}
|
||||
const repliesOf = (h, n, from = "triage") => waitFor(async () => {
|
||||
const list = (await h.state()).transcript.filter((e) => e.from === from);
|
||||
@@ -326,9 +625,7 @@ test("a session that ends mid-turn: the message it took is answered by the host,
|
||||
|
||||
test("a launcher that cannot run is a failed start: the message is not delivered, nothing reports running, and the next message is refused the same way", async (t) => {
|
||||
const root = scratch();
|
||||
const adapter = join(root, "adapter.sh");
|
||||
writeFileSync(adapter, "exit 0\n");
|
||||
const route = { admit: async () => ({ ok: true, adapter, harness: "fake", harnessEnv: {}, harnessMounts: [], turnEnd: "exit", model: "m", provider: "p", env: {} }) };
|
||||
const route = { admit: async () => answer(root) };
|
||||
const { h, dataRoot } = host(t, root, { route, bwrap: join(root, "no-such-launcher") });
|
||||
for (const text of ["one", "two"]) {
|
||||
await assert.rejects(h.message(text), (e) => e.code === "triage-start-failed", text);
|
||||
@@ -350,6 +647,17 @@ test("a reply that names its message answers it: a session killed after replying
|
||||
assert.deepEqual((await h.state()).transcript.filter((e) => e.from === "host"), []);
|
||||
});
|
||||
|
||||
test("a progress note that names its message does not answer it: a session killed after the note leaves an unanswered line", { skip, timeout: 120000 }, async (t) => {
|
||||
const root = scratch();
|
||||
const { h } = host(t, root, { route: stubRoute(root) });
|
||||
const sent = await h.message("NOTE-SLEEP");
|
||||
const [noted] = await repliesOf(h, 1);
|
||||
assert.deepEqual([noted.text.replace(/\n\n.*$/s, ""), noted.in_reply_to], ["still looking", sent.id]);
|
||||
process.kill(-h.pid, "SIGKILL");
|
||||
const [note] = await repliesOf(h, 1, "host");
|
||||
assert.deepEqual([note.in_reply_to, note.text], [sent.id, UNANSWERED]);
|
||||
});
|
||||
|
||||
test("a message the session never read gets a host line when it ends and leaves the queue", { skip, timeout: 120000 }, async (t) => {
|
||||
const root = scratch();
|
||||
const { h, dataRoot } = host(t, root, { route: stubRoute(root) });
|
||||
@@ -370,3 +678,497 @@ test("with devOwner set, a card's bus start carries the dev-owner flag", { skip,
|
||||
const [reply] = await repliesOf(h, 1);
|
||||
assert.deepEqual(reply.cards, [{ step: "bus-start", org: "alpha", command: "mosaic bus start alpha --dev-local-owner --detach" }]);
|
||||
});
|
||||
|
||||
// #1873 triage-cap. The host counts each turn into <dataRoot>/triage/usage.jsonl and holds the stored
|
||||
// cap through daily-quota. The usage source here reads <run>/session/spend.jsonl, which the stub
|
||||
// adapter's SPEND writes (or a test writes by hand), one entry per spend.
|
||||
const spendSource = {
|
||||
readTurn: async ({ root }) => {
|
||||
let text;
|
||||
try { text = readFileSync(join(root, "spend.jsonl"), "utf8"); }
|
||||
catch (e) { if (e.code === "ENOENT") return { status: "known", entries: {} }; throw e; }
|
||||
const entries = {};
|
||||
for (const l of text.split("\n").filter(Boolean)) {
|
||||
const { id, n } = JSON.parse(l);
|
||||
entries[id] = { input: n, output: 0, cacheWrite: 0, cacheRead: 0 };
|
||||
}
|
||||
return { status: "known", entries };
|
||||
},
|
||||
};
|
||||
const DAY = "2026-11-18T00:00:00.000Z";
|
||||
// Store a cap the way the owner's verb does: the route store, its since at `since`.
|
||||
// The data root's triage directory, as the host's ensureDirs makes it (a test writes before any start).
|
||||
const triageDir = (dataRoot) => {
|
||||
mkdirSync(join(dataRoot, "triage"), { recursive: true, mode: 0o700 });
|
||||
chmodSync(dataRoot, 0o700);
|
||||
chmodSync(join(dataRoot, "triage"), 0o700);
|
||||
};
|
||||
const storeCap = (dataRoot, limit, timeZone = "UTC", since = DAY) => triageDir(dataRoot) ??
|
||||
writeRoute(dataRoot, { model: "glm-5", harness: "claude-code", limit, timeZone, by: "owner" }, { now: () => new Date(since) });
|
||||
const usageLine = (over = {}) => ({ at: "2026-11-18T10:00:00.000Z", run: "triage-20261118100000", turn: "owner-0123456789ab", status: "known", input: 10, output: 0, cacheRead: 0, cacheWrite: 0, ...over });
|
||||
const usageLines = (dataRoot) => readUsage(usagePath(dataRoot)).records;
|
||||
let spendId = 0;
|
||||
const spend = (dataRoot, n) => {
|
||||
mkdirSync(join(dataRoot, "triage", "agent", "session"), { recursive: true });
|
||||
appendFileSync(join(dataRoot, "triage", "agent", "session", "spend.jsonl"), JSON.stringify({ id: `s${++spendId}`, n }) + "\n");
|
||||
};
|
||||
const counting = (root) => { const r = { asked: 0, admit: async () => { r.asked++; return answer(root); } }; return r; };
|
||||
// A launcher that keeps the relay open and leaves the capability it was handed, so a test can make
|
||||
// the runner's calls itself without the sandbox.
|
||||
function standIn(root) {
|
||||
const f = join(root, "stand-in");
|
||||
writeFileSync(f, `#!/bin/sh\n: > '${join(root, "spawned")}'\ncat > '${join(root, "stdin.json")}'\nexec sleep 600\n`, { mode: 0o755 });
|
||||
return f;
|
||||
}
|
||||
async function runnerClient(root, dataRoot) {
|
||||
await waitFor(() => { try { return JSON.parse(readFileSync(join(root, "stdin.json"), "utf8")).cap; } catch { return null; } });
|
||||
const cap = JSON.parse(readFileSync(join(root, "stdin.json"), "utf8")).cap;
|
||||
return new Client({ path: join(dataRoot, "triage", "agent", "relay.sock"), cap });
|
||||
}
|
||||
|
||||
test("cap: a hit before a start refuses with the next reset, asks no route and spawns nothing; at 00:00 in the stored zone it starts", { timeout: 30000 }, async (t) => {
|
||||
const root = scratch();
|
||||
let clock = Date.parse("2026-11-18T12:00:00.000Z");
|
||||
const route = counting(root);
|
||||
const { h, dataRoot } = host(t, root, { route, bwrap: spy(root), usageSource: spendSource, now: () => new Date(clock) });
|
||||
storeCap(dataRoot, 100);
|
||||
appendUsage(usagePath(dataRoot), usageLine({ input: 60, output: 30, cacheRead: 5, cacheWrite: 5 }));
|
||||
await assert.rejects(h.message("hello"), (e) => e.code === "triage-over-cap" && e.until === "2026-11-19T00:00:00.000Z");
|
||||
assert.equal(route.asked, 0, "the route is not asked over the cap");
|
||||
assert.equal(existsSync(join(root, "spawned")), false);
|
||||
assert.deepEqual((await h.state()).transcript, [], "a refused message is never written");
|
||||
const s = await h.state();
|
||||
assert.deepEqual([s.overCapUntil, s.resetsAt], ["2026-11-19T00:00:00.000Z", "2026-11-19T00:00:00.000Z"]);
|
||||
assert.deepEqual(s.usageToday, { count: 100, input: 60, output: 30, cacheWrite: 5, cacheRead: 5, unknownTurns: 0 });
|
||||
assert.deepEqual(s.cap, { limit: 100, reset: "00:00", timeZone: "UTC" });
|
||||
clock = Date.parse("2026-11-18T23:59:59.999Z");
|
||||
await assert.rejects(h.message("hello"), (e) => e.code === "triage-over-cap");
|
||||
clock = Date.parse("2026-11-19T00:00:00.000Z");
|
||||
const s2 = await h.state();
|
||||
assert.deepEqual([s2.overCapUntil, s2.usageToday.count, s2.resetsAt], [null, 0, "2026-11-20T00:00:00.000Z"]);
|
||||
await h.message("hello");
|
||||
await waitFor(() => existsSync(join(root, "spawned")));
|
||||
assert.equal(route.asked, 1);
|
||||
});
|
||||
|
||||
test("cap: the reset is 00:00 in the stored zone on a DST day (America/Chicago, a 25-hour day)", { timeout: 30000 }, async (t) => {
|
||||
const root = scratch();
|
||||
let clock = Date.parse("2026-11-01T12:00:00.000Z");
|
||||
const route = counting(root);
|
||||
const { h, dataRoot } = host(t, root, { route, bwrap: spy(root), usageSource: spendSource, now: () => new Date(clock) });
|
||||
storeCap(dataRoot, 100, "America/Chicago", "2026-10-31T00:00:00.000Z");
|
||||
// 2026-11-01 in Chicago runs from 05:00Z (CDT) to 06:00Z the next day (CST).
|
||||
appendUsage(usagePath(dataRoot), usageLine({ at: "2026-11-01T05:00:00.000Z", input: 100 }));
|
||||
await assert.rejects(h.message("a"), (e) => e.code === "triage-over-cap" && e.until === "2026-11-02T06:00:00.000Z");
|
||||
clock = Date.parse("2026-11-02T05:59:59.999Z");
|
||||
await assert.rejects(h.message("b"), (e) => e.code === "triage-over-cap" && e.until === "2026-11-02T06:00:00.000Z");
|
||||
clock = Date.parse("2026-11-02T06:00:00.000Z");
|
||||
await h.message("c");
|
||||
await waitFor(() => existsSync(join(root, "spawned")));
|
||||
assert.equal(route.asked, 1);
|
||||
// A line from before the day began does not count: 04:59:59.999Z was still 2026-10-31 in Chicago.
|
||||
const other = scratch();
|
||||
const o = host(t, other, { route: counting(other), bwrap: spy(other), usageSource: spendSource, now: () => new Date("2026-11-01T12:00:00.000Z") });
|
||||
storeCap(o.dataRoot, 100, "America/Chicago", "2026-10-30T00:00:00.000Z");
|
||||
appendUsage(usagePath(o.dataRoot), usageLine({ at: "2026-11-01T04:59:59.999Z", input: 100 }));
|
||||
assert.equal((await o.h.state()).usageToday.count, 0);
|
||||
});
|
||||
|
||||
test("cap: the relay writes one line per turn, known or unknown, and a session that ends mid-turn still gets its line", { timeout: 30000 }, async (t) => {
|
||||
const root = scratch();
|
||||
const logs = [];
|
||||
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z"), log: (l) => logs.push(l) });
|
||||
const first = await h.message("one");
|
||||
const c = await runnerClient(root, dataRoot);
|
||||
const run = JSON.parse(readFileSync(join(dataRoot, "triage", "agent", "session.json"), "utf8")).run;
|
||||
assert.match(run, /^triage-\d{14}$/);
|
||||
spend(dataRoot, 7); // before the turn: the baseline holds it
|
||||
assert.deepEqual(await c.call("usage.begin", { turn: first.id }), {});
|
||||
spend(dataRoot, 40);
|
||||
// A repeated begin for the same turn keeps the first baseline, so the tokens before it still count.
|
||||
assert.deepEqual(await c.call("usage.begin", { turn: first.id }), {});
|
||||
spend(dataRoot, 2);
|
||||
await c.call("usage.end", { turn: first.id });
|
||||
// A repeated end does not count the turn twice.
|
||||
await c.call("usage.end", { turn: first.id });
|
||||
await c.call("usage.begin", { turn: `${first.id}:retry` });
|
||||
await c.call("usage.end", { turn: `${first.id}:retry` }); // nothing new: unknown
|
||||
const lines = usageLines(dataRoot);
|
||||
assert.deepEqual(lines.map((l) => [l.run, l.turn, l.status, l.input, l.output, l.cacheRead, l.cacheWrite]), [
|
||||
[run, first.id, "known", 42, 0, 0, 0],
|
||||
[run, `${first.id}:retry`, "unknown", null, null, null, null],
|
||||
]);
|
||||
assert.ok(lines.every((l) => l.at === "2026-11-18T12:00:00.000Z"), "the line's time is the host's clock");
|
||||
assert.ok(logs.includes("triage host: usage.end repeated for a turn; not counted twice"));
|
||||
assert.ok(logs.includes("triage host: turn usage unknown: no-new-usage"));
|
||||
assert.equal(lstatSync(usagePath(dataRoot)).mode & 0o777, 0o600);
|
||||
// A turn begun and never ended: the session is killed; the host writes its line.
|
||||
await c.call("usage.begin", { turn: "owner-0123456789ff" });
|
||||
spend(dataRoot, 5);
|
||||
process.kill(-h.pid, "SIGKILL");
|
||||
await waitFor(() => !h.running);
|
||||
await h.stop();
|
||||
assert.deepEqual(usageLines(dataRoot).slice(2).map((l) => [l.turn, l.status, l.input]), [["owner-0123456789ff", "known", 5]]);
|
||||
assert.equal(usageLines(dataRoot).length, 3, "the killed turn has one line, not two");
|
||||
});
|
||||
|
||||
test("cap: a turn that crosses the cap finishes; the runner is then refused and every queued message gets a host line", { timeout: 30000 }, async (t) => {
|
||||
const root = scratch();
|
||||
const logs = [];
|
||||
const route = counting(root);
|
||||
const { h, dataRoot } = host(t, root, { route, bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z"), log: (l) => logs.push(l) });
|
||||
storeCap(dataRoot, 100);
|
||||
const first = await h.message("one");
|
||||
const c = await runnerClient(root, dataRoot);
|
||||
assert.deepEqual((await c.call("message.receive")).map((m) => m.id), [first.id]);
|
||||
await c.call("state");
|
||||
await c.call("usage.begin", { turn: first.id });
|
||||
// Queued while the turn runs, under the cap: admitted.
|
||||
const second = await h.message("two");
|
||||
const third = await h.message("three");
|
||||
spend(dataRoot, 150);
|
||||
// Inside the turn (a continuation's state read) the runner is not stopped.
|
||||
assert.equal((await c.call("state")).tasks.total, 0);
|
||||
await c.call("usage.end", { turn: first.id });
|
||||
await c.call("message.send", { to: "human", body: "answer", class: "RESULT", in_reply_to: first.id });
|
||||
await c.call("message.read", { id: first.id });
|
||||
// The next receive is refused: the session is gone, the queue drains with a host line each.
|
||||
await assert.rejects(c.call("message.receive"), (e) => e.code === "not-holder");
|
||||
const tr = (await h.state()).transcript;
|
||||
assert.deepEqual(tr.filter((e) => e.from === "triage").map((e) => [e.in_reply_to, e.text]), [[first.id, "answer"]]);
|
||||
const until = "2026-11-19T00:00:00.000Z";
|
||||
assert.deepEqual(tr.filter((e) => e.from === "host").map((e) => [e.in_reply_to, e.text]), [[second.id, overCapNotTaken(until)], [third.id, overCapNotTaken(until)]]);
|
||||
assert.ok(logs.includes("triage host: stopping the session at message.receive: over the daily token cap"), logs.join(" | "));
|
||||
await assert.rejects(h.message("four"), (e) => e.code === "triage-over-cap" && e.until === until);
|
||||
assert.equal(route.asked, 3);
|
||||
// The session ends: nothing is left to answer, so no second line.
|
||||
process.kill(-h.pid, "SIGKILL");
|
||||
await waitFor(() => !h.running);
|
||||
assert.equal((await h.state()).transcript.filter((e) => e.from === "host").length, 2);
|
||||
});
|
||||
|
||||
test("cap: messages taken together: once a turn crosses the cap, the next message's turn does not start", { timeout: 30000 }, async (t) => {
|
||||
const root = scratch();
|
||||
const logs = [];
|
||||
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z"), log: (l) => logs.push(l) });
|
||||
storeCap(dataRoot, 100);
|
||||
const [a, b] = await Promise.all([h.message("one"), h.message("two")]);
|
||||
const c = await runnerClient(root, dataRoot);
|
||||
assert.deepEqual((await c.call("message.receive")).map((m) => m.id), [a.id, b.id]);
|
||||
await c.call("state");
|
||||
await c.call("usage.begin", { turn: a.id });
|
||||
spend(dataRoot, 100);
|
||||
await c.call("usage.end", { turn: a.id });
|
||||
await c.call("message.send", { to: "human", body: "answer", class: "RESULT", in_reply_to: a.id });
|
||||
await c.call("message.read", { id: a.id });
|
||||
// The runner reads the state before each message's turn: refused, so b's turn never begins.
|
||||
await assert.rejects(c.call("state"), (e) => e.code === "not-holder");
|
||||
const hostLines = (await h.state()).transcript.filter((e) => e.from === "host");
|
||||
assert.deepEqual(hostLines.map((e) => [e.in_reply_to, e.text]), [[b.id, overCapNotTaken("2026-11-19T00:00:00.000Z")]]);
|
||||
assert.ok(logs.includes("triage host: stopping the session at state: over the daily token cap"), logs.join(" | "));
|
||||
assert.equal(usageLines(dataRoot).length, 1);
|
||||
});
|
||||
|
||||
test("cap: a restart counts what the file holds: a new host on the same data root refuses at the same count", { timeout: 30000 }, async (t) => {
|
||||
const root = scratch();
|
||||
const first = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
|
||||
storeCap(first.dataRoot, 100);
|
||||
const m = await first.h.message("one");
|
||||
const c = await runnerClient(root, first.dataRoot);
|
||||
await c.call("usage.begin", { turn: m.id });
|
||||
spend(first.dataRoot, 60);
|
||||
await c.call("usage.end", { turn: m.id });
|
||||
// The second turn is open when the host stops: its line is written on the way down.
|
||||
await c.call("usage.begin", { turn: "owner-0123456789ee" });
|
||||
spend(first.dataRoot, 45);
|
||||
await first.h.stop();
|
||||
assert.deepEqual(usageLines(first.dataRoot).map((l) => [l.turn, l.input]), [[m.id, 60], ["owner-0123456789ee", 45]]);
|
||||
const route = counting(root);
|
||||
const second = host(t, root, { route, bwrap: spy(root), usageSource: spendSource, now: () => new Date("2026-11-18T13:00:00.000Z") });
|
||||
assert.equal((await second.h.state()).usageToday.count, 105);
|
||||
await assert.rejects(second.h.message("two"), (e) => e.code === "triage-over-cap" && e.until === "2026-11-19T00:00:00.000Z");
|
||||
assert.equal(route.asked, 0);
|
||||
});
|
||||
|
||||
test("cap: a message whose turn began and was not answered gets the unanswered line when the cap stops the session", { timeout: 30000 }, async (t) => {
|
||||
const root = scratch();
|
||||
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
|
||||
storeCap(dataRoot, 100);
|
||||
const [a, b] = await Promise.all([h.message("one"), h.message("two")]);
|
||||
const c = await runnerClient(root, dataRoot);
|
||||
await c.call("message.receive");
|
||||
await c.call("usage.begin", { turn: a.id });
|
||||
spend(dataRoot, 100);
|
||||
await c.call("usage.end", { turn: a.id });
|
||||
// The turn's answer is lost (a halted processing state leaves a message unread): a stays queued.
|
||||
await assert.rejects(c.call("state"), (e) => e.code === "not-holder");
|
||||
const lines = (await h.state()).transcript.filter((e) => e.from === "host").map((e) => [e.in_reply_to, e.text]);
|
||||
assert.deepEqual(lines, [[a.id, UNANSWERED], [b.id, overCapNotTaken("2026-11-19T00:00:00.000Z")]]);
|
||||
});
|
||||
|
||||
test("cap: under the sandbox, the runner's turns are counted, a turn that crosses the cap answers, the queued message drains, and a restart counts the file", { skip, timeout: 120000 }, async (t) => {
|
||||
const root = scratch();
|
||||
const logs = [];
|
||||
const route = stubRoute(root);
|
||||
const clock = () => new Date("2026-11-18T12:00:00.000Z");
|
||||
const { h, dataRoot } = host(t, root, { route, usageSource: spendSource, now: clock, log: (l) => logs.push(l) });
|
||||
storeCap(dataRoot, 1000);
|
||||
const one = await h.message("SPEND 300");
|
||||
const echo = await h.message("plain");
|
||||
const replies = await repliesOf(h, 2);
|
||||
assert.deepEqual(replies.map((e) => [e.text.replace(/\n\n.*$/s, ""), e.in_reply_to]), [["spent 300", one.id], ["echo: plain", echo.id]]);
|
||||
const lines = usageLines(dataRoot);
|
||||
assert.deepEqual(lines.map((l) => [l.turn, l.status, l.input]), [[one.id, "known", 300], [echo.id, "unknown", null]]);
|
||||
assert.match(lines[0].run, /^triage-\d{14}$/);
|
||||
await h.stop();
|
||||
// A restart: a new host on the same data root counts the file.
|
||||
const again = host(t, root, { route, usageSource: spendSource, now: clock, log: (l) => logs.push(l) });
|
||||
const s = await again.h.state();
|
||||
assert.deepEqual([s.usageToday.count, s.usageToday.unknownTurns], [300, 1]);
|
||||
const big = await again.h.message("GATED-SPEND 800");
|
||||
await waitFor(() => existsSync(join(dataRoot, "triage", "agent", "work", "started")));
|
||||
const queued = await again.h.message("queued");
|
||||
writeFileSync(join(dataRoot, "triage", "agent", "work", "gate"), "x");
|
||||
const [answer] = await repliesOf(again.h, 3).then((l) => l.slice(2));
|
||||
assert.deepEqual([answer.text.replace(/\n\n.*$/s, ""), answer.in_reply_to], ["spent 800", big.id]);
|
||||
const [note] = await repliesOf(again.h, 1, "host");
|
||||
assert.deepEqual([note.in_reply_to, note.text], [queued.id, overCapNotTaken("2026-11-19T00:00:00.000Z")]);
|
||||
await waitFor(() => !again.h.running);
|
||||
assert.equal((await again.h.state()).lastExit.code, 22, "the runner ends as a session that is gone");
|
||||
assert.equal((await again.h.state()).transcript.filter((e) => e.from === "host").length, 1);
|
||||
await assert.rejects(again.h.message("after"), (e) => e.code === "triage-over-cap");
|
||||
assert.equal(usageLines(dataRoot).at(-1).input, 800);
|
||||
});
|
||||
|
||||
test("cap: a hit from another count before a queued message: the file is read again at each check", { timeout: 30000 }, async (t) => {
|
||||
const root = scratch();
|
||||
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
|
||||
storeCap(dataRoot, 1000);
|
||||
const first = await h.message("one");
|
||||
const c = await runnerClient(root, dataRoot);
|
||||
await c.call("message.receive");
|
||||
const second = await h.message("two");
|
||||
// Under the cap the runner takes the queued message.
|
||||
assert.deepEqual((await c.call("message.receive")).map((m) => m.id), [second.id]);
|
||||
await c.call("usage.begin", { turn: second.id });
|
||||
const third = await h.message("three");
|
||||
appendUsage(usagePath(dataRoot), usageLine({ input: 1000 }));
|
||||
// Over the cap inside second's turn: a continuation's state read is not stopped; the turn finishes.
|
||||
assert.equal((await c.call("state")).tasks.total, 0);
|
||||
spend(dataRoot, 4);
|
||||
await c.call("usage.end", { turn: second.id });
|
||||
await c.call("message.send", { to: "human", body: "answer", class: "RESULT", in_reply_to: second.id });
|
||||
await c.call("message.read", { id: second.id });
|
||||
await assert.rejects(c.call("message.receive"), (e) => e.code === "not-holder");
|
||||
const until = overCapNotTaken("2026-11-19T00:00:00.000Z");
|
||||
const hostLines = (await h.state()).transcript.filter((e) => e.from === "host").map((e) => [e.in_reply_to, e.text]);
|
||||
assert.deepEqual(hostLines.sort(), [[first.id, until], [third.id, until]].sort(), "every message still queued leaves it");
|
||||
assert.deepEqual(usageLines(dataRoot).at(-1).input, 4, "the finished turn is counted");
|
||||
});
|
||||
|
||||
test("cap: an idle session over the cap ends at its next receive, with nothing queued", { timeout: 30000 }, async (t) => {
|
||||
const root = scratch();
|
||||
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
|
||||
storeCap(dataRoot, 1000);
|
||||
const m = await h.message("one");
|
||||
const c = await runnerClient(root, dataRoot);
|
||||
await c.call("message.receive");
|
||||
await c.call("message.send", { to: "human", body: "answer", class: "RESULT", in_reply_to: m.id });
|
||||
await c.call("message.read", { id: m.id });
|
||||
appendUsage(usagePath(dataRoot), usageLine({ input: 1000 }));
|
||||
await assert.rejects(c.call("message.receive"), (e) => e.code === "not-holder");
|
||||
assert.equal((await h.state()).transcript.filter((e) => e.from === "host").length, 0, "nothing was queued, so no host line");
|
||||
});
|
||||
|
||||
test("cap: the count waits for a turn still being counted: usage.end answers after its line, and a message right after a mid-turn kill is refused", { timeout: 30000 }, async (t) => {
|
||||
const root = scratch();
|
||||
let gate = null; // while set, the usage source waits on it
|
||||
const close = () => { let release; const promise = new Promise((r) => { release = r; }); gate = { promise, release: () => { gate = null; release(); } }; return gate; };
|
||||
const gated = { readTurn: async (a) => { if (gate) await gate.promise; return spendSource.readTurn(a); } };
|
||||
const route = counting(root);
|
||||
// A failed assertion must not leave the gate shut: the host's stop waits for the usage chain. After hooks
|
||||
// run in the order they were added, so this one goes before the host's stop.
|
||||
t.after(() => gate?.release());
|
||||
const { h, dataRoot } = host(t, root, { route, bwrap: standIn(root), usageSource: gated, now: () => new Date("2026-11-18T12:00:00.000Z") });
|
||||
storeCap(dataRoot, 100);
|
||||
const m = await h.message("one");
|
||||
const c = await runnerClient(root, dataRoot);
|
||||
await c.call("message.receive");
|
||||
await c.call("usage.begin", { turn: m.id });
|
||||
spend(dataRoot, 60);
|
||||
let g = close();
|
||||
let answered = false;
|
||||
const end = c.call("usage.end", { turn: m.id }).then(() => { answered = true; });
|
||||
await new Promise((r) => setTimeout(r, 200));
|
||||
assert.equal(answered, false, "usage.end is not answered before its line is written");
|
||||
assert.equal(usageLines(dataRoot).length, 0);
|
||||
g.release();
|
||||
await end;
|
||||
assert.deepEqual(usageLines(dataRoot).map((l) => l.input), [60]);
|
||||
await c.call("message.send", { to: "human", body: "answer", class: "RESULT", in_reply_to: m.id });
|
||||
await c.call("message.read", { id: m.id });
|
||||
// A second turn crosses the cap and the session is killed before its usage.end.
|
||||
const two = await h.message("two");
|
||||
assert.deepEqual((await c.call("message.receive")).map((x) => x.id), [two.id]);
|
||||
await c.call("usage.begin", { turn: two.id });
|
||||
spend(dataRoot, 50);
|
||||
g = close();
|
||||
process.kill(-h.pid, "SIGKILL");
|
||||
await waitFor(() => !h.running);
|
||||
const asked = route.asked;
|
||||
setTimeout(() => g.release(), 200);
|
||||
await assert.rejects(h.message("three"), (e) => e.code === "triage-over-cap" && e.until === "2026-11-19T00:00:00.000Z");
|
||||
assert.equal(route.asked, asked, "the route is not asked: the open turn was counted first");
|
||||
assert.deepEqual(usageLines(dataRoot).map((l) => [l.turn, l.input]), [[m.id, 60], [two.id, 50]]);
|
||||
});
|
||||
|
||||
test("cap: a harness transcript path outside the run directory reads as no transcript, so the turn's line is unknown", { timeout: 30000 }, async (t) => {
|
||||
for (const label of ["a sibling of the run directory", "a path through .. from inside it"]) {
|
||||
const root = scratch();
|
||||
const logs = [];
|
||||
let target = null;
|
||||
const mod = async (name) => name === "fake" ? { envNames: DECLARED, installRoots: () => [join(root, "install")], transcript: ({ runDir }) => (target = label.startsWith("a sibling") ? join(runDir, "..", "outside") : `${runDir}/session/../../outside`) } : null;
|
||||
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, harnessModule: mod, now: () => new Date("2026-11-18T12:00:00.000Z"), log: (l) => logs.push(l) });
|
||||
const m = await h.message("one");
|
||||
const c = await runnerClient(root, dataRoot);
|
||||
mkdirSync(target, { recursive: true });
|
||||
await c.call("usage.begin", { turn: m.id });
|
||||
appendFileSync(join(target, "spend.jsonl"), JSON.stringify({ id: "x1", n: 5 }) + "\n");
|
||||
await c.call("usage.end", { turn: m.id });
|
||||
assert.deepEqual(usageLines(dataRoot).map((l) => [l.status, l.input]), [["unknown", null]], label);
|
||||
assert.ok(logs.includes("triage host: turn usage unknown: baseline-unknown"), label);
|
||||
}
|
||||
});
|
||||
|
||||
test("cap: a torn last line is refused, logged once, and counts as an unknown turn; the next line stands alone", { timeout: 30000 }, async (t) => {
|
||||
const root = scratch();
|
||||
const logs = [];
|
||||
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z"), log: (l) => logs.push(l) });
|
||||
storeCap(dataRoot, 1000);
|
||||
appendUsage(usagePath(dataRoot), usageLine({ input: 30 }));
|
||||
appendFileSync(usagePath(dataRoot), '{"at":"2026-11-18T11:00:00.000Z","run":"triage-2026111');
|
||||
const s = await h.state();
|
||||
assert.deepEqual([s.usageToday.count, s.usageToday.unknownTurns], [30, 1]);
|
||||
await h.state();
|
||||
assert.deepEqual(logs.filter((l) => /usage\.jsonl line/.test(l)), ["triage host: usage.jsonl line 2 is not a usage line; it counts as an unknown turn"]);
|
||||
// Under the cap, a message starts, and its turn's line is written after the torn one.
|
||||
const m = await h.message("one");
|
||||
const c = await runnerClient(root, dataRoot);
|
||||
await c.call("usage.begin", { turn: m.id });
|
||||
spend(dataRoot, 20);
|
||||
await c.call("usage.end", { turn: m.id });
|
||||
const lines = usageLines(dataRoot);
|
||||
assert.deepEqual(lines.map((l) => [l.turn, l.status, l.input]), [["owner-0123456789ab", "known", 30], ["line-2", "unknown", null], [m.id, "known", 20]]);
|
||||
const s2 = await h.state();
|
||||
assert.deepEqual([s2.usageToday.count, s2.usageToday.unknownTurns], [50, 1]);
|
||||
});
|
||||
|
||||
test("cap: a usage file with a wrong mode, a link or a non-file refuses every message and reads as not read", { timeout: 30000 }, async (t) => {
|
||||
const cases = [
|
||||
["mode 0644", (f) => { appendUsage(f, usageLine()); chmodSync(f, 0o644); }],
|
||||
["mode 0400", (f) => { appendUsage(f, usageLine()); chmodSync(f, 0o400); }],
|
||||
["a link", (f) => { writeFileSync(f + ".real", JSON.stringify(usageLine()) + "\n", { mode: 0o600 }); symlinkSync(f + ".real", f); }],
|
||||
["a directory", (f) => mkdirSync(f)],
|
||||
];
|
||||
for (const [label, make] of cases) {
|
||||
const root = scratch();
|
||||
const route = counting(root);
|
||||
const { h, dataRoot } = host(t, root, { route, bwrap: spy(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
|
||||
storeCap(dataRoot, 1000);
|
||||
make(usagePath(dataRoot));
|
||||
await assert.rejects(h.message("hello"), (e) => e.code === "triage-usage-unreadable" && e.message === "Triage's token usage could not be read", label);
|
||||
assert.equal(route.asked, 0, label);
|
||||
assert.equal(existsSync(join(root, "spawned")), false, label);
|
||||
const s = await h.state();
|
||||
assert.deepEqual([s.usageToday, s.cap, s.resetsAt], ["not read", { limit: 1000, reset: "00:00", timeZone: "UTC" }, "2026-11-19T00:00:00.000Z"], label);
|
||||
}
|
||||
});
|
||||
|
||||
test("cap: a line the host cannot write leaves the count short, so the stored cap refuses from then on", { timeout: 30000 }, async (t) => {
|
||||
const root = scratch();
|
||||
const logs = [];
|
||||
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z"), log: (l) => logs.push(l) });
|
||||
storeCap(dataRoot, 1000);
|
||||
const m = await h.message("one");
|
||||
const c = await runnerClient(root, dataRoot);
|
||||
await c.call("message.receive");
|
||||
await c.call("usage.begin", { turn: m.id });
|
||||
spend(dataRoot, 5);
|
||||
appendUsage(usagePath(dataRoot), usageLine({ input: 1 }));
|
||||
chmodSync(usagePath(dataRoot), 0o644);
|
||||
await c.call("usage.end", { turn: m.id });
|
||||
assert.ok(logs.includes("triage host: usage line not written: triage-usage-unreadable"), logs.join(" | "));
|
||||
chmodSync(usagePath(dataRoot), 0o600);
|
||||
assert.equal(usageLines(dataRoot).length, 1, "the file is as it was");
|
||||
// The file reads again, but a turn is missing from it: refused, and the state says not read.
|
||||
await assert.rejects(h.message("two"), (e) => e.code === "triage-usage-unreadable");
|
||||
assert.equal((await h.state()).usageToday, "not read");
|
||||
await c.call("message.send", { to: "human", body: "answer", class: "RESULT", in_reply_to: m.id });
|
||||
const later = await h.state();
|
||||
assert.equal(later.transcript.filter((e) => e.from === "host").length, 0);
|
||||
});
|
||||
|
||||
test("cap: with no route stored the fields say not set, nothing is refused for usage, and each turn's line is still written", { timeout: 30000 }, async (t) => {
|
||||
const root = scratch();
|
||||
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
|
||||
const s = await h.state();
|
||||
assert.deepEqual([s.usageToday, s.cap, s.resetsAt, s.overCapUntil], ["not set", "not set", "not set", null]);
|
||||
triageDir(dataRoot);
|
||||
// A usage file over any count, even a broken one, refuses nothing while no cap is stored.
|
||||
appendUsage(usagePath(dataRoot), usageLine({ input: 10 ** 9 }));
|
||||
appendFileSync(usagePath(dataRoot), "not json");
|
||||
const m = await h.message("one");
|
||||
const c = await runnerClient(root, dataRoot);
|
||||
assert.deepEqual((await c.call("message.receive")).map((x) => x.id), [m.id]);
|
||||
await c.call("state");
|
||||
await c.call("usage.begin", { turn: m.id });
|
||||
spend(dataRoot, 3);
|
||||
await c.call("usage.end", { turn: m.id });
|
||||
assert.deepEqual(usageLines(dataRoot).at(-1).input, 3);
|
||||
const two = await h.message("two");
|
||||
assert.deepEqual((await c.call("message.receive")).map((x) => x.id), [two.id]);
|
||||
});
|
||||
|
||||
test("cap: an unreadable route store refuses every message and reads as not read", { timeout: 30000 }, async (t) => {
|
||||
const root = scratch();
|
||||
const route = counting(root);
|
||||
const { h, dataRoot } = host(t, root, { route, bwrap: spy(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
|
||||
storeCap(dataRoot, 1000);
|
||||
chmodSync(join(dataRoot, "triage", "route.json"), 0o644);
|
||||
await assert.rejects(h.message("hello"), (e) => e.code === "triage-route-unreadable" && e.message === "the model route could not be read");
|
||||
assert.equal(route.asked, 0);
|
||||
const s = await h.state();
|
||||
assert.deepEqual([s.usageToday, s.cap, s.resetsAt], ["not read", "not read", "not read"]);
|
||||
// A running session: the store turns unreadable, the next receive is refused and the queue drains.
|
||||
const other = scratch();
|
||||
const r = host(t, other, { route: counting(other), bwrap: standIn(other), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
|
||||
storeCap(r.dataRoot, 1000);
|
||||
const m = await r.h.message("one");
|
||||
const c = await runnerClient(other, r.dataRoot);
|
||||
await c.call("message.receive");
|
||||
const two = await r.h.message("two");
|
||||
chmodSync(join(r.dataRoot, "triage", "route.json"), 0o644);
|
||||
await assert.rejects(c.call("message.receive"), (e) => e.code === "not-holder");
|
||||
const hostLines = (await r.h.state()).transcript.filter((e) => e.from === "host").map((e) => [e.in_reply_to, e.text]);
|
||||
assert.deepEqual(hostLines.sort(), [[m.id, USAGE_NOT_READ], [two.id, USAGE_NOT_READ]].sort());
|
||||
});
|
||||
|
||||
test("cap: the read carries usageToday, cap and resetsAt, with and without a stored route, and no path, key or env value", { timeout: 30000 }, async (t) => {
|
||||
const root = scratch();
|
||||
const env = { ...CONSOLE_ENV, ANTHROPIC_AUTH_TOKEN: "tok-secret-value" };
|
||||
const route = { admit: async () => answer(root, { env: { ANTHROPIC_BASE_URL: "http://127.0.0.1:9", ANTHROPIC_AUTH_TOKEN: "tok-secret-value" } }) };
|
||||
const { h, dataRoot } = host(t, root, { route, bwrap: spy(root), env, usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
|
||||
const unset = await triageView(h, { send: true });
|
||||
assert.deepEqual([unset.usageToday, unset.cap, unset.resetsAt], ["not set", "not set", "not set"]);
|
||||
storeCap(dataRoot, 500, "Asia/Kolkata", "2026-11-17T00:00:00.000Z");
|
||||
appendUsage(usagePath(dataRoot), usageLine({ at: "2026-11-18T11:00:00.000Z", input: 120, output: 3 }));
|
||||
await h.message("hello");
|
||||
const view = await triageView(h, { send: true });
|
||||
assert.deepEqual(view.usageToday, { count: 123, input: 120, output: 3, cacheWrite: 0, cacheRead: 0, unknownTurns: 0 });
|
||||
assert.deepEqual(view.cap, { limit: 500, reset: "00:00", timeZone: "Asia/Kolkata" });
|
||||
assert.equal(view.resetsAt, "2026-11-18T18:30:00.000Z");
|
||||
const text = JSON.stringify([unset, view]);
|
||||
for (const bad of [dataRoot, root, "usage.jsonl", "route.json", "glm-5", "claude-code", "tok-secret-value", "from-the-console", "since", "2026-11-17T00:00:00.000Z", "triage-2026"]) {
|
||||
assert.equal(text.includes(bad), false, bad);
|
||||
}
|
||||
});
|
||||
@@ -4,7 +4,7 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { request } from 'node:http';
|
||||
import { mkdtempSync, existsSync, readFileSync } from 'node:fs';
|
||||
import { mkdtempSync, mkdirSync, existsSync, readFileSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { startServer } from '../src/serve.mjs';
|
||||
import { createDevOwner } from '../src/dev-owner.mjs';
|
||||
@@ -12,6 +12,8 @@ import { consoleAccess } from '../src/access.mjs';
|
||||
import { createSsoSession } from './helpers/sso-session.mjs';
|
||||
import { NO_ROUTE_MESSAGE, capMessage, triageView, triageHTTP, SNAPSHOT_MAX_AGE_MS } from '../src/triage-http.mjs';
|
||||
import { createTriageHost, TriageError } from '../src/triage-host.mjs';
|
||||
import { writeRoute } from '../src/triage-route-store.mjs';
|
||||
import { appendUsage, usagePath } from '../src/triage-usage.mjs';
|
||||
|
||||
const LEAK = '/home/owner/.config/mosaic-dev/secret-token-0123456789';
|
||||
|
||||
@@ -239,6 +241,21 @@ test('with the real host and no route: the read says so in plain words, the writ
|
||||
assert.equal(JSON.parse(readFileSync(join(host.paths.health, 'snapshot.json'), 'utf8')).snapshotVersion, 1, 'the read wrote the snapshot');
|
||||
});
|
||||
|
||||
test('triage-cap: the read carries usageToday, cap and resetsAt from the real host, and no path, model or harness', async t => {
|
||||
const dataRoot = join(mkdtempSync(join(process.env.TMPDIR, 'th-')), 'data');
|
||||
mkdirSync(join(dataRoot, 'triage'), { recursive: true, mode: 0o700 });
|
||||
writeRoute(dataRoot, { model: 'glm-5', harness: 'claude-code', limit: 1000, timeZone: 'UTC', by: 'owner' }, { now: () => new Date('2026-11-18T00:00:00.000Z') });
|
||||
appendUsage(usagePath(dataRoot), { at: '2026-11-18T09:00:00.000Z', run: 'triage-20261118090000', turn: 'owner-0123456789ab', status: 'known', input: 30, output: 5, cacheRead: 2, cacheWrite: 1 });
|
||||
const host = createTriageHost({ dataRoot, readSnapshot: async () => null, now: () => new Date('2026-11-18T10:00:00.000Z'), bwrap: '/nonexistent/bwrap' });
|
||||
const f = await devServer(t, { triage: host });
|
||||
const read = await send(f.port, '/api/triage', { headers: { host: f.headers.host } });
|
||||
assert.equal(read.status, 200);
|
||||
assert.deepEqual(read.json.usageToday, { count: 38, input: 30, output: 5, cacheWrite: 1, cacheRead: 2, unknownTurns: 0 });
|
||||
assert.deepEqual(read.json.cap, { limit: 1000, reset: '00:00', timeZone: 'UTC' });
|
||||
assert.equal(read.json.resetsAt, '2026-11-19T00:00:00.000Z');
|
||||
for (const leak of [dataRoot, 'glm-5', 'claude-code', 'usage.jsonl', '"since"']) assert.equal(read.text.includes(leak), false, leak);
|
||||
});
|
||||
|
||||
test('the route module keeps its own write checks when called alone: --send, one exact Origin, one same-origin Fetch Metadata', async () => {
|
||||
const triage = fakeTriage();
|
||||
const call = async (headers, { send = true } = {}) => {
|
||||
|
||||
@@ -0,0 +1,424 @@
|
||||
// #1873 triage-route-store: the owner-only file that holds Triage's model route and daily cap,
|
||||
// <dataRoot>/triage/route.json (design report section 1, option C). Names only: no env value and
|
||||
// no key. The directory is 0700 and the file 0600, both the running user's; a link, a non-file,
|
||||
// a wrong mode or owner refuses. A write is compare-and-set on revision, atomic (temp file plus
|
||||
// rename in the same directory), and appends one receipt line.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { spawn } from "node:child_process";
|
||||
import { chmodSync, existsSync, linkSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, renameSync, rmSync, symlinkSync, unlinkSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import {
|
||||
CAP_FIELDS, ROUTE_FIELDS, ROUTE_VERSION, RouteStoreError, capFor, readRoute, routePaths, validRoute, writeRoute,
|
||||
} from "../src/triage-route-store.mjs";
|
||||
|
||||
const MODULE = new URL("../src/triage-route-store.mjs", import.meta.url).pathname;
|
||||
const NOW = "2026-11-18T12:00:00.000Z";
|
||||
const LATER = "2026-11-18T13:00:00.000Z";
|
||||
const at = (iso) => () => new Date(iso);
|
||||
const INPUT = { model: "glm-5", harness: "claude-code", limit: 200000, timeZone: "America/Chicago", by: "owner" };
|
||||
|
||||
function root(t) {
|
||||
const dir = mkdtempSync(join(process.env.TMPDIR, "trs-"));
|
||||
t.after(() => rmSync(dir, { recursive: true, force: true }));
|
||||
return dir;
|
||||
}
|
||||
const code = (c) => (e) => e instanceof RouteStoreError && e.code === c;
|
||||
// Unreadable with a given reason, so each check is shown to refuse on its own.
|
||||
const unreadableFor = (re) => (e) => code("triage-route-unreadable")(e) && re.test(e.message);
|
||||
const receipts = (dataRoot) => readFileSync(routePaths(dataRoot).receipts, "utf8").split("\n").filter(Boolean).map((l) => JSON.parse(l));
|
||||
const good = () => ({
|
||||
version: 1, revision: 1, model: "glm-5", harness: "claude-code",
|
||||
cap: { limit: 200000, since: NOW, reset: "00:00", timeZone: "America/Chicago" }, updatedAt: NOW, updatedBy: "owner",
|
||||
});
|
||||
// Write a doc straight to route.json at the right modes, to test the loader alone.
|
||||
function plant(dataRoot, doc) {
|
||||
const { dir, file } = routePaths(dataRoot);
|
||||
mkdirSync(dir, { recursive: true, mode: 0o700 });
|
||||
chmodSync(dir, 0o700);
|
||||
writeFileSync(file, typeof doc === "string" ? doc : JSON.stringify(doc), { mode: 0o600 });
|
||||
chmodSync(file, 0o600);
|
||||
}
|
||||
|
||||
test("paths, field names and the reset rule each live in one place", () => {
|
||||
const p = routePaths("/d");
|
||||
assert.deepEqual(p, { dir: "/d/triage", file: "/d/triage/route.json", receipts: "/d/triage/route-receipts.jsonl", lock: "/d/triage/route.lock" });
|
||||
assert.deepEqual([...ROUTE_FIELDS].sort(), ["cap", "harness", "model", "revision", "updatedAt", "updatedBy", "version"]);
|
||||
assert.deepEqual([...CAP_FIELDS].sort(), ["limit", "reset", "since", "timeZone"]);
|
||||
assert.equal(ROUTE_VERSION, 1);
|
||||
// A new cap starts its count now and resets at 00:00 in the owner's zone.
|
||||
assert.deepEqual(capFor({ limit: 5, timeZone: "UTC", previous: null, now: NOW }), { limit: 5, since: NOW, reset: "00:00", timeZone: "UTC" });
|
||||
// A later cap keeps its since, changed or not, as an organization's token limits do: lowering
|
||||
// the cap never drops the tokens already counted.
|
||||
const prev = { limit: 5, since: NOW, reset: "00:00", timeZone: "UTC" };
|
||||
assert.equal(capFor({ limit: 5, timeZone: "UTC", previous: prev, now: LATER }).since, NOW);
|
||||
assert.equal(capFor({ limit: 2, timeZone: "UTC", previous: prev, now: LATER }).since, NOW);
|
||||
assert.equal(capFor({ limit: 5, timeZone: "Europe/Berlin", previous: prev, now: LATER }).since, NOW);
|
||||
});
|
||||
|
||||
test("no file reads as not set; a write then reads back the same route", (t) => {
|
||||
const d = root(t);
|
||||
assert.equal(readRoute(d), null);
|
||||
const written = writeRoute(d, INPUT, { now: at(NOW) });
|
||||
assert.deepEqual(written, good());
|
||||
assert.deepEqual(readRoute(d), good());
|
||||
const { dir, file } = routePaths(d);
|
||||
assert.equal(lstatSync(dir).mode & 0o777, 0o700);
|
||||
assert.equal(lstatSync(file).mode & 0o777, 0o600);
|
||||
// The store holds names only: nothing shaped like an env value or a key.
|
||||
const text = readFileSync(file, "utf8");
|
||||
assert.doesNotMatch(text, /ANTHROPIC|_KEY|TOKEN|mst-|msk-|env/i);
|
||||
// A second write bumps the revision and keeps the cap's since when the cap is the same.
|
||||
const second = writeRoute(d, { ...INPUT, model: "glm-5.3-flash", revision: 1 }, { now: at(LATER) });
|
||||
assert.equal(second.revision, 2);
|
||||
assert.equal(second.model, "glm-5.3-flash");
|
||||
assert.equal(second.cap.since, NOW);
|
||||
assert.equal(second.updatedAt, LATER);
|
||||
assert.deepEqual(readRoute(d), second);
|
||||
// No temp file or lock is left behind.
|
||||
assert.deepEqual(readdirSync(dir).sort(), ["route-receipts.jsonl", "route.json"]);
|
||||
});
|
||||
|
||||
test("each write appends one receipt line with the old and the new value", (t) => {
|
||||
const d = root(t);
|
||||
writeRoute(d, INPUT, { now: at(NOW) });
|
||||
writeRoute(d, { ...INPUT, limit: 1000 }, { now: at(LATER) });
|
||||
const lines = receipts(d);
|
||||
assert.equal(lines.length, 2);
|
||||
assert.deepEqual(lines[0], { at: NOW, by: "owner", revision: 1, from: null, to: { model: "glm-5", harness: "claude-code", cap: good().cap } });
|
||||
assert.equal(lines[1].revision, 2);
|
||||
assert.deepEqual(lines[1].from, { model: "glm-5", harness: "claude-code", cap: good().cap });
|
||||
assert.equal(lines[1].to.cap.limit, 1000);
|
||||
assert.equal(lines[1].to.cap.since, NOW);
|
||||
assert.equal(lstatSync(routePaths(d).receipts).mode & 0o777, 0o600);
|
||||
// A refused write appends nothing.
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, revision: 1 }, { now: at(LATER) }), code("triage-route-stale"));
|
||||
assert.equal(receipts(d).length, 2);
|
||||
});
|
||||
|
||||
test("a stale revision refuses and leaves the file alone", (t) => {
|
||||
const d = root(t);
|
||||
writeRoute(d, INPUT, { now: at(NOW) });
|
||||
writeRoute(d, { ...INPUT, limit: 7, revision: 1 }, { now: at(NOW) });
|
||||
const before = readFileSync(routePaths(d).file, "utf8");
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, limit: 9, revision: 1 }, { now: at(LATER) }), code("triage-route-stale"));
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, limit: 9, revision: 3 }, { now: at(LATER) }), code("triage-route-stale"));
|
||||
assert.equal(readFileSync(routePaths(d).file, "utf8"), before);
|
||||
// Revision 0 means "only if there is no route yet".
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, revision: 0 }, { now: at(LATER) }), code("triage-route-stale"));
|
||||
const e = root(t);
|
||||
assert.equal(writeRoute(e, { ...INPUT, revision: 0 }, { now: at(NOW) }).revision, 1);
|
||||
});
|
||||
|
||||
test("refused input shapes on a write", (t) => {
|
||||
const d = root(t);
|
||||
// Each input check refuses with its own message, so none rests only on the final shape check.
|
||||
const bad = [
|
||||
[{ ...INPUT, model: "" }, /model must be/], [{ ...INPUT, model: "Has Space" }, /model must be/],
|
||||
[{ ...INPUT, model: "x".repeat(65) }, /model must be/], [{ ...INPUT, model: 5 }, /model must be/],
|
||||
[{ ...INPUT, harness: "" }, /harness must be/], [{ ...INPUT, harness: "Claude" }, /harness must be/],
|
||||
[{ ...INPUT, harness: "../pi" }, /harness must be/], [{ ...INPUT, harness: "x".repeat(33) }, /harness must be/],
|
||||
[{ ...INPUT, limit: -1 }, /the cap must be/], [{ ...INPUT, limit: 1.5 }, /the cap must be/],
|
||||
[{ ...INPUT, limit: "100" }, /the cap must be/], [{ ...INPUT, limit: 2 ** 53 }, /the cap must be/],
|
||||
[{ ...INPUT, timeZone: undefined }, /a time zone is required/], [{ ...INPUT, timeZone: "" }, /time zone is unknown/],
|
||||
[{ ...INPUT, timeZone: "Mars/Olympus_Mons" }, /time zone is unknown/], [{ ...INPUT, timeZone: "../etc" }, /time zone is unknown/],
|
||||
[{ ...INPUT, timeZone: "+05:00" }, /time zone is unknown/],
|
||||
[{ ...INPUT, by: "" }, /actor name/], [{ ...INPUT, by: "Owner Person" }, /actor name/],
|
||||
[{ ...INPUT, revision: -1 }, /revision must be/], [{ ...INPUT, revision: "1" }, /revision must be/], [{ ...INPUT, revision: 1.5 }, /revision must be/],
|
||||
[{ ...INPUT, env: { A: "b" } }, /unknown field env/], [{ ...INPUT, key: "mst-x" }, /unknown field key/],
|
||||
[null, /not an object/], [[INPUT], /not an object/],
|
||||
];
|
||||
for (const [input, message] of bad) {
|
||||
assert.throws(() => writeRoute(d, input, { now: at(NOW) }), (e) => code("triage-route-invalid")(e) && message.test(e.message), JSON.stringify(input));
|
||||
}
|
||||
// No time zone default: leaving it out refuses.
|
||||
const { timeZone, ...noZone } = INPUT;
|
||||
assert.throws(() => writeRoute(d, noZone, { now: at(NOW) }), (e) => code("triage-route-invalid")(e) && /a time zone is required/.test(e.message));
|
||||
assert.equal(readRoute(d), null);
|
||||
});
|
||||
|
||||
test("an unknown time zone refuses on write and on read", (t) => {
|
||||
const d = root(t);
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, timeZone: "Atlantis/Lost" }, { now: at(NOW) }), code("triage-route-invalid"));
|
||||
plant(d, { ...good(), cap: { ...good().cap, timeZone: "Atlantis/Lost" } });
|
||||
assert.throws(() => readRoute(d), code("triage-route-unreadable"));
|
||||
});
|
||||
|
||||
test("each refused stored shape reads as unreadable", (t) => {
|
||||
const g = good();
|
||||
const shapes = {
|
||||
"not json": "{",
|
||||
"an array": [g],
|
||||
"null": null,
|
||||
"unknown field": { ...g, env: {} },
|
||||
"unknown cap field": { ...g, cap: { ...g.cap, perTurn: 5 } },
|
||||
"missing version": (({ version, ...r }) => r)(g),
|
||||
"missing revision": (({ revision, ...r }) => r)(g),
|
||||
"missing model": (({ model, ...r }) => r)(g),
|
||||
"missing harness": (({ harness, ...r }) => r)(g),
|
||||
"missing cap": (({ cap, ...r }) => r)(g),
|
||||
"missing updatedAt": (({ updatedAt, ...r }) => r)(g),
|
||||
"missing updatedBy": (({ updatedBy, ...r }) => r)(g),
|
||||
"missing cap.limit": { ...g, cap: (({ limit, ...r }) => r)(g.cap) },
|
||||
"missing cap.since": { ...g, cap: (({ since, ...r }) => r)(g.cap) },
|
||||
"missing cap.reset": { ...g, cap: (({ reset, ...r }) => r)(g.cap) },
|
||||
"missing cap.timeZone": { ...g, cap: (({ timeZone, ...r }) => r)(g.cap) },
|
||||
"version 2": { ...g, version: 2 },
|
||||
"revision 0": { ...g, revision: 0 },
|
||||
"revision text": { ...g, revision: "1" },
|
||||
"bad model": { ...g, model: "A B" },
|
||||
"bad harness": { ...g, harness: "A/B" },
|
||||
"cap an array": { ...g, cap: [1] },
|
||||
"negative limit": { ...g, cap: { ...g.cap, limit: -1 } },
|
||||
"bad reset": { ...g, cap: { ...g.cap, reset: "24:00" } },
|
||||
"bad since": { ...g, cap: { ...g.cap, since: "2026-11-18" } },
|
||||
"bad updatedAt": { ...g, updatedAt: "yesterday" },
|
||||
"bad updatedBy": { ...g, updatedBy: "A B" },
|
||||
};
|
||||
for (const [name, doc] of Object.entries(shapes)) {
|
||||
const d = root(t);
|
||||
plant(d, doc);
|
||||
assert.throws(() => readRoute(d), code("triage-route-unreadable"), name);
|
||||
if (typeof doc !== "string") assert.equal(validRoute(doc), false, name);
|
||||
}
|
||||
assert.equal(validRoute(g), true);
|
||||
// A write over an unreadable file refuses too; it never replaces what it could not read.
|
||||
const d = root(t);
|
||||
plant(d, { ...g, env: {} });
|
||||
assert.throws(() => writeRoute(d, INPUT, { now: at(NOW) }), code("triage-route-unreadable"));
|
||||
});
|
||||
|
||||
test("an oversized file is refused before it is parsed", (t) => {
|
||||
const d = root(t);
|
||||
plant(d, JSON.stringify(good()) + " ".repeat(70000));
|
||||
assert.throws(() => readRoute(d), code("triage-route-unreadable"));
|
||||
});
|
||||
|
||||
test("a wrong mode on the directory or the file refuses", (t) => {
|
||||
for (const mode of [0o755, 0o750, 0o711, 0o500]) {
|
||||
const d = root(t);
|
||||
plant(d, good());
|
||||
chmodSync(routePaths(d).dir, mode);
|
||||
assert.throws(() => readRoute(d), code("triage-route-unreadable"), `dir ${mode.toString(8)}`);
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, revision: 1 }, { now: at(NOW) }), code("triage-route-unreadable"), `dir ${mode.toString(8)}`);
|
||||
chmodSync(routePaths(d).dir, 0o700);
|
||||
}
|
||||
for (const mode of [0o644, 0o640, 0o604, 0o400, 0o700]) {
|
||||
const d = root(t);
|
||||
plant(d, good());
|
||||
chmodSync(routePaths(d).file, mode);
|
||||
assert.throws(() => readRoute(d), code("triage-route-unreadable"), `file ${mode.toString(8)}`);
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, revision: 1 }, { now: at(NOW) }), code("triage-route-unreadable"), `file ${mode.toString(8)}`);
|
||||
}
|
||||
// A receipts file with the wrong mode refuses the write before the route changes.
|
||||
const d = root(t);
|
||||
writeRoute(d, INPUT, { now: at(NOW) });
|
||||
chmodSync(routePaths(d).receipts, 0o644);
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, limit: 1, revision: 1 }, { now: at(LATER) }), code("triage-route-unreadable"));
|
||||
assert.equal(readRoute(d).revision, 1);
|
||||
});
|
||||
|
||||
test("a wrong owner refuses on the directory and the file", (t) => {
|
||||
const d = root(t);
|
||||
plant(d, good());
|
||||
// Pretend to be another user: the store compares with the uid it is given.
|
||||
const other = process.getuid() + 1;
|
||||
// The files alone owned by someone else: route.json on read and write, the receipts on write.
|
||||
assert.throws(() => readRoute(d, { fileUid: other }), unreadableFor(/route\.json is not owned by this user/));
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, revision: 1 }, { now: at(NOW), fileUid: other }), unreadableFor(/route\.json is not owned by this user/));
|
||||
const e = root(t);
|
||||
writeRoute(e, INPUT, { now: at(NOW) });
|
||||
unlinkSync(routePaths(e).file);
|
||||
assert.throws(() => writeRoute(e, INPUT, { now: at(NOW), fileUid: other }), unreadableFor(/route-receipts\.jsonl is not owned by this user/));
|
||||
assert.equal(existsSync(routePaths(e).file), false);
|
||||
assert.throws(() => readRoute(d, { uid: other }), unreadableFor(/the triage directory is not owned by this user/));
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, revision: 1 }, { now: at(NOW), uid: other }), unreadableFor(/the triage directory is not owned by this user/));
|
||||
assert.equal(readRoute(d, { uid: process.getuid() }).revision, 1);
|
||||
});
|
||||
|
||||
test("a link at the file, the directory or the receipts refuses; so does a non-file", (t) => {
|
||||
// Symlink at route.json, to a valid route elsewhere.
|
||||
let d = root(t);
|
||||
plant(d, good());
|
||||
const elsewhere = join(d, "elsewhere.json");
|
||||
renameSync(routePaths(d).file, elsewhere);
|
||||
symlinkSync(elsewhere, routePaths(d).file);
|
||||
assert.throws(() => readRoute(d), code("triage-route-unreadable"));
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, revision: 1 }, { now: at(NOW) }), code("triage-route-unreadable"));
|
||||
// A hard link at route.json.
|
||||
d = root(t);
|
||||
plant(d, good());
|
||||
linkSync(routePaths(d).file, join(d, "hard.json"));
|
||||
assert.throws(() => readRoute(d), code("triage-route-unreadable"));
|
||||
// Symlink at the triage directory, to a real 0700 directory holding a valid route.
|
||||
d = root(t);
|
||||
const real = join(d, "real");
|
||||
mkdirSync(real, { mode: 0o700 });
|
||||
writeFileSync(join(real, "route.json"), JSON.stringify(good()), { mode: 0o600 });
|
||||
symlinkSync(real, routePaths(d).dir);
|
||||
assert.throws(() => readRoute(d), unreadableFor(/the triage directory is a link/));
|
||||
assert.throws(() => writeRoute(d, INPUT, { now: at(NOW) }), unreadableFor(/the triage directory is a link/));
|
||||
// A directory where route.json should be.
|
||||
d = root(t);
|
||||
mkdirSync(routePaths(d).file, { recursive: true, mode: 0o700 });
|
||||
chmodSync(routePaths(d).dir, 0o700);
|
||||
assert.throws(() => readRoute(d), unreadableFor(/route\.json is not a regular file/));
|
||||
// A file where the triage directory should be, at 0700 so only the directory check refuses it.
|
||||
d = root(t);
|
||||
writeFileSync(routePaths(d).dir, "x", { mode: 0o700 });
|
||||
chmodSync(routePaths(d).dir, 0o700);
|
||||
assert.throws(() => readRoute(d), unreadableFor(/the triage directory is not a directory/));
|
||||
assert.throws(() => writeRoute(d, INPUT, { now: at(NOW) }), unreadableFor(/the triage directory is not a directory/));
|
||||
// A symlink at the receipts file refuses the write and never follows it.
|
||||
d = root(t);
|
||||
writeRoute(d, INPUT, { now: at(NOW) });
|
||||
const target = join(d, "target.jsonl");
|
||||
writeFileSync(target, "", { mode: 0o600 });
|
||||
unlinkSync(routePaths(d).receipts);
|
||||
symlinkSync(target, routePaths(d).receipts);
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, revision: 1 }, { now: at(LATER) }), code("triage-route-unreadable"));
|
||||
assert.equal(readFileSync(target, "utf8"), "");
|
||||
assert.equal(readRoute(d).revision, 1);
|
||||
// A symlink at the lock path is not followed: the write refuses as busy.
|
||||
d = root(t);
|
||||
writeRoute(d, INPUT, { now: at(NOW) });
|
||||
symlinkSync(join(d, "nowhere"), routePaths(d).lock);
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, revision: 1 }, { now: at(LATER) }), code("triage-route-busy"));
|
||||
assert.equal(existsSync(join(d, "nowhere")), false);
|
||||
});
|
||||
|
||||
test("a write under a narrow umask still leaves 0700 and 0600, so the next read works", (t) => {
|
||||
const d = root(t);
|
||||
const old = process.umask(0o277);
|
||||
let written;
|
||||
try { written = writeRoute(d, INPUT, { now: at(NOW) }); } finally { process.umask(old); }
|
||||
const { dir, file, receipts: r } = routePaths(d);
|
||||
assert.equal(lstatSync(dir).mode & 0o777, 0o700);
|
||||
assert.equal(lstatSync(file).mode & 0o777, 0o600);
|
||||
assert.equal(lstatSync(r).mode & 0o777, 0o600);
|
||||
assert.deepEqual(readRoute(d), written);
|
||||
// An existing receipts file is never repaired: a wrong mode still refuses.
|
||||
chmodSync(r, 0o644);
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, revision: 1 }, { now: at(LATER) }), unreadableFor(/route-receipts\.jsonl mode is not 0600/));
|
||||
});
|
||||
|
||||
test("a failed temp write refuses and leaves the route; a failed receipt after the rename says so", (t) => {
|
||||
const d = root(t);
|
||||
writeRoute(d, INPUT, { now: at(NOW) });
|
||||
const before = readFileSync(routePaths(d).file, "utf8");
|
||||
const fail = (stage) => (s) => { if (s === stage) throw Object.assign(new Error("no space"), { code: "ENOSPC" }); };
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, limit: 1, revision: 1 }, { now: at(LATER), hook: fail("write") }), unreadableFor(/route\.json cannot be written/));
|
||||
assert.equal(readFileSync(routePaths(d).file, "utf8"), before);
|
||||
assert.deepEqual(readdirSync(routePaths(d).dir).sort(), ["route-receipts.jsonl", "route.json"]);
|
||||
assert.equal(receipts(d).length, 1);
|
||||
// After the rename the route has changed; the refusal names the new revision and the missing line.
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, limit: 1, revision: 1 }, { now: at(LATER), hook: fail("after-rename") }),
|
||||
(e) => code("triage-route-unrecorded")(e) && /changed to revision 2, but its receipt line was not written/.test(e.message));
|
||||
assert.equal(readRoute(d).revision, 2);
|
||||
assert.equal(receipts(d).length, 1);
|
||||
assert.deepEqual(readdirSync(routePaths(d).dir).sort(), ["route-receipts.jsonl", "route.json"]);
|
||||
});
|
||||
|
||||
test("the store makes its directory 0700 but never the data root", (t) => {
|
||||
const d = root(t);
|
||||
assert.throws(() => writeRoute(join(d, "missing"), INPUT, { now: at(NOW) }), code("triage-route-unreadable"));
|
||||
assert.equal(existsSync(join(d, "missing")), false);
|
||||
});
|
||||
|
||||
test("two writers in one process: the second, while the first holds the lock, refuses busy", (t) => {
|
||||
const d = root(t);
|
||||
writeRoute(d, INPUT, { now: at(NOW) });
|
||||
let inner = null;
|
||||
const won = writeRoute(d, { ...INPUT, limit: 1, revision: 1 }, {
|
||||
now: at(LATER),
|
||||
hook: (stage) => {
|
||||
if (stage !== "before-rename") return;
|
||||
try { writeRoute(d, { ...INPUT, limit: 2, revision: 1 }, { now: at(LATER) }); } catch (e) { inner = e; }
|
||||
},
|
||||
});
|
||||
assert.equal(won.cap.limit, 1);
|
||||
assert.ok(code("triage-route-busy")(inner), String(inner));
|
||||
assert.equal(readRoute(d).cap.limit, 1);
|
||||
assert.equal(receipts(d).length, 2);
|
||||
});
|
||||
|
||||
// A child process that loads the store and writes once; prints {ok, code}. `wait` makes it hold
|
||||
// at the given stage until the file `go` appears, so two children overlap for real.
|
||||
const CHILD = `
|
||||
import { writeRoute } from ${JSON.stringify(MODULE)};
|
||||
import { existsSync, writeFileSync } from "node:fs";
|
||||
const [dataRoot, limit, revision, wait, go, ready] = process.argv.slice(1);
|
||||
try {
|
||||
const r = writeRoute(dataRoot, { model: "glm-5", harness: "claude-code", limit: Number(limit), timeZone: "UTC", by: "owner", revision: Number(revision) }, {
|
||||
hook: (stage) => {
|
||||
if (stage === "crash") { if (wait === "crash") process.kill(process.pid, "SIGKILL"); return; }
|
||||
if (stage !== wait) return;
|
||||
writeFileSync(ready, "");
|
||||
const end = Date.now() + 20000;
|
||||
while (!existsSync(go) && Date.now() < end) Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 20);
|
||||
},
|
||||
});
|
||||
console.log(JSON.stringify({ ok: true, limit: r.cap.limit }));
|
||||
} catch (e) { console.log(JSON.stringify({ ok: false, code: e.code ?? String(e) })); }
|
||||
`;
|
||||
function child(args) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const p = spawn(process.execPath, ["--input-type=module", "-e", CHILD, ...args], { stdio: ["ignore", "pipe", "pipe"] });
|
||||
let out = "", err = "";
|
||||
p.stdout.setEncoding("utf8").on("data", (c) => { out += c; });
|
||||
p.stderr.setEncoding("utf8").on("data", (c) => { err += c; });
|
||||
p.on("error", reject);
|
||||
p.on("close", (status, signal) => resolve({ status, signal, out: out.trim(), err }));
|
||||
});
|
||||
}
|
||||
async function until(path) {
|
||||
const end = Date.now() + 20000;
|
||||
while (!existsSync(path)) {
|
||||
if (Date.now() > end) throw new Error(`timed out waiting for ${path}`);
|
||||
await new Promise((r) => setTimeout(r, 20));
|
||||
}
|
||||
}
|
||||
|
||||
test("two writer processes racing on one revision: one wins, one refuses", async (t) => {
|
||||
const d = root(t);
|
||||
writeRoute(d, { ...INPUT, timeZone: "UTC" }, { now: at(NOW) });
|
||||
const go = join(d, "go"), ready = join(d, "ready");
|
||||
// The first holds just before its rename; the second starts then, with the same revision.
|
||||
const first = child([d, "11", "1", "before-rename", go, ready]);
|
||||
await until(ready);
|
||||
const second = await child([d, "22", "1", "none", join(d, "unused"), join(d, "unused-ready")]);
|
||||
writeFileSync(go, "");
|
||||
const a = JSON.parse((await first).out), b = JSON.parse(second.out);
|
||||
assert.deepEqual(a, { ok: true, limit: 11 });
|
||||
assert.equal(b.ok, false);
|
||||
assert.equal(b.code, "triage-route-busy");
|
||||
assert.equal(readRoute(d).cap.limit, 11);
|
||||
assert.equal(readRoute(d).revision, 2);
|
||||
// After the winner, the loser's revision is stale.
|
||||
const again = JSON.parse((await child([d, "22", "1", "none", join(d, "u2"), join(d, "u3")])).out);
|
||||
assert.deepEqual(again, { ok: false, code: "triage-route-stale" });
|
||||
// Started together with no pause, exactly one of two wins.
|
||||
const both = await Promise.all([child([d, "33", "2", "none", join(d, "u4"), join(d, "u5")]), child([d, "44", "2", "none", join(d, "u6"), join(d, "u7")])]);
|
||||
const results = both.map((r) => JSON.parse(r.out));
|
||||
assert.equal(results.filter((r) => r.ok).length, 1, JSON.stringify(results));
|
||||
assert.ok(results.filter((r) => !r.ok).every((r) => ["triage-route-busy", "triage-route-stale"].includes(r.code)), JSON.stringify(results));
|
||||
assert.equal(readRoute(d).revision, 3);
|
||||
assert.equal(receipts(d).length, 3);
|
||||
});
|
||||
|
||||
test("a crash between the temp write and the rename leaves the old file", async (t) => {
|
||||
const d = root(t);
|
||||
const old = writeRoute(d, { ...INPUT, timeZone: "UTC" }, { now: at(NOW) });
|
||||
const r = await child([d, "99", "1", "crash", join(d, "go"), join(d, "ready")]);
|
||||
assert.equal(r.signal, "SIGKILL", r.err);
|
||||
assert.deepEqual(readRoute(d), old);
|
||||
assert.equal(receipts(d).length, 1);
|
||||
// The temp file is still there, and the dead writer's lock refuses the next write, naming it.
|
||||
const { dir, lock } = routePaths(d);
|
||||
assert.ok(readdirSync(dir).some((n) => n.endsWith(".tmp")), readdirSync(dir).join(","));
|
||||
assert.throws(() => writeRoute(d, { ...INPUT, revision: 1 }, { now: at(LATER) }), (e) => code("triage-route-busy")(e) && /not running/.test(e.message) && /route\.lock/.test(e.message));
|
||||
// The person removes the lock by hand; the next write works and the read ignores the temp file.
|
||||
unlinkSync(lock);
|
||||
assert.equal(writeRoute(d, { ...INPUT, revision: 1 }, { now: at(LATER) }).revision, 2);
|
||||
assert.equal(readRoute(d).revision, 2);
|
||||
});
|
||||
@@ -156,7 +156,7 @@ test("the snapshot file is one 0600 file, replaced whole", () => {
|
||||
assert.equal(JSON.parse(readFileSync(file, "utf8")).n, 2);
|
||||
});
|
||||
|
||||
test("organizations are read at most ORG_READS_AT_ONCE at a time and listed in registry order", async () => {
|
||||
test("config checks run at most ORG_READS_AT_ONCE at a time and organizations are listed in registry order", async () => {
|
||||
let live = 0, most = 0;
|
||||
const ids = Array.from({ length: 10 }, (_, i) => `org${i}`);
|
||||
const slow = async () => { live++; most = Math.max(most, live); await new Promise((r) => setTimeout(r, 5)); live--; return { valid: true, exit: 0, code: null }; };
|
||||
@@ -167,15 +167,19 @@ test("organizations are read at most ORG_READS_AT_ONCE at a time and listed in r
|
||||
assert.equal(snap.organizations.count, 10);
|
||||
});
|
||||
|
||||
// The timing tests below assert what was read, which no load changes, and a time bound far
|
||||
// above the one deadline they prove; the "begun late" test moves the deadline's clock by hand.
|
||||
test("the organization reads share one deadline: forty hung organizations take one timeout, not ten", async () => {
|
||||
let calls = 0;
|
||||
const calls = { busStatus: 0, processing: 0, orgConfig: 0 };
|
||||
const ids = Array.from({ length: 40 }, (_, i) => `org${i}`);
|
||||
const hang = () => { calls++; return new Promise(() => {}); };
|
||||
const hang = (name) => () => { calls[name]++; return new Promise(() => {}); };
|
||||
const began = Date.now();
|
||||
const snap = await composeSnapshot({ orgs: async () => ids.map((id) => ({ id, state: "active" })), busStatus: hang, processing: hang, orgConfig: hang }, { timeoutMs: 200 });
|
||||
const snap = await composeSnapshot({ orgs: async () => ids.map((id) => ({ id, state: "active" })), busStatus: hang("busStatus"), processing: hang("processing"), orgConfig: hang("orgConfig") }, { timeoutMs: 200 });
|
||||
const took = Date.now() - began;
|
||||
assert.ok(took < 800, `took ${took} ms`);
|
||||
assert.ok(calls < ids.length, `${calls} reads began`);
|
||||
// Ten timeouts would be 2000 ms; one is 200.
|
||||
assert.ok(took < 1500, `took ${took} ms`);
|
||||
// The config checks behind the first four hang to the deadline, and no later one begins.
|
||||
assert.ok(calls.orgConfig < ids.length, `${calls.orgConfig} config checks began`);
|
||||
assert.equal(snap.organizations.count, 40);
|
||||
assert.deepEqual(snap.organizations.list.map((o) => o.id), ids);
|
||||
for (const o of snap.organizations.list) assert.deepEqual([o.bus, o.processing, o.config], [NOT_READ, NOT_READ, NOT_READ], o.id);
|
||||
@@ -184,11 +188,35 @@ test("the organization reads share one deadline: forty hung organizations take o
|
||||
test("an organization begun late gets only what is left of the deadline", async () => {
|
||||
const ids = ["a0", "a1", "a2", "a3", "a4"];
|
||||
const hang = () => new Promise(() => {});
|
||||
// Three readers hang; a3 answers after 300 ms, and the worker that read it starts a4 with 100 ms left.
|
||||
const orgConfig = (id) => (id === "a3" ? new Promise((r) => setTimeout(() => r({ valid: true, exit: 0, code: null }), 300)) : hang());
|
||||
// The deadline's clock moves only by hand. Three readers hang; a3 answers at once and moves the
|
||||
// clock 300 ms, so the worker that read it starts a4 with 100 ms left. a4 answers after 250 ms:
|
||||
// inside a whole timeout, outside what is left.
|
||||
let t = 0;
|
||||
const answer = { valid: true, exit: 0, code: null };
|
||||
const orgConfig = (id) => {
|
||||
if (id === "a3") { t += 300; return answer; }
|
||||
if (id === "a4") return new Promise((r) => setTimeout(() => r(answer), 250));
|
||||
return hang();
|
||||
};
|
||||
const began = Date.now();
|
||||
const snap = await composeSnapshot({ orgs: async () => ids.map((id) => ({ id, state: "active" })), orgConfig }, { timeoutMs: 400 });
|
||||
const snap = await composeSnapshot({ orgs: async () => ids.map((id) => ({ id, state: "active" })), orgConfig }, { timeoutMs: 400, clock: () => t });
|
||||
const took = Date.now() - began;
|
||||
assert.ok(took < 600, `took ${took} ms`);
|
||||
assert.ok(took < 1500, `took ${took} ms`);
|
||||
assert.deepEqual(snap.organizations.list.map((o) => [o.id, o.config === NOT_READ ? "not read" : "read"]), [["a0", "not read"], ["a1", "not read"], ["a2", "not read"], ["a3", "read"], ["a4", "not read"]]);
|
||||
});
|
||||
|
||||
test("the bus and processing reads are not held behind the config checks: every organization's sockets are read", async () => {
|
||||
const ids = Array.from({ length: 10 }, (_, i) => `org${i}`);
|
||||
const hang = () => new Promise(() => {});
|
||||
// The first four config checks hang to the deadline and hold every config slot; the socket reads answer at once.
|
||||
const orgConfig = (id) => (ids.indexOf(id) < ORG_READS_AT_ONCE ? hang() : { valid: true, exit: 0, code: null });
|
||||
const snap = await composeSnapshot({
|
||||
orgs: async () => ids.map((id) => ({ id, state: "active" })),
|
||||
busStatus: async () => ({ healthy: true, pid: 1 }),
|
||||
processing: async () => ({ halted: false, invalid: false }),
|
||||
orgConfig,
|
||||
}, { timeoutMs: 300 });
|
||||
assert.deepEqual(snap.organizations.list.map((o) => o.id), ids);
|
||||
for (const o of snap.organizations.list) assert.deepEqual([o.bus.state, o.processing], ["running", { halted: false, invalid: false }], o.id);
|
||||
for (const o of snap.organizations.list.slice(0, ORG_READS_AT_ONCE)) assert.equal(o.config, NOT_READ, o.id);
|
||||
});
|
||||
@@ -0,0 +1,205 @@
|
||||
// #1873 triage-cap: Triage's usage file and its daily cap. One line per turn in
|
||||
// <dataRoot>/triage/usage.jsonl, held to the same file checks as the route store; the cap is the
|
||||
// daily-quota plugin's own rule over those lines, with the stored cap as its one leaf.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { chmodSync, linkSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, symlinkSync, writeFileSync } from "node:fs";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { join } from "node:path";
|
||||
import { mayLaunch, report } from "../../plugins/limit-check/daily-quota/index.mjs";
|
||||
import {
|
||||
USAGE_ACCOUNT, USAGE_FIELDS, appendUsage, capState, delta, limitsFor, nextReset, readUsage, snapshot, usagePath,
|
||||
} from "../src/triage-usage.mjs";
|
||||
|
||||
const scratch = () => mkdtempSync(join(process.env.TMPDIR, "tu-"));
|
||||
const fileIn = () => {
|
||||
const dir = join(scratch(), "triage");
|
||||
mkdirSync(dir, { mode: 0o700 });
|
||||
return join(dir, "usage.jsonl");
|
||||
};
|
||||
const line = (over = {}) => ({ at: "2026-11-18T12:00:00.000Z", run: "triage-20261118120000", turn: "owner-0123456789ab", status: "known", input: 10, output: 5, cacheRead: 2, cacheWrite: 1, ...over });
|
||||
const unknownLine = (over = {}) => line({ status: "unknown", input: null, output: null, cacheRead: null, cacheWrite: null, ...over });
|
||||
const cap = (over = {}) => ({ limit: 100, since: "2026-10-01T00:00:00.000Z", reset: "00:00", timeZone: "UTC", ...over });
|
||||
const ms = (s) => Date.parse(s);
|
||||
|
||||
test("the usage file is <dataRoot>/triage/usage.jsonl and a line holds exactly its eight fields", () => {
|
||||
assert.equal(usagePath("/d"), "/d/triage/usage.jsonl");
|
||||
assert.deepEqual(USAGE_FIELDS, ["at", "run", "turn", "status", "input", "output", "cacheRead", "cacheWrite"]);
|
||||
assert.equal(USAGE_ACCOUNT, "triage");
|
||||
});
|
||||
|
||||
test("each append writes one line, 0600, in field order; a torn last line is closed off before the next line", () => {
|
||||
const file = fileIn();
|
||||
appendUsage(file, line());
|
||||
appendUsage(file, unknownLine({ turn: "owner-0123456789ac" }));
|
||||
assert.equal(lstatSync(file).mode & 0o777, 0o600);
|
||||
const text = readFileSync(file, "utf8");
|
||||
assert.equal(text.split("\n").length, 3);
|
||||
assert.deepEqual(Object.keys(JSON.parse(text.split("\n")[0])), USAGE_FIELDS);
|
||||
// A crash in a write leaves a torn line. The next append starts on a line of its own.
|
||||
writeFileSync(file, '{"at":"2026-11-18T12:0', { flag: "a" });
|
||||
appendUsage(file, line({ turn: "owner-0123456789ad" }));
|
||||
const lines = readFileSync(file, "utf8").split("\n");
|
||||
assert.equal(lines.length, 5);
|
||||
assert.equal(JSON.parse(lines[3]).turn, "owner-0123456789ad");
|
||||
// A line that does not match its shape is never written.
|
||||
for (const bad of [{ ...line(), extra: 1 }, line({ at: "yesterday" }), line({ input: -1 }), line({ status: "unknown" }), unknownLine({ input: 3 }), line({ turn: "a b" }), line({ run: "" })]) {
|
||||
assert.throws(() => appendUsage(file, bad), (e) => e.code === "triage-usage-unreadable", JSON.stringify(bad));
|
||||
}
|
||||
assert.equal(readFileSync(file, "utf8").split("\n").length, 5);
|
||||
});
|
||||
|
||||
test("a narrow umask still leaves a new usage file at 0600", () => {
|
||||
const file = fileIn();
|
||||
const old = process.umask(0o277);
|
||||
try { appendUsage(file, line()); } finally { process.umask(old); }
|
||||
assert.equal(lstatSync(file).mode & 0o777, 0o600);
|
||||
});
|
||||
|
||||
// Each unsafe state refuses both the read and the append, and the append leaves the target alone.
|
||||
function unsafe(make) {
|
||||
const file = fileIn();
|
||||
const opts = make(file) ?? {};
|
||||
return { file, opts };
|
||||
}
|
||||
test("a wrong mode, a wrong owner, a symlink, a hard link or a non-file refuses the read and the append", () => {
|
||||
const cases = {
|
||||
"mode 0644": (f) => { appendUsage(f, line()); chmodSync(f, 0o644); },
|
||||
"mode 0400": (f) => { appendUsage(f, line()); chmodSync(f, 0o400); },
|
||||
"owner": (f) => { appendUsage(f, line()); return { uid: process.getuid() + 1 }; },
|
||||
"symlink": (f) => { const real = join(f, "..", "real.jsonl"); writeFileSync(real, "", { mode: 0o600 }); symlinkSync(real, f); },
|
||||
"dangling symlink": (f) => { symlinkSync(join(f, "..", "absent.jsonl"), f); },
|
||||
"hard link": (f) => { appendUsage(f, line()); linkSync(f, join(f, "..", "other.jsonl")); },
|
||||
"directory": (f) => { mkdirSync(f, { mode: 0o700 }); },
|
||||
"fifo": (f) => { execFileSync("mkfifo", ["-m", "600", f]); },
|
||||
};
|
||||
for (const [label, make] of Object.entries(cases)) {
|
||||
const { file, opts } = unsafe(make);
|
||||
const before = lstatSync(file);
|
||||
assert.throws(() => readUsage(file, opts), (e) => e.code === "triage-usage-unreadable", `read: ${label}`);
|
||||
assert.throws(() => appendUsage(file, line(), opts), (e) => e.code === "triage-usage-unreadable", `append: ${label}`);
|
||||
const after = lstatSync(file);
|
||||
assert.deepEqual([after.size, after.mode, after.ino], [before.size, before.mode, before.ino], `untouched: ${label}`);
|
||||
}
|
||||
// The dangling link's target is not made.
|
||||
const { file } = unsafe((f) => { symlinkSync(join(f, "..", "absent.jsonl"), f); });
|
||||
try { appendUsage(file, line()); } catch {}
|
||||
assert.throws(() => lstatSync(join(file, "..", "absent.jsonl")), (e) => e.code === "ENOENT");
|
||||
});
|
||||
|
||||
test("a usage file over the size bound refuses before it is parsed", () => {
|
||||
const file = fileIn();
|
||||
appendUsage(file, line());
|
||||
appendUsage(file, line({ turn: "owner-0123456789ac" }));
|
||||
const size = lstatSync(file).size;
|
||||
assert.equal(readUsage(file, { maxBytes: size }).records.length, 2);
|
||||
assert.throws(() => readUsage(file, { maxBytes: size - 1 }), (e) => e.code === "triage-usage-unreadable");
|
||||
});
|
||||
|
||||
test("no file reads as no turns; good lines read back as records for the triage account", () => {
|
||||
const file = fileIn();
|
||||
assert.deepEqual(readUsage(file), { records: [], bad: [] });
|
||||
appendUsage(file, line());
|
||||
appendUsage(file, unknownLine({ turn: "owner-0123456789ac" }));
|
||||
const { records, bad } = readUsage(file);
|
||||
assert.deepEqual(bad, []);
|
||||
assert.deepEqual(records, [{ account: "triage", ...line() }, { account: "triage", ...unknownLine({ turn: "owner-0123456789ac" }) }]);
|
||||
});
|
||||
|
||||
test("a torn last line is refused and counts as one unknown turn; a wrong-shaped line the same", () => {
|
||||
const file = fileIn();
|
||||
appendUsage(file, line());
|
||||
writeFileSync(file, '{"at":"2026-11-18T13:00:00.000Z","run":"triage-2026', { flag: "a" });
|
||||
const torn = readUsage(file);
|
||||
assert.equal(torn.records.length, 2);
|
||||
assert.deepEqual(torn.bad.map((b) => b.line), [2]);
|
||||
const [, b] = torn.records;
|
||||
assert.deepEqual([b.status, b.run, b.turn, b.input, b.output, b.cacheRead, b.cacheWrite], ["unknown", null, "line-2", null, null, null, null]);
|
||||
assert.ok(!Number.isNaN(Date.parse(b.at)), "a torn last line takes the file's change time");
|
||||
const [r] = report({ limits: limitsFor(cap()), records: torn.records, now: "2026-11-18T23:00:00.000Z" });
|
||||
assert.deepEqual([r.count, r.unknownTurns], [18, 1]);
|
||||
// Closed off by the next append, it stays one unknown turn, dated by the line after it.
|
||||
appendUsage(file, line({ at: "2026-11-18T14:00:00.000Z", turn: "owner-0123456789ac", input: 1, output: 0, cacheRead: 0, cacheWrite: 0 }));
|
||||
writeFileSync(file, JSON.stringify({ ...line(), input: "9" }) + "\n", { flag: "a" });
|
||||
const later = readUsage(file);
|
||||
assert.deepEqual(later.bad.map((x) => x.line), [2, 4]);
|
||||
assert.equal(later.records[1].at, "2026-11-18T14:00:00.000Z");
|
||||
const [again] = report({ limits: limitsFor(cap()), records: later.records, now: "2026-11-18T23:00:00.000Z" });
|
||||
assert.deepEqual([again.count, again.unknownTurns], [19, 2]);
|
||||
});
|
||||
|
||||
test("the limits copy the organization shape: one daily leaf for the triage account", () => {
|
||||
assert.deepEqual(limitsFor(cap()), { tokens: { daily: { triage: cap() } } });
|
||||
const records = [{ account: "triage", ...line({ input: 100, output: 0, cacheRead: 0, cacheWrite: 0 }) }];
|
||||
assert.deepEqual(mayLaunch({ limits: limitsFor(cap()), subject: { account: "triage" }, records, now: "2026-11-18T13:00:00.000Z" }).allow, false);
|
||||
});
|
||||
|
||||
test("the cap state: under, at and over the cap, unknown turns adding nothing, and the cap's since", () => {
|
||||
const now = ms("2026-11-18T18:00:00.000Z");
|
||||
const recs = (...counts) => counts.map((n, i) => ({ account: "triage", ...line({ turn: `owner-00000000000${i}`, input: n, output: 0, cacheRead: 0, cacheWrite: 0 }) }));
|
||||
const under = capState({ cap: cap(), records: recs(40, 59), now });
|
||||
assert.equal(under.allow, true);
|
||||
assert.deepEqual(under.usageToday, { count: 99, input: 99, output: 0, cacheRead: 0, cacheWrite: 0, unknownTurns: 0 });
|
||||
assert.equal(under.resetsAt, "2026-11-19T00:00:00.000Z");
|
||||
const at = capState({ cap: cap(), records: recs(40, 60), now });
|
||||
assert.equal(at.allow, false);
|
||||
assert.equal(at.usageToday.count, 100);
|
||||
const withUnknown = capState({ cap: cap(), records: [...recs(40), { account: "triage", ...unknownLine({ turn: "owner-00000000000f" }) }], now });
|
||||
assert.deepEqual([withUnknown.allow, withUnknown.usageToday.count, withUnknown.usageToday.unknownTurns], [true, 40, 1]);
|
||||
// Lines before the cap's since or before today's reset do not count.
|
||||
const old = [{ account: "triage", ...line({ at: "2026-11-17T23:59:59.999Z", input: 500, output: 0, cacheRead: 0, cacheWrite: 0 }) }];
|
||||
assert.equal(capState({ cap: cap(), records: old, now }).usageToday.count, 0);
|
||||
const since = [{ account: "triage", ...line({ at: "2026-11-18T10:00:00.000Z", input: 500, output: 0, cacheRead: 0, cacheWrite: 0 }) }];
|
||||
assert.equal(capState({ cap: cap({ since: "2026-11-18T11:00:00.000Z" }), records: since, now }).allow, true);
|
||||
// A cap of 0 takes no turn, as the plugin's own rule says.
|
||||
assert.equal(capState({ cap: cap({ limit: 0 }), records: [], now }).allow, false);
|
||||
});
|
||||
|
||||
test("the reset is 00:00 in the stored zone: the next one is the time shown and the day turns over there", () => {
|
||||
const z = (timeZone) => cap({ timeZone });
|
||||
assert.equal(new Date(nextReset(z("UTC"), ms("2026-11-18T18:00:00.000Z"))).toISOString(), "2026-11-19T00:00:00.000Z");
|
||||
assert.equal(new Date(nextReset(z("UTC"), ms("2026-11-18T00:00:00.000Z"))).toISOString(), "2026-11-19T00:00:00.000Z", "at the reset, the next is a day on");
|
||||
assert.equal(new Date(nextReset(z("UTC"), ms("2026-11-18T23:59:59.999Z"))).toISOString(), "2026-11-19T00:00:00.000Z");
|
||||
assert.equal(new Date(nextReset(z("Asia/Kolkata"), ms("2026-11-18T18:00:00.000Z"))).toISOString(), "2026-11-18T18:30:00.000Z");
|
||||
// A since after now: the reset after it.
|
||||
assert.equal(new Date(nextReset(cap({ since: "2026-11-19T06:00:00.000Z" }), ms("2026-11-18T18:00:00.000Z"))).toISOString(), "2026-11-20T00:00:00.000Z");
|
||||
});
|
||||
|
||||
test("a DST day: America/Chicago falls back on 2026-11-01 (25 hours) and springs forward on 2026-03-08 (23 hours)", () => {
|
||||
const chicago = cap({ timeZone: "America/Chicago", since: "2026-01-01T00:00:00.000Z", limit: 50 });
|
||||
// Fall back: the day runs from 05:00Z (00:00 CDT) to 06:00Z next day (00:00 CST).
|
||||
assert.equal(new Date(nextReset(chicago, ms("2026-11-01T12:00:00.000Z"))).toISOString(), "2026-11-02T06:00:00.000Z");
|
||||
assert.equal(new Date(nextReset(chicago, ms("2026-11-01T05:30:00.000Z"))).toISOString(), "2026-11-02T06:00:00.000Z", "inside the repeated hour's day");
|
||||
const fall = [
|
||||
{ account: "triage", ...line({ at: "2026-11-01T04:59:59.999Z", turn: "owner-000000000001", input: 30, output: 0, cacheRead: 0, cacheWrite: 0 }) },
|
||||
{ account: "triage", ...line({ at: "2026-11-01T05:00:00.000Z", turn: "owner-000000000002", input: 30, output: 0, cacheRead: 0, cacheWrite: 0 }) },
|
||||
{ account: "triage", ...line({ at: "2026-11-02T05:30:00.000Z", turn: "owner-000000000003", input: 30, output: 0, cacheRead: 0, cacheWrite: 0 }) },
|
||||
];
|
||||
const lateFall = capState({ cap: chicago, records: fall, now: ms("2026-11-02T05:59:59.999Z") });
|
||||
assert.deepEqual([lateFall.allow, lateFall.usageToday.count, lateFall.resetsAt], [false, 60, "2026-11-02T06:00:00.000Z"], "the 25th hour still counts the day");
|
||||
const nextDay = capState({ cap: chicago, records: fall, now: ms("2026-11-02T06:00:00.000Z") });
|
||||
assert.deepEqual([nextDay.allow, nextDay.usageToday.count, nextDay.resetsAt], [true, 0, "2026-11-03T06:00:00.000Z"]);
|
||||
// Spring forward: the day runs from 06:00Z (00:00 CST) to 05:00Z next day (00:00 CDT).
|
||||
assert.equal(new Date(nextReset(chicago, ms("2026-03-08T12:00:00.000Z"))).toISOString(), "2026-03-09T05:00:00.000Z");
|
||||
const spring = [{ account: "triage", ...line({ at: "2026-03-08T06:00:00.000Z", input: 60, output: 0, cacheRead: 0, cacheWrite: 0 }) }];
|
||||
assert.equal(capState({ cap: chicago, records: spring, now: ms("2026-03-09T04:59:59.999Z") }).allow, false);
|
||||
assert.equal(capState({ cap: chicago, records: spring, now: ms("2026-03-09T05:00:00.000Z") }).allow, true);
|
||||
});
|
||||
|
||||
test("the turn delta keeps the bus's rules: new entries summed, a rewritten or missing baseline unknown, no new usage unknown", () => {
|
||||
const e = (input) => ({ input, output: 1, cacheWrite: 0, cacheRead: 0 });
|
||||
const begin = snapshot({ status: "known", entries: { a: e(5) } });
|
||||
assert.deepEqual(delta(begin, snapshot({ status: "known", entries: { a: e(5), b: e(7), c: e(1) } })), { status: "known", input: 8, output: 2, cacheWrite: 0, cacheRead: 0 });
|
||||
assert.deepEqual(delta(begin, snapshot({ status: "known", entries: { a: e(5) } })), { status: "unknown", code: "no-new-usage" });
|
||||
assert.deepEqual(delta(begin, snapshot({ status: "known", entries: { a: e(6), b: e(1) } })), { status: "unknown", code: "transcript-rewritten" });
|
||||
assert.deepEqual(delta(begin, snapshot({ status: "known", entries: { b: e(1) } })), { status: "unknown", code: "transcript-rewritten" });
|
||||
assert.deepEqual(delta(null, snapshot({ status: "known", entries: { b: e(1) } })), { status: "unknown", code: "baseline-unknown" });
|
||||
assert.deepEqual(delta(snapshot({ status: "unknown", code: "transcript-size" }), snapshot({ status: "known", entries: {} })), { status: "unknown", code: "baseline-unknown" });
|
||||
assert.deepEqual(delta(begin, snapshot({ status: "unknown", code: "transcript-parse" })), { status: "unknown", code: "transcript-parse" });
|
||||
assert.deepEqual(snapshot({ status: "unknown", code: "anything" }), { status: "unknown", code: "source-unknown" });
|
||||
assert.deepEqual(snapshot({ status: "known", entries: { a: { input: -1, output: 0, cacheWrite: 0, cacheRead: 0 } } }), { status: "unknown", code: "source-invalid" });
|
||||
assert.deepEqual(snapshot({ status: "known", entries: { a: { ...e(1), extra: 0 } } }), { status: "unknown", code: "source-invalid" });
|
||||
assert.deepEqual(snapshot(null), { status: "unknown", code: "source-invalid" });
|
||||
const big = { status: "known", entries: { a: e(Number.MAX_SAFE_INTEGER), b: e(1) } };
|
||||
assert.deepEqual(delta(snapshot({ status: "known", entries: {} }), snapshot(big)), { status: "unknown", code: "usage-overflow" });
|
||||
});
|
||||
@@ -302,8 +302,9 @@ test('keyboard focus follows visual order and survives a data refresh', async t
|
||||
await b.navigate(`http://127.0.0.1:${server.address().port}/#/orgs/ddigital/organization`);
|
||||
await wait('document.querySelectorAll("[data-org-dashboard] .ws-doc").length > 0 && document.querySelectorAll("#workspace-nav .ws-proj").length === 3');
|
||||
const wide = await tabStops(b, 'document.querySelector("#sections a")');
|
||||
// The left column holds the sessions and, below them, the Workflows card (SP16); the right holds Settings.
|
||||
const regions = ['nav', 'head', 'Sessions and workflows', 'Settings'];
|
||||
// The left column holds the sessions and, below them, the Workflows card (SP16), then Autonomy and Starter
|
||||
// packs (SP18); the right holds Settings.
|
||||
const regions = ['nav', 'head', 'Sessions, workflows, autonomy and starter packs', 'Settings'];
|
||||
assert.deepEqual([...new Set(wide.map(s => s.region))].filter(r => r !== 'other'), regions);
|
||||
assertVisualOrder(wide.filter(s => s.region !== 'other'), regions, '1440');
|
||||
const keys = wide.map(s => s.key);
|
||||
|
||||
@@ -312,6 +312,9 @@ test('a stopped organization is not read: the page and card say the bus is stopp
|
||||
await wait(`document.querySelector('[data-org-dashboard] [data-wfg-card][data-bus-stopped]')`, 'stopped card');
|
||||
assert.match(await b.evaluate(`document.querySelector('[data-wfg-card]').textContent`), /Not read: the bus is stopped\./);
|
||||
assert.equal(await b.evaluate(`document.querySelector('[data-wfg-card] [data-wfg-none]')`), null);
|
||||
// SP18: the card is the page's one Workflows block; the old stopped panel under it said the same thing twice.
|
||||
assert.equal(await b.evaluate(`[...document.querySelectorAll('[data-org-dashboard] h2, [data-org-dashboard] h3')].filter(h => h.textContent.trim() === 'Workflows').length`), 1, 'one Workflows heading on a stopped organization page');
|
||||
assert.equal(await b.evaluate(`document.querySelector('[data-workflows]')`), null);
|
||||
assert.equal(reads.filter(id => id === 'gamma').length, 0, 'a stopped organization is not read');
|
||||
});
|
||||
|
||||
|
||||
+1
-1
@@ -46,6 +46,6 @@ if [ "${1:-}" = library ] && [ "${2:-}" = survey ]; then
|
||||
exec node "$REPO/packages/library/src/survey.mjs" "$@"
|
||||
fi
|
||||
case "${1:-}" in
|
||||
org|proxy|halt|resume|processing|usage|project|limits|inbox|decide|tasks|agents|trail|talk|stop|launches|bus|console|plugins|credentials|hosts|grants|name|names|follow|threads|thread|access|dev|run) exec node "$REPO/packages/cli/src/cli.mjs" "$@" ;;
|
||||
org|proxy|halt|resume|processing|usage|project|limits|inbox|decide|tasks|agents|trail|talk|stop|launches|bus|console|plugins|credentials|hosts|grants|name|names|follow|threads|thread|access|dev|run|triage) exec node "$REPO/packages/cli/src/cli.mjs" "$@" ;;
|
||||
esac
|
||||
exec node "$REPO/packages/seat/src/cli.mjs" "$@"
|
||||
@@ -113,9 +113,26 @@ lists.
|
||||
can write. The counts are the session's own report, not host evidence.
|
||||
- The extension's per-reply dedupe is in memory. After a restart the same
|
||||
reply can get a second notice.
|
||||
- On the first turn of a new Claude Code session the transcript may not
|
||||
hold the reply when Stop runs; the hook then checks
|
||||
`last_assistant_message` and logs an empty `entryId`.
|
||||
- Claude Code can run the Stop hook before it writes the turn to the
|
||||
transcript (seen on 2.1.319 to 2.1.325, on any turn). The Stop hook keeps,
|
||||
per session id, the uuid of the last prompt whose turn it checked, in
|
||||
`<log>.stop.json` next to its log. A transcript is stale while its last
|
||||
prompt has that uuid, or while its last reply does not match
|
||||
`last_assistant_message`. The hook reads it again until it is not, for up
|
||||
to 2 s. If it never catches up, the hook checks `last_assistant_message`
|
||||
alone, logs an empty `entryId` and `late: true`, and the turn's earlier
|
||||
replies go unchecked; with no `last_assistant_message` it checks nothing.
|
||||
Two cases are left. First, a late fallback records no prompt, because
|
||||
the hook never saw the turn. When that turn lands in the transcript, a
|
||||
later Stop can take it for its own: one with no
|
||||
`last_assistant_message`, or one whose reply has the same text as the
|
||||
late turn's last reply. That Stop checks the late turn's replies again.
|
||||
Second, a Stop hook that blocks makes Claude go on with no new prompt,
|
||||
so a second Stop in that turn finds the transcript stale by its prompt.
|
||||
It waits 2 s and checks `last_assistant_message` alone, late. Managed
|
||||
sessions do not reach this case: their only Stop hook never blocks. The
|
||||
state file sits in the run directory with the log, so it is the
|
||||
session's own record too.
|
||||
- The word/phrase lists are the mechanical subset of ms-unslop only, keyed to
|
||||
pattern ids in lists.json. Style judgments (voice, rhythm, structure) stay in
|
||||
the skill, not the gate.
|
||||
|
||||
@@ -77,20 +77,86 @@ function check(text, at) {
|
||||
}
|
||||
}
|
||||
|
||||
function stop(input) {
|
||||
const list = entries(input.transcript_path);
|
||||
// The transcript's last prompt (its uuid, or null) and the assistant messages with text since it.
|
||||
function lastTurn(path) {
|
||||
const list = entries(path);
|
||||
let start = -1;
|
||||
list.forEach((e, i) => { if (isPrompt(e)) start = i; });
|
||||
const turn = replies(list.slice(start + 1));
|
||||
if (!turn.length && typeof input.last_assistant_message === "string" && input.last_assistant_message.trim()) {
|
||||
turn.push({ id: "", text: input.last_assistant_message });
|
||||
const uuid = start >= 0 && typeof list[start].uuid === "string" && list[start].uuid ? list[start].uuid : null;
|
||||
return { prompt: uuid, replies: replies(list.slice(start + 1)) };
|
||||
}
|
||||
|
||||
// The Stop hook's own state, next to its log: per session id, the uuid of the last prompt whose
|
||||
// turn it checked. At most STATE_MAX sessions are kept, the latest last.
|
||||
const STATE = LOG ? `${LOG}.stop.json` : null;
|
||||
const STATE_MAX = 64;
|
||||
function stopState() {
|
||||
if (!STATE) return {};
|
||||
try {
|
||||
const value = JSON.parse(fs.readFileSync(STATE, "utf8"));
|
||||
return value !== null && typeof value === "object" && !Array.isArray(value) ? value : {};
|
||||
} catch { return {}; }
|
||||
}
|
||||
function recordStop(session, prompt) {
|
||||
if (!STATE) return;
|
||||
const state = stopState();
|
||||
delete state[session];
|
||||
state[session] = prompt;
|
||||
const keys = Object.keys(state);
|
||||
for (const k of keys.slice(0, Math.max(0, keys.length - STATE_MAX))) delete state[k];
|
||||
const tmp = `${STATE}.${process.pid}`;
|
||||
try {
|
||||
fs.writeFileSync(tmp, JSON.stringify(state) + "\n");
|
||||
fs.renameSync(tmp, STATE);
|
||||
} catch {
|
||||
try { fs.rmSync(tmp, { force: true }); } catch {}
|
||||
}
|
||||
}
|
||||
|
||||
// Claude Code can run the Stop hook before it writes the turn to the transcript, so the
|
||||
// transcript may still end with the last turn. It is stale while its last prompt is the one the
|
||||
// hook last checked for this session, or while its last reply is not the one the hook input
|
||||
// names in last_assistant_message. Read it again until it is not, for up to STOP_WAIT_MS (the
|
||||
// hook's own limit is 10 s). A transcript that never catches up gives last_assistant_message
|
||||
// alone, logged with late: true so a release where it never matches shows in the log; with no
|
||||
// last_assistant_message there is no reply to check, and nothing is logged.
|
||||
// Two cases are left. A late fallback records no prompt uuid, since the hook never saw the turn.
|
||||
// Once that turn lands, a later Stop with no last_assistant_message, or with a reply whose text
|
||||
// equals the late turn's last reply, takes it for its own and logs its replies again. And a Stop
|
||||
// hook that blocks makes Claude go on with no new prompt, so a second Stop in that turn finds the
|
||||
// transcript stale by the uuid rule, waits, and logs only last_assistant_message, late. Managed
|
||||
// sessions never reach the second case: their only Stop hook never blocks.
|
||||
const STOP_WAIT_MS = 2000;
|
||||
const flat = (s) => s.replace(/\s+/g, "");
|
||||
const pause = (ms) => Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ms);
|
||||
|
||||
function stop(input) {
|
||||
const told = typeof input.last_assistant_message === "string" && input.last_assistant_message.trim() ? input.last_assistant_message : null;
|
||||
const session = typeof input.session_id === "string" ? input.session_id : "";
|
||||
const state = stopState();
|
||||
const seen = Object.hasOwn(state, session) && typeof state[session] === "string" ? state[session] : null;
|
||||
const stale = (t) => (t.prompt !== null && t.prompt === seen) || (told !== null && !(t.replies.length && flat(t.replies.at(-1).text) === flat(told)));
|
||||
const deadline = Date.now() + STOP_WAIT_MS;
|
||||
let t = lastTurn(input.transcript_path);
|
||||
let turn = t.replies, late = false;
|
||||
while (stale(t)) {
|
||||
if (Date.now() >= deadline) {
|
||||
turn = told === null ? [] : [{ id: "", text: told }];
|
||||
late = true;
|
||||
break;
|
||||
}
|
||||
pause(25);
|
||||
t = lastTurn(input.transcript_path);
|
||||
turn = t.replies;
|
||||
}
|
||||
for (const r of turn) {
|
||||
const result = check(r.text, "stop");
|
||||
if (result === null) return;
|
||||
if (result.clean) log({ ev: "checked", clean: true, entryId: r.id, charsChecked: result.charsChecked });
|
||||
else log({ ev: "flagged", clean: false, entryId: r.id, charsChecked: result.charsChecked, findings: result.findings });
|
||||
const extra = late ? { late: true } : {};
|
||||
if (result.clean) log({ ev: "checked", clean: true, entryId: r.id, charsChecked: result.charsChecked, ...extra });
|
||||
else log({ ev: "flagged", clean: false, entryId: r.id, charsChecked: result.charsChecked, findings: result.findings, ...extra });
|
||||
}
|
||||
if (!late && t.prompt !== null) recordStop(session, t.prompt);
|
||||
}
|
||||
|
||||
// The entry ids the log already holds a notice for.
|
||||
|
||||
@@ -302,6 +302,92 @@ t("claude-hook: a broken lists.json logs broken_lists and no verdict; bad input
|
||||
assert.deepStrictEqual(events(bad).map((e) => e.ev), ["hook_failed"]);
|
||||
});
|
||||
|
||||
// Claude Code can run the Stop hook before it writes the turn to the transcript (seen on 2.1.319,
|
||||
// 2.1.322 and 2.1.325). The hook then saw the last turn's reply and logged it again.
|
||||
const jsonl = (list) => list.map((x) => JSON.stringify(x)).join("\n") + "\n";
|
||||
const lastTurn = [
|
||||
{ type: "user", uuid: "p1", message: { role: "user", content: "status?" } },
|
||||
line("assistant", "m1", "a1", [{ type: "text", text: "The build is green." }]),
|
||||
];
|
||||
const TELL_REPLY = "Let me delve into it. A pivotal step.";
|
||||
|
||||
t("claude-hook: Stop waits for a late transcript and checks this turn's reply, not the last turn's", () => {
|
||||
const transcript = writeTmp("transcript-late.jsonl", jsonl(lastTurn));
|
||||
const late = writeTmp("transcript-late-tail.jsonl", jsonl([
|
||||
{ type: "user", uuid: "p2", message: { role: "user", content: "and now?" } },
|
||||
line("assistant", "m2", "a2", [{ type: "text", text: TELL_REPLY }]),
|
||||
]));
|
||||
const log = tmp("hook-late.jsonl");
|
||||
// The writer runs on its own while the hook blocks, as claude's transcript writer does.
|
||||
const writer = require("node:child_process").spawn("sh", ["-c", 'sleep 0.3; cat "$1" >> "$2"', "sh", late, transcript], { stdio: "ignore" });
|
||||
const r = hook("stop", { transcript_path: transcript, last_assistant_message: TELL_REPLY }, { MOSAIC_UNSLOP_LOG: log });
|
||||
writer.kill();
|
||||
assert.strictEqual(r.status, 0, r.stderr);
|
||||
assert.deepStrictEqual(events(log).map((e) => [e.ev, e.entryId]), [["flagged", "a2"]]);
|
||||
});
|
||||
|
||||
t("claude-hook: Stop with a transcript that never catches up checks last_assistant_message alone", () => {
|
||||
const transcript = writeTmp("transcript-stale.jsonl", jsonl(lastTurn));
|
||||
const log = tmp("hook-stale.jsonl");
|
||||
const r = hook("stop", { transcript_path: transcript, last_assistant_message: TELL_REPLY }, { MOSAIC_UNSLOP_LOG: log });
|
||||
assert.strictEqual(r.status, 0, r.stderr);
|
||||
assert.deepStrictEqual(events(log).map((e) => [e.ev, e.entryId, e.late]), [["flagged", "", true]]);
|
||||
// A turn already in the transcript is checked at once, with its entry id.
|
||||
const now = tmp("hook-current.jsonl");
|
||||
const started = Date.now();
|
||||
hook("stop", { transcript_path: transcript, last_assistant_message: "The build is green." }, { MOSAIC_UNSLOP_LOG: now });
|
||||
assert.ok(Date.now() - started < 1500, "no wait when the transcript is current");
|
||||
assert.deepStrictEqual(events(now).map((e) => [e.ev, e.entryId, e.late]), [["checked", "a1", undefined]]);
|
||||
});
|
||||
|
||||
// The Stop hook keeps, per session and next to its log, the last prompt whose turn it checked. A
|
||||
// transcript that still ends with that prompt is stale, even with no last_assistant_message or
|
||||
// with a reply whose text equals the last turn's.
|
||||
const appendLater = (from, to) => require("node:child_process").spawn("sh", ["-c", 'sleep 0.3; cat "$1" >> "$2"', "sh", from, to], { stdio: "ignore" });
|
||||
|
||||
t("claude-hook: Stop with no last_assistant_message does not check a turn it checked again, and waits for the late tail", () => {
|
||||
const transcript = writeTmp("transcript-untold.jsonl", jsonl(lastTurn));
|
||||
const log = tmp("hook-untold.jsonl");
|
||||
const env = { MOSAIC_UNSLOP_LOG: log };
|
||||
assert.strictEqual(hook("stop", { session_id: "s1", transcript_path: transcript }, env).status, 0);
|
||||
assert.deepStrictEqual(events(log).map((e) => [e.ev, e.entryId]), [["checked", "a1"]]);
|
||||
// The next turn's Stop runs before its turn reaches the transcript.
|
||||
const writer = appendLater(writeTmp("transcript-untold-tail.jsonl", jsonl([
|
||||
{ type: "user", uuid: "p2", message: { role: "user", content: "and now?" } },
|
||||
line("assistant", "m2", "a2", [{ type: "text", text: TELL_REPLY }]),
|
||||
])), transcript);
|
||||
const r = hook("stop", { session_id: "s1", transcript_path: transcript }, env);
|
||||
writer.kill();
|
||||
assert.strictEqual(r.status, 0, r.stderr);
|
||||
assert.deepStrictEqual(events(log).map((e) => [e.ev, e.entryId]), [["checked", "a1"], ["flagged", "a2"]]);
|
||||
// A transcript that never catches up gives nothing to check: nothing is logged.
|
||||
assert.strictEqual(hook("stop", { session_id: "s1", transcript_path: transcript }, env).status, 0);
|
||||
assert.strictEqual(events(log).length, 2);
|
||||
// The state is per session: another session on the same log has checked nothing yet.
|
||||
hook("stop", { session_id: "s2", transcript_path: transcript }, env);
|
||||
assert.deepStrictEqual(events(log).slice(2).map((e) => [e.ev, e.entryId]), [["flagged", "a2"]]);
|
||||
});
|
||||
|
||||
t("claude-hook: Stop whose reply has the last turn's text does not take the stale transcript for this turn", () => {
|
||||
const transcript = writeTmp("transcript-same.jsonl", jsonl(lastTurn));
|
||||
const log = tmp("hook-same.jsonl");
|
||||
const env = { MOSAIC_UNSLOP_LOG: log };
|
||||
const told = { session_id: "s1", transcript_path: transcript, last_assistant_message: "The build is green." };
|
||||
assert.strictEqual(hook("stop", told, env).status, 0);
|
||||
assert.deepStrictEqual(events(log).map((e) => [e.ev, e.entryId]), [["checked", "a1"]]);
|
||||
const writer = appendLater(writeTmp("transcript-same-tail.jsonl", jsonl([
|
||||
{ type: "user", uuid: "p2", message: { role: "user", content: "still?" } },
|
||||
line("assistant", "m2", "a2", [{ type: "text", text: "The build is green." }]),
|
||||
])), transcript);
|
||||
const r = hook("stop", told, env);
|
||||
writer.kill();
|
||||
assert.strictEqual(r.status, 0, r.stderr);
|
||||
assert.deepStrictEqual(events(log).map((e) => [e.ev, e.entryId]), [["checked", "a1"], ["checked", "a2"]]);
|
||||
// A transcript that never catches up gives last_assistant_message alone, late.
|
||||
hook("stop", told, env);
|
||||
assert.deepStrictEqual(events(log).slice(2).map((e) => [e.ev, e.entryId, e.late]), [["checked", "", true]]);
|
||||
});
|
||||
|
||||
let failed = 0;
|
||||
for (const [name, ok] of results) { console.log(`${ok ? "PASS" : "FAIL"} ${name}`); if (!ok) failed++; }
|
||||
console.log(`${results.length - failed}/${results.length} passed`);
|
||||
|
||||
Reference in new issue
Block a user